[
{
"path": ".gitignore",
"content": ".DS_Store\nwip"
},
{
"path": "assets/main.rs",
"content": "fn main() {\n println!(\"hopefully github detects this file and categorizes my repo as a Rust repo\");\n}\n"
},
{
"path": "license-apache",
"content": " Apache License\n Version 2.0, January 2004\n http://www.apache.org/licenses/\n\nTERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION\n\n1. Definitions.\n\n \"License\" shall mean the terms and conditions for use, reproduction,\n and distribution as defined by Sections 1 through 9 of this document.\n\n \"Licensor\" shall mean the copyright owner or entity authorized by\n the copyright owner that is granting the License.\n\n \"Legal Entity\" shall mean the union of the acting entity and all\n other entities that control, are controlled by, or are under common\n control with that entity. For the purposes of this definition,\n \"control\" means (i) the power, direct or indirect, to cause the\n direction or management of such entity, whether by contract or\n otherwise, or (ii) ownership of fifty percent (50%) or more of the\n outstanding shares, or (iii) beneficial ownership of such entity.\n\n \"You\" (or \"Your\") shall mean an individual or Legal Entity\n exercising permissions granted by this License.\n\n \"Source\" form shall mean the preferred form for making modifications,\n including but not limited to software source code, documentation\n source, and configuration files.\n\n \"Object\" form shall mean any form resulting from mechanical\n transformation or translation of a Source form, including but\n not limited to compiled object code, generated documentation,\n and conversions to other media types.\n\n \"Work\" shall mean the work of authorship, whether in Source or\n Object form, made available under the License, as indicated by a\n copyright notice that is included in or attached to the work\n (an example is provided in the Appendix below).\n\n \"Derivative Works\" shall mean any work, whether in Source or Object\n form, that is based on (or derived from) the Work and for which the\n editorial revisions, annotations, elaborations, or other modifications\n represent, as a whole, an original work of authorship. For the purposes\n of this License, Derivative Works shall not include works that remain\n separable from, or merely link (or bind by name) to the interfaces of,\n the Work and Derivative Works thereof.\n\n \"Contribution\" shall mean any work of authorship, including\n the original version of the Work and any modifications or additions\n to that Work or Derivative Works thereof, that is intentionally\n submitted to Licensor for inclusion in the Work by the copyright owner\n or by an individual or Legal Entity authorized to submit on behalf of\n the copyright owner. For the purposes of this definition, \"submitted\"\n means any form of electronic, verbal, or written communication sent\n to the Licensor or its representatives, including but not limited to\n communication on electronic mailing lists, source code control systems,\n and issue tracking systems that are managed by, or on behalf of, the\n Licensor for the purpose of discussing and improving the Work, but\n excluding communication that is conspicuously marked or otherwise\n designated in writing by the copyright owner as \"Not a Contribution.\"\n\n \"Contributor\" shall mean Licensor and any individual or Legal Entity\n on behalf of whom a Contribution has been received by Licensor and\n subsequently incorporated within the Work.\n\n2. Grant of Copyright License. Subject to the terms and conditions of\n this License, each Contributor hereby grants to You a perpetual,\n worldwide, non-exclusive, no-charge, royalty-free, irrevocable\n copyright license to reproduce, prepare Derivative Works of,\n publicly display, publicly perform, sublicense, and distribute the\n Work and such Derivative Works in Source or Object form.\n\n3. Grant of Patent License. Subject to the terms and conditions of\n this License, each Contributor hereby grants to You a perpetual,\n worldwide, non-exclusive, no-charge, royalty-free, irrevocable\n (except as stated in this section) patent license to make, have made,\n use, offer to sell, sell, import, and otherwise transfer the Work,\n where such license applies only to those patent claims licensable\n by such Contributor that are necessarily infringed by their\n Contribution(s) alone or by combination of their Contribution(s)\n with the Work to which such Contribution(s) was submitted. If You\n institute patent litigation against any entity (including a\n cross-claim or counterclaim in a lawsuit) alleging that the Work\n or a Contribution incorporated within the Work constitutes direct\n or contributory patent infringement, then any patent licenses\n granted to You under this License for that Work shall terminate\n as of the date such litigation is filed.\n\n4. Redistribution. You may reproduce and distribute copies of the\n Work or Derivative Works thereof in any medium, with or without\n modifications, and in Source or Object form, provided that You\n meet the following conditions:\n\n (a) You must give any other recipients of the Work or\n Derivative Works a copy of this License; and\n\n (b) You must cause any modified files to carry prominent notices\n stating that You changed the files; and\n\n (c) You must retain, in the Source form of any Derivative Works\n that You distribute, all copyright, patent, trademark, and\n attribution notices from the Source form of the Work,\n excluding those notices that do not pertain to any part of\n the Derivative Works; and\n\n (d) If the Work includes a \"NOTICE\" text file as part of its\n distribution, then any Derivative Works that You distribute must\n include a readable copy of the attribution notices contained\n within such NOTICE file, excluding those notices that do not\n pertain to any part of the Derivative Works, in at least one\n of the following places: within a NOTICE text file distributed\n as part of the Derivative Works; within the Source form or\n documentation, if provided along with the Derivative Works; or,\n within a display generated by the Derivative Works, if and\n wherever such third-party notices normally appear. The contents\n of the NOTICE file are for informational purposes only and\n do not modify the License. You may add Your own attribution\n notices within Derivative Works that You distribute, alongside\n or as an addendum to the NOTICE text from the Work, provided\n that such additional attribution notices cannot be construed\n as modifying the License.\n\n You may add Your own copyright statement to Your modifications and\n may provide additional or different license terms and conditions\n for use, reproduction, or distribution of Your modifications, or\n for any such Derivative Works as a whole, provided Your use,\n reproduction, and distribution of the Work otherwise complies with\n the conditions stated in this License.\n\n5. Submission of Contributions. Unless You explicitly state otherwise,\n any Contribution intentionally submitted for inclusion in the Work\n by You to the Licensor shall be under the terms and conditions of\n this License, without any additional terms or conditions.\n Notwithstanding the above, nothing herein shall supersede or modify\n the terms of any separate license agreement you may have executed\n with Licensor regarding such Contributions.\n\n6. Trademarks. This License does not grant permission to use the trade\n names, trademarks, service marks, or product names of the Licensor,\n except as required for reasonable and customary use in describing the\n origin of the Work and reproducing the content of the NOTICE file.\n\n7. Disclaimer of Warranty. Unless required by applicable law or\n agreed to in writing, Licensor provides the Work (and each\n Contributor provides its Contributions) on an \"AS IS\" BASIS,\n WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or\n implied, including, without limitation, any warranties or conditions\n of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A\n PARTICULAR PURPOSE. You are solely responsible for determining the\n appropriateness of using or redistributing the Work and assume any\n risks associated with Your exercise of permissions under this License.\n\n8. Limitation of Liability. In no event and under no legal theory,\n whether in tort (including negligence), contract, or otherwise,\n unless required by applicable law (such as deliberate and grossly\n negligent acts) or agreed to in writing, shall any Contributor be\n liable to You for damages, including any direct, indirect, special,\n incidental, or consequential damages of any character arising as a\n result of this License or out of the use or inability to use the\n Work (including but not limited to damages for loss of goodwill,\n work stoppage, computer failure or malfunction, or any and all\n other commercial damages or losses), even if such Contributor\n has been advised of the possibility of such damages.\n\n9. Accepting Warranty or Additional Liability. While redistributing\n the Work or Derivative Works thereof, You may choose to offer,\n and charge a fee for, acceptance of support, warranty, indemnity,\n or other liability obligations and/or rights consistent with this\n License. However, in accepting such obligations, You may act only\n on Your own behalf and on Your sole responsibility, not on behalf\n of any other Contributor, and only if You agree to indemnify,\n defend, and hold each Contributor harmless for any liability\n incurred by, or claims asserted against, such Contributor by reason\n of your accepting any such warranty or additional liability.\n\nEND OF TERMS AND CONDITIONS\n"
},
{
"path": "license-mit",
"content": "Permission is hereby granted, free of charge, to any\nperson obtaining a copy of this software and associated\ndocumentation files (the \"Software\"), to deal in the\nSoftware without restriction, including without\nlimitation the rights to use, copy, modify, merge,\npublish, distribute, sublicense, and/or sell copies of\nthe Software, and to permit persons to whom the Software\nis furnished to do so, subject to the following\nconditions:\n\nThe above copyright notice and this permission notice\nshall be included in all copies or substantial portions\nof the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF\nANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED\nTO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A\nPARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT\nSHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY\nCLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION\nOF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR\nIN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER\nDEALINGS IN THE SOFTWARE.\n"
},
{
"path": "posts/chat-server.md",
"content": "# Beginner's Guide to Concurrent Programming: Coding a Multithreaded Chat Server using Tokio\n\n_04 May 2024 · #rust · #async · #concurrency · #tokio_\n\n\n\n\nTable of contents \n\n## Introduction\n\nI recently finished coding a multithreaded chat server using Tokio and I'm pretty happy with it. I'd like to share what I learned in this easy-to-follow step-by-step tutorial-style article. Let's get into it.\n\n> [!NOTE]\n> The full source code for every step can be found in [the examples directory](https://github.com/pretzelhammer/chat-server/tree/main/examples) of [this repository](https://github.com/pretzelhammer/chat-server).\n\n## 01\\) Simplest possible echo server\n\nLet's start by writing the simplest possible echo server.\n\n```rust\nuse tokio::{io::{AsyncReadExt, AsyncWriteExt}, net::TcpListener};\n\n#[tokio::main]\nasync fn main() -> anyhow::Result<()> {\n let server = TcpListener::bind(\"127.0.0.1:42069\").await?;\n let (mut tcp, _) = server.accept().await?;\n let mut buffer = [0u8; 16];\n loop {\n let n = tcp.read(&mut buffer).await?;\n if n == 0 {\n break;\n }\n let _ = tcp.write(&buffer[..n]).await?;\n }\n Ok(())\n}\n```\n\n`#[tokio::main]` is a procedural macro that removes some of the boilerplate of building a tokio runtime, and turns this:\n\n```rust\n#[tokio::main]\nasync fn my_async_fn() {\n todo!()\n}\n```\n\nRoughly into this:\n\n```rust\nfn main() {\n tokio::runtime::Builder::new_multi_thread()\n .enable_all()\n .build()\n .unwrap()\n .block_on(my_async_fn)\n}\n\nasync fn my_async_fn() {\n todo!()\n}\n```\n\nAnd as a quick refresher if we have an async function like this:\n\n```rust\nasync fn my_async_fn(t: T) -> T {\n todo!()\n}\n```\n\nIt roughly desugars into:\n\n```rust\nfn my_async_fn(t: T) -> impl Future {\n todo!()\n}\n```\n\nAnd a `Future` represents some form of asynchronous computation that we can `await` to get the result.\n\nLet's also use the `anyhow` crate for carefree error propagation. Any place where we might want to return `Result>` we can substitute it with `anyhow::Result` and get the same behavior.\n\nIn this line we're binding a TCP listener:\n\n```rust\nlet server = TcpListener::bind(\"127.0.0.1:42069\").await?;\n```\n\n> [!IMPORTANT]\n> This is `tokio::net::TcpListener` and not `std::net::TcpListener`. The former is async and the latter is sync. Also calling `bind` returns a `Future` which we **must** `await` for anything to happen because futures are lazy in Rust!\n\nAs a general rule of thumb, if there's a type that handles IO by the same name in both `tokio` and `std` we want to use the one in `tokio`.\n\nThe rest of the code should hopefully be straight-forward:\n\n```rust\nlet (mut tcp, _) = server.accept().await?;\nlet mut buffer = [0u8; 16];\nloop {\n let n = tcp.read(&mut buffer).await?;\n if n == 0 {\n break;\n }\n let _ = tcp.write(&buffer[..n]).await?;\n}\n```\n\nWe accept a connection, create a buffer, and then we read bytes from the connection into the buffer and write those bytes back to the connection in a loop until the connection closes.\n\nWe can connect to this server using a tool like `telnet` to see that it does in fact echo everything back to us:\n\n```console\n$ telnet 127.0.0.1 42069\n> my first e c h o server!\nmy first e c h o server!\n> hooray!\nhooray!\n```\n\n> [!TIP]\n> To quit `telnet` type `^]` (control + right square bracket) to enter command mode and type \"quit\" + ENTER.\n\nIf you'd like to mess around with the code yourself just `git clone` [this repository](https://github.com/pretzelhammer/chat-server) and you'll be able to quickly run any example with `just example {number}`. Then you can tinker with the source code at `examples/server-{number}.rs` to your heart's content. Once an example is running you can interact with it by running `just telnet`.\n\n## 02\\) Handling multiple connections serially\n\nThere's an annoying bug in our server: it quits after handling just one connection! If we try to `just telnet` more than once we get `telnet: Unable to connect to remote host: Connection refused` at which point we have to manually restart the server with `just example 01` again. 🤦\n\nHere's how we would fix it:\n\n```rust\nuse tokio::{io::{AsyncReadExt, AsyncWriteExt}, net::TcpListener};\n\n#[tokio::main]\nasync fn main() -> anyhow::Result<()> {\n let server = TcpListener::bind(\"127.0.0.1:42069\").await?;\n loop {\n let (mut tcp, _) = server.accept().await?;\n let mut buffer = [0u8; 16];\n loop {\n let n = tcp.read(&mut buffer).await?;\n if n == 0 {\n break;\n }\n let _ = tcp.write(&buffer[..n]).await?;\n }\n }\n}\n```\n\nWe just had to add another `loop` around our `server.accept()` line! Pretty easy, now we can run the updated example with `just example 02` and the server stays up regardless of how many times we run `just telnet` in a row.\n\n## 03\\) Modifying messages\n\nAs exciting as an echo server is, it would be even more exciting if it modified messages somehow. So how about we try adding a ❤️ emoji at the end of every echoed line? Here's what that would look like:\n\n```rust\nuse tokio::{io::{AsyncReadExt, AsyncWriteExt}, net::TcpListener};\n\n#[tokio::main]\nasync fn main() -> anyhow::Result<()> {\n let server = TcpListener::bind(\"127.0.0.1:42069\").await?;\n loop {\n let (mut tcp, _) = server.accept().await?;\n let mut buffer = [0u8; 16];\n loop {\n let n = tcp.read(&mut buffer).await?;\n if n == 0 {\n break;\n }\n // convert byte slice to a String\n let mut line = String::from_utf8(buffer[..n].to_vec())?;\n // remove line terminating chars added by telnet\n line.pop(); // remove \\n char\n line.pop(); // remove \\r char\n // add our own line terminator :)\n line.push_str(\" ❤️\\n\");\n let _ = tcp.write(line.as_bytes()).await?;\n }\n }\n}\n```\n\nDemo of our hearty echo server:\n\n```console\n$ just telnet\n> hello\nhello ❤️\n> it works!\nit works! ❤️\n```\n\nHowever if we write a message that's a little too long we'll see this bug:\n\n```console\n> this is the best day ever!\nthis is the be ❤️\n day ever ❤️\n```\n\nWelp! Nobody said this was gonna be easy. We can increase the fixed size of our buffer but by how much? We can use a growable buffer like a `Vec` but what if the client sends a _really, really long line_? We could solve these problems ourselves, but they're pretty common, so we can also offload them to someone else too.\n\n## 04\\) Parsing a stream of bytes as lines\n\nTokio offers a convenient and robust solution to our line problem in the `tokio-util` crate. Here's how we can use it:\n\n```rust\nuse futures::{SinkExt, StreamExt};\nuse tokio::net::TcpListener;\nuse tokio_util::codec::{FramedRead, FramedWrite, LinesCodec};\n\n#[tokio::main]\nasync fn main() -> anyhow::Result<()> {\n let server = TcpListener::bind(\"127.0.0.1:42069\").await?;\n loop {\n let (mut tcp, _) = server.accept().await?;\n let (reader, writer) = tcp.split();\n let mut stream = FramedRead::new(reader, LinesCodec::new());\n let mut sink = FramedWrite::new(writer, LinesCodec::new());\n while let Some(Ok(mut msg)) = stream.next().await {\n msg.push_str(\" ❤️\");\n sink.send(msg).await?;\n }\n }\n}\n```\n\nThere's a lot of new stuff in this example so let's go over it. The `split` method splits a `TcpStream` into a `ReadHalf` and `WriteHalf`. This is useful if we want to add these halves to different structs, or send them to different threads, or read and write to the same `TcpStream` concurrently (which we'll be doing later).\n\n`ReadHalf` implements `AsyncRead` and `WriteHalf` implements `AsyncWrite`, however as mentioned previously, these can be tedious and error-prone to work with directly, which why we bring in `LinesCodec`, `FramedRead`, and `FramedWrite`.\n\n`LinesCodec` handles the low-level details of converting a stream of bytes into a stream of UTF-8 strings delimited by newlines, and using it together with `FramedRead` we can wrap a `ReadHalf` to get an implementation of `Stream- >`, which is much easier to work with than an `AsyncRead`. A `Stream` is like the async version of an `Iterator`. For example, if we had a sync function like this:\n\n```rust\nfn iterate
(items: impl Iterator- ) {\n for item in items {\n todo!()\n }\n}\n```\n\nThe refactored async version would be:\n\n```rust\nuse futures::{Stream, StreamExt};\n\nasync fn iterate
(mut items: impl Stream- + Unpin) {\n while let Some(item) = items.next().await {\n todo!()\n }\n}\n```\n\nWe also use `LinesCodec` together with `FramedWrite` to wrap a `WriteHalf` to get an implementation of `Sink
`, which is much easier to work with than an `AsyncWrite`. As you've probably guessed, a `Sink` is the opposite of a `Stream`, it consumes values instead of producing values.\n\nThe rest of the code is straight-forward:\n\n```rust\nwhile let Some(Ok(mut msg)) = stream.next().await {\n msg.push_str(\" ❤️\");\n sink.send(msg).await?;\n}\n```\n\nWe get message from the stream, add a heart to it, and then send it to the sink. If we wanted to be fancy we could have also mapped the stream and forward it to the sink like this:\n\n```rust\nstream.map(|msg| {\n let mut msg = msg?;\n msg.push_str(\" ❤️\");\n Ok(msg)\n}).forward(sink).await?\n```\n\n`forward` returns a `Future` that completes when the `Stream` has been fully processed into the `Sink` and the `Sink` has been closed and flushed.\n\nNow our server correctly appends the heart regardless of message length:\n\n```console\n$ just telnet\n> this is a really really really long message kinda\nthis is a really really really long message kinda ❤️\n```\n\n## 05\\) Adding `/help` & `/quit` server commands\n\nTelnet is annoying to quit. The usual tricks of `esc`, `^C`, and `^D` don't work. We have to type `^]` to enter command mode and then type `quit` + ENTER. 🤦\n\nWe can make our server more user-friendly by implementing our own commands, so let's start with `/help` and `/quit`. `/help` will print out a list and description of all the commands our server supports and `/quit` will cause the server to close the connection (which will also cause telnet to quit).\n\nSo that these commands are discoverable lets send them immediately to every client that connects. Here's what everything put together looks like:\n\n```rust\nuse futures::{SinkExt, StreamExt};\nuse tokio::net::TcpListener;\nuse tokio_util::codec::{FramedRead, FramedWrite, LinesCodec};\n\nconst HELP_MSG: &str = include_str!(\"help.txt\");\n\n#[tokio::main]\nasync fn main() -> anyhow::Result<()> {\n let server = TcpListener::bind(\"127.0.0.1:42069\").await?;\n loop {\n let (mut tcp, _) = server.accept().await?;\n let (reader, writer) = tcp.split();\n let mut stream = FramedRead::new(reader, LinesCodec::new());\n let mut sink = FramedWrite::new(writer, LinesCodec::new());\n // send list of server commands to\n // the user as soon as they connect\n sink.send(HELP_MSG).await?;\n while let Some(Ok(mut msg)) = stream.next().await {\n // handle new /help command\n if msg.starts_with(\"/help\") {\n sink.send(HELP_MSG).await?;\n // handle new /quit command\n } else if msg.starts_with(\"/quit\") {\n break;\n // handle regular message\n } else {\n msg.push_str(\" ❤️\");\n sink.send(msg).await?;\n }\n }\n }\n}\n```\n\nLet's give it a spin:\n\n```console\n$ just telnet\nServer commands\n /help - prints this message\n /quit - quits server\n> /help # new command\nServer commands\n /help - prints this message\n /quit - quits server\n> woohoo it works\nwoohoo it works ❤️\n> /quit # new command\nConnection closed by foreign host.\n```\n\n## 06\\) Handling multiple connections concurrently\n\nThe biggest downside of our server is that it only handles one connection at a time! If we run `just telnet` in two separate terminals we'll notice our server will only respond to the first connection, and won't start responding to the second connection until the first quits. Although we've been using a lot of async APIs our current implementation behaves no differently than a sync single-threaded server. Let's change that:\n\n```rust\nuse futures::{SinkExt, StreamExt};\nuse tokio::net::{TcpListener, TcpStream};\nuse tokio_util::codec::{FramedRead, FramedWrite, LinesCodec};\n\nconst HELP_MSG: &str = include_str!(\"help.txt\");\n\n#[tokio::main]\nasync fn main() -> anyhow::Result<()> {\n let server = TcpListener::bind(\"127.0.0.1:42069\").await?;\n loop {\n let (tcp, _) = server.accept().await?;\n // spawn a separate task for\n // to handle every connection\n tokio::spawn(handle_user(tcp));\n }\n}\n\nasync fn handle_user(mut tcp: TcpStream) -> anyhow::Result<()> {\n let (reader, writer) = tcp.split();\n let mut stream = FramedRead::new(reader, LinesCodec::new());\n let mut sink = FramedWrite::new(writer, LinesCodec::new());\n sink.send(HELP_MSG).await?;\n while let Some(Ok(mut msg)) = stream.next().await {\n if msg.starts_with(\"/help\") {\n sink.send(HELP_MSG).await?;\n } else if msg.starts_with(\"/quit\") {\n break;\n } else {\n msg.push_str(\" ❤️\");\n sink.send(msg).await?;\n }\n }\n Ok(())\n}\n```\n\n`tokio::spawn` takes a `Future` and spawns an asynchronous \"task\" to complete it. The execution begins immediately so we don't have to `await` the returned join handle like we would a future. A \"task\" is like a native thread except instead of being managed by the OS it's managed by Tokio. You're probably already familiar with this concept by some of these other names: lightweight threads, green threads, user-space threads.\n\n## 07\\) Letting users kinda chat\n\nTo really get the party started we need to upgrade our echo server to a chat server that lets separate concurrent connections communicate with each other:\n\n> [!NOTE]\n> The code is starting to get long and difficult to read. All following examples will be presented as a heavily abbreviated diff highlighting the key changes, but you can still find the full source code for any example in [the examples directory](https://github.com/pretzelhammer/chat-server/tree/main/examples) of [this repository](https://github.com/pretzelhammer/chat-server). You can see a diff between any two examples by running `just diff {number} {number}`. For instance, to see the diff between this example and the previous example you would run `just diff 06 07`.\n\n```rust\n// ...\nasync fn main() -> anyhow::Result<()> {\n // ...\n // create broadcast channel\n let (tx, _) = broadcast::channel::(32);\n // ...\n // clone it for every connected client\n tokio::spawn(handle_user(tcp, tx.clone()));\n}\n\nasync fn handle_user(\n mut tcp: TcpStream,\n tx: Sender\n) -> anyhow::Result<()> {\n // ...\n // get a receiver from the sender\n let mut rx = tx.subscribe();\n // ...\n while let Some(Ok(mut user_msg)) = stream.next().await {\n // ...\n // send all messages to the channel\n tx.send(user_msg)?;\n // ...\n // receive all of our and others'\n // messages from the channel\n let peer_msg = rx.recv().await?;\n sink.send(peer_msg).await?;\n }\n // ...\n}\n```\n\nWe communicate between different clients using a broadcast channel. After creating the channel we get a `Sender` and a `Receiver` which we can `clone` any number of times and send to different threads. Each value sent via a `Sender` gets received by every `Receiver`, so the value type must implement `Clone`.\n\nBefore we would get a message from the client's stream and immediately echo it out to the client's sink. Now when we get a message from the client's stream we pass it through the broadcast channel before we get it back and then send it to the client's sink. Every client will receive their own and others' messages from the shared channel.\n\nLet's try out our new code by connecting with two clients at once:\n\n```console\n$ just telnet # concurrent client 1\n> 1: hello # msg 1\n1: hello ❤️\n> 1: anybody there? # msg 2\n1: anybody there? ❤️\n\n$ just telnet # concurrent client 2\n> 2: hey there # msg 3\n1: hello ❤️\n> 2: how are you # msg 4\n1: anybody there? ❤️\n> 2: i am right here # msg 5\n2: hey there ❤️\n> 2: wtf # msg 6\n2: how are you ❤️\n```\n\nEach client is seeing each other's messages but they seem kinda delayed and staggered for some reason. Something just isn't quite right here.\n\nThe bug in our code is here:\n\n```rust\n// the client must first send a message\nwhile let Some(Ok(mut user_msg)) = stream.next().await {\n // in order to receive a message\n let peer_msg = rx.recv().await?;\n // and these two things always alternate\n}\n```\n\nIn order to receive a message from a peer, we must first send a message. What if we want to be a lurker? Or what if our conversation partner is much more chatty than us? On the other hand, if we're the chatty one then we'll barely see any messages from our partner since we'll mostly see our own echoed output.\n\nTo solve this problem we need to be able to `await` two futures at once. In this case those futures are the ones created by `stream.next()`, which gets the next message from the client, and `rx.recv()`, which gets the next message from the channel.\n\n## 08\\) Letting users actually chat\n\n`tokio::select!` allows us to poll multiple futures at once:\n\n```rust\nasync fn handle_user(\n mut tcp: TcpStream,\n tx: Sender\n) -> anyhow::Result<()> {\n // ...\n loop {\n tokio::select! {\n user_msg = stream.next() => {\n // ...\n },\n peer_msg = rx.recv() => {\n // ...\n },\n }\n }\n // ...\n}\n```\n\nWe execute the match arm of whatever future completes first. The other future is dropped.\n\nNow if we try our server:\n\n```console\n$ just telnet # concurrent client 1\n> 1: hello # msg 1\n1: hello ❤️\n> 1: anybody there? # msg 2\n1: anybody there? ❤️\n2: i am right here ❤️\n2: how are you ❤️\n> 1: i am doing great # msg 5\n\n$ just telnet # concurrent client 2\n1: hello ❤️\n1: anybody there? ❤️\n> 2: i am right here # msg 3\n2: i am right here ❤️\n> 2: how are you? # msg 4\n2: how are you ❤️\n1: i am doing great ❤️\n```\n\nIt works! Anyway, celebrations aside, we need to talk about cancel safety. As mentioned before, Rust futures are lazy, and they only make progress while being polled. Polling is a little bit different than awaiting. To await a future means to poll it to completion. To poll a future means to ask it to make some progress, but it may not necessarily complete.\n\nOn one hand, this is great, because if we start polling a future and later decide we don't need its result anymore, we can stop polling it and we won't waste anymore CPU on doing useless work. On the other hand, this may not be so great if the future we're cancelling is in the middle of an important operation that if not completed may drop important data or may leave data in a corrupt state.\n\nLet's look at an example of \"cancelling\" a future. Cancelling is in quotes because it's not an explicit operation, it just means we started to poll a future but then stopped polling it before it completed.\n\n```rust\nuse tokio::time::sleep;\nuse std::time::Duration;\n\nasync fn count_to(num: u8) {\n for i in 1..=num {\n sleep(Duration::from_millis(100)).await;\n println!(\"{i}\");\n }\n}\n\n#[tokio::main]\nasync fn main() {\n println!(\"start counting\");\n // the select! macro polls each\n // future until one of them completes,\n // and then we execute the match arm\n // of the completed future and drop\n // all of the other futures\n tokio::select! {\n _ = count_to(3) => {\n println!(\"counted to 3\");\n },\n _ = count_to(10) => {\n println!(\"counted to 10\");\n },\n };\n println!(\"stop counting\");\n // this sleep is here to demonstrate\n // that the count_to(10) doesn't make\n // any progress after we stop polling\n // it, even if we go to sleep and do\n // nothing else for a while\n sleep(Duration::from_millis(1000)).await;\n}\n```\n\nThis program outputs:\n\n```\nstart counting\n1\n1\n2\n2\n3\n3\ncounted to 3\nstop counting\n```\n\nWe \"cancelled\" the `count_to(10)` future. In this simple toy example we don't care if the count completes or not so this future is cancel-safe in that sense, but if finishing the count was critical to our application then cancelling this future would be a problem. To make sure the future completes we can `await` it after the `tokio::select!`:\n\n```rust\n// ...\nasync fn main() {\n println!(\"start counting\");\n let count_to_10 = count_to(10);\n tokio::select! {\n _ = count_to(3) => {\n println!(\"counted to 3\");\n },\n _ = count_to_10 => { // ❌\n println!(\"counted to 10\");\n },\n };\n println!(\"stop counting\");\n println!(\"jk, keep counting\");\n count_to_10.await; // ❌\n println!(\"finished counting to 10\");\n}\n```\n\nThrows:\n\n```\nerror[E0382]: use of moved value: count_to_10\n```\n\nOh duh, we made the simplest mistake in the book, trying to use a value after we moved it. Let's pass a mutable reference instead:\n\n```rust\n// ...\nasync fn main() {\n println!(\"start counting\");\n let count_to_10 = count_to(10);\n tokio::select! {\n _ = count_to(3) => {\n println!(\"counted to 3\");\n },\n _ = &mut count_to_10 => { // ❌\n println!(\"counted to 10\");\n },\n };\n println!(\"stop counting\");\n println!(\"jk, keep counting\");\n count_to_10.await;\n println!(\"counted to 10\");\n}\n```\n\nNow throws:\n\n```\nerror[E0277]: {async fn body@src/main.rs:23:28: 28:2}\n cannot be unpinned\n -> src/main.rs:34:5\n |\n23 | async fn count_to(num: u8) {\n | ----------------- within this impl futures::Future\n...\n34 | / tokio::select! {\n35 | | _ = count_to(3) => {\n36 | | println!(\"counted to 3\");\n37 | | },\n... \n41 | | };\n | | ^\n | | |\n | |_____within impl futures::Future,\n | the trait Unpin is not implemented for\n | {async fn body@src/main.rs:23:28: 28:2},\n | which is required by &mut impl\n | futures::Future: futures::Future\n | required by a bound introduced by this call\n |\n = note: consider using the pin! macro\n consider using Box::pin if you need to access\n the pinned value outside of the current scope\n```\n\nWe need to \"pin\" our future. Okay, let's do what the compiler suggested:\n\n```rust\n#[tokio::main]\nasync fn main() {\n println!(\"start counting\");\n let count_to_10 = count_to(10);\n tokio::pin!(count_to_10); // ✔️\n tokio::select! {\n _ = count_to(3) => {\n println!(\"counted to 3\");\n },\n _ = &mut count_to_10 => {\n println!(\"counted to 10\");\n },\n };\n println!(\"stop counting\");\n println!(\"jk, keep counting\");\n count_to_10.await;\n println!(\"finished counting to 10\");\n}\n```\n\nCompiles and outputs:\n\n```\nstart counting\n1\n1\n2\n2\n3\n3\ncounted to 3\nstop counting\njk, keep counting\n4\n5\n6\n7\n8\n9\n10\nfinished counting to 10\n```\n\nTo pin something in Rust means to pin its location in memory. Once it is pinned it cannot be moved. The reason some futures need to be pinned before being polled is because under-the-hood they can contain self-referential pointers that would be invalidated if the future was ever moved.\n\nIf that last part flew over your head don't worry, I don't fully get it either. But fear not, here's a general algorithm we can follow to solve these kinds of problems when they arise:\n\n**1\\)** If we're writing generic code that takes a future or something that produces futures we can add `+ Unpin` to the trait bounds. So for example this doesn't compile:\n\n```rust\nuse futures::{Stream, StreamExt};\n\nasync fn iterate(\n mut items: impl Stream- \n) {\n while let Some(item) = items.next().await { // ❌\n todo!()\n }\n}\n```\n\nThrows:\n\n```\nerror[E0277]: impl Stream
- cannot be unpinned\n```\n\nBut if we sprinkle `Unpin` into the function signature it works:\n\n```rust\nasync fn iterate
(\n mut items: impl Stream- + Unpin // ✔️\n) {\n while let Some(item) = items.next().await {\n todo!()\n }\n}\n```\n\n**2\\)** However, let's say that causes compile errors elsewhere in our code, because we are passing a stream to this function isn't `Unpin`. We can remove the `Unpin` from the function signature and use the `pin!` macro to pin the stream within the function:\n\n```rust\nasync fn iterate
(\n mut items: impl Stream- \n) {\n tokio::pin!(items); // ✔️\n while let Some(item) = items.next().await {\n todo!()\n }\n}\n```\n\nThis pins it to the stack, so it cannot escape the current scope.\n\n**3\\)** If the pinned object needs to escape the current scope there's `Box::pin` to pin it in the heap:\n\n```rust\nasync fn iterate
(\n mut items: impl Stream- \n) {\n let mut items = Box::pin(items); // ✔️\n while let Some(item) = items.next().await {\n todo!()\n }\n}\n```\n\n**4\\)** Or we can ask the caller to figure out this detail for us:\n\n```rust\nasync fn iterate
+ ?Sized>(\n mut items: Pin<&mut S>\n) {\n while let Some(item) = items.next().await {\n todo!()\n }\n}\n```\n\nHowever in this case the caller is also us, so this doesn't help that much over solutions 2 & 3.\n\n> [!IMPORTANT]\n> In summary: we need to be mindful which futures are and aren't cancel-safe when we pass them to code that may not poll them to completion, e.g. `tokio::select!`. If you're writing a Rust library that polls futures you need to document if your library will poll the futures to completion or not. If you're writing a Rust library that produces futures you need to document which of the futures are and aren't cancel-safe. If you're using a Rust library that either polls or returns futures you need to carefully read its docs.\n\n## 09\\) Assigning names to users\n\nIn the current iteration of our chat server it's hard to follow who said what. We could prepend each connection's socket address to their message to disambiguate them, and if we did it would look something like this:\n\n```console\n$ just telnet\n> hello\n127.0.0.1:51270: hello\n```\n\nHowever that is both ugly and boring. Let's generate random names by combining an adjective with an animal and assign them to users when they join:\n\n```rust\npub static ADJECTIVES: [&str; 628] = [\n \"Mushy\",\n \"Starry\",\n \"Peaceful\",\n \"Phony\",\n \"Amazing\",\n \"Queasy\",\n // ...\n];\n\npub static ANIMALS: [&str; 243] = [\n \"Owl\",\n \"Mantis\",\n \"Gopher\",\n \"Robin\",\n \"Vulture\",\n \"Prawn\",\n // ...\n];\n\npub fn random_name() -> String {\n let adjective = fastrand::choice(ADJECTIVES).unwrap();\n let animal = fastrand::choice(ANIMALS).unwrap();\n format!(\"{adjective}{animal}\")\n}\n```\n\nHere's a sampling of some of the names this creates:\n\n```\nHushedLlama\nDimpledDinosaur\nUrbanMongoose\nYawningMinotaur\nRomanticRhino\nDapperPeacock\nPlasticCentaur\nBubblyChicken\nAnxiousGriffin\nSpicyAlpaca\nMindlessOctopus\nWealthyPelican\nCruelCapybara\nRegalFrog\nPinkPoodle\nQuirkyGazelle\nPoshGopher\nCarelessBobcat\nSomberWeasel\nZenMammoth\nDazzlingSquid\n```\n\nTo keep our main file clean and concise, let's put this functionality into our lib file and import it:\n\n```rust\nuse chat_server::random_name;\n\n// ...\n\nasync fn handle_user(\n mut tcp: TcpStream,\n tx: Sender\n) -> anyhow::Result<()> {\n // ...\n // generate random name\n let name = random_name();\n // ...\n // tell user their name\n sink.send(format!(\"You are {name}\")).await?;\n // ...\n user_msg = stream.next() => {\n // ...\n // prepend user's name to their messages\n tx.send(format!(\"{name}: {user_msg}\"))?;\n },\n // ...\n}\n```\n\nLet's give it a try:\n\n```console\n$ just chat\nYou are MeatyPuma\n> hello\nMeatyPuma: hello\nPeacefulGibbon: howdy\n```\n\nExcellent.\n\n> [!NOTE]\n> I switched from using `just telnet` to `just chat` because I got tired of using telnet and built a TUI chat client that's easier to use and looks nicer, which is what `just chat` runs.\n\n## 10\\) Letting users edit their names with `/name`\n\nWe'd like names to be unique across the server. We can enforce this by maintaining the names in a `HashSet`. However, since we'd also like to let users edit their names using the `/name` command we need to share this set between users running in different threads.\n\n> [!TIP]\n> To share mutable data across many threads we can wrap it with `Arc>`, which is like the thread-safe version of `Rc>` if you've ever used that before.\n\nLet's wrap our `Arc>>` in a new type to make using it more ergonomic:\n\n```rust\n// ...\n\n#[derive(Clone)]\nstruct Names(Arc>>);\n\nimpl Names {\n fn new() -> Self {\n Self(Arc::new(Mutex::new(HashSet::new())))\n }\n // returns true if name was inserted,\n // i.e. the name is unique\n fn insert(&self, name: String) -> bool {\n self.0.lock().unwrap().insert(name)\n }\n // returns unique name\n fn get_unique(&self) -> String {\n let mut name = random_name();\n let mut guard = self.0.lock().unwrap();\n while !guard.insert(name.clone()) {\n name = random_name();\n }\n name\n }\n}\n\n#[tokio::main]\nasync fn main() -> anyhow::Result<()> {\n // ...\n let names = Names::new();\n // ...\n tokio::spawn(handle_user(tcp, tx.clone(), names.clone()));\n}\n\nasync fn handle_user(\n mut tcp: TcpStream,\n tx: Sender,\n names: Names,\n) -> anyhow::Result<()> {\n // ...\n // get a unique name for new user\n let mut name = names.get_unique();\n // ...\n // tell them their name\n sink.send(format!(\"You are {name}\")).await?;\n // ...\n user_msg = stream.next() => {\n // ...\n // handle new /name command\n if user_msg.starts_with(\"/name\") {\n let new_name = user_msg\n .split_ascii_whitespace()\n .nth(1)\n .unwrap()\n .to_owned();\n // check if name is unique\n let changed_name = names.insert(new_name.clone());\n if changed_name {\n // notify everyone that user\n // changed their name\n tx.send(format!(\"{name} is now {new_name}\"))?;\n // remove previous name\n names.remove(&name);\n // set new name\n name = new_name;\n } else {\n // tell user that name is\n // already taken\n sink.send(\n format!(\"{new_name} is already taken\")\n ).await?;\n }\n }\n // ...\n },\n // ...\n}\n```\n\nLet's try it out:\n\n```console\n$ just chat\nServer commands\n /help - prints this message\n /name {name} - change name\n /quit - quits server\nYou are FancyYak\n> hello\nFancyYak: hello\n> /name pretzelhammer # new command\nFancyYak is now pretzelhammer\n> 🦀🦀🦀\npretzelhammer: 🦀🦀🦀\n```\n\n> [!CAUTION]\n> Rust promises that compiling safe programs are free of memory vulnerabilities, but it makes no promises that they will be free of deadlocks. When we add locks to our program we need to be careful to avoid creating deadlock scenarios.\n\nHere's some tips for avoiding deadlocks:\n\n**1\\)** Don't hold locks across await points\n\nAn `await` point is anywhere in an async function where `await` is called. When we call `await` we yield control back to the Tokio scheduler. If our yielded future holds a lock that means any executing futures won't be able to acquire it, in which case they will block forever while waiting on it, and the yielded lock-holding future won't get an opportunity to run again, and so we have a deadlock.\n\nThat was a bit abstract so let's run through a concrete example. Imagine we have a single-threaded Tokio runtime, with three futures ready to be polled:\n\n```\nTokio scheduler, future queue:\n+---------+---------+---------+\n| fut A | fut B | fut C |\n+---------+---------+---------+\n```\n\nSince this is a single-threaded runtime we can only execute one future at a time. Tokio polls the first future, future A, which runs code that looks like this:\n\n```rust\nasync do_stuff(mutex: Mutex) {\n // acquires lock\n let guard = mutex.lock().unwrap();\n // hits await point, i.e. yields to scheduler\n other_async_fn().await?;\n // releases lock\n dbg!(guard);\n}\n```\n\nWhen it hits the `await` point, the future goes back to the end of the queue:\n\n```\nTokio scheduler, future queue:\n+---------+---------+---------+\n| fut B | fut C | fut A* |\n+---------+---------+---------+\n* holding lock\n```\n\nThen Tokio tries to poll the next future, future B, and that future runs through the same code path, trying to acquire a lock to the same mutex that future A is currently holding! It will block forever! Future B cannot make progress until future A releases the lock, but future A cannot release the lock until future B yields back to the scheduler. We have a deadlock.\n\n_\"But what if we use an async mutex instead of a sync mutex?\"_\n\nIt is true that tip 1 applies to sync mutexes, like `std::sync::Mutex`, but not to async mutexes, like `tokio::sync::Mutex`. For mutexes designed to be used in an async context, we can hold their locks across `await` points, however they are slower. To quote the Tokio docs:\n\n> Contrary to popular belief, it is ok and often preferred to use the ordinary Mutex from the standard library in asynchronous code.\n>\n> The feature that the async mutex offers over the blocking mutex is the ability to keep it locked across an await point. This makes the async mutex more expensive than the blocking mutex, so the blocking mutex should be preferred in the cases where it can be used. The primary use case for the async mutex is to provide shared mutable access to IO resources such as a database connection. If the value behind the mutex is just data, it's usually appropriate to use a blocking mutex such as the one in the standard library.\n\nGenerally speaking, if we can structure our code to never to hold a lock across an `await` point it's better to use a sync mutex, and if we absolutely have to hold a lock across an `await` point then we would switch to an async mutex.\n\n**2\\)** Don't reaquire the same lock multiple times\n\nTrivial example:\n\n```rust\nfn main() {\n let mut mutex = Mutex::new(5);\n // acquire lock\n let g = mutex.lock().unwrap();\n // try to acquire lock again\n mutex.lock().unwrap(); // deadlocks\n}\n```\n\nAlthough the mistake is super obvious in the example above when it happens in Real CodeTM it's much harder to spot and debug.\n\n_\"I won't have to worry about this if I'm using a read-write lock, right? Since those are suppose to be able to give out many read locks to concurrent threads.\"_\n\nSurprisingly no, even reacquiring a read lock on a read-write lock twice in the same thread can produce a deadlock. To borrow a diagram from the standard library `RwLock` docs:\n\n```\n// Thread 1 | // Thread 2\nlet _rg = lock.read(); |\n | // will block\n | let _wg = lock.write();\n// may deadlock |\nlet _rg = lock.read(); |\n```\n\nAnd to quote the `RwLock` docs from the `parking_lot` crate:\n\n> This lock uses a task-fair locking policy which avoids both reader and writer starvation. This means that readers trying to acquire the lock will block even if the lock is unlocked when there are writers waiting to acquire the lock. Because of this, attempts to recursively acquire a read lock within a single thread may result in a deadlock.\n\nI guess we just have to be really careful 🤷\n\n**3\\)** Acquire locks in the same order everywhere\n\nIf we need to acquire multiple locks to safely perform some operation, we need to always acquire those locks in the same order, otherwise deadlocks can trivially occur, like here:\n\n```\n// Thread 1 | // Thread 2\nlet _a = a.lock(); | let _b = b.lock();\n// ... | // ...\nlet _b = b.lock(); | let _a = a.lock();\n```\n\n_\"My head is spinning from all of these gotchas. Surely there has to be an easier or better way to do all of this locking stuff?\"_\n\n**4\\)** Use lockfree data structures\n\nDoing this allows you to disregard tips 1-3, since lockfree data structures cannot deadlock. However, in general, lockfree data structures are slower than most of their lock-based counterparts.\n\n**5\\)** Use channels for everything\n\nDoing this also allows you to disregard tips 1-3, since channels cannot deadlock. I'm not well-read enough on this subject to comment on whether using channels for everything can degrade or improve the performance of a concurrent program vs using locks. I imagine the answer, like the answer to most computer science questions, is _\"it depends.\"_\n\nThis approach is also sometimes called the \"actor pattern\" and if you search for \"actor\" on cargo you'll find a lot of actor framework crates that supposedly help with structuring your program to follow this pattern.\n\nAnyway, that was long detour. Let's get back to our chat server.\n\n## 11\\) Freeing user's name if they disconnect\n\nWe have a bug in our code. Names are not removed from the set when a user disconnects, so after a name is taken it can never be used again, not until we restart the server. Unfortunately there's a tricky obstacle we have to overcome before we can fix this.\n\nThe obstacle is the user may disconnect due to an error, and we're using `?` everywhere in our `handle_user` function, which propagates the error up to `main`, but it shouldn't be `main`'s responsibility to clean up names, that's a detail that should remain within `handle_user`. We can get rid of `?` and pattern match on `Result`s everywhere but that's verbose and ugly. So what do we do when we have to get rid of a bunch of repetitive, ugly, verbose code? We use macros.\n\nAs a quick recap, remember that all blocks in Rust are expressions, and we can `break` out of a block with a value. A couple examples:\n\n```rust\nfn main() {\n // breaking from loop with value\n let value = loop {\n break \"value\";\n };\n assert_eq!(value, \"value\");\n\n // to break from a non-loop block\n // it needs to be labelled\n let value = 'label: {\n break 'label \"value\";\n };\n assert_eq!(value, \"value\");\n}\n```\n\nAlso, the `?` operator is not magic and can be implemented as a macro:\n\n```rust\nmacro_rules! question_mark {\n ($result:expr) => {\n match $result {\n Ok(ok) => ok,\n Err(err) => return Err(err.into()),\n }\n }\n}\n```\n\nWhich is everything we want, except the `return` should be a `break`, since we'd like to handle the errors within our function and not propagate them to the caller. So let's write a new macro and call it `b!` which is short for `break`:\n\n```rust\nmacro_rules! b {\n ($result:expr) => {\n match $result {\n Ok(ok) => ok,\n Err(err) => break Err(err.into()),\n }\n }\n}\n```\n\nAnd then we can refactor a function that propagates errors to its caller like this:\n\n```rust\nfn some_function() -> anyhow::Result<()> {\n // initialize state here\n loop {\n fallible_statement_1?;\n fallible_statement_2?;\n // etc\n }\n // clean up state here, but\n // this may never be reached\n // because the ? returns from\n // the function instead of\n // breaking from the loop\n Ok(())\n}\n```\n\nInto a function which catches and handles its own errors:\n\n```rust\nfn some_function() {\n // initialize state here\n let result = loop {\n b!(fallible_statement_1);\n b!(fallible_statement_2);\n // etc\n };\n // clean up state here, always reached\n if let Err(err) = result {\n // handle errors if necessary\n }\n // nothing to return anymore since\n // we take care of everything within\n // the function :)\n}\n```\n\nSo with all of that context out of the way, here's the updated code:\n\n```rust\n// ...\n\nasync fn handle_user(\n mut tcp: TcpStream,\n tx: Sender,\n names: Names,\n) -> anyhow::Result<()> {\n // ...\n // we now catch errors here\n let result: anyhow::Result<()> = loop {\n // all fallible statements\n // from before are now wrapped\n // with our b!() macro\n };\n // the line below is always reached\n // and the user's name is always freed,\n // regardless if they quit normally or\n // abruptly disconnected due to an error\n names.remove(&name);\n // return result to caller if they want\n // to do anything extra\n result\n}\n```\n\nNow if a user disconnects for any reason we will always reclaim their name.\n\n## 12\\) Adding a main room\n\nRight now all users are dumped into the same room and have nowhere else to go. It will be difficult to keep the conversation on one topic and hard to follow the discussion if multiple side-conversations start happening concurrently. We should add the ability to create and join different rooms within our server. As a first step let's refactor our current code to add everyone who joins the server into a default room, called `main`. The updated code:\n\n```rust\n// ...\n\nstruct Room {\n tx: Sender,\n}\n\nimpl Room {\n fn new() -> Self {\n let (tx, _) = broadcast::channel(32);\n Self {\n tx,\n }\n }\n}\n\nconst MAIN: &str = \"main\";\n\n#[derive(Clone)]\nstruct Rooms(Arc>>);\n\nimpl Rooms {\n fn new() -> Self {\n Self(Arc::new(RwLock::new(HashMap::new())))\n }\n fn join(&self, room_name: &str) -> Sender {\n // get read access\n let read_guard = self.0.read().unwrap();\n // check if room already exists\n if let Some(room) = read_guard.get(room_name) {\n return room.tx.clone();\n }\n // must drop read before acquiring write\n drop(read_guard);\n // create room if it doesn't yet exist\n // get write access\n let mut write_guard = self.0.write().unwrap();\n let room = write_guard\n .entry(room_name.to_owned())\n .or_insert(Room::new());\n room.tx.clone()\n }\n}\n\n#[tokio::main]\nasync fn main() -> anyhow::Result<()> {\n // ...\n let rooms = Rooms::new();\n // ...\n tokio::spawn(handle_user(tcp, names.clone(), rooms.clone()));\n}\n\nasync fn handle_user(\n mut tcp: TcpStream,\n names: Names,\n rooms: Rooms,\n) -> anyhow::Result<()> {\n // ...\n // when user connects to server\n // automatically put them in\n // the main room\n let room_name = MAIN.to_owned();\n let room_tx = rooms.join(&room_name);\n let mut room_rx = room_tx.subscribe();\n // notify everyone in room that\n // a new user has joined\n let _ = room_tx.send(format!(\"{name} joined {room_name}\"));\n // ...\n tokio::select! {\n user_msg = stream.next() => {\n // ...\n // send messages to the room\n // we're currently in\n b!(room_tx.send(format!(\"{name}: {user_msg}\")));\n },\n // receive messages from the\n // room we're currently in\n peer_msg = room_rx.recv() => {\n // ...\n },\n }\n // ...\n // notify everyone in room that\n // we have left\n let _ = room_tx.send(format!(\"{name} left {room_name}\"));\n // ...\n}\n```\n\nA `Room` is a wrapper around a `broadcast::Sender`, and `Rooms` is a wrapper around an `Arc>>` because we need to maintain a map of room names to broadcast channels and we'd like to share and modify this map across many threads.\n\nWe also added notification messages for when a user joins and leaves a room. Let's see what it looks like altogether:\n\n```console\n$ just chat\nYou are AwesomeVulture\nAwesomeVulture joined main\nJealousHornet joined main\nJealousHornet: we are at the main room!\n> can we create other rooms?\nAwesomeVulture: can we create other rooms?\nJealousHornet: not yet, back to work we go\nJealousHornet left main\n```\n\n## 13\\) Letting users join or create rooms with `/join`\n\nSince we implemented the `join` method earlier this will be pretty easy:\n\n```rust\n// ...\nasync fn handle_user(\n mut tcp: TcpStream,\n names: Names,\n rooms: Rooms,\n) -> anyhow::Result<()> {\n // ...\n // automatically join main room\n // on connect, as before\n let mut room_name = MAIN.to_owned();\n let mut room_tx = rooms.join(&room_name);\n let mut room_rx = room_tx.subscribe();\n // ...\n if user_msg.starts_with(\"/join\") {\n let new_room = user_msg\n .split_ascii_whitespace()\n .nth(1)\n .unwrap()\n .to_owned();\n // check if user is already in the room\n // they're trying to join\n if new_room == room_name {\n b!(sink.send(format!(\"You are in {room_name}\")).await);\n continue;\n }\n // notify current room that we've left\n b!(room_tx.send(format!(\"{name} left {room_name}\")));\n // join new room, this creates\n // the room if it doesn't\n // already exist\n room_tx = rooms.join(&new_room);\n room_rx = room_tx.subscribe();\n room_name = new_room;\n // notify new room that we have joined\n b!(room_tx.send(format!(\"{name} joined {room_name}\")));\n }\n // ...\n // notify our current room that we've left\n // on disconnect, as before\n let _ = room_tx.send(format!(\"{name} left {room_name}\"));\n // ...\n}\n```\n\nNow we can start pizza parties:\n\n```console\n$ just chat\nServer commands\n /help - prints this message\n /name {name} - change name\n /join {room} - joins room\n /quit - quits server\nYou are ElasticBonobo\nElasticBonobo joined main\nBlondCyclops joined main\n> /join pizza # new command\nElasticBonobo joined pizza\nBlondCyclops joined pizza\n> let's have a pizza party\nElasticBonobo: let's have a pizza party\nBlondCyclops: 🍕🥳\n```\n\n## 14\\) Listing all rooms with `/rooms`\n\nRight now pizza parties on the server are not very discoverable. If a user lands in the `main` room there's no way for them to know all the other users on the server are in `pizza`. Let's add a `/rooms` command that will list all of the rooms on server:\n\n```rust\n// ...\n\n#[derive(Clone)]\nstruct Rooms(Arc>>);\n\nimpl Rooms {\n fn list(&self) -> Vec<(String, usize)> {\n // iterate over rooms map\n let mut list: Vec<_> = self\n .0\n .read()\n .unwrap()\n .iter()\n // receiver_count tells us\n // the # of users in the room\n .map(|(name, room)| (\n name.to_owned(),\n room.tx.receiver_count(),\n ))\n .collect();\n list.sort_by(|a, b| {\n use std::cmp::Ordering::*;\n // sort rooms by # of users first\n match b.1.cmp(&a.1) {\n // and by alphabetical order second\n Equal => a.0.cmp(&b.0),\n ordering => ordering,\n }\n });\n list\n }\n}\n\n// ...\n\nasync fn handle_user(\n mut tcp: TcpStream,\n names: Names,\n rooms: Rooms,\n) -> anyhow::Result<()> {\n // ...\n // handle new /rooms command\n if user_msg.starts_with(\"/rooms\") {\n let rooms_list = rooms.list();\n let rooms_list = rooms_list\n .into_iter()\n .map(|(name, count)| format!(\"{name} ({count})\"))\n .collect::>()\n .join(\", \");\n b!(sink.send(format!(\"Rooms - {rooms_list}\")).await);\n }\n // ...\n}\n```\n\nNow everyone is invited to our pizza parties:\n\n```console\n$ just chat\nServer commands\n /help - prints this message\n /name {name} - change name\n /rooms - list rooms\n /join {room} - joins room\n /quit - quits server\nYou are SilentYeti\nSilentYeti joined main\n> /rooms # new command\nRooms - pizza (2), main (1)\n> /join pizza\nSilentYeti joined pizza\n> can i be part of this pizza party? 🥺\nSilentYeti: can i be part of this pizza party? 🥺\nBulkyApe: of course ❤️\nAmazingDragon: 🔥🔥🔥\n```\n\n## 15\\) Removing empty rooms\n\nWe have a bug, after a room is created it's never deleted, even after it becomes empty. After a while our server rooms list will look like this:\n\n```console\n> /rooms\nRooms - a (0), bunch (0), of (0), abandoned (0), rooms (0)\n```\n\nLet's fix that:\n\n```rust\n// ...\n\n#[derive(Clone)]\nstruct Rooms(Arc>>);\n\nimpl Rooms {\n // ...\n fn leave(&self, room_name: &str) {\n let read_guard = self.0.read().unwrap();\n let mut delete_room = false;\n if let Some(room) = read_guard.get(room_name) {\n // if the receiver count is 1 then\n // we're the last person in the room\n // and can remove it\n delete_room = room.tx.receiver_count() <= 1;\n }\n drop(read_guard);\n if delete_room {\n let mut write_guard = self.0.write().unwrap();\n write_guard.remove(room_name);\n }\n }\n fn change(\n &self,\n prev_room: &str,\n next_room: &str\n ) -> Sender {\n self.leave(prev_room);\n self.join(next_room)\n }\n // ...\n}\n\nasync fn handle_user(\n mut tcp: TcpStream,\n names: Names,\n rooms: Rooms,\n) -> anyhow::Result<()> {\n // ...\n if user_msg.starts_with(\"/join\") {\n // ...\n // now correctly deletes the room\n // we're leaving if it becomes empty\n room_tx = rooms.change(&room_name, &new_room);\n // ...\n }\n // ...\n // when we disconnect we also\n // need to leave and delete the\n // room if it's empty\n rooms.leave(&room_name);\n // ...\n}\n```\n\n## 16\\) Listing users in the room with `/users`\n\nUsers within a room are not discoverable. Let's add a `/users` command that will list the users in the current room. To do that we'll have to add a `HashSet` to the `Room` struct and update many of the `Rooms` methods to also take a user name when joining, changing, or leaving a room:\n\n```rust\n// ...\n\nstruct Room {\n // ...\n // keep track of the names\n // of the users in the room\n users: HashSet,\n}\n\nimpl Room {\n fn new() -> Self {\n // ...\n let users = HashSet::new();\n Self {\n // ...\n users,\n }\n }\n}\n\n#[derive(Clone)]\nstruct Rooms(Arc>>);\n\nimpl Rooms {\n // ...\n fn join(&self, room_name: &str, user_name: &str) -> Sender {\n // ...\n room.users.insert(user_name.to_owned());\n // ...\n }\n fn leave(&self, room_name: &str, user_name: &str) {\n // ...\n room.users.remove(user_name);\n // ...\n }\n // update user's name in room if they\n // changed it using the /name command\n fn change_name(\n &self,\n room_name: &str,\n prev_name: &str,\n new_name: &str\n ) {\n let mut write_guard = self.0.write().unwrap();\n if let Some(room) = write_guard.get_mut(room_name) {\n room.users.remove(prev_name);\n room.users.insert(new_name.to_owned());\n }\n }\n // returns list of users' names in the room\n fn list_users(&self, room_name: &str) -> Option> {\n self\n .0\n .read()\n .unwrap()\n .get(room_name)\n .map(|room| {\n // get users in room\n let mut users = room\n .users\n .iter()\n .cloned()\n .collect::>();\n // alphabetically sort\n // users by names\n users.sort();\n users\n })\n }\n}\n\n// ...\n\nasync fn handle_user(\n mut tcp: TcpStream,\n names: Names,\n rooms: Rooms,\n) -> anyhow::Result<()> {\n // ...\n // send our name when joining a room\n room_tx = rooms.join(&room_name, &name);\n // ...\n if user_msg.starts_with(\"/name\") {\n // ...\n if changed_name {\n // let room know we changed our name\n rooms.change_name(&room_name, &name, &new_name);\n // ...\n }\n // ...\n } else if user_msg.starts_with(\"/join\") {\n // ...\n // send our name when changing rooms\n room_tx = rooms.change(&room_name, &new_room, &name);\n // ...\n // handle new /users command\n } else if user_msg.starts_with(\"/users\") {\n let users_list = rooms\n .list_users(&room_name)\n .unwrap()\n .join(\", \");\n b!(sink.send(format!(\"Users - {users_list}\")).await);\n }\n // ...\n rooms.leave(&room_name, &name);\n // ...\n}\n```\n\nNow we can find our friends within rooms:\n\n```console\n$ just chat\nServer commands\n /help - prints this message\n /name {name} - change name\n /rooms - list rooms\n /join {room} - joins room\n /users - lists users in current room\n /quit - quits server\nYou are StarryDolphin\nStarryDolphin joined main\n> /users # new command\nUsers - ColorfulSheep, PaleHedgehog, StarryDolphin\n> hey colorful sheep! 👋\nStarryDolphin: hey colorful sheep! 👋\nColorfulSheep: good to see you again starry dolphin! 🙌\n```\n\n## 17\\) Optimizing performance\n\nWe're gonna bucket our performance optimizations into three categories: reducing heap allocations, reducing lock contention, and compiling for speed.\n\nThere's other categories, but I think these are the most relevant for our particular program.\n\n### Reducing heap allocations\n\nThe fastest code is the code that never runs. If we don't have to allocate something on the heap then we don't have to call the allocator.\n\n#### `String` -> `CompactString`\n\nWe have a lot of short strings. We can have thousands of users and hundreds of rooms on the server and user names and room names are both almost always shorter than 24 characters. Instead of storing them as `String`s we can store them as `CompactString`s. `String`s always store their data on the heap, but `CompactString`s will store strings shorter than 24 bytes on the stack, and will only heap allocate the string if it's longer than 24 bytes. If we enforce a maximum length of 24 ASCII characters for user and room names then we can guarantee we'll never have to perform any heap allocations for them.\n\n#### `Sender` -> `Sender>`\n\nAs you may remember, when we `send` something to a broadcast channel every call to `recv` clones that data. That means if a user sends a `String` message that is five paragraphs long in a room with 1000 other users we're going to have to clone that message 1000 times which means doing 1000 heap allocations. We know that after a message is sent it's immutable, so we don't need to send a `String`, we can convert the `String` to an `Arc` instead and send that, because cloning an `Arc` is very cheap since all it does is increment an atomic counter.\n\n#### Miscellaneous micro optimizations\n\nAfter combing through the code I found a couple places where we were carelessly allocating unnecessary `Vec`s and `String`s, mostly for the `/rooms` and `/users` commands, and made them both only allocate a single `String` when generating a response.\n\n### Reducing lock contention\n\nHigh lock contention increases how long threads need to wait for a lock to be free. Reducing lock contention reduces thread waiting time and increases program throughput.\n\n#### `Mutex>` -> `DashSet`\n\nWe store names in a `Mutex>`. That's one lock for potentially thousands of keys. Instead of putting a lock around the entire set, what if we could put a lock around every individual key in the set? A `DashSet` doesn't necessarily go that far, but it does split the data into several shards internally and each shard gets its own lock. Some ASCII diagrams to help explain:\n\n```\n+-------------------------------+\n| Mutex |\n| +---------------------------+ |\n| | HashSet | |\n| | +-----+-----+-----+-----+ | |\n| | | key | key | key | key | | |\n| | +-----+-----+-----+-----+ | |\n| +---------------------------+ |\n+-------------------------------+\n\n+-----------------------------------+\n| DashSet |\n| +---------------+---------------+ |\n| | RwLock | RwLock | |\n| | +-----+-----+ | +-----+-----+ | |\n| | | key | key | | | key | key | | |\n| | +-----+-----+ | +-----+-----+ | |\n| +-------------------+-----------+ |\n+-----------------------------------+\n```\n\nIn this case guarding the same amount of data but with more locks means each lock will get less contention between threads.\n\n#### `RwLock>` -> `DashMap`\n\nWe store rooms in a `RwLock>` but for the same reasons mentioned above we can use a `DashMap` to reduce lock contention.\n\n#### Better random name generation\n\nThere's a big issue in our random name generation. It's a [Birthday Problem](https://en.wikipedia.org/wiki/Birthday_problem) just in different clothes. Even if we have 600 unique adjectives and 250 unique animals and we can generate 150k unique names using those, we'd expect the probability of a collision in the first 1k generated names to be very low, right? Unfortunately no, after generating just 460 names there's already a >50% chance that there will be a collision, and after generating just 1k names the probability of a collision is >96%. More collisions means more time threads will spend fighting to find a unique name for every user that joins the server as the number of active users on the server climbs.\n\nI refactored the name generation to iterate through all possible name combinations in a pseudo-random fashion, so the generated names still appear random but now we can guarantee that for 600 unique adjectives and 250 unique animals that we will generate 150k unique names in a row without any collisions.\n\n#### System allocator -> jemalloc\n\njemalloc is suppose to be faster for multithreaded programs because it uses per-thread arenas which reduces contention for memory allocation. That sounds good to me, so let's change our allocator to jemalloc.\n\n### Compiling for speed\n\nThe default cargo `build` command is configured to quickly compile slow programs. Instead, we would like to slowly compile a fast program. In order to do that we need to add this to our Cargo.toml:\n\n```toml\n[profile.release]\ncodegen-units = 1\nlto = \"fat\"\n```\n\nAnd then execute the `build` command with these flags:\n\n```console\n$ RUSTFLAGS=\"-C target-cpu=native\" cargo build --release\n```\n\nAnyway, this section was a lot. You can see the full source code [here](https://github.com/pretzelhammer/chat-server/blob/main/examples/server-17.rs), and you can see a diff between the non-optimized and optimized versions by running `just diff 16 17`.\n\n## 18\\) Finishing touches\n\nWe've neglected logging, parsing command line arguments, and error handling thus far because they're boring and most people don't like reading about them. Let's speed through them.\n\nHere's how we can setup `tracing` to log to `stdout`:\n\n```rust\nuse std::io;\nuse tracing_subscriber::{fmt, EnvFilter, layer::SubscriberExt};\n\nfn setup_logging() {\n let subscriber = tracing_subscriber::registry()\n .with(EnvFilter::from_default_env())\n .with(fmt::Layer::new()\n .without_time()\n .compact()\n .with_ansi(true)\n .with_writer(io::stdout)\n );\n tracing::subscriber::set_global_default(subscriber)\n .expect(\"Unable to set a global subscriber\");\n}\n```\n\nAnd then after running `setup_logging` somewhere early in our main function we can call the `trace!`, `debug!`, `info!`, `warn!`, and `error!` macros from `tracing`, which all function similarly to `println!`. We also can customize the logging level via the environment, using the `RUST_LOG` environment variable.\n\nRight now our server always runs at `127.0.0.1` on port `42069`. We should let server admins be able to customize this without having to recompile our code. We can accept these parameters as command line arguments and parse them using the `clap` crate:\n\n```rust\nuse std::net::{IpAddr, SocketAddr, Ipv4Addr};\nuse clap::Parser;\n\nconst DEFAULT_IP: IpAddr = IpAddr::V4(Ipv4Addr::new(127, 0, 0, 1));\nconst DEFAULT_PORT: u16 = 42069;\n\n#[derive(Parser)]\n#[command(long_about = None)]\nstruct Cli {\n #[arg(short, long, default_value_t = DEFAULT_IP)]\n ip: IpAddr,\n\n #[arg(short, long, default_value_t = DEFAULT_PORT)]\n port: u16,\n}\n\nfn parse_socket_addr() -> SocketAddr {\n let cli = Cli::parse();\n SocketAddr::new(cli.ip, cli.port)\n}\n```\n\nFor error handling there's a bunch of little mundane things that we neglected, none of which are particularly interesting to write about.\n\nYou can see the full source code with all logging and error handling [here](https://github.com/pretzelhammer/chat-server/blob/main/examples/server-18.rs). To see a diff against the previous version of the code run `just diff 17 18`.\n\n## Conclusion\n\nWe learned a lot! The final full code for the server is [here](https://github.com/pretzelhammer/chat-server/blob/main/src/bin/chat-server.rs). You can run it with `just server`. To chat run `just chat`. And if it gets lonely run `just bots`.\n\n## Discuss\n\nDiscuss this article on\n- [Github](https://github.com/pretzelhammer/rust-blog/discussions/75)\n- [official Rust users forum](https://users.rust-lang.org/t/beginners-guide-to-concurrent-programming-coding-a-multithreaded-chat-server-using-tokio/110976)\n- [learnrust subreddit](https://www.reddit.com/r/learnrust/comments/1cmebbo/beginners_guide_to_concurrent_programming_coding/)\n- [rust subreddit](https://www.reddit.com/r/rust/comments/1cnz9p9/beginners_guide_to_concurrent_programming_coding/)\n\n## Further reading\n\n- [Common Rust Lifetime Misconceptions](./common-rust-lifetime-misconceptions.md)\n- [Tour of Rust's Standard Library Traits](./tour-of-rusts-standard-library-traits.md)\n- [Learning Rust in 2024](./learning-rust-in-2024.md)\n- [Using Rust in Non-Rust Servers to Improve Performance](./rust-in-non-rust-servers.md)\n- [Sizedness in Rust](./sizedness-in-rust.md)\n- [RESTful API in Sync & Async Rust](./restful-api-in-sync-and-async-rust.md)\n- [Learn Assembly with Entirely Too Many Brainfuck Compilers](./too-many-brainfuck-compilers.md)\n\n\n\n## Notifications\n\nGet notified when a new blog post gets published by\n- Subscribing to this repo's [releases RSS feed](https://github.com/pretzelhammer/rust-blog/releases.atom) or\n- Watching this repo's releases (click `Watch` → click `Custom` → select `Releases` → click `Apply`)\n"
},
{
"path": "posts/common-rust-lifetime-misconceptions.md",
"content": "# Common Rust Lifetime Misconceptions\n\n_19 May 2020 · #rust · #lifetimes_\n\n**Table of Contents**\n- [Intro](#intro)\n- [The Misconceptions](#the-misconceptions)\n - [1) `T` only contains owned types](#1-t-only-contains-owned-types)\n - [2) if `T: 'static` then `T` must be valid for the entire program](#2-if-t-static-then-t-must-be-valid-for-the-entire-program)\n - [3) `&'a T` and `T: 'a` are the same thing](#3-a-t-and-t-a-are-the-same-thing)\n - [4) my code isn't generic and doesn't have lifetimes](#4-my-code-isnt-generic-and-doesnt-have-lifetimes)\n - [5) if it compiles then my lifetime annotations are correct](#5-if-it-compiles-then-my-lifetime-annotations-are-correct)\n - [6) boxed trait objects don't have lifetimes](#6-boxed-trait-objects-dont-have-lifetimes)\n - [7) compiler error messages will tell me how to fix my program](#7-compiler-error-messages-will-tell-me-how-to-fix-my-program)\n - [8) lifetimes can grow and shrink at run-time](#8-lifetimes-can-grow-and-shrink-at-run-time)\n - [9) downgrading mut refs to shared refs is safe](#9-downgrading-mut-refs-to-shared-refs-is-safe)\n - [10) closures follow the same lifetime elision rules as functions](#10-closures-follow-the-same-lifetime-elision-rules-as-functions)\n- [Conclusion](#conclusion)\n- [Discuss](#discuss)\n- [Further Reading](#further-reading)\n- [Notifications](#notifications)\n\n\n\n## Intro\n\nI've held all of these misconceptions at some point and I see many beginners struggle with these misconceptions today. Some of my terminology might be non-standard, so here's a table of shorthand phrases I use and what I intend for them to mean.\n\n| Phrase | Shorthand for |\n|-|-|\n| `T` | 1) a set containing all possible types _or_ Trait for T {}\nimpl Trait for &T {} // ❌\nimpl Trait for &mut T {} // ❌\n```\n\nThe above program doesn't compile as expected:\n\n```none\nerror[E0119]: conflicting implementations of trait `Trait` for type `&_`\n --> src/lib.rs:3:1\n |\n2 | impl Trait for T {}\n | ------------------- first implementation here\n3 | impl Trait for &T {}\n | ^^^^^^^^^^^^^^^^^^^^ conflicting implementation for `&_`\n\nerror[E0119]: conflicting implementations of trait `Trait` for type `&mut _`\n --> src/lib.rs:4:1\n |\n2 | impl Trait for T {}\n | ------------------- first implementation here\n3 | impl Trait for &T {}\n4 | impl Trait for &mut T {}\n | ^^^^^^^^^^^^^^^^^^^^^^^^ conflicting implementation for `&mut _`\n```\n\nThe compiler doesn't allow us to define an implementation of `Trait` for `&T` and `&mut T` since it would conflict with the implementation of `Trait` for `T` which already includes all of `&T` and `&mut T`. The program below compiles as expected, since `&T` and `&mut T` are disjoint:\n\n```rust\ntrait Trait {}\nimpl Trait for &T {} // ✅\nimpl Trait for &mut T {} // ✅\n```\n\nAlthough it could probably go without saying, but for the sake of making sure nobody erroneously extrapolates the last couple examples, implementations for concrete types cannot overlap and this compiles just fine:\n\n```rust\ntrait Trait {}\nstruct Struct;\nimpl Trait for Struct {} // ✅\nimpl Trait for &Struct {} // ✅\nimpl Trait for &mut Struct {} // ✅\n```\n\n**Key Takeaways**\n- `T` is a superset of both `&T` and `&mut T`\n- `&T` and `&mut T` are disjoint sets\n\n\n### 2) if `T: 'static` then `T` must be valid for the entire program\n\n**Misconception Corollaries**\n- `T: 'static` should be read as _\"`T` has a `'static` lifetime\"_\n- `&'static T` and `T: 'static` are the same thing\n- if `T: 'static` then `T` must be immutable\n- if `T: 'static` then `T` can only be created at compile time\n\nMost Rust beginners get introduced to the `'static` lifetime for the first time in a code example that looks something like this:\n\n```rust\nfn main() {\n let str_literal: &'static str = \"str literal\";\n}\n```\n\nThey get told that `\"str literal\"` is hardcoded into the compiled binary and is loaded into read-only memory at run-time so it's immutable and valid for the entire program and that's what makes it `'static`. These concepts are further reinforced by the rules surrounding defining `static` variables using the `static` keyword.\n\n```rust\n// Note: This example is purely for illustrative purposes.\n// Never use `static mut`. It's a footgun. There are\n// safe patterns for global mutable singletons in Rust but\n// those are outside the scope of this article.\n\nstatic BYTES: [u8; 3] = [1, 2, 3];\nstatic mut MUT_BYTES: [u8; 3] = [1, 2, 3];\n\nfn main() {\n MUT_BYTES[0] = 99; // ❌ - mutating static is unsafe\n\n unsafe {\n MUT_BYTES[0] = 99;\n assert_eq!(99, MUT_BYTES[0]);\n }\n}\n```\n\nRegarding `static` variables\n- they can only be created at compile-time\n- they should be immutable, mutating them is unsafe\n- they're valid for the entire program\n\nThe `'static` lifetime was probably named after the default lifetime of `static` variables, right? So it makes sense that the `'static` lifetime has to follow all the same rules, right?\n\nWell yes, but a type _with_ a `'static` lifetime is different from a type _bounded by_ a `'static` lifetime. The latter can be dynamically allocated at run-time, can be safely and freely mutated, can be dropped, and can live for arbitrary durations.\n\nIt's important at this point to distinguish `&'static T` from `T: 'static`.\n\n`&'static T` is an immutable reference to some `T` that can be safely held indefinitely long, including up until the end of the program. This is only possible if `T` itself is immutable and does not move _after the reference was created_. `T` does not need to be created at compile-time. It's possible to generate random dynamically allocated data at run-time and return `'static` references to it via a memory leak, e.g.\n\n```rust\nuse rand;\n\n// generate random 'static str refs at run-time\nfn rand_str_generator() -> &'static str {\n let rand_string = rand::random::().to_string();\n Box::leak(rand_string.into_boxed_str())\n}\n```\n\n`T: 'static` is some `T` that can be safely held indefinitely long, including up until the end of the program. `T: 'static` includes all `&'static T` however it also includes all owned types, like `String`, `Vec`, etc. The owner of some data is guaranteed that data will never get invalidated as long as the owner holds onto it, therefore the owner can safely hold onto the data indefinitely long, including up until the end of the program. `T: 'static` should be read as _\"`T` can live at least as long as a `'static` lifetime\"_ not _\"`T` has a `'static` lifetime\"_. A program to help illustrate these concepts:\n\n```rust\nuse rand;\n\nfn drop_static(t: T) {\n std::mem::drop(t);\n}\n\nfn main() {\n let mut strings: Vec = Vec::new();\n for _ in 0..10 {\n if rand::random() {\n // all the strings are randomly generated\n // and dynamically allocated at run-time\n let string = rand::random::().to_string();\n strings.push(string);\n }\n }\n\n // strings are owned types so they can\n // live at least as long as 'static\n for mut string in strings {\n // all the strings are mutable\n string.push_str(\"a mutation\");\n // all the strings are droppable\n drop_static(string); // ✅\n }\n\n // all the strings have been invalidated before the end of the program\n println!(\"I am the end of the program\");\n}\n```\n\n**Key Takeaways**\n- `T: 'static` should be read as _\"`T` can live at least as long as a `'static` lifetime\"_\n- if `T: 'static` then `T` can be a borrowed type with a `'static` lifetime _or_ an owned type\n- since `T: 'static` includes owned types that means `T`\n - can be dynamically allocated at run-time\n - does not have to be valid for the entire program\n - can be safely and freely mutated\n - can be dynamically dropped at run-time\n - can have lifetimes of different durations\n\n\n\n### 3) `&'a T` and `T: 'a` are the same thing\n\nThis misconception is a generalized version of the one above.\n\n`&'a T` requires and implies `T: 'a` since a reference to `T` of lifetime `'a` cannot be valid for `'a` if `T` itself is not valid for `'a`. For example, the Rust compiler will never allow the construction of the type `&'static Ref<'a, T>` because if `Ref` is only valid for `'a` we can't make a `'static` reference to it.\n\n`T: 'a` includes all `&'a T` but the reverse is not true.\n\n```rust\n// only takes ref types that can outlive 'a\nfn t_ref<'a, T: 'a>(t: &'a T) {}\n\n// takes any types that can outlive 'a\nfn t_bound<'a, T: 'a>(t: T) {}\n\n// owned type which contains a reference\nstruct Ref<'a, T: 'a>(&'a T);\n\nfn main() {\n let string = String::from(\"string\");\n\n t_bound(&string); // ✅\n t_bound(Ref(&string)); // ✅\n t_bound(&Ref(&string)); // ✅\n\n t_ref(&string); // ✅\n t_ref(Ref(&string)); // ❌ - expected ref, found struct\n t_ref(&Ref(&string)); // ✅\n\n // string can outlive 'static which is longer than 'a\n t_bound(string); // ✅\n}\n```\n\n**Key Takeaways**\n- `T: 'a` is more general and more flexible than `&'a T`\n- `T: 'a` accepts owned types, owned types which contain references, and references\n- `&'a T` only accepts references\n- if `T: 'static` then `T: 'a` since `'static` >= `'a` for all `'a`\n\n\n\n### 4) my code isn't generic and doesn't have lifetimes\n\n**Misconception Corollaries**\n- it's possible to avoid using generics and lifetimes\n\nThis comforting misconception is kept alive thanks to Rust's lifetime elision rules, which allow you to omit lifetime annotations in functions because the Rust borrow checker will infer them following these rules:\n- every input ref to a function gets a distinct lifetime\n- if there's exactly one input lifetime it gets applied to all output refs\n- if there's multiple input lifetimes but one of them is `&self` or `&mut self` then the lifetime of `self` is applied to all output refs\n- otherwise output lifetimes have to be made explicit\n\nThat's a lot to take in so let's look at some examples:\n\n```rust\n// elided\nfn print(s: &str);\n\n// expanded\nfn print<'a>(s: &'a str);\n\n// elided\nfn trim(s: &str) -> &str;\n\n// expanded\nfn trim<'a>(s: &'a str) -> &'a str;\n\n// illegal, can't determine output lifetime, no inputs\nfn get_str() -> &str;\n\n// explicit options include\nfn get_str<'a>() -> &'a str; // generic version\nfn get_str() -> &'static str; // 'static version\n\n// illegal, can't determine output lifetime, multiple inputs\nfn overlap(s: &str, t: &str) -> &str;\n\n// explicit (but still partially elided) options include\nfn overlap<'a>(s: &'a str, t: &str) -> &'a str; // output can't outlive s\nfn overlap<'a>(s: &str, t: &'a str) -> &'a str; // output can't outlive t\nfn overlap<'a>(s: &'a str, t: &'a str) -> &'a str; // output can't outlive s & t\nfn overlap(s: &str, t: &str) -> &'static str; // output can outlive s & t\nfn overlap<'a>(s: &str, t: &str) -> &'a str; // no relationship between input & output lifetimes\n\n// expanded\nfn overlap<'a, 'b>(s: &'a str, t: &'b str) -> &'a str;\nfn overlap<'a, 'b>(s: &'a str, t: &'b str) -> &'b str;\nfn overlap<'a>(s: &'a str, t: &'a str) -> &'a str;\nfn overlap<'a, 'b>(s: &'a str, t: &'b str) -> &'static str;\nfn overlap<'a, 'b, 'c>(s: &'a str, t: &'b str) -> &'c str;\n\n// elided\nfn compare(&self, s: &str) -> &str;\n\n// expanded\nfn compare<'a, 'b>(&'a self, &'b str) -> &'a str;\n```\n\nIf you've ever written\n- a function which takes or returns references\n- a struct method which takes or returns references\n- a generic function\n- a trait object (more on this later)\n- a closure (more on this later)\n\nthen your code has generic elided lifetime annotations all over it.\n\n**Key Takeaways**\n- almost all Rust code is generic code and there's elided lifetime annotations everywhere\n\n\n\n### 5) if it compiles then my lifetime annotations are correct\n\n**Misconception Corollaries**\n- Rust's lifetime elision rules for functions are always right\n- Rust's borrow checker is always right, technically _and semantically_\n- Rust knows more about the semantics of my program than I do\n\nIt's possible for a Rust program to be technically compilable but still semantically wrong. Take this for example:\n\n```rust\nstruct ByteIter<'a> {\n remainder: &'a [u8]\n}\n\nimpl<'a> ByteIter<'a> {\n fn next(&mut self) -> Option<&u8> {\n if self.remainder.is_empty() {\n None\n } else {\n let byte = &self.remainder[0];\n self.remainder = &self.remainder[1..];\n Some(byte)\n }\n }\n}\n\nfn main() {\n let mut bytes = ByteIter { remainder: b\"1\" };\n assert_eq!(Some(&b'1'), bytes.next());\n assert_eq!(None, bytes.next());\n}\n```\n\n`ByteIter` is an iterator that iterates over a slice of bytes. We're skipping the `Iterator` trait implementation for conciseness. It seems to work fine, but what if we want to check a couple bytes at a time?\n\n```rust\nfn main() {\n let mut bytes = ByteIter { remainder: b\"1123\" };\n let byte_1 = bytes.next();\n let byte_2 = bytes.next();\n if byte_1 == byte_2 { // ❌\n // do something\n }\n}\n```\n\nUh oh! Compile error:\n\n```none\nerror[E0499]: cannot borrow `bytes` as mutable more than once at a time\n --> src/main.rs:20:18\n |\n19 | let byte_1 = bytes.next();\n | ----- first mutable borrow occurs here\n20 | let byte_2 = bytes.next();\n | ^^^^^ second mutable borrow occurs here\n21 | if byte_1 == byte_2 {\n | ------ first borrow later used here\n```\n\nI guess we can copy each byte. Copying is okay when we're working with bytes but if we turned `ByteIter` into a generic slice iterator that can iterate over any `&'a [T]` then we might want to use it in the future with types that may be very expensive or impossible to copy and clone. Oh well, I guess there's nothing we can do about that, the code compiles so the lifetime annotations must be right, right?\n\nNope, the current lifetime annotations are actually the source of the bug! It's particularly hard to spot because the buggy lifetime annotations are elided. Let's expand the elided lifetimes to get a clearer look at the problem:\n\n```rust\nstruct ByteIter<'a> {\n remainder: &'a [u8]\n}\n\nimpl<'a> ByteIter<'a> {\n fn next<'b>(&'b mut self) -> Option<&'b u8> {\n if self.remainder.is_empty() {\n None\n } else {\n let byte = &self.remainder[0];\n self.remainder = &self.remainder[1..];\n Some(byte)\n }\n }\n}\n```\n\nThat didn't help at all. I'm still confused. Here's a hot tip that only Rust pros know: give your lifetime annotations descriptive names. Let's try again:\n\n```rust\nstruct ByteIter<'remainder> {\n remainder: &'remainder [u8]\n}\n\nimpl<'remainder> ByteIter<'remainder> {\n fn next<'mut_self>(&'mut_self mut self) -> Option<&'mut_self u8> {\n if self.remainder.is_empty() {\n None\n } else {\n let byte = &self.remainder[0];\n self.remainder = &self.remainder[1..];\n Some(byte)\n }\n }\n}\n```\n\nEach returned byte is annotated with `'mut_self` but the bytes are clearly coming from `'remainder`! Let's fix it.\n\n```rust\nstruct ByteIter<'remainder> {\n remainder: &'remainder [u8]\n}\n\nimpl<'remainder> ByteIter<'remainder> {\n fn next(&mut self) -> Option<&'remainder u8> {\n if self.remainder.is_empty() {\n None\n } else {\n let byte = &self.remainder[0];\n self.remainder = &self.remainder[1..];\n Some(byte)\n }\n }\n}\n\nfn main() {\n let mut bytes = ByteIter { remainder: b\"1123\" };\n let byte_1 = bytes.next();\n let byte_2 = bytes.next();\n std::mem::drop(bytes); // we can even drop the iterator now!\n if byte_1 == byte_2 { // ✅\n // do something\n }\n}\n```\n\nNow that we look back on the previous version of our program it was obviously wrong, so why did Rust compile it? The answer is simple: it was memory safe.\n\nThe Rust borrow checker only cares about the lifetime annotations in a program to the extent it can use them to statically verify the memory safety of the program. Rust will happily compile programs even if the lifetime annotations have semantic errors, and the consequence of this is that the program becomes unnecessarily restrictive.\n\nHere's a quick example that's the opposite of the previous example: Rust's lifetime elision rules happen to be semantically correct in this instance but we unintentionally write a very restrictive method with our own unnecessary explicit lifetime annotations.\n\n```rust\n#[derive(Debug)]\nstruct NumRef<'a>(&'a i32);\n\nimpl<'a> NumRef<'a> {\n // my struct is generic over 'a so that means I need to annotate\n // my self parameters with 'a too, right? (answer: no, not right)\n fn some_method(&'a mut self) {}\n}\n\nfn main() {\n let mut num_ref = NumRef(&5);\n num_ref.some_method(); // mutably borrows num_ref for the rest of its lifetime\n num_ref.some_method(); // ❌\n println!(\"{:?}\", num_ref); // ❌\n}\n```\n\nIf we have some struct generic over `'a` we almost never want to write a method with a `&'a mut self` receiver. What we're communicating to Rust is _\"this method will mutably borrow the struct for the entirety of the struct's lifetime\"_. In practice this means Rust's borrow checker will only allow at most one call to `some_method` before the struct becomes permanently mutably borrowed and thus unusable. The use-cases for this are extremely rare but the code above is very easy for confused beginners to write and it compiles. The fix is to not add unnecessary explicit lifetime annotations and let Rust's lifetime elision rules handle it:\n\n```rust\n#[derive(Debug)]\nstruct NumRef<'a>(&'a i32);\n\nimpl<'a> NumRef<'a> {\n // no more 'a on mut self\n fn some_method(&mut self) {}\n\n // above line desugars to\n fn some_method_desugared<'b>(&'b mut self){}\n}\n\nfn main() {\n let mut num_ref = NumRef(&5);\n num_ref.some_method();\n num_ref.some_method(); // ✅\n println!(\"{:?}\", num_ref); // ✅\n}\n```\n\n**Key Takeaways**\n- Rust's lifetime elision rules for functions are not always right for every situation\n- Rust does not know more about the semantics of your program than you do\n- give your lifetime annotations descriptive names\n- try to be mindful of where you place explicit lifetime annotations and why\n\n\n\n### 6) boxed trait objects don't have lifetimes\n\nEarlier we discussed Rust's lifetime elision rules _for functions_. Rust also has lifetime elision rules for trait objects, which are:\n- if a trait object is used as a type argument to a generic type then its lifetime bound is inferred from the containing type\n - if there's a unique bound from the containing then that's used\n - if there's more than one bound from the containing type then an explicit bound must be specified\n- if the above doesn't apply then\n - if the trait is defined with a single lifetime bound then that bound is used\n - if `'static` is used for any lifetime bound then `'static` is used\n - if the trait has no lifetime bounds then its lifetime is inferred in expressions and is `'static` outside of expressions\n\nAll of that sounds super complicated but can be simply summarized as _\"a trait object's lifetime bound is inferred from context.\"_ After looking at a handful of examples we'll see the lifetime bound inferences are pretty intuitive so we don't have to memorize the formal rules:\n\n```rust\nuse std::cell::Ref;\n\ntrait Trait {}\n\n// elided\ntype T1 = Box;\n// expanded, Box has no lifetime bound on T, so inferred as 'static\ntype T2 = Box;\n\n// elided\nimpl dyn Trait {}\n// expanded\nimpl dyn Trait + 'static {}\n\n// elided\ntype T3<'a> = &'a dyn Trait;\n// expanded, &'a T requires T: 'a, so inferred as 'a\ntype T4<'a> = &'a (dyn Trait + 'a);\n\n// elided\ntype T5<'a> = Ref<'a, dyn Trait>;\n// expanded, Ref<'a, T> requires T: 'a, so inferred as 'a\ntype T6<'a> = Ref<'a, dyn Trait + 'a>;\n\ntrait GenericTrait<'a>: 'a {}\n\n// elided\ntype T7<'a> = Box>;\n// expanded\ntype T8<'a> = Box + 'a>;\n\n// elided\nimpl<'a> dyn GenericTrait<'a> {}\n// expanded\nimpl<'a> dyn GenericTrait<'a> + 'a {}\n```\n\nConcrete types which implement traits can have references and thus they also have lifetime bounds, and so their corresponding trait objects have lifetime bounds. Also you can implement traits directly for references which obviously have lifetime bounds:\n\n```rust\ntrait Trait {}\n\nstruct Struct {}\nstruct Ref<'a, T>(&'a T);\n\nimpl Trait for Struct {}\nimpl Trait for &Struct {} // impl Trait directly on a ref type\nimpl<'a, T> Trait for Ref<'a, T> {} // impl Trait on a type containing refs\n```\n\nAnyway, this is worth going over because it often confuses beginners when they refactor a function from using trait objects to generics or vice versa. Take this program for example:\n\n```rust\nuse std::fmt::Display;\n\nfn dynamic_thread_print(t: Box) {\n std::thread::spawn(move || {\n println!(\"{}\", t);\n }).join();\n}\n\nfn static_thread_print(t: T) { // ❌\n std::thread::spawn(move || {\n println!(\"{}\", t);\n }).join();\n}\n```\n\nIt throws this compile error:\n\n```none\nerror[E0310]: the parameter type `T` may not live long enough\n --> src/lib.rs:10:5\n |\n9 | fn static_thread_print(t: T) {\n | -- help: consider adding an explicit lifetime bound...: `T: 'static +`\n10 | std::thread::spawn(move || {\n | ^^^^^^^^^^^^^^^^^^\n |\nnote: ...so that the type `[closure@src/lib.rs:10:24: 12:6 t:T]` will meet its required lifetime bounds\n --> src/lib.rs:10:5\n |\n10 | std::thread::spawn(move || {\n | ^^^^^^^^^^^^^^^^^^\n```\n\nOkay great, the compiler tells us how to fix the issue so let's fix the issue.\n\n```rust\nuse std::fmt::Display;\n\nfn dynamic_thread_print(t: Box) {\n std::thread::spawn(move || {\n println!(\"{}\", t);\n }).join();\n}\n\nfn static_thread_print(t: T) { // ✅\n std::thread::spawn(move || {\n println!(\"{}\", t);\n }).join();\n}\n```\n\nIt compiles now but these two functions look awkward next to each other, why does the second function require a `'static` bound on `T` where the first function doesn't? That's a trick question. Using the lifetime elision rules Rust automatically infers a `'static` bound in the first function so both actually have `'static` bounds. This is what the Rust compiler sees:\n\n```rust\nuse std::fmt::Display;\n\nfn dynamic_thread_print(t: Box) {\n std::thread::spawn(move || {\n println!(\"{}\", t);\n }).join();\n}\n\nfn static_thread_print(t: T) {\n std::thread::spawn(move || {\n println!(\"{}\", t);\n }).join();\n}\n```\n\n**Key Takeaways**\n- all trait objects have some inferred default lifetime bounds\n\n\n\n### 7) compiler error messages will tell me how to fix my program\n\n**Misconception Corollaries**\n- Rust's lifetime elision rules for trait objects are always right\n- Rust knows more about the semantics of my program than I do\n\nThis misconception is the previous two misconceptions combined into one example:\n\n```rust\nuse std::fmt::Display;\n\nfn box_displayable(t: T) -> Box { // ❌\n Box::new(t)\n}\n```\n\nThrows this error:\n\n```none\nerror[E0310]: the parameter type `T` may not live long enough\n --> src/lib.rs:4:5\n |\n3 | fn box_displayable(t: T) -> Box {\n | -- help: consider adding an explicit lifetime bound...: `T: 'static +`\n4 | Box::new(t)\n | ^^^^^^^^^^^\n |\nnote: ...so that the type `T` will meet its required lifetime bounds\n --> src/lib.rs:4:5\n |\n4 | Box::new(t)\n | ^^^^^^^^^^^\n```\n\nOkay, let's fix it how the compiler is telling us to fix it, nevermind the fact that it's automatically inferring a `'static` lifetime bound for our boxed trait object without telling us and its recommended fix is based on that unstated fact:\n\n```rust\nuse std::fmt::Display;\n\nfn box_displayable(t: T) -> Box { // ✅\n Box::new(t)\n}\n```\n\nSo the program compiles now... but is this what we actually want? Probably, but maybe not. The compiler didn't mention any other fixes but this would have also been appropriate:\n\n```rust\nuse std::fmt::Display;\n\nfn box_displayable<'a, T: Display + 'a>(t: T) -> Box { // ✅\n Box::new(t)\n}\n```\n\nThis function accepts all the same arguments as the previous version plus a lot more! Does that make it better? Not necessarily, it depends on the requirements and constraints of our program. This example is a bit abstract so let's take a look at a simpler and more obvious case:\n\n```rust\nfn return_first(a: &str, b: &str) -> &str { // ❌\n a\n}\n```\n\nThrows:\n\n```none\nerror[E0106]: missing lifetime specifier\n --> src/lib.rs:1:38\n |\n1 | fn return_first(a: &str, b: &str) -> &str {\n | ---- ---- ^ expected named lifetime parameter\n |\n = help: this function's return type contains a borrowed value, but the signature does not say whether it is borrowed from `a` or `b`\nhelp: consider introducing a named lifetime parameter\n |\n1 | fn return_first<'a>(a: &'a str, b: &'a str) -> &'a str {\n | ^^^^ ^^^^^^^ ^^^^^^^ ^^^\n```\n\nThe error message recommends annotating both inputs and the output with the same lifetime. If we did this our program would compile but this function would overly-constrain the return type. What we actually want is this:\n\n```rust\nfn return_first<'a>(a: &'a str, b: &str) -> &'a str { // ✅\n a\n}\n```\n\n**Key Takeaways**\n- Rust's lifetime elision rules for trait objects are not always right for every situation\n- Rust does not know more about the semantics of your program than you do\n- Rust compiler error messages suggest fixes which will make your program compile which is not that same as fixes which will make you program compile _and_ best suit the requirements of your program\n\n\n\n### 8) lifetimes can grow and shrink at run-time\n\n**Misconception Corollaries**\n- container types can swap references at run-time to change their lifetime\n- Rust borrow checker does advanced control flow analysis\n\nThis does not compile:\n\n```rust\nstruct Has<'lifetime> {\n lifetime: &'lifetime str,\n}\n\nfn main() {\n let long = String::from(\"long\");\n let mut has = Has { lifetime: &long };\n assert_eq!(has.lifetime, \"long\");\n\n {\n let short = String::from(\"short\");\n // \"switch\" to short lifetime\n has.lifetime = &short;\n assert_eq!(has.lifetime, \"short\");\n\n // \"switch back\" to long lifetime (but not really)\n has.lifetime = &long;\n assert_eq!(has.lifetime, \"long\");\n // `short` dropped here\n }\n\n assert_eq!(has.lifetime, \"long\"); // ❌ - `short` still \"borrowed\" after drop\n}\n```\n\nIt throws:\n\n```none\nerror[E0597]: `short` does not live long enough\n --> src/main.rs:11:24\n |\n11 | has.lifetime = &short;\n | ^^^^^^ borrowed value does not live long enough\n...\n15 | }\n | - `short` dropped here while still borrowed\n16 | assert_eq!(has.lifetime, \"long\");\n | --------------------------------- borrow later used here\n```\n\nThis also does not compile, throws the exact same error as above:\n\n```rust\nstruct Has<'lifetime> {\n lifetime: &'lifetime str,\n}\n\nfn main() {\n let long = String::from(\"long\");\n let mut has = Has { lifetime: &long };\n assert_eq!(has.lifetime, \"long\");\n\n // this block will never run\n if false {\n let short = String::from(\"short\");\n // \"switch\" to short lifetime\n has.lifetime = &short;\n assert_eq!(has.lifetime, \"short\");\n\n // \"switch back\" to long lifetime (but not really)\n has.lifetime = &long;\n assert_eq!(has.lifetime, \"long\");\n // `short` dropped here\n }\n\n assert_eq!(has.lifetime, \"long\"); // ❌ - `short` still \"borrowed\" after drop\n}\n```\n\nLifetimes have to be statically verified at compile-time and the Rust borrow checker only does very basic control flow analysis, so it assumes every block in an `if-else` statement and every match arm in a `match` statement can be taken and then chooses the shortest possible lifetime for the variable. Once a variable is bounded by a lifetime it is bounded by that lifetime _forever_. The lifetime of a variable can only shrink, and all the shrinkage is determined at compile-time.\n\n**Key Takeaways**\n- lifetimes are statically verified at compile-time\n- lifetimes cannot grow or shrink or change in any way at run-time\n- Rust borrow checker will always choose the shortest possible lifetime for a variable assuming all code paths can be taken\n\n\n\n### 9) downgrading mut refs to shared refs is safe\n\n**Misconception Corollaries**\n- re-borrowing a reference ends its lifetime and starts a new one\n\nYou can pass a mut ref to a function expecting a shared ref because Rust will implicitly re-borrow the mut ref as immutable:\n\n```rust\nfn takes_shared_ref(n: &i32) {}\n\nfn main() {\n let mut a = 10;\n takes_shared_ref(&mut a); // ✅\n takes_shared_ref(&*(&mut a)); // above line desugared\n}\n```\n\nIntuitively this makes sense, since there's no harm in re-borrowing a mut ref as immutable, right? Surprisingly no, as the program below does not compile:\n\n```rust\nfn main() {\n let mut a = 10;\n let b: &i32 = &*(&mut a); // re-borrowed as immutable\n let c: &i32 = &a;\n dbg!(b, c); // ❌\n}\n```\n\nThrows this error:\n\n```none\nerror[E0502]: cannot borrow `a` as immutable because it is also borrowed as mutable\n --> src/main.rs:4:19\n |\n3 | let b: &i32 = &*(&mut a);\n | -------- mutable borrow occurs here\n4 | let c: &i32 = &a;\n | ^^ immutable borrow occurs here\n5 | dbg!(b, c);\n | - mutable borrow later used here\n```\n\nA mutable borrow does occur, but it's immediately and unconditionally re-borrowed as immutable and then dropped. Why is Rust treating the immutable re-borrow as if it still has the mut ref's exclusive lifetime? While there's no issue in the particular example above, allowing the ability to downgrade mut refs to shared refs does indeed introduce potential memory safety issues:\n\n```rust\nuse std::sync::Mutex;\n\nstruct Struct {\n mutex: Mutex\n}\n\nimpl Struct {\n // downgrades mut self to shared str\n fn get_string(&mut self) -> &str {\n self.mutex.get_mut().unwrap()\n }\n fn mutate_string(&self) {\n // if Rust allowed downgrading mut refs to shared refs\n // then the following line would invalidate any shared\n // refs returned from the get_string method\n *self.mutex.lock().unwrap() = \"surprise!\".to_owned();\n }\n}\n\nfn main() {\n let mut s = Struct {\n mutex: Mutex::new(\"string\".to_owned())\n };\n let str_ref = s.get_string(); // mut ref downgraded to shared ref\n s.mutate_string(); // str_ref invalidated, now a dangling pointer\n dbg!(str_ref); // ❌ - as expected!\n}\n```\n\nThe point here is that when you re-borrow a mut ref as a shared ref you don't get that shared ref without a big gotcha: it extends the mut ref's lifetime for the duration of the re-borrow even if the mut ref itself is dropped. Using the re-borrowed shared ref is very difficult because it's immutable but it can't overlap with any other shared refs. The re-borrowed shared ref has all the cons of a mut ref and all the cons of a shared ref and has the pros of neither. I believe re-borrowing a mut ref as a shared ref should be considered a Rust anti-pattern. Being aware of this anti-pattern is important so that you can easily spot it when you see code like this:\n\n```rust\n// downgrades mut T to shared T\nfn some_function(some_arg: &mut T) -> &T;\n\nstruct Struct;\n\nimpl Struct {\n // downgrades mut self to shared self\n fn some_method(&mut self) -> &Self;\n\n // downgrades mut self to shared T\n fn other_method(&mut self) -> &T;\n}\n```\n\nEven if you avoid re-borrows in function and method signatures Rust still does automatic implicit re-borrows so it's easy to bump into this problem without realizing it like so:\n\n```rust\nuse std::collections::HashMap;\n\ntype PlayerID = i32;\n\n#[derive(Debug, Default)]\nstruct Player {\n score: i32,\n}\n\nfn start_game(player_a: PlayerID, player_b: PlayerID, server: &mut HashMap) {\n // get players from server or create & insert new players if they don't yet exist\n let player_a: &Player = server.entry(player_a).or_default();\n let player_b: &Player = server.entry(player_b).or_default();\n\n // do something with players\n dbg!(player_a, player_b); // ❌\n}\n```\n\nThe above fails to compile. `or_default()` returns a `&mut Player` which we're implicitly re-borrowing as `&Player` because of our explicit type annotations. To do what we want we have to:\n\n```rust\nuse std::collections::HashMap;\n\ntype PlayerID = i32;\n\n#[derive(Debug, Default)]\nstruct Player {\n score: i32,\n}\n\nfn start_game(player_a: PlayerID, player_b: PlayerID, server: &mut HashMap) {\n // drop the returned mut Player refs since we can't use them together anyway\n server.entry(player_a).or_default();\n server.entry(player_b).or_default();\n\n // fetch the players again, getting them immutably this time, without any implicit re-borrows\n let player_a = server.get(&player_a);\n let player_b = server.get(&player_b);\n\n // do something with players\n dbg!(player_a, player_b); // ✅\n}\n```\n\nKinda awkward and clunky but this is the sacrifice we make at the Altar of Memory Safety.\n\n**Key Takeaways**\n- try not to re-borrow mut refs as shared refs, or you're gonna have a bad time\n- re-borrowing a mut ref doesn't end its lifetime, even if the ref is dropped\n\n\n\n### 10) closures follow the same lifetime elision rules as functions\n\nThis is more of a Rust Gotcha than a misconception.\n\nClosures, despite being functions, do not follow the same lifetime elision rules as functions.\n\n```rust\nfn function(x: &i32) -> &i32 {\n x\n}\n\nfn main() {\n let closure = |x: &i32| x; // ❌\n}\n```\n\nThrows:\n\n```none\nerror: lifetime may not live long enough\n --> src/main.rs:6:29\n |\n6 | let closure = |x: &i32| x;\n | - - ^ returning this value requires that `'1` must outlive `'2`\n | | |\n | | return type of closure is &'2 i32\n | let's call the lifetime of this reference `'1`\n```\n\nAfter desugaring we get:\n\n```rust\n// input lifetime gets applied to output\nfn function<'a>(x: &'a i32) -> &'a i32 {\n x\n}\n\nfn main() {\n // input and output each get their own distinct lifetimes\n let closure = for<'a, 'b> |x: &'a i32| -> &'b i32 { x };\n // note: the above line is not valid syntax, but we need it for illustrative purposes\n}\n```\n\nThere's no good reason for this discrepancy. Closures were first implemented with different type inference semantics than functions and now we're stuck with it forever because to unify them at this point would be a breaking change. So how can we explicitly annotate a closure's type? Our options include:\n\n```rust\nfn main() {\n // cast to trait object, becomes unsized, oops, compile error\n let identity: dyn Fn(&i32) -> &i32 = |x: &i32| x;\n\n // can allocate it on the heap as a workaround but feels clunky\n let identity: Box &i32> = Box::new(|x: &i32| x);\n\n // can skip the allocation and just create a static reference\n let identity: &dyn Fn(&i32) -> &i32 = &|x: &i32| x;\n\n // previous line desugared :)\n let identity: &'static (dyn for<'a> Fn(&'a i32) -> &'a i32 + 'static) = &|x: &i32| -> &i32 { x };\n\n // this would be ideal but it's invalid syntax\n let identity: impl Fn(&i32) -> &i32 = |x: &i32| x;\n\n // this would also be nice but it's also invalid syntax\n let identity = for<'a> |x: &'a i32| -> &'a i32 { x };\n\n // since \"impl trait\" works in the function return position\n fn return_identity() -> impl Fn(&i32) -> &i32 {\n |x| x\n }\n let identity = return_identity();\n\n // more generic version of the previous solution\n fn annotate(f: F) -> F where F: Fn(&T) -> &T {\n f\n }\n let identity = annotate(|x: &i32| x);\n}\n```\n\nAs I'm sure you've already noticed from the examples above, when closure types are used as trait bounds they do follow the usual function lifetime elision rules.\n\nThere's no real lesson or insight to be had here, it just is what it is.\n\n**Key Takeaways**\n- every language has gotchas 🤷\n\n\n\n## Conclusion\n\n- `T` is a superset of both `&T` and `&mut T`\n- `&T` and `&mut T` are disjoint sets\n- `T: 'static` should be read as _\"`T` can live at least as long as a `'static` lifetime\"_\n- if `T: 'static` then `T` can be a borrowed type with a `'static` lifetime _or_ an owned type\n- since `T: 'static` includes owned types that means `T`\n - can be dynamically allocated at run-time\n - does not have to be valid for the entire program\n - can be safely and freely mutated\n - can be dynamically dropped at run-time\n - can have lifetimes of different durations\n- `T: 'a` is more general and more flexible than `&'a T`\n- `T: 'a` accepts owned types, owned types which contain references, and references\n- `&'a T` only accepts references\n- if `T: 'static` then `T: 'a` since `'static` >= `'a` for all `'a`\n- almost all Rust code is generic code and there's elided lifetime annotations everywhere\n- Rust's lifetime elision rules are not always right for every situation\n- Rust does not know more about the semantics of your program than you do\n- give your lifetime annotations descriptive names\n- try to be mindful of where you place explicit lifetime annotations and why\n- all trait objects have some inferred default lifetime bounds\n- Rust compiler error messages suggest fixes which will make your program compile which is not that same as fixes which will make you program compile _and_ best suit the requirements of your program\n- lifetimes are statically verified at compile-time\n- lifetimes cannot grow or shrink or change in any way at run-time\n- Rust borrow checker will always choose the shortest possible lifetime for a variable assuming all code paths can be taken\n- try not to re-borrow mut refs as shared refs, or you're gonna have a bad time\n- re-borrowing a mut ref doesn't end its lifetime, even if the ref is dropped\n- every language has gotchas 🤷\n\n\n\n## Discuss\n\nDiscuss this article on\n- [learnrust subreddit](https://www.reddit.com/r/learnrust/comments/gmrcrq/common_rust_lifetime_misconceptions/)\n- [official Rust users forum](https://users.rust-lang.org/t/blog-post-common-rust-lifetime-misconceptions/42950)\n- [rust subreddit](https://www.reddit.com/r/rust/comments/golrsx/common_rust_lifetime_misconceptions/)\n- [Hackernews](https://news.ycombinator.com/item?id=23279731)\n- [Github](https://github.com/pretzelhammer/rust-blog/discussions)\n\n\n## Further Reading\n\n- [Tour of Rust's Standard Library Traits](./tour-of-rusts-standard-library-traits.md)\n- [Beginner's Guide to Concurrent Programming: Coding a Multithreaded Chat Server using Tokio](./chat-server.md)\n- [Learning Rust in 2024](./learning-rust-in-2024.md)\n- [Using Rust in Non-Rust Servers to Improve Performance](./rust-in-non-rust-servers.md)\n- [Sizedness in Rust](./sizedness-in-rust.md)\n- [RESTful API in Sync & Async Rust](./restful-api-in-sync-and-async-rust.md)\n- [Learn Assembly with Entirely Too Many Brainfuck Compilers](./too-many-brainfuck-compilers.md)\n\n\n\n## Notifications\n\nGet notified when a new blog post gets published by\n- Subscribing to this repo's [releases RSS feed](https://github.com/pretzelhammer/rust-blog/releases.atom) or\n- Watching this repo's releases (click `Watch` → click `Custom` → select `Releases` → click `Apply`)\n"
},
{
"path": "posts/learning-rust-in-2020.md",
"content": "# Learning Rust in 2020\n\n_09 May 2020 · #rust · #programming · #exercises_\n\n**Table of Contents**\n- [Intro](#intro)\n- [TL;DR](#tldr)\n- [Practical Rust Resource Reviews](#practical-rust-resource-reviews)\n - [HackerRank](#hackerrank)\n - [Project Euler](#project-euler)\n - [LeetCode](#leetcode)\n - [Codewars](#codewars)\n - [Advent of Code](#advent-of-code)\n - [Rustlings](#rustlings)\n - [Exercism](#exercism)\n- [Conclusion](#conclusion)\n- [Discuss](#discuss)\n- [Further Reading](#further-reading)\n- [Notifications](#notifications)\n\n\n\n## Intro\n\nWhen I started learning Rust I made the mistake of following the advice to read [The Book](https://doc.rust-lang.org/book/title-page.html) first. While it's a great resource, it's pretty overwhelming for a beginner to get told _\"If you'd like to learn this programming language the best way to start is to read this 20 chapter book!\"_ Most people give up before they even get started when they get advice like this. Nobody ever told someone to read a 20 chapter book just to get started with Javascript or Python. Rust's learning curve is no joke but you gotta give the people what they want, and they want to program, not read about programming. Programming is fun and reading about programming is not as fun.\n\nThe first 10% of this article is gonna be me giving you advice on how to learn Rust in 2020 following a _practical hands-on coding_ approach. This is the good part of the article. You can safely exit after this part (I'll tell you when). The remaining 90% of this article is me ranting about how most online coding challenge sites have poor support for Rust.\n\n\n\n## TL;DR\n\nIf you're a total Rust newbie and want to learn as much as possible in just one day you should read fasterthanlime's excellent [A half-hour to learn Rust](https://fasterthanli.me/blog/2020/a-half-hour-to-learn-rust/) and then checkout the awesome [Rustlings](https://github.com/rust-lang/rustlings) repo and complete the exercises.\n\nIf you're a Rust beginner you should get started on [Exercism's Rust Track](https://exercism.io/tracks/rust). If you get stuck you should ask your friends Google and StackOverflow for help. I recommend taking the time to get comfortable reading and navigating the [Rust Standard Library Docs](https://doc.rust-lang.org/std/) which is amazing and has simple practical examples for how to use everything inside of it. [Rust by Example](https://doc.rust-lang.org/rust-by-example/) is also a really good high-level reference that you can use to quickly learn Rust syntax and features. If you want to gain a deeper understanding of a certain Rust concept only then do I recommend finding the appropriate chapter in [The Book](https://doc.rust-lang.org/book/title-page.html) to read. The best part of completing an exercise on Exercism is that you get access to all the solutions by other members which you can sort by most-starred to see particularly idiomatic or clever solutions. This is a great way to learn!\n\nAt this point you're probably an advanced beginner and can find your own path. If you need more guidance and would like to continue working on small simple programs I recommend doing the exercises from the [Advent of Code 2018 Calendar](https://adventofcode.com/2018). The reason why I specifically recommended the 2018 calendar is because once you're finished with an exercise you can compare your solution to [BurntSushi's Advent of Code 2018 Rust solutions](https://github.com/BurntSushi/advent-of-code). BurntSushi writes really clean, readable, idiomatic Rust code. Reading the code of an experienced Rustacean will teach you as much as the exercises themselves.\n\nExit now, the good part of the article is over.\n\n\n\n## Practical Rust Resource Reviews\n\n_Alternative title: Reviews of Free Online Resources a Rust Beginner can use to Practice Writing Small Simple Rust Programs_\n\nMost of these resources weren't specifically created for the purpose of teaching Rust, however they can all be used to learn and practice Rust and many of them explicitly support Rust submissions and provide Rust-specific versions of problems.\n\nThe resources are ordered from worst to best.\n\n\n\n### [HackerRank](https://www.hackerrank.com)\n\nRust is a supported language on HackerRank except you aren't allowed to submit Rust solutions to most of the problems on their site. I tried to upload my solution directly and they refused it:\n\n\n\nThis is really strange because I was able to browse Rust solutions for the problem above submitted by other HackerRank users, so it's possible to submit a Rust solution somehow. I tried Googling this issue but Google didn't return any useful results. There's no way for me to evaluate HackerRank other than to tell you not to waste your time with it like I did.\n\n\n\n### [Project Euler](https://projecteuler.net/archives)\n\nWhen I first started to learn programming back in 2012 I commonly heard _\"If you wanna get up to speed quickly in a new programming language solve some Project Euler problems with it!\"_ which was okay advice at the time since there were not many other alternatives but in my opinion Project Euler has very little to do with programming. Project Euler problems are more math problems than they are programming problems. Their challenge lies almost entirely in the mathematical reasoning required to reach the solution as the programming required is usually trivial. I would not recommend solving Project Euler problems as a way to learn Rust unless you're very mathematically inclined and have some nostalgia for the site.\n\n\n\n### [LeetCode](https://leetcode.com/problemset/all/)\n\nRust is a supported language on LeetCode. For every problem on LeetCode you get a solution template which usually contains a single unimplemented function which you then have to implement and submit in order to solve the problem. For more involved problems the solution template might include a `struct` and an `impl` block with several unimplemented methods. Unfortunately, these solution templates are not created by humans, they are automatically generated, which results in a lot of really awkward and unidiomatic Rust code. Examples:\n\n| LeetCode generated Rust | Idiomatic Rust |\n|-|-|\n| tree problems represent links as `Option>>` | `Option>>` is overkill for tree links and `Option>` works just as well and is much easier to work with |\n| methods which obviously mutate self still borrow it immutably, e.g. `fn insert(&self, val: i32)` | methods that mutate self need to borrow it mutably, e.g. `fn insert(&mut self, val: i32)` |\n| signed 32-bit integers are used for all numbers, even if the problem is undefined for negative integers, e.g. `fn nth_fib(n: i32) -> i32` | problems which are undefined for negative integers should use unsigned integers, e.g. `fn nth_fib(n: u32) -> u32` |\n| functions always take ownership of their arguments, even if it's unnecessary, e.g. `fn sum(nums: Vec) -> i32` | if you don't need ownership then borrow `fn sum(nums: &[i32]) -> i32` |\n| functions sometimes ignore basic error cases, e.g. for `fn get_max(nums: Vec) -> i32` what `i32` should be returned if `nums` is empty? | if a result might be undefined the return type should be wrapped in an `Option`, e.g. `fn get_max(nums: &[i32]) -> Option` |\n\nOther LeetCode issues, specific to Rust:\n- LeetCode doesn't allow you to pull in 3rd-party dependencies in solutions. Normally I think this is okay for most languages but Rust in particular has a pretty slim standard library which doesn't even include regex support so a lot of the more complex string parsing problems on LeetCode are pointlessly difficult to solve in Rust but have otherwise trivial solutions in other languages which have regex support in their standard libraries.\n- None of the problems in the `concurrency` category accept solutions in Rust. What? Fearless concurrency is one of Rust's major selling points!\n- After solving a problem you can go to the problem's comments section to see other user's solutions (as many users like to publish their solutions there) but because Rust isn't very popular on LeetCode sometimes you won't find any Rust solutions ;(\n\nGeneral LeetCode issues:\n- LeetCode has a surprising amount of very low quality problems. Problems can be liked and disliked by users but problems are never removed even if they hit very high dislike ratios. I've seen lots of problems with 100+ votes and 80%+ dislike ratios and I don't understand why they are kept on the site.\n- Problem difficulty ratings are kinda off. Problems are rated as Easy, Medium, or Hard but there are many Easy problems with lower solve rates than many Hard problems.\n- Not all problems accept solutions in all languages, and you can't filter problems by which languages they accept. None of the graph problems on LeetCode accept Rust solutions, for example.\n- LeetCode blocks \"premium\" problems behind a steep monthly paywall but doesn't offer any kind of premium free-trial so there's no telling if the quality is actually any better than the free problems.\n\nThings LeetCode does right:\n- Solutions to problems are tested against a suite of secret unit tests, but if you fail a particular test case they show you the failed case.\n- All of the generated Rust code at least follows rustfmt conventions.\n\n\n\n### [Codewars](https://www.codewars.com/join?language=rust)\n\nCodewars is a misleading name. There's no war going on at Codewars. There's no time limit to solve problems and your solutions aren't judged on their speed of execution or memory usage. You aren't in competition with anyone else. This isn't a bad thing, just worth pointing out.\n\nRust is a supported language on Codewars. For every problem on Codewars you get a solution template which usually contains a single unimplemented function which you then have to implement and submit in order to solve the problem. These solution templates are created by humans, including humans who aren't familiar with Rust, so you occasionally get some awkward and unidiomatic Rust. Examples:\n\n| Codewars' Rust Problems | Idiomatic Rust |\n|-|-|\n| sometimes don't follow rustfmt conventions, e.g. `fn makeUppercase(s:&str)->String` | always follows rustfmt conventions, e.g. `fn make_uppercase(s: &str) -> String` |\n| sometimes takes signed integer arguments for problems that aren't defined for negative integers, e.g. `fn nth_fib(n: i32) -> i32` | if a problem isn't defined for negative integers use unsigned integer arguments, e.g. `fn nth_fib(n: u32) -> u32` |\n| sometimes a problem asks you to return `-1` for the null case, e.g. `fn get_index(needle: i32, haystack: &[i32]) -> i32` | if a result can be null the return type should be wrapped in an `Option`, e.g. `fn get_index(needle: i32, haystack: &[i32]) -> Option` |\n| sometimes don't take advantage of deref coercion, e.g. `fn do_stuff(s: &String, list: &Vec)` | takes advantage of deref coercion, e.g. `fn do_stuff(s: &str, list: &[i32])` |\n\nAll of the issues above only happen sometimes since there are Rustaceans of various skill-levels on Codewars translating problems to Rust. This is a huge step up from LeetCode where all of the generated Rust problem code is consistently unidiomatic. However, the Rust community on Codewars as a whole might lean towards the inexperienced side since I've seen some highly upvoted \"idiomatic\" solutions that were also a bit on the awkward side. Examples:\n\n| Codewars' highest upvoted Rust solutions | Idiomatic Rust |\n|-|-|\n| sometimes use an explicit return at the end of a function block, e.g. `return result;` | blocks are evaluated as expressions and implicitly return their last item, an explicit return at the end of a function block is unnecessary, e.g. `result` |\n| often use compact formatting to make the solution look more concise | should follow rustfmt conventions |\n| sometimes make unnecessary allocations, e.g. `str_slice.to_string().chars()` | if you don't need to allocate then don't, e.g. `str_slice.chars()` |\n| often try to solve the problem using nothing but iterators at the cost of everything else | iterators are expressive and idiomatic, but if you have to chain 15 of them in a row and there are multiple levels of nested iterators in-between then perhaps you should consider refactoring to use some helper functions, intermediate variables, and maybe even a for-loop |\n\nAgain, the issues above only happen sometimes. An experienced Rustacean can spot them easily but there are a lot of Rust newbies on these sites who have no clue they are learning anti-patterns.\n\nOther Codewars issues, specific to Rust:\n- Rust doesn't seem that popular on Codewars, the site has 9000 exercises but only 300 of them have been translated to Rust ;(\n\nOther general Codewars issues:\n- Your solution is tested against a suite of secret unit tests, if you fail one of the secret unit tests you aren't shown the failed test case. This is especially annoying if the test case tests for an edge case that wasn't clearly communicated in the problem description.\n\nThings Codewars does right:\n- There's a small whitelist of 3rd-party dependencies you can use to help solve problems with Rust. This whitelist includes: rand, chrono, regex, serde, itertools, and lazy_static which helps round out Rust's standard library and puts it more on par with other languages.\n- You can filter problems by language.\n- Submitting a solution to a problem also automatically publishes the solution. You can view and upvote other members' solutions. You can sort solutions by most upvotes to see particularly concise and clever solutions, which sometimes will also be very idiomatic (but sometimes not, as explained above).\n- Problem difficulty grading is pretty good! Instead of grading problems as Easy, Medium, or Hard like LeetCode, Codewars chooses to grade problems from easiest to hardest as: 8 kyu, 7 kyu, 6 kyu, 5 kyu, 4 kyu, 3 kyu, 2 kyu, 1 kyu. I completed 60 problems in the 8 kyu - 4 kyu range and every level felt a little more difficult than the last, which aligned with my expectations.\n\n\n\n### [Advent of Code](https://adventofcode.com/)\n\nAdvent of Code is totally language-agnostic. This would seem like a minus at first but seeing how horribly HackerRank, LeetCode, and Codewars handle their support for Rust on their sites it's actually a plus. Advent of Code also gets placed above the previously mentioned sites because AoC's exercises are really interesting, diverse, and high quality in my opinion.\n\nGeneral AoC issues:\n- After you finish an exercise there's no way to see other people's Rust solutions unless you search from them on Google, and even after you find some there's no telling how good or idiomatic they are.\n\nTo solve the above issue I recommend going through the 2018 Calendar problems and comparing your solutions to [BurntSushi's AoC 2018 Rust solutions](https://github.com/BurntSushi/advent-of-code). BurntSushi writes really clean, readable, idiomatic Rust code. If you want to go through the 2019 Calendar then I recommend comparing your solutions to [bcmyers' AoC 2019 Rust solutions](https://github.com/bcmyers/aoc2019). The reason I specifically suggest bcmyers' is because he made a [youtube playlist of him coding up the solutions](https://www.youtube.com/playlist?list=PLQXBtq4j4Ozkx3r4eoMstdkkOG98qpBfg) and he does a great job of explaining his thought process and why he's doing what he's doing while he's coding.\n\nThings AoC got right:\n- High quality, interesting, curated exercises that are tied together with a narrative.\n- Language agnostic, so while it doesn't teach you any Rust patterns it at least doesn't teach you any Rust anti-patterns either.\n\n\n\n### [Rustlings](https://github.com/rust-lang/rustlings)\n\nRustlings is sooo good. All Rustlings exercises are hand-crafted for Rust with love and it's a wonderful breath of fresh air. Finally, a set of exercises that really teach you idiomatic Rust!\n\nIf you're a total Rust newbie you should absolutely checkout [Rustlings](https://github.com/rust-lang/rustlings) and get started on the exercises. I highly recommend reading fasterthanlime's [A half-hour to learn Rust](https://fasterthanli.me/blog/2020/a-half-hour-to-learn-rust/) first as it'll get you up to speed on a lot of Rust syntax and concepts super quickly.\n\nI have only 1 tiny Rustlings criticism: there are some sudden difficulty spikes in the \"error-handling\" and \"conversions\" exercises that I could see some users getting overwhelmed by. I assume most probably make it through, or at least I hope.\n\nI also have 1 tiny non-criticism: it's too short. This is a non-criticism because it's one of Rustlings design goals to be a quick and gentle introduction to Rust but it's so good that of course I wish it was somehow longer.\n\n\n\n### [Exercism](https://exercism.io/tracks/rust)\n\nExercism has a Rust track, which is a collection of exercises roughly ordered by subject and difficulty. The Rust track shares a lot of exercises in common with other tracks, but all of the exercises were translated to Rust by experienced Rustaceans and don't suffer from any of the awkward unidiomatic Rust issues that are common on LeetCode and Codewars. There are about a dozen Rust-specific problems that require you to implement a standard library trait, or write a macro, or write a parallel solution using multiple threads, or write unsafe Rust code. These exercises are by far the highlights of the track and I wish there were more of them. Exercism is second only to Rustlings as a resource for learning Rust. The only reason I placed it above Rustlings is Rustlings can be completed in an evening and Exercism's Rust track will take at least a month to complete so it just has a lot more content.\n\nExercism issues, specific to the Rust track:\n- \"Mentored mode\" is useless, as most of the Rust mentors on the site are inactive, and the students heavily outnumber them, so it's much better to go through a track in \"practice mode\".\n- There are 92 exercises but a good chunk of them don't really teach you anything new so they kinda feel like busywork. They could probably cut ~20 exercises from the track to make it feel a lot tighter.\n\nThings Exercism does right:\n- All problems are translated to Rust or written for Rust by experienced Rustaceans.\n- There are problems which specifically teach Rust's idioms, design patterns, and unique features.\n- Problem difficulties are fairly graded, easy problems are easy, medium problems are medium, hard problems are hard.\n- You can include whatever 3rd-party dependencies that you want in your solutions.\n- All unit tests are public, if you're failing a test you know exactly why.\n- After you submit a solution you can browse other user's solutions, and you can sort solutions by which received the most stars.\n\n\n\n## Conclusion\n\nSame as the [TL;DR](#tldr) :)\n\n\n\n## Discuss\n\nDiscuss this article on\n- [learnrust subreddit](https://www.reddit.com/r/learnrust/comments/ggj8tf/learning_rust_in_2020/)\n- [official Rust users forum](https://users.rust-lang.org/t/blog-post-learning-rust-in-2020/42373)\n- [rust subreddit](https://www.reddit.com/r/rust/comments/gie64f/learning_rust_in_2020/)\n- [Hackernews](https://news.ycombinator.com/item?id=23160975)\n- [Github](https://github.com/pretzelhammer/rust-blog/discussions)\n\n\n## Further Reading\n\n- [Common Rust Lifetime Misconceptions](./common-rust-lifetime-misconceptions.md)\n- [Tour of Rust's Standard Library Traits](./tour-of-rusts-standard-library-traits.md)\n- [Beginner's Guide to Concurrent Programming: Coding a Multithreaded Chat Server using Tokio](./chat-server.md)\n- [Learning Rust in 2024](./learning-rust-in-2024.md)\n- [Using Rust in Non-Rust Servers to Improve Performance](./rust-in-non-rust-servers.md)\n- [Sizedness in Rust](./sizedness-in-rust.md)\n- [RESTful API in Sync & Async Rust](./restful-api-in-sync-and-async-rust.md)\n- [Learn Assembly with Entirely Too Many Brainfuck Compilers](./too-many-brainfuck-compilers.md)\n\n\n\n## Notifications\n\nGet notified when a new blog post gets published by\n- Subscribing to this repo's [releases RSS feed](https://github.com/pretzelhammer/rust-blog/releases.atom) or\n- Watching this repo's releases (click `Watch` → click `Custom` → select `Releases` → click `Apply`)\n"
},
{
"path": "posts/learning-rust-in-2024.md",
"content": "# Learning Rust in 2024\n\n_21 September 2024 · #rust · #coding · #exercises_\n\n**Table of contents**\n- [TL;DR](#tldr)\n- [0\\) Reference material](#0-reference-material)\n- [1\\) Read A half hour to learn Rust](#1-read-a-half-hour-to-learn-rust)\n- [2\\) Complete rustlings](#2-complete-rustlings)\n- [3\\) Spend 10 hours coding in Rust](#3-spend-10-hours-coding-in-rust)\n - [100 Exercises to Learn Rust](#100-exercises-to-learn-rust)\n - [The Rust track on Exercism](#the-rust-track-on-exercism)\n - [Advent of Code](#advent-of-code)\n - [Tutorials](#tutorials)\n- [4\\) Read Common Rust Lifetime Misconceptions](#4-read-common-rust-lifetime-misconceptions)\n- [5\\) Spend another 10 hours coding in Rust](#5-spend-another-10-hours-coding-in-rust)\n- [6\\) Read Tour of Rust's Standard Library Traits](#6-read-tour-of-rusts-standard-library-traits)\n- [What's next?](#whats-next)\n- [Honorable mentions](#honorable-mentions)\n- [Discuss](#discuss)\n- [Further Reading](#further-reading)\n- [Notifications](#notifications)\n\nThis is my opinionated guide on how to go from knowing nothing about Rust to being kinda okay at Rust quickly as possible. This is what I would tell myself if I was starting today.\n\n\n\n## TL;DR\n\n1. Read [A half hour to learn Rust](https://fasterthanli.me/articles/a-half-hour-to-learn-rust) (30 - 60 mins)\n2. Complete [rustlings](https://github.com/rust-lang/rustlings) (2 - 3 hours)\n3. Spend 10 hours coding in Rust (8 - 12 hours)\n4. Read [Common Rust Lifetime Misconceptions](./common-rust-lifetime-misconceptions.md) (30 - 60 mins)\n5. Spend another 10 hours coding in Rust (8 - 12 hours)\n6. Read [Tour of Rust's Standard Library Traits](./tour-of-rusts-standard-library-traits.md) (2 - 4 hours)\n\nAnd that's it, after a short 19 to 30 hours you'll go from being a Rust beginner to a Rust advanced beginner 😎\n\n\n\n## 0\\) Reference material\n\nWhile learning Rust it's useful to have [Rust by Example](https://doc.rust-lang.org/rust-by-example/index.html) and the [Rust Standard Library docs](https://doc.rust-lang.org/std/) open in a couple browser tabs. If you get stuck on something you can probably find the answer by searching in either of those two resources, as both have a search bar at the top if you need to quickly find something.\n\nOther good alternatives are StackOverflow and ChatGPT (or whatever LLM you prefer). Almost every Rust beginner question has been asked and answered on StackOverflow and is indexed on Google. As for ChatGPT, it may get the answer wrong like half the time, but it will at least point you in the general direction of where you should search next to find a good answer.\n\nIf you've tried all of those things but still cannot get unstuck, then here are some beginner-friendly online Rust communities where you can ask questions: the [/r/rust](https://www.reddit.com/r/rust/) subreddit, the [/r/learnrust](https://www.reddit.com/r/learnrust/) subreddit, and the [official Rust users forum](https://users.rust-lang.org/).\n\nAlso it's a good idea to have [Rust Playground](https://play.rust-lang.org/) open in a browser tab. It's an online code editor and compiler for Rust code. It's great for tinkering with small examples to reinforce what you're learning while reading.\n\n\n\n## 1\\) Read [A half hour to learn Rust](https://fasterthanli.me/articles/a-half-hour-to-learn-rust)\n\nI recommend [A half hour to learn Rust](https://fasterthanli.me/articles/a-half-hour-to-learn-rust) above all other resources similar to it because it introduces Rust syntax and concepts in a very systematic and ground-up approach. It starts with the simplest possible example and then incrementally adds a little bit more complexity in every following example. The sequencing is very logical and easy to follow.\n\n\n\n## 2\\) Complete [rustlings](https://github.com/rust-lang/rustlings)\n\n[Rustlings](https://github.com/rust-lang/rustlings) is an ordered collection of tiny Rust programming exercises. Each exercise is a tiny Rust program that doesn't compile or is failing its unit tests. Your job for every exercise is to fix the program so it compiles and passes its unit tests.\n\nBefore starting rustlings you need to have Rust installed. I recommend using [rustup](https://rustup.rs/) to manage Rust installations. Once you have `rustup` you just have to run `rustup update` to update Rust on your machine any time.\n\nWhen coding Rust, you can use whatever code editor you want, but I recommend using one that supports [rust-analyzer](https://rust-analyzer.github.io/). I use [VS Code](https://code.visualstudio.com/) with [this extension](https://marketplace.visualstudio.com/items?itemName=rust-lang.rust-analyzer) to enable rust-analyzer.\n\nGetting started with rustlings is very easy because we can install it using `cargo`. Cargo is Rust's dependency manager and build tool, and comes with all Rust installations. If you installed Rust using `rustup` then you already have it.\n\nRun `cargo install rustlings`. By default, it will put the rustlings binary in `$HOME/.cargo/bin`, so you should add that path to your `$PATH` variable if it's not in there already, e.g.\n\n```sh\nexport PATH=\"$HOME/.cargo/bin:$PATH\"\n```\n\nNow run `rustlings`, which will create a `rustlings/` directory for you with all of the exercises in it. Open that directory in a code editor and then run the `rustlings` command within the directory and follow its instructions. It will tell you which exercise to work on. Complete the exercise and then the command will tell which exercise to work on next. Rinse and repeat until you're done.\n\n\n\n## 3\\) Spend 10 hours coding in Rust\n\nThis can be whatever you want, but for your first 10 hours I would suggest sticking to single-threaded sync Rust. There's enough novel and challenging Rust concepts\\* to learn within those parameters to easily keep you busy for at least 10 hours, and many people take much longer to fully wrap their heads around them. Multithreaded async Rust adds a bunch of additional complex concepts\\*\\*, so starting with it before you have the basics down will likely be overwhelming.\n\n\\*: ownership, borrowing, lifetimes, lifetime elision, mutable vs immutable references, sizedness, deref coercion, traits, trait objects, generics, const generics, static vs dynamic dispatch, error handling, iterators, modules, declarative macros, procedural macros, just to name a few.\n\n\\*\\*: futures, pinning, polling, awaiting, send, sync, cancellation, streams, sinks, locks, atomics, channels, actors, just to name a few.\n\nIf you have no ideas on what to code then I have a few suggestions.\n\n\n\n### [100 Exercises to Learn Rust](https://rust-exercises.com/100-exercises/)\n\nIf you enjoyed the difficulty and pacing of rustlings then you will enjoy 100 Exercises to Learn Rust (100ELR). Although made by different authors, 100ELR feels like a natural extension of rustlings and clearly drew a lot of inspiration from it.\n\nTo get started clone the [mainmatter/100-exercises-to-learn-rust](https://github.com/mainmatter/100-exercises-to-learn-rust) repo to your machine. Install their test suite runner using `cargo install --locked workshop-runner`. Then start reading the [accompanying book](https://rust-exercises.com/100-exercises/). It has 100 chapters and at the end of each chapter it tells you what exercise to do in the repo. Once you complete an exercise run `wr` in the project root to check that your solution passes the unit tests, and if it does continue on to the next chapter. Rinse and repeat until you're done.\n\nAlthough 100 chapters sounds like a lot they're all quite short so you'll get through them faster than you think. Completing the entire book likely take most people about 15 to 25 hours.\n\nHowever, if you found rustlings too easy or too slow and would like to try something more challenging, then you may want to skip 100ELR.\n\n\n\n### The [Rust track](https://exercism.org/tracks/rust) on Exercism\n\nThe [Rust track](https://exercism.org/tracks/rust) on Exercism is the best collection of small Rust exercises on the internet aside from rustlings and 100ELR. What makes it so good is that many of the exercises were crafted by Rustaceans, and are meant to teach you the aspects of Rust that make it unique and interesting compared to other programming languages. There's an exercise where you have to write a declarative macro! There's even exercise where you have to write unsafe code!\n\nTo get started you have to make an account on Exercism and enroll in the Rust track (both are free). You can then complete the exercises online using Exercism's online code editor or by downloading the exercises to your machine, completing them using your code editor, and submitting your solutions to Exercism using the `exercism` cli tool.\n\nI recommend using the `exercism` cli tool. There are many advantages. First, your code editor has rust-analyzer and Exercism's online code editor doesn't. Second, you can use dark mode in your editor but you can't use dark mode on Exercism unless you pay (it's a premium feature). Third, you can track your solutions using git. Fourth, you can choose which unit tests to run while you're developing your solution, instead of running the full test suite every time. The official Exercism instructions on how to do the exercises locally on your machine are [here](https://exercism.org/docs/using/solving-exercises/working-locally).\n\nEach track on Exercism has a \"learning mode\" and a \"practice mode.\" At the moment the Rust track's \"learning mode\" is disabled while the Exercism team reworks it. Regardless, I think the \"practice mode\" is better because it lets you do the exercises in whatever order you want. So while you may not have the option right now, if the option between a \"learning mode\" and a \"practice mode\" comes back to the Rust track in the future I'd recommend picking practice mode.\n\nWhile the exercises in the Rust track are ordered, the order is arbitrary and following it isn't important. With the exception of the [Luhn](https://exercism.org/tracks/rust/exercises/luhn), [Luhn From](https://exercism.org/tracks/rust/exercises/luhn-from), and [Luhn Trait](https://exercism.org/tracks/rust/exercises/luhn-trait) exercises, all of the other exercises are totally unrelated to each other and can be done in whatever order.\n\nThe best part of completing an exercise is that it gives you the ability to see and star other people's solutions. After completing an exercise I recommend checking these out, especially the most starred solutions, as they can teach you elegant and idiomatic parts of Rust that you may have not discovered on your own.\n\nFurthermore, the exercises are rated by difficulty: easy, medium, or hard. Currently there's 99 total exercises in the Rust track and I would say that about 30 are really good and the other 59 are just generic exercises that were copy & pasted from other tracks. Less than 0.1% of people complete the entire track. I wouldn't be surprised if less than 1% of people even complete half of the track. I have done all of the exercises so I'll recommend which ones I think are the best, since you're likely not going to do all of them anyway.\n\nYou should start with the easy exercises. Difficulty-wise they're slightly more challenging than what you'd find in rustlings or 100ELR. Quality-wise all of them are okay. None of them will take you longer than 20 minutes to complete. Also, if completing the easy exercises stops being rewarding then move on to the medium exercises, don't push yourself to complete all the easy ones if you don't feel like you're getting anything out of them anymore.\n\nAnyway, as for the medium exercises, these are the best ones in my opinion (in no particular order):\n- [PaaS I/O](https://exercism.org/tracks/rust/exercises/paasio)\n- [Simple Linked List](https://exercism.org/tracks/rust/exercises/simple-linked-list)\n- [Fizzy](https://exercism.org/tracks/rust/exercises/fizzy)\n- [Robot Name](https://exercism.org/tracks/rust/exercises/robot-name)\n- [Triangle](https://exercism.org/tracks/rust/exercises/triangle)\n- [Robot Simulator](https://exercism.org/tracks/rust/exercises/robot-simulator)\n- [Accumulate](https://exercism.org/tracks/rust/exercises/accumulate)\n- [Word Count](https://exercism.org/tracks/rust/exercises/word-count)\n- [Grep](https://exercism.org/tracks/rust/exercises/grep)\n- [Luhn](https://exercism.org/tracks/rust/exercises/luhn)\n- [Luhn From](https://exercism.org/tracks/rust/exercises/luhn-from)\n- [Luhn Trait](https://exercism.org/tracks/rust/exercises/luhn-trait)\n- [Clock](https://exercism.org/tracks/rust/exercises/clock)\n- [Space Age](https://exercism.org/tracks/rust/exercises/space-age)\n- [Sublist](https://exercism.org/tracks/rust/exercises/sublist)\n- [Binary Search](https://exercism.org/tracks/rust/exercises/binary-search)\n- [ETL](https://exercism.org/tracks/rust/exercises/etl)\n- [Grade School](https://exercism.org/tracks/rust/exercises/grade-school)\n- [Hamming](https://exercism.org/tracks/rust/exercises/hamming)\n- [Isogram](https://exercism.org/tracks/rust/exercises/isogram)\n- [Nucleotide Count](https://exercism.org/tracks/rust/exercises/nucleotide-count)\n\nDepending on which ones you do they can take you anywhere from 10 to 45 minutes.\n\nOnce you're ready to bump up the difficulty again, these are the best hard exercises in my opinion (in no particular order):\n- [Macros](https://exercism.org/tracks/rust/exercises/macros)\n- [Parallel Letter Frequency](https://exercism.org/tracks/rust/exercises/parallel-letter-frequency)\n- [Xorcism](https://exercism.org/tracks/rust/exercises/xorcism)\n- [React](https://exercism.org/tracks/rust/exercises/react)\n- [Circular Buffer](https://exercism.org/tracks/rust/exercises/circular-buffer)\n- [Forth](https://exercism.org/tracks/rust/exercises/forth)\n- [Doubly Linked List](https://exercism.org/tracks/rust/exercises/doubly-linked-list)\n\nDepending on which ones you do they can take anywhere from 30 to 90 minutes.\n\nIf you do a bunch of easy exercises, plus the medium and hard ones I recommend, it will probably take you around 20 hours. I think completing the entire track will probably be double that, at around 40 hours. With that said don't push yourself to finish the entire track if you feel yourself starting to lose interest, it'd be better to change gears and continue your Rust coding journey trying something else next, because the most important thing is that you keep having fun while learning.\n\n\n\n### [Advent of Code](https://adventofcode.com/)\n\n[Advent of Code](https://adventofcode.com/) (AoC) is a collection of programming puzzles. Every year since 2015 a new puzzle has been released every day from December 1st to 25th. Each puzzle has 2 parts. The first 10-15 puzzles of each year are usually easy to medium difficulty, and the last 10-15 puzzles of each year are usually hard. The puzzles are designed to be language agnostic, meaning you can solve them using any programming language. So if your goal is to learn Rust then just doing the puzzles is not going to help you very much, but if you complete a puzzle and then review an experienced Rustacean's solution that could help teach you a lot of Rust.\n\nThe experienced Rustacean in this case would be fasterthanlime. On top of publishing his solutions for the 2020 and 2022 AoC puzzles he also wrote beginner-friendly blog posts explaining his thought process while solving each one. If you're going to do AoC to learn Rust then you should start with the 2020 puzzles and after completing each one read the corresponding fasterthanlime blog post on it. I wouldn't recommend going far past puzzle 10 for any given year because they can start to get really challenging and you'll be spending most of your time busting your brain trying to figure out how to solve the puzzle instead of learning Rust.\n\n\n\n#### Setting up\n\nFirst, create an account on [AoC](https://adventofcode.com/). Second, since AoC is language agnostic you have to set up your own cargo project with your own scaffolding in order to solve the puzzles. Doing this can be very educational but it's also pretty boring. I'd suggest skipping that and using this Github template instead: [fspoettel/advent-of-code-rust](https://github.com/fspoettel/advent-of-code-rust). Click on `Use this template` -> `Create a new repository`, then `git clone` the created repository to your machine. In `.cargo/config.toml` set `AOC_YEAR` to `2020`.\n\nAlso you can install the `aoc-cli` tool to fetch all instructions and inputs from your AoC account to your machine. To do that run `cargo install aoc-cli --version 0.12.0`, then create the file `$HOME/.adventofcode.session` and paste your AoC session cookie into it. To get your session cookie, press `F12` while anywhere on the AoC website to open your browser's developer tools, then look for `Cookies` under the `Application` or `Storage` tabs and copy the `session` cookie value.\n\nOnce all of that is done you can run `cargo download {day}` within your project directory to generate the scaffolding and download the puzzle for that day. Once you think you've solved it you can run `cargo solve {day}` and then submit your final result via the AoC website to verify. For day 1 the commands would be `cargo download 1` and `cargo solve 1`.\n\n\n\n#### Year 2020\n\nAs mentioned before, make sure to set `AOC_YEAR` to `2020` in `.cargo/config.toml`. After completing a puzzle read fasterthanlime's solution on his blog, here's direct links to his solutions to the first 10 puzzles:\n\n1. [AoC 2020 Day 1](https://fasterthanli.me/series/advent-of-code-2020/part-1)\n1. [AoC 2020 Day 2](https://fasterthanli.me/series/advent-of-code-2020/part-2)\n1. [AoC 2020 Day 3](https://fasterthanli.me/series/advent-of-code-2020/part-3)\n1. [Aoc 2020 Day 4](https://fasterthanli.me/series/advent-of-code-2020/part-4)\n1. [AoC 2020 Day 5](https://fasterthanli.me/series/advent-of-code-2020/part-5)\n1. [AoC 2020 Day 6](https://fasterthanli.me/series/advent-of-code-2020/part-6)\n1. [AoC 2020 Day 7](https://fasterthanli.me/series/advent-of-code-2020/part-7)\n1. [AoC 2020 Day 8](https://fasterthanli.me/series/advent-of-code-2020/part-8)\n1. [AoC 2020 Day 9](https://fasterthanli.me/series/advent-of-code-2020/part-9)\n1. [AoC 2020 Day 10](https://fasterthanli.me/series/advent-of-code-2020/part-10)\n\n\n\n#### Year 2022\n\nIf you completed at least 10 of the 2020 puzzles then year 2022 is the next best year to do. Clone another copy of [fspoettel/advent-of-code-rust](https://github.com/fspoettel/advent-of-code-rust) and set `AOC_YEAR` to `2022` in `.cargo/config.toml` in the project directory. Then here's direct links to fasterthanlime's 2022 solutions:\n\n1. [AoC 2022 Day 1](https://fasterthanli.me/series/advent-of-code-2022/part-1)\n1. [AoC 2022 Day 2](https://fasterthanli.me/series/advent-of-code-2022/part-2)\n1. [AoC 2022 Day 3](https://fasterthanli.me/series/advent-of-code-2022/part-3)\n1. [AoC 2022 Day 4](https://fasterthanli.me/series/advent-of-code-2022/part-4)\n1. [AoC 2022 Day 5](https://fasterthanli.me/series/advent-of-code-2022/part-5)\n1. [AoC 2022 Day 6](https://fasterthanli.me/series/advent-of-code-2022/part-6)\n1. [AoC 2022 Day 7](https://fasterthanli.me/series/advent-of-code-2022/part-7)\n1. [AoC 2022 Day 8](https://fasterthanli.me/series/advent-of-code-2022/part-8)\n1. [AoC 2022 Day 9](https://fasterthanli.me/series/advent-of-code-2022/part-9)\n1. [AoC 2022 Day 10](https://fasterthanli.me/series/advent-of-code-2022/part-10)\n\n\n\n### Tutorials\n\nThe goal of most Rust tutorials is to show you how some piece of software could be implemented in Rust, not to necessarily to teach you Rust, and so many tutorial authors assume their audience will already be somewhat competent with the language and will omit a bunch of beginner explanations for the sake of keeping the tutorial focused and concise.\n\nWith that said, writing a non-trivial piece of software can be a lot more fun and rewarding than completing a bunch of small exercises, so this approach can be preferable for keeping your motivations high, especially if you're building something you're genuinely interested and excited about.\n\nI wish I could recommend some tutorials here but there's so many of them out there that it would be hard for me to go through all of them and rank them by their quality. The quality probably isn't that important though, what's important is that you're writing Rust code and enjoying your time, so pick any tutorial that looks fun and follow along without sweating the details.\n\nI will give a shoutout to [Codecrafters](https://codecrafters.io) though. Their tutorials are clearly very high quality and they have Rust starter templates for all of them. In every step of every tutorial they give you instructions and hints on how to implement the next part of the program, and once you're done you submit your code and it runs against a test suite to check if everything was implemented correctly.\n\nA big downside of Codecrafters is that it costs $40/month. They occasionally run promotions where a tutorial will be free for a month, so depending on when you check their site you may get lucky and a tutorial that catches your eye will be free. However, their pricing suggests they target software companies looking to train their engineers, and not individual engineers themselves. If you're employed at such a company, check if your company provides you a discretionary training allowance for courses like Codecrafters, which you could use to cover the cost.\n\n\n\n## 4\\) Read [Common Rust Lifetime Misconceptions](./common-rust-lifetime-misconceptions.md)\n\nDisclaimer: I wrote this.\n\nReading [Common Rust Lifetime Misconceptions](./common-rust-lifetime-misconceptions.md) will help you understand why half of the stuff you tried in your last 10 hours of Rust coding didn't work 🙂\n\nJokes aside, understanding lifetimes is probably the biggest stumbling block that many beginners struggle to get over when learning Rust, and this article dispels all of the most common misconceptions that people have about lifetimes that cause them confusion and frustration.\n\nThe reason I recommend it after at least 10 hours of practical Rust coding experience is because I think it might be too much technical information too soon for absolute beginners, who may find it more overwhelming than helpful.\n\n\n\n## 5\\) Spend another 10 hours coding in Rust\n\nAs before if you're not sure what to code my suggestions remain the same: [100 Exercises to Learn Rust](https://rust-exercises.com/100-exercises/), the [Rust track](https://exercism.org/tracks/rust) on Exercism, [Advent of Code](https://adventofcode.com/), or tutorials.\n\nAlso since you now have some Rust experience under your belt feel free to try coding something in multithreaded async Rust if you're feeling adventurous.\n\n\n\n## 6\\) Read [Tour of Rust's Standard Library Traits](./tour-of-rusts-standard-library-traits.md)\n\nDisclaimer: I wrote this.\n\nTraits are the main way to write polymorphic code in Rust, so they're used everywhere, especially the ones from the standard library. [Tour of Rust's Standard Library Traits](./tour-of-rusts-standard-library-traits.md) gives a guided tour of the most popular standard library traits, their methods, how to use them, and when to implement them for your own types. It's pretty thorough, and shares a ton of useful tips. And although the article is lengthy you don't have to read the entire thing to get value out of it, so don't let its size daunt you.\n\n\n\n## What's next?\n\nCongrats. You made it. You definitely know enough Rust to forge your own path forward. Good luck and have fun.\n\n\n\n## Honorable mentions\n\nThese are other Rust beginner resources that I really like but wasn't able to find a spot for them in my Rust learning guide.\n\n\n\n### [Tour of Rust](https://tourofrust.com/)\n\nIt's really good. However, it covers a lot of the same ground that other resources in the guide already cover, so it was omitted for the sake of keeping the guide streamlined.\n\n\n\n### [Comprehensive Rust](https://google.github.io/comprehensive-rust/)\n\nThis is a Rust course developed by the Android team at Google. It's a collection of slides which are meant to be presented by a speaker experienced with Rust, but the book version has the speaker notes at the bottom of each page so that it can be learned from as a standalone resource.\n\nIt's very concise and fast-paced, and also has sections covering parts of Rust that are typically omitted from other beginner resources, such as [bare metal Rust](https://google.github.io/comprehensive-rust/bare-metal.html) and [concurrency in Rust](https://google.github.io/comprehensive-rust/concurrency/welcome.html). However, since it's better presented by a speaker it didn't make sense to put it into my guide for self-learners.\n\n\n\n### [Clear explanation of Rust's module system](https://www.sheshbabu.com/posts/rust-module-system/)\n\nThis is a great article. I wasn't sure where to put it in the guide because it could go anywhere, but ideally it comes right when the learner starts breaking their Rust code across multiple files, which comes at a different point for every person in their Rust coding journey. Anyway, whenever that point comes for you, read this article then.\n\n\n\n## Discuss\n\nDiscuss this article on\n- [Github](https://github.com/pretzelhammer/rust-blog/discussions/84)\n- [official Rust users forum](https://users.rust-lang.org/t/learning-rust-in-2024)\n- [learnrust subreddit](https://www.reddit.com/r/learnrust/comments/1fnlvd8/learning_rust_in_2024/)\n- [rust subreddit](https://www.reddit.com/r/rust/comments/1fod8u9/learning_rust_in_2024/)\n\n\n## Further reading\n\n- [Common Rust Lifetime Misconceptions](./common-rust-lifetime-misconceptions.md)\n- [Tour of Rust's Standard Library Traits](./tour-of-rusts-standard-library-traits.md)\n- [Beginner's Guide to Concurrent Programming: Coding a Multithreaded Chat Server using Tokio](./chat-server.md)\n- [Using Rust in Non-Rust Servers to Improve Performance](./rust-in-non-rust-servers.md)\n- [Sizedness in Rust](./sizedness-in-rust.md)\n- [RESTful API in Sync & Async Rust](./restful-api-in-sync-and-async-rust.md)\n- [Learn Assembly with Entirely Too Many Brainfuck Compilers](./too-many-brainfuck-compilers.md)\n\n\n\n## Notifications\n\nGet notified when a new blog post gets published by\n- Subscribing to this repo's [releases RSS feed](https://github.com/pretzelhammer/rust-blog/releases.atom) or\n- Watching this repo's releases (click `Watch` → click `Custom` → select `Releases` → click `Apply`)\n"
},
{
"path": "posts/restful-api-in-sync-and-async-rust.md",
"content": "\n# RESTful API in Sync & Async Rust\n\n_11 May 2021 · #rust · #diesel · #rocket · #sqlx · #actix-web_\n\n**Table of Contents**\n\n- [Intro](#intro)\n- [General](#general)\n - [Project Setup](#project-setup)\n - [Loading Environment Variables w/dotenv](#loading-environment-variables-wdotenv)\n - [Handling Dates & Times w/chrono](#handling-dates--times-wchrono)\n - [Logging w/fern](#logging-wfern)\n - [JSON Serialization w/serde](#json-serialization-wserde)\n - [Domain Modeling](#domain-modeling)\n- [Sync Implementation](#sync-implementation)\n - [SQL Schema Migrations w/diesel-cli](#sql-schema-migrations-wdiesel-cli)\n - [Executing SQL Queries w/Diesel](#executing-sql-queries-wdiesel)\n - [Mapping DB Enums to Rust Enums](#mapping-db-enums-to-rust-enums)\n - [Fetching Data](#fetching-data)\n - [Inserting Data](#inserting-data)\n - [Updating Data](#updating-data)\n - [Deleting Data](#deleting-data)\n - [Using a Connection Pool w/r2d2](#using-a-connection-pool-wr2d2)\n - [Refactoring DB Operations Into a Module](#refactoring-db-operations-into-a-module)\n - [HTTP Routing w/Rocket](#http-routing-wrocket)\n - [Routing Basics](#routing-basics)\n - [GET Requests](#get-requests)\n - [POST & PATCH Requests](#post--patch-requests)\n - [DELETE Requests](#delete-requests)\n - [Refactoring API Routes Into a Module](#refactoring-api-routes-into-a-module)\n - [Authentication](#authentication)\n- [Async Implementation](#async-implementation)\n - [SQL Schema Migrations w/sqlx-cli](#sql-schema-migrations-wsqlx-cli)\n - [Executing SQL Queries w/sqlx](#executing-sql-queries-wsqlx)\n - [Fetching Data](#fetching-data-1)\n - [Inserting Data](#inserting-data-1)\n - [Updating Data](#updating-data-1)\n - [Deleting Data](#deleting-data-1)\n - [Compile-Time Verification of SQL Queries](#compile-time-verification-of-sql-queries)\n - [Using a Connection Pool w/sqlx](#using-a-connection-pool-wsqlx)\n - [Refactoring DB Operations Into a Module](#refactoring-db-operations-into-a-module-1)\n - [HTTP Routing w/actix-web](#http-routing-wactix-web)\n - [Routing Basics](#routing-basics-1)\n - [GET Requests](#get-requests-1)\n - [POST & PATCH Requests](#post--patch-requests-1)\n - [DELETE Requests](#delete-requests-1)\n - [Refactoring API Routes Into a Module](#refactoring-api-routes-into-a-module-1)\n - [Authentication](#authentication-1)\n- [Benchmarks](#benchmarks)\n - [Servers](#servers)\n - [Methodology](#methodology)\n - [Measuring Resource Usage](#measuring-resource-usage)\n - [Results](#results)\n - [Read-Only Workload](#read-only-workload)\n - [Reads + Writes Workload](#reads--writes-workload)\n- [Concluding Thoughts](#concluding-thoughts)\n - [Diesel vs sqlx](#diesel-vs-sqlx)\n - [Rocket vs actix-web](#rocket-vs-actix-web)\n - [Sync Rust vs Async Rust](#sync-rust-vs-async-rust)\n - [Rust vs JS](#rust-vs-js)\n - [In Summary](#in-summary)\n- [Discuss](#discuss)\n- [Further Reading](#further-reading)\n- [Notifications](#notifications)\n\n\n\n## Intro\n\nLet's implement a RESTful API server in Rust for an imaginary Kanban-style project management app. A popular real-world example of such an app is Trello:\n\n\n\nOn its surface Kanban is pretty simple: there's a board and cards. The board represents a project. The cards represent tasks. The position of the cards on the board represents the state and progress of the tasks. The simplest boards have 3 columns for tasks which are: queued (to do), in progress (doing), and done (done).\n\nDespite being simple on its surface, Kanban, and all kinds of project management software in general, is a literal bottomless pit of complexity. There's a million things we could implement, and after we finish the first million things there would be a million more. However, since I'm trying to write a single article and not an entire book series let's keep the feature scope tiny.\n\nThe server should support the ability to:\n- Create boards\n - Boards have names\n- Get a list of all boards\n- Delete boards\n- Create cards\n - Can associate cards with boards\n - Cards have descriptions and statuses\n- Get a list of all the cards on a board\n- Get a board summary: count of all the cards on the board grouped by their status\n- Update cards\n- Delete cards\n\nAnd that's it! To make this project slightly more interesting let's also include token-based authentication for all of the server's endpoints, but let's keep it simple: as long as a request contains a valid token it has access to all of the boards and cards.\n\nFurthermore, to satisfy my own curiosity, and to maximize the educationalness of this article, we're going to write two implementations together: one using sync Rust and the other using async Rust. The first implementation will use r2d2, Diesel, and Rocket. The second implementation will use sqlx, and actix-web. Here's a quick preview of the crates we'll be using for this project:\n\nGeneral crates\n- dotenv (loading environment variables)\n- log + fern (logging)\n- chrono (date & time handling)\n- serde + serde_json (JSON de/serialization)\n\nSync crates\n- diesel-cli (DB schema migrations)\n- diesel + diesel-derive-enum (ORM / building SQL queries)\n- r2d2 (DB connection pool)\n- rocket + rocket_contrib (HTTP routing)\n\nAsync crates\n- sqlx-cli (DB schema migrations)\n- sqlx (executing SQL queries & DB connection pool)\n- actix-web (HTTP routing)\n- futures (general future-related utilities)\n\nAfter finishing both sync and async implementations we'll run some benchmarks to see which has better performance, because everyone loves benchmarks.\n\n\n\n## General\n\n\n\n### Project Setup\n\nAll of the boring instructions for setting this project up, like installing Docker and running locally, are in the [companion code repository](https://github.com/pretzelhammer/kanban). For this article let's focus entirely on the fun part: the Rust!\n\nAfter the initial setup we have this empty `Cargo.toml` file:\n\n```toml\n# Cargo.toml\n\n[package]\nname = \"kanban\"\nversion = \"0.1.0\"\nedition = \"2018\"\n```\n\nAnd this empty main file:\n\n```rust\n// src/main.rs\n\nfn main() {\n println!(\"Hello, world!\");\n}\n```\n\n\n### Loading Environment Variables w/dotenv\n\ncrates\n- dotenv\n\n```diff\n# Cargo.toml\n\n[package]\nname = \"kanban\"\nversion = \"0.1.0\"\nedition = \"2018\"\n\n+[dependencies]\n+dotenv = \"0.15\"\n```\n\nThis crate does one small simple job: it loads variables from an `.env` in the current working directory and adds them to the program's environment variables. Here's the general `.env` file we'll be using:\n\n```bash\n# .env\n\nLOG_LEVEL=INFO\nLOG_FILE=server.log\nDATABASE_URL=postgres://postgres@localhost:5432/postgres \n```\n\nUpdated main file which uses `dotenv`:\n\n```rust\n// src/main.rs\n\ntype StdErr = Box;\n\nfn main() -> Result<(), StdErr> {\n // loads env variables from .env\n dotenv::dotenv()?;\n\n // example\n assert_eq!(\"INFO\", std::env::var(\"LOG_LEVEL\").unwrap());\n\n Ok(())\n}\n```\n\n\n\n### Handling Dates & Times w/chrono\n\ncrates\n- chrono\n\n```diff\n# Cargo.toml\n\n[package]\nname = \"kanban\"\nversion = \"0.1.0\"\nedition = \"2018\"\n\n[dependencies]\ndotenv = \"0.15\"\n+chrono = \"0.4\"\n```\n\nRust's go-to library for handling dates & times is chrono. We're not using the dependency in our project _just yet_ but will very soon after we add a few more dependencies.\n\n\n\n### Logging w/fern\n\ncrates\n- log\n- fern\n\n```diff\n# Cargo.toml\n\n[package]\nname = \"kanban\"\nversion = \"0.1.0\"\nedition = \"2018\"\n\n[dependencies]\ndotenv = \"0.15\"\nchrono = \"0.4\"\n+log = \"0.4\"\n+fern = \"0.6\"\n```\n\nLog is Rust's logging facade library. It provides the high-level logging API but we still need to pick an implementation, and the implementation we're going to use is the fern crate. Fern allows us to easily customize the logging format and also chain multiple outputs so we can log to stderr and a file if we wanted to. After adding log and fern let's encapsulate all of the logging configuration and initialization into its own module:\n\n```rust\n// src/logger.rs\n\nuse std::env;\nuse std::fs;\nuse log::{debug, error, info, trace, warn};\n\npub fn init() -> Result<(), fern::InitError> {\n // pull log level from env\n let log_level = env::var(\"LOG_LEVEL\").unwrap_or(\"INFO\".into());\n let log_level = log_level\n .parse::()\n .unwrap_or(log::LevelFilter::Info);\n\n let mut builder = fern::Dispatch::new()\n .format(|out, message, record| {\n out.finish(format_args!(\n \"[{}][{}][{}] {}\",\n chrono::Local::now().format(\"%H:%M:%S\"),\n record.target(),\n record.level(),\n message\n ))\n })\n .level(log_level)\n // log to stderr\n .chain(std::io::stderr());\n\n // also log to file if one is provided via env\n if let Ok(log_file) = env::var(\"LOG_FILE\") {\n let log_file = fs::File::create(log_file)?;\n builder = builder.chain(log_file);\n }\n\n // globally apply logger\n builder.apply()?;\n\n trace!(\"TRACE output enabled\");\n debug!(\"DEBUG output enabled\");\n info!(\"INFO output enabled\");\n warn!(\"WARN output enabled\");\n error!(\"ERROR output enabled\");\n\n Ok(())\n}\n```\n\nAnd then add that module to our main file:\n\n```diff\n// src/main.rs\n\n+mod logger;\n\ntype StdErr = Box;\n\nfn main() -> Result<(), StdErr> {\n dotenv::dotenv()?;\n+ logger::init()?;\n\n Ok(())\n}\n```\n\nIf we run the program now, since `INFO` is the default logging level, here's what we'd see:\n\n```bash\n$ cargo run\n[08:36:30][kanban::logger][INFO] INFO output enabled\n[08:36:30][kanban::logger][WARN] WARN output enabled\n[08:36:30][kanban::logger][ERROR] ERROR output enabled\n```\n\n\n### JSON Serialization w/serde\n\ncrates\n- serde\n- serde_json\n\n```diff\n# Cargo.toml\n\n[package]\nname = \"kanban\"\nversion = \"0.1.0\"\nedition = \"2018\"\n\n[dependencies]\ndotenv = \"0.15\"\n- chrono = \"0.4\"\n+ chrono = { version = \"0.4\", features = [\"serde\"] }\nlog = \"0.4\"\nfern = \"0.6\"\n+ serde = { version = \"1.0\", features = [\"derive\"] }\n+ serde_json = \"1.0\"\n```\n\nPro-tip: when adding a new dependency to a project it's good to look through existing dependencies to see if they have the new dependency as a feature flag. In this case chrono has serde as a feature flag, which if enabled, adds `serde::Serialize` and `serde::Deserialize` impls to all of chrono's types. This will allow us to use chrono types in our own structs later which we will also derive `serde::Serialize` and `serde::Deserialize` impls for.\n\n\n\n### Domain Modeling\n\nOkay, let's start modeling our domain. We know we will have boards so:\n\n```rust\n#[derive(serde::Serialize)]\n#[serde(rename_all = \"camelCase\")]\npub struct Board {\n pub id: i64,\n pub name: String,\n pub created_at: chrono::DateTime,\n}\n```\n\nUnpacking the new stuff:\n- `#[derive(serde::Serialize)]` derives a `serde::Serialize` impl for `Board` which will allow us to serialize it to JSON using the `serde_json` crate.\n- `#[serde(rename_all = \"camelCase\")]` renames all of the snake_case member identifiers to camelCase when serializing (or vice versa when deserializing). This is because it's a convention to use snake_case names in Rust but JSON is often produced and consumed by JS code and the JS convention is to use camelCase for member identifiers.\n- Making `id` an `i64` instead of an `u64` might seem like an odd choice but since we're using PostgreSQL as our DB we have to do this because PostgreSQL only supports signed integer types.\n- A `created_at` member is always useful to have, if for no other reason than to be able to sort entities by chronological order when no better sort order is available.\n\nOkay, let's add cards and statuses:\n\n```rust\n#[derive(serde::Serialize)]\n#[serde(rename_all = \"camelCase\")]\npub struct Card {\n pub id: i64,\n pub board_id: i64,\n pub description: String,\n pub status: Status,\n pub created_at: chrono::DateTime,\n}\n\n#[derive(serde::Serialize, serde::Deserialize)]\n#[serde(rename_all = \"camelCase\")]\npub enum Status {\n Todo,\n Doing,\n Done,\n}\n```\n\nSince we'd also like to support returning a \"board summary\" which contains the count of all of the cards on a board grouped by their status here's the model for that:\n\n```rust\n#[derive(serde::Serialize)]\npub struct BoardSummary {\n pub todo: i64,\n pub doing: i64,\n pub done: i64,\n}\n```\n\nWhen using the API to create a new board users can provide the board name but not its id, since that will be set by the DB, so we need a model for that as well:\n\n```rust\n#[derive(serde::Deserialize)]\npub struct CreateBoard {\n pub name: String,\n}\n```\n\nLikewise users can also create cards. When creating a card let's assume we only want users to provide the new card's description and what board it should be associated with. The new card will get the default todo status and will get its id set by the DB:\n\n```rust\n#[derive(serde::Deserialize)]\n#[serde(rename_all = \"camelCase\")]\npub struct CreateCard {\n pub board_id: i64,\n pub description: String,\n}\n```\n\nWhen updating a card let's assume we want users to only be able to update the description or the status. It would be pretty weird if we allowed them to move cards between boards which is a pretty unusual feature in most project management apps:\n\n```rust\n#[derive(serde::Deserialize)]\npub struct UpdateCard {\n pub description: String,\n pub status: Status,\n}\n```\n\nThrow all of those into their own module and we get:\n\n```rust\n// src/models.rs\n\n// for GET requests\n\n#[derive(serde::Serialize)]\n#[serde(rename_all = \"camelCase\")]\npub struct Board {\n pub id: i64,\n pub name: String,\n pub created_at: chrono::DateTime,\n}\n\n#[derive(serde::Serialize)]\n#[serde(rename_all = \"camelCase\")]\npub struct Card {\n pub id: i64,\n pub board_id: i64,\n pub description: String,\n pub status: Status,\n pub created_at: chrono::DateTime,\n}\n\n#[derive(serde::Serialize, serde::Deserialize)]\n#[serde(rename_all = \"camelCase\")]\npub enum Status {\n Todo,\n Doing,\n Done,\n}\n\n#[derive(serde::Serialize)]\npub struct BoardSummary {\n pub todo: i64,\n pub doing: i64,\n pub done: i64,\n}\n\n// for POST requests\n\n#[derive(serde::Deserialize)]\npub struct CreateBoard {\n pub name: String,\n}\n\n#[derive(serde::Deserialize)]\n#[serde(rename_all = \"camelCase\")]\npub struct CreateCard {\n pub board_id: i64,\n pub description: String,\n}\n\n// for PATCH requests\n\n#[derive(serde::Deserialize)]\npub struct UpdateCard {\n pub description: String,\n pub status: Status,\n}\n```\n\nAnd the updated main file:\n\n```diff\n// src/main.rs\n\nmod logger;\n+mod models;\n\ntype StdErr = Box;\n\nfn main() -> Result<(), StdErr> {\n dotenv::dotenv()?;\n logger::init()?;\n\n Ok(())\n}\n```\n\n\n## Sync Implementation\n\n\n\n### SQL Schema Migrations w/diesel-cli\n\ncrates\n- diesel-cli\n\n```bash\ncargo install diesel_cli\n```\n\nIf the above command doesn't work at first, it's likely because we don't have all the development libraries for all of diesel-cli's supported databases. Since we're just using PostgreSQL, we can make sure the development libraries are installed with these commands:\n\n```bash\n# macOS\nbrew install postgresql\n\n# ubuntu\napt-get install postgresql libpq-dev\n```\n\nAnd then we can tell cargo to only install diesel-cli with support for PostgreSQL:\n\n```bash\ncargo install diesel_cli --no-default-features --features postgres\n```\n\nOnce we have diesel-cli installed we can use it to create new migrations and execute pending migrations. diesel-cli figures out which DB to connect to by checking the `DATABASE_URL` environment variable, which it will also load from an `.env` file if one exists in the current working directory.\n\nAssuming the DB is currently running and a `DATABASE_URL` environment variable is present, here's the first diesel-cli command we'd run to bootstrap our project:\n\n```bash\ndiesel setup\n```\n\nWith this diesel-cli creates a `migrations` directory where we can generate and write our DB schema migrations. Let's generate our first migration:\n\n```bash\ndiesel migration generate create_boards\n```\n\nThis will create a new directory, e.g. `migrations/---_create_boards`, with an `up.sql` and `down.sql` which is where we'll write our SQL queries. The \"up\" query is for creating or modifying our DB schema, in this case creating a boards table:\n\n```sql\n-- create_boards up.sql\nCREATE TABLE IF NOT EXISTS boards (\n id BIGSERIAL PRIMARY KEY,\n name TEXT NOT NULL,\n created_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT (CURRENT_TIMESTAMP AT TIME ZONE 'utc')\n);\n\n-- seed db with some test data for local dev\nINSERT INTO boards\n(name)\nVALUES\n('Test board 1'),\n('Test board 2'),\n('Test board 3');\n```\n\nAnd the \"down\" query is for reverting the schema changes made in the \"up\" query, in this case dropping the created boards table:\n\n```sql\n-- create_boards down.sql\nDROP TABLE IF EXISTS boards;\n```\n\nWe will also need to store some cards:\n\n```bash\ndiesel migration generate create_cards\n```\n\nThe up query for cards:\n\n```sql\n-- create_cards up.sql\nCREATE TYPE STATUS_ENUM AS ENUM ('todo', 'doing', 'done');\n\nCREATE TABLE IF NOT EXISTS cards (\n id BIGSERIAL PRIMARY KEY,\n board_id BIGINT NOT NULL,\n description TEXT NOT NULL,\n status STATUS_ENUM NOT NULL DEFAULT 'todo',\n created_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT (CURRENT_TIMESTAMP AT TIME ZONE 'utc'),\n CONSTRAINT board_fk\n FOREIGN KEY (board_id)\n REFERENCES boards(id)\n ON DELETE CASCADE\n);\n\n-- seed db with some test data for local dev\nINSERT INTO cards\n(board_id, description, status)\nVALUES\n(1, 'Test card 1', 'todo'),\n(1, 'Test card 2', 'doing'),\n(1, 'Test card 3', 'done'),\n(2, 'Test card 4', 'todo'),\n(2, 'Test card 5', 'todo'),\n(3, 'Test card 6', 'done'),\n(3, 'Test card 7', 'done');\n```\n\nAnd the down query for cards:\n\n```sql\n-- create_cards down.sql\nDROP TABLE IF EXISTS cards;\n```\n\nAfter writing our migrations we can run them with this command:\n\n```bash\ndiesel migration run\n```\n\nThis executes the migrations in chronological order and also writes a Diesel schema file which should look like something like this at this point:\n\n```rust\n// src/schema.rs\n\ntable! {\n boards (id) {\n id -> Int8,\n name -> Text,\n created_at -> Timestamptz,\n }\n}\n\ntable! {\n cards (id) {\n id -> Int8,\n board_id -> Int8,\n description -> Text,\n status -> Status_enum,\n created_at -> Timestamptz,\n }\n}\n\njoinable!(cards -> boards (board_id));\n\nallow_tables_to_appear_in_same_query!(\n boards,\n cards,\n);\n```\n\nThe above file will always be generated by diesel-cli and is not something we should ever try to edit by hand, however the `diesel setup` command from earlier also generates a `diesel.toml` configuration file which we can edit if we need to configure or modify how the Diesel schema is generated:\n\n```toml\n# diesel.toml\n\n[print_schema]\nfile = \"src/schema.rs\"\n```\n\nThis will be useful to know for later.\n\n\n\n### Executing SQL Queries w/Diesel\n\ncrates\n- diesel\n\n```diff\n# Cargo.toml\n\n[package]\nname = \"kanban\"\nversion = \"0.1.0\"\nedition = \"2018\"\n\n[dependencies]\ndotenv = \"0.15\"\nchrono = { version = \"0.4\", features = [\"serde\"] }\nlog = \"0.4\"\nfern = \"0.6\"\nserde = { version = \"1.0\", features = [\"derive\"] }\nserde_json = \"1.0\"\n+diesel = { version = \"1.4\", features = [\"postgres\", \"chrono\"] }\n```\n\nWe enable the postgres and chrono feature flags since we'll be connecting to PostgreSQL and deserializing PostgreSQL's timestamp types to chrono types. Updated main file:\n\n```diff\n// src/main.rs\n\n+#[macro_use]\n+extern crate diesel;\n\nmod logger;\nmod models;\n+mod schema;\n\ntype StdErr = Box;\n\nfn main() -> Result<(), StdErr> {\n dotenv::dotenv()?;\n logger::init()?;\n\n Ok(())\n}\n```\n\nBefore decorating our models with Diesel's derive macros we have to bring the generated types from the Diesel schema file into scope:\n\n```diff\n// src/models.rs\n\n+use crate::schema::*;\n\n// etc\n```\n\nHowever, if we currently try to run `cargo check` we'd get this:\n\n```none\nerror[E0412]: cannot find type `Status_enum` in this scope\n --> src/schema.rs:14:19\n |\n14 | status -> Status_enum,\n | ^^^^^^^^^^^ not found in this scope\n```\n\nUh oh, now what?\n\n\n\n#### Mapping DB Enums to Rust Enums\n\nAs you may recall from the preceding section we defined an enum type in PostgreSQL like so:\n\n```sql\nCREATE TYPE STATUS_ENUM AS ENUM ('todo', 'doing', 'done');\n```\n\nUnfortunately Diesel does not support mapping DB enums to Rust enums out-of-the-box in a convenient way, so we have to pull in an unofficial 3rd-party library to do this for us:\n\n```diff\n# Cargo.toml\n\n[package]\nname = \"kanban\"\nversion = \"0.1.0\"\nedition = \"2018\"\n\n[dependencies]\ndotenv = \"0.15\"\nchrono = { version = \"0.4\", features = [\"serde\"] }\nlog = \"0.4\"\nfern = \"0.6\"\nserde = { version = \"1.0\", features = [\"derive\"] }\nserde_json = \"1.0\"\ndiesel = { version = \"1.4\", features = [\"postgres\", \"chrono\"] }\n+diesel-derive-enum = { version = \"1.1\", features = [\"postgres\"] }\n```\n\nThen we decorate the enum type with the following derive macro and attribute:\n\n```diff\n-#[derive(serde::Serialize, serde::Deserialize)]\n+#[derive(serde::Serialize, serde::Deserialize, diesel_derive_enum::DbEnum)]\n#[serde(rename_all = \"camelCase\")]\n+#[DieselType = \"Status_enum\"]\npub enum Status {\n Todo,\n Doing,\n Done,\n}\n```\n\nThe derive macro generates a new type called `Status_enum` within the same scope, and this type can be used by Diesel to map the `STATUS_ENUM` DB enum to our `Status` Rust enum.\n\nThen we update `diesel.toml` with this:\n\n```diff\n[print_schema]\nfile = \"src/schema.rs\"\n+import_types = [\"diesel::sql_types::*\", \"crate::models::Status_enum\"]\n```\n\nThen we re-generate the Diesel schema file using this command:\n\n```bash\ndiesel print-schema > ./src/schema.rs\n```\n\nWhich updates our Diesel schema file to this:\n\n```diff\n// src/schema.rs\n\ntable! {\n+ use diesel::sql_types::*;\n+ use crate::models::Status_enum;\n\n boards (id) {\n id -> Int8,\n name -> Text,\n created_at -> Timestamptz,\n }\n}\n\ntable! {\n+ use diesel::sql_types::*;\n+ use crate::models::Status_enum;\n\n cards (id) {\n id -> Int8,\n board_id -> Int8,\n description -> Text,\n status -> Status_enum,\n created_at -> Timestamptz,\n }\n}\n\njoinable!(cards -> boards (board_id));\n\nallow_tables_to_appear_in_same_query!(\n boards,\n cards,\n);\n```\n\nAnd now when we run `cargo check` everything type checks just fine. Wow, that was quite the annoying hassle, but thankfully it's over.\n\n\n\n#### Fetching Data\n\nAlright, now let's impl some Diesel traits for our models by using the handy derive macros:\n\n```diff\n-#[derive(serde::Serialize)]\n+#[derive(serde::Serialize, diesel::Queryable)]\n#[serde(rename_all = \"camelCase\")]\npub struct Board {\n pub id: i64,\n pub name: String,\n pub created_at: chrono::DateTime,\n}\n```\n\nDeriving `diesel::Queryable` for a struct allows it to be returned as the result of a Diesel query. The order of the struct's members must match the order of the columns in the rows returned from the Diesel query.\n\nThe schema file generated by Diesel exports a DSL we can use to craft DB queries with. One of the major benefits of using the DSL is that it gives us verification at compile-time that all of our SQL queries are syntactically and semantically correct! Here's some commented examples:\n\n```rust\nuse diesel::prelude::*;\nuse diesel::PgConnection;\n\n// generated by diesel-cli from DB schema\nuse crate::schema::boards;\n\n// handwritten by us\nuse crate::models::Board;\n\n// example of connecting to PostgreSQL\nfn get_connection() -> PgConnection {\n dotenv::dotenv().unwrap();\n let db_url = env::var(\"DATABASE_URL\").unwrap();\n PgConnection::establish(&db_url).unwrap()\n}\n\n// fetches all boards from the boards table,\n// diesel can map the DB rows to the Board model\n// because we derived Queryable for it and the\n// DB row data matches the Board's members\nfn all_boards(conn: &PgConnection) -> Vec {\n boards::table\n .load(conn)\n .unwrap()\n}\n\n// fetching boards in chronological order\nfn all_boards_chronological(conn: &PgConnection) -> Vec {\n boards::table\n .order_by(boards::created_at.asc())\n .load(conn)\n .unwrap()\n}\n\n// fetching a board by its id\nfn board_by_id(conn: &PgConnection, board_id: i64) -> Board {\n boards::table\n .filter(boards::id.eq(board_id))\n .first(conn)\n .unwrap()\n}\n\n// fetching boards by their exact name\nfn board_by_name(conn: &PgConnection, board_name: &str) -> Vec {\n boards::table\n .filter(boards::name.eq(board_name))\n .load(conn)\n .unwrap()\n}\n\n// fetching boards if their names contain some string\nfn board_name_contains(conn: &PgConnection, contains: &str) -> Vec {\n // in LIKE queries \"%\" means \"match zero or more of any character\"\n let contains = format!(\"%{}%\", contains);\n boards::table\n .filter(boards::name.ilike(contains))\n .load(conn)\n .unwrap()\n}\n\n// fetching boards created within the past 24 hours\nfn recent_boards(conn: &PgConnection) -> Vec {\n let past_day = chrono::Utc::now() - chrono::Duration::days(1);\n boards::table\n .filter(boards::created_at.ge(past_day))\n .load(conn)\n .unwrap()\n}\n\n// fetching boards create within the past 24 hours\n// AND whose name contains some string\n// by chaining filter methods\nfn recent_boards_and_name_contains(conn: &PgConnection, contains: &str) -> Vec {\n let contains = format!(\"%{}%\", contains);\n let past_day = chrono::Utc::now() - chrono::Duration::days(1);\n boards::table\n .filter(boards::name.ilike(contains))\n .filter(boards::created_at.ge(past_day))\n .load(conn)\n .unwrap()\n}\n\n// fetching boards create within the past 24 hours\n// AND whose name contains some string\n// by composing predicate expressions\nfn recent_boards_and_name_contains_2(conn: &PgConnection, contains: &str) -> Vec {\n let contains = format!(\"%{}%\", contains);\n let past_day = chrono::Utc::now() - chrono::Duration::days(1);\n let predicate = boards::name.ilike(contains).and(boards::created_at.ge(past_day));\n boards::table\n .filter(predicate)\n .load(conn)\n .unwrap()\n}\n\n// fetching boards create within the past 24 hours\n// OR whose name contains some string\n// by chaining filter methods\nfn recent_boards_or_name_contains(conn: &PgConnection, contains: &str) -> Vec {\n let contains = format!(\"%{}%\", contains);\n let past_day = chrono::Utc::now() - chrono::Duration::days(1);\n boards::table\n .filter(boards::name.ilike(contains))\n .or_filter(boards::created_at.ge(past_day))\n .load(conn)\n .unwrap()\n}\n\n// fetching boards create within the past 24 hours\n// OR whose name contains some string\n// by composing predicate expressions\nfn recent_boards_or_name_contains_2(conn: &PgConnection, contains: &str) -> Vec {\n let contains = format!(\"%{}%\", contains);\n let past_day = chrono::Utc::now() - chrono::Duration::days(1);\n let predicate = boards::name.ilike(contains).or(boards::created_at.ge(past_day));\n boards::table\n .filter(predicate)\n .load(conn)\n .unwrap()\n}\n```\n\nOkay, let's also derive `diesel::Queryable` for cards:\n\n```diff\n-#[derive(serde::Serialize)]\n+#[derive(serde::Serialize, diesel::Queryable)]\n#[serde(rename_all = \"camelCase\")]\npub struct Card {\n pub id: i64,\n pub board_id: i64,\n pub description: String,\n pub status: Status,\n pub created_at: chrono::DateTime,\n}\n```\n\nWe went over a bunch of query examples above but here's a few more:\n\n```rust\nuse diesel::prelude::*;\nuse diesel::PgConnection;\n\n// generated by diesel-cli from DB schema\nuse crate::schema::cards;\n\n// handwritten by us\nuse crate::models::{Card, Status};\n\n// fetch all cards\nfn all_cards(conn: &PgConnection) -> Vec {\n cards::table\n .load(conn)\n .unwrap()\n}\n\n// fetch cards by board\nfn cards_by_board(conn: &PgConnection, board_id: i64) -> Vec {\n cards::table\n .filter(cards::board_id.eq(board_id))\n .load(conn)\n .unwrap()\n}\n\n// fetch cards by status\nfn cards_by_status(conn: &PgConnection, status: Status) -> Vec {\n cards::table\n .filter(cards::status.eq(status))\n .load(conn)\n .unwrap()\n}\n```\n\nSo it seems like we can craft almost every query our server needs to support using Diesel's DSL. _Almost_. One of the queries we'd like to support is returning a \"board summary\" which is the count of all the cards within a board grouped by their status. This is how we'd write the SQL query:\n\n```sql\nSELECT count(*), status\nFROM cards\nWHERE cards.board_id = $1\nGROUP BY status;\n```\n\nUnfortunately we cannot use Diesel's generated DSL to craft this query. Like all ORMs though, Diesel provides a method to run SQL directly, and that method is `diesel::sql_query` so let's talk about how we'd use that to get our board summary.\n\nFirst we need to create a new model which will be the result of our query and derive `diesel::QueryableByName` for it as well as decorate its members with the diesel types they map to:\n\n```rust\n#[derive(Default, serde::Serialize)]\npub struct BoardSummary {\n pub todo: i64,\n pub doing: i64,\n pub done: i64,\n}\n\n// this will be the result of our diesel::sql_query query\n#[derive(diesel::QueryableByName)]\npub struct StatusCount {\n #[sql_type = \"diesel::sql_types::BigInt\"]\n pub count: i64,\n #[sql_type = \"Status_enum\"]\n pub status: Status,\n}\n\n// converting from a list of StatusCounts to a BoardSummary\nimpl From> for BoardSummary {\n fn from(counts: Vec) -> BoardSummary {\n let mut summary = BoardSummary::default();\n for StatusCount { count, status } in counts {\n match status {\n Status::Todo => summary.todo += count,\n Status::Doing => summary.doing += count,\n Status::Done => summary.done += count,\n }\n }\n summary\n }\n}\n```\n\nWhy do structs which are the result of \"regular\" Diesel queries have to derive `diesel::Queryable` but structs which are the result of `diesel::sql_query` queries have to derive `diesel::QueryableByName`? According to the Diesel docs the former deserializes DB row columns by index and the latter deserializes DB row columns by name, and the first one is more performant but the latter is a bit more foolproof, hence its use for arbitrary SQL queries.\n\nAnyway, after jumping through all of the hoops above we can finally implement a function to fetch board summaries:\n\n```rust\n// fetching summary of cards on a board by id\nfn board_summary(conn: &PgConnection, board_id: i64) -> BoardSummary {\n diesel::sql_query(format!(\n \"SELECT count(*), status FROM cards WHERE cards.board_id = {} GROUP BY status\",\n board_id\n ))\n .load::(conn)\n .unwrap()\n .into()\n}\n```\n\n\n\n#### Inserting Data\n\nWe don't want to just fetch boards, we'd like to create them as well! As a reminder, the `CreateBoard` model only has a `name` member because the `id` and `created_at` members will be set by the DB. And this is how we'd update our `CreateBoard` model to use it with Diesel's DSL:\n\n```diff\n// boards has to be in scope to be used in table_name attribute\nuse crate::schema::boards;\n\n-#[derive(serde::Deserialize)]\n+#[derive(serde::Deserialize, diesel::Insertable)]\n+#[table_name = \"boards\"]\npub struct CreateBoard {\n pub name: String,\n}\n```\n\nAfter deriving an `diesel::Insertable` impl for `CreateBoard` we can use it directly in Diesel insert queries:\n\n```rust\nuse diesel::prelude::*;\nuse diesel::PgConnection;\n\nuse crate::schema::boards;\nuse crate::models::{Board, CreateBoard};\n\n// create and return new board from CreateBoard model\nfn create_board(conn: &PgConnection, create_board: CreateBoard) -> Board {\n diesel::insert_into(boards::table)\n .values(&create_board)\n .get_result(conn) // return inserted board result\n .unwrap()\n}\n\n// same as above except without using the CreateBoard model\nfn create_board_2(conn: &PgConnection, board_name: String) -> Board {\n diesel::insert_into(boards::table)\n .values(boards::name.eq(board_name))\n .get_result(conn) // return inserted board result\n .unwrap()\n}\n```\n\nWe follow the same steps for the `CreateCard` struct:\n\n```diff\n// cards has to be in scope to be used in table_name attribute\nuse crate::schema::cards;\n\n- #[derive(serde::Deserialize)]\n+ #[derive(serde::Deserialize, diesel::Insertable)]\n#[serde(rename_all = \"camelCase\")]\n+ #[table_name = \"cards\"]\npub struct CreateCard {\n pub board_id: i64,\n pub description: String,\n}\n```\n\nExample create functions:\n\n```rust\nuse diesel::prelude::*;\nuse diesel::PgConnection;\n\nuse crate::schema::cards;\nuse crate::models::{Card, CreateCard};\n\n// create and return new card using CreateCard model\nfn create_card(conn: &PgConnection, create_card: CreateCard) -> Card {\n diesel::insert_into(cards::table)\n .values(&create_card)\n .get_result(conn) // return inserted card result\n .unwrap()\n}\n\n// same as above except without using CreateCard model\nfn create_card_2(conn: &PgConnection, board_id: i64, description: String) -> Card {\n diesel::insert_into(cards::table)\n .values((cards::board_id.eq(board_id), cards::description.eq(description)))\n .get_result(conn) // returns inserted card result\n .unwrap()\n}\n```\n\n\n\n#### Updating Data\n\nIf we derive `diesel::AsChangeSet` for the `UpdateCard` model:\n\n```diff\n// cards has to be in scope to be used in table_name attribute\nuse crate::schema::cards;\n\n-#[derive(serde::Deserialize)]\n+#[derive(serde::Deserialize, diesel::AsChangeset)]\n+#[table_name = \"cards\"]\npub struct UpdateCard {\n pub description: String,\n pub status: Status,\n}\n```\n\nWe can now pass `UpdateCard` directly to the `set` method of Diesel update queries:\n\n```rust\nuse diesel::prelude::*;\nuse diesel::PgConnection;\n\nuse crate::schema::cards;\nuse crate::models::{Card, UpdateCard};\n\n// update card in DB using UpdateCard model\nfn update_card(conn: &PgConnection, card_id: i64, update_card: UpdateCard) -> Card {\n diesel::update(cards::table.filter(cards::id.eq(card_id)))\n .set(update_card)\n .get_result(conn) // return updated card result\n .unwrap()\n}\n\n// same as above except without using model\nfn update_card_2(conn: &PgConnection, card_id: i64, description: String, status: Status) -> Card {\n diesel::update(cards::table.filter(cards::id.eq(card_id)))\n .set((cards::description.eq(description), cards::status.eq(status)))\n .get_result(conn) // return updated card result\n .unwrap()\n}\n```\n\n\n\n#### Deleting Data\n\nSome delete query examples:\n\n```rust\nuse diesel::prelude::*;\nuse diesel::PgConnection;\nuse crate::schema::{boards, cards};\n\n// delete all boards\nfn delete_all_boards(conn: &PgConnection) {\n diesel::delete(boards::table)\n .execute(conn)\n .unwrap();\n}\n\n// delete a board by its id\nfn delete_board_by_id(conn: &PgConnection, board_id: i64) {\n diesel::delete(boards::table.filter(boards::id.eq(board_id)))\n .execute(conn)\n .unwrap();\n}\n\n// delete all cards\nfn delete_all_cards(conn: &PgConnection) {\n diesel::delete(cards::table)\n .execute(conn)\n .unwrap();\n}\n\n// delete a card by its id\nfn delete_card_by_id(conn: &PgConnection, card_id: i64) {\n diesel::delete(cards::table.filter(cards::id.eq(card_id)))\n .execute(conn)\n .unwrap();\n}\n\n// delete all the cards on a board\nfn delete_cards_by_board(conn: &PgConnection, board_id: i64) {\n diesel::delete(cards::table.filter(cards::board_id.eq(board_id)))\n .execute(conn)\n .unwrap();\n}\n\n// delete all done cards on a board\nfn delete_done_cards_by_board(conn: &PgConnection, board_id: i64) {\n diesel::delete(\n cards::table\n .filter(cards::board_id.eq(board_id))\n .filter(cards::status.eq(Status::Done)),\n )\n .execute(conn)\n .unwrap();\n}\n```\n\n\n\n#### Using a Connection Pool w/r2d2\n\ncrates\n- r2d2\n\nCreating DB connections is expensive, so let's use a connection pool library to handle the hard work of managing and reusing DB connections for us. The library we're going to use is r2d2, which comes as a standalone crate but can also can come packaged within Diesel if we enable the r2d2 feature flag, so let's do that:\n\n```diff\n# Cargo.toml\n\n[package]\nname = \"kanban\"\nversion = \"0.1.0\"\nedition = \"2018\"\n\n[dependencies]\ndotenv = \"0.15\"\nchrono = { version = \"0.4\", features = [\"serde\"] }\nlog = \"0.4\"\nfern = \"0.6\"\nserde = { version = \"1.0\", features = [\"derive\"] }\nserde_json = \"1.0\"\n-diesel = { version = \"1.4\", features = [\"postgres\", \"chrono\"] }\n+diesel = { version = \"1.4\", features = [\"postgres\", \"chrono\", \"r2d2\"] }\ndiesel-derive-enum = { version = \"1.1\", features = [\"postgres\"] }\n```\n\nHere's how we'd create a pool with the default configuration options:\n\n```rust\nuse diesel::prelude::*;\nuse diesel::PgConnection;\nuse diesel::r2d2;\n\ntype PgPool = r2d2::Pool>;\n\nfn get_pool() -> PgPool {\n dotenv::dotenv().unwrap();\n let db_url = env::var(\"DATABASE_URL\").unwrap();\n let manager = r2d2::ConnectionManager::new(db_url);\n let pool = r2d2::Pool::new(manager).unwrap();\n pool\n}\n\nfn use_pool(pool: &PgPool) {\n let connection = pool.get().unwrap();\n // use connection\n}\n```\n\n\n\n#### Refactoring DB Operations Into a Module\n\nThe rest of our application doesn't need to know or care that we're using Diesel and r2d2, so let's abstract all of those implementation details away within a `Db` struct, which we'll put into a `db` module:\n\n```rust\n// src/db.rs\n\nuse diesel::prelude::*;\nuse diesel::PgConnection;\nuse diesel::r2d2;\n\nuse crate::StdErr;\nuse crate::models::*;\nuse crate::schema::*;\n\ntype PgPool = r2d2::Pool>;\n\npub struct Db {\n pool: PgPool,\n}\n\nimpl Db {\n pub fn connect() -> Result {\n let db_url = env::var(\"DATABASE_URL\")?;\n let manager = r2d2::ConnectionManager::new(db_url);\n let pool = r2d2::Pool::new(manager)?;\n Ok(Db { pool })\n }\n\n pub fn boards(&self) -> Result, StdErr> {\n let conn = self.pool.get()?;\n Ok(boards::table.load(&conn)?)\n }\n\n pub fn board_summary(&self, board_id: i64) -> Result {\n let conn = self.pool.get()?;\n let counts: Vec = diesel::sql_query(format!(\n \"select count(*), status from cards where cards.board_id = {} group by status\",\n board_id\n ))\n .load(&conn)?;\n Ok(counts.into())\n }\n\n pub fn create_board(&self, create_board: CreateBoard) -> Result {\n let conn = self.pool.get()?;\n let board = diesel::insert_into(boards::table)\n .values(&create_board)\n .get_result(&conn)?;\n Ok(board)\n }\n\n pub fn delete_board(&self, board_id: i64) -> Result<(), StdErr> {\n let conn = self.pool.get()?;\n diesel::delete(boards::table.filter(boards::id.eq(board_id))).execute(&conn)?;\n Ok(())\n }\n\n pub fn cards(&self, board_id: i64) -> Result, StdErr> {\n let conn = self.pool.get()?;\n let cards = cards::table\n .filter(cards::board_id.eq(board_id))\n .load(&conn)?;\n Ok(cards)\n }\n\n pub fn create_card(&self, create_card: CreateCard) -> Result {\n let conn = self.pool.get()?;\n let card = diesel::insert_into(cards::table)\n .values(create_card)\n .get_result(&conn)?;\n Ok(card)\n }\n\n pub fn update_card(&self, card_id: i64, update_card: UpdateCard) -> Result {\n let conn = self.pool.get()?;\n let card = diesel::update(cards::table.filter(cards::id.eq(card_id)))\n .set(update_card)\n .get_result(&conn)?;\n Ok(card)\n }\n\n pub fn delete_card(&self, card_id: i64) -> Result<(), StdErr> {\n let conn = self.pool.get()?;\n diesel::delete(\n cards::table.filter(cards::id.eq(card_id)),\n )\n .execute(&conn)?;\n Ok(())\n }\n}\n```\n\nUpdated main file:\n\n```diff\n// src/main.rs\n\n#[macro_use]\nextern crate diesel;\n\n+mod db;\nmod logger;\nmod models;\nmod schema;\n\ntype StdErr = Box;\n\nfn main() -> Result<(), StdErr> {\n dotenv::dotenv()?;\n logger::init()?;\n\n Ok(())\n}\n```\n\n\n### HTTP Routing w/Rocket\n\ncrates\n- rocket\n- rocket_contrib\n\n```diff\n# Cargo.toml\n\n[package]\nname = \"kanban\"\nversion = \"0.1.0\"\nedition = \"2018\"\n\n[dependencies]\ndotenv = \"0.15\"\nchrono = { version = \"0.4\", features = [\"serde\"] }\nlog = \"0.4\"\nfern = \"0.6\"\nserde = { version = \"1.0\", features = [\"derive\"] }\nserde_json = \"1.0\"\ndiesel = { version = \"1.4\", features = [\"postgres\", \"chrono\", \"r2d2\"] }\ndiesel-derive-enum = { version = \"1.1\", features = [\"postgres\"] }\n+rocket = \"0.4\"\n+rocket_contrib = \"0.4\"\n```\n\nRocket v0.4 depends on some nightly features, some of which are only available on compiler version 1.53.0 since they're later removed, so we'll need to install a specific nightly compiler and then switch to using these commands\n\n```bash\n# install rustc 1.53.0-nightly (07e0e2ec2 2021-03-24)\nrustup install nightly-2021-03-24\n\n# set it as default\nrustup default nightly-2021-03-24\n```\n\nWe also have to add some compiler feature flags to the top of our main:\n\n```diff\n// src/main.rs\n\n+// required for rocket macros to work\n+#![feature(proc_macro_hygiene, decl_macro)]\n\n#[macro_use]\nextern crate diesel;\n\nmod db;\nmod logger;\nmod models;\nmod schema;\n\ntype StdErr = Box;\n\nfn main() -> Result<(), StdErr> {\n dotenv::dotenv()?;\n logger::init()?;\n\n Ok(())\n}\n```\n\nWhile we're here let's throw together a quick Rocket hello world:\n\n```diff\n// required for rocket macros to work\n#![feature(proc_macro_hygiene, decl_macro)]\n\n#[macro_use]\nextern crate diesel;\n\nmod db;\nmod logger;\nmod models;\nmod routes;\nmod schema;\n\ntype StdErr = Box;\n\n+#[rocket::get(\"/\")]\n+fn hello_world() -> &'static str {\n+ \"Hello, world!\"\n+}\n\nfn main() -> Result<(), StdErr> {\n dotenv::dotenv()?;\n logger::init()?;\n\n+ rocket::ignite()\n+ .mount(\"/\", rocket::routes![hello_world])\n+ .launch();\n\n Ok(())\n}\n```\n\nThe code above does exactly what you think it does. Also, if we `cargo run` it:\n\n```bash\n$ cargo run\n[13:07:21][launch][INFO] Configured for development.\n[13:07:21][launch_][INFO] address: localhost\n[13:07:21][launch_][INFO] port: 8000\n[13:07:21][launch_][INFO] log: normal\n[13:07:21][launch_][INFO] workers: 32\n[13:07:21][launch_][INFO] secret key: generated\n[13:07:21][launch_][INFO] limits: forms = 32KiB\n[13:07:21][launch_][INFO] keep-alive: 5s\n[13:07:21][launch_][INFO] read timeout: 5s\n[13:07:21][launch_][INFO] write timeout: 5s\n[13:07:21][launch_][INFO] tls: disabled\n[13:07:21][rocket::rocket][INFO] Mounting /:\n[13:07:21][_][INFO] GET / (hello_world)\n[13:07:21][launch][INFO] Rocket has launched from http://localhost:8000\n```\n\nLook at all those beautiful logs! I bet you were wondering if, and when, all the logging setup work we did forever ago was going to become useful. Well that time is now! Of course, being good developers who care about being able to debug issues when they arise, we should be logging way more in our application, but I've been intentionally omitting the logging statements because they're noisy and have little educational value. Please imagine that they are there.\n\n\n\n#### Routing Basics\n\nWe can decorate functions with Rocket's procedural macros to turn them into request handlers, example:\n\n```rust\n#[rocket::get(\"/\")]\nfn index() -> &'static str {\n \"I'm the index route!\"\n}\n\n#[rocket::get(\"/nested/route\")]\nfn index() -> &'static str {\n \"I'm some nested route!\"\n}\n```\n\nData can be extracted from the path using `\")]\nfn greet(name: String) -> String {\n format!(\"Hello, {}!\", name)\n}\n```\n\nAny type which impls `rocket::request::FromParam` can be extracted from the path:\n\n```rust\nuse rocket::request::FromParam;\n\n// Rocket provides FromParam impls for\n// - all stdlib number types\n// - bool\n// - String\n// - &str\n// - some other misc types\n#[rocket::get(\"/echo////etc\")]\nfn echo_path(string: String, num: usize, maybe: bool) -> String {\n format!(\"got string {}, num {}, and maybe {}\", string, num, maybe)\n}\n\n// Rocket also provides FromParam impls for\n// - Option where T: FromParam\n// - returns Some(T) on success, None otherwise\n// - Result where T: FromParam\n// - returns Ok(T) on success, Err(::Error) otherwise\n\n// example custom type\nstruct EvenNumber(i32);\n\n// example FromParam impl\nimpl<'a> FromParam<'a> for EvenNumber {\n type Error = &'static str;\n\n fn from_param(param: &'a RawStr) -> Result {\n let result = i32::from_param(param);\n if let Ok(num) = result {\n if num % 2 == 0 {\n Ok(EvenNumber(num))\n } else {\n Err(\"param not an even number\")\n }\n } else {\n Err(\"param not a number\")\n }\n }\n}\n\n// extracting our own custom defined type\n#[rocket::get(\"/\")]\nfn even_param(even: EvenNumber) -> String {\n format!(\"got even number {}\", even.0)\n}\n```\n\nA request handler's return type can be anything that impls `rocket::response::Responder`:\n\n```rust\nuse std::io;\nuse std::fs::File;\nuse rocket::{http, request, response};\nuse rocket::response::{Response, Responder};\n\n// Rocket provides Responder impls for\n// - &str\n// - String\n// - &[u8]\n// - Vec\n// - File\n// - ()\n// - some other misc types\n#[rocket::get(\"/cargo\")]\nfn returns_cargo() -> File {\n File::open(\"Cargo.toml\").unwrap()\n}\n\n// Rocket also provides Responder impls for\n// - Option where T: Responder\n// - returns T if Some(T), 404 Not Found if None\n// - Result where T: Responder, E: Debug\n// - returns T if Ok(T), 500 Internal Server Error if Err(E)\n\n// example custom type\nstruct EvenNumber(i32);\n\n// example Responder impl\nimpl<'r> Responder<'r> for EvenNumber {\n fn respond_to(self, _req: &request::Request<'_>) -> response::Result<'r> {\n Response::build()\n .status(http::Status::Ok)\n .raw_header(\"X-Number-Parity\", \"Even\")\n .sized_body(io::Cursor::new(format!(\"returning even number {}\", self.0)))\n .ok()\n }\n}\n\n// returning custom defined type\n#[rocket::get(\"/even\")]\nfn returns_even() -> EvenNumber {\n EvenNumber(2)\n}\n```\n\n\n\n#### GET Requests\n\nThe first problem we need to solve is how to pass our `Db` to our Rocket request handlers. We can achieve this by first passing the `Db` instance to the Rocket application builder via the `manage` method which tells Rocket this is some application state that it should manage for us:\n\n```diff\n// src/main.rs\n\n// required for rocket macros to work\n#![feature(proc_macro_hygiene, decl_macro)]\n\n#[macro_use]\nextern crate diesel;\n\nmod db;\nmod logger;\nmod models;\nmod schema;\nmod routes;\n\ntype StdErr = Box;\n\n#[rocket::get(\"/\")]\nfn hello_world() -> &'static str {\n \"Hello, world!\"\n}\n\nfn main() -> Result<(), StdErr> {\n dotenv::dotenv()?;\n logger::init()?;\n\n+ let db = db::Db::connect()?;\n\n rocket::ignite()\n+ .manage(db)\n .mount(\"/\", rocket::routes![hello_world])\n .launch();\n\n Ok(())\n}\n```\n\nThen we can retrieve the managed application state from Rocket in our request handlers by wrapping the data type we wish to receive with `rocket::State` as one of our function parameters:\n\n```rust\nuse rocket::State;\nuse crate::db::Db;\n\n#[rocket::get(\"/use/db\")]\nfn use_db(db: State) {\n // use db\n}\n```\n\nWe can automatically return JSON by wrapping the return type with the `rocket_contrib::json::Json` type which impls `Responder` and will serialize the wrapped type to JSON using serde and set the response header `Content-Type: application/json`, example:\n\n```rust\nuse rocket_contrib::json::Json;\nuse crate::models::Status;\n\n#[rocket::get(\"/example/json\")]\nfn return_json() -> Json {\n Json(Status::Todo)\n}\n```\n\nOkay, we now have enough context to write all of our GET request handlers:\n\n```rust\nuse rocket::State;\nuse rocket_contrib::json::Json;\n\nuse crate::StdErr;\nuse crate::db::Db;\nuse crate::models::{Board, Card, BoardSummary};\n\n// GET requests\n\n#[rocket::get(\"/boards\")]\nfn boards(db: State) -> Result>, StdErr> {\n db.boards().map(Json)\n}\n\n#[rocket::get(\"/boards//summary\")]\nfn board_summary(db: State, board_id: i64) -> Result, StdErr> {\n db.board_summary(board_id).map(Json)\n}\n\n#[rocket::get(\"/boards//cards\")]\nfn cards(db: State, board_id: i64) -> Result>, StdErr> {\n db.cards(board_id).map(Json)\n}\n```\n\n\n\n#### POST & PATCH Requests\n\n`rocket_contrib::json::Json` not only impls `Responder` but also impls `rocket::data::FromData` which means we can use it as a function parameter to a request handler and Rocket will attempt deserialize the request's JSON body into the type we specify, so here's how we'd write the POST & PATCH request handlers:\n\n```rust\nuse rocket::State;\nuse rocket_contrib::json::Json;\n\nuse crate::StdErr;\nuse crate::db::Db;\nuse crate::models::{Board, CreateBoard, Card, CreateCard, UpdateCard};\n\n// POST requests\n\n#[rocket::post(\"/boards\", data = \"\")]\nfn create_board(db: State, create_board: Json) -> Result, StdErr> {\n db.create_board(create_board.0).map(Json)\n}\n\n#[rocket::post(\"/cards\", data = \"\")]\nfn create_card(db: State, create_card: Json) -> Result, StdErr> {\n db.create_card(create_card.0).map(Json)\n}\n\n// PATCH requests\n\n#[rocket::patch(\"/cards/\", data = \"\")]\nfn update_card(\n db: State,\n card_id: i64,\n update_card: Json,\n) -> Result, StdErr> {\n db.update_card(card_id, update_card.0).map(Json)\n}\n```\n\n\n\n#### DELETE Requests\n\nThe DELETE request handlers:\n\n```rust\nuse rocket::State;\nuse rocket_contrib::json::Json;\n\nuse crate::StdErr;\nuse crate::db::Db;\n\n#[rocket::delete(\"/boards/\")]\nfn delete_board(db: State, board_id: i64) -> Result<(), StdErr> {\n db.delete_board(board_id)\n}\n\n#[rocket::delete(\"/cards/\")]\nfn delete_card(db: State, card_id: i64) -> Result<(), StdErr> {\n db.delete_card(card_id)\n}\n```\n\n\n\n#### Refactoring API Routes Into A Module\n\nLet's clean up our code and put all of the API routes into their own module:\n\n```rust\n// src/routes.rs\n\nuse rocket::http;\nuse rocket::response;\nuse rocket::request\nuse rocket::State;\nuse rocket_contrib::json::Json;\n\nuse crate::StdErr;\nuse crate::db::Db;\nuse crate::models::{Board, CreateBoard, BoardSummary, Card, CreateCard, UpdateCard};\n\n// board routes\n\n#[rocket::get(\"/boards\")]\nfn boards(db: State) -> Result>, StdErr> {\n db.boards().map(Json)\n}\n\n#[rocket::post(\"/boards\", data = \"\")]\nfn create_board(db: State, create_board: Json) -> Result, StdErr> {\n db.create_board(create_board.0).map(Json)\n}\n\n#[rocket::get(\"/boards//summary\")]\nfn board_summary(db: State, board_id: i64) -> Result, StdErr> {\n db.board_summary(board_id).map(Json)\n}\n\n#[rocket::delete(\"/boards/\")]\nfn delete_board(db: State, board_id: i64) -> Result<(), StdErr> {\n db.delete_board(board_id)\n}\n\n// card routes\n\n#[rocket::get(\"/boards//cards\")]\nfn cards(db: State, board_id: i64) -> Result>, StdErr> {\n db.cards(board_id).map(Json)\n}\n\n#[rocket::post(\"/cards\", data = \"\")]\nfn create_card(db: State, create_card: Json) -> Result, StdErr> {\n db.create_card(create_card.0).map(Json)\n}\n\n#[rocket::patch(\"/cards/\", data = \"\")]\nfn update_card(\n db: State,\n card_id: i64,\n update_card: Json,\n) -> Result, StdErr> {\n db.update_card(card_id, update_card.0).map(Json)\n}\n\n#[rocket::delete(\"/cards/\")]\nfn delete_card(db: State, card_id: i64) -> Result<(), StdErr> {\n db.delete_card(card_id)\n}\n\n// single public function which returns all API routes\n\npub fn api() -> Vec {\n rocket::routes![\n boards,\n create_board,\n board_summary,\n delete_board,\n cards,\n create_card,\n update_card,\n delete_card,\n ]\n}\n```\n\nUpdated main file:\n\n```diff\n// src/main.rs\n\n// required for rocket macros to work\n#![feature(proc_macro_hygiene, decl_macro)]\n\n#[macro_use]\nextern crate diesel;\n\nmod db;\nmod logger;\nmod models;\n+mod routes;\nmod schema;\n\ntype StdErr = Box;\n\n#[rocket::get(\"/\")]\nfn hello_world() -> &'static str {\n \"Hello, world!\"\n}\n\nfn main() -> Result<(), StdErr> {\n dotenv::dotenv()?;\n logger::init()?;\n\n let db = db::Db::connect()?;\n\n rocket::ignite()\n .manage(db)\n .mount(\"/\", rocket::routes![hello_world])\n+ .mount(\"/api\", routes::api())\n .launch();\n\n Ok(())\n}\n```\n\nWe're almost there! Only one small piece of the puzzle is missing: authentication.\n\n\n\n#### Authentication\n\nLet's implement token-based authentication for our RESTful API server. When a request comes in, we'll first check if it has a `Authorization: Bearer ` header, and if not reject it immediately. Otherwise, we validate the `` by checking that it exists in a `tokens` DB table, and if it's valid we respond to the request.\n\nSince this is a toy project we're not going to implement different access permissions per token, all tokens will have access to all boards and all cards. Also, let's skip the implementing the handler which generates and returns tokens because it's mostly boilerplate and not relevant to us implementing authentication.\n\nAnyway, we're gonna need to create a `tokens` table first:\n\n```bash\ndiesel migration generate create_tokens\n```\n\nSQL query to create the `tokens` table:\n\n```sql\n-- create_tokens up.sql\nCREATE TABLE IF NOT EXISTS tokens (\n id TEXT PRIMARY KEY,\n expired_at TIMESTAMP WITH TIME ZONE NOT NULL\n);\n\n-- seed db with some test data for local dev\nINSERT INTO tokens\n(id, expired_at)\nVALUES\n('LET_ME_IN', (CURRENT_TIMESTAMP + INTERVAL '15 minutes') AT TIME ZONE 'utc');\n```\n\nSQL query to drop the `tokens` table:\n\n```sql\n-- create_tokens down.sql\nDROP TABLE IF EXISTS tokens;\n```\n\nThen we run the new migration with:\n\n```bash\ndiesel migration run\n```\n\nThe above command also has the side-effect of updating the generated Diesel schema file with the new tokens table metadata:\n\n```diff\n// src/schema.rs\n\ntable! {\n use diesel::sql_types::*;\n use crate::models::Status_enum;\n\n boards (id) {\n id -> Int8,\n name -> Text,\n created_at -> Timestamptz,\n }\n}\n\ntable! {\n use diesel::sql_types::*;\n use crate::models::Status_enum;\n\n cards (id) {\n id -> Int8,\n board_id -> Int8,\n description -> Text,\n status -> Status_enum,\n created_at -> Timestamptz,\n }\n}\n\n+table! {\n+ use diesel::sql_types::*;\n+ use crate::models::Status_enum;\n+\n+ tokens (id) {\n+ id -> Text,\n+ expired_at -> Timestamptz,\n+ }\n+}\n\njoinable!(cards -> boards (board_id));\n\nallow_tables_to_appear_in_same_query!(\n boards,\n cards,\n+ tokens,\n);\n```\n\nThen we derive a `diesel::Queryable` impl for the `Token` model:\n\n```diff\n// src/models.rs\n\nuse crate::schema::*;\n\n+// for authentication\n+\n+#[derive(diesel::Queryable)]\n+pub struct Token {\n+ pub id: String,\n+ pub expired_at: chrono::DateTime,\n+}\n\n// other models\n```\n\nAnd then add the following method to our `Db` struct to validate tokens:\n\n```diff\nuse std::env;\n\nuse diesel::prelude::*;\nuse diesel::r2d2;\nuse diesel::PgConnection;\n\nuse crate::models::*;\nuse crate::schema::*;\nuse crate::StdErr;\n\ntype PgPool = r2d2::Pool>;\n\npub struct Db {\n pool: PgPool,\n}\n\nimpl Db {\n pub fn connect() -> Result {\n let db_url = env::var(\"DATABASE_URL\")?;\n let manager = r2d2::ConnectionManager::new(db_url);\n let pool = r2d2::Pool::new(manager)?;\n Ok(Db { pool })\n }\n\n+ // token methods\n+\n+ pub fn validate_token(&self, token_id: &str) -> Result {\n+ let conn = self.pool.get()?;\n+ let token = tokens::table\n+ .filter(tokens::id.eq(token_id))\n+ .filter(tokens::expired_at.ge(diesel::dsl::now))\n+ .first(&conn)?;\n+ Ok(token)\n+ }\n\n // other methods\n}\n```\n\nWe can implement authentication in Rocket in a couple ways, by using either Rocket middleware or Rocket request guards. The official Rocket docs recommend the latter so let's go with that.\n\nA request guard is any type within a handler's parameters which impls `rocket::request::FromRequest`. So all we have to do is just impl that for `Token` and then add `Token` parameters to all our request handlers and we're golden. Here's the `FromRequest` impl on `Token`:\n\n```rust\nuse rocket::request::{FromRequest, Request, Outcome};\nuse crate::models::Token;\n\nimpl<'a, 'r> FromRequest<'a, 'r> for Token {\n type Error = &'static str;\n fn from_request(request: &'a Request<'r>) -> Outcome {\n // get request headers\n let headers = request.headers();\n\n // check that Authorization header exists\n let maybe_auth_header = headers.get_one(\"Authorization\");\n if maybe_auth_header.is_none() {\n return Outcome::Failure((\n http::Status::Unauthorized,\n \"missing Authorization header\",\n ));\n }\n\n // and is well-formed\n let auth_header = maybe_auth_header.unwrap();\n let mut auth_header_parts = auth_header.split_ascii_whitespace();\n let maybe_auth_type = auth_header_parts.next();\n if maybe_auth_type.is_none() {\n return Outcome::Failure((\n http::Status::Unauthorized,\n \"malformed Authorization header\",\n ));\n }\n\n // and uses the Bearer token authorization method\n let auth_type = maybe_auth_type.unwrap();\n if auth_type != \"Bearer\" {\n return Outcome::Failure((\n http::Status::BadRequest,\n \"invalid Authorization type\",\n ));\n }\n\n // and the Bearer token is present\n let maybe_token_id = auth_header_parts.next();\n if maybe_token_id.is_none() {\n return Outcome::Failure((http::Status::Unauthorized, \"missing Bearer token\"));\n }\n let token_id = maybe_token_id.unwrap();\n\n // we can use request.guard::() to get a T from a request\n // which includes managed application state like our Db\n let outcome_db = request.guard::>();\n let db: State = match outcome_db {\n Outcome::Success(db) => db,\n _ => return Outcome::Failure((http::Status::InternalServerError, \"internal error\")),\n };\n\n // validate token\n let token_result = db.validate_token(token_id);\n match token_result {\n Ok(token) => Outcome::Success(token),\n Err(_) => Outcome::Failure((\n http::Status::Unauthorized,\n \"invalid or expired Bearer token\",\n )),\n }\n }\n}\n```\n\nNow we add a `Token` parameter to every request handler we want to guard, which is effectively how we implement authentication:\n\n```diff\n// src/routes.rs\n\nuse rocket::http;\nuse rocket::State;\nuse rocket::request::{FromRequest, Request, Outcome};\nuse rocket_contrib::json::Json;\n\nuse crate::db::Db;\nuse crate::models::*;\nuse crate::StdErr;\n\nimpl<'a, 'r> FromRequest<'a, 'r> for Token {\n type Error = &'static str;\n fn from_request(request: &'a Request<'r>) -> Outcome {\n // impl\n }\n}\n\n// board routes\n\n#[rocket::get(\"/boards\")]\n-fn boards(db: State) -> Result>, StdErr> {\n+fn boards(db: State, _t: Token) -> Result>, StdErr> {\n db.boards().map(Json)\n}\n\n#[rocket::post(\"/boards\", data = \"\")]\n-fn create_board(db: State, create_board: Json) -> Result, StdErr> {\n+fn create_board(db: State, create_board: Json, _t: Token) -> Result, StdErr> {\n db.create_board(create_board.0).map(Json)\n}\n\n#[rocket::get(\"/boards//summary\")]\n-fn board_summary(db: State, board_id: i64) -> Result, StdErr> {\n+fn board_summary(db: State, board_id: i64, _t: Token) -> Result, StdErr> {\n db.board_summary(board_id).map(Json)\n}\n\n#[rocket::delete(\"/boards/\")]\n- fn delete_board(db: State, board_id: i64) -> Result<(), StdErr> {\n+ fn delete_board(db: State, board_id: i64, _t: Token) -> Result<(), StdErr> {\n db.delete_board(board_id)\n}\n\n// card routes\n\n#[rocket::get(\"/boards//cards\")]\n-fn cards(db: State, board_id: i64) -> Result>, StdErr> {\n+fn cards(db: State, board_id: i64, _t: Token) -> Result