\n![\"\"](\"http://i.imgur.com/pYJSi.png\" "\\"Hosted")
\n
\n\nThough not particularly exciting, it serves the purpose of verifying that the library can at least be loaded up and successfully complete a trivial task. Because Ray has some external dependencies that must be manually installed, this test is especially important.\n\nIf we look a little more carefully at the rendered content and compare it to our implementation code, we get a few hints about how Ray works. For example, we can infer that the default background color is black and the default text color is white. We can also infer that it displays the first _scene_ by default without explicitly telling it which scene to render. We also see that it looks like Ray's coordinate system places y=0 at the top of the screen. This placement is pretty common for graphics systems, but it's always good to get the question of \"Which way is up?\" out of the way as early as possible.\n\nIt wouldn't be hard to come up with more questions that might be answerable by tweaking this example a bit, but when I first start learning a new library, I try not to be too adventurous. So rather than getting bogged down in the details, I revisited the documentation to figure out how to render a rectangle to the screen.\n\n## Step 2: Render a red 20x20 square to the screen\n\nRendering text was a nice start, but because most of this game hinges on manipulting polygons, not words, it was important to test out some basic drawing operations right away. Because Ray's documentation includes a whole section on polygons, this next step was quite easy to work through.\n\n### Implementation\n\nThe simple program here shares the same boilerplate code as the previous \"Hello world\" example but simply swaps out the text rendering code with some polygon manipulation code.\n\n```ruby\nrequire 'ray'\n\nRay.game \"Test\" do\n register { add_hook :quit, method(:exit!) }\n\n scene :square do\n @rect = Ray::Polygon.rectangle([0, 0, 20, 20], Ray::Color.red)\n @rect.pos = [200, 200]\n\n render do |win|\n win.draw @rect\n end\n end\n\n scenes << :square\nend\n```\n\n### Results\n\nThe following screenshot shows what was rendered to the screen after I made this small change:\n\n\n\n![\"\"](\"http://i.imgur.com/3iXgj.png\" "\\"Hosted")
\n
\n\nAfter comparing the results to the implementation code, it became clear to me that in order to use Ray effectively, I'd need to begin thinking in terms of vector graphics and matrix transformations. In particular, the example demonstrates that Ray represents its drawable objects using an abstract coordinate system for points and edges and then translates those coordinates to determine where they end up being rendered on the screen. This is why we define the square with a top-left corner of (0,0) and then later explicitly set the position to (200,200).\n\nKnowing the math behind 2D transformations is not essential for completing this exercise, but a basic background in those concepts wouldn't hurt. I kept forgetting that this was how Ray worked under the hood while working on this article, which caused some of my debugging sessions to drag on longer than they should have. If you're following along at home and attempting to do each step before reading how I did it, it might not hurt for you to brush up on [the basic math involved in 2D graphics](http://www.willamette.edu/~gorr/classes/GeneralGraphics/Transforms/transforms2d.htm) before continuing with the exercise.\n\nOnce I got a square rendered on the screen, the next step was to make it move.\n\n## Step 3: Get the red square to follow the mouse pointer\n\nEven though the final plans called for this to be a game you play using the arrow keys on your keyboard instead of a mouse, the `on :mouse_motion` example in Ray's documentation was staring me in the face and provided too much instant gratification to skip over.\n\n### Implementation\n\nThis code shows the changes that I made to make the square follow the mouse pointer around the screen. If you are trying to run these examples as you read along, simply replace the scene code from step 2 with this new implementation. All the other boilerplate code will remain the same throughout the rest of this article.\n\n```ruby\nscene :square do\n @rect = Ray::Polygon.rectangle([0, 0, 20, 20], Ray::Color.red)\n @rect.pos = [200,200]\n\n on :mouse_motion do |pos|\n @rect.pos = pos\n end\n\n render do |win|\n win.draw @rect\n end\nend\n```\n\n### Results\n\nThis video shows the red square following the mouse pointer around the screen:\n\n\n\nOnce I got this code working, I was able to get a rough sense of how Ray handles its main event loop. The `on()` method allows you to define observers for various events. Any matching callbacks get triggered on each tick, before the `render` code gets executed. The `:mouse_motion` event was an easy one to start with because it simply yields the position of the mouse pointer on each tick, but the general concept could be applied just as well to key press events.\n\nBut before messing with handling keyboard interaction, I decided to take a quick glance at what kind of object the `on :mouse_motion` observer was yielding. I thought it was possible that these would be just simple two-element arrays, but after doing a few printline statements, realized that they were `Ray::Vector2` objects. A [brief source dive](https://github.com/Mon-Ouie/ray/blob/master/lib/ray/vector.rb) brought me up to speed on what to expect from this sort of object; then I moved on to the next step.\n\n## Step 4: Move the square to the left using the left arrow\n\nI initially tripped up on this step because I didn't understand that the `:key_press` event gets triggered only when the key is initially pressed and does not trigger repeatedly while a key is held down. However, once I found the matching `:key_release` event and an example that used both of them, I was able to make some progress by implementing some simple transactional logic.\n\n### Implementation\n\nThe following code uses an instance variable `@moving_left` to track whether the square needs to continue moving left. Whenever `@moving_left` is true, it uses vector addition to translate the current position of the rectangle.\n\n```ruby\nscene :square do\n @rect = Ray::Polygon.rectangle([0, 0, 20, 20], Ray::Color.red)\n @rect.pos = [200,200]\n\n on :key_press, key(:left) do\n @moving_left = true\n end\n\n on :key_release, key(:left) do\n @moving_left = false\n end\n\n render do |win|\n win.draw @rect\n @rect.pos += [-1,0] if @moving_left\n end\nend\n```\n\n### Results\n\nThe following video shows the red square creeping slowly to the left each time I hold down the left arrow key:\n\n\n\nAfter I got this step working, I investigated a couple more things about Ray through experimentation. My tinkering caused me to discover that the `key()` method actually converts the symbolic value `:left` into a `Ray::Key` object, which is a simple container that looks up the key code for you. I also found out that the position of a drawable object appears to be immutable, so you can't do things like `@rect.pos.x -= 1` and expect it to work. Instead, you need to do vector addition and then assign a new position object. This design decision would have made a lot more sense to me if I kept the mathematical underpinnings of vector graphics in mind while working in this step, but instead, it just lead me to scratch my head for a while.\n\n## Step 5: Allow all arrow keys to move the square\n\nI could have repeated the general approach I took in step 4 to get all my arrow keys working, but it would have been tedious. If I read the documentation a little more closely before starting step 4, I would have seen that Ray's author pretty much says exactly that in one of his examples.\n\n### Implementation\n\nThe following code uses the conditionless callback `always` to run some code on each tick and checks whether a key is being held down by calling the aptly named `holding?` method that I overlooked in step 4.\n\n```ruby\nscene :square do\n @rect = Ray::Polygon.rectangle([0, 0, 20, 20], Ray::Color.red)\n @rect.pos = [200,200]\n\n always do\n @rect.pos += [-1, 0] if holding?(:left)\n @rect.pos += [1, 0] if holding?(:right)\n @rect.pos += [0, -1] if holding?(:up)\n @rect.pos += [0, 1] if holding?(:down)\n end\n\n render do |win|\n win.draw @rect\n end\nend\n```\n\n### Results\n\nAfter making this change, the red square was able to move in all directions, as shown in the following video. Moving diagonally simply requires holding down two keys at once (i.e., holding up and left moves northwest across the screen).\n\n\n\nThe main thing that I noticed was that moving the red square around was tedious because it was moving so slowly. I investigated a few options, including changing Ray's default frame rate, but my wife quickly talked me into doing something much simpler.\n\n## Step 6: Make the square move a bit faster\n\nThis step involved tweaking the distance traveled by the red square on each tick, thus increasing its speed.\n\n### Implementation\n\nIn the following code, I changed the distance that the red square moves when a key is held down from 1 to 2, effectively doubling its speed.\n\n```ruby\nscene :square do\n @rect = Ray::Polygon.rectangle([0, 0, 20, 20], Ray::Color.red)\n @rect.pos = [200,200]\n\n always do\n @rect.pos += [-2, 0] if holding?(:left)\n @rect.pos += [2, 0] if holding?(:right)\n @rect.pos += [0, -2] if holding?(:up)\n @rect.pos += [0, 2] if holding?(:down)\n end\n\n render do |win|\n win.draw @rect\n end\nend\n```\n\n### Results\n\nThis video shows the faster-moving rectangle. Jumping a distance of two pixels at a time still looks like smooth motion, so this approach definitely was more simple than any of the other ideas I had in mind.\n\n\n\nThis was the first time that I started feeling the desire to refactor things: updating four values when I could have updated one seemed a bit tedious. However, I try to keep a semistrict policy of not refactoring unless I am in deep pain for the first few hours of working with a new tool. The reason I do this is to allow my mind to work in a purely creative mode, avoiding invoking the inner \"judge\" that I talked about in [Practicing Ruby 2.2](http://practicingruby.com/articles/2). Take this note as fair warning, though: there will be more repetitive code to come before this exercise is completed!\n\nAt this point, I had a red square moving at a speed that looks comparable to how things tend to move in old-school arcade games. Because the novelty value of moving a little square around in a void wears off pretty quickly, the next step was to introduce some other game objects into the mix.\n\n## Step 7: Display 20 randomly placed 10x10 white squares\n\nIn this step, I introduced the goodies that our red rectangle is meant to collect. Researching collision detection at this point would only complicate things, so instead I focused on the visual aspect of things as well as some simple bounds testing.\n\n### Implementation\n\nThe following code generates 20 random squares and renders them completely within the visible area on the screen. It does not introduce any new Ray concepts, so it should be pretty easy to follow.\n\n```ruby\nscene :square do\n @rect = Ray::Polygon.rectangle([0, 0, 20, 20], Ray::Color.red)\n @rect.pos = [200,200]\n\n max_x = window.size.width - 20\n max_y = window.size.height - 20\n\n @goodies = 20.times.map do\n x = rand(max_x) + 10\n y = rand(max_y) + 10\n \n g = Ray::Polygon.rectangle([0,0,10,10])\n g.pos = [x,y]\n\n g\n end\n\n always do\n @rect.pos += [-2, 0] if holding?(:left)\n @rect.pos += [2, 0] if holding?(:right)\n @rect.pos += [0, -2] if holding?(:up)\n @rect.pos += [0, 2] if holding?(:down)\n end\n\n render do |win|\n @goodies.each { |g| win.draw(g) }\n win.draw @rect\n end\nend\n```\n\n### Results\n\nThe following screenshot demonstrates what this effect ended up looking like. It's almost like a starry night! \n\n\n\n![\"\"](\"http://i.imgur.com/yLqJW.png\" "\\"Hosted")
\n
\n\nAdding bounds checking to make sure the white squares would be rendered within the visible area of the screen reminded me that I should have done something similar to prevent the red square from moving beyond the edge of the screen as well.\n\n## Step 8: Keep the red square from leaving the screen\n\nThe next step was to implement a rudimentary means of keeping the red square from completely disappearing from the screen.\n\n### Implementation\n\nThe following code checks to make sure that the top-left corner of the red square never exits the screen by updating its position only if the new location is within the screen's dimensions. I show only the updated `always` callback because it was the only thing that changed.\n\n```ruby\nalways do\n if @rect.pos.x - 2 > 0\n @rect.pos += [-2, 0] if holding?(:left)\n end\n\n if @rect.pos.x + 2 < window.size.width\n @rect.pos += [2, 0] if holding?(:right)\n end\n\n if @rect.pos.y - 2 > 0\n @rect.pos += [0, -2] if holding?(:up)\n end\n\n if @rect.pos.y + 2 < window.size.height\n @rect.pos += [0, 2] if holding?(:down)\n end\nend\n```\n\n### Results\n\nThe following video shows bounds checking behavior that is slightly different than the previous implementation code; my original code used (-10,-10) rather than (0,0) as the abstract origin for my rectangle. If you run the code yourself, your rectangle will get closer to the edge at times than what this video shows.\n\n\n\nIn retrospect, this code was a bit buggy, as it really should have been looking at all the corners of the square, not just the top-left corner. But because it was good enough to keep the red square from completely sailing off into the void, I decided to save the fix as a problem for later. Putting it off would be a bad idea if I were writing production code, but thankfully the rules for spiking are different.\n\nThe next step was to get over my tensions about this buggy and unrefactored code and get my red square to interact with the white squares.\n\n## Step 9: Remove white squares when they get covered up \n\nIn this step, we finally need to think about collision detection: specifically, how to determine when one rectangle is contained within another. It turns out that Ray provides some helpers for this, but it took a source dive for me to find them, and a lot of experimentation to figure out how exactly to use them.\n\n### Implementation\n\nThe following code uses the `Array#to_rect` core extension that Ray provides for creating `Ray::Rect` objects. This object provides basic collision detection routines, including an `inside?` method that can be used to determine whether one rectangle is completely contained within another. On each tick, any of the white squares that are contained with the bounds of the red square get removed.\n\n```ruby\nalways do\n # same code as in step 8 goes here\n\n @goodies.reject! { |e|\n goodie = [e.pos.x, e.pos.y, 10, 10].to_rect\n \n goodie.inside?([@rect.pos.x, @rect.pos.y, 20, 20])\n }\nend\n```\n\n### Results\n\nThe following video demonstrates collecting goodies. To make things a bit more challenging, I made it so that you must completely cover the white squares rather than simply touching them.\n\n\n\nOnce I figured out how to use `Ray::Rect`, implementing this functionality was relatively straightforward. However, my early confusion about `Ray::Polygon.rectangle` made me think that it returned a `Ray::Rect` object, which it does not. After digging through the source for both `Polygon` and `Rect` at both the Ruby level and the C level, I could not find an easy way to automatically convert a rectangular polygon into a `Rect` object, maybe because Ray is still a pretty young library, or maybe because of a design decision. \n\nRather than dwelling on that question, I just manually instantiated `Ray::Rect` objects via `Array#to_rect` so that I could keep moving on. This is the exact point at which I thought that perhaps I should introduce some sort of data model for my game objects that could implement `to_rect` on and remove some of this duplication, but I once again brushed those tensions aside in favor of moving on to something new.\n\n## Step 10: Display \"You win\" when all white squares are gone\n\nIn this step, I introduced the winning game condition, which is removing all the white squares from the screen.\n\n### Implementation\n\nOnly a minor modification to the `render` callback was needed to complete this step. We simply check whether the array of white squares is empty, and if so, render the phrase \"YOU WIN\" to the screen similar to the way we rendered \"Hello World\" in step 1.\n\n```ruby\nrender do |win|\n if @goodies.empty?\n win.draw text(\"YOU WIN\", :at => [100,100], :size => 60)\n else\n @goodies.each { |g| win.draw(g) }\n win.draw @rect\n end\nend\n```\n\n### Results\n\nThe following video demonstrates that the game can now be won. You may want to fast-forward a bit, as it takes a while to collect all those white squares.\n\n\n\nThis was a really simple step, so there isn't much more to say about it. The next step was to introduce baddies into the game. \n\n## Step 11: Add five randomly placed 15x15 blue squares\n\nIn this step, I placed some blue squares in random locations around the screen to serve as our baddies. As in step 7, I focused on the visual aspect of things and didn't immediately jump into collision detection or movement rules.\n\n### Implementation\n\nThe following code shows the changes that needed to be made to get the blue squares onto the screen. They are very similar to those in step 7, but if you want to see the full context, you can view [a snapshot of the game's source code for this step](https://github.com/elm-city-craftworks/goodies_and_baddies/blob/41110cc71d3f94231754313fec47d1ad6a87e902/game.rb\n) on github.\n\n```ruby\nscene :square do\n # same code as step 10 goes here\n\n @baddies = 5.times.map do\n x = rand(max_x) + 15\n y = rand(max_y) + 15\n g = Ray::Polygon.rectangle([0,0,15,15], Ray::Color.blue)\n g.pos += [x,y]\n g\n end\n \n always do\n # ... same as step 10 goes here\n end\n\n render do |win|\n if @goodies.empty?\n win.draw text(\"YOU WIN\", :at => [100,100], :size => 60)\n else\n @goodies.each { |g| win.draw(g) }\n @baddies.each { |g| win.draw(g) }\n win.draw @rect\n end\n end\nend\n```\n\n### Results\n\nThe following screenshot shows what the randomized blue squares look like:\n\n\n\n![\"\"](\"http://i.imgur.com/Q9mwQ.png\" "\\"Hosted")
\n
\n\nThis step was pretty much a direct repeat of what I did in step 7, so there isn't a whole lot of interesting things to discuss here. The next step was to get these blue squares to be more than just pretty drawings by making them deadly.\n\n## Step 12: Display \"You lose\" on collision with a blue square\n\nIn this step, I introduce a losing condition, which marks the point where my program actually becomes a functional game, even if it's a very boring one.\n\n### Implementation\n\nRevisiting the `Ray::Rect` source code, I found that it also provides a simple `collide?` method that tells you whether any part of a given rectangle intersects with another. The following code uses this feature to make it so that even if a single point of a blue rectangle touches the red one, the game ends in a loss. If this excerpt is too hard to follow without the surrounding context, check out [the source code of the game at this step](https://github.com/elm-city-craftworks/goodies_and_baddies/blob/5595b3fe43decd1f06f1376cc2bd1cfd9c24ec06/game.rb) on github.\n\n```ruby \nscene :square do\n # same code as in step 11\n\n always do\n # same code as in step 11\n\n @game_over ||= @baddies.any? { |e|\n baddie = [e.pos.x, e.pos.y, 15, 15].to_rect\n baddie.collide?([@rect.pos.x, @rect.pos.y, 20,20])\n }\n end\n\n render do |win|\n if @goodies.empty?\n win.draw text(\"YOU WIN\", :at => [100,100], :size => 60)\n elsif @game_over\n win.draw text(\"YOU LOSE\", :at => [100,100], :size => 60)\n else\n @goodies.each { |g| win.draw(g) }\n @baddies.each { |g| win.draw(g) }\n win.draw @rect\n end\n end\nend\n```\n\n### Results\n\nThis video shows that the game ends in failure as soon as the red square touches a blue square:\n\n\n\nIn this step, I explicitly built even more `Ray::Rect` objects, pushing me even closer to the breaking point—a point at which refactoring was not simply desirable but absolutely necessary. But with only one step left to implement before completing the exercise, I pressed on.\n\n## Step 13: Make the blue squares follow the red square\n\nThis final step makes the game a whole lot more interesting and even somewhat fun. There are lots of ways that you could code the movement rules for the baddies, but I went with the simplest one: proceed in a straight line toward the red square on each tick.\n\n### Implementation\n\nThis code should be fairly self-explanatory, as it does not introduce any new Ray concepts. It uses a simple algorithm for moving each blue square towards the red square that randomizes the distance traveled on each tick by choosing a number between 0 and 2.5. The [final source code for the game](https://github.com/elm-city-craftworks/goodies_and_baddies/blob/cdf0fe1b59fe2416886e94da6e45c2affc1dc111/game.rb) is available on github.\n\n```ruby\nscene :square do\n # same code as in step 12\n\n always do\n # same code as in step 12\n\n @baddies.each do |e|\n if e.pos.x < @rect.pos.x\n e.pos += [rand*2.5,0]\n else\n e.pos -= [rand*2.5,0]\n end\n\n if e.pos.y < @rect.pos.y\n e.pos += [0, rand*2.5]\n else\n e.pos -= [0, rand*2.5]\n end\n end\n end\n\n render do |win|\n # same code as in step 12\n end\nend\n```\n\n### Results\n\nThe following video shows a complete run of the game, ending in victory. Before you try it out yourself and end up frustrated, please note that I recorded about 20 losses before getting conditions favorable enough for me to win.\n\n\n\nAt this point, I accomplished my goal of having a fairly interesting playable game in 13 small steps. If I wanted to go further, I would first go back and comprehensively refactor this code, and I would also study Ray in a more detailed fashion. However, I was thrilled to be able to get this far without doing that.\n\n## Reflections\n\nHopefully, seeing my process of learning new things has been useful to you. Everyone says you should work in baby steps, but it is my experience that many intermediate developers have a much different idea of what a 'small step' is than more skilled developers tend to have. Even with my level of experience, I consistently find that the programmers that I look up to have a much more refined sense of simplicity and focus than I do. \n\nOne of the most beneficial aspects of taking things one step at a time is that doing so isolates the risk of running into unknown-unknowns and lets you handle them individually. There were many times when holes in my own understanding of how Ray works combined with holes in its documentation caused me to get confused or frustrated. However, the feeling of struggling with a single issue is much more manageable than thinking about dozens of potential blockers simultaneously.\n\nThere is also something to be said for instant gratification. The smaller your steps are, the sooner you see some measureable progress. Each successful step forward gives you a small feeling of satisfaction that motivates you to take on the next challenge. This feeling is a key reason why many people like doing test-driven development, and it can be applied to a broad range of practices.\n\nThe one thing that I often reevaluate while working in this style is to what extent I should be refactoring as I go. Writing about my process today made me even more uncertain about whether it makes sense to let the code get so ugly just for the sake of preventing judgmental thoughts from arising. However, I feel like the question of whether to refactor as you go is largely a matter of personal preference. That said, I'm very curious to hear what your experience was like while working through this exercise, as well as what you thought of the approach I took. So what do you think?\n\n"
},
{
"path": "articles/v2/007-unobtrusive-ruby.md",
"content": "When Mike Burns outlined his vision of [Unobtrusive Ruby](http://robots.thoughtbot.com/post/10125070413/unobtrusive-ruby), I initially thought it was going to be a hit with the community. However, a lack of specific examples led to a critical backlash and caused the post to generate more heat than light. This is unfortunate, because the ideas he outlined are quite valuable and shouldn't be overlooked.\n\nIn this article, I share my own interpretation of what Unobtrusive Ruby means, based on the the points that Mike outlined. I can't guarantee that my take on this is what Mike had in mind, but it should be interesting to those who wanted a better-defined roadmap than what he provided. To get this most out of this article, I recommend going back and [reading what Mike wrote](http://robots.thoughtbot.com/post/10125070413/unobtrusive-ruby) before continuing. Try to think about what these concepts mean to you, and then compare them to what I've outlined here.\n\nThe following guidelines are the ones Mike laid out, but I've replaced his explanations with my own in the hopes that attacking these ideas from a second angle will be useful.\n\n## Take objects, not classes \n\nBecause Ruby lets us pass classes around like any other object, we have a way to do dependency injection that isn't as common in other languages. For example, we can write something like the following code:\n\n```ruby\nclass Roster\n def initialize \n @participants = [] \n end \n \n def <<(new_participant) \n @participants << new_participant \n end \n \n def participant_names \n @participants.map { |e| e.full_name } \n end \n \n def print(printer=RosterPrinter)\n printer.new(participant_names).print\n end \nend\n\nclass RosterPrinter \n def initialize(participant_names) \n @participant_names = participant_names \n end \n \n def print\n puts \"Participants:\\n\" + \n @participant_names.map { |e| \"* #{e}\" }.join(\"\\n\") \n end \nend \n```\n\nThis feels clever, but this form of dependency injection is more brittle than it\nneeds to be. A [recent conversation with Derick\nBailey](http://blog.rubybestpractices.com/posts/gregory/055-issue-23-solid-design.html#comment-317367342)\non my [SOLID\narticle](http://blog.rubybestpractices.com/posts/gregory/055-issue-23-solid-design.html)\nabout a very similar example made me realize just how much we tend to pass\nclasses around unnecessarily when we could be directly passing the objects our\nclass depends on. With that in mind, Derick helped me refactor the previous example to the more flexible design shown here. \n\n```ruby\nclass Roster\n def initialize \n @participants = [] \n end \n \n def <<(new_participant) \n @participants << new_participant \n end \n \n def participant_names \n @participants.map { |e| e.full_name } \n end \n \n def print(printer=RosterPrinter.new)\n printer.print(@participants)\n end \nend\n\nclass RosterPrinter \n def print(participant_names)\n puts \"Participants:\\n\" + \n participant_names.map { |e| \"* #{e}\" }.join(\"\\n\") \n end \nend \n```\n\nAlthough this is a subtle change, it has a major impact on the way `Roster` and its printer object relate to one another. The `Roster#print` method originally had a dependency on both the constructor of its printer object and its `print()` instance method. Our new code reduces that coupling by depending only on the existence of a `print()` method in the general case. The following examples demonstrate the added flexibility that this new approach offers us.\n\n```ruby\n# does not provide a .new() method\nmodule FunctionalPrinter\n def self.print(participant_names)\n puts \"Participants:\\n\" + \n participant_names.map { |e| \"* #{e}\" }.join(\"\\n\") \n end\nend\n\nrequire \"prawn\"\n\n# has a different constructor than RosterPrinter\nclass PDFPrinter\n def initialize(filename)\n @document = Prawn::Document.new\n @filename = filename\n end\n\n def print(participant_names)\n @document.text(\"Participants\", :size => 16)\n\n participant_names.each do |e|\n @document.text(\"- #{e}\")\n end\n\n @document.render_file(@filename)\n end\nend\n\nroster = Roster.new\nroster << \"Gregory Brown\" << \"Jia Wu\" << \"Jordan Byron\"\n\nputs \"USING DEFAULT\"\nroster.print\n\nputs \"USING FUNCTIONAL PRINTER\"\nroster.print(FunctionalPrinter)\n\nputs \"USING PDF PRINTER (see roster.pdf)\"\nroster.print(PDFPrinter.new(\"roster.pdf\"))\n``` \n\nBoth `FunctionalPrinter` and `PDFPrinter` demonstrate corner cases that our original code did not account for. By following the guideline of accepting objects rather than classes as the arguments to our methods, our new design can accomodate these types of objects without modification of the `Roster#print` code. As you can see from these examples, this way makes life easier for us. \n\n## Never require inheritance\n\nSome libraries that we use strongly encourage us to create subclasses of the objects they provide; others flat-out force us to do so. `ActiveRecord` is an example of a library that is almost useless without inheritance but is also very complicated and would be hard to use as an example. The following code is a much simpler example of a tool that expects you to use subclasses of its provided `Plugin` class to get the job done.\n\n```ruby\nmodule Inspector\n def self.analyze(data)\n Plugin.registered_plugins.each { |e| e.new.analyze(data) }\n end\n\n class Plugin\n def self.inherited(base)\n registered_plugins << base\n end\n\n def self.registered_plugins\n @registered_plugins ||= []\n end\n\n def analyze(data)\n raise NotImplementedError\n end\n end\nend\n\nclass WordCountPlugin < Inspector::Plugin\n def analyze(data)\n word_count = data.split(/ /).length\n puts \"Content contained #{word_count} words\"\n end\nend\n\nclass WordLengthPlugin < Inspector::Plugin\n def analyze(data)\n longest = data.split(/ /).map { |e| e.length }.max\n puts \"Longest word contained #{longest} characters\"\n end\nend\n\nInspector.analyze(\"This is a test of the watcher plugins\")\n\n## OUTPUTS\nContent contained 8 words\nLongest word contained 7 characters \n```\n\nUsing the `inherited` hook is a clever way to implictly register plugins, but this approach comes with a number of downsides. For example, you are forced to make all your plugins inherit from `Inspector::Plugin`, which means your class can't be a subclass of anything else. Additionally, you need to use a class even if a module would make more sense. This is a direct consequence of the \"interface taking classes rather than ordinary objects\" issue and cannot be easily avoided. If you throw in things like having to be aware of possible Liskov Substitution Principle violations, it becomes clear that an API that forces you to use subclassing is not exactly flexible.\n\nThis code shows a much more flexible alternative that completely removes the dependency on class inheritance:\n\n```ruby\nmodule Inspector\n def self.analyze(data)\n registered_plugins.each { |e| e.analyze(data) }\n end\n\n def self.registered_plugins\n @registered_plugins ||= []\n end\nend\n\nmodule WordCountPlugin\n def self.analyze(data)\n word_count = data.split(/ /).length\n puts \"Content contained #{word_count} words\"\n end\nend\n\nmodule WordLengthPlugin\n def self.analyze(data)\n longest = data.split(/ /).map { |e| e.length }.max\n puts \"Longest word contained #{longest} characters\"\n end\nend\n\nInspector.registered_plugins << WordCountPlugin << WordLengthPlugin\nInspector.analyze(\"This is a test of the watcher plugins\")\n```\n\nNow any object that has a valid `analyze` method will work as a plugin, as long as you explicitly register it with `Inspector`. This approach results in much cleaner-looking code for this trivial implementation, but the general strategy can also can be used in situations where inheritance is still a part of the picture. The following code shows plugins that inherit from a parent class coexisting with plugins built from scratch.\n\n```ruby\nmodule Inspector\n def self.analyze(data)\n registered_plugins.each { |e| e.analyze(data) }\n end\n\n def self.registered_plugins\n @registered_plugins ||= []\n end\n\n class Plugin\n def self.verify(&block)\n validations << block\n end\n\n def self.validations\n @validations ||= []\n end\n\n def self.analyze(&block)\n define_method :analyze do |data|\n validate(data) \n block.call(data)\n end\n end\n\n def validate(data)\n raise unless self.class.validations.all? { |v| v.call(data) }\n end\n end\nend\n\nclass WordCountPlugin < Inspector::Plugin\n verify { |data| data.is_a?(String) }\n\n analyze do |data|\n word_count = data.split(/ /).length\n puts \"Content contained #{word_count} words\"\n end\nend\n\nmodule WordLengthPlugin\n # same as before, not inheriting from anything\nend\n\nInspector.registered_plugins << WordCountPlugin.new << WordLengthPlugin\nInspector.analyze(\"This is a test of the watcher plugins\")\n```\n\nThis small change makes a big difference in flexibility and leaves some of the decision making up to your users rather than forcing them down a particular path. Using the default approach of inheriting from `Inspector::Plugin` will feel nearly identical to an inheritance-only approach to the user. However, if more customizations are needed, this design provides an easy way to build plugins from scratch. Because it is so easy to implement this pattern, it is probably worth doing even if your immediate needs don't call for such flexibility.\n\n## Inject dependencies\n\nI covered dependency injection and the dependency inversion principle at length in [Practicing Ruby 1.23](http://blog.rubybestpractices.com/posts/gregory/055-issue-23-solid-design.html). Rather than repeating those details here, I'd like you to read that article as well as the comments from [Derick Bailey](http://blog.rubybestpractices.com/posts/gregory/055-issue-23-solid-design.html#comment-317367342). My conversation with Derick taught me a thing or two about the subtle distinctions between dependency injection (a technique) and dependency inversion (a design principle) that are hard to summarize but still well worth working through if you want to solidify your understanding of these two very important concepts.\n\nAs an example of some practical code that uses dependency injection, let's look at the Highline command-line library. In ordinary usage, Highline outputs everything to `STDIN` and `STDOUT`. You can install HighLine via RubyGems and run the following example to get a feel for how it works.\n\n```ruby\nrequire \"highline\"\n\nconsole = HighLine.new\nname = console.ask(\"What is your name?\") \nconsole.say(\"Hello #{name}\")\n```\n\nIn ordinary cases, this is exactly what we want: ordinary input and output from your console. However, to test HighLine, we made it possible to inject input and output objects to use in place of `STDIN` and `STDOUT`. The next example shows the use of `StringIO` objects, which is what we use in our unit tests. \n\n```ruby\nrequire \"stringio\"\nrequire \"highline\"\n\ninput = StringIO.new(\"Gregory\\n\")\ninput.rewind # set seek pos back to 0\n\noutput = StringIO.new\n\nconsole = HighLine.new(input, output)\nname = console.ask(\"What is your name?\")\nconsole.say(\"Hello #{name}!\")\n\noutput.rewind\nputs output.read\n```\n\nInterestingly enough, this 'feature' of HighLine has caused it to be used in a number of contexts that we didn't anticipate. For example, it is occasionally used in GUI programs for its input validation features and is sometimes used in noninteractive scripts for its text formatting features. If we had directly worked against `STDIN` and `STDOUT`, these ways of using HighLine would not be possible without ugly hacks.\n\n## Unit tests are fast\n\nPersonally, I am not obsessive about unit test performance. Many Rubyists care a lot about this and advocate the heavy use of mock objects to speed up your tests. Mike points out in his article that the combination of \"mock objects, a lack of inheritance, and injected dependencies\" will make your tests fast, and that's basically true. \n\nDependency injection does facilitate the use of mock objects, and our HighLine example demonstrates why that is the case. A lack of inheritance might imply that your method call chains are shorter and that you have fewer dependencies, but it's not as strong of a metric as he makes it seem. Eventually, object composition can end up being more or less the same as inheritance in complexity, and in those cases you may still end up with slow tests. Composed object systems are easier to decouple, which is probably what he's getting at here.\n\nFor some practical examples of how you can use mocks to decouple your tests from your code and speed things up a bit, see [Practicing Ruby 1.20](http://blog.rubybestpractices.com/posts/gregory/052-issue-20-thoughts-on-mocking.html). If you read through that article, you'll find out why I don't care so much about limiting my dependencies to the object under test. But when all else is considered equal, fewer dependencies mean less code, which probably means faster tests.\n\nHaving a performant test suite is certainly ideal; it's just a matter of weighing he costs of fine-tuning your test suite versus the benefits that faster test runs provide. Personally, I felt that Mike somewhat overemphasized this particular point, but many well-known Rubyists would disagree with me on this one.\n\n## Require no interface\n\nMike suggests that your library code should allow your users to name their methods however they want and should be designed to consume rather than be consumed. He then recognizes that this design may not always be possible and concedes that if you need the user to implement certain features, you should limit this functionality to one or two methods at most. This is great general advice, and if you look at Ruby itself, we can find some good examples.\n\nThe `Enumerable` module is capable of providing the vast majority of its features if the user implements only an `each` method. If you want to use things like `sort`, you just need the yielded elements to implement a `<=>` method. All features in `Enumerable` can thus be supported by \"one or two methods\" implemented by the user, which makes it a very good API, considering the great wealth of functionality it provides.\n\nHowever, `Enumerator` takes things a step further by requiring no interface at all. You can name the methods of the target collection anything you want; you merely tell `Enumerator` which method it should delegate its `each` method to when you initialize an `Enumerator`. See the following example to see how flexible this approach is.\n\n```ruby\nclass RandomInfiniteList\n def generate\n loop do\n yield rand\n end\n end\nend\n\nenum = Enumerator.new(RandomInfiniteList.new, :generate)\np enum.next\np enum.take(10)\n```\n\nIn this way, `Enumerator` can be made to turn any object with an iterator method into an `Enumerable` object, regardless of what the name of that method is. This feature can be useful when you're working with unidiomatic Ruby objects that provide an iterator but do not mix in `Enumerable`.\n\nWe can also look at an example that's a bit less abstract. If we look back at the `Inspector` example that we were using to discuss how to avoid hard inheritance requirements, we can see that it requires only a small interface for plugins to conform to. Although it isn't so bad that each plugin needed an `analyze` method in our previous iteration, we can make some modifications so that it depends on no interface at all, which may bring us a step closer to what Mike was hinting at.\n\nThe following example shows what an `Inspector` class that implements an \"expects no interface\" API style might look like. To keep things interesting, I've left the implementation of this class up to you. If you get stuck, feel free to leave a comment asking for hints on how to build this out.\n\n```ruby\n# delegates to the WordLengthPlugin module\nInspector.report(\"Word Length\") do |data|\n WordLengthPlugin.analyze(data)\nend\n\n# implements the report as an inline function\nInspector.report(\"Word Count\") do |data|\n word_count = data.split(/ /).length\n puts \"Content contained #{word_count} words\"\nend\n\nInspector.analyze(\"This is a test of the watcher plugins\")\n```\n\n## Data, not action\n\nThis guideline seems to boil down to the idea that your API calls should be simple \"data in, data out\" operations whenever that level of simplicity is easily within reach. After thinking about this concept a bit, I realized that a pair of common Ruby operations serve as perfect examples.\n\nSuppose that you want to open a binary file and read its contents into a `String`. You could write the following code, which will get the job done:\n\n```ruby\nfile = File.open(\"foo.jpg\", \"rb\")\nimage_data = file.read\nfile.close\n```\n\nBut this approach is the correct way to do things only if you want to work explicitly with the `File` object and control exactly when it gets opened and closed. If all you want to do is read the contents of a binary file, this is three actions for what should be one \"data in, data out\" operation. Fortunately, Ruby recognizes this as a common use case and provides a nicer API that you can use instead:\n\n```ruby\nimage_data = File.binread(\"foo.jpg\")\n```\n\nIn this example, we pass the filename and get back the contents of that file data. Though we have less control over the overall process, we also get to ignore irrelevant details like exactly when the file is opened and closed. This is what I think that Mike probably meant when he said \"data, not action,\" and it can be applied when designing your own APIs. To drive the point home, let's look at another example.\n\nThe following code shows the generic form of doing a GET request via the `Net::HTTP` standard library. It is not the most terse way to use `Net::HTTP`, but it is one of the most common.\n\n```ruby\nrequire 'net/http'\n\nurl = URI.parse('http://www.google.com')\nres = Net::HTTP.start(url.host, url.port) do |http|\n http.get('/')\nend\n\nputs res.body\n```\n\nThere is a whole lot going on here, including explicit `URI` parsing and explicit calls to `get()`. But as you saw with `File.open` versus `File.binread`, Ruby provides a convenient alternative for this very common operation. The open-uri standard library makes it possible to write the following code instead:\n\n```ruby\nrequire \"open-uri\"\n\nputs open(\"http://google.com\").read\n```\n\nOnce again, we see a series of complex actions being converted into a simple \"data in, data out\" operation. In this case, we are converting a string that represents a URI into an IO object via the globally available `open()` method. This approach makes it possible for us to not think about explicitly parsing our string into a `URI` object and lets us ignore the details about starting a HTTP connection and explicitly making a GET request. When all we want is the source of a web page, it's great to be able to ignore these details.\n\n## Always be coding\n\nAs I reflect on Mike's guidelines for Unobtrusive Ruby, it all seems to come down to making life easier for your users by limiting the impact your system has on them. Give your users small, flexible APIs that do not demand much of their systems, and they will have better luck using your code. This seems like a noble set of goals to me, and I hope my examples demonstrate the same spirit that Mike wanted to encourage in his post.\n\nHowever, I always worry about using design guidelines like these as some sort of absolute set of commandments. There were a whole lot of words like \"always\" and \"never\" in the original Unobtrusive Ruby post that, if left unchecked, could cause more harm than good. For me, context is king, and these ideas seem much more important for those who are writing code that is intended for widespread reuse than they might be for ordinary application development. That said, the examples I've shown here demonstrate that you can often be on the right side of these guidelines with little to no additional effort. Therefore, if we can keep the ideas behind Unobtrusive Ruby in the back of our minds and apply them when it's an easy fit, we may well end up improving our code.\n\nThis article is meant to be a conversation starter, not gospel. Please share your own thoughts on what Unobtrusive Ruby means to you, either via a comment here or on your own blog. I think it's a conversation worth having, even if we haven't quite nailed down all the definitions yet. If we end up getting an interesting discussion going, I'll invite Mike to check out what we've discussed and see what he thinks about it.\n\nSo what do you think? Was my code unobtrusive enough for you? If not, why not? ;)\n"
},
{
"path": "articles/v2/008-singleton-pattern.md",
"content": "Many design patterns that originated in other object-oriented languages have elegant Ruby translations. However, the [Singleton](http://en.wikipedia.org/wiki/Singleton_pattern) stands out as a construct that seems to have no good way to implement in Ruby. In this article, I will walk through the different options and explain why they all have something wrong with them. But first, we need a working definition of the singleton pattern to make sure we're on the same page.\n\nPut briefly, the singleton pattern is a clever way of implementing global objects that you never need to explicitly instantiate. Not to be confused with Ruby's mostly unrelated concept of a \"singleton class,\" the singleton pattern is applied when only a single instance of an object is needed across an entire application. Typical examples include objects that represent configuration data, global logging systems, and other similar structures. But there are also some subtle use cases in Ruby due to the fact that classes and modules are objects. For example, we have the `Math` module on which we can call methods such as `Math.sin()`, and `Math.cos()`. The `Math` module is acting as a singleton object in this context, even if it's not immediately obvious to the user. Keep in mind while reading this article that I've lumped this sort of use case in with the more traditional ones, as it shifts the perspective somewhat.\n\nThere are a lot of different ways to implement this general pattern in Ruby, but as I've already mentioned, they all pretty much suck. That said, studying this problem can teach us a thing or two about the subtleties (and warts) of the Ruby object system. As you read along, try figuring out the downsides of each implementation before moving on to read my explanations. This exercise will make the article more interesting and may even uncover some fresh ideas that I haven't considered yet.\n\n### Using the `Singleton` module provided by the standard library\n\nRuby provides a standard library to assist in implementing the singleton pattern. The following code (originally from Practicing Ruby 1.25) illustrates how you can use this library to build a simple logger object.\n\n```ruby\nrequire \"singleton\"\n\nclass SimpleLogger\n include Singleton\n\n def initialize\n @output = []\n end\n\n attr_reader :output\n\n def error(message)\n output << formatted_message(message, \"ERROR\")\n end\n\n def info(message)\n output << formatted_message(message, \"INFO\")\n end\n\n def write(filename)\n File.open(filename, \"a\") { |f| f << output.join }\n end\n\n private\n\n def formatted_message(message, message_type)\n \"#{Time.now} | #{message_type}: #{message}\\n\"\n end\nend\n```\n\nBy including the `Singleton` module, we make it so that it is no longer possible to create an instance of the `SimpleLogger` class in an ordinary way.\n\n```\n>> logger = SimpleLogger.new\nNoMethodError: private method `new' called for SimpleLogger:Class\n from (irb):2\n```\n\nThis behavior makes sense, because the point of the singleton pattern is to prevent multiple instances of a given object from being created. This example code shows how to get at a `SimpleLogger` instance in a way that guarantees that only one will be created.\n\n```ruby\nlogger = SimpleLogger.instance\n\nlogger.error(\"Some serious problem\")\nlogger.info(\"Something you might want to know\")\nlogger.write(\"log.txt\")\n```\n\nThis interface is a bit cumbersome to work with, but it gets the job done, and on its own isn't too bad. However, disabling `new` and adding an `instance` method isn't all that the `Singleton` module does. It also does all of the following things:\n\n* Overrides `inherited()` on the class to ensure that subclasses also retain `Singleton` behavior\n* Overrides `dup()`/`clone()` on the class to ensure that copied classes also retain `Singleton` behavior\n* Overrides `_load()` to call `instance()`, modifying `Marshal` loading behavior to return the single instance\n* Overrides `_dump()` to strip state information when serializing via `Marshal`\n* Overrides `dup()`/`clone()` on the instance to raise a `TypeError`, preventing cloning or duping of the instance\n\nWhen you think about what a singleton object is actually meant to be, these changes make sense. However, many Rubyists look at this and see a whole lot of complexity without a lot of direct benefits. This impression causes many folks to avoid the use of the `Singleton` module in favor of implementations that are a bit more low ceremony. These implementations tend to ignore some of the edge cases that the `Singleton` module accounts for but are much easier to understand.\n\n### Using a class consisting of only class methods\n\nThe following code uses ordinary class methods as an alternative to the previous approach. We explicitly call `undef_method` to make it so that instances of this class cannot be created, but otherwise this code is a vanilla Ruby class definition.\n\n```ruby\nclass SimpleLogger\n class << self\n undef_method :new\n\n def output\n @output ||= []\n end\n\n def error(message)\n output << formatted_message(message, \"ERROR\")\n end\n\n def info(message)\n output << formatted_message(message, \"INFO\")\n end\n\n def write(filename)\n File.open(filename, \"a\") { |f| f << output.join }\n end\n\n private\n\n def formatted_message(message, message_type)\n \"#{Time.now} | #{message_type}: #{message}\\n\"\n end\n end\nend\n```\n\nUsing this class is very simple, as the entire API consists of class method calls.\n\n```ruby\nSimpleLogger.error(\"Some serious problem\")\nSimpleLogger.info(\"Something you might want to know\")\nSimpleLogger.write(\"log.txt\")\n```\n\nThis approach isn't too bad, but it has its own set of caveats. The use of `undef_method` to disable the `new` method makes our `Class` object into something that is some ways class-like, but isn't quite a class anymore. From a purity standpoint, something just feels wrong about a construct that can exist in an inheritance hierarchy but cannot be instantiated. There is also the question of whether it ever really makes sense to create a subclass of a singleton object.\n\nFor a number of reasons, these philosophical issues tend to push folks in the direction of Ruby's `Module` construct, which at first glance seems to address some of these problems.\n\n### Using a module consisting of only module methods\n\nEvery module is an object that cannot exist in a hierarchy and cannot be instantiated but otherwise holds similar properties of `Class` objects. Note how similar this code is to our previous example.\n\n```ruby\nmodule SimpleLogger\n class << self\n def output\n @output ||= []\n end\n\n def error(message)\n output << formatted_message(message, \"ERROR\")\n end\n\n def info(message)\n output << formatted_message(message, \"INFO\")\n end\n\n def write(filename)\n File.open(filename, \"a\") { |f| f << output.join }\n end\n\n private\n\n def formatted_message(message, message_type)\n \"#{Time.now} | #{message_type}: #{message}\\n\"\n end\n end\nend\n```\n\nThe two approaches are so similar that they look identical from the end user's perspective:\n\n```ruby\nSimpleLogger.error(\"Some serious problem\")\nSimpleLogger.info(\"Something you might want to know\")\nSimpleLogger.write(\"log.txt\")\n```\n\nOf course, if we look under the hood, we find that these two implementations are quite different. Although it's true that we've effectively made it impossible to create a subclass of `SimpleLogger` and that we didn't have to explicitly disable the `new` method because `Module` does not provide one, we now are faced with the problem that this module can be mixed into other objects.\n\nJust as a class can have methods at the class level and the instance level, a module can have methods at the module level and the \"mixin\" level. Our `SimpleLogger` code defines all of its methods at the module level, which means that mixing it into an object via `include` or `extend` will not add any new functionality to the object it gets mixed into. From a purity standpoint, this approach is pretty much identical to the \"useless instances\" that would be possible for us to create if we allowed calls to the `new` method in our class-based `SimpleLogger`. Modules therefore don't actually give us much of an advantage over classes after all.\n\nTo make matters more confusing, Ruby provides us with a couple additional ways to use modules to implement the singleton pattern that bring new kinds of complexity into the mix.\n\n### Using a module with `module_function`\n\nIf I were to create a list of Ruby's most confusing features, `module_function` would be near the top. It is a keyword (like `private` and `protected`), which allows you to specify certain methods within a module to be callable at the module level. This feature seems useful at a glance and is even used by Ruby's `Math` module. The interesting thing about module methods is that they serve as public methods on the module itself but get mixed into other objects as private methods. \n\nThis example code demonstrates directly calling methods on the `Math` module, which looks similar to our previous module-based singleton pattern example.\n\n```ruby\nclass Point\n def initialize(x,y)\n @x = x\n @y = y\n end\n\n attr_reader :x, :y\n\n def distance_to(other_point)\n Math.hypot(other_point.x - x, other_point.y - y)\n end\nend\n\npoint_a = Point.new(0,0)\npoint_b = Point.new(4,3)\n\np point_a.distance_to(point_b)\n```\n\nIf we instead include the `Math` module into the `Point` class, we see that the behavior is different than defining methods directly on the `Math` module because its functionality does get mixed into `Point`.\n\n```ruby\nclass Point\n include Math\n\n def initialize(x,y)\n @x = x\n @y = y\n end\n\n attr_reader :x, :y\n\n def distance_to(other_point)\n hypot(other_point.x - x, other_point.y - y)\n end\nend\n\npoint_a = Point.new(0,0)\npoint_b = Point.new(4,3)\n\np point_a.distance_to(point_b)\n```\n\n\nThis pattern of having a module that doubles as a mixin and a singleton object probably has limited applications, but it seems reasonable for the `Math` module because each method provided by `Math` is purely functional and is also unlikely to clash with other features within a given class. But even if mixing in the `Math` module is convenient and relatively safe, we wouldn't want our `Point` object to expose the features that the `Math` object provides via its public API. This is where we notice that `module_function` anticipates this potential problem and attempts to solve it by making all mixed-in methods private.\n\n```\n>> point_a.hypot(4,3)\nNoMethodError: private method `hypot' called for #\n\n$ cat -n data/gettysburg.txt \n 1 Four score and seven years ago, our fathers brought forth on this continent a\n 2 new nation, conceived in Liberty and dedicated to the proposition that all men\n 3 are created equal.\n 4 \n 5 Now we are engaged in a great civil war, testing whether that nation, or any\n 6 nation so conceived and so dedicated, can long endure. We are met on a great\n 7 battle-field of that war. We have come to dedicate a portion of that field as a\n 8 final resting place for those who here gave their lives that that nation might\n 9 live. It is altogether fitting and proper that we should do this.\n 10 \n 11 But, in a larger sense, we can not dedicate -- we can not consecrate -- we can\n 12 not hallow -- this ground. The brave men, living and dead, who struggled here\n 13 have consecrated it far above our poor power to add or detract. The world will\n 14 little note nor long remember what we say here, but it can never forget what\n 15 they did here. It is for us the living, rather, to be dedicated here to the\n 16 unfinished work which they who fought here have thus far so nobly advanced. It\n 17 is rather for us to be here dedicated to the great task remaining before us --\n 18 that from these honored dead we take increased devotion to that cause for which\n 19 they gave the last full measure of devotion -- that we here highly resolve that\n 20 these dead shall not have died in vain -- that this nation, under God, shall\n 21 have a new birth of freedom -- and that government of the people, by the people,\n 22 for the people, shall not perish from the earth.\n\n\nOn my system, `cat` seems to assume a fixed-width column with space for up to six digits. This format looks great for any file with fewer than a million lines in it, but eventually breaks down once you cross that boundary.\n\n```\n$ ruby -e \"1_000_000.times { puts 'blah' }\" | cat -n | tail\n999991 blah\n999992 blah\n999993 blah\n999994 blah\n999995 blah\n999996 blah\n999997 blah\n999998 blah\n999999 blah\n1000000 blah\n```\n\nThis design decision makes implementing the formatting code for this feature a whole lot easier. The `RCat::Display#print_labeled_line` method shows that it's possible to implement this kind of formatting with a one-liner:\n\n```ruby\ndef print_labeled_line(line)\n print \"#{line_number.to_s.rjust(6)}\\t#{line}\" \nend\n```\n\nAlthough the code in this example is sufficient for our needs in `rcat`, it's worth mentioning that `String` also supports the `ljust` and `center` methods. All three of these justification methods can optionally take a second argument, which causes them to use an arbitrary string as padding rather than a space character; this feature is sometimes useful for creating things like ASCII status bars or tables.\n\nI've worked on a lot of different command-line report formats before, and I can tell you that streamable, fixed-width output is the easiest kind of reporting you'll come by. Things get a lot more complicated when you have to support variable-width columns or render elements that span multiple rows and columns. I won't get into the details of how to do those things here, but feel free to leave a comment if you're interested in hearing more on that topic.\n\n### Error handling and exit codes\n\nThe techniques we've covered so far are enough to get most of `rcat`'s tests passing, but the following three scenarios require a working knowledge of how Unix commands tend to handle errors. Read through them and do the best you can to make sense of what's going on.\n\n```ruby\n`cat #{gettysburg_file}`\ncat_success = $?\n\n`rcat #{gettysburg_file}`\nrcat_success = $?\n\nunless cat_success.exitstatus == 0 && rcat_success.exitstatus == 0\n fail \"Failed 'cat and rcat success exit codes match\"\nend\n\n############################################################################\n\ncat_out, cat_err, cat_process = Open3.capture3(\"cat some_invalid_file\")\nrcat_out, rcat_err, rcat_process = Open3.capture3(\"rcat some_invalid_file\") \n\nunless cat_process.exitstatus == 1 && rcat_process.exitstatus == 1\n fail \"Failed 'cat and rcat exit codes match on bad file\"\nend\n\nunless rcat_err == \"rcat: No such file or directory - some_invalid_file\\n\"\n fail \"Failed 'cat and rcat error messages match on bad file'\"\nend\n\n############################################################################\n\n\ncat_out, cat_err, cat_proccess = Open3.capture3(\"cat -x #{gettysburg_file}\")\nrcat_out,rcat_err, rcat_process = Open3.capture3(\"rcat -x #{gettysburg_file}\") \n\nunless cat_process.exitstatus == 1 && rcat_process.exitstatus == 1\n fail \"Failed 'cat and rcat exit codes match on bad switch\"\nend\n\nunless rcat_err == \"rcat: invalid option: -x\\nusage: rcat [-bns] [file ...]\\n\"\n fail \"Failed 'rcat provides usage instructions when given invalid option\"\nend\n```\n\nThe first test verifies exit codes for successful calls to `cat` and `rcat`. In Unix programs, exit codes are a means to pass information back to the shell about whether a command finished successfully. The right way to signal that things worked as expected is to return an exit code of 0, which is exactly what Ruby does whenever a program exits normally without error.\n\nWhenever we run a shell command in Ruby using backticks, a `Process::Status` object is created and is then assigned to the `$?` global variable. This object contains (among other things) the exit status of the command that was run. Although it looks a bit cryptic, we're able to use this feature to verify in our first test that both `cat` and `rcat` finished their jobs successfully without error.\n\nThe second and third tests require a bit more heavy lifting because in these scenarios, we want to capture not only the exit status of these commands, but also whatever text they end up writing to the STDERR stream. To do so, we use the `Open3` standard library. The `Open3.capture3` method runs a shell command and then returns whatever was written to STDOUT and STDERR, as well as a `Process::Status` object similar to the one we pulled out of `$?` earlier. \n\nIf you look at _bin/rcat_, you'll find the code that causes these tests to pass:\n\n```ruby\nbegin\n RCat::Application.new(ARGV).run\nrescue Errno::ENOENT => err\n abort \"rcat: #{err.message}\"\nrescue OptionParser::InvalidOption => err\n abort \"rcat: #{err.message}\\nusage: rcat [-bns] [file ...]\"\nend\n```\n\nThe `abort` method provides a means to write some text to STDERR and then exit with a nonzero code. The previous code provides functionality equivalent to the following, more explicit code:\n\n```ruby\nbegin\n RCat::Application.new(ARGV).run\nrescue Errno::ENOENT => err\n $stderr.puts \"rcat: #{err.message}\"\n exit(1)\nrescue OptionParser::InvalidOption => err\n $stderr.puts \"rcat: #{err.message}\\nusage: rcat [-bns] [file ...]\"\n exit(1)\nend\n```\n\nLooking back on things, the errors I've rescued here are somewhat low level, and\nit might have been better to rescue them where they occur and then reraise\ncustom errors provided by `RCat`. This approach would lead to code similar to\nwhat is shown below:\n\n```ruby\nbegin\n RCat::Application.new(ARGV).run\nrescue RCat::Errors::FileNotFound => err\n # ...\nrescue RCat::Errors::InvalidParameter => err\n # ..\nend\n```\n\nRegardless of how these exceptions are labeled, it's important to note that I intentionally let them bubble all the way up to the outermost layer and only then rescue them and call `Kernel#exit`. Intermingling `exit` calls within control flow or modeling logic makes debugging nearly impossible and also makes automated testing a whole lot harder.\n\nAnother thing to note about this code is that I write my error messages to `STDERR` rather than `STDOUT`. Unix-based systems give us these two different streams for a reason: they let us separate debugging output and functional output so that they can be redirected and manipulated independently. Mixing the two together makes it much more difficult for commands to be chained together in a pipeline, going against the [Unix philosophy](http://en.wikipedia.org/wiki/Unix_philosophy).\n\nError handling is a topic that could easily span several articles. But when it comes to building command-line applications, you'll be in pretty good shape if you remember just two things: use `STDERR` instead of `STDOUT` for debugging output, and make sure to exit with a nonzero status code if your application fails to do what it is supposed to do. Following those two simple rules will make your application play a whole lot nicer with others.\n\n### Reflections\n\nHoly cow, this was a hard article to write! When I originally decided to write a `cat` clone, I worried that the example would be too trivial and boring to be worth writing about. However, once I actually implemented it and sat down to write this article, I realized that building command-line applications that respect Unix philosophy and play nice with others is harder than it seems on the surface.\n\nRather than treating this article as a definitive reference for how to build good command-line applications, perhaps we can instead use it as a jumping-off point for future topics to cover in a more self-contained fashion. I'd love to hear your thoughts on what topics in particular interested you and what areas you think should have been covered in greater detail.\n\n> NOTE: If you'd like to learn more about this topic, consider doing the Practicing Ruby self-guided course on [Streams, Files, and Sockets](https://practicingruby.com/articles/study-guide-1?u=dc2ab0f9bb). You've already completed one of its reading exercises by working through this article!\n" }, { "path": "articles/v2/010-from-requirements-discovery-to-release.md", "content": "Every time we start a greenfield software project, we are faced with the overwhelming responsibility of creating something from nothing. Because the path from the requirements discovery phase to the first release of a product has so many unexpected twists and turns, the whole process can feel a bit unforgiving and magical. This feeling is a big part of what makes programming hard, even for experienced developers.\n\nFor the longest time, I relied heavily on my intuition to get myself kick-started on new projects. I didn't have a clear sense of what my creative process was, but I could sense that my fear of the unknown started to melt away as I gained more experience as a programmer. Having a bit of confidence in my own abilities made me more productive, but not knowing where that confidence came from made it impossible for me to cultivate it in others. Treating my creative process as a black box also made it meaningless for me to compare my approach to anyone else's. Eventually, I got fed up with these limitations and decided that I wanted to do something to overcome them.\n\nMy angle of approach was fairly simple: I decided to take a greenfield project from the idea phase to an initial open source release while documenting the entire process. I thought this information might provide a useful starting point for identifying patterns in how I work and also a basis of comparison for other folks. As I reviewed my notes from this exercise and compared them to my previous experiences, I was thrilled to see that a clear pattern did emerge. This article summarizes what I learned about my own process; I hope it will also be helpful to you.\n\n### Brainstorming for project ideas\n\nThe process of coming up with an idea for a software project (or perhaps any creative work) is highly dynamic. The best ideas tend to evolve quite a bit from whatever the original spark of inspiration was. If you are not constrained to solving a particular problem, it can be quite rewarding to allow yourself to wander a bit and see where you end up. Evolving an idea is like starting with a base recipe for a dish and then tweaking a couple ingredients at a time until you end up with something delicious. The story of how this particular project started should illustrate just how much mutation can happen in the early stages of creating something new.\n\nA few days before writing this article, I was trying to come up with ideas for another Practicing Ruby article I had planned to write. I wanted to do something on event-driven programming and thought that some sort of tower defense game might be a fun example to play with. However, the ideas I had in mind were too complicated, so I gradually simplified my game ideas until they turned into something vaguely resembling a simple board game.\n\nEventually, I forgot that my main goal was to get an article written and decided to focus on developing my board game ideas instead. With my wife's help, over the course of a weekend I managed to come up with a fairly playable board game that bore no resemblence to a tower defense game and would serve as a terrible event-driven programming exercise. However, I still wanted to implement a software version of the game because it would make the experience much easier for us to analyze and share with others.\n\nMy intuition said that the project would take me a day or so to build and that it'd be sufficiently interesting to take notes on for my \"documenting the creative process\" exercise. This gut feeling was enough to convince me to take the plunge, so I cleared the whiteboards in my office in preparation for an impromptu design session.\n\n### Establishing the 10,000-foot view\n\nWhether you're building a game or modeling a complex business process, you need to define lots of terms before you can go about describing the interactions of your system. When you consider the fact that complex dependencies can make it hard to change names later, it's hard to overstate the importance of this stage of the process. For this reason, it's always a good idea to start a new project by defining some terms for some of the most important components and interactions that you'll be working with. My first whiteboard sketch focused on exactly that:\n\n
\n![](\"http://farm7.static.flickr.com/6229/6283525185_35bd4c96a8_z.jpg\")
\n
\n\nHaving a sense of the overall structure of the game in somewhat more formal terms made it possible for me to begin mapping these concepts onto object relationships. The following image shows my first crack at figuring out what classes I'd need and how they would interact with each other:\n\n\n\n![](\"http://farm7.static.flickr.com/6049/6283524127_032ab93d77_z.jpg\")
\n
\n\nIt's worth noting that in both of these diagrams, I was making no attempt at being exhaustive, nor was I expecting these designs to survive beyond an initial spike. But because moving boxes and arrows around on a whiteboard is easier than rewriting code, I tend to start off any moderately complex project this way.\n\nWith just these two whiteboard sketches, I had most of what I needed to start coding. The only important thing left to be done before I could fire up my text editor was coming up with a suitable name for the game. After trying and failing at finding a variant of \"All your base\" that wasn't an existing gem name, I eventually settled on \"Stack Wars.\" I picked this name because \na big part of the physical game has to do with building little stacks of army tiles in the territories you control. Despite the fact that the name doesn't mean much in the electronic version, it was an unclaimed name that could easily be _CamelCased_ and _snake_cased_, so I decided to go with it.\n\nAs important as naming considerations are, getting bogged down in them can be just as harmful as paying no attention to the problem at all. For this reason, I decided to leave some of the details of the game in my head so that I could postpone some naming decisions until I saw how the code was coming together. That decision allowed me to start coding a bit earlier at the cost of having a bit of an incomplete roadmap.\n\n### Picking some low-hanging fruit\n\nEvery time I start a new project, I try to identify a small task that I can finish quickly so that I can get some instant gratification. I find an early success to be important for my morale, and it also serves as a gentle way to test some of my assumptions about the project.\n\nI try to avoid starting with the boring stuff like setting up boilerplate code and building trivial container objects. Instead, I typically attempt to build a small but useful end-to-end feature. For the purposes of this game, an ASCII representation of the battlefield seemed like a good place to start. I started this task by creating a file called _sample_ui.txt_ with the contents you see here:\n\n```\n 0 1 2 3 4 5 6 7 8 \n BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB\n 0 (___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)\n | | | | | | | | |\n 1 (___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)\n | | | | | | | | |\n 2 (___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)\n | | | | | | | | |\n 3 (___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)\n | | | | | | | | |\n 4 (___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)\n | | | | | | | | |\n 5 (___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)\n | | | | | | | | |\n 6 (___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)\n | | | | | | | | |\n 7 (B 2)--(___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)\n | | | | | | | | |\n 8 (___)--(W 2)--(___)--(___)--(___)--(___)--(___)--(___)--(___)\n WWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWW\n```\n\nIn order to implement this visualization, I needed to make some decisions about how the battlefield data was going to be represented, but I wanted to defer as much of that work as possible. After [asking for some feedback about this problem](https://gist.github.com/1310883), I opted to write the visualization code against a simple array of arrays of Ruby primitives that could be trivially be transformed to and from JSON. Within a few minutes, I had a script that was generating similar output to my original sketch:\n\n```ruby\nrequire \"json\"\n\ndata = JSON.parse(File.read(ARGV[0]))\n\ncolor_to_symbol = { \"black\" => \"B\", \"white\" => \"W\" }\n\nheader = \" 0 1 2 3 4 5 6 7 8\\n\"\nseparator = \" | | | | | | | | |\\n\"\n\nborder_b = \" BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB\\n\"\nborder_w = \" WWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWW\\n\"\n\nbattlefield_text = data.map.with_index do |row, row_index|\n row_text = row.map do |color, strength|\n if color == \"unclaimed\"\n \"(___)\"\n else\n \"(#{color_to_symbol[color]}#{strength.to_s.rjust(2)})\"\n end\n end.join(\"--\")\n\n \"#{row_index.to_s.rjust(3)} #{row_text}\\n\"\nend.join(separator)\n\nputs [header, border_b, battlefield_text, border_w].join\n```\n\nAlthough this script is a messy little hack, it got me started on the project in a way that was immediately useful. In the process of creating this visualization tool, I ended up thinking about a lot of tangentially related issues. In particular, I started to brainstorm about the following topics:\n\n* What fixture data I would need for testing various game actions\n* What the coordinate system for the `Battlefield` would be\n* What data the `Territory` object would need to contain\n* What format to use for inputting moves via the command-line interface\n\nThe fact that I was thinking about all of these things was a sign that my initial spike was successful. However, it was also a sign that I should spend some time laying out the foundation for a real object-oriented project rather than continuing to hack things together as if I were writing a ball of Perl scripts.\n\n### Laying out some scaffolding\n\nAlthough you don't necessarily need to worry about writing super-clean code for a first release of a project, it is important to at least lay down the basic groundwork, which makes it possible to replace bad code with good code later. By introducing a `TextDisplay` object, I was able to reduce the _stackwars-viewer_ script to the following code:\n\n```ruby\n#!/usr/bin/env ruby\n\nrequire \"json\"\nrequire_relative \"../lib/stack_wars\"\n\ndata = JSON.parse(File.read(ARGV[0]))\n\nputs StackWars::TextDisplay.new(data)\n```\n\nAfter the initial extraction of the code from my script, I thought about how much time I wanted to invest in refactoring `TextDisplay`. I ended up deciding that because this game will eventually have a GUI that completely replaces its command-line interface, I shouldn't put too much effort into code that would soon be deleted. However, I couldn't resist making it at least a tiny bit more readable for the time being:\n\n```ruby\nmodule StackWars\n class TextDisplay\n COLOR_SYM = { \"black\" => \"B\", \"white\" => \"W\" }\n HEADER = \"#{' '*7}#{(0..8).to_a.join(' '*6)}\"\n SEPARATOR = \"#{' '*6} #{9.times.map { '|' }.join(' '*6)}\"\n\n BLACK_BORDER = \"#{' '*5}#{COLOR_SYM['black']*61}\"\n WHITE_BORDER = \"#{' '*5}#{COLOR_SYM['white']*61}\"\n\n def initialize(battlefield)\n @battlefield = battlefield\n end\n\n def to_s\n battlefield_text = @battlefield.map.with_index do |row, row_index|\n row_text = row.map do |color, strength|\n if color == \"unclaimed\"\n \"(___)\"\n else\n \"(#{COLOR_SYM[color]}#{strength.to_s.rjust(2)})\"\n end\n end.join(\"--\")\n\n \"#{row_index.to_s.rjust(3)} #{row_text}\\n\"\n end.join(\"#{SEPARATOR}\\n\")\n\n [HEADER, BLACK_BORDER, battlefield_text.chomp, WHITE_BORDER].join(\"\\n\")\n end\n end\nend\n```\n\nAfter writing this code, I wondered whether I should tackle the building of a proper `Battlefield` class that would take the raw data for each cell and wrap it in a `Territory` object. I was hesitant to make both of these changes at once, so I ended up compromising by creating a `Battlefield` class that simply wrapped the nested array of primitives for now:\n\n```ruby\nmodule StackWars\n class Battlefield\n def self.from_json(json_file)\n new(JSON.parse(File.read(json_file)))\n end\n\n def initialize(territories)\n @territories = territories\n end\n\n def to_a\n Marshal.load(Marshal.dump(@territories))\n end\n\n # loses instance variables, but better than hitting to_s() by default\n alias_method :inspect, :to_s\n\n def to_s\n TextDisplay.new(to_a).to_s\n end\n end\nend\n```\n\nWith this new object in place, I was able to further simplify the _stackwars-viewer_ script, leading to the trivial code shown here:\n\n```ruby\nrequire_relative \"../lib/stack_wars\"\n\nputs StackWars::Battlefield.from_json(ARGV[0])\n```\n\nThe benefit of doing these minor extractions is that it makes it possible to focus on the relationships between the objects in a system rather than their implementations. You can always refactor implementation code later, but interfaces are hard to untangle once you start wiring things up to them. This is why it is important to start thinking about the ingress and egress points of your objects as early as possible, even if you're still allowing yourself to write quick and dirty implementation code.\n\nThe benefits of laying the proper groundwork for your project and keeping things nicely organized are hard to see in the early stages but are extremely clear later when things get more complex.\n\n### Starting to chip away at the hard parts\n\nUnless you are an incredibly good software designer, odds are good that some aspects of your project will be harder to work on than others. There is even a funny quote that hints at this phenomenon: _\"The first 90 percent of the code accounts for the first 90 percent of the development time. The remaining 10 percent of the code accounts for the other 90 percent of the development time.\"_\n\nTo avoid this sort of situation, it is important to maintain a balance between easy tasks and more difficult tasks. Starting a project with an easy task is a great way to get the ball rolling, but if you don't tackle some challenging aspects of your project early on, you may find yourself having to rewrite a ton of code later. The hard parts of your project are what test your overall design as well as your understanding of the problem domain.\n\nWith this in mind, I knew it was time to take a closer look at some of the game actions in Stack Wars. Because the FORTIFY action must be implemented before any of the other game actions become meaningful, I decided to start there. The following code was my initial stab at figuring out what I needed to build in order to get this feature working:\n\n```ruby\ndef fortify(position)\n position.add_army(active_player.color)\n active_player.reserves -= 1\nend\n```\n\nUntil this point in the project, I had been avoiding writing formal tests because I had a mixture of trivial code and throwaway code. But now that I was about to work on some Serious Business, I decided to try test-driving things. After a fair amount of struggling, I decided to add _mocha_ into the mix and begin test-driving a `Game` class through the use of mock objects:\n\n```ruby\nrequire_relative \"../test_helper\"\n\ndescribe \"StackWars::Game\" do\n\n let(:territory) { mock(\"territory\") }\n let(:battlefield) { mock(\"battlefield\") }\n\n subject { StackWars::Game.new(battlefield) }\n\n it \"must be able to alternate players\" do\n subject.active_player.color.must_equal :black\n\n subject.start_new_turn\n subject.active_player.color.must_equal :white\n\n subject.start_new_turn\n subject.active_player.color.must_equal :black\n end\n\n it \"must be able to fortify positions\" do\n subject.expects(:territory_at).with([0,1]).returns(territory)\n territory.expects(:fortify).with(subject.active_player)\n\n subject.fortify([0,1])\n end\nend\n```\n\nTaking this approach made it possible for me to test whether the `Game` class was able to delegate `fortify` calls to territories, even though I had not yet implemented the `Territory` class. It gave me a pretty nice way to look at the problem from the outside in and resulted in a clean-looking `Game` class:\n\n```ruby\nmodule StackWars\n class Game\n def initialize(battlefield)\n @players = [Player.new(\"black\"), Player.new(\"white\")].cycle\n @battlefield = battlefield\n start_new_turn \n end\n\n attr_reader :active_player\n\n def fortify(position)\n territory = territory_at(position) \n \n territory.fortify(active_player)\n end\n\n def start_new_turn\n @active_player = @players.next\n end\n\n private\n\n def territory_at(position)\n @battlefield[*position]\n end\n end\nend\n```\n\nHowever, the problem remained that this code hinged on a number of features that were not implemented yet. This frustration caused me to begin working on getting the basic functionality in place for a `Territory` class without writing tests for its behaviors up front. I used a combination of the _stackwars-viewer_ tool and irb to verify that the `Territory` objects that I had shoehorned into the system were working as expected.\n\nAfter making it so that the `Battlefield` object contained a nested array of `Territory` objects, I went back and wrote some unit tests for `Territory`. The tests ended up being fairly long and tedious, but the implementation code for `Territory#fortify` ended up being quite simple and worked as expected:\n\n```ruby\nmodule StackWars\n class Territory\n # other methods omitted\n\n def fortify(player)\n if controlled_by?(player)\n player.deploy_army\n\n @army_strength += 1\n @occupant ||= player.color\n else\n raise Errors::IllegalMove\n end\n end\n end\nend\n```\n\nGetting the `Territory` tests to go green felt good, but I wasn't satisfied. Now that I had implemented a game action, I wanted to see it in real use. This itch lead me to write a simple script that simulated players fortifying their positions, which resulted in the following output:\n\n```\n 0 1 2 3 4 5 6 7 8\n WWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWW\n 0 (___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)\n | | | | | | | | |\n 1 (___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)\n | | | | | | | | |\n 2 (___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)\n | | | | | | | | |\n 3 (___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)\n | | | | | | | | |\n 4 (___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)\n | | | | | | | | |\n 5 (___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)\n | | | | | | | | |\n 6 (B 2)--(___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)\n | | | | | | | | |\n 7 (___)--(W 2)--(___)--(___)--(___)--(___)--(___)--(___)--(___)\n | | | | | | | | |\n 8 (___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)\n BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB\n\nFortifying black position at (0,6)\n\n 0 1 2 3 4 5 6 7 8\n WWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWW\n 0 (___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)\n | | | | | | | | |\n 1 (___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)\n | | | | | | | | |\n 2 (___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)\n | | | | | | | | |\n 3 (___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)\n | | | | | | | | |\n 4 (___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)\n | | | | | | | | |\n 5 (___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)\n | | | | | | | | |\n 6 (B 3)--(___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)\n | | | | | | | | |\n 7 (___)--(W 2)--(___)--(___)--(___)--(___)--(___)--(___)--(___)\n | | | | | | | | |\n 8 (___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)\n BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB\n\nFortifying white baseline position at (2,0)\n\n 0 1 2 3 4 5 6 7 8\n WWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWW\n 0 (___)--(___)--(W 1)--(___)--(___)--(___)--(___)--(___)--(___)\n | | | | | | | | |\n 1 (___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)\n | | | | | | | | |\n 2 (___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)\n | | | | | | | | |\n 3 (___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)\n | | | | | | | | |\n 4 (___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)\n | | | | | | | | |\n 5 (___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)\n | | | | | | | | |\n 6 (B 3)--(___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)\n | | | | | | | | |\n 7 (___)--(W 2)--(___)--(___)--(___)--(___)--(___)--(___)--(___)\n | | | | | | | | |\n 8 (___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)--(___)\n BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB\n```\n\nSeeing this example run gave me a great feeling because it was the first sign that something resembling a game was under development. However, the path to get to this point was long and arduous, even though this was by far the easiest action to implement. \n\n### Slamming into the wall\n\nThere comes a time in every reasonably complicated project at which you end up biting off more than you can chew. The exact reason for this will vary from project to project: perhaps you overlooked something in your design, or misunderstood a key requirement, or maybe you just let your code get too messy and it reached a point where it could no longer be extended without heavy refactoring. This sort of thing is normal and to be expected, but is also a critical turning point in your project. \n\nIf you aren't careful, the setbacks caused by slamming into the wall can really shake your spirits and negatively affect your productivity. Having unrealistic expectations that a certain task will be easy to complete is a surefire way to trigger this effect. That's what happened to me on this project; I hope the following story will serve as a cautionary tale.\n\nAfter implementing the FORTIFY action, I thought I would be able to repeat the process for MOVE, ATTACK, and INVADE. Anticipating this approach, I roughed out a replacement for `Game#fortify()` called `Game#play()`, which took into account all game actions and selected the right one based on the circumstances:\n\n```ruby\ndef play(pos1, pos2=nil)\n if pos2.nil?\n territory = territory_at(pos1)\n \n territory.fortify(active_player)\n else\n from = territory_at(pos1)\n to = territory_at(pos2)\n\n raise Errors::IllegalMove unless battlefield.adjacent?(from, to)\n\n if to.occupied_by?(opponent)\n attack(from, to)\n else\n move_army(from, to)\n end\n end\n \n start_new_turn\nend\n```\n\nHowever, as soon as I looked at this method definition, I knew for sure that testing this code with mocks would be at best brittle and at worst downright misleading. On top of that, the code introduced several new concepts that would need to trickle down into the rest of the system. I tried to think through how to simplify things so that this could be more easily tested, but quickly grew frustrated and ended up abandoning the idea of test-driving this functionality.\n\nInstead, I decided that the problem was that I didn't have a running game console that displayed the battlefield and accepted inputs. I thought that even a buggy system that allowed me to interact with the game in a tangible way would be better than writing a ton of tedious tests against what might end up being the wrong interface. This decision caused me to begin to modify the system in any number of haphazard ways until I got a functioning game console.\n\nI did eventually get something working, but it was so fragile that I ended up enlisting my wife's help in QA testing it until it sort of looked and felt like a working game. Unfortunately, the hot fixes I was applying while she found new ways to break things caused more bugs to surface. Eventually, I gave up on the project for the evening and decided to come back to it with fresh eyes in the morning.\n\n### Searching for a pragmatic middle path\n\nConventional wisdom says that if a particular bit of code is especially hard to test, a structural flaw in your design might be to blame. Because testability and extensibility are linked together, it is a good idea to listen to your tests when they make life hard for you. But there certainly are times when we need to temporarily sacrifice purity in the name of practicality.\n\nThe fact that I had a working Stack Wars implementation but no confidence that it wasn't super buggy left me in a sticky position: I wanted to make sure that the code would stabilize, but I didn't want to rework the design from scratch. The base design of the code was more than good enough for a first release; I just wanted to iron the wrinkles out and find a way to refactor with a bit of confidence that each change I made wasn't going to break everything all over again.\n\nTo accomplish this goal, I started by making my manual testing process more efficient. I made it so that my game console would fire up a game in which each player had only 2 armies in reserve rather than 27. This change made it possible to play an entire game in a fraction of the time that a real game would take, but still allowed me to test all the game actions. I used this faster manual feedback loop to quickly eliminate the bugs that I had introduced the night before, and I also tried to be more careful about the fixes I applied.\n\nOnce I got things to a reasonable level of stability, I realized that I could then build a fairly good integration test by replaying moves from a real, complete game. After a few more tweaks, my wife and I managed to make it through a game without a crash. I then set up a demo script that would replay these moves one by one until it reached the end of the game. Once I got that stage working, I extracted it into an integration test that reads the moves from a JSON file, calls `Game#play` for each one, and then does some validations to make sure the game ended as expected:\n\n```ruby\ndescribe \"A full game of stack wars\" do\n let(:moves) do\n moves_file = \"#{File.dirname(__FILE__)}/../fixtures/moves.json\"\n JSON.load(File.read(moves_file))\n end\n\n let(:battlefield) { StackWars::Battlefield.new }\n\n let(:game) { StackWars::Game.new(battlefield) }\n\n it \"must end as expected\" do\n message = catch(:game_over) do\n moves.each { |move| game.play(*move) }\n end\n\n message.must_equal \"white won!\"\n\n white = game.active_player\n black = game.opponent\n\n battlefield.deployed_armies(white).must_equal(4)\n battlefield.deployed_armies(black).must_equal(0)\n\n white.reserves.must_equal(0)\n black.reserves.must_equal(0)\n\n white.successful_invasions.must_equal(6)\n black.successful_invasions.must_equal(4)\n end\nend\n```\n\nHaving this integration test in place will make it possible for me to come back later and refactor the codebase to make it more testable without the fear of breaking things. Although unit tests offer more in the way of documenting how the codebase is meant to work and provide more precisely located feedback upon failure, this single test is good enough to ensure that I don't introduce new critical bugs into the application without noticing them.\n\nIn retrospect, it seems like integration testing is more important than exhaustive unit testing in the very early phases of a hard-to-test project. It is less of a time investment to create some black box testing and such testing is less likely to be thrown out as subsystems change rapidly during the initial phases of development.\n\n### Shipping the 0.1.0 release\n\nIt is important to remember that a 0.1.0 release of an open source project is basically just a way to communicate an idea to your fellow programmers. If you label a release 0.1.0, no one is going to expect feature completeness, stability, or even a particularly high level of code quality. What they will expect is for you to have attempted to make your project comprehensible to them and ideally to have done a good job of making it easy to get involved in your project. I tried to keep these priorities in mind while preparing Stack Wars for release. \n\nAdding the full game test was an important first step for making the codebase release-ready. People who try out the game are going to want to be able to submit bug reports and possibly add new bits of functionality. Without tests to verify that their changes don't break stuff, it would be much harder to safely accept their contributions.\n\nSome additional code cleanup was also necessary: I removed a bunch of tests and examples that were no longer relevant and shifted around some of the code within each class to make it easier to read. In general, it is a good idea to remove anything that is not actively in use, as well as anything that isn't quite working correctly whenever you release code. This step lessens the chances of confusion and frustration when someone tries to read your code. \n\nI did not bother with API documentation just yet because so much is still subject to change, but I did write up a basic [README](https://github.com/sandal/stack_wars/blob/master/README.md) with instructions for those who want to play-test the game as well as those who might want to hack on its code. I also wrote a detailed writeup of the [game rules](https://github.com/sandal/stack_wars/blob/master/RULES.txt) because folks will need to learn the game before they can understand how this program works.\n\nIn addition to documentation and cleanup, I did what I could to make it very easy to try out the game. Running `stack_wars rules` will display the game rules so that you don't need to go back to the source code or your web browser to know how the game is played. Additionally, I made it possible to run through the sample game that Jia and I played just by typing `stack_wars demo`. The sole reason these features exist is to make the software more accessible to new users, which I hope will increase the chance that those users become contributors at some point in the future. But even if most people download my software without ever contributing to it in some way, I still care a lot about the experience they have while using something I created.\n\nYou can try things out for yourself by following the instructions in the [README](https://github.com/sandal/stack_wars/blob/master/README.md); this video will give you a sense of what the first release of this project ended up looking like:\n\n\n\n\n
\n\nAlthough in the grand scheme of things it may not look like much, I am pretty happy with it for a 0.1.0 release!\n\n### Reflections\n\nThe more I think about it, the more I realize that the cycle I've outlined in this article is pretty much the one I go through whenever I'm starting a new project. There are some things about my process that I like, and some things that I don't. However, knowing that there is a pattern I tend to follow makes me think that I can now work towards optimizing it over time.\n\nThe thing I found fascinating about this exercise is that it really drove home the point that software development is about a lot more than just writing code. There are a whole range of skills involved in bringing a software project from the idea phase to even the most humble first release, and it seems like it'd be good for us to spend time optimizing the whole process rather than just our purely code-oriented skills.\n\nBecause I've never attempted anything quite like this exercise before, I'm really curious to hear your thoughts on this article. Please leave a comment, even if you're the type that typically lurks, with whatever your initial gut reaction may be. If this is a topic that interests you, please also share your thoughts on how we might be able to dig even deeper in future experiments.\n"
},
{
"path": "articles/v2/011-domain-specific-api-construction.md",
"content": "Many people are attracted to Ruby because of its flexible syntax. Through various tricks and techniques, it is possible for Ruby APIs to blend seamlessly into a wide range of domains. \n\nIn this article, we will investigate how domain-specific APIs are constructed by implementing simplified versions of popular patterns seen in the wild. Hopefully, this exploration will give you a better sense of the tools you have at your disposal as well as a chance to see various Ruby metaprogramming concepts being used in context.\n\n### Implementing `attr_accessor`\n\nOne of the first things every beginner to Ruby learns is how to use `attr_accessor`. It has the appearance of a macro or keyword, but is actually just an ordinary method defined at the module level. To illustrate that this is the case, we can easily implement it ourselves.\n\n```ruby\nclass Module\n def attribute(*attribs)\n attribs.each do |a|\n define_method(a) { instance_variable_get(\"@#{a}\") }\n define_method(\"#{a}=\") { |val| instance_variable_set(\"@#{a}\", val) }\n end\n end\nend\n\nclass Person\n attribute :name, :email\nend\n\nperson = Person.new\nperson.name = \"Gregory Brown\"\n\np person.name #=> \"Gregory Brown\"\n```\n\nIn order to understand what is going on here, you need to think a little bit about what a class object in Ruby actually is. A class object is an instance of the class called `Class`, which is in turn a subclass of the `Module` class. When an instance method is added to the `Module` class definition, that method becomes available as a class method on all classes.\n\nAt the class/module level, it is possible to call `define_method`, which will in turn add new instance methods to the class/module that it gets called on. So when the `attribute()` method gets called on `Person`, a pair of methods get defined on `Person` for each attribute. The end result is that functionally equivalent code to what is shown below gets dynamically generated:\n\n```ruby\nclass Person\n def name\n @name\n end\n\n def name=(val)\n @name = val\n end\n\n def email\n @email\n end\n\n def email=(val)\n @email = val\n end\nend\n```\n\nThis is powerful stuff. As soon as you recognize that things like `attr_accessor` are not some special keywords or macros that only the Ruby interpreter can define, a ton of possibilities open up.\n\n### Implementing a Rails-style `before_filter` construct\n\nRails uses class methods all over the place to make it look and feel like its own dialect of Ruby. As a single example, it is possible to register callbacks to be run before a given controller action is executed using the `before_filter` feature. The simplified example below is a rough approximation of what this functionality looks like in Rails.\n\n```ruby\nclass ApplicationController < BasicController\n before_filter :authenticate\n\n def authenticate\n puts \"authenticating current user\"\n end\nend\n\nclass PeopleController < ApplicationController \n before_filter :locate_person, :only => [:show, :edit, :update]\n\n def show\n puts \"showing a person's data\"\n end\n\n def edit\n puts \"displaying a person edit form\"\n end\n\n def update\n puts \"committing updated person data to the database\"\n end\n\n def create\n puts \"creating a new person\"\n end\n\n def locate_person\n puts \"locating a person\"\n end\nend\n```\n\nSuppose that `BasicController` provides us with the `before_filter` method as well as an `execute` method which will execute a given action, but first trigger any `before_filter` callbacks. Then we'd expect the `execute` method to have the following behavior. \n\n```ruby\ncontroller = PeopleController.new\n\nputs \"EXECUTING SHOW\"\ncontroller.execute(:show)\n\nputs\n\nputs \"EXECUTING CREATE\"\ncontroller.execute(:create)\n\n=begin -- expected output --\n\nEXECUTING SHOW\nauthenticating current user\nlocating a person\nshowing a person's data\n\nEXECUTING CREATE\nauthenticating current user\ncreating a new person \n\n=end \n```\n\nImplementing this sort of behavior isn't as trivial as implementing a clone of `attr_accessor`, because in this scenario we need to manipulate some class level data. Things are further complicated by the fact that we want filters to propagate down through the inheritance chain, allowing a given class to apply both its own filters as well as the filters of its ancestors. Fortunately, Ruby provides facilities to deal with both of these concerns, resulting in the following implementation of our `BasicController` class:\n\n```ruby\nclass BasicController\n def self.before_filters\n @before_filters ||= []\n end\n\n def self.before_filter(callback, params={})\n before_filters << params.merge(:callback => callback)\n end\n\n def self.inherited(child_class)\n before_filters.reverse_each { |f| child_class.before_filters.unshift(f) }\n end\n\n def execute(action)\n matched_filters = self.class.before_filters.select do |f| \n f[:only].nil? || f[:only].include?(action) \n end\n\n matched_filters.each { |f| send f[:callback] }\n send(action)\n end\nend\n```\n\nIn this code, we store our filters as an array of hashes on each class, and use the `before_filters` method as a way of lazily initializing that array. Whenever a subclass gets created, the parent class copies its filters to the front of list of filters that the child class will continue to build up. This allows downward propagation of filters through the inheritance chain. If that sounds confusing, exploring in irb a bit might help clear up what ends up happening as a result of this `inherited` hook.\n\n```\n>> BasicController.before_filters.map { |e| e[:callback] }\n=> []\n>> ApplicationController.before_filters.map { |e| e[:callback] }\n=> [:authenticate]\n>> PeopleController.before_filters.map { |e| e[:callback] }\n=> [:authenticate, :locate_person]\n```\n\nFrom here, it should be pretty easy to see how the `execute` method works. It simply looks up this list of filters and selects the ones relevant to the provided action. It then uses `send` to call each callback that was matched, and finally calls the target action. \n\nWhile we've only gone through two examples of class level macros so far, the techniques used between the two of them cover most of what you'll need to know to understand virtually all uses of this pattern in other scenarios. If we really wanted to dig in deeper, we could go over some other tricks such as using class methods to mix modules into classes on-demand (a common pattern in Rails plugins), but instead I'll leave those concepts for you to explore on your own and move on to some other interesting patterns.\n\n### Implementing a cheap counterfeit of Mail's API\n\nHistorically, sending email in Ruby has always been an ugly and cumbersome process. However, the Mail gem changed all of that not too long ago. Using Mail, sending a message can be as simple as the code shown below.\n\n```ruby\nMail.deliver do\n from \"gregory.t.brown@gmail.com\"\n to \"test@test.com\"\n subject \"Hello world\"\n body \"Hi there! This isn't spam, I swear\"\nend\n```\n\nThe nice thing about Mail is that in addition to this convenient syntax, it is still possible to work with a more ordinary looking API as well.\n\n```ruby\nmail = Mail::Message.new\n\nmail.from = \"gregory.t.brown@gmail.com\"\nmail.to = \"test@test.com\"\nmail.body = \"Hi there! This isn't spam, I swear\"\nmail.subject = \"Hello world\"\n\nmail.deliver\n```\n\nIf we ignore the actual sending of email and focus on the interface to the object, implementing a dual purpose API like this is surprisingly easy. The code below defines a class that provides a matching API to the examples shown above.\n\n```ruby\nclass FakeMail\n def self.deliver(&block)\n mail = Message.new(&block)\n mail.deliver\n end\n\n class Message\n def initialize(&block)\n instance_eval(&block) if block\n end\n\n attr_writer :from, :to, :subject, :body\n\n def from(text=nil)\n return @from unless text \n\n self.from = text\n end\n\n def to(text=nil)\n return @to unless text\n\n self.to = text\n end\n\n def subject(text=nil)\n return @subject unless text\n\n self.subject = text\n end\n\n def body(text=nil)\n return @body unless text\n\n self.body = text\n end\n\n # this is just a placeholder for a real delivery method\n def deliver\n puts \"Delivering a message from #{from} to #{to} \"+\n \"with the subject '#{subject}' and \"+\n \"the body '#{body}'\"\n end\n end\nend\n```\n\nThere are only two things that make this class definition different from that of the ordinary class definitions we see in elementary Ruby textbooks. The first is that the constructor for `FakeMail::Message` accepts an optional block to run through `instance_eval`, and the second is that the class provides accessor methods which can act as both a reader and a writer depending on whether an argument is given or not. These two special features go hand in hand, as the following example demonstrates:\n\n```ruby\n FakeMail.deliver do \n # this looks ugly, but would be necessary if using ordinary attr_accessors\n self.from = \"gregory.t.brown@gmail.com\"\n\n end\n\n mail = FakeMail::Message.new\n\n # when you strip away the syntactic sugar, this looks ugly too.\n mail.from \"gregory.t.brown@gmail.com\"\n```\n\nThis approach to implementing this pattern is fairly common and shows up in a lot of different Ruby projects, including my own libraries. By accepting a bit more complexity in our accessor code, we end up with a more palatable API in both scenarios, and it feels like a good trade. However, the dual purpose accessors always felt like a bit of a hack to me, and I recently found a different approach that is I think is a bit more solid. The code below shows how I would attack this problem in new projects:\n\n```ruby\nclass FakeMail\n\n def self.deliver(&block)\n mail = MessageBuilder.new(&block).message\n mail.deliver\n end\n\n class MessageBuilder\n def initialize(&block)\n @message = Message.new\n instance_eval(&block) if block\n end\n\n attr_reader :message\n\n def from(text)\n message.from = text\n end\n\n def to(text)\n message.to = text\n end\n\n def subject(text)\n message.subject = text\n end\n\n def body(text)\n message.body = text\n end\n end\n\n class Message\n attr_accessor :from, :to, :subject, :body\n\n def deliver\n puts \"Delivering a message from #{from} to #{to} \"+\n \"with the subject '#{subject}' and \"+\n \"the body '#{body}'\"\n end\n end\nend\n```\n\nThis code is a drop-in replacement for what I wrote before, but is quite different under the hood. Rather than putting the syntactic sugar directly onto the `Message` object, I create a `MessageBuilder` object for this purpose. When `FakeMail.deliver` is called, the `MessageBuilder` object ends up being the target context for the block to be evaluated in rather than the `Message` object. This effectively splits the code the implements the sugary interface from the code that implements the domain model, eliminating the need for dual purpose accessors.\n\nThere is another benefit that comes along with taking this approach, but it is more subtle. Whenever we use `instance_eval`, it evaluates the block as if you were executing your statements within the object it was called on. This means it is possible to bypass private methods and otherwise mess around with objects in ways that are typically reserved for internal use. By switching the context to a simple facade object whose only purpose is to provide some domain specific API calls for the user, it's less likely that someone will accidentally call internal methods or otherwise stomp on the internals of the system's domain objects.\n\nIt's worth mentioning that even this improved approach to implementing an `instance_eval` based interface comes with its own limitations. For example, whenever you use `instance_eval`, it makes it so that `self` within the block points to the object the block is being evaluated against rather than the object in the the surrounding scope. The closure property of Ruby code blocks makes it possible to access local variables, but if you reference instance variables, they will refer to the object your block is being evaluated against. This can confuse beginners and even some more experienced Ruby developers. \n\nIf you want to use this style of API, your best bet is to reserve it for things that are relatively simple and configuration-like in nature. As things get more complex the limitations of this approach become more and more painful to deal with. That having been said, valid use cases for this pattern occur often enough that you should be comfortable implementing it whenever it makes sense to do so.\n\nThe next pattern is one that you probably WON'T use all that often, but is perhaps the best example of how far you can stretch Ruby's syntax and behaviors to fit your own domain.\n\n### Implementing a shoddy version of XML Builder\n\nOne of the first libraries that impressed me as a Ruby newbie was Jim Weirich's XML Builder. The fact that you could create a single Ruby object that magically knew how to convert arbitrary method calls into an XML structure seemed like pure voodoo to me at the time. \n\n```ruby\nrequire \"builder\"\n\nbuilder = Builder::XmlMarkup.new\n\nxml = builder.class do |roster|\n roster.student { |s| s.name(\"Jim\"); s.phone(\"555-1234\") }\n roster.student { |s| s.name(\"Jordan\"); s.phone(\"123-1234\") }\n roster.student { |s| s.name(\"Greg\"); s.phone(\"567-1234\") }\nend\n\nputs xml\n\n=begin -- expected output --\n\n\n\n
\n\nTo implement the map generation code, I put together a simple `Location` object which is essentially an infinite two dimensional doubly linked list. Notice how the class definition below makes extensive use of the common `||=` idiom to handle the lazy evaluation and caching.\n\n```ruby\nclass Location\n def self.[](x,y)\n @locations ||= {}\n @locations[[x,y]] ||= new(x,y)\n end \n\n def initialize(x,y)\n @x = x\n @y = y\n @color = [:green, :green, :blue].sample\n end\n\n def ==(other)\n [x,y] == [other.x, other.y]\n end \n\n attr_reader :x, :y, :color\n\n def north\n @north ||= Location[@x,@y-1] \n end\n\n def south\n @south ||= Location[@x,y+1]\n end\n\n def east\n @east ||= Location[@x+1, @y] \n end\n\n def west\n @west ||= Location[@x-1, @y] \n end\n\n def neighbors\n [north, south, east, west]\n end\nend\n```\n\nWhile this technique works fine and is the traditional way to achieve lazy evaluation in Ruby, it feels a bit primitive. Ruby does not provide a general purpose construct for lazy evaluation, but if it did, it would allow us to write code similar to what you see below:\n\n```ruby\nclass Location\n def self.[](x,y)\n @locations ||= {}\n @locations[[x,y]] ||= new(x,y)\n end \n\n def initialize(x,y)\n @x = x\n @y = y\n @color = [:green, :green, :blue].sample\n\n @north = LazyObject.new { Location[@x,@y-1] }\n @south = LazyObject.new { Location[@x,y+1] }\n @east = LazyObject.new { Location[@x+1, @y] }\n @west = LazyObject.new { Location[@x-1, @y] }\n end\n\n def ==(other)\n [x,y] == [other.x, other.y]\n end \n\n def neighbors\n [north, south, east, west]\n end\n\n attr_reader :x, :y, :color, :north, :south, :east, :west\nend\n```\n\nSuch an object can be implemented as a simple proxy which delays the execution of a callback until the results are actually needed. The following code illustrates one way to do that:\n\n```ruby\nclass LazyObject < BasicObject\n def initialize(&callback)\n @callback = callback\n end\n\n def __result__\n @__result__ ||= @callback.call\n end\n\n def method_missing(*a, &b)\n __result__.send(*a, &b)\n end\nend\n```\n\nAnother option would be to use [lazy.rb](http://moonbase.rydia.net/software/lazy.rb/), which provides similar functionality via `Lazy::Promise` objects that get instantiated via the `Lazy.promise` method:\n\n```ruby\nrequire \"lazy\"\n\nclass Location\n def self.[](x,y)\n @locations ||= {}\n @locations[[x,y]] ||= new(x,y)\n end \n\n def initialize(x,y)\n @x = x\n @y = y\n @color = [:green, :green, :blue].sample\n\n @north = Lazy.promise { Location[@x,@y-1] }\n @south = Lazy.promise { Location[@x,y+1] }\n @east = Lazy.promise { Location[@x+1, @y] }\n @west = Lazy.promise { Location[@x-1, @y] }\n end\n\n def ==(other)\n [x,y] == [other.x, other.y]\n end\n\n def neighbors\n [north, south, east, west]\n end\n\n attr_reader :x, :y, :color, :north, :south, :east, :west\nend\n```\n\nThis approach provides a thread safe solution and prevents us from having to reinvent the wheel. The only downside is that _lazy.rb_ is a bit dated and generates some warnings on Ruby 1.9 due to the way it implements its core proxy object. But whether you use _lazy.rb_ or roll your own lazy object, it is important to understand that the difference between this pattern and the common Ruby idiom of delaying execution via cached method calls is more than just a matter of aesthetics. To illustrate why that is the case, we can can consider the difference in behavior between the two approaches when `Location#neighbors` is called.\n\nIn the original example that explictly defines the `north`, `south`, `east`, and `west` methods, the first time the `neighbors` method is called, four `Location` objects are created. This means that the following line of code will generate all four neighboring `Location` objects even if not all of them are needed to answer the question it asks:\n\n```ruby\ngreen_neighbor = location.neighbors.find { |loc| loc.color == :green }\n```\n\nBy contrast, a `Location` object that uses some form of lazy object would behave differently here. Because `Enumerable#find` returns as soon as it finds a single object which matches its conditions, the `Location#color` method will not necessarily get called on each of the neighboring locations. This means that in the best case scenario, only one new `Location` object would end up getting created. While this particular example is a bit contrived, it's not hard to see why this is a desireable characteristic of lazy objects that cannot be easily emulated via the standard Ruby idiom for delayed execution.\n\n### Future objects\n\nLazy objects provide certain performance benefits in the sense that they make it possible to avoid unnecessary computation, but they don't do anything to improve the perceived waiting time for any computations that actually need to be run. This is where future objects come in handy.\n\nA future object is essentially an object which immediately begins doing some processing in the background but only blocks if the results are demanded before the thread has finished executing. The example below demonstrates how this sort of object can come in handy for building a simple non-blocking download manager:\n\n```ruby\nrequire \"open-uri\"\nrequire \"lazy\"\n\nclass DownloadManager\n def initialize\n @downloads = []\n end\n\n def save(url, filename)\n downloads << Lazy.future { File.binwrite(filename, open(url).read) }\n end\n\n def finish_all_downloads\n downloads.each { |d| Lazy.demand(d) }\n end\n\n private\n\n attr_reader :downloads\nend\n\ndownloader = DownloadManager.new \n\ndownloader.save(\"http://prawn.majesticseacreature.com/manual.pdf\", \"manual.pdf\")\nputs \"Starting Prawn manual download\"\n\ndownloader.save(\"http://sandal.github.com/rbp-book/pdfs/rbp_1-0.pdf\", \"rbp_1-0.pdf\")\nputs \"Starting download of Ruby Best Practices book\"\n\nputs \"Waiting for downloads to finish...\"\ndownloader.finish_all_downloads\n```\n\nIn this particular example the callback doesn't return a meaningful value, and so the `DownloadManager#finish_all_downloads` method makes use of `Lazy.demand` to force each future to wrap up its computations. However, the following example demonstrates that the future objects that _lazy.rb_ provides can also be used as transparent proxy objects:\n\n```ruby\nrequire \"open-uri\"\nrequire \"lazy\"\n\nclass Download\n def initialize(url, filename)\n @filename = filename\n @contents = open(url).read\n end\n\n def save\n File.binwrite(@filename, @contents)\n end\nend\n\nclass DownloadManager\n def initialize\n @downloads = []\n end\n\n def save(url, filename)\n downloads << Lazy.future { Download.new(url, filename) }\n end\n\n def finish_all_downloads\n downloads.each { |d| d.save }\n end\n\n private\n\n attr_reader :downloads\nend\n\ndownloader = DownloadManager.new \n\ndownloader.save(\"http://prawn.majesticseacreature.com/manual.pdf\", \n \"manual.pdf\")\nputs \"Starting Prawn manual download\"\n\ndownloader.save(\"http://sandal.github.com/rbp-book/pdfs/rbp_1-0.pdf\", \n \"rbp_1-0.pdf\")\nputs \"Starting download of Ruby Best Practices book\"\n\nputs \"Waiting for downloads to finish...\"\ndownloader.finish_all_downloads\n```\n\nIn both examples, the future object will block as long as necessary to allow the computations to complete, but only when it is forced to do so. Until this occurs, other operations can continue in parallel and/or block the execution of these future objects. This means in the best case scenario, a computation will end up being completed before it is actually needed, and the results will be returned from the future object's cache at that time.\n\nWhile implementing a generic future object from scratch would not be difficult for anyone who has experience working with threads, concurrency is a weak point for me and I rather not confuse folks by giving potentially bad advice through a naive implementation of my own. Those who are really itching to see how such an object is implemented should look at the [lazy.rb source code](https://github.com/mental/lazy/blob/master/lib/lazy.rb#L138-146), but if you treat future objects as black boxes you just need to know a few basic things about Ruby's thread model to make use of this construct effectively.\n\nThe most important thing to keep in mind is that thread scheduling in standard Ruby is affected by a global interpreter lock (GIL) which makes it so that most computations end up blocking the execution of other threads. Alternative implementations such as JRuby and Rubinius remove this lock, but in standard Ruby this basically means that threads are mostly useful for backgrounding operations such as file and network I/O. This is because unlike most computations, I/O operations will give other threads a chance to run while waiting on their data to become available. Because lazy.rb's implementation is thread based, future objects inherit the same set of restrictions. The other thing to be aware of is that Ruby does not explicitly join all of its threads once the main execution thread completes. This means that if I did not explicitly call `downloader.finish_all_downloads` in the previous example, the threads spun up by my future objects would be terminated if the main thread finished up before the downloads were completed. This may be obvious to anyone with a background in concurrency, but I scratched by head for a bit because of this issue.\n\nOther than those issues, future objects pretty much allow you to solve some basic concurrency problems without knowing a whole lot about how to work with low level concurrency primitives. While the example I've shown here is a bit dull, I can imagine this technique might come in handy for things like sending emails or doing some time intensive computations that are part of an interactive reporting system. Both of these are problems I've had to solve before using tools like [Resque](https://github.com/defunkt/resque), but simple future objects might prove to be a lightweight \nalternative. I'd be curious to hear from our readers who have some concurrency experience whether that seems like a good idea or not, and also whether you have ideas for other potential applications of future objects.\n\n## Reflections\n\nThe general concept of wrapping data in results objects isn't that exciting, but the notion of lazy objects and future objects show that results objects can be imbued with rich behaviors that can make our code more flexible and easier to understand. \n\nWhile Rubyists are no strangers to benefits of lazy evaluation, the process of writing this article has lead me to believe that we can probably benefit from having some higher level constructs to work with. However, explaining my thoughts on that would take a whole other article.\n\nSimilarly, it seems that Ruby provides all the basic tooling necessary for concurrency, even when you take into account the limitations of standard Ruby due to its GIL. It would be nice if we could establish some good patterns and constructs for making this kind of programming more accessible to the amateur. Such constructs may end up hiding some of the details that experienced developers care about, but would likely lead to more performant code without sacrificing maintainability or learnability.\n\nOn a closing note, the fact that there are two interesting subcategories of results objects hints that there may be more left to discover. This is similarly true for patterns about arguments. I can feel in my gut that there are other patterns out there just waiting to be discovered, but cannot think of any off the top of my head at the moment. Have you seen anything in the wild that hints at how we can expand on these ideas? If so, please leave a comment! \n\n"
},
{
"path": "articles/v2/README.md",
"content": "These articles are from Practicing Ruby's second volume, which ran from \n2011-08-23 to 2011-11-30. The manuscripts in this folder correspond to the\nfollowing articles on practicingruby.com:\n\n* [Issue 2.1: Ways to load code](http://practicingruby.com/articles/shared/tmxmprhfrpwq) (2011.08.23)\n* [Issue 2.2: How to attack sticky problems](http://practicingruby.com/articles/shared/bhftubljbomqpmifbibmzmptlxhoin) (2011.08.30)\n* [Issue 2.3: A closure is a double edged sword](http://practicingruby.com/articles/shared/mvzhovpjbghr) (2011.09.06)\n* [Issue 2.4: Implementing Enumerable and Enumerator in Ruby](http://practicingruby.com/articles/shared/ggcwduoyfqmz) (2011.09.13)\n* [Issue 2.5: Thoughts on regression testing](http://practicingruby.com/articles/shared/ggcwduoyfqmz) (2011.09.20)\n* [Issue 2.6: Learning new things, step by step](http://practicingruby.com/articles/shared/vbmlgkdtahzd) (2011.09.27)\n* [Issue 2.7: \"Unobtrusive Ruby\" in practice](http://practicingruby.com/articles/shared/ozkzbsdmagcm) (2011.10.04)\n* [Issue 2.8: Ruby and the singleton pattern don't get along](http://practicingruby.com/articles/shared/jleygxejeopq) (2011.10.11)\n* [Issue 2.9: Building Unix-style command line applications](http://practicingruby.com/articles/shared/qyxvmrgmhuln) (2011.10.18)\n* [Issue 2.10: From requirements discovery to release](http://practicingruby.com/articles/shared/nlhxgszkgenq) (2011.10.27)\n* [Issue 2.11: Domain specific API construction](http://practicingruby.com/articles/shared/iptocucwujtj) (2011.11.02)\n* [Issue 2.12: Working with binary file formats](http://practicingruby.com/articles/shared/iptocucwujtj) (2011.11.09)\n* [Issue 2.13: Designing business reporting applications](http://practicingruby.com/articles/shared/gthgvfebjvyn) (2011.11.17)\n* [Issue 2.14: Thoughts on \"Arguments and Results\", Part 1](http://practicingruby.com/articles/shared/vpxpovppchww) (2011.11.23)\n* [Issue 2.15: Thoughts on \"Arguments and Results\", Part 2](http://practicingruby.com/articles/shared/wdykkrmdfjvf) (2011.11.30)\n"
},
{
"path": "articles/v3/001-quality-software.md",
"content": "I ended the second volume of Practicing Ruby by launching an exploration into\nthe uncomfortable question of what it means to write good code. To investigate\nthe topic, I began to compile a wiki full of small case studies for each of the\ndifferent properties outlined by [ISO/IEC\n9126](http://en.wikipedia.org/wiki/ISO/IEC_9126) -- an international standard\nfor evaluating software quality. While I made good headway on this project\nbefore taking a break for the holidays, I left some of it unfinished and\npromised to kick off this new volume by presenting my completed work.\n\nWhile it is possible to read through the wiki by [starting at the overview on\nthe homepage](https://github.com/elm-city-craftworks/code_quality/wiki) and then\nclicking through it page by page, there is a tremendous amount of content there\non lots of disjoint topics. To help find your way through these materials, I've\nsummarized their contents below so that you know what to expect.\n\n### Functionality concerns\n\nWhile we all know that getting our software to work correctly is important, the\nfunctional qualities of our software are often not emphasized as much as they\nshould be. Issues to consider in this area include:\n\n* The [suitability](https://github.com/elm-city-craftworks/code_quality/wiki/Suitability) \nof our software for serving its intended purpose. As an example, I note the\ndifferences between the _open-uri_ vs. _net/http_ standard libraries and suggest\nthat while they have some overlap in functionality, they are aimed at very\ndifferent use cases.\n\n* The [accuracy](https://github.com/elm-city-craftworks/code_quality/wiki/Accuracy) \nof our software in meeting its requirements. As an example, I discuss how a\nsmall bug in Gruff made the entire library unusable for running a particular\nreport, even though it otherwise was well suited for the problem.\n\n* The [interoperability](https://github.com/elm-city-craftworks/code_quality/wiki/Interoperability) \nof our software and how it effects our ability to fit seamlessly into the user's\nenvironment. As an example, I discuss at a high level the benefits of using the\nRack webserver interface as compared to writing adapters that directly connect\nweb frameworks with web servers.\n\n* The [security](https://github.com/elm-city-craftworks/code_quality/wiki/Security) \nof our software and how it affects the safety of our users and the systems our\nsoftware runs on. As an example, I discuss a small twitter bot I wrote for\ndemonstration purposes that had a operating system command injection\nvulnerability, and also show how I fixed the problem.\n\n### Reliability concerns\n\nEven if our software does what it is supposed to do, if it does not do so\nreliably, it will not do a good job at making users happy. Issues to consider in\nthis area include:\n\n* The [maturity](https://github.com/elm-city-craftworks/code_quality/wiki/Maturity) \nof our software, i.e. the gradual reduction of unexpected defects over time. As\nan example, I discuss some regression tests we've written for the Practicing\nRuby web application.\n\n* The [fault tolerance](https://github.com/elm-city-craftworks/code_quality/wiki/Fault-Tolerance) \nof our software, i.e. how easy it is for us to mitigate the impact of failures\nin our code. As an example, I discuss at a high level about how ActiveRecord\nimplements error handling around failed validations, and show a pure Ruby\napproximation for how to build something similar.\n\n* The [recoverability](https://github.com/elm-city-craftworks/code_quality/wiki/Recoverability) \nof our software when dealing with certain kinds of failures. As an example, I\ndiscuss various features that resque-retry provide for trying to recover from\nbackground job failures.\n\n### Usability concerns\n\nOnce we have code that does it's job correctly and does it well, we still need\nto think about how pleasant of an experience we create for our users. Issue to\nconsider in this area include:\n\n* The [understandability](https://github.com/elm-city-craftworks/code_quality/wiki/Understandability) \nof our software, in particular how well its functionality is organized and how\nwell documented it is. As an example, I extract some guidelines for writing a\ngood README file using Sinatra's README as a reference.\n\n* The [learnability](https://github.com/elm-city-craftworks/code_quality/wiki/Learnability)\nof our software, i.e. how easy it is to discover new ways of using the software\nbased on what the user already knows. As an example, I discuss a change we made\nto the Prawn graphics API to make things more consistent and easier to learn.\n\n* The [operability](https://github.com/elm-city-craftworks/code_quality/wiki/Operability) \nof our software, particularly whether we give our users the control and\nflexibility they need to get their job done. As an example, I discuss how most\nMarkdown processors in Ruby function as black boxes, and how RedCarpet 2 takes a\ndifferent approach that makes it much easier to customize.\n\n* The [attractiveness](https://github.com/elm-city-craftworks/code_quality/wiki/Attractiveness) \nof our software. As an example, I show the difference between low level and high\nlevel interfaces for interacting with the Cairo graphics library, and illustrate\nhow the use of syntactic sugar can influence user behavior.\n\n### Efficiency concerns\n\nRuby has had a reputation for being a slow, resource intensive programming\nlanguage. As a result, we need to rely on some special tricks to make sure that\nour code is fast enough to meet the needs of our users. Issues to consider in\nthis area include:\n\n* The [performance](https://github.com/elm-city-craftworks/code_quality/wiki/Performance)\nof our software. As an example, I talk at a very high level about the\ncomputationally expensive nature of PNG alpha channel splitting, and how C\nextensions can be used to solve that problem.\n\n* The [resource utilization](https://github.com/elm-city-craftworks/code_quality/wiki/Resource-Utilization)\ncharacteristics of our software. While this most frequently means memory and\ndisk space usage, there are lots of different resources our programs use. As an\nexample, I talk about the fairly elegant use of file locking in the PStore\nstandard library.\n\n### Maintainability concerns\n\nNo matter how good our software is, it will ultimately be judged by how well it\ncan change and grow over time. This is the area we tend to spend most of our\ntime studying, because difficult to maintain projects make us miserable as\nprogrammers. Issues to consider in this area include:\n\n* The [analyzability](https://github.com/elm-city-craftworks/code_quality/wiki/Analyzability) \nof our software, i.e. how easy it is for us to reason about our code. As an\nexample, I discuss at a high level how the Flog utility assigns scores to\nmethods based on their complexity, and how that can be used to identify areas of\nyour code that need refactoring.\n\n* The [changeability](https://github.com/elm-city-craftworks/code_quality/wiki/Changeability)\nof our software, which is commonly considered the holy grail of software design.\nAs an example, I point out connascence as a mental model for reasoning about the\nrelationships between software components and how easy or hard they are to\nchange.\n\n* The [stability](https://github.com/elm-city-craftworks/code_quality/wiki/Stability)\nof our software, in particular how much impact changes have on users. As an\nexample, I talk about the merits of designing unobtrusive APIs for reducing the\namount of moving parts in our code.\n\n* The [testability](https://github.com/elm-city-craftworks/code_quality/wiki/Testability)\nof our software. As an example, I discuss how useful the SOLID principles are in\nmaking our code easier to test.\n\n### Portability concerns\n\nOne thing we don't think about often in Ruby, perhaps not often enough, is how\neasy it is for folks to get our software up and running in environments other\nthan our own. While writing code in a high level language does get us away from\nsome of the problems that system programmers need to consider, there are still\nplatform and environment issues that deserve our attention. Issues to consider\nin this area include:\n\n* The [adaptability](https://github.com/elm-city-craftworks/code_quality/wiki/Adaptability) \nof our software to the user's environment. As an example, I discuss at a high\nlevel the approach HighLine takes to shield the user from having to write low\nlevel console interaction code.\n\n* The [installability](https://github.com/elm-city-craftworks/code_quality/wiki/Installability) \nof our software. As an example, I discuss some general thoughts about the \nstate of installing Ruby software, and look into an interesting approach to \nsetting up a Rails application in Jordan Byron's Mission of Mercy clinic \nmanagement project.\n\n* The [co-existence](https://github.com/elm-city-craftworks/code_quality/wiki/Co-existence) \nof our software with other software in the user's environment. As an example, \nI discuss how conflicting monkey patches led me on a wild goose chase in \none of my Rails applications.\n\n* The [replaceability](https://github.com/elm-city-craftworks/code_quality/wiki/Replaceability) \nof our software as well as the ability for our software to act as a drop in \nreplacement for other tools. Because I feel this concept is one baked into the UNIX and open \nsource culture, I don't provide a specific case study but instead point out several applications \nof this idea in the wild.\n\n### Reflections\n\nSpending several weeks studying this topic just so I can *start* a discussion\nwith our readers has been a painful, but enlightening experience for me. As you\ncan see from the giant laundry list of concerns listed above, the concept of\nsoftware quality is much deeper than something like [The Four Simple Rules of\nDesign](http://www.c2.com/cgi/wiki?XpSimplicityRules) might imply. \n\nIt is no surprise that we yearn for something more simple than what I've\noutlined here, but I cannot in good conscience remove any of the focus areas\noutlined by ISO/IEC 9126 as being unimportant when it comes to software quality.\nWhile we cannot expect that all of our software will be shining examples of all\nof these properties all of the time, we do have a responsibility for knowing how\nto spot the tensions between these various concerns and we must do our best to\nresolve them in a smart way.\n\nWhile our intuition and experience may allow us to address most of the issues\nI've outlined here at a subconscious level, I feel that more work needs to be\ndone for us to seriously consider ourselves good engineers. The real challenge\nfor me personally is to figure out how to continue to study these topics without\nstifling my creativity, willingness to experiment, and ability to make decisions\nwithout becoming overwhelmed.\n\nI look forward to hearing your own thoughts on this topic, because it is one\nthat we probably need to work through together if we want to make any real\nprogress.\n"
},
{
"path": "articles/v3/002-building-excellent-examples.md",
"content": "Good code examples are the secret sauce that makes Practicing Ruby a high-quality learning resource. That said, the art of building excellent examples is one that I think all programmers should practice, not just those folks out there trying to teach for a living. The ability to express ideas clearly through well-focused snippets of code is key to writing good tests, documentation, bug reports, code reviews, demonstrations, and a whole lot of other stuff, too.\n\nIn this article, I've identified five patterns I use for expressing ideas through code examples. These techniques run the gamut from building contrived \"Hello World\" programs to crafting full-scale sample applications using a literate programming style. Each technique has its own strengths and weaknesses, and I've done what I could to outline them where possible.\n\nAlthough this isn't necessarily a comprehensive list, it can help you start to improve the way you write your examples and also serves as a good jumping-off point for further discussion on the topic.\n\n### Contrived examples\n\nFor any given programming language or software library, the odds are pretty good that the first example you'll run is a contrived \"Hello World\" program. The following example is taken from the Sinatra web framework but is similar in spirit to pretty much every other \"Hello World\" application out there:\n\n```ruby\nrequire 'sinatra'\n\nget '/hi' do\n \"Hello World!\"\nend\n```\n\nThis kind of example seems quite useless on the surface and is neither interesting nor educational. As it turns out, these characteristics are precisely what make this \"Hello World\" program perfect! The contrived nature of the example allows it to serve as a simple sanity check for someone who is trying out Sinatra for the first time.\n\nIf you try to run this example and find that it doesn't work correctly, there are only a few possible points of failure. In most cases, not even getting a \"Hello World\" program to run correctly can be blamed on one of three things: out-of-date documentation, issues with your environment, or user error. The fact that there are very few moving parts makes it much easier for you to determine the source of your problem than it would be if the example were significantly more complex. This ease of debugging is precisely why most introductory tutorials start off with a \"Hello World\" program rather than something more exciting.\n\nAlthough the most common use case for contrived examples is to construct \"Hello World\" applications, there are other use cases for this technique as well. In particular, contrived examples are a good fit for discussions about syntactic or structural differences between two pieces of code. As an example, consider a short tutorial that explains why a user might want to use Ruby's `attr_reader` functionality. It could start by showing a `Person` class that implements accessors explicitly:\n\n```ruby\nclass Person\n def initialize(name, email)\n @name = name\n @email = email\n end\n\n def name\n @name\n end\n\n def email \n @email\n end\nend\n```\n\nA followup example could then be provided to show how to simplify the code via `attr_reader`:\n\n```ruby\nclass Person\n def initialize(name, email)\n @name = name\n @email = email\n end\n\n attr_reader :name, :email\nend\n```\n\nThis scenario is very simplistic when compared to the class definitions we write in real projects, but the absence of complicated functionality makes it easier for the reader to focus on the syntactic differences between the two examples. It also allows the novice Ruby programmer to think of the difference between explicitly defining accessors and using `attr_reader` as a simple structural transformation rather than something with complex semantic differences. Although this mental model is not 100 percent accurate, it emphasizes the big picture, which is what actually matters for a novice programmer. The simplicity of these examples makes the general pattern much easier to remember, which justifies hiding a few things behind the curtain to be revealed later.\n\nUnfortunately, the ability of contrived examples to hide the semantics of our programming constructs is just as often a drawback as it is an asset. The more complex a concept is, the more dangerous it is to present simplistic examples rather than working through more realistic scenarios. For example, it is common for object-oriented programming tutorials to use real-world objects and hierarchies to explain how class inheritance works, but the disconnect between these models and the kinds that real software projects implement is so great that this approach completely obfuscates the real power and purpose of object-oriented programming. By choosing a scenario that may feel natural to the reader but does not fit naturally with the underlying programming constructs, this sort of tutorial fails to emphasize the right details and leaves the door open for a wide range of misconceptions. These incorrect assumptions end up getting in the way of learning real object-oriented programming techniques rather than helping develop an understanding of them.\n\nI could easily rant on this topic, but someone else did it for me by writing a great mailing list post entitled [Goodbye, shitty Car extends Vehicle object-orientation tutorial](http://lists.canonical.org/pipermail/kragen-tol/2011-August/000937.html). Despite the somewhat inflammatory title, it is a very insightful post, and I strongly recommend reading it if you want to see a strong argument for the limitations of contrived examples as teaching tools.\n\nFiguring out where to draw the line between when it is appropriate to use a contrived example and when to use one that is based on a practical application is tricky. In general, I try to keep in mind that the purpose of a contrived example is specifically to remove context from the picture. Outside of \"Hello World\" programs and simple syntactic transformations, a lack of context hurts more than it helps, and so I try to avoid contrived examples as much as I can for pretty much every other use case. \n\n### Cheap counterfeits\n\nOne of my favorite techniques for teaching programming concepts is to construct cheap counterfeits that emulate the surface-level behavior of a more complicated structure. These \"poor man's implementations\" are similar to contrived examples in that they can hide as much complexity as they'd like from the reader but, because they are grounded by some realistic scenario, do not suffer from being totally disconnected from practical applications.\n\nI have used this technique extensively throughout Practicing Ruby and my other written works, and it almost always works out well. In fact, the issue on [Implementing Enumerable and Enumerator in Ruby](http://practicingruby.com/articles/4) was entirely based on this strategy and turned out to be one of the most popular articles I've written for this journal. Although you are probably already very familiar with this pattern as a Practicing Ruby reader, I can still provide a bit of extra insight by decomposing it for you.\n\nThe purpose of building a cheap counterfeit is not to gain a deep understanding of how a certain construct actually works. Instead, the purpose of a counterfeit is to teach people how to steal ideas from other interesting bits of code for their own needs. For example, take the previous `attr_reader` example:\n\n```ruby\nclass Person\n def initialize(name, email)\n @name = name\n @email = email\n end\n\n attr_reader :name, :email\nend\n```\n\nThis is a great feature, because it replaces tedious boilerplate methods with a concise declarative statement. But without some sort of explanation as to how it works, `attr_reader` feels pretty magical and might be perceived as a special case that the Ruby internals are responsible for handling. This misconception can easily be cleared up by showing how to implement a cheap counterfeit version of `attr_reader` in application code:\n\n```ruby\nclass Module\n def my_attr_reader(*args)\n args.each do |a|\n define_method(a) { instance_variable_get(\"@#{a}\") }\n end\n end\nend\n\nclass Person\n def initialize(name, email)\n @name = name\n @email = email\n end\n\n my_attr_reader :name, :email\nend\n```\n\nIf teaching programmers how to use `attr_reader` is like treating them to a nice fish dinner, teaching them how to implement it is like giving them a fishing pole and showing them how to catch their own meals. Seeing a practical use of `define_method` opens the doors for a huge range of other applications, all of which hinge on the simple concept of dynamic method definition. For example, a similar technique could be used to convert hideous method names like `test_a_user_must_be_able_to_log_in` into the elegant syntax shown here:\n\n```ruby\ntest \"A user must be able to log in\" do\n # your test code here\nend\n```\n\nThere are countless other applications of dynamic method definition, many of which I expect Practicing Ruby readers are already familiar with. The point here is that a single example that demystifies a certain technique can make a huge difference in what possibilities someone sees in a given system. This payoff is what makes cheap counterfeits such a tremendously good teaching tool.\n\nAn important thing to keep in mind, however, is that this technique is useful mostly for teaching concepts, as opposed to showing someone how a feature is really implemented. If you actually look into the implementation of `attr_reader`, you'll find a number of edge cases that this cheap counterfeit example does not take into consideration. Although these subtleties are not especially relevant if you're just trying to give a contextualized example of how `define_method` can be used, they would be important to point out if you were trying to write a specification for how `attr_reader` is meant to work, which is why cheap counterfeits are not a substitute for case studies of real code but instead serve a different purpose entirely.\n\n### Simplified examples\n\nDigging directly into the source code of a project is the most direct way to understand how its features are implemented, but it can be a somewhat disorienting process. Production code in all but the most trivial projects tends to accumulate edge cases, error-checking code, and other bits of cruft that make it harder to see what the core ideas are. When giving a talk about how something is implemented or writing documentation for potential contributors, it is sometimes helpful to provide simplified examples that demonstrate the key functionality while minimizing distractions.\n\nSuppose I want to do a lightning talk about how [MiniTest](https://github.com/seattlerb/minitest) is implemented, with the goal of attracting new contributors to the project. In a talk like that, I'd definitely need to discuss a bit about how assertions work. A logical place to start might be the `Assertions#assert` method:\n\n```ruby\ndef assert test, msg = nil\n msg ||= \"Failed assertion, no message given.\"\n self._assertions += 1\n unless test then\n msg = msg.call if Proc === msg\n raise MiniTest::Assertion, msg\n end\n true\nend\n```\n\nThe implementation of `assert` is simple enough that I could probably show it as-is without losing my audience. But if I keep in mind that this code is going to be shown on a slide for just a few seconds, I might show the following simplified example instead:\n\n```ruby\ndef assert(test, msg=nil)\n msg ||= \"Failed assertion, no message given.\"\n\n raise(MiniTest::Assertion, msg) unless test\n true\nend\n```\n\nThis code omits some implementation details, but it preserves the main idea, which is that MiniTest's assertions work by raising an exception when a test fails. The fact that `assert` is where the number of assertions is counted is fairly obvious and only adds noise when you want to get a rough idea for how the code works at a glance. Likewise, the fact that a message can be passed in as a `Proc` object rather than a string is an interesting but obscure edge case that does not need to be emphasized. By removing these two statements from the method definition, the core behavior is easier to notice.\n\nThe process of creating a simplified example starts with looking at the original source code and then determining which details are essential to expressing the idea you want to express and which details can be considered background noise. The next step is to construct an example that serves as a functional subset of the original implementation when used within a certain context. You don't want to deviate too much from the original idea, but you can clean up the syntax a bit where appropriate to make the example easier to understand. From there, you can treat the example as a substitute for the real implementation for the purposes of demonstration; you just need to make sure to point out that you have simplified things a bit.\n\nWith this MiniTest example, the simplified version of the code is only slightly less complicated than the original, so the benefits of using this technique are a bit subdued. In practice, you're much more likely to run into situations in which there are dozens of lines of implementation code but only a handful of them are central to the idea that you are trying to express. In those situations, this pattern is especially effective at cutting through the cruft to get at the real meat of the problem you want to focus on. However, it's worth keeping in mind that even relatively small and easy-to-understand chunks of code can be simplified if they happen to include statements that are not directly relevant to the point you are trying to make.\n\n### Reduced examples\n\nA reduced example is one that reproduces a certain behavior in the most simple possible way. This technique is most commonly used for putting together bug reports and is one of the most important skills you can have as a software developer.\n\nIn the [Ruby Best Practices](http://rubybestpractices.com/) book, I told a story about a bug that was spotted in Prawn and how we reduced the original report to something much more simple in order to discover the root cause of the problem. Because this is still the best example I've found of this process in action, I'll summarize that story here rather than telling a new one.\n\nA developer sent us the following bug report to demonstrate that our code for generating fixed-width columns of text had a problem that was causing page breaks to be inserted unnecessarily:\n\n```ruby\nPrawn::Document.generate(\"span.pdf\") do\n span(350, :position => :center) do \n text \"Here's some centered text in a 350 point column. \" * 100\n end\n \n text \"Here's my sentence.\"\n\n bounding_box([50,300], :width => 400) do \n text \"Here's some default bounding box text. \" * 10 \n\n pos = bounds.absolute_left - margin_box.absolute_left\n span(bounds.width, :position => pos) do\n text \"The rain in Spain falls mainly on the plains. \" * 300 \n end\n end\n\n text \"Here's my second sentence.\"\nend\n```\n\nIn this example, he expected all of his text to be rendered on one page and was trying to show that each time he used the `span` construct, an unnecessary page break was created. He showed that this was the case both within the default page boundaries and within a manually specified bounding box. As far as user reports go, this example was pretty good, because it was specifically designed to show the problem he was having and was clearly not just some broken production code that he wanted help with.\n\nThat having been said, an understanding of how Prawn works under the hood made it possible to simplify this example quite a bit, even before investigating further. Because the default page boundaries in Prawn are implemented in terms of bounding boxes and the `bounding_box` method just temporarily swaps those dimensions with new ones, the second part of this report was superfluous. Removing it got the reproducible sample down to the example shown here:\n\n```ruby\nPrawn::Document.generate(\"span.pdf\") do\n span(350) do \n text \"Here's some text in a 350pt wide column. \" * 20\n end\n\n text \"This text should appear on the same page as the spanning text\"\nend\n```\n\nIn making this reduction, I also did some other minor cleanup chores such as reworking the text to be self-documenting and removing the `:position => :center` option for `span`, because it didn't affect the outcome. At this point, even someone without experience in how Prawn works would be able to more easily spot the problem in the example.\n\nAlthough `span` is not a trivial construct, it had only two possible points of failure: the `bounding_box` method and the `canvas` method. Because `bounding_box` is fundamental to pretty much everything Prawn does and we had plenty of evidence that it was working as expected, we turned our attention to `canvas`.\n\nThe purpose of `canvas` is to execute the contents of a block while ignoring all document margins and bounding boxes in place, essentially converting everything to absolute coordinates on the page. After the block is executed, it is supposed to keep the text pointer wherever it left off, which means that it should not trigger a pagebreak unless the text flows beyond the bottom of the page. To test this behavior, we coded up the following example:\n\n```ruby\nPrawn::Document.generate(\"canvas_sets_y_to_0.pdf\") do \n canvas { text \"Some text at the absolute top left of the page\" }\n text \"This text should not be after a pagebreak\" \nend\n```\n\nAfter running this example, we noticed that it exhibited the same defect that we saw in the user's bug report. Because this method is almost as deep down the Prawn call chain as you can go, it became clear that at this point we had our reduced example. The benefit of drilling down like this became apparent when we converted our sample code into a regression test:\n\n```ruby\nclass CanvasTest < Test::Unit::TestCase\n def setup \n @pdf = Prawn::Document.new\n end\n\n def test_canvas_should_not_reset_y_to_zero \n after_text_position = nil\n\n @pdf.canvas do \n @pdf.text \"Hello World\" \n after_text_position = @pdf.y\n end\n\n assert_equal after_text_position, @pdf.y \n end\nend\n```\n\nAfter seeing this test fail and applying a quick patch that got it to go green, we went back and ran the original bug report the user provided us with. As predicted, the bad behavior went away and things were once again working as expected.\n\nThe benefits of reducing the example before writing a regression test were tremendous. Not only was the test easier to write, but it also ended up capturing the problem at a much lower level than it would have if we immediately set in with codifying the bug report as a unit test. In addition to these benefits, the reduction process itself greatly simplified the debugging process, as it allowed us to proceed methodically to find the root of the problem.\n\nI've mostly used this reduction technique while debugging, but it can also be a useful way to find your way around a complex codebase. By starting with a practical example that exercises the system from the outermost layer, you can drill down into the code and trace your way through the call chains to find how some particular aspect of the software works. Although this is a less direct approach than just reading the documentation, it will give you a better fundamental understanding of how the system hangs together, and it's a fun way to practice code reading.\n\nWhether you are exploring a new codebase or tracking down a bug, the reduction process limits the scope of the things you need to think about, allowing you to dedicate your attention in a more focused way. This effect is somewhat similar to what we find when we make use of simplified examples but is more of a drilling down process than it is a pruning process. Both have their merits, and they can even be used in combination at times.\n\n### Sample applications \n\nAlthough all of the techniques I've discussed so far can be quite useful for studying, investigating, and teaching about specific issues, none of them are particularly suitable for demonstrating big-picture topics. When you want to emphasize how things come together, as opposed to how each individual part works, nothing beats the combination of a sample application with a good walkthrough tutorial. Several issues from Practicing Ruby Volume 2 made use of this format and were very well received by the readers here.\n\nIn [Learning new things step-by-step](http://practicingruby.com/articles/6), I built a small game for the purpose of demonstrating how to develop software in tiny bite-sized chunks. The nice thing about this approach is that it allows the reader to follow along at home (either mentally or by literally running the code themselves), while proceeding at their own pace. I hope it also encourages folks to experiment and draw their own conclusions rather than just rigidly following a predefined script.\n\nIn [Building Unix-style command line applications](http://practicingruby.com/articles/9), I tackled the creation of a Ruby clone of the Unix `cat` utility by focusing on distinct areas of functionality, one at a time. This is a slightly less linear format than the step-by-step approach of the game development article, but it allows readers to look at the complete application from several different angles, depending on the topics that interested them most.\n\nFinally, in [Designing business reporting applications](http://practicingruby.com/articles/13), I totally break away from linearity by presenting the source code of a full application in literate programming style. This approach allows readers to study the real implementation code and my commentary side by side and to bounce around as they see fit. This lack of explicit structure encourages readers to explore in a free-form fashion rather than focusing on some predefined areas of interests.\n\nAlthough these articles were some of the most successful ones that I've published here at Practicing Ruby, they were also among the most challenging to write. I had to apply a much higher standard of writing clear and concise code than I would if I were simply trying to make a project easy enough for me to maintain on my own. The prose was tricky to organize, because it's hard to decide which areas to emphasize and which to gloss over in a complete application. For these reasons, sample applications can be a cumbersome and time-consuming learning resource to produce. However, the investment seems to be well worth it in the end.\n\n### Reflections\n\nWriting good examples can be seriously hard work. This is why all too often we see people overusing contrived examples or simply attempting to pass off unrefined snippets of production code as learning materials. However, code examples in all of their myriad forms lay the foundation for how we communicate our ideas as software developers.\n\nAn important thing to remember when writing code examples is that the process is in many ways similar to writing prose. If we simply spit out a brain dump without thinking about how it will be interpreted and understood by others, we will end up with crappy results. But if we remember that the main goal of writing our examples is to communicate an idea to our fellow programmers, we naturally begin to ask the questions that lead us to improve our work.\n\nI hope that by sharing these few patterns with you, I've given you some useful ideas for how to improve your code communication skills. Following the patterns I've outlined here will lead you to writing better examples for your documentation, bug reports, unit tests, tutorials, and quite a few other things as well. \n\nThough the techniques I've shown here are ones that work well in a wide range of contexts, I am sure there are other approaches worth learning about. If you've seen a novel use of code examples in the wild, please let me know! I'd also be happy to hear any other thoughts you have on this topic, and I wouldn't mind helping a few folks come up with good examples for the projects they're working on. If you've got something you want me to take a look at, just leave a comment and I'll be sure to get back to you.\n"
},
{
"path": "articles/v3/003-turing-tarpit.md",
"content": "> NOTE: This article describes an interactive challenge that was done in\n> realtime at the time it was published. You can still make use of it\n> by starting from the initial version of the code, but if you'd\n> rather skip to the end results, be sure to read [Issue 3.4](https://practicingruby.com/articles/spiral-staircase-of-refactoring).\n\nA programming language that is endlessly flexible but difficult to use because\nof its lack of support for common operations is known as a [Turing\ntarpit](http://en.wikipedia.org/wiki/Turing_tarpit). While a range of esoteric\nprogramming languages fall under this category, the [Brainfuck\nlanguage](http://en.wikipedia.org/wiki/Brainfuck) is one that stands out for its\nextreme minimalism.\n\nBrainfuck manages to be Turing complete in only 8 operations, which means that\ndespite being functionally equivalent to Ruby in a theoretical sense, it offers\nvirtually none of the conveniences that a modern programmer has come to expect.\nYou don't need to go any farther than its \"Hello World\" program to see that it\nisn't a language that any sane person would want to work in:\n\n```ruby\n++++++++++[>+++++++>++++++++++>+++>+<<<<-]>++.>+.+++++++..+++.>++.\n<<+++++++++++++++.>.+++.------.--------.>+.>.\n```\n\nHowever, the simplicity of the language is not without at least some merit. A\nconsequence of having a somewhat trivial to parse syntax and very basic\nsemantics is that while Brainfuck may be one of the hardest languages to *use*,\nit simultaneously happens to be one of the easiest languages to *implement*.\nWhile it's not quite as easy a task as working on a code kata, it takes roughly\nthe same order of magnitude of effort to produce a fully functioning Brainfuck\ninterpreter. The key difference is that you end up with some software that is\nmuch deeper than your average bowling score calculator after you've completed\nthe exercise.\n\nA functioning Brainfuck interpreter is complex enough where you can begin to ask\nserious questions about code quality and overall design. It is also something\nthat you can build new functionality on top of in a ton of interesting ways. So\nin this exercise, the real payoff comes after you've run your first \"Hello\nWorld\" example and have a working interpreter to play with. The downside is that\nit might take you a day or two of work to get to that point, and after all that\neffort, you might need to think about doing something that's just a bit more\nmeaningful with your life. But like most things that involve this sort of\ndrudgery, being a Practicing Ruby subscriber can help you skip the boring stuff\nand get right to the juicy parts.\n\nRather than a typical article, this issue is instead an interactive challenge\nfor our readers to try at home. I've posted a [simple and functional Brainfuck\ninterpreter](https://github.com/elm-city-craftworks/turing_tarpit/tree/starting_point) on GitHub,\nand I'm inviting all of you to look for ways of improving it. The code itself is\ngood in places and not so good in others, and I've intentionally left in some\nthings that can be improved. Your challenge, if you choose to accept it, is as\nfollows:\n\n* If you spot things in the code that can be improved, let me know!\n* If you spot a bug, file a bug report via github issues and optionally send a pull request that includes a failing test.\n* If you have time to make a refactoring or improvement yourself, fork the project and submit pull requests\n* If you want to add documentation patches, those are welcome too!\n* Feel free to work on this with friends and colleagues, even if they aren't Practicing Ruby subscribers.\n\nIn [Issue 3.4](https://practicingruby.com/articles/spiral-staircase-of-refactoring), I will\ngo over the improvements we made to this project as a group, and discuss why\nthey're worthwhile. Because I know there are several things that need\nimprovement in this code that are pretty general in nature, I'm reasonably sure\nthat article will be a generally interesting discussion on software design and\nRuby idioms, as opposed to a collection of esoterica. But since I don't know\nwhat to expect from your contributions, the exact contents will be a surprise\neven to me.\n"
},
{
"path": "articles/v3/004-climbing-the-spiral-staircase-of-refactoring.md",
"content": "In [Issue 3.3](http://practicingruby.com/articles/shared/bwgflabwncjv), I presented a proof-of-concept Ruby implementation of the [Brainfuck programming language](http://en.wikipedia.org/wiki/Brainfuck) and challenged Practicing Ruby readers to improve upon it. After receiving several patches that helped move things along, I sat down once again to clean up the code even further. What I came to realize as I worked on my revisions is that the refactoring process is very similar to climbing a spiral staircase. Each structural change to the code simultaneously left the project back where it started along one vector while moving it forward along another.\n\nBecause we often look at the merits of a given refactoring technique within the context of a single transition from worse code to better code, it's easy to mistakenly assume that the refactoring process is much more linear than it actually is. In this article, I've tried to capture a much wider angle view of how refactoring really works in the wild. The end result is a story which I hope will spark some good discussions about how we can improve our code quality over time.\n\n### Prologue. Everything has to start somewhere\n\nI decided to name my interpreter [Turing Tarpit](http://en.wikipedia.org/wiki/Turing_tarpit), because that term is perfectly apt for describing languages like Brainfuck. In a nutshell, the term refers to any language which is infinitely flexible, yet nearly impossible to use for anything practical. It turns out that building this sort of mind trap for programmers is quite easy to do.\n\nMy first iteration was easy enough to build, and consisted of three objects: a `Tape`, an `Interpreter`, and a `Scanner`. The rough breakdown of responsibilities was something like this:\n\n* The [Tape object](https://github.com/elm-city-craftworks/turing_tarpit/blob/starting_point/lib/turing_tarpit.rb#L103-149) implemented something similar to the storage mechanism in a [Turing machine](http://en.wikipedia.org/wiki/Turing_machine#Informal_description). It provided mechanisms for accessing and modifying numeric values in cells, as well as a way to increment and decrement the pointer that determined which cell to operate on.\n\n* The [Interpreter object](https://github.com/elm-city-craftworks/turing_tarpit/blob/starting_point/lib/turing_tarpit.rb#L7-34) served as a mapping between Brainfuck's symbolic operators and the operations provided by the `Tape` object. It also implemented the I/O functionality required by Brainfuck.\n\n* The [Scanner object](https://github.com/elm-city-craftworks/turing_tarpit/blob/starting_point/lib/turing_tarpit.rb#L36-69) was responsible for taking a Brainfuck source file as input and transforming it into a stream of operations that could be handled by the `Interpreter` object. For the most part this simply meant reading the source file one character at a time, but this object also needed to account for Brainfuck's forward and backward jump operations.\n\nWhile my initial implementation was reasonably clean for a proof-of-concept, it definitely had room for improvement. I decided to ask for feedback early in the hopes that folks would find and fix the things I knew were problematic while simultaneously checking my blindspots for issues that I hadn't noticed myself.\n\n### Act I. Getting a fresh perspective on the problem\n\nSome of the issues brought up by contributors were fairly obvious housekeeping chores, but nonetheless made the project nicer to work with:\n\n* Steve Klabnik [requested a way to run the whole test suite at once](https://github.com/elm-city-craftworks/turing_tarpit/pull/3) instead of file by file. He had provided a patch with a Rakefile, but since the project didn't have any immediate need for other rake tasks, we ended up deciding that a simple _test/suite.rb_ file would be sufficient. Notes were added to the README on how to run the tests.\n\n* Renato Riccieri [broke the classes out into individual files](https://github.com/elm-city-craftworks/turing_tarpit/pull/6). The original implementation had everything in _lib/turing_tarpit.rb_, simply for convenience reasons while spiking. Breaking the classes into individual files brought the project more in line with [standard Ruby packaging conventions](http://chneukirchen.github.com/rps/).\n\n* Benoit Daloze [refactored some ugly output code](https://github.com/elm-city-craftworks/turing_tarpit/pull/2) to use `putc(char)` instead of `print(\"\" << char)`. Since the latter was obviously a hack due to my lack of awareness of the `putc` method, this was a welcome contribution.\n\nAfter this initial round of cleanup, we ended up thinking through a pair of more substantial problems: the inconsitent use of private accessors, and a proposed refactoring to break up the `Scanner` object into two separate objects, a `Tokenizer` and a `Scanner`.\n\n**The story behind my recent private accessor experiments**\n\nRyan LeCompte was the one to bring up [the question about private accessors](https://github.com/elm-city-craftworks/turing_tarpit/issues/1), and was curious about why I had used them in some places but referenced instance variables directly in others. The main reason for this was simply that the use of private accessors is a new experiment for me, and so in my haste of getting a first version out the door, I remembered to use them in some places but not in others.\n\nThis project in particular posed certain challenges for using private accessors conveniently. A specific example of where I ran into some weird edge cases can easily be seen in the `Tape` object:\n\n```ruby\nmodule TuringTarpit\n class Tape\n def initialize\n self.pointer_position = 0\n # ...\n end\n\n def increment_pointer\n self.pointer_position = pointer_position + 1\n end\n\n # ...\n\n private\n\n attr_writer :pointer_position\n end\nend\n```\n\nIf you just glance quickly at this class definition, it is very tempting to try to refactor `increment_pointer` so that it uses convenient `+=` syntax, resulting in something like the code below:\n\n```ruby\ndef increment_pointer\n self.pointer_position += 1\nend\n```\n\nIn most cases, this refactoring would be a good one because it makes the code slightly less verbose without sacrificing readability. However, it turns out that Ruby does not extend the same private method special casing to `self.foo += something` as it does to `self.foo = something`. This means that if you attempt to refactor this code to use `+=` it ends up raising a `NoMethodError`. Because this is definitely a downside of using private accessors, it's reasonable to ask why you'd bother to use them in the first place rather than using public accessors or simply referring to instance variables directly.\n\nThe best reason I can find for making use of accessors in general vs. instance variables is simply that the former are much more flexible. New behavior such as validations or transformations can be added later by changing what used to be vanilla accessors into ordinary method definitions. Additionally, if you accidentally introduce a typo into your code, you will get a `NoMethodError` right away rather than having to track down why your attribute is `nil` when you didn't expect it to be in some completely different place in your code.\n\nThe problem with making accessors public is that it hints to the consumer that it is meant to be touched and used, which is often not the case at all, especially for writers. While Ruby makes it trivial to circumvent privacy protections, a private method communicates to the user that it is meant to be treated as an implementation detail and should not be depended on. So the reason for using a private accessor is the same as the reason for using a private method: to mark the accessor as part of the internals of the object.\n\nThe interesting thing I stumbled across in this particular project is that if you take this technique to the extreme, it is possible to build entire applications without ever explicitly referencing an instance variable. It comes at the cost of the occasional weird edge case when calling private methods internally, but makes it possible to treat instance variables as a whole as a _language implementation detail_, rather than an _application implementation detail_. Faced with the opportunity to at least experiment with that idea, I decided to make the entire Turing Tarpit codebase completely free of instance variables, which ended up taking very little effort.\n\nThe jury is still out on whether or not this is a good idea, but I plan to keep trying the idea out in my projects and see whether I run into any more issues. If I don't experience problems, I'd say this technique is well worth it because it emphasizes message-passing rather than state manipulation in our objects. \n\n**Splitting up the Scanner object**\n\nAfter helping out with a few of the general housekeeping chores, Steve Klabnik then turned his attention to one of the weakest spots in the code, the `Scanner` object. He pointed out that having an object with dependencies on a whole lot of private methods is a bit of a code smell, and focused specifically on the `Scanner#next` method. The original implementation looked like this:\n\n```ruby\nmodule TuringTarpit\n class Scanner\n # ...\n\n def next(cell_value)\n validate_index\n\n element = @chars[@index]\n \n case element\n when \"[\"\n jump_forward if cell_value.zero?\n\n consume\n element = @chars[@index]\n when \"]\"\n if cell_value.zero?\n while element == \"]\"\n consume\n element = @chars[@index]\n validate_index\n end\n else\n jump_back\n consume\n element = @chars[@index]\n end\n end\n \n consume\n element\n end\n end\nend\n```\n\nSteve pointed out that the `Scanner#next` method was really doing more of a tokenizing operation, and that most of the scanning work was actually being done by the various private methods that were being used to traverse the underlying string. He prepared a patch which made this relationship explicit by introducing a `Tokenizer` object which would provide a method to replace `Scanner#next`. His newly introduced object allowed for a re-purposing of the `Scanner` object which allowed its methods to become public:\n\n```ruby\nmodule TuringTarpit\n class Tokenizer\n # ...\n\n def next(cell_value)\n scanner.validate_index\n\n element = scanner.current_char\n\n case element\n when \"[\"\n scanner.jump_forward if cell_value.zero?\n\n scanner.consume\n element = scanner.current_char\n when \"]\"\n if cell_value.zero?\n while element == \"]\"\n scanner.consume\n element = scanner.current_char\n scanner.validate_index\n end\n else\n scanner.jump_back\n scanner.consume\n element = scanner.current_char\n end\n end\n\n scanner.consume\n element\n end\n end\nend\n```\n\nThe thing in particular I liked about this patch is that it abstracted away some of the tedious index operations that were originally present in `Scanner#next`. As much as possible I prefer to isolate anything that can cause off-by-one errors or other such nonsense, and this refactoring did a good job of addressing that issue.\n\nThe interesting thing about this refactoring is that while I intended to work on the same area of the code if no one else patched it, I had planned to approach it in a very different way. My original idea was to implement some sort of generic stream datastructure and reuse it in both `Scanner` and `Tape`. However, seeing that Steve's patch at least partly addressed my concerns while possibly opening some new avenues as well, I abandoned that idea and merged his work instead.\n\n### Act II. Building a better horse\n\nAfter applying the various patches from the folks who participated in this challenge, the code was in a much better place than where it started. However, much work was still left to be done!\n\nIn particular, the code responsible for turning Brainfuck syntax into a stream of operations still needed a lot of work. The `Tokenizer` class that Steve introduced was an improvement, but without further revisions would simply serve as a layer of indirection rather than as an abstraction. Zed Shaw describes the difference between these two concepts very eloquently in his essay [Indirection Is Not Abstraction](http://zedshaw.com/essays/indirection_is_not_abstraction.html) by stating that _\"Abstraction is used to reduce complexity. Indirection is used to reduce coupling or dependence.\"_\n\nAs far as the `Tokenizer` object goes, Steve's patch reducing coupling somewhat by pushing some of the implementation details down into the `Scanner` object. However, the procedure is pretty much identical with the exception of the lack of explicit indexing code, and so the baseline complexity actually increases because what was once done by one object is now split across two objects.\n\nTo address this problem, the dividing lines between the two objects needed to be leveraged so that they could interact with each other at a higher level. It took me a while to think through the problem, but in doing so I realized that I could now push more functionality down into the `Scanner` object so that `Tokenizer#next` ended up with fewer moving parts. After some major gutting and re-arranging, I ended up with a method that looked like this:\n\n```ruby\nmodule TuringTarpit\n class Tokenizer\n # ...\n\n def next(cell_value)\n case scanner.next_char\n when Scanner::FORWARD_JUMP\n if cell_value.zero?\n scanner.jump_forward\n else\n scanner.next_char\n end\n when Scanner::BACKWARD_JUMP\n if cell_value.zero?\n scanner.skip_while(Scanner::BACKWARD_JUMP)\n else\n scanner.jump_back\n end\n end\n\n scanner.current_char\n end\n end\nend\n```\n\nAfter this refactoring, the `Tokenizer#next` method was a good deal more abstract in a number of ways:\n\n* It expected the `Scanner` to handle validations itself rather than telling it when to check the index \n\n* It no longer referenced Brainfuck syntax and instead used constants provided by the `Scanner`\n\n* It eliminated a lot of cumbersome assignments by reworking its algorithm so that `Scanner#current_char` always referenced the right character at the end of the scanning routine.\n\n* It expected the `Scanner` to remain internally consistent, rather than handling edge cases itself.\n\nThese reductions in complexity made a hugely positive impact on the readability and understandability of the `Tokenizer#next` method. While all of these changes could have technically been made before the split between the `Scanner` and `Tokenizer` happened, cutting the knot into two pieces certainly made untangling things easier. This is why indirection and abstraction often go hand in hand, despite the fact that they are very different concepts from one another.\n\n### Act III. Mountains are once again merely mountains\n\nAfter building on top of Steve's work to simplify the syntax-processing code even further, I finally felt like that part of the project was in decent shape. I then decided to turn my attention back to the `Interpreter` object, since it had not received any love from the challenge participants. The original code for it looked something like this:\n\n```ruby\nmodule TuringTarpit\n class Interpreter\n def run\n loop do\n case tokenizer.next(tape.cell_value)\n when \"+\"\n tape.increment_cell_value\n when \"-\"\n tape.decrement_cell_value\n when \">\"\n tape.increment_pointer\n when \"<\"\n tape.decrement_pointer\n when \".\"\n putc(tape.cell_value)\n when \",\"\n value = STDIN.getch.bytes.first\n next if value.zero?\n\n tape.cell_value = value\n end\n end\n end\n end\nend\n```\n\nWhile this implementation wasn't too bad, there were two things I didn't like about it. The first issue was that it directly referenced Brainfuck syntax, which sort of defeated the purpose of having the tokenizer be syntax independent. The second problem was that I found the case statement to feel a bit brittle and limiting. What I really wanted was a dynamic dispatcher similar to the following method:\n\n```ruby\ndef run\n loop do\n if operation = tokenizer.next(evaluator.cell_value)\n tape.send(operation)\n end\n end\nend\n```\n\nIn order to introduce this kind of functionality, I'd need to find a place to introduce a simple mapping from Brainfuck syntax to operation names. I already had the keys and values in mind, I just needed to find a place to put them:\n\n```ruby\nOPERATIONS = { \"+\" => :increment_cell_value,\n \"-\" => :decrement_cell_value,\n \">\" => :increment_pointer,\n \"<\" => :decrement_pointer,\n \".\" => :output_cell_value,\n \",\" => :input_cell_value }\n```\n\nFiguring out how to make this work was surprisingly challenging. I found that the extra layers of indirection between the `Tape` and the `Scanner` meant that any change made too far down the chain would need to be echoed all the way up it, and that changes made towards the top felt tacked on and out of place. This eventually led me to question what the separation between `Scanner` and `Tokenizer` was really gaining me, as well as the separation between `Interpreter` and `Tape`.\n\nAfter a fair amount of ruminating, I decided to take my four objects and join them together at the seams so that only two remained. The `Scanner` and `Tokenizer` ended up getting joined back together to form a new `Interpreter` class. The job of the `Interpreter` is to take Brainfuck syntax and turn it into a stream of operations. You can get a rough idea of how it all came together by checking out the following code:\n\n```ruby\nmodule TuringTarpit\n class Interpreter\n FORWARD_JUMP = \"[\"\n BACKWARD_JUMP = \"]\"\n\n OPERATIONS = { \"+\" => :increment_cell_value,\n \"-\" => :decrement_cell_value,\n \">\" => :increment_pointer,\n \"<\" => :decrement_pointer,\n \".\" => :output_cell_value,\n \",\" => :input_cell_value }\n\n def next_operation(cell_value)\n case next_char\n when FORWARD_JUMP\n if cell_value.zero?\n jump_forward\n else\n skip_while(FORWARD_JUMP)\n end\n when BACKWARD_JUMP\n if cell_value.zero?\n skip_while(BACKWARD_JUMP)\n else\n jump_back\n end\n end\n\n OPERATIONS[current_char]\n end\n\n # ... lots of private methods are back, but now fine-tuned.\n end\nend\n```\n\nThe old `Interpreter` object and `Tape` object were also merged together, forming a single object I ended up calling `Evaluator`. The job of the `Evaluator` object is to take a stream of operations provided by the newly defined `Interpreter` object and then execute them against a Turing Machine like data structure. In essence, the `Evaluator` object is nothing more than the original `Tape` object I implemented along with a few extra methods which account for the things the original `Interpreter` object was meant to do:\n\n```ruby\nmodule TuringTarpit\n class Evaluator \n def self.run(interpreter)\n evaluator = new\n\n loop do\n if operation = interpreter.next_operation(evaluator.cell_value)\n evaluator.send(operation)\n end\n end\n end\n\n def output_cell_value\n putc(cell_value)\n end\n\n def input_cell_value\n value = $stdin.getch.ord\n return if value.zero?\n\n self.cell_value = value\n end\n\n # other methods same as original Tape methods\n end\nend\n```\n\nI had mixed feelings about recombining these objects, because to some extent it felt like a step backwards to me. In particular, I think this refactoring resulted in some minor violations of the [Single Responsibility Principle](http://en.wikipedia.org/wiki/Single_responsibility_principle), and increased the overall coupling of the system somewhat. However, the independence of the four different objects the system previously consisted of seemed artificial at best. To the extent that they could be changed easily or swapped out for one another, I could not think of a single practical reason why I'd actually want that kind of flexibility. In this particular situation it turned out that recombining the objects greatly reduced their communications overhead, and so was worth the loss in generality.\n\n### Epilogue. Sending the ship out to sea\n\nI was really tempted to keep noodling on the design of this project, because even in my final version of the code I still felt that I could have done better. But at a certain point I decided that I could end up getting caught in this trap forever, and the only way to free myself from it was to wrap up my work and just ship the damn thing. This ultimately meant that I had to take care of several chores that neither I nor the various participants in this challenge bothered to work on earlier:\n\n* I added a [set of integration tests](https://github.com/elm-city-craftworks/turing_tarpit/blob/act3/test/integration/evaluator_test.rb) which ran the `Evaluator` against a couple sample Brainfuck programs to make sure we had some decent end-to-end testing support. Found a couple bugs that way.\n\n* I set up and ran [simplecov](https://github.com/colszowka/simplecov) to check whether my tests were at least *running* all the implementation code, and ended up spotting a faulty test which wasn't actually getting run.\n\n* I added a [bin/turing_tarpit](https://github.com/elm-city-craftworks/turing_tarpit/blob/act3/bin/turing_tarpit) file so that you can execute Brainfuck programs without building a Ruby shim first. \n\n* Did the usual gemspec + Gemfile dance and pushed a 1.0.0 gem to rubygems.org. Typically I'd call a project in its early stages a 0.1.0 release, but I honestly don't see myself working on this much more so I might as well call it 'production ready'.\n\nAfter I wrapped up all these chores, I decided to go back and check out what my [flog](https://github.com/seattlerb/flog) complexity scores were for each stage in this process. It turns out that the final version was the least complex, with the lowest overall score, lowest average score, and lowest top-score by method. The original implementation came in second place, and the other two iterations were in a distant third and fourth place. While that gave me some reassurances, it doesn't mean much except for that Flog seems to really hate external method calls.\n\n### Reflections\n\nThis has been one of my favorite articles to write for Practicing Ruby so far. It forced me to look at the refactoring process in a much more introspective way than I have typically done in the past, and gave me a chance to interact with some of our awesome readers. I do think it ended up raising more questions and challenges in my mind than it did give me answers and reassurances, but I suppose that's a sign that learning happened.\n\nWhile I found it very hard to summarize the refactoring lifecycle for this project, my hope is that I've at least given you a glimpse of the spiral staircase metaphor I chose to name this article after. If it didn't end up making you feel too dizzy, I'd love to hear your thoughts about this exercise as well as what your own process is like when it comes to refactoring code.\n"
},
{
"path": "articles/v3/005-framework-design-and-implementation.md",
"content": "Ruby is a great language for building application frameworks, particularly\nmicro-frameworks. The sad thing is that by the time most frameworks become\npopular, they end up quite complicated. To discover the joy of building reusable\nscaffolding for others, it's necessary to take a look at where the need for that\nscaffolding comes from in the first place.\n\nIn the January 2012 core skills session at Mendicant University, I asked our\nstudents to each build multi-user email based applications. While the students\nwere working on very different projects, there was a ton of boilerplate that was\ncommon between them all. Because it was too painful to watch the same bits of\ncode get written again and again in slightly different ways, I decided to build\na tiny framework to solve this problem.\n\nIn this issue of Practicing Ruby and the one that follows it, I'm going to have\nyou work through the code I wrote and help me figure out what goes into building\na good application framework. The goal for this issue is to generate ideas and\nquestions about the codebase. All of what we learn from this exercise will be\nneatly packaged up and synthesized in time for Issue 3.6, but for now I'm\nlooking for folks to get their hands dirty.\n\n## The Challenge\n\nI would like you to spend at least the same amount of time you'd ordinarily\nspend reading a Practicing Ruby article actively reading and working through\n[Newman 0.1.1](https://github.com/mendicant-original/newman/tree/v0.1.1), my\nmicro-framework for email based applications. I have intentionally left the\nsource uncommented for two reasons: to get you to practice your code reading\nskills and to get your candid feedback on the strengths and weaknesses of my\noverall design without influencing you too much.\n\nAs you read the code, don't just passively click through files on github!\nInstead, pull down the source and play with it: Run the examples if you can, or\neven better, build your own examples. Try to break stuff if you think you might\nbe able to find a bug or two, or try to add a new feature you find interesting.\nThis is an open sandbox to play in!\n\nOnce you have managed to find your way around, you're encouraged to start\nactively collaborating. I'll be available via the #newman IRC channel and\n[newman@librelist.org](newman@librelist.org) to listen to any ideas or questions\nyou have. Of course, feel free to use Github for bug reports, feature requests,\nand comments on pull requests / commits.\n\nIn this code I've tried to apply pretty much everything I've ever taught via\nPracticing Ruby whenever there was an opportunity to do so. I've also broken\naway from established Ruby conventions in places to explore new ideas. Reading\nit will be worth your time, and if you actively involve yourself in the\nconversations around it, you'll be sure to level up your Ruby skills in no time. \n\n**One last thing: Don't be afraid to ask where to get started if you feel stuck.\nThe purpose of this exercise is to learn, and I will do what I can to help you\nget a lot out of this challenge.**\n"
},
{
"path": "articles/v3/006-framework-design-and-implementation.md",
"content": "In [Issue 3.5](http://practicingruby.com/articles/22), I challenged Practicing Ruby subscribers to read through and play with the uncommented source code of [Newman 0.1.1](https://github.com/mendicant-original/newman/tree/v0.1.1), the first release of my micro-framework for building email-centric applications. My hope was that by looking through the implementation of a framework in its very early stages of development, readers would be able to familiarize themselves with the kinds of challenges involved in building this sort of project.\n\nIf you didn't participate in that challenge, I recommend spending an hour or two working through it now before reading the rest of this article. My feeling was (and is) that because framework development is about taking care of a thousand tiny details, it's important to see where this kind of project begins before you can really appreciate where it ends up. Assuming you've gone ahead and done that, we can move on to this week's exercise.\n\n### The challenge revisited\n\nI had originally planned to provide a nice annotated walk-through of Newman's implementation up front, but then decided it'd be better if you had a chance to explore it without much guidance before sharing my own explanations of how it all hangs together.\n\nHowever, this would be little more than an exercise in code reading if I didn't revisit that challenge and provide you with comprehensive implementation notes. With that in mind, you can now read [the fully documented source code](http://mendicant-original.github.com/newman/lib/newman.html), complete with little bits of design ruminations peppered throughout the code. This ought to answer some of the questions that were rattling around in the back of your mind, and even if it didn't, it may spark new questions or ideas that you can share with me as they arise.\n\nJust reading through the documented codebase should teach you a lot about how a micro-framework can be built in Ruby. But the process of doing so might still leave you feeling a bit disoriented because it provides a view of the big picture in terms of dozens of microscopic snapshots rather than exposing a handful of bright-line items to focus on. With that in mind, I've decided to outline a few of the recurring obstacles I kept running into even in the first week of development on this project, as I think they'll be friction points for frameworks of all varieties.\n\n### Lessons learned the hard way\n\nThere is a ton of content to read in that source walk-through, so I'll try to keep these points brief in the hopes that they'll spark some discussions that might possibly lead to further investigation in future articles. But these are the main things to watch out for if you're designing your own framework or contributing to someone else's early stage project:\n\n**1) Dealing with global state sucks**\n\nIn the context of Newman, I had to deal with global state in the form of configuration settings, logging, mailer objects, and persistence layers. The first version of Newman had to resort to turning many of its objects into singleton objects because any object which manipulates global state can have global side effects. \n\nThe viral nature of singleton objects was something that I rarely encountered in application or even library code, but became blindingly apparent in working on this framework. For example, Newman's first version shipped with a `Newman::Mailer` object, but because this object was using Mail's global settings functionality, it was not practical to ever create more than one `Newman::Mailer` object. This was really annoying, because it meant that Newman would be limited to monitoring a single inbox per process, which seems like an artificial restriction. But because a `Newman::Server` object is essentially a router designed to work bridge our mailer object to our application objects, it too needed to become a singleton object!\n\nWe eventually were able to work around this for the most part by using some low level APIs provided by the mail gem, and this will pave the way for multi-inbox support in Newman in the near future. But I was sort of shocked at how much impact depending on a singleton object can have on the overall flexibility of a framework, because I had not experienced this problem in my usual work on applications and libraries.\n\nFor the things that'd be ordinarily implemented as singleton objects, such as loggers or configuration objects, I took care to make it so that even if in practice the system makes use of globally available state, the structure allows for that to be changed with minimal impact. As an example of this, you can see that Newman actually passes its setting objects, data storage objects, and loggers down the call chain to any object that needs them rather than having those objects reference a constant or global variable. This makes it possible for isolation to occur at any point in the chain, and makes it so that Newman has very few hard dependencies on shared state, with its use only being a matter of convenience. The unfortunate side effect of this sort of design is a bit of repetitive code, but I've tried to minimize that where possible by providing convenience constructors and factory methods that make this job easier.\n\n**2) Handling application errors in server software is hard**\n\nIn the first version of Newman, any application error would cause the whole server to come crashing down with it. This is definitely not the right way to do things, at least not by default, as it means that a problem with a single callback in a single application can bring down a whole process that is otherwise working as expected.\n\nYou might think that the solution to this is to simply rescue application errors and log them, and that is more-or-less the approach I chose to solving this problem. However, I quickly ran into an issue with this in testing. I didn't want a bunch of verbose log output while running my integration tests, so I ran the server with logging turned off. But soon enough, I was getting tests which were failing but giving me no feedback at all as to why that was happening. I eventually discovered that this was due to application errors being swallowed silently, causing the application to fail to respond but not give any feedback that would help with debugging them. The code was not raising an exception, so the tests were not halting with an error, they were just failing.\n\nTo solve this issue, I added a server configuration object which allowed toggling exception raising on and off. When that setting was enabled, the server would halt, which is exactly the behavior I wanted in my tests. This did the trick and it's been mostly smooth sailing since then. But the question remains: what are the best defaults to use for this sort of thing? I'm thinking that for test mode, logging should be off and exception raising should be on, and for the default runtime behavior, it should be exactly the opposite. Is that sane? I don't really know, but I suppose it's worth a try.\n\nAnother open question I have here is how much effort should be put into only rescuing certain kinds of errors. In most cases, \"log and move on\" seems like the right behavior from the server's perspective, but could this lead to weird edge cases? As the framework designer, it's my job to help make it hard for the application developer to shoot himself in the foot. But unfortunately right now I don't really know where to aim. More research is required here.\n\n**3) Frameworks are tricky to test**\n\nThe whole point of a framework is to tie together a bunch of loose odds and ends to produce a cohesive environment for application development. Writing tests for applications developed within a framework should be *easier* than writing tests for applications developed standalone, but writing tests for the framework itself present new and interesting challenges that aren't typically encountered in ordinary application development.\n\nWhen I first started working on Newman, I wasn't writing any tests at all\nbecause I had no faith that any of the objects I was creating were going to\nsurvive first contact with real use cases. Instead, I focused on building little\nexample applications which moved the functionality along and served as a way of\nmanual testing the code in a fairly predictable way. But after even a modest\namount of functionality was built, the lack of automated testing made it so that\neach new change to the system involved a long, cumbersome, and error prone\nmanual testing session, to the point where it was no longer practical.\n\nFrom there, I decided to build some simple \"live tests\" that would essentially\nscript away the manual checking I was doing, running the example programs\nautomatically, and automating the sending and checking of email to give me a\nsimple red/green check. The process of introducing these changes required me to\nmake some changes to the system, such as allowing the system to run tick by tick\nrather than in a busy-wait loop, among other things. This cut down some of the\nmanual testing time and made the test plan more standardized, but was still very\nbrittle because cleaning up after a failed test was still a manual process.\n\nSooner or later, we introduced some surface-level test doubles, such as a simple\nTestMailer which could serve as a stand-in replacement for the real mail object.\nWith this object in place it was possible to convert some of the live tests to a\nset of acid tests which were capable of testing the system from end to end\nreaching all systems except for the actual mail interactions. This was a huge\nimprovement because it made basically the same tests run in fractions of a\nsecond rather than half a minute, but I'm still glad it's not where we started\nat. Why? Simply because email is a super messy domain to work in and five\nminutes of manual testing will expose many problems that hours of work on finely\ncrafted automated tests would never catch unless you happen to be an email\nexpert (I'm not). The only thing I regret is that I should have developed these\ntests concurrently with my manual testing, rather than waiting until the pain\nbecame so great that the project just ground to a halt.\n\nEven after these improvements, Newman 0.2.0 still ended up shipping with no unit\ntests. Part of this is because of the lack of available time I had to write\nthem, but the other part is that it still feels like a challenge for me to\nmeaningfully test isolated portions of the framework since they literally will\nnever be useful in isolation. I'm stuck in between a rock and a hard place,\nbecause the use of mock objects feels too artificial, but dragging in the real\ndependencies is an exercise in tedium. I'd love some guidance on how to test\nthis sort of code effectively and will be looking into how things like Sinatra\nand Rails do their testing to see if I can learn anything there.\n\nBut one thing is for sure, I wouldn't suggest trying to build a framework unit\ntest by unit test. The problem domain is way too messy with way too many\ninterlocking parts for that to be practical. I think if I took the TDD approach\nI'd still be working on getting the most basic interactions working in Newman\ntoday rather than talking about it's second major release. Still, maybe I'm just\ndoing it wrong?\n\n**4) Defining a proper workflow takes more effort than it seems**\n\nThis obstacle is hard to describe succinctly, so I won't try to do so. But the\nmain point I want to stress is that many frameworks are essentially nothing more\nthan a glorified tool for ferrying a bunch of request data around and building\nup a response in the end. But deciding where to sneak in extension points, and\nwhere to give the application developer control vs. where to make a decision for\nthem is very challenging.\n\nTry to start with one of Newman's examples and trace the path from the point\nwhere an email is received to the point where a response gets sent out. Then let\nme know what you think of what I've done, and perhaps give me some ideas for how\nI can do it better. I'm still not happy with the decomposition as it is, but I'm\nstruggling to figure out how to fix it.\n\n**5) Modularity is great, but comes at a cost**\n\nAs soon as you decide to make things modular, you have to be very careful about\nbaking assumptions into your system. This means making interfaces between\nobjects as simple as possible, limiting the amount of dependencies on shared\nstate, and providing generic adapter objects to wrap specific implementation\ndetails in some contexts. This is another thing that's hard to express in a\nconcise way, but the point is that modularity is a lot more complicated when you\nare not just concerned about reusability/replaceability within a single\napplication, but instead within an entire class of applications, all with\nsomewhat different needs.\n\nI've been trying to ask myself the question of whether a given bit of\nfunctionality really will ever be customized by a third-party extension, and if\nI think it will be, I've been trying to imagine a specific use case. If I can't\nfind one, I decide to avoid generalizing my constructs. However, this is a\ndangerous game and finding the right balance between making a system highly\ncustomizable and making it cohesive is a real challenge. It's where all the fun\nproblems come from with framework design, but is also the source of a lot of\nheadaches.\n\n### Reflections\n\nI'm afraid dear Practicing Rubyist that once again I've raised more questions\nthan answers. I always worry when I find myself over my own head that perhaps\nI've lost some of you in the process. But I want to emphasize the fact that this\njournal is meant to chronicle a collective learning process for all of us, not\njust a laundry list of developer protips that I can pull off the top of my head.\nEven if this series of articles was hard to digest, it will pave the way for\nmore neatly synthesized works in the future.\n\nAlso, don't underestimate your ability to contribute something to this\nconversation! If any ideas or questions popped up in your head while reading\nthrough these notes, please share them without thinking about whether or not\nyour insights would be worth sharing. I've been continuously impressed by the\nquality of the feedback around here, so I'd love to hear what you think.\n"
},
{
"path": "articles/v3/007-criteria-for-disciplined-inheritance.md",
"content": "Inheritance is a key concept in most object-oriented languages, but applying it\nskillfully can be challenging in practice. Back in 1989, [M.\nSakkinen](http://users.jyu.fi/~sakkinen/) wrote a paper called [Disciplined\ninheritance](http://scholar.google.com/scholar?cluster=5893037045851782349&hl=en&as_sdt=0,7&sciodt=0,7)\nthat addresses these problems and offers some useful criteria for working around\nthem. Despite being over two decades old, this paper is extremely relevant to\nthe modern Ruby programmer.\n\nSakkinen's central point seems to be that most traditional uses of inheritance lead to poor encapsulation, bloated object contracts, and accidental namespace collisions. He provides two patterns for disciplined inheritance and suggests that by normalizing the way that we model things, we can apply these two patterns to a very wide range of scenarios. He goes on to show that code that conforms to these design rules can easily be modeled as ordinary object composition, exposing a solid alternative to traditional class-based inheritance.\n\nThese topics are exactly what this two-part article will cover, but before we can address them, we should establish what qualifies as inheritance in Ruby. The general term is somewhat overloaded, so a bit of definition up front will help start us off on the right foot. \n\n### Flavors of Ruby inheritance\n\nAlthough classical inheritance is centered on the concept of class-based hierarchies, modern object-oriented programming languages provide many different mechanisms for code sharing. Ruby is no exception: it provides four common ways to model inheritance-based relationships between objects.\n\n* Classes provide a single-inheritance model similar to what is found in many other object-oriented languages, albeit lacking a few privacy features.\n\n* Modules provide a mechanism for modeling multiple inheritance, which is easier to reason about than C++ style class inheritance but is more powerful than Java's interfaces.\n\n* Transparent delegation techniques make it possible for a child object to dynamically forward messages to a parent object. This technique has similar effects as class-/module-based modeling on the child object's contract but preserves encapsulation between the objects.\n\n* Simple aggregation techniques make it possible to compose objects for the purpose of code sharing. This technique is most useful when the subobject is not meant to be a drop-in replacement for the superobject.\n\nAlthough most problems can be modeled using any one of these techniques, they each have their own strengths and weaknesses. Throughout both parts of this article, I'll point out the trade-offs between them whenever it makes sense to do so.\n\n### Modeling incidental inheritance \n\nSakkinen describes **incidental inheritance** as the use of an inheritance-based modeling approach to share implementation details between dissimiliar objects. That is to say that child (consumer) objects do not have an _is-a_ relationship to their parents (dependencies) and therefore do not need to provide a superset of their parent's functionality.\n\nIn theory, incidental inheritance is easy to implement in a disciplined way because it does not impose complex constraints on the relationships between objects within a system. As long as the child object is capable of working without errors for the behaviors it is meant to provide, it does not need to take special care to adhere to the [Liskov Substitution Principle](http://blog.rubybestpractices.com/posts/gregory/055-issue-23-solid-design.html). In fact, the child needs only to expose and interact with the bits of functionality from the parent object that are specifically relevant to its domain.\n\nRegardless of the model of inheritance used, Sakkinen's paper suggests that child objects should rely only on functionality provided by immediate ancestors. This is essentially an inheritance-oriented parallel to the [Law of Demeter](http://en.wikipedia.org/wiki/Law_of_Demeter) and sounds like good advice to follow whenever it is practical to do so. However, this constraint would be challenging to enforce at the language level in Ruby and may not be feasible to adhere to in every imaginable scenario. In practice, the lack of adequate privacy controls in Ruby make traditional class hierarchies or module mixins quite messy for incidental inheritance, which complicates things a bit. But before we discuss that problem any further, we should establish what incidental inheritance looks like from several different angles in Ruby.\n\nIn the following set of examples, I construct a simple `Report` object that computes the sum and average of numbers listed in a text file. I break this problem into three distinct parts: a component that provides functionality similar to Ruby's `Enumerable` module, a component that uses those features to do simple calculations on numerical data, and a component that outputs the final report. The contrived nature of this scenario should make it easier to examine the structural differences between Ruby's various ways of implementing inheritance relationships, but be sure to keep some more realistic scenarios in the back of your mind as you work through these examples. \n\nThe classical approach of using a class hierarchy for code sharing is worth looking at, even if most practicing Rubyists would quickly identify this as the wrong approach to this particular problem. It serves as a good baseline for identifying the problems introduced by inheritance and how to overcome them. As you read through the following code, think of its strengths and weaknesses, as well as any alternative ways to model this scenario that you can come up with.\n\n```ruby\nclass EnumerableCollection\n def count\n c = 0\n each { |e| c += 1 }\n c\n end\n\n # Samnang's implementation from Issue 2.4\n def reduce(arg=nil) \n return reduce {|s, e| s.send(arg, e)} if arg.is_a?(Symbol)\n\n result = arg\n each { |e| result = result ? yield(result, e) : e }\n\n result\n end\nend\n\nclass StatisticalCollection < EnumerableCollection\n def sum\n reduce(:+) \n end\n\n def average\n sum / count.to_f\n end \nend\n\nclass StatisticalReport < StatisticalCollection\n def initialize(filename)\n self.input = filename\n end\n\n def to_s\n \"The sum is #{sum}, and the average is #{average}\"\n end\n\n private \n\n attr_accessor :input\n\n def each\n File.foreach(input) { |e| yield(e.chomp.to_i) }\n end\nend\n\nputs StatisticalReport.new(\"numbers.txt\")\n```\n\nThrough its inheritance-based relationships, `StatisticalReport` is able to act as a simple presenter object while relying on other reusable components to crunch the numbers for it. The `EnumerableCollection` and `StatisticalCollection` objects do most of the heavy lifting while managing to remain useful for a wide range of different applications. The division of responsibilities between these components is reasonably well defined, and if you ignore the underlying mechanics of the style of inheritance being used here, this example is a good demonstration of effective code reuse.\n\nUnfortunately, the devil is in the details. When viewed from a different angle, it's easy to see a wide range of problems that exist even in this very simple application of class-based inheritance:\n\n1. `EnumerableCollection` and `StatisticalCollection` can be instantiated, but\nit is not possible to do anything meaningful with them as they are currently\nwritten. Although it's not always a bad idea to make use of abstract\nclasses, valid uses of that pattern typically invert the relationship shown\nhere, with the child object filling in a missing piece so that its parent can do\na complex job.\n\n2. Although `StatisticalReport` relies on only two relatively generic methods from `StatisticalCollection` and `StatisticalCollection` similarly relies on only two methods from `EnumerableCollection`, the use of class inheritance forces a rigid hierarchical relationship between the objects. Even if it's not especially awkward to say a `StatisticalCollection` is an `EnumerableCollection`, it's definitely weird to say that a `StatisticalReport` is also an `EnumerableCollection`. What makes matters worse is that this sort of modeling prevents `StatisticalReport` from inheriting from something more related to its domain such as a `HtmlReport` or something similar. As my [favorite OOP rant](http://lists.canonical.org/pipermail/kragen-tol/2011-August/000937.html) proclaims, class hierarchies do not exist simply to satisfy our inner Linnaeus.\n\n3. There is no encapsulation whatsoever between the components in this system. The purely functional nature of both `EnumerableCollection` and `Statistics` make this less of a practical concern in this particular example but is a dangerous characteristic of all code that uses class-based inheritance in Ruby. Any instance variables created within a `StatisticalReport` object will be directly accessible in method calls all the way up its ancestor chain, and the same goes for any methods that `StatisticalReport` defines. Although a bit of discipline can help prevent this from becoming a problem in most simple uses of class inheritance, deep method resolution paths can make accidental collisions of method definitions or instance variable names a serious risk. Such a risk might be mitigated by the introduction of class-specific privacy controls, but they do not exist in Ruby. \n\n4. As a consequence of points 2 and 3, the `StatisticalReport` object ends up with a bloated contract that isn't representative of its domain model. It'd be awkward to call `StatisticalReport#count` or `StatisticalReport#reduce`, but if those inherited methods are not explicitly marked as private in the `StatisticalReport` definition, they will still be callable by clients of the `StatisticalReport` object. Once again, the stateless nature of this program makes the effects less damning in this particular example, but it doesn't take much effort to imagine the inconsistencies that could arise due to this problem. In addition to real risks of unintended side effects, this kind of modeling makes it harder to document the interface of the `StatisticalReport` in a natural way and diminishes the usefulness of Ruby's reflective capabilities.\n\nAt least some of these issues can be resolved through the use of Ruby's module-based mixin functionality. The following example shows how our class-based code can be trivially refactored to use modules instead. Once again, as you read through the code, think of its strengths and weaknesses as well as how you might approach the problem differently if it were up to you to design this system.\n\n```ruby\nmodule SimplifiedEnumerable\n def count\n c = 0\n each { |e| c += 1 }\n c\n end\n\n # Samnang's implementation from Issue 2.4\n def reduce(arg=nil) \n return reduce {|s, e| s.send(arg, e)} if arg.is_a?(Symbol)\n\n result = arg\n each { |e| result = result ? yield(result, e) : e }\n\n result\n end\nend\n\nmodule Statistics\n def sum\n reduce(:+) \n end\n\n def average\n sum / count.to_f\n end \nend\n\nclass StatisticalReport\n include SimplifiedEnumerable\n include Statistics\n\n def initialize(filename)\n self.input = filename\n end\n\n def to_s\n \"The sum is #{sum}, and the average is #{average}\"\n end\n\n private \n\n attr_accessor :input\n\n def each\n File.foreach(input) { |e| yield(e.chomp.to_i) }\n end\nend\n\nputs StatisticalReport.new(\"numbers.txt\")\n```\n\nUsing module mixins does not improve the encapsulation of the components in the system or solve the problem of `StatisticalReport` inheriting methods that aren't directly related to its problem domain, but it does alleviate some of the other problems that Ruby's class-based inheritance causes. In particular, it makes it no longer possible to create instances of objects that wouldn't be useful to use as standalone objects and also loosens the dependencies between the components in the system.\n\nAlthough the `Statistics` and `SimplifiedEnumerable` modules are still not capable of doing anything useful without being tied to some other object, the relationship between them is much looser. When the two are mixed into the `StatisticalReport` object, an implicit relationship between `Statistics` and `SimplifiedEnumerable` exists due to the calls to `reduce` and `count` from within the `Statistics` module, but this relationship is an implementation detail rather than a structural constraint. To see the difference yourself, think about how easy it would be to switch `StatisticalReport` to use Ruby's `Enumerable` module instead of the `SimplifiedEnumerable` module I provided and compare that to the class-based implementation of this scenario.\n\nThe bad news is that the way that modules solve some of the problems that we discovered about class hierarchies in Ruby ends up making some of the other problems even worse. Because modules tend to provide a whole lot of functionality based on a very thin contract with the object they get mixed into, they are one of the leading causes of child obesity. For example, swapping my `SimplifiedEnumerable` module for Ruby's `Enumerable` method would cause a net increase of 42 new methods that could be directly called on `StatisticalReport`. And now, rather than having a single path to follow in `StatisticalReport` to determine its ancestry chain, there are two. A nice feature of mixins is that they have fairly simple rules about how they get added to the method lookup path to avoid some of the complexities involved in class-based multiple inheritance, but you still need to memorize those rules and be aware of the combinatorial effects of module inclusion.\n\nAs it turns out, modules are a pragmatic compromise that is convenient to use but only slightly more well-behaved than traditional class inheritance. In simple situations, they work just fine, but for more complex systems they end up requiring an increasing amount of discipline to use effectively. Nonetheless, modules tend to be used ubiquitously in Ruby programs despite these problems. A naïve observer might assume that this is a sign that we don't have a better way of doing things in Ruby, but they would be mostly wrong.\n\nAll the problems discussed so far with inheritance can be solved via simple aggregation techniques. For strong evidence of that claim, take a look at the refactored code shown here. As in the previous examples, keep an eye out for the pros and cons of this modeling strategy, and think about what you might do differently.\n\n```ruby\nclass StatisticalCollection\n def initialize(data)\n self.data = data\n end\n\n def sum\n data.reduce(:+) \n end\n\n def average\n sum / data.count.to_f\n end \n\n private\n\n attr_accessor :data\nend\n\nclass StatisticalReport\n def initialize(filename)\n self.input = filename\n \n self.stats = StatisticalCollection.new(each)\n end\n\n def to_s\n \"The sum is #{stats.sum}, and the average is #{stats.average}\"\n end\n\n private \n\n attr_accessor :input, :stats\n\n def each\n return to_enum(__method__) unless block_given?\n\n File.foreach(input) { |e| yield(e.chomp.to_i) }\n end\nend\n\nputs StatisticalReport.new(\"numbers.txt\")\n```\n\nThe first thing you'll notice is that the code is much shorter, as if by magic, but really it's because I completely cheated here and got rid of my counterfeit `Enumerable` object so that I could expose a potentially good idiom for dealing with iteration in an aggregation-friendly way. Feel free to mentally replace the object passed to `StatisticalCollection`'s constructor with something like the code shown here if you don't want me to get away with parlor tricks:\n\n```ruby\nrequire \"forwardable\"\n\nclass EnumerableCollection\n extend Forwardable\n\n # Forwardable bypasses privacy, which is what we want here.\n delegate :each => :data\n\n def initialize(data)\n self.data = data\n end\n\n def count\n c = 0\n each { |e| c += 1 }\n c\n end\n\n # Samnang's implementation from Issue 2.4\n def reduce(arg=nil) \n return reduce {|s, e| s.send(arg, e)} if arg.is_a?(Symbol)\n\n result = arg\n each { |e| result = result ? yield(result, e) : e }\n\n result\n end\n\n private\n\n attr_accessor :data\nend\n```\n\nRegardless of what iteration strategy we end up using, the following points are worth noting about the way we've modeled our system this time around:\n\n1. There are three components in this system, all of which are useful and testable as standalone objects.\n\n2. The relationships between all three components are purely indirect, and the coupling between the objects is limited to the names and behavior of the methods called on them rather than their complete surfaces.\n\n3. There is strict encapsulation between the three components: each have their own namespace, and each can enforce their own privacy controls. It's possible of course to side-step these protections, but they are at least enabled by default. The issue of accidental naming collisions between methods or variables of objects is completely eliminated.\n\n4. As a result of points 2 and 3, the surface of each object is kept narrowly in line with its own domain. In fact, the public interface of `StatisticalReport` has been reduced to its constructor and the `to_s` method, making it as thin as possible while still being useful. \n\nThere are certainly downsides to using aggregation; it is not a golden hammer by any means. But when it comes to **incidental inheritance**, it seems to be the right tool for the job more often than not. I'd love to hear counterarguments to this claim, though, so please do share them if you have something in mind that you don't think would gracefully fit this style of modeling.\n\n### Reflections\n\nAlthough it may be a bit hard to see why disciplined inheritance matters in the trivial scenario we've been talking about throughout this article, it become increasingly clear as systems become more complex. Most scenarios that involve incidental inheritance are actually relatively horizontal problems in nature, but the use of class-based inheritance or module mixins forces a vertical method lookup path that can become very unwieldy, to say the least. When taken to the extremes, you end up with objects like `ActiveRecord::Base`, which has a path that is 43 levels deep, or `Prawn::Document`, which has a 26-level-deep path. In the case of Prawn, at least, this is just pure craziness that I am ashamed to have unleashed upon the world, even if it seemed like a good idea at the time.\n\nIn a language like Ruby that lacks both multiple inheritance and true class-specific privacy for variables and methods, using class-based hierarchies or module mixins for complex forms of incidental inheritance requires a tremendous amount of discipline. For that reason, the extra effort involved in refactoring towards an aggregation-based design pales in comparison to the maintenance headaches caused by following the traditional route. For example, in both `Prawn` and `ActiveRecord`, aggregation would make it possible to flatten that chain by an order of magnitude while simultaneously reducing the chance of namespace collisions, dependencies on lookup order, and accidental side effects due to state mutations. It seems like the cost of somewhat more verbose code would be well worth it in these scenarios.\n\nIn Issue 3.8, we will move on to discuss an essential form of inheritance that Sakkinen refers to as **completely consistent inheritance**. Exploring that topic will get us closer to the concept of mathematical subtypes, which are much more interesting at the theoretical level than incidental inheritance relationships are. But because Ruby's language features make even the simple relationships described in this issue somewhat challenging to manage in an elegant way, I am still looking forward to hearing your ideas and questions about the things I've covered so far.\n\nA major concern I have about incidental inheritance is that I still don't have a\nclear sense of where to draw the line between the two extremes I've outlined in\nthis article. I definitely want to look further into this area, so please leave\na comment if you don't mind sharing your thoughts.\n\n\n"
},
{
"path": "articles/v3/008-criteria-for-disciplined-inheritance.md",
"content": "In [Issue 3.7](http://practicingruby.com/articles/24), I started to explore the criteria laid out by Sakkinen's\n[Disciplined Inheritance](http://scholar.google.com/scholar?cluster=5893037045851782349&hl=en&as_sdt=0,7&sciodt=0,7), \na language-agnostic paper published more than two decades ago that is surprisingly \nrelevant to the modern Ruby programmer. In this issue, we continue where Issue 3.7 \nleft off: on the question of how to maintain complete compatibility between\nparent and child objects in inheritance-based domain models. Or, to put it another way,\nthis article explores how to reuse code safely within a system—\nwithout it becoming a maintenance nightmare.\n\nAfter taking a closer look at what Sakkinen exposed regarding this topic, I came to\nrealize that the ideas he presented were strikingly similar to the [Liskov Substitution\nPrinciple](http://en.wikipedia.org/wiki/Liskov_Substitution_Principle). In fact,\nthe extremely dynamic nature of Ruby makes \nestablishing [a behavioral notion of subtyping](http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.39.1223) (Liskov and Wing 1993)\na prerequisite for developing disciplined inheritance practices. \nAs a result, this article refers to Liskov's work more than Sakkinen's, \neven though both papers have extremely interesting things to say on this topic. \n\n### Defining a behavioral subtype \n\nBoth Sakkinen and Liskov describe the essence of the inheritance relationship as \nthe ability of a child object to serve as a drop-in replacement wherever\nits parent object can be used. I've greatly simplified the concept by\nstating it in such a general fashion, but this is the thread that ties\ntheir independent works together. \n\nLiskov goes a step farther than Sakkinen by defining two kinds of \nbehavioral subtypes: children that extend the behavior specified by their \nparents, and children that constrain the behavior specified by their parents. \nThese concepts are not mutually exclusive, but because each brings up\nits own set of challenges, it is convenient to separate them in this\nfashion.\n\nBoth Sakkinen and Liskov emphasize that the abstract concept of subtyping \nis much more about the observable behavior of objects than it is about\nwhat exactly is going on under the hood. This concept is a natural way of thinking\nfor Rubyists, and it is worth keeping in mind as you read through the rest\nof this article. In particular, when we talk about the type of an object,\nwe are focusing on what that object *does*, not what it *is*.\n\nAlthough the concept of a behavioral subtype sounds like a direct analogue for\nwhat we commonly refer to as \"duck typing\" in Ruby, the former is about\nthe full contract of an object rather than how it acts under certain\ncircumstances. I go into more detail about the differences between\nthese concepts toward the end of this article,\nbut before we can discuss them meaningfully, we need to take a look\nat Liskov's two types of behavioral subtyping and how they can\nbe implemented.\n\n### Behavioral subtypes as extensions\n\nWhether you realize it or not, odds are good that you are already familiar with using behavioral subtypes as extensions. Whenever we inherit from `ActiveRecord::Base` or mix `Enumerable` into one of our objects, we're making use of this concept. In essence, the purpose of an extension is to bolt new behavior on top of an existing type to form a new subtype.\n\nTo ensure that our child objects maintain the substitution principle, we need to make sure that any new behavior and modifications introduced by extensions follow a few simple rules. In particular, all new functionality must be either completely transparent to the parent object or defined in terms of the parent object's functionality. Changing the signature of a method provided by the parent object would be considered an incompatible change, as would directly modifying instance variables referenced by the parent object. These strict rules may seem like overkill, but they are the only way to guarantee that your extended subtypes will be drop-in replacements for their supertypes.\n\nIn practice, obeying these rules is not as hard as it seems. For example, suppose we wanted to extend `Prawn::Document` so that it implements some helpers for article typesetting:\n\n```ruby\nPrawn::Article.generate(\"test.pdf\") do\n h1 \"Criteria for Disciplined Inheritance\"\n \n para %{\n This is an example of building a Prawn-based article\n generator through the use of a behavioral subtype as\n an extension. It's about as wonderful and self-referential\n as you might expect.\n }\n\n h2 \"Benefits of behavioral subtyping\"\n\n para %{\n The benefits of behavioral subtyping cannot be directly\n known without experiencing them for yourself.\n }\n\n para %{\n But if you REALLY get stuck, try asking Barbara Liskov.\n }\nend\n```\n\nThe most simple way to implement this sort of domain language would be to create a subclass of `Prawn::Document`, as shown in the following example:\n\n```ruby\nmodule Prawn\n class Article < Document\n include Measurements\n\n def h1(contents)\n text(contents, :size => 24)\n move_down in2pt(0.3)\n end\n\n def h2(contents)\n move_down in2pt(0.1)\n text(contents, :size => 16)\n move_down in2pt(0.2)\n end\n\n def para(contents)\n text(contents.gsub(/\\s+/, \" \"))\n move_down in2pt(0.1)\n end\n end\nend\n```\n\nAs far as Liskov is concerned, `Prawn::Article` is a perfectly legitimate extension because instances of it are drop-in substitutes for `Prawn::Document` objects. In fact, this sort of extension is trivial to prove to be a behavioral subtype because it is defined purely in terms of public methods that are provided by its parents (`Prawn::Document` and `Prawn::Measurements`). Because the functionality added is so straightforward, the use of subclassing here might just be the right tool for the job. \n\nThe downside of using subclassing is that even minor alterations to program requirements can cause encapsulation-related issues to become a real concern. For example, if we decide that we want to add a pair of instance variables that control the fonts used for headers and paragraphs, it would be hard to guarantee that these variables wouldn't clash with the data contained within `Prawn::Document` objects. We can assume that calls to public methods provided by the parent object are safe, but we cannot say the same about referencing instance variables, so a delegation-based model starts to look more appealing.\n\nSuppose we wanted to support the following API, but via delegation rather than subclassing:\n\n```ruby\nPrawn::Article.generate(\"test.pdf\") do\n header_font \"Courier\"\n paragraph_font \"Helvetica\"\n\n h1 \"Criteria for Disciplined Inheritance\"\n \n para %{\n This is an example of building a Prawn-based article\n generator through the use of a behavioral subtype as\n an extension. It's about as wonderful and self-referential\n as you might expect.\n }\n\n h2 \"Benefits of behavioral subtyping\"\n\n para %{\n The benefits of behavioral subtyping cannot be directly\n known without experiencing them for yourself.\n }\n\n para %{\n But if you REALLY get stuck, try asking Barbara Liskov.\n }\nend\n```\n\nUsing a `method_missing` hook and the `Prawn::Article.generate` class method, it\nis fairly easy to implement this DSL:\n\n```ruby\nmodule Prawn\n class Article\n def self.generate(*args, &block)\n Prawn::Document.generate(*args) do |pdf|\n new(pdf).instance_eval(&block)\n end\n end\n\n def initialize(document)\n self.document = document \n document.extend(Prawn::Measurements)\n\n # set defaults so that @paragraph_font and @header_font are never nil.\n paragraph_font \"Times-Roman\"\n header_font \"Times-Roman\"\n end\n\n def h1(contents)\n font(header_font) do\n text(contents, :size => 24)\n move_down in2pt(0.3)\n end\n end\n\n def h2(contents)\n font(header_font) do\n move_down in2pt(0.1)\n text(contents, :size => 16)\n move_down in2pt(0.2)\n end\n end\n\n def para(contents)\n font(paragraph_font) do\n text(contents.gsub(/\\s+/, \" \"))\n move_down in2pt(0.1)\n end\n end\n\n def paragraph_font(font=nil)\n return @paragraph_font = font if font\n\n @paragraph_font\n end\n\n def header_font(font=nil)\n return @header_font = font if font\n\n @header_font\n end\n\n def method_missing(id, *args, &block)\n document.send(id, *args, &block)\n end\n\n private\n\n attr_accessor :document\n end\nend\n```\n\nTaking this approach involves writing more code and adds some complexity. However, that is a small price to pay for the peace of mind that comes with cleanly separating the data contained within the `Prawn::Article` and `Prawn::Document` objects. This design also makes it harder for `Prawn::Article` to have name clashes with `Prawn::Document`'s private methods and forces any private method calls to `Prawn::Document` to be done explicitly. Because transparent delegation exposes the full contract of the parent object, it is still necessary for the child object to maintain full compatibility with those methods in the same manner that a class-inheritance-based model would. Nonetheless, this pattern provides a safer way to implement subtypes because it avoids incidental clashes, which could otherwise occur easily.\n\nAlthough the examples we've looked at so far—combined with your own experiences—should give you a good sense of how to extend code via behavioral subtypes, there are some common pitfalls I have glossed over in order to keep things simple. I'll get back to those before the end of the article, but for now let's turn our attention to the other kind of subtypes Liskov describes in her paper. She refers to them as _constrained subtypes_, but I call them _restriction subtypes_ as an easy-to-remember mirror image of the _extension subtype_ concept.\n\n### Behavioral subtypes as restrictions\n\nJust as subtypes can be used to extend the behavior of a supertype, they can also be used to restrict generic behaviors by providing more specific implementations of them. The example Liskov uses in her paper illustrates how a stack structure can be viewed as a restriction on the more general concept of a bag.\n\nIn its most simple form, a bag is essentially nothing more than a set that can contain duplicates. Items can be added and removed from a bag, and it is possible to determine whether the bag contains a given item. However, much like with a set, order is not guaranteed. The following code, which is somewhat of a contrived example, implements a `Bag` object similar to the one described in Liskov's paper:\n\n```ruby\nContainerFullError = Class.new(StandardError)\nContainerEmptyError = Class.new(StandardError)\n\nclass Bag\n def initialize(limit)\n self.items = [] \n self.limit = limit\n end\n\n def push(obj)\n raise ContainerFullError unless data.length < limit\n\n data.shuffle!\n data.push(obj)\n end\n\n def pop\n raise ContainerEmptyError if data.empty?\n\n data.shuffle!\n data.pop\n end\n\n def include?(obj)\n data.include?(obj)\n end\n\n private\n\n attr_accessor :items, :limit\nend\n```\n\nThe challenge in determining whether a `Stack` object can meaningfully be considered a subtype of this sort of structure is that we need to find a way to describe the functionality of a bag so that it is general enough to allow for interesting subtypes to exist but specific enough to allow the `Bag` object to be used on its own in a predictable way. Because Ruby lacks the design-by-contract features that Liskov depends on in her paper, we need to describe this specification verbally rather than relying on our tools to enforce them for us. Something like the following list of rules is roughly similar to what she describes more formally in her work:\n\n1) A bag has `items` and a size `limit`.\n\n2) A bag has a `push` operation, which adds a new object to the bag's `items`.\n\n* If the current number of `items` is less than the `limit`, the new object is added to the bag's `items`.\n\n* Otherwise, a `ContainerFullError` is raised.\n\n3) A bag has a `pop` operation, which removes an object from the bag's `items` and returns it as a result.\n\n* If the bag has no `items`, a `ContainerEmptyError` is raised.\n\n* Otherwise, one object is removed from the bag's `items` and returned.\n\n4) A bag has an `include?` operation, which indicates whether the provided object is one of bag's `items`.\n\n* If the bag's `items` contains the provided object, `true` is returned.\n\n* Otherwise, `false` is returned.\n\nWith these rules in mind, we can see that the following `Stack` object satisfies the definition of a bag while simultaneously introducing a predictable ordering to `items`:\n\n```ruby\nContainerFullError = Class.new(StandardError)\nContainerEmptyError = Class.new(StandardError)\n\nclass Stack\n def initialize(limit)\n self.items = [] \n self.limit = limit\n end\n\n def push(obj)\n raise ContainerFullError unless data.length < limit\n\n data.push(obj)\n end\n\n def pop\n raise ContainerEmptyError if data.empty?\n\n data.pop\n end\n\n def include?(obj)\n data.include?(obj)\n end\n\n private\n\n attr_accessor :items, :limit\nend\n```\n\nWith this example code in mind, we can specify the behavior of a stack in the following way:\n\n1) A stack is a bag.\n\n2) A stack's `pop` operation follows a last-in, first-out (LIFO) order.\n\nBecause the ordering requirements of a stack don't conflict with the defining characteristics of a bag, a stack can be substituted for a bag without any issues. The key thing to keep in mind here is that restriction subtypes can create additional constraints on top of what was specified by their supertypes but cannot loosen the constraints put upon them by their ancestors in any way. For example, based on the way we defined bag objects, we would not be able to return `nil` instead of raising `ContainerEmptyError` when `pop` is called on an empty stack, even if that seems like a fairly innocuous change.\n\nOnce again, maintaining this sort of discipline may seem on the surface to be more trouble than it is worth. However, these kinds of assumptions are baked into useful patterns such as the [template method pattern](http://en.wikipedia.org/wiki/Template_method_pattern) and are also key to designing type hierarchies for all sorts of data structures. A good example of these concepts in action can be found in the way Ruby organizes its numeric types. The class hierarchy is shown here, but be sure to check out Ruby's documentation if you want to get a sense of how exactly these classes hang together.\n\n![](\"http://i.imgur.com/ObKrf.jpg\")
\n\nWhether you are designing extension subtypes or restriction subtypes, it is unfortunately easier to get things wrong than it is to get them right, due to all the subtle issues that need to be considered. For that reason, we'll now take a look at a few examples of flawed behavioral subtypes and how to go about fixing them.\n\n### Examples of flawed behavioral subtypes\n\nTo test your understanding of behavior subtype compatibility while simultaneously exposing some common pitfalls, I provide the following three flawed examples for you to study. As you read through them, try to figure out what the subtype compatibility problem is and how you might go about solving it.\n\n1) Suppose we want to add an equality operator to the bag structure. A sample operator is provided here for the `Bag` object, which conforms to the following newly specified\nfeature: \"Two bags are considered equal if they have equivalent items and size limits\". What problems will we encounter in implementing a bag-compatible equality operator for the `Stack` object? \n\n```ruby\nclass Bag\n # other code similar to before\n\n def ==(other)\n [data.sort, limit] == [other.sort, other.limit]\n end\n\n protected \n \n # NOTE: Implementing == is one of the few legitimate uses of \n # protected methods / attributes\n attr_accessor :data, :limit\nend\n```\n\n2) Suppose we have two mutable objects, a `Rectangle` and a `Square`, and we wish to implement `Square` as a restriction of `Rectangle`. Given the following implementation of a `Rectangle` object, what problems will be encountered in defining a `Square` object?\n\n```ruby\nclass Rectangle\n def area\n width * height\n end\n\n attr_accessor :width, :height\nend\n```\n\n3) Suppose we have a `PersistentSet` object that delegates all method calls to the `Set` object provided by Ruby's standard library, as shown in the following code. Why is this not a compatible subtype, even though it does not explicitly modify the behavior of any of `Set`'s operations?\n\n```ruby\nrequire \"set\"\nrequire \"pstore\"\n\nclass PersistentSet \n def initialize(filename)\n self.store = PStore.new(filename)\n\n store.transaction { store[:data] ||= Set.new }\n end\n\n def method_missing(name, *args, &block)\n store.transaction do \n store[:data].send(name, *args, &block)\n end\n end\n\n private\n\n attr_accessor :store\nend\n```\n\nTo avoid spoiling the fun of finding and fixing the defects with these examples yourself, I've hidden my explanation of the [problems](https://gist.github.com/15b50f918c88bccd6eac) and [solutions](https://gist.github.com/3f53d4094759c0508e19) on a pair of gists. Please spend some time on this exercise before reading the spoilers, as you'll learn a lot more that way!\n\nA huge hint is that the first problem is based on an issue discussed in [Liskov's paper](http://www.cs.cmu.edu/~wing/publications/LiskovWing94.pdf) and the second and third problems are discussed in an [article about LSP](http://www.objectmentor.com/resources/articles/lsp.pdf) by Bob Martin. However, please note that their solutions are not exactly the most natural fit for Ruby, so there is still room for some creativity here.\n\n### Behavioral subtyping versus duck typing\n\nBetween this article and the topics discussed in [Issue 3.7](http://practicingruby.com/articles/24), this two-part series offers a fairly comprehensive view of disciplined inheritance practices for the Ruby programmer. However, as I hinted toward the beginning of this article, there is the somewhat looser concept of duck typing that deserves a mention if we really want to see the whole picture.\n\nWhat duck typing and behavioral subtypes have in common is that both concepts rely on what an object can do rather than what exactly it is. They differ in that behavioral subtypes seem to be more about the behavior of an entire object and duck typing is about how a given object behaves within a certain context. Duck typing can be a good deal more flexible than behavioral subtyping in that sense, because typically it involves an object implementing a meaningful response to a single message rather than an entire suite of behaviors. You can find a ton of examples of duck typing in use in Ruby, but perhaps the easiest to spot is the ubiquitous use of the `to_s` method.\n\nBy implementing a `to_s` method in our objects, we are able to indicate to Ruby that our object has a meaningful string representation, which can then be used in a wide range of contexts. Among other things, the `to_s` method is automatically called by irb when an `inspect` method is not also provided, called by the `Kernel#puts` method on whatever object you pass to it, and called automatically on the result of any expression executed via string interpolation. Implementing a meaningful `to_s` method is not exactly a form of behavioral subtyping but is still a very useful form of code sharing. [Issue 1.14](http://blog.rubybestpractices.com/posts/gregory/046-issue-14-duck-typing.html) and [Issue 1.15](http://blog.rubybestpractices.com/posts/gregory/047-issue-15-duck-typing-2.html) cover duck typing in great detail, but this single example is enough to point out the merits of this technique and how much simpler it is than the topics discussed in this article.\n\n### Reflections\n\nA true challenge for any practicing Rubyist is finding a balance between the free-wheeling culture of Ruby development and the more rigorous approaches of our predecessors. Disciplined inheritance techniques will make our lives easier, and knowing what a behavioral subtype is and how to design one will surely come in handy on any moderately complex project. However, we should keep our eyes trained on how these issues relate to maintainability, understandability, and changeability rather than obsessing about how they can lead us to mathematically pure designs.\n\nI think there is room for another article on the practical applications of these ideas, in which I might talk about applying some design-by-contract concepts to Ruby programs or how to develop shared unit tests that make it easier to test for compatibility when implementing subtypes. But I don't plan to work on that article immediately, so for now we can sort out those issues via comments on this article. If you have any suggestions for how to tie these ideas back to real problems, or questions on how to apply them to the things you've been working on, please share your thoughts. \n"
},
{
"path": "articles/v3/009-using-games-to-practice-domain-modeling.md",
"content": "As programmers, it is literally our job to make [domain models](http://en.wikipedia.org/wiki/Domain_model) understandable to computers. While this can be some of the most creative work we do, it also tends to be the most challenging. The inherent difficulty of designing and implementing conceptual models leads many to develop their problem solving skills through a painful process of trial and error rather than some form of deliberate practice. However, that is a path paved with sorrows, and we can do better.\n\nDefining problem spaces and navigating within them does get easier as you become more experienced. But if you only work with complex domain models while you are knee deep in production code, you'll find that many useful modeling patterns will blend in with application-specific details and quickly fade into the background without being noticed. Instead, what is needed is a testbed for exploring these ideas that is complex enough to mirror some of the problems you're likely to encounter in your daily work, but inconsequential enough to ensure that your practical needs for working code won't get in the way of exploring new ideas. \n\nWhile there are a number of ways to create a good learning environment for studying domain modeling, my favorite approach is to try to clone bits of functionality from various games I play when I'm not coding. In this article, I'll walk you through an example of this technique by demonstrating how to model a simplified version of the [Minecraft crafting system](http://www.minecraftwiki.net/wiki/Crafting).\n\n### Defining the problem space\n\n> **NOTE:** Those who haven't played Minecraft before may want to spend a few minutes watching this video [tutorial about crafting](http://www.youtube.com/watch?v=AKktiCsCPWE) or skimming [the game's wiki page](http://www.minecraftwiki.net/wiki/Crafting) on the topic before continuing. However, because I only focus on a few very basic ideas about the system for this exercise, you don't need to be a Minecraft player in order to enjoy this article.\n\nThe crafting table is a key component in Minecraft because it provides the player with a way to turn natural resources into useful tools, weapons, and construction materials. Stripped down to its bare essence, the function of the crafting table is essentially to convert various input items laid out in a 3x3 grid into some quantity of a different type of item. For example, a single block of wood can be converted into four wooden planks, a pair of wooden planks can be combined to produce four sticks, and a stick combined with a piece of coal will produce four torches. Virtually all objects in the Minecraft world can be built in this fashion, as long as the player has the necessary materials and knows the rules about how to combine them together. \n\nBecause positioning of input items within the crafting table's grid is significant, players need to make use of recipes to learn how various input items can be combined to produce new objects. To make recipes easier for the player to memorize, the game allows for a bit of flexibility in the way things are arranged, as long as the basic structure of the layout is preserved. In particular, the input items for recipes can be horizontally and vertically shifted as long as they remain within the 3x3 grid, and the system also knows how to match mirror images as well. However, after accounting for these variants, there is a direct mapping from the inputs to the outputs in the crafting system.\n\nAs of 2012-02-27, Minecraft supports 174 crafting recipes. This is a small enough number where even a naïve data model would likely be fast enough to not cause any usability problems, even if you consider the fact that most of those recipes can be shifted around in various ways. But in the interest of showing off some neat Ruby data modeling tricks, I've decided to try to implement this model in an efficient way. In doing so, I found out that inputs can be checked for corresponding outputs in constant time, and that there are some useful constraints that make it so that only a few variants need to be checked in most cases in order to find a match for the player's input items.\n\nMy [finished model](https://github.com/elm-city-craftworks/crafting_table) ended up consisting of three parts: A `Recipe` object responsible for codifying the layout of input items and generating variants based on that layout, a `Cookbook` object which maps recipes to their outputs, and an `Importer` object which generates a cookbook object from CSV formatted recipe data. In the following sections, I will take a look at each of these objects and point out any interesting details about them.\n\n### Modeling recipes \n\n> **NOTE:** To keep my implementation code easy to follow, I have simplified the recipe model somewhat so that it does not consider mirror images of recipes to be equivalent. Implementing that sort of behavior could be a fun exercise for the reader, and would make this model a closer match to what Minecraft actually implements.\n\nThe challenge involved in modeling Minecraft recipes is that you need to treat horizontal and vertically shifted item sets as being equivalent to one other. Or in other words, as long as the shape of an item set is preserved, there is a bit of flexibility about where you can place items on the table. For example, all of the recipes below are considered to be equivalent to one another:\n\n\n\nA naïve approach to the problem will lead you to checking up to 25 variants for each recipe, only to find out that most of them are invalid mutations of the original item set that place at least one item outside of the 3x3 grid. Some simple checks can be put in place to throw out invalid variants, but it is better to never generate them at all. \n\n> **UPDATE 2012-03-01**: Based on a suggestion by [Shane Emmons](http://community.mendicantuniversity.org/people/semmons99), I worked on a better approach to this problem after this article was published. The basic idea is that rather than generating recipe variants, you instead normalize the recipes into a single common layout on demand. Check out the [updated code here](https://github.com/elm-city-craftworks/crafting_table/blob/607a4d8fc958c2e746b899c43b5cbb01301b3c6b/lib/crafting_table/recipe.rb). The solution described below is still interesting though, so feel free to read on anyway!\n\nThe approach I ended up taking is to compute margins surrounding each item that indicate how they can be shifted. As each new item gets added to the recipe, its margins and the margins of the current item set are intersected to obtain a new set of boundaries. The following diagram demonstrates the process of adding three items (B, C, A) to the grid sequentially, with each newly added item reducing the number of equivalent recipes that can be generated:\n\n\n\nThis process is very efficient because it involves simple numerical computations at insert time, rather than processing the whole item set at once. With that in mind, my implementation of `Recipe#[]=` updates the margins for the item set right away whenever a new item is added:\n\n```ruby\nrecipe[0,0] = \"B\"\np recipe.send(:margins) #=> {:top=>2, :left=>0, :right=>2, :bottom=>0}\n\nrecipe[1,0] = \"C\"\np recipe.send(:margins) #=> {:top=>2, :left=>0, :right=>1, :bottom=>0} \n\nrecipe[0,1] = \"A\"\np recipe.send(:margins) #=> {:top=>1, :left=>0, :right=>1, :bottom=>0} \n```\n\nThe following code shows how `Recipe#[]=` is implemented. In particular, it demonstrates that item set margins are directly updated on insert, but variant layouts are not generated until later.\n\n```ruby\nmodule CraftingTable\n class Recipe\n TABLE_WIDTH = 3\n TABLE_HEIGHT = 3\n\n def initialize\n self.ingredients = {}\n self.margins = { :top => Float::INFINITY, \n :left => Float::INFINITY, \n :right => Float::INFINITY,\n :bottom => Float::INFINITY }\n\n self.variants = Set.new\n self.variants_need_updating = false\n end\n\n # ... various unrelated details omitted ...\n\n def []=(x,y,ingredient_type)\n raise ArgumentError unless (0...TABLE_WIDTH).include?(x)\n raise ArgumentError unless (0...TABLE_HEIGHT).include?(y)\n\n # storing positions as vectors makes variant computations easier\n ingredients[Vector[x,y]] = ingredient_type\n\n update_margins(x,y)\n\n self.variants_need_updating = true\n end\n\n private\n\n attr_accessor :margins, :ingredients, :variants_need_updating\n\n def update_margins(x,y)\n margins[:left] = [x, margins[:left] ].min\n margins[:right] = [TABLE_WIDTH-x-1, margins[:right] ].min\n margins[:bottom] = [y, margins[:bottom]].min\n margins[:top] = [TABLE_HEIGHT-y-1, margins[:top] ].min\n end\n end\nend\n```\n\nI deferred the process of generating variants simply because doing so at insert time would cause many unnecessary intermediate computations to be done for multi-item recipes. While such a small number of possible variations pretty much guarantees there won't be performance issues whether or not lazy evaluation is used, I wanted to use this situation as a chance to think through how I would model variant generation if efficiency was a real concern. In production code, premature optimization is the root of all evil, but when you're in deliberate practice mode it can be quite fun.\n\nI ultimately decided to generate the variants on demand when two `Recipe` objects are compared to one another. As you can see from the following code, my implementation of `Recipe#==` causes both recipe objects involved in the test to update their variants if necessary:\n\n```ruby\nmodule CraftingTable\n class Recipe\n # ... various unrelated details omitted ...\n\n def ==(other)\n return false unless self.class == other.class\n\n variants == other.variants\n end\n\n protected\n\n def variants\n update_variants if variants_need_updating\n\n @variants\n end\n end\nend\n```\n\nWhile the high level interface for `Recipe` comparison is easy to follow, the way I ended up generating the underlying variant data is a bit messy. The implementation details for `Recipe#update_variants` are shown below, but the rough idea here is that I compute a set of `valid_offsets` and then use them to do vector addition to translate items to different coordinates within the grid. After performing this transformation, I wrap the variant data in a `Set` object so that they can easily be compared in an order-independent fashion. Assuming all this happens successfully, the `variants_need_updating` flag gets set to `false` to indicate that the variant data is now up to date. \n\n```ruby\nmodule CraftingTable\n class Recipe\n \n # ... various unrelated details omitted ...\n\n private\n\n attr_accessor :margins, :ingredients, :variants_need_updating\n attr_writer :variants\n\n def update_variants\n raise InvalidRecipeError if ingredients.empty?\n\n variant_data = valid_offsets.map do |x,y|\n ingredients.each_with_object({}) do |(position, content), variant|\n new_position = position + Vector[x,y]\n\n variant[new_position] = content\n end\n end\n\n self.variants = Set[*variant_data]\n self.variants_need_updating = false\n end\n\n def valid_offsets\n horizontal = (-margins[:left]..margins[:right]).to_a\n vertical = (-margins[:bottom]..margins[:top]).to_a\n\n horizontal.product(vertical)\n end\n end\nend\n```\n\nAn interesting thing to note about this design is that variants are purely implementation details that are not exposed via the public API. While the large amount of code I've shelved in private methods seems to indicate that there might be an object to extract here, I like the idea that from the outside perspective, the equivalence relationship between recipes is established without having to do any sort of explicit check to see whether two different layouts share a common variant. To see the true benefits of this kind of information hiding, we can take a look at how it affects the design of the cookbook.\n\n### Modeling a cookbook\n\nOne of the first things I noticed about this problem domain was that the mapping of inputs to outputs were a natural fit for a hash structure. While it took a while to sort out the details, I eventually was able to put together a `Cookbook` object that works in the manner shown below:\n\n```ruby\ncookbook = CraftingTable::Cookbook.new\ntorch_recipe = CraftingTable::Recipe.new\n\ntorch_recipe[1,1] = \"coal\"\ntorch_recipe[1,0] = \"stick\"\n\ncookbook[torch_recipe] = [\"torch\", 4]\n\n# ---\n\nuser_recipe = CraftingTable::Recipe.new\nuser_recipe[2,2] = \"coal\"\nuser_recipe[2,1] = \"stick\"\n\np cookbook[user_recipe] #=> [\"torch\", 4]\n```\n\nThe final implementation of this object turned out to be incredibly simple, although it required some minor extensions to the `Recipe` object in order to work correctly. Take a look at the following class definition to see just how little `CraftingTable::Cookbook` is doing under the hood:\n\n```ruby\nmodule CraftingTable\n\n # This is the complete definition of my Cookbook object, with no omissions!\n\n class Cookbook\n def initialize\n self.recipes = {}\n end\n\n def [](recipe)\n recipes[recipe]\n end\n\n def []=(recipe, output)\n if recipes[recipe]\n raise ArgumentError, \"A variant of this recipe is already defined!\"\n end\n\n recipes[recipe] = output\n end\n\n private\n\n attr_accessor :recipes\n end\nend\n```\n\nOn the surface, the class seems to have only two tangible features to it: it severely narrows the interface to the hash it wraps so that it becomes nothing more than a simple key/value store, and it forces single assignment semantics. However, when we look at how the object is actually used, we see that there is an implicit dependency on some deeper, more domain specific logic. Revisiting the usage example from before, you can see that the `Cookbook` object treats variants of the same recipes as if they were the same hash key. \n\n```ruby\n# ... unimportant boilerplate omitted\n\ntorch_recipe[1,1] = \"coal\"\ntorch_recipe[1,0] = \"stick\"\n\ncookbook[torch_recipe] = [\"torch\", 4]\n\n# ---\n\nuser_recipe[2,2] = \"coal\"\nuser_recipe[2,1] = \"stick\"\n\np cookbook[user_recipe] #=> [\"torch\", 4]\n```\n\nIf you haven't already dug into the source to locate where this bit of magic comes from, it has to do with the fact that Ruby provides hooks that allow you to use complex objects as hash keys. In particular, customizing the way that objects are used as hash keys involves overriding the `Object#hash` and `Object#eql?` methods. If you take a closer look at the `Recipe` object, you'll see it does define both of these methods:\n\n```ruby\nmodule CraftingTable\n # ... various unrelated details omitted ...\n\n class Recipe\n def ==(other)\n return false unless self.class == other.class\n\n variants == other.variants\n end\n\n # this is the standard idiom, as in most cases == should be the same as eql?\n alias_method :eql?, :==\n\n def hash\n variants.hash\n end\n end\nend\n```\n\nWhile I don't want to get bogged down in the details of how these hooks work, the basic idea is that the `hash` method returns a numeric identifier which determines which bucket to store the value in. When a provided key hashes to the same number of an object already in the hash, the `eql?` method determines whether the keys are actually equivalent. Because `Recipe#hash` simply delegates to `Set#hash`, all item sets with the same elements have the same hash value, even if their order differs. Likewise, when `eql?` is called, it ends up delegating to `Set#==` which has the same semantics. If you trace your way through the usage example, you'll find that because `torch_recipe` and `user_recipe` generate the same variants, they also can stand in for one another as hash keys due to these overridden methods.\n\nWithout a doubt, this is a *clever* technique. But I'm still on the fence about whether it is a good approach or not. On the one hand, it makes use of a well defined hook that Ruby provides which seems to be well suited for the problem we're trying to model. On the other hand, it is not a very explicit way of building an API at all, and requires a non-trivial understanding of low level features of Ruby to fully understand this code. This is a common tension whenever designing Ruby objects: Matz assumes we're all a lot smarter and a lot more responsible than we might consider ourselves. \n\nI decided to go this route because in learning exercises I like to push my boundaries a bit and see where it takes me. But if this were production code, I would think about going with a slighly less elegant but more explicit approach. For example, I might have made the `Recipe#variants` method public and then did something similar to the following code in the `Cookbook#[]` method:\n\n```ruby\nmodule CraftingTable\n class Cookbook\n def [](recipe)\n variant = recipe.variants.find { |v| recipes[v] }\n\n recipes[variant]\n end\n\n # ...\n end\nend\n```\n\nThat said, I would love to hear your thoughts on this particular pattern. Sometimes when a technique is rare, it's hard to tell whether it seems unintuitive because it is actually hard to understand, or if it just feels that way because it isn't familiar territory.\n\n### Modeling a recipe importer\n\nWith the interesting modeling out of the way, all that remains to talk about is how to get data imported into cookbooks in a way that doesn't require a lot of tedious assignment statements. For this purpose, I built a simple `Importer` object which takes a CSV file as input and builds up a `Cookbook` object from it.\n\nThe data format consists of multiline records separated by an empty line, as shown below:\n\n```ruby\ntorch,4\n-,-,-\n-,coal,-\n-,stick,-\n\ncrafting_table,1\n-,-,-\nwooden_plank,wooden_plank,-\nwooden_plank,wooden_plank,-\n```\n\nWhile the data isn't pretty as a raw CSV file, this format makes it convenient to edit the data via a spreadsheet program, and doing so provides a pretty nice layout of the input grid. Once the file is written up, it ends up getting used in the manner shown below:\n\n```ruby\ncookbook = CraftingTable::Importer.cookbook_from_csv(recipe_file)\nuser_recipe_1 = CraftingTable::Recipe.new\n\nuser_recipe_1[1,0] = \"stick\"\nuser_recipe_1[1,1] = \"coal\"\n\np cookbook[user_recipe_1] #=> [\"torch\", 4]\n\nuser_recipe_2 = CraftingTable::Recipe.new\n\nuser_recipe_2[0,0] = \"wooden_plank\"\nuser_recipe_2[0,1] = \"wooden_plank\"\nuser_recipe_2[1,0] = \"wooden_plank\"\nuser_recipe_2[1,1] = \"wooden_plank\"\n\np cookbook[user_recipe_1] #=> [\"crafting_table\", 1]\n```\n\nThe implementation of the `Importer` object is mostly an uninspired procedural hack, with the only interesting detail of it being that it manually iterates over the CSV data using `CSV.new` in combination with a `File` object as yet another unnecessary-yet-educational efficiency optimization:\n\n```ruby\nmodule CraftingTable\n Importer = Object.new\n\n class << Importer\n def cookbook_from_csv(filename)\n cookbook = Cookbook.new\n\n File.open(filename) do |f|\n csv = CSV.new(f)\n\n until f.eof?\n product, quantity = csv.gets\n\n grid = [csv.gets, csv.gets, csv.gets]\n\n cookbook[recipe_from_grid(grid)] = [product, quantity.to_i]\n \n csv.gets\n end\n end\n\n cookbook\n end\n\n def recipe_from_grid(grid)\n recipe = Recipe.new\n\n last_row = Recipe::TABLE_WIDTH - 1\n last_col = Recipe::TABLE_HEIGHT - 1\n\n ((0..last_row).to_a).product((0..last_col).to_a) do |x,y|\n row = x\n col = last_col - y\n \n next if grid[col][x] =~ /-/\n\n recipe[x,y] = grid[col][x]\n end\n\n recipe\n end\n end\nend\n```\n\nThis object is boring enough that I originally had planned to not implement it at all, in favor of having a `Cookbook.from_csv` method and perhaps a `Recipe.from_grid` method. However, I am increasingly growing suspicious of the presence of too many factory methods on objects, and worried that I'd be mixing the concerns of data extraction and data manipulation too much by doing that. In particular, I would have had to figure out a way to avoid directly referencing the \"-\" string as an indicator of an empty cell in `Recipe.from_grid`, and I didn't want to focus my energy on that because it felt like a waste of time.\n\nThis code represents a small compromise in that it isolates something that doesn't quite have a natural home so that it can be refactored later into something more elegant. Because this is a bolt-on feature, I felt comfortable making that trade so that I could focus more on the heart of the problem. However, if data import needs became more complex, this code would almost certainly need to be refactored into something more well organized.\n\n### Reflections\n\nHopefully this article has given you a strong sense of how deep even seemingly simple game constructs can be if you really think them through. In my experience, this phenomenon is strikingly similar to the kinds of complexity that arise naturally in even moderately complicated business applications. The main difference is that in a practice environment, you don't need to worry about how much money you're costing someone else by spending as much time thinking about the problem as you do writing implementation code.\n\nWhile doing deliberate practice of this variety, it is perfectly acceptable to actively seek out ways to induce analysis paralysis, premature optimization, and extreme over-engineering. In fact, the closer you get to feeling like your solution is completely overkill for the problem at hand, the more likely it is that you're going to learn something useful from the exercise. Experiencing the tensions that arise from this kind of perfectionism in a low-risk environment can make it a lot easier to take a middle of the road path when dealing with your day to day work.\n\nThe thing I like most about this sort of exercise is that it will often lead you to come across patterns or techniques that actually are directly applicable in practical scenarios. Whenever I stumble across a technique which is just as easy to implement as a more commonly used alternative but is more robust in some way, I tend to experiment with introducing those ideas into my production code to see how they work out for me. Sometimes these experiments work and other times they don't, but they always improve my understanding of why I do things the way I do.\n\nWhile I remain a firm believer in the idea that deliberate practice should be done only in moderation and that there is no substitute for working on real problems that matter to you, the occasional sitting or two spent on shaking up what you think you know about this craft is well worth the effort. There are lots of different ways to do that, but this is the way that works for me. I'd love to hear what you think of it, and would also like to hear what other ways you've tried to hone your problem solving skills. \n"
},
{
"path": "articles/v3/010-robustness.md",
"content": "Writing robust code is always challenging, even when dealing with extremely well controlled environments. But when you enter the danger zone where software failures can result in data loss or extended service interruptions, coding for robustness becomes essential even if it is inconvenient to do so. \n\nIn this article, I will share some of the lessons I've learned about building\nstable software through my work on the [Newman mail framework](https://github.com/mendicant-original/newman). While the techniques I've discovered so far are fairly ordinary, it was easy to underestimate their importance in the early stages of the project's development. My hope is that by exposing my stumbling points, it will save others from making the same mistakes.\n\n### Lesson 1: Collect enough information about your workflow.\n\nIn many contexts, collecting the information you need to analyze a failure is the easy part of the debugging process. When your environment is well controlled, a good stack trace combined with a few well placed `puts` statements are often all you need to start reproducing an error in your development environment. Unfortunately, these well-worn strategies are not nearly as effective for debugging application frameworks.\n\nTo get a clearer sense of the problem, consider that Newman's server software knows almost nothing about the applications it runs, nor does it know much of anything about the structures of the emails it is processing. It also cannot assume that its interactions with external IMAP and SMTP servers will be perfectly stable. In this kind of environment, something can go wrong at every possible turn. This means that in order to find out where and why a failure occured, it is necessary to make the sequence of events easier to analyze by introducing some sort of logging system.\n\nA good place to start when introducing event logging is with the outermost layer of the system. In Newman's case, this means tracking information about every incoming and outgoing email, as shown below: \n\n```\nI, [2012-03-10T12:46:57.274091 #9841] INFO -- REQUEST: \n{:from=>[\"gregory_brown@letterboxes.org\"], :to=>[\"test+ping@test.com\"],\n:bcc=>nil, :subject=>\"hi there!\", :reply_to=>nil}\n\nI, [2012-03-10T12:46:57.274896 #9841] INFO -- RESPONSE: \n{:from=>[\"test@test.com\"], :to=>[\"gregory_brown@letterboxes.org\"], \n:bcc=>nil, :subject=>\"pong\", :reply_to=>nil}\n```\n\nBecause Newman currently only understands how to filter messages based on their TO and SUBJECT fields, the standard log information is fairly helpful for basic application debugging needs. However, when dealing with complex problems, it is nice to be able to see [the raw contents of the messages](https://raw.github.com/gist/01fbab481a21f4d43bbf/0778e1a0ae887e6423bce985298e3f8d60eb37a0/gistfile1.txt). Rather than choosing one or the other, Newman handles both log formats by outputting them at different log levels:\n\n```ruby\nmodule Newman\n module EmailLogger\n def log_email(logger, prefix, email)\n logger.debug(prefix) { \"\\n#{email}\" }\n logger.info(prefix) { email_summary(email) }\n end\n\n private\n\n def email_summary(email)\n { :from => email.from,\n :to => email.to,\n :bcc => email.bcc,\n :subject => email.subject,\n :reply_to => email.reply_to }\n end\n end\nend\n```\n\nHaving the ability to dynamically decide what level of detail your log output should contain is one of the main advantages of using a proper logging system instead of directly outputting messages to the console. While it would be possible to implement some sort of information filtering mechanism without using a formal logging system, doing so would involve reinventing many of the things that the `Logger` standard library already provides for you.\n\nThe cost of introducing a logging system is that once you depend on logs for your debugging information, some form of exception logging becomes absolutely necessary. Because failures can be very context-dependent, deciding how handle them can be tricky. \n\n### Lesson 2: Plan for various kinds of predictable failures.\n\nBecause Newman does not know anything about the applications it runs except that they all implement a `call` method, it is not possible to be selective about what kinds of errors to handle. Instead, a catch-all mechanism is implemented in the `process_request` method:\n\n```ruby\nmodule Newman\n class Server\n def process_request(request, response)\n apps.each do |app|\n app.call(:request => request, \n :response => response, \n :settings => settings,\n :logger => logger)\n end\n\n return true\n rescue StandardError => e\n if settings.service.raise_exceptions\n raise\n else\n logger.info(\"APP ERROR\") { e.inspect }\n logger.debug(\"APP ERROR\") { \"#{e.inspect}\\n\" + e.backtrace.join(\"\\n \") }\n\n return false\n end\n end\n end\nend\n```\n\nIf you trace the execution path through this method, you'll find that there are three possible outcomes. If everything worked as expected, the method simply returns true. However, if an exception is raised by one of the applications, then the `raise_exceptions` configuration setting is used to determine whether to simply re-raise the exception or log the error and return false.\n\nThe reason `Newman::Server#process_request` is implemented in this somewhat awkward way is that it is necessary to let the application developer determine whether or not application errors should crash the server. Generally speaking, this would be a bad behavior in production, because it means that a single fault in an edge case of a single feature could halt a whole service that is otherwise working as expected. However, when it comes to writing tests, it might be nice for applications to raise their exceptions rather than quietly writing stack traces to a log file and moving on. This pair of competing concerns explains why the `raise_exceptions` configuration option exists, even if it leads to ugly implementation code.\n\nWhile `Newman::Server#process_request` does a good job of handling application errors, there are a range of failures that can happen as a result of server operations as well. This means that `Newman::Server#tick` needs to implement its own exception handling and logging, as shown below:\n\n```ruby\nmodule Newman\n class Server\n def tick \n mailer.messages.each do |request| \n response = mailer.new_message(:to => request.from, \n :from => settings.service.default_sender)\n\n process_request(request, response) && response.deliver\n end\n rescue StandardError => e\n logger.fatal(\"SERVER ERROR\") { \"#{e.inspect}\\n\" + e.backtrace.join(\"\\n \") }\n raise\n end\n end\nend\n```\n\nWhile it may be possible to recover from some of the errors that occur at the server level, there are many problems which simply cannot be recovered from automatically. For this reason, `Newman::Server#tick` always re-raises the exceptions it encounters after logging them as fatal errors. While implementing this method in such a conservative way helps shield against dangerous failures, it does not completely prevent them from occurring. Sadly, that is a lesson I ended up learning the hard way.\n\n### Lesson 3: Reduce the impact of catastrophic failures. \n\nA few days before this article was published, I accidentally introduced an\ninfinite send/receive loop into the experimental Newman-based mailing list system [MailWhale](https://github.com/mendicant-original/mail_whale). I caught the problem right away, but not before my email provider banned me for 1 hour for exceeding my send quota. In the few minutes of chaos before I figured out what was going wrong, there was a window of time in which any incoming emails would simply be dropped, resulting in data loss.\n\nIt's painful to imagine what would have happened if this failure occured while someone wasn't actively babysitting the server. While the process was crashing with a `Net::SMTPFatalError` each time cron ran it, this happened after reading all incoming mail. As a result, the incoming mail would get dropped from the inbox without any response, failing silently. Once the quota was lifted, a single email would cause the server to start thrashing again, eventually leading to a permanent ban. In addition to these problems, anyone using the mailing list would be bombarded with at least a few duplicate emails before the quota kicked in each time. Although I was fortunate to not live out this scenario, the mere thought of it sends chills down my spine.\n\nWhile the infinite loop I introduced could probably be avoided by doing some simple checks in Newman, the problem of the server failing repeatedly is a general defect that could cause all sorts of different problems down the line. To solve this problem, I've implemented a simplified version of the [circuit breaker](http://en.wikipedia.org/wiki/Circuit_breaker_design_pattern) pattern in MailWhale, as shown below:\n\n```ruby\nrequire \"fileutils\"\n\n# unimportant details omitted...\n\nbegin\n if File.exists?(\"server.lock\")\n abort(\"Server is locked because of an unclean shutdown. Check \"+\n \"the logs to see what went wrong, and delete server.lock \"+\n \"if the problem has been resolved\") \n end\n\n server.tick\nrescue Exception # used to match *all* exceptions\n FileUtils.touch(\"server.lock\")\n raise \nend\n```\n\nWith this small change, any exception raised by the server will cause a lock file to be written out to disk, which will then be detected the next time the server runs. As long as the `server.lock` file is present, the server will immediately shut itself down rather than continuing on with its processing. This forces someone (or some other automated process) to intervene in order for the server to resume normal operations. As a result, repeated failure is a whole lot less likely to occur. \n\nIf this circuit breaker were in place when I triggered the original infinite loop, I would have still exceeded my quota, but the only data loss would be the first request the server failed to respond to. All email that was sent in the interim would remain in the inbox until the problem was fixed, and there would be no chance that the server would continue to thrash without someone noticing that an unclean shutdown had occurred. This is clearly a better behavior, and perhaps this is how things should have been implemented in the first place.\n\nOf course, we now have the problem that this code is a bit too aggressive. There are likely to be many kinds of failures which are transient in nature, and shutting down the server and hard-locking it like this feels overkill for those scenarios. However, I am gradually learning that it is better to whitelist things than blacklist them when you can't easily enumerate what can possibly go wrong. For that reason I've chosen to go with this extremely conservative solution, but I will need to put this technique through its paces a bit before I can decide whether it is really the right approach. \n\n### Reflections\n\nI originally planned to cover many more lessons in this article, but the more I worked on it, the more I realized my own lack of experience in producing truly robust software. When it comes to email, it's like the entire problem space is one big fuzz test: there seems to be an infinite amount of ways for things to crash and burn.\n\nIn addition to the few issues I have already outlined, Newman is going to need to jump many more hurdles before it can be considered stable. In particular, I need to sort out the following problems:\n\n* Sometimes connections via IMAP can hang indefinitely, so some sort of timeout logic needs to be introduced. To deal with this, I'm thinking of looking into the [retriable](https://github.com/kamui/retriable) gem.\n\n* In one of my test runs of our simple ping/pong application, I ended up causing newman to reply to a Gmail mailer daemon, which caused a request/response loop to occur. Thankfully, Gmail's daemon gave up after a few tries, but if it didn't I would have ended up melting my inbox again. This means that Newman will need some way to deal with bounced emails. We've looked at some options for this, but most involve some pretty messy heuristics that make the cure look worse than the disease.\n\n* Currently it is relatively straightforward to write automated tests to reproduce known issues, but very hard for me to come up with realistic test scenarios in a proactive way. This means that while we can shore up Newman's stability over time in this fashion, we'll always be trailing behind on the problems we haven't encountered yet. I need to look into whether there are some email-based acid tests I can run the server against.\n\n* There is still a great deal of ugliness / fragility in the way Newman does its exception handling. The techniques I've shown in this article are meant to be considered a rough starting point, not a set of best practices. I plan to re-read Avdi Grimm's [Exceptional Ruby](http://exceptionalruby.com/) and see what ideas I can apply from it. When I first read that book I thought many of the techniques it recommended were overkill for day to day Ruby applications, but several of them may be just what Newman needs.\n\nThe bad news is that all of the above problems seem challenging enough to deal with, but they're likely to be just the first set of roadblocks on the highway to the danger zone. There are still a lot of unknown-unknowns that may get in my way. The good news is that because I can take my time while working on this project, the uncertainty of things is part of what makes this a fun problem to work on.\n\nHave you ever had a similar experience of coding in a dangerous and poorly-defined environment before? If so, I'd love to hear your story, as well as any advice you might have for me.\n"
},
{
"path": "articles/v3/README.md",
"content": "These articles are from Practicing Ruby's third volume, which ran from\n2012-01-03 to 2012-03-13. The manuscripts in this folder correspond to the\nfollowing articles on practicingruby.com:\n\n* [Issue 3.1: The qualities of great software](https://practicingruby.com/articles/shared/hhgcloeuoega) (2012.01.03)\n* [Issue 3.2: Patterns for building excellent examples](https://practicingruby.com/articles/shared/wfsyjrkiwidq) (2012.01.10)\n* [Issue 3.3: Exploring the depths of a Turing tarpit](https://practicingruby.com/articles/shared/bwgflabwncjv) (2012.01.17)\n* [Issue 3.4: Climbing the spiral staircase of refactoring](https://practicingruby.com/articles/shared/ndejcopauhne) (2012.01.25)\n* [Issue 3.5: Framework design and implementation, Part 1](https://practicingruby.com/articles/shared/rtzdzdwwzfxm) (2012.02.02)\n* [Issue 3.6: Framework design and implementation, Part 2](https://practicingruby.com/articles/shared/exckjeiytsaw) (2012.02.08)\n* [Issue 3.7: Criteria for disciplined inheritance, Part 1](https://practicingruby.com/articles/shared/uvgdkprzmoqf) (2012.02.15)\n* [Issue 3.8: Criteria for disciplined inheritance, Part 2](https://practicingruby.com/articles/shared/lxgettcjiggh) (2012.02.21)\n* [Issue 3.9: Using games to practice domain modeling](https://practicingruby.com/articles/shared/ihlfxtwgquny) (2012.02.28)\n* [Issue 3.10: Lessons learned from coding in the danger zone](https://practicingruby.com/articles/shared/lwvegkvhalqr) (2012.03.13)\n\nIf you enjoy what you read here, please subscribe to [Practicing Ruby](http://practicingruby.com). These articles would not exist without the support of our paid subscribers.\n"
},
{
"path": "articles/v4/001-testing-costs-benefits.md",
"content": "Over the last several years, Ruby programmers have gained a reputation of being\n*test obsessed* -- a designation that many of our community members consider to\nbe a badge of honor. While I share their enthusiasm to some extent, I can't help but notice\nhow dangerous it is to treat any single methodology as if it were a panacea.\n\nOur unchecked passion about [test-driven\ndevelopment](http://en.wikipedia.org/wiki/Test-driven_development) (TDD) has paved the way for deeply\ndogmatic thinking to become our cultural norm. As a result, many vocal members\nof our community have oversold the benefits of test-driven development \nwhile downplaying or outright ignoring some of its costs. While I don't doubt\nthe good intentions of those who have advocated TDD in this\nway, I feel strongly that this tendency to play fast and loose with very complex\nideas ends up generating more heat than light.\n\nTo truly evaluate the impact that TDD can have on our work, we need to go \nbeyond the anecdotes of our community leaders and seek answers to \ntwo important questions:\n\n> 1) What evidence-based arguments are there for using TDD? \n\n> 2) How can we evaluate the costs and benefits of TDD in our own work?\n\nIn this article, I will address both of these questions and share with you my\nplans to investigate the true costs and benefits of TDD in a more rigorous and\nintrospective way than I have done in the past. My hope is that by considering a\nbroad spectrum of concerns with a fair amount of precision, I will be able to\nshare relevant experiences that may help you challenge and test your own \nassumptions about test-driven development.\n\n### What evidence-based arguments are there for using TDD? \n\nBefore publishing this article, I conducted a survey that collected thoughts\nfrom Practicing Ruby readers about the costs and benefits of test-driven\ndevelopment that they have personally experienced. Over 50 individuals responded, and\nas you might expect there was a good deal of diversity in replies. However, the\nfollowing common assumptions about TDD stood out:\n\n```diff\n+ Increased confidence in developers working on test-driven codebases\n+ Increased protection from defects, especially regressions\n+ Better code quality (in particular, less coupling and higher cohesion)\n+ Tests as a replacement/supplement to other forms of documentation\n+ Improved maintainability and changeability of codebases\n+ Ability to refactor without fear of breaking things\n+ Ability of tests to act as a \"living specification\" of expected behavior\n+ Earlier detection of misunderstandings/ambiguities in requirements\n+ Smaller production codebases with more simple designs\n+ Easier detection of flaws in the interactions between objects\n+ Reduced need for manual testing\n+ Faster feedback loop for discovering whether an implementation is correct\n- Slower per-feature development work because tests take a lot of time to write\n- Steep learning curve due to so many different testing tools / methodologies\n- Increased cost of test maintenance as projects get larger\n- Some time wasted on fixing \"brittle tests\"\n- Effectiveness is highly dependent on experience/discipline of dev team\n- Difficulty figuring out where to get started on new projects\n- Reduced ability to quickly produce quick and dirty prototypes\n- Difficulty in evaluating how much time TDD costs vs. how much it saves\n- Reduced productivity due to slow test runs\n- High setup costs\n```\n\nBefore conducting this survey, I compiled [my own list of\nassumptions](https://gist.github.com/2277788) about test-driven \ndevelopment, and I was initially relieved to see that there was a high degree of\noverlap between my intuition and the experiences that Practicing Ruby \nreaders had reported on. However, my hopes of finding some solid ground to stand\non were shattered when I realized that virtually all of these claims did not have\nany conclusive empirical evidence to support them.\n\nSearching the web for answers, I stumbled across a great [three-part\narticle](http://scrumology.com/the-benefits-of-tdd-are-neither-clear-nor-are-they-immediately-apparent/)\n called \"The benefits of TDD are neither clear nor are they immediately\napparent\", which presents a fairly convincing argument that we don't know as\nmuch about the effect of TDD on our craft as we think we do. The whole article is\nworth reading, but this paragraph in [part\n3](http://scrumology.com/the-benefits-of-tdd-why-tdd-part-3/) really grabbed my\nattention:\n\n> Eighteen months ago, I would have said that TDD was a slam dunk. Now that I’ve taken the time to look at the papers more closely … and actually read more than just the introduction and conclusion … I would say that the only honest conclusion is that TDD results in more tests and by implication, fewer defects. Any other conclusions such as better design, better APIs, simpler design, lower complexity, increased productivity, more maintainable code etc., are simply not supported.\n\nThroughout the article, the author emphasizes that he believes in the value of\nTDD and seems to think that the inconsistency of rigor and quality in the\nstudies at least partially explain why their results do not mirror the \nexpectations of practitioners. He even offers some standards for what he \nbelieves would make for more reliable studies on TDD:\n\n> My off-the-top-of-my-head list of criteria for such a study, includes (a) a multi year study with a minimum of 3 years consecutive years (b) a study of several teams (c) team sizes must be 7 (+/-2) team members and have (d) at least 4 full time developers. Finally, (e) it needs to be a study of a product in production, as opposed to a study based on student exercises. Given such as study it would be difficult to argue their conclusions, whatever they be.\n\nHis points (c) and (d) about team size seem subject to debate, but it is fair\nto say that studies should at least consider many different team sizes as\nopposed to focusing on individual developers exclusively. All other points he\nmakes seem essential to ensuring that results remain tied to reality, but he\ngoes on to conclude that his requirements are so complicated and costly to \nimplement that it could explain why all existing studies fall short of this gold\nstandard.\n\nIntrigued by this article, I went on to look into whether there were other, more\nauthoritative sources of information about the overall findings of research on\ntest-driven development. As luck would have it, the O'Reilly book on\nevidence-based software engineering ([Making\nSoftware](http://www.amazon.com/Making-Software-Really-Works-Believe/dp/0596808321)) had a chapter on this\ntopic called \"How effective is test-driven development?\" which followed a\nsimilar story arc.\n\nIn this chapter, five researchers present the result of their systematic review of \nquantitative studies on test driven development. After analyzing what published \nliterature says about internal quality, external quality, productivity, \nand correctness testing, the researchers found some evidence that both \ncorrectness testing and external quality are improved through TDD. However, \nupon limiting the scope to well-defined studies only, the positive effect \non external quality disappears, and even the effect on correctness \ntesting weakens significantly. In other words, their conclusion matched the\nconclusions of the previously mentioned article: *there is simply not a whole lot of\nscience supporting our feverish advocacy of TDD and its benefits.*\n\nWhile the lack of rigorous and conclusive evidence is disconcerting, it is not \nnecessarily a sign that our perception of the costs and benefits of \nTDD is invalid. Instead, we should treat these findings as an invitation to\nslow down and look at our own decision making process in a more careful and\nintrospective way. \n\n### How can we evaluate the costs and benefits of TDD in our own work?\n\nBecause there are very few evidence-supported generalizations that can be made\nabout test-driven development, we each have the responsibility to discover for\nourselves what effects the red-green-refactor cycle truly has on our work. But\nbased on my personal experience, many of us have a long way to go before we can\neven begin to answer this question.\n\nIn the process of preparing this article, I ended up identifying three\nguidelines that I feel are essential for any sort of introspective evaluation. I\nhave listed them below, along with some brief notes on how I have failed\nmiserably at meeting these preconditions when it comes to analyzing TDD.\n\n---\n\n**1) We must be aware of our assumptions, and be willing to test them.**\n\n_How I failed to do this:_ As someone who learned TDD primarily because other smart people\ntold me it was the right way to do things, my personal opinions about testing\nwere developed reactively rather than proactively. As a result, I have ignored certain \nobservations and amplified others to fit a particular mental model that is\nmostly informed by gut reactions rather than reasoned choices.\n\n**2) We must be aware of our limitations and try to overcome them.**\n\n_How I failed to do this:_ My mixed feelings towards TDD are in part due to my\nown lack of effort to fully understand the methodology. \nWhile I may have done enough formal practice to have some basic intuitive sense of what\nthe red-green-refactor cycle is like, I have never been able to sustain \na pure TDD workflow over the entire lifetime of any reasonably complex\nproject that I have worked on. As a result, it is likely that I have been \nblaming testing tools and methodologies for my some of my own deficiencies.\n\n**3) We must be continuously mindful of context and avoid over-generalization.**\n\n_How I failed to do this:_ I have always been irked by the lack of sufficient context in literature about\ntest-driven development, but I have found myself guilty of committing a similar\ncrime on numerous occasions. Even when I have tried to use specific examples to support\nmy arguments, I have often failed to consider that my working environment is very\ndifferent than that of most programmers. As a result, I have made more than few\nsweeping generalizations which could be invalid at best and misleading at worst.\n\n---\n\nIf I had to guess why I approached TDD in such a haphazard way despite my\ntendency to treat other areas of software development with a lot more \ncareful attention, I would say it was a combination of immaturity and a \ndeeply overcommitted work schedule. When I first learned Ruby in 2004, I \nstudied just enough about software testing and the TDD workflow to get \nby, and then after that only brushed up on my software testing skills \nwhen it was absolutely essential to do so. There was simply too much to learn\nabout and not enough time, and so I never ended up giving TDD as much attention \nas it might have deserved.\n\nLike most things that get learned in this fashion, my knowledge of software\ntesting in the test-driven style is full of gaping holes and \ndark corners. Until recently this is something I have always been able to work\naround, but my role as a teacher has forced me to identify this as a real weak \nspot of mine that needs to be dealt with. \n\n### Looking at TDD from a fresh perspective\n\nRelearning the fundamentals of test-driven development is the only way \nI am ever going to come up with a coherent explanation for [my \nassumptions](https://gist.github.com/2277788) about the costs and benefits of this kind of workflow, \nand is also the only way that I will be able to break free from various \nmisconceptions that I have been carrying around for the better part of \na decade.\n\nFor a period of 90 days from 2012-04-10 to 2012-07-09, I plan to follow \ndisciplined TDD practices as much as possible. The exact process I want \nto adopt is reflected in the handy-dandy flow chart shown below:\n\n\n\n
\n\nThis is a workflow that I am already quite familiar with and have practiced\nbefore, but the difference this time around is that I'm going to avoid cutting\ncorners. In the past, I have usually started projects by spiking a rough\nprototype before settling into a TDD workflow, and that may have dampened the\neffect that writing tests up front could have had on my early design process in\nthose projects. I have also practicing refactoring in the large rather than the\nsmall fairly often, favoring a Red-Green-Red-Green-...-Red-Green-Refactor\npattern which almost certainly lead to more brittle tests and implementations\nthan I might have been able to come up with if I were more disciplined.\nThroughout this three month trial period, I plan to think long and hard before\nmaking any deviations from standard practice, and will be sure to note whenever\nI do so.\n\nThe benefit of revisiting this methodology as an experienced developer is that I\nhave a whole lot more confidence in my ability to be diligent in my efforts. In\nparticular, I plan to take careful notes during each and every coding session\nabout my TDD struggles and triumphs, which I will associate with particular\nchangesets on particular projects. Before writing this article I did a test run\nof how this might work out, and you can \n[check out these notes](https://gist.github.com/2286918) to get a sense of what \nI am shooting for. I think the [github compare\nview](https://github.com/sandal/puzzlenode-solutions/compare/9070...3b79) will \nreally come in handy for this kind of note-taking, as it will allow me to track \nmy progress with a high degree of precision. \n\nI don't plan to simply use these notes for subjectively analyzing my own\nprogress, but also expect to use them as a way to seek out advice and help from\nmy friends who seem to have strongly integrated test-driven development into their\nworking practices. Having particular code samples to share along with some additional \ncontext will go a long way towards helping me ask the right kinds of \nquestions that will move me forward. Each time I reach a stumbling point or\ndiscover a pattern that is influencing my work (for better or for worse), I will\nrequest some feedback from someone who might be able to help. When I was\nlearning TDD the first time around I might have avoided asking \"stupid\nquestions\" as a way to hide my ignorance, but this time I am intentionally\ntrying to expose my weaknesses so that they can be dealt with.\n\nAfter this 90 day period of disciplined study and practice of test-driven\ndevelopment, I will collect my notes and attempt to summarize my findings.\nIf I have enough interesting results to share, I will publish them in Issue 4.12\nof Practicing Ruby towards the end of July 2012. At that time, I will also\nattempt to take a slightly more informed guess at the \"cost and benefits\"\nquestion that lead me to write this article in the first place, and will comment\non how this disciplined period of practice has influenced my assumptions about\nTDD.\n\n### Predictions about what will be discovered\n\nWhile certain things are best left to be a mystery, there are a few predictions\ncan make about the outcomes of this project. These are mostly \"just\nfor fun\", but also may help reveal some of my biases and expectations:\n\n* I expect that I will reverse my position on several criticisms of test-driven\n development as I learn more about practicing it properly.\n\n* I expect that I will understand more of the claims that I feel are either\n overstated or lacking in context, and will either be able a more balanced\n view of them or meaningfully express my reservations about them.\n\n* I expect that I will stop exclusively doing pure test-driven development as\n soon as this trial period is over, but think it is very likely that I will\n use TDD more often and more skillfully in the future.\n\n* I expect to be just as frustrated about the extra work involved in TDD\n by the end of this study as I am now.\n\n* I expect that simply by measuring my progress and reflecting on it, that I\n will learn a lot of interesting things that aren't related to TDD at all,\n and that will help me write better Practicing Ruby articles!\n\nI will do my best not to allow these predictions to become self-fulfilling\nprophecies and just go with the flow, but I feel it is important to expose \nthe lens that I will be viewing my experiences through.\n\n### Limitations of this method of study\n\nThe method I am using to reflect on my studies is to some extent a legitimate\nform of qualitative research that may be useful for more than just improving\nmy own skillset. I am essentially conducting a diary study, which is \nthe [same technique that Donald Knuth used](http://books.google.com/books?id=DxuGi5h2-HEC&lpg=PA58&dq=Reading%20Qualitative%20Research%20Knuth&pg=PA58#v=onepage&q=Reading%20Qualitative%20Research%20Knuth&f=false)\nin an attempt to categorize the different kinds of errors found in TeX. This \ntechnique is also used in marketing and usability\nresearch, and can provide interesting insights into the experiences of\nindividuals with sufficient context to be analyzed in a fairly rigorous way.\nHowever, I am not a scientist and this is not a scientific study, and so there\nare a ton of limitations can threaten the validity of any claims made about \nthe results of this project.\n\nThe first and most obvious limitation is that this is a self-study, and that I\nam already chock full of my own assumptions and biases. My main goal is to learn\nmore about TDD and come up with better reasons for the decisions I make about\nhow I practice software testing, but it is impossible for me to wipe the slate\ncompletely clean and serve as an objective source of information on this topic.\n\nOn top of this, I will be discussing things entirely in terms of my experiences\nand won't have many objective measures to work with. My hope is that tagging\nmy notes with links back to particular changesets will make it possibly to apply \nsome quantitative measures after this study is completed, but it is hard to say\nwhether that will be feasible or whether it would even mean anything if I\nattempted to do that. Without hard numbers, my results will not be\ndirectly comparable to anyone else's nor can it say anything about the average\ndeveloper's experience.\n\nLastly, when I look back on my notes from the 90 day period, it may be hard for\nme to reestablish the context of the early days of the study. This means that my\nfinal report may be strongly biased by whatever ends up happening towards the \nend of the trial period. While I expect that I will be able to make some high-level \ncomparisons across the whole time period, I will not be able to precisely \ncompare my experiences on day 5 with my experiences on day 85 even if I take\nvery detailed notes. This may cause some important insights to get lost in the\nshuffle.\n\nMy hope is that by staying communicative during this study and by sharing most\nor all of my raw data (code, notes, etc.), the effects of these limitations will\nbe reduced so that others can still gain something useful from my\nefforts. At the very least, this transparency will allow individuals \nto decide for themselves to what extent my conclusions match up with my\nevidence, and whether my results are relevant to other contexts.\n\n### Some things you can do to help me\n\nOne thing I know about Practicing Ruby readers is that you folks really enjoy \nimproving the craft of software development. That is the reason why I decided \nto announce my plans for this study via an article here rather than \nsomewhere else. If you would like to support this project,\nthere are a few ways you can contribute.\n\n**If you have a few seconds to spare:** You can spread the word about this\nproject by sharing this article with your friends and colleagues. This will help\nme make sure to get adequate critical review from the community, which is a key\npart of the improvement process. To create a share link, just click the handy\ndandy robot down in the bottom right corner of the screen.\n\n**If you have a few minutes to spare:** You can leave a comment sharing your\nthoughts on this article as well as any questions or suggestions you might have\nfor me. I take all reader feedback to heart, and comments are one of the best\nways that you can support my work on these articles. \n\n**If you have a few minutes to spare each week:** You can subscribe to the\n[mendicant-research](http://lists.rubymendicant.org/listinfo.cgi/mendicant-research-rubymendicant.org)\nmailing list, where I plan to post my questions about TDD as\nI study, as well as any interesting problems I run into or helpful learning\nresources I end up using. I am also going to invite a few folks from the Ruby\ncommunity that I think have specific skills that will help me with this study,\nbut I feel that every practicing Rubyist could be a meaningful contributor to\nthese discussions.\n\n**If you have a large amount of free time:** You can try to do this study along with me.\nI can't promise that I'll have time during the 90 day period to regularly review\nyour progress, but I can definitely help you get set up and also would love to\ncompare notes at the end of the trial period. If this is something that\ninterests you, please post to the\n[mendicant-research](http://lists.rubymendicant.org/listinfo.cgi/mendicant-research-rubymendicant.org) mailing list and I'll provide additional details.\n\nAny little bit of effort you spend on helping me make this project better will\nabsolutely be appreciated! Our readers are what make this journal what it is, I\njust work here. :wink:\n"
},
{
"path": "articles/v4/002-moving-beyond-the-first-release.md",
"content": "In Issue 2.10, I described the path I took [from requirements discovery to\nrelease](http://practicingruby.com/articles/10) for a small game I created,\nincluding various corners I had cut to get an initial release out the door.\nWhile that article was very well received, it left an important question\nunanswered: **How do you transform a prototype into a product?**\n\nTo answer that question, I have built a new little game for us to play with. But\nthis time around, rather than outlining the path from the idea phase to a basic\nproof-of-concept, I will instead focus on what it takes to turn a rough prototype into a\ngame that might actually be fun to play. Because that is a pretty big topic, I\nhave split this article into two parts. This first part describes the initial\nprototype and my plans for improving it, and the second part will reflect on the\nchallenges I encountered while trying to make those improvements. \n\n### Blind: A Minesweeper-inspired game with a massive twist\n\nThe game I created for this article is based on a very simple concept:\nNavigating a mine field in search of an exit. While the premise is a bit\ndifferent from the classic Minesweeper game, the basic idea of carefully trying\nto figure out where the mines are without getting yourself blown up is\npreserved. The graphic below gives a rough approximation what the structure of\nthe Blind world is like:\n\n\n![](\"http://i.imgur.com/lvRaj.png\")
\n
\n\nThe game starts with the player positioned at the center of the safe zone, a\nbuffer zone that exists to prevent mines from being detonated as soon as the\nplayer spawns. Within the mine field itself, a large quantity of mines are\nrandomly spawned and there is no way for the player to know in advance exactly\nwhere they will be. This area is also where the exit gets randomly positioned,\nforcing the player to navigate the mine field to find their way out of the\nlevel. \n\nThere is one way to win and two ways to lose a game of Blind. As you might imagine,\ngetting too close to a mine is one of the ways to lose; the other is to wander off\nthe edge of the map into deep space. To do this, the player needs to ignore a\npretty obvious warning sign, but this loss condition helps make sure the player\nstays within a well defined perimeter throughout the game. The only way to win\nis to find the exit, which can be easy or hard depending on the positions and\nquantities of mines in the minefield.\n\nFrom this description, you might be imagining some sort of simple\ntwo-dimensional arcade game, complete with a hard to control little ship\n(perhaps similar to Asteroids), but as I mentioned before, this game has a\nmassive twist: It has no graphics at all! Instead, it relies entirely on\n[positional audio](http://en.wikipedia.org/wiki/3D_audio_effect) to represent its world. To get a taste of what the gameplay is like, grab a pair of headphones and play the video shown below.\n\n> **NOTE:** I have turned on the debugging output so that this video can be played\non mute by those who either dislike loud noises or are reading this article\nin a place where they can't play audio. However, it is worth noting that the\ngame is designed to be played without any visual feedback at all,\nand that while testing it I have typically played with my eyes closed! \n\n\n\n\n
\n\nThe \"audio only\" twist is enough to inject some excitement into a very boring\ngame concept, but right now the game is not particularly enjoyable to play because it is poorly\nbalanced. I will talk about ideas I have for improving that later, but for now\nwe should move on to discuss the implementation code that got me to the point\nwhere I could make this short video.\n\n### An overview of Blind's initial implementation\n\nI was originally going to use Blind as a demonstration how to implement\nlayered applications that have clean separation between their business logic and\nUI code, so the code quality is better than what I usually start out with in proof-of-concept projects. \nWhile you might be able to read the [full implementation]() without too much effort, \nthe following outline will give you an overview of how the codebase is structured without \ndragging you too far out into the weeds.\n\n**[Blind::Game]() triggers events in response to the player's movements**\n\nOne decision I made early on in the process of building Blind was that I wanted\nthe main `Blind::Game` object to be based on a publish/subscribe model. Whenever\nI have worked on games in the past, I have always struggled to figure out how to\nencapsulate the rules without writing extremely brittle code, and this time\naround I think I found a nice happy medium. \n\nIf you look at the `Game#move`\nmethod below, you can see that while it captures all the different kinds of\nevents that can occur within the game, it leaves it up to someone else to\ndetermine what should happen in response to those events. This flexibility will\nhopefully come in handy as the game rules evolve over time.\n\n```ruby\nmodule Blind\n class Game\n # ...\n \n def move(dx, dy)\n x,y = world.current_position.to_a\n\n r1 = world.current_region\n r2 = world.move_to(x + dx, y + dy)\n\n if r1 != r2\n broadcast_event(:leave_region, r1)\n broadcast_event(:enter_region, r2)\n end\n\n mines = world.mine_positions\n\n if mines.find { |e| world.distance(e) < MINE_DETONATION_RANGE }\n broadcast_event(:mine_detonated)\n end\n\n if world.distance(world.exit_position) < EXIT_ACTIVATION_RANGE\n broadcast_event(:exit_located)\n end\n end\n end\nend\n```\n\nWhile I will show some more examples of this event system when we discuss Blind's\npresentation layer, the following tests hint at how the publish/subscribe\nsystem works. In a nutshell, the `Game#on_event` method sets up callbacks that get executed whenever `Game#broadcast_event` is called with a matching key.\n\n```ruby\n it \"must trigger an event when a mine is detonated\" do\n detonated = false\n\n game.on_event(:mine_detonated) { detonated = true }\n\n mine = world.mine_positions.first\n\n game.move(mine.x - Blind::Game::MINE_DETONATION_RANGE, mine.y)\n\n refute detonated, \"should not be detonated before \" +\n \"player is in the mine's range\"\n\n game.move(1, 0)\n\n assert detonated, \"should detonate when player is in the mine's range\"\n end\n```\n\nWhile I am fairly happy with the implementation of `Blind::Game`, both the\nimplementation code and the tests hint at the current dependency on a pair of\nmagic numbers stored in the `EXIT_ACTIVATION_RANGE` and\n`MINE_DETONATION_RANGE` constants. This was done purely for the sake of\nconvenience, but I imagine they will need to be parameterized at some point in\nfuture to make the game rules more customizable.\n\nAnother potential pitfall of this design is that its open-ended flexibility may\nprove to be a double edged sword. I will be looking at this closely as I\nincrease the complexity of the game rules, but what I want to avoid is having a\nbig chunk of the game logic spill over into the client code unnecessarily. This\nwasn't a major concern with the initial implementation, but it is definitely\nsomething to look out for later.\n\nLastly, the event system as it currently is implemented assumes that there is a\nsingle subscriber for each published event. This is an arbitrary limitation that\ncan easily be lifted, but right now is a limitation to be aware that may\nneed to be dealt with later.\n\n**[Blind::World]() models the layout of the game world**\n\nIn practice, the `Blind::World` class has been working out reasonable well.\nHowever, it is a bit of a structural method due to its broad spectrum of\nresponsibilities. In particular, `Blind::World` can be used to:\n\n* Track where the mines, exit, and player are in the world\n* Compute the distance between the player and any other object in the world\n* Determine the region the player is currently in\n* Move the player to an arbitrary location in the world\n* Generate random locations within the minefield\n\nWhile you might be able to group some of these concepts together, it is clear\nthat when taken as a set, these features fail to represent a single cohesive\nobject. In particular, the methods provided by this object don't have much in\ncommon when it comes to the level of abstraction they operate at. For example,\nthe `World#distance` method and the `World#random_minefield_position` have so\nlittle in common that it is hard to look at them side by side without getting\na bit of a headache.\n\n```ruby\nmodule Blind\n class World\n\n # simple delegation to the underlying Point object\n def distance(other)\n current_position.distance(other)\n end\n\n # ...\n\n # a non-trivial trigonometric function\n def random_minefield_position\n angle = rand(0..2*Math::PI)\n length = rand(MINE_FIELD_RANGE)\n\n x = length*Math.cos(angle)\n y = length*Math.sin(angle)\n\n Blind::Point.new(x.to_i,y.to_i)\n end\n end\nend\n```\n\nAnother major limitation of the `Blind::World` object is that similar to \n`Blind::Game`, it is chock full of magic numbers representing the sizes\nand position of the various regions. While it might be reasonable to \nprovide sensible defaults, locking these values down to exact numbers\nwill make world customization harder. I imagine this is something \nI will need to refactor sooner rather than later if I want to make\nthe game more interesting to play.\n\nWith all of these structural problems, you might expect the `Blind::World`\nobject to be quite cumbersome to work with, but so far I haven't really\nhad problems with it. I expect that the next round of improvements to Blind \nwill change that, but I prefer to refactor based on needs rather than\ngut feelings about what a good design might look like.\n\n**[Blind::Point]() implements a simple generic point structure**\n\nThis object is simple enough where you can read its full implementation before we discuss it further:\n\n```ruby\nrequire \"matrix\"\n\nmodule Blind\n class Point\n def initialize(x,y)\n @data = Vector[x,y]\n end\n\n def x\n data[0]\n end\n\n def y\n data[1]\n end\n\n def distance(other)\n (self.data - other.data).r\n end\n\n def ==(other)\n distance(other).zero?\n end\n\n def to_a\n [x,y]\n end\n\n def to_s\n \"(#{x}, #{y})\"\n end\n\n protected\n \n attr_reader :data\n end\nend\n```\n\nThe idea of having a generic class for representing points makes perfect sense for this game, because we need to do point math all over the place. I like that `Blind::Point` is an immutable object, because it reduces the possibility for weird corruptions to happen. However, the current implementation is not really putting the dependency on Ruby's `Vector` object to good use. I had originally built the class this way because I expected to be doing a lot of calculations, but then somehow got in the habit of manually doing the math on the individual components. \n\nI assume that as Blind gets more complex, I will do more and more point math, and that will inspire me to delegate a few more operations to the `Vector` object. I may also refactor several of Blind's API calls to take `Blind::Point` objects rather than explicit x and y arguments, which would encourage better use of this object. This kind of inconsistency is common in the early stages any project, because the boundary lines between the various objects in the system have not quite solidified yet. The good news is that these tensions tend to work themselves out gradually over time.\n\n\n**[Blind::UI::JukeBox]() is responsible for constructing the various sounds used in the game**\n\nThe highly dynamic nature of the sounds in Blind made it worthwhile to introduce a small abstraction for loading and manipulating audio files. The example below demonstrates how both simple and complex sounds can be created by `Blind::UI::JukeBox`:\n\n```ruby\nmodule Blind\n module UI\n JukeBox = Object.new\n \n class << JukeBox \n def explosion\n new_sound(\"grenade\")\n end\n\n def phone(position)\n new_sound(\"telephone\") do |s|\n s.pos = [position.x, position.y, 0]\n s.looping = true\n\n s.play\n end\n end\n\n def mines(positions)\n step = 0\n step_size = 1/positions.count.to_f\n\n positions.map do |pos|\n new_sound(\"beep\") do |s|\n s.pos = [pos.x, pos.y, 1]\n s.looping = true\n s.pitch = step + rand(step_size)\n\n s.play\n\n step += step_size\n end\n end\n end\n\n # ... several other sounds\n end\n end\nend\n```\n\nOne thing I like about this code is that it gave me a chance to use one of my favorite techniques: hiding ugly code via block-based APIs. While it looks pretty, the `JukeBox#new_sound` method is in reality nothing more than a tiny bit of syntactic sugar built on top of the underlying `Ray::Sound` object:\n\n```ruby\nmodule Blind\n module UI\n JukeBox = Object.new\n \n class << JukeBox\n def new_sound(name) \n filename = \"#{File.dirname(__FILE__)}/../../../data/#{name}.wav\"\n \n Ray::Sound.new(filename).tap do |s|\n yield s if block_given?\n end\n end\n end\n end\nend\n```\n\nWhen I had first built this method, it was designed to store the `Ray::Sound` objects in a hash that was keyed by the sound name. However, I eventually ended up deciding that it'd be best to let the client determine if and how sound objects should be cached, and so this method (and the `JukeBox` object as a whole) became a bit more simple as a result of that. Of course, it does complicate things for the `Blind::UI::GamePresenter` object, which has essentially become a dumping ground for all the functionality that didn't fit well in the other components that Blind is made up of.\n\n**[Blind::UI::GamePresenter]() bridges the gap between the game logic and the\nUI**\n\nSimilar to `Blind::World`, the `Blind::UI::GamePresenter` object suffers from a\nbit of an identity crisis. On the one hand, some of its methods do look like\nsimple presentation-related features:\n\n```ruby\nmodule Blind\n module UI\n class GamePresenter\n def lose_game(message)\n silence_sounds\n\n sound = sounds[:explosion]\n sound.play\n\n self.game_over_message = message\n end\n\n def win_game(message)\n silence_sounds\n\n sound = sounds[:celebration]\n sound.play\n\n self.game_over_message = message\n end\n end\n end\nend\n```\n\nHowever, there are just as many examples of methods that seem to be tacked\non to this object that perhaps would have been better off on another object:\n\n```ruby\nmodule Blind\n module UI\n class GamePresenter\n\n # requires domain knowledge about Blind::UI::JukeBox\n def silence_sounds\n sounds.each do |name, sound|\n case name\n when :mines\n sound.each { |s| s.stop }\n else\n sound.stop\n end\n end\n end\n\n # touches every attribute provided by Blind::World\n def to_s\n \"Player position #{world.current_position}\\n\"+\n \"Region #{world.current_region}\\n\"+\n \"Mines\\n #{world.mine_positions.each_slice(5)\n .map { |e| e.join(\", \") }.join(\"\\n\")}\\n\"+\n \"Exit\\n #{world.exit_position}\"\n end\n end\n end\nend\n```\n\nAnd just for good measure, `Blind::UI::GamePresenter` implements a few\nmethods that seem to be closer to the logical layer rather than the\npresentation layer:\n\n```ruby\nmodule Blind\n module UI\n class GamePresenter\n \n # if we want to change a game rule, we'd need to update\n # the GamePresenter object. That seems a bit strange.\n def setup_events\n game.on_event(:enter_region, :danger_zone) do\n self.in_danger_zone = true\n end\n\n game.on_event(:leave_region, :danger_zone) do\n self.in_danger_zone = false\n end\n\n game.on_event(:enter_region, :deep_space) do\n lose_game(\"you drifted off into deep space! you lose!\")\n end\n\n game.on_event(:mine_detonated) do\n lose_game(\"you got blasted by a mine! you lose!\")\n end\n\n game.on_event(:exit_located) do\n win_game(\"you found the exit! you win!\")\n end\n end\n\n # this triggers a \"SURPRISE MATH ATTACK!\" as in Blind::World\n def detect_danger_zone\n if in_danger_zone\n min = Blind::World::DANGER_ZONE_RANGE.min\n max = Blind::World::DANGER_ZONE_RANGE.max\n\n sounds[:siren].volume = \n ((world.distance(world.center_position) - min) / max.to_f) * 100\n else\n sounds[:siren].volume = 0\n end\n end\n end\n end\nend\n```\n\nThis extremely messy design is a consequence of trying to make good design\ndecisions elsewhere. Whenever I was in doubt about whether something was a\nlogical concern or a presentation concern, I error on pushing the code out of\nthe domain models and into the UI. This helped the objects which implemented\npure game logic stay simple and lean, but without finding a good place for all\nthis other code to go, it ended up getting slapped together in a haphazard \n\"procedural programming with objects\" style. I am hoping to clean up this\nobject eventually, but I didn't have any brilliant ideas for how to do so\nbefore publishing this article.\n\n**The [bin/blind]() executable implements trivial UI boilerplate code**\n\nThe main benefit of the `Blind::UI::GamePresenter` class is that it makes it\npossible for the Ray-based UI code to be almost entirely logic-free. This \nleads to a very clear main program loop:\n\n```ruby\nalways do\n if game.finished?\n message = game.game_over_message\n else\n game.detect_danger_zone\n \n game.move( 0.0, -0.2) if holding?(:w)\n game.move( 0.0, 0.2) if holding?(:s)\n game.move(-0.2, 0.0) if holding?(:a)\n game.move( 0.2, 0.0) if holding?(:d)\n\n position = game.player_position\n\n Ray::Audio.pos = [position.x, position.y, 0]\n end\nend\n```\n\nMy hope is that I will be able to preserve the simplicity of this runner file\neven if I end up having to radically restructure the `Blind::UI::GamePresenter`\nobject. Because the interface used by this script is very narrow, I don't expect\nthat will be a problem.\n \n### What would make Blind a more enjoyable game?\n\nNow that I have given you a ton of context about the various strengths and\nweaknesses of Blind's codebase, we can talk about some ways to improve its\ngameplay.\n\n**Customizable world maps**\n\nWhile randomization can help make games have a higher replay value, full\nrandomization can result in a pretty inconsistent gaming experience. I would\nlike to add either a mechanism for loading in pre-defined world maps, or\nbuild a more customizable random world generator that allows to define\na bunch of different factors that affect gameplay.\n\n> **CHALLENGES:** In order to implement this feature, I am going to need to deal with reducing\nthe dependency on hard-coded numeric values throughout the system. I will\nwant to be able to control things like the size of the regions or the blast\nradius of a mine, and currently those things are not configurable at runtime.\n\n**Level-based organization**\n\nOnce I have the ability to support multiple different maps in the game, I would\nlike to be able to chain them together in a sequence to form levels. This will\nmake it possible to start with easy maps and progress to harder ones, which may\nbe a bit less of a disorienting experience for the player than the game\ncurrently is.\n\n> **CHALLENGES:** This feature does not require a ton of rework to the base\nimplementation, because we will be to easily modify the handler for the `:exit_located`\nevent and have it advance to the next level rather than end the game. However, I\nwant to make sure to re-think the `Blind::UI::GamePresenter` object before doing\nthis so that I can avoid accumulating even more logic in the wrong place.\n\n**Multiple lives per game**\n\nHaving to start over from the beginning will become more and more annoying as\nmore levels are added, so I will introduce the concept of \"lives\" in some form\nor another. I haven't decided yet exactly what the mechanics for this feature\nwill be like, but it should make death somewhat less tragic and irritating\nfor the player.\n\n> **CHALLENGES:** Similar to adding support for levels, this probably won't \nrequire many structural changes to the game's current implementation. But as\nI mentioned before, we need a better home for our event handling code moving\nforward.\n\n**Moving enemies that chase the player**\n\nHaving to slowly navigate a mine field in two dimensional space is\nnerve-wracking enough, but getting chased through one would be terrifying!\nI want to add some sort of flying baddies that will chase the player around\nthe minefield. There are lots of different ways I could possibly implement\nthis, but those are more game design questions moreso than technical questions.\n\n> **CHALLENGES:** Adding more game elements that require their position to be \nupdated will force me to think harder about the limitations of the \ncurrent `Blind::Point` class and the overall design of \nthe `Blind::World` class. I may need to end up refactoring both of\nthose objects, depending on how complex the functionality for these new\ngame elements are. Additionally, new event types and event handlers will\nbe created, and new sounds will need to be added. In other words, making\nthis change will force me to touch pretty much every object in the system.\n\n**A defense mechanism for the player**\n\nI would like to add some way for the player to protect themselves from harm.\nThis will likely be some sort of limited-use shield that is effective against\nmine detonations, flying baddies, or both.\n\n> **CHALLENGES:** Adding this functionality will require some sort of new event\ntype, and probably some modification to existing events. It will involve less\nrework than the flying baddies feature, but will require more changes to the\nexisting system than most of the other features I have proposed. The good news\nis that apart from those changes, the feature itself should be easy to\nimplement.\n\nI may not get around to implementing all of these features by the time Issue 4.3\nis published, but I will definitely tackle at least a few of them between now\nand then. Regardless of how things turn out, I think we will end up with some\ninteresting problems to discuss.\n\n### Some things you can do to help me\n\n**If you have a few seconds to spare** and know someone who has some experience\nwith game development, please share this article and ask them to get in touch\nwith me.\n\n**If you have a few minutes to spare**, please share your thoughts about this\narticle. I would be happy to hear whatever is on your mind, whether it is about\nthe game itself, its codebase, or just a gut reaction to something I have said\nin this article.\n\n**If you can spare an extra hour or two of your time**, please pull down the\ncode and try to get the game up and running, and then take a closer look\nat its implementation. Once you have done that, get in touch with me with\nany questions and suggestions about points I didn't manage to cover in\nthis article.\n\nAnd lastly, if you want to stay on top of changes as I work towards implementing\nthe code that will be discussed in Issue 4.3, please hang out in the\n**#mendicant** channel on Freenode. I may occasionally ask the folks there to test things\nfor me or give me feedback on small snippets of code while I am working.\n"
},
{
"path": "articles/v4/004-cheap-counterfeits-jekyll.md",
"content": "While it may not seem like it at first, you can learn a great deal about Ruby by building something as simple as a static website generator. Although the task itself may seem a bit dull, it provides an opportunity to practice a wide range of Ruby idioms that can be applied elsewhere whenever you need to manipulate text-based data or muck around with the filesystem. Because text and files are everywhere, this kind of practice can have a profound impact on your ability to write elegant Ruby code.\n\nUnfortunately, there are two downsides to building a static site generator as a learning exercise: it involves a fairly large time commitment, and in the end you will probably be better off using [Jekyll](http://github.com/mojombo/jekyll) rather than maintaining your own project. But don't despair, I wrote this article specifically with those two points in mind!\n\nIn order to make it easier for us to study text and file processing tricks, I broke off a small chunk of Jekyll's functionality and implemented a simplified demo app called [Jackal](http://github.com/elm-city-craftworks/jackal). Although it would be a horrible idea to attempt to use this barely functional counterfeit to maintain a blog or website, it works great as a tiny but context-rich showcase for some very handy Ruby idioms.\n\n### A brief overview of Jackal's functionality\n\nThe best way to get a feel for what Jackal can do is to [grab it from Github](https://github.com/elm-city-craftworks/jackal) and follow the instructions in the README. However, because it only implements a single feature, you should be able to get a full sense of how it works from the following overview.\n\nSimilar to Jekyll, the main purpose of Jackal is to convert Markdown-formatted posts and their metadata into HTML files. For example, suppose we have a file called **_posts/2012-05-09-tiniest-kitten.markdown** with the following contents:\n\n```\n---\ncategory: essays\ntitle: The tiniest kitten\n---\n\n# The Tiniest Kitten\n\nIs not nearly as **small** as you might think she is.\n```\n\nJackal's job is to split the metadata from the content in this file and then generate a new file called **_site/essays/2012/05/09/tiniest_kitten.html** that ends up looking like this:\n\n\n```html\nThe Tiniest Kitten
\n\nIs not nearly as small as you might think she is.
\n```\n\nIf Jackal were a real static site generator, it would support all sorts of fancy features like layouts and templates, but I found that I was able to generate enough \"teaching moments\" without those things, and so this is pretty much all there is to it. You may want to spend a few more minutes [reading its source](http://github.com/elm-city-craftworks/jackal) before moving on, but if you understand this example, you will have no trouble understanding the rest of this article.\n\nNow that you have some sense of the surrounding context, I will take you on a guided tour of through various points of interest in Jackal's implementation, highlighting the parts that illustrate generally useful techniques.\n\n### Idioms for text processing\n\nWhile working on solving this problem, I noticed a total of four text processing idioms worth mentioning.\n\n**1) Enabling multi-line mode in patterns**\n\nThe first step that Jackal (and Jekyll) need to take before further processing can be done on source files is to split the YAML-based metadata from the post's content. In Jekyll, the following code is used to split things up:\n\n```ruby\nif self.content =~ /^(---\\s*\\n.*?\\n?)^(---\\s*$\\n?)/m\n self.content = $POSTMATCH\n self.data = YAML.load($1)\nend\n```\n\nThis is a fairly vanilla use of regular expressions, and is pretty easy to read even if you aren't especially familiar with Jekyll itself. The main interesting thing about it that it uses the `/m` modifier to make it so that the pattern is evaluated in multiline-mode. In this particular example, this simply makes it so that the group which captures the YAML metadata can match multiple lines without explicitly specifying the intermediate `\\n` characters. The following contrived example should help you understand what that means if you are still scratching your head:\n\n```\n>> \"foo\\nbar\\nbaz\\nquux\"[/foo\\n(.*)quux/, 1]\n=> nil\n>> \"foo\\nbar\\nbaz\\nquux\"[/foo\\n(.*)quux/m, 1]\n=> \"bar\\nbaz\\n\"\n```\n\nWhile this isn't much of an exciting idiom for those who have a decent understanding of regular expressions, I know that for many patterns can be a mystery, and so I wanted to make sure to point this feature out. It is great to use whenever you need to match a semi-arbritrary blob of content that can span many lines.\n\n**2) Using MatchData objects rather than global variables**\n\nWhile it is not necessarily terrible to use variables like `$1` and `$POSTMATCH`, I tend to avoid them whenever it is not strictly necessary to use them. I find that using `String#match` feels a lot more object-oriented and is more aesthetically pleasing:\n\n```ruby\nif md = self.content.match(/^(---\\s*\\n.*?\\n?)^(---\\s*$\\n?)/m)\n self.content = md.post_match\n self.data = md[1]\nend\n```\n\nIf you combine this with the use of Ruby 1.9's named groups, your code ends up looking even better. The following example is what I ended up using in Jackal:\n\n```ruby\nif (md = contents.match(/^(?\n\n
\n\nIn the video, the communications delay is set at only a couple of seconds, but it\ncan be set arbitrarily high, which makes it possible to simulate the full 14-\nminute-plus delay between Earth and Mars. No matter what the delay is set at, the\nrover queues up commands as they come in and sends its responses one \nat a time as its tasks are completed. The entire simulator is only a couple of\npages of code. It consists of the following objects and responsibilities:\n\n* [SpaceExplorer::Radio](https://github.com/elm-city-craftworks/space_explorer/blob/pr-5.2/lib/space_explorer/radio.rb) relays messages on a time delay.\n* [SpaceExplorer::MissionControl](https://github.com/elm-city-craftworks/space_explorer/blob/pr-5.2/lib/space_explorer/mission_control.rb) communicates with the rover.\n* [SpaceExplorer::Rover](https://github.com/elm-city-craftworks/space_explorer/blob/pr-5.2/lib/space_explorer/rover.rb) communicates with mission control and updates the map.\n* [SpaceExplorer::World](https://github.com/elm-city-craftworks/space_explorer/blob/pr-5.2/lib/space_explorer/world.rb) implements the simulated world map.\n\nAs I implemented this system, I took care to abide by Smyth's recommendation\nthat methods not return meaningful values. Although I wasn't so pedantic as\nto explicitly return `nil` from each function, I treated them as void functions\ninternally, so none of the simulator's features depend on the return value \nof the methods I implemented. To see the effect this approach had on\noverall system design, we can trace a command's execution from end to end while\npaying attention to what is going on under the hood.\n\nI'd like to walk you through how `SNAPSHOT` works, simply because it has the\nlargest number of moving parts to it. As you saw in the video, \n`SNAPSHOT` is used to get back a 5x5 ASCII \"picture\" of the area around the \nrover, which can be used to aid navigation. In the following example, \n`@` is the rover, `-` represents empty spaces, and `X` represents boulders:\n\n```\n20:35| seacreature| !SNAPSHOT\n20:35| roboseacreature| X - - X -\n20:35| roboseacreature| X X - X X\n20:35| roboseacreature| - X @ X X\n20:35| roboseacreature| - - X X -\n20:35| roboseacreature| - - - - -\n```\n\nAs you may have already guessed, the user interface for this project is\nIRC-based, which is a convenient (if ugly) medium for experimenting with\nasynchronous communications. A bot that is responsible for running\nthe simulation monitors the channel for commands, which can be any\nmessage that starts with an exclamation point. When these messages are\ndetected, they are passed on a `MissionControl` object for processing. The\ncallback that monitors the channel and passes messages along to that \nobject is shown here:\n\n```ruby\nbot.on(:message, /\\A!(.*)/) do |m, command|\n mission_control.send_command(command)\nend\n```\n\nThe `MissionControl` object is nothing more than a bridge between the UI \nand a `Radio` object, so the `send_command` method passes the \ncommand along without modification:\n\n```ruby\nmodule SpaceExplorer\n class MissionControl\n def send_command(command)\n @radio_link.transmit(command)\n end\n end\nend\n```\n\nThe `Radio` instance that `@radio_link` points to holds a\nreference to a `Rover` object, which is where the `SNAPSHOT` command will be\nprocessed. Before it gets there, `Radio#transmit` enforces a\ntransmission delay through the use of a very coarse-grained timer mechanism:\n\n```ruby\nmodule SpaceExplorer\n class Radio\n def transmit(command)\n raise \"Target not defined\" unless defined?(@target)\n\n Thread.new do\n start_time = Time.now\n\n sleep 1 while Time.now - start_time < @delay\n\n @target.receive_command(command) \n end\n end\n end\nend\n```\n\nIt's important to point out here that `Radio#transmit` is designed to work with\nan arbitrary delay, so it isn't practical for it to block execution and\nreturn a value. Instead, it spins off a background thread that will eventually\ncall the `receive_command` callback method on its `@target` object, which in this case is a\n`Rover` instance.\n\nThe implementation of the `Rover` object is more interesting than the\nobjects we've looked at so far because it implements the\n[Actor model](http://en.wikipedia.org/wiki/Actor_model). \nWhenever `Rover#receive_command` is called, commands are not\nprocessed directly but are instead placed on a threadsafe queue that then gets\nacted upon in a first-come, first-serve basis. This approach allows the `Rover` to do \nits tasks sequentially while continuing to accept requests as they come in. To\nunderstand how that works, think about how `SNAPSHOT` gets handled by the \nfollowing code:\n\n```ruby\nrequire \"thread\"\n\nmodule SpaceExplorer\n class Rover\n def initialize(world, radio_link)\n @world = world\n @radio_link = radio_link\n\n @queue = Queue.new\n\n Thread.new { loop { process_command(@queue.pop) } }\n end\n\n def receive_command(command)\n @queue.push(command)\n end\n\n def process_command(command)\n case command\n when \"PING\"\n @radio_link.transmit(\"PONG\")\n when \"NORTH\", \"SOUTH\", \"EAST\", \"WEST\" \n @world.move(command)\n when \"SNAPSHOT\"\n @world.snapshot { |text| @radio_link.transmit(\"\\n#{text}\") }\n else\n # do nothing\n end\n end\n end\nend\n```\n\nWhen the `receive_command` callback is triggered by the `Radio` object,\nthe method pushes that command onto a queue, which should happen nearly\ninstantaneously in practice. At this point, the command has finished its\noutbound trip and is ready to be processed.\n\nAfter the `Rover` object handles any tasks that were already queued up,\n`SNAPSHOT` is passed to the `process_command` method, where the \nfollowing line gets executed:\n\n```ruby\n@world.snapshot { |text| @radio_link.transmit(\"\\n#{text}\") }\n```\n\nThis code looks a little weird because it isn't immediately obvious why a block\nis being used here. Instead, we might expect the following code under \nordinary circumstances:\n\n```ruby\n@radio_link.transmit(\"\\n#{@world.snapshot}\") \n```\n\nHowever, taking this approach would be a subtle violation of Smyth's LoD,\nbecause it would require `World#snapshot` to have a meaningful return value,\nintroducing additional coupling. In this case, the coupling is\ntemporal rather than structural, which makes it harder to spot.\n\nThe main difference between the two examples is that the latter has a strong\nconnascence of timing and the former does not. In the value-returning example,\nif `@world.snapshot` were not simply generating a trivial ASCII diagram but\nactually controlling hardware on a Mars rover to take an image, we might expect\nit to take some amount of time to respond. If it were a large enough amount of\ntime, it wouldn't be practical to block while waiting for a response, so the\ncall to `RadioLink#transmit` would need to be backgrounded. This would also be\ntrue for any caller that made use of `World#snapshot`.\n\nBy using a code block (which is really just a lightweight, anonymous callback\nmechanism), we can push the responsibility of whether to run the\ncomputations in a background thread into the `World` object, making that\ndecision completely invisible to its callers. As an added bonus, `World` can\nalso be more responsible about failure handling as well, because it decides \nif and when to execute the callback and how to handle unexpected situations.\n\nIn practical scenarios, the advantages and disadvantages of whether \nviolate Smyth's law would need to be weighed out, but in this case I've\nintentionally tried to apply it first and then attempt to justify it. For this\nparticular example, I can see the approach as being worthwhile even if it \nmakes for slightly more ugly code.\n\nOf course, no attempt at purity is ultimately successful, and if you take a look\nat `World#snapshot`, you will see that this is where I finally throw Smyth's LoD\nout the window for the sake of practicality. Feel free to focus on the structure\nof the code rather than the algorithm used to process the map, as that is what \nmatters most in this article:\n\n```ruby\nmodule SpaceExplorer\n class World\n DELTAS = (-2..2).to_a.product((-2..2).to_a)\n\n # ...\n\n def snapshot\n snapshot = DELTAS.map do |rowD, colD|\n if colD == 0 && rowD == 0\n \"@\"\n else\n @data[@row + rowD][@col + colD]\n end\n end\n\n text = snapshot.each_slice(5).map { |e| e.join(\" \") }.join(\"\\n\")\n\n yield text\n end\n end\nend\n```\n\nAmong other things, we see here the familiar chain of `Enumerable` methods \nslammed together, all of which return values that are not immediate parts of\nthe `World` object:\n\n```ruby\ntext = snapshot.each_slice(5).map { |e| e.join(\" \") }.join(\"\\n\")\n```\n\nAlthough I could probably have written some cumbersome adapters to make this\ncode conform to Smyth's LoD, I think that would be a wasteful attempt to follow\nthe letter of the law rather than its spirit. This is especially true when you\nconsider that Smyth and many other early adopters of the classical Law of\nDemeter were working in languages that had a clear separation between objects\nand data structures, so they would not necessarily have considered core\nstructures to be \"objects\" in the proper sense. In Ruby, our core structures are\nfull-blown objects, but that does not mean they need to follow the same rules \nas our domain objects.\n\nI would love it if you'd share a comment with your own thoughts about \nthe philosophical divide between data structures and domain objects, and\nalso encourage you to read [this post from Bob Martin](https://sites.google.com/site/unclebobconsultingllc/active-record-vs-objects) \non the topic, but I won't dwell on the point for now. We still have\nwork to do!\n\nWith the output in hand, all that remains to be done is to ferry it back to the\nIRC channel that requested it. Looking back at the relevant portion of `Rover#process_command`, \nyou can see that the yielded text from `World#snapshot` is passed on to \nanother `Radio` object:\n\n```ruby\n@world.snapshot { |text| @radio_link.transmit(\"\\n#{text}\") }\n```\n\nThis `Radio` object holds a reference to the `MissionControl` object that sent\nthe original `SNAPSHOT` command, and the path back to it is identical to the\npath the command took to get to the `Rover` object, just in reverse. I won't\nexplain that process again in detail, as all that really matters is that\n`MissionControl#receive_command` eventually gets run. This method is just as\nboring as the `send_command` method we looked at earlier, serving as a direct\nbridge to the UI. I've used a Cinch-based IRC bot in this example, but anything\nwith a `msg()` method will do:\n\n```ruby\nmodule SpaceExplorer\n class MissionControl\n # ...\n\n def receive_command(command)\n @narrator.msg(command)\n end\n end\nend\n```\n\nAt this point, a message is sent to the IRC channel and the out-and-back trip is\ncompleted. Despite being a fairly complicated feature, Smyth's LoD was mostly\nfollowed throughout, and things got weird in only a few places. That said,\nif you have a devious mind, you are likely to have already realized that the\nrelative simplicity of this code is deceptive, because there are\nso many places things can go wrong. Let's talk a little more about \nthat now.\n\n### GROUP PROJECT: Exploring our options for failure handling\n\nSmyth's Law of Demeter promises three main consequences: less complex method\ndefinitions, a decrease in temporal coupling, and a robust way of handling\nfailures. Although the example I've been using provides some evidence for the\nfirst two claims, I intentionally avoided working on error handling to leave\nsomething for us to think through together.\n\nYour challenge, if you choose to accept it, is to think about what can go wrong\nin this simulation and to come up with ways to handle those problems without\nviolating Smyth's LoD. Off the top of my head, I can think of several trivial\nproblems that exist in this code, but I'm sure there are many other things that I\nhaven't considered.\n\nIf you want to start with some low-hanging fruit, think about what happens when\nan invalid command is sent, or what happens when the rover moves off the edge of\nthe fixed-size map it is currently using. If you want to get fancy, think about\nwhether the rover ought to have some safety mechanism that will prevent it from\ndriving into boulders, which it is currently perfectly happy to do. Or, if you\nwant to get creative, find your own way of breaking things, and feel free to ask\nme clarifying questions as you go.\n\nAny level of participation is welcome, ranging from asking a \"What if?\"\nquestion after reading through the code a bit to grand-scale patches that make\nour clunky little rover bulletproof. As I said at the beginning of this\narticle, my purpose in introducing Smyth's LoD to you was to start a\nconversation, and I think this is a fun way to do exactly that.\n\nThe [full source for the simulator](https://github.com/elm-city-craftworks/space_explorer) \nis ready for you to tinker with, so go forth and break stuff!\n\n### Reflections\n\nAlthough I am fairly happy with how the simulator experiment turned out, it is \nhard to draw very many conclusions from it. In very small greenfield \nprojects, it is hard to see how any design principle will ultimately \ninfluence the full software development lifecycle. That having been said,\nit did serve as a great testbed for exploring these ideas and can be a \nstepping stone toward trying these techniques in more practical settings.\n\nI tend to think of software principles as being descriptive rather than\nprescriptive; they provide us with convenient labels for particular approaches\nto problems that already exist in the wild. If you've seen or worked on some\ncode that reminds you of the ideas that Smyth's Law of Demeter attempts to\ncapture, I'd love to hear about it.\n\nI'd also love to hear about whatever doubts have been nagging you as\nyou worked your way through this article. Every software design strategy has its\nstrengths and weaknesses, and sometimes we make the mistake of emphasizing the\ngood parts while downplaying the bad parts, especially when we study new things.\nWith that in mind, your curmudgeonly comments are most welcome, as they tend to \nbring some balance along with them.\n\n> **NOTE:** I owe a huge hat-tip to [David\n> Black](http://twitter.com/david_a_black), as he was the inspiration for\nthis article. He and I were collaborating on a more traditional\ntreatment of the Law of Demeter; we each found our own divergent ideas to\ninvestigate, but I definitely would not have written this article if he hadn't\nshared his thoughtful explorations with me.\n"
},
{
"path": "articles/v5/003-evented-io.md",
"content": "*This issue of Practicing Ruby was contributed by Magnus Holm ([@judofyr][judofyr]), \na Ruby programmer from Norway. Magnus works on various open source \nprojects (including the [Camping][camping] web framework),\nand writes articles over at [the timeless repository][timeless].*\n\nWorking with network I/O in Ruby is so easy: \n\n```ruby\nrequire 'socket'\n\n# Start a server on port 9234\nserver = TCPServer.new('0.0.0.0', 9234)\n\n# Wait for incoming connections\nwhile io = server.accept\n io << \"HTTP/1.1 200 OK\\r\\n\\r\\nHello world!\"\n io.close\nend\n\n# Visit http://localhost:9234/ in your browser.\n```\n\nBoom, a server is up and running! Working in Ruby has some disadvantages, though: we\ncan handle only one connection at a time. We can also have only one *server*\nrunning at a time. There's no understatement in saying that these constraints\ncan be quite limiting. \n\nThere are several ways to improve this situation, but lately we've seen an\ninflux of event-driven solutions. [Node.js][nodejs] is just an event-driven I/O-library\nbuilt on top of JavaScript. [EventMachine][em] has been a solid solution in the Ruby\nworld for several years. Python has [Twisted][twisted], and Perl has so many that they even\nhave [an abstraction around them][anyevent].\n\nAlthough these solutions might seem like silver bullets, there are subtle details that\nyou'll have to think about. You can accomplish a lot by following simple rules\n(\"don't block the thread\"), but I always prefer to know precisely what I'm\ndealing with. Besides, if doing regular I/O is so simple, why does\nevent-driven I/O have to be looked at as black magic?\n\nTo show that they are nothing to be afraid of, we are going to implement an \nI/O event loop in this article. Yep, that's right; we'll capture the core \npart of EventMachine/Node.js/Twisted in about 150 lines of Ruby. It won't \nbe performant, it won't be test-driven, and it won't be solid, but it will \nuse the same concepts as in all of these great projects. We will start \nby looking at a minimal chat server example and then discuss \nhow to build the infrastructure that supports it.\n\n## Obligatory chat server example\n\nBecause chat servers seem to be the event-driven equivalent of a\n\"hello world\" program, we will keep with that tradition here. The\nfollowing example shows a trivial `ChatServer` object that uses\nthe `IOLoop` that we'll discuss in this article:\n\n```ruby\nclass ChatServer\n def initialize\n @clients = []\n @client_id = 0\n end\n\n def <<(server)\n server.on(:accept) do |stream|\n add_client(stream)\n end\n end\n\n def add_client(stream)\n id = (@client_id += 1)\n send(\"User ##{id} joined\\n\")\n\n stream.on(:data) do |chunk|\n send(\"User ##{id} said: #{chunk}\")\n end\n\n stream.on(:close) do\n @clients.delete(stream)\n send(\"User ##{id} left\")\n end\n\n @clients << stream\n end\n\n def send(msg)\n @clients.each do |stream|\n stream << msg\n end\n end\nend\n\n# usage\n\nio = IOLoop.new\nserver = ChatServer.new\n\nserver << io.listen('0.0.0.0', 1234)\n\nio.start\n```\n\nTo play around with this server, run [this script][chatserver] and then open up\na couple of telnet sessions to it. You should be able to produce something like the\nfollowing with a bit of experimentation:\n\n```\n# from User #1's console:\n$ telnet 127.0.0.1 1234\n\nUser #2 joined\nUser #2 said: Hi\nHi\nUser #1 said: Hi\nUser #2 said: Bye\nUser #2 left\n\n# from User #2's console (quits after saying Bye)\n$ telnet 127.0.0.1 1234\n\nUser #1 said: Hi\nBye\nUser #2 said: Bye\n```\n\nIf you don't have the time to try out this code right now,\ndon't worry: as long as you understand the basic idea behind it, you'll be fine.\nThis chat server is here to serve as a practical example to help you \nunderstand [the code we'll be discussing][chatserver] throughout this article.\n\nNow that we have a place to start from, let's build our event system.\n\n## Event handling\n\nFirst of all we need, obviously, events! With no further ado:\n\n```ruby\nmodule EventEmitter\n def _callbacks\n @_callbacks ||= Hash.new { |h, k| h[k] = [] }\n end\n\n def on(type, &blk)\n _callbacks[type] << blk\n self\n end\n\n def emit(type, *args)\n _callbacks[type].each do |blk|\n blk.call(*args)\n end\n end\nend\n\nclass HTTPServer\n include EventEmitter\nend\n\nserver = HTTPServer.new\nserver.on(:request) do |req, res|\n res.respond(200, 'Content-Type' => 'text/html')\n res << \"Hello world!\"\n res.close\nend\n\n# When a new request comes in, the server will run:\n# server.emit(:request, req, res)\n\n```\n\n`EventEmitter` is a module that we can include in classes that can send and\nreceive events. In one sense, this is the most important part of our event\nloop: it defines how we use and reason about events in the system. Modifying it\nlater will require changes all over the place. Although this particular\nimplementation is a bit more simple than what you'd expect from a real \nlibrary, it covers the fundamental ideas that are common to all\nevent-based systems.\n\n## The IO loop\n\nNext, we need something to fire up these events. As you will see in\nthe following code, the general flow of an event loop is simple:\ndetect new events, run their associated callbacks, and then repeat\nthe whole process again.\n\n```ruby\nclass IOLoop\n # List of streams that this IO loop will handle.\n attr_reader :streams\n\n def initialize\n @streams = []\n end\n \n # Low-level API for adding a stream.\n def <<(stream)\n @streams << stream\n stream.on(:close) do\n @streams.delete(stream)\n end\n end\n\n # Some useful helpers:\n def io(io)\n stream = Stream.new(io)\n self << stream\n stream\n end\n\n def open(file, *args)\n io File.open(file, *args)\n end\n\n def connect(host, port)\n io TCPSocket.new(host, port)\n end\n\n def listen(host, port)\n server = Server.new(TCPServer.new(host, port))\n self << server\n server.on(:accept) do |stream|\n self << stream\n end\n server\n end\n\n # Start the loop by calling #tick over and over again.\n def start\n @running = true\n tick while @running\n end\n\n # Stop/pause the event loop after the current tick.\n def stop\n @running = false\n end\n\n def tick\n @streams.each do |stream|\n stream.handle_read if stream.readable?\n stream.handle_write if stream.writable?\n end\n end\nend\n```\n\nNotice here that `IOLoop#start` blocks everything until `IOLoop#stop` is called.\nEverything after `IOLoop#start` will happen in callbacks, which means that the\ncontrol flow can be surprising. For example, consider the following code:\n\n```ruby\nl = IOLoop.new\n\nruby = i.connect('ruby-lang.org', 80) # 1\nruby << \"GET / HTTP/1.0\\r\\n\\r\\n\" # 2\n\n# Print output\nruby.on(:data) do |chunk|\n puts chunk # 3\nend\n\n# Stop IO loop when we're done\nruby.on(:close) do\n l.stop # 4\nend\n\nl.start # 5\n```\n\nYou might think that you're writing data in step 2, but the\n`<<` method actually just stores the data in a local buffer.\nIt's not until the event loop has started (in step 5) that the data\nactually gets sent. The `IOLoop#start` method triggers `#tick` to be run in a loop, which\ndelegates to `Stream#handle_read` and `Stream#handle_write`. These methods \nare responsible for doing any necessary I/O operations and then triggering\nevents such as `:data` and `:close`, which you can see being used in steps 3 and 4. We'll take a look at how `Stream` is implemented later, but for now \nthe main thing to take away from this example is that event-driven code \ncannot be read in top-down fashion as if it were procedural code.\n\nStudying the implementation of `IOLoop` should also reveal why it's \nso terrible to block inside a callback. For example, take a look at this \ncall graph:\n\n```\n# indentation means that a method/block is called\n# deindentation means that the method/block returned\n\ntick (10 streams are readable)\n stream1.handle_read\n stream1.emit(:data)\n your callback\n\n stream2.handle_read\n stream2.emit(:data)\n your callback\n you have a \"sleep 5\" inside here\n\n stream3.handle_read\n stream3.emit(:data)\n your callback\n ...\n```\n\nBy blocking inside the second callback, the I/O loop has to wait 5 seconds \nbefore it's able to call the rest of the callbacks. This wait is\nobviously a bad thing, and it is important\nto avoid such a situation when possible. Of course, nonblocking\ncallbacks are not enough—the event loop also needs to make use of nonblocking\nI/O. Let's go over that a bit more now.\n\n## IO events\n\nAt the most basic level, there are only two events for an `IO` object:\n\n1. Readable: The `IO` is readable; data is waiting for us. \n2. Writable: The `IO` is writable; we can write data.\n\nThese might sound a little confusing: how can a client know that the server\nwill send us data? It can't. Readable doesn't mean \"the server will send us\ndata\"; it means \"the server has already sent us data.\" In that case, the data\nis handled by the kernel in your OS. Whenever you read from an `IO` object, you're\nactually just copying bytes from the kernel. If the receiver does not read \nfrom `IO`, the kernel's buffer will become full and the sender's `IO` will \nno longer be writable. The sender will then have to wait until the \nreceiver can catch up and free up the kernel's buffer. This situation is\nwhat makes nonblocking `IO` operations tricky to work with.\n\nBecause these low-level operations can be tedious to handle manually, the \ngoal of an I/O loop is to trigger some more usable events for application\nprogrammers:\n\n1. Data: A chunk of data was sent to us.\n2. Close: The IO was closed.\n3. Drain: We've sent all buffered outgoing data.\n4. Accept: A new connection was opened (only for servers).\n\nAll of this functionality can be built on top of Ruby's `IO` objects with\na bit of effort.\n\n## Working with the Ruby IO object\n\nThere are various ways to read from an `IO` object in Ruby:\n\n```ruby\ndata = io.read\ndata = io.read(12)\ndata = io.readpartial(12)\ndata = io.read_nonblock(12)\n```\n\n* `io.read` reads until the `IO` is closed (e.g., end of file, server closes the\nconnection, etc.) \n\n* `io.read(12)` reads until it has received exactly 12 bytes.\n\n* `io.readpartial(12)` waits until the `IO` becomes readable, then it reads *at\nmost* 12 bytes. So if a server sends only 6 bytes, `readpartial` will return\nthose 6 bytes. If you had used `read(12)`, it would wait until 6 more bytes were\nsent.\n\n* `io.read_nonblock(12)` will read at most 12 bytes if the IO is readable. It\nraises `IO::WaitReadable` if the `IO` is not readable.\n\nFor writing, there are two methods:\n\n```ruby\nlength = io.write(str)\nlength = io.write_nonblock(str)\n```\n\n* `io.write` writes the whole string to the `IO`, waiting until the `IO` becomes\nwritable if necessary. It returns the number of bytes written (which should\nalways be equal to the number of bytes in the original string).\n\n* `io.write_nonblock` writes as many bytes as possible until the `IO` becomes\nnonwritable, returning the number of bytes written. It raises `IO::WaitWritable`\nif the `IO` is not writable.\n\nThe challenge when both reading and writing in a nonblocking fashion is knowing \nwhen it is possible to do so and when it is necessary to wait.\n\n## Getting real with IO.select\n\nWe need some mechanism for knowing when we can read or write to our\nstreams, but I'm not going to implement `Stream#readable?` or `#writable?`. It's \na terrible solution to loop over every stream object in Ruby and check whether it's\nreadable/writable over and over again. This is really just not a job for Ruby;\nit's too far away from the kernel.\n\nLuckily, the kernel exposes ways to efficiently detect readable and writable\nI/O streams. The simplest cross-platform method is called select(2) \nand is available in Ruby as `IO.select`:\n\n```\nIO.select(read_array [, write_array [, error_array [, timeout]]])\n\nCalls select(2) system call. It monitors supplied arrays of IO objects and waits\nuntil one or more IO objects are ready for reading, ready for writing, or have\nerrors. It returns an array of those IO objects that need attention. It returns \nnil if the optional timeout (in seconds) was supplied and has elapsed.\n```\n\nWith this knowledge, we can write a much better `#tick` method:\n\n```ruby\nclass IOLoop\n def tick\n r, w = IO.select(@streams, @streams)\n r.each do |stream|\n stream.handle_read\n end\n \n w.each do |stream|\n stream.handle_write\n end\n end\nend\n```\n\n`IO.select` will block until some of our streams become readable or writable\nand then return those streams. From there, it is up to those streams to do \nthe actual data processing work.\n\n## Handling streaming input and output \n\nNow that we've used the `Stream` object in various examples, you may \nalready have an idea of what its responsibilities are. But let's first take a look at how it is implemented:\n\n```ruby\nclass Stream\n # We want to bind/emit events.\n include EventEmitter\n\n def initialize(io)\n @io = io\n # Store outgoing data in this String.\n @writebuffer = \"\"\n end\n\n # This tells IO.select what IO to use.\n def to_io; @io end\n\n def <<(chunk)\n # Append to buffer; #handle_write is doing the actual writing.\n @writebuffer << chunk\n end\n \n def handle_read\n chunk = @io.read_nonblock(4096)\n emit(:data, chunk)\n rescue IO::WaitReadable\n # Oops, turned out the IO wasn't actually readable.\n rescue EOFError, Errno::ECONNRESET\n # IO was closed\n emit(:close)\n end\n \n def handle_write\n return if @writebuffer.empty?\n length = @io.write_nonblock(@writebuffer)\n # Remove the data that was successfully written.\n @writebuffer.slice!(0, length)\n # Emit \"drain\" event if there's nothing more to write.\n emit(:drain) if @writebuffer.empty?\n rescue IO::WaitWritable\n rescue EOFError, Errno::ECONNRESET\n emit(:close)\n end\nend\n```\n\n`Stream` is nothing more than a wrapper around a Ruby `IO` object that\nabstracts away all the low-level details of reading and writing that were\ndiscussed throughout this article. The `Server` object we make use of \nin `IOLoop#listen` is implemented in a similar fashion but is focused\non accepting incoming connections instead:\n\n```ruby\nclass Server\n include EventEmitter\n\n def initialize(io)\n @io = io\n end\n\n def to_io; @io end\n \n def handle_read\n sock = @io.accept_nonblock\n emit(:accept, Stream.new(sock))\n rescue IO::WaitReadable\n end\n\n def handle_write\n # do nothing\n end\nend\n```\n\nNow that you've studied how these low-level objects work, you should\nbe able to revisit the full [source code for the Chat Server\nexample][chatserver] and understand exactly how it works. If you\ncan do that, you know how to build an evented I/O loop from scratch.\n\n### Conclusions\n\nAlthough the basic ideas behind event-driven I/O systems are easy to understand, \nthere are many low-level details that complicate things. This article discussed some of these ideas, but there are many others that would need\nto be considered if we were trying to build a real event library. Among\nother things, we would need to consider the following problems:\n\n* Because our event loop does not implement timers, it is difficult to do\na number of important things. Even something as simple as keeping a \nconnection open for a set period of time can be painful without built-in\nsupport for timers, so any serious event library must support them. It's\nworth pointing out that `IO#select` does accept a timeout parameter, and\nit would be possible to make use of it fairly easily within this codebase.\n\n* The event loop shown in this article is susceptible to [back pressure][bp],\nwhich occurs when data continues to be buffered infinitely even if it\nhas not been accepted for processing yet. Because our event loop \nprovides no mechanism for signaling that its buffers are full, incoming\ndata will accumulate and have a similar effect to a memory leak until\nthe connection is closed or the data is accepted.\n\n* The performance of select(2) is linear, which means that handling \n10,000 streams will take 10,000x as long as handling a single stream. \nAlternative solutions do exist at the kernel, but many are not \ncross-platform and are not exposed to Ruby by default. If you have \nhigh performance needs, you may want to look into the [nio4r][nio4r] \nproject, which attempts to solve this problem in a clean way by \nwrapping the libev library.\n\nThe challenges involved in getting the details right in event loops\nare the real reason why tools like EventMachine and Node.js exist. These systems\nallow application programmers to gain the benefits of event-driven I/O without\nhaving to worry about too many subtle details. Still, knowing how they work under the hood\nshould help you make better use of these tools, and should also take away some\nof the feeling that they are a kind of deep voodoo that you'll never\ncomprehend. Event-driven I/O is perfectly understandable; it is just a bit \nmessy.\n\n[chatserver]: https://gist.githubusercontent.com/practicingruby/3612925/raw/315e7bfc5de7a029606b3885d71953acb84f112e/ChatServer.rb \n[timeless]: http://timelessrepo.com\n[camping]: https://github.com/camping\n[judofyr]: http://twitter.com/judofyr\n[nodejs]: http://nodejs.org\n[em]: http://rubyeventmachine.com\n[twisted]: http://twistedmatrix.com\n[anyevent]: http://metacpan.org/module/AnyEvent\n[libev]: http://software.schmorp.de/pkg/libev.html\n[libuv]: https://github.com/joyent/libuv\n[nio4r]: https://github.com/tarcieri/nio4r\n[bp]: http://en.wikipedia.org/wiki/Back_pressure#Back_pressure_in_information_technology\n"
},
{
"path": "articles/v5/004-service-apis.md",
"content": "*This article was contributed by Carol Nichols\n([@carols10cents](http://twitter.com/carols10cents),\n[carols10cents@rstat.us](https://rstat.us/users/Carols10cents)), one\nof the active maintainers of [rstat.us](https://rstat.us). Carol is\nalso involved in the Pittsburgh Ruby community, and is a co-organizer of the\n[Steel City Ruby Conf](http://steelcityrubyconf.org/).*\n\n[Rstat.us](https://rstat.us) is a microblogging site that is similar to Twitter, but \nbased on the [OStatus](http://ostatus.org/about) open standard. It's designed to be federated so\nthat anyone can run an instance of rstat.us on their own domain while still being\nable to follow people on other domains. Although rstat.us is an active project \nwhich has a lot to offer its users, the lack of an API has limited its\nadoption. In particular, an API would facilitate the development of mobile\nclients, which are a key part of what makes microblogging convenient for many people.\n\nTwo different types of APIs have been considered for possible implementation\nin rstat.us: a hypermedia API using an open microblogging spec and a JSON API that is\ncompatible with Twitter's API. In this article, we'll compare these two API styles \nin the context of rstat.us, and discuss the decision that the project's\ndevelopers have made after weighing out the options.\n\n## Hypermedia API\n\nHypermedia APIs currently have a reputation for being complicated and hard to\nunderstand, but they're really nothing to be scared of. There are many, many\narticles about what hypermedia is or is not, but the general definition that\nmade hypermedia click for me is that a hypermedia API returns links in its\nresponses that the client then uses to make its next calls. This means that the\nserver does not have a set of URLs with parameters documented for you up front;\nit has documentation of the controls that you will see within the responses.\n\nThe specific hypermedia API type that we are considering for rstat.us is one\nthat complies with the [Application-Level Profile Semantics (ALPS) microblogging\nspec](http://amundsen.com/hypermedia/profiles/). This spec is an experiment\nstarted by Mike Amundsen to explore the advantages and disadvantages of multiple\nclient and server implementations agreeing only on what particular values for\nthe XHTML attributes `class`, `id`, `rel`, and `name` signify. The spec does not\ncontain any URLs, example queries, or example responses.\n\nHere is a subset of the ALPS spec attributes and definitions; these have to do\nwith the rendering of one status update and its metadata:\n\n- li.message - A representation of a single message\n- span.message-text - The text of a message posted by a user\n- span.user-text - The user nickname text\n- a with rel 'message' - A reference to a message representation\n\nThis is one way you could render an update that is compatible with these attributes:\n\n```html\n\n```\n\nAnd this is another way that is also compatible:\n\n```html\n\n```\n\nNotice some of the differences between the two:\n\n- All the elements being siblings vs some nested within each other\n- Only having the ALPS attribute values vs having other classes and rels as well\n- Only having the ALPS elements vs having the `` element \nbetween the `
Kernel#\\`, `IO.popen`, `Process.spawn`, `open`, `shell`, `open3`,\n`pty`, and probably more. All of these ship with Ruby, some in the core and\nothers in the standard library.\n\nAll of these spawning methods boil down to the same pattern, but we're not going\nto implement them all. To save time, we'll stick with implementing `system` and\nthe backtick method. Either of these methods can be called\nwith a shell command as the argument. Both handle the command in slightly\ndifferent ways with slightly different outputs:\n\n``` \nsystem('ls -l') #=> true\nsystem('ls -l *.rb | ack Product') #=> true\nsystem('boohoo') #=> nil\n`git log -n1 --format=%h^` #=> 51e7a1c\n`hostname` #=> jessebook\n```\n\nLet's start building them.\n\n## Harnessing ourselves with tests\n\nBefore we dive into spawning process head first, let's rein ourselves in a\nbit. If we're going to reimplement what Ruby already provides, we're going to\nneed a way to test our implementation and make sure that it performs the same\nway that Ruby does. Enter [Rubyspec](http://rubyspec.org).\n\n> The RubySpec project aims to write a complete executable specification for the\n> Ruby programming language that is syntax-compatible with RSpec. RSpec is\n> essentially a DSL (domain-specific language) for describing the behavior of\n> code. This project contains specs that describe Ruby language syntax, core\n> library classes, and standard library classes.\n\nRubySpec provides a specification for the Ruby language itself, and we want to\nreimplement a part of the Ruby language; therefore, we can use RubySpec\nto test our implementation.\n\nTo use these specs to drive our implementation, we need to get two\nthings: RubySpec itself, and its testing library mspec. You can check\nout [this README](https://github.com/rubyspec/rubyspec/blob/master/README) \nfor installation instructions. To verify that things are working as \nexpected, try running the kernel tests from within the RubySpec project\ndirectory:\n\n```bash\n$ mspec core/kernel\n```\n\nTo run our custom code against these tests, we can use\nthe familiar `-r` option with `mspec` to require a file that redefines\nthe methods we want to override. Let's do that, while at the same time \nrunning the `Kernel.system` specs:\n\n```bash\n$ touch practicing_spawning.rb\n$ mspec -r ./practicing_spawning.rb core/kernel/system_spec.rb\n```\n\nShould be all green so far!\n\n## Breaking the test\n\nLet's begin our implementation by causing the tests to fail:\n\n```ruby\n# practicing_spawning.rb\nmodule Kernel\n def system(*args)\n end\n\n private :system\nend\n```\n\nThe very first spec says that `system` should be private. I set that up right\naway because it's not the interesting part. If we run the `system` specs again,\nwe get our first of several failures:\n\n```console\n1)\nKernel#system executes the specified command in a subprocess FAILED\nExpected (STDOUT): \"a\\n\"\n but got: \"\"\n```\n\nThis failure directly relates to the following spec:\n\n```ruby\nit \"executes the specified command in a subprocess\" do\n lambda { @object.system(\"echo a\") }.should output_to_fd(\"a\\n\")\nend\n```\n\nIf you've ever used the `system` method, this test should be easy to\nunderstand. It says that shelling out to `echo` should output the echoed string.\nIf you [dig into](https://github.com/rubyspec/mspec/blob/master/lib/mspec/matchers/output_to_fd.rb#L68-70)\n the `output_to_fd` method that's part of `mspec`, you'll see that it's\nexpecting this output on `STDOUT`.\n\n## fork and subprocesses\n\nThe failing spec title says that `system` spawns a subprocess. If you're\ncreating new processes on a Unix system, that means using `fork`:\n\n> ------------------------------------------------------------------------------\n> Kernel.fork [{ block }] -> fixnum or nil \n> Process.fork [{ block }] -> fixnum or nil\n> \n> ------------------------------------------------------------------------------\n> \n> Creates a subprocess. If a block is specified, that block is run in the\n> subprocess, and the subprocess terminates with a status of zero. Otherwise,\n> the fork call returns twice, once in the parent, returning the process ID of\n> the child, and once in the child, returning nil.\n\nThis bit of Ruby documentation gives you an idea of what `fork` does. It's\nconceptually similar to going on a hike and coming to a fork in the trail. The\ntrail represents the execution of a process over time. Whereas humans can only\npick one path, when a process is forked it literally continues down both\nbranches of the trail in parallel. What was one process becomes two independent \nprocesses. This behavior is specified by the fork(2) manpage:\n\n> Fork() causes creation of a new process. The new process (child process) is\n> an exact copy of the calling process (parent process) [...]\n\nWhen you `fork`, you start with one process and end up with two processes that\nare *exactly the same*. In some cases, this means that everything is copied from\none process to the other. But if [copy-on-write\nsemantics](http://en.wikipedia.org/wiki/Copy-on-write) are implemented,\nthe two processes may physically share memory until one of them tries to\nmodify it; then each gets its own copy written out.\n\nAlthough understanding `fork` is certainly helpful, we still haven't quite figured\nout how to implement the `system` method. We know that we can take our Ruby \nprocess and create a copy of it with `fork`, but how do we then turn the \nnew child process into an `echo` process?\n\n## fork + exec\n\nThe `fork` + `exec` pattern for spawning processes is the blueprint upon which\nmost process spawning is built. We've already looked at `fork`, so what\nabout `exec`?\n\n`exec` transforms the current process into another process. Using\n`exec`, you can transform a Ruby process into an `ls` process, another Ruby\nprocess, or an `echo` process:\n\n```ruby\nputs 'hi from Ruby'\nexec('ls')\nputs 'bye from Ruby' # will never be reached\n```\n\nThis program will never get to the last line of Ruby code. Once it has performed\n`exec('ls')`, the Ruby program no longer exists. It has been transformed to `ls`.\nSo there's no possible way for it to get back to this Ruby program and finish\nexecution.\n\n## Finally, a passing test\n\nWith `fork` and `exec`, we now have the building blocks that we need to implement\nour own `system` method. Here's the most basic implementation:\n\n```ruby\n# practicing_spawning.rb\nmodule Kernel\n def system(*args)\n\n # Create a new subprocess that will just exec the requested program.\n pid = fork { exec(*args) }\n\n # Because fork() allows both processes to work in parallel, we must tell the\n # parent process to wait for the child to exit. Otherwise, the parent would\n # continue in parallel with the child and would be unable to process its\n # return value.\n _, status = Process.waitpid2(pid)\n status.success?\n end\n\n private :system\nend\n```\n\nIf we run this against the same spec as before, more tests pass, but\nnot all of them. Still, getting that initial spec to pass means that we're headed\nin the right direction.\n\nThere are three very simple Unix programming primitives in use here: `fork`,\n`exec`, and `wait`. We've already talked about `fork` and `exec`, the\ncornerstone of Unix process spawning. The third player here, `wait`, is often\nused in unison with these two. It tells the parent process to wait for the child\nprocess before continuing, rather than continuing execution in parallel. This is\na pretty common pattern when spawning shell commands, because you usually want to\nwait for the output of the command.\n\nIn this case, we collect the status of the child when it exits and return the\nresult of `success?`. This result is `true` for a successful exit status code (i.e., 0)\nand `false` for any other value.\n\n## Getting back to green\n\nNow we need to get the rest of the `system` specs passing. In\nthe remainder of the failures, we see the following output:\n\n```console\n1) \nKernel#system returns nil when command execution fails FAILED\nExpected false to be nil\n\n\n
\n\nI knew right away that this was exactly the kind of simulation I wanted to\nbuild, but it honestly didn't even cross my mind to attempt to read the Clojure\ncode and port it to Ruby. One quick glance at its [source code][sim] reminded me\njust how much I wanted to learn a Lisp dialect some day, but it definitely\nwasn't going to be today! I didn't have time to go dust off the books on my\nshelf that I never read, or to watch the [2.5 hour long video][hickey] of the\ntalk that this code came from. \n\nSo instead of doing all that, I set off to build my own implementation from\nscratch by cobbling together the bits of information I had collected\ninto something that sort of worked. Using the general description of ant\nbehavior as my guide, I left it up to my imagination to fill in the details, and\nwithin an hour or so I had built something that had ants moving around the\nscreen. Unfortunately, my little family of ants seemed to have come from a\nfailed evolutionary branch, because they didn't do what they were supposed to\ndo! They'd wander around randomly, get stuck, choose the wrong food sources, and\ngenerally misbehave in all sorts of painful ways. Thinking that I needed a\nbreak, I stepped away from the project for a day so that I could come back to it\nwith a fresh perspective.\n\nThe next day, I did end up getting something vaguely resembling an ant colony to\nappear on my screen. The behavior was not perfect, but it illustrated the main \nidea of the algorithm:\n\n\n\n
\n\nIt was fairly easy to get to this point, but then it became extremely hard to \nimprove upon the simulator. Ant colony optimization has a lot of variables to it\n(i.e. things like the size of the world, number of ants, number of food sources,\npheremone decay rate, amount of pheremone dropped per iteration, etc). Changing\nany one of these things can influence the effectiveness of the others. When you\ncombine this variability with an implementation where the actual behaviors were \nhalf-baked and possibly buggy, you end up with a big mess that is hard to debug,\nand even harder to understand. Knowing that my code was in really bad shape,\nI was ready to give up.\n\nAlthough it took a lot of rumination to get me there, the lightbulb eventually\nturned on: maybe reading the Clojure implementation wasn't such a bad idea after\nall! I had initially thought that learning the algorithm would be much easier\nthan learning the semantics and syntax of a new language, but two days of hard\nwork and mediocre results lead me to re-evaluate that assumption. At the very\nleast, I could spend an afternoon with the Clojure code. Even if the ants still frightened\nand confused me in the end, I'd at least learn how to read some code from a language \nthat I had always wanted to study anyway. \n\nI think you can guess what happened next: within a couple of hours, I not only\nfully understood Rich Hickey's implementation, but I had also learned dozens\nupon dozens of Clojure features, including a few that have no direct analogue in\nRuby. While it may have been a result of frustration \ndriven development, I was genuinely surprised at what a great way this was to\nlearn a new language while also studying a programming problem that I was\ninterested in.\n\nThroughout the rest of this article, I will attempt to demonstrate that given\nthe right example, even a few dozen lines of code can teach you a tremendous\namount of useful things about a language that you've never worked with before.\nIf you are new to Clojure programming, you'll be able to follow along\nand experience the same benefits that I did; if you already know the\nlanguage, you can use this as an exercise in developing a beginner's mindset. In\neither case, I think you'll be surprised at how much we can extract\nfrom such a small chunk of code.\n\nTo keep things simple, we won't bother to read the complex bits of code that \nimplements ant behavior. Instead, we'll start from the bottom up and\ntake a look at how this simulation models its world and the ants within it.\nAlthough this won't help us understand how things get set into action, it will\ngive us plenty of opportunities to learn some interesting Clojure\nfeatures.\n\n## Modeling the world\n\nThe following code is responsible for creating a blank slate world with 80x80\ndimensions:\n\n```clojure\n(defstruct cell :food :pher) ;may also have :ant and :home\n\n;dimensions of square world\n(def dim 80)\n\n;world is a 2d vector of refs to cells\n(def world \n (apply vector \n (map (fn [_] \n (apply vector (map (fn [_] (ref (struct cell 0 0))) \n (range dim)))) \n (range dim))))\n\n(defn place [[x y]]\n (-> world (nth x) (nth y)))\n```\n\nEven if this is the first time you've ever seen a Clojure program, you could take an\neducated guess at what is going on in at least a few of these lines of code:\n\n```clojure\n(defstruct cell :food :pher) ; this defines a Struct-like thing\n\n(def dim 80) ; this defines a named value, setting dim=80\n\n(defn place [[x y]]\n (-> world (nth x) (nth y))) ; this looks like an accessor into a\n ; two-dimensional grid\n```\n\nThe code in the `world` definition is much more complicated, but it \nhas a helpful comment that describes what it is: a 2D vector of \nrefs to cells. This hint gives us some useful keywords to search for \nin [Clojure's API docs][clojure-doc]. With a bit of effort, it is possible\nto use this code sample and Clojure's documentation to learn all the \nfollowing things about the language:\n\n1. The [Map][Map] collection is Clojure's equivalent to Ruby's `Hash` object.\n1. The [StructMap][StructMap] collection is a `Map` with some predefined keys\nthat cannot be removed. They can be defined using `defstruct`, and are instantiated\nvia `struct`.\n1. The `(def ...)` construct is a [special form][def] that defines global variables \nwithin a namespace, but it is considered bad style to treat these variables as\nif they were mutable.\n1. The `(defn ...)` construct is a macro which among other things provides\nsyntactic sugar for defining functions with named parameters.\n1. The [Vector][Vector] collection has core functionality which is similar to\nRuby's `Array` object. Vectors can be instantiated using the `vector` function or\nvia the `[]` literal syntax, and their elements are accessed using \nthe `nth` function.\n1. All collections in Clojure implement a [Sequence][Sequence]\ninterface that is similar to Ruby's `Enumerable` module. It provides various\nfunctions that Ruby programmers are already familiar with, such as `map`, `reduce`, \n`sort` But because most of these functions return lazy sequences, they\nbehave slightly differently than their Ruby counterparts.\n1. The [Ref][Ref] construct is a transactional reference, which is one of\nClojure's concurrency primitives. In a nutshell, wrapping state in a `Ref`\nmakes it so that state can only be modified from within a transaction, ensuring\nthread safety.\n1. Among other things, the `range` function provides behavior similar to the enumerator \nform of Ruby's `Integer#times` method. \n1. The `apply` function provides functionality similar to Ruby's splat operator\n(`*`), passing the elements of a sequence as arguments to a function.\n1. The [-> macro][->] provides syntactic sugar for function composition, which\ncan make chaining function calls easier.\n\nBased on this laundry list of concepts to learn, it is easy to see from this\nexample alone that much like Ruby, Clojure is a very rich language that is\ncapable of concisely expressing very complex ideas. With that in mind, it is\nhelpful to use Clojure's REPL to experiment while learning, much as we'd do\nwith `irb` in Ruby. Once again using the code sample as a guide, an \nexploration such as the one that follows can go a long way towards \nverifying our understanding of what we learned from the documentation:\n\n```clojure\nuser=> (defstruct cell :food :pher)\n; #'user/cell\nuser=> (struct cell 1 4)\n; {:food 1, :pher 4}\nuser=> [ [ :a :b :c ] [ :d :e :f ] ]\n; [[:a :b :c] [:d :e :f]]\nuser=> (def data [[:a :b :c] [:d :e :f]])\n; #'user/data\nuser=> (nth (nth data 1) 2)\n; :f\nuser=> (nth (nth data 2) 1)\n; IndexOutOfBoundsException clojure.lang.PersistentVector.arrayFor \n; (PersistentVector.java:106)\nuser=> (nth (nth data 0) 1)\n; :b\nuser=> (-> data (nth 1) (nth 2))\n; :f\nuser=> (map (fn [x] (* x 2)) [1 2 3])\n; (2 4 6)\nuser=> (vector (map (fn [x] (* x 2)) [1 2 3]))\n; [(2 4 6)]\nuser=> (apply vector (map (fn [x] (* x 2)) [1 2 3]))\n; [2 4 6]\nuser=> (range 5)\n; (0 1 2 3 4)\nuser=> (apply vector (map (fn [x] (struct cell 0 0)) (range 5)))\n; [{:food 0, :pher 0} {:food 0, :pher 0} {:food 0, :pher 0} \n; {:food 0, :pher 0} {:food 0, :pher 0}]\n```\n\nKnowing what we now know, it is possible to imagine a loose translation of the\noriginal Clojure code sample into Ruby, if we account for a few cavaets:\n\n1. Most `Enumerable` methods return `Array` objects, which are not lazily\nevaluated. Some support for lazy sequences exist in Ruby 2.0, but we'll\nnot bother with that in our translation because it'd only create more\nwork for us.\n\n2. We don't have a direct analogy to Clojure's `Ref` construct, but we can\npretend that we do for the purposes of this example.\n\n3. We don't have anything baked into the language which implements a `Hash` with\nsome required keys and some optional ones. But such behavior could be\nemulated by building a custom `Cell` object. \n\n4. We don't have destructuring in the parameter lists for our\nfunctions, so we need to handle destructuring manually within the bodies\nof our methods rather than their signatures.\n\nKeeping these points in mind, here's a semi-literal translation of Clojure code\nto Ruby:\n\n```ruby\nDIM = 80 \nWORLD = DIM.times.map do # 1\n DIM.times.map { Ref.new(Cell.new(0, 0)) } # 2,3\n end\n\ndef place(pos) \n x, y = pos # 4\n WORLD[x][y]\nend\n```\n\nWhile the two languages cannot be categorically compared by such a coarse\nexercise in syntactic gymnastics, it does help the similarities and \ndifferences between the languages stand out a bit more. This allows us to reuse\nthe knowledge we already have, and also exposes the gaps in our \nunderstanding that need to be filled in.\n\n> **SIDE QUEST:** The remaining two sections in this article will repeat this\nsame basic process on two more small chunks of code from the ant simulator. If you have some\nfree time and an interest in learning Clojure, you may want to start\nwith the initial code samples in each section and try to figure them out on\nyour own, and *then* come back to read my notes. If you decide to\ntry this out, please share a comment with what you've learned.\n\nNow that we've tackled one concrete feature from this program, it will be much\neasier to understand the rest. There's a lot left to learn, so let's keep\nmoving!\n\n## Modeling an ant\n\nThe following code is responsible for initializing an ant at a \ngiven location within the world:\n\n```clojure\n(defstruct ant :dir) ;may also have :food\n\n(defn create-ant \n \"create an ant at the location, returning an ant agent on the location\"\n [loc dir]\n (dosync\n (let [p (place loc)\n a (struct ant dir)]\n (alter p assoc :ant a)\n (agent loc))))\n```\n\nBecause we already have a rudimentary understanding of how `StructMap` works,\nand how to define functions, we can skip over some of the boilerplate\nand get right to the good stuff:\n\n```clojure\n(dosync ; 1\n (let [p (place loc) ; 2\n a (struct ant dir)]\n (alter p assoc :ant a) ; 3\n (agent loc))) ; 4\n```\n\nDigging back into Clojure's API docs, we can learn four new things \nfrom this code sample:\n\n1. The [dosync][dosync] macro starts a transaction,\nwhich among other things, makes it possible to modify `Ref`\nstructures in a thread-safe way.\n\n1. The [let][let] macro allows you to make use of named values within\na lexical scope. This construct appears to be roughly similar to the \nconcept of block-local variables in Ruby.\n\n1. The [alter][alter] function is used for modifying the contents of a `Ref`\nstructure, and can only be called within a transaction.\n\n1. The [Agent][Agent] construct is another one of Clojure's concurrency\nprimitives. This structure provides an interesting state-centric alternative\nto the actor model of concurrency: rather than encapsulating behavior that acts\nupon external state, agents encapsulate state which is *acted upon* by external\nbehaviors.\n\nOf course, in order to verify that we understand what the documentation is\ntelling us, nothing beats a bit of casual experimentation in the REPL:\n\n```clojure\nuser=> (let [x 10 y 20] (+ x y))\n; 30\nuser=> (let [x 10] (let [y 20] (+ x y)))\n; 30\nuser=> (let [x 10] (let [y 20]) y)\n; CompilerException java.lang.RuntimeException: Unable to resolve \n; symbol: y in this context, compiling:(NO_SOURCE_PATH:3) \nuser=> (def foo (ref { :x 1 :y 1}) )\n; #'user/foo\nuser=> foo\n; #\n\n
\n\nWhile this search effort may seem highly organized, it is the\nresult of very simple decisions made by individual ants. On each\ntick of the simulation, each ant decides its next action based only on its\ncurrent location and the three adjacent locations ahead of it. But \nbecause ants can indirectly communicate via their environment, complex \nbehavior arises in the aggregate.\n\nEmergence and self-organization are popular concepts in programming, but far too many\ndevelopers start and end their explorations into these ideas with [Conway's Game of Life][conway]. \nIn this article, I will help you see these fascinating properties in a new\nlight by demonstrating the role they play in [ant colony optimization (ACO)][aco] algorithms.\n\n> **NOTE:** There are many ways to simulate ant behavior, some of which can be quite useful\nfor a wide range of search applications. For this article, I have built\na fairly naïve simulation that is meant to loosely mimic the kind of ant\nbehavior you can observe in the natural world. This article *may* be useful as a \nbrief introduction to ACO, but be sure to dig deeper if you are interested in\npractical applications. My goal is to provide a great example of emergent \nbehavior, NOT a great reference for nature-inspired search algorithms.\n\n## Modeling the state of an ant colony\n\nThis simulated world consists of many cells: some are food sources, \nsome are part of the colony's nest, and the rest are an\nopen field that needs to be traversed. Each cell can contain a single \nant facing in one of the eight directions you'd find on a compass. \nAs the ants move around the world, they mark the cells they visit with\na trail of pheromones that helps them find their way between their \nnest and nearby food sources. Pheromones accumulate as more ants \ntravel across a given trail, but they also gradually evaporate. \nThe combination of these two properties of pheromones helps \nants find efficient paths to nearby food sources.\n\nSubtle changes to any of these rules can yield very different outcomes, \nand finding an optimal result will necessarily involve some\nexperimentation. Knowing that, it makes sense for the simulator to \nhave a data model that is divorced from its domain logic. Many\nbehavioral changes can be made without altering the\nunderlying data model, and that allows the `Ant`, `Cell`, and `World` constructs to\nbe defined as simple value objects as shown below:\n\n```ruby\nmodule AntSim \n class Ant\n def initialize(direction, location)\n self.direction = direction\n self.location = location\n end\n\n attr_accessor :food, :direction, :location\n end\n\n class Cell\n def initialize(food, home_pheremone, food_pheremone)\n self.food = food \n self.home_pheremone = home_pheremone\n self.food_pheremone = food_pheremone\n end\n\n attr_accessor :food, :home_pheremone, :food_pheremone, :ant, :home\n end\n\n class World\n def initialize(world_size)\n self.size = world_size\n self.data = size.times.map { size.times.map { Cell.new(0,0,0) } }\n end\n\n def [](location)\n x,y = location\n\n data[x][y]\n end\n\n def sample\n data[rand(size)][rand(size)]\n end\n\n def each\n data.each_with_index do |col,x| \n col.each_with_index do |cell, y| \n yield [cell, [x, y]]\n end\n end\n end\n\n private\n\n attr_accessor :data, :size\n end\nend\n```\n\nThese classes are somewhat peculiar in that they are very state-centric and \ndo not encapsulate any interesting domain logic. Although it won't win us\nobject-oriented style points, designing things this way decouples the state of \nthe simulated world from both the events that happen within it and the \noptimization algorithms that run against it. These objects\nrepresent only the nouns of our system, leaving it up to their collaborators \nto supply the verbs.\n\n## Moving around the world\n\nThe ants in this system are surprisingly limited in their behavior. On each \nand every iteration, their entire decision making process can result \nin exactly one of the following outcomes:\n\n\n\nMost of these actions are extremely localized. Turning does not affect any\ncells, while moving only affects the cell the ant currently occupies\nand the one immediately in front of it. However, taking or dropping food\ntriggers a pheromone update, affecting every cell the ant has \nvisited since the last time it updated its trails. This can have far-reaching\neffects on the behavior of the rest of the colony, even though each individual\nant can only sense the pheromone levels of its own cell and the three cells\ndirectly in front of it. While natural ants must drop pheromone\ncontinuously as they walk, artificial ants can improve upon nature by\nupdating entire paths instantaneously.\n\nAn object that implements these behaviors needs to know about the structure of\nthe `Ant`, `Cell`, and `World` objects, but it still does not\nneed to know much about the core domain logic of the simulator. What we want is\nan `Actor` that understands its world and how to play specific roles within it, \nbut does not attempt to define the broader story arc:\n\n```ruby\nrequire \"set\"\n\nmodule AntSim\n class Actor\n DIR_DELTA = [[0, -1], [ 1, -1], [ 1, 0], [ 1, 1],\n [0, 1], [-1, 1], [-1, 0], [-1, -1]]\n\n def initialize(world, ant)\n self.world = world\n self.ant = ant\n\n self.history = Set.new\n end\n\n attr_reader :ant\n\n def turn(amt)\n ant.direction = (ant.direction + amt) % 8\n\n self\n end\n\n def move\n history << here\n\n new_location = neighbor(ant.direction)\n\n ahead.ant = ant\n here.ant = nil\n\n ant.location = new_location\n\n self\n end\n\n def drop_food\n here.food += 1\n ant.food = false\n\n self\n end\n\n def take_food\n here.food -= 1\n ant.food = true\n\n self\n end\n\n def mark_food_trail\n history.each do |old_cell|\n old_cell.food_pheremone += 1 unless old_cell.food > 0 \n end\n\n history.clear\n\n self\n end\n\n def mark_home_trail\n history.each do |old_cell|\n old_cell.home_pheremone += 1 unless old_cell.home\n end\n\n history.clear\n\n self\n end\n\n def foraging?\n !ant.food\n end\n\n def here\n world[ant.location]\n end\n\n def ahead\n world[neighbor(ant.direction)]\n end\n\n def ahead_left\n world[neighbor(ant.direction - 1)]\n end\n\n def ahead_right\n world[neighbor(ant.direction + 1)]\n end\n \n def nearby_places\n [ahead, ahead_left, ahead_right]\n end\n\n private\n\n def neighbor(direction)\n x,y = ant.location\n\n dx, dy = DIR_DELTA[direction % 8]\n\n [(x + dx) % world.size, (y + dy) % world.size]\n end\n\n attr_accessor :world, :history\n attr_writer :ant\n end\nend\n```\n\nOf course, now that we have crossed the line from pure data models to an object\nwhich actually does something, it is impossible to implement meaningful behavior\nwithout making certain assumptions that will affect the capabilities of the \nrest of the system. The `Actor` class draws two significant lines in the sand that\nare easy to overlook on a quick glance:\n\n1. Storing history data in a `Set` rather than an `Array` makes it so\nthat when this object updates pheromone trails, it only takes into account\nwhat cells were visited, not how many times they were visited or in what order\nthey were traversed.\n\n2. The modular arithmetic performed in the `neighbor` function treats the world\nas if it were a [torus][torus], instead of a plane. This means that the\nleftmost column and the rightmost column of the map are adjacent to one \nanother, as are the top and bottom rows. This allows ants to easily wrap around\nthe edges of the map, but also establishes connections between cells that you\nmay not intuitively think of as being close to one another. Without a\nthree-dimensional visualization, it is hard to show that the top right corner of\nthe map and the bottom left corner are actually adjacent to one another.\n\nOf course, the purpose of the `Actor` class is to hide these details from\nthe rest of the system. As long as its collaborators can operate within these \nconstraints, the `Actor` object can be treated as a magic black box that knows\nhow to make ants move around the world and do interesting things. To see why\nthat is useful, check out the `Simulator#iterate` function which drives the\nsimulator's main event loop:\n\n```ruby\nmodule AntSim\n class Simulator\n # ... other functions ...\n\n def iterate\n actors.each do |actor|\n optimizer = Optimizer.new(actor.here, actor.nearby_places)\n \n if actor.foraging?\n action = optimizer.seek_food\n else\n action = optimizer.seek_home\n end\n\n case action\n when :drop_food\n actor.drop_food.mark_food_trail.turn(4)\n when :take_food\n actor.take_food.mark_home_trail.turn(4)\n when :move_forward\n actor.move\n when :turn_left\n actor.turn(-1)\n when :turn_right\n actor.turn(1)\n else\n raise NotImplementedError, action.inspect\n end\n end\n\n sleep ANT_SLEEP\n end\n end\nend\n```\n\nHere we can see that the `Simulator` acts as a bridge that translates\nthe `Optimizer` object's very abstract suggestions into concrete\nactions for the `Actor` to carry out. The design of the `Actor` object gives the\n`Simulator` just enough control to make some small adjustments to the process,\nbut not so much that it needs to be bogged down with the details.\n\n## Finding food and bringing it home\n\nNow that we know the state of the world and how it can be manipulated, it is\ntime to discuss how to produce the kind of behavior that you saw in the\nvideo at the beginning of this article. Perhaps unsurprisingly, the life of the\neveryday worker ant is actually fairly mundane.\n\nEvery ant in this simulation is always either searching for food to bring back\nto the nest, or trying to return home with the food it found. As soon \nan ant accomplishes one of these tasks, it immediately transitions to the other,\nnot bothering to take even a moment to bask in fruits of its labor. The\nfollowing outline describes what the ants in this simulation are \"thinking\" \nat any given point in time, assuming that they haven't managed to \nbecome self-aware...\n\n**When searching for food:**\n\n1. If the current cell has food in it and it is NOT part of the nest, \npick up some food.\n\n2. Otherwise, check the cell directly in front of me. If it has food in it, is\nnot part of the nest, and it is not occupied by another ant, move there.\n\n3. If not, rank the three adjacent cells in front of me based\non the amount of food they contain, and how intense their `food_pheremone`\nlevels are. I will *usually* choose to move or turn towards the cell with\nhighest ranking, but I will randomly deviate from this pattern on occasion\nso that I can explore some uncharted territory.\n\n**When searching for the nest:**\n\n1. If the current cell is part of the nest, drop the food I am carrying.\n\n2. Otherwise, check the cell directly in front of me. If it is part of the nest,\nand it is not occupied by another ant, move there.\n\n3. If not, rank the three adjacent cells in front of me based\non whether or not they are part of the nest, and how intense their `home_pheremone`\nlevels are. I will *usually* choose to move or turn towards the cell with\nhighest ranking, but I will randomly deviate from this pattern on occasion\nso that I can explore some uncharted territory.\n\nTranslating these ideas into code is very straightforward, especially\nif you treat the underlying mathematical formulas as a black box:\n\n```ruby\nmodule AntSim\n class Optimizer\n # ...\n\n def seek_food\n if here.food > 0 && (! here.home)\n :take_food\n elsif ahead.food > 0 && (! ahead.home ) && (! ahead.ant )\n :move_forward\n else\n food_ranking = rank_by { |cell| cell.food }\n pher_ranking = rank_by { |cell| cell.food_pheremone }\n\n ranks = combined_ranks(food_ranking, pher_ranking)\n follow_trail(ranks)\n end\n end\n\n def seek_home\n if here.home\n :drop_food\n elsif ahead.home && (! ahead.ant)\n :move_forward\n else\n home_ranking = rank_by { |cell| cell.home ? 1 : 0 }\n pher_ranking = rank_by { |cell| cell.home_pheremone }\n\n ranks = combined_ranks(home_ranking, pher_ranking)\n follow_trail(ranks)\n end\n end\n\n def follow_trail(ranks)\n choice = wrand([ ahead.ant ? 0 : ranks[ahead],\n ranks[ahead_left],\n ranks[ahead_right]])\n\n [:move_forward, :turn_left, :turn_right][choice]\n end\n \n\n # ...\n end\nend\n```\n\nIf you understand the general idea behind this algorithm, don't worry about the\nexact computations that the `Optimizer` uses unless you are\nplanning on researching Ant Colony Optimization in much greater detail. While I\nunderstand what my own code is doing, I'll admit that I mostly \ncargo-cult copied the probabilistic methods \nfrom [Rich Hickey's simulator][hickey] while sprinkling in a few minor tweaks \nhere and there. That said, if you want to see exactly how I hacked things\ntogether, feel free to check out \nthe [full Optimizer class definition][optimizer].\n\nWhat I personally find much more interesting than the nuts and bolt of\n*how* this algorithm works is to think about *why* it works.\n\n## How the hive mind emerges\n\nAs we discussed in the previous section, ants are attracted to pheromone, and\nthat makes them more likely to follow the trails left behind by other ants than\nthey are to venture out on their own. However, when ants first start exploring\na new space, there are no trails to follow and so they are forced to wander\naround randomly until a food source is found.\n\nGenerally speaking, ants that take a shorter path from the nest to a food\nsource will arrive there sooner than ants that take a longer path. If they\nfollow their own pheromone trail back to the nest, they will also return home\nsooner than those who are traversing longer paths. By the time ants who have\ntaken a longer path return home, the ants on the shortest paths have already\nwent back out in search of additional food, which increases the pheromone levels\non their trails.\n\nThis process on its own would bias the ant colony to prefer shorter paths over\nlonger ones, but the optimization would be somewhat sluggish and might tend to\nproduce solutions that work well locally but aren't nearly as attractive\nglobally. To get better results, the system needs a bit of entropy thrown into\nthe mix.\n\nBecause the behavior of ants has a certain amount of randomness to it,\nthe occasional deviation from established paths are fairly common. Even if the\nfluctuations are small, each tiny shortcut that allows an ant to get between two\npoints along a path in a shorter amount of time ultimately contributes to\nfinding an optimal solution. This means that even an ant who goes wildly off\ncourse and starves to death nowhere near the nest can make a meaningful\ncontribution to the colony if even some tiny segment of its path serves to\nshorten an existing well-worn trail.\n\nWhen you add in the fact that pheromones are volatile and tend to evaporate over\ntime, an upper limit emerges for how much a bad path or a local optimization can\ninfluence the colony's decision making. Evaporation is also a key part of what\nallows the ants to change course when a food source is exhausted, or an obstacle\nstands in the way of an established path.\n\nPheromone decay is something that can be modeled in many ways, but the easiest\nway of simulating it is to gradually reduce the pheromone at every cell in the \nworld on a regular interval. For an example of this approach, check out\n`Simulator#evaporate`:\n\n```ruby\nmodule AntSim\n class Simulator\n def evaporate\n world.each do |cell, (x,y)| \n cell.home_pheremone *= EVAP_RATE \n cell.food_pheremone *= EVAP_RATE\n end\n end\n end\nend\n```\n\nSo if you take the basic positive feedback loop caused by pheromone attraction\nand mix in a bit of probabilistic exploration and the gradual evaporation of trails, you end\nup with a fairly robust optimization process. It truly is remarkable that \nthese basic factors can combine to create a very\neffective search heuristic, especially when you consider the fact that what\nwe've discussed here is only a crude approximation of the tip of the iceberg\nwhen it comes to [Ant Colony Optimization][aco].\n\n## Reflections\n\nEmergent behaviors in computing problems have always fascinated me, even though I\nhave not spent nearly enough time studying them to understand them well. I feel\nsimilarly about a lot of other things in life, ranging from the board game Go,\nto the spread of memes throughout communities both online and offline.\n\nThere is something deep and almost spiritual in the realization that the\nextremely complex behaviors can emerge from very simple systems with very few\nrules, and a complete lack of central organization. It forces us to call into\nquestion everything we experience and to wonder whether there is some elegant\nexplanation for it all!\n\n[conway]: http://en.wikipedia.org/wiki/Conway%27s_Game_of_Life\n[aco]: http://en.wikipedia.org/wiki/Ant_colony_optimization\n[torus]: http://en.wikipedia.org/wiki/Torus\n[hickey]: https://gist.github.com/1093917\n[rubyantsim]: https://github.com/elm-city-craftworks/practicing-ruby-examples/tree/master/v5/009\n[optimizer]: https://github.com/elm-city-craftworks/practicing-ruby-examples/blob/master/v5/009/lib/ant_sim/optimizer.rb\n"
},
{
"path": "articles/v5/010-prototypes.md",
"content": "*This article was written by Avdi Grimm. Avdi is a [Ruby Rogue][rogue], a\nconsulting pair programmer, and the head chef at [RubyTapas][tapas]. He writes\nabout software development at [Virtuous Code.][virtuous]*\n\nWhen you think of the term *object-oriented programming*, one of the\nfirst associated words that springs to mind is probably *classes*. For\nmost of its history, the OOP paradigm has been almost inextricably\nlinked with the idea of classes. Classes serve as *object factories*:\nthey hold the blueprint for new objects, and can be called upon to\nmanufacture as many as needed. Each object, or *instance*, has its\nstate, but each derives its behavior from the class. Classes, in turn,\nshare behavior through inheritance. In most OO programs, the class\nstructure is the primary organizing principle.\n\nEven though classes have gone hand-in-hand with OOP for decades, they\naren't the only way to build families of objects with shared behavior.\nThe most common alternative to *class-based* programming is\n*prototype-based* programming. Languages that use prototypes rather than\nclasses include [Self][self], [Io][io], and (most well known of all) JavaScript.\n\nRuby comes from the class-based school of OO language design. But it's\nflexible enough that with a little cleverness, we can experiment with\nprototype-style coding. In this article that's just what we'll do.\n\n[self]: http://en.wikipedia.org/wiki/Self_(programming_language\n[io]: http://en.wikipedia.org/wiki/Io_(programming_language)\n[rogue]: http://rubyrogues.com/\n[tapas]: http://devblog.avdi.org/rubytapas/\n[virtuous]: http://devblog.avdi.org/\n\n## Getting started\n\nSo how do we write OO programs without classes? Let's explore this\nquestion in Ruby. We'll use the example of a text-adventure game in the\nstyle of \"[Colossal Cave\nAdventure](http://en.wikipedia.org/wiki/Colossal_Cave_Adventure)\". This\nis one of my favorite programming examples for object-oriented systems,\nsince it involves modeling a virtual world of interacting objects,\nincluding characters, items, and interconnected rooms.\n\nWe open up an interactive Ruby session, and start typing. We begin with\nan `adventurer` object. This object will serve as our avatar in the\ngame's world, translating our commands into interactions between\nobjects:\n\n```ruby\nadventurer = Object.new\n```\n\nThe first ability we give to our adventurer is the ability to look at\nits surroundings. The `look` command will cause the adventurer to output\na description of its current location:\n\n```ruby\nclass << adventurer\n attr_accessor :location\n\n def look\n puts location.description\n end\nend\n```\n\nThen we add a starting location, called `end_of_road`, and put the\nadventurer in that location:\n\n```ruby\nend_of_road = Object.new\ndef end_of_road.description\n <\n\n
\n\n\nRather than attempting to rush off to a particular destination that I have planned for you in advance, I'd like you to try forging your own path and see where it takes you. Start with the project's [README](https://github.com/elm-city-craftworks/weiqi/blob/master/README.md) and then follow your interests from there.\n\nOnce you've had a chance to take a look around, consider either coming back here to discuss the things you found interesting, or continuing the conversation over on Github via tickets and pull requests. The code has its good and bad parts (as you'll soon find out), so there is definitely plenty to discuss.\n\nAnd most importantly, please let me know whether this particular experiment seemed to work for you or not. You're welcome to leave a comment below, or if you'd be more comfortable giving feedback in private, you can email me at: **gregory@practicingruby.com**.\n"
},
{
"path": "articles/v6/003-actors.md",
"content": "> This issue was a collaboration with [Alberto Fernández Capel][afcapel], a Ruby developer\nfrom Spain. Although it has been through many revisions since\nwe started, Alberto's ideas, code, and explanations provided\nan excellent starting point that lead us to publish this article.\n\nConventional wisdom says that concurrent programming is hard, especially in \nRuby. This basic assumption is what lead many Rubyists to take an interest\nin languages like Erlang and Scala -- their baked in support for \nthe [actor model][actors] is meant to make concurrent systems \nmuch easier for everyday programmers to implement and understand.\n\nBut do you really need to look outside of Ruby to find concurrency primitives\nthat can make your work easier? The answer to that question probably \ndepends on the levels of concurrency and availability that you require, but\nthings have definitely been shaping up in recent years. In particular, \nthe [Celluloid][celluloid] framework has brought us a convenient and clean way to implement\nactor-based concurrent systems in Ruby.\n\nIn order to appreciate what Celluloid can do for you, you first need to\nunderstand what the actor model is, and what benefits it offers over the\ntraditional approach of directly using threads and locks for concurrent \nprogramming. In this article, we'll try to shed some light on those points by\nsolving a classic concurrency puzzle in three ways: Using Ruby's built-in\nprimitives (threads and mutex locks), using the Celluloid framework, and using a\nminimal implementation of the actor model that we'll build from scratch.\n\nBy the end of this article, you certainly won't be a concurrency expert\nif you aren't already, but you'll have a nice head start on some\nbasic concepts that will help you decide how to tackle concurrent programming\nwithin your own projects. Let's begin!\n\n## The Dining Philosophers Problem\n\nThe [Dining Philosophers][philosophers] problem was formulated by Edsger Djisktra in 1965 to\nillustrate the kind of issues we can find when multiple processes compete to\ngain access to exclusive resources.\n\nIn this problem, five philosophers meet to have dinner. They sit at a round\ntable and each one has a bowl of rice in front of them. There are also five\nchopsticks, one between each philosopher. The philosophers spent their time\nthinking about _The Meaning of Life_. Whenever they get\nhungry, they try to eat. But a philosopher needs a chopstick in each\nhand in order to grab the rice. If any other\nphilosopher has already taken one of those chopsticks, the hungry\nphilosopher will wait until that chopstick is available.\n\nThis problem is interesting because if it is not properly solved it can easily\nlead to deadlock issues. We'll take a look at those issues soon, but first let's\nconvert this problem domain into a few basic Ruby objects.\n\n### Modeling the table and its chopsticks\n\nAll three of the solutions we'll discuss in this article rely on a `Chopstick`\nclass and a `Table` class. The definitions of both classes are shown below:\n\n```ruby\nclass Chopstick\n def initialize\n @mutex = Mutex.new\n end\n\n def take\n @mutex.lock\n end\n\n def drop\n @mutex.unlock\n\n rescue ThreadError\n puts \"Trying to drop a chopstick not acquired\"\n end\n\n def in_use?\n @mutex.locked?\n end\nend\n\nclass Table\n def initialize(num_seats)\n @chopsticks = num_seats.times.map { Chopstick.new }\n end\n\n def left_chopstick_at(position)\n index = (position - 1) % @chopsticks.size\n @chopsticks[index]\n end\n\n def right_chopstick_at(position)\n index = (position + 1) % @chopsticks.size\n @chopsticks[index]\n end\n\n def chopsticks_in_use\n @chopsticks.select { |f| f.in_use? }.size\n end\nend\n```\n\nThe `Chopstick` class is just a thin wrapper around a regular Ruby mutex \nthat will ensure that two philosophers can not grab the same chopstick \nat the same time. The `Table` class deals with the geometry of the problem; \nit knows where each seat is at the table, which chopstick is to the left \nor to the right of that seat, and how many chopsticks are currently in use.\n\nNow that you've seen the basic domain objects that model this problem, we'll\nlook at different ways of implementing the behavior of the philosophers. \nWe'll start with what *doesn't* work.\n\n## A solution that leads to deadlocks\n\nThe `Philosopher` class shown below would seem to be the most straightforward\nsolution to this problem, but has a fatal flaw that prevents it from being\nthread safe. Can you spot it?\n\n```ruby\nclass Philosopher\n def initialize(name)\n @name = name\n end\n\n def dine(table, position)\n @left_chopstick = table.left_chopstick_at(position)\n @right_chopstick = table.right_chopstick_at(position)\n\n loop do\n think\n eat\n end\n end\n\n def think\n puts \"#{@name} is thinking\"\n end\n\n def eat\n take_chopsticks\n\n puts \"#{@name} is eating.\"\n\n drop_chopsticks\n end\n\n def take_chopsticks\n @left_chopstick.take\n @right_chopstick.take\n end\n\n def drop_chopsticks\n @left_chopstick.drop\n @right_chopstick.drop\n end\nend\n```\n\nIf you're still scratching your head, consider what happens when each\nphilosopher object is given its own thread, and all the philosophers attempt to\neat at the same time. \n\nIn this naive implementation, it is\npossible to reach a state in which every philosopher picks up their left-hand\nchopstick, leaving no chopsticks on the table. In that scenario, every\nphilosopher would simply wait forever for their right-hand chopstick to \nbecome available -- resulting in a deadlock. You can reproduce the problem\nby running the following code:\n\n```ruby\nnames = %w{Heraclitus Aristotle Epictetus Schopenhauer Popper}\n\nphilosophers = names.map { |name| Philosopher.new(name) }\ntable = Table.new(philosophers.size)\n\nthreads = philosophers.map.with_index do |philosopher, i|\n Thread.new { philosopher.dine(table, i) }\nend\n\nthreads.each(&:join)\nsleep\n```\n\nRuby is smart enough to inform you of what went wrong, so you should end up\nseeing a backtrace that looks something like this:\n\n```console\nAristotle is thinking\nPopper is eating.\nPopper is thinking\nEpictetus is eating.\nEpictetus is thinking\nHeraclitus is eating.\nHeraclitus is thinking\nSchopenhauer is eating.\nSchopenhauer is thinking\n\ndining_philosophers_uncoordinated.rb:79:in `join': deadlock detected (fatal)\n from dining_philosophers_uncoordinated.rb:79:in `each'\n from dining_philosophers_uncoordinated.rb:79:in `- \" + \n candidates.select { |e| Integer(e['effort']) <= effort }\n .sample(3)\n .map { |e| \"
- #{e['name']} (#{e['label']}) \" }\n .join + \n \"
How about these meals?
\n <%= @selected %>\n \n\n```\n\nIt's worth noting that the code above is about at the level of complexity\nwhere more formal development practices start to pay off. But since I \nmanaged to squeeze three weeks of active use out of the tool before\ngetting to this point, I definitely won't mind doing some cleanup work \nif and when I decide to add more features to it.\n\n**Discussion**\n\nThe main thing I hope you will take away from this article is that \"keeping\nthings simple\" is term that we say often but rarely practice. This can have a\npainful effect on our daily work, but is disasterous for our side projects,\nbecause we often work on them with a tight time budget. \n\nSpeaking from personal experience, I've lost count of how many Rails applications \nskeletons I've built that started with big dreams and ended up with nothing more \nthan a couple database models, a few half-finished CRUD forms, and an\nauthentication system, but no actual features to speak of. I guess I am\njust extremely good at overestimating how much time and motivation I'll\nhave for building the things I think of day to day.\n\nIncreasingly, I've been trying to think of software as a support system for\nsolving human problems, rather than some sort of artifact that holds intrinsic\nvalue. Software is extremely expensive to build and maintain, so it pays to\nwrite as little code as possible. This does not just mean writing terse\nprograms: it means spending more time and creativity on practical problem\nsolving, so that you can focus your energy on making people's lives\neasier rather than obsessing over technical issues. Adopting this mindset\ncan lead you to being more thoughtful when you build software for others,\nbut it also serves as a reminder that you can and should enjoy the fruits \nof your own labor.\n\nEven though programming can be fun in its own right, you don't need to view\nevery software project as an opportunity to solve interesting coding puzzles.\nWhen measured in terms of functional value rather than implementation details,\nsometimes the most elegant solution is a script that you cobbled together during\na lunch break, because it cost you almost nothing but still managed to do\nsomething useful for you. These opportunities appear around every corner, you\njust need to be prepared to take advantage of them when they arise. I hope\nthe story I've shared in this article has helped you learn what to look out for.\n\n*Do you have an example of some code you wrote that took very little effort but\nstill ended up being very useful for you? If so, please share your story in\nthe comments section below.*\n" }, { "path": "articles/v6/README.md", "content": "The articles in this folder are from Practicing Ruby's sixth volume, which ran from\nJanuary 2013 to May 2013.\n\nYou can also read them for free online at practicingruby.com.\n" }, { "path": "articles/v7/001-simulating-tiny-worlds.md", "content": "As programmers we are very comfortable with the idea of using\nsoftware to solve concrete problems. However, it is easy to\nunderestimate the role that code can play in problem-solving itself, and that blindspot can hinder our creativity in a number of ways.\n\nIn this article, I will walk you through some fun examples that demonstrate how code can be used as an exploratory thinking tool, and then reflect upon how that kind of workflow might be applied to everyday programming tasks.\n\n## Setting the stage\n\nThe source of my motivation for writing on this topic is the [StarLogo](http://education.mit.edu/starlogo/) programming environment and Mitchel Resnick's excellent book \"Turtles, Termites, and Traffic Jams\", both of which illustrate the potential for software to be used as a mind-expanding thinking tool.\n\nAs the title of the book implies, StarLogo is an environment that facilitates simplistic modeling of scenarios that occur in the natural world. The purpose of the tool is not to create environments that closely emulate reality, but instead, to encourage exploration and discovery in simple, tightly constrained microcosms. Apart from being an intellectual curiosity, this sort of toolset provides a powerful way to intuitively experience deep concepts that range from self-organization and emergent behavior to massive parallelism.\n\nIn the spirit of exploration, I won't attempt to make a case for those claims by way of a top-down explanation. Instead, we'll now walk through a few scenarios that are easily implemented using StarLogo-style modeling. The examples I've chosen are based on ideas from StarLogo and Resnick's book, but I have ported them to JRuby to allow you to explore the concepts without having to familiarize yourself with a new environment first. The engine I built is called [Terrarium](https://github.com/sandal/terrarium), and it is very much a rough prototype, but it should still be good enough to introduce you to these ideas with minimal friction.\n\n## Scenario I: Forest fires\n\nThe environment in the StarLogo model consists of a two-dimensional grid of \"patches\", which are in some ways similar to cellular automata models such as Conway's Game of Life. \n\nUsing only patch color to represent state, we could apply the following ruleset to simulate a rough sketch of a forest fire:\n\n1. Start by building a forest. For the sake of simplicity, we can begin with an empty grid and then randomly paint some of its patches green.\n\n2. To ignite our fire, we can pick a random patch in the grid and paint it red.\n\n3. Each green patch then needs to repeatedly check to see if any of its neighbors are red, and if so, it becomes red itself, spreading the fire.\n\nApplying these three trivial rules results in the following behavior:\n\n\n\nAlthough this animation should be fairly straightforward to understand, it is worth pointing out one small detail about the geometry of a StarLogo-style world: rather than being an infinite grid like Conway's Game of Life, it is a torus, where the left side is connected to the right, and the top is connected to the bottom. This explains why the fire (which starts at the bottom of the screen) quickly overflows onto the top of the screen in this animation.\n\nThe code that was used to generate this visualization is shown below, and is nearly a direct translation of the rules shown above:\n\n```ruby\nTerrarium::Scenario.define do\n # Rule 1: Build the forest\n patches do\n with_probability(0.5) { set_color :green } \n end\n\n # Rule 2. Start the fire\n random_patch { set_color :red }\n\n # Rule 3. Spread the fire\n patches! do\n if color == :green && neighbors.any? { |e| e.color == :red }\n set_color :red\n end\n end\nend\n```\n\nIt is here where you can catch the first glimpse of what I meant by \"code as a thinking tool\". With Terrarium as our engine and StarLogo-style data modeling, we don't need to think at all about the structure or inner workings of our program, but instead can immediately turn our ideas into code. This takes what would cost us hours in upfront modeling and reduces it to minutes of effort.\n\nBeing able to work at this very high level of abstraction allows us to try variations and experiments as soon as we think of them. A simple idea to try out with this model is to see how the fire spreads at various levels of tree density. You will find that at 50% (which is what is shown above), the fire will pretty much always spread across the forest, but at 30%, the opposite is true. Is there a critical tipping point between those two bounds? If so, why is it there? These are the kinds of thoughts that arise when you can focus on ideas rather than code.\n\n## Scenario II: Infectious disease\n\nAs you may have guessed from the name of the language, StarLogo also implements the *turtle graphics* model found in the Logo programming language. Both languages were developed by same research group at MIT, and so if you are familiar with Logo turtles, you will find that StarLogo's creatures move around the world in a similar way to their classical ancestors.\n\nHowever, that is where the similarities end. While the average Logo turtle lives a solitary life, StarLogo's creatures can be commanded en-masse, in groups of hundreds or thousands. Where the Logo turtle is mostly used for drawing lines (albeit in some very clever ways), the StarLogo creature is capable of having much more complex interactions with its world, including the other creatures in it.\n\nTake for example the problem of modeling the spread of a contagious disease through a population of creatures. If we allow ourselves to paper over any inconsistencies with reality by using a bit of imagination, the following rules are sufficient for emulating this scenario:\n\n1. Arrange a group of healthy creatures into a crowd\n3. Infect some of the creatures with the disease\n4. Allow the creatures to slowly move about their world\n5. The disease will spread from sick to healthy creatures whenever they come into contact with each other.\n6. After a set period of \"sick time\", the creature will either die or recover, based on probability. (Recovered creatures can be re-infected if they come into contact with sick creatures, dead creatures simply disappear.)\n\nWhen applied to a population of 200 StarLogo creatures, these rules produce a pattern similar to what is shown in the following animation:\n\n\n\nHere we see the disease quickly spreading from a few infected individuals to the majority of the population. However, the rate of infection then dampens due to the following factors:\n\n* As the creatures wander around, they become less densely packed together, which reduces the frequency at which they transmit disease to one another.\n\n* If a creature eventually dies from an infection, that stops it from continuing to spread the disease, because it gets removed from the world upon its death.\n\n* If the creature recovers, it can be reinfected, but by then the creatures have already spread out enough to prevent rapid chain reactions from occuring.\n\nAll of these conditions are effected by a number of variables, including population size, population density, duration of sick time, number of initially infected creatures, speed of movement of the creatures, and the probability of death vs. recovery in the infected population. In addition to this, the whole system is subject to some degree of fluctuation due to the randomness in both the movement and initial layout of the population.\n\nTaking a purely analytical approach towards thinking through the relationships between all of these variables would be a challenging task to say the least. However, it does not take much specialized knowledge at all to model this problem using StarLogo-style creatures. In fact, the code below is all you need to implement this scenario. Try reading it one rule at a time while looking at the animation, and you should be able to piece together the main concepts even if you've never heard of StarLogo before reading this article:\n\n```ruby\nTerrarium::Scenario.define do\n healthy_color = :cyan\n sick_color = :yellow\n\n initial_population = 200\n crowd_range = 5..15\n sick_time = 5\n infection_density = 0.02\n movement_speed = 0.2\n \n create_creatures(initial_population)\n \n # rule 1: arrange a group of healthy creatures into a crowd\n creatures do \n lt rand(0..359)\n fd rand(crowd_range)\n\n data[:sick_time] = 0\n set_color healthy_color\n end\n\n # rule 2: infect some creatures\n creatures do\n with_probability(infection_density) do\n set_color sick_color\n \n data[:sick_time] = sick_time\n end\n end\n\n # rule 3: allow the creatures to move about randomly\n creatures! do\n lt rand(1..40)\n rt rand(1..40)\n\n fd movement_speed\n end\n\n # rule 4: spread disease on contact\n creatures! do\n next unless color == healthy_color\n\n if nearby_creatures.any? { |e| e.color == sick_color }\n set_color sick_color\n \n data[:sick_time] = sick_time\n end\n end\n\n # rule 5: recover or die based on probability \n creatures!(1) do\n next unless color == sick_color\n\n if data[:sick_time] > 0\n data[:sick_time] -= 1 \n else\n coinflip ? set_color(healthy_color) : destroy\n end\n end\nend\n```\n\nBecause it's the live interactions in this system that are complex and not its rules, you cannot easily predict the patterns that will emerge from this program by simply reading its source code. However, by repeatedly running the program and testing various assumptions you have about the system, you can rapidly gain an intuitive sense for the patterns that arise. In that sense, exploratory programming environments can have an effect similar to that of plotting a mathematical formula: although they can't give you a precise answer to your question, they can very quickly communicate the main points of a story.\n\n## Scenario III: Rabbits in a cabbage patch\n\nAs you may have already guessed, StarLogo's data model doesn't just give you creatures and patches, but it also supports interactions between the two. Because both the creatures and patches can encapsulate arbitrarily complex data, and because StarLogo provides a solid API for various kinds of common tasks, the richness of behavior that can be expressed through these interactions is mind boggling.\n\nThe full StarLogo environment can tackle problems like ant foraging behavior with ease, a problem that I labored with for weeks and spent two issues of Practicing Ruby on ([Issue 5.8](https://practicingruby.com/articles/92) and [Issue 5.9](https://practicingruby.com/articles/93)). However, the features I've ported from StarLogo into the Terrarium project are somewhat limited, so we'll tackle a more basic scenario that will still give you a sense of how creatures and patches can interact with one another.\n\nWe'll now take a stab at implementing a simple ecosystem in which hungry rabbits wander around doing what rabbits tend to do: eating, procreating, and dying. This is the sort of predator/prey modeling problem that you might find on a school math test, but we'll approach it informally rather than brushing up on our differential equations.\n\nHere are the rules that will get our ecosystem up and running:\n\n1. Create a cabbage patch by randomly coloring some patches green\n2. On each iteration of the simulation, give each patch a small chance to sprout cabbage, facilitating regrowth.\n3. Arrange a crowd of rabbits in the cabbage patch.\n4. Allow the rabbits to wander randomly around the cabbage patch\n5. Rabbits eat any cabbage they encounter. This sets the patch color back to black, and increases the energy of the rabbits.\n6. Rabbits gradually lose energy over time. If their energy is fully depleted, they die.\n7. Rabbits also breed (asexually!) when they have enough energy. The parent's energy is reduced, and then it produces an exact clone of itself at its current location.\n\nOnce set into action, these constaints give rise to the dynamic system you see in the animation below. To make sense of what's going on, ignore the rabbits and focus on the oscillating growing and shrinking of the cabbage patch:\n\n\n\nWhat you're seeing happen here is a basic cycle that tends to proceed in the following fashion:\n\n* Whenever the rabbits have plenty of cabbage to eat, they breed, and their population numbers rise.\n\n* As the rabbit population rises, the cabbage gets eaten more rapidly, reducing the amount of total food available to the rabbits in the cabbage patch.\n\n* As food sources dwindle, rabbits tend to stop breeding and some also die of starvation, causing their population levels to drop.\n\n* A smaller rabbit population leads to slower cabbage consumption, which results in rapid regrowth and plenty of cabbage for the rabbits to eat.\n\n* This in turn leads the rabbits to stop dying from starvation and start breeding again, starting the cycle all over again.\n\nThe fact that we've reproduced this cycle is not a particularly profound result: you could have guessed it without ever bothering to create a simulation. However, if you treat the basic problem as a starting point and then continue your explorations from there, many more surprising results can be found. \n\nIn my casual experiments I found that the system is surprisingly tolerant to singular catastrophic events (such as killing off 90% of the rabbits or the cabbage), because the two populations naturally force each other into balance. However, very small changes to the rate of cabbage regrowth, or to the amount of energy the rabbits gain from eating the cabbage can have disasterous effects that lead to extinction. I found these patterns interesting, because they were opposite to my intuition. \n\nPerhaps a more significant point though is that I doubt I would have even thought to try out those ideas if I were working with a formal equation rather than a dynamic and lively visualization. Because I'm not a visually-oriented learner, this really surprised me!\n\nThe full source code for this scenario is shown below, and you're should skim it at least, but you don't need to get bogged down in the details unless you plan to play around with StarLogo or my Terrarium engine after you're done reading this article. If you're feeling a bit tired by now, you can skip right past it to the next section without losing too much. \n\n```ruby\nTerrarium::Scenario.define do\n cabbage_density = 0.5 \n regrowth_rate = 0.02\n initial_population = 200\n initial_energy = 8\n food_energy = 5\n hatch_threshold = 10\n hatched_energy = 0.25\n\n cabbage_color = :green\n rabbit_color = :white\n soil_color = :black\n\n # rule 1: create cabbage patch\n patches do \n set_color soil_color\n with_probability(cabbage_density) { set_color(cabbage_color) } \n end\n\n # rule 2: cabbage regrowth\n patches! do\n with_probability(regrowth_rate) { set_color(cabbage_color) } \n end\n\n create_creatures(200)\n\n # rule 3: arrange a crowd of rabbits\n creatures do \n lt rand(0..359)\n fd rand(5..25)\n\n data[:energy] = initial_energy\n\n set_color rabbit_color\n end\n\n # rule 4: let the rabbits wander\n creatures! { rt(rand(1...40)); lt(rand(1..40)); fd(1) }\n \n # rule 5: rabbits eat any cabbage they encounter, gaining energy\n creatures! do \n update_patch do |patch| \n if patch.color == cabbage_color\n patch.set_color soil_color\n data[:energy] += food_energy\n end\n end\n end\n \n # rule 6: the rabbits are always losing energy\n creatures! { data[:energy] -= 1 }\n\n # rule 7: when the rabbits run out of energy, they die\n creatures! { destroy if data[:energy] < 1 }\n\n # rule 8: when rabbits have enough energy, they clone themselves\n # (but it costs them some energy)\n creatures! do \n if data[:energy] > hatch_threshold\n data[:energy] *= hatched_energy\n\n hatch\n end\n end\nend\n```\n\n## Exploratory programming as a first-class paradigm?\n\nEven though we've managed to pack a lot of interesting behavior into a small amount of code, the examples I've shown here barely scratch the surface of StarLogo and capabilities. While my Terrarium engine is nothing more than a poor man's implementation of a few of StarLogo's features, the full StarLogo language is elegantly designed and carefully thought out.\n\nBut the goal of this article was not to introduce you to a shiny piece of technological infrastucture, it was meant to get you thinking about a different kind of workflow than what we tend to use day to day. Even through the smudged window I've had you look through, it should be clear to see that the style of programming used in StarLogo has several powerful benefits:\n\n1. Thoughts can be expressed directly\n2. Feedback is given continuously \n3. Failure comes at a very low cost\n4. The problem domain is well constrained\n5. Objects can be directly acted upon\n\nWhile most of the tools I use when I'm programming have at least some of these positive traits, it's rare to experience the effect of all of them simultaneously. However, a few positive examples do come \nto mind. In particular, the various web browser development tools (like Firebug or the tools that ship with Chrome) support this kind of workflow.\n\nWhen it comes to frontend web development tools, I've always been amazed at how much it is possible to incrementally evolve a design by tweaking various page elements until you're happy with them. I think that much of the effectiveness of this technique is due to the benefits listed above. Here is a specific example to illustrate that point:\n\n1. If you want to change a font size of a given block of text, it's as easy as clicking that text and editing a single attribute.\n\n2. You see the results immediately on your screen. \n\n3. If you don't like the results, you can easily revert your changes. And if you made a mistake when you were editing things, it should be immediately obvious based on what does (or doesn't) get displayed on the screen.\n\n4. Although the environment is very sophisticated, the scope is constrained enough where the available actions are fairly clear at any given point in time. \n\n5. Finally, because you are often looking at things within the scope of a single element that you are working with directly, you can use extremely localized thinking without harmful consequences.\n\nUnfortunately, I can't easily come up with similar examples when it comes to backend web frameworks. If you narrow the scope, similar workflows can be applied to very simple HTTP services running on Sinatra, but once you need anything more complex than that it becomes much too broad of a problem to solve.\n\nTo be fair, Rails has some elements baked into it that facilitate a certain amount of exploratory programming (the console, scaffolding, etc.). However, these features have always felt to me as if they were not taken nearly far enough, and that there is still room for a much higher level toolkit, even if it would only be useful for rapid prototyping. \n\nIn an ideal world, I would love to be able to describe a useful full-stack feature in a web application in a dozen lines or less, but I've never seen anything that gets me even close to that level of abstraction. Of course, web architecture is sufficiently obtuse to make this a genuinely hard problem to solve, so I'm not surprised that there isn't an obvious solution out there just yet.\n\nBut web programming (particularly general-purpose web programming) is really at a lower level than where this paradigm really could shine. It seems to me that there is nearly infinite possibility for what one might call \"domain-specific development environments\". For example, could we build programmable tools for book publishers that sit somewhere between a WYSIWYG editor and DocBook XML? Could we build drop-in management panels for business metrics that can be programmed at a high enough level that an analyst could use them with minimal help from their programming team? Is there hope that we can put these kinds of high-powered but easy-to-use tools into the hands of musicians, artists, teachers, and charity volunteers?\n\nPerhaps the best use of a general purpose programming language it to build domain-specific environments that help cross a bridge from low-level infrastructure to high-level ideas. But because this is all just a pie-in-the-sky dream that may never end up becoming a reality, I will let you be the judge! Please share your thoughts in the comments below.\n\n\n\n\n\n\n" }, { "path": "articles/v7/002-http-server.md", "content": "*This article was written by Luke Francl, a Ruby developer living in\nSan Francisco. He is a developer at [Swiftype](https://swiftype.com) where he\nworks on everything from web crawling to answering support requests.*\n\nImplementing a simpler version of a technology that you use every day can\nhelp you understand it better. In this article, we will apply this\ntechnique by building a simple HTTP server in Ruby.\n\nBy the time you're done reading, you will know how to serve files from your\ncomputer to a web browser with no dependencies other than a few standard\nlibraries that ship with Ruby. Although the server\nwe build will not be robust or anywhere near feature complete,\nit will allow you to look under the hood of one of the most fundamental\npieces of technology that we all use on a regular basis.\n\n## A (very) brief introduction to HTTP\n\nWe all use web applications daily and many of us build\nthem for a living, but much of our work is done far above the HTTP level.\nWe'll need come down from the clouds a bit in order to explore\nwhat happens at the protocol level when someone clicks a \nlink to *http://example.com/file.txt* in their web browser. \n\nThe following steps roughly cover the typical HTTP request/response lifecycle:\n\n1) The browser issues an HTTP request by opening a TCP socket connection to\n`example.com` on port 80. The server accepts the connection, opening a\nsocket for bi-directional communication.\n\n2) When the connection has been made, the HTTP client sends a HTTP request:\n\n```\nGET /file.txt HTTP/1.1\nUser-Agent: ExampleBrowser/1.0\nHost: example.com\nAccept: */*\n```\n\n3) The server then parses the request. The first line is the Request-Line which contains\nthe HTTP method (`GET`), Request-URI (`/file.txt`), and HTTP version (`1.1`).\nSubsequent lines are headers, which consists of key-value pairs delimited by `:`. \nAfter the headers is a blank line followed by an optional message body (not shown in\nthis example).\n\n4) Using the same connection, the server responds with the contents of the file:\n\n```\nHTTP/1.1 200 OK\nContent-Type: text/plain\nContent-Length: 13\nConnection: close\n\nhello world\n```\n\n5) After finishing the response, the server closes the socket to terminate the connection.\n\nThe basic workflow shown above is one of HTTP's most simple use cases,\nbut it is also one of the most common interactions handled by web servers.\nLet's jump right into implementing it!\n\n## Writing the \"Hello World\" HTTP server\n\nTo begin, let's build the simplest thing that could possibly work: a web server\nthat always responds \"Hello World\" with HTTP 200 to any request. The following\ncode mostly follows the process outlined in the previous section, but is\ncommented line-by-line to help you understand its implementation details:\n\n```ruby\nrequire 'socket' # Provides TCPServer and TCPSocket classes\n\n# Initialize a TCPServer object that will listen\n# on localhost:2345 for incoming connections.\nserver = TCPServer.new('localhost', 2345)\n\n# loop infinitely, processing one incoming\n# connection at a time.\nloop do\n\n # Wait until a client connects, then return a TCPSocket\n # that can be used in a similar fashion to other Ruby\n # I/O objects. (In fact, TCPSocket is a subclass of IO.)\n socket = server.accept\n\n # Read the first line of the request (the Request-Line)\n request = socket.gets\n\n # Log the request to the console for debugging\n STDERR.puts request\n\n response = \"Hello World!\\n\"\n\n # We need to include the Content-Type and Content-Length headers\n # to let the client know the size and type of data\n # contained in the response. Note that HTTP is whitespace\n # sensitive, and expects each header line to end with CRLF (i.e. \"\\r\\n\")\n socket.print \"HTTP/1.1 200 OK\\r\\n\" +\n \"Content-Type: text/plain\\r\\n\" +\n \"Content-Length: #{response.bytesize}\\r\\n\" +\n \"Connection: close\\r\\n\"\n\n # Print a blank line to separate the header from the response body,\n # as required by the protocol.\n socket.print \"\\r\\n\"\n\n # Print the actual response body, which is just \"Hello World!\\n\"\n socket.print response\n\n # Close the socket, terminating the connection\n socket.close\nend\n```\nTo test your server, run this code and then try opening `http://localhost:2345/anything`\nin a browser. You should see the \"Hello world!\" message. Meanwhile, in the output for\nthe HTTP server, you should see the request being logged:\n\n```\nGET /anything HTTP/1.1\n```\n\nNext, open another shell and test it with `curl`:\n\n```\ncurl --verbose -XGET http://localhost:2345/anything\n```\n\nYou'll see the detailed request and response headers:\n\n```\n* About to connect() to localhost port 2345 (#0)\n* Trying 127.0.0.1... connected\n* Connected to localhost (127.0.0.1) port 2345 (#0)\n> GET /anything HTTP/1.1\n> User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7\n OpenSSL/0.9.8r zlib/1.2.3\n> Host: localhost:2345\n> Accept: */*\n>\n< HTTP/1.1 200 OK\n< Content-Type: text/plain\n< Content-Length: 13\n< Connection: close\n<\nHello world!\n* Closing connection #0\n```\n\nCongratulations, you've written a simple HTTP server! Now we'll \nbuild a more useful one.\n\n## Serving files over HTTP\n\nWe're about to build a more realistic program that is capable of \nserving files over HTTP, rather than simply responding to any request\nwith \"Hello World\". In order to do that, we'll need to make a few \nchanges to the way our server works.\n\nFor each incoming request, we'll parse the `Request-URI` header and translate it into\na path to a file within the server's public folder. If we're able to find a match, we'll\nrespond with its contents, using the file's size to determine the `Content-Length`,\nand its extension to determine the `Content-Type`. If no matching file can be found,\nwe'll respond with a `404 Not Found` error status.\n\nMost of these changes are fairly straightforward to implement, but mapping the\n`Request-URI` to a path on the server's filesystem is a bit more complicated due\nto security issues. To simplify things a bit, let's assume for the moment that a\n`requested_file` function has been implemented for us already that can handle\nthis task safely. Then we could build a rudimentary HTTP file server in the following way:\n\n```ruby\nrequire 'socket'\nrequire 'uri'\n\n# Files will be served from this directory\nWEB_ROOT = './public'\n\n# Map extensions to their content type\nCONTENT_TYPE_MAPPING = {\n 'html' => 'text/html',\n 'txt' => 'text/plain',\n 'png' => 'image/png',\n 'jpg' => 'image/jpeg'\n}\n\n# Treat as binary data if content type cannot be found\nDEFAULT_CONTENT_TYPE = 'application/octet-stream'\n\n# This helper function parses the extension of the\n# requested file and then looks up its content type.\n\ndef content_type(path)\n ext = File.extname(path).split(\".\").last\n CONTENT_TYPE_MAPPING.fetch(ext, DEFAULT_CONTENT_TYPE)\nend\n\n# This helper function parses the Request-Line and\n# generates a path to a file on the server.\n\ndef requested_file(request_line)\n # ... implementation details to be discussed later ...\nend\n\n# Except where noted below, the general approach of\n# handling requests and generating responses is\n# similar to that of the \"Hello World\" example\n# shown earlier.\n\nserver = TCPServer.new('localhost', 2345)\n\nloop do\n socket = server.accept\n request_line = socket.gets\n\n STDERR.puts request_line\n\n path = requested_file(request_line)\n\n # Make sure the file exists and is not a directory\n # before attempting to open it.\n if File.exist?(path) && !File.directory?(path)\n File.open(path, \"rb\") do |file|\n socket.print \"HTTP/1.1 200 OK\\r\\n\" +\n \"Content-Type: #{content_type(file)}\\r\\n\" +\n \"Content-Length: #{file.size}\\r\\n\" +\n \"Connection: close\\r\\n\"\n\n socket.print \"\\r\\n\"\n\n # write the contents of the file to the socket\n IO.copy_stream(file, socket)\n end\n else\n message = \"File not found\\n\"\n\n # respond with a 404 error code to indicate the file does not exist\n socket.print \"HTTP/1.1 404 Not Found\\r\\n\" +\n \"Content-Type: text/plain\\r\\n\" +\n \"Content-Length: #{message.size}\\r\\n\" +\n \"Connection: close\\r\\n\"\n\n socket.print \"\\r\\n\"\n\n socket.print message\n end\n\n socket.close\nend\n```\n\nAlthough there is a lot more code here than what we saw in the\n\"Hello World\" example, most of it is routine file manipulation\nsimilar to the kind we'd encounter in everyday code. Now there\nis only one more feature left to implement before we can serve\nfiles over HTTP: the `requested_file` method.\n\n## Safely converting a URI into a file path\n\nPractically speaking, mapping the Request-Line to a file on the \nserver's filesystem is easy: you extract the Request-URI, scrub \nout any parameters and URI-encoding, and then finally turn that \ninto a path to a file in the server's public folder:\n\n```ruby\n# Takes a request line (e.g. \"GET /path?foo=bar HTTP/1.1\")\n# and extracts the path from it, scrubbing out parameters\n# and unescaping URI-encoding.\n#\n# This cleaned up path (e.g. \"/path\") is then converted into\n# a relative path to a file in the server's public folder\n# by joining it with the WEB_ROOT.\ndef requested_file(request_line)\n request_uri = request_line.split(\" \")[1]\n path = URI.unescape(URI(request_uri).path)\n\n File.join(WEB_ROOT, path)\nend\n```\n\nHowever, this implementation has a very bad security problem that has affected\nmany, many web servers and CGI scripts over the years: the server will happily\nserve up any file, even if it's outside the `WEB_ROOT`.\n\nConsider a request like this:\n\n```\nGET /../../../../etc/passwd HTTP/1.1\n```\n\nOn my system, when `File.join` is called on this path, the \"..\" path components\nwill cause it escape the `WEB_ROOT` directory and serve the `/etc/passwd` file.\nYikes! We'll need to sanitize the path before use in order to prevent this\nkind of problem.\n\n> **Note:** If you want to try to reproduce this issue on your own machine,\nyou may need to use a low level tool like *curl* to demonstrate it. Some browsers change the path to remove the \"..\" before sending a request to the server.\n\nBecause security code is notoriously difficult to get right, we will borrow our\nimplementation from [Rack::File](https://github.com/rack/rack/blob/master/lib/rack/file.rb).\nThe approach shown below was actually added to `Rack::File` in response to a [similar\nsecurity vulnerability](http://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2013-0262) that\nwas disclosed in early 2013:\n\n```ruby\ndef requested_file(request_line)\n request_uri = request_line.split(\" \")[1]\n path = URI.unescape(URI(request_uri).path)\n\n clean = []\n\n # Split the path into components\n parts = path.split(\"/\")\n\n parts.each do |part|\n # skip any empty or current directory (\".\") path components\n next if part.empty? || part == '.'\n # If the path component goes up one directory level (\"..\"),\n # remove the last clean component.\n # Otherwise, add the component to the Array of clean components\n part == '..' ? clean.pop : clean << part\n end\n\n # return the web root joined to the clean path\n File.join(WEB_ROOT, *clean)\nend\n```\n\nTo test this implementation (and finally see your file server in action), \nreplace the `requested_file` stub in the example from the previous section \nwith the implementation shown above, and then create an `index.html` file \nin a `public/` folder that is contained within the same directory as your\nserver script. Upon running the script, you should be able to \nvisit `http://localhost:2345/index.html` but NOT be able to reach any\nfiles outside of the `public/` folder.\n\n## Serving up index.html implicitly\n\nIf you visit `http://localhost:2345` in your web browser, you'll see a 404 Not\nFound response, even though you've created an index.html file. Most real web\nservers will serve an index file when the client requests a directory. Let's\nimplement that.\n\nThis change is more simple than it seems, and can be accomplished by adding\na single line of code to our server script:\n\n```diff\n# ...\npath = requested_file(request_line)\n\n+ path = File.join(path, 'index.html') if File.directory?(path)\n\nif File.exist?(path) && !File.directory?(path)\n# ...\n```\n\nDoing so will cause any path that refers to a directory to have \"/index.html\" appended to\nthe end of it. This way, `/` becomes `/index.html`, and `/path/to/dir` becomes\n`path/to/dir/index.html`.\n\nPerhaps surprisingly, the validations in our response code do not need\nto be changed. Let's recall what they look like and then examine why\nthat's the case:\n\n```ruby\nif File.exist?(path) && !File.directory?(path)\n # serve up the file...\nelse\n # respond with a 404\nend\n```\n\nSuppose a request is received for `/somedir`. That request will automatically be converted by our server into `/somedir/index.html`. If the index.html exists within `/somedir`, then it will be served up without any problems. However, if `/somedir` does not contain an `index.html` file, the `File.exist?` check will fail, causing the server to respond with a 404 error code. This is exactly what we want!\n\nIt may be tempting to think that this small change would make it possible to remove the `File.directory?` check, and in normal circumstances you might be able to safely do with it. However, because leaving it in prevents an error condition in the edge case where someone attempts to serve up a directory named `index.html`, we've decided to leave that validation as it is.\n\nWith this small improvement, our file server is now pretty much working as we'd expect it to. If you want to play with it some more, you can grab the [complete source code](https://github.com/elm-city-craftworks/practicing-ruby-examples/tree/master/v7/002) from GitHub.\n\n## Where to go from here\n\nIn this article, we reviewed how HTTP works, then built a simple web\nserver that can serve up files from a directory. We've also examined\none of the most common security problems with web applications and\nfixed it. If you've made it this far, congratulations! That's a lot\nto learn in one day.\n\nHowever, it's obvious that the server we've built is extremely limited.\nIf you want to continue in your studies, here are a few recommendations\nfor how to go about improving the server:\n\n* According to the HTTP 1.1 specification, a server must minimally\nrespond to GET and HEAD to be compliant. Implement the HEAD response.\n* Add error handling that returns a 500 response to the client\nif something goes wrong with the request.\n* Make the web root directory and port configurable.\n* Add support for POST requests. You could implement CGI by executing\na script when it matches the path, or implement \nthe [Rack spec](http://rack.rubyforge.org/doc/SPEC.html) to\nlet the server serve Rack apps with `call`.\n* Reimplement the request loop using [GServer](http://www.ruby-doc.org/stdlib-2.0/libdoc/gserver/rdoc/GServer.html)\n(Ruby's generic threaded server) to handle multiple connections.\n\nPlease do share your experiences and code if you decide to try any of\nthese ideas, or if you come up with some improvement ideas of your own.\nHappy hacking!\n\n*We'd like to thank Eric Hodel, Magnus Holm, Piotr Szotkowski, and \nMathias Lafeldt for reviewing this article and providing feedback \nbefore we published it.*\n\n> NOTE: If you'd like to learn more about this topic, consider doing the Practicing Ruby self-guided course on [Streams, Files, and Sockets](https://practicingruby.com/articles/study-guide-1?u=dc2ab0f9bb). You've already completed one of its reading exercises by working through this article!\n\n> SEE ALSO: A [similar HTTP server](https://github.com/emilyhorsman/practicing-ruby-examples/blob/v7_002python/v7/002python27/http_server.py) written in Python, contributed by [Emily Horsman](http://emilyhorsman.com/).\n" }, { "path": "articles/v7/003-stats.md", "content": "> This issue was a collaboration with my wife, Jia Wu. Jia is an associate scientist at the Yale Child Study Center, where she spends a good portion of her time analyzing brainwave data from various EEG experiments. Although this article focuses on very basic concepts, her background in statistical programming was very helpful whenever I got stuck on something. That said, if you find any mistakes in this article, you can blame me, not her.\n\nOne human quirk that fascinates me is the huge disparity between our moment-to-moment experiences and our perception of past events. This is something that I've read about a lot in pop-psych books, and also is one of the main reasons that I practice insight meditation. However, it wasn't until I read Daniel Kahneman's book \"Thinking, Fast and Slow\" that I realized just how strongly separated our *experiencing self* is from our *remembering self*. \n\nIn both Kahneman's book and [his talk at TED\n2010](http://www.ted.com/talks/daniel_kahneman_the_riddle_of_experience_vs_memory.html),\nhe uses a striking example comparing two colonoscopy patients who recorded their\npain levels periodically throughout their procedure. Although modern sedation\ntechniques have made this a much less painful procedure, no anethesia was used\nduring this study, which pretty much guaranteed that both patients would\nbe in for an unpleasant experience.\n\nFrom the data Kahneman shows, the first patient had a much shorter procedure \nand reported much less overall pain than the \nsecond patient. However, when asked later about how painful their colonoscopy \nwere, the first patient remembered it to be much more unpleasant than \nthe second patient did. How can that be?\n\nAs it turns out, how an event ends has a lot to do with how we will perceive the overall experience when we recall it down the line. In the colonoscopy study, the first patient reported a high pain spike immediately before the end of their procedure, where the second patient had pain that was gradually reduced before the procedure ended. This is the explanation Kahneman offers as to why the first patient remembered their colonoscopy to be far worse of an experience than the second patient remembered it to be. \n\nThis disparity between experience and memory isn't just a one-off observation -- it's a robust finding, and it is has been repeated in many different contexts. The lesson to be learned here is that we cannot trust our remembering mind to give a faithful account of the things we experience day-to-day. The unfortunate cost that comes along with this reality is that we're not as good about making judgements about our own well being as we could be if we did not have this cognitive limitation.\n\nI thought about this idea for a long time, particularly as it related to my day-to-day happiness. Like most software developers (and probably *all* writers), my work has a lot of highs and lows to it -- so my gut feeling was that my days could be neatly divided into good days and bad days. But because Kahneman had taught me that my intuitions couldn't be trusted, I eventually set out to turn this psychological problem into an engineering problem by recording and analyzing my own mood ratings over time.\n\n## Designing an informal experiment\n\nI wanted my mood study to be rigorous enough to be meaningful on a personal level, but I had no intentions of conducting a tightly controlled scientific study. What I really wanted was to build a simple breadcrumb trail of mood ratings so that I didn't need to rely on memory alone to gauge my overall sense of well-being over time.\n\nAfter thinking through various data collection strategies, I eventually settled on SMS messages as my delivery mechanism. The main reason for going this route was that I needed a polling device that could follow me everywhere, but one that wouldn't badly disrupt whatever I was currently doing. Because I use a terrible phone that pretty much can only be used for phone calls and texting, this approach made it possible for me to regularly update my mood rating without getting sucked into all the things that would distract me on a computer.\n\nTo make data entry easy, I used a simple numerical scale for tracking my mood:\n\n* Very Happy (9): No desire to change anything about my current experience.\n* Happy (7-8): Pleased by the current experience, but may still be slightly tired, distracted, or anxious.\n* Neutral (5-6): Not bothered by my current experience, but not necessarily enjoying it.\n* Unhappy (3-4): My negative feelings are getting in the way of me doing what I want to do.\n* Very Unhappy (1-2): Unable to do what I want because I am overwhelmed with negative feelings.\n\nOriginally I had intended to collect these mood updates over the course of several weeks without any specific questions in mind. However, Jia convinced me that having at least a general sense of what questions I was interested in would help me organize the study better -- so I started to think about what I might be able to observe from this seemingly trivial dataset.\n\nAfter a short brainstorming session, we settled on the following general questions:\n\n* How stable is my mood in general? In other words, how much variance is there over a given time period?\n* Are there any patterns in the high and low points that I experience each day? How far apart are the two?\n* Does day of the week and time of day have any effect on my mood?\n\nThese questions helped me ensure that the data I intended to collect was sufficient. Once we confirmed that was the case, we were ready to start writing some code!\n\n## Building the necessary tools\n\nTo run this study, I used two small toolchains: one for data collection, and one for reporting.\n\nThe job of the data collection toolchain was primarily to deal with sending and receiving text messages at randomized intervals. It stored my responses into database records similar to what you see below:\n\n```\n[{:id=>485, :message=>\"8\", :recorded_at=>1375470054},\n {:id=>484, :message=>\"8\", :recorded_at=>1375465032},\n {:id=>483, :message=>\"8\", :recorded_at=>1375457397},\n {:id=>482, :message=>\"9\", :recorded_at=>1375450750},\n {:id=>481, :message=>\"8\", :recorded_at=>1375411347}, ...]\n```\n\nTo support this workflow, I relied almost entirely on external services, including Twilio and Heroku. As a result, the whole data collection toolchain I built consisted of around 80 lines of code spread across two simple [rake tasks](https://github.com/sandal/dwbh/blob/pr-7.3/Rakefile) and a small Sinatra-based [web service](https://github.com/sandal/dwbh/blob/pr-7.3/dwbh.rb). Here's the basic storyline that describes how these two little programs work:\n\n1. Every ten minutes between 8:00am and 11:00pm each day, the randomizer in the `app:remind` task gets run. It has a 1:6 chance of triggering a mood update reminder.\n\n2. Whenever the randomizer sends a reminder, it does so by hitting the `/send-reminder` route on my web service, which causes Twilio to deliver a SMS message to my phone.\n\n3. I respond to those messages with a mood rating. This causes Twilio to fire a webhook that hits the `/record-mood` route on the Sinatra app with the message data as GET parameters. The response data along with a timestamp are then stored in a database for later processing.\n\n4. Some time later, the reporting toolchain will hit the `/mood-logs.csv` route to download a dump of the whole dataset, which includes the raw data shown above along with a few other computed fields that make reporting easier.\n\nAfter a bit of hollywood magic involving a menagerie of R scripts, some more rake tasks, and a bit of Prawn-based PDF generation code, the reporting toolchain ends up spitting out a [two-page PDF report](http://notes.practicingruby.com/docs/7.3-mood-report.pdf) that looks like what you see below:\n\n[](http://notes.practicingruby.com/docs/7.3-mood-report.pdf)\n\nWe'll be discussing some of the details about how the various graphs get generated and the challenges involved in implementing them later on in this article, but if you want to get a sense of what the Ruby glue code looks in the reporting toolchain, I'd recommend looking at its [Rakefile](https://github.com/elm-city-craftworks/practicing-ruby-examples/blob/pr-7.3/v7/003/Rakefile). The tasks it provides allow me to type `rake generate-report` in a console and cause the following chain of events to happen:\n\n1. The latest mood data get downloaded from the Sinatra app in CSV format.\n\n2. All of the R-based graphing scripts are run, outputting a bunch of image files.\n\n3. A PDF is generated to cleanly present those images in a single document.\n\n4. The CSV data and image files are then be deleted, because they're no longer needed.\n\nBetween this reporting code and the data aggregation toolchain, I ended up with a system that has been very easy to work with for the many weeks that I have been running this study. The whole user experience boils down to pressing a couple buttons on my phone when I'm prompted to do so, and then typing a single command to generate reports whenever I want to take a look at them.\n\nAt a first glance, the way this system is implemented may look a bit like its hung together with shoestrings and glue, but the very loose coupling between its components has made it easy to both work on individual pieces in isolation, and to make significant changes without a ton of rework. It seems like the [worse is better](http://en.wikipedia.org/wiki/Worse_is_better) mantra applies well to this sort of project.\n\nI'd be happy to discuss the design of these two toolchains with you once you've finished this article, but for now let's look at what all those graphs are saying about my mood.\n\n## Analyzing the results\n\nThe full report for my mood study consists of four different graphs generated via the R stats language, each of which gives us a different way of looking at the data:\n\n* Figure 1 provides a view of the average mood ratings across the whole time period\n* Figure 2 tracks the daily minimum and maximums for the whole time period.\n* Figure 3 shows the average mood rating and variance broken out by day of week\n* Figure 4 plots the distribution of the different mood ratings at various times of day.\n\nThe order above is the same as that of the PDF report, and it is essentially sorted by the largest time scales down to the shortest ones. Since that is a fairly natural way to look at this data, we'll discuss it in the same order in this article.\n\n---\n\n**Figure 1 ([view source code](https://github.com/elm-city-craftworks/practicing-ruby-examples/blob/pr-7.3/v7/003/moving-summary.R)):**\n\n\n\nI knew as soon as I started working on this study that I'd want to somehow capture the general trend of the entire data series, but I didn't anticipate how noisy it would be to [plot nearly 500 data points](http://i.imgur.com/NlIlgMI.png), many of which were too close together to visually distinguish from one another. To lessen the noise, I decided to plot a moving average instead of the individual ratings over time, which is what you see in **Figure 1** above.\n\nIt's important to understand the tradeoffs here: by smoothing out the data, I lost the ability to see what the individual ratings were at any given time. However, I gained the ability to easily discern the following bits of useful information:\n\n* How my experiences over a period of a couple days compare to the global average (green horizontal line), and to the global standard deviation (gray horizontal lines). This information could tell me whether my day-to-day experience has been improving or getting worse over time, and also how stable the swings in my mood have been recently compared to what might be considered \"typical\" for me across a large time span.\n\n* Whether my recent mood updates indicated that my mood was trending upward or downward, and roughly how long I could expect that to last.\n\nWithout rigorous statistical analysis and a far less corruptable means of studying myself, these bits of information could never truly predict the future or even be used as the primary basis for decision making. However, the extra information has been helping me put my mind in a historical perspective that isn't purely based on my remembered experiences, and that alone has turned out to be extremely useful to me.\n\n> **Implementation notes ([view source code](https://github.com/elm-city-craftworks/practicing-ruby-examples/blob/pr-7.3/v7/003/moving-summary.R)):**\n>\n> I chose to use an exponentially-smoothed weighted average here, mostly because I wanted to see the trend line change direction as quickly as possible whenever new points of data hinted that my mood was getting better or worse over time. There are lots of different techniques for doing weighted averages, and this one is actually a little more complicated than some of the other options out there. If I had to implement the computations myself I may have chosen a more simple method. But since an exponential moving average function already existed in the [TTR package](http://rss.acs.unt.edu/Rdoc/library/TTR/html/MovingAverages.html), it didn't really cost me any extra effort to model things this way.\n\n>I had first seen this technique used in [The Hacker's Diet](http://www.fourmilab.ch/hackdiet/www/subsection1_2_4_0_4.html#SECTION0240400000000000000), where it proved to be a useful means of cancelling out the noise of daily weight fluctuations so that you could see if you were actually gaining or losing weight. I was hoping it would have the same effect for me with my mood monitoring, and so far it has worked as well as I expected it would.\n\n>\n>It's also worth noting that in this graph, the curve represents something close to a continous time scale. To accomplish this, I converted the UNIX timestamps into fractional days from the moment the study had started. It's not perfect, but it has the neat effect of making visible changes to the graph after even a single new data point has been recorded.\n\n---\n\n**Figure 2 ([view source code](https://github.com/elm-city-craftworks/practicing-ruby-examples/blob/pr-7.3/v7/003/daily-min-max.R)):**\n\n\n\n\nIn a purely statistical sense, the highest and lowest values reported for each day might not be especially relevant. However, the nature of this particular study made me feel it was important to track them. After all, even if the \"average\" mood for two days were both around 7, a day where the lowest mood rating was a 1 will certainly be different sort of day than one where the lowest rating was a 5! For this reason, **Figure 2** shows the extreme high and low for each day in the study. This information is useful for the following purposes:\n\n* Determining what my daily peak experiences are like on average. For example, we can see from this data that there was only one day where I didn't report at least a single rating of 7 or higher, and that most days my high point was either an 8 or 9. \n\n* Determining what my daily low points are like on average. Reading the data shown above, we can see that there were only three days in the entire study that I reported a low rating of 1, but that about one in five days had a low rating of 4 or less. \n\n* Visualizing the range between high and low points on a daily basis. This can be seen by looking at the space between the two lines: the smaller the distance, the smaller the range of the mood swing for that day.\n\nA somewhat obvious limitation of this visualization is that the range of moods recorded in a day do not necessarily reflect the range of moods actually experienced throughout that day. In most of the other ways I've sliced up the dataset, we can hope that averaging will smooth out some of the ill effects of missing information, but this view in particular can be easily corrupted by a single \"missed event\" per day. The key point here is that **Figure 2** can only be viewed as a rough sketch of the overall trend, and not a precise picture of day-to-day experience.\n\n> **Implementation notes ([view source code](https://github.com/elm-city-craftworks/practicing-ruby-examples/blob/pr-7.3/v7/003/daily-min-max.R)):**\n>\n> This was an extremely straightforward graph to produce using standard R functions, so there isn't too much to discuss about it. However, it's worth pointing out for folks who are unfamiliar with R that the support for data aggregation built into the language is excellent. Here is the code that takes the raw mood log entries and rolls them up by daily minimum and maximum:\n>\n> `data_max <- aggregate(rating ~ day, data, max)`\n>\n> `data_min <- aggregate(rating ~ day, data, min)`\n>\n> Because R is such a special-purpose language, it includes many neat data manipulation features similar to this one.\n\n---\n\n**Figure 3 ([view source code](https://github.com/elm-city-craftworks/practicing-ruby-examples/blob/pr-7.3/v7/003/day-of-week.R)):**\n\n\n\n\nThis visualization shows the mean and standard deviation for all mood updates broken out by day of week. Looking at my mood data in this way provides the following information:\n\n* Whether or not certain days of the week have better mood ratings on average than others.\n* Whether or not certain days of the week have more consistent mood ratings than others.\n* What the general ups-and-downs look like in a typical week in my life\n\nIf you look at the data points shown in **Figure 3** above, you'll see that the high points (Monday and Friday) stand out noticeably from the low points (Wednesday and Saturday). However, to see whether that difference is significant or not, we need to be confident that what we're observing isn't simply a result of random fluctuations and noise. This is where some basical statistical tests are needed.\n\nTo test for difference in the averages between days, we ran a one-way ANOVA test, and then did a pairwise test with FDR correction. Based on these tests we were able to show a significant difference (p < 0.01) between Monday+Wednesday, Monday+Saturday, and Friday+Saturday. The difference between Wednesday+Friday was not significant, but was close (p = 0.0547). I don't want to get into a long and distracting stats tangent here, but if you are curious about what the raw results of the computations ended up looking like, take a look at [this gist](https://gist.github.com/sandal/6147469).\n\n> **Implementation notes ([view source code](https://github.com/elm-city-craftworks/practicing-ruby-examples/blob/pr-7.3/v7/003/day-of-week.R)):**\n\n> An annoying thing about R is that despite having very powerful graphing functionality built into the language, it does not have a standard feature for drawing error bars. We use a small [helper function](https://github.com/elm-city-craftworks/practicing-ruby-examples/blob/pr-7.3/v7/003/helpers.R#L2-L5) to handle this work, which is based on code we found in [this blog post](http://bmscblog.wordpress.com/2013/01/23/error-bars-with-r ).\n\n> Apart from the errorbars issue and the calls to various statistical reporting functions, this code is otherwise functionally similar to what is used to generate **Figure 2**.\n\n---\n\n**Figure 4 ([view source code](https://github.com/elm-city-craftworks/practicing-ruby-examples/blob/pr-7.3/v7/003/frequency.R)):**\n\n\n\n\nThe final view of the data shows the distribution of mood ratings broken out by time of day. Because the number of mood ratings recorded in each time period weren't evenly distributed, I decided to plot the frequency of the mood rating values by percentage rather than total count for each. Presenting the data this way allows the five individual graphs to be directly compared to one another, because it ensures that they all use the same scale.\n\nWhenever I look at this figure, it provides me with the following information:\n\n* How common various rating values are, broken out by time of day.\n* How stable my mood is at a given time of day\n* What parts of the day are more or less enjoyable than others on average\n\nThe most striking pattern I saw from the data shown above was that the percentage of negative and negative-leaning ratings gradually increased throughout the day, up until 8pm, and then they rapidly dropped back down to similar levels as the early morning. In the 8am-11am time period, mood ratings of five or under account for about 7% of the overall distribution, but in the 5pm to 8pm slot, they account for about 20% of the ratings in that time period. Finally, the whole thing falls off a cliff in the 8pm-11pm slot and the ratings of five or lower drop back down to under 7%. It will be interesting to see whether or not this pattern holds up over time.\n\n> **Implementation notes ([view source code](https://github.com/elm-city-craftworks/practicing-ruby-examples/blob/pr-7.3/v7/003/frequency.R)):**\n\n> Building this particular visualization turned out to be more complicated than I had hoped for it to be. It may be simply due to my relative inexperience with R, but I found the `hist()` function to be cumbersome to work with due to a bunch of awkward defaults. For example, the default settings caused the mood ratings of 1 and 2 to be grouped together, for reasons I still only vaguely understand. Also, the way that I implemented grouping by time period can probably be improved greatly.\n\n> Feedback on how to clean up this code is welcome!\n\n## Mapping a story to the data\n\nBecause this was a very personal study, and because the data itself has very low scientific validity, I shouldn't embellish the patterns I observed with wild guesses about their root causes. However, I can't resist, so here are some terrible narrations for you to enjoy!\n\n*I learned that although genuine bad days are actually somewhat rare in my life, when they're bad, they can be really bad:*\n\n\n\n*I learned that I probably need to get better at relaxing during my days off:* \n\n\n\n*I learned that like most people, as I get tired it's easier for me to get into a bad mood, and that rest helps recharge my batteries:*\n\n\n\nAlthough these lessons may not be especially profound, it is fun to see even rudimentary evidence for them in the data I collected. If I keep doing this study, I can use these observations to try out some different things in the hopes of optimizing my day-to-day sense of well being.\n\n## Reflections\n\nGiven that this article started with a story about a colonoscopy and ended with an animated GIF, I think it's best to leave it up to you to draw your own conclusions about what you can take away from it. But I would definitely love to hear your thoughts on any part of this project, so please do share them!\n" }, { "path": "articles/v7/004-incremental.md", "content": "When you look at this photograph of highway construction, what do you see?\n\n\n\nIf your answer was \"ugly urban decay\", then you are absolutely right! But because this construction project is only a few miles away from my house, I can tell you a few things about it that reveal a far more interesting story:\n\n* On the far left side of the photo, you can see the first half of a newly constructed suspension bridge. At the time this picture was taken, it was serving five lanes of northbound traffic.\n\n* Directly next to that bridge, cars are driving southbound on what was formerly the northbound side of our old bridge, serving 3 lanes of traffic.\n\n* Dominating the rest of the photograph is the mostly deconstructed southbound side of our old bridge, a result of several months of active work.\n\nSo with those points in mind, what you are looking at here is an *incremental improvement* to a critical traffic bottleneck along the main route between New York City and Boston. This work was accomplished with hardly any service interruptions, despite the incredibly tight constraints on the project. This is legacy systems work at the highest level, and there is much we can learn from it that applies equally well to code as it does to concrete.\n\n## Case study: Improving one of Practicing Ruby's oldest features\n\nNow that we've set the scene with a colorful metaphor, it is time to see how these ideas can influence the way we work on software projects. To do that, I will walk you through a major change we made to practicingruby.com that involved a fair amount of legacy coding headaches. You will definitely see some ugly code along the way, but hopefully a bit of cleverness will shine through as well.\n\nThe improvement that we will discuss is a complete overhaul of Practicing Ruby's content sharing features. Although I've encouraged our readers to share our articles openly since our earliest days, several awkward implementation details made this a confusing process:\n\n* You couldn't just copy-paste links to articles. You needed to explictly click a share button that would generate a public share link for you.\n\n* If you did copy-paste an internal link from the website rather than explicitly generating a share link, those who clicked on that link would be immediately asked for registration information without warning. This behavior was a side-effect of how we did authorization and not an intentional \"feature\", but it was super annoying to folks who encountered it.\n\n* If you visited a public share link while logged in, you'd see the guest view rather than the subscriber view, and you'd need to click a \"log in\" button to see the comments, navbar, etc.\n\n* Both internal paths and share paths were completely opaque (e.g. \"articles/101\" and \"/articles/shared/zmkztdzucsgv\"), making it hard to know what a URL pointed to without\nvisiting it.\n \nDespite these flaws, subscribers did use Practicing Ruby's article sharing mechanism. They also made use of the feature in ways we didn't anticipate -- for example, it became the standard workaround for using Instapaper to read our content offline. As time went on, we used this feature for internal needs as well, whether it was to give away free samples, or to release old content to the public. To make a long story short, one of our most awkward features eventually also became one of the most important.\n\nWe avoided changing this system for quite a long while because we always had something else to work on that seemed more urgent. But after enough time had passed, we decided to pay down our debts. In particular, we wanted to make the following changes:\n\n* We wanted to switch to subscriber-based share tokens rather than generating a new share token for each and every article. As long as a token was associated with an active subscriber, it could then be used to view any of our articles.\n\n* We wanted to clean up and unify our URL scheme. Rather than having internal path like \"/articles/101\" and share path like \"/articles/shared/zmkztdzucsgv\", we would have a single path for both purposes that looked like this:\n\n```\n/articles/improving-legacy-systems?u=dc2ab0f9bb\n```\n\n* We wanted to make sure to be smart about authorization. Guests who visited a link with a valid share key would always see the \"guest view\" of that article, and logged in subscribers would always see the \"subscriber view\". If a key was invalid or missing, the guest would be explicitly told that the page was protected, rather than dropped into our registration process without warning.\n\n* We wanted to make sure to make our links easy to share by copy-paste, whether it was from anywhere within our web interface, from the browser location bar, or even in the emails we send to subscribers. This meant making sure we put your share token pretty much anywhere you might click on an article link.\n\nLaying out this set of requirements helped us figure out where the destination was, but we knew intuitively that the path to get there would be a long and winding road. The system we initially built for sharing articles did not take any of these concepts into account, and so we would need to find a way to shoehorn them in without breaking old behavior in any significant way. We also would need to find a way to do this *incrementally*, to avoid releasing a ton of changes to our system at once that could be difficult to debug and maintain. The rest of this article describes how we went on to do exactly that, one pull request at a time.\n\n> **NOTE:** Throughout this article, I link to the \"files changed\" view of pull requests to give you a complete picture of what changed in the code, but understanding every last detail is not important. It's fine to dig deep into some pull requests while skimming or skipping others.\n\n## Step 1: Deal with authorization failures gracefully\n\nWhen we first started working on practicingruby.com, we thought it would be convenient to automatically handle Github authentication behind the scenes so that subscribers rarely needed to explicitly click a \"sign in\" button in order to read articles. This is a good design idea, but we only really considered the happy path while building and testing it.\n\nMany months down the line, we realized that people would occasionally share internal links to our articles by accident, rather than explicitly generating public links. Whenever that happened, the visitor would be put through our entire registration process without warning, including:\n\n* Approving our use of Github to authorize their account\n* Going through an email confirmation process\n* Getting prompted for credit card information\n\nMost would understandably abandon this process part of the way through. In the best case scenario, our application's behavior would be seen as very confusing, though I'm sure for many it felt downright rude and unpleasant. It's a shame that such a bad experience could emerge from what was actually good intentions both on our part and on whoever shared a link to our content in the first place. Think of what a different experience it might have been if the visitor had been redirected to our landing page where they could see the following message:\n\n\n\nAlthough that wouldn't be quite as nice as getting free access to an article that someone wanted to share with them, it would at least avoid any confusion about what had just happened. My first attempt at introducing this kind of behavior into the system looked like what you see below:\n\n```ruby\nclass ApplicationController < ApplicationController::Base\n # ...\n \n def authenticate\n return if current_authorization \n \n flash[:notice] = \n \"That page is protected. Please sign in or sign up to continue\"\n \n store_location\n redirect_to(root_path)\n end\nend \n```\n\nWe deployed this code and for a few days, it seemed to be a good enough stop-gap measure for resolving this bug, even if it meant that subscribers might need to click a \"sign in\" button a little more often. However, I realized that it was a bit too naive of a solution when I received an email asking why it was necessary to click \"sign in\" in order to make the \"subscribe\" button work. My quick fix had broken our registration system. :cry:\n\nUpon hearing that bad news, I immediately pulled this code out of production after writing a test that proved this problem existed on my feature branch but not in master. A few days later, I put together a quick fix that got my tests passing. My solution was to extract a helper method that decided how to handle authorization failures. The default behavior would be to redirect to the root page and display an error message as we did above, but during registrations, we would automatically initiate a Github authentication as we had done in the past:\n\n```ruby\nclass ApplicationController < ApplicationController::Base\n # ...\n \n def authenticate\n return if current_authorization \n \n store_location\n redirect_on_auth_failure\n end\n \n def redirect_on_auth_failure\n flash[:notice] = \n \"That page is protected. Please sign in or sign up to continue\"\n \n redirect_to(root_path)\n end\nend\n\nclass RegistrationController < ApplicationController\n # ...\n \n def redirect_on_auth_failure\n redirect_to login_path \n end\nend \n```\n\nThis code, though not especially well designed, seemed to get the job done without too much trouble. It also served as a useful reminder that I should be on the lookout for holes in the test suite, which in retrospect should have been obvious given the awkward behavior of the original code. As they say, hindsight is 20/20!\n\n\n> HISTORY: Deployed 2013-07-26 and then reverted a few days later due to the registration bug mentioned above. Redeployed on 2013-08-06, then merged three days later. \n>\n>[View complete diff](https://github.com/elm-city-craftworks/practicing-ruby-web/pull/145/files)\n\n## Step 2: Add article slugs\n\nWhen we first started working on practicingruby.com, we didn't put much thought to what our URLs looked like. In the first few weeks, we were rushing to get features like syntax highlighting and commenting out the door while keeping up with the publication schedule, and so we didn't have much energy to think about the minor details.\n\nEven if it made sense at the time, this is one decision I came to regret. In particular, I disliked the notion that the paths that subscribers saw (e.g. \"/articles/101\") were completely different than the ones we generated for public viewing (e.g. \"/articles/shared/zmkztdzucsgv\"), with no direct way to associate the two. When you add in the fact that both of these URL schemes are opaque, it definitely stood out as a poor design decision on our part.\n\nTechnically speaking, it would be possible to unify the two different schemes using subscriber tokens without worrying about the descriptiveness of the URLs, perhaps using paths like \"/articles/101?u=dc20f9bb\". However, since we would need to be messing around with article path generation as it was, it seemed like a good idea to make those paths much more attractive by adding slugs. The goal was to have a path like: \"/articles/improving-legacy-systems?u=dc2ab0f9bb\". \n\nBecause we knew article slugs would be easy to implement, we decided to build and ship them before moving on to the more complicated changes we had planned to make. The pair of methods below are the most interesting implementation details from this changeset:\n\n```ruby\nclass Article < ActiveRecord::Base\n # ...\n\n def self.[](key)\n find_by_slug(key) || find_by_id(key)\n end\n\n def to_param\n if slug.present?\n slug\n else\n id.to_s\n end\n end\nend\n```\n\nThe `Article[]` method is a drop-in replacement for `Article.find` that allows lookup by slug or by id. This means that both `Article[101]` and `Article['improving-legacy-code']` are valid calls, each of them returning an `Article` object. Because we only call `Article.find()` in a few places in our codebase, it was easy to swap those calls out to use `Article[]` instead.\n\nThe `Article#to_params` method is used internally by Rails to generate paths. So wherever `article_url` or `article_path` get called with an `Article` object, this method will be called to determine what gets returned. If the article has a slug associated, it'll return something like \"/articles/improving-legacy-code\". If it doesn't have a slug set yet, it will return the familiar opaque database ids, i.e. \"/articles/101\".\n\nThere is a bit of an inconsistency in this design worth noting: I chose to override the `to_params` method, but not the `find` method on my model. However, since the former is a method that is designed to be overridden and the latter might be surprising to override, I felt somewhat comfortable with this design decision.\n\nAlthough it's not worth showing the code for it, I also added a redirect to the new style URLs whenever a slug existed for an article. By doing this, I was able to effectively deprecate the old URL style without breaking existing links. While we won't ever disable lookup by database ID, this at least preserves some consistency at the surface level of the application.\n\n> HISTORY: Deployed 2013-08-16 and then merged the next day. Adding slugs to articles was a manual process that I completed a few days after the feature shipped.\n>\n> [View complete diff](https://github.com/elm-city-craftworks/practicing-ruby-web/pull/155/files)\n\n## Step 3: Add subscriber share tokens\n\nIn theory it should have been nearly trivial to implement subscriber-based share tokens. After all, we were simply generating a random string for each subscriber and then appending it to the end of article URLs as a GET parameter (e.g. \"u=dc20f9bb\"). In practice, there were many edge cases that would complicate our implementation.\n\nThe ideal situation would be to override the `article_path` and `article_url` methods to add the currently logged in user's share token to any article links throughout the application. However, we weren't able to find a single place within the Rails call chain where such a global override would make sense. It would easy enough to get this kind of behavior in both our views and controllers by putting the methods in a helper and then mixing that helper into our ApplicationController, but it wasn't easy to take the same approach in our tests and mailers. To make matters worse, some of the places we wanted to use these path helpers would have access to the ones rails provided by default, but would not include our overrides, and so we'd silently lose the behavior we wanted to add.\n\nWe were unable to find an elegant solution to this problem, but eventually settled on a compromise. We built a low level object for generating the URLs with subscriber tokens, as shown below:\n\n```ruby\nclass ArticleLink\n include Rails.application.routes.url_helpers\n\n def initialize(article, params)\n self.article = article\n self.params = params\n end\n\n def path(token)\n article_path(article, params_with_token(token))\n end\n\n def url(token)\n article_url(article, params_with_token(token))\n end\n\n private\n\n attr_accessor :params, :article\n\n def params_with_token(token)\n {:u => token}.merge(params)\n end\nend\n```\n\nThen in our `ApplicationHelper`, we added the following bits of glue code:\n\n```ruby\nmodule ApplicationHelper\n def article_url(article, params={})\n return super unless current_user\n\n ArticleLink.new(article, params).url(current_user.share_token)\n end\n\n def article_path(article, params={})\n return super unless current_user\n\n ArticleLink.new(article, params).path(current_user.share_token)\n end\nend\n```\n\nAdding these simple shims made it so that we got the behavior we wanted in the ordinary use cases of `article_url` and `article_path`, which were in our controllers and views. In our mailers and tests, we opted to use the `ArticleLink` object directly, because we needed to explicitly pass in tokens in those areas anyway. Because it was impossible for us to make this code completely DRY, this convention-based design was the best we could come up with.\n\nAs part of this changeset, I modified the redirection code that I wrote when we were introducing slugs to also take tokens into account. If a subscriber visited a link that didn't include a share token, it would rewrite the URL to include their token. This was yet another attempt at introducing a bit of consistency where there previously was none.\n\n> HISTORY: Deployed code to add tokens upon visiting an article on 2013-08-20, then did a second deploy to update the archives and library links the next day, merged on 2013-08-23.\n>\n> [View complete diff](https://github.com/elm-city-craftworks/practicing-ruby-web/pull/158/files)\n\n## Step 4: Redesign and improve broadcast mailer\n\nI use a very basic web form in our admin panel to send email announcements out to Practicing Ruby subscribers. Originally, this feature relied on sending messages in batches, which was the simple thing to do when we assumed we'd be sending an identical message to everyone:\n\n```ruby\nclass BroadcastMailer < ActionMailer::Base\n def deliver_broadcast(message={})\n @body = message[:body]\n\n user_batches(message) do |users|\n mail(\n :to => \"gregory@practicingruby.com\",\n :bcc => users,\n :subject => message[:subject]\n ).deliver\n end\n end\n\n private\n\n def user_batches(message)\n yield(message[:to]) && return if message[:commit] == \"Test\"\n\n User.where(:notify_updates => true).to_notify.\n find_in_batches(:batch_size => 25) do |group|\n yield group.map(&:contact_email)\n end\n end\nend\n```\n\nDespite being a bit of a hack, this code served us well enough for a fairly long time. It even supported a basic \"test mode\" that allowed me to send a broadcast email to myself before sending it out everyone. However, the design would need to change greatly if we wanted to include share tokens in the article links we emailed to subscribers. We'd need to send out individual emails rather than sending batched messages, and we'd also need to implement some sort of basic mail merge functionality to handle article link generation.\n\nI don't want to get too bogged down in details here, but this changeset turned out to be far more complicated than I expected. For starters, the way we were using `ActionMailer` in our original code was incorrect, and we were relying on undefined behavior without realizing it. Because the `BroadcastMailer` had been working fine for us in production and its (admittedly mediocre) tests were passing, we didn't notice the problem until we attempted to change its behavior. After attempting to introduce code that looked like this, I started to get all sorts of confusing test failures:\n\n```ruby\nclass BroadcastMailer < ActionMailer::Base\n # NOTE: this is an approximation, but it captures the basic idea...\n def deliver_broadcast(message={})\n @body = message[:body]\n\n User.where(:notify_updates => true).to_notify.each do |user|\n mail(:to => user.contact_email, :subject => message[:subject]).deliver\n end\n end\nend\n```\n\nEven though this code appeared to work as expected in development (sending individual emails to each recipient), in my tests, `ActionMailer::Base.deliveries` was returning N copies of the first email sent in this loop. After some more playing around with ActionMailer and semi-fruitless internet searches, I concluded that this was because we weren't using the mailers in the officially sanctioned way. We'd need to change our code so that the mailer returned a `Mail` object, rather than handling the delivery for us.\n\nBecause I didn't want that logic to trickle up into the controller, and because I expected things might get more complicated as we kept adding more features to this object, I decided to introduce an intermediate service object to handle some of the work for us, and then greatly simplify the mailer object. I also wanted to make the distinction between sending a test message and sending a message to everyone more explicit, so I took the opportunity to do that as well. The resulting code ended up looking something similar to what you see below:\n\n```ruby\nclass Broadcaster\n def self.notify_subscribers(params)\n BroadcastMailer.recipients.each do |email|\n BroadcastMailer.broadcast(params, email).deliver\n end\n end\n\n def self.notify_testers(params)\n BroadcastMailer.broadcast(params, params[:to]).deliver\n end\nend\n\nclass BroadcastMailer < ActionMailer::Base\n def self.recipients\n User.where(:notify_updates => true).to_notify.map(&:contact_email)\n end\n\n def broadcast(message, email)\n mail(:to => email,\n :subject => message[:subject])\n end\nend\n```\n\nWith this code in place, I had successfully converted the batch email delivery to individual emails. It was time to move on to adding a bit of code that would give me mail-merge functionality. I decided to use Mustache for this purpose, which would allow me to write emails that look like this:\n\n```\nHere is an awesome article I wrote:\n\n{{#article}}improving-legacy-systems{{/article}}\n```\n\nMustache would then run some code behind the scenes and turn that message body into the following output:\n\n```\nHere is an awesome article I wrote:\n\nhttp://practicingruby.com/articles/improving-legacy-systems?u=dc20f9bb\n```\n\nAs a proof of concept, I wrote a bit of code that handled the article link expansion, but didn't handle share tokens yet. It only took two extra lines in `BroadcastMailer#broadcast` to add this support:\n\n```ruby\nclass BroadcastMailer < ActionMailer::Base\n # ...\n \n def broadcast(message, email)\n article_finder = ->(e) { article_url(Article[e]) }\n\n @body = Mustache.render(message[:body], :article => article_finder)\n\n mail(:to => email,\n :subject => message[:subject])\n end\nend\n```\n\nI deployed this code in production and sent myself a couple test emails, verifying that the article links were getting expanded as I expected them to. I had planned to work on adding the user tokens immediately after running those live tests, but at that moment realized that I had overlooked an important issue related to performance.\n\nPrevious to this changeset, the `BroadcastMailer` was responsible for sending about 16 emails at a time (25 people per email). But now, it would be sending about 400 of them! Even though we use a DelayedJob worker to handle the actual delivery of the messages, it might take some significant amount of time to insert 400 custom-generated emails into the queue. Rather than investigating that problem right away, I decided to get myself some rest and tackle it the next day with Jordan. \n\n> HISTORY: Deployed on 2013-08-22, and then merged the next day.\n>\n> [View complete diff](https://github.com/elm-city-craftworks/practicing-ruby-web/pull/162/files)\n\n## Step 5: Test broadcast mailer's performance\n\nBefore we could go any farther with our work on the broadcast mailer, we needed to check the performance implications of switching to non-batched emails. We didn't need to do a very scientific test -- we just needed to see how severe the slowdown was. Because our previous code ran without a noticeable delay, pretty much anything longer than a second or two would be concerning to us.\n\nTo conduct our test, we first populated our development environment with 2000 users (about 5x as many active users as we had on Practicing Ruby at the time). Then, we posted a realistic email in the broadcast mailer form, and kept an eye on the messages that were getting queued up via the Rails console. After several seconds we hadn't even queued up 100 jobs, so it became clear that performance very well could be a concern.\n\nTo double check our estimates, and to form a more realistic test, we temporarily disabled our DelayedJob worker on the server and then ran the broadcast mailer in our live environment. Although the mailer did finish up queuing its messages without the request timing out, it took about half a minute to do so. With this information in hand, we cleared out the test jobs so that they wouldn't actually be delivered, and then spent a bit of time lost in thought.\n\nUltimately, we learned several important things from this little experiment:\n\n1. The mail building and queuing process was definitely slow enough to worry us.\n2. In the worst case scenario, I would be able to deal with a 30 second delay in delivering broadcasts, but we would need to fix this problem if we wanted to unbatch other emails of ours, such as comment notifications.\n3. The most straightforward way to deal with this problem would be to run the entire mail building and queuing process in the background.\n\nThe first two points were not especially surprising to us, but the third concerned us a bit. While we have had good luck using DelayedJob in conjunction with the MailHopper gem to send email, we had some problems in the past with trying to handle arbitrary jobs with it. We suspected this had to do with some of our dependencies being outdated, but never had time to investigate properly. With our fingers crossed, we decided to hope for the best and plan for the worst.\n\n## Step 6: Process broadcast mails using DelayedJob\n\nOur first stab at backgrounding the work done by\n`Broadcaster.notify_subscribers` was to simply change the call to\n`Broadcaster.delay.notify_subscribers`. \n\nIn theory, this small change should have done the\ntrick: the method is conceptually nothing more than a \"fire and forget\"\nfunction that did not need to interact in any way with its caller. But after\nspending a long time staring at an incredibly confusing error log, we\nrealized that it wasn't safe to assume that DelayedJob would cleanly serialize\na Rails `params` hash. Constructing our own hash to pass into the\n`Broadcaster.notify_subscribers` method resolved those issues, and we ended up\nwith the following code in `BroadcastsController`:\n\n```ruby\nmodule Admin\n class BroadcastsController < ApplicationController\n def create\n # ...\n\n # build our own hash to avoid DelayedJob serialization issues\n message = { :subject => params[:subject],\n :body => params[:body] } \n\n if params[:commit] == \"Test\"\n message[:to] = params[:to]\n\n Broadcaster.notify_testers(message)\n else\n Broadcaster.delay.notify_subscribers(message)\n end\n\n # ...\n end\n end\nend\n```\n\nAfter tweaking our test suite slightly to take this change into account, we\nwere back to green fairly quickly. We experimented with the delayed broadcasts\nlocally and found that it resolved our slowness issue in the UI. The worker\nwould still take a little while to build all those mails and get them queued\nup, but since it was being done in the background it no longer was much of a\nconcern to us.\n\nWe were cautiously optimistic that this small change might fix our issues, so\nwe deployed the code to production and did another live test. Unfortunately,\nthis lead us to a new error condition, and so we had to go back to the drawing board. \nEventually we came across [this Github issue](https://github.com/collectiveidea/delayed_job/issues/350), which hinted (indirectly) that we might be running into one of the many issues with YAML parsing on Ruby 1.9.2.\n\nWe could have attempted to do yet another workaround to avoid updating our\nRuby version, but we knew that this was not the first, second, or even third time that we had been bitten by the fact that we were still running an ancient\nand poorly supported version of Ruby. In fact, we realized that wiping the\nslate clean and provisioning a whole new VPS might be the way to go, because\nthat way we could upgrade all of our platform dependencies at once.\n\nSo with that in mind, Jordan went off to work on getting us a new production\nenvironment set up, and we temporarily put this particular changeset on hold.\nThere was still plenty of work for me to do that didn't rely on upgrading our\nproduction environment, so I kept working against our old server while he tried to spin up a new one.\n\n> HISTORY: Deployed for live testing on 2013-08-23 but then immediately pulled\n> from production upon failure. Redeployed to our new server on 2013-08-30,\n> then merged the following day.\n>\n> [View complete diff](https://github.com/elm-city-craftworks/practicing-ruby-web/pull/164)\n\n## Step 7: Support share tokens in broadcast mailer\n\nNow that we had investigated the performance issues with the mailer and had a plan in place to fix them, it was time for me to finish what I had planned to work on in the first place: adding share tokens to article links in emails.\n\nThe changes to `BroadcastMailer` were fairly straightforward: pass a `User` rather than an email address into the `broadcast` method, and then use `ArticleLink` to generate a customized link based on the subscriber's share token: \n\n```ruby\nclass BroadcastMailer < ActionMailer::Base\n def self.recipients\n User.where(:notify_updates => true).to_notify\n end\n\n def broadcast(message, subscriber)\n article_finder = ->(e) { \n ArticleLink.new(Article[e]).url(subscriber.share_token) \n }\n\n @body = Mustache.render(message[:body], :article => article_finder)\n\n mail(:to => subscriber.contact_email,\n :subject => message[:subject])\n end\nend\n```\n\nThe only complication of rewiring `BroadcastMailer` this way is that it broke our test mailer functionality. Because the test mailer could send a message to any email address (whether there was an account associated with it or not), we wouldn't be able to look up a valid `User` record to pass to the `BroadcastMailer`. The code below shows my temporary solution to this API compatibility problem:\n\n```ruby\nclass Broadcaster\n # ...\n \n def self.notify_testers(params)\n subscriber = Struct.new(:contact_email, :share_token)\n .new(params[:to], \"testtoken\")\n\n BroadcastMailer.broadcast(params, subscriber).deliver\n end\nend\n```\n\nUsing a `Struct` object to generate an interface shim as I've done here is not the most elegant solution, but it gets the job done. A better solution would be to create a container object that could be used by both `notify_subscribers` and `notify_testers`, but I wasn't ready to make that design decision yet.\n\nWith these changes in place, I was able to do some live testing to verify that we had managed to get share tokens into our article links. Now all that remained was to add the logic that would allow these share tokens to permit guest access to articles.\n\n> HISTORY: Deployed 2013-08-24, then merged on 2013-08-29.\n>\n> [View complete diff](https://github.com/elm-city-craftworks/practicing-ruby-web/pull/165)\n\n## Step 8: Allow guest access to articles via share tokens\n\nWith all the necessary underplumbing in place, I was finally ready to model the new sharing mechanism. The end goal was to support the following behavior:\n\n- Subscribers see the full article w. comments whenever they are logged in\n- With a valid token in the URL, guests see our \"shared article\" view.\n- Without a valid token, guests see the \"protected page\" error\n- Links that use a token from an expired account are disabled\n- Old-style share links redirect to the new-style subscriber token links\n\nThe main challenge was that there wasn't an easy way to separate these concepts from each other, at least not in a meaningful way. However, we were able to reuse large chunks of existing code to do this, so most of the work was just tedious rewiring of controller actions while layering in a few more tests here and there.\n\nThe changes that needed to be made to support these behaviors were not that hard to make, but I did feel concerned about how complicated our `ArticlesController#show` action was getting. Including the relevant filters, here is what it looked like after all the changes were made (skim it, but don't bother trying to understand it!):\n\n```ruby\nclass ArticlesController < ApplicationController\n before_filter :find_article, :only => [:show, :edit, :update, :share]\n before_filter :update_url, :only => [:show]\n before_filter :validate_token, :only => [:show]\n\n skip_before_filter :authenticate, :only => [:show, :shared, :samples]\n skip_before_filter :authenticate_user, :only => [:show, :shared, :samples]\n\n def show\n store_location\n decorate_article\n\n if current_user\n mixpanel.track(\"Article Visit\", :title => @article.subject,\n :user_id => current_user.hashed_id)\n\n @comments = CommentDecorator.decorate(@article.comments\n .order(\"created_at\"))\n else\n shared_by = User.find_by_share_token(params[:u]).hashed_id\n\n mixpanel.track(\"Shared Article Visit\", :title => @article.subject,\n :shared_by => shared_by)\n\n render \"shared\"\n end\n end\n\n private\n\n def find_article\n @article = Article[params[:id]]\n\n render_http_error(404) unless @article\n end\n\n def update_url\n slug_needs_updating = @article.slug.present? && params[:id] != @article.slug\n missing_token = current_user && params[:u].blank?\n\n redirect_to(article_path(@article)) if slug_needs_updating || missing_token\n end\n\n def validate_token\n return if current_user.try(:active?)\n\n unless params[:u].present? && \n User.find_by_share_token_and_status(params[:u], \"active\")\n attempt_user_login # helper that calls authenticate + authenticate_user\n end\n end\nend\n```\n\nThis is clearly not a portrait of healthy code! In fact, it looks suspiciously similiar to the code samples that \"lost the plot\" in Avdi Grimm's contributed article on [confident coding](https://practicingruby.com/articles/confident-ruby). That said, it's probably more fair to say that there wasn't much of a well defined plot when this code was written in the first place, and my attempts to modify it only muddied things further. \n\nIt was hard for me to determine whether or not I should attempt to refactor this code right away or wait until later. From a purely technical perspective, the answer was obvious that this code needed to be cleaned up. But looking at it from another angle, I wanted to make sure that the external behavior of the system was what I actually wanted before I invested more time into optimizing its implementation. I didn't have insight at this point in time to answer that question, so I decided to leave the code messy for the time being until I had a chance to see how well the new sharing mechanism performed in production. \n\n> HISTORY: Deployed on 2013-08-26 and then merged on 2013-08-29.\n>\n> [View complete diff](https://github.com/elm-city-craftworks/practicing-ruby-web/pull/173)\n\n## Step 9: Get practicingruby.com running on our new VPS\n\nWhile I kept working on the sharing mechanism, Jordan was busy setting up a new production environment for us. We set up a temporary subdomain new.practicingruby.com for this purpose, and that allowed us to do some live testing while he got everything up and running.\n\nTechnically speaking, we only need to upgrade our Ruby version in order to fix the problems we encountered with DelayedJob, so spinning up a new VPS instance purely for that purpose might sound a bit overkill at first glance. However, starting with a blank slate environment allowed us to upgrade the rest of our serverside dependencies at a relaxed pace, without worrying about potentially causing large amounts of site downtime. Spinning up the new environment in parallel before decommissioning the old one also meant that we could always switch back to our old environment if we encountered any problems during the migration to the new server.\n\nOn 2013-08-30, we decided to migrate to the new environment. The first step was to pull in the delayed broadcast mailer code and do live tests similar to the ones we had done earlier. After we found that those went smoothly, we decided to do a complete end-to-end test by using the new system to deliver an announcement about the planned maintenance downtime. That worked without any issues, and so at that time we were ready to perform the cut over.\n\nWe made sure to copy over all the data from our old environment immediately after putting it into maintenance mode, and then we updated our DNS entries to point practicingruby.com at the new server. After the DNS records propagated, we used a combination of watching our logs and our analytics dashboard (Mixpanel) to see how things were going. There were only two minor hiccups before everything got back to normal:\n\n* We had a small issue with our Oauth configuration on Github, but resolving it was trivial after we realized that it was not in fact a DNS-related problem, but an issue on our end.\n\n* We realized as soon as we spun up our cron jobs on the new server that our integration with Mailchimp's API had broken. The gem we were using was not Ruby 2.0 compatible, but we never realized this in development because we use it only in a tiny cleanup script that runs behind the scenes on the server. Thankfully because we had isolated this dependency from our application code, [changing it was very easy](https://github.com/elm-city-craftworks/practicing-ruby-web/pull/177/files).\n\nThese two issues were the only problems we needed to debug under pressure throughout all the work we described in this article. Given how much we changed under the hood, I am quite proud of that fact.\n\n## Reflections\n\nThe state of practicingruby.com immediately after our server migration was roughly comparable to that of the highway construction photograph that you saw at the beginning of this article: some improvements had been made, but there was still plenty of old cruft left around, and lots of work left to be done before things could be considered finished. My goal in writing this article was not to show a beautiful end result, but instead to illustrate a process that is seldom discussed.\n\nIn the name of preserving realism, I dragged you through some of our oldest and worst code, and also showed you some newer code that isn't much nicer looking than our old stuff. Along the way, we used countless techniques that feel more like plumbing work than interesting programming work. Each step along the way, we used a different technique to glue one bit of code to another bit of code without breaking old behaviors, because there was no good one-size fits all solution to turn to. We got the job done, but we definitely got our hands dirty in the process.\n\nI feel fairly confident that some of the changes I showed in this article are ones that I will be thankful for in the long haul, while others I will come to regret. The trouble of course is knowing which will be which, and only time and experience can get me there. But hopefully by sharing my own experiences with you, you can learn something from my mistakes, too!\n\n> Special thanks goes to Jordan Byron (the maintainer of practicingruby.com) for collaborating with me on this article, and for helping Practicing Ruby run smoothly over the years.\n" }, { "path": "articles/v7/005-low-level.md", "content": "> This issue of Practicing Ruby was directly inspired by Nick Morgan's\n> [Easy 6502](http://skilldrick.github.io/easy6502/) tutorial. While\n> the Ruby code in this article is my own, the bytecode for the\n> Snake6502 game was shamelessly stolen from Nick. Be sure to check\n> out [Easy 6502](http://skilldrick.github.io/easy6502/) if this topic \n> interests you; it's one of the best programming tutorials I've ever seen.\n\n\nThe sea of numbers you see below is about as close to the metal as programming gets:\n\n```\n0600: 20 06 06 20 38 06 20 0d 06 20 2a 06 60 a9 02 85 \n0610: 02 a9 04 85 03 a9 11 85 10 a9 10 85 12 a9 0f 85 \n0620: 14 a9 04 85 11 85 13 85 15 60 a5 fe 85 00 a5 fe \n0630: 29 03 18 69 02 85 01 60 20 4d 06 20 8d 06 20 c3 \n0640: 06 20 19 07 20 20 07 20 2d 07 4c 38 06 a5 ff c9 \n0650: 77 f0 0d c9 64 f0 14 c9 73 f0 1b c9 61 f0 22 60 \n0660: a9 04 24 02 d0 26 a9 01 85 02 60 a9 08 24 02 d0 \n0670: 1b a9 02 85 02 60 a9 01 24 02 d0 10 a9 04 85 02 \n0680: 60 a9 02 24 02 d0 05 a9 08 85 02 60 60 20 94 06 \n0690: 20 a8 06 60 a5 00 c5 10 d0 0d a5 01 c5 11 d0 07 \n06a0: e6 03 e6 03 20 2a 06 60 a2 02 b5 10 c5 10 d0 06 \n06b0: b5 11 c5 11 f0 09 e8 e8 e4 03 f0 06 4c aa 06 4c \n06c0: 35 07 60 a6 03 ca 8a b5 10 95 12 ca 10 f9 a5 02 \n06d0: 4a b0 09 4a b0 19 4a b0 1f 4a b0 2f a5 10 38 e9 \n06e0: 20 85 10 90 01 60 c6 11 a9 01 c5 11 f0 28 60 e6 \n06f0: 10 a9 1f 24 10 f0 1f 60 a5 10 18 69 20 85 10 b0 \n0700: 01 60 e6 11 a9 06 c5 11 f0 0c 60 c6 10 a5 10 29 \n0710: 1f c9 1f f0 01 60 4c 35 07 a0 00 a5 fe 91 00 60 \n0720: a2 00 a9 01 81 10 a6 03 a9 00 81 10 60 a2 00 ea \n0730: ea ca d0 fb 60 \n```\n\nAlthough you probably can't tell by looking at it, what you see here\nis assembled machine code for the venerable 6502 processor that powered \nmany of the classic video games of the 1980s. When executed in simulated\nenvironment, this small set of cryptic instructions produces a minimal\nversion of the Snake arcade game, as shown below:\n\n\n\nIn this article, we will build a stripped down 6502 simulator \nin JRuby that is complete enough to run this game. If you haven't done much \nlow-level programming before, don't worry! Most of what follows is \njust ordinary Ruby code. I will also be showing you a ton of examples \nalong the way, and those should help keep you on track. You might also\nwant to grab [full source code](https://github.com/sandal/vintage) for \nthe simulator, so that you can experiment with it while \nreading through this article.\n\n## Warmup exercise: Reverse engineering Snake6502\n\nAn interesting property of machine code is that if you know its structure,\nyou can convert it back into assembly language. Among other things,\nthe ability to disassemble machine code is useful for debugging and\nexploration purposes. Let's try this out on Snake6502! \n\nThe output below shows memory locations, machine code, and assembly code for the\nfirst 28 instructions of the game. These instructions are responsible for\ninitializing the state of the snake and the apple before the main event \nloop kicks off. You don't need to understand exactly how they work right\nnow, just try to get a feel for how the code in the `hexdump` column corresponds\nto the code in the `assembly` column:\n\n```\naddress hexdump assembly\n------------------------------\n$0600 20 06 06 JSR $0606\n$0603 20 38 06 JSR $0638\n$0606 20 0d 06 JSR $060d\n$0609 20 2a 06 JSR $062a\n$060c 60 RTS\n$060d a9 02 LDA #$02\n$060f 85 02 STA $02\n$0611 a9 04 LDA #$04\n$0613 85 03 STA $03\n$0615 a9 11 LDA #$11\n$0617 85 10 STA $10\n$0619 a9 10 LDA #$10\n$061b 85 12 STA $12\n$061d a9 0f LDA #$0f\n$061f 85 14 STA $14\n$0621 a9 04 LDA #$04\n$0623 85 11 STA $11\n$0625 85 13 STA $13\n$0627 85 15 STA $15\n$0629 60 RTS\n$062a a5 fe LDA $fe\n$062c 85 00 STA $00\n$062e a5 fe LDA $fe\n$0630 29 03 AND #$03\n$0632 18 CLC\n$0633 69 02 ADC #$02\n$0635 85 01 STA $01\n$0637 60 RTS\n```\n\nIf you look at the output carefully, you'll be able to notice some patterns even\nif you don't understand what the instructions themselves are meant to do. For\nexample, each instruction is made up of between 1-3 bytes of machine code. The\nfirst byte in each instruction tells us what operation it is, and the remaining\nbytes (if any) form its operand.\n\nIf you take a look at the first four instructions, it is easy to see that the\nopcode `20` corresponds to the `JSR` instruction. Forming its operand is\nsimilarly straightforward, because it's the same number in both places, \njust with opposite byte order:\n\n```\n20 06 06 -> JSR $0606 \n20 38 06 -> JSR $0638\n20 0d 06 -> JSR $060d\n20 2a 06 -> JSR $062a\n```\n\nIf you ignore the symbols in front of the numbers for the moment, mapping single\nbyte operands is even easier, because they're represented the same way in both\nthe machine code and the assembly code. Knowing that the `85` opcode maps\nto the `STA` operation, it should be easy to see how `11, 13, 15` map to\n`$11, $13, $15` in the following example:\n\n```\n85 11 -> STA $11\n85 13 -> STA $13\n85 15 -> STA $15\n```\n\nBut the symbols in front of the numbers in assembly language obviously mean\nsomething. If you carefully look at the machine code, you'll be able to find\nthat the same operation can have multiple different opcodes, each of which\nidentify a particular kind of operand:\n\n```\na9 0f -> LDA #$0f\na5 fe -> LDA $fe\n```\n\nWithout getting into too much detail here, the example above shows us that both\n`a9` and `a5` correspond to the `LDA` instruction. The difference between the\ntwo opcodes is that `a9` treats its operand as an immediate value, and `a5` \ninterprets it as a memory address. In assembly code, this difference is\nrepresented syntactically (`#$xx` vs. `$xx`), but in the machine code we must\nrely on numbers alone.\n\nThe various ways of interpreting operands (called \"addressing modes\") are\nprobably the most confusing part of working with 6502 code. There are\nabout a dozen of them, and to get Snake6502 running, we need to implement\nmost of them. The good news is that every addressing mode is just a\nroundabout way of converting an operand into a particular address in memory, and once you have that\naddress, the operations themselves do not care about how you computed it.\nOnce you sweep all that stuff under the rug, you can end up with clean\noperation definitions like this:\n\n```ruby\n# NOTE: 'e' refers to the address that was computed from the instruction's\n# operand and addressing mode.\n\nLDA { cpu[:a] = mem[e] } \nSTA { mem[e] = cpu[:a] }\n```\n\nThis realization also tells us that the memory module will not need to take\naddressing modes into account as long as they're precomputed elsewhere. With\nthat in mind, let's get started building a storage model for our simulator.\nWe'll deal with the hairy problem of addressing modes later.\n\n## Memory\n\nExcept for a few registers that are used to store intermediate\ncomputations, the 6502 processor relies on its memory for pretty much\neverything. Program code, data, and the system stack all reside in \nthe same 16-bit addressing space. Even flow control is entirely\ndependent on memory: the program counter itself is nothing more\nthan an address that is used to look up the next instruction to run.\n\nThis \"all in one bucket\" approach is a double-edged sword. It makes it harder to\nwrite safe programs, but the tradeoff is that the storage model itself is very\nsimple. Conceptually, the memory module is nothing more than a mapping \nbetween 16-bit addresses and 8-bit values:\n\n```ruby\ndescribe \"Storage\" do\n let(:mem) { Vintage::Storage.new }\n\n it \"can get and set values\" do\n mem[0x1337] = 0xAE\n\n mem[0x1337].must_equal(0xAE)\n end\n\n # ...\nend\n```\n\nBut because the program counter keeps track of a 'current location' \nin memory at any point in time, there is a lot more we can do with \nthis simple structure. Let's walk through the remaining tests \nfor `Vintage::Storage` to see what else it implements.\n\n**Program loading**\n\nWhen a program is loaded into memory, there is nothing special about the \nway it is stored, it's just like any other data. In a real 6502 processer,\na register is used to store the address of the \nnext instruction to be run, and that address is used to read an opcode\nfrom memory. In our simulator, we can let the `Storage` class keep track \nof this number for us, incrementing it whenever we call \nthe `Storage#next` method.\n\nThe following test shows how to load a program and then walk its code one byte at a time:\n\n```ruby\nit \"can load a bytecode sequence into memory and traverse it\" do\n bytes = [0x20, 0x06, 0x06]\n\n mem.load(bytes)\n mem.pc.must_equal(program_offset) # load() does not increment counter\n\n bytes.each { |b| mem.next.must_equal(b) }\n\n mem.pc.must_equal(program_offset + 3)\nend\n```\n\nThe starting position of the program can be an arbitrary location, but\nto maintain compatibility with the simulator from the Easy6502 tutorial, we\ninitialize the program counter to `0x600`:\n\n```ruby\nlet(:program_offset) { Vintage::Storage::PROGRAM_OFFSET }\n\nit \"sets an initial position of $0600\" do\n program_offset.must_equal(0x0600)\n\n mem.pc.must_equal(program_offset)\nend\n```\n\n**Flow control + branching**\n\nVery rudimentary flow control is supported by setting the \nprogram counter to a particular address, which causes the \nprocessor to `jump` to the instruction at that address:\n\n```ruby\nit \"implements jump\" do\n mem.jump(program_offset + 0xAB)\n\n mem.pc.must_equal(program_offset + 0xAB)\nend\n```\n\nBranching can be implemented by only calling `jump` when a\ncondition is met:\n\n```ruby\nit \"implements conditional branching\" do\n big = 0xAB\n small = 0x01\n\n # a false condition does not affect mem.pc\n mem.branch(small > big, program_offset + 5)\n mem.pc.must_equal(program_offset)\n\n # true condition jumps to the provided address\n mem.branch(big > small, program_offset + 5)\n mem.pc.must_equal(program_offset + 5)\nend\n```\n\nThis test case is a bit contrived, so let's take a look at \nsome real Snake6502 code that illustrates how branching meant to be used:\n\n```\n$064d a5 ff LDA $ff # read the last key pressed on the keyboard\n$064f c9 77 CMP #$77 # check if the key was \"w\" (ASCII code 0x77)\n$0651 f0 0d BEQ $0660 # if so, jump forward to $0660 \n$0653 c9 64 CMP #$64 # check if the key was \"d\" (ASCII code 0x64)\n$0655 f0 14 BEQ $066b # if so, jump forward to $066b\n$0657 c9 73 CMP #$73 # check if the key was \"s\" (ASCII code 0x73)\n$0659 f0 1b BEQ $0676 # if so, jump forward to $0676\n$065b c9 61 CMP #$61 # check if the key was \"a\" (ASCII code 0x61)\n$065d f0 22 BEQ $0681 # if so, jump forward to $0681\n```\n\nPresumably, the code at `$0660` starts a procedure that moves the snake's\nhead up, the code at `$066b` moves it to the right, and so on. In other words,\nif one of these `BEQ` instructions finds a match, it will jump to the right place \nin the code to handle the relevant condition. But if no match is found, the \nprocessor will happily continue on to whatever code comes after this set of \ninstructions in the program.\n\nThe tricky thing about using instructions that rely on `jump` (and consequently,\n`branch`) is that they are essentially GOTO statements. When you see one of\nthese statements in the code, you know exactly what instruction will be executed\nnext, but there's no way of telling if it will ever return to the location\nit was called from. To get around this problem, we need support for subroutines\nthat know how to return to where they've been called from. And to implement\n*those*, we need a system stack.\n\n**Stack operations**\n\nHere are the tests for how we'd like our stack to behave:\n\n```ruby\nlet(:stack_origin) { Vintage::Storage::STACK_ORIGIN }\nlet(:stack_offset) { Vintage::Storage::STACK_OFFSET }\n\nit \"has a 256 element stack between 0x0100-0x01ff\" do\n stack_offset.must_equal(0x0100)\n stack_origin.must_equal(0xff) # this value gets added to the offset\nend\n\nit \"implements stack-like behavior\" do\n mem.sp.must_equal(stack_origin)\n\n mem.push(0x01)\n mem.push(0x03)\n mem.push(0x05)\n\n mem.sp.must_equal(stack_origin - 3)\n\n mem.pull.must_equal(0x05)\n mem.pull.must_equal(0x03)\n mem.pull.must_equal(0x01)\n\n mem.sp.must_equal(stack_origin)\nend\n```\n\nAs the tests indirectly suggest, the stack is a region in memory \nbetween`$0100` and `$01ff`, indexed by a stack pointer (`sp`).\nEach time a value is pushed onto the stack, the value of the \nstack pointer is decremented, and each time a value is pulled, \nthe pointer is incremented. This makes it so that the stack\npointer always tells you where the \"top of the stack\" is.\n\n**Subroutines**\n\nWith a stack in place, we'll have most of what we need to implement\n\"Jump to subroutine\" (`jsr`) and \"Return from subroutine\" (`rts`)\nfunctionality. The behavior of these features will end up \nlooking something like this:\n\n```ruby\nit \"implements jsr/rts\" do\n mem.jsr(0x0606)\n mem.jsr(0x060d)\n\n mem.pc.must_equal(0x060d)\n\n mem.rts\n mem.pc.must_equal(0x0606)\n\n mem.rts\n mem.pc.must_equal(program_offset)\nend\n```\n\nTo make the above test pass, `jsr` needs to `push` the current \nprogram counter onto the stack before executing a `jump` to the \nspecified address. Later when `rts` is called, the address is\npulled out of the stack, and then another `jump` is executed\nto bring you back to where the last `jsr` command was executed.\nThis works fine even in nested subroutine calls, due to the\nnature of how stacks work.\n\nThe only tricky part is that addresses are 16-bit values, but \nstack entries are limited to single byte values. To get around\nthis problem, we need a couple helper functions to convert\na 16-bit number into two bytes, and vice-versa:\n\n```ruby\nit \"can convert two bytes into a 16 bit integer\" do\n mem.int16([0x37, 0x13]).must_equal(0x1337)\nend\n\nit \"can convert a 16 bit integer into two bytes\" do\n mem.bytes(0x1337).must_equal([0x37, 0x13])\nend\n```\n\nThese helpers will also come in handy later, when we need to deal with\naddressing modes.\n\n**Implementation**\n\nBehavior-wise, there is a lot of functionality here. In a high level\nenvironment it would feel a lot like we were mixing distinct concerns,\nbut at the low level we're working at it's understandable that nearly\ninfinite flexibility is desireable.\n\nDespite the conceptual complexity, the `Storage` class is extremely easy to \nimplement. In fact, it takes less than 80 lines of code if you don't\nworry about validations and robustness:\n\n```ruby\nmodule Vintage\n class Storage\n PROGRAM_OFFSET = 0x0600\n STACK_OFFSET = 0x0100\n STACK_ORIGIN = 0xff\n\n def initialize\n @memory = Hash.new(0)\n @pc = PROGRAM_OFFSET\n @sp = STACK_ORIGIN\n end\n\n attr_reader :pc, :sp\n\n def load(bytes)\n index = PROGRAM_OFFSET\n\n bytes.each_with_index { |c,i| @memory[index+i] = c }\n end\n\n def [](address)\n @memory[address]\n end\n\n def []=(address, value)\n @memory[address] = (value & 0xff)\n end\n\n def next\n @memory[@pc].tap { @pc += 1 }\n end\n\n def jump(address)\n @pc = address\n end\n\n def branch(test, address)\n return unless test\n\n @pc = address\n end\n\n def jsr(address)\n low, high = bytes(@pc)\n\n push(low)\n push(high)\n\n jump(address)\n end\n\n def rts\n h = pull\n l = pull\n\n @pc = int16([l, h])\n end\n\n def push(value)\n @memory[STACK_OFFSET + @sp] = value\n @sp -= 1\n end\n\n def pull\n @sp += 1\n\n @memory[STACK_OFFSET + @sp]\n end\n\n def int16(bytes)\n bytes.pack(\"c*\").unpack(\"v\").first\n end\n\n def bytes(num)\n [num].pack(\"v\").unpack(\"c*\")\n end\n end\nend\n```\n\nFor such boring code, its a bit surprising to think that it can be a fundamental\nbuilding block for generic computing. Keep in mind of course that we're building\na simulation and not a real piece of hardware, and we're doing it in one of the\nhighest level languages you can use.\n\nIf it already feels like we're cheating, just wait until you see the next trick!\n\n## Memory-mapped I/O\n\nTo implement Snake6502, our simulator needs to be able to generate random\nnumbers, read keyboard input, and also display graphics on the screen. None of\nthese features are directly supported by the 6502 instruction set, so that means\nthat every individual system had to come up with its own way of doing things.\nThis is one of many things that causes machine code (especially old-school\nmachine code) to not be directly portable from one system to another.\n\nBecause we're trying to get Snake6502 to run in our simulator without modifying\nits bytecode, we're more-or-less constrained to following the approach used by\nthe Easy6502 simulator: memory-mapped I/O.\n\nThis approach is actually very easy to implement in a simulated environment: you\nadd hooks around certain memory addresses so that when they are accessed, they\nexecute some custom code rather than directly reading or writing a \nvalue to memory. In the case of Snake6502, we expect the following behaviors:\n\n* Reading from `$fe` returns a random 8-bit integer.\n* Reading from `$ff` retrieves the ASCII code of the last key \npressed on the keyboard.\n* Writing to addresses between `$0200` to `$05ff` will render\npixels to the screen. (`$0200` is the top-left corner\nof the 32x32 display, and `$05ff` is the bottom-right corner.)\n\nThese features could be added directly to the `Storage` class, but it would\nfeel a bit awkward to clutter up a generic module with some very specific edge\ncases. For that reason, it is probably better to implement them as a module\nmixin: \n\n```ruby\nmodule Vintage\n module MemoryMap\n RANDOMIZER = 0xfe\n KEY_PRESS = 0xff\n PIXEL_ARRAY = (0x0200..0x05ff)\n\n attr_accessor :ui\n\n def [](address)\n case address\n when RANDOMIZER\n rand(0xff)\n when KEY_PRESS\n ui.last_keypress\n else\n super\n end\n end\n\n def []=(k, v)\n super\n\n if PIXEL_ARRAY.include?(k)\n ui.update(k % 32, (k - 0x0200) / 32, v % 16)\n end\n end\n end\nend\n```\n\nYou probably already have a good idea of how `MemoryMap` works from seeing\nits implementation, but it wouldn't hurt to see an example of how it is\nused before we move on. Here's how to display a single pixel on the \nscreen, randomly varying its color until the spacebar (ASCII code 0x20) \nis pressed:\n\n```ruby\nmem = Vintage::Storage.new\nmem.extend(Vintage::MemoryMap)\n\nmem.ui = Vintage::Display.new \n\n(mem[0x0410] = mem[0xfe]) until mem[0xff] == 0x20 \n```\n\nIt's worth noting that this is the only code in the entire simulator that\ndirectly depends on a connection to some sort of user interface, and the\nprotocol consists of just two methods: `ui.update(x, y, color)` and\n`ui.last_keypress`. In our case, we use a JRuby-based GUI, but anything\nelse could be substituted as long as it implemented these two methods.\n\nAt this point, our storage model is pretty much complete. We now can \nturn our attention to various number crunching features.\n\n## Registers and Flags\n\nIn order to get Snake6502 to run, we need all six of\nthe programmable registers that the processor provides. We've handled two of\nthem already (the stack pointer and the program counter), so we just have four\nmore to implement: A, X, Y, and P. A few design constraints will help make this\nwork go a whole lot faster:\n\n* Most of the operations that can be done on A are done the same way on X and Y,\nso we can implement some generic functions that operate on all three of them.\n\n* We can implement the status register (P) as a collection of individual\nattributes, rather than seven 1-bit flags packs into a single byte.\n\n* Because Snake6502 only relies on the (c)arry, (n)egative, and (z)ero flags\nfrom the status register, we can skip implementing the other four status flags \nand still have a playable game.\n\nWith those limitations in mind, let's work through some specs to understand\nhow this model ought to behave. For starters, we'll be building a `Vintage::CPU` \nthat implements three registers and three flags, initializing them all to \nzero by default:\n\n```ruby\ndescribe \"CPU\" do\n let(:cpu) { Vintage::CPU.new }\n\n let(:registers) { [:a, :x, :y] }\n let(:flags) { [:c, :n, :z] }\n \n it \"initializes registers and flags to zero\" do\n (registers + flags).each { |e| cpu[e].must_equal(0) }\n end\n\n #...\nend\n```\n\nIt will be possible to directly set registers via the `#[]=` method, because\nthe behavior will be the same for all three registers:\n\n```ruby\nit \"allows directly setting registers\" do\n registers.each do |e|\n value = rand(0xff)\n\n cpu[e] = value\n cpu[e].must_equal(value)\n end\nend\n```\n\nHowever, because flags don't have the same update semantics as registers, we \nwill not allow directly setting them via `#[]=`:\n\n```ruby\nit \"does not allow directly setting flags\" do\n flags.each do |e|\n value = rand(0xff)\n\n err = -> { cpu[e] = value }.must_raise(ArgumentError)\n err.message.must_equal \"#{e.inspect} is not a register\"\n end\nend\n```\n\nThe carry flag (c) can toggled via the `set_carry` and `clear_carry` \nmethods. We'll need this later for getting the `CPU` into\na clean state whenever we do addition and subtraction \noperations:\n\n```ruby\nit \"allows setting the c flag via set_carry and clear_carry\" do\n cpu.set_carry\n expect_flags(:c => 1)\n\n cpu.clear_carry\n expect_flags(:c => 0)\nend\n```\n\nSome other instructions will require us to set the carry flag\nbased on arbitrary conditions, so we'll need support for that as well:\n\n```ruby\nit \"allows conditionally setting the c flag via carry_if\" do\n # true condition\n x = 3\n cpu.carry_if(x > 1)\n\n expect_flags(:c => 1)\n\n # false condition\n x = 0\n cpu.carry_if(x > 1)\n\n expect_flags(:c => 0)\nend\n```\n\nThe N and Z flags are set based on whatever result the `CPU` last processed:\n\n```ruby\nit \"sets z=1 when a result is zero, sets z=0 otherwise\" do\n cpu.result(0)\n expect_flags(:z => 1)\n\n cpu.result(0xcc)\n expect_flags(:z => 0)\nend\n\nit \"sets n=1 when result is 0x80 or higher, n=0 otherwise\" do\n cpu.result(rand(0x80..0xff))\n expect_flags(:n => 1)\n\n cpu.result(rand(0x00..0x7f))\n expect_flags(:n => 0)\nend\n```\n\nThe `result` method also returns a number truncated to fit in a single byte,\nbecause pretty much every place we could store a number in this system\nexpects 8-bit integers:\n\n```ruby\nit \"truncates results to fit in a single byte\" do\n cpu.result(0x1337).must_equal(0x37)\nend \n```\n\nTo help keep the `CPU` in a consistent state and to simplify the work\ninvolved in many of the 6502 instructions, we automatically call `cpu.result`\nwhenever a register is set via `CPU#[]=`. The tests below show the \nthe effects of that behavior:\n\n```ruby\n it \"implicitly calls result() when registers are set\" do\n registers.each do |e|\n cpu[e] = 0x100\n \n cpu[e].must_equal(0)\n expect_flags(:z => 1, :n => 0)\n\n cpu[e] -= 1\n \n cpu[e].must_equal(0xff)\n expect_flags(:z => 0, :n => 1)\n end\n end\n```\n\nHere's an implementation that satisfies all of the tests we've seen so far:\n\n```ruby\nmodule Vintage\n class CPU\n def initialize\n @registers = { :a => 0, :x => 0, :y => 0 }\n @flags = { :z => 0, :c => 0, :n => 0 }\n end\n\n def [](key)\n @registers[key] || @flags.fetch(key)\n end\n\n def []=(key, value)\n unless @registers.key?(key)\n raise ArgumentError, \"#{key.inspect} is not a register\" \n end\n\n @registers[key] = result(value)\n end\n\n def set_carry\n @flags[:c] = 1\n end\n\n def clear_carry\n @flags[:c] = 0\n end\n\n def carry_if(test)\n test ? set_carry : clear_carry\n end\n\n def result(number)\n number &= 0xff\n\n @flags[:z] = (number == 0 ? 1 : 0)\n @flags[:n] = number[7]\n\n number\n end\n end\nend\n```\n \nPutting it all together, the role of the `CPU` class is mostly just to do some\nbasic numerical housekeeping that will make implementing 6502 instructions\neasier. Consider for example, the `CMP` and `BEQ` operations, which can\nbe used together to form a primitive sort of `if` statement. We saw these two\noperations used together in the earlier example of keyboard input handling:\n\n```\n$064f c9 77 CMP #$77 # check if the key was \"w\" (ASCII code 0x77)\n$0651 f0 0d BEQ $0660 # if so, jump forward to $0660 \n```\n\nUsing a combination of the `CPU` and `Storage` objects we've already built, we'd\nbe able to define the `CMP` and `BEQ` operations as shown below:\n\n```ruby\nCMP do \n cpu.carry_if(cpu[:a] >= mem[e])\n\n cpu.result( cpu[:a] - mem[e] )\nend\n\nBEQ { mem.branch(cpu[:z] == 1, e) }\n```\n\nEven if we ignore the `cpu.carry_if` call, we know from what we've seen\nalready that if `CPU#result` is called with a zero value, it will set the Z flag\nto 1. We also know that when `Storage#branch` is called with a true value, it\nwill jump to the specified address, otherwise it will do nothing at all. Putting\nthose two facts together with the Snake6502 shown above tells us that if the\nvalue in the A register is `0x77`, execution will jump to `$0600`.\n\nAt this point, we're starting to see how 6502 instructions can be\nmapped onto the objects we've already built, and that means we're \nclose to the finish line. Before we get there, we only have two obstacles\nto clear: implementing addressing modes to handle operands, and building\na program runner that knows how to map raw 6502 code to the operation \ndefinitions shown above.\n\n## Addressing Modes\n\n> **NOTE:** The explanation that follows barely scrapes the surface of\nthis topic. If you want to really understand 6502 addressing modes, you should check\nout the [relevant section](http://skilldrick.github.io/easy6502/#addressing)\nin the Easy6502 tutorial.\n\nIn the very first exercise where we disassembled the first few instructions\nof Snake6502, we discovered the presence of several addressing modes\nthat cause operands to be interpreted in various different ways. To get\nthe game running, we will need to handle a total of eight different \naddressing modes.\n\nThis is a lot of different ways to generate an address, and its intimidating \nto realize we're only implementing an incomplete subset of what the 6502 processor \nprovides. However, its important to keep in mind that the only data structure \nwe have to work with is a simple mapping from 16-bit integers to 8-bit \nintegers. Among other things, clever indexing can give us the functionality we'd\nexpect from variables, references, and arrays -- all the stuff that doesn't have\na direct representation in machine code.\n\nI'm going to show the definitions for all of the addressing modes used by\nSnake6502 below, which probably won't make much sense at first glance. But try\nto see if you can figure out what some of this code doing:\n\n```ruby\nmodule Vintage\n module Operand\n def self.read(mem, mode, x, y)\n case mode\n when \"#\" # Implicit \n nil\n when \"@\" # Relative\n offset = mem.next\n\n mem.pc + (offset <= 0x80 ? offset : -(0xff - offset + 1)) \n when \"IM\" # Immediate\n mem.pc.tap { mem.next }\n when \"ZP\" # Zero Page\n mem.next\n when \"ZX\" # Zero Page, X\n mem.next + x\n when \"AB\" # Absolute\n mem.int16([mem.next, mem.next])\n when \"IX\" # Indexed Indirect\n e = mem.next\n\n mem.int16([mem[e + x], mem[e + x + 1]])\n when \"IY\" # Indirect Indexed\n e = mem.next\n\n mem.int16([mem[e], mem[e+1]]) + y\n else\n raise NotImplementedError, mode.inspect\n end\n end\n end\nend\n```\n\nNow let's walk through them one-by-one. You can refer to the source code above as needed\nto make sense of the following examples.\n\n1) The implicit addressing mode is meant for instructions that either don't operate \non a memory address at all, or can infer the address internally. An example\nwe've already seen is the `RTS` operations that is used to return from a subroutine --\nit gets its data from the stack rather than from an operand, making it a single\nbyte instruction. \n\n2) The relative addressing mode is used by branches only. Consider\nthe following example:\n\n```\n$0651 f0 0d BEQ $0660 # if Z=1, jump to $0660 \n```\n\nBy the time the `$0d` operand is read, the program counter will be set to\n`$0653`. If you add these two numbers together, you get the address to jump to\nif Z=1: `$0660`.\n\n3) Immediate addressing is used when you want to have an instruction work on the\noperand itself. To do so, we return the operand's address, then increment the \nprogram counter as normal. In the example below, the computed address (`e`) \nis `0x0650`, and `mem[e] == 0x77`:\n\n```\n$064f c9 77 CMP #$77\n```\n\n4) Zero page addressing is straightforward, it is simply refers to any address\nbetween `$00` and `$ff`. These are convenient for storing program data in, and\nare faster to access because they do not require combining two bytes into a 16\nbit integer. We've already seen copious use of this address mode throughout\nthe examples in this article, particularly when working with keyboard input\n(`$ff`) and random number generation (`$fe`).\n\n5) Zero page, X indexing is used for iterating over some simple sequences in\nmemory. For example, Snake6502 stores the position of each part of the snakes\nbody in byte pairs starting at memory location `$10`. Using this addressing\nmode, it is possible to walk over the array by simply incrementing the X\nregister as you go.\n\n6) We've also seen plenty of examples of absolute addressing, especially when\nlooking at `JSR` operations. The only complication involved in processing\nthese addresses is that two bytes need to be read and then assembled into\na 16bit integer. But since we've had to do that in several places already,\nit should be easy enough to understand.\n\n7) Indexed indirect addressing gives us a way to dynamically compute an address\nfrom other addresses that we've stored in memory. That sounds really confusing,\nbut the following example should help clear it up. The code below is responsible\nfor moving the snake by painting a white pixel at its updated head position, and\npainting a black pixel at its old tail position:\n\n```\n$0720 a2 00 LDX #$00\n$0722 a9 01 LDA #$01\n$0724 81 10 STA ($10,X) \n$0726 a6 03 LDX $03\n$0728 a9 00 LDA #$00\n$072a 81 10 STA ($10,X) \n```\n\nThe first three lines are hardcoded to look at memory locations `$10` and `$11` \nto form an address in the pixel array that refers to the new head of the \nsnake. The next three lines do something similar for the tail of the snake,\nbut with a twist: because the length of the snake is dynamic, it needs to\nbe looked up from memory. This value is stored in memory location `$03`.\nSo to unpack the whole thing, `STA ($10, X)` will take the address `$10`, add to\nit the number of bytes in the whole snake array, and then look up the address\nstored in the last position of that array. That address points to the snake's\ntail in the pixel array, which ends up getting set to black by this instruction.\n\n8) Indirect indexed addressing gives us yet another way to walk over multibyte\nstructures. In nake6502, this addressing mode is only used for drawing the\napple on the screen. Its position is stored in a 16-bit value stored \nin `$00` and `$01`, and the following code is used to set its color to a \nrandom value:\n\n```\n$0719 a0 00 LDY #$00\n$071b a5 fe LDA $fe\n$071d 91 00 STA ($00),Y\n```\n\nThere are bound to be more interesting uses of these addressing modes, but we\nwe've certainly covered enough ground for now! Don't worry if you didn't\nunderstand this section that well, it took me many times reading the Easy6502\ntutorial and the source code for Snake6502 before I figured these out myself.\n\n## 6502 Simulator (finally!)\n\nWe are now finally at the point where all the hard stuff is done, and all that\nremains is to wire up the simulator itself. In other words, it's time for\nthe fun part of the project.\n\nThe input for the simulator will be a binary file containing the\nassembled program code for Snake6502. The bytes in that file not meant to\nbe read as printable characters, but they can be inspected using a hex editor:\n\n```\n$ hexdump examples/snake.rom\n0000000 20 06 06 20 38 06 20 0d 06 20 2a 06 60 a9 02 85\n0000010 02 a9 04 85 03 a9 11 85 10 a9 10 85 12 a9 0f 85\n0000020 14 a9 04 85 11 85 13 85 15 60 a5 fe 85 00 a5 fe\n0000030 29 03 18 69 02 85 01 60 20 4d 06 20 8d 06 20 c3\n0000040 06 20 19 07 20 20 07 20 2d 07 4c 38 06 a5 ff c9\n0000050 77 f0 0d c9 64 f0 14 c9 73 f0 1b c9 61 f0 22 60\n0000060 a9 04 24 02 d0 26 a9 01 85 02 60 a9 08 24 02 d0\n0000070 1b a9 02 85 02 60 a9 01 24 02 d0 10 a9 04 85 02\n0000080 60 a9 02 24 02 d0 05 a9 08 85 02 60 60 20 94 06\n0000090 20 a8 06 60 a5 00 c5 10 d0 0d a5 01 c5 11 d0 07\n00000a0 e6 03 e6 03 20 2a 06 60 a2 02 b5 10 c5 10 d0 06\n00000b0 b5 11 c5 11 f0 09 e8 e8 e4 03 f0 06 4c aa 06 4c\n00000c0 35 07 60 a6 03 ca 8a b5 10 95 12 ca 10 f9 a5 02\n00000d0 4a b0 09 4a b0 19 4a b0 1f 4a b0 2f a5 10 38 e9\n00000e0 20 85 10 90 01 60 c6 11 a9 01 c5 11 f0 28 60 e6\n00000f0 10 a9 1f 24 10 f0 1f 60 a5 10 18 69 20 85 10 b0\n0000100 01 60 e6 11 a9 06 c5 11 f0 0c 60 c6 10 a5 10 29\n0000110 1f c9 1f f0 01 60 4c 35 07 a0 00 a5 fe 91 00 60\n0000120 a2 00 a9 01 81 10 a6 03 a9 00 81 10 60 a2 00 ea\n0000130 ea ca d0 fb 60\n0000135\n```\n\nThe challenge that is left to be completed is to process\nthe opcodes and operands in this file and turn them into\na running program. To do that, we will make use of a CSV file \nthat lists the operation name and addressing mode for each opcode \nfound in file:\n\n```\n00,BRK,#\n10,BPL,@\n18,CLC,#\n20,JSR,AB\n# ... rest of instructions go here ...\nE6,INC,ZP\nE8,INX,#\nE9,SBC,IM\nF0,BEQ,@\n```\n\nOnce we know the addressing mode for a given operation, we can read its\noperand and turn it into an address (denoted by `e`). And once we have *that*, \nwe can execute the commands that are defined in following DSL:\n\n```ruby\n# NOTE: This file contains definitions for every instruction used \n# by Snake6502. Most of the functionality here is a direct result\n# of simple calls to Vintage::Storage and Vintage::CPU instances.\n\nNOP { }\nBRK { raise StopIteration }\n\n\nLDA { cpu[:a] = mem[e] }\nLDX { cpu[:x] = mem[e] }\nLDY { cpu[:y] = mem[e] }\n\nTXA { cpu[:a] = cpu[:x] }\n\nSTA { mem[e] = cpu[:a] }\n\n## Counters\n\nINX { cpu[:x] += 1 }\nDEX { cpu[:x] -= 1 }\n\nDEC { mem[e] = cpu.result(mem[e] - 1) }\nINC { mem[e] = cpu.result(mem[e] + 1) } \n\n## Flow control\n\nJMP { mem.jump(e) }\n\nJSR { mem.jsr(e) }\nRTS { mem.rts }\n\nBNE { mem.branch(cpu[:z] == 0, e) }\nBEQ { mem.branch(cpu[:z] == 1, e) }\nBPL { mem.branch(cpu[:n] == 0, e) }\nBCS { mem.branch(cpu[:c] == 1, e) }\nBCC { mem.branch(cpu[:c] == 0, e) }\n\n## Comparisons\n\nCPX do \n cpu.carry_if(cpu[:x] >= mem[e])\n\n cpu.result(cpu[:x] - mem[e]) \nend\n\nCMP do \n cpu.carry_if(cpu[:a] >= mem[e])\n\n cpu.result(cpu[:a] - mem[e]) \nend\n\n\n## Bitwise operations\n\nAND { cpu[:a] &= mem[e] }\nBIT { cpu.result(cpu[:a] & mem[e]) }\n\nLSR do\n t = (cpu[:a] >> 1) & 0x7F\n \n cpu.carry_if(cpu[:a][0] == 1)\n cpu[:a] = t\nend\n\n## Arithmetic\n\nSEC { cpu.set_carry }\nCLC { cpu.clear_carry }\n\nADC do \n t = cpu[:a] + mem[e] + cpu[:c]\n\n cpu.carry_if(t > 0xff)\n cpu[:a] = t\nend\n\nSBC do\n t = cpu[:a] - mem[e] - (cpu[:c] == 0 ? 1 : 0)\n\n cpu.carry_if(t >= 0)\n cpu[:a] = t\nend\n```\n\nWe can treat both the opcode lookup CSV and the instructions definitions DSL \nas configuration files, to be loaded into the configuration object \nshown below:\n\n```ruby\nrequire \"csv\"\n\nmodule Vintage\n class Config\n CONFIG_DIR = \"#{File.dirname(__FILE__)}/../../config\"\n\n def initialize(name)\n load_codes(name)\n load_definitions(name)\n end\n\n attr_reader :definitions, :codes\n\n private\n\n def load_codes(name)\n csv_data = CSV.read(\"#{CONFIG_DIR}/#{name}.csv\")\n .map { |r| [r[0].to_i(16), [r[1].to_sym, r[2]]] }\n\n @codes = Hash[csv_data]\n end\n\n def load_definitions(name)\n @definitions = {}\n\n instance_eval(File.read(\"#{CONFIG_DIR}/#{name}.rb\"))\n end\n\n def method_missing(id, *a, &b)\n return super unless id == id.upcase\n\n @definitions[id] = b\n end\n end\nend\n```\n\nThen finally, we can tie everything together with a `Simulator` object that\ninstantiates all the objects we need, and kicks off a program execution loop:\n\n```ruby\nmodule Vintage\n class Simulator\n EvaluationContext = Struct.new(:mem, :cpu, :e)\n \n def self.run(file, ui)\n config = Vintage::Config.new\n cpu = Vintage::CPU.new\n mem = Vintage::Storage.new\n\n mem.extend(MemoryMap)\n mem.ui = ui\n \n mem.load(File.binread(file).bytes)\n\n loop do\n code = mem.next\n\n op, mode = config.codes[code]\n if name\n e = Operand.read(mem, mode, cpu[:x], cpu[:y])\n\n EvaluationContext.new(mem, cpu, e)\n .instance_exec(&config.definitions[op])\n else\n raise LoadError, \"No operation matches code: #{'%.2x' % code}\"\n end\n end\n end\n end\nend\n```\n\nAt this point, you're ready to play Snake! Or if you've been following closely\nalong with this article all the way to the end, you're probably more likely to\nhave a cup of coffee or take a nap from information overload. Either way,\ncongratulations for making it all the way through this long and winding\nissue of Practicing Ruby!\n\n## Further Reading\n\nThis article and the [Vintage simulator](http://github.com/sandal/vintage) is built on top of a ton of other\npeople's ideas and learning resources. Here are some of the works I referred to\nwhile researching this topic:\n\n* [Easy 6502](http://skilldrick.github.io/easy6502/) by Nick Morgan\n* [Mos Technology 6502](http://en.wikipedia.org/wiki/MOS_Technology_6502) @ Wikipedia\n* [Rockwell 6502 Programmer's Manual](http://homepage.ntlworld.com/cyborgsystems/CS_Main/6502/6502.htm) by Bluechip\n* [NMos 6502 opcodes](http://www.6502.org/tutorials/6502opcodes.html) by John Pickens\n* [r6502](https://github.com/joelanders/r6502) by Joe Landers\n" }, { "path": "articles/v7/006-infrastructure.md", "content": "> This issue of Practicing Ruby was a collaboration with Mathias Lafeldt\n([@mlafeldt](https://twitter.com/mlafeldt)), an Infrastructure\nDeveloper living in Hamburg, Germany. If Mathias had to choose the one\nInternet meme that best describes his work, it would certainly be\n_Automate all the things!_ \n\nFor at least as long as Ruby has been popular among web developers, it has also\nbeen recognized as a useful tool for system administration work. Although it was\nfirst used as a clean alternative to Perl for adhoc scripting, Ruby quickly\nevolved to the point where it became an excellent platform for large scale \ninfrastructure automation projects. \n\nIn this article, we'll explore realistic code that handles various system\nautomation tasks, and discuss what benefits the automated approach has over \ndoing things the old-fashioned way. We'll also see first-hand what it means to treat\n\"infrastructure as code\", and the impact it has on building maintainable systems. \n\n## Prologue: Why does infrastructure automation matter?\n\nTwo massive infrastructure automation systems have been built in \nRuby ([Puppet][puppet] and [Chef][chef]), both of which have entire open-source\necosystems supporting them. But because these frameworks were built by and for\nsystem administrators, infrastructure automation is often viewed as a\nspecialized skillset by Ruby programmers, rather than something that everyone\nshould learn. This is probably an incorrect viewpoint, but it is one that is\neasy to hold without realizing the consequences.\n\nSpeaking from my own experiences, I had always assumed that infrastructure\nautomation was a problem that mattered mostly for large-scale public web\napplications, internet service providers, and very complicated enterprise\nprojects. In those kinds of environments, the cost of manually setting up\nservers would obviously be high enough to justify using a \nsophisticated automation framework. But because I never encountered those\nscenarios in my own work, I was content to do things the old-fashioned way:\nreading lots of \"works for me\" instructions from blog posts, manually typing\ncommands on the console, and swearing loudly whenever I broke something. For\nthings that really matter or tasks that seemed too tough for me to do on my own,\nI'd find someone else to take care of it for me.\n\nThe fundamental problem was that my system-administration related pain wasn't \nsevere enough to motivate me to learn a whole new way of doing things. Because\nI never got curious enough about the topic, I didn't realize that infrastructure \nautomation has other benefits beyond eliminating the costs\nof doing repetitive and error-prone manual configuration work. In particular,\nI vastly underestimated the value of treating \"infrastructure as code\",\nespecially as it relates to creating systems that are abstract, modular,\ntestable, understandable, and utterly hackable. Narrowing the problem down to\nthe single issue of reducing repetitive labor, I had failed to see that\ninfrastructure automation has the potential to eliminate an entire class of\nproblems associated with manual system configuration.\n\nTo help me get unstuck from this particular viewpoint, Mathias Lafeldt offered\nto demonstrate to me why infrastructure automation matters, even if you aren't\nmaintaining hundreds of servers or spending dozens of hours a week babysitting\nproduction systems. To teach me this lesson, Mathias built a [Chef cookbook][pr-cookbook] to completely automate the process of building an environment suitable for running [Practicing Ruby's web application][pr-web], starting with nothing but a bare Ubuntu\nLinux installation. The early stages of this process weren't easy: Jordan and I\nhad to answer more questions about our system setup than I\never thought would be necessary. But as things fell into place and\nrecipes started getting written, the benefits of being able to conceptualize a\nsystem as code rather than as an amorphous blob of configuration files and\ninterconnected processes began to reveal themselves.\n\nThe purpose of this article is not to teach you how to get up and running with\nChef, nor is it meant to explain every last detail of the cookbook that\nMathias built for us. Instead, it will help you learn about the core concepts of\ninfrastructure automation the same way I did: by tearing apart a handful of real\nuse cases and seeing what you can understand about them. If you've never used\nan automated system administration workflow before, or if you've only ever run\ncookbooks that other people have provided for you, this article will give you a\nmuch better sense of why the idea of treating \"infrastructure as code\" matters.\nIf you already know the answer to that question, you may still benefit from\nlooking at the problem from a beginner's mindset. In either case, we have\na ton of code to work our way through, so let's get started!\n\n## A recipe for setting up Ruby \n\nLet's take a look at how Chef can be used \nto manage a basic Ruby installation. As you can see below, Chef\nuses a pure Ruby domain-specific language for defining its recipes,\nso it should be easy to read even if you've never worked with\nthe framework before:\n\n```ruby\ninclude_recipe \"ruby_build\"\n\nruby_version = node[\"practicingruby\"][\"ruby\"][\"version\"]\n\nruby_build_ruby(ruby_version) { prefix_path \"/usr/local\" }\n\nbash \"update-rubygems\" do\n code \"gem update --system\"\n not_if \"gem list | grep -q rubygems-update\"\nend\n\ngem_package \"bundler\"\n```\n\nAt the high level, this recipe is responsible for handling the following tasks: \n\n1. Installing the `ruby-build` command line tool.\n2. Using `ruby-build` to compile and install Ruby to `/usr/local`.\n3. Updating RubyGems to the latest version.\n4. Installing the bundler gem.\n\nUnder the hood, a lot more is happening. Let's take a closer look at each\nstep to understand a bit more about how Chef recipes work.\n\n**Installing ruby-build**\n\n```ruby\ninclude_recipe \"ruby_build\"\n```\n\nIncluding the default recipe from the [ruby_build cookbook](https://github.com/fnichol/chef-ruby_build) \nin our own code takes care of installing the `ruby-build` command line utility, \nand also handles installing a bunch of low-level packages that are required to compile Ruby \non an Ubuntu system. But all of this work happens behind the scenes -- we just need \nto make use of the `ruby_build_ruby` command this cookbook provides and the rest will be \ntaken care of for us.\n\n**Compiling and installing Ruby**\n\n```ruby\nruby_version = node[\"practicingruby\"][\"ruby\"][\"version\"]\n\nruby_build_ruby(ruby_version) { prefix_path \"/usr/local\" }\n```\n\nIn our recipe, the version of Ruby we want to install is not specified\nexplicitly, but instead set elsewhere using Chef's attribute system.\nIn the cookbook's [default attributes file][pr-cookbook-attributes], you'll find an entry that\nlooks like this:\n\n```ruby\ndefault[\"practicingruby\"][\"ruby\"][\"version\"] = \"2.0.0-p247\"\n```\n\nChef has a very flexible and very complicated [attribute management system][chef-attributes], but its main purpose is the same as any configuration \nsystem: to keep source code as generic as possible by not hard-coding\napplication-specific values. By getting these values out of the\nsource file and into well-defined locations, it also makes it\neasy to see all of our application-specific configuration \ndata at once.\n\n**Updating RubyGems**\n\n```ruby\nbash \"update-rubygems\" do\n code \"gem update --system\"\n not_if \"gem list | grep -q rubygems-update\"\nend\n```\n\nIn this code we make use of a couple shell commands, the \nfirst of which is obviously responsible for updating RubyGems.\nThe second command is a guard that prevents the gem update\ncommand from running more than once.\n\nMost actions in Chef have similar logic baked into them to\nmake sure operations are only carried out when necessary. These \nguard clauses are handled internally whenever there is a well defined \ncondition to check for, so you don't need to think about them often.\nIn the case of shell commands the operation is potentially arbitrary,\nso a custom guard clause is necessary.\n\n**Installing bundler**\n\n```ruby\ngem_package \"bundler\"\n```\n\nThis command is roughly equivalent to typing `gem install bundler` on the\ncommand line. Because we installed Ruby into `/usr/local`, it will be used as\nour system Ruby, and so we can use `gem_package` without any additional\nsettings. More complicated system setups would involve a bit more\ncode than what you see above, but for our purposes we're able to keep \nthings simple.\n\nPutting all of these ideas together, we end up not just with an understanding of\nhow to go about installing Ruby using a Chef recipe, but also a glimpse\nof a few of the benefits of treating \"infrastructure as code\". As we\ncontinue to work through more complicated examples, those benefits\nwill become even more obvious.\n\n## A recipe for setting up process monitoring \n\nNow that we've tackled a simple example of a Chef recipe, let's work through \na more interesting one. The following code is what we use for installing\nand configuring the [God][god] process monitoring framework:\n\n```ruby\ninclude_recipe \"practicingruby::_ruby\"\n\ngem_package \"god\"\n\ndirectory \"/etc/god\" do\n owner \"root\"\n group \"root\"\n mode \"0755\"\nend\n\nfile \"/etc/god/master.conf\" do\n owner \"root\"\n group \"root\"\n mode \"0644\"\n notifies :restart, \"service[god]\"\n\n home = node[\"practicingruby\"][\"deploy\"][\"home_dir\"] \n god_file = \"#{home}/current/config/delayed_job.god\"\n\n content \"God.load('#{god_file}') if File.file?('#{god_file}')\"\nend\n\ncookbook_file \"/etc/init/god.conf\" do\n source \"god.upstart\"\n owner \"root\"\n group \"root\"\n mode \"0644\"\nend\n\nservice \"god\" do\n provider Chef::Provider::Service::Upstart\n action [:enable, :start]\nend\n```\n\nThe short story about this recipe is that it handles the following tasks:\n\n1. Installing the `god` gem.\n2. Setting up some configuration files for `god`.\n3. Registering `god` as a service to run at system boot.\n4. Starting the `god` service as soon as the recipe is run.\n\nBut that's just the 10,000 foot view -- let's get down in the weeds a bit.\n\n**Installing god via RubyGems**\n\n```ruby\ninclude_recipe \"practicingruby::_ruby\"\n\ngem_package \"god\"\n```\n\nGod is distributed as a gem, so we need to make sure Ruby is installed\nbefore we can make use of it. To do this, we include the Ruby installation\nrecipe that was shown earlier. If the Ruby recipe hasn't run yet, it will\nbe executed now, but if it has already run then `include_recipe` will\ndo nothing at all. In either case, we can be sure that we have a\nworking Ruby configuration by the time the `gem_package` command is called.\n\nThe `gem_package` command itself works exactly the same way as it did when we\nused it to install Bundler in the Ruby recipe, so there's nothing new to say\nabout it.\n\n**Setting up a master configuration file**\n\n```ruby\ndirectory \"/etc/god\" do\n owner \"root\"\n group \"root\"\n mode \"0755\"\nend\n\nfile \"/etc/god/master.conf\" do\n owner \"root\"\n group \"root\"\n mode \"0644\"\n notifies :restart, \"service[god]\"\n\n home = node[\"practicingruby\"][\"deploy\"][\"home_dir\"] \n god_file = \"#{home}/current/config/delayed_job.god\"\n\n content \"God.load('#{god_file}') if File.file?('#{god_file}')\"\nend\n```\n\nA master configuration file is typically used with God to load\nall of the process-specific configuration files for a whole system \nwhen God starts up. In our case, we only have one process to watch, \nso our master configuration is a simple one-line shim that points at the\n[delayed_job.god][pr-web-dj] file that is deployed alongside our Rails \napplication.\n\nBecause our `/etc/god/master.conf` file is so trivial, we directly specify \nits contents in the recipe itself rather than using one of Chef's more\ncomplicated mechanisms for dealing with configuration files. In this\nparticular case, manually creating the file would certainly involve\nless work, but we'd lose some of the benefits that Chef is providing here.\n\nIn particular, it's worth noticing that file permissions and ownership\nare explicitly specified in the recipe, that the actual location\nof the file is configurable, and that Chef will send a notification\nto restart God whenever this file changes. All of these things\nare the sort of minor details that are easily forgotten when\nmanually managing configuration files on servers.\n\n**Running god as a system service**\n\nGod needs to be running at all times, so we want to make sure that it started on\nsystem reboot and cleanly terminated when the system is shut down. To do that, we\ncan configure God to run as an Upstart service. To do that, we need to create\nyet another configuration file:\n\n```ruby\ncookbook_file \"/etc/init/god.conf\" do\n source \"god.upstart\"\n owner \"root\"\n group \"root\"\n mode \"0644\"\nend\n```\n\nThe `cookbook_file` command used here is similar to the `file` command, but has a\nspecialized purpose: To copy files from a cookbook's `files` directory to\nsome location on the system being automated. In this case, we're\nusing the `files/default/god.upstart` cookbook file as our source, and it\nlooks like this:\n\n```\ndescription \"God is a monitoring framework written in Ruby\"\n\nstart on runlevel [2345]\nstop on runlevel [!2345]\n\npre-start exec god -c /etc/god/master.conf\npost-stop exec god terminate\n```\n\nHere we can see exactly what commands are going to be used to start and \nshutdown God, as well as the runlevels that it will be started and\nstopped on. We can also see that the `/etc/god/master.conf` file we\ncreated earlier will be loaded by God whenever it starts up.\n\nNow all that remains is to enable the service to run when the system\nboots, and also tell it to start up right now:\n\n```ruby\nservice \"god\" do\n provider Chef::Provider::Service::Upstart\n action [:enable, :start]\nend\n```\n\nIt's worth mentioning here that if we didn't explicitly specify the\n`Service::Upstart` provider, Chef would expect the service\nconfiguration file to be written as a [System-V init\nscript][god-init], which are written at a much lower level of abstraction. There\nisn't anything wrong with doing things that way, but Upstart\nscripts are definitely more readable. \n\nBy this point, we've already seen how Chef can be used to install packages,\nmanage configuration files, run arbitrary shell commands, \nand set up system services. That knowledge alone will take you far,\nbut let's look at one more recipe to discover a few more \nadvanced features before we wrap things up.\n\n## A recipe for setting up an Nginx web server \n\nThe recipe we use for configuring Nginx is the most complicated one in\nPracticing Ruby's cookbook, but it mostly just combines and expands upon the\nconcepts we've already discussed. Try to see what you can\nunderstand of it before reading the explanations that follow, but don't\nworry if every last detail isn't immediately clear to you:\n\n```ruby\nnode.set[\"nginx\"][\"worker_processes\"] = 4\nnode.set[\"nginx\"][\"worker_connections\"] = 768\nnode.set[\"nginx\"][\"default_site_enabled\"] = false\n\ninclude_recipe \"nginx::default\"\n\nssl_dir = ::File.join(node[\"nginx\"][\"dir\"], \"ssl\")\ndirectory ssl_dir do\n owner \"root\"\n group \"root\"\n mode \"0600\"\nend\n\ndomain_name = node[\"practicingruby\"][\"rails\"][\"host\"]\nbash \"generate-ssl-files\" do\n cwd ssl_dir\n flags \"-e\"\n code <<-EOS\n DOM=#{domain_name}\n openssl genrsa -out $DOM.key 4096\n openssl req -new -batch -subj \"/CN=$DOM\" -key $DOM.key -out $DOM.csr\n openssl x509 -req -days 365 -in $DOM.csr -signkey $DOM.key -out $DOM.crt\n rm $DOM.csr\n EOS\n notifies :reload, \"service[nginx]\"\n not_if { ::File.exists?(::File.join(ssl_dir, domain_name + \".crt\")) }\nend\n\ntemplate \"#{node[\"nginx\"][\"dir\"]}/sites-available/practicingruby\" do\n source \"nginx_site.erb\"\n owner \"root\"\n group \"root\"\n mode \"0644\"\n variables(:domain_name => domain_name)\nend\n\nnginx_site \"practicingruby\" do\n enable true\nend\n```\n\nWhen you put all the pieces together, this recipe is responsible for the\nfollowing tasks:\n\n1. Overriding some default Nginx configuration values.\n2. Installing Nginx and managing it as a service.\n3. Generating a self-signed SSL certificate based on a configurable domain name.\n4. Using a template to generate a site-specific configuration file.\n5. Enabling Nginx to serve up our Rails application.\n\nIn this recipe even more than the others we've looked at, a lot of the details\nare handled behind the scenes. Let's dig a bit deeper to see what's really\ngoing on.\n\n**Installing and configuring Nginx**\n\nWe rely on the nginx cookbook to do most of the hard work of \nsetting up our web server for us. Apart\nfrom overriding a few default attributes, we only need to include the\n`nginx:default` recipe into our own code to install the relevant software \npackages, generate an `nginx.conf` file, and to provide all the necessary\ninit scripts to manage Nginx as a service. The following four lines\nof code take care of all of that for us:\n\n```ruby\nnode.set[\"nginx\"][\"worker_processes\"] = 4\nnode.set[\"nginx\"][\"worker_connections\"] = 768\nnode.set[\"nginx\"][\"default_site_enabled\"] = false\n\ninclude_recipe \"nginx::default\"\n```\n\nThe interesting thing to notice here is that unlike the typical server\nconfiguration file, only the things we explicitly changed are visible here.\nAll the rest of the defaults are set automatically for us, and we don't\nneed to be concerned with their values until the time comes when we decide we\nneed to change them. By hiding all the details that do not matter to us,\nChef recipes tend to be much more intention revealing than\nthe typical server configuration file.\n\n**Generating SSL keys**\n\nIn a real production environment, we would probably copy SSL credentials\ninto place rather than generating them on the fly. However, since\nthis particular cookbook provides a blueprint for building an experimental testbed\nrather than an exact clone of our live system, we handle this task internally to make the system a little bit more developer-friendly.\n\nThe basic idea behind the following code is that we want to generate an SSL\ncertificate and private key for whatever domain name you'd like, so that \nit is possible to serve up the application over SSL within a virtualized \nstaging environment. But since that is somewhat of an obscure use case, you\ncan focus on what interesting Chef features are being used\nin the following code rather than the particular shell code being executed:\n\n```ruby\nssl_dir = ::File.join(node[\"nginx\"][\"dir\"], \"ssl\")\ndirectory ssl_dir do\n owner \"root\"\n group \"root\"\n mode \"0600\"\nend\n\ndomain_name = node[\"practicingruby\"][\"rails\"][\"host\"]\nbash \"generate-ssl-files\" do\n cwd ssl_dir\n flags \"-e\"\n code <<-EOS\n DOM=#{domain_name}\n openssl genrsa -out $DOM.key 4096\n openssl req -new -batch -subj \"/CN=$DOM\" -key $DOM.key -out $DOM.csr\n openssl x509 -req -days 365 -in $DOM.csr -signkey $DOM.key -out $DOM.crt\n rm $DOM.csr\n EOS\n notifies :reload, \"service[nginx]\"\n not_if { ::File.exists?(::File.join(ssl_dir, domain_name + \".crt\")) }\nend\n```\n\nAs you read through this code, you may have noticed that `::File` is used\ninstead of `File`, which looks a bit awkward. The problem here is that\nChef defines its own `File` class that ends up having a naming collision with\nRuby's core class. So to safely make use of Ruby's `File` class, we need to\nexplicitly do our constant lookup from the top-level namespace. This is just a\nsmall side effect of how Chef's recipe DSL is implemented, but it is\nworth noting to clear up any confusion.\n\nWith that distraction out of the way, we can skip right over the `directory`\ncode which we've seen in earlier recipes, and turn our attention to the `bash`\ncommand and its options. This example is far more interesting than the one we\nused to update RubyGems earlier, because in addition to specifying a command to\nexecute and a `not_if` guard clause, it also does all of the following things:\n\n* Switches the working directory to the SSL directory we created within our Nginx directory.\n* Sets the `-e` flag, which will abort the script if any command fails to run successfully.\n* Uses a service notification to tell Nginx to reload its configuration files\n\nFrom this we see that executing shell code via a Chef recipe isn't quite the\nsame thing as simply running some commands in a console. The entire surrounding\ncontext is also specified and verified, making it a whole lot more likely\nthat things will work the way you expect them to. If these benefits were\nharder to see in the Ruby installation recipe, they should be easier to\nrecognize now.\n\n**Configuring Nginx to serve up Practicing Ruby**\n\nAlthough the [nginx cookbook](https://github.com/opscode-cookbooks/nginx) takes care \nof setting up our `nginx.conf` file for us, it does not manage site \nconfigurations for us. We need to take care of that ourselves and\ntweak some settings dynamically, so that means telling our\nrecipe to make use of a template:\n\n```ruby\ntemplate \"#{node[\"nginx\"][\"dir\"]}/sites-available/practicingruby\" do\n source \"nginx_site.erb\"\n owner \"root\"\n group \"root\"\n mode \"0644\"\n variables(:domain_name => domain_name)\nend\n```\n\nThe [full template](https://github.com/elm-city-craftworks/practicing-ruby-cookbook/blob/master/templates/default/nginx_site.erb)\nis a rather long file full of the typical Nginx boilerplate, but the small\nexcerpt below shows how it is customized using ERB to insert some dynamic\ncontent:\n\n```erb\nserver {\n listen 80;\n server_name <%= \"#{@domain_name} www.#{@domain_name}\" %>;\n rewrite ^ https://$server_name$request_uri? permanent;\n}\n```\n\nOnce the configuration file is generated and stored in the right place, we\nenable it using the following command:\n\n```ruby\nnginx_site \"practicingruby\" do\n enable true\nend\n```\n\nUnder the hood, the [nxensite](https://github.com/Dreyer/nxensite) script is used \nto do the actual work of enabling the site, but that implementation detail is \ndeliberately kept hidden from view.\n\nAt this point, we have studied enough features of Chef to establish a basic\nliteracy that will facilitate reading a wide range of recipes with only\na little bit of effort. At the very least, you now have enough\nknowledge to make sense of every recipe in Practicing Ruby's cookbook.\n\n## A cookbook for building a (mostly) complete Rails environment \n\nThe goal of this article was to give you a sense of what kinds of building\nblocks that Chef recipes are made up of so that you could see various\ninfrastructure automation concepts in practice. If you feel like you've\nmade it that far, you may now be interested in looking at how a complete\nautomation project is sewn together.\n\nThe full [Practicing Ruby cookbook][pr-cookbook] contains a total of eight recipes,\nthree of which we've already covered in this article. The five recipes\nwe did not discuss are responsible for handling the\nfollowing chores:\n\n* Creating and managing a deployment user account to be used by Capistrano.\n* Installing PostgreSQL and configuring a database for use with our Rails app.\n* Configuring Unicorn and managing it as an Upstart service.\n* Setting up some folders and files needed to deploy our Rails app.\n* Installing and managing MailCatcher as a service, to make email testing easier.\n\nIf you are curious about how these recipes work, go ahead and read them! Many\nare thin wrappers around external cookbook dependencies, and none of them use\nany Chef features that we haven't already discussed. Attempting to\nmake sense of how these recipes work would be a great way to test your \nunderstanding of what we covered in this article.\n\nIf you want to take things a step farther, you can actually try to provision a\nproduction-like environment for Practicing Ruby on your own system. The\ncookbook's [README file](https://github.com/elm-city-craftworks/practicing-ruby-cookbook#readme) is fairly detailed, and we have things set up to work within a\nvirtual machine that can run in isolation without having a negative impact\non your own development environment. We also simplify a few things to make\nsetup easier, such as swapping out GitHub authentication for OmniAuth developer\nmode, making most service integrations optional, and other little tweaks that\nmake it possible to try things out without having to do a bunch of \nconfiguration work.\n\nI absolutely recommend trying to run our cookbook on your own to learn a whole\nlot more about Chef, but fair warning: to do so you will need to become familiar\nwith the complex network of underplumbing that we intentionally avoided\ndiscussing in this article. It's not too hard to work your way through, but\nexpect some turbulence along the way.\n\n## Epilogue: What are the costs of infrastructure automation?\n\nThe process of learning from Practicing Ruby's cookbook, and the act\nof writing this article really convinced me that I had greatly underestimated\nthe potential benefits that infrastructure automation has to offer. However, it\nis important to be very clear on one point: there's no such thing as a \nfree lunch.\n\nAt my current stage of understanding, I feel the same about Chef as I do about\nRails: impressed by its vast capabilities, convinced of its utility, and shocked\nby its complexity. There are a tremendous amount of moving parts that you need\nto understand before it becomes useful, and many layers of subsystems that need\nto be wired up before you can actually get any of your recipes to run.\n\nAnother concern is that \"infrastructure as code\" comes with the drawbacks\nassociated with code and not just the benefits. Third-party cookbooks vary in\nquality and sometimes need to be patched or hacked to get them to work the way\nyou want, and some abstractions are leaky and leave you doing some tedious work \nat a lower level than you'd want. Dependency management is also complicated: using external cookbooks means introducing at least one more fragile package \ninstaller into your life.\n\nIn the case of Chef in particular, it is also a bit strange that although its\ninterface is mostly ordinary Ruby code, it has developed in a somewhat parallel\nuniverse where the user is assumed to know a lot about system administration,\nand very little about Ruby. This leads to some design choices that aren't\nnecessarily bad, but are at least surprising to an experienced Ruby developer.\n\nAnd as for infrastructure automation as a whole, well... it doesn't fully free\nyou from knowing quite a few details about the systems you are trying to manage.\nIt does allow you to express ideas at a higher level, but you still need to\nbe able to peel back the veneer and dive into some low level system\nadministration concepts whenever something doesn't work the way you expect it\nwould or doesn't support the feature you want to use via its high level\ninterface. In that sense, an automated system will not necessarily reduce\nlearning costs, it just has you doing a different kind of learning.\n\nDespite all these concerns, I have to say that this is one skillset that I wish\nI had picked up years ago, and I fully intend to look for opportunities\nto apply these ideas in my own projects. I hope after reading this article,\nyou will try to do the same, and then share your stories about your experiences.\n\n## Recommendations for further reading\n\nDespite having a very complex ecosystem, the infrastructure automation world\n(and especially the Chef community) have a ton of useful documentation that is\nfreely available and easy to get started with. Here are a few resources to try\nout if you want to continue exploring this topic on your own:\n\n* [Opscode Chef documentation](http://docs.opscode.com): The official Chef documentation; comprehensive and really well organized. \n\n* [Opscode public cookbooks](https://github.com/opscode-cookbooks): You can learn a lot by reading some of the most widely-used cookbooks in the Chef community. For complex examples, definitely check out the [apache2](https://github.com/opscode-cookbooks/apache2) and [mysql](https://github.com/opscode-cookbooks/mysql) cookbooks.\n\n* [#learnchef](https://learnchef.opscode.com/): A collection of tutorials and screencasts designed to help you learn Chef.\n\n* [Common Idioms in Chef Recipes](http://www.opscode.com/blog/2013/09/04/demystifying-common-idioms-in-chef-recipes/): Explanation of (possibly surprising) idioms that sometimes appear in recipe code.\n\n* [Learning Chef](http://mlafeldt.github.io/blog/2012/09/learning-chef): A friendly introduction to Chef written by Mathias.\n\nIf you've got some experience with infrastructure automation and have found\nother tutorials or articles that you like which aren't listed here, please leave\na comment. Mathias will also be watching the comments for this article, so\ndon't be afraid to ask any general questions you have about infrastructure\nautomation or Chef, too.\n\nThanks for making it all the way to the end of this article, and happy automating!\n\n[puppet]: http://projects.puppetlabs.com/projects/puppet\n[chef]: http://www.opscode.com/chef/\n[pr-cookbook]: https://github.com/elm-city-craftworks/practicing-ruby-cookbook/tree/1.0.8\n[pr-cookbook-attributes]: https://github.com/elm-city-craftworks/practicing-ruby-cookbook/blob/1.0.8/attributes/default.rb\n[pr-web]: https://github.com/elm-city-craftworks/practicing-ruby-web\n[chef-attributes]: http://docs.opscode.com/essentials_cookbook_attribute_files.html\n[God]: http://godrb.com/\n[god-init]: https://raw.github.com/elm-city-craftworks/practicing-ruby-cookbook/37ca12dc6432dfee955a70b6f2cc288e40782733/files/default/god.sh\n[pr-web-dj]: https://github.com/elm-city-craftworks/practicing-ruby-web/blob/master/config/delayed_job.god\n\n" }, { "path": "articles/v7/007-wumpus.md", "content": " [Hunt the Wumpus][wikipedia] is a hide-and-seek game that takes place in an underground\ncave network full of interconnected rooms. To win the game, the player\nneeds to locate the evil Wumpus and kill it while avoiding various different \nhazards that are hidden within in the cave.\n\nOriginally written by Gregory Yob in the 1970s, this game is traditionally\nplayed using a text-based interface, which leaves plenty up to the\nplayer's imagination, and also makes programming easier for those who\nwant to build Wumpus-like games of their own.\n\nBecause of its simple but clever nature, Hunt the Wumpus has been ported \nto many different platforms and programming languages over the last several\ndecades. In this article, you will discover why this blast from the past \nserves as an excellent example of creative computing, and you'll also \nlearn how to implement it from scratch in Ruby.\n\n## Gameplay demonstration\n\nThere are only two actions available to the player throughout the game: to move\nfrom room to room, or to shoot arrows into nearby rooms in an attempt to kill \nthe Wumpus. Until the player knows for sure where the Wumpus is, most of their actions \nwill be dedicated to moving around the cave to gain a sense of its layout:\n\n You are in room 1.\n Exits go to: 2, 8, 5\n -----------------------------------------\n What do you want to do? (m)ove or (s)hoot? m\n Where? 2\n -----------------------------------------\n You are in room 2.\n Exits go to: 1, 10, 3\n -----------------------------------------\n What do you want to do? (m)ove or (s)hoot? m\n Where? 10\n -----------------------------------------\n You are in room 10.\n Exits go to: 2, 11, 9\n\nEven after only a couple actions, the player can start to piece together\na map of the cave's topography, which will help them avoid getting lost\nas they continue their explorations:\n\n\n\nPlay continues in this fashion, with the player wandering around until \na hazard is detected:\n\n What do you want to do? (m)ove or (s)hoot? m\n Where? 11\n -----------------------------------------\n You are in room 11.\n Exits go to: 10, 8, 20\n -----------------------------------------\n What do you want to do? (m)ove or (s)hoot? m\n Where? 20\n -----------------------------------------\n You are in room 20.\n You feel a cold wind blowing from a nearby cavern.\n Exits go to: 11, 19, 17\n\nIn this case, the player has managed to get close\nto a bottomless pit, which is detected by the presence of\na cold wind emanating from an adjacent room.\n\nBecause hazards are sensed indirectly, the player needs to use a deduction\nprocess to know for sure which hazards are in what rooms. With the knowledge of\nthe cave layout so far, the only thing that is for certain is there is at least one\npit nearby, with both rooms 17 and 19 being possible candidates. One of them\nmight be safe, but there is also a chance that BOTH rooms contain pits.\nIn a literal sense, the player might have reached a dead end:\n\n\n\nA risky player might chance it and try one of the two rooms, but\nthat isn't a smart way to play. The safe option is to \nbacktrack in search of a different path through the cave:\n\n What do you want to do? (m)ove or (s)hoot? m\n Where? 11\n -----------------------------------------\n You are in room 11.\n Exits go to: 10, 8, 20\n -----------------------------------------\n What do you want to do? (m)ove or (s)hoot? m\n Where? 8\n -----------------------------------------\n You are in room 8.\n You smell something terrible nearby\n Exits go to: 11, 1, 7\n\nChanging directions ends up paying off. Upon entering room 8,\nthe terrible smell that is sensed indicates that the Wumpus is nearby,\nand because rooms 1 and 11 have already been visited, there\nis only one place left for the Wumpus to be hiding:\n\n What do you want to do? (m)ove or (s)hoot? s\n Where? 7\n -----------------------------------------\n YOU KILLED THE WUMPUS! GOOD JOB, BUDDY!!!\n\nAt the end of the hunt, the player's map ended up looking like this:\n\n\n\nIn less fortunate circumstances, the player would need to do a lot more\nexploration before they could be certain about where the Wumpus \nwas hiding. Other hazards might also be encountered, including giant bats \nthat are capable of moving the player to a random location in the cave.\nBecause all these factors are randomized in each new game, Hunt the Wumpus\ncan be played again and again without ever encountering an identical\ncave layout.\n\nWe will discuss more about the game rules throughout the rest of this\narticle, but the few concepts illustrated in this demonstration are more \nthan enough for us to start modeling some of the key game objects.\nLet's get to work!\n\n## Implementing \"Hunt the Wumpus\" from scratch\n\nLike many programs from its era, Hunt the Wumpus was designed to \nbe hackable. If you look at one of the [original publications][atari]\nabout the game, you can see that the author actively encourages\ntweaking its rules, and even includes the full source code \nof the game.\n\nBefore you rush off to study the original implementation, remember that \nit was written four decades ago in BASIC. Unless you consider yourself\na technological archaeologist, it's probably not the best way to\nlearn about the game. With that in mind, I've put together a learning\nexercise that will guide you through implementing some of the core \ngame concepts of Hunt the Wumpus -- without getting bogged down in\nspecific game rules or having to write boring user interface code.\n\nIn particular, I want you to implement three classes that I have \nalready written the tests for:\n\n1. A `Wumpus::Room` class to manage hazards and connections between rooms\n2. A `Wumpus::Cave` class to manage the overall topography of the cave\n3. A `Wumpus::Player` class that handles sensing and encountering hazards\n\nYou can work through this exercise by [cloning its git repository][wumpus-diy], \nand following the instructions in the README. I have put the tests for each \nclass on its own branch, so that you can merge them into your own code \none at a time until you end up with a complete passing test suite.\n\nOnce these three classes are written, you'll be able to use my UI code \nand game logic to play a rousing round of Hunt the Wumpus. You'll\nalso be able to compare your own work to my [reference implementation][wumpus-ref]\nof the game, and discuss any questions or thoughts with me about\nthe differences between our approaches.\n\nThroughout the rest of this article, I will provide design and implementation\nnotes for each class, as well as a brief overview of how the game rules for\nHunt the Wumpus can be implemented using these objects. These notes\nshould help you interpret what the test suite is actually asking\nyou to build, and will also help you understand my reference\nimplementation.\n\n> **NOTE:** If you're short on time or aren't in the mood for hacking\nright now, you can still get a lot out of this exercise by simply \nthinking about how you'd write the code to pass the provided test \nsuite, and then looking my implementation. But it's definitely\nbetter to at least *try* to write some code yourself, even\nif you don't complete the full exercise.\n\n## Modeling rooms \n\nStructurally speaking, rooms and their connections form a simple undirected graph:\n\n\n\nOur `Room` class will manage these connections, and also make it easy \nto query and manipulate the hazards that can be found in a room --\nincluding bats, pits, and the wumpus itself. In particular, we will\nbuild an object with the following attributes and behaviors:\n\n```ruby\ndescribe \"A room\" do\n it \"has a number\"\n it \"may contain hazards\"\n\n describe \"with neighbors\" do\n it \"has two-way connections to neighbors\"\n it \"knows the numbers of all neighboring rooms\"\n it \"can choose a neighbor randomly\"\n it \"is not safe if it has hazards\" \n it \"is not safe if its neighbors have hazards\"\n it \"is safe when it and its neighbors have no hazards\"\n end\nend\n```\n\nLet's walk through each of these requirements individually and fill\nin the necessary details.\n\n1) Every room has an identifying number that helps the player keep \ntrack of where they are:\n\n```ruby\ndescribe \"A room\" do\n let(:room) { Wumpus::Room.new(12) }\n\n it \"has a number\" do\n room.number.must_equal(12)\n end\n\n # ...\nend\n```\n\n2) Rooms may contain hazards, which can be added or removed as the \ngame progresses:\n\n```ruby\nit \"may contain hazards\" do \n # rooms start out empty\n assert room.empty?\n\n # hazards can be added\n room.add(:wumpus)\n room.add(:bats)\n\n # a room with hazards isn't empty\n refute room.empty?\n\n # hazards can be detected by name\n assert room.has?(:wumpus)\n assert room.has?(:bats)\n\n refute room.has?(:alf)\n\n # hazards can be removed\n room.remove(:bats)\n refute room.has?(:bats)\nend\n```\n\n3) Each room can be connected to other rooms in the cave:\n\n```ruby\ndescribe \"with neighbors\" do\n let(:exit_numbers) { [11, 3, 7] }\n\n before do\n exit_numbers.each { |i| room.connect(Wumpus::Room.new(i)) }\n end\n\n # ...\nend\n```\n\n4) One-way paths are not allowed, i.e. all connections between rooms are\nbidirectional:\n\n```ruby\nit \"has two-way connections to neighbors\" do\n exit_numbers.each do |i| \n # a neighbor can be looked up by room number\n room.neighbor(i).number.must_equal(i)\n\n # Room connections are bidirectional\n room.neighbor(i).neighbor(room.number).must_equal(room)\n end\nend\n```\n\n5) Each room knows all of its exits, which consist of\nall neighboring room numbers:\n\n```ruby\nit \"knows the numbers of all neighboring rooms\" do\n room.exits.must_equal(exit_numbers)\nend\n```\n\n6) Neighboring rooms can be selected at random, which is\nuseful for certain game events:\n\n```ruby\nit \"can choose a neighbor randomly\" do\n exit_numbers.must_include(room.random_neighbor.number)\nend\n```\n\n7) A room is considered safe only if there are no hazards within it\nor any of its neighbors:\n\n```ruby\nit \"is not safe if it has hazards\" do\n room.add(:wumpus)\n\n refute room.safe?\nend\n\nit \"is not safe if its neighbors have hazards\" do\n room.random_neighbor.add(:wumpus)\n\n refute room.safe?\nend\n\nit \"is safe when it and its neighbors have no hazards\" do\n assert room.safe?\nend\n```\n\n**Implementation notes**\n\nBecause this object only handles basic data tranformations, it\nshouldn't be hard to implement. But if you get stuck, you\ncan always look at [my version of the Wumpus::Room class][wumpus-room].\n\n## Modeling the cave\n\nAlthough a game of Hunt the Wumpus can be played with an arbitrary cave layout,\nthe traditional Wumpus cave is based on the [dodecahedron][]. To\nmodel things this way, a room is placed at each vertex, and the edges form\nthe connections between rooms. If you squash the structure to fit in a\ntwo-dimensional space, you end up with the following graph:\n\n\n\nEven though it would be technically possible to construct this structure without\na collection object by connecting rooms together in an ad-hoc fashion,\ntraversing the structure and manipulating it would be cumbersome. For that\nreason, we will build a `Wumpus::Cave` object with the following properties:\n\n```ruby\ndescribe \"A cave\" do\n it \"has 20 rooms that each connect to exactly three other rooms\" \n it \"can select rooms at random\"\n it \"can move hazards from one room to another\"\n it \"can add hazards at random to a specific number of rooms\"\n it \"can find a room with a particular hazard\"\n it \"can find a safe room to serve as an entrance\"\nend\n```\n\nSome of these features a bit tricky to explain comprehensively through\ntests, but the following examples should give you a basic idea of\nhow they're meant to work.\n\n1) The cave has 20 rooms, and each room is connected to exactly \nthree other rooms:\n\n```ruby\ndescribe \"A cave\" do\n let(:cave) { Wumpus::Cave.dodecahedron }\n let(:rooms) { (1..20).map { |i| cave.room(i) } }\n\n it \"has 20 rooms that each connect to exactly three other rooms\" do\n rooms.each do |room|\n room.neighbors.count.must_equal(3)\n \n assert room.neighbors.all? { |e| e.neighbors.include?(room) }\n end\n end\nend\n```\n\nThe intent here is to loosly verify that the layout is dodecahedron \nshaped, but it is more of a sanity check than a strict validation.\nA stronger check would require us to compute things like minimal\ncycles for each point, which would make for a much more \ncomplicated test.\n\nIn my implementation I use a JSON file that hard-codes the \nconnections between each room explicitly rather than trying to \nautomatically generate the layout, so this test is mostly just to catch errors \nwith that configuration file. If you reuse the [dodecahredon.json][json] \nfile in your own code, it should make passing these tests easy.\n\n2) Rooms in the cave can be selected randomly:\n\n```ruby\nit \"can select rooms at random\" do\n sampling = Set.new\n\n # see test/helper.rb for how this assertion works\n must_eventually(\"randomly select each room\") do\n new_room = cave.random_room \n sampling << new_room\n\n sampling == Set[*rooms] \n end\nend\n```\n\nThis feature is important for implementing the behavior of giant bats, who move\nthe player to a random location in the cave. It is also useful for hazard\nplacement, as we'll see later. The way I test the behavior is a bit awkward,\nbut the basic idea is that if you keep selecting rooms at random, you'll\neventually hit every room in the cave.\n\n3) Hazards can be moved from one room to another:\n\n```ruby\nit \"can move hazards from one room to another\" do\n room = cave.random_room\n neighbor = room.neighbors.first\n\n room.add(:bats)\n\n assert room.has?(:bats)\n refute neighbor.has?(:bats)\n\n cave.move(:bats, :from => room, :to => neighbor)\n\n refute room.has?(:bats)\n assert neighbor.has?(:bats)\nend\n```\n\nThis test shows bats being moved from a random room to\none of its neighbors, but `Cave#move` can used to move any hazard\nbetween any two rooms in the cave, even if they are not\nadajecent to each other.\n\n4) Hazards can be randomly distributed throughout the cave:\n\n```ruby\nit \"can add hazards at random to a specific number of rooms\" do\n cave.add_hazard(:bats, 3)\n\n rooms.select { |e| e.has?(:bats) }.count.must_equal(3)\nend\n```\n\nFor the most part, the work to be done here is just to pick\nsome rooms at random and add hazards\nto them. However, because there is no sense in adding a single\ntype of hazard to a room more than once, `Cave#add_hazard`\nshould take care to select only rooms that do not already have\nthe specified hazard in them. This is hinted at by the specs,\nbut because the check is a loose one, just keep this detail\nin mind while implementing this method.\n\n5) Rooms can be looked up based on the hazards they contain:\n\n```ruby\nit \"can find a room with a particular hazard\" do\n cave.add_hazard(:wumpus, 1)\n\n assert cave.room_with(:wumpus).has?(:wumpus)\nend\n```\n\nIn my implementation, I just grab the first room that matches the \ncriteria, but any matching room would be acceptable. It\nwould also make sense to have a `Cave#all_rooms_with` method, but it isn't needed for a basic implementation\nof the game.\n\n6) A safe entrance can be located:\n\n```ruby\nit \"can find a safe room to serve as an entrance\" do\n cave.add_hazard(:wumpus, 1)\n cave.add_hazard(:pit, 3)\n cave.add_hazard(:bats, 3)\n\n entrance = cave.entrance\n\n assert entrance.safe?\nend\n```\n\nThis is where the `Wumpus::Room#safe?` method comes in handy. Picking any room\nthat passes that condition is enough to get the job done here.\n\n**Implementation notes**\n\nThe desired behavior of the `Wumpus::Cave` class is admittedly a bit\nunderspecified here, but in many cases minor variations won't effect\ngameplay all that much. Some of these operations are also intentionally \na bit more general than what is strictly needed for the game, to permit \nsome experimentation with rule changes once you have a working implementation. \n\nThis was a challenging object for me to design and test, because many \nof the features which are intuitively obvious are hard to specify \nformally. Do the best you can with building it, and refer\nto [my implementation of the Wumpus::Cave class][wumpus-cave] whenever \nyou hit any snags.\n\n## Modeling the player\n\nDespite the complexity of the cave layout, most game events in \nHunt the Wumpus are triggered by local conditions based on the \nplayer's current room and its direct neighbors. For example, \nimagine that the player is positioned in Room #1 as shown in \nfollowing diagram:\n\n\n\nWith this setup, the player would sense the nearby hazards,\nresulting in the following output:\n\n You are in room 1.\n You hear a rustling sound nearby\n You smell something terrible nearby\n Exits go to: 2, 3, 4\n\nOrdinarily we'd need to do some investigation work to discover which hazards\nwere where, but because this is a contrived scenario, we don't \nneed to guess. Knowing the layout of the neighborhood, we can enumerate the \npossible outcomes for any player action:\n\n* The player will encounter the wumpus upon moving into room 2.\n* The player will encounter bats upon moving into room 3.\n* The player will not encounter any hazards in room 4.\n* The player can shoot into room 2 to kill the wumpus.\n* The player will miss the wumpus by shooting into room 3 or 4.\n\nIf you take this single example and generalize it, you'll find that every turn\nof Hunt the Wumpus involves only three distinct kinds of events:\n\n```ruby\ndescribe \"the player\" do\n it \"can sense hazards in neighboring rooms\" \n it \"can encounter hazards when entering a room\"\n it \"can perform actions on neighboring rooms\" \nend\n```\n\nWith these requirements in mind, it is possible for us to model \nthe `Wumpus::Player` class as an event-driven object that handles \neach event type listed above. The only state it needs to explicitly\nmaintain is a reference to the room currently being explored: everything \nelse can be managed externally through callbacks. You'll see why this\nis useful when we look at how the game rules are implemented later,\nbut for now just try to follow along as best as you can.\n\nThe test setup for the `Wumpus::Player` class is a bit complicated, mostly \nbecause we need to reconstruct something similar to the layout shown in the\nprevious diagram in order to meaningfully test its behavior:\n\n```ruby\ndescribe \"the player\" do\n let(:player) { Wumpus::Player.new }\n\n let(:empty_room) { Wumpus::Room.new(1) }\n\n let(:wumpus_room) do\n Wumpus::Room.new(2).tap { |e| e.add(:wumpus) }\n end\n\n let(:bat_room) do\n Wumpus::Room.new(3).tap { |e| e.add(:bats) }\n end\n\n # ...\nend\n```\n\n\nIn addition to wiring up some rooms, I also register all of the events we're \ninterested in tracking during setup, using some dummy callbacks that are\nmeant to serve as stand-ins for real game logic. This is not an\nelegant way of building a test harness, but it gets the job done:\n\n```ruby\nlet(:sensed) { Set.new }\nlet(:encountered) { Set.new }\n\nbefore do\n empty_room.connect(bat_room)\n empty_room.connect(wumpus_room)\n\n player.sense(:bats) do\n sensed << \"You hear a rustling\"\n end\n\n player.sense(:wumpus) do\n sensed << \"You smell something terrible\"\n end\n\n player.encounter(:wumpus) do\n encountered << \"The wumpus ate you up!\"\n end\n\n player.encounter(:bats) do\n encountered << \"The bats whisk you away!\"\n end\n\n player.action(:move) do |destination|\n player.enter(destination)\n end\nend\n```\n\nOnce all of that is taken care of, the callbacks can be tested in isolated\nscenarios:\n\n```ruby\nit \"can sense hazards in neighboring rooms\" do\n player.enter(empty_room)\n player.explore_room\n\n sensed.must_equal(Set[\"You hear a rustling\", \"You smell something terrible\"])\n \n assert encountered.empty?\nend\n\nit \"can encounter hazards when entering a room\" do\n player.enter(bat_room)\n encountered.must_equal(Set[\"The bats whisk you away!\"])\n \n assert sensed.empty? \nend\n\nit \"can perform actions on neighboring rooms\" do\n player.act(:move, wumpus_room)\n player.room.must_equal(wumpus_room)\n\n encountered.must_equal(Set[\"The wumpus ate you up!\"])\n assert sensed.empty?\nend\n```\n\nThese test cases verify that the right callbacks have been called\nby manipulating simple sets of strings, but the real use case for \nthe `Wumpus::Player` class is to trigger operations on \ngame objects as well as the user interface. If you are having\ntrouble imagining what that would look like, it may help to \nread ahead a bit further before attempting to get these \ntests to pass.\n\n**Implementation notes:**\n\nLike the `Wumpus::Cave` class, this object is underspecified, but you probably\ndon't need to build something identical to [my implementation of Wumpus::Player][wumpus-player]\nin order to get the game to run. However, you may want to make an effort\nto ensure that callbacks are triggered in the order that they are registered,\notherwise you can run into some interesting edge cases when more than one\ncondition is satisfied at the same time.\n\n## Defining the game rules\n\nWith a foundation in place, implementing the game logic for Hunt the\nWumpus is very easy. My version of the game simplifies the rules, but\nhopefully still captures the spirit of the original. \n\nAs you walk through the following code, you can treat the\n`Wumpus::Narrator` object as a black box. This is a boring object that\nonly does some basic I/O under the hood, so your time\nwould be better spent focusing on the game logic.\n\nWith that caveat out of the way, let's take a look at how Hunt the Wumpus can be\nimplemented in terms of the three game objects we just built. To get started, we\nneed a cave!\n\n```ruby\ncave = Wumpus::Cave.dodecahedron\n```\n\nThis cave will contain three pits, three giant bats, and the most evil and\nstinky Wumpus you could ever imagine:\n\n```ruby\ncave.add_hazard(:wumpus, 1)\ncave.add_hazard(:pit, 3)\ncave.add_hazard(:bats, 3)\n```\n\nWe also need a player to navigate the cave, and a narrator to regale us with\ntales about the player's adventures:\n\n```ruby\nplayer = Wumpus::Player.new\nnarrator = Wumpus::Narrator.new\n```\n\nWhenever a player senses a hazard nearby, the narrator will give us a hint\nof what kind of trouble lurks just around the bend:\n\n```ruby\nplayer.sense(:bats) do\n narrator.say(\"You hear a rustling sound nearby\") \nend\n\nplayer.sense(:wumpus) do\n narrator.say(\"You smell something terrible nearby\")\nend\n\nplayer.sense(:pit) do\n narrator.say(\"You feel a cold wind blowing from a nearby cavern.\")\nend\n```\n\nIf upon entering a room the player encounters the Wumpus, it\nwill become startled. We'll discuss the detailed consequences\nof this later, but the basic idea is that it will cause the\nWumpus to either run away to an adjacent room, or to gobble\nthe player up:\n\n```ruby\nplayer.encounter(:wumpus) do\n player.act(:startle_wumpus, player.room)\nend\n```\n\nWhen bats are encountered, the narrator will inform us of\nthe event, then a random room will be selected to drop\nthe player off in. If any hazards are encountered\nin that room, the effects will be applied immediately,\npossibly leading to the player's demise.\n\nBut assuming that the player managed to survive the flight, \nthe bats will take up residence in the new location. This\ncan make navigation very complicated, because stumbling\nback into that room will cause the player to be moved\nto yet another random location:\n\n```ruby\nplayer.encounter(:bats) do\n narrator.say \"Giant bats whisk you away to a new cavern!\"\n\n old_room = player.room\n new_room = cave.random_room\n\n player.enter(new_room)\n\n cave.move(:bats, from: old_room, to: new_room)\nend\n```\n\nIf the player happens to come across a bottomless pit, the\nstory ends immediately, even though the player's journey\nwill probably go on forever:\n\n```ruby\nplayer.encounter(:pit) do\n narrator.finish_story(\"You fell into a bottomless pit. Enjoy the ride!\")\nend\n```\n\nThe player's actions are what ultimately ends up triggering game events. \nThe movement action is straightforward: it simply updates the player's\ncurrent location and then fires callbacks for any hazards encountered:\n\n```ruby\nplayer.action(:move) do |destination|\n player.enter(destination)\nend\n```\n\nShooting is more complicated, although the way it is implemented here\nis still a simplification of how the original game worked. In Gregory Yob's\nversion, you had only five arrows, but they could travel a distance of up to\nfive rooms, even shooting around corners if you knew the right path. In my\nversion, arrows are unlimited but can only fire into neighboring rooms.\n\nIf the player shoots into the room that the Wumpus is hiding in, the beast \nis slayed and the story ends happily ever after. If instead the player shoots\ninto the wrong room, then no matter where the Wumpus is in the cave, it will \nbe startled by the sound.\n\n```ruby\nplayer.action(:shoot) do |destination|\n if destination.has?(:wumpus)\n narrator.finish_story(\"YOU KILLED THE WUMPUS! GOOD JOB, BUDDY!!!\") \n else\n narrator.say(\"Your arrow missed!\")\n\n player.act(:startle_wumpus, cave.room_with(:wumpus))\n end\nend\n```\n\nWhen the Wumpus is startled, it will either stay where it is or move into\none of its neighboring rooms. The player will be able to hear the Wumpus\nmove anywhere in the cave, even if it is not in a nearby room.\n\nIf the Wumpus is in the same room as the player at the end of this process,\nit will gobble the player up and the game will end in sadness and tears:\n\n```ruby\nplayer.action(:startle_wumpus) do |old_wumpus_room|\n if [:move, :stay].sample == :move\n new_wumpus_room = old_wumpus_room.random_neighbor\n cave.move(:wumpus, from: old_wumpus_room, to: new_wumpus_room)\n\n narrator.say(\"You heard a rumbling in a nearby cavern.\")\n end\n\n if player.room.has?(:wumpus)\n narrator.finish_story(\"You woke up the wumpus and he ate you!\")\n end\nend\n```\n\nAnd that pretty much sums it up. I omitted a few lines of boilerplate\ncode that fire up the main event loop, but this pretty much covers\nall of the code that implements the actual game rules. It is designed\nto be very hackable, so please do experiment with it however you'd like.\n\nIf you want to review the full game executable without the intermingled\ncommentary, please see [the bin/wumpus script][wumpus-script].\n\n## Additional Exercises\n\nHopefully by working through this article you've seen for yourself why Hunt the\nWumpus is both fun to play and fun to implement. If you are looking for more\nthings to try, I'd suggest the following activities:\n\n* Limit the number of arrows that the player can shoot, and end the game when\nthe player runs out of arrows.\n\n* Try implementing the \"crooked arrow\" behavior of the original Wumpus game. To\ndo this allow the player to specify a path of up to five rooms. Whenever the\nplayer guesses an incorrect path, have the arrow to bounce into a random room.\nIf the arrow ends up hitting the player because of this, they lose!\n\n* Make it harder to guess the connections between rooms by randomizing\nthe room numbers for each new game while keeping the overall shape the same.\n\n* Try out one of the alternative cave layouts described in Gregory Yob's\nfollowup publication about [Wumpus 2][atari-2].\n\n* Add new hazards of your own, or other types of game objects that\nare beneficial, or provide some more depth to the story.\n\n* Implement a solver bot that plays the game automatically.\n\n* Build a better user interface for the game, either improving the text-based\nUI or attempting something using a GUI or web-based interface. You should\nonly need to edit the `Wumpus::Narrator` and `Wumpus::Console` objects\nin order to replace the current interface.\n\n* Keep the game behavior the same, but try out a different design than the one\nI provided here and/or improve the test suite.\n\nIf you try out any of these extra credit exercises, please share your work. I'd\nbe very interested to see what you come up with. Until then, happy hacking!\n\n[atari]: http://www.atariarchives.org/bcc1/showpage.php?page=247\n[atari-2]: http://www.atariarchives.org/bcc2/showpage.php?page=244\n[wumpus-ref]: https://github.com/elm-city-craftworks/wumpus/tree/reference_implementation\n[wumpus-diy]: https://github.com/elm-city-craftworks/wumpus\n[wumpus-room]: https://github.com/elm-city-craftworks/wumpus/blob/reference_implementation/lib/wumpus/room.rb\n[wumpus-cave]: https://github.com/elm-city-craftworks/wumpus/blob/reference_implementation/lib/wumpus/cave.rb\n[wumpus-player]: https://github.com/elm-city-craftworks/wumpus/blob/reference_implementation/lib/wumpus/player.rb\n[wumpus-script]: https://github.com/elm-city-craftworks/wumpus/blob/reference_implementation/bin/wumpus\n[wikipedia]: http://en.wikipedia.org/wiki/Hunt_the_Wumpus\n[dodecahedron]: http://en.wikipedia.org/wiki/Dodecahedron\n[json]: https://raw.github.com/elm-city-craftworks/wumpus/reference_implementation/data/dodecahedron.json\n" }, { "path": "articles/v7/008-language-learning.md", "content": "[ ] Day 1 review\n[ ] Day 2 review\n[ ] Day 3 review\n[ ] Day 4 review\n[ ] Day 5 review\n[ ] Explain four step system and projects\n[ ] Add explanations of code\n[ ] Connect togther narrative\n\n\n----------\n\nhttp://alistapart.com/article/writing-is-thinking\n\nFirst of all, why did this excerpt from your experience stand out to you, personally? \nWas this the moment something clicked for you regarding your work?\n\nSecondly, why do you think things turned out the way they did? Were you surprised? \nDo you do things differently now as a result? When you spell this out, \nit’s the difference between journaling for yourself and writing for an audience.\n\nFinally, is this something others in your line of work are prone to miss? \nIs it a rookie error, or something more like an industry-wide oversight? \nIf you’ve tried to search online for similar opinions, do you get a lot \nof misinformation? Or is the good information simply not in a place\nwhere others in your field are likely to see it?\n\n----------\n\n**Summarize my preliminaries and the four step structure, then make\ncase study presentation chronological but free-form narrative \n(i.e. don't put in clear headers for exercise, book, project, etc, make it \nmore like a journal -- give the reader a 'riding shotgun' view of the\naction, with a brief summary at the end of each day. Include evolution\nin thought process and new realizations as I go. Wrap up with a where\nto next section.**\n\n** Look up dates for everything to linearize it, but don't be afraid to do mild\nediting / resequencing for clarity**\n\n## What is trivial code literacy?\n\n**Starting with fizzbuzz may work, but consider condensing or replacing with a\nshorter anecdote (the chinese one?)**\n\n(rewrite to be a bit more positive, and to illustrate\nRuby vs. Erlang, and Reading vs. Writing)\n\nProgrammers often joke about how ridiculous it is to use the FizzBuzz problem as\na screening test, because it is so easy to solve. Derived from a word\ngame that's used to test the division skills of small children, FizzBuzz is\nabout as conceptually trivial as computing challenges get:\n\n> Write a program that prints the numbers from 1 to 100. But for multiples \nof three print “Fizz” instead of the number and for the multiples of \nfive print “Buzz”. For numbers which are multiples of both three and \nfive print “FizzBuzz”.\n\nAny working programmer or hobbyist that's built absolutely any software could\nsolve this problem in their sleep, as long as they were allowed to use a language \nthey were already comfortable with. But suppose the interviewer asked a Ruby\nprogrammer who had never worked in a functional programming language before to \nproduce an Erlang solution instead. What would that programmer need to learn in\norder to pass the test? Let's figure that out by working backwards from the \nfollowing solution:\n\n```erlang\n-module(fizzbuzz).\n-export([run/0]).\n\nrun() -> \n io:format(\"~p~n\", [lists:map(fun transform/1, lists:seq(1,100))]).\n\ntransform(X) when X rem 15 =:= 0 -> \"FizzBuzz\";\ntransform(X) when X rem 5 =:= 0 -> \"Fizz\";\ntransform(X) when X rem 3 =:= 0 -> \"Buzz\";\ntransform(X) -> X.\n```\n\nLet's assume that simply producing the source code shown above wouldn't be good\nenough to pass the test, the programmer would actually need to execute it and\nshow that it produces the correct output, too. Right out of the gate, that means\nunderstanding how to install Erlang, compile Erlang modules, and then call\nfunctions on them. If that was all done successfully, the programmer might\nproduce something like the following output:\n\n```erlang\n$ erl\n1> c(fizzbuzz).\n{ok,fizzbuzz}\n2> fizzbuzz:run().\n[1,2,\"Buzz\",4,\"Fizz\",\"Buzz\",7,8,\"Buzz\",\"Fizz\",11,\"Buzz\",13,14,\"Fizzbuzz\",16,\n 17,\"Buzz\",19,\"Fizz\",\"Buzz\",22,23,\"Buzz\",\"Fizz\",26,\"Buzz\",28,29,\"Fizzbuzz\",31,\n 32,\"Buzz\",34,\"Fizz\",\"Buzz\",37,38,\"Buzz\",\"Fizz\",41,\"Buzz\",43,44,\"Fizzbuzz\",46,\n 47,\"Buzz\",49,\"Fizz\",\"Buzz\",52,53,\"Buzz\",\"Fizz\",56,\"Buzz\",58,59,\"Fizzbuzz\",61,\n 62,\"Buzz\",64,\"Fizz\",\"Buzz\",67,68,\"Buzz\",\"Fizz\",71,\"Buzz\",73,74,\"Fizzbuzz\",76,\n 77,\"Buzz\",79,\"Fizz\",\"Buzz\",82,83,\"Buzz\",\"Fizz\",86,\"Buzz\",88,89,\"Fizzbuzz\",91,\n 92,\"Buzz\",94,\"Fizz\",\"Buzz\",97,98,\"Buzz\",\"Fizz\"]\n```\n\nBut imagine the interviewer was not convinced by this alone, and wanted the\nprogrammer to walk through the code statement-by-statement and explain it.\n\n```erlang\n-module(fizzbuzz).\n```\n\nThis code defines the module name, but it also implies what the filename should\nbe (`fizzbuzz.erl`). If the programmer didn't know the two must match,\nautoloading would not work correctly when the Erlang `c()` shell command \nwas used.\n\n```erlang\n-export([run/0]).\n```\n\nThis code is necessary to make it possible to call the `fizzbuzz:run()` function\nexternally, because all Erlang functions are private by default. The programmer\nwould also need to explain that `run/0` means \n\"the run function with zero arguments\", demonstrating an understanding of the\nconcept of *function arity*.\n\n```erlang\nrun() -> \n io:format(\"~p~n\", [lists:map(fun transform/1, lists:seq(1,100))]).\n```\n\nThis line of code has a ton of features crammed into it, including calls to both\nthe `io` and `lists` standard libraries, along with the syntax for passing\nan existing function as an argument to another function \n(e.g. `fun transform/1`).\n\n```erlang\ntransform(X) when X rem 15 =:= 0 -> \"FizzBuzz\";\ntransform(X) when X rem 5 =:= 0 -> \"Fizz\";\ntransform(X) when X rem 3 =:= 0 -> \"Buzz\";\ntransform(X) -> X.\n```\n\nFinally, we see function overloading and guards, both concepts that don't exist\nin Ruby, but are commonly used in Erlang.\n\nWhen you add up all of these points, it takes a whole lot of knowledge for even\nan experienced programmer to write such a trivial program in a language they're\nunfamiliar with. When you throw in things like familiarizing yourself with new\nsyntax and grammar rules, it becomes easy to see that trivial code literacy\ndemands a whole lot more understanding of a language than it appears to\nat a first glance.\n\n\n```\nStart every day coding, end every day thinking.\n\n1. Warmup exercise (30 mins)\n\nMake sure to have these ready the night before, pick stuff\nthat you can work on right away without having to study \nin advance. \n\nThey can either be book exercises or stuff\nfrom other sources, but they should be self-verifiable\nfor correctness. Goal is not to finish but just to \nlearn as much as possible.\n\n2. Book reading and exercises (90 mins)\n\nJumping around chapters is OK, but reading whole chapters\nat a time is encouraged. Read what is most related to\nthe projects you're working on. \n\nGive reading a higher priority over exercises during this \ntime, and do only exercises related to the current reading.\n\n3. Work on projects (90 mins)\n\nStart with bowling score calculator, then dining philosophers,\nthen IRC rover bot if time permits. Focus on getting working\ncode first, before worrying about correct code. But once you\nhave a working solution, figure out how to make it right,\nand get help if necessary for style questions.\n\n4. Review today's work and do next day's prep work (30 mins)\n\nPrepare questions, TODO lists, and exercises for the next\nday. Reflect on what was learned today, possibly looking\nup tangential points that you didn't have time for in the\nday, or seek solutions to exercises already published online,\nor write notes asking for help. \n\nTry to do all four hours in a single day if possible,\notherwise try to do 1+2 and 3+4 in two sessions as close\ntogether as possible.\n```\n\nProjects summary (what and why)\n-----------------\n\nBook learning\n-------------\n\nTyped in, ran, and tinkered with nearly every code snipped from \nCH 1 to CH 14. (First several chapters before the \"practice week\"\nbrought me up to fizz-buzz level knowledge, skipped a couple \nchapters, but wrote dozens of functions)\n\nWent off on several tangents (find a couple examples)\n\nFind some book learning highlights from each day, and note tie-ins with\nexercises / projects.\n\n(what's shown in this article is actually just a small fraction\nof the code I typed, including all book snippets and most exercises\n(get a count or rough estimate)\n\nMaybe show the fizzbuzz example?\n\nDaily Review\n------------\n\nCover this by adding a wrap-up paragraph or two at the end of each day entry.\nSummarize the most important lessons learned, the pitfalls and triumphs,\nand what I had planned to do the next day.\n\nNext actions\n------------\n\nWhat did I leave undone? What could I have done next?\n\n* Process termination still not 100% clear to me\n* I still suck at concurrency concepts\n\nWrapup\n------\n\nRe-state the four step system and its benefits in a couple paragraphs,\ninvite others to try it.\n\nLearning is cyclical. Always go back and see how your new knowledge might have\nbeen applied to old problems, particularly wherever you struggled before.\n\n\nRaw journal notes + checklists\n-------------------------------\n\nShare, don't share, summarize?\n\n\nPreliminaries: December 26 - Jan 5 (12 hrs)\n-------------------------------------------\n\nSummarize what was studied / learned during this time period.\n\nMaybe create a bulleted list of \"What I already know about erlang\"\nbased on Ch 1-6 and my prior knowledge.\n\n\n* structures: Atoms, Integers, Floats, Tuples, Lists, Records, Strings(`*`)\n* constructs: Modules, annotations, functions.\n* workflow: shell, compilation\n* pattern matching, list comprehensions, guards, recursive coding, \n single assignment\n* Using io:format to print out output\n* Basic error handling\n* Standard library features like `erlang:*` and `lists:*` \n\n(Probably more)\n\nConsider showing fizzbuzz example for this.\n\nDay 1: January 6 (Monday)\n-------------------------------------------\n\n### Finding the smallest element of a list\n\nGood refresheer on pattern matching and recursive coding style (from Journal)\n\nOriginal solution:\n\n```erlang\n-module(mylists).\n-export([minimum/1]).\n\nminimum([H|T]) -> minimum(T, H).\n\nminimum([], Min) -> Min;\nminimum([H|T], Min) -> \n case Smallest < H of\n true -> minimum(T, Min);\n false -> minimum(T, H)\n end.\n```\n\nIn retrospect:\n\n```erlang\n-module(mylists).\n-export([minimum/1]).\n\nminimum([Min|T]) -> minimum(T, Min).\n\nminimum([], Min) -> Min;\n\nminimum([H|T], Min) when Min < H -> minimum(T, Min);\n\nminimum([Min|T], _) -> minimum(T, Min).\n```\n\n## (Almost working ping-pong) \n\nI didn't have much to go on yet, but the file server example and hello world\nexample were useful in preparing this. It's always good to save all exercises\nyou work on / projects you work on because they become your library for\nlooking up features in context.\n\nAlmost works, but has a bug in it! Deal with that later.\n\n```erlang\n-module(ping_pong).\n-export([start/1, loop/1]).\n\nstart(Message) -> spawn(ping_pong, loop, [Message]).\n\nloop(Message) ->\n receive\n {Client, N} ->\n io:format(\"~p Received: ~s~n\", [self(), Message]),\n\n %% FIXME: Find out why this isn't working! %%\n case N of\n 1 -> Client ! { self(), N -1 }, exit(self(), ok);\n 0 -> exit(self(), ok);\n true -> Client ! { self(), N - 1 }\n end\n end,\n loop(Message).\n```\n\n## Reading notes\n\nAlready read Ch 1-6 (skimmed 5) during preliminaries doing most of their\nexercises. Decided to skip Ch 7 on binary processing, since none of my projects\nwould need it.\n\nFocus for the day is on Ch 8, a misc. grab bag of erlang features. \n\n> TODO: Try to find features introduced in this chapter and point them out\nif they're shown in other code samples, particularly in this day's project\nor the next day's warmup exercise.\n\nPossible points of interest:\n\n* Code loading (pp124-126)\n* Macros / preprocessor\n* Annotations\n* Non-short circuiting logic\n* List subtraction\n* Process dictionary\n\n## Bowling\n\nDiscuss the first data modeling challenges. Note that at this point, I'm not\neven sure what the differences between lists and tuples are (see the journal).\n\nNote how pattern matching makes non-uniformity less awkward than in Ruby\n(e.g. `{10}` vs `{A, B}`)\n\nNote how I got hung up on `true -> ...` in case for a while, but eventually\ncame to realize that catchall would be something like `_ -> ...`. Discovering\nthe same bug in this code helped me realize what I'd need to do to fix\nthe other ping-pong code, but only after sleeping on it (maybe split\nthis explanation up into two parts, one in today's review, the other in the\nnext day). \n\nNote how it was fun to write simple unit tests this way, if brittle.\n\n```erlang\n-module(bowling).\n-export([score/1, test/0]).\n\nscore([]) -> 0;\nscore([H|T]) -> \n case H of\n {A} ->\n\n [H1|T1] = T,\n\n case H1 of\n { B, C } -> A + B + C + score(T);\n { B } -> \n case T1 of\n [] -> 0;\n _ ->\n [H2|_] = T1,\n case H2 of\n { C, _ } -> A + B + C + score(T);\n { C } -> A + B + C + score(T)\n end\n end\n end;\n\n {A, B} when 10 =:= A + B ->\n [H1|_] = T,\n\n case H1 of\n {C, _} -> A + B + C + score(T);\n {C} -> A + B + C + score(T)\n end;\n\n\n {A, B} -> H, A + B + score(T)\n end.\n\ntest() -> \n 9 = score([{7, 2}]), \n\n % 4 6 8 9 5 0 3 1 0 4\n 40 = score([{1,3},{2,4},{3,5},{5,4},{2,3},{0,0},{1,2},{1,0},{0,0},{1,3}]),\n\n % 4 6 8 S:12 5 0 3 1 0 4\n 43 = score([{1,3},{2,4},{3,5},{5,5},{2,3},{0,0},{1,2},{1,0},{0,0},{1,3}]),\n\n % 4 6 8 S:15 5 0 3 1 0 4\n 46 = score([{1,3},{2,4},{3,5},{10},{2,3},{0,0},{1,2},{1,0},{0,0},{1,3}]),\n\n % 4 6 8 S:14 S:11 1 3 1 0 4\n 52 = score([{1,3},{2,4},{3,5},{5,5},{4,6},{1,0},{1,2},{1,0},{0,0},{1,3}]),\n\n % 4 6 8 S:21 S:13 3 3 1 0 4\n 63 = score([{1,3},{2,4},{3,5},{10},{10},{1,2},{1,2},{1,0},{0,0},{1,3}]),\n\n 300 = score([{10},{10},{10},{10},{10},{10},{10},{10},{10},{10},{10},{10}]),\n\n % 30 30 30 30 29 20 20 30 30 30 \n 279 = score([{10},{10},{10},{10},{10},{10},{9,1},{10},{10},{10},{10},{10}]),\n\n ok.\n```\n\n## Day wrapup notes\n\n> The code I wrote for my bowling score calculator has no error handling, and\nis a terrible mess of case statements, but it seems to be working! \n\nDay 2: January 7 (Monday)\n-------------------------------------------\n\nOriginally planned to spend a whole extra\nday on sequential erlang, but then realized\nthat concurrency is more interesting to me. \n(unsure to put this point at the head, the summary, or omit it)\n\n## Working ping pong + Refactored ping-pong\n\nRealized `true -> ...` case wasn't working but didn't connect it\nto yesterday's notes yet, switch to `if`.\n\n(working example)\n\n```erlang\n-module(ping_pong).\n-export([start/1, loop/1]).\n\nstart(N) -> \n Ping = spawn(ping_pong, loop, [ping]),\n Pong = spawn(ping_pong, loop, [pong]),\n\n Ping ! { Pong, N },\n done.\n\nloop(Message) ->\n receive\n {Client, N} ->\n io:format(\"[~p] ~p Received: ~s ~n\", [N, self(), Message]),\n\n if \n N > 2 -> Client ! { self(), N - 1 }, loop(Message);\n N =:= 2 -> Client ! { self(), N - 1 };\n true -> void\n end\n end.\n```\n\nThen realize that I literally used the same pattern in my bowling example, but\nused `case`, and switched back to it:\n\n```erlang\n-module(ping_pong).\n-export([start/1, loop/1]).\n\nstart(N) -> \n Ping = spawn(ping_pong, loop, [ping]),\n Pong = spawn(ping_pong, loop, [pong]),\n\n Ping ! { Pong, N },\n done.\n\nloop(Message) ->\n receive\n {Client, N} ->\n io:format(\"[~p] ~p Received: ~s ~n\", [N, self(), Message]),\n\n case N of\n 1 -> done;\n 2 -> Client ! { self(), N - 1 }, done;\n _ -> Client ! { self(), N - 1 }, loop(Message)\n end\n end.\n```\n\nHere is where I struggled with process termination. Do we explicitly\ncall exit() when the process is done? Simply let the loop terminate?\nLeave the loop running in a refreshed state? Is it a concern to have\nprocesses accumulating in a zombie state of some sort? The exercise indicates\nthat we should make sure to terminate the processes gracefully, but I'm\nunsure what that means in Erlang.\n\nED NOTE: Did not look this up at the time, but there is some relevant \ndiscussion here: http://stackoverflow.com/questions/14515480/processes-exiting-normally\n\nSeems that a function simply returning does not terminate a process in erlang.\n\n### Reading notes\n\nDecided to skip Ch 9 on type system and Ch 10 on compiler tools, because neither\nwere essential for my projects and I was concerned about the limited time I'd\nhave in the practice week.\n\nPossible points of interest:\n\n* Tuple modules\n* Message processing semantics (esp unmatched messages) -- 194\n (contrast this to our actor article, at least my understanding of it\n being roughly equivalent to a work queue. Possible look at Celluloid)\n* More confusion around code loading\n* Request/response pattern + RPC style\n* Receive timeouts\n* Register\n* TCO caveat\n* Processes are cheap (seems like a broken record on this)\n* Boilerplate ??? (maybe not)\n\nFind a way to discuss the above without drawing too much from the book,\nmaybe pick a few topics and show original examples.\n\n## Refactored bowling\n\nNote the piecewise problem decomposition, and my concerns about being too\ngolf-ish, brittle. But also not my feeling of how it helps clarity / \nelegance / simplicity, and linerizes the code to eliminate nested conditionals.\n\n```erlang\n-module(bowling).\n-export([score/1, test/0]).\n\nscore([]) -> 0;\nscore([{10}|T]) -> strike(T) + score(T);\nscore([{A,B}|T]) when 10 =:= A + B -> spare(T) + score(T);\nscore([{A,B}|T]) -> A + B + score(T).\n\nstrike([]) -> 0;\nstrike([{10}]) -> 0;\nstrike([{10}, {10}|_]) -> 30;\nstrike([{10}, {Ball2, _}|_]) -> 20 + Ball2;\nstrike([{Ball1, Ball2}|_]) -> 10 + Ball1 + Ball2.\n\nspare([]) -> 0;\nspare([{10}|_]) -> 20;\nspare([{NextBall, _}|_]) -> 10 + NextBall.\n\ntest() -> \n 9 = score([{7, 2}]), \n\n % 4 6 8 9 5 0 3 1 0 4\n 40 = score([{1,3},{2,4},{3,5},{5,4},{2,3},{0,0},{1,2},{1,0},{0,0},{1,3}]),\n\n % 4 6 8 S:12 5 0 3 1 0 4\n 43 = score([{1,3},{2,4},{3,5},{5,5},{2,3},{0,0},{1,2},{1,0},{0,0},{1,3}]),\n\n % 4 6 8 S:15 5 0 3 1 0 4\n 46 = score([{1,3},{2,4},{3,5},{10},{2,3},{0,0},{1,2},{1,0},{0,0},{1,3}]),\n\n % 4 6 8 S:14 S:11 1 3 1 0 4\n 52 = score([{1,3},{2,4},{3,5},{5,5},{4,6},{1,0},{1,2},{1,0},{0,0},{1,3}]),\n\n % 4 6 8 S:21 S:13 3 3 1 0 4\n 63 = score([{1,3},{2,4},{3,5},{10},{10},{1,2},{1,2},{1,0},{0,0},{1,3}]),\n\n 300 = score([{10},{10},{10},{10},{10},{10},{10},{10},{10},{10},{10},{10}]),\n\n % 30 30 30 30 29 20 20 30 30 30 \n 279 = score([{10},{10},{10},{10},{10},{10},{9,1},{10},{10},{10},{10},{10}]),\n\n % 30 30 30 30 29 20 20 29 20 20 \n 258 = score([{10},{10},{10},{10},{10},{10},{9,1},{10},{10},{9,1},{10}]),\n\n % 30 30 30 30 29 20 20 29 20 20 \n 257 = score([{10},{10},{10},{10},{10},{10},{9,1},{10},{10},{9,1},{9,1}]),\n\n ok.\n```\n\n(refactoring also applies to list:min, though I didn't know\nit at the time)\n\n## Inital very broken dining philosophers\n\nhttps://github.com/sandal/erlang-practice/commit/df17dddec3588bff7b73417d0290c48beb2cf6bf\n\n(But maybe Chopstick is almost right? **CHECK THIS**)\n\n## Day wrapup notes\n\nUsed most of my wrapup time on reading, because I had exercises queued\nup for the next day (register race, ring).\n\nDay 3: January 8 (Wednesday) -- cut short\n-------------------------------------------\n\n(May want to streamline the discussion for this day because of its lack of consistency with the\nother days, but see how it plays out.)\n\n## Register race condition\n\nNote my struggle and lessons learned from this, even though I couldn't solve \nit myself. Note epiphany about Erlang not being totally immune to race\nconditions, and my understanding of the problem. Also note request/response\npattern seen again here for sync (point out synthesis used to get from\nhere to dining philosophers).\n\n```erlang\n-module(concurrency).\n-export([start/2]).\n\n% Solution is from http://forums.pragprog.com/forums/27/topics/124\n\nstart(Atom, Fun) ->\n Registrant = self(),\n spawn(\n fun() ->\n try register(Atom, self()) of\n true ->\n Registrant ! true,\n Fun()\n catch\n error:badarg ->\n Registrant ! false\n end\n end),\n receive\n true -> true;\n false -> erlang:error(badarg)\n end.\n```\n\n( consider cleaning up code )\n\n\nMy understanding after investigation (see journal for more details):\n\n> It's not possible to use `whereis` to verify that a process has been\nregistered or not, because both processes could pass that test BEFORE their call\nto `register` is processed. If one process completes the register process, the\nother will fail, but only after `start()` returns, which is not what the\nexercise calls for.\n\n> To verify success or failure BEFORE `start()` returns, the spawned processes\ncommunicates back to the process that spawned them about their status. The\nparent process uses `receive` to wait for a response, ensuring that failure is\ncommunicated at the time the method returns, not after.\n\nThis problem is more general than the race condition, it also hints at how to\nwrite functions that fail sychronously. Unsure whether synchronous failure\nis an edge case in Erlang or not (where it would be the default in\nsingle-threaded Ruby)\n\n### Reading notes (discuss Philo research?)\n\nPossible points of interest:\n\n* Self-calling `Fun`. Why did I want this? For quick spawn examples???\n* \"Let some other process fix the error\" and \"Let it crash\"\n* Corrective vs. defensive programming\n* Some extra research into DIning Philoosopher: Chandry/Misra and a waiter.\n\n## Day wrapup notes\n\nDidn't go as planned, mostly just did some work on exercise and a whole lot of\nsearching around for answers. Also didn't get that far on Dining Philosophers,\nexcept to research strategy. But rather than attempt to make up for lost time,\nI just let it go.\n\nDay 4: January 9 (Thursday)\n-------------------------------------------\n\n## Process Ring\n\nnote dining philosophers synthesis.\n(or maybe not, because we didn't go chandry/misra)\n\nnote mixed feelings about academic nature of exercise\nnote usefulness of drawing a picture\n\n```erlang\n-module(ring(Probably more).\n-export([send/2, loop/1]).\n\nsend(N, M) ->\n Head = build(N),\n Head ! { deliver, howdy, N*M }.\n\nstart() ->\n spawn(ring, loop, [void]).\n\nconnect(From, To) ->\n From ! {observe, To}.\n\nbuild(N) ->\n First = start(),\n Last = build(N-1, First),\n\n connect(Last, First),\n First.\n\nbuild(0, Current) -> Current;\n\nbuild(N, Current) ->\n Next = start(),\n connect(Current, Next),\n build(N-1, Next).\n\nloop(Observer) ->\n receive\n {observe, NewObserver} -> \n io:format(\"~p is now observing ~p\", [self(), NewObserver]),\n loop(NewObserver);\n {deliver, _, 0} ->\n io:format(\"Done sending messages!~n\"),\n loop(Observer);\n {deliver, Message, Count} when Observer =/= void ->\n io:format(\"[~p], ~p is sending message ~p to ~p~n\", \n [Count, self(), Message, Observer]),\n Observer ! {deliver, Message, Count - 1},\n loop(Observer)\n end.\n```\n\n## Reading notes\n\nPoints of interest:\n\n* Process linking and monitoring\n* Various ways of error signalling\n* Linking + monitoring patterns / firewalls\n\n(most time spent on book code samples)\n\n\n## Dining philosophers\n\n\n```erlang\n-module(philosophers).\n-export([dine/0, loop/3]).\n\ndine() ->\n [C1, C2, C3, C4, C5] = [chopstick:start(X) || X <- [1,2,3,4,5]],\n dine(C1, C2, C3, C4, C5).\n\n\ndine(C1, C2, C3, C4, C5) -> \n Aristotle = spawn(philosophers, loop, [\"Aristotle\", C1, C2]),\n Popper = spawn(philosophers, loop, [\"Popper\", C2, C3]),\n Epictetus = spawn(philosophers, loop, [\"Epictetus\", C3, C4]),\n Heraclitus = spawn(philosophers, loop, [\"Heraclitus\", C4, C5]),\n Schopenhauer = spawn(philosophers, loop, [\"Schopenhauer\", C1, C5]),\n\n Aristotle ! Popper ! Epictetus ! Heraclitus ! Schopenhauer ! think.\n\nloop(Philosopher, LeftChopstick, RightChopstick) ->\n receive\n think -> \n io:format(\"~p is thinking.~n\", [Philosopher]),\n timer:sleep(1000),\n\n self() ! eat;\n eat -> \n LeftChopstick ! {take, self()},\n\n receive\n {cs, FirstChopstick} ->\n io:format(\"~p picked up chopstick ~p~n\", [Philosopher, FirstChopstick]), \n\n RightChopstick ! {take, self()},\n receive\n {cs, SecondChopstick} ->\n io:format(\"~p picked up chopstick ~p~n\", [Philosopher, SecondChopstick]),\n io:format(\"~p is eating.~n\", [Philosopher]),\n timer:sleep(1000)\n end\n end,\n\n LeftChopstick ! {drop, self()},\n RightChopstick ! {drop, self()},\n\n io:format(\"~p is done eating, releases chopsticks ~p and ~p~n\",\n [Philosopher, FirstChopstick, SecondChopstick]),\n\n self() ! think\n end,\n\n loop(Philosopher, LeftChopstick, RightChopstick).\n```\n\n```erlang\n-module(chopstick).\n-export([start/1, loop/2]).\n\nstart(Number) ->\n spawn(chopstick, loop, [Number, nobody]).\n\nloop(Number, Owner) ->\n receive\n {take, Owner} -> loop(Number, Owner);\n {take, NewOwner} when Owner =:= nobody -> \n NewOwner ! {cs, Number},\n loop(Number, NewOwner);\n {drop, Owner} ->\n loop(Number, nobody)\n end.\n```\n\n(note request/response synchronization and similarity to\nN-ring problem, and also why I used the naive solution)\n\nNote what I learned about Erlang from this example, even when using the naive\nsolution.\n\nNever really thought through the many ways of solving this problem.\n\n## wrapup\n\n* Note about syntax errors (journal)\n* Thoughts about asynchronous message delivery\n\n\nDay 5: January 10 (Friday)\n-------------------------------------------\n\n### Start with monitor\n\nNote points of interest here, mainly error handling.\n\n```erlang\n-module(errors).\n-export([my_spawn/3]).\n\nmy_spawn(Mod, Func, Args) ->\n Pid = spawn(Mod, Func, Args),\n {T1, _} = statistics(wall_clock),\n\n spawn(fun() ->\n Ref = monitor(process, Pid),\n receive\n { 'DOWN', Ref, process, Pid, Why } ->\n io:format(\"~p went down with reason: ~p~n\", [Pid, Why]),\n\n {T2, _} = statistics(wall_clock),\n\n io:format(\"~p was alive for ~p seconds~n\", [Pid, (T2-T1)/1000])\n end\n end),\n\n Pid.\n```\n\n## Reading notes summarized here in transition\n * Die together workers and parent monitor\n * Keepalive\n * Distributed erlang, session names + cookie based auth\n * KVM uses process dictionary, what about tuple modules? (i guess it'd run\n into register problems and remote RPC issues)\n * Didn't quite get `nl()` to work\n * Recurring problems with stuck processes + testing callbacks\n (move to review instead?)\n\n## Trivial process\n\n(explain tangent)\n\n```erlang\n-module(trivial_process).\n-export([start/0, loop/1]).\n\nstart() -> spawn(?MODULE, loop, [1]).\n\nloop(N) ->\n io:format(\"Tick ~p.~n\", [N]),\n\n receive\n _ -> error(\"Boom!\")\n after 1000 ->\n loop(N+1)\n end.\n```\n\n\n## Rover\n\n\n```erlang\n-module(world).\n-export([start/1, loop/3]).\n\nstart(Filename) ->\n spawn(world, loop, [read(Filename), 11, 13]).\n\nloop(MapData, Row, Col) ->\n receive \n {Caller, MsgID, snapshot} ->\n Caller ! { self(), MsgID, {snapshot, snapshot(MapData, Row, Col)}},\n loop(MapData, Row, Col);\n {Caller, MsgID, move_north} ->\n Caller ! { self(), MsgID, {move_north, Row-1, Col}},\n loop(MapData, Row-1, Col);\n {Caller, MsgID, move_south} ->\n Caller ! { self(), MsgID, {move_south, Row+1, Col}},\n loop(MapData, Row+1, Col);\n {Caller, MsgID, move_east} ->\n Caller ! { self(), MsgID, {move_east, Row, Col+1}},\n loop(MapData, Row, Col+1);\n {Caller, MsgID, move_west} ->\n Caller ! { self(), MsgID, {move_west, Row, Col-1}},\n loop(MapData, Row, Col-1)\n end.\n\n\nread(Filename) ->\n { ok, MapBinary } = file:read_file(Filename),\n MapText = binary_to_list(MapBinary),\n list_to_tuple(\n [ list_to_tuple(string:tokens(X, \" \")) || \n X <- string:tokens(MapText, \"\\n\")]).\n\n% TODO: add @ sign to indicate current location.\nsnapshot(Map, Row, Col) ->\n RowWindow = [ element(RowD, Map) || \n RowD <- lists:seq(Row - 2, Row + 2) ],\n\n string:join(\n lists:map(fun(RowData) ->\n string:join(\n [element(ColD, RowData) || ColD <- lists:seq(Col - 2, Col + 2)], \n \" \")\n end, \n RowWindow), \n \"\\n\") ++ \"\\n\".\n```\n\n```erl\n-module(radio).\n-export([start/1, loop/1]).\n-define(TRANSMISSION_DELAY, 5000).\n\nstart(Controller) -> spawn(radio, loop, [Controller]).\n\nloop(Controller) ->\n receive\n { transmit, Pid, Message } ->\n erlang:send_after(?TRANSMISSION_DELAY, Pid, {self(), erlang:make_ref(), Message});\n Message ->\n erlang:send_after(?TRANSMISSION_DELAY, Controller, Message)\n end,\n\n loop(Controller).\n```\n\n\n```erl\n-module(controller).\n-export([start/0, loop/0]).\n\nstart() -> spawn(controller, loop, []).\n\nloop() ->\n receive\n {_, MsgId, {snapshot, MapData} } ->\n io:format(\"~s~n~n(msg id: ~p)~n\", [MapData, MsgId]);\n Any -> io:format(\"Received message: ~p~n\", [Any])\n end,\n loop().\n```\n\nRover (note ugly map parsing code, consider attempting a refactor,\nnote ease of concurrency stuff)\n\n## Daily wrapup\n\nThis project has given me a very visceral lesson in the differences between:\n\n* Reading code\n* Running example code\n* Trying out coding exercises\n* Working on toy projects\n* Working on real projects\n\nLearning a language is an N-dimensional activity, it's surprising that we\ntend to have a much more simplistic view of what is involved (or at least I do).\nSo much of a willingness to theorize and discuss that which we have very little\npractical experience with.\n" }, { "path": "articles/v7/008-oss-quality.md", "content": "> This article was written in collaboration with Eric Hodel\n> ([@drbrain](http://twitter.com/drbrain)), a developer from Seattle. \n> Eric is a Ruby core team member, and he also maintains RubyGems \n> and RDoc. \n\nA big challenge in managing open source projects is that their codebases tend\nto decay as they grow. This isn't due to a lack of technically skilled\ncontributors, but instead is a result of the gradual loss of understandability \nthat comes along with any long-term and open-ended project \nthat has an distributed team of volunteers supporting it.\n\nOnce a project becomes more useful, it naturally attracts a more\ndiverse group of developers who are interested in adapting the codebase to\nmeet their own needs. Patches are submitted by contributors who do not fully \nunderstand a project's implementation, and maintainers merge these patches \nwithout fully understanding the needs of their contributors. Maintainers \nmay also struggle to remember the reasoning behind any of their own code that they \nhaven't touched in a while, but they still need to be able to work with it.\n\nAs a result of both of these influencing factors, mistaken assumptions tend to \nproliferate as a project grows, and with them come bugs and undefined behaviors. \nWhen direct knowledge of the codebase becomes limited and unreliable, it's easy to \nlet code quality standards slip without fully realizing the potential\nfor future problems. \n\nIf bad code continues to accumulate in this fashion, improving one part of a \na project usually means breaking something else in the \nprocess. Once a maintainer starts spending most of their time fixing bugs, \nit gets hard to move their project forward in meaningful \nways. This is where open source development stops being fun, and starts feeling \nlike a painful chore.\n\nNot all projects need to end up this way, though. As long as project maintainers \nmake sure to keep the quality arrow pointing upwards over the long haul, \nany bad code that temporarily accumulates in a project can always be replaced with \nbetter code whenever things start getting painful. The real challenge is to \nestablish healthy maintenance practices that address quality issues \nin a consistent and sustainable way. \n\n### Developing a process-oriented approach towards quality\n\nIn this article, we'll discuss three specific tactics we've used in \nour own projects that can be applied at any stage in the software \ndevelopment lifecycle. These are not quick fixes; they are helpful \nhabits that drive up understandability and code quality more and more\nas you continue to practice them. The good news is that even though \nit might be challenging to keep up with these efforts on a daily basis, \nthe recommendations themselves are very simple:\n\n1. Let external changes drive incremental quality improvements\n2. Treat all code with inadequate testing as legacy code\n3. Expand functionality via well-defined extension points \n\nWe'll now take a look at each of these guidelines individually and walk \nyou through some examples of how we've put them into practice in RDoc, \nRubyGems, and Prawn -- three projects that have had their own share of \nquality issues over the years, but continue to serve very diverse \ncommunities of users and contributors.\n\n### 1) Let external changes drive incremental quality improvements\n\nAlthough there is often an endless amount of cleanup work that can\nbe done in mature software projects, there is rarely enough\navailable development time to invest in these efforts. For programmers \nworking on open source in their spare time, it is hard enough\nto keep up with new incoming requests, so most preventative maintenance \nwork ends up being deferred indefinitely. When cleanup efforts do happen,\nthey tend to be done in concentrated bursts and then things go back\nto business-as-usual from there.\n\nA better approach is to pay down technical debts little by little, not as a\ndistinct activity but as part of responding to ordinary change requests. There\nare only two rules to remember when applying this technique in your daily work:\n\n* Try to avoid making the codebase worse with each new change, or at least\nminimize new maintenance costs as much as possible.\n* If there is an easy way to improve the code while doing everyday work, \ngo ahead and invest a little bit of effort now to make future changes easier. \n\nThe amount of energy spent on meeting these two guidelines should be proportional\nto the perceived risks and rewards of the change request itself, but typically\nit doesn't take a lot of extra effort. It may mean spending an extra 10 minutes on a\npatch that would take an hour to develop, or an extra hour on a patch that would\ntake a day to prepare. In any case, it should feel like an obviously good\ninvestment that is well worth the cost you are paying for it.\n\nThere is a great example in Prawn that illustrates this technique being used,\nand if you want to see it in its raw form, you can check out [this pull\nrequest](https://github.com/prawnpdf/prawn/pull/587) from Matt Patterson.\n\nMatt's request was to change the way that Prawn's image loading\nfeature detected whether it was working with an I/O object or a path to\na file on disk. Initially Prawn assumed that any object responding to `read` \nwould be treated as an I/O object, but this was too loose of a test and\ncaused some subtle failures when working with `Pathname` objects.\n\nThe technical details of the change are not important here, so don't worry if\nyou don't understand them. Instead, just look at the method that would need to\nbe altered to fix this problem, and ask yourself whether you would feel\ncomfortable making a change to it:\n\n```ruby\ndef build_image_object(file)\n file.rewind if file.respond_to?(:rewind)\n file.binmode if file.respond_to?(:binmode)\n\n if file.respond_to?(:read)\n image_content = file.read\n else\n raise ArgumentError, \"#{file} not found\" unless File.file?(file) \n image_content = File.binread(file)\n end\n \n image_sha1 = Digest::SHA1.hexdigest(image_content)\n\n if image_registry[image_sha1]\n info = image_registry[image_sha1][:info]\n image_obj = image_registry[image_sha1][:obj]\n else\n info = Prawn.image_handler.find(image_content).new(image_content)\n\n min_version(info.min_pdf_version) if info.respond_to?(:min_pdf_version)\n\n image_obj = info.build_pdf_object(self)\n image_registry[image_sha1] = {:obj => image_obj, :info => info}\n end\n\n [image_obj, info]\nend\n```\n\nAlthough this probably isn't the absolute worst code you have ever seen, \nit isn't very easy to read. Because it takes on many responsibilities,\nit's hard to even summarize what it is supposed to do! Fortunately for Matt,\nthe part that he would need to change was only the first few lines of the \nmethod, which are reasonably easy to group together:\n\n```ruby\ndef build_image_object(file)\n file.rewind if file.respond_to?(:rewind)\n file.binmode if file.respond_to?(:binmode)\n\n if file.respond_to?(:read)\n image_content = file.read\n else\n raise ArgumentError, \"#{file} not found\" unless File.file?(file) \n image_content = File.binread(file)\n end\n\n # ... everything else \nend\n```\n\nThe quick fix would have been to edit these lines directly, but Matt recognized\nthe opportunity to isolate a bit of related functionality and make the code a\nlittle bit better in the process of doing so. Pushing these lines of code down\ninto a helper method and tweaking them slightly resulted in the following\ncleanup to the `build_image_object` method:\n\n```ruby\ndef build_image_object(file)\n io = verify_and_open_image(file)\n image_content = io.read\n\n # ... everything else\nend\n```\n\nIn the newly created helper method, Matt introduced his desired change, \nwhich is much easier to understand in isolation than it would have been in the \noriginal `build_image_object` method definition. In particular, he changed\nthe duck typing test to look for `rewind` rather than `read`, in the hopes\nthat it would be a more reliable way to detect I/O-like objects. Everything \nelse would be wrapped in a `Pathname` instance:\n\n```ruby\ndef verify_and_open_image(io_or_path)\n if io_or_path.respond_to?(:rewind)\n io = io_or_path\n\n io.rewind\n\n io.binmode if io.respond_to?(:binmode)\n return io\n end\n\n io_or_path = Pathname.new(io_or_path)\n raise ArgumentError, \"#{io_or_path} not found\" unless io_or_path.file?\n \n io_or_path.open('rb')\nend\n```\n\nAt this point, he could have submitted a pull request, because the tests were\nstill green and the new behavior was working as expected. However, the issue\nhe had set out to fix in the first place wasn't causing Prawn's tests to fail,\nand that was a sign that there was some undefined behavior at the root of\nthis problem. Although Prawn had some tests for reading images referenced by\n`Pathname` objects, it only had done its checks at a high level, and did not\nverify that the PDF output was being rendered correctly.\n\nA test would be needed at the lower level to verify that the output was no\nlonger corrupted, but this kind of testing is slightly tedious to do in Prawn.\nNoticing this rough spot, Matt created an RSpec matcher to make this kind of\ntesting easier to do in the future:\n\n```ruby\nRSpec::Matchers.define :have_parseable_xobjects do\n match do |actual|\n expect { PDF::Inspector::XObject.analyze(actual.render) }.not_to raise_error\n true\n end\n failure_message_for_should do |actual|\n \"expected that #{actual}'s XObjects could be successfully parsed\"\n end\nend\n```\n\nFinally, he provided a few test cases to demonstrate that his patch\nfixed the problem he was interested in, and also covered some other \ncommon use cases as well:\n\n```ruby\ncontext \"setting the length of the bytestream\" do\n it \"should correctly work with images from Pathname objects\" do\n info = @pdf.image(Pathname.new(@filename))\n expect(@pdf).to have_parseable_xobjects\n end\n\n it \"should correctly work with images from IO objects\" do\n info = @pdf.image(File.open(@filename, 'rb'))\n expect(@pdf).to have_parseable_xobjects\n end\n\n it \"should correctly work with images from IO objects not set to mode rb\" do\n info = @pdf.image(File.open(@filename, 'r'))\n expect(@pdf).to have_parseable_xobjects\n end\nend\n```\n\nWhen you put all of these changes together, the total value of this patch\nis much greater than the somewhat obscure bug it fixed. By addressing\nsome minor pain points as he worked, Matt also improved Prawn in the\nfollowing ways:\n\n* The `build_image_object` method is now more understandable because one \nof its responsibilities has been broken out into its own method.\n\n* The `verify_and_open_image` method allows us to group together all the\nbasic guard clauses for determining how to read the image data, \nmaking it easier to see exactly what those rules are.\n\n* The added tests clarify the intended behavior of Prawn's image loading\nmechanism.\n\n* The newly added RSpec matcher will help us to do more\nPDF-level checks in future tests.\n\nNone of these changes required a specific and focused effort of refactoring or redesign,\nit just involved a bit of attention to detail and a willingness to make minor\nimprovements that would pay off for someone else in the future.\n\nAs a project maintainer, you cannot expect contributors to put this level of\neffort into their patches -- Matt really went above and beyond here. However, \nyou can definitely look for these kind of opportunities yourself during review \ntime, and either ask the contributor to make some revisions, or make them yourself \nbefore you merge in new changes. No matter who ends up doing the work, little by\nlittle these kinds of incremental cleanup efforts can turn a rough codebase into\nsomething pleasant to work with.\n\n### 2) Treat all code without adequate testing as legacy code\n\nHistorically, we've defined legacy code as code that was written long before our\ntime, without any consideration for our current needs. However, any untested\ncode can also be considered legacy code[^1], because it often\nhas many of the same characteristics that make outdated systems difficult to\nwork with. Open source projects evolve quickly, and even very clean code\ncan cause a lot of headaches if its intended behavior is left undefined.\n\nTo guard against the negative impacts of legacy code, it helps \nto continuously update your project's test suite so \nthat it constantly reflects your current understanding of the problem domain\nyou are working in. A good starting point is to make sure that your project\nhas good code coverage and that you keep your builds green in CI.\nOnce you've done that, the next step is to go beyond the idea of just having\nlots of tests and start focusing on making your test suite more capable \nof catching problems before they leak out into released code.\n\nHere are some things to keep in mind when considering the potential \nimpact that new changes will have on your project's stability:\n\n* Any behavior change introduced without test\ncoverage has a good chance of causing a defect or \naccidentally breaking backwards-compatibility in a future release.\n\n* A passing test suite is not proof that a change is well-defined\nand defect-free.\n\n* The only reliable way to verify that existing features have \nwell-defined behavior and good test coverage is to\nreview their code manually.\n\n* Contributors often don't understand your project's problem domain \nor its codebase well enough to know how to write good tests for\ntheir changes without some guidance.\n\nThese points are not meant to imply that each and every pull request\nought to be gone over with a fine-tooth comb -- they're only meant to \nserve as a reminder that maintaining a high quality test suite is\na harder problem than we often make it out to be. The same ideas\nof favoring incremental improvements over heroic efforts that\nwe discussed earlier also apply here. There is no need to\nrush towards a perfect test suite all at once, as long as it improves\non average over time.\n\nWe'll now look at a [pull request](https://github.com/rubygems/rubygems/pull/781/files) \nthat Brian Fletcher submitted to\nRubyGems for a good example of how these ideas can be applied \nin practice.\n\nBrian's request was to add support for Base64 encoded usernames and\npasswords in gem request URLs. Because RubyGems already supported\nthe use of HTTP Basic Auth with unencoded usernames and passwords in\nURLs, this was an easy change to make. The desired URL decoding functionality\nwas already implemented by `Gem::UriFormatter`, so the\ninitial commit for this pull request involved changing just a single line \nof code:\n\n```diff\n request = @request_class.new @uri.request_uri\n \n unless @uri.nil? || @uri.user.nil? || @uri.user.empty? then\n- request.basic_auth @uri.user, @uri.password\n+ request.basic_auth Gem::UriFormatter.new(@uri.user).unescape,\n+ Gem::UriFormatter.new(@uri.password).unescape\n end\n \n request.add_field 'User-Agent', @user_agent\n```\n\nOn the surface, this looks like a fairly safe change to make. Because it only\nadds support for a new edge case, it should preserve the original behavior\nfor URLs that did not need to be unescaped. No new test failures were introduced\nby this patch, and a quick look at the test suite shows that `Gem::UriFormatter`\nhas some tests covering its behavior.\n\nAs far as changes go, this one is definitely low risk. But if you dig in a\nlittle bit deeper, you can find a few things to worry about: \n\n* Even though only a single line of code was changed, that line of code\nwas at the beginning of a method that is almost 90 lines long. This isn't\nnecessarily a problem, but it should at least be a warning sign to slow\ndown and take a closer look at things.\n\n* A quick look at the test suite reveals that although there were tests\nfor the `unescape` method provided by `GemUri::Formatter`, there were no tests \nfor the use of Basic Auth in gem request URLs, which means the behavior\nthis patch was modifying was not formally defined. Because of this, we can't\nbe sure that a subtle incompatibility wasn't introduced by this patch, \nand we wouldn't know if one was introduced later due to a change to\n`GemUri::Formatter`, either.\n\n* The new behavior introduced by this patch also wasn't verified, which\nmeans that it could have possibly been accidentally removed in a future \nrefactoring or feature patch. Another contributor could easily assume \nthat URL decoding was incidental rather than intentional without\ntests that indicated otherwise.\n\nThese are the kind of problems that a detailed review can discover \nwhich are often invisible at the surface level. However, a much more\nefficient maintenance policy is to simply assume one or more of the \nabove problems exist whenever a change is introduced without tests, \nand then either add tests yourself or ask contributors to add them\nbefore merging. \n\nIn this case, Eric asked Brian to add a test after giving him some guidance \non how to go about implementing it. For reference, this was his exact request:\n\n> Can you add a test for this to test/rubygems/test_gem_request.rb?\n>\n> You should be able to examine the request object through the block #fetch yields to.\n\nIn response, Brian dug in and noticed that the base case of\nusing HTTP Basic Auth wasn't covered by the tests. So rather than simply \nadding a test for the new behavior he added, he went ahead and wrote tests \nfor both cases:\n\n```ruby\nclass TestGemRequest < Gem::TestCase\n def test_fetch_basic_auth\n uri = URI.parse \"https://user:pass@example.rubygems/specs.\" +\n Gem.marshal_version\n @request = Gem::Request.new(uri, Net::HTTP::Get, nil, nil)\n conn = util_stub_connection_for :body => :junk, :code => 200\n\n response = @request.fetch\n\n auth_header = conn.payload['Authorization']\n\n assert_equal \"Basic #{Base64.encode64('user:pass')}\".strip, auth_header\n end\n\n def test_fetch_basic_auth_encoded\n uri = URI.parse \"https://user:%7BDEScede%7Dpass@example.rubygems/specs.\" +\n Gem.marshal_version\n @request = Gem::Request.new(uri, Net::HTTP::Get, nil, nil)\n conn = util_stub_connection_for :body => :junk, :code => 200\n\n response = @request.fetch\n\n auth_header = conn.payload['Authorization']\n\n assert_equal \"Basic #{Base64.encode64('user:{DEScede}pass')}\".strip, \n auth_header\n end\nend\n```\n\nIt is hard to overstate the difference between a patch with these tests\nadded to it and one without tests. The original commit introduced a new\ndependency and more complex logic into a feature that lacked formal definition\nof its behavior. But as soon as these tests are added to the change request,\nRubyGems gains support for a new special condition in gem request URLs while \ntightening up the definition of the original behavior. The tests\nalso serve to protect both conditions from breaking without being noticed \nin the future.\n\nTaken individually, the risks of accepting untested patches are\nsmall enough that they don't seem important enough to worry about when you are pressed\nfor time. But in the aggregate, the effects of untested code will pile up until your\ncodebase really does become unworkable legacy code. For that reason, establishing\ngood habits about reviewing and shoring up tests on each new change can make a\nhuge difference in long-term maintainability.\n\n### 3) Expand functionality via well-defined extension points \n\nMost open source projects can benefit from having two clearly defined interfaces:\none for end-users, and one for developers who want to extend its functionality.\nThis point may seem tangentially related to code quality and maintainability,\nbut a well-defined extension API can greatly increase a project's stability.\n\nWhen its possible to add new functionality to a project without patching its\ncodebase directly, it becomes easier to separate essential features that most\npeople will need from features that are only relevant in certain rare \ncontexts. The ability to support external add-ons in a transparent way also\nmakes it possible to try experiments outside of your main codebase and then\nonly merge in features that prove to be both stable and widely used.\n\nEven within the scope of a single codebase, explicitly defining a layer one\nlevel beneath the surface forces you to think about what the common points\nof interaction are between your project's features. It also makes testing\neasier, because feature implementations tend to get slimmed down as the\nextension API becomes more capable. Each part can then be tested in \nisolation without having to think about large amorphous blobs\nof internal dependencies.\n\nIt may be hard to figure out how to create an extension API when you first start\nworking on a project, because at that time you probably don't know much\nabout the ways that people will need to extend its core behavior, and you may\nnot even have a good sense of what its core feature set should be! This is\ncompletely acceptable, and it makes sense to focus exclusively on your high-level \ninterface at first. But as your project matures, you can use the following guidelines to\nincrementally bring a suitable extension API into existence:\n\n* With each new feature request, ask yourself whether it could be implemented\nas an external add-on without patching your project's codebase. If not, figure \nout what extension points would make it possible to do so.\n\n* For any of your features that have become difficult to work with or overly\ncomplex, think about what extension points would need to be added in\norder to extract those features into external add-ons.\n\n* For any essential features that have clearly related functionality, \nfigure out what it would take to re-implement them on top of well defined \nextension points rather than relying on lots of private internal code.\n\nAt first, you may start by carrying out these design considerations as simple\nthought experiments that will indirectly influence the way you implement \nthings. Later, you can take them more seriously and seek to support\nnew functionality via external add-ons rather than merging new features \nunless there is a very good reason to do otherwise. Every project needs to\ndiscover the right balance for itself, but the basic idea is that the value of a\nclear extension API increases the longer a project is in active use.\n\nBecause RDoc has been around for a very long time and has a fairly decent extension \nAPI, it is a good library to look at for examples of what this technique has\nto offer. Without asking Eric for help, I looked into what it would take to autolink \nGithub issues, commits, and version tags in RDoc output. This isn't something I had \na practical use for, but I figured it would be a decent way to test how easily I \ncould extend the RDoc parser.\n\nI started with the following text as my input data: \n\n```\nPlease see #125, #127, and #159\n\nAlso see @bed324 and v0.14.0\n```\n\nMy goal was to produce the following HTML output after telling RDoc what repository\nthat these issues, commits, and tags referred to:\n\n```\nPlease see #125,\n#127, and \n#159
\n\nAlso see
![](\"../../assets/images/header.png\")
\n <%= @body %>\n