[
  {
    "path": ".gems",
    "content": "cutest:1.2.2\nbyebug:5.0.0\ndisque:0.0.6\ncelluloid:0.17.0\n"
  },
  {
    "path": ".gitignore",
    "content": "*.gem\nnodes.conf\n.deps\n"
  },
  {
    "path": ".travis.yml",
    "content": "language: ruby\nrvm:\n  - 2.2\nsudo: false\ncache:\n  bundler: false\n  directories:\n    - disque\n    - gems\nbefore_install:\n  - test -d disque/.git || git clone -n https://github.com/antirez/disque.git\n  - cd disque && git fetch origin && git checkout -f origin/master && make && cd ..\ninstall:\n  - export GEM_HOME=$PWD/gems/$RUBY_VERSION\n  - export GEM_PATH=$GEM_HOME:$GEM_PATH\n  - export PATH=$GEM_HOME/bin:$PWD/disque/src:$PATH\n  - cat .gems* | xargs gem install\nbefore_script: disque-server --daemonize yes\nscript: make test\n"
  },
  {
    "path": "CONTRIBUTING.md",
    "content": "### The Soveran Contribution Guidelines.\n\n(taken from some project at https://github.com/soveran)\n\nThis code tries to solve a particular problem with a very simple\nimplementation. We try to keep the code to a minimum while making\nit as clear as possible. The design is very likely finished, and\nif some feature is missing it is possible that it was left out on\npurpose. That said, new usage patterns may arise, and when that\nhappens we are ready to adapt if necessary.\n\nA good first step for contributing is to meet us on IRC and discuss\nideas. We spend a lot of time on #lesscode at freenode, always ready\nto talk about code and simplicity. If connecting to IRC is not an\noption, you can create an issue explaining the proposed change and\na use case. We pay a lot of attention to use cases, because our\ngoal is to keep the code base simple. Usually the result of a\nconversation is the creation of a different tool.\n\nPlease don't start the conversation with a pull request. The code\nshould come at last, and even though it may help to convey an idea,\nmore often than not it draws the attention to a particular\nimplementation.\n"
  },
  {
    "path": "LICENSE",
    "content": "The MIT License\n\nCopyright (c) 2015 Pablo Astigarraga | pote <pote@tardis.com.uy>\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in\nall copies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN\nTHE SOFTWARE.\n"
  },
  {
    "path": "Makefile",
    "content": "ifndef GEM_HOME\n  $(error GEM_HOME not set.)\nendif\n\nPACKAGES := disc\nVERSION_FILE := lib/disc/version.rb\n\nDEPS := ${GEM_HOME}/installed\nVERSION := $(shell sed -ne '/.*VERSION *= *\"\\(.*\\)\".*/s//\\1/p' <$(VERSION_FILE))\nGEMS := $(addprefix pkg/, $(addsuffix -$(VERSION).gem, $(PACKAGES)))\n\nexport RUBYLIB := lib:test:$(RUBYLIB)\n\nall: test $(GEMS)\n\nconsole: $(DEPS)\n\tirb -r disc\n\ntest: $(DEPS)\n\tcutest ./test/**/*_test.rb\n\nclean:\n\trm pkg/*.gem\n\nrelease: $(GEMS)\n\tgit tag v$(VERSION)\n\tgit push --tags\n\tfor gem in $^; do gem push $$gem; done\n\npkg/%-$(VERSION).gem: %.gemspec $(VERSION_FILE) | pkg\n\tgem build $<\n\tmv $(@F) pkg/\n\n$(DEPS): $(GEM_HOME) .gems\n\tcat .gems | xargs gem install && touch $(GEM_HOME)/installed\n\npkg $(GEM_HOME):\n\tmkdir -p $@\n\n.PHONY: all test release clean\n"
  },
  {
    "path": "README.md",
    "content": "# Disc [![Build Status](https://travis-ci.org/pote/disc.svg?branch=master)](https://travis-ci.org/pote/disc)\n\nDisc fills the gap between your Ruby service objects and [antirez](http://antirez.com/)'s wonderful [Disque](https://github.com/antirez/disque) backend.\n\n\n![Disc Wars!](https://cloud.githubusercontent.com/assets/437/8634016/b63ee0f8-27e6-11e5-9a78-51921bd32c88.jpg)\n\n## Basic Usage\n\n1.  Install the gem\n\n  ```bash\n  $ gem install disc\n  ```\n\n2. Write your jobs\n\n  ```ruby\n  require 'disc'\n\n  class CreateGameGrid\n    include Disc::Job\n    disc queue: 'urgent'\n\n    def perform(type)\n      # perform rather lengthy operations here.\n    end\n  end\n  ```\n\n3. Enqueue them to perform them asynchronously\n\n  ```ruby\n  CreateGameGrid.enqueue('light_cycle')\n  ```\n\n\n4. Create a file that requires anything needed for your jobs to run\n\n  ```ruby\n# disc_init.rb\n  # Require here anything that your application needs to run,\n  # like ORMs and your models, database configuration, etc.\n  Dir['./jobs/**/*.rb'].each { |job| require job }\n  ```\n\n5. Run as many Disc Worker processes as you wish, requiring your `disc_init.rb` file\n\n  ```bash\n  $ QUEUES=urgent,default disc -r ./disc_init.rb\n  ```\n\n4. Or enqueue them to be performed at some time in the future, or on a queue other than it's default.\n\n  ```ruby\n  CreateGameGrid.enqueue(\n    'disc_arena',\n    at: DateTime.new(2020, 12, 31),\n    queue: 'not_so_important'\n  )\n  ```\n\n## Disc Configuration\n\nDisc takes its configuration from environment variables.\n\n| ENV Variable       |  Default Value   | Description\n|:------------------:|:-----------------|:------------|\n| `QUEUES`           | 'default'        | The list of queues that `Disc::Worker` will listen to, it can be a single queue name or a list of comma-separated queues |\n| `DISC_CONCURRENCY` | '25'             | Amount of threads to spawn when Celluloid is available. |\n| `DISQUE_NODES`     | 'localhost:7711' | This is the list of Disque servers to connect to, it can be a single node or a list of comma-separated nodes |\n| `DISQUE_AUTH`      | ''               | Authorization credentials for Disque. |\n| `DISQUE_TIMEOUT`   | '100'            | Time in milliseconds that the client will wait for the Disque server to acknowledge and replicate a job |\n| `DISQUE_CYCLE`     | '1000'           | The client keeps track of which nodes are providing more jobs, after the amount of operations specified in cycle it tries to connect to the preferred node. |\n\n\n## Disc Jobs\n\n`Disc::Job` is a module you can include in your Ruby classes, this allows a Disc worker process to execute the code in them by adding a class method (`#enqueue`) with the following signature:\n\n```Ruby\ndef enqueue(arguments, at: nil, queue: nil, **options)\nend\n```\n\nSignature documentation follows:\n\n```ruby\n## Disc's `#enqueue` is the main user-facing method of a Disc job, it\n#  enqueues a job with a given set of arguments in Disque, so it can be\n#  picked up by a Disc worker process.\n#\n## Parameters:\n#\n## `arguments`  - an optional array of arguments with which to execute\n#                 the job's `perform` method.\n#\n# `at`          - an optional named parameter specifying a moment in the\n#                 future in which to run the job, must respond to\n#                 `#to_time`.\n#\n## `queue`      - an optional named parameter specifying the name of the\n#                 queue in which to store the job, defaults to the class\n#                 Disc queue or to 'default' if no Disc queue is specified\n#                 in the class.\n#\n##  `**options` - an optional hash of options to forward internally to\n#                 [disque-rb](https://github.com/soveran/disque-rb)'s\n#                 `#push` method, valid options are:\n#\n##  `replicate: <count>`  - specifies the number of nodes the job should\n#                           be replicated to.\n#\n### `delay: <sec>`        - specifies a delay time in seconds for the job\n#                           to be delivered to a Disc worker, it is ignored\n#                           if using the `at` parameter.\n#\n### `ttl: <sec>`          - specifies the job's time to live in seconds:\n#                           after this time, the job is deleted even if\n#                           it was not successfully delivered. If not\n#                           specified, the default TTL is one day.\n#\n### `maxlen: <count>`     - specifies that if there are already <count>\n#                           messages queued for the specified queue name,\n#                           the message is refused.\n#\n### `async: true`         - asks the server to let the command return ASAP\n#                           and replicate the job to other nodes in the background.\n#\n#\n### CAVEATS\n#\n## For convenience, any object can be passed as the `arguments` parameter,\n#  `Array()` will be used internally to preserve the array structure.\n#\n## The `arguments` parameter is serialized for storage using `Disc.serialize`\n#  and Disc workers picking it up use `Disc.deserialize` on it, both methods\n#  use standard library json but can be overriden by the user\n#\n```\n\nYou can see [Disque's ADDJOB documentation](https://github.com/antirez/disque#addjob-queue_name-job-ms-timeout-replicate-count-delay-sec-retry-sec-ttl-sec-maxlen-count-async) for more details\n\nWhen a Disc worker process is assigned a job, it will create a new intance of the job's class and execute the `perform` method with whatever arguments were previously passed to `#enqueue`.\n\nExample:\n\n```ruby\nclass ComplexJob\n  include Disc::Job\n  disc queue: 'urgent'\n\n  def perform(first_parameter, second_parameter)\n    # do things...\n  end\nend\n\n\nComplexJob.enqueue(['first argument', { second: 'argument' }])\n```\n\n### Managing Jobs\n\nDisc jobs can be managed by knowing their disque ID, this id is returned by the `#enqueue` method so you can control the job or query it's state from your application code.\n\n```ruby\nEchoer.enqueue('test')\n#=> \"DIa18101491133639148a574eb30cd2e12f25dcf8805a0SQ\"\n```\n\nThe disque ID is also available from within the context of an executing job, you can access it via `self.disque_id` if you wish to do things like notify Disque that a long-running job is still being executed.\n\n```ruby\nclass LongJob\n  include Disc::Job\n\n  def perform(first_parameter, second_parameter)\n    # Do things that take a while.\n\n    Disc.disque.call('WORKING', self.disque_id)\n\n    # Do more things that take a while.\n  end\nend\n```\n\n#### Job Status\n\nAfter a job is enqueued, you can check it's current status like so:\n\n```ruby\nEchoer.enqueue('test')\n#=> \"DIa18101491133639148a574eb30cd2e12f25dcf8805a0SQ\"\n\nDisc[\"DIa18101491133639148a574eb30cd2e12f25dcf8805a0SQ\"]\n#=> {\n  \"arguments\"=>[\"test\"],\n  \"class\"=>\"Echoer\",\n  \"id\"=>\"DIa18101491133639148a574eb30cd2e12f25dcf8805a0SQ\",\n  \"queue\"=>\"test\",\n  \"state\"=>\"queued\",\n  \"repl\"=>1,\n  \"ttl\"=>86391,\n  \"ctime\"=>1462488116652000000,\n  \"delay\"=>0,\n  \"retry\"=>8640,\n  \"nacks\"=>0,\n  \"additional-deliveries\"=>0,\n  \"nodes-delivered\"=>[\"a18101496d562e412a459c6b114561efe95c57cc\"],\n  \"nodes-confirmed\"=>[],\n  \"next-requeue-within\"=>8630995,\n  \"next-awake-within\"=>8630495,\n  \"body\"=>\"{\\\"class\\\":\\\"Echoer\\\",\\\"arguments\\\":[\\\"test\\\"]}\"\n}\n```\n\nThis information might vary, as it's retreived from Disque via the [`SHOW`](https://github.com/antirez/disque#show-job-id) command, only `arguments` and `class` are filled in by Disc, which are added by using `Disc.deserialize` on the `body` value.\n\n#### Do everything Disque can\n\nAccess to the disque ID allows us to leverage the Disque API to manage the job, you can execute Disque commands via the `Disc.disque.call()` method, see [the Disque API](https://github.com/antirez/disque#main-api) to see all the commands available.\n\n### Job Serialization\n\nJob information (their arguments, and class) need to be serialized in order to be stored\nin Disque, to this end Disc uses the `Disc.serialize` and `Disc.deserialize` methods.\n\nBy default, these methods use the Ruby standard library json implementation in order to serialize and deserialize job data, this has a few implications:\n\n1. Arguments passed to a job's `#enqueue` method need to be serializable by `Disc.serialize` and parsed back by `Disc.deserialize`, so by default you can't pass complex Ruby objects like a `user` model, instead, pass `user.id`, and use that from your job code.\n\n2. You can override `Disc.serialize` and `Disc.deserialize` to use a different JSON implementation, or MessagePack, or whatever else you wish.\n\n\n## Error handling\n\nWhen a job raises an exception, `Disc.on_error` is invoked with the error and\nthe job data. By default, this method prints the error to standard error, but\nyou can override it to report the error to your favorite error aggregator.\n\n``` ruby\n# On disc_init.rb\ndef Disc.on_error(exception, job)\n  # ... report the error\nend\n\nDir[\"./jobs/**/*.rb\"].each { |job| require job }\n```\n\n### Job Definition\n\nThe error handler function gets the data of the current job as a Hash, that has the following schema.\n\n|               |                                                       |\n|:-------------:|:------------------------------------------------------|\n| `'class'`     | (String) The Job class.                               |\n| `'arguments'` | (Array) The arguments passed to perform.              |\n| `'queue'`     | (String) The queue from which this job was picked up. |\n| `'disque_id'` | (String) Disque's job ID.                             |\n\n\n## Testing modes\n\nDisc includes a testing mode, so you can run your test suite without a need to run a Disque server.\n\n### Enqueue mode\n\nBy default, Disc places your jobs in an in-memory hash, with each queue being a key in the hash and values being an array.\n\n```ruby\nrequire 'disc'\nrequire 'disc/testing'\n\nrequire_relative 'examples/returner'\nDisc.enqueue! #=> This is the default mode for disc/testing so you don't need to specify it,\n              #   you can use this method to go back to the enqueue mode if you switch it.\n\n\nReturner.enqueue('test argument')\n\nDisc.queues\n#=> {\"default\"=>[{:arguments=>[\"test argument\"], :class=>\"Returner\", :options=>{}}]}\n\nReturner.enqueue('another test')\n#=> => {\"default\"=>[{:arguments=>[\"test argument\"], :class=>\"Returner\", :options=>{}}, {:arguments=>[\"another test\"], :class=>\"Returner\", :options=>{}}]}\n```\n\nYou can still flush the queues just as you would running on regular mode.\n\n```ruby\nDisc.flush\n\nDisc.queues\n#=> {}\n```\n\n### Inline mode\n\nYou also have the option for Disc to execute jobs immediately when `#enqueue` is called.\n\n```ruby\nrequire 'disc'\nrequire 'disc/testing'\n\nrequire_relative 'examples/returner'\nDisc.inline!\n\nReturner.enqueue('test argument')\n#=> 'test argument'\n```\n\n## [Optional] Celluloid integration\n\nDisc workers run just fine on their own, but if you happen to be using\n[Celluloid](https://github.com/celluloid/celluloid) you might want Disc to take\nadvantage of it and spawn multiple worker threads per process, doing this is\ntrivial! Just require Celluloid before your init file:\n\n```bash\n$ QUEUES=urgent,default disc -r celluloid/current -r ./disc_init.rb\n```\n\nWhenever Disc detects that Celluloid is available it will use it to  spawn a\nnumber of threads equal to the `DISC_CONCURRENCY` environment variable, or 25 by\ndefault.\n\n## [Optional] Rails and ActiveJob integration\n\nYou can use Disc easily in Rails without any more hassle, but if you'd like to use it via [ActiveJob](http://edgeguides.rubyonrails.org/active_job_basics.html) you can use the adapter included in this gem.\n\n```ruby\n# Gemfile\ngem 'disc'\n\n# config/application.rb\nmodule YourApp\n  class Application < Rails::Application\n    require 'active_job/queue_adapters/disc_adapter'\n    config.active_job.queue_adapter = :disc\n  end\nend\n\n# app/jobs/clu_job.rb\n\nclass CluJob < ActiveJob::Base\n  queue_as :urgent\n\n  def perform(*args)\n    # Try to take over The Grid here...\n  end\nend\n\n# disc_init.rb\nrequire ::File.expand_path('../config/environment', __FILE__)\n\n# Wherever you want\nCluJob.perform_later(a_bunch_of_arguments)\n```\n\nDisc is run in the exact same way, for this example it'd be:\n\n```bash\n$ QUEUES=urgent disc -r ./disc_init.rb\n```\n\n## Similar Projects\n\nIf you want to use Disque but Disc isn't cutting it for you then you should take a look at [Havanna](https://github.com/djanowski/havanna), a project by my friend [@djanowski](https://twitter.com/djanowski).\n\n## License\n\nThe code is released under an MIT license. See the [LICENSE](./LICENSE) file for\nmore information.\n\n## Acknowledgements\n\n* To [@foca](https://github.com/foca) for helping me ship a quality thing and putting up with my constant whining.\n* To [@antirez](https://github.com/antirez) for Redis, Disque, and his refreshing way of programming wonderful tools.\n* To [@soveran](https://github.com/soveran) for pushing me to work on this and publishing gems that keep me enjoying ruby.\n* To [all contributors](https://github.com/pote/disc/graphs/contributors)\n\n## Sponsorship\n\nThis open source tool is proudly sponsored by [13Floor](http://13Floor.org)\n\n![13Floor](./13Floor-circulo-1.png)\n"
  },
  {
    "path": "bin/disc",
    "content": "#!/usr/bin/env ruby\n\nif ARGV.empty?\n  $stdout.puts('Usage: disc -r FILE')\n\n  exit(1)\nend\n\nstop = proc do\n  if defined?(Disc)\n    Disc::Worker.stop\n  else\n    exit 0\n  end\nend\n\ntrap(:INT,  &stop)\ntrap(:TERM, &stop)\n\nrequire 'clap'\nrequire_relative '../lib/disc'\nrequire_relative '../lib/disc/worker'\n\nClap.run ARGV,\n  \"-r\" => lambda { |file| require file }\n\nif defined?(Celluloid)\n  $stdout.puts(\n    \"[Notice] Disc running in celluloid mode! Current DISC_CONCURRENCY is\\\n #{ Integer(ENV.fetch('DISC_CONCURRENCY', '25')) }.\"\n  )\n\n  Disc::Worker.send(:include, Celluloid)\n\n  if defined?(Celluloid::SupervisionGroup)\n    # Deprecated as of Celluloid 0.17, but still supported via \"backported mode\"\n    class Disc::WorkerGroup < Celluloid::SupervisionGroup\n      pool Disc::Worker,\n            size: Integer(ENV.fetch('DISC_CONCURRENCY', '25')),\n            as: :worker_pool,\n            args: [{ run: true }]\n    end\n\n    Disc::WorkerGroup.run\n  else\n    Disc::Worker.pool(\n      size: Integer(ENV.fetch('DISC_CONCURRENCY', '25')),\n      args: [{ run: true }]\n    )\n  end\nelse\n  $stdout.puts(\"[Notice] Disc running in non-threaded mode\")\n  Disc::Worker.run\nend\n\n"
  },
  {
    "path": "disc.gemspec",
    "content": "require_relative \"lib/disc/version\"\n\nGem::Specification.new do |s|\n  s.name        = 'disc'\n  s.version     = Disc::VERSION\n  s.summary     = 'A simple and powerful Disque job implementation'\n  s.description = 'Easily define and run background jobs using Disque'\n  s.authors     = ['pote']\n  s.email       = ['pote@tardis.com.uy']\n  s.homepage    = 'https://github.com/pote/disc'\n  s.license     = 'MIT'\n  s.files       = `git ls-files`.split(\"\\n\")\n\n  s.executables.push('disc')\n\n  s.add_dependency('disque', '~> 0.0.6')\n  s.add_dependency('clap', '~> 1.0')\nend\n"
  },
  {
    "path": "examples/disc_init.rb",
    "content": "$:.unshift('lib')\nDir.glob(\"./examples/**/*.rb\") { |f| require f }\n"
  },
  {
    "path": "examples/echoer.rb",
    "content": "require 'disc'\n\nclass Echoer\n  include Disc::Job\n  disc queue: 'test'\n\n  def perform(first, second, third)\n    puts \"First: #{ first }, Second: #{ second }, Third: #{ third }\"\n  end\nend\n"
  },
  {
    "path": "examples/failer.rb",
    "content": "require 'disc'\n\ndef Disc.on_error(exception, job)\n  $stdout.puts('<insert error reporting here>')\n  $stdout.puts(exception.message)\n  $stdout.puts(job)\nend\n\nclass Failer\n  include Disc::Job\n  disc queue: 'test'\n\n  def perform(string)\n    raise string\n  end\nend\n"
  },
  {
    "path": "examples/greeter.rb",
    "content": "require 'disc'\n\nclass Greeter\n  include Disc::Job\n  disc queue: 'test_medium'\n\n  def perform(string)\n    $stdout.puts(string)\n  end\nend\n"
  },
  {
    "path": "examples/identifier.rb",
    "content": "require 'disc'\n\nclass Identifier\n  include Disc::Job\n  disc queue: 'test'\n\n  def perform\n    $stdout.puts(\"Working with Disque ID: #{ self.disque_id }\")\n  end\nend\n"
  },
  {
    "path": "examples/returner.rb",
    "content": "class Returner\n  include Disc::Job\n\n  def perform(argument)\n    return argument\n  end\nend\n"
  },
  {
    "path": "lib/active_job/queue_adapters/disc_adapter.rb",
    "content": "require 'date'\nrequire 'msgpack'\nrequire 'disc/worker'\n\nmodule ActiveJob\n  module QueueAdapters\n    class DiscAdapter\n      def self.enqueue(job)\n        enqueue_at(job, nil)\n      end\n\n      def self.enqueue_at(job, timestamp)\n        Disc.disque.push(\n          job.queue_name,\n          Disc.serialize({\n            class: job.class.name,\n            arguments: job.arguments\n          }),\n          Disc.disque_timeout,\n          delay: timestamp.nil? ? nil : (timestamp.to_time.to_i - DateTime.now.to_time.to_i)\n        )\n      end\n    end\n  end\nend\n"
  },
  {
    "path": "lib/disc/errors.rb",
    "content": "class Disc\n  class Error < StandardError; end\n\n  class UnknownJobClassError  < Error; end\n  class NonParsableJobError   < Error; end\n  class NonJobClassError      < Error; end\nend\n"
  },
  {
    "path": "lib/disc/job.rb",
    "content": "module Disc::Job\n  attr_accessor :disque_id\n\n  def self.included(base)\n    base.extend(ClassMethods)\n  end\n\n  module ClassMethods\n    def disque\n      defined?(@disque) ? @disque : Disc.disque\n    end\n\n    def disque=(disque)\n      @disque = disque\n    end\n\n    def disc(queue: nil, **options)\n      @queue = queue\n      @disc_options = options\n    end\n\n    def disc_options\n      @disc_options ||= {}\n    end\n\n    def queue\n      @queue || Disc.default_queue\n    end\n\n    def perform(arguments)\n      self.new.perform(*arguments)\n    end\n\n    ## Disc's `#enqueue` is the main user-facing method of a Disc job, it\n    #  enqueues a job with a given set of arguments in Disque, so it can be\n    #  picked up by a Disc worker process.\n    #\n    ## Parameters:\n    #\n    ## `arguments`  - an optional array of arguments with which to execute\n    #                 the job's #perform method.\n    #\n    ## `at`         - an optional named parameter specifying a moment in the\n    #                 future in which to run the job, must respond to\n    #                 `#to_time`.\n    #\n    ## `queue`      - an optional named parameter specifying the name of the\n    #                 queue in which to store the job, defaults to the class\n    #                 Disc queue or to 'default' if no Disc queue is specified\n    #                 in the class.\n    #\n    ##  `**options` - an optional hash of options to forward internally to\n    #                 [disque-rb](https://github.com/soveran/disque-rb)'s\n    #                 `#push` method, valid options are:\n    #\n    ##  `replicate: <count>`  - specifies the number of nodes the job should\n    #                           be replicated to.\n    #\n    ### `delay: <sec>`        - specifies a delay time in seconds for the job\n    #                           to be delivered to a Disc worker, it is ignored\n    #                           if using the `at` parameter.\n    #\n    ### `ttl: <sec>`          - specifies the job's time to live in seconds:\n    #                           after this time, the job is deleted even if\n    #                           it was not successfully delivered. If not\n    #                           specified, the default TTL is one day.\n    #\n    ### `maxlen: <count>`     - specifies that if there are already <count>\n    #                           messages queued for the specified queue name,\n    #                           the message is refused.\n    #\n    ### `async: true`         - asks the server to let the command return ASAP\n    #                           and replicate the job to other nodes in the background.\n    #\n    #\n    ### CAVEATS\n    #\n    ## For convenience, any object can be passed as the `arguments` parameter,\n    #  `Array()` will be used internally to preserve the array structure.\n    #\n    ## The `arguments` parameter is serialized for storage using `Disc.serialize`\n    #  and Disc workers picking it up use `Disc.deserialize` on it, both methods\n    #  use standard library json but can be overriden by the user\n    #\n    def enqueue(args = [], at: nil, queue: nil, **options)\n      options = disc_options.merge(options).tap do |opt|\n        opt[:delay] = at.to_time.to_i - DateTime.now.to_time.to_i unless at.nil?\n      end\n\n      disque.push(\n        queue || self.queue,\n        Disc.serialize({\n          class: self.name,\n          arguments: Array(args)\n        }),\n        Disc.disque_timeout,\n        options\n      )\n    end\n  end\nend\n"
  },
  {
    "path": "lib/disc/testing.rb",
    "content": "class Disc\n  def self.queues\n    @queues ||= {}\n  end\n\n  def self.testing_mode\n    @testing_mode ||= 'enqueue'\n  end\n\n  def self.enqueue!\n    @testing_mode = 'enqueue'\n  end\n\n  def self.inline!\n    @testing_mode = 'inline'\n  end\n\n  def self.flush\n    @queues = {}\n  end\n\n  def self.qlen(queue)\n    return 0 if Disc.queues[queue].nil?\n\n    Disc.queues[queue].length\n  end\n\n  def self.enqueue(klass,  arguments, at: nil, queue: nil, **options)\n    job_attrs = { arguments: arguments, class: klass, options: options }\n    if queues[queue].nil?\n      queues[queue] = [job_attrs]\n    else\n      queues[queue] << job_attrs\n    end\n\n    job_attrs\n  end\nend\n\nmodule Disc::Job::ClassMethods\n  def enqueue(args = [], at: nil, queue: nil, **options)\n    case Disc.testing_mode\n    when 'enqueue'\n      Disc.enqueue(\n        self.name,\n        Array(args),\n        queue: queue || self.queue,\n        at: at,\n        **options)\n    when 'inline'\n      self.perform(*args)\n    else\n      raise \"Unknown Disc testing mode, this shouldn't happen\"\n    end\n  end\nend\n"
  },
  {
    "path": "lib/disc/version.rb",
    "content": "class Disc\n  VERSION = \"0.2.0\"\nend\n"
  },
  {
    "path": "lib/disc/worker.rb",
    "content": "require 'disc'\n\n\nclass Disc::Worker\n  attr_reader :disque,\n              :queues,\n              :timeout,\n              :count\n\n  def self.current\n    @current ||= new\n  end\n\n  def self.run\n    current.run\n  end\n\n  def self.stop\n    current.stop\n  end\n\n  def initialize(options = {})\n    @disque = options.fetch(:disque, Disc.disque)\n    @queues = options.fetch(\n      :queues,\n      ENV.fetch('QUEUES', Disc.default_queue)\n    ).split(',')\n    @count = Integer(\n      options.fetch(\n        :count,\n        ENV.fetch('DISQUE_COUNT', '1')\n      )\n    )\n    @timeout = Integer(\n      options.fetch(\n        :timeout,\n        ENV.fetch('DISQUE_TIMEOUT', '2000')\n      )\n    )\n\n    self.run if options[:run]\n    self\n  end\n\n  def stop\n    @stop = true\n  end\n\n  def run\n    $stdout.puts(\"Disc::Worker listening in #{queues}\")\n    loop do\n      jobs = disque.fetch(from: queues, timeout: timeout, count: count)\n      Array(jobs).each do |queue, msgid, serialized_job|\n        begin\n          job_instance, arguments = Disc.load_job(serialized_job, msgid)\n          job_instance.perform(*arguments)\n          disque.call('ACKJOB', msgid)\n          $stdout.puts(\"Completed #{ job_instance.class.name } id #{ msgid }\")\n        rescue => err\n          Disc.on_error(err, {\n            disque_id: msgid,\n            queue: queue,\n            class: defined?(job_instance) ? job_instance.class.name : '',\n            arguments: defined?(arguments) ? arguments : []\n          })\n        end\n      end\n\n      break if @stop\n    end\n  ensure\n    disque.quit\n  end\nend\n"
  },
  {
    "path": "lib/disc.rb",
    "content": "require 'date'\nrequire 'disque'\nrequire 'json'\n\nclass Disc\n  def self.disque\n    @disque ||= Disque.new(\n      ENV.fetch('DISQUE_NODES', 'localhost:7711'),\n      auth: ENV.fetch('DISQUE_AUTH', nil),\n      cycle: Integer(ENV.fetch('DISQUE_CYCLE', '1000'))\n    )\n  end\n\n  def self.disque=(disque)\n    @disque = disque\n  end\n\n  def self.disque_timeout\n    @disque_timeout ||= 100\n  end\n\n  def self.disque_timeout=(timeout)\n    @disque_timeout = timeout\n  end\n\n  def self.default_queue\n    @default_queue ||= 'default'\n  end\n\n  def self.default_queue=(queue)\n    @default_queue = queue\n  end\n\n  def self.qlen(queue)\n    disque.call('QLEN', queue)\n  end\n\n  def self.flush\n    Disc.disque.call('DEBUG', 'FLUSHALL')\n  end\n\n  def self.on_error(exception, job)\n    $stderr.puts exception\n  end\n\n  def self.serialize(args)\n    JSON.dump(args)\n  end\n\n  def self.deserialize(data)\n    JSON.parse(data)\n  end\n\n  def self.[](disque_id)\n    job_data = disque.call(\"SHOW\", disque_id)\n    return nil if job_data.nil?\n\n    job_data = Hash[*job_data]\n    job_data['arguments'] = Disc.deserialize(job_data['body'])['arguments']\n    job_data['class'] = Disc.deserialize(job_data['body'])['class']\n\n    job_data\n  end\n\n  ## Receives:\n  #\n  #   A string containing data serialized by `Disc.serialize`\n  #\n  ## Returns:\n  #\n  #   An array containing:\n  #\n  #     * An instance of the given job class\n  #     * An array of arguments to pass to the job's `#perorm` class.\n  #\n  def self.load_job(serialized_job, disque_id = nil)\n    begin\n      job_data = Disc.deserialize(serialized_job)\n    rescue => err\n      raise Disc::NonParsableJobError.new(err)\n    end\n\n    begin\n      job_class = Object.const_get(job_data['class'])\n    rescue => err\n      raise Disc::UnknownJobClassError.new(err)\n    end\n\n    begin\n      job_instance = job_class.new\n      job_instance.disque_id = disque_id\n    rescue => err\n      raise Disc::NonJobClassError.new(err)\n    end\n\n    return [job_instance, job_data['arguments']]\n  end\nend\n\nrequire_relative 'disc/errors'\nrequire_relative 'disc/job'\nrequire_relative 'disc/version'\n"
  },
  {
    "path": "test/disc_test.rb",
    "content": "require 'cutest'\nrequire 'disc'\n\nrequire_relative '../examples/echoer'\n\nprepare do\n  Disc.disque_timeout = 1 # 1ms so we don't wait at all.\n  Disc.flush\nend\n\nscope do\n  test 'Disc should be able to communicate with Disque' do\n    assert !Disc.disque.nil?\n\n    assert_equal 'PONG', Disc.disque.call('PING')\n  end\n\n  test 'we get easy access to the job via the job id with Disc[job_id]' do\n    job_id = Echoer.enqueue(['one argument', { random: 'data' }, 3])\n\n    job_data = Disc[job_id]\n\n    assert_equal 'Echoer', job_data['class']\n    assert_equal 'queued', job_data['state']\n    assert_equal 3, job_data['arguments'].count\n    assert_equal 'one argument', job_data['arguments'].first\n  end\n\n  test 'we can query the length of a given queue with Disc.qlen' do\n    Echoer.enqueue(['one argument', { random: 'data' }, 3])\n\n    assert_equal 1, Disc.qlen(Echoer.queue)\n  end\n\n  test 'Disc.flush deletes everything in the queue' do\n    Echoer.enqueue(['one argument', { random: 'data' }, 3])\n    Disc.flush\n\n    assert_equal 0, Disc.qlen(Echoer.queue)\n  end\n\n  test 'Disc.load_job returns a job instance and arguments' do\n    serialized_job = Disc.serialize(\n      { class: 'Echoer', arguments: ['one argument', { random: 'data' }, 3] }\n    )\n\n    job_instance, arguments = Disc.load_job(serialized_job)\n\n    assert_equal Echoer, job_instance.class\n    assert arguments.is_a?(Array)\n    assert_equal 3, arguments.count\n    assert_equal 'one argument', arguments.first\n  end\n\n  test 'Disc.load_job raises appropriate errors ' do\n    begin\n      job_instance, arguments = Disc.load_job('gibberish')\n      assert_equal 'Should not reach this point', false\n    rescue => err\n      assert err.is_a?(Disc::Error)\n      assert err.is_a?(Disc::NonParsableJobError)\n    end\n\n    serialized_job = Disc.serialize(\n      { class: 'NonExistantDiscJobClass', arguments: [] }\n    )\n    begin\n      job_instance, arguments = Disc.load_job(serialized_job)\n      assert_equal 'Should not reach this point', false\n    rescue => err\n      assert err.is_a?(Disc::Error)\n      assert err.is_a?(Disc::UnknownJobClassError)\n    end\n  end\nend\n"
  },
  {
    "path": "test/job_test.rb",
    "content": "require 'cutest'\nrequire 'disc'\n\nrequire_relative '../examples/echoer'\n\nprepare do\n  Disc.disque_timeout = 1 # 1ms so we don't wait at all.\n  Disc.flush\nend\n\nscope do\n  test 'jobs are enqueued to the correct Disque queue with appropriate parameters and class' do\n    job_id = Echoer.enqueue(['one argument', { random: 'data' }, 3])\n\n    jobs = Array(Disc.disque.fetch(from: ['test'], timeout: Disc.disque_timeout, count: 1))\n    assert jobs.any?\n    assert_equal 1, jobs.count\n\n    jobs.first.tap do |queue, id, serialized_job|\n      job = Disc.deserialize(serialized_job)\n\n      assert job.has_key?('class')\n      assert job.has_key?('arguments')\n\n      assert_equal 'Echoer', job['class']\n      assert_equal job_id, id\n\n      args = job['arguments']\n      assert_equal 3, args.size\n      assert_equal 'one argument', args[0]\n      assert_equal({ 'random' => 'data' }, args[1])\n      assert_equal(3, args[2])\n    end\n  end\n\n  test 'enqueue at timestamp behaves properly' do\n    job_id = Echoer.enqueue(['one argument', { random: 'data' }, 3], at: Time.now + 1)\n\n    jobs = Array(Disc.disque.fetch(from: ['test'], timeout: Disc.disque_timeout, count: 1))\n    assert jobs.empty?\n\n    sleep 0.5\n    jobs = Array(Disc.disque.fetch(from: ['test'], timeout: Disc.disque_timeout, count: 1))\n    assert jobs.empty?\n\n    sleep 0.5\n    jobs = Array(Disc.disque.fetch(from: ['test'], timeout: Disc.disque_timeout, count: 1))\n    assert jobs.any?\n    assert_equal 1, jobs.size\n\n    jobs.first.tap do |queue, id, serialized_job|\n      assert_equal 'test', queue\n      assert_equal job_id, id\n      job = Disc.deserialize(serialized_job)\n      assert job.has_key?('class')\n      assert job.has_key?('arguments')\n      assert_equal 'Echoer', job['class']\n      assert_equal 3, job['arguments'].size\n    end\n  end\n\n  test 'enqueue supports replicate' do\n    error = Echoer.enqueue(['one argument', { random: 'data' }, 3], replicate: 100) rescue $!\n\n    assert_equal RuntimeError, error.class\n    assert_equal \"NOREPL Not enough reachable nodes for the requested replication level\", error.message\n  end\n\n  test 'enqueue supports delay' do\n    job_instance = Echoer.enqueue(['one argument', { random: 'data' }, 3], delay: 2)\n\n    jobs = Array(Disc.disque.fetch(from: ['test'], timeout: Disc.disque_timeout, count: 1))\n    assert jobs.empty?\n\n    sleep 1\n    jobs = Array(Disc.disque.fetch(from: ['test'], timeout: Disc.disque_timeout, count: 1))\n    assert jobs.empty?\n\n    sleep 2\n    jobs = Array(Disc.disque.fetch(from: ['test'], timeout: Disc.disque_timeout, count: 1))\n    assert jobs.any?\n    assert_equal 1, jobs.size\n  end\n\n  test 'enqueue supports retry' do\n    job_instance = Echoer.enqueue(['one argument', { random: 'data' }, 3], retry: 1)\n\n    jobs = Array(Disc.disque.fetch(from: ['test'], timeout: Disc.disque_timeout, count: 1))\n    assert jobs.any?\n    assert_equal 1, jobs.size\n\n    sleep 1.5\n    jobs = Array(Disc.disque.fetch(from: ['test'], timeout: Disc.disque_timeout, count: 1))\n    assert jobs.any?\n    assert_equal 1, jobs.size\n  end\n\n  test 'enqueue supports ttl' do\n    job_instance = Echoer.enqueue(['one argument', { random: 'data' }, 3], ttl: 1)\n\n    sleep 1.5\n    jobs = Array(Disc.disque.fetch(from: ['test'], timeout: Disc.disque_timeout, count: 1))\n    assert jobs.empty?\n  end\n\n  test 'enqueue supports maxlen' do\n    Echoer.enqueue(['one argument', { random: 'data' }, 3], maxlen: 1)\n    error = Echoer.enqueue(['one argument', { random: 'data' }, 3], maxlen: 1) rescue $!\n\n    assert_equal RuntimeError, error.class\n    assert_equal \"MAXLEN Queue is already longer than the specified MAXLEN count\", error.message\n  end\n\n  test 'enqueue supports async' do\n    job_instance = Echoer.enqueue(['one argument', { random: 'data' }, 3], async: true)\n\n    sleep 1 # async is too fast to reliably assert an empty queue, let's wait instead\n    jobs = Array(Disc.disque.fetch(from: ['test'], timeout: Disc.disque_timeout, count: 1))\n    assert jobs.any?\n    assert_equal 1, jobs.size\n  end\nend\n"
  },
  {
    "path": "test/process_test.rb",
    "content": "require 'cutest'\nrequire 'disc'\nrequire 'pty'\nrequire 'timeout'\n\nrequire_relative '../examples/echoer'\nrequire_relative '../examples/failer'\nrequire_relative '../examples/identifier'\n\nprepare do\n  Disc.disque_timeout = 1 # 1ms so we don't wait at all.\n  Disc.flush\nend\n\nscope do\n  # Runs a given command, yielding the stdout (as an IO) and the PID (a String).\n  # Makes sure the process finishes after the block runs.\n  def run(command)\n    out, _, pid = PTY.spawn(command)\n    yield out, pid\n  ensure\n    Process.kill(\"KILL\", pid)\n    sleep 0.1 # Make sure we give it time to finish.\n  end\n\n  # Checks whether a process is running.\n  def is_running?(pid)\n    Process.getpgid(pid)\n    true\n  rescue Errno::ESRCH\n    false\n  end\n\n  test 'Disc.on_error will catch unhandled exceptions and keep disc alive' do\n    failer = Failer.enqueue('this can only end positively')\n\n    run('QUEUES=test ruby -Ilib bin/disc -r ./examples/failer') do |cout, pid|\n      output = Timeout.timeout(1) { cout.take(5) }\n      assert output.grep(/<insert error reporting here>/).any?\n      assert output.grep(/this can only end positively/).any?\n      assert output.grep(/Failer/).any?\n\n      assert is_running?(pid)\n      assert_equal 0, Disc.qlen(Failer.queue)\n    end\n  end\n\n  test 'jobs are executed' do\n    Echoer.enqueue(['one argument', { random: 'data' }, 3])\n\n    run('QUEUES=test ruby -Ilib bin/disc -r ./examples/echoer') do |cout, pid|\n      output = Timeout.timeout(1) { cout.take(3) }\n      assert output.grep(/First: one argument, Second: {\"random\"=>\"data\"}, Third: 3/).any?\n      assert_equal 0, Disc.qlen(Echoer.queue)\n    end\n  end\n\n  test 'running jobs have access to their Disque job ID' do\n    disque_id = Identifier.enqueue\n\n    run('QUEUES=test ruby -Ilib bin/disc -r ./examples/identifier') do |cout, pid|\n      output = Timeout.timeout(1) { cout.take(3) }\n      assert output.grep(/Working with Disque ID: #{ disque_id }/).any?\n\n      assert_equal 0, Disc.qlen(Identifier.queue)\n    end\n  end\nend\n"
  },
  {
    "path": "test/testing_test.rb",
    "content": "# Yo dawg I put some testing in your testing so you can test while you test.\n\nrequire 'disc'\nrequire 'disc/testing'\n\nrequire_relative '../examples/echoer'\nrequire_relative '../examples/returner'\n\nprepare do\n  Disc.disque_timeout = 1 # 1ms so we don't wait at all.\n  Disc.enqueue!\n  Disc.flush\nend\n\nscope do\n  test \"testing mode should not enqueue jobs into Disque\" do\n    Echoer.enqueue(['one argument', { random: 'data' }, 3])\n    assert_equal 0, Disc.disque.call('QLEN', 'test')\n    assert_equal 1, Disc.qlen('test')\n\n    # Flush should still work though\n    Disc.flush\n    assert_equal 0, Disc.qlen('test')\n  end\n\n  test \"simple enqueuing should work \" do\n    Echoer.enqueue('one thing')\n\n    assert_equal 1, Disc.queues['test'].count\n    assert Disc.queues['test'].first.has_key?(:arguments)\n    assert_equal 1, Disc.queues['test'].first[:arguments].count\n    assert_equal 'one thing', Disc.queues['test'].first[:arguments].first\n  end\n\n  test \"testing mode enqueue jobs into an in-memory list by default\" do\n    Echoer.enqueue(['one argument', { random: 'data' }, 3])\n\n    assert_equal 1, Disc.queues['test'].count\n    assert Disc.queues['test'].first.has_key?(:arguments)\n    assert_equal 3, Disc.queues['test'].first[:arguments].count\n    assert_equal 'one argument', Disc.queues['test'].first[:arguments].first\n    assert_equal 'Echoer', Disc.queues['test'].first[:class]\n  end\n\n  test \"testing mode enqueue jobs into an in-memory list by default\" do\n    Echoer.enqueue(['one argument', { random: 'data' }, 3])\n    assert_equal 'one argument', Disc.queues['test'].first[:arguments].first\n  end\n\n  test \"ability to run jobs inline\" do\n    Disc.inline!\n    assert_equal 'this is an argument',  Returner.enqueue('this is an argument')\n  end\nend\n"
  }
]