## Project Status
Barbeque is used on production at Cookpad.
## What's Barbeque?
Barbeque is a job queue system that consists of:
- Web console to manage jobs
- Web API to queue a job
- Worker to execute a job
A job for Barbeque is a command you configured on web console.
A message serialized by JSON and a job name are given to the command when performed.
In Barbeque worker, they are done on Docker container.
## Why Barbeque?
- You can achieve job-level auto scaling using tools like [Amazon ECS](https://aws.amazon.com/ecs/) [EC2 Auto Scaling group](https://aws.amazon.com/autoscaling/)
- For Amazon ECS, Barbeque has Hako executor
- You don't have to manage infrastructure for each application like Resque or Sidekiq
For details, see [Scalable Job Queue System Built with Docker // Speaker Deck](https://speakerdeck.com/k0kubun/scalable-job-queue-system-built-with-docker).
## Deployment
### Web API & console
Install barbeque.gem to an empty Rails app and mount `Barbeque::Engine`.
And deploy it as you like.
You also need to prepare MySQL, Amazon SQS and Amazon S3.
#### For sandbox environment
Barbeque's enqueue API tries to be independent of MySQL by design.
Although that design policy, verifying the enqueued job is useful in some environment (such as sandboxed environment).
Passing `BARBEQUE_VERIFY_ENQUEUED_JOBS=1` to the Web API server enables the feature that verifies the enqueued job by accessing MySQL.
### Worker
```bash
$ rake barbeque:worker BARBEQUE_QUEUE=default
```
The rake task launches four worker processes.
- Two runners
- receives message from SQS queue, starts job execution and stores its identifier to the database
- One execution poller
- gets execution status and reflect it to the database
- One retry poller
- gets retried execution status and reflect it to the database
## Usage
Web API documentation is available at [doc/toc.md](./doc/toc.md).
### Ruby
[barbeque\_client.gem](https://github.com/cookpad/barbeque_client) has API client and ActiveJob integration.
## Executor
Barbeque executor can be customized in config/barbeque.yml. Executor is responsible for starting executions and getting status of executions.
Barbeque has currently two executors.
### Docker (default)
Barbeque::Executor::Docker starts execution by `docker run --detach` and gets status by `docker inspect`.
### Hako
Barbeque::Executor::Hako starts execution by `hako oneshot --no-wait` and gets status from S3 task notification.
#### Requirement
You must configure CloudWatch Events for putting S3 task notification.
See Hako's documentation for detail.
https://github.com/eagletmt/hako/blob/master/docs/ecs-task-notification.md
## License
The gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).
================================================
FILE: Rakefile
================================================
begin
require 'bundler/setup'
rescue LoadError
puts 'You must `gem install bundler` and `bundle install` to run rake tasks'
end
require 'rdoc/task'
RDoc::Task.new(:rdoc) do |rdoc|
rdoc.rdoc_dir = 'rdoc'
rdoc.title = 'Barbeque'
rdoc.options << '--line-numbers'
rdoc.rdoc_files.include('README.md')
rdoc.rdoc_files.include('lib/**/*.rb')
end
require 'bundler/gem_tasks'
require File.expand_path('../spec/dummy/config/application', __FILE__)
namespace :plotly do
desc 'Update plotly.js to specified version'
task :update, [:version] do |t, args|
sh "curl -sSfL https://github.com/plotly/plotly.js/archive/v#{args[:version]}.tar.gz | tar zxf - plotly.js-#{args[:version]}/dist/plotly-basic.js -O > vendor/assets/javascripts/plotly-basic.js"
end
end
namespace :bootstrap do
desc 'Update Bootstrap to specified version'
task :update, [:version] do |t, args|
version = args.fetch(:version)
zipfile = "bootstrap-v#{version}.zip"
sh "curl -sSfL -o #{zipfile} https://github.com/twbs/bootstrap/releases/download/v#{version}/bootstrap-#{version}-dist.zip"
sh "bsdtar xf #{zipfile} --strip-components 2 -C vendor/assets/stylesheets bootstrap-#{version}-dist/css/bootstrap.css"
sh "bsdtar xf #{zipfile} --strip-components 2 -C vendor/assets/javascripts bootstrap-#{version}-dist/js/bootstrap.js"
end
end
namespace :adminlte do
desc 'Update AdminLTE to specified version'
task :update, [:version] do |t, args|
version = args.fetch(:version)
tarball = "adminlte-v#{version}.tar.gz"
sh "curl -sSfL -o #{tarball} https://github.com/ColorlibHQ/AdminLTE/archive/refs/tags/v#{version}.tar.gz"
sh "tar zxf #{tarball} --strip-components 3 -C vendor/assets/stylesheets AdminLTE-#{version}/dist/css/AdminLTE.css AdminLTE-#{version}/dist/css/skins/skin-blue.css"
sh "tar zxf #{tarball} --strip-components 3 -C vendor/assets/javascripts AdminLTE-#{version}/dist/js/adminlte.js"
end
end
Rails.application.load_tasks
================================================
FILE: app/assets/config/barbeque_manifest.js
================================================
//= link_directory ../javascripts/barbeque .js
//= link_directory ../stylesheets/barbeque .css
================================================
FILE: app/assets/images/barbeque/.keep
================================================
================================================
FILE: app/assets/javascripts/barbeque/application.js
================================================
// This is a manifest file that'll be compiled into application.js, which will include all the files
// listed below.
//
// Any JavaScript file within this directory, lib/assets/javascripts, vendor/assets/javascripts,
// or any plugin's vendor/assets/javascripts directory can be referenced here using a relative path.
//
// It's not advisable to add code directly here, but if you do, it'll appear at the bottom of the
// compiled file. JavaScript code in this file should be added after the last require_* statement.
//
// Read Sprockets README (https://github.com/rails/sprockets#sprockets-directives) for details
// about supported directives.
//
//= require jquery
//= require bootstrap
//= require jquery_ujs
//= require adminlte
//= require plotly-basic
//= require_tree .
================================================
FILE: app/assets/javascripts/barbeque/job_definitions.js
================================================
jQuery(function($) {
if (!document.querySelector('.barbeque_job_definitions_controller')) {
return;
}
$('.use_slack_notification').bind('change', function(event) {
const enabledField = $('.slack_notification_field');
if (event.target.value === 'true') {
enabledField.removeClass('active');
} else {
enabledField.addClass('active');
}
});
$('.enable_retry_configuration').bind('change', function(event) {
const enabledField = $('.retry_configuration_field');
if (event.target.value === 'true') {
enabledField.removeClass('active');
} else {
enabledField.addClass('active');
}
});
});
================================================
FILE: app/assets/javascripts/barbeque/job_queues.js
================================================
jQuery(function($) {
if (!document.querySelector('.barbeque_job_queues_controller')) {
return;
}
const sqsDiv = document.getElementById('sqs-attributes');
if (!sqsDiv) {
return;
}
const { url, metricsUrl } = sqsDiv.dataset;
$.getJSON(url).done(data => {
renderBox(sqsDiv, 'SQS queue metrics', metricsUrl, data);
if (data.dlq) {
const dlqDiv = document.getElementById('sqs-dlq-attributes');
renderBox(dlqDiv, 'SQS dead-letter queue metrics', metricsUrl, data.dlq);
}
}).fail(jqxhr => {
const errorMessage = document.createElement('div');
errorMessage.classList.add('alert');
errorMessage.classList.add('alert-danger');
errorMessage.appendChild(document.createTextNode(`Server returned ${jqxhr.status}: ${jqxhr.statusText}`));
const indicator = sqsDiv.querySelector('.loading-indicator');
if (indicator) {
indicator.parentNode.removeChild(indicator);
}
sqsDiv.appendChild(errorMessage);
});
});
const renderBox = (div, title, metricsUrl, data) => {
const box = document.createElement('div');
box.classList.add('box');
const boxHeader = document.createElement('div');
boxHeader.classList.add('box-header');
const boxTitle = document.createElement('h3');
boxTitle.classList.add('box-title');
boxTitle.classList.add('with_padding');
boxTitle.appendChild(document.createTextNode(title));
boxHeader.appendChild(boxTitle);
const boxBody = document.createElement('div');
boxBody.classList.add('box-body');
box.appendChild(boxHeader);
box.appendChild(boxBody);
const table = document.createElement('table');
table.classList.add('table');
table.classList.add('table-bordered');
const thead = document.createElement('thead');
const theadTr = document.createElement('tr');
thead.appendChild(theadTr);
const tbody = document.createElement('tbody');
const tbodyTr = document.createElement('tr');
tbody.appendChild(tbodyTr);
for (const [name, value] of Object.entries(data.attributes)) {
const th = document.createElement('th');
th.appendChild(document.createTextNode(name));
theadTr.appendChild(th);
const td = document.createElement('td');
td.appendChild(document.createTextNode(value));
tbodyTr.appendChild(td);
}
table.appendChild(thead);
table.appendChild(tbody);
boxBody.appendChild(table);
const indicator = div.querySelector('.loading-indicator');
if (indicator) {
indicator.parentNode.removeChild(indicator);
}
div.appendChild(box);
const metrics = {
NumberOfMessagesSent: 'Sum',
ApproximateNumberOfMessagesVisible: 'Sum',
ApproximateNumberOfMessagesNotVisible: 'Sum',
ApproximateAgeOfOldestMessage: 'Maximum',
};
const row = document.createElement('div');
row.classList.add('row');
boxBody.appendChild(row);
for (const [metricName, statistic] of Object.entries(metrics)) {
$.getJSON(`${metricsUrl}?queue_name=${data.queue_name}&metric_name=${metricName}&statistic=${statistic}`).done(data => {
renderChart(row, data);
}).fail(jqxhr => {
const errorMessage = document.createElement('div');
errorMessage.classList.add('alert');
errorMessage.classList.add('alert-danger');
errorMessage.appendChild(document.createTextNode(`Failed to load SQS metrics ${metricName}: ${jqxhr.status}: ${jqxhr.statusText}`));
return div.appendChild(errorMessage);
});
}
};
const renderChart = function(row, data) {
const div = document.createElement('div');
div.classList.add('col-md-3');
const chartDiv = document.createElement('div');
div.appendChild(chartDiv);
div.dataset.label = data.label;
// Insert charts ordered by label name
let inserted = false;
for (const child of row.children) {
if (data.label < child.dataset.label) {
row.insertBefore(div, child);
inserted = true;
break;
}
}
if (!inserted) {
row.appendChild(div);
}
return Plotly.plot(chartDiv, [{
type: 'scatter',
x: data.datapoints.map(point => point.timestamp),
y: data.datapoints.map(point => point.value),
}], {
title: data.label,
});
};
================================================
FILE: app/assets/stylesheets/barbeque/application.css
================================================
/*
*= require bootstrap
*= require AdminLTE
*= require skins/skin-blue
*= require barbeque/job_definitions
*/
:root {
--barbeque-border-color: #ececec;
}
.box-body {
padding: 12px;
}
.box-header .box-title.with_padding {
padding: 6px;
font-size: 22px;
}
.table.table-bordered {
margin-bottom: 12px;
border-color: var(--barbeque-border-color);
}
.table.table-bordered td, .table.table-bordered th {
border-color: var(--barbeque-border-color);
}
================================================
FILE: app/assets/stylesheets/barbeque/job_definitions.css
================================================
.barbeque_job_definitions_controller .use_slack_notification_wrapper label {
font-weight: normal;
}
.barbeque_job_definitions_controller .use_slack_notification {
margin: 0 2px 0 8px;
}
.barbeque_job_definitions_controller .slack_notification_field {
display: none;
}
.barbeque_job_definitions_controller .slack_notification_field label {
font-weight: normal;
}
.barbeque_job_definitions_controller .slack_notification_field.active {
display: block;
}
.barbeque_job_definitions_controller .retry_configuration_wrapper label {
font-weight: normal;
}
.barbeque_job_definitions_controller .enable_retry_configuration {
margin: 0 2px 0 8px;
}
.barbeque_job_definitions_controller .retry_configuration_field {
display: none;
}
.barbeque_job_definitions_controller .retry_configuration_field label {
font-weight: normal;
}
.barbeque_job_definitions_controller .retry_configuration_field.active {
display: block;
}
.barbeque_job_definitions_controller .table.table-bordered td,
.barbeque_job_definitions_controller .table.table-bordered th {
overflow-wrap: anywhere;
min-width: 100px;
}
================================================
FILE: app/controllers/barbeque/api/application_controller.rb
================================================
require 'garage'
class Barbeque::Api::ApplicationController < ActionController::API
before_action :force_json_format
include Garage::ControllerHelper
rescue_from ActiveRecord::RecordNotFound do |exception|
respond_with_error(404, 'record_not_found', exception.message)
end
rescue_from WeakParameters::ValidationError do |exception|
respond_with_error(400, 'invalid_parameter', exception.message)
end
private
# @param [Integer] status_code HTTP status code
# @param [String] error_code Must be unique
# @param [String] message Error message for API client, not for end user.
def respond_with_error(status_code, error_code, message)
render json: { status_code: status_code, error_code: error_code, message: message }, status: status_code
end
# This is required to use ActionController::API with Garage
def force_json_format
request.format = :json
end
end
================================================
FILE: app/controllers/barbeque/api/job_executions_controller.rb
================================================
require 'barbeque/maintenance'
class Barbeque::Api::JobExecutionsController < Barbeque::Api::ApplicationController
include Garage::RestfulActions
validates :create do
string :application, required: true, description: 'Application name of the job'
string :job, required: true, description: 'Class of Job to be enqueued'
string :queue, required: true, description: 'Queue name to enqueue a job'
any :message, required: true, description: 'Free-format JSON'
integer :delay_seconds, description: 'Set message timer of SQS'
end
rescue_from Barbeque::MessageEnqueuingService::BadRequest do |exc|
render status: 400, json: { error: exc.message }
end
private
def require_resources
protect_resource_as Barbeque::Api::JobExecutionResource
end
def require_resource
model = Barbeque::JobExecution.find_or_initialize_by(message_id: params[:message_id])
@resource = Barbeque::Api::JobExecutionResource.new(model)
rescue ActiveRecord::StatementInvalid, Mysql2::Error::ConnectionError => e
if Barbeque::Maintenance.database_maintenance_mode?
Barbeque::ExceptionHandler.handle_exception(e)
@resource = Barbeque::Api::DatabaseMaintenanceResource.new(e)
else
raise e
end
end
def create_resource
message_id = enqueue_message
model = Barbeque::JobExecution.new(message_id: message_id)
@resource = Barbeque::Api::JobExecutionResource.new(model)
end
# @return [String] id of a message queued to SQS.
def enqueue_message
Barbeque::MessageEnqueuingService.new(
application: params[:application],
job: params[:job],
queue: params[:queue],
message: params[:message],
delay_seconds: params[:delay_seconds],
).run
end
# Link to job_execution isn't available if it isn't dequeued yet
def location
if @resource.id
super
else
nil
end
end
def respond_with_resource_options
if @resource.is_a?(Barbeque::Api::DatabaseMaintenanceResource)
super.merge(status: 503)
else
super
end
end
end
================================================
FILE: app/controllers/barbeque/api/job_retries_controller.rb
================================================
class Barbeque::Api::JobRetriesController < Barbeque::Api::ApplicationController
include Garage::RestfulActions
# http://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_SendMessage.html
SQS_MAX_DELAY_SECONDS = 900
validates :create do
integer :delay_seconds, only: 0..SQS_MAX_DELAY_SECONDS
end
private
def require_resources
protect_resource_as Barbeque::Api::JobRetryResource
end
def create_resource
result = retry_message
Barbeque::JobRetry.new(message_id: result.message_id).to_resource
end
def retry_message
Barbeque::MessageRetryingService.new(
message_id: params[:job_execution_message_id],
delay_seconds: params[:delay_seconds].to_i,
).run
end
end
================================================
FILE: app/controllers/barbeque/api/revision_locks_controller.rb
================================================
class Barbeque::Api::RevisionLocksController < Barbeque::Api::ApplicationController
include Garage::RestfulActions
validates :create do
string :revision, required: true, description: 'Docker image revision to lock'
end
private
def require_resources
protect_resource_as Barbeque::Api::RevisionLockResource
end
def require_resource
@resource = Barbeque::App.find_by!(name: params[:app_name])
end
def create_resource
app = Barbeque::App.find_by!(name: params[:app_name])
image = Barbeque::DockerImage.new(app.docker_image)
image.tag = params[:revision]
app.update!(docker_image: image.to_s)
Barbeque::Api::RevisionLockResource.new(app)
end
def destroy_resource
image = Barbeque::DockerImage.new(@resource.docker_image)
image.tag = 'latest'
@resource.update!(docker_image: image.to_s)
Barbeque::Api::RevisionLockResource.new(@resource)
end
end
================================================
FILE: app/controllers/barbeque/application_controller.rb
================================================
module Barbeque
class ApplicationController < ActionController::Base
protect_from_forgery with: :exception
end
end
================================================
FILE: app/controllers/barbeque/apps_controller.rb
================================================
class Barbeque::AppsController < Barbeque::ApplicationController
def index
@apps = Barbeque::App.all
end
def show
@app = Barbeque::App.find(params[:id])
end
def new
@app = Barbeque::App.new
end
def edit
@app = Barbeque::App.find(params[:id])
end
def create
@app = Barbeque::App.new(params.require(:app).permit(:name, :docker_image, :description))
if @app.save
redirect_to @app, notice: 'App was successfully created.'
else
render :new
end
end
def update
@app = Barbeque::App.find(params[:id])
# Name can't be changed after it's created.
if @app.update(params.require(:app).permit(:docker_image, :description))
redirect_to @app, notice: 'App was successfully updated.'
else
render :edit
end
end
def destroy
@app = Barbeque::App.find(params[:id])
@app.destroy
redirect_to root_path, notice: 'App was successfully destroyed.'
end
end
================================================
FILE: app/controllers/barbeque/job_definitions_controller.rb
================================================
class Barbeque::JobDefinitionsController < Barbeque::ApplicationController
def index
@job_definitions = Barbeque::JobDefinition.all
end
def show
@job_definition = Barbeque::JobDefinition.find(params[:id])
@job_executions = @job_definition.job_executions.order(id: :desc).page(params[:page])
@retry_config = @job_definition.retry_config
@status = params[:status].presence.try(&:to_i)
if @status
@job_executions = @job_executions.where(status: @status)
end
end
def new
@job_definition = Barbeque::JobDefinition.new
@job_definition.build_slack_notification
@job_definition.build_retry_config
if params[:job_definition]
@job_definition.assign_attributes(new_job_definition_params)
end
end
def edit
@job_definition = Barbeque::JobDefinition.find(params[:id])
unless @job_definition.slack_notification
@job_definition.build_slack_notification
end
unless @job_definition.retry_config
@job_definition.build_retry_config
end
end
def create
attributes = new_job_definition_params.merge(command: command_array)
@job_definition = Barbeque::JobDefinition.new(attributes)
if @job_definition.save
redirect_to @job_definition, notice: 'Job definition was successfully created.'
else
render :new
end
end
def update
@job_definition = Barbeque::JobDefinition.find(params[:id])
attributes = params.require(:job_definition).permit(
:description,
slack_notification_attributes: slack_notification_params,
retry_config_attributes: retry_config_params,
).merge(command: command_array)
if @job_definition.update(attributes)
redirect_to @job_definition, notice: 'Job definition was successfully updated.'
else
render :edit
end
end
def destroy
@job_definition = Barbeque::JobDefinition.find(params[:id])
@job_definition.sns_subscriptions.each do |sns_subscription|
Barbeque::SnsSubscriptionService.new.unsubscribe(sns_subscription)
end
@job_definition.destroy
redirect_to job_definitions_url, notice: 'Job definition was successfully destroyed.'
end
def stats
@job_definition = Barbeque::JobDefinition.find(params[:job_definition_id])
@days = (params[:days] || 3).to_i
end
def execution_stats
job_definition = Barbeque::JobDefinition.find(params[:job_definition_id])
days = (params[:days] || 3).to_i
now = Time.zone.now
render json: job_definition.execution_stats(days.days.ago(now), now)
end
private
def slack_notification_params
%i[id channel notify_success notify_failure_only_if_retry_limit_reached failure_notification_text _destroy]
end
def retry_config_params
%i[id retry_limit base_delay max_delay jitter _destroy]
end
def command_array
Shellwords.split(params.require(:job_definition)[:command])
end
def new_job_definition_params
params.require(:job_definition).permit(
:job,
:app_id,
:description,
slack_notification_attributes: slack_notification_params,
retry_config_attributes: retry_config_params,
)
end
end
================================================
FILE: app/controllers/barbeque/job_executions_controller.rb
================================================
class Barbeque::JobExecutionsController < Barbeque::ApplicationController
ID_REGEXP = /\A[0-9]+\z/
def show
if ID_REGEXP === params[:message_id]
job_execution = Barbeque::JobExecution.find_by(id: params[:message_id])
if job_execution
redirect_to(job_execution)
return
end
end
@job_execution = Barbeque::JobExecution.find_by!(message_id: params[:message_id])
# Return 404 when job_definition or app is deleted
@job_definition = Barbeque::JobDefinition.find(@job_execution.job_definition_id)
@app = Barbeque::App.find(@job_definition.app_id)
@log = @job_execution.execution_log
@job_retries = @job_execution.job_retries.order(id: :desc)
end
def retry
@job_execution = Barbeque::JobExecution.find_by!(message_id: params[:job_execution_message_id])
raise ActionController::BadRequest unless @job_execution.retryable?
result = Barbeque::MessageRetryingService.new(message_id: @job_execution.message_id).run
@job_execution.retried!
redirect_to @job_execution, notice: "Succeed to retry (message_id=#{result.message_id})"
end
end
================================================
FILE: app/controllers/barbeque/job_queues_controller.rb
================================================
require 'aws-sdk-cloudwatch'
require 'aws-sdk-sqs'
require 'barbeque/config'
class Barbeque::JobQueuesController < Barbeque::ApplicationController
def index
@job_queues = Barbeque::JobQueue.all
end
def show
@job_queue = Barbeque::JobQueue.find(params[:id])
end
def new
@job_queue = Barbeque::JobQueue.new
end
def edit
@job_queue = Barbeque::JobQueue.find(params[:id])
end
def create
@job_queue = Barbeque::JobQueue.new(params.require(:job_queue).permit(:name, :description))
@job_queue.queue_url = create_queue(@job_queue).queue_url if @job_queue.valid?
if @job_queue.save
redirect_to @job_queue, notice: 'Job queue was successfully created.'
else
render :new
end
end
def update
@job_queue = Barbeque::JobQueue.find(params[:id])
# Name can't be changed after it's created.
if @job_queue.update(params.require(:job_queue).permit(:description))
redirect_to @job_queue, notice: 'Job queue was successfully updated.'
else
render :edit
end
end
def destroy
@job_queue = Barbeque::JobQueue.find(params[:id])
@job_queue.destroy
redirect_to job_queues_url, notice: 'Job queue was successfully destroyed.'
end
def sqs_attributes
job_queue = Barbeque::JobQueue.find(params[:id])
attributes = self.class.sqs_client.get_queue_attributes(
queue_url: job_queue.queue_url,
attribute_names: %w[
ApproximateNumberOfMessages
ApproximateNumberOfMessagesNotVisible
RedrivePolicy
QueueArn
],
).attributes
dlq_metrics =
if attributes['RedrivePolicy']
dlq_arn = JSON.parse(attributes['RedrivePolicy']).fetch('deadLetterTargetArn')
dlq_name = queue_name_from_arn(dlq_arn)
dlq_url = self.class.sqs_client.get_queue_url(queue_name: dlq_name).queue_url
dlq_attributes = self.class.sqs_client.get_queue_attributes(
queue_url: dlq_url,
attribute_names: %w[
ApproximateNumberOfMessages
ApproximateNumberOfMessagesNotVisible
],
).attributes.transform_values(&:to_i)
{
queue_name: dlq_name,
attributes: dlq_attributes,
}
else
nil
end
render json: {
queue_name: queue_name_from_arn(attributes['QueueArn']),
attributes: {
'ApproximateNumberOfMessages' => attributes['ApproximateNumberOfMessages'].to_i,
'ApproximateNumberOfMessagesNotVisible' => attributes['ApproximateNumberOfMessagesNotVisible'].to_i,
},
dlq: dlq_metrics,
}
end
def sqs_metrics
queue_name = params[:queue_name]
unless queue_name.present?
render status: 400, json: { error: 'params[:queue_name] is required' }
return
end
metric_name = params[:metric_name]
unless metric_name.present?
render status: 400, json: { error: 'params[:metric_name] is required' }
return
end
statistic = params[:statistic]
unless statistic.present?
render status: 400, json: { error: 'params[:statistic] is required' }
return
end
now = Time.zone.now
from = now - 24.hours
resp = self.class.cloudwatch_client.get_metric_statistics(
namespace: 'AWS/SQS',
metric_name: metric_name,
dimensions: [
{ name: 'QueueName', value: queue_name },
],
start_time: from,
end_time: now,
period: compute_minimum_period(from, now),
statistics: [statistic],
)
render json: {
label: resp.label,
datapoints: resp.datapoints.sort_by(&:timestamp).map { |datapoint|
{
timestamp: Time.zone.at(datapoint.timestamp),
value: datapoint[statistic.underscore],
}
},
}
end
private
# @return [Aws::SQS::Types::CreateQueueResult] A struct which has only queue_url.
def create_queue(job_queue)
Aws::SQS::Client.new.create_queue(
queue_name: job_queue.sqs_queue_name,
attributes: {
'ReceiveMessageWaitTimeSeconds' => Barbeque.config.sqs_receive_message_wait_time.to_s,
},
)
end
def self.sqs_client
@sqs_client ||= Aws::SQS::Client.new
end
def self.cloudwatch_client
@cloudwatch_client ||= Aws::CloudWatch::Client.new
end
def queue_name_from_arn(arn)
arn.slice(/[^:]+\z/)
end
def compute_minimum_period(start_time, end_time, maximum_datapoint: 1440)
# - Datapoints cannot exceed 1,440
# - Period must be a multiple of 60
maximum_datapoint = [maximum_datapoint, 1440].min.to_f
minimum_period = ((end_time - start_time) / maximum_datapoint).ceil
r = minimum_period % 60
if r == 0
minimum_period
else
minimum_period - r + 60
end
end
end
================================================
FILE: app/controllers/barbeque/job_retries_controller.rb
================================================
class Barbeque::JobRetriesController < Barbeque::ApplicationController
def show
@job_retry = Barbeque::JobRetry.find(params[:id])
@job_execution = @job_retry.job_execution
# Return 404 when job_definition or app is deleted
@job_definition = Barbeque::JobDefinition.find(@job_execution.job_definition_id)
@app = Barbeque::App.find(@job_definition.app_id)
if params[:job_execution_message_id] != @job_execution.message_id
redirect_to([@job_execution, @job_retry])
end
@execution_log = @job_execution.execution_log
@retry_log = @job_retry.execution_log
end
end
================================================
FILE: app/controllers/barbeque/monitors_controller.rb
================================================
class Barbeque::MonitorsController < Barbeque::ApplicationController
def index
now = Time.zone.now
from = 6.hours.ago(now.beginning_of_hour)
rows = Barbeque::JobExecution.find_by_sql([<You may have mistyped the address or the page may have moved.
If you are the application owner check the logs for more information.
Maybe you tried to change something you didn't have access to.
If you are the application owner check the logs for more information.
If you are the application owner check the logs for more information.