Full Code of ko1/allocation_tracer for AI

Repository: ko1/allocation_tracer
Branch: master
Commit: f4ac1e8c15f8
Files: 15
Total size: 55.2 KB

Directory structure:
gitextract_gpj107i2/

├── .gitignore
├── .travis.yml
├── Gemfile
├── LICENSE.txt
├── README.md
├── Rakefile
├── allocation_tracer.gemspec
├── ext/
│   └── allocation_tracer/
│       ├── allocation_tracer.c
│       └── extconf.rb
├── lib/
│   ├── allocation_tracer/
│   │   ├── trace.rb
│   │   └── version.rb
│   ├── allocation_tracer.rb
│   └── rack/
│       └── allocation_tracer.rb
└── spec/
    ├── allocation_tracer_spec.rb
    └── spec_helper.rb

================================================
FILE CONTENTS
================================================

================================================
FILE: .gitignore
================================================
*.gem
*.rbc
.bundle
.config
.yardoc
Gemfile.lock
InstalledFiles
_yardoc
coverage
doc/
lib/bundler/man
pkg
rdoc
spec/reports
test/tmp
test/version_tmp
tmp
*.bundle
*.DS_Store


================================================
FILE: .travis.yml
================================================
language: ruby
rvm:
  - 2.1.0
  - 2.2.0
  - 2.3.0
before_install:
  - gem update bundler


================================================
FILE: Gemfile
================================================
source 'https://rubygems.org'

# Specify your gem's dependencies in allocation_tracer.gemspec
gemspec


================================================
FILE: LICENSE.txt
================================================
Copyright (c) 2014 Koichi Sasada

MIT License

Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:

The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.


================================================
FILE: README.md
================================================
# ObjectSpace::AllocationTracer

This module allows to trace object allocation.

This feature is similar to https://github.com/SamSaffron/memory_profiler
and https://github.com/srawlins/allocation_stats. But this feature
focused on `age' of objects.

This gem was separated from gc_tracer.gem.

[![Build Status](https://travis-ci.org/ko1/allocation_tracer.svg?branch=master)](https://travis-ci.org/ko1/allocation_tracer)

## Installation

Add this line to your application's Gemfile:

    gem 'allocation_tracer'

And then execute:

    $ bundle

Or install it yourself as:

    $ gem install allocation_tracer

## Usage

### Allocation tracing

You can trace allocation and aggregate information. Information includes:

* count - how many objects are created.
* total_age - total age of objects which created here
* max_age - age of longest living object created here
* min_age - age of shortest living object created here

Age of Object can be calculated by this formula: [current GC count] - [birth time GC count]

For example:

```ruby
require 'allocation_tracer'
require 'pp'

pp ObjectSpace::AllocationTracer.trace{
  50_000.times{|i|
    i.to_s
    i.to_s
    i.to_s
  }
}
```

will show

```
{["test.rb", 6]=>[50000, 0, 47440, 0, 1, 0],
 ["test.rb", 7]=>[50000, 4, 47452, 0, 6, 0],
 ["test.rb", 8]=>[50000, 7, 47456, 0, 6, 0]}
```

In this case,
* 50,000 objects are created at `test.rb:6' and
  * 0 old objects created.
  * 47,440 is total age of objects created at this line (average age of object created at this line is 47440/50000 = 0.9488).
  * 0 is minimum age
  * 1 is maximum age.
  * 0 total memory consumption without RVALUE

You can also specify `type' in GC::Tracer.setup_allocation_tracing() to
specify what should be keys to aggregate like that.

```ruby
require 'allocation_tracer'
require 'pp'

ObjectSpace::AllocationTracer.setup(%i{path line type})

result = ObjectSpace::AllocationTracer.trace do
  50_000.times{|i|
    a = [i.to_s]
    b = {i.to_s => nil}
    c = (i.to_s .. i.to_s)
  }
end

pp result
```

and you will get:

```
{["test.rb", 8, :T_STRING]=>[50000, 15, 49165, 0, 16, 0],
 ["test.rb", 8, :T_ARRAY]=>[50000, 12, 49134, 0, 16, 0],
 ["test.rb", 9, :T_STRING]=>[100000, 27, 98263, 0, 16, 0],
 ["test.rb", 9, :T_HASH]=>[50000, 16, 49147, 0, 16, 8998848],
 ["test.rb", 10, :T_STRING]=>[100000, 36, 98322, 0, 16, 0],
 ["test.rb", 10, :T_STRUCT]=>[50000, 16, 49147, 0, 16, 0]}
```

Interestingly, you can not see array creations in a middle of block:

```ruby
require 'allocation_tracer'
require 'pp'

ObjectSpace::AllocationTracer.setup(%i{path line type})

result = ObjectSpace::AllocationTracer.trace do
  50_000.times{|i|
    [i.to_s]
    nil
  }
end

pp result
```

and it prints:

```
{["test.rb", 8, :T_STRING]=>[25015, 5, 16299, 0, 2, 0]}
```

There are only string creation. This is because unused array creation is
ommitted by optimizer.

Simply you can require `allocation_tracer/trace' to start allocation
tracer and output the aggregated information into stdout at the end of
program.

```ruby
require 'allocation_tracer/trace'

# Run your program here
50_000.times{|i|
  i.to_s
  i.to_s
  i.to_s
}
```

and you will see:

```
path    line    count   old_count       total_age       min_age max_age total_memsize
...rubygems/core_ext/kernel_require.rb 55     18       1       23      1       6       358
...lib/allocation_tracer/lib/allocation_tracer/trace.rb       6       2      012      6       6       0
test.rb 0       1       0       0       0       0       0
test.rb 5       50000   4       41492   0       5       0
test.rb 6       50000   3       41490   0       5       0
test.rb 7       50000   7       41497   0       5       0
```

(tab separated columns)

### Total Allocations / Free

Allocation tracer collects the total number of allocations and frees during the
`trace` block.  After the block finishes executing, you can examine the total
number of allocations / frees per object type like this:

```ruby
require 'allocation_tracer'

ObjectSpace::AllocationTracer.trace do
  1000.times {
    ["foo", {}]
  }
end
p allocated: ObjectSpace::AllocationTracer.allocated_count_table
p freed: ObjectSpace::AllocationTracer.freed_count_table
```

The output of the script will look like this:

```
{:allocated=>{:T_NONE=>0, :T_OBJECT=>0, :T_CLASS=>0, :T_MODULE=>0, :T_FLOAT=>0, :T_STRING=>1000, :T_REGEXP=>0, :T_ARRAY=>1000, :T_HASH=>1000, :T_STRUCT=>0, :T_BIGNUM=>0, :T_FILE=>0, :T_DATA=>0, :T_MATCH=>0, :T_COMPLEX=>0, :T_RATIONAL=>0, :unknown=>0, :T_NIL=>0, :T_TRUE=>0, :T_FALSE=>0, :T_SYMBOL=>0, :T_FIXNUM=>0, :T_UNDEF=>0, :T_NODE=>0, :T_ICLASS=>0, :T_ZOMBIE=>0}}
{:freed=>{:T_NONE=>0, :T_OBJECT=>0, :T_CLASS=>0, :T_MODULE=>0, :T_FLOAT=>0, :T_STRING=>1871, :T_REGEXP=>41, :T_ARRAY=>226, :T_HASH=>7, :T_STRUCT=>41, :T_BIGNUM=>0, :T_FILE=>50, :T_DATA=>25, :T_MATCH=>47, :T_COMPLEX=>0, :T_RATIONAL=>0, :unknown=>0, :T_NIL=>0, :T_TRUE=>0, :T_FALSE=>0, :T_SYMBOL=>0, :T_FIXNUM=>0, :T_UNDEF=>0, :T_NODE=>932, :T_ICLASS=>0, :T_ZOMBIE=>0}}
```

### Lifetime table

You can collect lifetime statistics with
ObjectSpace::AllocationTracer.lifetime_table method.

```ruby
require 'pp'
require 'allocation_tracer'

ObjectSpace::AllocationTracer.lifetime_table_setup true
result = ObjectSpace::AllocationTracer.trace do
  100000.times{
    Object.new
    ''
  }
end
pp ObjectSpace::AllocationTracer.lifetime_table
```

will show

```
{:T_OBJECT=>[3434, 96563, 0, 0, 1, 0, 0, 2],
 :T_STRING=>[3435, 96556, 2, 1, 1, 1, 1, 1, 2]}
```

This output means that the age of 3434 T_OBJECT objects are 0, 96563
objects are 1 and 2 objects are 7. Also the age of 3435 T_STRING
objects are 0, 96556 objects are 1 and so on.

Note that these numbers includes living objects and dead objects.  For
dead objects, age means lifetime. For living objects, age means
current age.

## Rack middleware

You can use AllocationTracer via rack middleware.

```ruby
require 'rack'
require 'sinatra'
require 'rack/allocation_tracer'

use Rack::AllocationTracerMiddleware

get '/' do
  'foo'
end
```

When you access to `http://host/allocation_tracer/` then you can see a table of allocation tracer.

You can access the following pages.

* http://host/allocation_tracer/
* http://host/allocation_tracer/allocated_count_table
* http://host/allocation_tracer/freed_count_table_page
* http://host/allocation_tracer/lifetime_table

The following pages are demonstration Rails app on Heroku environment.

* http://protected-journey-7206.herokuapp.com/allocation_tracer/
* http://protected-journey-7206.herokuapp.com/allocation_tracer/allocated_count_table
* http://protected-journey-7206.herokuapp.com/allocation_tracer/freed_count_table_page
* http://protected-journey-7206.herokuapp.com/allocation_tracer/lifetime_table

Source code of this demo app is https://github.com/ko1/tracer_demo_rails_app. You only need to modify like https://github.com/ko1/tracer_demo_rails_app/blob/master/config.ru to use it on Rails.

## Development

Compile the code:

```
$ rake compile
```

Run the tests:

```
$ rake spec
```

## Contributing

1. Fork it ( http://github.com/ko1/allocation_tracer/fork )
2. Create your feature branch (`git checkout -b my-new-feature`)
3. Commit your changes (`git commit -am 'Add some feature'`)
4. Push to the branch (`git push origin my-new-feature`)
5. Create new Pull Request

## Author

Koichi Sasada from Heroku, Inc.



================================================
FILE: Rakefile
================================================
require "bundler/gem_tasks"
require "rake/extensiontask"
require 'rspec/core/rake_task'

spec = Gem::Specification.load('allocation_tracer.gemspec')

Rake::ExtensionTask.new("allocation_tracer", spec){|ext|
  ext.lib_dir = "lib/allocation_tracer"
}

RSpec::Core::RakeTask.new('spec' => 'compile')

task default: :spec

task :run => 'compile' do
  ruby %q{-I ./lib test.rb}
end


================================================
FILE: allocation_tracer.gemspec
================================================
# coding: utf-8
lib = File.expand_path('../lib', __FILE__)
$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
require 'allocation_tracer/version'

Gem::Specification.new do |spec|
  spec.name          = "allocation_tracer"
  spec.version       = ObjectSpace::AllocationTracer::VERSION
  spec.authors       = ["Koichi Sasada"]
  spec.email         = ["ko1@atdot.net"]
  spec.summary       = %q{allocation_tracer gem adds ObjectSpace::AllocationTracer module.}
  spec.description   = %q{allocation_tracer gem adds ObjectSpace::AllocationTracer module.}
  spec.homepage      = "https://github.com/ko1/allocation_tracer"
  spec.license       = "MIT"

  spec.extensions    = %w[ext/allocation_tracer/extconf.rb]
  spec.required_ruby_version = '>= 2.1.0'

  spec.files         = `git ls-files -z`.split("\x0")
  spec.executables   = spec.files.grep(%r{^bin/}) { |f| File.basename(f) }
  spec.test_files    = spec.files.grep(%r{^(test|spec|features)/})
  spec.require_paths = ["lib"]

  spec.add_development_dependency "bundler", "~> 1.5"
  spec.add_development_dependency "rake"
  spec.add_development_dependency "rake-compiler"
  spec.add_development_dependency "rspec"
end


================================================
FILE: ext/allocation_tracer/allocation_tracer.c
================================================
/*
 * allocation tracer: adds GC::Tracer::start_allocation_tracing
 *
 * By Koichi Sasada
 * created at Thu Apr 17 03:50:38 2014.
 */

#include "ruby/ruby.h"
#include "ruby/debug.h"
#include <assert.h>

size_t rb_obj_memsize_of(VALUE obj); /* in gc.c */

static VALUE rb_mAllocationTracer;

struct traceobj_arg {
    int running;
    int keys, vals;
    st_table *object_table;     /* obj (VALUE)      -> allocation_info */
    st_table *str_table;        /* cstr             -> refcount */

    st_table *aggregate_table;  /* user defined key -> [count, total_age, max_age, min_age] */
    struct allocation_info *freed_allocation_info;

    /* */
    size_t **lifetime_table;
    size_t allocated_count_table[T_MASK];
    size_t freed_count_table[T_MASK];
};

struct allocation_info {
    struct allocation_info *next;

    /* all of information don't need marking. */
    int living;
    VALUE flags;
    VALUE klass;
    size_t generation;
    size_t memsize;

    /* allocator info */
    const char *path;
    unsigned long line;
};

#define MAX_KEY_DATA 4

#define KEY_PATH    (1<<1)
#define KEY_LINE    (1<<2)
#define KEY_TYPE    (1<<3)
#define KEY_CLASS   (1<<4)

#define MAX_VAL_DATA 6

#define VAL_COUNT     (1<<1)
#define VAL_OLDCOUNT  (1<<2)
#define VAL_TOTAL_AGE (1<<3)
#define VAL_MIN_AGE   (1<<4)
#define VAL_MAX_AGE   (1<<5)
#define VAL_MEMSIZE   (1<<6)

static char *
keep_unique_str(st_table *tbl, const char *str)
{
    st_data_t n;

    if (str && st_lookup(tbl, (st_data_t)str, &n)) {
	char *result;

	st_insert(tbl, (st_data_t)str, n+1);
	st_get_key(tbl, (st_data_t)str, (st_data_t *)&result);

	return result;
    }
    else {
	return NULL;
    }
}

static const char *
make_unique_str(st_table *tbl, const char *str, long len)
{
    if (!str) {
	return NULL;
    }
    else {
	char *result;

	if ((result = keep_unique_str(tbl, str)) == NULL) {
	    result = (char *)ruby_xmalloc(len+1);
	    strncpy(result, str, len);
	    result[len] = 0;
	    st_add_direct(tbl, (st_data_t)result, 1);
	}
	return result;
    }
}

static void
delete_unique_str(st_table *tbl, const char *str)
{
    if (str) {
	st_data_t n;

	if (st_lookup(tbl, (st_data_t)str, &n) == 0) rb_bug("delete_unique_str: unreachable");

	if (n == 1) {
	    st_delete(tbl, (st_data_t *)&str, NULL);
	    ruby_xfree((char *)str);
	}
	else {
	    st_insert(tbl, (st_data_t)str, n-1);
	}
    }
}

struct memcmp_key_data {
    int n;
    st_data_t data[MAX_KEY_DATA];
};

static int
memcmp_hash_compare(st_data_t a, st_data_t b)
{
    struct memcmp_key_data *k1 = (struct memcmp_key_data *)a;
    struct memcmp_key_data *k2 = (struct memcmp_key_data *)b;
    return memcmp(&k1->data[0], &k2->data[0], k1->n * sizeof(st_data_t));
}

static st_index_t
memcmp_hash_hash(st_data_t a)
{
    struct memcmp_key_data *k = (struct memcmp_key_data *)a;
    return rb_memhash(k->data, sizeof(st_data_t) * k->n);
}

static const struct st_hash_type memcmp_hash_type = {
    memcmp_hash_compare, memcmp_hash_hash
};

static struct traceobj_arg *tmp_trace_arg; /* TODO: Do not use global variables */

static struct traceobj_arg *
get_traceobj_arg(void)
{
    if (tmp_trace_arg == 0) {
	tmp_trace_arg = ALLOC_N(struct traceobj_arg, 1);
	MEMZERO(tmp_trace_arg, struct traceobj_arg, 1);
	tmp_trace_arg->running = 0;
	tmp_trace_arg->keys = 0;
	tmp_trace_arg->vals = VAL_COUNT | VAL_OLDCOUNT | VAL_TOTAL_AGE | VAL_MAX_AGE | VAL_MIN_AGE | VAL_MEMSIZE;
	tmp_trace_arg->aggregate_table = st_init_table(&memcmp_hash_type);
	tmp_trace_arg->object_table = st_init_numtable();
	tmp_trace_arg->str_table = st_init_strtable();
	tmp_trace_arg->freed_allocation_info = NULL;
	tmp_trace_arg->lifetime_table = NULL;
    }
    return tmp_trace_arg;
}

static int
free_keys_i(st_data_t key, st_data_t value, void *data)
{
    ruby_xfree((void *)key);
    return ST_CONTINUE;
}

static int
free_values_i(st_data_t key, st_data_t value, void *data)
{
    ruby_xfree((void *)value);
    return ST_CONTINUE;
}

static int
free_key_values_i(st_data_t key, st_data_t value, void *data)
{
    ruby_xfree((void *)key);
    ruby_xfree((void *)value);
    return ST_CONTINUE;
}

static void
delete_lifetime_table(struct traceobj_arg *arg)
{
    int i;
    if (arg->lifetime_table) {
	for (i=0; i<T_MASK; i++) {
	    free(arg->lifetime_table[i]);
	}
	free(arg->lifetime_table);
	arg->lifetime_table = NULL;
    }
}

static void
clear_traceobj_arg(void)
{
    struct traceobj_arg * arg = get_traceobj_arg();

    st_foreach(arg->aggregate_table, free_key_values_i, 0);
    st_clear(arg->aggregate_table);
    st_foreach(arg->object_table, free_values_i, 0);
    st_clear(arg->object_table);
    st_foreach(arg->str_table, free_keys_i, 0);
    st_clear(arg->str_table);
    arg->freed_allocation_info = NULL;
    delete_lifetime_table(arg);
}

static struct allocation_info *
create_allocation_info(void)
{
    return (struct allocation_info *)ruby_xmalloc(sizeof(struct allocation_info));
}

static void
free_allocation_info(struct traceobj_arg *arg, struct allocation_info *info)
{
    delete_unique_str(arg->str_table, info->path);
    ruby_xfree(info);
}

static void
newobj_i(VALUE tpval, void *data)
{
    struct traceobj_arg *arg = (struct traceobj_arg *)data;
    struct allocation_info *info;
    rb_trace_arg_t *tparg = rb_tracearg_from_tracepoint(tpval);
    VALUE obj = rb_tracearg_object(tparg);
    VALUE path = rb_tracearg_path(tparg);
    VALUE line = rb_tracearg_lineno(tparg);
    VALUE klass = Qnil;
    switch(BUILTIN_TYPE(obj)) {
        case T_NODE:
        case T_IMEMO:
            break;
        default:
            klass = RBASIC_CLASS(obj);
    }
    const char *path_cstr = RTEST(path) ? make_unique_str(arg->str_table, RSTRING_PTR(path), RSTRING_LEN(path)) : NULL;

    if (st_lookup(arg->object_table, (st_data_t)obj, (st_data_t *)&info)) {
	if (info->living) {
	    /* do nothing. there is possibility to keep living if FREEOBJ events while suppressing tracing */
	}
	/* reuse info */
	delete_unique_str(arg->str_table, info->path);
    }
    else {
	info = create_allocation_info();
    }

    info->next = NULL;
    info->flags = RBASIC(obj)->flags;
    info->living = 1;
    info->memsize = 0;
    info->klass = (RTEST(klass) && !RB_TYPE_P(obj, T_NODE)) ? rb_class_real(klass) : Qnil;
    info->generation = rb_gc_count();

    info->path = path_cstr;
    info->line = NUM2INT(line);

    st_insert(arg->object_table, (st_data_t)obj, (st_data_t)info);

    arg->allocated_count_table[BUILTIN_TYPE(obj)]++;
}

/* file, line, type, klass */
#define MAX_KEY_SIZE 4

static void
aggregate_each_info(struct traceobj_arg *arg, struct allocation_info *info, size_t gc_count)
{
    st_data_t key, val;
    struct memcmp_key_data key_data;
    size_t *val_buff;
    size_t age = (int)(gc_count - info->generation);
    int i = 0;

    if (arg->keys & KEY_PATH) {
	key_data.data[i++] = (st_data_t)info->path;
    }
    if (arg->keys & KEY_LINE) {
	key_data.data[i++] = (st_data_t)info->line;
    }
    if (arg->keys & KEY_TYPE) {
	key_data.data[i++] = (st_data_t)(info->flags & T_MASK);
    }
    if (arg->keys & KEY_CLASS) {
	key_data.data[i++] = info->klass;
    }
    key_data.n = i;
    key = (st_data_t)&key_data;

    if (st_lookup(arg->aggregate_table, key, &val) == 0) {
	struct memcmp_key_data *key_buff = ruby_xmalloc(sizeof(struct memcmp_key_data));
	key_buff->n = key_data.n;

	for (i=0; i<key_data.n; i++) {
	    key_buff->data[i] = key_data.data[i];
	}
	key = (st_data_t)key_buff;

	/* count, old count, total age, max age, min age */
	val_buff = ALLOC_N(size_t, 6);
	val_buff[0] = val_buff[1] = val_buff[2] = 0;
	val_buff[3] = val_buff[4] = age;
	val_buff[5] = 0;

	if (arg->keys & KEY_PATH) keep_unique_str(arg->str_table, info->path);

	st_insert(arg->aggregate_table, (st_data_t)key_buff, (st_data_t)val_buff);
    }
    else {
	val_buff = (size_t *)val;
    }

    val_buff[0] += 1;
#ifdef FL_PROMOTED
    if (info->flags & FL_PROMOTED) val_buff[1] += 1;
#elif defined(FL_PROMOTED0) && defined(FL_PROMOTED1)
    if (info->flags & FL_PROMOTED0 &&
	info->flags & FL_PROMOTED1) val_buff[1] += 1;
#endif
    val_buff[2] += age;
    if (val_buff[3] > age) val_buff[3] = age; /* min */
    if (val_buff[4] < age) val_buff[4] = age; /* max */
    val_buff[5] += info->memsize;
}

static void
aggregate_freed_info(void *data)
{
    size_t gc_count = rb_gc_count();
    struct traceobj_arg *arg = (struct traceobj_arg *)data;
    struct allocation_info *info = arg->freed_allocation_info;

    arg->freed_allocation_info = NULL;

    if (arg->running) {
	while (info) {
	    struct allocation_info *next_info = info->next;
	    aggregate_each_info(arg, info, gc_count);
	    free_allocation_info(arg, info);
	    info = next_info;
	}
    }
}

static void
move_to_freed_list(struct traceobj_arg *arg, VALUE obj, struct allocation_info *info)
{
    if (arg->freed_allocation_info == NULL) {
	rb_postponed_job_register_one(0, aggregate_freed_info, arg);
    }

    info->next = arg->freed_allocation_info;
    arg->freed_allocation_info = info;
    st_delete(arg->object_table, (st_data_t *)&obj, (st_data_t *)&info);
}

static void
add_lifetime_table(size_t **lines, int type, struct allocation_info *info)
{
    size_t age = rb_gc_count() - info->generation;
    size_t *line = lines[type];
    size_t len, i;

    if (line == NULL) {
	len = age + 1;
	line = lines[type] = calloc(1 + len, sizeof(size_t));
	assert(line != NULL);
	line[0] = len;
    }
    else {
	len = line[0];

	if (len < age + 1) {
	    size_t old_len = len;
	    len = age + 1;
	    line = lines[type] = realloc(line, sizeof(size_t) * (1 + len));

	    assert(line != NULL);

	    for (i=old_len; i<len; i++) {
		line[i+1] = 0;
	    }

	    line[0] = len;
	}
    }

    line[1 + age]++;
}

static void
freeobj_i(VALUE tpval, void *data)
{
    struct traceobj_arg *arg = (struct traceobj_arg *)data;
    rb_trace_arg_t *tparg = rb_tracearg_from_tracepoint(tpval);
    VALUE obj = rb_tracearg_object(tparg);
    struct allocation_info *info;

    if (st_lookup(arg->object_table, (st_data_t)obj, (st_data_t *)&info)) {

	info->flags = RBASIC(obj)->flags;
	info->memsize = rb_obj_memsize_of(obj);

	move_to_freed_list(arg, obj, info);

	if (arg->lifetime_table) {
	    add_lifetime_table(arg->lifetime_table, BUILTIN_TYPE(obj), info);
	}
    }

    arg->freed_count_table[BUILTIN_TYPE(obj)]++;
}

static void
check_tracer_running(void)
{
    struct traceobj_arg * arg = get_traceobj_arg();

    if (!arg->running) {
	rb_raise(rb_eRuntimeError, "not started yet");
    }
}

static void
enable_newobj_hook(void)
{
    VALUE newobj_hook;

    check_tracer_running();

    if (!rb_ivar_defined(rb_mAllocationTracer, rb_intern("newobj_hook"))) {
	rb_raise(rb_eRuntimeError, "not started.");
    }
    newobj_hook = rb_ivar_get(rb_mAllocationTracer, rb_intern("newobj_hook"));
    if (rb_tracepoint_enabled_p(newobj_hook)) {
	rb_raise(rb_eRuntimeError, "newobj hooks is already enabled.");
    }

    rb_tracepoint_enable(newobj_hook);
}

static void
disable_newobj_hook(void)
{
    VALUE newobj_hook;

    check_tracer_running();

    if ((!rb_ivar_defined(rb_mAllocationTracer, rb_intern("newobj_hook"))) || ((newobj_hook = rb_ivar_get(rb_mAllocationTracer, rb_intern("newobj_hook"))) == Qnil)) {
	rb_raise(rb_eRuntimeError, "not started.");
    }
    if (rb_tracepoint_enabled_p(newobj_hook) == Qfalse) {
	rb_raise(rb_eRuntimeError, "newobj hooks is already disabled.");
    }

    rb_tracepoint_disable(newobj_hook);
}

static void
start_alloc_hooks(VALUE mod)
{
    VALUE newobj_hook, freeobj_hook;
    struct traceobj_arg *arg = get_traceobj_arg();

    if (!rb_ivar_defined(rb_mAllocationTracer, rb_intern("newobj_hook"))) {
	rb_ivar_set(rb_mAllocationTracer, rb_intern("newobj_hook"), newobj_hook = rb_tracepoint_new(0, RUBY_INTERNAL_EVENT_NEWOBJ, newobj_i, arg));
	rb_ivar_set(rb_mAllocationTracer, rb_intern("freeobj_hook"), freeobj_hook = rb_tracepoint_new(0, RUBY_INTERNAL_EVENT_FREEOBJ, freeobj_i, arg));
    }
    else {
	newobj_hook = rb_ivar_get(rb_mAllocationTracer, rb_intern("newobj_hook"));
	freeobj_hook = rb_ivar_get(rb_mAllocationTracer, rb_intern("freeobj_hook"));
    }

    rb_tracepoint_enable(newobj_hook);
    rb_tracepoint_enable(freeobj_hook);
}

static VALUE
stop_alloc_hooks(VALUE self)
{
    struct traceobj_arg * arg = get_traceobj_arg();
    check_tracer_running();

    {
	VALUE newobj_hook = rb_ivar_get(rb_mAllocationTracer, rb_intern("newobj_hook"));
	VALUE freeobj_hook = rb_ivar_get(rb_mAllocationTracer, rb_intern("freeobj_hook"));
	rb_tracepoint_disable(newobj_hook);
	rb_tracepoint_disable(freeobj_hook);

	clear_traceobj_arg();

	arg->running = 0;
    }

    return Qnil;
}

static VALUE
type_sym(int type)
{
    static VALUE syms[T_MASK] = {0};

    if (syms[0] == 0) {
	int i;
	for (i=0; i<T_MASK; i++) {
	    switch (i) {
#define TYPE_NAME(t) case (t): syms[i] = ID2SYM(rb_intern(#t)); break;
		TYPE_NAME(T_NONE);
		TYPE_NAME(T_OBJECT);
		TYPE_NAME(T_CLASS);
		TYPE_NAME(T_MODULE);
		TYPE_NAME(T_FLOAT);
		TYPE_NAME(T_STRING);
		TYPE_NAME(T_REGEXP);
		TYPE_NAME(T_ARRAY);
		TYPE_NAME(T_HASH);
		TYPE_NAME(T_STRUCT);
		TYPE_NAME(T_BIGNUM);
		TYPE_NAME(T_FILE);
		TYPE_NAME(T_MATCH);
		TYPE_NAME(T_COMPLEX);
		TYPE_NAME(T_RATIONAL);
		TYPE_NAME(T_NIL);
		TYPE_NAME(T_TRUE);
		TYPE_NAME(T_FALSE);
		TYPE_NAME(T_SYMBOL);
		TYPE_NAME(T_FIXNUM);
		TYPE_NAME(T_UNDEF);
#ifdef T_IMEMO /* introduced from Rub 2.3 */
		TYPE_NAME(T_IMEMO);
#endif
		TYPE_NAME(T_NODE);
		TYPE_NAME(T_ICLASS);
		TYPE_NAME(T_ZOMBIE);
		TYPE_NAME(T_DATA);
	      default:
		syms[i] = ID2SYM(rb_intern("unknown"));
		break;
#undef TYPE_NAME
	    }
	}
    }

    return syms[type];
}

struct arg_and_result {
    int update;
    struct traceobj_arg *arg;
    VALUE result;
};

static int
aggregate_result_i(st_data_t key, st_data_t val, void *data)
{
    struct arg_and_result *aar = (struct arg_and_result *)data;
    struct traceobj_arg *arg = aar->arg;
    VALUE result = aar->result;
    size_t *val_buff = (size_t *)val;
    struct memcmp_key_data *key_buff = (struct memcmp_key_data *)key;
    VALUE v, oldv, k = rb_ary_new();
    int i = 0;

    i = 0;
    if (arg->keys & KEY_PATH) {
	const char *path = (const char *)key_buff->data[i++];
	if (path) {
	    rb_ary_push(k, rb_str_new2(path));
	}
	else {
	    rb_ary_push(k, Qnil);
	}
    }
    if (arg->keys & KEY_LINE) {
	rb_ary_push(k, INT2FIX((int)key_buff->data[i++]));
    }
    if (arg->keys & KEY_TYPE) {
	int sym_index = key_buff->data[i++];
	rb_ary_push(k, type_sym(sym_index));
    }
    if (arg->keys & KEY_CLASS) {
	VALUE klass = key_buff->data[i++];
	if (RTEST(klass) && BUILTIN_TYPE(klass) == T_CLASS) {
	    klass = rb_class_real(klass);
	    rb_ary_push(k, klass);
	    /* TODO: actually, it is dangerous code because klass can be sweeped */
	    /*       So that class specifier is hidden feature                   */
	}
	else {
	    rb_ary_push(k, Qnil);
	}
    }

#define MIN(a, b) ((a) < (b) ? (a) : (b))
#define MAX(a, b) ((a) > (b) ? (a) : (b))

    if (aar->update && (oldv = rb_hash_aref(result, k)) != Qnil) {
	v = rb_ary_new3(6,
			INT2FIX(val_buff[0] + (size_t)FIX2INT(RARRAY_AREF(oldv, 0))), /* count */
			INT2FIX(val_buff[1] + (size_t)FIX2INT(RARRAY_AREF(oldv, 1))), /* old count */
			INT2FIX(val_buff[2] + (size_t)FIX2INT(RARRAY_AREF(oldv, 2))), /* total_age */
			INT2FIX(MIN(val_buff[3], (size_t)FIX2INT(RARRAY_AREF(oldv, 3)))), /* min age */
			INT2FIX(MAX(val_buff[4], (size_t)FIX2INT(RARRAY_AREF(oldv, 4)))), /* max age */
			INT2FIX(val_buff[5] + (size_t)FIX2INT(RARRAY_AREF(oldv, 5)))); /* memsize_of */
    }
    else {
	v = rb_ary_new3(6,
			INT2FIX(val_buff[0]), INT2FIX(val_buff[1]),
			INT2FIX(val_buff[2]), INT2FIX(val_buff[3]),
			INT2FIX(val_buff[4]), INT2FIX(val_buff[5]));
    }

    rb_hash_aset(result, k, v);

    return ST_CONTINUE;
}

static int
aggregate_live_object_i(st_data_t key, st_data_t val, void *data)
{
    VALUE obj = (VALUE)key;
    struct allocation_info *info = (struct allocation_info *)val;
    size_t gc_count = rb_gc_count();
    struct traceobj_arg *arg = (struct traceobj_arg *)data;

    if (BUILTIN_TYPE(obj) == (info->flags & T_MASK)) {
	VALUE klass = RBASIC_CLASS(obj);
	info->flags = RBASIC(obj)->flags;
	info->klass = (RTEST(klass) && !RB_TYPE_P(obj, T_NODE)) ? rb_class_real(klass) : Qnil;
    }

    aggregate_each_info(arg, info, gc_count);

    return ST_CONTINUE;
}

static int
lifetime_table_for_live_objects_i(st_data_t key, st_data_t val, st_data_t data)
{
    struct allocation_info *info = (struct allocation_info *)val;
    VALUE h = (VALUE)data;
    int type = info->flags & T_MASK;
    VALUE sym = type_sym(type);
    size_t age = rb_gc_count() - info->generation;
    VALUE line;
    size_t count, i;

    if ((line = rb_hash_aref(h, sym)) == Qnil) {
	line = rb_ary_new();
	rb_hash_aset(h, sym, line);
    }

    for (i=RARRAY_LEN(line); i<age+1; i++) {
	rb_ary_push(line, INT2FIX(0));
    }

    count = NUM2SIZET(RARRAY_AREF(line, age));
    RARRAY_ASET(line, age, SIZET2NUM(count + 1));

    return ST_CONTINUE;
}

static VALUE
aggregate_result(struct traceobj_arg *arg)
{
    struct arg_and_result aar;
    aar.result = rb_hash_new();
    aar.arg = arg;

    while (arg->freed_allocation_info) {
	aggregate_freed_info(arg);
    }

    /* collect from recent-freed objects */
    aar.update = 0;
    st_foreach(arg->aggregate_table, aggregate_result_i, (st_data_t)&aar);

    {
	st_table *dead_object_aggregate_table = arg->aggregate_table;

	/* make live object aggregate table */
	arg->aggregate_table = st_init_table(&memcmp_hash_type);
	st_foreach(arg->object_table, aggregate_live_object_i, (st_data_t)arg);

	/* aggregate table -> Ruby hash */
	aar.update = 1;
	st_foreach(arg->aggregate_table, aggregate_result_i, (st_data_t)&aar);

	/* remove live object aggregate table */
	st_foreach(arg->aggregate_table, free_key_values_i, 0);
	st_free_table(arg->aggregate_table);

	arg->aggregate_table = dead_object_aggregate_table;
    }

    /* lifetime table */
    if (arg->lifetime_table) {
	VALUE h = rb_hash_new();
	int i;

	rb_ivar_set(rb_mAllocationTracer, rb_intern("lifetime_table"), h);

	for (i=0; i<T_MASK; i++) {
	    size_t *line = arg->lifetime_table[i];

	    if (line) {
		size_t len = line[0], j;
		VALUE ary = rb_ary_new();
		VALUE sym = type_sym(i);

		rb_hash_aset(h, sym, ary);

		for (j=0; j<len; j++) {
		    rb_ary_push(ary, SIZET2NUM(line[j+1]));
		}
	    }
	}

	st_foreach(arg->object_table, lifetime_table_for_live_objects_i, (st_data_t)h);
    }

    return aar.result;
}

/*
 *
 *  call-seq:
 *     ObjectSpace::AllocationTracer.result  -> hash
 *
 *  Returns the current allocated results
 *
 *  If you need to know the results of allocation tracing
 *  without pausing or stopping tracing you can use this method.
 *
 *  Example:
 *
 *    require 'allocation_tracer'
 *
 *    ObjectSpace::AllocationTracer.trace do
 *      3.times do |i|
 *        a = "#{i}"
 *        puts ObjectSpace::AllocationTracer.result
 *      end
 *    end
 *
 *    # => {["scratch.rb", 5]=>[2, 0, 0, 0, 0, 0]}
 *    # => {["scratch.rb", 5]=>[4, 0, 0, 0, 0, 0], ["scratch.rb", 6]=>[16, 0, 0, 0, 0, 0]}
 *    # => {["scratch.rb", 5]=>[6, 0, 0, 0, 0, 0], ["scratch.rb", 6]=>[38, 0, 0, 0, 0, 0]}
 *
 */
static VALUE
allocation_tracer_result(VALUE self)
{
    VALUE result;
    struct traceobj_arg *arg = get_traceobj_arg();

    disable_newobj_hook();
    result = aggregate_result(arg);
    enable_newobj_hook();
    return result;
}

/*
 *
 *  call-seq:
 *     ObjectSpace::AllocationTracer.clear  -> NilClass
 *
 *  Clears the current allocated results
 *
 *  If you need to clear the results of allocation tracing
 *  without stopping tracing you can use this method.
 *
 */
static VALUE
allocation_tracer_clear(VALUE self)
{
    clear_traceobj_arg();
    return Qnil;
}

/*! Used in allocation_tracer_trace
*   to ensure that a result is returned.
*/
static VALUE
allocation_tracer_trace_i(VALUE self)
{
    rb_yield(Qnil);
    return allocation_tracer_result(self);
}

/*
 *
 *  call-seq:
 *     ObjectSpace::AllocationTracer.trace { |a, b| block } -> array
 *     ObjectSpace::AllocationTracer.start                  -> NilClass
 *
 *  Traces object allocations inside of the block
 *
 *  Objects created inside of the block will be tracked by the tracer.
 *  If the method is called without a block, the tracer will start
 *  and continue until ObjectSpace::AllocationTracer.stop is called.
 *
 *  Output can be customized with ObjectSpace::AllocationTracer.setup.
 *
 *  Example:
 *
 *     pp ObjectSpace::AllocationTracer.trace{
 *       50_000.times{|i|
 *         i.to_s
 *         i.to_s
 *         i.to_s
 *       }
 *     }
 *
 *     # => {["test.rb", 6]=>[50000, 0, 47440, 0, 1, 0],
 *           ["test.rb", 7]=>[50000, 4, 47452, 0, 6, 0],
 *           ["test.rb", 8]=>[50000, 7, 47456, 0, 6, 0]}
 *
 */
static VALUE
allocation_tracer_trace(VALUE self)
{
    struct traceobj_arg * arg = get_traceobj_arg();

    if (arg->running) {
	rb_raise(rb_eRuntimeError, "can't run recursivly");
    }
    else {
	arg->running = 1;
	if (arg->keys == 0) arg->keys = KEY_PATH | KEY_LINE;
	start_alloc_hooks(rb_mAllocationTracer);

	if (rb_block_given_p()) {
	    return rb_ensure(allocation_tracer_trace_i, self, stop_alloc_hooks, Qnil);
	}
    }

    return Qnil;
}

/*
 *
 *  call-seq:
 *     ObjectSpace::AllocationTracer.stop                  -> array
 *
 *  Stops allocation tracing if currently running
 *
 *  When allocation tracing is started via ObjectSpace::AllocationTracer.start
 *  it will continue until this method is called.
 *
 *  Example:
 *    pp ObjectSpace::AllocationTracer.stop
 *
 *    # => {["test.rb", 6]=>[50000, 0, 47440, 0, 1, 0],
 *          ["test.rb", 7]=>[50000, 4, 47452, 0, 6, 0],
 *          ["test.rb", 8]=>[50000, 7, 47456, 0, 6, 0]}
 *
 */
static VALUE
allocation_tracer_stop(VALUE self)
{
    VALUE result;

    disable_newobj_hook();
    result = aggregate_result(get_traceobj_arg());
    stop_alloc_hooks(self);
    return result;
}

/*
 *
 *  call-seq:
 *     ObjectSpace::AllocationTracer.pause                  -> NilClass
 *
 *  Pauses allocation tracing
 *
 *  Use in conjunction with ObjectSpace::AllocationTracer.start and
 *  ObjectSpace::AllocationTracer.stop.
 *
 */
static VALUE
allocation_tracer_pause(VALUE self)
{
    disable_newobj_hook();
    return Qnil;
}

/*
 *
 *  call-seq:
 *     ObjectSpace::AllocationTracer.resume                  -> NilClass
 *
 *  Resumes allocation tracing if previously paused
 *
 *  See ObjectSpace::AllocationTracer.pause to pause allocation tracing.
 *
 */
static VALUE
allocation_tracer_resume(VALUE self)
{
    enable_newobj_hook();
    return Qnil;
}

/*
 *
 *  call-seq:
 *     ObjectSpace::AllocationTracer.setup([symbol])       -> NilClass
 *
 *  Change the format that results will be returned.
 *
 *  Takes an array of symbols containing the order you would like the output
 *  to be returned. Valid options:
 *
 *    - :path
 *    - :line
 *    - :type
 *    - :class
 *
 *  Example:
 *
 *     ObjectSpace::AllocationTracer.setup(%i{path line type})
 *
 *     result = ObjectSpace::AllocationTracer.trace do
 *       50_000.times{|i|
 *         a = [i.to_s]
 *         b = {i.to_s => nil}
 *         c = (i.to_s .. i.to_s)
 *       }
 *     end
 *
 *     pp result
 *
 *     # => {["test.rb", 8, :T_STRING]=>[50000, 15, 49165, 0, 16, 0],
 *           ["test.rb", 8, :T_ARRAY]=>[50000, 12, 49134, 0, 16, 0],
 *           ["test.rb", 9, :T_STRING]=>[100000, 27, 98263, 0, 16, 0],
 *           ["test.rb", 9, :T_HASH]=>[50000, 16, 49147, 0, 16, 8998848],
 *           ["test.rb", 10, :T_STRING]=>[100000, 36, 98322, 0, 16, 0],
 *           ["test.rb", 10, :T_STRUCT]=>[50000, 16, 49147, 0, 16, 0]}
 *
 */
static VALUE
allocation_tracer_setup(int argc, VALUE *argv, VALUE self)
{
    struct traceobj_arg * arg = get_traceobj_arg();

    if (arg->running) {
	rb_raise(rb_eRuntimeError, "can't change configuration during running");
    }
    else {
	if (argc >= 1) {
	    int i;
	    VALUE ary = rb_check_array_type(argv[0]);

	    arg->keys = 0;

	    for (i=0; i<(int)RARRAY_LEN(ary); i++) {
		if (RARRAY_AREF(ary, i) == ID2SYM(rb_intern("path"))) arg->keys |= KEY_PATH;
		else if (RARRAY_AREF(ary, i) == ID2SYM(rb_intern("line"))) arg->keys |= KEY_LINE;
		else if (RARRAY_AREF(ary, i) == ID2SYM(rb_intern("type"))) arg->keys |= KEY_TYPE;
		else if (RARRAY_AREF(ary, i) == ID2SYM(rb_intern("class"))) arg->keys |= KEY_CLASS;
		else {
		    rb_raise(rb_eArgError, "not supported key type");
		}
	    }
	}
    }

    if (argc == 0) {
	arg->keys = KEY_PATH | KEY_LINE;
    }
    return Qnil;
}

/*
 *
 *  call-seq:
 *     ObjectSpace::AllocationTracer.header   -> array
 *
 *  Return headers
 *
 *  Example:
 *
 *     puts ObjectSpace::AllocationTracer.header
 *     => [:path, :line, :count, :old_count, :total_age, :min_age, :max_age, :total_memsize]
 *
 */
VALUE
allocation_tracer_header(VALUE self)
{
    VALUE ary = rb_ary_new();
    struct traceobj_arg * arg = get_traceobj_arg();

    if (arg->keys & KEY_PATH) rb_ary_push(ary, ID2SYM(rb_intern("path")));
    if (arg->keys & KEY_LINE) rb_ary_push(ary, ID2SYM(rb_intern("line")));
    if (arg->keys & KEY_TYPE) rb_ary_push(ary, ID2SYM(rb_intern("type")));
    if (arg->keys & KEY_CLASS) rb_ary_push(ary, ID2SYM(rb_intern("class")));

    if (arg->vals & VAL_COUNT) rb_ary_push(ary, ID2SYM(rb_intern("count")));
    if (arg->vals & VAL_OLDCOUNT) rb_ary_push(ary, ID2SYM(rb_intern("old_count")));
    if (arg->vals & VAL_TOTAL_AGE) rb_ary_push(ary, ID2SYM(rb_intern("total_age")));
    if (arg->vals & VAL_MIN_AGE) rb_ary_push(ary, ID2SYM(rb_intern("min_age")));
    if (arg->vals & VAL_MAX_AGE) rb_ary_push(ary, ID2SYM(rb_intern("max_age")));
    if (arg->vals & VAL_MEMSIZE) rb_ary_push(ary, ID2SYM(rb_intern("total_memsize")));
    return ary;
}

/*
 *
 *  call-seq:
 *     ObjectSpace::AllocationTracer.lifetime_table_setup(true)   -> NilClass
 *
 * Enables tracing for the generation of objects
 *
 * See ObjectSpace::AllocationTracer.lifetime_table for an example.
 */
static VALUE
allocation_tracer_lifetime_table_setup(VALUE self, VALUE set)
{
    struct traceobj_arg * arg = get_traceobj_arg();

    if (arg->running) {
	rb_raise(rb_eRuntimeError, "can't change configuration during running");
    }

    if (RTEST(set)) {
	if (arg->lifetime_table == NULL) {
	    arg->lifetime_table = (size_t **)calloc(T_MASK, sizeof(size_t **));
	}
    }
    else {
	delete_lifetime_table(arg);
    }

    return Qnil;
}

/*
 *
 *  call-seq:
 *     ObjectSpace::AllocationTracer.lifetime_table   -> hash
 *
 * Returns generations for objects
 *
 * Count is for both living (retained) and dead (freed) objects.
 *
 * The key is the type of objects, for example `T_OBJECT` for Ruby objects
 * or `T_STRING` for Ruby strings.
 *
 * The value is an array containing a count of the objects, the index is
 * the generation.
 *
 * Example:
 *
 *     require 'pp'
 *     require 'allocation_tracer'
 *
 *     ObjectSpace::AllocationTracer.lifetime_table_setup true
 *     result = ObjectSpace::AllocationTracer.trace do
 *       100000.times{
 *         Object.new
 *         ''
 *       }
 *     end
 *     pp ObjectSpace::AllocationTracer.lifetime_table
 *     # => {:T_OBJECT=>[3434, 96563, 0, 0, 1, 0, 0, 2],
 *           :T_STRING=>[3435, 96556, 2, 1, 1, 1, 1, 1, 2]}
 */
static VALUE
allocation_tracer_lifetime_table(VALUE self)
{
    VALUE result = rb_ivar_get(rb_mAllocationTracer, rb_intern("lifetime_table"));
    rb_ivar_set(rb_mAllocationTracer, rb_intern("lifetime_table"), Qnil);
    return result;
}


/*
 *
 *  call-seq:
 *     ObjectSpace::AllocationTracer.allocated_count_table   -> hash
 *
 * Returns allocation count totals for Ruby object types
 *
 * Returns a hash showing the number of each type of object that has been allocated.
 *
 * Example:
 *
 *     require 'allocation_tracer'
 *
 *     ObjectSpace::AllocationTracer.trace do
 *       1000.times {
 *         ["foo", {}]
 *       }
 *     end
 *     p allocated: ObjectSpace::AllocationTracer.allocated_count_table
 *     {:allocated=>{:T_NONE=>0, :T_OBJECT=>0, :T_CLASS=>0, :T_MODULE=>0, :T_FLOAT=>0, :T_STRING=>1000, :T_REGEXP=>0, :T_ARRAY=>1000, :T_HASH=>1000, :T_STRUCT=>0, :T_BIGNUM=>0, :T_FILE=>0, :T_DATA=>0, :T_MATCH=>0, :T_COMPLEX=>0, :T_RATIONAL=>0, :unknown=>0, :T_NIL=>0, :T_TRUE=>0, :T_FALSE=>0, :T_SYMBOL=>0, :T_FIXNUM=>0, :T_UNDEF=>0, :T_NODE=>0, :T_ICLASS=>0, :T_ZOMBIE=>0}}
 *
 *     p freed: ObjectSpace::AllocationTracer.freed_count_table
 *     {:freed=>{:T_NONE=>0, :T_OBJECT=>0, :T_CLASS=>0, :T_MODULE=>0, :T_FLOAT=>0, :T_STRING=>1871, :T_REGEXP=>41, :T_ARRAY=>226, :T_HASH=>7, :T_STRUCT=>41, :T_BIGNUM=>0, :T_FILE=>50, :T_DATA=>25, :T_MATCH=>47, :T_COMPLEX=>0, :T_RATIONAL=>0, :unknown=>0, :T_NIL=>0, :T_TRUE=>0, :T_FALSE=>0, :T_SYMBOL=>0, :T_FIXNUM=>0, :T_UNDEF=>0, :T_NODE=>932, :T_ICLASS=>0, :T_ZOMBIE=>0}}
 */
static VALUE
allocation_tracer_allocated_count_table(VALUE self)
{
    struct traceobj_arg * arg = get_traceobj_arg();
    VALUE h = rb_hash_new();
    int i;

    for (i=0; i<T_MASK; i++) {
	rb_hash_aset(h, type_sym(i), SIZET2NUM(arg->allocated_count_table[i]));
    }

    return h;
}

/*
 *
 *  call-seq:
 *     ObjectSpace::AllocationTracer.freed_count_table   -> hash
 *
 * Returns freed count totals for Ruby object types
 *
 * Returns a hash showing the number of each type of object that has been freed.
 *
 * See ObjectSpace::AllocationTracer.allocated_count_table for example usage
 *
 */
static VALUE
allocation_tracer_freed_count_table(VALUE self)
{
    struct traceobj_arg * arg = get_traceobj_arg();
    VALUE h = rb_hash_new();
    int i;

    for (i=0; i<T_MASK; i++) {
	rb_hash_aset(h, type_sym(i), SIZET2NUM(arg->freed_count_table[i]));
    }

    return h;
}

void
Init_allocation_tracer(void)
{
    VALUE rb_mObjSpace = rb_const_get(rb_cObject, rb_intern("ObjectSpace"));
    VALUE mod = rb_mAllocationTracer = rb_define_module_under(rb_mObjSpace, "AllocationTracer");

    /* allocation tracer methods */
    rb_define_module_function(mod, "trace", allocation_tracer_trace, 0);
    rb_define_module_function(mod, "start", allocation_tracer_trace, 0);
    rb_define_module_function(mod, "stop", allocation_tracer_stop, 0);
    rb_define_module_function(mod, "pause", allocation_tracer_pause, 0);
    rb_define_module_function(mod, "resume", allocation_tracer_resume, 0);

    rb_define_module_function(mod, "result", allocation_tracer_result, 0);
    rb_define_module_function(mod, "clear", allocation_tracer_clear, 0);
    rb_define_module_function(mod, "setup", allocation_tracer_setup, -1);
    rb_define_module_function(mod, "header", allocation_tracer_header, 0);

    rb_define_module_function(mod, "lifetime_table_setup", allocation_tracer_lifetime_table_setup, 1);
    rb_define_module_function(mod, "lifetime_table", allocation_tracer_lifetime_table, 0);

    rb_define_module_function(mod, "allocated_count_table", allocation_tracer_allocated_count_table, 0);
    rb_define_module_function(mod, "freed_count_table", allocation_tracer_freed_count_table, 0);
}


================================================
FILE: ext/allocation_tracer/extconf.rb
================================================
require 'mkmf'
create_makefile('allocation_tracer/allocation_tracer')


================================================
FILE: lib/allocation_tracer/trace.rb
================================================
require 'allocation_tracer'

# ObjectSpace::AllocationTracer.setup(%i{path line})
ObjectSpace::AllocationTracer.trace

at_exit{
  results = ObjectSpace::AllocationTracer.stop

  puts ObjectSpace::AllocationTracer.header.join("\t")
  results.sort_by{|k, v| k}.each{|k, v|
    puts (k+v).join("\t")
  }
}




================================================
FILE: lib/allocation_tracer/version.rb
================================================
module ObjectSpace::AllocationTracer
  VERSION = "0.6.3"
end


================================================
FILE: lib/allocation_tracer.rb
================================================
require "allocation_tracer/version"
require "allocation_tracer/allocation_tracer"

module ObjectSpace::AllocationTracer

  def self.output_lifetime_table table
    out = (file = ENV['RUBY_ALLOCATION_TRACER_LIFETIME_OUT']) ? open(File.expand_path(file), 'w') : STDOUT
    max_lines = table.inject(0){|r, (_type, lines)| r < lines.size ? lines.size : r}
    out.puts "type\t" + (0...max_lines).to_a.join("\t")
    table.each{|type, line|
      out.puts "#{type}\t#{line.join("\t")}"
    }
  end

  def self.collect_lifetime_table
    ObjectSpace::AllocationTracer.lifetime_table_setup true

    if block_given?
      begin
        ObjectSpace::AllocationTracer.trace do
          yield
        end
        result = ObjectSpace::AllocationTracer.lifetime_table
        output_lifetime_table(result)
      ensure
        ObjectSpace::AllocationTracer.lifetime_table_setup false
      end
    else
      ObjectSpace::AllocationTracer.trace
    end
  end

  def self.collect_lifetime_table_stop
    ObjectSpace::AllocationTracer.stop
    result = ObjectSpace::AllocationTracer.lifetime_table
    ObjectSpace::AllocationTracer.lifetime_table_setup false
    output_lifetime_table(result)
    result
  end
end


================================================
FILE: lib/rack/allocation_tracer.rb
================================================
#
# Rack middleware
#

require 'allocation_tracer'

module Rack
  module AllocationTracerMiddleware
    def self.new *args
      TotalTracer.new *args
    end

    class Tracer
      def initialize app
        @app = app
        @sort_order = (0..7).to_a
      end

      def allocation_trace_page result, env
        if /\As=(\d+)/ =~ env["QUERY_STRING"]
          top = $1.to_i
          @sort_order.unshift top if @sort_order.delete top
        end

        table = result.map{|(file, line, klass), (count, oldcount, total_age, min_age, max_age, memsize)|
          ["#{Rack::Utils.escape_html(file)}:#{'%04d' % line}",
            Rack::Utils.escape_html(klass ? klass.name : '<internal>'),
            count, oldcount, total_age / Float(count), min_age, max_age, memsize]
        }

        begin
          table = table.sort_by{|vs|
            ary = @sort_order.map{|i| Numeric === vs[i] ? -vs[i] : vs[i]}
          }
        rescue
          ts = []
          table.each{|*cols|
            cols.each.with_index{|c, i|
              h = (ts[i] ||= Hash.new(0))
              h[c.class] += 1
            }
          }
          return "<pre>Sorting error\n" + Rack::Utils.escape_html(h.inspect) + "</pre>"
        end

        headers = %w(path class count old_count average_age min_age max_age memsize).map.with_index{|e, i|
          "<th><a href='./?s=#{i}'>#{e}</a></th>"
        }.join("\n")
        header = "<tr>#{headers}</tr>"
        body = table.map{|cols|
          "<tr>" + cols.map{|c| "<td>#{c}</td>"}.join("\n") + "</tr>"
        }.join("\n")
        "<table>#{header}#{body}</table>"
      end

      def count_table_page count_table
        text = count_table.map{|k, v| "%-10s\t%8d" % [k, v]}.join("\n")
        "<pre>#{text}</pre>"
      end

      def allocated_count_table_page
        count_table_page ObjectSpace::AllocationTracer.allocated_count_table
      end

      def freed_count_table_page
        count_table_page ObjectSpace::AllocationTracer.freed_count_table
      end

      def lifetime_table_page
        table = []
        max_age = 0
        ObjectSpace::AllocationTracer.lifetime_table.each{|type, ages|
          max_age = [max_age, ages.size - 1].max
          table << [type, *ages]
        }
        headers = ['type', *(0..max_age)].map{|e| "<th>#{e}</th>"}.join("\n")
        body =  table.map{|cols|
          "<tr>" + cols.map{|c| "<td>#{c}</td>"}.join("\n") + "</tr>"
        }.join("\n")
        "<table border='1'><tr>#{headers}</tr>\n#{body}</table>"
      end

      def call env
        if /\A\/allocation_tracer(?:\/|$)/ =~ env["PATH_INFO"]
          result = ObjectSpace::AllocationTracer.result
          ObjectSpace::AllocationTracer.pause

          begin
            html = case env["PATH_INFO"]
                   when /lifetime_table/
                     lifetime_table_page
                   when /allocated_count_table/
                     allocated_count_table_page
                   when /freed_count_table/
                     freed_count_table_page
                   else
                     allocation_trace_page result, env
                   end
            #
            [200, {"Content-Type" => "text/html"}, [html]]
          ensure
            ObjectSpace::AllocationTracer.resume
          end
        else
          @app.call env
        end
      end
    end

    class TotalTracer < Tracer
      def initialize *args
        super
        ObjectSpace::AllocationTracer.setup %i(path line class)
        ObjectSpace::AllocationTracer.lifetime_table_setup true
        ObjectSpace::AllocationTracer.start
      end
    end
  end
end


================================================
FILE: spec/allocation_tracer_spec.rb
================================================
require 'spec_helper'
require 'tmpdir'
require 'fileutils'

describe ObjectSpace::AllocationTracer do
  describe 'ObjectSpace::AllocationTracer.trace' do
    it 'should includes allocation information' do
      line = __LINE__ + 2
      result = ObjectSpace::AllocationTracer.trace do
        Object.new
      end

      expect(result.length).to be >= 1
      expect(result[[__FILE__, line]]).to eq [1, 0, 0, 0, 0, 0]
    end

    it 'should run twice' do
      line = __LINE__ + 2
      result = ObjectSpace::AllocationTracer.trace do
        Object.new
      end
      #GC.start
      # p result
      expect(result.length).to be >= 1
      expect(result[[__FILE__, line]]).to eq [1, 0, 0, 0, 0, 0]
    end

    it 'should analyze many objects' do
      line = __LINE__ + 3
      result = ObjectSpace::AllocationTracer.trace do
        50_000.times{|i|
          i.to_s
          i.to_s
          i.to_s
        }
      end

      GC.start
      #pp result

      expect(result[[__FILE__, line + 0]][0]).to be >= 50_000
      expect(result[[__FILE__, line + 1]][0]).to be >= 50_000
      expect(result[[__FILE__, line + 2]][0]).to be >= 50_000
    end

    it 'should count old objects' do
      a = nil
      line = __LINE__ + 2
      result = ObjectSpace::AllocationTracer.trace do
        a = 'x' # it will be old object
        32.times{GC.start}
      end

      expect(result.length).to be >= 1
      _, old_count, * = *result[[__FILE__, line]]
      expect(old_count).to be == 1
    end

    it 'should acquire allocated memsize' do
      line = __LINE__ + 2
      result = ObjectSpace::AllocationTracer.trace do
        _ = 'x' * 1234 # danger
        GC.start
      end

      expect(result.length).to be >= 1
      size = result[[__FILE__, line]][-1]
      expect(size).to be > 1234 if size > 0
    end

    it 'can be paused and resumed' do
      line = __LINE__ + 2
      result = ObjectSpace::AllocationTracer.trace do
        Object.new
        ObjectSpace::AllocationTracer.pause
        Object.new # ignore tracing
        ObjectSpace::AllocationTracer.resume
        Object.new
      end

      expect(result.length).to be 2
      expect(result[[__FILE__, line    ]]).to eq [1, 0, 0, 0, 0, 0]
      expect(result[[__FILE__, line + 4]]).to eq [1, 0, 0, 0, 0, 0]
    end

    it 'can be get middle result' do
      middle_result = nil
      line = __LINE__ + 2
      result = ObjectSpace::AllocationTracer.trace do
        Object.new
        middle_result = ObjectSpace::AllocationTracer.result
        Object.new
      end

      expect(result.length).to be 2
      expect(result[[__FILE__, line    ]]).to eq [1, 0, 0, 0, 0, 0]
      expect(result[[__FILE__, line + 2]]).to eq [1, 0, 0, 0, 0, 0]

      expect(middle_result.length).to be 1
      expect(middle_result[[__FILE__, line    ]]).to eq [1, 0, 0, 0, 0, 0]
    end

    describe 'stop when not started yet' do
      it 'should raise RuntimeError' do
        expect do
          ObjectSpace::AllocationTracer.stop
        end.to raise_error(RuntimeError)
      end
    end

    describe 'pause when not started yet' do
      it 'should raise RuntimeError' do
        expect do
          ObjectSpace::AllocationTracer.pause
        end.to raise_error(RuntimeError)
      end
    end

    describe 'resume when not started yet' do
      it 'should raise RuntimeError' do
        expect do
          ObjectSpace::AllocationTracer.resume
        end.to raise_error(RuntimeError)
      end
    end

    describe 'when starting recursively' do
      it 'should raise RuntimeError' do
        expect do
          ObjectSpace::AllocationTracer.trace{
            ObjectSpace::AllocationTracer.trace{}
          }
        end.to raise_error(RuntimeError)
      end
    end

    describe 'with different setup' do
      it 'should work with type' do
        line = __LINE__ + 3
        ObjectSpace::AllocationTracer.setup(%i(path line type))
        result = ObjectSpace::AllocationTracer.trace do
          _a = [Object.new]
          _b = {Object.new => 'foo'}
        end

        expect(result.length).to be 5
        expect(result[[__FILE__, line, :T_OBJECT]]).to eq [1, 0, 0, 0, 0, 0]
        expect(result[[__FILE__, line, :T_ARRAY]]).to eq [1, 0, 0, 0, 0, 0]
        # expect(result[[__FILE__, line + 1, :T_HASH]]).to eq [1, 0, 0, 0, 0]
        expect(result[[__FILE__, line + 1, :T_OBJECT]]).to eq [1, 0, 0, 0, 0, 0]
        expect(result[[__FILE__, line + 1, :T_STRING]]).to eq [1, 0, 0, 0, 0, 0]
      end

      it 'should work with class' do
        line = __LINE__ + 3
        ObjectSpace::AllocationTracer.setup(%i(path line class))
        result = ObjectSpace::AllocationTracer.trace do
          _a = [Object.new]
          _b = {Object.new => 'foo'}
        end

        expect(result.length).to be 5
        expect(result[[__FILE__, line, Object]]).to eq [1, 0, 0, 0, 0, 0]
        expect(result[[__FILE__, line, Array]]).to eq [1, 0, 0, 0, 0, 0]
        # expect(result[[__FILE__, line + 1, Hash]]).to eq [1, 0, 0, 0, 0, 0]
        expect(result[[__FILE__, line + 1, Object]]).to eq [1, 0, 0, 0, 0, 0]
        expect(result[[__FILE__, line + 1, String]]).to eq [1, 0, 0, 0, 0, 0]
      end

      it 'should have correct headers' do
        ObjectSpace::AllocationTracer.setup(%i(path line))
        expect(ObjectSpace::AllocationTracer.header).to eq [:path, :line, :count, :old_count, :total_age, :min_age, :max_age, :total_memsize]
        ObjectSpace::AllocationTracer.setup(%i(path line class))
        expect(ObjectSpace::AllocationTracer.header).to eq [:path, :line, :class, :count, :old_count, :total_age, :min_age, :max_age, :total_memsize]
        ObjectSpace::AllocationTracer.setup(%i(path line type class))
        expect(ObjectSpace::AllocationTracer.header).to eq [:path, :line, :type, :class, :count, :old_count, :total_age, :min_age, :max_age, :total_memsize]
      end

      it 'should set default setup' do
        ObjectSpace::AllocationTracer.setup()
        expect(ObjectSpace::AllocationTracer.header).to eq [:path, :line, :count, :old_count, :total_age, :min_age, :max_age, :total_memsize]
      end
    end
  end

  describe 'collect lifetime_table' do
    before do
      ObjectSpace::AllocationTracer.lifetime_table_setup true
    end

    after do
      ObjectSpace::AllocationTracer.lifetime_table_setup false
    end

    it 'should make lifetime table' do
      ObjectSpace::AllocationTracer.trace do
        100000.times{
          Object.new
          ''
        }
      end
      table = ObjectSpace::AllocationTracer.lifetime_table

      expect(table[:T_OBJECT].inject(&:+)).to be >= 10_000
      expect(table[:T_STRING].inject(&:+)).to be >= 10_000
      expect(table[:T_NONE]).to be nil
    end

    it 'should return nil when ObjectSpace::AllocationTracer.lifetime_table_setup is false' do
      ObjectSpace::AllocationTracer.lifetime_table_setup false

      ObjectSpace::AllocationTracer.trace do
        100000.times{
          Object.new
          ''
        }
      end

      table = ObjectSpace::AllocationTracer.lifetime_table

      expect(table).to be nil
    end

    it 'should return nil getting it twice' do
      ObjectSpace::AllocationTracer.trace do
        100000.times{
          Object.new
          ''
        }
      end

      table = ObjectSpace::AllocationTracer.lifetime_table
      table = ObjectSpace::AllocationTracer.lifetime_table

      expect(table).to be nil
    end
  end

  describe 'ObjectSpace::AllocationTracer.collect_lifetime_table' do
    it 'should collect lifetime table' do
      table = ObjectSpace::AllocationTracer.collect_lifetime_table do
        100000.times{
          Object.new
          ''
        }
      end

      expect(table[:T_OBJECT].inject(&:+)).to be >= 10_000
      expect(table[:T_STRING].inject(&:+)).to be >= 10_000
      expect(table[:T_NONE]).to be nil
    end
  end

  describe 'ObjectSpace::AllocationTracer.allocated_count_table' do
    it 'should return a Hash object' do
      h = ObjectSpace::AllocationTracer.allocated_count_table
      expect(h[:T_NONE]).to be 0
    end
  end

  describe 'ObjectSpace::AllocationTracer.freed_count_table' do
    it 'should return a Hash object' do
      h = ObjectSpace::AllocationTracer.freed_count_table
      expect(h[:T_NONE]).to be 0
    end
  end
end


================================================
FILE: spec/spec_helper.rb
================================================
require 'rubygems'
require 'bundler/setup'
require 'allocation_tracer'

RSpec.configure do |config|
  config.mock_framework = :rspec
end