Showing preview only (224K chars total). Download the full file or copy to clipboard to get everything.
Repository: ice799/memprof
Branch: master
Commit: 23e8bac0bf80
Files: 43
Total size: 212.5 KB
Directory structure:
gitextract_chiyp8m1/
├── .gitignore
├── LICENSE
├── README.md
├── Rakefile
├── bin/
│ └── memprof
├── ext/
│ ├── arch.h
│ ├── bin_api.h
│ ├── elf.c
│ ├── extconf.rb
│ ├── i386.c
│ ├── i386.h
│ ├── json.c
│ ├── json.h
│ ├── mach.c
│ ├── memprof.c
│ ├── mmap.h
│ ├── tracer.c
│ ├── tracer.h
│ ├── tracers/
│ │ ├── fd.c
│ │ ├── gc.c
│ │ ├── memcache.c
│ │ ├── memory.c
│ │ ├── mysql.c
│ │ ├── objects.c
│ │ ├── postgres.c
│ │ ├── resources.c
│ │ ├── sql.c
│ │ └── sql.h
│ ├── tramp.c
│ ├── tramp.h
│ ├── util.c
│ ├── util.h
│ ├── x86_64.c
│ ├── x86_64.h
│ ├── x86_gen.c
│ └── x86_gen.h
├── lib/
│ └── memprof/
│ ├── middleware.rb
│ ├── signal.rb
│ └── tracer.rb
├── memprof.gemspec
└── spec/
├── memprof_spec.rb
├── memprof_uploader_spec.rb
└── tracing_spec.rb
================================================
FILE CONTENTS
================================================
================================================
FILE: .gitignore
================================================
*.o
*.so
*.bundle
Makefile
*.dSYM
*.log
*.gem
ext/dst/*
ext/src/yajl-1.0.9/*
================================================
FILE: LICENSE
================================================
Copyright (c) 2009-2015 Joe Damato, Aman Gupta, and Jake Douglas
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
================================================
FILE: README.md
================================================
# NOTE
This project is no longer maintained. It does not work for any Ruby version
above Ruby 1.8.7.
# memprof
(c) Joe Damato
@joedamato
http://timetobleed.com
Memprof is a Ruby level memory profiler that can help you find reference
leaks in your application.
Memprof can also do very lightweight function call tracing to figure out
which system and library calls are happening in your code.
# Installation
gem install memprof
# API
## Memprof.stats
Memprof.start
12.times{ "abc" }
Memprof.stats
Memprof.stop
Start tracking file/line information for objects created after calling
`Memprof.start`, and print out a summary of file:line/class pairs
created.
12 file.rb:2:String
*Note*: Call `Memprof.stats` again after `GC.start` to see which objects
are cleaned up by the garbage collector:
Memprof.start
10.times{ $last_str = "abc" }
puts '=== Before GC'
Memprof.stats
puts '=== After GC'
GC.start
Memprof.stats
Memprof.stop
After `GC.start`, only the very last instance of `"abc"` will still
exist:
=== Before GC
10 file.rb:2:String
=== After GC
1 file.rb:2:String
*Note*: Use `Memprof.stats("/path/to/file")` to write results to a file.
*Note*: Use `Memprof.stats!` to clear out tracking data after printing
out results.
## Memprof.track
Simple wrapper for `Memprof.stats` that will start/stop memprof around a
given block of ruby code.
Memprof.track{
100.times{ "abc" }
100.times{ 1.23 + 1 }
100.times{ Module.new }
}
For the block of ruby code, print out file:line/class pairs for
ruby objects created.
100 file.rb:2:String
100 file.rb:3:Float
100 file.rb:4:Module
*Note*: You can call GC.start at the end of the block to print out only
objects that are 'leaking' (i.e. objects that still have inbound
references).
*Note*: Use `Memprof.track("/path/to/file")` to write the results to a
file instead of stdout.
## Memprof.dump
Memprof.dump{
"hello" + "world"
}
Dump out all objects created in a given ruby block as detailed json
objects.
{
"_id": "0x15e5018",
"file": "file.rb",
"line": 2,
"type": "string",
"class_name": "String",
"length": 10,
"data": "helloworld"
}
*Note*: Use `Memprof.dump("/path/to/filename")` to write the json output
to a file, one per line.
## Memprof.dump_all
Memprof.dump_all("myapp_heap.json")
Dump out all live objects inside the Ruby VM to `myapp_heap.json`, one
per line.
### [memprof.com](http://memprof.com) heap visualizer
# load memprof before requiring rubygems, so objects created by
# rubygems itself are tracked by memprof too
require `gem which memprof/signal`.strip
require 'rubygems'
require 'myapp'
Installs a `URG` signal handler and starts tracking file/line
information for newly created ruby objects. When the process receives
`SIGURG`, it will fork and call `Memprof.dump_all` to write out the
entire heap to a json file.
Use the `memprof` command to send the signal and upload the heap to
[memprof.com](http://memprof.com):
memprof --pid <PID> --name my_leaky_app --key <API_KEY>
## Memprof.trace
require 'open-uri'
require 'mysql'
require 'memcached'
Memprof.trace{
10.times{ Module.new }
10.times{ GC.start }
10.times{ open('http://google.com/') }
10.times{ Mysql.connect.query("select 1+2") }
10.times{ Memcached.new.get('memprof') }
}
For a given block of ruby code, count:
- number of objects created per type
- number of calls to and time spent in GC
- number of calls to and time spent in connect/read/write/select
- number of calls to and time spent in mysql queries
- number of calls to and responses to memcached commands
- number of calls to and bytes through malloc/realloc/free
The resulting json report looks like:
{
"objects": {
"created": 10,
"types": {
"module": 10, # Module.new
}
},
"gc": {
"calls": 10, # GC.start
"time": 0.17198
},
"fd": {
"connect": {
"calls": 10, # open('http://google.com')
"time": 0.0110
}
},
"mysql": {
"queries": 10, # Mysql.connect.query("select 1+2")
"time": 0.0006
},
"memcache": {
"get": {
"calls": 10, # Memcached.new.get('memprof')
"responses": {
"notfound": 10
}
}
}
}
*Note*: To write json to a file instead, set `Memprof.trace_filename =
"/path/to/file.json"`
## Memprof.trace_request
Memprof.trace_request(env){ @app.call(env) }
Like `Memprof.trace`, but assume an incoming Rack request and include
information about the request itself.
{
"start" : 1272424769750716,
"tracers" : {
/* ... */
},
"rails" : {
"controller" : "home",
"action" : "index"
},
"request" : {
"REQUEST_URI" : "/home",
"REQUEST_METHOD" : "GET",
"REMOTE_ADDR" : "127.0.0.1",
"QUERY_STRING" : null
},
"time" : 1.3442
}
# Middlewares
## Memprof::Middleware
require 'memprof/middleware'
config.middlewares.use(Memprof::Middleware)
Wrap each request in a `Memprof.track` to print out all object
location/type pairs created during that request.
*Note*: It is preferable to run this in staging or production mode with
Rails applications, since development mode creates a lot of unnecessary
objects during each request.
*Note*: To force a GC run before printing out a report, pass in
`:force_gc => true` to the middleware.
## Memprof::Tracer
require 'memprof/tracer'
config.middleware.insert(0, Memprof::Tracer)
Wrap each request in a `Memprof.trace_request` and write results to
`/tmp/memprof_tracer-PID.json`
## Memprof::Filter
Similar to `Memprof::Tracer`, but for legacy Rails 2.2 applications.
class ApplicationController < ActionController::Base
require 'memprof/tracer'
around_filter(Memprof::Filter)
end
# Compatibility
Memprof supports all 1.8.x (MRI and REE) VMs, as long as they are 64-bit
and contain debugging symbols. For best results, use RVM to compile ruby
and make sure you are on a 64-bit machine.
The following ruby builds are not supported:
- Ruby on small/medium EC2 instances (32-bit machines)
- OSX's default system ruby (no debugging symbols, fat 32/64-bit
binary)
*Note*: Many linux distributions do not package debugging symbols by
default. You can usually install these separately, for example using
`apt-get install libruby1.8-dbg`
## Coming soon
- support for Ruby 1.9
- support for i386/i686 ruby builds
# Credits
- Jake Douglas for the Mach-O Snow Leopard support
- Aman Gupta for various bug fixes and other cleanup
- Rob Benson for initial 1.9 support and cleanup
- Paul Barry for force_gc support in `Memprof::Middleware`
================================================
FILE: Rakefile
================================================
task :spec do
Dir.chdir('ext') do
sh "make clean" rescue nil
sh "ruby extconf.rb"
sh "make"
end
sh "ruby spec/memprof_spec.rb"
end
task :default => :spec
# Should be used like:
# rake --trace ci[1.8.7,shared]
task :ci, [:ruby_type, :lib_option] do |t, args|
ruby_type, lib_option = args[:ruby_type], args[:lib_option]
raise "#{ruby_type} is not a supported ruby version" unless ["1.8.6", "1.8.7", "ree"].include?(ruby_type)
raise "#{lib_option} is not a supported " unless ["shared", "static"].include?(lib_option)
lib_option = case lib_option
when "static"
"--disable-shared"
when "shared"
"--enable-shared"
end
sh "/usr/bin/env bash -c \"
source ~/.rvm/scripts/rvm &&
rvm install #{ruby_type} --reconfigure -C #{lib_option} &&
rvm #{ruby_type} --symlink memprof &&
memprof_gem install bacon\""
Dir.chdir('ext') do
sh '/usr/bin/env bash -c "make clean"' rescue nil
sh "~/.rvm/bin/memprof_ruby extconf.rb"
sh '/usr/bin/env bash -c "make"'
end
sh "~/.rvm/bin/memprof_ruby spec/memprof_spec.rb"
end
================================================
FILE: bin/memprof
================================================
#!/usr/bin/env ruby
require 'rubygems'
require 'optparse'
require 'restclient'
require 'term/ansicolor'
MEMPROF_URL = "https://memprof.com/upload"
MEMPROF_BANNER = <<-EOF
Memprof Uploader
http://www.memprof.com
======================
EOF
class MemprofUploader
include Term::ANSIColor
def initialize(args=ARGV)
puts MEMPROF_BANNER
@parser = OptionParser.new do |opts|
opts.banner = "Usage:"
opts.on("-p", "--pid <pid>", Integer, "PID of the process to dump (required)") {|arg| @pid = arg }
opts.on("-n", "--name <name>", "Name for your dump (required)") {|arg| @name = arg }
opts.on("-k", "--key <key>", "Memprof.com API key (required)") {|arg| @key = arg }
opts.on("-d", "--[no-]delete", "Delete dump file after uploading (default true)") {|arg| @delete_dump = arg }
opts.on("-s", "--seconds <seconds>",
Integer, "Seconds to wait for the dump (default 300)") {|arg| @secs = arg }
opts.on("-t", "--[no-]test", "Test run (don't actually upload) (default false)") {|arg| @test = arg }
opts.on("-f", "--file <path>", "Upload specific json dump (optional)") {|arg| @file = arg }
opts.on("--put-my-data-on-the-internet", "Confirm that you understand\n" +
"memprof.com will show all your \n".rjust(80) +
"internal data on the internet (required)".rjust(80)) {|arg| @confirmed = true}
opts.on("--info") do
require 'rbconfig'
puts RUBY_DESCRIPTION if defined? RUBY_DESCRIPTION
puts "CFLAGS='#{Config::CONFIG["CFLAGS"]}' ./configure #{Config::CONFIG["configure_args"]}"
bin = "#{Config::CONFIG['bindir']}/#{Config::CONFIG['ruby_install_name']}"
puts `file #{bin}`
if RUBY_PLATFORM =~ /darwin/
puts `otool -L #{bin}`
else
puts `ldd #{bin}`
end
puts
exit!
end
end
begin
@parser.parse!
rescue Exception => e
if e.kind_of?(SystemExit)
raise e
else
fail_with(e.message)
end
end
# Make this default to true if the user didn't pass any preference.
@delete_dump = true if @delete_dump.nil?
# Make this default to 60 if the user didn't pass the number of seconds.
@secs ||= 300
if @file
fail_with("File not found: #{@file}") unless File.exists?(@file)
end
if @pid.nil? and @file.nil?
fail_with("Missing PID! (-p PID)")
elsif @name.nil? || @name.empty?
fail_with("Missing name! (-n 'my application')")
elsif @key.nil? || @key.empty?
fail_with("Missing API key! (-k KEY)")
elsif !@confirmed
fail_with("\nERROR: You MUST CONFIRM that you understand ALL YOUR CODE "+
"WILL BE PUBLICLY \nAVAILABLE ON THE INTERNET by passing " +
"--put-my-data-on-the-internet", true)
end
end
def run!
dump_filename = @file ? compress_file(@file, "/tmp/#{File.basename @file}.gz") : compress_file(get_dump_filename(@pid))
begin
upload_dump(dump_filename, @name, @key)
ensure
if @delete_dump
File.delete(dump_filename)
puts "\nDeleted dump file."
end
end
puts "Finished!"
end
private
def compress_file(filename, output=nil)
puts "Compressing with gzip..."
output ||= "#{filename}.gz"
`gzip -c '#{filename}' > '#{output}'`
output
end
def get_dump_filename(pid)
# Get a listing of files before we signal for the new dump
# We'll compare against this later to see which is the new file.
old_files = Dir.glob("/tmp/memprof-#{pid}-*")
signal_process(pid)
timeout = 30
puts "Waiting #{timeout} seconds for process #{pid} to create a new dump..."
file = nil
timeout.times do |i|
sleep 1 unless file = (Dir.glob("/tmp/memprof-#{pid}-*") - old_files).first and break
fail_with("Timed out after waiting #{timeout} seconds. Make sure you added require '#{File.expand_path('../../lib/memprof/signal', __FILE__)}' to your application.") if i+1 == timeout
end
puts "\nFound file #{file}"
if file =~ /\.IN_PROGRESS/
file = file.sub(/\.IN_PROGRESS/, "")
puts "Dump in progress. Waiting #{@secs} seconds for it to complete..."
@secs.times do |i|
sleep 1 unless File.exist?(file) and break
fail_with("Timed out after waiting #{@secs} seconds") if i+1 == @secs
end
end
file
end
def upload_dump(filename, name, key)
return if @test
file = File.open(filename, "r")
puts "\nUploading to memprof.com..."
response = RestClient.post(MEMPROF_URL, :upload => file, :name => name, :key => key)
puts "Response from server:"
p response.to_s
end
def fail_with(str, yell=true)
puts @parser.to_s
puts
if yell
print red, bold, str, reset
puts
else
puts "\n" + str
end
puts
exit(1)
end
def signal_process(pid)
begin
Process.kill("URG", pid)
rescue Errno::ESRCH
fail_with("No such process #{pid}!")
end
puts "Signaled process #{pid} with SIGURG"
end
end
MemprofUploader.new(ARGV).run!
================================================
FILE: ext/arch.h
================================================
#if !defined (_ARCH_H_)
#define _ARCH_H_
/*
* The only supported architectures at this time are i{3-6}86 and amd64. All
* other architectures should fail.
*/
#if defined(_ARCH_i386_) || defined(_ARCH_i686_)
#include "i386.h"
#elif defined(_ARCH_x86_64_)
#include "x86_64.h"
#else
#error "Unsupported architecture! Cannot continue compilation."
#endif
#include "x86_gen.h"
/*
* arch_get_st2_tramp - architecture specific stage 2 trampoline getter
*
* This function will return a pointer to an area of memory that contains
* the stage 2 trampoline for this architecture.
*
* This function will also set the out parameter size equal to the size of this
* region, if size is not NULL.
*/
void *
arch_get_st2_tramp(size_t *size);
/*
* arch_insert_st1_tramp - architecture specific stage 1 trampoline insert
*
* Given:
* start - a start address to attempt to insert a trampoline at
* trampee - the address of the function to trampoline
* tramp - a pointer to the trampoline
*
* This function will inspect the start address to determine if an instruction
* which transfers execution to the trampee is present. If so, this function
* will rewrite the instruction to ensure that tramp is called instead.
*
* Returns 0 on success, 1 otherwise.
*/
int
arch_insert_st1_tramp(void *start, void *trampee, void *tramp);
/*
* arch_get_inline_st2_tramp - architecture specific inline stage 2 tramp getter
*
* This function will return a pointer to an area of memory that contains the
* stage 2 inline trampoline for this architecture.
*
* This function will also set the out parameter size equal to the size of this
* region, if size is not NULL.
*/
void *
arch_get_inline_st2_tramp(size_t *size);
/*
* Given:
* - addr - The base address of an instruction sequence.
*
* - marker - This is the marker to search for which will indicate that the
* instruction sequence has been located.
*
* - trampoline - The address of the handler to redirect execution to.
*
* - table_entry - Address of where the stage 2 trampoline code will reside
*
* This function will:
* Insert and setup the stage 1 and stage 2 trampolines if addr points to an
* instruction that could be from the inlined add_freelist function.
*
* This function returns 1 on failure and 0 on success.
*/
int
arch_insert_inline_st2_tramp(void *addr, void *marker, void *trampoline, void *table_entry);
#endif
================================================
FILE: ext/bin_api.h
================================================
#if !defined(BIN_API__)
#define BIN_API__
#include <stddef.h>
#include <stdint.h>
/*
* Some useful foward declarations for function protoypes here and elsewhere.
*/
struct tramp_st2_entry;
struct inline_tramp_st2_entry;
/*
* bin_init - Initialize the binary format specific layer.
*/
void
bin_init();
/*
* bin_find_symbol - find a symbol
*
* Given:
* - sym - a symbol name
* - size - optional out parameter
* - search_libs - 0 to search only ruby/libruby, 1 to search other
* libraries, too.
*
* This function will search for the symbol sym and return its address if
* found, or NULL if the symbol could not be found.
*
* Optionally, this function will set its out parameter size equal to the
* size of the symbol.
*/
void *
bin_find_symbol(const char *sym, size_t *size, int search_libs);
/*
* bin_find_symbol_name - find a symbol's name
*
* Given:
* - sym - a symbol address
*
* This function will search for the symbol sym and return its name if
* found, or NULL if the symbol could not be found.
*/
const char *
bin_find_symbol_name(void *sym);
/*
* bin_allocate_page - allocate a page suitable for trampolines
*
* This function will allocate a page of memory in the right area of the
* virtual address space and with the appropriate permissions for stage 2
* trampoline code to live and execute.
*/
void *
bin_allocate_page(void);
/*
* bin_type_size - Return size (in bytes) of a given type.
*
* Given:
* - type - a string representation of a type
*
* This function will return the size (in bytes) of the type. If no such type
* can be found, return 0.
*/
size_t
bin_type_size(const char *type);
/*
* bin_type_member_offset - Return the offset (in bytes) of member in type.
*
* Given:
* - type - a string representation of a type
* - member - a string representation of a field int he type
*
* This function will return the offset (in bytes) of member in type.
*
* On failure, this function returns -1.
*/
int
bin_type_member_offset(const char *type, const char *member);
/*
* bin_update_image - Update a binary image in memory
*
* Given:
* - trampee - the name of the symbol to hook
* - tramp - the stage 2 trampoline entry
* - orig_func - out parameter storing the address of the function that was
* originally being called.
*
* this function will update the binary image so that all calls to trampee will
* be routed to tramp.
*
* Returns 0 on success.
*/
int
bin_update_image(const char *trampee, struct tramp_st2_entry *tramp, void **orig_func);
#endif
================================================
FILE: ext/elf.c
================================================
#if defined(HAVE_ELF)
#define _GNU_SOURCE
#include "bin_api.h"
#include "arch.h"
#include "util.h"
#include <assert.h>
#include <dwarf.h>
#include <err.h>
#include <error.h>
#include <errno.h>
#include <fcntl.h>
#include <libdwarf.h>
#include <libelf/gelf.h>
#include <libgen.h>
#include <limits.h>
#include <link.h>
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <sysexits.h>
#include <unistd.h>
#include <sys/mman.h>
#include <sys/stat.h>
#include <sys/types.h>
extern struct memprof_config memprof_config;
/* The size of a PLT entry */
#define PLT_ENTRY_SZ (16)
/* The system-wide debug symbol directory */
/* XXX this should be set via extconf and not hardcoded */
#define DEBUGDIR "/usr/lib/debug"
/* Keep track of whether this ruby is built with a shared library or not */
static int libruby = 0;
static Dwarf_Debug dwrf = NULL;
/* Set of ELF specific state about this Ruby binary/shared object */
static struct elf_info *ruby_info = NULL;
struct elf_info {
int fd;
Elf *elf;
GElf_Addr base_addr;
void *text_segment;
size_t text_segment_len;
GElf_Addr got_addr;
GElf_Addr relplt_addr;
Elf_Data *relplt;
size_t relplt_count;
GElf_Addr plt_addr;
Elf_Data *plt;
size_t plt_size;
size_t plt_count;
Elf_Data *debuglink_data;
GElf_Ehdr ehdr;
Elf_Data *dynsym;
size_t dynsym_count;
const char *dynstr;
GElf_Shdr symtab_shdr;
Elf_Data *symtab_data;
const char *filename;
struct elf_info *debug_data;
};
/* These callback return statuses are used to tell the invoker of the callback
* (walk_linkmap) whether to continue walking or bail out.
*/
typedef enum {
CB_CONTINUE,
CB_EXIT,
} linkmap_cb_status;
/* A callback type that specifies a function that can be fired on each link map
* entry.
*/
typedef linkmap_cb_status (*linkmap_cb)(struct link_map *, void *);
typedef void (*linkmap_lib_cb)(struct elf_info *lib, void *data);
static void open_elf(struct elf_info *info);
static int dissect_elf(struct elf_info *info, int find_debug);
static void *find_plt_addr(const char *symname, struct elf_info *info);
static void walk_linkmap(linkmap_cb cb, void *data);
/*
* plt_entry - procedure linkage table entry
*
* This struct is intended to be "laid onto" a piece of memory to ease the
* parsing, use, modification, and length calculation of PLT entries.
*
* For example:
* jmpq *0xaaf00d(%rip) # jump to GOT entry
*
* # the following instructions are only hit if function has not been
* # resolved previously.
*
* pushq $0x4e # push ID
* jmpq 0xcafebabefeedface # invoke the link linker
*/
struct plt_entry {
unsigned char jmp[2];
int32_t jmp_disp;
/* There is no reason (currently) to represent the pushq and jmpq
* instructions which invoke the linker.
*
* We don't need those to hook the GOT; we only need the jmp_disp above.
*
* TODO represent the extra instructions
*/
unsigned char pad[10];
} __attribute__((__packed__));
/*
* get_got_addr - given a PLT entry, return the global offset table entry that
* the entry uses.
*/
static void *
get_got_addr(struct plt_entry *plt, struct elf_info *info)
{
void *addr = NULL;
assert(plt != NULL);
assert(plt->jmp[0] == 0xff);
if (plt->jmp[1] == 0x25) {
#if defined(_ARCH_x86_64_)
// jmpq *0x2ccf3a(%rip)
addr = (void *)&(plt->pad) + plt->jmp_disp;
#else
// jmp *0x81060f0
addr = (void *)(plt->jmp_disp);
#endif
} else if (plt->jmp[1] == 0xa3) {
// jmp *0x130(%ebx)
addr = (void *)(info->base_addr + info->got_addr + plt->jmp_disp);
}
dbg_printf("PLT addr: %p, .got.plt slot: %p\n", plt, addr);
return addr;
}
/*
* overwrite_got - given the address of a PLT entry, overwrite the address
* in the GOT that the PLT entry uses with the address in tramp.
*
* returns the original function address
*/
static void *
overwrite_got(void *plt, const void *tramp, struct elf_info *info)
{
assert(plt != NULL);
assert(tramp != NULL);
void *ret = NULL;
memcpy(&ret, get_got_addr(plt, info), sizeof(void *));
copy_instructions(get_got_addr(plt, info), &tramp, sizeof(void *));
dbg_printf("GOT value overwritten to: %p, from: %p\n", tramp, ret);
return ret;
}
struct plt_hook_data {
const char *sym;
void *addr;
};
struct dso_iter_data {
linkmap_lib_cb cb;
void *passthru;
};
static linkmap_cb_status
for_each_dso_cb(struct link_map *map, void *data)
{
struct dso_iter_data *iter_data = data;
struct elf_info curr_lib;
/* skip a few things we don't care about */
if (strstr(map->l_name, "linux-vdso")) {
dbg_printf("found vdso (skipping): %s\n", map->l_name);
return CB_CONTINUE;
} else if (strstr(map->l_name, "ld-linux")) {
dbg_printf("found ld-linux (skipping): %s\n", map->l_name);
return CB_CONTINUE;
} else if (strstr(map->l_name, "libruby")) {
dbg_printf("found libruby (skipping): %s\n", map->l_name);
return CB_CONTINUE;
} else if (strstr(map->l_name, "memprof")) {
dbg_printf("found memprof (skipping): %s\n", map->l_name);
return CB_CONTINUE;
} else if (!map->l_name || map->l_name[0] == '\0') {
dbg_printf("found an empty string (skipping)\n");
return CB_CONTINUE;
}
memset(&curr_lib, 0, sizeof(curr_lib));
dbg_printf("trying to open elf object: %s\n", map->l_name);
curr_lib.filename = map->l_name;
open_elf(&curr_lib);
if (curr_lib.elf == NULL) {
dbg_printf("opening the elf object (%s) failed! (skipping)\n", map->l_name);
return CB_CONTINUE;
}
curr_lib.base_addr = map->l_addr;
curr_lib.filename = map->l_name;
if (dissect_elf(&curr_lib, 0) == 2) {
dbg_printf("elf file, %s hit an unrecoverable error (skipping)\n", map->l_name);
elf_end(curr_lib.elf);
close(curr_lib.fd);
return CB_CONTINUE;
}
dbg_printf("dissected the elf file: %s, base: %lx\n",
curr_lib.filename, (unsigned long)curr_lib.base_addr);
iter_data->cb(&curr_lib, iter_data->passthru);
elf_end(curr_lib.elf);
close(curr_lib.fd);
return CB_CONTINUE;
}
static void
for_each_dso(linkmap_lib_cb cb, void *passthru)
{
struct dso_iter_data data;
data.cb = cb;
data.passthru = passthru;
walk_linkmap(for_each_dso_cb, &data);
}
static void
hook_required_objects(struct elf_info *info, void *data)
{
assert(info != NULL);
struct plt_hook_data *hook_data = data;
void *trampee_addr = NULL;
if ((trampee_addr = find_plt_addr(hook_data->sym, info)) != NULL) {
dbg_printf("found: %s @ %p\n", hook_data->sym, trampee_addr);
overwrite_got(trampee_addr, hook_data->addr, info);
}
return;
}
/*
* do_bin_allocate_page - internal page allocation routine
*
* This function allocates a page suitable for stage 2 trampolines. This page
* is allocated based on the location of the text segment of the Ruby binary
* or libruby.
*
* The page has to be located in a 32bit window from the Ruby code so that
* jump and call instructions can redirect execution there.
*
* This function returns the address of the page found or NULL if no page was
* found.
*/
static void *
do_bin_allocate_page(struct elf_info *info)
{
void * ret = NULL, *addr = NULL;
uint32_t count = 0;
if (!info)
return NULL;
if (libruby) {
/* There is a libruby. Start at the end of the text segment and search for
* a page.
*/
addr = info->text_segment + info->text_segment_len;
for (; count < USHRT_MAX; addr += memprof_config.pagesize, count += memprof_config.pagesize) {
ret = mmap(addr, memprof_config.pagesize, PROT_WRITE|PROT_READ|PROT_EXEC, MAP_ANON|MAP_PRIVATE, -1, 0);
if (ret != MAP_FAILED) {
memset(ret, 0x90, memprof_config.pagesize);
return ret;
}
}
} else {
/* if there is no libruby, use the linux specific MAP_32BIT flag which will
* grab a page in the lower 4gb of the address space.
*/
assert((size_t)info->text_segment <= UINT_MAX);
#ifndef MAP_32BIT
#define MAP_32BIT 0 // no MAP_32BIT defined on certain 32bit systems
#endif
return mmap(NULL, memprof_config.pagesize, PROT_WRITE|PROT_READ|PROT_EXEC, MAP_ANON|MAP_PRIVATE|MAP_32BIT, -1, 0);
}
return NULL;
}
/*
* bin_allocate_page - allocate a page suitable for holding stage 2 trampolines
*
* This function is just a wrapper which passes through some internal state.
*/
void *
bin_allocate_page()
{
return do_bin_allocate_page(ruby_info);
}
/*
* get_plt_addr - architecture specific PLT entry retrieval
*
* Given the internal data and an index, this function returns the address of
* the PLT entry at that address.
*
* A PLT entry takes the form:
*
* jmpq *0xfeedface(%rip)
* pushq $0xaa
* jmpq 17110
*/
static inline GElf_Addr
get_plt_addr(struct elf_info *info, size_t ndx) {
assert(info != NULL);
dbg_printf("file: %s, base: %lx\n", info->filename, (unsigned long)info->base_addr);
return info->base_addr + info->plt_addr + (ndx + 1) * PLT_ENTRY_SZ;
}
/*
* find_got_addr - find the global offset table entry for specific symbol name.
*
* Given:
* - syname - the symbol name
* - info - internal information about the ELF object to search
*
* This function searches the .rela.plt section of an ELF binary, searcing for
* entries that match the symbol name passed in. If one is found, the address
* of corresponding entry in .plt is returned.
*/
static void *
find_plt_addr(const char *symname, struct elf_info *info)
{
assert(symname != NULL);
size_t i = 0;
if (info == NULL) {
info = ruby_info;
}
assert(info != NULL);
/* Search through each of the .rela.plt entries */
for (i = 0; i < info->relplt_count; i++) {
GElf_Rel rel;
GElf_Rela rela;
GElf_Sym sym;
GElf_Addr addr;
void *ret = NULL;
const char *name;
if (info->relplt->d_type == ELF_T_RELA) {
ret = gelf_getrela(info->relplt, i, &rela);
if (ret == NULL
|| ELF64_R_SYM(rela.r_info) >= info->dynsym_count
|| gelf_getsym(info->dynsym, ELF64_R_SYM(rela.r_info), &sym) == NULL)
continue;
} else if (info->relplt->d_type == ELF_T_REL) {
ret = gelf_getrel(info->relplt, i, &rel);
if (ret == NULL
|| ELF64_R_SYM(rel.r_info) >= info->dynsym_count
|| gelf_getsym(info->dynsym, ELF64_R_SYM(rel.r_info), &sym) == NULL)
continue;
} else {
dbg_printf("unknown relplt entry type: %d\n", info->relplt->d_type);
continue;
}
name = info->dynstr + sym.st_name;
/* The name matches the name of the symbol passed in, so get the PLT entry
* address and return it.
*/
if (strcmp(symname, name) == 0) {
addr = get_plt_addr(info, i);
return (void *)addr;
}
}
return NULL;
}
/*
* do_bin_find_symbol - internal symbol lookup function.
*
* Given:
* - sym - the symbol name to look up
* - size - an optional out argument holding the size of the symbol
* - elf - an elf information structure
*
* This function will return the address of the symbol (setting size if desired)
* or NULL if nothing can be found.
*/
static void *
do_bin_find_symbol(const char *sym, size_t *size, struct elf_info *elf)
{
const char *name = NULL;
assert(sym != NULL);
assert(elf != NULL);
ElfW(Sym) *esym, *lastsym = NULL;
if (elf->symtab_data && elf->symtab_data->d_buf) {
dbg_printf("checking symtab....\n");
esym = (ElfW(Sym)*) elf->symtab_data->d_buf;
lastsym = (ElfW(Sym)*) ((char*) elf->symtab_data->d_buf + elf->symtab_data->d_size);
assert(esym <= lastsym);
for (; esym < lastsym; esym++){
/* ignore numeric/empty symbols */
if ((esym->st_value == 0) ||
(ELF32_ST_BIND(esym->st_info)== STB_NUM))
continue;
name = elf_strptr(elf->elf, elf->symtab_shdr.sh_link, (size_t)esym->st_name);
if (name && strcmp(name, sym) == 0) {
if (size) {
*size = esym->st_size;
}
dbg_printf("Found symbol: %s in symtab\n", sym);
return elf->base_addr + (void *)esym->st_value;
}
}
}
if (elf->dynsym && elf->dynsym->d_buf) {
dbg_printf("checking dynsymtab....\n");
esym = (ElfW(Sym) *) elf->dynsym->d_buf;
lastsym = (ElfW(Sym) *) ((char *) elf->dynsym->d_buf + elf->dynsym->d_size);
for (; esym < lastsym; esym++){
/* ignore numeric/empty symbols */
if ((esym->st_value == 0) ||
(elf->dynstr == 0) ||
(ELF32_ST_BIND(esym->st_info)== STB_NUM))
continue;
name = elf->dynstr + esym->st_name;
if (name && strcmp(name, sym) == 0) {
if (size) {
*size = esym->st_size;
}
dbg_printf("Found symbol: %s in dynsym\n", sym);
return elf->base_addr + (void *)esym->st_value;
}
}
}
dbg_printf("Couldn't find symbol: %s in dynsym\n", sym);
return NULL;
}
static void
find_symbol_cb(struct elf_info *info, void *data)
{
assert(info != NULL);
void *trampee_addr = NULL;
struct plt_hook_data *hook_data = data;
void *ret = do_bin_find_symbol(hook_data->sym, NULL, info);
if (ret) {
hook_data->addr = ret;
dbg_printf("found %s @ %p, fn addr: %p\n", hook_data->sym, trampee_addr,
hook_data->addr);
}
}
/*
* bin_find_symbol - find the address of a given symbol and set its size if
* desired.
*
* This function is just a wrapper for the internal symbol lookup function.
*/
void *
bin_find_symbol(const char *sym, size_t *size, int search_libs)
{
void *ret = do_bin_find_symbol(sym, size, ruby_info);
if (!ret && search_libs) {
dbg_printf("Didn't find symbol: %s in ruby, searching other libs\n", sym);
struct plt_hook_data hd;
hd.sym = sym;
hd.addr = NULL;
for_each_dso(find_symbol_cb, &hd);
ret = hd.addr;
}
return ret;
}
/*
* do_bin_find_symbol_name - internal symbol name lookup function.
*
* Given:
* - sym - the symbol address to look up
* - elf - an elf information structure
*
* This function will return the name of the symbol
* or NULL if nothing can be found.
*/
static const char *
do_bin_find_symbol_name(void *sym, struct elf_info *elf)
{
char *name = NULL;
void *ptr;
assert(sym != NULL);
assert(elf != NULL);
assert(elf->symtab_data != NULL);
assert(elf->symtab_data->d_buf != NULL);
ElfW(Sym) *esym = (ElfW(Sym)*) elf->symtab_data->d_buf;
ElfW(Sym) *lastsym = (ElfW(Sym)*) ((char*) elf->symtab_data->d_buf + elf->symtab_data->d_size);
assert(esym <= lastsym);
for (; esym < lastsym; esym++){
/* ignore weak/numeric/empty symbols */
if ((esym->st_value == 0) ||
(ELF32_ST_BIND(esym->st_info)== STB_WEAK) ||
(ELF32_ST_BIND(esym->st_info)== STB_NUM))
continue;
ptr = elf->base_addr + (void *)esym->st_value;
name = elf_strptr(elf->elf, elf->symtab_shdr.sh_link, (size_t)esym->st_name);
if (ptr == sym)
return name;
}
return NULL;
}
/*
* Do the same thing as in bin_find_symbol above, but compare addresses and return the string name.
*/
const char *
bin_find_symbol_name(void *sym) {
return do_bin_find_symbol_name(sym, ruby_info);
}
/*
* bin_update_image - update the ruby binary image in memory.
*
* Given -
* trampee - the name of the symbol to hook
* tramp - the stage 2 trampoline entry
* orig_func - out parameter storing the address of the function that was
* originally called.
*
* This function will update the ruby binary image so that all calls to trampee
* will be routed to tramp.
*
* Returns 0 on success
*/
int
bin_update_image(const char *trampee, struct tramp_st2_entry *tramp, void **orig_func)
{
void *trampee_addr = NULL;
assert(trampee != NULL);
assert(tramp != NULL);
assert(tramp->addr != NULL);
/* first check if the symbol is in the PLT */
trampee_addr = find_plt_addr(trampee, NULL);
if (trampee_addr) {
void *ret = NULL;
dbg_printf("Found %s in the PLT, inserting tramp...\n", trampee);
ret = overwrite_got(trampee_addr, tramp->addr, ruby_info);
assert(ret != NULL);
if (orig_func) {
*orig_func = ret;
dbg_printf("setting orig function: %p\n", *orig_func);
}
} else {
trampee_addr = bin_find_symbol(trampee, NULL, 0);
dbg_printf("Couldn't find %s in the PLT...\n", trampee);
if (trampee_addr) {
unsigned char *byte = ruby_info->text_segment;
size_t count = 0;
int num = 0;
assert(byte != NULL);
if (orig_func) {
*orig_func = trampee_addr;
}
for(; count < ruby_info->text_segment_len; byte++, count++) {
if (arch_insert_st1_tramp(byte, trampee_addr, tramp) == 0) {
num++;
}
}
dbg_printf("Inserted %d tramps for: %s\n", num, trampee);
}
}
dbg_printf("Trying to hook %s in other libraries...\n", trampee);
struct plt_hook_data data;
data.addr = tramp->addr;
data.sym = trampee;
for_each_dso(hook_required_objects, &data);
dbg_printf("Done searching other libraries for %s\n", trampee);
return 0;
}
static Dwarf_Die
check_die(Dwarf_Die die, const char *search, Dwarf_Half type)
{
char *name = 0;
Dwarf_Error error = 0;
Dwarf_Half tag = 0;
int ret = 0;
int res = dwarf_diename(die,&name,&error);
if (res == DW_DLV_ERROR) {
printf("Error in dwarf_diename\n");
exit(1);
}
if (res == DW_DLV_NO_ENTRY) {
return 0;
}
res = dwarf_tag(die,&tag,&error);
if (res != DW_DLV_OK) {
printf("Error in dwarf_tag\n");
exit(1);
}
if (tag == type && strcmp(name, search) == 0){
//printf("tag: %d name: '%s' die: %p\n",tag,name,die);
ret = 1;
}
dwarf_dealloc(dwrf,name,DW_DLA_STRING);
return ret ? die : 0;
}
static Dwarf_Die
search_dies(Dwarf_Die die, const char *name, Dwarf_Half type)
{
int res = DW_DLV_ERROR;
Dwarf_Die cur_die=die;
Dwarf_Die child = 0;
Dwarf_Error error;
Dwarf_Die ret = 0;
ret = check_die(cur_die, name, type);
if (ret)
return ret;
for(;;) {
Dwarf_Die sib_die = 0;
res = dwarf_child(cur_die,&child,&error);
if (res == DW_DLV_ERROR) {
printf("Error in dwarf_child\n");
exit(1);
}
if (res == DW_DLV_OK) {
ret = search_dies(child,name,type);
if (ret) {
if (cur_die != die && cur_die != ret)
dwarf_dealloc(dwrf,cur_die,DW_DLA_DIE);
return ret;
}
}
/* res == DW_DLV_NO_ENTRY */
res = dwarf_siblingof(dwrf,cur_die,&sib_die,&error);
if (res == DW_DLV_ERROR) {
printf("Error in dwarf_siblingof\n");
exit(1);
}
if (res == DW_DLV_NO_ENTRY) {
/* Done at this level. */
break;
}
/* res == DW_DLV_OK */
if (cur_die != die)
dwarf_dealloc(dwrf,cur_die,DW_DLA_DIE);
cur_die = sib_die;
ret = check_die(cur_die, name, type);
if (ret)
return ret;
}
return 0;
}
static Dwarf_Die
find_die(const char *name, Dwarf_Half type)
{
Dwarf_Die ret = 0;
Dwarf_Unsigned cu_header_length = 0;
Dwarf_Half version_stamp = 0;
Dwarf_Unsigned abbrev_offset = 0;
Dwarf_Half address_size = 0;
Dwarf_Unsigned next_cu_header = 0;
Dwarf_Error error;
int cu_number = 0;
Dwarf_Die no_die = 0;
Dwarf_Die cu_die = 0;
int res = DW_DLV_ERROR;
for (;;++cu_number) {
no_die = 0;
cu_die = 0;
res = DW_DLV_ERROR;
res = dwarf_next_cu_header(dwrf, &cu_header_length, &version_stamp, &abbrev_offset, &address_size, &next_cu_header, &error);
if (res == DW_DLV_ERROR) {
printf("Error in dwarf_next_cu_header\n");
exit(1);
}
if (res == DW_DLV_NO_ENTRY) {
/* Done. */
return 0;
}
/* The CU will have a single sibling, a cu_die. */
res = dwarf_siblingof(dwrf,no_die,&cu_die,&error);
if (res == DW_DLV_ERROR) {
printf("Error in dwarf_siblingof on CU die \n");
exit(1);
}
if (res == DW_DLV_NO_ENTRY) {
/* Impossible case. */
printf("no entry! in dwarf_siblingof on CU die \n");
exit(1);
}
ret = search_dies(cu_die,name,type);
if (cu_die != ret)
dwarf_dealloc(dwrf,cu_die,DW_DLA_DIE);
if (ret)
break;
}
/* traverse to the end to reset */
while ((dwarf_next_cu_header(dwrf, &cu_header_length, &version_stamp, &abbrev_offset, &address_size, &next_cu_header, &error)) != DW_DLV_NO_ENTRY);
return ret ? ret : 0;
}
/*
* find_libruby_cb - a callback which attempts to locate libruby in the linkmap
*
* This callback is fired from walk_linkmap for each item on the linkmap. This
* function searchs for libruby and stores data about it in the object passed
* through.
*
* This function return CB_EXIT once libruby is found to terminate the link map
* walker. If libruby isn't found, this function returns CB_CONTINUE.
*/
static linkmap_cb_status
find_libruby_cb(struct link_map *map, void *data)
{
struct elf_info *lib = data;
assert(map != NULL);
assert(data != NULL);
if (strstr(map->l_name, "libruby")) {
dbg_printf("Found a libruby.so!\n");
if (lib) {
lib->base_addr = (GElf_Addr)map->l_addr;
lib->filename = strdup(map->l_name);
}
libruby = 1;
return CB_EXIT;
}
return CB_CONTINUE;
}
/*
* walk_linkmap - walk the linkmap firing a callback along the way.
*
* Given:
* - cb - a callback function
* - data - and any private data to pass through (optional)
*
* This function will crawl the linkmap and fire the callback on each item
* found.
*
* If the callback returns CB_CONTINUE, this function will move to the next
* (if any) item in the link map.
*
* If the callback returns CB_EXIT, this function will return immediately.
*/
static void
walk_linkmap(linkmap_cb cb, void *data)
{
struct link_map *map = _r_debug.r_map;
assert(map);
while (map) {
dbg_printf("Found a linkmap entry: %s\n", map->l_name);
if (cb(map, data) == CB_EXIT)
break;
map = map->l_next;
}
return;
}
/*
* has_libruby - check if this ruby binary is linked against libruby.so
*
* This function checks if the curreny binary is linked against libruby. If
* so, it sets libruby = 1, and fill internal state in the elf_info structure.
*
* Returns 1 if this binary is linked to libruby.so, 0 if not.
*/
static int
has_libruby(struct elf_info *lib)
{
libruby = 0;
walk_linkmap(find_libruby_cb, lib);
return libruby;
}
size_t
bin_type_size(const char *name)
{
Dwarf_Unsigned size = 0;
Dwarf_Error error;
int res = DW_DLV_ERROR;
Dwarf_Die die = 0;
die = find_die(name, DW_TAG_structure_type);
if (die) {
res = dwarf_bytesize(die, &size, &error);
dwarf_dealloc(dwrf,die,DW_DLA_DIE);
if (res == DW_DLV_OK)
return size;
}
return 0;
}
int
bin_type_member_offset(const char *type, const char *member)
{
Dwarf_Error error;
int res = DW_DLV_ERROR;
Dwarf_Die die = 0, child = 0;
Dwarf_Attribute attr = 0;
die = find_die(type, DW_TAG_structure_type);
if (die) {
child = search_dies(die, member, DW_TAG_member);
dwarf_dealloc(dwrf,die,DW_DLA_DIE);
if (child) {
res = dwarf_attr(child, DW_AT_data_member_location, &attr, &error);
if (res == DW_DLV_OK) {
Dwarf_Locdesc *locs = 0;
Dwarf_Signed num = 0;
res = dwarf_loclist(attr, &locs, &num, &error);
if (res == DW_DLV_OK && num > 0) {
return locs[0].ld_s[0].lr_number;
}
}
}
}
return -1;
}
/*
* open_elf - Opens a file from disk and gets the elf reader started.
*
* Given a filename, this function attempts to open the file and start the
* elf reader.
*
* Returns an elf_info object that must be freed by the caller.
*/
static void
open_elf(struct elf_info *info)
{
assert(info != NULL);
if (elf_version(EV_CURRENT) == EV_NONE)
errx(EX_SOFTWARE, "ELF library initialization failed: %s", elf_errmsg(-1));
if ((info->fd = open(info->filename, O_RDONLY, 0)) < 0)
err(EX_NOINPUT, "open \%s\" failed", info->filename);
if ((info->elf = elf_begin(info->fd, ELF_C_READ, NULL)) == NULL)
errx(EX_SOFTWARE, "elf_begin() failed: %s.", elf_errmsg(-1));
if (elf_kind(info->elf) != ELF_K_ELF)
errx(EX_DATAERR, "%s is not an ELF object.", info->filename);
return;
}
static char *
get_debuglink_info(struct elf_info *elf, unsigned long *crc_out)
{
char *basename = (char *) elf->debuglink_data->d_buf;
unsigned long offset = strlen(basename) + 1;
offset = (offset + 3) & ~3;
memcpy(crc_out, elf->debuglink_data->d_buf + offset, 4);
return basename;
}
static int
verify_debug_checksum(const char *filename, unsigned long crc)
{
struct stat stat_buf;
void *region = NULL;
int fd = open(filename, O_RDONLY);
unsigned long crc_check = 0;
if (fd == -1) {
dbg_printf("Couldn't open debug file: %s, because: %s\n", filename, strerror(errno));
return 1;
}
if (fstat(fd, &stat_buf) == -1) {
dbg_printf("Couldn't stat debug file: %s, because: %s\n", filename, strerror(errno));
return 1;
}
region = mmap(NULL, stat_buf.st_size, PROT_READ, MAP_PRIVATE, fd, 0);
if (region == MAP_FAILED) {
dbg_printf("mapping a section of size: %zd has failed!\n", stat_buf.st_size);
return 1;
}
crc_check = gnu_debuglink_crc32(crc_check, region, stat_buf.st_size);
dbg_printf("supposed crc: %lx, computed crc: %lx\n", crc, crc_check);
munmap(region, stat_buf.st_size);
close(fd);
return !(crc == crc_check);
}
static int
find_debug_syms(struct elf_info *elf)
{
/*
* XXX TODO perhaps allow users to specify an env var (or something) pointing
* to where the debug symbols live?
*/
unsigned long crc = 0;
char *basename = get_debuglink_info(elf, &crc);
char *debug_file = NULL, *dir = NULL;
char *tmp = strdup(elf->filename);
assert(basename != NULL);
dbg_printf(".gnu_debuglink base file name: %s, crc: %lx\n", basename, crc);
dir = dirname(tmp);
debug_file = calloc(1, strlen(DEBUGDIR) + strlen(dir) +
strlen("/") + strlen(basename) + 1);
strncat(debug_file, DEBUGDIR, strlen(DEBUGDIR));
strncat(debug_file, dir, strlen(dir));
strncat(debug_file, "/", strlen("/"));
strncat(debug_file, basename, strlen(basename));
elf->debug_data = calloc(1, sizeof(*elf->debug_data));
elf->debug_data->filename = debug_file;
dbg_printf("Possible debug symbols in: %s\n", elf->debug_data->filename);
if (verify_debug_checksum(elf->debug_data->filename, crc)) {
dbg_printf("Checksum verification of debug file: %s failed!", elf->debug_data->filename);
free(debug_file);
free(elf->debug_data);
return 1;
}
open_elf(elf->debug_data);
if (dissect_elf(elf->debug_data, 0) != 0) {
/* free debug_data */
dbg_printf("Dissection of debug data failed!\n");
free(debug_file);
free(elf->debug_data);
return 1;
}
elf->symtab_data = elf->debug_data->symtab_data;
elf->symtab_shdr = elf->debug_data->symtab_shdr;
/* XXX free stuff */
/* LEAK elf_end(elf->elf); */
elf->elf = elf->debug_data->elf;
close(elf->debug_data->fd);
dbg_printf("Finished dissecting debug data\n");
return 0;
}
/*
* dissect_elf - Parses and stores internal data about an ELF object.
*
* Given an elf_info structure, this function will attempt to parse the object
* and store important state needed to rewrite the object later.
*
* TODO better error handling
*
* Returns 1 on hard errors.
* Returns 2 on recoverable errors (missing symbol table).
* Returns 0 on success.
*/
static int
dissect_elf(struct elf_info *info, int find_debug)
{
assert(info != NULL);
size_t shstrndx = 0, j = 0;
Elf *elf = info->elf;
Elf_Scn *scn = NULL;
GElf_Shdr shdr;
int ret = 0;
if (elf_getshdrstrndx(elf, &shstrndx) == -1) {
dbg_printf("getshstrndx() failed: %s.", elf_errmsg(-1));
ret = 1;
goto out;
}
if (gelf_getehdr(elf, &(info->ehdr)) == NULL) {
dbg_printf("Couldn't get elf header.");
ret = 1;
goto out;
}
/* search each ELF header and store important data for each header... */
while ((scn = elf_nextscn(elf, scn)) != NULL) {
if (gelf_getshdr(scn, &shdr) != &shdr) {
dbg_printf("getshdr() failed: %s.", elf_errmsg(-1));
ret = 1;
goto out;
}
/*
* The .dynamic section contains entries that are important to memprof.
* Specifically, the .rela.plt section information. The .rela.plt section
* indexes the .plt, which will be important for hooking functions in
* shared objects.
*/
if (shdr.sh_type == SHT_DYNAMIC) {
Elf_Data *data;
data = elf_getdata(scn, NULL);
/* for each entry in the dyn section... */
for (j = 0; j < shdr.sh_size / shdr.sh_entsize; ++j) {
GElf_Dyn dyn;
if (gelf_getdyn(data, j, &dyn) == NULL) {
dbg_printf("Couldn't get .dynamic data from loaded library.");
ret = 1;
goto out;
}
if (dyn.d_tag == DT_JMPREL) {
info->relplt_addr = dyn.d_un.d_ptr;
}
else if (dyn.d_tag == DT_PLTGOT) {
info->got_addr = dyn.d_un.d_ptr;
}
else if (dyn.d_tag == DT_PLTRELSZ) {
info->plt_size = dyn.d_un.d_val;
}
}
}
/*
* The .dynsym section has useful pieces, too, like the dynamic symbol
* table. This table is used when walking the .rela.plt section.
*/
else if (shdr.sh_type == SHT_DYNSYM) {
Elf_Data *data;
info->dynsym = elf_getdata(scn, NULL);
info->dynsym_count = shdr.sh_size / shdr.sh_entsize;
if (info->dynsym == NULL || elf_getdata(scn, info->dynsym) != NULL) {
dbg_printf("Couldn't get .dynsym data ");
ret = 1;
goto out;
}
scn = elf_getscn(elf, shdr.sh_link);
if (scn == NULL || gelf_getshdr(scn, &shdr) == NULL) {
dbg_printf("Couldn't get section header.");
ret = 1;
goto out;
}
data = elf_getdata(scn, NULL);
if (data == NULL || elf_getdata(scn, data) != NULL
|| shdr.sh_size != data->d_size) {// condition true on 32bit: || data->d_off) {
dbg_printf("Couldn't get .dynstr data\n");
ret = 1;
goto out;
}
info->dynstr = data->d_buf;
}
/*
* Pull out information (start address and length) of the .text section.
*/
else if (shdr.sh_type == SHT_PROGBITS &&
(shdr.sh_flags == (SHF_ALLOC | SHF_EXECINSTR)) &&
strcmp(elf_strptr(elf, shstrndx, shdr.sh_name), ".text") == 0) {
info->text_segment = (void *)shdr.sh_addr + info->base_addr;
info->text_segment_len = shdr.sh_size;
}
/*
* Pull out information (start address) of the .plt section.
*/
else if (shdr.sh_type == SHT_PROGBITS) {
if (strcmp(elf_strptr(elf, shstrndx, shdr.sh_name), ".plt") == 0) {
info->plt_addr = shdr.sh_addr;
} else if (strcmp(elf_strptr(elf, shstrndx, shdr.sh_name), ".got.plt") == 0) {
info->got_addr = shdr.sh_addr;
} else if (strcmp(elf_strptr(elf, shstrndx, shdr.sh_name), ".gnu_debuglink") == 0) {
dbg_printf("gnu_debuglink section found\n", shdr.sh_size);
if ((info->debuglink_data = elf_getdata(scn, NULL)) == NULL ||
info->debuglink_data->d_size == 0) {
dbg_printf(".gnu_debuglink section existed, but wasn't readable.\n");
ret = 2;
goto out;
}
dbg_printf("gnu_debuglink section read (size: %zd)\n", shdr.sh_size);
}
}
/*
* The symbol table is also needed for bin_find_symbol
*/
else if (shdr.sh_type == SHT_SYMTAB) {
info->symtab_shdr = shdr;
if ((info->symtab_data = elf_getdata(scn, info->symtab_data)) == NULL ||
info->symtab_data->d_size == 0) {
dbg_printf("shared lib has a broken symbol table. Is it stripped? "
"memprof only works on shared libs that are not stripped!\n");
ret = 2;
goto out;
}
}
}
/* If this object has no symbol table there's nothing else to do but fail */
if (!info->symtab_data) {
if (info->debuglink_data) {
dbg_printf("binary is stripped, but there is debug symbol information. memprof will try to read debug symbols in.\n");
} else {
dbg_printf("binary is stripped, and no debug symbol info was found. memprof only works on binaries that are not stripped!\n");
}
ret = 1;
}
/*
* Walk the sections, pull out, and store the .plt section
*/
for (j = 1; j < info->ehdr.e_shnum; j++) {
scn = elf_getscn(elf, j);
if (scn == NULL || gelf_getshdr(scn, &shdr) == NULL) {
dbg_printf("Couldn't get section header from library.");
ret = 1;
goto out;
}
if (shdr.sh_addr == info->relplt_addr
&& shdr.sh_size == info->plt_size) {
info->relplt = elf_getdata(scn, NULL);
info->relplt_count = shdr.sh_size / shdr.sh_entsize;
if (info->relplt == NULL || elf_getdata(scn, info->relplt) != NULL) {
dbg_printf("Couldn't get .rel*.plt data from");
ret = 1;
goto out;
}
break;
}
}
out:
if (find_debug && ret == 1) {
if (info->debuglink_data) {
find_debug_syms(info);
} else {
dbg_printf("=== WARNING: Object %s was STRIPPED and had no debuglink section. Nothing left to try.\n", info->filename);
}
}
return ret;
}
/*
* bin_init - initialize the binary parsing/modification layer.
*
* This function starts the elf parser and sets up internal state.
*/
void
bin_init()
{
Dwarf_Error dwrf_err;
ruby_info = calloc(1, sizeof(*ruby_info));
if (!ruby_info) {
errx(EX_UNAVAILABLE, "Unable to allocate memory to start binary parsing layer");
}
if (!has_libruby(ruby_info)) {
dbg_printf("This ruby binary has no libruby\n");
char *filename = calloc(1, 255);
if (readlink("/proc/self/exe", filename, 255) == -1) {
errx(EX_UNAVAILABLE, "Unable to follow /proc/self/exe symlink: %s", strerror(errno));
}
ruby_info->filename = filename;
ruby_info->base_addr = 0;
dbg_printf("The path to the binary is: %s\n", ruby_info->filename);
}
open_elf(ruby_info);
if (dissect_elf(ruby_info, 1) == 2) {
errx(EX_DATAERR, "Error trying to parse elf file: %s\n", ruby_info->filename);
}
if (dwarf_elf_init(ruby_info->elf, DW_DLC_READ, NULL, NULL, &dwrf, &dwrf_err) != DW_DLV_OK) {
errx(EX_DATAERR, "unable to read debugging data from binary. was it compiled with -g? is it unstripped?");
}
dbg_printf("bin_init finished\n");
}
#endif
================================================
FILE: ext/extconf.rb
================================================
if RUBY_VERSION >= "1.9"
STDERR.puts "\n\n"
STDERR.puts "***************************************************************************************"
STDERR.puts "************************** ruby 1.9 is not supported (yet) =( *************************"
STDERR.puts "***************************************************************************************"
exit(1)
end
require 'mkmf'
require 'fileutils'
CWD = File.expand_path(File.dirname(__FILE__))
def sys(cmd)
puts " -- #{cmd}"
unless ret = xsystem(cmd)
raise "#{cmd} failed, please report to memprof@tmm1.net with pastie.org link to #{CWD}/mkmf.log"
end
ret
end
###
# yajl
yajl = File.basename('yajl-1.0.9.tar.gz')
dir = File.basename(yajl, '.tar.gz')
unless File.exists?("#{CWD}/dst/lib/libyajl_ext.a")
puts "(I'm about to compile yajl.. this will definitely take a while)"
Dir.chdir('src') do
FileUtils.rm_rf(dir) if File.exists?(dir)
sys("tar zxvf #{yajl}")
Dir.chdir("#{dir}/src") do
sys("sed -i -e 's,yajl,json,g' *.h *.c api/*.h")
Dir['{,api/}yajl*.{h,c}'].each do |file|
FileUtils.mv file, file.gsub('yajl', 'json')
end
FileUtils.mkdir_p "api/json"
%w[ common parse gen ].each do |f|
FileUtils.cp "api/json_#{f}.h", 'api/json/'
end
File.open("extconf.rb",'w') do |f|
f.puts "require 'mkmf'; $INCFLAGS[0,0] = '-I./api/ '; create_makefile 'libyajl'"
end
sys("#{Config::CONFIG['bindir']}/#{Config::CONFIG['ruby_install_name']} extconf.rb")
sys("make")
if RUBY_PLATFORM =~ /darwin/
sys("libtool -static -o libyajl_ext.a #{Dir['*.o'].join(' ')}")
else
sys("ar rv libyajl_ext.a #{Dir['*.o'].join(' ')}")
end
FileUtils.mkdir_p "#{CWD}/dst/lib"
FileUtils.cp 'libyajl_ext.a', "#{CWD}/dst/lib"
FileUtils.mkdir_p "#{CWD}/dst/include"
FileUtils.cp_r 'api/json', "#{CWD}/dst/include/"
end
end
end
$LIBPATH.unshift "#{CWD}/dst/lib"
$INCFLAGS[0,0] = "-I#{CWD}/dst/include "
unless have_library('yajl_ext') and have_header('json/json_gen.h')
raise 'Yajl build failed'
end
def add_define(name)
$defs.push("-D#{name}")
end
if RUBY_PLATFORM =~ /linux/
###
# libelf
libelf = File.basename('libelf-0.8.13.tar.gz')
dir = File.basename(libelf, '.tar.gz')
unless File.exists?("#{CWD}/dst/lib/libelf_ext.a")
puts "(I'm about to compile libelf.. this will definitely take a while)"
Dir.chdir('src') do
FileUtils.rm_rf(dir) if File.exists?(dir)
sys("tar zxvf #{libelf}")
Dir.chdir(dir) do
ENV['CFLAGS'] = '-fPIC'
sys("./configure --prefix=#{CWD}/dst --disable-nls --disable-shared")
sys("make")
sys("make install")
end
end
Dir.chdir('dst/lib') do
FileUtils.ln_s 'libelf.a', 'libelf_ext.a'
end
end
$LIBPATH.unshift "#{CWD}/dst/lib"
$INCFLAGS[0,0] = "-I#{CWD}/dst/include "
unless have_library('elf_ext', 'gelf_getshdr')
raise 'libelf build failed'
end
###
# libdwarf
libdwarf = File.basename('libdwarf-20091118.tar.gz')
dir = File.basename(libdwarf, '.tar.gz').sub('lib','')
unless File.exists?("#{CWD}/dst/lib/libdwarf_ext.a")
puts "(I'm about to compile libdwarf.. this will definitely take a while)"
Dir.chdir('src') do
FileUtils.rm_rf(dir) if File.exists?(dir)
sys("tar zxvf #{libdwarf}")
Dir.chdir("#{dir}/libdwarf") do
ENV['CFLAGS'] = "-fPIC -I#{CWD}/dst/include"
ENV['LDFLAGS'] = "-L#{CWD}/dst/lib"
sys("./configure")
sys("make")
FileUtils.cp 'libdwarf.a', "#{CWD}/dst/lib/libdwarf_ext.a"
FileUtils.cp 'dwarf.h', "#{CWD}/dst/include/"
FileUtils.cp 'libdwarf.h', "#{CWD}/dst/include/"
end
end
end
unless have_library('dwarf_ext')
raise 'libdwarf build failed'
end
is_elf = true
add_define 'HAVE_ELF'
add_define 'HAVE_DWARF'
end
if have_header('mach-o/dyld.h')
is_macho = true
add_define 'HAVE_MACH'
# XXX How to determine this properly? RUBY_PLATFORM reports "i686-darwin10.2.0" on Snow Leopard.
add_define "_ARCH_x86_64_"
sizes_of = [
'RVALUE',
'struct heaps_slot'
]
offsets_of = {
'struct heaps_slot' => %w[ slot limit ],
'struct BLOCK' => %w[ body var cref self klass wrapper block_obj orig_thread dyna_vars scope prev ],
'struct METHOD' => %w[ klass rklass recv id oid body ]
}
addresses_of = [
# 'add_freelist',
# 'rb_newobj',
# 'freelist',
# 'heaps',
# 'heaps_used',
# 'finalizer_table'
]
expressions = []
sizes_of.each do |type|
name = type.sub(/^struct\s*/,'')
expressions << ["sizeof__#{name}", "sizeof(#{type})"]
end
offsets_of.each do |type, members|
name = type.sub(/^struct\s*/,'')
members.each do |member|
expressions << ["offset__#{name}__#{member}", "(size_t)&(((#{type} *)0)->#{member})"]
end
end
addresses_of.each do |name|
expressions << ["address__#{name}", "&#{name}"]
end
pid = fork{sleep while true}
output = IO.popen('gdb --interpreter=mi --quiet', 'w+') do |io|
io.puts "attach #{pid}"
expressions.each do |name, expr|
io.puts "-data-evaluate-expression #{expr.dump}"
end
io.puts 'quit'
io.puts 'y'
io.close_write
io.read
end
Process.kill 9, pid
attach, *results = output.grep(/^\^/).map{ |l| l.strip }
if results.find{ |l| l =~ /^\^error/ }
raise "Unsupported platform: #{results.inspect}"
end
values = results.map{ |l| l[/value="(.+?)"/, 1] }
vars = Hash[ *expressions.map{|n,e| n }.zip(values).flatten ].each do |name, val|
add_define "#{name}=#{val.split.first}"
end
end
arch = RUBY_PLATFORM[/(.*)-linux/,1]
case arch
when "universal"
arch = 'x86_64'
when 'i486'
arch = 'i386'
end
add_define "_ARCH_#{arch}_"
if ENV['MEMPROF_DEBUG'] == '1'
add_define "_MEMPROF_DEBUG"
$preload = ["\nCFLAGS = -Wall -Wextra -fPIC -ggdb3 -O0"]
end
if is_elf or is_macho
$objs = Dir['{.,*}/*.c'].map{ |file| file.gsub(/\.c(pp)?$/, '.o') }
create_makefile('memprof')
makefile = File.read('Makefile')
makefile.gsub!('-c $<', '-o $(patsubst %.c,%.o,$<) -c $<')
makefile.gsub!(/(CLEANOBJS\s*=)/, '\1 */*.o')
File.open('Makefile', 'w+'){ |f| f.puts(makefile) }
else
raise 'unsupported platform'
end
================================================
FILE: ext/i386.c
================================================
#if defined (_ARCH_i386_) || defined(_ARCH_i686_)
#include <stdint.h>
#include <string.h>
#include <sys/mman.h>
#include "arch.h"
#include "x86_gen.h"
/* This is the stage 1 inline trampoline for hooking the inlined add_freelist
* function .
*
* NOTE: The original instruction mov %reg, freelist is 7 bytes wide,
* whereas jmpq $displacement is only 5 bytes wide. We *must* pad out
* the next two bytes. This will be important to remember below.
*/
struct inline_st1_tramp {
unsigned char jmp;
uint32_t displacement;
unsigned char pad;
} __attribute__((__packed__)) inline_st1_tramp = {
.jmp = '\xe9',
.displacement = 0,
.pad = '\x90',
};
struct inline_st1_base_short {
unsigned char mov;
void *addr;
} __attribute__((__packed__)) inline_st1_short = {
.mov = '\xa3',
.addr = 0,
};
struct inline_st1_base_long {
unsigned char mov;
unsigned char src_reg;
void *addr;
} __attribute__((__packed__)) inline_st1_long = {
.mov = '\x89',
.src_reg = 0,
.addr = 0
};
static int
arch_check_ins(unsigned char *base)
{
/* if the byte is 0xa3 then we're moving from %eax, so
* the length is only 5, so we don't need the pad.
*
* otherwise, we're moving from something else, so the
* length is going to be 6 and we need a NOP.
*/
/* is it a mov instruction? */
if (*base == 0xa3)
return 0;
else if (*base == 0x89)
return 1;
return -1;
}
int
arch_insert_inline_st2_tramp(void *addr, void *marker, void *trampoline, void *table_entry)
{
struct inline_st1_base_long *long_base = addr;
struct inline_st1_base_short *short_base = addr;
struct inline_st1_tramp *st1_tramp = addr;
void *mov_target = NULL;
size_t pad_length = 0;
if ((pad_length = arch_check_ins(addr)) == -1)
return 1;
if (pad_length == 0) {
mov_target = short_base->addr;
default_inline_st2_tramp.mov = 0x90;
default_inline_st2_tramp.src_reg = 0xa3;
inline_st1_tramp.displacement = table_entry - (void *)(short_base + 1);
default_inline_st2_tramp.jmp_displacement = (void *)(short_base + 1) - (table_entry + sizeof(default_inline_st2_tramp));
} else {
mov_target = long_base->addr;
default_inline_st2_tramp.mov = long_base->mov;
default_inline_st2_tramp.src_reg = long_base->src_reg;
inline_st1_tramp.displacement = table_entry - (void *)(long_base + 1) + 1;
default_inline_st2_tramp.jmp_displacement = (void *)(long_base + 1) - (table_entry + sizeof(default_inline_st2_tramp));
}
if (marker == mov_target) {
default_inline_st2_tramp.mov_addr= default_inline_st2_tramp.frame.freelist = marker;
default_inline_st2_tramp.frame.fn_addr = trampoline;
if (pad_length) {
copy_instructions(addr, &inline_st1_tramp, sizeof(inline_st1_tramp));
} else {
copy_instructions(addr, &inline_st1_tramp, sizeof(inline_st1_tramp) - 1);
}
memcpy(table_entry, &default_inline_st2_tramp, sizeof(default_inline_st2_tramp));
return 0;
}
return 1;
}
#endif
================================================
FILE: ext/i386.h
================================================
#if !defined(_i386_h_)
#define _i386_h_
#include <stdint.h>
#include "arch.h"
/*
* This is the "normal" stage 2 trampoline with a default entry pre-filled
*/
static struct tramp_st2_entry {
unsigned char ebx_save;
unsigned char mov;
void *addr;
unsigned char call[2];
unsigned char ebx_restore;
unsigned char ret;
} __attribute__((__packed__)) default_st2_tramp = {
.ebx_save = 0x53, /* push ebx */
.mov = 0xbb, /* mov addr into ebx */
.addr = 0, /* this is filled in later */
.call = {0xff, 0xd3}, /* calll *ebx */
.ebx_restore = 0x5b, /* pop ebx */
.ret = 0xc3, /* ret */
};
/*
* This is the inline stage 2 trampoline with a default entry pre-filled
*/
static struct inline_tramp_st2_entry {
/* this block will be filled in at runtime to replicate the overwritten
* instruction.
*/
unsigned char mov;
unsigned char src_reg;
void *mov_addr;
/* this frame will arrange freelist to be passed as an argument to
* the third and final trampoline (C level).
*/
struct {
unsigned char push_ebx;
unsigned char pushl[2];
void * freelist;
unsigned char mov_ebx;
void * fn_addr;
unsigned char call[2];
unsigned char pop_ebx;
unsigned char restore_ebx;
} __attribute__((__packed__)) frame;
/* this block jumps back to resume execution */
unsigned char jmp;
uint32_t jmp_displacement;
} __attribute__((__packed__)) default_inline_st2_tramp = {
.mov = 0x89,
.src_reg = 0,
.mov_addr = 0,
.frame = {
.push_ebx = 0x53,
.pushl = {0xff, 0x35},
.freelist = 0,
.mov_ebx = 0xbb,
.fn_addr = 0,
.call = {0xff, 0xd3},
.pop_ebx = 0x5b,
.restore_ebx = 0x5b,
},
.jmp = 0xe9,
.jmp_displacement = 0,
};
#endif
================================================
FILE: ext/json.c
================================================
#ifndef _GNU_SOURCE
#define _GNU_SOURCE
#endif
#include <assert.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include "json.h"
void
json_gen_reset(json_gen gen)
{
json_gen_clear(gen);
assert (gen->state[gen->depth] == json_gen_complete);
gen->state[gen->depth] = json_gen_start;
gen->print(gen->ctx, "\n", 1);
}
json_gen_status
json_gen_cstr(json_gen gen, const char * str)
{
if (!str || str[0] == 0)
return json_gen_null(gen);
else
return json_gen_string(gen, (unsigned char *)str, strlen(str));
}
json_gen_status
json_gen_format(json_gen gen, char *format, ...)
{
va_list args;
char *str;
int bytes_printed = 0;
json_gen_status ret;
va_start(args, format);
bytes_printed = vasprintf(&str, format, args);
assert(bytes_printed != -1);
va_end(args);
ret = json_gen_string(gen, (unsigned char *)str, strlen(str));
free(str);
return ret;
}
json_gen_status
json_gen_pointer(json_gen gen, void* ptr)
{
return json_gen_format(gen, "0x%x", ptr);
}
================================================
FILE: ext/json.h
================================================
#if !defined(__JSON__H_)
#define __JSON__H_
#include <stdarg.h>
#include <json/json_gen.h>
/* HAX: copied from internal json_gen.c (PATCH json before building instead)
*/
typedef enum {
json_gen_start,
json_gen_map_start,
json_gen_map_key,
json_gen_map_val,
json_gen_array_start,
json_gen_in_array,
json_gen_complete,
json_gen_error
} json_gen_state;
struct json_gen_t
{
unsigned int depth;
unsigned int pretty;
const char * indentString;
json_gen_state state[YAJL_MAX_DEPTH];
json_print_t print;
void * ctx; /* json_buf */
/* memory allocation routines */
json_alloc_funcs alloc;
};
/* END HAX
*/
void
json_gen_reset(json_gen gen);
json_gen_status
json_gen_cstr(json_gen gen, const char * str);
json_gen_status
json_gen_format(json_gen gen, char *format, ...);
json_gen_status
json_gen_pointer(json_gen gen, void* ptr);
#endif
================================================
FILE: ext/mach.c
================================================
#if defined(HAVE_MACH)
#include "arch.h"
#include "bin_api.h"
#include "mmap.h"
#include "util.h"
#include <assert.h>
#include <dlfcn.h>
#include <err.h>
#include <errno.h>
#include <inttypes.h>
#include <limits.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sysexits.h>
#include <sys/mman.h>
#include <sys/stat.h>
#include <mach-o/dyld.h>
#include <mach-o/getsect.h>
#include <mach-o/loader.h>
#include <mach-o/ldsyms.h>
#include <mach-o/nlist.h>
#include <mach-o/dyld_images.h>
#include <mach-o/fat.h>
struct mach_config {
const struct mach_header *hdr;
const struct nlist_64 *symbol_table;
const struct nlist_64 **sorted_symbol_table;
const struct section_64 *symstub_sect;
const char *string_table;
uint32_t symbol_count;
uint32_t string_table_size;
intptr_t image_offset;
const struct mach_header* load_addr;
uint32_t nindirectsyms;
uint32_t indirectsymoff;
struct mmap_info file;
const char *filename;
unsigned int index;
};
struct symbol_data {
const char *name;
void *address;
uint32_t size;
uint32_t index;
};
static struct mach_config ruby_img_cfg;
extern struct memprof_config memprof_config;
/*
* The jmp instructions in the dyld stub table are 6 bytes,
* 2 bytes for the instruction and 4 bytes for the offset operand
*
* This jmp does not jump to the offset operand, but instead
* looks up an absolute address stored at the offset and jumps to that.
* Offset is the offset from the address of the _next_ instruction sequence.
*
* We need to deference the address at this offset to find the real
* target of the dyld stub entry.
*/
struct dyld_stub_entry {
unsigned char jmp[2];
int32_t offset;
} __attribute((__packed__));
/* Return the jmp target of a stub entry */
static inline void*
get_dyld_stub_target(struct dyld_stub_entry *entry) {
// If the instructions match up, then dereference the address at the offset
if (entry->jmp[0] == 0xff && entry->jmp[1] == 0x25)
return *((void**)((void*)(entry + 1) + entry->offset));
return NULL;
}
/* Set the jmp target of a stub entry */
static inline void
set_dyld_stub_target(struct dyld_stub_entry *entry, void *addr) {
void *target = (void *)(entry+1) + entry->offset;
copy_instructions(target, &addr, sizeof(void *));
}
static inline const char*
get_symtab_string(struct mach_config *img_cfg, uint32_t stroff);
static void
extract_symbol_data(struct mach_config *img_cfg, struct symbol_data *sym_data);
static int
find_dyld_image_index(const struct mach_header_64 *hdr);
static void *
find_stub_addr(const char *symname, struct mach_config *img_cfg)
{
uint64_t i = 0, nsyms = 0;
uint32_t symindex = 0;
assert(img_cfg && symname);
const struct section_64 *sect = img_cfg->symstub_sect;
if (!sect)
return NULL;
nsyms = sect->size / sect->reserved2;
for (; i < nsyms; i ++) {
uint32_t currsym = sect->reserved1 + i;
uint64_t stubaddr = sect->offset + (i * sect->reserved2);
uint32_t symoff = 0;
assert(currsym <= img_cfg->nindirectsyms);
/* indirect sym entries are just 32bit indexes into the symbol table to the
* symbol the stub is referring to.
*/
symoff = img_cfg->indirectsymoff + (i * 4);
memcpy(&symindex, (char*)img_cfg->hdr + symoff, 4);
symindex = symindex & ((uint32_t) ~(INDIRECT_SYMBOL_LOCAL | INDIRECT_SYMBOL_ABS));
const struct nlist_64 *ent = img_cfg->symbol_table + symindex;
const char *string = get_symtab_string(img_cfg, ent->n_un.n_strx);
if (strcmp(symname, string+1) == 0) {
if (stubaddr) {
dbg_printf("address of stub in %s for %s is %" PRIx64 " + %p = ", img_cfg->filename, string, stubaddr, img_cfg->load_addr);
stubaddr = (uint64_t)img_cfg->load_addr + stubaddr;
dbg_printf("%" PRIx64 "\n", stubaddr);
return (void *)stubaddr;
}
}
}
dbg_printf("couldn't find address of stub: %s in %s\n", symname, img_cfg->filename);
return NULL;
}
/*
* Search all entries in a stub table for the stub that corresponds to trampee_addr,
* and overwrite to point at our trampoline code.
* Returns 0 if any tramps were successfully inserted.
*/
static int
update_dyld_stub_table(void *table, uint64_t len, void *trampee_addr, struct tramp_st2_entry *tramp)
{
int ret = -1;
struct dyld_stub_entry *entry = (struct dyld_stub_entry*) table;
void *max_addr = table + len;
for(; (void*)entry < max_addr; entry++) {
void *target = get_dyld_stub_target(entry);
if (trampee_addr == target) {
set_dyld_stub_target(entry, tramp->addr);
ret = 0;
}
}
return ret;
}
/*
* Get all DSOs
*/
static const struct dyld_all_image_infos *
dyld_get_all_images() {
static const struct dyld_all_image_infos* (*_dyld_get_all_image_infos)() = NULL;
static const struct dyld_all_image_infos *images = NULL;
if (!_dyld_get_all_image_infos) {
_dyld_lookup_and_bind("__dyld_get_all_image_infos", (void**)&_dyld_get_all_image_infos, NULL);
assert(_dyld_get_all_image_infos != NULL);
}
if (!images) {
images = _dyld_get_all_image_infos();
assert(images != NULL);
}
return images;
}
/*
* Get info for particular DSO
*/
static const struct dyld_image_info *
dyld_get_image_info_for_index(int index) {
const struct dyld_all_image_infos *images = dyld_get_all_images();
// Stupid indexes into the infoArray don't match indexes used elsewhere, so we have to loop
unsigned int i;
const struct mach_header *hdr = _dyld_get_image_header(index);
for(i=0; i < _dyld_image_count(); i++) {
const struct dyld_image_info image = images->infoArray[i];
if (hdr == image.imageLoadAddress)
return &(images->infoArray[i]);
}
return NULL;
}
/*
* This function tells us if the passed header index is something
* that we should try to update (by looking at it's filename)
* Only try to update the running executable, or files that match
* "libruby.dylib" or "*.bundle" (other C gems)
*/
static const struct mach_header *
should_update_image(int index) {
const struct mach_header *hdr = _dyld_get_image_header(index);
/* Don't update if it's the memprof bundle */
if ((void*)hdr == &_mh_bundle_header)
return NULL;
/* If it's the ruby executable, do it! */
if ((void*)hdr == &_mh_execute_header)
return hdr;
/* otherwise, check to see if its a bundle or libruby */
const struct dyld_image_info *image = dyld_get_image_info_for_index(index);
size_t len = strlen(image->imageFilePath);
if (len >= 6) {
const char *possible_bundle = (image->imageFilePath + len - 6);
if (strcmp(possible_bundle, "bundle") == 0)
return hdr;
}
if (len >= 13) {
const char *possible_libruby = (image->imageFilePath + len - 13);
if (strcmp(possible_libruby, "libruby.dylib") == 0)
return hdr;
}
return NULL;
}
/*
* Attempts to update all necessary code in a given 'section' of a Mach-O image, to redirect
* the given function to the trampoline. This function takes care of both normal calls as well as
* shared library cases.
* Returns 0 if any tramps were successfully inserted.
*/
static int
update_mach_section(const struct mach_header *header, const struct section_64 *sect, intptr_t slide, void *trampee_addr, struct tramp_st2_entry *tramp) {
int ret = -1;
uint64_t len = 0;
/*
* We should be able to calculate this information from the section_64 struct ourselves,
* but I encountered problems going that route, so using this helper function is fine.
*
* The segment "__TEXT" means "executable code and other read-only data." Segments have "sections", like "__text", "__const", etc.
* We want "__text" for normal function calls, and "__symbol_stub" (with variations like "__symbol_stub1") for shared lib stubs.
*/
void *section = getsectdatafromheader_64((const struct mach_header_64*)header, "__TEXT", sect->sectname, &len) + slide;
if (strncmp(sect->sectname, "__symbol_stub", 13) == 0) {
if (update_dyld_stub_table(section, sect->size, trampee_addr, tramp) == 0) {
ret = 0;
}
return ret;
}
if (strcmp(sect->sectname, "__text") == 0) {
size_t count = 0;
for(; count < len; section++, count++) {
if (arch_insert_st1_tramp(section, trampee_addr, tramp) == 0) {
ret = 0;
}
}
}
return ret;
}
/*
* For a given Mach-O image, iterates over all segments and their sections, passing
* the sections to update_mach_section for potential tramping.
* Returns 0 if any tramps were successfully inserted.
*/
static int
update_bin_for_mach_header(const struct mach_header *header, intptr_t slide, void *trampee_addr, struct tramp_st2_entry *tramp) {
int ret = -1;
int i, j;
int lc_count = header->ncmds;
/* Load commands start immediately after the Mach header.
* This as a char* because we need to step it forward by an arbitrary number of bytes,
* specified in the 'cmdsize' field of each load command.
*/
const char *lc = ((const char*) header) + sizeof(struct mach_header_64);
/* Check all the load commands in the object to see if they are segment commands */
for (i = 0; i < lc_count; i++, lc += ((struct load_command*)lc)->cmdsize) {
if (((struct load_command*)lc)->cmd == LC_SEGMENT_64) {
/* If it's a segment command, we want to iterate over each of it's sections. */
const struct segment_command_64 *seg = (const struct segment_command_64 *) lc;
const struct section_64 * sect = (const struct section_64*)(lc + sizeof(struct segment_command_64));
/* Sections start immediately after the segment_command, and are included in the segment's "cmdsize" */
int section_count = (seg->cmdsize - sizeof(struct segment_command_64)) / sizeof(struct section_64);
/* Attempt to tramp the sections */
for (j=0; j < section_count; j++, sect++) {
if (update_mach_section(header, sect, slide, trampee_addr, tramp) == 0) {
ret = 0;
}
}
}
}
return ret;
}
/* This function takes a pointer to an *in process* mach_header_64
* and returns it's image index, which is required to specify the image
* in many dyld functions.
*
* This will NOT work for mach objects read manually from a file, since
* that's just arbitrary data and the dyld system knows nothing about it.
*/
static int
find_dyld_image_index(const struct mach_header_64 *hdr) {
uint32_t i;
for (i = 0; i < _dyld_image_count(); i++) {
const struct mach_header_64 *tmphdr = (const struct mach_header_64*) _dyld_get_image_header(i);
if (hdr == tmphdr)
return i;
}
errx(EX_SOFTWARE, "Could not find image index");
/* this is to quiet a GCC warning. might be a bug because errx is marked
* __dead2/noreturn
*/
return -1;
}
/*
* This function compares two nlist_64 structures by their n_value field (address, usually).
* It is used by qsort in extract_symbol_table.
*/
static int
nlist_cmp(const void *obj1, const void *obj2) {
const struct nlist_64 *nlist1 = *(const struct nlist_64**) obj1;
const struct nlist_64 *nlist2 = *(const struct nlist_64**) obj2;
if (nlist1->n_value == nlist2->n_value)
return 0;
else if (nlist1->n_value < nlist2->n_value)
return -1;
else
return 1;
}
/*
* This function sets the passed pointers to a buffer containing the nlist_64 entries,
* a buffer containing the string data, the number of entries, and the size of the string buffer.
*
* The string names of symbols are stored separately from the symbol table.
* The symbol table entries contain a string 'index', which is an offset into this region.
*
* !!! This function allocates memory. symbol_table and string_table should be freed when no longer used !!!
*/
static void
extract_symbol_table(const struct mach_header_64 *hdr, struct mach_config *img_cfg) {
const struct nlist_64 **new_symtbl;
char *new_strtbl;
uint32_t i, j;
assert(hdr);
assert(img_cfg);
const struct load_command *lc = (const struct load_command *)(hdr + 1);
for (i = 0; i < hdr->ncmds; i++, (lc = (const struct load_command *)((char *)lc + lc->cmdsize))) {
if (lc->cmd == LC_SYMTAB) {
// dbg_printf("found an LC_SYMTAB load command.\n");
const struct symtab_command *sc = (const struct symtab_command*) lc;
const struct nlist_64 *file_symtbl = (const struct nlist_64*)((const char*)hdr + sc->symoff);
new_symtbl = malloc(sc->nsyms * sizeof(struct nlist_64*));
new_strtbl = malloc(sc->strsize);
memcpy(new_strtbl, (char*)hdr + sc->stroff, sc->strsize);
for (j = 0; j < sc->nsyms; j++)
new_symtbl[j] = file_symtbl + j;
qsort(new_symtbl, sc->nsyms, sizeof(struct nlist_64*), &nlist_cmp);
img_cfg->symbol_table = file_symtbl;
img_cfg->sorted_symbol_table = new_symtbl;
img_cfg->symbol_count = sc->nsyms;
img_cfg->string_table = new_strtbl;
img_cfg->string_table_size = sc->strsize;
} else if (lc->cmd == LC_DYSYMTAB) {
// dbg_printf("found an LC_DYSYMTAB load command.\n");
const struct dysymtab_command *dynsym = (const struct dysymtab_command *) lc;
img_cfg->nindirectsyms = dynsym->nindirectsyms;
img_cfg->indirectsymoff = dynsym->indirectsymoff;
} else if (lc->cmd == LC_SEGMENT_64) {
// dbg_printf("found an LC_SEGMENT_64 load command.\n");
const struct segment_command_64 *seg = (const struct segment_command_64 *) lc;
uint32_t i = 0;
const struct section_64 *asect = (const struct section_64 *)(seg + 1);
for(; i < seg->nsects; i++, asect++) {
/*
* setting up data to find the indirect symbol tables.
*/
/* if this section hsa no symbol stubs, then we don't care about it */
if ((asect->flags & SECTION_TYPE) != S_SYMBOL_STUBS)
continue;
if (asect->reserved2 == 0) {
dbg_printf("!!! Found an LC_SEGMET_64 which was marked as having stubs,"
" but does not have reserved2 set!! %16s.%16s (skipping)\n", asect->segname, asect->sectname);
continue;
}
// dbg_printf("Found a section with symbol stubs: %16s.%16s.\n", asect->segname, asect->sectname);
img_cfg->symstub_sect = asect;
}
} else {
// dbg_printf("found another load command that is not being tracked: %" PRId32 "\n", lc->cmd);
}
}
assert(img_cfg->symbol_table && img_cfg->string_table);
}
/*
* Return the string at the given offset into the symbol table's string buffer
*/
static inline const char*
get_symtab_string(struct mach_config *img_cfg, uint32_t stroff) {
assert(img_cfg);
assert(img_cfg->string_table != NULL);
assert(stroff < img_cfg->string_table_size);
return img_cfg->string_table + stroff;
}
/*
* Lookup the address, size, and symbol table index of a symbol given a symbol_data
* If sym_data is passed with the name set, this function will attempt to fill
* in the address, etc. If it is passed with the address set, it will attempt
* to fill in the name.
*/
static void
extract_symbol_data(struct mach_config *img_cfg, struct symbol_data *sym_data)
{
uint32_t i, j;
assert(img_cfg->symbol_table != NULL);
assert(img_cfg->symbol_count > 0);
for (i=0; i < img_cfg->symbol_count; i++) {
// const struct nlist_64 *nlist_entry = img_cfg->sorted_symbol_table[i];
const struct nlist_64 *nlist_entry = img_cfg->symbol_table + i;
const char *string = NULL;
string = get_symtab_string(img_cfg, nlist_entry->n_un.n_strx);
/* Add the slide to get the *real* address in the process. */
const uint64_t addr = nlist_entry->n_value;
void *ptr = (void*)(addr + img_cfg->image_offset);
/*
* If the user passes a name, match against the name
* If the user passes an address, match against that.
*/
if ((sym_data->name && string && strcmp(sym_data->name, string+1) == 0) || (sym_data->address && ptr == sym_data->address)) {
if (!sym_data->address)
sym_data->address = ptr;
if (!sym_data->name)
sym_data->name = string+1;
sym_data->index = i;
const struct nlist_64 *next_entry = NULL;
/*
* There can be multiple entries in the symbol table with the same n_value (address).
* This means that the 'next' one isn't always good enough. We have to make sure it's
* really a different symbol.
*/
j = 1;
while (next_entry == NULL) {
const struct nlist_64 *tmp_entry = img_cfg->sorted_symbol_table[i + j];
if (nlist_entry->n_value != tmp_entry->n_value)
next_entry = tmp_entry;
j++;
}
/*
* Subtract our address from the address of the next symbol to get it's rough size.
* My observation is that the start of the next symbol will be padded to 16 byte alignment from the end of this one.
* This should be fine, since the point of getting the size is just to minimize scan area for tramp insertions.
*/
sym_data->size = (next_entry->n_value - addr);
break;
}
}
}
static void
free_mach_config(struct mach_config *cfg) {
if (cfg == &ruby_img_cfg)
return;
munmap_file(&cfg->file);
free(cfg);
}
static struct mach_config *
mach_config_for_index(unsigned int index) {
if (index >= _dyld_image_count())
return NULL;
if (index == ruby_img_cfg.index)
return &ruby_img_cfg;
const struct dyld_image_info *image = dyld_get_image_info_for_index(index);
struct mach_config *cfg = calloc(1, sizeof(struct mach_config));
cfg->index = index;
cfg->file.name = cfg->filename = image->imageFilePath;
if (mmap_file_open(&cfg->file) < 0)
errx(EX_OSFILE, "Failed to fread() file %s", cfg->filename);
cfg->image_offset = _dyld_get_image_vmaddr_slide(index);
cfg->load_addr = image->imageLoadAddress;
struct mach_header_64 *hdr = (struct mach_header_64*) cfg->file.data;
assert(hdr);
if (hdr->magic == FAT_CIGAM) {
unsigned int j;
struct fat_header *fat = (struct fat_header *)hdr;
for(j=0; j < OSSwapInt32(fat->nfat_arch); j++) {
struct fat_arch *arch = (struct fat_arch *)((char*)fat + sizeof(struct fat_header) + sizeof(struct fat_arch) * j);
if (OSSwapInt32(arch->cputype) == CPU_TYPE_X86_64) {
hdr = (struct mach_header_64 *)(cfg->file.data + OSSwapInt32(arch->offset));
break;
}
}
}
if (hdr->magic != MH_MAGIC_64) {
printf("Magic for Ruby Mach-O file doesn't match\n");
munmap_file(&cfg->file);
free(cfg);
return NULL;
}
extract_symbol_table(hdr, cfg);
cfg->hdr = (const struct mach_header *)hdr;
return cfg;
}
void *
bin_find_symbol(const char *symbol, size_t *size, int search_libs) {
struct symbol_data sym_data;
memset(&sym_data, 0, sizeof(struct symbol_data));
sym_data.name = symbol;
extract_symbol_data(&ruby_img_cfg, &sym_data);
if (!sym_data.address && search_libs) {
int i, header_count = _dyld_image_count();
for (i=0; i < header_count; i++) {
const struct dyld_image_info *image = dyld_get_image_info_for_index(i);
if ((void*)image->imageLoadAddress == &_mh_bundle_header ||
(void*)image->imageLoadAddress == &_mh_execute_header)
continue;
struct mach_config *cfg = mach_config_for_index(i);
if (cfg) {
extract_symbol_data(cfg, &sym_data);
if (sym_data.address == image->imageLoadAddress) { // wtf? this happens for mysql_api.bundle with mysql_real_query
sym_data.address = 0;
} else if (sym_data.address) {
if (cfg->image_offset == 0) // another wtf? happens on libSystem.dylib where we need to add load address, but libmysqlclient.dylib etc are fine
sym_data.address = (char*)image->imageLoadAddress + (size_t)sym_data.address;
dbg_printf("found symbol %s in %s: %p\n", sym_data.name, image->imageFilePath, sym_data.address);
free_mach_config(cfg);
break;
}
free_mach_config(cfg);
}
}
}
if (size)
*size = sym_data.size;
return sym_data.address;
}
/*
* Do the same thing as in bin_find_symbol above, but compare addresses and return the string name.
*/
const char *
bin_find_symbol_name(void *symbol) {
struct symbol_data sym_data;
memset(&sym_data, 0, sizeof(struct symbol_data));
sym_data.address = symbol;
extract_symbol_data(&ruby_img_cfg, &sym_data);
return sym_data.name;
}
/*
* I will explain bin_update_image with imaginary Ruby code:
*
* Process.mach_images.each do |image|
* image.segments.each do |segment|
* segment.sections.each do |section|
* if section.name == "__text"
* tramp_normal_callsites(section)
* elsif section.name =~ /__symbol_stub/ && image.filename =~ /libruby\.dylib|bundle/
* tramp_dyld_stubs(section)
* end
* end
* end
* end
*/
int
bin_update_image(const char *trampee, struct tramp_st2_entry *tramp, void **orig_function)
{
int ret = -1;
int i;
int header_count = _dyld_image_count();
void *trampee_addr = bin_find_symbol(trampee, NULL, 0);
// Go through all the mach objects that are loaded into this process
for (i=0; i < header_count; i++) {
const struct mach_header *current_hdr = NULL;
if ((void*)_dyld_get_image_header(i) == &_mh_bundle_header)
continue; // always ignore memprof.bundle
struct mach_config *cfg = mach_config_for_index(i);
if (cfg->filename && strstr(cfg->filename, "libSystem") != NULL) {
free_mach_config(cfg);
continue; // ignore libSystem
}
void *stub = find_stub_addr(trampee, cfg);
if (stub) {
struct dyld_stub_entry *entry = (struct dyld_stub_entry *)stub;
if (orig_function)
*orig_function = get_dyld_stub_target(entry);
set_dyld_stub_target(entry, tramp->addr);
ret = 0;
} else if (trampee_addr) {
if ((current_hdr = should_update_image(i)) == NULL)
continue;
if (update_bin_for_mach_header(current_hdr, _dyld_get_image_vmaddr_slide(i), trampee_addr, tramp) == 0) {
ret = 0;
if (orig_function)
*orig_function = trampee_addr;
}
}
free_mach_config(cfg);
}
return ret;
}
void *
do_bin_allocate_page(struct mach_config *cfg)
{
void *ret = NULL;
void *addr = (void *)cfg->load_addr;
size_t i = 0;
dbg_printf("ruby loaded at: %p\n", addr);
/*
* XXX no clue how large the text segment is, so guess.
* TODO remove this.
*/
addr += 65535;
for (; i < INT_MAX - memprof_config.pagesize; i += memprof_config.pagesize, addr += memprof_config.pagesize) {
ret = mmap(addr, memprof_config.pagesize, PROT_WRITE|PROT_READ|PROT_EXEC,
MAP_ANON|MAP_PRIVATE, -1, 0);
if (ret != MAP_FAILED) {
dbg_printf("found a page at: %p\n", ret);
memset(ret, 0x90, memprof_config.pagesize);
return ret;
}
}
return NULL;
}
void *
bin_allocate_page()
{
return do_bin_allocate_page(&ruby_img_cfg);
}
size_t
bin_type_size(const char *type)
{
(void) type;
return 0;
}
int
bin_type_member_offset(const char *type, const char *member)
{
(void) type;
(void) member;
return -1;
}
void
bin_init()
{
void *ptr = NULL;
int index = 0;
Dl_info info;
memset(&ruby_img_cfg, 0, sizeof(struct mach_config));
// We can use this is a reasonably sure method of finding the file
// that the Ruby junk resides in.
ptr = dlsym(RTLD_DEFAULT, "rb_newobj");
if (!ptr)
errx(EX_SOFTWARE, "Could not find rb_newobj in this process. WTF???");
if (!dladdr(ptr, &info) || !info.dli_fname)
errx(EX_SOFTWARE, "Could not find the Mach object associated with rb_newobj.");
ruby_img_cfg.file.name = ruby_img_cfg.filename = info.dli_fname;
if (mmap_file_open(&ruby_img_cfg.file) < 0)
errx(EX_OSFILE, "Failed to fread() file %s", ruby_img_cfg.filename);
struct mach_header_64 *hdr = (struct mach_header_64*) ruby_img_cfg.file.data;
assert(hdr);
ruby_img_cfg.hdr = (const struct mach_header *)hdr;
if (hdr->magic != MH_MAGIC_64)
errx(EX_SOFTWARE, "Magic for Ruby Mach-O file doesn't match");
index = find_dyld_image_index((const struct mach_header_64*) info.dli_fbase);
ruby_img_cfg.image_offset = _dyld_get_image_vmaddr_slide(index);
ruby_img_cfg.index = index;
ruby_img_cfg.load_addr = dyld_get_image_info_for_index(index)->imageLoadAddress;
extract_symbol_table(hdr, &ruby_img_cfg);
assert(ruby_img_cfg.symbol_table != NULL);
assert(ruby_img_cfg.string_table != NULL);
assert(ruby_img_cfg.symbol_count > 0);
// XXX: do not free this, since we're using the symbol and string tables from inside the file
// free(hdr);
}
#endif
================================================
FILE: ext/memprof.c
================================================
#include <ruby.h>
#ifndef _GNU_SOURCE
#define _GNU_SOURCE
#endif
#include <assert.h>
#include <err.h>
#include <fcntl.h>
#include <inttypes.h>
#include <stddef.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sysexits.h>
#include <st.h>
#include <intern.h>
#include <node.h>
#include "arch.h"
#include "bin_api.h"
#include "tracer.h"
#include "tramp.h"
#include "util.h"
/*
* bleak_house stuff
*/
static VALUE eUnsupported;
static int track_objs = 0;
static int memprof_started = 0;
static st_table *objs = NULL;
/*
* stuff needed for heap dumping
*/
static VALUE (*rb_classname)(VALUE);
static RUBY_DATA_FUNC *rb_bm_mark;
static RUBY_DATA_FUNC *rb_blk_free;
static RUBY_DATA_FUNC *rb_thread_mark;
struct memprof_config memprof_config;
/*
* memprof config struct init
*/
static void init_memprof_config_base();
static void init_memprof_config_extended();
struct obj_track {
VALUE obj;
char *source;
int line;
int len;
struct timeval time[];
};
static VALUE gc_hook;
static void **ptr_to_rb_mark_table_add_filename = NULL;
static void (*rb_mark_table_add_filename)(char*);
static void (*rb_add_freelist)(VALUE);
static int
ree_sourcefile_mark_each(st_data_t key, st_data_t val, st_data_t arg)
{
struct obj_track *tracker = (struct obj_track *)val;
assert(tracker != NULL);
if (tracker->source)
rb_mark_table_add_filename(tracker->source);
return ST_CONTINUE;
}
static int
mri_sourcefile_mark_each(st_data_t key, st_data_t val, st_data_t arg)
{
struct obj_track *tracker = (struct obj_track *)val;
assert(tracker != NULL);
if (tracker->source)
(tracker->source)[-1] = 1;
return ST_CONTINUE;
}
/* Accomodate the different source file marking techniques of MRI and REE.
*
* The function pointer for REE changes depending on whether COW is enabled,
* which can be toggled at runtime. We need to deference it to get the
* real function every time we come here, as it may have changed.
*/
static void
sourcefile_marker()
{
if (ptr_to_rb_mark_table_add_filename) {
rb_mark_table_add_filename = *ptr_to_rb_mark_table_add_filename;
assert(rb_mark_table_add_filename != NULL);
st_foreach(objs, ree_sourcefile_mark_each, (st_data_t)NULL);
} else {
st_foreach(objs, mri_sourcefile_mark_each, (st_data_t)NULL);
}
}
static VALUE
newobj_tramp()
{
VALUE ret = rb_newobj();
struct obj_track *tracker = NULL;
if (track_objs && objs) {
tracker = malloc(sizeof(*tracker) + sizeof(struct timeval));
if (tracker) {
if (ruby_current_node && ruby_current_node->nd_file &&
*ruby_current_node->nd_file) {
tracker->source = ruby_current_node->nd_file;
tracker->line = nd_line(ruby_current_node);
} else if (ruby_sourcefile) {
tracker->source = ruby_sourcefile;
tracker->line = ruby_sourceline;
} else {
tracker->source = NULL;
tracker->line = 0;
}
tracker->obj = ret;
tracker->len = 1;
/* TODO a way for the user to disallow time tracking */
if (gettimeofday(&tracker->time[0], NULL) == -1) {
perror("gettimeofday failed. Continuing anyway, error");
}
rb_gc_disable();
st_insert(objs, (st_data_t)ret, (st_data_t)tracker);
rb_gc_enable();
} else {
fprintf(stderr, "Warning, unable to allocate a tracker. "
"You are running dangerously low on RAM!\n");
}
}
return ret;
}
static void
freelist_tramp(unsigned long rval)
{
struct obj_track *tracker = NULL;
if (rb_add_freelist) {
rb_add_freelist(rval);
}
if (track_objs && objs) {
st_delete(objs, (st_data_t *) &rval, (st_data_t *) &tracker);
if (tracker) {
free(tracker);
}
}
}
static int
objs_free(st_data_t key, st_data_t record, st_data_t arg)
{
struct obj_track *tracker = (struct obj_track *)record;
free(tracker);
return ST_DELETE;
}
static int
objs_tabulate(st_data_t key, st_data_t record, st_data_t arg)
{
st_table *table = (st_table *)arg;
struct obj_track *tracker = (struct obj_track *)record;
char *source_key = NULL;
unsigned long count = 0;
char *type = NULL;
int bytes_printed = 0;
switch (TYPE(tracker->obj)) {
case T_NONE:
type = "__none__"; break;
case T_BLKTAG:
type = "__blktag__"; break;
case T_UNDEF:
type = "__undef__"; break;
case T_VARMAP:
type = "__varmap__"; break;
case T_SCOPE:
type = "__scope__"; break;
case T_NODE:
type = "__node__"; break;
default:
if (RBASIC(tracker->obj)->klass) {
type = (char*) rb_obj_classname(tracker->obj);
} else {
type = "__unknown__";
}
}
bytes_printed = asprintf(&source_key, "%s:%d:%s", tracker->source ? tracker->source : "__null__", tracker->line, type);
assert(bytes_printed != -1);
st_lookup(table, (st_data_t)source_key, (st_data_t *)&count);
if (st_insert(table, (st_data_t)source_key, ++count)) {
free(source_key);
}
return ST_CONTINUE;
}
struct results {
char **entries;
size_t num_entries;
};
static int
objs_to_array(st_data_t key, st_data_t record, st_data_t arg)
{
struct results *res = (struct results *)arg;
unsigned long count = (unsigned long)record;
char *source = (char *)key;
int bytes_printed = 0;
bytes_printed = asprintf(&(res->entries[res->num_entries++]), "%7li %s", count, source);
assert(bytes_printed != -1);
free(source);
return ST_DELETE;
}
static VALUE
memprof_start(VALUE self)
{
if (!memprof_started) {
insert_tramp("rb_newobj", newobj_tramp);
insert_tramp("add_freelist", freelist_tramp);
memprof_started = 1;
}
if (track_objs == 1)
return Qfalse;
track_objs = 1;
return Qtrue;
}
static VALUE
memprof_stop(VALUE self)
{
/* TODO: remove trampolines and set memprof_started = 0 */
if (track_objs == 0)
return Qfalse;
track_objs = 0;
st_foreach(objs, objs_free, (st_data_t)0);
return Qtrue;
}
static int
memprof_strcmp(const void *obj1, const void *obj2)
{
char *str1 = *(char **)obj1;
char *str2 = *(char **)obj2;
return strcmp(str2, str1);
}
static VALUE
memprof_stats(int argc, VALUE *argv, VALUE self)
{
st_table *tmp_table;
struct results res;
size_t i;
VALUE str;
FILE *out = NULL;
if (!track_objs)
rb_raise(rb_eRuntimeError, "object tracking disabled, call Memprof.start first");
rb_scan_args(argc, argv, "01", &str);
if (RTEST(str)) {
out = fopen(StringValueCStr(str), "w");
if (!out)
rb_raise(rb_eArgError, "unable to open output file");
}
track_objs = 0;
tmp_table = st_init_strtable();
st_foreach(objs, objs_tabulate, (st_data_t)tmp_table);
res.num_entries = 0;
res.entries = malloc(sizeof(char*) * tmp_table->num_entries);
st_foreach(tmp_table, objs_to_array, (st_data_t)&res);
st_free_table(tmp_table);
qsort(res.entries, res.num_entries, sizeof(char*), &memprof_strcmp);
for (i=0; i < res.num_entries; i++) {
fprintf(out ? out : stderr, "%s\n", res.entries[i]);
free(res.entries[i]);
}
free(res.entries);
if (out)
fclose(out);
track_objs = 1;
return Qnil;
}
static VALUE
memprof_stats_bang(int argc, VALUE *argv, VALUE self)
{
memprof_stats(argc, argv, self);
st_foreach(objs, objs_free, (st_data_t)0);
return Qnil;
}
static void
json_print(void *ctx, const char * str, unsigned int len)
{
FILE *out = (FILE *)ctx;
size_t written = 0;
while(1) {
written += fwrite(str + written, sizeof(char), len - written, out ? out : stdout);
if (written == len) break;
}
if (str && len > 0 && str[0] == '\n' && out)
fflush(out);
}
static VALUE
memprof_track(int argc, VALUE *argv, VALUE self)
{
if (!rb_block_given_p())
rb_raise(rb_eArgError, "block required");
memprof_start(self);
rb_yield(Qnil);
memprof_stats(argc, argv, self);
memprof_stop(self);
return Qnil;
}
static json_gen_status
json_gen_id(json_gen gen, ID id)
{
if (id) {
if (id < 100)
return json_gen_format(gen, ":%c", id);
else
return json_gen_format(gen, ":%s", rb_id2name(id));
} else
return json_gen_null(gen);
}
static json_gen_status
json_gen_value(json_gen gen, VALUE obj)
{
if (FIXNUM_P(obj))
return json_gen_integer(gen, NUM2LONG(obj));
else if (NIL_P(obj) || obj == Qundef)
return json_gen_null(gen);
else if (obj == Qtrue)
return json_gen_bool(gen, 1);
else if (obj == Qfalse)
return json_gen_bool(gen, 0);
else if (SYMBOL_P(obj))
return json_gen_id(gen, SYM2ID(obj));
else
return json_gen_pointer(gen, (void*)obj);
}
static json_gen_config fancy_conf = { .beautify = 1, .indentString = " " };
static json_gen_config basic_conf = { .beautify = 0, .indentString = " " };
static json_gen
json_for_args(int argc, VALUE *argv)
{
FILE *out = NULL;
VALUE str;
rb_scan_args(argc, argv, "01", &str);
if (RTEST(str)) {
out = fopen(StringValueCStr(str), "w");
if (!out)
rb_raise(rb_eArgError, "unable to open output file");
}
if (!out)
out = stderr;
json_gen gen = json_gen_alloc2((json_print_t)&json_print, out == stderr ? &fancy_conf : &basic_conf, NULL, (void*)out);
return gen;
}
static void
json_free(json_gen gen)
{
FILE *out = (FILE*)gen->ctx;
if (out != stderr)
fclose(out);
json_gen_free(gen);
}
static VALUE
memprof_trace(int argc, VALUE *argv, VALUE self)
{
if (!rb_block_given_p())
rb_raise(rb_eArgError, "block required");
json_gen gen = json_for_args(argc, argv);
trace_set_output(gen);
json_gen_map_open(gen);
trace_invoke_all(TRACE_RESET);
trace_invoke_all(TRACE_START);
VALUE ret = rb_yield(Qnil);
trace_invoke_all(TRACE_DUMP);
trace_invoke_all(TRACE_STOP);
json_gen_map_close(gen);
json_gen_reset(gen);
json_free(gen);
trace_set_output(NULL);
return ret;
}
static int
each_request_entry(st_data_t key, st_data_t record, st_data_t arg)
{
json_gen gen = (json_gen)arg;
VALUE k = (VALUE)key;
VALUE v = (VALUE)record;
if (RTEST(v) && BUILTIN_TYPE(v) == T_STRING && RTEST(k) && BUILTIN_TYPE(k) == T_STRING &&
RSTRING_PTR(k)[0] >= 65 && RSTRING_PTR(k)[0] <= 90) {
json_gen_cstr(gen, StringValueCStr(k));
json_gen_cstr(gen, StringValueCStr(v));
}
return ST_CONTINUE;
}
static VALUE tracing_json_filename = Qnil;
static json_gen tracing_json_gen = NULL;
static VALUE
memprof_trace_filename_set(int argc, VALUE *argv, VALUE self)
{
if (tracing_json_gen) {
json_free(tracing_json_gen);
tracing_json_gen = NULL;
}
if (!RTEST(*argv)) {
tracing_json_filename = Qnil;
} else {
tracing_json_gen = json_for_args(argc, argv);
tracing_json_filename = *argv;
}
return tracing_json_filename;
}
static VALUE
memprof_trace_filename_get(VALUE self)
{
return tracing_json_filename;
}
static VALUE
memprof_trace_request(VALUE self, VALUE env)
{
if (!rb_block_given_p())
rb_raise(rb_eArgError, "block required");
uint64_t start_time;
uint64_t end_time;
char str_time[32];
json_gen gen;
if (tracing_json_gen)
gen = tracing_json_gen;
else
gen = json_for_args(0, NULL);
json_gen_map_open(gen);
json_gen_cstr(gen, "start");
start_time = timeofday_ms();
sprintf(str_time, "%" PRIu64, start_time);
json_gen_number(gen, str_time, strlen(str_time));
json_gen_cstr(gen, "tracers");
json_gen_map_open(gen);
trace_set_output(gen);
trace_invoke_all(TRACE_RESET);
trace_invoke_all(TRACE_START);
start_time = timeofday_ms();
VALUE ret = rb_yield(Qnil);
end_time = timeofday_ms();
trace_invoke_all(TRACE_DUMP);
trace_invoke_all(TRACE_STOP);
json_gen_map_close(gen);
if (RTEST(env) && TYPE(env) == T_HASH) {
VALUE val, str;
val = rb_hash_aref(env, rb_str_new2("action_controller.request.path_parameters"));
if (!RTEST(val))
val = rb_hash_aref(env, rb_str_new2("action_dispatch.request.parameters"));
if (RTEST(val) && TYPE(val) == T_HASH) {
json_gen_cstr(gen, "rails");
json_gen_map_open(gen);
str = rb_hash_aref(val, rb_str_new2("controller"));
if (RTEST(str) && TYPE(str) == T_STRING) {
json_gen_cstr(gen, "controller");
json_gen_cstr(gen, RSTRING_PTR(str));
}
str = rb_hash_aref(val, rb_str_new2("action"));
if (RTEST(str) && TYPE(str) == T_STRING) {
json_gen_cstr(gen, "action");
json_gen_cstr(gen, RSTRING_PTR(str));
}
json_gen_map_close(gen);
}
json_gen_cstr(gen, "request");
json_gen_map_open(gen);
// struct RHash *hash = RHASH(env);
// st_foreach(hash->tbl, each_request_entry, (st_data_t)gen);
#define DUMP_HASH_ENTRY(key) do { \
str = rb_hash_aref(env, rb_str_new2(key)); \
if (RTEST(str) && \
TYPE(str) == T_STRING && \
RSTRING_PTR(str)) { \
json_gen_cstr(gen, key); \
json_gen_cstr(gen, RSTRING_PTR(str)); \
} \
} while(0)
// DUMP_HASH_ENTRY("HTTP_USER_AGENT");
DUMP_HASH_ENTRY("REQUEST_PATH");
DUMP_HASH_ENTRY("PATH_INFO");
DUMP_HASH_ENTRY("REMOTE_ADDR");
DUMP_HASH_ENTRY("REQUEST_URI");
DUMP_HASH_ENTRY("REQUEST_METHOD");
DUMP_HASH_ENTRY("QUERY_STRING");
json_gen_map_close(gen);
}
if (RTEST(ret) && TYPE(ret) == T_ARRAY) {
json_gen_cstr(gen, "response");
json_gen_map_open(gen);
json_gen_cstr(gen, "code");
json_gen_value(gen, RARRAY_PTR(ret)[0]);
json_gen_map_close(gen);
}
json_gen_cstr(gen, "time");
json_gen_integer(gen, end_time-start_time);
json_gen_map_close(gen);
json_gen_reset(gen);
if (gen != tracing_json_gen)
json_free(gen);
return ret;
}
#include "json.h"
#include "env.h"
#include "rubyio.h"
#include "re.h"
#ifndef RARRAY_PTR
#define RARRAY_PTR(ary) RARRAY(ary)->ptr
#endif
#ifndef RARRAY_LEN
#define RARRAY_LEN(ary) RARRAY(ary)->len
#endif
#ifndef RSTRING_PTR
#define RSTRING_PTR(str) RSTRING(str)->ptr
#endif
#ifndef RSTRING_LEN
#define RSTRING_LEN(str) RSTRING(str)->len
#endif
static int
each_hash_entry(st_data_t key, st_data_t record, st_data_t arg)
{
json_gen gen = (json_gen)arg;
VALUE k = (VALUE)key;
VALUE v = (VALUE)record;
json_gen_array_open(gen);
json_gen_value(gen, k);
json_gen_value(gen, v);
json_gen_array_close(gen);
return ST_CONTINUE;
}
static int
each_ivar(st_data_t key, st_data_t record, st_data_t arg)
{
json_gen gen = (json_gen)arg;
ID id = (ID)key;
VALUE val = (VALUE)record;
const char *name = rb_id2name(id);
json_gen_cstr(gen, name ? name : "(none)");
json_gen_value(gen, val);
return ST_CONTINUE;
}
static char *
nd_type_str(VALUE obj)
{
switch(nd_type(obj)) {
#define ND(type) case NODE_##type: return #type;
ND(METHOD); ND(FBODY); ND(CFUNC); ND(SCOPE);
ND(BLOCK); ND(IF); ND(CASE); ND(WHEN);
ND(OPT_N); ND(WHILE); ND(UNTIL); ND(ITER);
ND(FOR); ND(BREAK); ND(NEXT); ND(REDO);
ND(RETRY); ND(BEGIN); ND(RESCUE); ND(RESBODY);
ND(ENSURE); ND(AND); ND(OR); ND(NOT);
ND(MASGN); ND(LASGN); ND(DASGN); ND(DASGN_CURR);
ND(GASGN); ND(IASGN); ND(CDECL); ND(CVASGN);
ND(CVDECL); ND(OP_ASGN1); ND(OP_ASGN2); ND(OP_ASGN_AND);
ND(OP_ASGN_OR); ND(CALL); ND(FCALL); ND(VCALL);
ND(SUPER); ND(ZSUPER); ND(ARRAY); ND(ZARRAY);
ND(HASH); ND(RETURN); ND(YIELD); ND(LVAR);
ND(DVAR); ND(GVAR); ND(IVAR); ND(CONST);
ND(CVAR); ND(NTH_REF); ND(BACK_REF); ND(MATCH);
ND(MATCH2); ND(MATCH3); ND(LIT); ND(STR);
ND(DSTR); ND(XSTR); ND(DXSTR); ND(EVSTR);
ND(DREGX); ND(DREGX_ONCE); ND(ARGS); ND(ARGSCAT);
ND(ARGSPUSH); ND(SPLAT); ND(TO_ARY); ND(SVALUE);
ND(BLOCK_ARG); ND(BLOCK_PASS); ND(DEFN); ND(DEFS);
ND(ALIAS); ND(VALIAS); ND(UNDEF); ND(CLASS);
ND(MODULE); ND(SCLASS); ND(COLON2); ND(COLON3)
ND(CREF); ND(DOT2); ND(DOT3); ND(FLIP2);
ND(FLIP3); ND(ATTRSET); ND(SELF); ND(NIL);
ND(TRUE); ND(FALSE); ND(DEFINED); ND(NEWLINE);
ND(POSTEXE); ND(ALLOCA); ND(DMETHOD); ND(BMETHOD);
ND(MEMO); ND(IFUNC); ND(DSYM); ND(ATTRASGN);
ND(LAST);
default:
return "unknown";
}
}
static inline void
obj_dump_class(json_gen gen, VALUE obj)
{
if (RBASIC(obj)->klass) {
json_gen_cstr(gen, "class");
json_gen_value(gen, RBASIC(obj)->klass);
VALUE name = rb_classname(RBASIC(obj)->klass);
if (RTEST(name)) {
json_gen_cstr(gen, "class_name");
json_gen_cstr(gen, RSTRING_PTR(name));
}
}
}
/* TODO
* look for FL_EXIVAR flag and print ivars
* print more detail about Proc/struct BLOCK in T_DATA if freefunc == blk_free
* add Memprof.dump_all for full heap dump
* print details on different types of nodes (nd_next, nd_lit, nd_nth, etc)
*/
static void
obj_dump(VALUE obj, json_gen gen)
{
int type;
json_gen_map_open(gen);
json_gen_cstr(gen, "_id");
json_gen_value(gen, obj);
struct obj_track *tracker = NULL;
if (st_lookup(objs, (st_data_t)obj, (st_data_t *)&tracker) && BUILTIN_TYPE(obj) != T_NODE) {
json_gen_cstr(gen, "file");
json_gen_cstr(gen, tracker->source);
json_gen_cstr(gen, "line");
json_gen_integer(gen, tracker->line);
json_gen_cstr(gen, "time");
json_gen_integer(gen, (tracker->time[0].tv_sec * 1000000) + tracker->time[0].tv_usec);
}
json_gen_cstr(gen, "type");
switch (type=BUILTIN_TYPE(obj)) {
case T_DATA:
json_gen_cstr(gen, "data");
obj_dump_class(gen, obj);
if (DATA_PTR(obj)) {
json_gen_cstr(gen, "data");
json_gen_pointer(gen, DATA_PTR(obj));
}
if (RDATA(obj)->dfree == (RUBY_DATA_FUNC)rb_blk_free) {
void *val;
VALUE ptr;
val = *(void**)(DATA_PTR(obj) + memprof_config.offset_BLOCK_body);
if (val) {
json_gen_cstr(gen, "nd_body");
json_gen_pointer(gen, val);
}
val = *(void**)(DATA_PTR(obj) + memprof_config.offset_BLOCK_var);
if (val) {
json_gen_cstr(gen, "nd_var");
json_gen_pointer(gen, val);
}
val = *(void**)(DATA_PTR(obj) + memprof_config.offset_BLOCK_cref);
if (val) {
json_gen_cstr(gen, "nd_cref");
json_gen_pointer(gen, val);
}
val = *(void**)(DATA_PTR(obj) + memprof_config.offset_BLOCK_dyna_vars);
if (val) {
json_gen_cstr(gen, "vars");
json_gen_pointer(gen, val);
}
val = *(void**)(DATA_PTR(obj) + memprof_config.offset_BLOCK_scope);
if (val) {
json_gen_cstr(gen, "scope");
json_gen_pointer(gen, val);
}
ptr = *(VALUE*)(DATA_PTR(obj) + memprof_config.offset_BLOCK_self);
json_gen_cstr(gen, "self");
json_gen_value(gen, ptr);
ptr = *(VALUE*)(DATA_PTR(obj) + memprof_config.offset_BLOCK_klass);
json_gen_cstr(gen, "klass");
json_gen_value(gen, ptr);
ptr = *(VALUE*)(DATA_PTR(obj) + memprof_config.offset_BLOCK_orig_thread);
json_gen_cstr(gen, "thread");
json_gen_value(gen, ptr);
ptr = *(VALUE*)(DATA_PTR(obj) + memprof_config.offset_BLOCK_wrapper);
if (RTEST(ptr)) {
json_gen_cstr(gen, "wrapper");
json_gen_value(gen, ptr);
}
ptr = *(VALUE*)(DATA_PTR(obj) + memprof_config.offset_BLOCK_block_obj);
if (RTEST(ptr)) {
json_gen_cstr(gen, "block");
json_gen_value(gen, ptr);
}
/* TODO: is .prev actually useful? refers to non-heap allocated struct BLOCKs,
* but we don't print out any information about those
*/
/*
json_gen_cstr(gen, "prev");
json_gen_array_open(gen);
val = *(void**)(DATA_PTR(obj) + memprof_config.offset_BLOCK_prev);
while (val) {
json_gen_pointer(gen, val);
prev = val;
val = *(void**)(ptr + memprof_config.offset_BLOCK_prev);
if (prev == val)
break;
}
json_gen_array_close(gen);
*/
} else if (RDATA(obj)->dmark == (RUBY_DATA_FUNC)rb_bm_mark) {
VALUE ptr;
ID id, mid;
ptr = *(VALUE*)(DATA_PTR(obj) + memprof_config.offset_METHOD_klass);
if (RTEST(ptr)) {
json_gen_cstr(gen, "klass");
json_gen_value(gen, ptr);
}
ptr = *(VALUE*)(DATA_PTR(obj) + memprof_config.offset_METHOD_rklass);
if (RTEST(ptr)) {
json_gen_cstr(gen, "rklass");
json_gen_value(gen, ptr);
}
ptr = *(VALUE*)(DATA_PTR(obj) + memprof_config.offset_METHOD_recv);
if (RTEST(ptr)) {
json_gen_cstr(gen, "recv");
json_gen_value(gen, ptr);
}
ptr = *(VALUE*)(DATA_PTR(obj) + memprof_config.offset_METHOD_body);
if (RTEST(ptr)) {
json_gen_cstr(gen, "node");
json_gen_value(gen, ptr);
}
mid = *(ID*)(DATA_PTR(obj) + memprof_config.offset_METHOD_id);
if (mid) {
json_gen_cstr(gen, "mid");
json_gen_id(gen, mid);
}
id = *(ID*)(DATA_PTR(obj) + memprof_config.offset_METHOD_oid);
if (id && id != mid) {
json_gen_cstr(gen, "oid");
json_gen_id(gen, id);
}
} else if (RDATA(obj)->dmark == (RUBY_DATA_FUNC)rb_thread_mark) {
rb_thread_t th = (rb_thread_t)DATA_PTR(obj);
if (th == rb_curr_thread) {
json_gen_cstr(gen, "current");
json_gen_bool(gen, 1);
} else {
if (th->dyna_vars) {
json_gen_cstr(gen, "varmap");
json_gen_pointer(gen, th->dyna_vars);
}
json_gen_cstr(gen, "node");
json_gen_pointer(gen, th->node);
json_gen_cstr(gen, "cref");
json_gen_pointer(gen, th->cref);
char *status;
switch (th->status) {
case THREAD_TO_KILL:
status = "to_kill";
break;
case THREAD_RUNNABLE:
status = "runnable";
break;
case THREAD_STOPPED:
status = "stopped";
break;
case THREAD_KILLED:
status = "killed";
break;
default:
status = "unknown";
}
json_gen_cstr(gen, "status");
json_gen_cstr(gen, status);
#define WAIT_FD (1<<0)
#define WAIT_SELECT (1<<1)
#define WAIT_TIME (1<<2)
#define WAIT_JOIN (1<<3)
#define WAIT_PID (1<<4)
json_gen_cstr(gen, "wait_for");
json_gen_array_open(gen);
if (th->wait_for & WAIT_FD)
json_gen_cstr(gen, "fd");
if (th->wait_for & WAIT_SELECT)
json_gen_cstr(gen, "select");
if (th->wait_for & WAIT_TIME)
json_gen_cstr(gen, "time");
if (th->wait_for & WAIT_JOIN)
json_gen_cstr(gen, "join");
if (th->wait_for & WAIT_PID)
json_gen_cstr(gen, "pid");
json_gen_array_close(gen);
if (th->wait_for & WAIT_FD) {
json_gen_cstr(gen, "fd");
json_gen_integer(gen, th->fd);
}
#define DELAY_INFTY 1E30
if (th->wait_for & WAIT_TIME) {
json_gen_cstr(gen, "delay");
if (th->delay == DELAY_INFTY)
json_gen_cstr(gen, "infinity");
else
json_gen_double(gen, th->delay - timeofday());
}
if (th->wait_for & WAIT_JOIN) {
json_gen_cstr(gen, "join");
json_gen_value(gen, th->join->thread);
}
}
json_gen_cstr(gen, "priority");
json_gen_integer(gen, th->priority);
if (th == rb_main_thread) {
json_gen_cstr(gen, "main");
json_gen_bool(gen, 1);
}
if (th->next && th->next != rb_main_thread) {
json_gen_cstr(gen, "next");
json_gen_value(gen, th->next->thread);
}
if (th->prev && th->prev != th && (th->prev == rb_main_thread || th->prev != th->next)) {
json_gen_cstr(gen, "prev");
json_gen_value(gen, th->prev->thread);
}
if (th->locals) {
json_gen_cstr(gen, "variables");
json_gen_map_open(gen);
st_foreach(th->locals, each_ivar, (st_data_t)gen);
json_gen_map_close(gen);
}
}
break;
case T_STRUCT:
json_gen_cstr(gen, "struct");
obj_dump_class(gen, obj);
break;
case T_FILE:
json_gen_cstr(gen, "file");
obj_dump_class(gen, obj);
OpenFile *file = RFILE(obj)->fptr;
if (file->f) {
json_gen_cstr(gen, "fileno");
json_gen_integer(gen, fileno(file->f));
}
if (file->f2) {
json_gen_cstr(gen, "fileno2");
json_gen_integer(gen, fileno(file->f2));
}
if (file->pid) {
json_gen_cstr(gen, "pid");
json_gen_integer(gen, file->pid);
}
if (file->path) {
json_gen_cstr(gen, "path");
json_gen_cstr(gen, file->path);
}
if (file->mode) {
json_gen_cstr(gen, "mode");
json_gen_array_open(gen);
if (file->mode & FMODE_READABLE)
json_gen_cstr(gen, "readable");
if (file->mode & FMODE_WRITABLE)
json_gen_cstr(gen, "writable");
if (file->mode & FMODE_READWRITE)
json_gen_cstr(gen, "readwrite");
if (file->mode & FMODE_APPEND)
json_gen_cstr(gen, "append");
if (file->mode & FMODE_CREATE)
json_gen_cstr(gen, "create");
if (file->mode & FMODE_BINMODE)
json_gen_cstr(gen, "binmode");
if (file->mode & FMODE_SYNC)
json_gen_cstr(gen, "sync");
if (file->mode & FMODE_WBUF)
json_gen_cstr(gen, "wbuf");
if (file->mode & FMODE_RBUF)
json_gen_cstr(gen, "rbuf");
if (file->mode & FMODE_WSPLIT)
json_gen_cstr(gen, "wsplit");
if (file->mode & FMODE_WSPLIT_INITIALIZED)
json_gen_cstr(gen, "wsplit_initialized");
json_gen_array_close(gen);
}
break;
case T_FLOAT:
json_gen_cstr(gen, "float");
obj_dump_class(gen, obj);
json_gen_cstr(gen, "data");
json_gen_double(gen, RFLOAT(obj)->value);
break;
case T_BIGNUM:
json_gen_cstr(gen, "bignum");
obj_dump_class(gen, obj);
json_gen_cstr(gen, "negative");
json_gen_bool(gen, RBIGNUM(obj)->sign == 0);
json_gen_cstr(gen, "length");
json_gen_integer(gen, RBIGNUM(obj)->len);
json_gen_cstr(gen, "data");
json_gen_string(gen, RBIGNUM(obj)->digits, RBIGNUM(obj)->len);
break;
case T_MATCH:
json_gen_cstr(gen, "match");
obj_dump_class(gen, obj);
json_gen_cstr(gen, "data");
json_gen_value(gen, RMATCH(obj)->str);
break;
case T_REGEXP:
json_gen_cstr(gen, "regexp");
obj_dump_class(gen, obj);
json_gen_cstr(gen, "length");
json_gen_integer(gen, RREGEXP(obj)->len);
json_gen_cstr(gen, "data");
json_gen_cstr(gen, RREGEXP(obj)->str);
break;
case T_SCOPE:
json_gen_cstr(gen, "scope");
struct SCOPE *scope = (struct SCOPE *)obj;
if (scope->local_tbl) {
int i = 0;
int n = scope->local_tbl[0];
VALUE *list = &scope->local_vars[-1];
VALUE cur = *list++;
if (RTEST(cur)) {
json_gen_cstr(gen, "node");
json_gen_value(gen, cur);
}
if (n) {
json_gen_cstr(gen, "variables");
json_gen_map_open(gen);
while (n--) {
cur = *list++;
i++;
if (!rb_is_local_id(scope->local_tbl[i]))
continue;
json_gen_id(gen, scope->local_tbl[i]);
json_gen_value(gen, cur);
}
json_gen_map_close(gen);
}
}
break;
case T_NODE:
json_gen_cstr(gen, "node");
json_gen_cstr(gen, "node_type");
json_gen_cstr(gen, nd_type_str(obj));
json_gen_cstr(gen, "file");
json_gen_cstr(gen, RNODE(obj)->nd_file);
json_gen_cstr(gen, "line");
json_gen_integer(gen, nd_line(obj));
json_gen_cstr(gen, "node_code");
json_gen_integer(gen, nd_type(obj));
#define PRINT_ID(sub) json_gen_id(gen, RNODE(obj)->sub.id)
#define PRINT_VAL(sub) json_gen_value(gen, RNODE(obj)->sub.value)
int nd_type = nd_type(obj);
json_gen_cstr(gen, "n1");
switch(nd_type) {
case NODE_LVAR:
case NODE_DVAR:
case NODE_IVAR:
case NODE_CVAR:
case NODE_GVAR:
case NODE_CONST:
case NODE_ATTRSET:
case NODE_LASGN:
case NODE_IASGN:
case NODE_DASGN:
case NODE_CVASGN:
case NODE_CVDECL:
case NODE_GASGN:
case NODE_DASGN_CURR:
case NODE_BLOCK_ARG:
case NODE_CDECL:
case NODE_VALIAS:
PRINT_ID(u1);
break;
case NODE_OP_ASGN2:
if (RNODE(obj)->u3.id > 1000000)
PRINT_VAL(u1);
else
PRINT_ID(u1);
break;
case NODE_SCOPE: {
ID *tbl = RNODE(obj)->nd_tbl;
json_gen_array_open(gen);
if (tbl) {
int size = tbl[0];
int i = 3;
for (; i < size+1; i++) {
json_gen_id(gen, tbl[i]);
}
}
json_gen_array_close(gen);
break;
}
case NODE_IFUNC:
case NODE_CFUNC: {
const char *name = bin_find_symbol_name((void*)RNODE(obj)->u1.value);
json_gen_format(gen, "0x%x: %s", RNODE(obj)->u1.value, name ? name : "???");
break;
}
default:
PRINT_VAL(u1);
}
json_gen_cstr(gen, "n2");
switch(nd_type) {
case NODE_CALL:
case NODE_FBODY:
case NODE_DEFN:
case NODE_ATTRASGN:
case NODE_FCALL:
case NODE_VCALL:
case NODE_COLON2:
case NODE_COLON3:
case NODE_BACK_REF:
case NODE_DEFS:
case NODE_VALIAS:
PRINT_ID(u2);
break;
case NODE_OP_ASGN1:
if (RNODE(obj)->nd_mid == 0)
json_gen_cstr(gen, ":||");
else if (RNODE(obj)->nd_mid == 1)
json_gen_cstr(gen, ":&&");
else
PRINT_ID(u2);
break;
case NODE_OP_ASGN2:
if (RNODE(obj)->u3.id > 1000000) {
PRINT_VAL(u2);
} else {
if (RNODE(obj)->nd_mid == 0)
json_gen_cstr(gen, ":||");
else if (RNODE(obj)->nd_mid == 1)
json_gen_cstr(gen, ":&&");
else
PRINT_ID(u2);
}
break;
case NODE_DREGX:
case NODE_DREGX_ONCE:
case NODE_NTH_REF:
case NODE_IFUNC:
case NODE_CFUNC:
case NODE_NEWLINE:
json_gen_integer(gen, RNODE(obj)->u2.argc);
break;
case NODE_BLOCK:
case NODE_ARRAY:
if (RNODE(obj)->u2.node == RNODE(obj))
json_gen_null(gen);
else
PRINT_VAL(u2);
break;
default:
PRINT_VAL(u2);
}
json_gen_cstr(gen, "n3");
switch(nd_type) {
case NODE_ARGS:
json_gen_integer(gen, RNODE(obj)->u3.cnt);
break;
case NODE_OP_ASGN2:
if (RNODE(obj)->u3.id > 1000000)
PRINT_VAL(u3);
else
PRINT_ID(u3);
break;
default:
PRINT_VAL(u3);
}
break;
case T_STRING:
json_gen_cstr(gen, "string");
obj_dump_class(gen, obj);
json_gen_cstr(gen, "length");
json_gen_integer(gen, RSTRING_LEN(obj));
if (FL_TEST(obj, ELTS_SHARED|FL_USER3)) {
json_gen_cstr(gen, "shared");
json_gen_value(gen, RSTRING(obj)->aux.shared);
json_gen_cstr(gen, "flags");
json_gen_array_open(gen);
if (FL_TEST(obj, ELTS_SHARED))
json_gen_cstr(gen, "elts_shared");
if (FL_TEST(obj, FL_USER3))
json_gen_cstr(gen, "str_assoc");
json_gen_array_close(gen);
} else {
json_gen_cstr(gen, "data");
json_gen_string(gen, (unsigned char *)RSTRING_PTR(obj), RSTRING_LEN(obj));
}
break;
case T_VARMAP:
json_gen_cstr(gen, "varmap");
obj_dump_class(gen, obj);
struct RVarmap *vars = (struct RVarmap *)obj;
if (vars->next) {
json_gen_cstr(gen, "next");
json_gen_value(gen, (VALUE)vars->next);
}
if (vars->id) {
json_gen_cstr(gen, "data");
json_gen_map_open(gen);
json_gen_id(gen, vars->id);
json_gen_value(gen, vars->val);
json_gen_map_close(gen);
}
break;
case T_CLASS:
case T_MODULE:
case T_ICLASS:
json_gen_cstr(gen, type==T_CLASS ? "class" : type==T_MODULE ? "module" : "iclass");
obj_dump_class(gen, obj);
json_gen_cstr(gen, "name");
VALUE name = rb_classname(obj);
if (RTEST(name))
json_gen_cstr(gen, RSTRING_PTR(name));
else
json_gen_cstr(gen, 0);
json_gen_cstr(gen, "super");
json_gen_value(gen, RCLASS(obj)->super);
if (RTEST(RCLASS(obj)->super)) {
json_gen_cstr(gen, "super_name");
VALUE super_name = rb_classname(RCLASS(obj)->super);
if (RTEST(super_name))
json_gen_cstr(gen, RSTRING_PTR(super_name));
else
json_gen_cstr(gen, 0);
}
if (FL_TEST(obj, FL_SINGLETON)) {
json_gen_cstr(gen, "singleton");
json_gen_bool(gen, 1);
}
if (RCLASS(obj)->iv_tbl && RCLASS(obj)->iv_tbl->num_entries) {
json_gen_cstr(gen, "ivars");
json_gen_map_open(gen);
st_foreach(RCLASS(obj)->iv_tbl, each_ivar, (st_data_t)gen);
json_gen_map_close(gen);
}
if (RCLASS(obj)->m_tbl && RCLASS(obj)->m_tbl->num_entries) {
json_gen_cstr(gen, "methods");
json_gen_map_open(gen);
st_foreach(RCLASS(obj)->m_tbl, each_ivar, (st_data_t)gen);
json_gen_map_close(gen);
}
break;
case T_OBJECT:
json_gen_cstr(gen, "object");
obj_dump_class(gen, obj);
struct RClass *klass = RCLASS(obj);
if (klass->iv_tbl && klass->iv_tbl->num_entries) {
json_gen_cstr(gen, "ivars");
json_gen_map_open(gen);
st_foreach(klass->iv_tbl, each_ivar, (st_data_t)gen);
json_gen_map_close(gen);
}
break;
case T_ARRAY:
json_gen_cstr(gen, "array");
obj_dump_class(gen, obj);
struct RArray *ary = RARRAY(obj);
json_gen_cstr(gen, "length");
json_gen_integer(gen, ary->len);
if (FL_TEST(obj, ELTS_SHARED)) {
json_gen_cstr(gen, "shared");
json_gen_value(gen, ary->aux.shared);
} else if (ary->len) {
json_gen_cstr(gen, "data");
json_gen_array_open(gen);
int i;
for(i=0; i < ary->len; i++)
json_gen_value(gen, ary->ptr[i]);
json_gen_array_close(gen);
}
break;
case T_HASH:
json_gen_cstr(gen, "hash");
obj_dump_class(gen, obj);
struct RHash *hash = RHASH(obj);
json_gen_cstr(gen, "length");
if (hash->tbl)
json_gen_integer(gen, hash->tbl->num_entries);
else
json_gen_integer(gen, 0);
json_gen_cstr(gen, "default");
json_gen_value(gen, hash->ifnone);
if (hash->tbl && hash->tbl->num_entries) {
json_gen_cstr(gen, "data");
//json_gen_map_open(gen);
json_gen_array_open(gen);
st_foreach(hash->tbl, each_hash_entry, (st_data_t)gen);
json_gen_array_close(gen);
//json_gen_map_close(gen);
}
break;
default:
json_gen_cstr(gen, "unknown");
obj_dump_class(gen, obj);
}
json_gen_cstr(gen, "code");
json_gen_integer(gen, BUILTIN_TYPE(obj));
json_gen_map_close(gen);
}
extern st_table *rb_global_tbl;
static int
globals_each_dump(st_data_t key, st_data_t record, st_data_t arg)
{
json_gen_id((json_gen)arg, (ID)key);
json_gen_value((json_gen)arg, rb_gvar_get((void*)record));
return ST_CONTINUE;
}
static int
finalizers_each_dump(st_data_t key, st_data_t val, st_data_t arg)
{
json_gen gen = (json_gen)arg;
json_gen_array_open(gen);
json_gen_value(gen, (VALUE)key);
json_gen_value(gen, (VALUE)val);
json_gen_array_close(gen);
return ST_CONTINUE;
}
static void
memprof_dump_globals(json_gen gen)
{
json_gen_map_open(gen);
json_gen_cstr(gen, "_id");
json_gen_cstr(gen, "globals");
json_gen_cstr(gen, "type");
json_gen_cstr(gen, "globals");
json_gen_cstr(gen, "variables");
json_gen_map_open(gen);
st_foreach(rb_global_tbl, globals_each_dump, (st_data_t)gen);
json_gen_map_close(gen);
json_gen_map_close(gen);
json_gen_reset(gen);
}
static void
memprof_dump_stack_frame(json_gen gen, struct FRAME *frame)
{
json_gen_map_open(gen);
json_gen_cstr(gen, "_id");
json_gen_pointer(gen, frame);
json_gen_cstr(gen, "type");
json_gen_cstr(gen, "frame");
json_gen_cstr(gen, "self");
json_gen_value(gen, frame->self);
if (frame->last_class) {
json_gen_cstr(gen, "last_class");
json_gen_value(gen, frame->last_class);
}
if (frame->orig_func) {
json_gen_cstr(gen, "orig_func");
json_gen_id(gen, frame->orig_func);
}
if (frame->last_func && frame->last_func != frame->orig_func) {
json_gen_cstr(gen, "last_func");
json_gen_id(gen, frame->last_func);
}
if (frame->node) {
json_gen_cstr(gen, "node");
json_gen_pointer(gen, (void*)frame->node);
}
if (frame->prev) {
json_gen_cstr(gen, "prev");
json_gen_pointer(gen, (void*)frame->prev);
}
if (frame->tmp) {
json_gen_cstr(gen, "tmp");
json_gen_pointer(gen, (void*)frame->tmp);
}
json_gen_map_close(gen);
json_gen_reset(gen);
if (frame->prev) {
memprof_dump_stack_frame(gen, frame->prev);
}
}
static void
memprof_dump_stack(json_gen gen)
{
memprof_dump_stack_frame(gen, ruby_frame);
}
static void
memprof_dump_lsof(json_gen gen)
{
VALUE cmd = rb_str_new2("lsof -np ");
VALUE pid = rb_funcall(rb_mProcess, rb_intern("pid"), 0);
rb_str_append(cmd, rb_funcall(pid, rb_intern("to_s"), 0));
VALUE lsof = rb_funcall(rb_cObject, '`', 1, cmd);
if (RTEST(lsof)) {
VALUE newline = rb_str_new2("\n");
VALUE lines = rb_funcall(lsof, rb_intern("split"), 1, newline);
int i;
for (i=1; i < RARRAY_LEN(lines); i++) {
VALUE parts = rb_funcall(RARRAY_PTR(lines)[i], rb_intern("split"), 2, Qnil, INT2FIX(9));
json_gen_map_open(gen);
json_gen_cstr(gen, "_id");
json_gen_format(gen, "lsof:%d", i);
json_gen_cstr(gen, "type");
json_gen_cstr(gen, "lsof");
json_gen_cstr(gen, "fd");
json_gen_cstr(gen, RSTRING_PTR(RARRAY_PTR(parts)[3]));
json_gen_cstr(gen, "fd_type");
json_gen_cstr(gen, RSTRING_PTR(RARRAY_PTR(parts)[4]));
json_gen_cstr(gen, "fd_name");
json_gen_cstr(gen, RSTRING_PTR(RARRAY_PTR(parts)[RARRAY_LEN(parts)-1]));
json_gen_map_close(gen);
json_gen_reset(gen);
}
}
}
static void
memprof_dump_ps(json_gen gen)
{
VALUE cmd = rb_str_new2("ps -o rss,vsize -p ");
VALUE pid = rb_funcall(rb_mProcess, rb_intern("pid"), 0);
rb_str_append(cmd, rb_funcall(pid, rb_intern("to_s"), 0));
VALUE ps = rb_funcall(rb_cObject, '`', 1, cmd);
if (RTEST(ps)) {
VALUE newline = rb_str_new2("\n");
VALUE lines = rb_funcall(ps, rb_intern("split"), 1, newline);
if (RARRAY_LEN(lines) == 2) {
VALUE parts = rb_funcall(RARRAY_PTR(lines)[1], rb_intern("split"), 0);
json_gen_map_open(gen);
json_gen_cstr(gen, "_id");
json_gen_cstr(gen, "ps");
json_gen_cstr(gen, "type");
json_gen_cstr(gen, "ps");
json_gen_cstr(gen, "rss");
json_gen_cstr(gen, RSTRING_PTR(RARRAY_PTR(parts)[0]));
json_gen_cstr(gen, "vsize");
json_gen_cstr(gen, RSTRING_PTR(RARRAY_PTR(parts)[1]));
json_gen_map_close(gen);
json_gen_reset(gen);
}
}
}
static void
memprof_dump_finalizers(json_gen gen)
{
st_table *finalizer_table = *(st_table **)memprof_config.finalizer_table;
if (finalizer_table) {
json_gen_map_open(gen);
json_gen_cstr(gen, "_id");
json_gen_cstr(gen, "finalizers");
json_gen_cstr(gen, "type");
json_gen_cstr(gen, "finalizers");
json_gen_cstr(gen, "data");
json_gen_array_open(gen);
st_foreach(finalizer_table, finalizers_each_dump, (st_data_t)gen);
json_gen_array_close(gen);
json_gen_map_close(gen);
json_gen_reset(gen);
}
}
static int
objs_each_dump(st_data_t key, st_data_t record, st_data_t arg)
{
obj_dump((VALUE)key, (json_gen)arg);
json_gen_reset((json_gen)arg);
return ST_CONTINUE;
}
static VALUE
memprof_dump(int argc, VALUE *argv, VALUE self)
{
VALUE ret = Qnil;
int old = track_objs;
if (rb_block_given_p()) {
memprof_start(self);
ret = rb_yield(Qnil);
} else if (!track_objs)
rb_raise(rb_eRuntimeError, "object tracking disabled, call Memprof.start first");
track_objs = 0;
json_gen gen = json_for_args(argc, argv);
st_foreach(objs, objs_each_dump, (st_data_t)gen);
json_free(gen);
if (rb_block_given_p())
memprof_stop(self);
track_objs = old;
return ret;
}
static VALUE
memprof_dump_all(int argc, VALUE *argv, VALUE self)
{
if (memprof_config.heaps == NULL ||
memprof_config.heaps_used == NULL ||
memprof_config.sizeof_RVALUE == 0 ||
memprof_config.sizeof_heaps_slot == 0 ||
memprof_config.offset_heaps_slot_slot == SIZE_MAX ||
memprof_config.offset_heaps_slot_limit == SIZE_MAX)
rb_raise(eUnsupported, "not enough config data to dump heap");
char *heaps = *(char**)memprof_config.heaps;
int heaps_used = *(int*)memprof_config.heaps_used;
char *p, *pend;
int i, limit;
VALUE str;
char *filename = NULL;
char *in_progress_filename = NULL;
FILE *out = NULL;
rb_scan_args(argc, argv, "01", &str);
if (RTEST(str)) {
filename = StringValueCStr(str);
size_t filename_len = strlen(filename);
in_progress_filename = alloca(filename_len + 13);
memcpy(in_progress_filename, filename, filename_len);
memcpy(in_progress_filename + filename_len, ".IN_PROGRESS\0", 13);
out = fopen(in_progress_filename, "w");
if (!out)
rb_raise(rb_eArgError, "unable to open output file");
}
json_gen_config conf = { .beautify = 0, .indentString = " " };
json_gen gen = json_gen_alloc2((json_print_t)&json_print, &conf, NULL, (void*)out);
track_objs = 0;
memprof_dump_finalizers(gen);
memprof_dump_globals(gen);
memprof_dump_stack(gen);
for (i=0; i < heaps_used; i++) {
p = *(char**)(heaps + (i * memprof_config.sizeof_heaps_slot) + memprof_config.offset_heaps_slot_slot);
limit = *(int*)(heaps + (i * memprof_config.sizeof_heaps_slot) + memprof_config.offset_heaps_slot_limit);
pend = p + (memprof_config.sizeof_RVALUE * limit);
while (p < pend) {
if (RBASIC(p)->flags) {
obj_dump((VALUE)p, gen);
json_gen_reset(gen);
}
p += memprof_config.sizeof_RVALUE;
}
}
memprof_dump_lsof(gen);
memprof_dump_ps(gen);
json_gen_clear(gen);
json_gen_free(gen);
if (out) {
fclose(out);
rename(in_progress_filename, filename);
}
track_objs = 1;
return Qnil;
}
static void
init_memprof_config_base() {
memset(&memprof_config, 0, sizeof(memprof_config));
memprof_config.offset_heaps_slot_limit = SIZE_MAX;
memprof_config.offset_heaps_slot_slot = SIZE_MAX;
memprof_config.pagesize = getpagesize();
assert(memprof_config.pagesize);
}
static void
init_memprof_config_extended() {
/* If we don't have add_freelist, find the functions it gets inlined into */
memprof_config.add_freelist = bin_find_symbol("add_freelist", NULL, 0);
/*
* Sometimes gc_sweep gets inlined in garbage_collect
* (e.g., on REE it gets inlined into garbage_collect_0).
*/
if (memprof_config.add_freelist == NULL) {
memprof_config.gc_sweep = bin_find_symbol("gc_sweep",
&memprof_config.gc_sweep_size, 0);
if (memprof_config.gc_sweep == NULL)
memprof_config.gc_sweep = bin_find_symbol("garbage_collect_0",
&memprof_config.gc_sweep_size, 0);
if (memprof_config.gc_sweep == NULL)
memprof_config.gc_sweep = bin_find_symbol("garbage_collect",
&memprof_config.gc_sweep_size, 0);
memprof_config.finalize_list = bin_find_symbol("finalize_list",
&memprof_config.finalize_list_size, 0);
memprof_config.rb_gc_force_recycle = bin_find_symbol("rb_gc_force_recycle",
&memprof_config.rb_gc_force_recycle_size, 0);
memprof_config.freelist = bin_find_symbol("freelist", NULL, 0);
}
memprof_config.classname = bin_find_symbol("classname", NULL, 0);
memprof_config.bm_mark = bin_find_symbol("bm_mark", NULL, 0);
memprof_config.blk_free = bin_find_symbol("blk_free", NULL, 0);
memprof_config.thread_mark = bin_find_symbol("thread_mark", NULL, 0);
memprof_config.rb_mark_table_add_filename = bin_find_symbol("rb_mark_table_add_filename", NULL, 0);
/* Stuff for dumping the heap */
memprof_config.heaps = bin_find_symbol("heaps", NULL, 0);
memprof_config.heaps_used = bin_find_symbol("heaps_used", NULL, 0);
memprof_config.finalizer_table = bin_find_symbol("finalizer_table", NULL, 0);
#ifdef sizeof__RVALUE
memprof_config.sizeof_RVALUE = sizeof__RVALUE;
#else
memprof_config.sizeof_RVALUE = bin_type_size("RVALUE");
#endif
#ifdef sizeof__heaps_slot
memprof_config.sizeof_heaps_slot = sizeof__heaps_slot;
#else
memprof_config.sizeof_heaps_slot = bin_type_size("heaps_slot");
#endif
#ifdef offset__heaps_slot__limit
memprof_config.offset_heaps_slot_limit = offset__heaps_slot__limit;
#else
memprof_config.offset_heaps_slot_limit = bin_type_member_offset("heaps_slot", "limit");
#endif
#ifdef offset__heaps_slot__slot
memprof_config.offset_heaps_slot_slot = offset__heaps_slot__slot;
#else
memprof_config.offset_heaps_slot_slot = bin_type_member_offset("heaps_slot", "slot");
#endif
#ifdef offset__BLOCK__body
memprof_config.offset_BLOCK_body = offset__BLOCK__body;
#else
memprof_config.offset_BLOCK_body = bin_type_member_offset("BLOCK", "body");
#endif
#ifdef offset__BLOCK__var
memprof_config.offset_BLOCK_var = offset__BLOCK__var;
#else
memprof_config.offset_BLOCK_var = bin_type_member_offset("BLOCK", "var");
#endif
#ifdef offset__BLOCK__cref
memprof_config.offset_BLOCK_cref = offset__BLOCK__cref;
#else
memprof_config.offset_BLOCK_cref = bin_type_member_offset("BLOCK", "cref");
#endif
#ifdef offset__BLOCK__prev
memprof_config.offset_BLOCK_prev = offset__BLOCK__prev;
#else
memprof_config.offset_BLOCK_prev = bin_type_member_offset("BLOCK", "prev");
#endif
#ifdef offset__BLOCK__self
memprof_config.offset_BLOCK_self = offset__BLOCK__self;
#else
memprof_config.offset_BLOCK_self = bin_type_member_offset("BLOCK", "self");
#endif
#ifdef offset__BLOCK__klass
memprof_config.offset_BLOCK_klass = offset__BLOCK__klass;
#else
memprof_config.offset_BLOCK_klass = bin_type_member_offset("BLOCK", "klass");
#endif
#ifdef offset__BLOCK__orig_thread
memprof_config.offset_BLOCK_orig_thread = offset__BLOCK__orig_thread;
#else
memprof_config.offset_BLOCK_orig_thread = bin_type_member_offset("BLOCK", "orig_thread");
#endif
#ifdef offset__BLOCK__wrapper
memprof_config.offset_BLOCK_wrapper = offset__BLOCK__wrapper;
#else
memprof_config.offset_BLOCK_wrapper = bin_type_member_offset("BLOCK", "wrapper");
#endif
#ifdef offset__BLOCK__block_obj
memprof_config.offset_BLOCK_block_obj = offset__BLOCK__block_obj;
#else
memprof_config.offset_BLOCK_block_obj = bin_type_member_offset("BLOCK", "block_obj");
#endif
#ifdef offset__BLOCK__scope
memprof_config.offset_BLOCK_scope = offset__BLOCK__scope;
#else
memprof_config.offset_BLOCK_scope = bin_type_member_offset("BLOCK", "scope");
#endif
#ifdef offset__BLOCK__dyna_vars
memprof_config.offset_BLOCK_dyna_vars = offset__BLOCK__dyna_vars;
#else
memprof_config.offset_BLOCK_dyna_vars = bin_type_member_offset("BLOCK", "dyna_vars");
#endif
#ifdef offset__METHOD__klass
memprof_config.offset_METHOD_klass = offset__METHOD__klass;
#else
memprof_config.offset_METHOD_klass = bin_type_member_offset("METHOD", "klass");
#endif
#ifdef offset__METHOD__rklass
memprof_config.offset_METHOD_rklass = offset__METHOD__rklass;
#else
memprof_config.offset_METHOD_rklass = bin_type_member_offset("METHOD", "rklass");
#endif
#ifdef offset__METHOD__recv
memprof_config.offset_METHOD_recv = offset__METHOD__recv;
#else
memprof_config.offset_METHOD_recv = bin_type_member_offset("METHOD", "recv");
#endif
#ifdef offset__METHOD__id
memprof_config.offset_METHOD_id = offset__METHOD__id;
#else
memprof_config.offset_METHOD_id = bin_type_member_offset("METHOD", "id");
#endif
#ifdef offset__METHOD__oid
memprof_config.offset_METHOD_oid = offset__METHOD__oid;
#else
memprof_config.offset_METHOD_oid = bin_type_member_offset("METHOD", "oid");
#endif
#ifdef offset__METHOD__body
memprof_config.offset_METHOD_body = offset__METHOD__body;
#else
memprof_config.offset_METHOD_body = bin_type_member_offset("METHOD", "body");
#endif
int heap_errors_printed = 0;
if (memprof_config.heaps == NULL)
heap_errors_printed += fprintf(stderr,
"Failed to locate heaps\n");
if (memprof_config.heaps_used == NULL)
heap_errors_printed += fprintf(stderr,
"Failed to locate heaps_used\n");
if (memprof_config.sizeof_RVALUE == 0)
heap_errors_printed += fprintf(stderr,
"Failed to determine sizeof(RVALUE)\n");
if (memprof_config.sizeof_heaps_slot == 0)
heap_errors_printed += fprintf(stderr,
"Failed to determine sizeof(heaps_slot)\n");
if (memprof_config.offset_heaps_slot_limit == SIZE_MAX)
heap_errors_printed += fprintf(stderr,
"Failed to determine offset of heaps_slot->limit\n");
if (memprof_config.offset_heaps_slot_slot == SIZE_MAX)
heap_errors_printed += fprintf(stderr,
"Failed to determine offset of heaps_slot->slot\n");
if (heap_errors_printed)
fprintf(stderr, "You won't be able to dump your heap!\n");
int errors_printed = 0;
/* If we can't find add_freelist, we need to make sure we located the functions that it gets inlined into. */
if (memprof_config.add_freelist == NULL) {
if (memprof_config.gc_sweep == NULL) {
errors_printed += fprintf(stderr,
"Failed to locate add_freelist (it's probably inlined, but we couldn't find it there either!)\n");
errors_printed += fprintf(stderr,
"Failed to locate gc_sweep, garbage_collect_0, or garbage_collect\n");
}
if (memprof_config.gc_sweep_size == 0)
errors_printed += fprintf(stderr,
"Failed to determine the size of gc_sweep/garbage_collect_0/garbage_collect: %zd\n",
memprof_config.gc_sweep_size);
if (memprof_config.finalize_list == NULL)
errors_printed += fprintf(stderr,
"Failed to locate finalize_list\n");
if (memprof_config.finalize_list_size == 0)
errors_printed += fprintf(stderr,
"Failed to determine the size of finalize_list: %zd\n",
memprof_config.finalize_list_size);
if (memprof_config.rb_gc_force_recycle == NULL)
errors_printed += fprintf(stderr,
"Failed to locate rb_gc_force_recycle\n");
if (memprof_config.rb_gc_force_recycle_size == 0)
errors_printed += fprintf(stderr,
"Failed to determine the size of rb_gc_force_recycle: %zd\n",
memprof_config.rb_gc_force_recycle_size);
if (memprof_config.freelist == NULL)
errors_printed += fprintf(stderr,
"Failed to locate freelist\n");
}
if (memprof_config.classname == NULL)
errors_printed += fprintf(stderr,
"Failed to locate classname\n");
if (errors_printed) {
VALUE ruby_build_info = rb_eval_string("require 'rbconfig'; RUBY_DESCRIPTION + '\n' + RbConfig::CONFIG['CFLAGS'];");
/* who knows what could happen */
if (TYPE(ruby_build_info) == T_STRING)
fprintf(stderr, "%s\n", StringValuePtr(ruby_build_info));
fprintf(stderr, "\nTry installing debug symbols (apt-get install libruby1.8-dbg or similar), or using a "
"Ruby built with RVM (http://rvm.beginrescueend.com/)\n\n");
errx(EX_SOFTWARE, "If that doesn't work, please email this output to bugs@memprof.com");
}
}
void
Init_memprof()
{
VALUE memprof = rb_define_module("Memprof");
eUnsupported = rb_define_class_under(memprof, "Unsupported", rb_eStandardError);
rb_define_singleton_method(memprof, "start", memprof_start, 0);
rb_define_singleton_method(memprof, "stop", memprof_stop, 0);
rb_define_singleton_method(memprof, "stats", memprof_stats, -1);
rb_define_singleton_method(memprof, "stats!", memprof_stats_bang, -1);
rb_define_singleton_method(memprof, "track", memprof_track, -1);
rb_define_singleton_method(memprof, "dump", memprof_dump, -1);
rb_define_singleton_method(memprof, "dump_all", memprof_dump_all, -1);
rb_define_singleton_method(memprof, "trace", memprof_trace, -1);
rb_define_singleton_method(memprof, "trace_request", memprof_trace_request, 1);
rb_define_singleton_method(memprof, "trace_filename", memprof_trace_filename_get, 0);
rb_define_singleton_method(memprof, "trace_filename=", memprof_trace_filename_set, -1);
objs = st_init_numtable();
init_memprof_config_base();
bin_init();
init_memprof_config_extended();
create_tramp_table();
install_malloc_tracer();
install_gc_tracer();
install_objects_tracer();
install_fd_tracer();
install_mysql_tracer();
install_postgres_tracer();
install_memcache_tracer();
install_resources_tracer();
gc_hook = Data_Wrap_Struct(rb_cObject, sourcefile_marker, NULL, NULL);
rb_global_variable(&gc_hook);
rb_classname = memprof_config.classname;
rb_add_freelist = memprof_config.add_freelist;
rb_bm_mark = memprof_config.bm_mark;
rb_blk_free = memprof_config.blk_free;
rb_thread_mark = memprof_config.thread_mark;
ptr_to_rb_mark_table_add_filename = memprof_config.rb_mark_table_add_filename;
assert(rb_classname);
return;
}
================================================
FILE: ext/mmap.h
================================================
/*
* Following code taken from http://thebends.googlecode.com/svn/trunk/mach-o/mmap.c
* Copyright: Allen Porter <allen@thebends.org>
*/
#ifndef __MMAP_H__
#define __MMAP_H__
#include <assert.h>
#include <errno.h>
#include <fcntl.h>
#include <mach/vm_param.h>
#include <stdlib.h>
#include <stdio.h>
#include <sys/stat.h>
#include <sys/mman.h>
#include <unistd.h>
/*
* Simple utililty methods for mmaping files.
*/
struct mmap_info {
const char *name; /* filename */
int fd; /* file descriptor */
size_t data_size; /* size of file */
void* data; /* mmap'd writable contents of file */
};
/*
* Open a file, memory map its contents, and populate mmap_info. Requires a
* filename.
*/
int mmap_file_open(struct mmap_info *file_info) {
assert(file_info != NULL);
file_info->fd = open(file_info->name, O_RDONLY);
if (file_info->fd < 0) {
perror("open");
return -1;
}
struct stat sb;
if (fstat(file_info->fd, &sb) < 0) {
perror("fstat");
return -1;
}
file_info->data_size = (size_t)sb.st_size;
size_t map_size = file_info->data_size;
file_info->data = mmap(0, map_size,
PROT_READ,
MAP_FILE | MAP_SHARED,
file_info->fd, /* offset */0);
if (file_info->data == MAP_FAILED) {
perror("mmap");
return -1;
}
return 0;
}
/*
* Unmemory map and close the file in file_info.
*/
int munmap_file(struct mmap_info *file_info) {
if (munmap(file_info->data, file_info->data_size) < 0) {
perror("munmap");
return -1;
}
if (close(file_info->fd) < 0) {
perror("close");
return -1;
}
return 0;
}
#endif /* __MMAP_H__ */
================================================
FILE: ext/tracer.c
================================================
#include <assert.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include "json.h"
#include "tracer.h"
#include "util.h"
static json_gen tracing_json_gen = NULL;
/*
XXX if we ever need a linked list for anything else ever, remove this crap
and switch to a generic macro-based linked list implementation
*/
struct tracer_list {
struct tracer *tracer;
struct tracer_list *next;
};
static struct tracer_list *tracer_list = NULL;
int
trace_insert(struct tracer *trace)
{
assert(trace != NULL);
struct tracer_list *entry = malloc(sizeof(*entry));
entry->tracer = trace;
entry->next = tracer_list;
tracer_list = entry;
return 0;
}
int
trace_remove(const char *id)
{
struct tracer_list *tmp, *prev;
tmp = prev = tracer_list;
while (tmp) {
if (strcmp(id, tmp->tracer->id) == 0) {
tmp->next = tmp->next;
free(tmp->tracer);
free(tmp);
return 0;
}
prev = tmp;
tmp = tmp->next;
}
return 1;
}
static void
do_trace_invoke(struct tracer *trace, trace_fn fn)
{
switch (fn) {
case TRACE_START:
trace->start();
break;
case TRACE_STOP:
trace->stop();
break;
case TRACE_RESET:
trace->reset();
break;
case TRACE_DUMP:
json_gen_cstr(tracing_json_gen, trace->id);
json_gen_map_open(tracing_json_gen);
trace->dump(tracing_json_gen);
json_gen_map_close(tracing_json_gen);
break;
default:
dbg_printf("invoked a non-existant trace function type: %d", fn);
assert(1==0);
}
return;
}
int
trace_invoke_all(trace_fn fn)
{
struct tracer_list *tmp = tracer_list;
while (tmp) {
do_trace_invoke(tmp->tracer, fn);
tmp = tmp->next;
}
return 0;
}
int
trace_invoke(const char *id, trace_fn fn)
{
struct tracer_list *tmp = tracer_list;
while (tmp) {
if (strcmp(id, tmp->tracer->id) == 0) {
do_trace_invoke(tmp->tracer, fn);
}
tmp = tmp->next;
}
return 0;
}
void
trace_set_output(json_gen gen)
{
tracing_json_gen = gen;
}
json_gen
trace_get_output()
{
return tracing_json_gen;
}
================================================
FILE: ext/tracer.h
================================================
#if !defined(__TRACER__H_)
#define __TRACER__H_
#include "json.h"
struct tracer {
char *id;
void (*start)();
void (*stop)();
void (*reset)();
void (*dump)(json_gen);
};
typedef enum {
TRACE_START,
TRACE_STOP,
TRACE_RESET,
TRACE_DUMP,
} trace_fn;
int
trace_insert(struct tracer *trace);
int
trace_remove(const char *id);
int
trace_invoke_all(trace_fn fn);
int
trace_invoke(const char *id, trace_fn fn);
void
trace_set_output(json_gen gen);
json_gen
trace_get_output();
/* for now, these will live here */
extern void install_malloc_tracer();
extern void install_gc_tracer();
extern void install_fd_tracer();
extern void install_mysql_tracer();
extern void install_postgres_tracer();
extern void install_objects_tracer();
extern void install_memcache_tracer();
extern void install_resources_tracer();
#endif
================================================
FILE: ext/tracers/fd.c
================================================
#include <assert.h>
#include <errno.h>
#include <poll.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/select.h>
#include <sys/socket.h>
#include <sys/time.h>
#include <unistd.h>
#include "arch.h"
#include "bin_api.h"
#include "json.h"
#include "tracer.h"
#include "tramp.h"
#include "util.h"
struct memprof_fd_stats {
size_t read_calls;
uint32_t read_time;
size_t read_requested_bytes;
size_t read_actual_bytes;
size_t write_calls;
uint32_t write_time;
size_t write_requested_bytes;
size_t write_actual_bytes;
size_t recv_calls;
uint32_t recv_time;
ssize_t recv_actual_bytes;
size_t connect_calls;
uint32_t connect_time;
size_t select_calls;
uint32_t select_time;
size_t poll_calls;
uint32_t poll_time;
};
static struct tracer tracer;
static struct memprof_fd_stats stats;
static ssize_t
read_tramp(int fildes, void *buf, size_t nbyte) {
uint64_t millis = 0;
int err;
ssize_t ret;
millis = timeofday_ms();
ret = read(fildes, buf, nbyte);
err = errno;
millis = timeofday_ms() - millis;
stats.read_time += millis;
stats.read_calls++;
stats.read_requested_bytes += nbyte;
if (ret > 0)
stats.read_actual_bytes += ret;
errno = err;
return ret;
}
static ssize_t
write_tramp(int fildes, const void *buf, size_t nbyte) {
uint64_t millis = 0;
int err;
ssize_t ret;
millis = timeofday_ms();
ret = write(fildes, buf, nbyte);
err = errno;
millis = timeofday_ms() - millis;
stats.write_time += millis;
stats.write_calls++;
stats.write_requested_bytes += nbyte;
if (ret > 0)
stats.write_actual_bytes += ret;
errno = err;
return ret;
}
static ssize_t
recv_tramp(int socket, void *buffer, size_t length, int flags) {
uint64_t millis = 0;
int err;
ssize_t ret;
millis = timeofday_ms();
ret = recv(socket, buffer, length, flags);
err = errno;
millis = timeofday_ms() - millis;
stats.recv_time += millis;
stats.recv_calls++;
if (ret > 0)
stats.recv_actual_bytes += ret;
errno = err;
return ret;
}
static int
connect_tramp(int socket, const struct sockaddr *address, socklen_t address_len) {
uint64_t millis = 0;
int err, ret;
millis = timeofday_ms();
ret = connect(socket, address, address_len);
err = errno;
millis = timeofday_ms() - millis;
stats.connect_time += millis;
stats.connect_calls++;
errno = err;
return ret;
}
static int
select_tramp(int nfds, fd_set *readfds, fd_set *writefds, fd_set *errorfds, struct timeval *timeout)
{
uint64_t millis = 0;
int ret, err;
millis = timeofday_ms();
ret = select(nfds, readfds, writefds, errorfds, timeout);
err = errno;
millis = timeofday_ms() - millis;
stats.select_time += millis;
stats.select_calls++;
errno = err;
return ret;
}
static int
poll_tramp(struct pollfd fds[], nfds_t nfds, int timeout)
{
uint64_t millis = 0;
int ret, err;
millis = timeofday_ms();
ret = poll(fds, nfds, timeout);
err = errno;
millis = timeofday_ms() - millis;
stats.poll_time += millis;
stats.poll_calls++;
errno = err;
return ret;
}
static void
fd_trace_start() {
static int inserted = 0;
if (!inserted)
inserted = 1;
else
return;
insert_tramp("read", read_tramp);
insert_tramp("write", write_tramp);
insert_tramp("poll", poll_tramp);
#ifdef HAVE_MACH
insert_tramp("select$DARWIN_EXTSN", select_tramp);
#else
insert_tramp("select", select_tramp);
insert_tramp("connect", connect_tramp);
insert_tramp("recv", recv_tramp);
#endif
}
static void
fd_trace_stop() {
}
static void
fd_trace_reset() {
memset(&stats, 0, sizeof(stats));
}
static void
fd_trace_dump(json_gen gen) {
if (stats.read_calls > 0) {
json_gen_cstr(gen, "read");
json_gen_map_open(gen);
json_gen_cstr(gen, "calls");
json_gen_integer(gen, stats.read_calls);
json_gen_cstr(gen, "time");
json_gen_integer(gen, stats.read_time);
json_gen_cstr(gen, "requested");
json_gen_integer(gen, stats.read_requested_bytes);
json_gen_cstr(gen, "actual");
json_gen_integer(gen, stats.read_actual_bytes);
json_gen_map_close(gen);
}
if (stats.write_calls > 0) {
json_gen_cstr(gen, "write");
json_gen_map_open(gen);
json_gen_cstr(gen, "calls");
json_gen_integer(gen, stats.write_calls);
json_gen_cstr(gen, "time");
json_gen_integer(gen, stats.write_time);
json_gen_cstr(gen, "requested");
json_gen_integer(gen, stats.write_requested_bytes);
json_gen_cstr(gen, "actual");
json_gen_integer(gen, stats.write_actual_bytes);
json_gen_map_close(gen);
}
if (stats.recv_calls > 0) {
json_gen_cstr(gen, "recv");
json_gen_map_open(gen);
json_gen_cstr(gen, "calls");
json_gen_integer(gen, stats.recv_calls);
json_gen_cstr(gen, "time");
json_gen_integer(gen, stats.recv_time);
json_gen_cstr(gen, "actual");
json_gen_integer(gen, stats.recv_actual_bytes);
json_gen_map_close(gen);
}
if (stats.connect_calls > 0) {
json_gen_cstr(gen, "connect");
json_gen_map_open(gen);
json_gen_cstr(gen, "calls");
json_gen_integer(gen, stats.connect_calls);
json_gen_cstr(gen, "time");
json_gen_integer(gen, stats.connect_time);
json_gen_map_close(gen);
}
if (stats.select_calls > 0) {
json_gen_cstr(gen, "select");
json_gen_map_open(gen);
json_gen_cstr(gen, "calls");
json_gen_integer(gen, stats.select_calls);
json_gen_cstr(gen, "time");
json_gen_integer(gen, stats.select_time);
json_gen_map_close(gen);
}
if (stats.poll_calls > 0) {
json_gen_cstr(gen, "poll");
json_gen_map_open(gen);
json_gen_cstr(gen, "calls");
json_gen_integer(gen, stats.poll_calls);
json_gen_cstr(gen, "time");
json_gen_integer(gen, stats.poll_time);
json_gen_map_close(gen);
}
}
void install_fd_tracer()
{
tracer.start = fd_trace_start;
tracer.stop = fd_trace_stop;
tracer.reset = fd_trace_reset;
tracer.dump = fd_trace_dump;
tracer.id = "fd";
trace_insert(&tracer);
}
================================================
FILE: ext/tracers/gc.c
================================================
#include <assert.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/time.h>
#include <sys/resource.h>
#include "arch.h"
#include "bin_api.h"
#include "json.h"
#include "tracer.h"
#include "tramp.h"
#include "util.h"
struct memprof_gc_stats {
size_t gc_calls;
uint32_t gc_time;
uint32_t gc_utime;
uint32_t gc_stime;
};
static struct tracer tracer;
static struct memprof_gc_stats stats;
static void (*orig_garbage_collect)();
static void
gc_tramp()
{
uint64_t millis = 0;
struct rusage usage_start, usage_end;
millis = timeofday_ms();
getrusage(RUSAGE_SELF, &usage_start);
orig_garbage_collect();
getrusage(RUSAGE_SELF, &usage_end);
millis = timeofday_ms() - millis;
stats.gc_time += millis;
stats.gc_calls++;
stats.gc_utime += TVAL_TO_INT64(usage_end.ru_utime) - TVAL_TO_INT64(usage_start.ru_utime);
stats.gc_stime += TVAL_TO_INT64(usage_end.ru_stime) - TVAL_TO_INT64(usage_start.ru_stime);
}
static void
gc_trace_start() {
static int inserted = 0;
if (!inserted)
inserted = 1;
else
return;
orig_garbage_collect = bin_find_symbol("garbage_collect", NULL, 0);
assert(orig_garbage_collect != NULL);
dbg_printf("orig_garbage_collect: %p\n", orig_garbage_collect);
insert_tramp("garbage_collect", gc_tramp);
}
static void
gc_trace_stop() {
}
static void
gc_trace_reset() {
memset(&stats, 0, sizeof(stats));
}
static void
gc_trace_dump(json_gen gen) {
json_gen_cstr(gen, "calls");
json_gen_integer(gen, stats.gc_calls);
json_gen_cstr(gen, "time");
json_gen_integer(gen, stats.gc_time);
json_gen_cstr(gen, "utime");
json_gen_integer(gen, stats.gc_utime);
json_gen_cstr(gen, "stime");
json_gen_integer(gen, stats.gc_stime);
}
void install_gc_tracer()
{
tracer.start = gc_trace_start;
tracer.stop = gc_trace_stop;
tracer.reset = gc_trace_reset;
tracer.dump = gc_trace_dump;
tracer.id = "gc";
trace_insert(&tracer);
}
================================================
FILE: ext/tracers/memcache.c
================================================
#include <assert.h>
#include <errno.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/time.h>
#include "arch.h"
#include "bin_api.h"
#include "json.h"
#include "tracer.h"
#include "tramp.h"
#include "util.h"
struct memprof_memcache_stats {
size_t get_calls;
size_t get_responses[45];
size_t set_calls;
size_t set_responses[45];
};
static struct tracer tracer;
static struct memprof_memcache_stats stats;
static const char* (*_memcached_lib_version)(void);
static char* (*_memcached_get)(void *ptr, const char *key, size_t key_length, size_t *value_length, uint32_t *flags, void *error);
static int (*_memcached_set)(void *ptr, const char *key, size_t key_length, const char *value, size_t value_length, time_t expiration, uint32_t flags);
static char*
memcached_get_tramp(void *ptr, const char *key, size_t key_length, size_t *value_length, uint32_t *flags, void *error)
{
char* ret = _memcached_get(ptr, key, key_length, value_length, flags, error);
stats.get_calls++;
int err = *(int*)error;
stats.get_responses[err > 42 ? 44 : err]++;
return ret;
}
static int
memcached_set_tramp(void *ptr, const char *key, size_t key_length, const char *value, size_t value_length, time_t expiration, uint32_t flags)
{
int ret = _memcached_set(ptr, key, key_length, value, value_length, expiration, flags);
stats.set_calls++;
stats.set_responses[ret > 42 ? 44 : ret]++;
return ret;
}
static void
memcache_trace_start() {
static int inserted = 0;
if (!inserted)
inserted = 1;
else
return;
_memcached_lib_version = bin_find_symbol("memcached_lib_version", NULL, 1);
if (_memcached_lib_version) {
const char *version = _memcached_lib_version();
if (strcmp(version, "0.32") == 0) {
_memcached_get = bin_find_symbol("memcached_get", NULL, 1);
insert_tramp("memcached_get", memcached_get_tramp);
_memcached_set = bin_find_symbol("memcached_set", NULL, 1);
insert_tramp("memcached_set", memcached_set_tramp);
}
}
}
static void
memcache_trace_stop() {
}
static void
memcache_trace_reset() {
memset(&stats, 0, sizeof(stats));
}
static void
memcache_trace_dump_results(json_gen gen, size_t responses[])
{
int i;
json_gen_cstr(gen, "responses");
json_gen_map_open(gen);
for (i=0; i < 45; i++) {
if (responses[i]) {
switch (i) {
case 0:
json_gen_cstr(gen, "success");
break;
case 16:
json_gen_cstr(gen, "notfound");
break;
case 44:
json_gen_cstr(gen, "unknown");
break;
default:
json_gen_format(gen, "%d", i);
}
json_gen_integer(gen, responses[i]);
}
}
json_gen_map_close(gen);
}
static void
memcache_trace_dump(json_gen gen) {
if (stats.get_calls > 0) {
json_gen_cstr(gen, "get");
json_gen_map_open(gen);
json_gen_cstr(gen, "calls");
json_gen_integer(gen, stats.get_calls);
memcache_trace_dump_results(gen, stats.get_responses);
json_gen_map_close(gen);
}
if (stats.set_calls > 0) {
json_gen_cstr(gen, "set");
json_gen_map_open(gen);
json_gen_cstr(gen, "calls");
json_gen_integer(gen, stats.set_calls);
memcache_trace_dump_results(gen, stats.set_responses);
json_gen_map_close(gen);
}
}
void install_memcache_tracer()
{
tracer.start = memcache_trace_start;
tracer.stop = memcache_trace_stop;
tracer.reset = memcache_trace_reset;
tracer.dump = memcache_trace_dump;
tracer.id = "memcache";
trace_insert(&tracer);
}
================================================
FILE: ext/tracers/memory.c
================================================
#include <assert.h>
#include <errno.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include "arch.h"
#include "bin_api.h"
#include "json.h"
#include "tracer.h"
#include "tramp.h"
#include "util.h"
struct memprof_memory_stats {
size_t malloc_bytes_requested;
size_t calloc_bytes_requested;
size_t realloc_bytes_requested;
size_t malloc_bytes_actual;
size_t calloc_bytes_actual;
size_t realloc_bytes_actual;
size_t free_bytes_actual;
size_t malloc_calls;
size_t calloc_calls;
size_t realloc_calls;
size_t free_calls;
};
static struct tracer tracer;
static struct memprof_memory_stats stats;
static size_t (*malloc_usable_size)(void *ptr);
static void *
malloc_tramp(size_t size)
{
void *ret = NULL;
int err;
ret = malloc(size);
err = errno;
stats.malloc_bytes_requested += size;
stats.malloc_calls++;
if (ret)
stats.malloc_bytes_actual += malloc_usable_size(ret);
errno = err;
return ret;
}
static void *
calloc_tramp(size_t nmemb, size_t size)
{
void *ret = NULL;
int err;
ret = calloc(nmemb, size);
err = errno;
stats.calloc_bytes_requested += (nmemb * size);
stats.calloc_calls++;
if (ret)
stats.calloc_bytes_actual += malloc_usable_size(ret);
errno = err;
return ret;
}
static void *
realloc_tramp(void *ptr, size_t size)
{
void *ret = NULL;
int err;
ret = realloc(ptr, size);
err = errno;
stats.realloc_bytes_requested += size;
stats.realloc_calls++;
if (ret)
stats.realloc_bytes_actual += malloc_usable_size(ret);
errno = err;
return ret;
}
static void
free_tramp(void *ptr)
{
if (ptr)
stats.free_bytes_actual += malloc_usable_size(ptr);
stats.free_calls++;
free(ptr);
}
static void
malloc_trace_start()
{
static int inserted = 0;
if (!inserted)
inserted = 1;
else
return;
if (!malloc_usable_size) {
malloc_usable_size = bin_find_symbol("MallocExtension_GetAllocatedSize", NULL, 1);
if (!malloc_usable_size) {
dbg_printf("tcmalloc was not found...\n");
malloc_usable_size = bin_find_symbol("malloc_usable_size", NULL, 1);
}
if (!malloc_usable_size) {
dbg_printf("malloc_usable_size was not found...\n");
malloc_usable_size = bin_find_symbol("malloc_size", NULL, 1);
}
assert(malloc_usable_size != NULL);
dbg_printf("malloc_usable_size: %p\n", malloc_usable_size);
}
insert_tramp("malloc", malloc_tramp);
insert_tramp("realloc", realloc_tramp);
insert_tramp("calloc", calloc_tramp);
insert_tramp("free", free_tramp);
}
static void
malloc_trace_stop()
{
}
static void
malloc_trace_reset()
{
memset(&stats, 0, sizeof(stats));
}
static void
malloc_trace_dump(json_gen gen)
{
if (stats.malloc_calls > 0) {
json_gen_cstr(gen, "malloc");
json_gen_map_open(gen);
json_gen_cstr(gen, "calls");
json_gen_integer(gen, stats.malloc_calls);
json_gen_cstr(gen, "requested");
json_gen_integer(gen, stats.malloc_bytes_requested);
json_gen_cstr(gen, "actual");
json_gen_integer(gen, stats.malloc_bytes_actual);
json_gen_map_close(gen);
}
if (stats.realloc_calls > 0) {
json_gen_cstr(gen, "realloc");
json_gen_map_open(gen);
json_gen_cstr(gen, "calls");
json_gen_integer(gen, stats.realloc_calls);
json_gen_cstr(gen, "requested");
json_gen_integer(gen, stats.realloc_bytes_requested);
json_gen_cstr(gen, "actual");
json_gen_integer(gen, stats.realloc_bytes_actual);
json_gen_map_close(gen);
}
if (stats.calloc_calls > 0) {
json_gen_cstr(gen, "calloc");
json_gen_map_open(gen);
json_gen_cstr(gen, "calls");
json_gen_integer(gen, stats.calloc_calls);
json_gen_cstr(gen, "requested");
json_gen_integer(gen, stats.calloc_bytes_requested);
json_gen_cstr(gen, "actual");
json_gen_integer(gen, stats.calloc_bytes_actual);
json_gen_map_close(gen);
}
if (stats.free_calls > 0) {
json_gen_cstr(gen, "free");
json_gen_map_open(gen);
json_gen_cstr(gen, "calls");
json_gen_integer(gen, stats.free_calls);
json_gen_cstr(gen, "actual");
json_gen_integer(gen, stats.free_bytes_actual);
json_gen_map_close(gen);
}
}
void install_malloc_tracer()
{
tracer.start = malloc_trace_start;
tracer.stop = malloc_trace_stop;
tracer.reset = malloc_trace_reset;
tracer.dump = malloc_trace_dump;
tracer.id = "memory";
trace_insert(&tracer);
}
================================================
FILE: ext/tracers/mysql.c
================================================
#include <assert.h>
#include <errno.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/time.h>
#include "arch.h"
#include "bin_api.h"
#include "json.h"
#include "tracer.h"
#include "tracers/sql.h"
#include "tramp.h"
#include "util.h"
struct memprof_mysql_stats {
size_t query_calls;
uint32_t query_time;
size_t query_calls_by_type[sql_UNKNOWN+1];
uint32_t query_time_by_type[sql_UNKNOWN+1];
};
static struct tracer tracer;
static struct memprof_mysql_stats stats;
static int (*orig_real_query)(void *mysql, const char *stmt_str, unsigned long length);
static int (*orig_send_query)(void *mysql, const char *stmt_str, unsigned long length);
static int
real_query_tramp(void *mysql, const char *stmt_str, unsigned long length) {
enum memprof_sql_type type;
uint64_t millis = 0;
int ret;
millis = timeofday_ms();
ret = orig_real_query(mysql, stmt_str, length);
millis = timeofday_ms() - millis;
stats.query_time += millis;
stats.query_calls++;
type = memprof_sql_query_type(stmt_str, length);
stats.query_time_by_type[type] += millis;
stats.query_calls_by_type[type]++;
return ret;
}
static int
send_query_tramp(void *mysql, const char *stmt_str, unsigned long length) {
enum memprof_sql_type type;
int ret;
ret = orig_send_query(mysql, stmt_str, length);
stats.query_calls++;
type = memprof_sql_query_type(stmt_str, length);
stats.query_calls_by_type[type]++;
return ret;
}
static void
mysql_trace_start() {
static int inserted = 0;
if (!inserted)
inserted = 1;
else
return;
orig_real_query = bin_find_symbol("mysql_real_query", NULL, 1);
if (orig_real_query)
insert_tramp("mysql_real_query", real_query_tramp);
orig_send_query = bin_find_symbol("mysql_send_query", NULL, 1);
if (orig_send_query)
insert_tramp("mysql_send_query", send_query_tramp);
}
static void
mysql_trace_stop() {
}
static void
mysql_trace_reset() {
memset(&stats, 0, sizeof(stats));
}
static void
mysql_trace_dump(json_gen gen) {
enum memprof_sql_type i;
if (stats.query_calls > 0) {
json_gen_cstr(gen, "queries");
json_gen_integer(gen, stats.query_calls);
json_gen_cstr(gen, "time");
json_gen_integer(gen, stats.query_time);
json_gen_cstr(gen, "types");
json_gen_map_open(gen);
for (i=0; i<=sql_UNKNOWN; i++) {
json_gen_cstr(gen, memprof_sql_type_str(i));
json_gen_map_open(gen);
json_gen_cstr(gen, "queries");
json_gen_integer(gen, stats.query_calls_by_type[i]);
json_gen_cstr(gen, "time");
json_gen_integer(gen, stats.query_time_by_type[i]);
json_gen_map_close(gen);
}
json_gen_map_close(gen);
}
}
void install_mysql_tracer()
{
tracer.start = mysql_trace_start;
tracer.stop = mysql_trace_stop;
tracer.reset = mysql_trace_reset;
tracer.dump = mysql_trace_dump;
tracer.id = "mysql";
trace_insert(&tracer);
}
================================================
FILE: ext/tracers/objects.c
================================================
#include <assert.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include "arch.h"
#include "bin_api.h"
#include "json.h"
#include "tracer.h"
#include "tramp.h"
#include "util.h"
#include "ruby.h"
struct memprof_objects_stats {
size_t newobj_calls;
size_t types[T_MASK+1];
};
static struct tracer tracer;
static struct memprof_objects_stats stats;
static VALUE (*orig_rb_newobj)();
static VALUE last_obj = 0;
static VALUE gc_hook = 0;
static void
record_last_obj()
{
if (last_obj) {
stats.types[BUILTIN_TYPE(last_obj)]++;
last_obj = 0;
}
}
static VALUE
objects_tramp() {
record_last_obj();
stats.newobj_calls++;
last_obj = orig_rb_newobj();
return last_obj;
}
static void
objects_trace_start() {
static int inserted = 0;
if (!inserted)
inserted = 1;
else
return;
orig_rb_newobj = bin_find_symbol("rb_newobj", NULL, 0);
assert(orig_rb_newobj != NULL);
dbg_printf("orig_rb_newobj: %p\n", orig_rb_newobj);
insert_tramp("rb_newobj", objects_tramp);
}
static void
objects_trace_stop() {
}
static void
objects_trace_reset() {
memset(&stats, 0, sizeof(stats));
last_obj = 0;
}
static inline char*
type_string(int type) {
switch (type) {
case T_NONE:
return "none";
case T_NIL:
return "nil";
case T_OBJECT:
return "object";
case T_CLASS:
return "class";
case T_ICLASS:
return "iclass";
case T_MODULE:
return "module";
case T_FLOAT:
return "float";
case T_STRING:
return "string";
case T_REGEXP:
return "regexp";
case T_ARRAY:
return "array";
case T_FIXNUM:
return "fixnum";
case T_HASH:
return "hash";
case T_STRUCT:
return "struct";
case T_BIGNUM:
return "bignum";
case T_FILE:
return "file";
case T_TRUE:
return "true";
case T_FALSE:
return "false";
case T_DATA:
return "data";
case T_MATCH:
return "match";
case T_SYMBOL:
return "symbol";
case T_BLKTAG:
return "blktag";
case T_UNDEF:
return "undef";
case T_VARMAP:
return "varmap";
case T_SCOPE:
return "scope";
case T_NODE:
return "node";
default:
return "unknown";
}
}
static void
objects_trace_dump(json_gen gen) {
int i;
record_last_obj();
json_gen_cstr(gen, "created");
json_gen_integer(gen, stats.newobj_calls);
json_gen_cstr(gen, "types");
json_gen_map_open(gen);
for (i=0; i<T_MASK+1; i++) {
if (stats.types[i] > 0) {
json_gen_cstr(gen, type_string(i));
json_gen_integer(gen, stats.types[i]);
}
}
json_gen_map_close(gen);
}
void install_objects_tracer()
{
if (!gc_hook) {
gc_hook = Data_Wrap_Struct(rb_cObject, record_last_obj, NULL, NULL);
rb_global_variable(&gc_hook);
}
tracer.start = objects_trace_start;
tracer.stop = objects_trace_stop;
tracer.reset = objects_trace_reset;
tracer.dump = objects_trace_dump;
tracer.id = "objects";
trace_insert(&tracer);
}
================================================
FILE: ext/tracers/postgres.c
================================================
#include <assert.h>
#include <errno.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/time.h>
#include "arch.h"
#include "bin_api.h"
#include "json.h"
#include "tracer.h"
#include "tracers/sql.h"
#include "tramp.h"
#include "util.h"
struct memprof_postgres_stats {
size_t query_calls;
size_t query_calls_by_type[sql_UNKNOWN+1];
};
static struct tracer tracer;
static struct memprof_postgres_stats stats;
static void * (*orig_PQexec)(void *postgres, const char *stmt);
static void *
PQexec_tramp(void *postgres, const char *stmt) {
enum memprof_sql_type type;
void *ret;
ret = orig_PQexec(postgres, stmt);
stats.query_calls++;
type = memprof_sql_query_type(stmt, strlen(stmt));
stats.query_calls_by_type[type]++;
return ret;
}
static void
postgres_trace_start() {
static int inserted = 0;
if (!inserted)
inserted = 1;
else
return;
orig_PQexec = bin_find_symbol("PQexec", NULL, 1);
if (orig_PQexec)
insert_tramp("PQexec", PQexec_tramp);
}
static void
postgres_trace_stop() {
}
static void
postgres_trace_reset() {
memset(&stats, 0, sizeof(stats));
}
static void
postgres_trace_dump(json_gen gen) {
enum memprof_sql_type i;
if (stats.query_calls > 0) {
json_gen_cstr(gen, "queries");
json_gen_integer(gen, stats.query_calls);
json_gen_cstr(gen, "types");
json_gen_map_open(gen);
for (i=0; i<=sql_UNKNOWN; i++) {
json_gen_cstr(gen, memprof_sql_type_str(i));
json_gen_map_open(gen);
json_gen_cstr(gen, "queries");
json_gen_integer(gen, stats.query_calls_by_type[i]);
json_gen_map_close(gen);
}
json_gen_map_close(gen);
}
}
void install_postgres_tracer()
{
tracer.start = postgres_trace_start;
tracer.stop = postgres_trace_stop;
tracer.reset = postgres_trace_reset;
tracer.dump = postgres_trace_dump;
tracer.id = "postgres";
trace_insert(&tracer);
}
================================================
FILE: ext/tracers/resources.c
================================================
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/time.h>
#include <sys/resource.h>
#include "json.h"
#include "tracer.h"
#include "tramp.h"
#include "util.h"
struct memprof_resources_stats {
long nsignals;
long inblock;
long oublock;
int64_t utime;
int64_t stime;
};
static struct tracer tracer;
static struct memprof_resources_stats stats;
static void
resources_trace_start() {
struct rusage usage;
getrusage(RUSAGE_SELF, &usage);
stats.nsignals = -usage.ru_nsignals;
stats.inblock = -usage.ru_inblock;
stats.oublock = -usage.ru_oublock;
stats.stime = -TVAL_TO_INT64(usage.ru_stime);
stats.utime = -TVAL_TO_INT64(usage.ru_utime);
}
static void
resources_trace_dump(json_gen gen) {
{ // calculate diff before dump, since stop is called after dump
struct rusage usage;
getrusage(RUSAGE_SELF, &usage);
stats.nsignals += usage.ru_nsignals;
stats.inblock += usage.ru_inblock;
stats.oublock += usage.ru_oublock;
stats.stime += TVAL_TO_INT64(usage.ru_stime);
stats.utime += TVAL_TO_INT64(usage.ru_utime);
}
json_gen_cstr(gen, "signals");
json_gen_integer(gen, stats.nsignals);
json_gen_cstr(gen, "inputs");
json_gen_integer(gen, stats.inblock);
json_gen_cstr(gen, "outputs");
json_gen_integer(gen, stats.oublock);
json_gen_cstr(gen, "stime");
json_gen_integer(gen, stats.stime);
json_gen_cstr(gen, "utime");
json_gen_integer(gen, stats.utime);
}
static void
resources_trace_stop() {
}
static void
resources_trace_reset() {
}
void install_resources_tracer()
{
tracer.start = resources_trace_start;
tracer.stop = resources_trace_stop;
tracer.reset = resources_trace_reset;
tracer.dump = resources_trace_dump;
tracer.id = "resources";
trace_insert(&tracer);
}
================================================
FILE: ext/tracers/sql.c
================================================
#include <tracers/sql.h>
enum memprof_sql_type
memprof_sql_query_type(const char *stmt, unsigned long length)
{
int i;
for (i=0; i<length && i<10; i++) {
switch (stmt[i]) {
case ' ':
case '\n':
case '\r':
continue;
break;
case 'S':
case 's':
return sql_SELECT;
case 'I':
case 'i':
return sql_INSERT;
case 'U':
case 'u':
return sql_UPDATE;
case 'D':
case 'd':
return sql_DELETE;
default:
return sql_UNKNOWN;
}
}
return sql_UNKNOWN;
}
const char *
memprof_sql_type_str(enum memprof_sql_type type)
{
switch (type) {
case sql_SELECT:
return "select";
case sql_UPDATE:
return "update";
case sql_INSERT:
return "insert";
case sql_DELETE:
return "delete";
default:
case sql_UNKNOWN:
return "unknown";
}
}
================================================
FILE: ext/tracers/sql.h
================================================
#if !defined(_sql_h_)
#define _sql_h_
enum memprof_sql_type {
sql_SELECT,
sql_UPDATE,
sql_INSERT,
sql_DELETE,
sql_UNKNOWN // last
};
enum memprof_sql_type
memprof_sql_query_type(const char *stmt, unsigned long length);
const char *
memprof_sql_type_str(enum memprof_sql_type);
#endif
================================================
FILE: ext/tramp.c
================================================
#include <ruby.h>
#ifndef _GNU_SOURCE
#define _GNU_SOURCE
#endif
#include <sysexits.h>
#include <sys/mman.h>
#include <err.h>
#include <assert.h>
#include "arch.h"
#include "bin_api.h"
#include "util.h"
#define FREELIST_INLINES 3
/*
* trampoline specific stuff
*/
static struct tramp_st2_entry *tramp_table;
static size_t tramp_size;
/*
* inline trampoline specific stuff
*/
static size_t inline_tramp_size;
static struct inline_tramp_st2_entry *inline_tramp_table;
extern struct memprof_config memprof_config;
void
create_tramp_table()
{
size_t i;
void *region, *ent, *inline_ent;
size_t tramp_sz = 0, inline_tramp_sz = 0;
ent = arch_get_st2_tramp(&tramp_sz);
inline_ent = arch_get_inline_st2_tramp(&inline_tramp_sz);
assert(ent && inline_ent);
region = bin_allocate_page();
if (region == MAP_FAILED)
errx(EX_SOFTWARE, "Failed to allocate memory for stage 1 trampolines.");
tramp_table = region;
inline_tramp_table = region + memprof_config.pagesize / 2;
for (i = 0; i < (memprof_config.pagesize / 2) / tramp_sz; i++) {
memcpy(tramp_table + i, ent, tramp_sz);
}
for (i = 0; i < (memprof_config.pagesize / 2) / inline_tramp_sz; i++) {
memcpy(inline_tramp_table + i, inline_ent, inline_tramp_sz);
}
}
static void
hook_freelist(int entry, void *tramp)
{
size_t sizes[FREELIST_INLINES];
void *freelist_inliners[FREELIST_INLINES];
void *freelist = NULL;
unsigned char *byte = NULL;
int i = 0, tramps_completed = 0;
assert(memprof_config.gc_sweep != NULL);
assert(memprof_config.finalize_list != NULL);
assert(memprof_config.rb_gc_force_recycle != NULL);
assert(memprof_config.freelist != NULL);
assert(memprof_config.gc_sweep_size > 0);
assert(memprof_config.finalize_list_size > 0);
assert(memprof_config.rb_gc_force_recycle_size > 0);
freelist_inliners[0] = memprof_config.gc_sweep;
freelist_inliners[1] = memprof_config.finalize_list;
freelist_inliners[2] = memprof_config.rb_gc_force_recycle;
sizes[0] = memprof_config.gc_sweep_size;
sizes[1] = memprof_config.finalize_list_size;
sizes[2] = memprof_config.rb_gc_force_recycle_size;
freelist = memprof_config.freelist;
/* start the search for places to insert the inline tramp */
byte = freelist_inliners[i];
while (i < FREELIST_INLINES) {
if (arch_insert_inline_st2_tramp(byte, freelist, tramp,
&inline_tramp_table[entry]) == 0) {
/* insert occurred, so increment internal counters for the tramp table */
entry++;
inline_tramp_size++;
/*
* add_freelist() only gets inlined *ONCE* into any of the 3 functions
* that we're scanning, so move on to the next 'inliner' after we
* tramp the first instruction we find. REE's gc_sweep has 2 calls,
* but this gets optimized into a single inlining and a jmp to it.
* older patchlevels of 1.8.7 don't have an add_freelist(), but the
* instruction should be the same.
*/
tramps_completed++;
i++;
byte = freelist_inliners[i];
continue;
}
/* if we've looked at all the bytes in this function... */
if ((size_t)((void *)byte - freelist_inliners[i]) >= sizes[i]) {
/* move on to the next function */
i++;
byte = freelist_inliners[i];
}
byte++;
}
if (tramps_completed != 3)
errx(EX_SOFTWARE, "Inline add_freelist tramp insertion failed! "
"Only inserted %d tramps.", tramps_completed);
}
void
insert_tramp(const char *trampee, void *tramp)
{
void *trampee_addr = bin_find_symbol(trampee, NULL, 1);
int inline_ent = inline_tramp_size;
if (trampee_addr == NULL) {
if (strcmp("add_freelist", trampee) == 0) {
/* XXX super hack */
inline_tramp_size++;
hook_freelist(inline_ent, tramp /* freelist_tramp() */);
} else {
errx(EX_SOFTWARE, "Failed to locate required symbol %s", trampee);
}
} else {
tramp_table[tramp_size].addr = tramp;
if (bin_update_image(trampee, &tramp_table[tramp_size], NULL) != 0)
errx(EX_SOFTWARE, "Failed to insert tramp for %s", trampee);
tramp_size++;
}
}
================================================
FILE: ext/tramp.h
================================================
#if !defined(TRAMP__)
#define TRAMP__
/*
* create_tramp_table - create the trampoline tables.
*/
void
create_tramp_table();
/*
* insert_tramp - insert a trampoline.
*
* Given:
* - trampee: function in which we want to install the trampoline.
* - tramp: pointer to the function to be called from the trampoline.
*
* This function is responsible for installing the requested trampoline
* at the location of "trampee". This results in tramp() being called
* whenever trampee() is executed.
*/
void
insert_tramp(const char *trampee, void *tramp);
#endif
================================================
FILE: ext/util.c
================================================
#include <stdlib.h>
#include <time.h>
#include <util.h>
#include <sys/time.h>
/* This is the CRC function used by GNU. Stripped executables may contain a
* section .gnu_debuglink which holds the name of an elf object with debug
* information and a checksum.
* !!!! DO NOT MODIFY THIS FUNCTION !!!!
* TODO create specs for this!
*/
unsigned long
gnu_debuglink_crc32(unsigned long crc, unsigned char *buf, size_t len)
{
static const unsigned int crc32_table[256] = {
0x00000000, 0x77073096, 0xee0e612c, 0x990951ba, 0x076dc419,
0x706af48f, 0xe963a535, 0x9e6495a3, 0x0edb8832, 0x79dcb8a4,
0xe0d5e91e, 0x97d2d988, 0x09b64c2b, 0x7eb17cbd, 0xe7b82d07,
0x90bf1d91, 0x1db71064, 0x6ab020f2, 0xf3b97148, 0x84be41de,
0x1adad47d, 0x6ddde4eb, 0xf4d4b551, 0x83d385c7, 0x136c9856,
0x646ba8c0, 0xfd62f97a, 0x8a65c9ec, 0x14015c4f, 0x63066cd9,
0xfa0f3d63, 0x8d080df5, 0x3b6e20c8, 0x4c69105e, 0xd56041e4,
0xa2677172, 0x3c03e4d1, 0x4b04d447, 0xd20d85fd, 0xa50ab56b,
0x35b5a8fa, 0x42b2986c, 0xdbbbc9d6, 0xacbcf940, 0x32d86ce3,
0x45df5c75, 0xdcd60dcf, 0xabd13d59, 0x26d930ac, 0x51de003a,
0xc8d75180, 0xbfd06116, 0x21b4f4b5, 0x56b3c423, 0xcfba9599,
0xb8bda50f, 0x2802b89e, 0x5f058808, 0xc60cd9b2, 0xb10be924,
0x2f6f7c87, 0x58684c11, 0xc1611dab, 0xb6662d3d, 0x76dc4190,
0x01db7106, 0x98d220bc, 0xefd5102a, 0x71b18589, 0x06b6b51f,
0x9fbfe4a5, 0xe8b8d433, 0x7807c9a2, 0x0f00f934, 0x9609a88e,
0xe10e9818, 0x7f6a0dbb, 0x086d3d2d, 0x91646c97, 0xe6635c01,
0x6b6b51f4, 0x1c6c6162, 0x856530d8, 0xf262004e, 0x6c0695ed,
0x1b01a57b, 0x8208f4c1, 0xf50fc457, 0x65b0d9c6, 0x12b7e950,
0x8bbeb8ea, 0xfcb9887c, 0x62dd1ddf, 0x15da2d49, 0x8cd37cf3,
0xfbd44c65, 0x4db26158, 0x3ab551ce, 0xa3bc0074, 0xd4bb30e2,
0x4adfa541, 0x3dd895d7, 0xa4d1c46d, 0xd3d6f4fb, 0x4369e96a,
0x346ed9fc, 0xad678846, 0xda60b8d0, 0x44042d73, 0x33031de5,
0xaa0a4c5f, 0xdd0d7cc9, 0x5005713c, 0x270241aa, 0xbe0b1010,
0xc90c2086, 0x5768b525, 0x206f85b3, 0xb966d409, 0xce61e49f,
0x5edef90e, 0x29d9c998, 0xb0d09822, 0xc7d7a8b4, 0x59b33d17,
0x2eb40d81, 0xb7bd5c3b, 0xc0ba6cad, 0xedb88320, 0x9abfb3b6,
0x03b6e20c, 0x74b1d29a, 0xead54739, 0x9dd277af, 0x04db2615,
0x73dc1683, 0xe3630b12, 0x94643b84, 0x0d6d6a3e, 0x7a6a5aa8,
0xe40ecf0b, 0x9309ff9d, 0x0a00ae27, 0x7d079eb1, 0xf00f9344,
0x8708a3d2, 0x1e01f268, 0x6906c2fe, 0xf762575d, 0x806567cb,
0x196c3671, 0x6e6b06e7, 0xfed41b76, 0x89d32be0, 0x10da7a5a,
0x67dd4acc, 0xf9b9df6f, 0x8ebeeff9, 0x17b7be43, 0x60b08ed5,
0xd6d6a3e8, 0xa1d1937e, 0x38d8c2c4, 0x4fdff252, 0xd1bb67f1,
0xa6bc5767, 0x3fb506dd, 0x48b2364b, 0xd80d2bda, 0xaf0a1b4c,
0x36034af6, 0x41047a60, 0xdf60efc3, 0xa867df55, 0x316e8eef,
0x4669be79, 0xcb61b38c, 0xbc66831a, 0x256fd2a0, 0x5268e236,
0xcc0c7795, 0xbb0b4703, 0x220216b9, 0x5505262f, 0xc5ba3bbe,
0xb2bd0b28, 0x2bb45a92, 0x5cb36a04, 0xc2d7ffa7, 0xb5d0cf31,
0x2cd99e8b, 0x5bdeae1d, 0x9b64c2b0, 0xec63f226, 0x756aa39c,
0x026d930a, 0x9c0906a9, 0xeb0e363f, 0x72076785, 0x05005713,
0x95bf4a82, 0xe2b87a14, 0x7bb12bae, 0x0cb61b38, 0x92d28e9b,
0xe5d5be0d, 0x7cdcefb7, 0x0bdbdf21, 0x86d3d2d4, 0xf1d4e242,
0x68ddb3f8, 0x1fda836e, 0x81be16cd, 0xf6b9265b, 0x6fb077e1,
0x18b74777, 0x88085ae6, 0xff0f6a70, 0x66063bca, 0x11010b5c,
0x8f659eff, 0xf862ae69, 0x616bffd3, 0x166ccf45, 0xa00ae278,
0xd70dd2ee, 0x4e048354, 0x3903b3c2, 0xa7672661, 0xd06016f7,
0x4969474d, 0x3e6e77db, 0xaed16a4a, 0xd9d65adc, 0x40df0b66,
0x37d83bf0, 0xa9bcae53, 0xdebb9ec5, 0x47b2cf7f, 0x30b5ffe9,
0xbdbdf21c, 0xcabac28a, 0x53b39330, 0x24b4a3a6, 0xbad03605,
0xcdd70693, 0x54de5729, 0x23d967bf, 0xb3667a2e, 0xc4614ab8,
0x5d681b02, 0x2a6f2b94, 0xb40bbe37, 0xc30c8ea1, 0x5a05df1b,
0x2d02ef8d
};
unsigned char *end;
crc = ~crc & 0xffffffff;
for (end = buf + len; buf < end; ++buf)
crc = crc32_table[(crc ^ *buf) & 0xff] ^ (crc >> 8);
return ~crc & 0xffffffff;
}
double
timeofday()
{
struct timeval tv;
#ifdef CLOCK_MONOTONIC
struct timespec tp;
if (clock_gettime(CLOCK_MONOTONIC, &tp) == 0) {
return (double)tp.tv_sec + (double)tp.tv_nsec * 1e-9;
}
#endif
gettimeofday(&tv, NULL);
return (double)tv.tv_sec + (double)tv.tv_usec * 1e-6;
}
uint64_t
timeofday_ms()
{
struct timeval tv;
gettimeofday(&tv, NULL);
return (uint64_t)tv.tv_sec*1e3 + (uint64_t)tv.tv_usec*1e-3;
}
================================================
FILE: ext/util.h
================================================
#if !defined(__util_h__)
#define __util_h__
#include <stdint.h>
#if defined(_MEMPROF_DEBUG)
#include <stdio.h>
#define dbg_printf(...) do {\
fprintf(stderr, "%s:%d ", __FILE__, __LINE__);\
fprintf(stderr, __VA_ARGS__);\
} while (0)
#else
#define dbg_printf(...)
#endif
#define ASSERT_ON_COMPILE(pred) \
switch(0){case 0:case pred:;}
struct memprof_config {
void *gc_sweep;
size_t gc_sweep_size;
void *finalize_list;
size_t finalize_list_size;
void *rb_gc_force_recycle;
size_t rb_gc_force_recycle_size;
void *freelist;
void *classname;
void *add_freelist;
void *rb_mark_table_add_filename;
void *bm_mark;
void *blk_free;
void *thread_mark;
void *heaps;
void *heaps_used;
void *finalizer_table;
size_t sizeof_RVALUE;
size_t sizeof_heaps_slot;
size_t offset_heaps_slot_limit;
size_t offset_heaps_slot_slot;
size_t offset_BLOCK_body;
size_t offset_BLOCK_var;
size_t offset_BLOCK_cref;
size_t offset_BLOCK_prev;
size_t offset_BLOCK_self;
size_t offset_BLOCK_klass;
size_t offset_BLOCK_wrapper;
size_t offset_BLOCK_orig_thread;
size_t offset_BLOCK_block_obj;
size_t offset_BLOCK_scope;
size_t offset_BLOCK_dyna_vars;
size_t offset_METHOD_klass;
size_t offset_METHOD_rklass;
size_t offset_METHOD_recv;
size_t offset_METHOD_id;
size_t offset_METHOD_oid;
size_t offset_METHOD_body;
size_t pagesize;
};
/* This is the CRC function used by GNU. Stripped executables may contain a
* section .gnu_debuglink which holds the name of an elf object with debug
* information and a checksum.
*/
unsigned long
gnu_debuglink_crc32 (unsigned long crc, unsigned char *buf, size_t len);
/* Copy of timeofday() implementation inside ruby 1.8, used w/ thread state */
double
timeofday();
/* Use this function for time tracking. It will (interally) try to use an
* appropriately granual timing function.
*/
uint64_t
timeofday_ms();
#define TVAL_TO_INT64(tv) ((int64_t)tv.tv_sec*1e3 + (int64_t)tv.tv_usec*1e-3)
#endif
================================================
FILE: ext/x86_64.c
================================================
#if defined (_ARCH_x86_64_)
#include <assert.h>
#include <stdint.h>
#include <string.h>
#include "arch.h"
#include "x86_gen.h"
#include "util.h"
/*
* inline_st1_tramp - inline stage 1 trampoline
*
* This is the stage 1 inline trampoline that will replace the mov instruction
* that updates freelist from the inlined function add_freelist.
*
* Note that the mov instruction is 7 bytes wide, so this trampoline needs two
* bytes of NOPs to keep it 7 bytes wide.
*
* In order to use this structure, you must set the displacement field to a
* 32bit displacement from the next instruction to the stage 2 trampoline.
*
* TODO replace the 2, 1 byte NOPs with a wider 16bit NOP.
*
* Original code:
*
* mov REGISTER, freelist # update the head of the freelist
*
* size = 7 bytes
*
* Code after tramp:
*
* jmp 0xfeedface(%rip) # jump to stage 2 trampoline
* nop # 1 byte NOP pad
* nop # 1 byte NOP pad
*
* size = 7 bytes
*/
struct inline_st1_tramp {
unsigned char jmp;
int32_t displacement;
unsigned char pad[2];
} __attribute__((__packed__)) inline_st1_tramp = {
.jmp = 0xe9,
.displacement = 0,
.pad = {0x90, 0x90},
};
/*
* inline_st1_base - inline stage 1 base instruction
*
* This structure is designed to be "laid onto" a piece of memory to ease the
* parsing, modification, and length calculation of the original instruction
* that will be overwritten with a jmp to the stage 2 trampoline.
*
* In order to use this structure, you must set the displacement, rex, and
* rex bytes to accurately represent the original instruction.
*/
struct inline_st1_base {
unsigned char rex;
unsigned char mov;
unsigned char src_reg;
int32_t displacement;
} __attribute__((__packed__)) inline_st1_mov = {
.rex = 0,
.mov = 0x89,
.src_reg = 0,
.displacement = 0
};
/*
* arch_check_ins - architecture specific instruction check
*
* This function checks the opcodes at a specific adderss to see if
* they could be a move instruction.
*
* Returns 1 if the address matches a mov, 0 otherwise.
*/
static int
arch_check_ins(struct inline_st1_base *base)
{
assert(base != NULL);
/* is it a mov instruction? */
if (base->mov == 0x89 &&
/* maybe. read the REX byte to find out for sure */
(base->rex == 0x48 ||
base->rex == 0x4c)) {
/* success */
return 1;
}
return 0;
}
/*
* arch_insert_inline_st2_tramp - architecture specific stage 2 tramp insert
*
* Given:
* - addr - The base address of an instruction sequence.
*
* - marker - This is the marker to search for which will indicate that the
* instruction sequence has been located.
*
* - trampoline - The address of the handler to redirect execution to.
*
* - table_entry - Address of where the stage 2 trampoline code will reside
*
* This function will:
* Insert and setup the stage 1 and stage 2 trampolines if addr points to an
* instruction that could be from the inlined add_freelist function.
*
* This function returns 1 on failure and 0 on success.
*/
int
arch_insert_inline_st2_tramp(void *addr, void *marker, void *trampoline, void *table_entry)
{
assert(addr != NULL);
assert(marker != NULL);
assert(trampoline != NULL);
assert(table_entry != NULL);
struct inline_st1_base *base = addr;
struct inline_tramp_st2_entry *entry = table_entry;
ASSERT_ON_COMPILE(sizeof(struct inline_st1_base) ==
sizeof(struct inline_st1_tramp));
if (!arch_check_ins(base))
return 1;
/* Sanity check. Ensure that the displacement from freelist to the next
* instruction matches the mov_target. If so, we know this mov is
* updating freelist.
*/
if (marker - (void *)(base + 1) == base->displacement) {
/* Before the stage 1 trampoline gets written, we need to generate
* the code for the stage 2 trampoline. Let's copy over the REX byte
* and the byte which mentions the source register into the stage 2
* trampoline.
*/
default_inline_st2_tramp.rex = base->rex;
default_inline_st2_tramp.src_reg = base->src_reg;
/* Setup the stage 1 trampoline. Calculate the displacement to
* the stage 2 trampoline from the next instruction.
*
* REMEMBER!!!! The next instruction will be NOP after our stage 1
* trampoline is written. This is 5 bytes into the structure, even
* though the original instruction we overwrote was 7 bytes.
*/
inline_st1_tramp.displacement = table_entry - (void *)(addr + 5);
copy_instructions(addr, &inline_st1_tramp, sizeof(inline_st1_tramp));
/* Finish setting up the stage 2 trampoline. */
/* calculate the displacement to freelist from the next instruction.
*
* This is used to replicate the original instruction we overwrote.
*/
default_inline_st2_tramp.mov_displacement = marker - (void *)&(entry->frame);
/* fill in the displacement to freelist from the next instruction.
*
* This is to arrange for the new value in freelist to be in %rdi, and as such
* be the first argument to the C handler. As per the amd64 ABI.
*/
default_inline_st2_tramp.frame.rdi_source_displacement = marker - (void *)&(entry->frame.align_rsp);
/* jmp back to the instruction after stage 1 trampoline was inserted
*
* This can be 5 or 7, it doesn't matter. If its 5, we'll hit our 2
* NOPS. If its 7, we'll land directly on the next instruction.
*/
default_inline_st2_tramp.jmp_displacement = (addr + sizeof(*base)) -
(table_entry + sizeof(default_inline_st2_tramp));
/* write the address of our C level trampoline in to the structure */
default_inline_st2_tramp.frame.addr = trampoline;
memcpy(table_entry, &default_inline_st2_tramp, sizeof(default_inline_st2_tramp));
return 0;
}
return 1;
}
#endif
================================================
FILE: ext/x86_64.h
================================================
#if !defined(_x86_64_h_)
#define _x86_64_h_
#include <stdint.h>
#include "arch.h"
/*
* tramp_st2_entry - stage 2 trampoline entry
*
* This trampoline calls a handler function via the callee saved register %rbx.
* The handler function is stored in the field 'addr'.
*
* A default pre-filled (except addr, of course) version of this trampoline is
* provided so that the opcodes do not need to be filled in every time it is
* used. You only need to set the addr field of default_st2_tramp and you are
* ready to roll.
*
* This trampoline is the assembly code:
*
* push %rbx # save %rbx
* push %rbp # save previous stack frame's %rbp
* mov %rsp, %rbp # update %rbp to be current stack pointer
* andl 0xFFFFFFFFFFFFFFF0, %rsp # align stack pointer as per the ABI
* mov ADDR, %rbx # move address of
gitextract_chiyp8m1/
├── .gitignore
├── LICENSE
├── README.md
├── Rakefile
├── bin/
│ └── memprof
├── ext/
│ ├── arch.h
│ ├── bin_api.h
│ ├── elf.c
│ ├── extconf.rb
│ ├── i386.c
│ ├── i386.h
│ ├── json.c
│ ├── json.h
│ ├── mach.c
│ ├── memprof.c
│ ├── mmap.h
│ ├── tracer.c
│ ├── tracer.h
│ ├── tracers/
│ │ ├── fd.c
│ │ ├── gc.c
│ │ ├── memcache.c
│ │ ├── memory.c
│ │ ├── mysql.c
│ │ ├── objects.c
│ │ ├── postgres.c
│ │ ├── resources.c
│ │ ├── sql.c
│ │ └── sql.h
│ ├── tramp.c
│ ├── tramp.h
│ ├── util.c
│ ├── util.h
│ ├── x86_64.c
│ ├── x86_64.h
│ ├── x86_gen.c
│ └── x86_gen.h
├── lib/
│ └── memprof/
│ ├── middleware.rb
│ ├── signal.rb
│ └── tracer.rb
├── memprof.gemspec
└── spec/
├── memprof_spec.rb
├── memprof_uploader_spec.rb
└── tracing_spec.rb
SYMBOL INDEX (295 symbols across 33 files)
FILE: ext/bin_api.h
type tramp_st2_entry (line 9) | struct tramp_st2_entry
type inline_tramp_st2_entry (line 10) | struct inline_tramp_st2_entry
type tramp_st2_entry (line 99) | struct tramp_st2_entry
FILE: ext/elf.c
type memprof_config (line 28) | struct memprof_config
type elf_info (line 43) | struct elf_info
type elf_info (line 45) | struct elf_info {
type linkmap_cb_status (line 84) | typedef enum {
type linkmap_cb_status (line 92) | typedef linkmap_cb_status (*linkmap_cb)(struct link_map *, void *);
type elf_info (line 93) | struct elf_info
type elf_info (line 95) | struct elf_info
type elf_info (line 96) | struct elf_info
type elf_info (line 97) | struct elf_info
type plt_entry (line 115) | struct plt_entry {
type plt_entry (line 134) | struct plt_entry
type elf_info (line 134) | struct elf_info
type elf_info (line 165) | struct elf_info
type plt_hook_data (line 177) | struct plt_hook_data {
type dso_iter_data (line 182) | struct dso_iter_data {
function linkmap_cb_status (line 187) | static linkmap_cb_status
function for_each_dso (line 240) | static void
function hook_required_objects (line 249) | static void
type elf_info (line 278) | struct elf_info
function GElf_Addr (line 335) | static inline GElf_Addr
type elf_info (line 354) | struct elf_info
type elf_info (line 419) | struct elf_info
function find_symbol_cb (line 480) | static void
type plt_hook_data (line 507) | struct plt_hook_data
type elf_info (line 528) | struct elf_info
function bin_update_image (line 583) | int
function Dwarf_Die (line 644) | static Dwarf_Die
function Dwarf_Die (line 676) | static Dwarf_Die
function Dwarf_Die (line 728) | static Dwarf_Die
function linkmap_cb_status (line 798) | static linkmap_cb_status
function walk_linkmap (line 833) | static void
function has_libruby (line 858) | static int
function bin_type_size (line 866) | size_t
function bin_type_member_offset (line 886) | int
function open_elf (line 925) | static void
type elf_info (line 947) | struct elf_info
function verify_debug_checksum (line 958) | static int
function find_debug_syms (line 991) | static int
function dissect_elf (line 1062) | static int
function bin_init (line 1249) | void
FILE: ext/extconf.rb
function sys (line 14) | def sys(cmd)
function add_define (line 74) | def add_define(name)
FILE: ext/i386.c
type inline_st1_tramp (line 18) | struct inline_st1_tramp {
type inline_st1_base_short (line 28) | struct inline_st1_base_short {
type inline_st1_base_long (line 36) | struct inline_st1_base_long {
function arch_check_ins (line 46) | static int
function arch_insert_inline_st2_tramp (line 66) | int
FILE: ext/i386.h
type tramp_st2_entry (line 10) | struct tramp_st2_entry {
type inline_tramp_st2_entry (line 30) | struct inline_tramp_st2_entry {
FILE: ext/json.c
function json_gen_reset (line 12) | void
function json_gen_status (line 21) | json_gen_status
function json_gen_status (line 30) | json_gen_status
function json_gen_status (line 49) | json_gen_status
FILE: ext/json.h
type json_gen_state (line 10) | typedef enum {
type json_gen_t (line 21) | struct json_gen_t
FILE: ext/mach.c
type mach_config (line 29) | struct mach_config {
type symbol_data (line 46) | struct symbol_data {
type mach_config (line 53) | struct mach_config
type memprof_config (line 54) | struct memprof_config
type dyld_stub_entry (line 68) | struct dyld_stub_entry {
type dyld_stub_entry (line 76) | struct dyld_stub_entry
function set_dyld_stub_target (line 86) | static inline void
type mach_config (line 93) | struct mach_config
type mach_config (line 96) | struct mach_config
type symbol_data (line 96) | struct symbol_data
type mach_header_64 (line 99) | struct mach_header_64
type mach_config (line 102) | struct mach_config
type section_64 (line 107) | struct section_64
type nlist_64 (line 128) | struct nlist_64
function update_dyld_stub_table (line 150) | static int
type dyld_all_image_infos (line 170) | struct dyld_all_image_infos
type dyld_all_image_infos (line 172) | struct dyld_all_image_infos
type dyld_all_image_infos (line 173) | struct dyld_all_image_infos
type dyld_image_info (line 191) | struct dyld_image_info
type dyld_all_image_infos (line 193) | struct dyld_all_image_infos
type mach_header (line 197) | struct mach_header
type dyld_image_info (line 200) | struct dyld_image_info
type mach_header (line 215) | struct mach_header
type mach_header (line 217) | struct mach_header
type dyld_image_info (line 228) | struct dyld_image_info
function update_mach_section (line 254) | static int
function update_bin_for_mach_header (line 291) | static int
function find_dyld_image_index (line 331) | static int
function nlist_cmp (line 354) | static int
function extract_symbol_table (line 377) | static void
type mach_config (line 454) | struct mach_config
function extract_symbol_data (line 468) | static void
function free_mach_config (line 525) | static void
type mach_config (line 534) | struct mach_config
type dyld_image_info (line 542) | struct dyld_image_info
type mach_config (line 543) | struct mach_config
type mach_config (line 543) | struct mach_config
type mach_header_64 (line 552) | struct mach_header_64
type mach_header_64 (line 552) | struct mach_header_64
type fat_header (line 557) | struct fat_header
type fat_header (line 557) | struct fat_header
type fat_arch (line 560) | struct fat_arch
type fat_arch (line 560) | struct fat_arch
type fat_header (line 560) | struct fat_header
type fat_arch (line 560) | struct fat_arch
type mach_header_64 (line 563) | struct mach_header_64
type mach_header (line 577) | struct mach_header
type symbol_data (line 584) | struct symbol_data
type symbol_data (line 586) | struct symbol_data
type dyld_image_info (line 595) | struct dyld_image_info
type mach_config (line 601) | struct mach_config
type symbol_data (line 629) | struct symbol_data
type symbol_data (line 631) | struct symbol_data
function bin_update_image (line 655) | int
type mach_config (line 704) | struct mach_config
function bin_type_size (line 737) | size_t
function bin_type_member_offset (line 744) | int
function bin_init (line 752) | void
FILE: ext/memprof.c
type memprof_config (line 43) | struct memprof_config
type obj_track (line 51) | struct obj_track {
function ree_sourcefile_mark_each (line 64) | static int
function mri_sourcefile_mark_each (line 75) | static int
function sourcefile_marker (line 93) | static void
function VALUE (line 105) | static VALUE
function freelist_tramp (line 147) | static void
function objs_free (line 164) | static int
function objs_tabulate (line 172) | static int
type results (line 213) | struct results {
function objs_to_array (line 218) | static int
function VALUE (line 233) | static VALUE
function VALUE (line 249) | static VALUE
function memprof_strcmp (line 262) | static int
function VALUE (line 270) | static VALUE
function VALUE (line 316) | static VALUE
function json_print (line 324) | static void
function VALUE (line 337) | static VALUE
function json_gen_status (line 350) | static json_gen_status
function json_gen_status (line 362) | static json_gen_status
function json_gen (line 382) | static json_gen
function json_free (line 403) | static void
function VALUE (line 412) | static VALUE
function each_request_entry (line 440) | static int
function VALUE (line 459) | static VALUE
function VALUE (line 477) | static VALUE
function VALUE (line 483) | static VALUE
function each_hash_entry (line 611) | static int
function each_ivar (line 626) | static int
function obj_dump_class (line 677) | static inline void
function obj_dump (line 699) | static void
function globals_each_dump (line 1400) | static int
function finalizers_each_dump (line 1408) | static int
function memprof_dump_globals (line 1419) | static void
function memprof_dump_stack_frame (line 1440) | static void
function memprof_dump_stack (line 1492) | static void
function memprof_dump_lsof (line 1498) | static void
function memprof_dump_ps (line 1536) | static void
function memprof_dump_finalizers (line 1571) | static void
function objs_each_dump (line 1594) | static int
function VALUE (line 1602) | static VALUE
function VALUE (line 1627) | static VALUE
function init_memprof_config_base (line 1702) | static void
function init_memprof_config_extended (line 1711) | static void
function Init_memprof (line 1928) | void
FILE: ext/mmap.h
type mmap_info (line 22) | struct mmap_info {
function mmap_file_open (line 33) | int mmap_file_open(struct mmap_info *file_info) {
function munmap_file (line 61) | int munmap_file(struct mmap_info *file_info) {
FILE: ext/tracer.c
type tracer_list (line 16) | struct tracer_list {
type tracer_list (line 21) | struct tracer_list
function trace_insert (line 23) | int
function trace_remove (line 36) | int
function do_trace_invoke (line 56) | static void
function trace_invoke_all (line 83) | int
function trace_invoke (line 94) | int
function trace_set_output (line 107) | void
function json_gen (line 113) | json_gen
FILE: ext/tracer.h
type tracer (line 6) | struct tracer {
type trace_fn (line 14) | typedef enum {
type tracer (line 22) | struct tracer
FILE: ext/tracers/fd.c
type memprof_fd_stats (line 19) | struct memprof_fd_stats {
type tracer (line 44) | struct tracer
type memprof_fd_stats (line 45) | struct memprof_fd_stats
function read_tramp (line 47) | static ssize_t
function write_tramp (line 68) | static ssize_t
function recv_tramp (line 89) | static ssize_t
function connect_tramp (line 109) | static int
function select_tramp (line 126) | static int
function poll_tramp (line 144) | static int
function fd_trace_start (line 162) | static void
function fd_trace_stop (line 184) | static void
function fd_trace_reset (line 188) | static void
function fd_trace_dump (line 193) | static void
function install_fd_tracer (line 266) | void install_fd_tracer()
FILE: ext/tracers/gc.c
type memprof_gc_stats (line 15) | struct memprof_gc_stats {
type tracer (line 22) | struct tracer
type memprof_gc_stats (line 23) | struct memprof_gc_stats
function gc_tramp (line 26) | static void
function gc_trace_start (line 45) | static void
function gc_trace_stop (line 61) | static void
function gc_trace_reset (line 65) | static void
function gc_trace_dump (line 70) | static void
function install_gc_tracer (line 85) | void install_gc_tracer()
FILE: ext/tracers/memcache.c
type memprof_memcache_stats (line 15) | struct memprof_memcache_stats {
type tracer (line 23) | struct tracer
type memprof_memcache_stats (line 24) | struct memprof_memcache_stats
function memcached_set_tramp (line 39) | static int
function memcache_trace_start (line 48) | static void
function memcache_trace_stop (line 70) | static void
function memcache_trace_reset (line 74) | static void
function memcache_trace_dump_results (line 79) | static void
function memcache_trace_dump (line 106) | static void
function install_memcache_tracer (line 127) | void install_memcache_tracer()
FILE: ext/tracers/memory.c
type memprof_memory_stats (line 14) | struct memprof_memory_stats {
type tracer (line 30) | struct tracer
type memprof_memory_stats (line 31) | struct memprof_memory_stats
function free_tramp (line 91) | static void
function malloc_trace_start (line 102) | static void
function malloc_trace_stop (line 132) | static void
function malloc_trace_reset (line 137) | static void
function malloc_trace_dump (line 143) | static void
function install_malloc_tracer (line 193) | void install_malloc_tracer()
FILE: ext/tracers/mysql.c
type memprof_mysql_stats (line 16) | struct memprof_mysql_stats {
type tracer (line 24) | struct tracer
type memprof_mysql_stats (line 25) | struct memprof_mysql_stats
function real_query_tramp (line 30) | static int
function send_query_tramp (line 50) | static int
function mysql_trace_start (line 64) | static void
function mysql_trace_stop (line 82) | static void
function mysql_trace_reset (line 86) | static void
function mysql_trace_dump (line 91) | static void
function install_mysql_tracer (line 120) | void install_mysql_tracer()
FILE: ext/tracers/objects.c
type memprof_objects_stats (line 14) | struct memprof_objects_stats {
type tracer (line 19) | struct tracer
type memprof_objects_stats (line 20) | struct memprof_objects_stats
function record_last_obj (line 26) | static void
function VALUE (line 35) | static VALUE
function objects_trace_start (line 43) | static void
function objects_trace_stop (line 59) | static void
function objects_trace_reset (line 63) | static void
function objects_trace_dump (line 127) | static void
function install_objects_tracer (line 146) | void install_objects_tracer()
FILE: ext/tracers/postgres.c
type memprof_postgres_stats (line 16) | struct memprof_postgres_stats {
type tracer (line 21) | struct tracer
type memprof_postgres_stats (line 22) | struct memprof_postgres_stats
type memprof_sql_type (line 27) | enum memprof_sql_type
function postgres_trace_start (line 39) | static void
function postgres_trace_stop (line 53) | static void
function postgres_trace_reset (line 57) | static void
function postgres_trace_dump (line 62) | static void
function install_postgres_tracer (line 83) | void install_postgres_tracer()
FILE: ext/tracers/resources.c
type memprof_resources_stats (line 12) | struct memprof_resources_stats {
type tracer (line 22) | struct tracer
type memprof_resources_stats (line 23) | struct memprof_resources_stats
function resources_trace_start (line 25) | static void
function resources_trace_dump (line 39) | static void
function resources_trace_stop (line 70) | static void
function resources_trace_reset (line 74) | static void
function install_resources_tracer (line 78) | void install_resources_tracer()
FILE: ext/tracers/sql.c
function memprof_sql_query_type (line 3) | enum memprof_sql_type
type memprof_sql_type (line 41) | enum memprof_sql_type
FILE: ext/tracers/sql.h
type memprof_sql_type (line 4) | enum memprof_sql_type {
type memprof_sql_type (line 12) | enum memprof_sql_type
type memprof_sql_type (line 16) | enum memprof_sql_type
FILE: ext/tramp.c
type tramp_st2_entry (line 21) | struct tramp_st2_entry
type inline_tramp_st2_entry (line 28) | struct inline_tramp_st2_entry
type memprof_config (line 30) | struct memprof_config
function create_tramp_table (line 32) | void
function hook_freelist (line 59) | static void
function insert_tramp (line 123) | void
FILE: ext/util.c
function gnu_debuglink_crc32 (line 13) | unsigned long
function timeofday (line 78) | double
function timeofday_ms (line 93) | uint64_t
FILE: ext/util.h
type memprof_config (line 19) | struct memprof_config {
FILE: ext/x86_64.c
type inline_st1_tramp (line 39) | struct inline_st1_tramp {
type inline_st1_base (line 59) | struct inline_st1_base {
function arch_check_ins (line 79) | static int
function arch_insert_inline_st2_tramp (line 117) | int
FILE: ext/x86_64.h
type tramp_st2_entry (line 30) | struct tramp_st2_entry {
type inline_tramp_st2_entry (line 128) | struct inline_tramp_st2_entry {
FILE: ext/x86_gen.c
function arch_insert_st1_tramp (line 22) | int
FILE: ext/x86_gen.h
type st1_base (line 20) | struct st1_base {
function copy_instructions (line 44) | static void
FILE: lib/memprof/middleware.rb
type Memprof (line 2) | module Memprof
class Middleware (line 3) | class Middleware
method initialize (line 4) | def initialize(app, opts = {})
method call (line 8) | def call(env)
FILE: lib/memprof/tracer.rb
type Memprof (line 7) | module Memprof
class Tracer (line 12) | class Tracer
method initialize (line 13) | def initialize(app)
method call (line 16) | def call(env)
type Filter (line 26) | module Filter
function filter (line 27) | def self.filter(controller)
FILE: spec/memprof_spec.rb
function filename (line 12) | def filename
function filedata (line 16) | def filedata
FILE: spec/tracing_spec.rb
function filename (line 18) | def filename
function filedata (line 22) | def filedata
function filename (line 113) | def filename
function filedata (line 117) | def filedata
Condensed preview — 43 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (230K chars).
[
{
"path": ".gitignore",
"chars": 78,
"preview": "*.o\n*.so\n*.bundle\nMakefile\n*.dSYM\n*.log\n*.gem\next/dst/*\next/src/yajl-1.0.9/*\n\n"
},
{
"path": "LICENSE",
"chars": 1090,
"preview": "Copyright (c) 2009-2015 Joe Damato, Aman Gupta, and Jake Douglas\n\nPermission is hereby granted, free of charge, to any "
},
{
"path": "README.md",
"chars": 6983,
"preview": "# NOTE\n\nThis project is no longer maintained. It does not work for any Ruby version\nabove Ruby 1.8.7.\n\n# memprof\n\n(c) Jo"
},
{
"path": "Rakefile",
"chars": 1071,
"preview": "task :spec do\n Dir.chdir('ext') do\n sh \"make clean\" rescue nil\n sh \"ruby extconf.rb\"\n sh \"make\"\n end\n sh \"ru"
},
{
"path": "bin/memprof",
"chars": 5331,
"preview": "#!/usr/bin/env ruby\n\nrequire 'rubygems'\nrequire 'optparse'\nrequire 'restclient'\nrequire 'term/ansicolor'\n\nMEMPROF_URL = "
},
{
"path": "ext/arch.h",
"chars": 2428,
"preview": "#if !defined (_ARCH_H_)\n#define _ARCH_H_\n\n/*\n * The only supported architectures at this time are i{3-6}86 and amd64. Al"
},
{
"path": "ext/bin_api.h",
"chars": 2550,
"preview": "#if !defined(BIN_API__)\n#define BIN_API__\n#include <stddef.h>\n#include <stdint.h>\n\n/*\n * Some useful foward declarations"
},
{
"path": "ext/elf.c",
"chars": 34377,
"preview": "#if defined(HAVE_ELF)\n#define _GNU_SOURCE\n#include \"bin_api.h\"\n#include \"arch.h\"\n#include \"util.h\"\n\n#include <assert.h>\n"
},
{
"path": "ext/extconf.rb",
"chars": 6289,
"preview": "if RUBY_VERSION >= \"1.9\"\n STDERR.puts \"\\n\\n\"\n STDERR.puts \"***********************************************************"
},
{
"path": "ext/i386.c",
"chars": 2976,
"preview": "#if defined (_ARCH_i386_) || defined(_ARCH_i686_)\n\n#include <stdint.h>\n#include <string.h>\n\n#include <sys/mman.h>\n\n#incl"
},
{
"path": "ext/i386.h",
"chars": 1832,
"preview": "#if !defined(_i386_h_)\n#define _i386_h_\n\n#include <stdint.h>\n#include \"arch.h\"\n\n/*\n * This is the \"normal\" stage 2 tramp"
},
{
"path": "ext/json.c",
"chars": 1011,
"preview": "#ifndef _GNU_SOURCE\n#define _GNU_SOURCE\n#endif\n\n#include <assert.h>\n#include <stdio.h>\n#include <stdlib.h>\n#include <str"
},
{
"path": "ext/json.h",
"chars": 903,
"preview": "#if !defined(__JSON__H_)\n#define __JSON__H_\n\n#include <stdarg.h>\n#include <json/json_gen.h>\n\n/* HAX: copied from interna"
},
{
"path": "ext/mach.c",
"chars": 24590,
"preview": "#if defined(HAVE_MACH)\n\n#include \"arch.h\"\n#include \"bin_api.h\"\n#include \"mmap.h\"\n#include \"util.h\"\n\n#include <assert.h>\n"
},
{
"path": "ext/memprof.c",
"chars": 55277,
"preview": "#include <ruby.h>\n\n#ifndef _GNU_SOURCE\n#define _GNU_SOURCE\n#endif\n\n#include <assert.h>\n#include <err.h>\n#include <fcntl."
},
{
"path": "ext/mmap.h",
"chars": 1682,
"preview": "/*\n * Following code taken from http://thebends.googlecode.com/svn/trunk/mach-o/mmap.c\n * Copyright: Allen Porter <allen"
},
{
"path": "ext/tracer.c",
"chars": 2094,
"preview": "#include <assert.h>\n#include <stdio.h>\n#include <stdlib.h>\n#include <string.h>\n\n#include \"json.h\"\n#include \"tracer.h\"\n#i"
},
{
"path": "ext/tracer.h",
"chars": 834,
"preview": "#if !defined(__TRACER__H_)\n#define __TRACER__H_\n\n#include \"json.h\"\n\nstruct tracer {\n char *id;\n void (*start)();\n voi"
},
{
"path": "ext/tracers/fd.c",
"chars": 6009,
"preview": "#include <assert.h>\n#include <errno.h>\n#include <poll.h>\n#include <stdio.h>\n#include <stdlib.h>\n#include <string.h>\n#inc"
},
{
"path": "ext/tracers/gc.c",
"chars": 1934,
"preview": "#include <assert.h>\n#include <stdio.h>\n#include <stdlib.h>\n#include <string.h>\n#include <sys/time.h>\n#include <sys/resou"
},
{
"path": "ext/tracers/memcache.c",
"chars": 3525,
"preview": "#include <assert.h>\n#include <errno.h>\n#include <stdio.h>\n#include <stdlib.h>\n#include <string.h>\n#include <sys/time.h>\n"
},
{
"path": "ext/tracers/memory.c",
"chars": 4390,
"preview": "#include <assert.h>\n#include <errno.h>\n#include <stdio.h>\n#include <stdlib.h>\n#include <string.h>\n\n#include \"arch.h\"\n#in"
},
{
"path": "ext/tracers/mysql.c",
"chars": 2906,
"preview": "#include <assert.h>\n#include <errno.h>\n#include <stdio.h>\n#include <stdlib.h>\n#include <string.h>\n#include <sys/time.h>\n"
},
{
"path": "ext/tracers/objects.c",
"chars": 3024,
"preview": "#include <assert.h>\n#include <stdio.h>\n#include <stdlib.h>\n#include <string.h>\n\n#include \"arch.h\"\n#include \"bin_api.h\"\n#"
},
{
"path": "ext/tracers/postgres.c",
"chars": 1905,
"preview": "#include <assert.h>\n#include <errno.h>\n#include <stdio.h>\n#include <stdlib.h>\n#include <string.h>\n#include <sys/time.h>\n"
},
{
"path": "ext/tracers/resources.c",
"chars": 1785,
"preview": "#include <stdio.h>\n#include <stdlib.h>\n#include <string.h>\n#include <sys/time.h>\n#include <sys/resource.h>\n\n#include \"js"
},
{
"path": "ext/tracers/sql.c",
"chars": 909,
"preview": "#include <tracers/sql.h>\n\nenum memprof_sql_type\nmemprof_sql_query_type(const char *stmt, unsigned long length)\n{\n int i"
},
{
"path": "ext/tracers/sql.h",
"chars": 298,
"preview": "#if !defined(_sql_h_)\n#define _sql_h_\n\nenum memprof_sql_type {\n sql_SELECT,\n sql_UPDATE,\n sql_INSERT,\n sql_DELETE,\n "
},
{
"path": "ext/tramp.c",
"chars": 4102,
"preview": "#include <ruby.h>\n\n#ifndef _GNU_SOURCE\n#define _GNU_SOURCE\n#endif\n\n#include <sysexits.h>\n#include <sys/mman.h>\n#include "
},
{
"path": "ext/tramp.h",
"chars": 569,
"preview": "#if !defined(TRAMP__)\n#define TRAMP__\n\n/*\n * create_tramp_table - create the trampoline tables.\n */\nvoid\ncreate_tramp_ta"
},
{
"path": "ext/util.c",
"chars": 4373,
"preview": "#include <stdlib.h>\n#include <time.h>\n#include <util.h>\n\n#include <sys/time.h>\n\n/* This is the CRC function used by GNU."
},
{
"path": "ext/util.h",
"chars": 2004,
"preview": "#if !defined(__util_h__)\n#define __util_h__\n\n#include <stdint.h>\n\n#if defined(_MEMPROF_DEBUG)\n#include <stdio.h>\n#define"
},
{
"path": "ext/x86_64.c",
"chars": 5902,
"preview": "#if defined (_ARCH_x86_64_)\n\n#include <assert.h>\n#include <stdint.h>\n#include <string.h>\n\n#include \"arch.h\"\n#include \"x8"
},
{
"path": "ext/x86_64.h",
"chars": 6235,
"preview": "#if !defined(_x86_64_h_)\n#define _x86_64_h_\n\n#include <stdint.h>\n#include \"arch.h\"\n\n/*\n * tramp_st2_entry - stage 2 tram"
},
{
"path": "ext/x86_gen.c",
"chars": 1841,
"preview": "#include <assert.h>\n#include <stdint.h>\n#include \"arch.h\"\n#include \"x86_gen.h\"\n\n/*\n * arch_insert_st1_tramp - architectu"
},
{
"path": "ext/x86_gen.h",
"chars": 2103,
"preview": "#if !defined(_x86_gen_)\n#define _x86_gen_\n\n#include <assert.h>\n#include <stdint.h>\n#include <stdlib.h>\n#include <string."
},
{
"path": "lib/memprof/middleware.rb",
"chars": 438,
"preview": "require File.expand_path('../../memprof', __FILE__)\nmodule Memprof\n class Middleware\n def initialize(app, opts = {})"
},
{
"path": "lib/memprof/signal.rb",
"chars": 341,
"preview": "begin\n require File.expand_path('../../memprof', __FILE__)\nrescue LoadError\n require File.expand_path('../../../ext/me"
},
{
"path": "lib/memprof/tracer.rb",
"chars": 940,
"preview": "begin\n require File.expand_path('../../memprof', __FILE__)\nrescue LoadError\n require File.expand_path('../../../ext/me"
},
{
"path": "memprof.gemspec",
"chars": 636,
"preview": "spec = Gem::Specification.new do |s|\n s.name = 'memprof'\n s.version = '0.3.10'\n s.summary = 'Ruby Memory Profiler'\n "
},
{
"path": "spec/memprof_spec.rb",
"chars": 2370,
"preview": "require File.dirname(__FILE__) + \"/../ext/memprof\"\n\nrequire 'rubygems'\nrequire 'bacon'\nBacon.summary_on_exit\n\nrequire 't"
},
{
"path": "spec/memprof_uploader_spec.rb",
"chars": 4502,
"preview": "require 'rubygems'\nrequire 'bacon'\nBacon.summary_on_exit\n\ndescribe \"MemprofUploader\" do\n\n it \"should display help outpu"
},
{
"path": "spec/tracing_spec.rb",
"chars": 3197,
"preview": "require File.dirname(__FILE__) + \"/../ext/memprof\"\n\nrequire 'rubygems'\nrequire 'bacon'\nBacon.summary_on_exit\n\nrequire 't"
}
]
About this extraction
This page contains the full source code of the ice799/memprof GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 43 files (212.5 KB), approximately 62.6k tokens, and a symbol index with 295 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.