Showing preview only (1,152K chars total). Download the full file or copy to clipboard to get everything.
Repository: geocommons/geocoder
Branch: master
Commit: 2500432a06d3
Files: 137
Total size: 12.4 MB
Directory structure:
gitextract_g0bu7lk9/
├── .gitignore
├── History.txt
├── LICENSE.txt
├── Makefile
├── Manifest.txt
├── README.rdoc
├── REST.rdoc
├── TODO.txt
├── bin/
│ └── rebuild_metaphones
├── build/
│ ├── build_indexes
│ ├── rebuild_cluster
│ ├── sql/
│ │ ├── cluster.sql
│ │ ├── convert.sql
│ │ ├── create.sql
│ │ ├── index.sql
│ │ ├── place.sql
│ │ └── setup.sql
│ ├── tiger2009_import
│ └── tiger_import
├── conf/
│ ├── geocoder-us/
│ │ ├── geocoder.ru
│ │ └── unicorn.rb
│ └── init/
│ └── geocoder-us.conf
├── debian/
│ ├── README.Debian
│ ├── changelog
│ ├── compat
│ ├── control
│ ├── copyright
│ ├── default
│ ├── docs
│ ├── geocoder-us.postinst
│ ├── geocoder-us.prerm
│ ├── rules
│ └── source/
│ └── format
├── demos/
│ ├── api/
│ │ ├── server.rb
│ │ └── views/
│ │ └── index.erb
│ ├── cli.rb
│ ├── demo/
│ │ ├── app/
│ │ │ ├── ext/
│ │ │ │ └── geocodewrap.rb
│ │ │ └── views/
│ │ │ ├── index.builder
│ │ │ └── index.erb
│ │ ├── config/
│ │ │ ├── bootstraps.rb
│ │ │ └── geoenvironment.rb
│ │ ├── config.ru
│ │ ├── geocoder_helper.rb
│ │ ├── geocom_geocode.rb
│ │ ├── main.rb
│ │ ├── rakefile.rb
│ │ └── tmp/
│ │ └── restart.txt
│ ├── parse.rb
│ └── simpledemo/
│ ├── views/
│ │ ├── index.builder
│ │ └── index.erb
│ └── ws.rb
├── doc/
│ ├── Makefile
│ ├── html4css1.css
│ ├── lookup.rst
│ ├── parsing.rst
│ └── voidspace.css
├── gemspec
├── lib/
│ └── geocoder/
│ ├── us/
│ │ ├── address.rb
│ │ ├── constants.rb
│ │ ├── database.rb
│ │ ├── metaphone.rb
│ │ ├── numbers.rb
│ │ └── rest.rb
│ └── us.rb
├── navteq/
│ ├── README
│ ├── convert.sql
│ ├── navteq_import
│ └── prepare.sql
├── setup.rb
├── src/
│ ├── Makefile
│ ├── README
│ ├── liblwgeom/
│ │ ├── Makefile
│ │ ├── box2d.c
│ │ ├── lex.yy.c
│ │ ├── liblwgeom.h
│ │ ├── lwalgorithm.c
│ │ ├── lwalgorithm.h
│ │ ├── lwcircstring.c
│ │ ├── lwcollection.c
│ │ ├── lwcompound.c
│ │ ├── lwcurvepoly.c
│ │ ├── lwgeom.c
│ │ ├── lwgeom_api.c
│ │ ├── lwgparse.c
│ │ ├── lwgunparse.c
│ │ ├── lwline.c
│ │ ├── lwmcurve.c
│ │ ├── lwmline.c
│ │ ├── lwmpoint.c
│ │ ├── lwmpoly.c
│ │ ├── lwmsurface.c
│ │ ├── lwpoint.c
│ │ ├── lwpoly.c
│ │ ├── lwsegmentize.c
│ │ ├── lwutil.c
│ │ ├── measures.c
│ │ ├── postgis_config.h
│ │ ├── ptarray.c
│ │ ├── vsprintf.c
│ │ ├── wktparse.h
│ │ ├── wktparse.lex
│ │ ├── wktparse.tab.c
│ │ ├── wktparse.tab.h
│ │ └── wktparse.y
│ ├── libsqlite3_geocoder/
│ │ ├── Makefile
│ │ ├── Makefile.nix
│ │ ├── Makefile.redhat
│ │ ├── extension.c
│ │ ├── extension.h
│ │ ├── levenshtein.c
│ │ ├── metaphon.c
│ │ ├── util.c
│ │ └── wkb_compress.c
│ ├── metaphone/
│ │ ├── Makefile
│ │ ├── README
│ │ ├── extension.c
│ │ └── metaphon.c
│ └── shp2sqlite/
│ ├── Makefile
│ ├── Makefile.macosx
│ ├── Makefile.nix
│ ├── Makefile.redhat
│ ├── dbfopen.c
│ ├── getopt.c
│ ├── getopt.h
│ ├── shapefil.h
│ ├── shp2sqlite.c
│ └── shpopen.c
└── test/
├── address.rb
├── benchmark.rb
├── constants.rb
├── data/
│ ├── address-sample.csv
│ ├── db-test.csv
│ └── locations.csv
├── database.rb
├── generate.rb
├── numbers.rb
└── run.rb
================================================
FILE CONTENTS
================================================
================================================
FILE: .gitignore
================================================
*.o
*.so
*.gem
pkg/
bin/shp2sqlite
src/shp2sqlite/shp2sqlite
src/liblwgeom/liblwgeom.a
doc/*.html
*.log
================================================
FILE: History.txt
================================================
=== 1.0.0 / 2009-06-02
* 1 major enhancement
* Birthday!
================================================
FILE: LICENSE.txt
================================================
GNU LESSER GENERAL PUBLIC LICENSE
Version 3, 29 June 2007
Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
This version of the GNU Lesser General Public License incorporates
the terms and conditions of version 3 of the GNU General Public
License, supplemented by the additional permissions listed below.
0. Additional Definitions.
As used herein, "this License" refers to version 3 of the GNU Lesser
General Public License, and the "GNU GPL" refers to version 3 of the GNU
General Public License.
"The Library" refers to a covered work governed by this License,
other than an Application or a Combined Work as defined below.
An "Application" is any work that makes use of an interface provided
by the Library, but which is not otherwise based on the Library.
Defining a subclass of a class defined by the Library is deemed a mode
of using an interface provided by the Library.
A "Combined Work" is a work produced by combining or linking an
Application with the Library. The particular version of the Library
with which the Combined Work was made is also called the "Linked
Version".
The "Minimal Corresponding Source" for a Combined Work means the
Corresponding Source for the Combined Work, excluding any source code
for portions of the Combined Work that, considered in isolation, are
based on the Application, and not on the Linked Version.
The "Corresponding Application Code" for a Combined Work means the
object code and/or source code for the Application, including any data
and utility programs needed for reproducing the Combined Work from the
Application, but excluding the System Libraries of the Combined Work.
1. Exception to Section 3 of the GNU GPL.
You may convey a covered work under sections 3 and 4 of this License
without being bound by section 3 of the GNU GPL.
2. Conveying Modified Versions.
If you modify a copy of the Library, and, in your modifications, a
facility refers to a function or data to be supplied by an Application
that uses the facility (other than as an argument passed when the
facility is invoked), then you may convey a copy of the modified
version:
a) under this License, provided that you make a good faith effort to
ensure that, in the event an Application does not supply the
function or data, the facility still operates, and performs
whatever part of its purpose remains meaningful, or
b) under the GNU GPL, with none of the additional permissions of
this License applicable to that copy.
3. Object Code Incorporating Material from Library Header Files.
The object code form of an Application may incorporate material from
a header file that is part of the Library. You may convey such object
code under terms of your choice, provided that, if the incorporated
material is not limited to numerical parameters, data structure
layouts and accessors, or small macros, inline functions and templates
(ten or fewer lines in length), you do both of the following:
a) Give prominent notice with each copy of the object code that the
Library is used in it and that the Library and its use are
covered by this License.
b) Accompany the object code with a copy of the GNU GPL and this license
document.
4. Combined Works.
You may convey a Combined Work under terms of your choice that,
taken together, effectively do not restrict modification of the
portions of the Library contained in the Combined Work and reverse
engineering for debugging such modifications, if you also do each of
the following:
a) Give prominent notice with each copy of the Combined Work that
the Library is used in it and that the Library and its use are
covered by this License.
b) Accompany the Combined Work with a copy of the GNU GPL and this license
document.
c) For a Combined Work that displays copyright notices during
execution, include the copyright notice for the Library among
these notices, as well as a reference directing the user to the
copies of the GNU GPL and this license document.
d) Do one of the following:
0) Convey the Minimal Corresponding Source under the terms of this
License, and the Corresponding Application Code in a form
suitable for, and under terms that permit, the user to
recombine or relink the Application with a modified version of
the Linked Version to produce a modified Combined Work, in the
manner specified by section 6 of the GNU GPL for conveying
Corresponding Source.
1) Use a suitable shared library mechanism for linking with the
Library. A suitable mechanism is one that (a) uses at run time
a copy of the Library already present on the user's computer
system, and (b) will operate properly with a modified version
of the Library that is interface-compatible with the Linked
Version.
e) Provide Installation Information, but only if you would otherwise
be required to provide such information under section 6 of the
GNU GPL, and only to the extent that such information is
necessary to install and execute a modified version of the
Combined Work produced by recombining or relinking the
Application with a modified version of the Linked Version. (If
you use option 4d0, the Installation Information must accompany
the Minimal Corresponding Source and Corresponding Application
Code. If you use option 4d1, you must provide the Installation
Information in the manner specified by section 6 of the GNU GPL
for conveying Corresponding Source.)
5. Combined Libraries.
You may place library facilities that are a work based on the
Library side by side in a single library together with other library
facilities that are not Applications and are not covered by this
License, and convey such a combined library under terms of your
choice, if you do both of the following:
a) Accompany the combined library with a copy of the same work based
on the Library, uncombined with any other library facilities,
conveyed under the terms of this License.
b) Give prominent notice with the combined library that part of it
is a work based on the Library, and explaining where to find the
accompanying uncombined form of the same work.
6. Revised Versions of the GNU Lesser General Public License.
The Free Software Foundation may publish revised and/or new versions
of the GNU Lesser General Public License from time to time. Such new
versions will be similar in spirit to the present version, but may
differ in detail to address new problems or concerns.
Each version is given a distinguishing version number. If the
Library as you received it specifies that a certain numbered version
of the GNU Lesser General Public License "or any later version"
applies to it, you have the option of following the terms and
conditions either of that published version or of any later version
published by the Free Software Foundation. If the Library as you
received it does not specify a version number of the GNU Lesser
General Public License, you may choose any version of the GNU Lesser
General Public License ever published by the Free Software Foundation.
If the Library as you received it specifies that a proxy can decide
whether future versions of the GNU Lesser General Public License shall
apply, that proxy's public statement of acceptance of any version is
permanent authorization for you to choose that version for the
Library.
================================================
FILE: Makefile
================================================
all:
make -C src install
gem build gemspec
test: all
ruby -Ilib tests/run.rb
install: all
# gem install *.gem
clean:
make -C src clean
rm -f lib/geocoder/us/sqlite3.so
rm -f *.gem
================================================
FILE: Manifest.txt
================================================
History.txt
Manifest.txt
README.rdoc
Rakefile
lib/geocoder/us/database.rb
lib/geocoder/us/numbers.rb
lib/geocoder/us/address.rb
lib/geocoder/us/constants.rb
tests/database.rb
tests/numbers.rb
tests/generate.rb
tests/run.rb
tests/address.rb
tests/benchmark.rb
tests/constants.rb
tests/data/address-sample.csv
tests/data/locations.csv
tests/data/db-test.csv
================================================
FILE: README.rdoc
================================================
= Geocoder::US
Geocoder::US 2.0 is a software package designed to geocode US street
addresses. Although it is primarily intended for use with the US Census
Bureau's free TIGER/Line dataset, it uses an abstract US address data model
that can be employed with other sources of US street address range data.
Geocoder::US 2.0 implements a Ruby interface to parse US street addresses, and
perform fuzzy lookup against an SQLite 3 database. Geocoder::US is designed to
return the best matches found, with geographic coordinates interpolated from
the street range dataset. Geocoder::US will fill in missing information, and
it knows about standard and common non-standard postal abbreviations, ordinal
versus cardinal numbers, and more.
Geocoder::US 2.0 is shipped with a free US ZIP code data set, compiled from
public domain sources.
== Synopsis
>> require 'geocoder/us'
>> db = Geocoder::US::Database.new("/opt/tiger/geocoder.db")
>> p db.geocode("1600 Pennsylvania Av, Washington DC")
[{:pretyp=>"", :street=>"Pennsylvania", :sufdir=>"NW", :zip=>"20502",
:lon=>-77.037528, :number=>"1600", :fips_county=>"11001", :predir=>"",
:precision=>:range, :city=>"Washington", :lat=>38.898746, :suftyp=>"Ave",
:state=>"DC", :prequal=>"", :sufqual=>"", :score=>0.906, :prenum=>""}]
== Web service
There is a small example webservice included in the gem at lib/geocoder/rest.rb. It requires Sinatra to run as well as setting the GEOCODER_DB environment variable. To run it:
GEOCODER_DB=/path/to/geocoder/geocoder.db ruby /path/to/gem/geocoder/lib/geocoder/us/rest.rb
You can then query the GeoCommons geocoder by calling the web service like:
http://localhost:9393/geocode.html?q=303+11th St NE,Washington,DC
== Optional Options
dbtype Geocoder::US::Database.new("/opt/tiger/geocoder.db", {:dbtype => 1})
The dbtype option is used when your datasbase encodes it's geometry blogs according to a special format. The
first and default value, is in a series of little-endian 4-byte ints ("V*")
The second is for ('CVVD*') format. Use option value 2 for this second type
debug Geocoder::US::Database.new("/opt/tiger/geocoder.db", {:debug => false})
cache Geocoder::US::Database.new("/opt/tiger/geocoder.db", {:cache => 50000})
The cache_size argument is measured in kilobytes and is used to set the SQLite cache size;
larger values will trade memory for speed in long-running processes.
== Prerequisites
To build Geocoder::US, you will need gcc/g++, make, bash or equivalent, the
standard *NIX 'unzip' utility, and the SQLite 3 executable and development
files installed on your system.
To use the Ruby interface, you will need the 'Text' gem installed from
rubyforge. To run the tests, you will also need the 'fastercsv' gem.
Additionally, you will need a custom build of the 'sqlite3-ruby' gem that
supports loading extension modules in SQLite. You can get a patched version of
this gem from http://github.com/schuyler/sqlite3-ruby/. Until the sqlite3-ruby
maintainers roll in the relevant patch, you will need *this* version.
*NOTE*: If you do not have /usr/include/sqlite3ext.h installed, then your
sqlite3 binaries are probably not configured to support dynamic extension
loading. If not, you *must* compile and install SQLite from source, or rebuild
your system packages. This is not believed to be a problem on Debian/Ubuntu,
but is known to be a problem with Red Hat/CentOS.
*NOTE*: If you *do* have to install from source, make sure that the
source-installed 'sqlite3' program is in your path before proceeding (and not
the system-installed version), using `which sqlite3`. Also, be sure that you've
added your source install prefix (usually /usr/local) to /etc/ld.so.conf (or
its moral equivalent) and that you've run /sbin/ldconfig.
== Thread safety
SQLite 3 is not designed for concurrent use of a single database handle across
multiple threads. Therefore, to prevent segfaults, Geocoder::US::Database
implements a global mutex that wraps all database access. The use of this mutex
will ensure stability in multi-threaded applications, but incurs a performance
penalty. However, since the database is read-only from Ruby, there's no reason
in principle why multi-threaded apps can't each have their own database handle.
To disable the mutex for better performance, you can do the following:
* Read the following and make sure you understand them:
* http://www.sqlite.org/faq.html#q6
* http://www.sqlite.org/cvstrac/wiki?p=MultiThreading
* Make sure you have compiled SQLite 3 with thread safety enabled.
* Instantiate a separate Geocoder::US::Database object for *each* thread
in your Ruby script, and pass :threadsafe => true to new() to disable mutex
synchronization.
Per the SQLite 3 documentation, do *not* attempt to retain a
Geocoder::US::Database object across a fork! "Problems will result if you do."
== Building Geocoder::US
Unpack the source and run 'make'. This will compile the SQLite 3 extension
needed by Geocoder::US, the Shapefile import utility, and the Geocoder-US
gem.
You can run 'make install' as root to install the gem systemwide.
== Generating a Geocoder::US Database
Build the package from source as described above. Generating the database
involves three basic steps:
* Import the Shapefile data into an SQLite database.
* Build the database indexes.
* Optionally, rebuild the database to cluster indexed rows.
We will presume that you are building a Geocoder::US database from TIGER/Line,
and that you have obtained the complete set of TIGER/Line ZIP files, and put
the entire tree in /opt/tiger. Please adjust these instructions as needed.
A full TIGER/Line database import takes ten hours to run on a normal Amazon
EC2 instance, and takes up a little over 5 gigabytes after all is said and
done. You will need to have at least 12 gigabytes of free disk space *after*
downloading the TIGER/Line dataset, if you are building the full database.
=== Import TIGER/Line
From inside the Geocoder::US source tree, run the following:
$ build/tiger_import /opt/tiger/geocoder.db /opt/tiger
This will unpack each TIGER/Line ZIP file to a temporary directory, and
perform the extract/transform/load sequence to incrementally build the
database. The process takes about 10-12 hours on a normal Amazon EC2 instance,
or about 5 CPU hours flat out on a modern PC. Note that not all TIGER/Line
source files contain address range information, so you will see error messages
for some counties, but this is normal.
If you only want to import specific counties, you can pipe a list of
TIGER/Line county directories to tiger_import on stdin. For example,
the following will install just the data for the state of Delaware:
$ ls -d /opt/tiger/10_DELAWARE/1* | build/tiger_import ~/delaware.db
The tiger_import process uses a binary utility, shp2sqlite, which is derived
from shp2pgsql, which ships with PostGIS. The shp2sqlite utility converts
.shp and .dbf files into SQL suitable for import into SQLite. This SQL
is then piped into the sqlite3 command line tool, where it is loaded into
temporary tables, and then a set of static SQL statements (kept in the sql/
directory) are used to transform this data and import it into the database
itself.
== Build metaphones using Ruby metaphone
run bin/rebuild_metaphones /opt/tiger/geocoder.db
This creates the metaphones using Ruby's metaphone function and will produce better geocoding results.
=== Build the indexes
After the database import is complete, you will want to construct the database
indexes:
$ build/build_indexes /opt/tiger/geocoder.db
This process takes 25 minutes on an EC2 instance (8 CPU minutes), but it's a
*lot* faster than building the indexes incrementally during the import
process. Basically, this process simply feeds SQL statements to the sqlite3
utility to construct the indexes on the existing database.
=== Cluster the database tables (optional)
As a final optional step, you can cluster the database tables according to
their indexes, which will make the database smaller, and lookups faster. This
process will take an hour or two, and may be a micro-optimization.
$ build/rebuild_cluster /opt/tiger/geocoder.db
You will need as much free disk space to run rebuild_cluster as the database
takes up, because the process essentially reconstructs the database in a new
file, and then it renames the new database over top of the old.
== Running the unit tests
From within the source tree, you can run the following:
$ ruby tests/run.rb
This tests the libraries, except for the database routines. If you have a
database built, you can run the test harness like so:
$ ruby tests/run.rb /opt/tiger/geocoder.db
The full test suite may take 30 or so seconds to run completely.
== License
Geocoder::US 2.0 was based on earlier work by Schuyler Erle on
a Perl module of the same name. You can find it at
http://search.cpan.org/~sderle/.
Geocoder::US 2.0 was written by Schuyler Erle, of Entropy Free LLC,
with the gracious support of FortiusOne, Inc. Please send bug reports,
patches, kudos, etc. to patches at geocoder.us.
Copyright (c) 2009 FortiusOne, Inc.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
================================================
FILE: REST.rdoc
================================================
GET /1.0/geocode/address.json
The geocode/address endpoint returns the interpolated latitude and longitude of
a US street address or street intersection. When given a US city or ZIP code,
the approximate center point of that place will be returned instead.
The geocoder attempts to return the most accurate possible result, including,
where possible, correcting the given street type, city, or postal code, and
identifying and correcting misspellings in the street or city name in the given
address.
Currently, address geocoding only works in the United States.
Parameters:
q = a string containing a US street address.
Returns a GeoJSON feature collection:
{
"type": "FeatureCollection",
"features": [
{
"type": "Feature",
"properties": {
"number": "41",
"street": "Decatur St",
"city": "San Francisco",
"state": "CA",
"zip": "94103",
"fips_county": "06075",
"score": 1.0,
"precision":"range"
},
"geometry": {
"type": "Point",
"coordinates": [-122.406032, 37.772502]
}
}
],
"address":"41 Decatur St, San Francisco CA 94103"
}
Each address match in the feature collection contains some combination of the
following properties:
number
The building number of the address. When a building number is not
included in a range stored in the address database, the nearest
known building number will be returned in its place.
street
The name of the street found in the database that matches the address,
given in a normalized form.
street1 / street2
When an address is parsed as an intersection, the intersecting streets
are returned as `street1` and `street2` in place of the `number` and
`street` fields.
city
The city matching the given address. In the US, this is typically
determined from the matching ZIP code, so, for ZIP codes that cover
more than one named place, the results may be different from what you
expect, but will still be suitable for postal addressing.
state
The two letter postal abbreviation of the state containing the matching
address.
zip
In the US, the five digit ZIP code of the matching address.
plus4
In the US, the ZIP+4 extension parsed from the address, if any. This
extension is not actually used in the geocoding process, but is
returned for convenience.
fips_county
In the US, the FIPS 6-4 code of the county containing the address.
prenum / sufnum
If the building number has a non-numeric prefix, it will be returned in
`prenum`. Ditto `sufnum` for non-numeric suffixes.
precision
The qualitative precision of the geocode. The value will be one of
`intersection`, 'range`, `street`, `zip`, or `city`.
score
The percentage of text match between the given address and the geocoded
result, expressed as a float between 0 and 1. A higher score indicates
a closer match. Results with a score below 0.5 should be regarded with
care.
================================================
FILE: TODO.txt
================================================
1. Check interpolate measure: scale longitude or not?
5. Intersections...
- import ALL linestrings (even those with without ranges)
- throw away internal points on lines that don't have ranges
7. Documentation (*)
8. Make SQLite memory cache size an option to the Database constructor
9. Precision and accuracy measure
10. Street line set back
================================================
FILE: bin/rebuild_metaphones
================================================
#!/usr/bin/ruby
require 'rubygems'
require 'sqlite3'
require 'text'
if(ARGV.length < 1)
print "Missing SQLite file parameter"
printf "\nUSAGE:\n"
print """./rebuild_metaphones [SQLite File]
[SQLite File] - SQLite file generated by the geocoder.
"""
exit
end
@db = SQLite3::Database.new(ARGV[0])
@db.create_function("metaphone", 2) do |func, string, len|
test = string.to_s.gsub(/\W/o, "")
if test =~ /^(\d+)/o
mph = $1
elsif test =~ /^([wy])$/io
mph = $1
else
mph = Text::Metaphone.metaphone test
end
func.result = mph[0...len.to_i]
end
sql = "update place set city_phone = metaphone(city,5)"
@db.execute sql
@db.close
================================================
FILE: build/build_indexes
================================================
#!/bin/bash
BASE=$(dirname $0)
PATH=$PATH:$BASE/bin
SQL="$BASE/sql"
# Just run the SQL that constructs the indexes.
sqlite3 $1 < ${SQL}/index.sql
================================================
FILE: build/rebuild_cluster
================================================
#!/bin/bash
BASE=$(dirname $0)
PATH=$PATH:$BASE/bin
SQL="$BASE/sql"
OLD_DB=$1
DATABASE=${OLD_DB}.$$
[ -r $DATABASE ] && echo "$DATABASE already exists." && exit -1
[ ! -r $OLD_DB ] && echo "Can't read $OLD_DB." && exit -1
# Create a shiny new database, attach the old one,
# extract the data from it, and then index that.
# Finally, overwrite the old database with the new one.
( cat ${SQL}/create.sql && \
echo "ATTACH DATABASE '${OLD_DB}' AS old;" && \
cat ${SQL}/cluster.sql && \
echo "DETACH DATABASE old;" && \
cat ${SQL}/index.sql && \
echo "ANALYZE;" ) | sqlite3 $DATABASE \
&& mv $DATABASE $OLD_DB
================================================
FILE: build/sql/cluster.sql
================================================
.echo on
-- turn off various pragmas to make SQLite faster
PRAGMA temp_store=MEMORY;
PRAGMA journal_mode=OFF;
PRAGMA synchronous=OFF;
PRAGMA cache_size=500000;
PRAGMA count_changes=0;
BEGIN TRANSACTION;
-- order the contents of each table by their indexes to reduce
-- the number of disk pages that need to be read on each query.
INSERT INTO place SELECT * FROM old.place ORDER BY zip, priority;
INSERT INTO edge SELECT * FROM old.edge ORDER BY tlid;
INSERT INTO feature SELECT * FROM old.feature ORDER BY street_phone, zip;
INSERT INTO feature_edge SELECT * FROM old.feature_edge ORDER BY fid;
INSERT INTO range SELECT * FROM old.range ORDER BY tlid;
COMMIT;
================================================
FILE: build/sql/convert.sql
================================================
BEGIN;
-- start by indexing the temporary tables created from the input data.
CREATE INDEX featnames_tlid ON tiger_featnames (tlid);
CREATE INDEX addr_tlid ON tiger_addr (tlid);
CREATE INDEX edges_tlid ON tiger_edges (tlid);
-- generate a summary table matching each edge to one or more ZIPs
-- for those edges that are streets and have a name
CREATE TEMPORARY TABLE linezip AS
SELECT DISTINCT tlid, zip FROM (
SELECT tlid, zip FROM tiger_addr a
UNION
SELECT tlid, zipr AS zip FROM tiger_edges e
WHERE e.mtfcc LIKE 'S%' AND zipr <> "" AND zipr IS NOT NULL
UNION
SELECT tlid, zipl AS zip FROM tiger_edges e
WHERE e.mtfcc LIKE 'S%' AND zipl <> "" AND zipl IS NOT NULL
) AS whatever;
CREATE INDEX linezip_tlid ON linezip (tlid);
-- generate features from the featnames table for each desired edge
-- computing the metaphone hash of the name in the process.
-- CREATE TEMPORARY TABLE sqlite_sequence (
-- name VARCHAR(255),
-- seq INTEGER);
CREATE TEMPORARY TABLE feature_bin (
fid INTEGER PRIMARY KEY AUTOINCREMENT,
street VARCHAR(100),
street_phone VARCHAR(5),
paflag BOOLEAN,
zip CHAR(5));
INSERT OR IGNORE INTO sqlite_sequence (name, seq) VALUES ('feature_bin',0);
UPDATE sqlite_sequence
SET seq=(SELECT max(fid) FROM feature)
WHERE name="feature_bin";
INSERT INTO feature_bin
SELECT DISTINCT NULL, fullname, metaphone(name,5), paflag, zip
FROM linezip l, tiger_featnames f
WHERE l.tlid=f.tlid AND name <> "" AND name IS NOT NULL;
CREATE INDEX feature_bin_idx ON feature_bin (street, zip);
INSERT INTO feature_edge
SELECT DISTINCT fid, f.tlid
FROM linezip l, tiger_featnames f, feature_bin b
WHERE l.tlid=f.tlid AND l.zip=b.zip
AND f.fullname=b.street AND f.paflag=b.paflag;
-- SELECT min(fid),max(fid) FROM feature_bin;
INSERT INTO feature
SELECT * FROM feature_bin;
-- generate edges from the edges table for each desired edge, running
-- a simple compression on the WKB geometry (because they're all
-- linestrings).
INSERT OR IGNORE INTO edge
SELECT l.tlid, compress_wkb_line(the_geom) FROM
(SELECT DISTINCT tlid FROM linezip) AS l, tiger_edges e
WHERE l.tlid=e.tlid AND fullname <> "" AND fullname IS NOT NULL;
-- generate all ranges from the addr table, stripping off any non-digit
-- prefixes and putting them in a separate column.
INSERT INTO range
SELECT tlid, digit_suffix(fromhn), digit_suffix(tohn),
nondigit_prefix(fromhn), zip, side
FROM tiger_addr;
END;
DROP TABLE feature_bin;
DROP TABLE linezip;
DROP TABLE tiger_addr;
DROP TABLE tiger_featnames;
DROP TABLE tiger_edges;
================================================
FILE: build/sql/create.sql
================================================
-- initialize the database tables.
-- 'place' contains the gazetteer of place names.
CREATE TABLE place(
zip CHAR(5),
city VARCHAR(100),
state CHAR(2),
city_phone VARCHAR(5),
lat NUMERIC(9,6),
lon NUMERIC(9,6),
status CHAR(1),
fips_class CHAR(2),
fips_place CHAR(7),
fips_county CHAR(5),
priority char(1));
-- 'edge' stores the line geometries and their IDs.
CREATE TABLE edge (
tlid INTEGER(10) PRIMARY KEY,
geometry BLOB);
-- 'feature' stores the name(s) and ZIP(s) of each edge.
CREATE TABLE feature (
fid INTEGER PRIMARY KEY,
street VARCHAR(100),
street_phone VARCHAR(5),
paflag BOOLEAN,
zip CHAR(5));
-- 'feature_edge' links each edge to a feature.
CREATE TABLE feature_edge (
fid INTEGER,
tlid INTEGER);
-- 'range' stores the address range(s) for each edge.
CREATE TABLE range (
tlid INTEGER(10),
fromhn INTEGER(6),
tohn INTEGER(6),
prenum VARCHAR(12),
zip CHAR(5),
side CHAR(1));
================================================
FILE: build/sql/index.sql
================================================
.echo on
PRAGMA temp_store=MEMORY;
PRAGMA journal_mode=MEMORY;
PRAGMA synchronous=OFF;
PRAGMA cache_size=500000;
PRAGMA count_changes=0;
-- create indexes for all the relevant ways each table is queried.
CREATE INDEX place_city_phone_state_idx ON place (city_phone, state);
CREATE INDEX place_zip_priority_idx ON place (zip, priority);
CREATE INDEX feature_street_phone_zip_idx ON feature (street_phone, zip);
CREATE INDEX feature_edge_fid_idx ON feature_edge (fid);
CREATE INDEX range_tlid_idx ON range (tlid);
================================================
FILE: build/sql/place.sql
================================================
[File too large to display: 11.4 MB]
================================================
FILE: build/sql/setup.sql
================================================
-- create temporary tables to hold the TIGER/Line data before it's
-- transformed and loaded into the permanent tables.
--
-- this file was made by running 'shp2pgsql -p' on each of the
-- TIGER/Line shapefiles and then massaging the result by hand.
--
PRAGMA temp_store=MEMORY;
PRAGMA journal_mode=MEMORY;
PRAGMA synchronous=OFF;
PRAGMA cache_size=500000;
PRAGMA count_changes=0;
CREATE TEMPORARY TABLE "tiger_edges" (
"statefp" varchar(2),
"countyfp" varchar(3),
"tlid" int8,
"tfidl" int8,
"tfidr" int8,
"mtfcc" varchar(5),
"fullname" varchar(100),
"smid" varchar(22),
"lfromadd" varchar(12),
"ltoadd" varchar(12),
"rfromadd" varchar(12),
"rtoadd" varchar(12),
"zipl" varchar(5),
"zipr" varchar(5),
"featcat" varchar(1),
"hydroflg" varchar(1),
"railflg" varchar(1),
"roadflg" varchar(1),
"olfflg" varchar(1),
"passflg" varchar(1),
"divroad" varchar(1),
"exttyp" varchar(1),
"ttyp" varchar(1),
"deckedroad" varchar(1),
"artpath" varchar(1),
"persist" varchar(1),
"gcseflg" varchar(1),
"offsetl" varchar(1),
"offsetr" varchar(1),
"tnidf" int8,
"tnidt" int8,
"the_geom" blob
);
-- SELECT AddGeometryColumn('','edges','the_geom','-1','MULTILINESTRING',2);
CREATE TEMPORARY TABLE "tiger_featnames" (
"tlid" int8,
"fullname" varchar(100),
"name" varchar(100),
"predirabrv" varchar(15),
"pretypabrv" varchar(50),
"prequalabr" varchar(15),
"sufdirabrv" varchar(15),
"suftypabrv" varchar(50),
"sufqualabr" varchar(15),
"predir" varchar(2),
"pretyp" varchar(3),
"prequal" varchar(2),
"sufdir" varchar(2),
"suftyp" varchar(3),
"sufqual" varchar(2),
"linearid" varchar(22),
"mtfcc" varchar(5),
"paflag" varchar(1));
CREATE TEMPORARY TABLE "tiger_addr" (
"tlid" int8,
"fromhn" varchar(12),
"tohn" varchar(12),
"side" varchar(1),
"zip" varchar(5),
"plus4" varchar(4),
"fromtyp" varchar(1),
"totyp" varchar(1),
"fromarmid" int4,
"toarmid" int4,
"arid" varchar(22),
"mtfcc" varchar(5));
================================================
FILE: build/tiger2009_import
================================================
#!/bin/bash
TMP="/tmp/tiger-import.$$"
SHPS="edges"
DBFS="featnames addr"
BASE=$(dirname $0)
PATH=$PATH:$BASE
SQL="$BASE/sql"
HELPER_LIB="$BASE/../lib/geocoder/us/sqlite3.so"
DATABASE=$1
shift
mkdir -p $TMP || exit 1
# Initialize the database if it doesn't exist.
[ ! -r $DATABASE ] && cat ${SQL}/{create,place}.sql | sqlite3 $DATABASE
# Marshal the county directories to import.
#
# If no directory was given on the command-line, read a list from STDIN.
if [ x"$1" = x"" ]; then
cat
else
# Otherwise, find all of the contents of each state directory.
ls -d $1/[0-9]* | while read state; do
ls -d ${state}/[0-9]*
done
fi | while read county; do
echo "--- $county"
# Unpack the county files into the temp directory.
for file in $SHPS $DBFS; do
ZIP=$(ls ${county}/*_${file}.zip 2>/dev/null)
SHP=$(ls ${county}/*_${file}.* 2>/dev/null)
if [ x"$ZIP" != x"" ]; then
unzip -q $ZIP -d $TMP
elif [ x"$SHP" != x"" ]; then
ln -s $SHP $TMP
fi
done
# Generate an SQL stream to feed into the sqlite3 binary.
# Start by loading the helper libs and initializing the temporary tables
# that will hold the TIGER data before ETL.
(echo ".load $HELPER_LIB" && \
cat ${SQL}/setup.sql && \
for file in $SHPS; do
# Convert each Shapefile into SQL statements.
shp2sqlite -aS ${TMP}/*_${file}.shp tiger_${file}
done && \
for file in $DBFS; do
# Convert each DBF into SQL statements likewise.
shp2sqlite -an ${TMP}/*_${file}.dbf tiger_${file}
done && \
cat ${SQL}/convert.sql) | sqlite3 $DATABASE
# Finally, do the transform/load phase (convert.sql)
# and clean up the temporary files.
rm -f $TMP/*
done 2>&1 | tee import-$$.log
rm -rf $TMP
================================================
FILE: build/tiger_import
================================================
#!/bin/bash
TMP="/tmp/tiger-import.$$"
SHPS="edges"
DBFS="featnames addr"
BASE=$(dirname $0)
PATH=$PATH:$BASE
SQL="$BASE/sql"
HELPER_LIB="$BASE/../lib/geocoder/us/sqlite3.so"
SHP2SQLITE=../src/shp2sqlite/shp2sqlite
DATABASE=$1
SOURCE=$2
shift
shift
mkdir -p $TMP || exit 1
# Initialize the database if it doesn't exist.
# Added places back in after adding the "drop if exists" directive to the SQL file. theduckylittle, 2011/12/15
[ ! -r $DATABASE ] && cat ${SQL}/{create,place}.sql | sqlite3 $DATABASE
#[ ! -r $DATABASE ] && cat ${SQL}/create.sql | sqlite3 $DATABASE
# Marshal the county directories to import.
#
# If no directory was given on the command-line, read a list of county IDs from STDIN.
if [ x"$1" != x"" ]; then
cat
else
# Otherwise, find all of the IDs from the contents of the directory structure.
ls $SOURCE/tl_*_edges.zip | while read file; do
file=$(basename $file)
code=${file##tl_????_}
echo ${code%%_edges.zip}
done
fi | sort | while read code; do
echo "--- $code"
# Unpack the county files into the temp directory.
for file in $SHPS $DBFS; do
ZIP=$(ls $SOURCE/*_${code}_${file}.zip 2>/dev/null)
SHP=$(ls $SOURCE/*_${code}_${file}.* 2>/dev/null)
if [ x"$ZIP" != x"" ]; then
unzip -q $ZIP -d $TMP
elif [ x"$SHP" != x"" ]; then
ln -s $SHP $TMP
fi
done
# Generate an SQL stream to feed into the sqlite3 binary.
# Start by loading the helper libs and initializing the temporary tables
# that will hold the TIGER data before ETL.
(echo ".load $HELPER_LIB" && \
cat ${SQL}/setup.sql && \
for file in $SHPS; do
# Convert each Shapefile into SQL statements.
${SHP2SQLITE} -aS ${TMP}/*_${file}.shp tiger_${file}
done && \
for file in $DBFS; do
# Convert each DBF into SQL statements likewise.
shp2sqlite -an ${TMP}/*_${file}.dbf tiger_${file}
done && \
cat ${SQL}/convert.sql) | sqlite3 $DATABASE
# Finally, do the transform/load phase (convert.sql)
# and clean up the temporary files.
rm -f $TMP/*
done 2>&1 | tee import-$$.log
rm -rf $TMP
================================================
FILE: conf/geocoder-us/geocoder.ru
================================================
require 'sinatra'
disable :run, :reload
require 'geocoder/us/rest'
run Sinatra::Application
================================================
FILE: conf/geocoder-us/unicorn.rb
================================================
worker_processes 4
user "www-data", "www-data"
listen "/var/run/geocoder-us/unicorn.sock", :backlog => 64
pid "/var/run/geocoder-us/unicorn.pid"
stderr_path "/var/log/geocoder-us/geocoder-err.log"
stdout_path "/var/log/geocoder-us/geocoder-out.log"
# Have each process listen on a local port for debugging purposes.
after_fork do |server, worker|
addr = "127.0.0.1:#{40000 + worker.nr}"
server.listen(addr, :tries => 1, :delay => 5, :tcp_nopush => true)
end
================================================
FILE: conf/init/geocoder-us.conf
================================================
description "geocoder.us"
start on runlevel [2345]
stop on runlevel [!2345]
respawn
expect daemon
script
. /etc/default/geocoder-us
unicorn -c /etc/geocoder-us/unicorn.rb /etc/geocoder-us/geocoder.ru
end script
================================================
FILE: debian/README.Debian
================================================
geocoder-us for Debian
----------------------
The Geocoder::US package is a Ruby library that uses a database built from the
US Census Bureau's TIGER/Line data to interpolate a latitude/longitude
coordinate for a given US street address.
Binary shared objects
---------------------
The Geocoder::US module depends on being able to load a native extension module
in its SQLite driver. For this reason, a version of libsqlite-ruby >= 1.3.0 is
needed. The module is built and included in the .deb as `sqlite.so`, and it is
installed in the same directory as the Ruby modules. This may not be ideal, but
this makes it easy for the Geocoder::US library to find it there; otherwise, a
configuration option would be necessary.
REST API server
---------------
The library's API centers on a single method 'geocode' to the
Geocoder::US::Database class that takes an address string and returns a list of
dicts containing the most likely matches with coordinates.
The `geocode` method is wrapped in a very simple Sinatra application with a single
endpoint `/geocode` and a single argument `q`, which returns the result of the
geocode method in JSON format.
The Sinatra web framework does not support running as a daemon on its own, so
the Thin web server is used as a container for the application. This package
creates an `/etc/geocoder-us` directory containing two files:
`/etc/geocoder-us/geocoder.ru` is the "rackup" adapter between Thin and Sinatra
and should probably not be changed. This file doesn't have to live in /etc, but
I couldn't figure out where else to put it.
`/etc/geocoder-us/thin.yml` contains the configuration options to run the Thin
server. This file as packaged runs the REST server as the www-data user on port
8080. This file *probably* doesn't need to be changed, but if the server starts
doing weird things, different options to control Thin's behavior can be set
here.
The package creates `/var/log/geocoder-us` and `/var/run/geocoder-us`
directories for the Thin log file and PID file, respectively, and chowns them
to www-data.
An init script is also included in `/etc/init.d/geocoder-us`. It is heavily
hacked from the default Debian init.ex script to support the weirdnesses of
Thin, but it is LSB compliant and supports the `status` command.
Where to put the database
-------------------------
The location of the database file should be set in `/etc/default/geocoder-us`.
The package creates a `/var/lib/geocoder-us` directory and configures the
database location by default to be `/var/lib/geocoder-us/geocoder.db`. If you
have an EBS volume containing a file called `geocoder.db`, for example, you can
just mount the volume at `/var/lib/geocoder-us` and then start the server and
all will be well.
================================================
FILE: debian/changelog
================================================
geocoder-us (2.0.1pre-1sg66) lucid; urgency=low
[ Schuyler Erle ]
* Remove "la" from feature type affixes, because it probably is used
less frequently then "La" as part of a Spanish place name.
[ SimpleGeo Nerds ]
-- SimpleGeo Nerds <nerds@simplegeo.com> Sat, 18 Jun 2011 19:06:47 +0000
geocoder-us (2.0.1pre-1sg65) lucid; urgency=low
[ Schuyler Erle ]
* Fix a bug in edge interpolation caused by zero-length segments.
[ SimpleGeo Nerds ]
-- SimpleGeo Nerds <nerds@simplegeo.com> Thu, 16 Jun 2011 22:18:47 +0000
geocoder-us (2.0.1pre-1sg64) lucid; urgency=low
[ Paul Lathrop ]
* Fix upstart job.
[ SimpleGeo Nerds ]
-- SimpleGeo Nerds <nerds@simplegeo.com> Wed, 08 Jun 2011 23:57:47 +0000
geocoder-us (2.0.1pre-1sg63) lucid; urgency=low
[ Schuyler Erle ]
* Remove Timeout block from geocode endpoint for being generally
unsafe. The timeout can happen elsewhere (e.g. Unicorn or Gate).
[ SimpleGeo Nerds ]
-- SimpleGeo Nerds <nerds@simplegeo.com> Wed, 08 Jun 2011 22:45:46 +0000
geocoder-us (2.0.1pre-1sg62) lucid; urgency=low
[ Paul Lathrop ]
* Load the geocoder database location from the env first, then argv.
This is not optimal, because it makes sense to have the command-line
over-ride the environment, however when we run this under unicorn,
argv[0] is set to something that is *not* a database location.
* Remove cruft.
* Use unicorn to run the geocoder.
* Don't run as root.
* Reverting "Remove cruft." Resurrect these files I shouldn't have
deleted. This reverts commit
434f9d3aa40e71a20b5d2cf9b8dad1802282957a.
[ SimpleGeo Nerds ]
-- SimpleGeo Nerds <nerds@simplegeo.com> Wed, 08 Jun 2011 22:10:48 +0000
geocoder-us (2.0.1pre-1sg61) lucid; urgency=low
[ Schuyler Erle ]
* Add test for intersections.
[ SimpleGeo Nerds ]
-- SimpleGeo Nerds <nerds@simplegeo.com> Sun, 05 Jun 2011 01:56:46 +0000
geocoder-us (2.0.1pre-1sg60) lucid; urgency=low
[ Wade Simmons ]
* Revert "Open a new database connection for every request." This
reverts commit 5a3e3dd5ea745f15482cb16b28b2d09a6753710d.
[ Wade Simmons and Derek Smith ]
* don't swallow exceptions when creating/destroying temporary
databases
* ensure that all prepared statements are closed, and make sure the
temporary databases are cleaned up if there is an exception
* add a TODO to possibly remove the Timeout.timeout
[ SimpleGeo Nerds ]
-- SimpleGeo Nerds <nerds@simplegeo.com> Sun, 05 Jun 2011 01:35:46 +0000
geocoder-us (2.0.1pre-1sg59) lucid; urgency=low
[ Schuyler Erle ]
* Fix missing require timeout?
[ SimpleGeo Nerds ]
-- SimpleGeo Nerds <nerds@simplegeo.com> Fri, 03 Jun 2011 22:57:47 +0000
geocoder-us (2.0.1pre-1sg58) lucid; urgency=low
[ Schuyler Erle ]
* Open a new database connection for every request.
[ SimpleGeo Nerds ]
-- SimpleGeo Nerds <nerds@simplegeo.com> Fri, 03 Jun 2011 22:49:47 +0000
geocoder-us (2.0.1pre-1sg57) lucid; urgency=low
[ Schuyler Erle ]
* Replace thin configs (undoing commit
2bd5dd8e9ae5210cd19692a4dfc557129ef07f58)
[ SimpleGeo Nerds ]
-- SimpleGeo Nerds <nerds@simplegeo.com> Fri, 03 Jun 2011 22:40:48 +0000
geocoder-us (2.0.1pre-1sg56) lucid; urgency=low
[ Paul Lathrop ]
* Fix upstart script so it creates a pidfile.
[ SimpleGeo Nerds ]
-- SimpleGeo Nerds <nerds@simplegeo.com> Wed, 25 May 2011 21:40:46 +0000
geocoder-us (2.0.1pre-1sg55) lucid; urgency=low
[ Schuyler Erle ]
* Bug fix the previous commit.
[ SimpleGeo Nerds ]
-- SimpleGeo Nerds <nerds@simplegeo.com> Fri, 20 May 2011 00:32:48 +0000
geocoder-us (2.0.1pre-1sg54) lucid; urgency=low
[ Schuyler Erle ]
* Don't cache statements anymore.
[ SimpleGeo Nerds ]
-- SimpleGeo Nerds <nerds@simplegeo.com> Fri, 20 May 2011 00:18:47 +0000
geocoder-us (2.0.1pre-1sg53) lucid; urgency=low
[ Schuyler Erle ]
* Update metaphones in place table.
* All tests now pass.
[ SimpleGeo Nerds ]
-- SimpleGeo Nerds <nerds@simplegeo.com> Mon, 16 May 2011 20:59:37 +0000
geocoder-us (2.0.1pre-1sg52) lucid; urgency=low
[ Schuyler Erle ]
* Update tiger_import for TIGER/Line 2010.
* Remove obvious place duplicates.
* Update places for 2010 using TIGER/Line and Geonames.
* Final update to new place database.
[ SimpleGeo Nerds ]
-- SimpleGeo Nerds <nerds@simplegeo.com> Mon, 16 May 2011 19:34:35 +0000
geocoder-us (2.0.1pre-1sg51) lucid; urgency=low
[ Schuyler Erle ]
* Try moving the temporary intersection table into a separate in-
memory database to eliminate locking contention.
[ SimpleGeo Nerds ]
-- SimpleGeo Nerds <nerds@simplegeo.com> Thu, 12 May 2011 19:17:24 +0000
geocoder-us (2.0.1pre-1sg50) lucid; urgency=low
* UNRELEASED
-- SimpleGeo Nerds <nerds@simplegeo.com> Tue, 03 May 2011 00:09:08 +0000
geocoder-us (2.0.1pre-1sg49) lucid; urgency=low
* UNRELEASED
-- SimpleGeo Nerds <nerds@simplegeo.com> Fri, 15 Apr 2011 21:31:56 +0000
geocoder-us (2.0.1pre-1sg48) lucid; urgency=low
* UNRELEASED
-- SimpleGeo Nerds <nerds@simplegeo.com> Tue, 05 Apr 2011 23:21:46 +0000
geocoder-us (2.0.1pre-1sg47) lucid; urgency=low
[ Schuyler Erle ]
* Slightly better: Explicitly close statements on the temporary
intersection table to release locks. Also checked cached statements
to see if they've been closed. This (hopefully) eliminates the need
to flush all statements after the table alteration.
[ SimpleGeo Nerds ]
-- SimpleGeo Nerds <nerds@simplegeo.com> Tue, 05 Apr 2011 18:24:10 +0000
geocoder-us (2.0.1pre-1sg46) lucid; urgency=low
[ Schuyler Erle ]
* Oh, and trap SQLite3::LockedException, because maybe that'll
magically fix it.
[ SimpleGeo Nerds ]
-- SimpleGeo Nerds <nerds@simplegeo.com> Tue, 05 Apr 2011 18:06:09 +0000
geocoder-us (2.0.1pre-1sg45) lucid; urgency=low
[ Schuyler Erle ]
* Throw a few more flush_statements in, because the call to DROP TABLE
seems to be wedging on "database table is locked" though who alone
knows why. Stupid SQLite. Grr.
[ SimpleGeo Nerds ]
-- SimpleGeo Nerds <nerds@simplegeo.com> Tue, 05 Apr 2011 17:51:10 +0000
geocoder-us (2.0.1pre-1sg44) lucid; urgency=low
* UNRELEASED
-- SimpleGeo Nerds <nerds@simplegeo.com> Tue, 29 Mar 2011 23:53:42 +0000
geocoder-us (2.0.1pre-1sg43) lucid; urgency=low
[ Schuyler Erle ]
* Fix a weird race condition when geocoding intersections.
[ SimpleGeo Nerds ]
-- SimpleGeo Nerds <nerds@simplegeo.com> Thu, 17 Feb 2011 22:11:48 +0000
geocoder-us (2.0.1pre-1sg42) lucid; urgency=low
[ Schuyler Erle ]
* Break tests to demonstrate Brooklyn/Manhattan regression.
* Fix Brooklyn/Manhattan regression. Hooray, test-driven development!
* Fix the "Mountain View, CA" bug where "normalizing" a street type in
a city name would prevent the city from getting removed from the
street parts in Address.city=.
* Add more tests to cover recent bugs.
[ SimpleGeo Nerds ]
-- SimpleGeo Nerds <nerds@simplegeo.com> Wed, 09 Feb 2011 22:37:07 +0000
geocoder-us (2.0.1pre-1sg41) lucid; urgency=low
[ Schuyler Erle ]
* Remove "Brooklyn, NY" as a place for ZIP 14729. Because that's
plainly WRONG.
* Pretty print output from CLI demo.
* Make all tests pass, post street side offset.
[ SimpleGeo Nerds ]
-- SimpleGeo Nerds <nerds@simplegeo.com> Fri, 04 Feb 2011 00:34:10 +0000
geocoder-us (2.0.1pre-1sg39) lucid; urgency=low
[ Schuyler Erle ]
* Fix a NaN bug caused by some TIGER/Line from/to house numbers being
equal.
[ SimpleGeo Nerds ]
-- SimpleGeo Nerds <nerds@simplegeo.com> Wed, 12 Jan 2011 02:30:40 +0000
geocoder-us (2.0.1pre-1sg38) lucid; urgency=low
[ Schuyler Erle ]
* Don't run directly through thin; this causes segfaults on our Debian
systems.
* Remove thin configs.
* Go back to using embedded metaphone function, which will improve
result quality.
* Move the database open back out to the init step.
[ SimpleGeo Nerds ]
-- SimpleGeo Nerds <nerds@simplegeo.com> Wed, 12 Jan 2011 02:00:40 +0000
geocoder-us (2.0.1pre-1sg37) lucid; urgency=low
* UNRELEASED
-- SimpleGeo Nerds <nerds@simplegeo.com> Tue, 04 Jan 2011 19:24:45 +0000
geocoder-us (2.0.1pre-1sg36) lucid; urgency=low
* UNRELEASED
-- SimpleGeo Nerds <nerds@simplegeo.com> Thu, 23 Dec 2010 17:46:19 +0000
geocoder-us (2.0.1pre-1sg35) lucid; urgency=low
[ dsmith ]
* Added a health endpoint to the geocoder
[ SimpleGeo Nerds ]
-- SimpleGeo Nerds <nerds@simplegeo.com> Mon, 20 Dec 2010 19:08:59 +0000
geocoder-us (2.0.1pre-1sg34) lucid; urgency=low
* UNRELEASED
-- SimpleGeo Nerds <nerds@simplegeo.com> Mon, 20 Dec 2010 17:39:41 +0000
geocoder-us (2.0.1pre-1sg33) lucid; urgency=low
[ Schuyler Erle ]
* Add street-side offsets of 7.5m by default.
[ SimpleGeo Nerds ]
-- SimpleGeo Nerds <nerds@simplegeo.com> Mon, 29 Nov 2010 23:12:51 +0000
geocoder-us (2.0.1pre-1sg32) lucid; urgency=low
* UNRELEASED
-- SimpleGeo Nerds <nerds@simplegeo.com> Wed, 24 Nov 2010 22:20:44 +0000
geocoder-us (2.0.1pre-1sg31) lucid; urgency=low
* UNRELEASED
-- SimpleGeo Nerds <nerds@simplegeo.com> Mon, 15 Nov 2010 19:35:15 +0000
geocoder-us (2.0.1pre-1sg29) lucid; urgency=low
* UNRELEASED
-- SimpleGeo Nerds <nerds@simplegeo.com> Tue, 09 Nov 2010 21:28:24 +0000
geocoder-us (2.0.1pre-1sg29) lucid; urgency=low
[ Schuyler Erle ]
* Refactor rest.rb slightly for speed?
[ SimpleGeo Nerds ]
-- SimpleGeo Nerds <nerds@simplegeo.com> Thu, 04 Nov 2010 21:54:45 +0000
geocoder-us (2.0.1pre-1sg28) lucid; urgency=low
[ Schuyler Erle ]
* All-singing, all-dancing REST server.
[ SimpleGeo Nerds ]
-- SimpleGeo Nerds <nerds@simplegeo.com> Thu, 04 Nov 2010 19:18:25 +0000
geocoder-us (2.0.1pre-1sg27) lucid; urgency=low
[ Schuyler Erle ]
* Work around for segfaults in the Metaphone C code.
* Add Upstart config.
* Remove default and init script, start/stop in postinst/prerm.
* Fix up prerm and postinst scripts.
* Don't daemonize when running with Upstart.
* Tweak respawn rate.
* Load a new database handle on every request. Hrrr.
[ SimpleGeo Nerds ]
-- SimpleGeo Nerds <nerds@simplegeo.com> Thu, 04 Nov 2010 18:50:00 +0000
geocoder-us (2.0.1pre-1sg21) lucid; urgency=low
* UNRELEASED
-- SimpleGeo Nerds <nerds@simplegeo.com> Fri, 01 Oct 2010 22:27:07 +0000
geocoder-us (2.0.1pre-1sg21) lucid; urgency=low
* UNRELEASED
-- SimpleGeo Nerds <nerds@simplegeo.com> Fri, 01 Oct 2010 22:21:08 +0000
geocoder-us (2.0.1pre-1sg21) lucid; urgency=low
* UNRELEASED
-- SimpleGeo Nerds <nerds@simplegeo.com> Fri, 01 Oct 2010 22:12:09 +0000
geocoder-us (2.0.1pre-1sg21) lucid; urgency=low
* UNRELEASED
-- SimpleGeo Nerds <nerds@simplegeo.com> Fri, 01 Oct 2010 21:52:07 +0000
geocoder-us (2.0.1pre-1sg21) lucid; urgency=low
* UNRELEASED
-- SimpleGeo Nerds <nerds@simplegeo.com> Fri, 01 Oct 2010 21:18:23 +0000
geocoder-us (2.0.1pre-1sg20) lucid; urgency=low
* UNRELEASED
-- SimpleGeo Nerds <nerds@simplegeo.com> Fri, 01 Oct 2010 21:07:24 +0000
geocoder-us (2.0.1pre-1sg19) lucid; urgency=low
* UNRELEASED
-- SimpleGeo Nerds <nerds@simplegeo.com> Fri, 01 Oct 2010 20:57:25 +0000
geocoder-us (2.0.1pre-1sg18) lucid; urgency=low
* UNRELEASED
-- SimpleGeo Nerds <nerds@simplegeo.com> Fri, 01 Oct 2010 20:45:23 +0000
geocoder-us (2.0.1pre-1sg17) lucid; urgency=low
* UNRELEASED
-- SimpleGeo Nerds <nerds@simplegeo.com> Fri, 01 Oct 2010 20:35:26 +0000
geocoder-us (2.0.1pre-1sg16) lucid; urgency=low
* UNRELEASED
-- SimpleGeo Nerds <nerds@simplegeo.com> Fri, 01 Oct 2010 20:24:24 +0000
geocoder-us (2.0.1pre-1sg15) lucid; urgency=low
* UNRELEASED
-- SimpleGeo Nerds <nerds@simplegeo.com> Fri, 01 Oct 2010 20:12:32 +0000
geocoder-us (2.0.1pre-1sg14) lucid; urgency=low
* UNRELEASED
-- SimpleGeo Nerds <nerds@simplegeo.com> Thu, 30 Sep 2010 22:18:36 +0000
geocoder-us (2.0.1pre-1sg13) lucid; urgency=low
* UNRELEASED
-- SimpleGeo Nerds <nerds@simplegeo.com> Thu, 30 Sep 2010 22:07:45 +0000
geocoder-us (2.0.1pre-1sg12) lucid; urgency=low
* UNRELEASED
-- SimpleGeo Nerds <nerds@simplegeo.com> Thu, 30 Sep 2010 21:56:56 +0000
geocoder-us (2.0.1pre-1sg11) lucid; urgency=low
* UNRELEASED
-- SimpleGeo Nerds <nerds@simplegeo.com> Thu, 30 Sep 2010 21:30:13 +0000
geocoder-us (2.0.1pre-1sg10) lucid; urgency=low
* UNRELEASED
-- SimpleGeo Nerds <nerds@simplegeo.com> Thu, 30 Sep 2010 20:58:14 +0000
geocoder-us (2.0.1pre-1sg9) lucid; urgency=low
* UNRELEASED
-- SimpleGeo Nerds <nerds@simplegeo.com> Thu, 30 Sep 2010 19:16:45 +0000
geocoder-us (2.0.1pre-1sg3) lucid; urgency=low
* UNRELEASED
-- SimpleGeo Nerds <nerds@simplegeo.com> Thu, 30 Sep 2010 18:49:24 +0000
geocoder-us (2.0.1pre-1sg2) lucid; urgency=low
[ Ian Eure ]
* Build the SQLite extension.
[ SimpleGeo Nerds ]
-- SimpleGeo Nerds <nerds@simplegeo.com> Tue, 24 Aug 2010 20:39:11 +0000
geocoder-us (2.0.1pre-1sg1) lucid; urgency=low
[ Ian Eure ]
* Add source format.
* Build-depend on ruby1.8, build with CDBS, update standards-version.
* Strip +x bit from stuff that doesn't need it.
* Update copyright.
* Update mainteiners, fix section, add .
* Fix installation of stuff in /var.
[ SimpleGeo Nerds ]
-- SimpleGeo Nerds <nerds@simplegeo.com> Tue, 24 Aug 2010 02:33:44 +0000
geocoder-us (2.0.1pre-1sg0) unstable; urgency=low
* Initial release
-- Schuyler Erle <schuyler@simplegeo.com> Sat, 07 Aug 2010 00:51:40 +0000
================================================
FILE: debian/compat
================================================
7
================================================
FILE: debian/control
================================================
Source: geocoder-us
Section: ruby
Priority: extra
Maintainer: SimpleGeo Nerds <nerds@simplegeo.com>
Uploaders: Schuyler Erle <schuyler@simplegeo.com>
Build-Depends: debhelper (>= 7), libsqlite3-dev, ruby1.8, cdbs, ruby-pkg-tools
Standards-Version: 3.8.4
Homepage: http://github.com/simplegeo/geocoder/
Package: geocoder-us
Architecture: any
Depends: ${misc:Depends}, ${shlibs:Depends}, ruby1.8, libsqlite3-ruby (>= 1.3.0), libsinatra-ruby, libjson-ruby, unicorn
Description: A US address geocoder.
A US address geocoder. Requires a suitable database.
================================================
FILE: debian/copyright
================================================
This work was packaged for Debian by:
Schuyler Erle <schuyler@simplegeo.com> on Sat, 07 Aug 2010 00:51:40 +0000
It was downloaded from http://github.com/simplegeo/geocoder/
Upstream Author(s):
Schuyler Erle <schuyler@simplegeo.com>
Copyright:
(c) 2009 FortiusOne, Inc.
License:
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This package is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
The Debian packaging is:
Copyright (C) 2010 SimpleGeo, Inc.
and is licensed under the GPL version 3, see `/usr/share/common-licenses/GPL-3'.
================================================
FILE: debian/default
================================================
# Defaults for geocoder-us upstart job
# sourced by /etc/init/geocoder-us.conf
# installed at /etc/default/geocoder-us by maintainer scripts
# Set the location of the geocoder database.
export GEOCODER_DB="/var/lib/geocoder-us/geocoder.db"
================================================
FILE: debian/docs
================================================
History.txt
Manifest.txt
README.rdoc
TODO.txt
TODO.txt
================================================
FILE: debian/geocoder-us.postinst
================================================
#!/bin/sh
# postinst script for #PACKAGE#
#
# see: dh_installdeb(1)
set -e
# summary of how this script can be called:
# * <postinst> `configure' <most-recently-configured-version>
# * <old-postinst> `abort-upgrade' <new version>
# * <conflictor's-postinst> `abort-remove' `in-favour' <package>
# <new-version>
# * <postinst> `abort-remove'
# * <deconfigured's-postinst> `abort-deconfigure' `in-favour'
# <failed-install-package> <version> `removing'
# <conflicting-package> <version>
# for details, see http://www.debian.org/doc/debian-policy/ or
# the debian-policy package
case "$1" in
configure)
# just make sure that /usr/bin/thin can write its PID file and logs
chown www-data /var/run/geocoder-us
chown www-data /var/log/geocoder-us
start geocoder-us || /bin/true
;;
abort-upgrade|abort-remove|abort-deconfigure)
;;
*)
echo "postinst called with unknown argument \`$1'" >&2
exit 1
;;
esac
# dh_installdeb will replace this with shell code automatically
# generated by other debhelper scripts.
#DEBHELPER#
exit 0
================================================
FILE: debian/geocoder-us.prerm
================================================
#!/bin/sh
set -e
case "$1" in
remove|deconfigure)
stop geocoder-us || true
;;
upgrade)
;;
failed-upgrade)
;;
*)
echo "prerm called with unknown argument \`$1'" >&2
exit 0
;;
esac
# dh_installdeb will replace this with shell code automatically
# generated by other debhelper scripts.
exit 0
================================================
FILE: debian/rules
================================================
#!/usr/bin/make -f
include /usr/share/cdbs/1/rules/debhelper.mk
include /usr/share/ruby-pkg-tools/1/class/ruby-setup-rb.mk
# Add here any variable or target overrides you need.
build/geocoder-us::
make -C $(CURDIR)/src/libsqlite3_geocoder
install -m 0644 $(CURDIR)/src/libsqlite3_geocoder/*.so \
$(CURDIR)/lib/geocoder/us/sqlite3.so
install/geocoder-us::
install -d -m 0755 $(DEB_DESTDIR)var/lib/geocoder-us \
$(DEB_DESTDIR)var/run/geocoder-us \
$(DEB_DESTDIR)var/log/geocoder-us
================================================
FILE: debian/source/format
================================================
3.0 (quilt)
================================================
FILE: demos/api/server.rb
================================================
require 'rubygems'
require 'sinatra'
require 'geocoder/us/database'
require 'json'
set :port, 8080
@@db = Geocoder::US::Database.new("/home/sderle/geocoder/california.db")
get '/geocode.json' do
if params[:q]
(@@db.geocode params[:q]).to_json
else
status 400
"parameter 'q' is missing"
end
end
get '/' do
unless params[:q].nil?
@records = @@db.geocode params[:q]
end
erb :index
end
================================================
FILE: demos/api/views/index.erb
================================================
<!DOCTYPE html
PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">
<head>
<style type="text/css">
html {font-family: Arial, sans-serif;}
table {font-size: 8pt; border-collapse: collapse;}
td { border: 1px solid black; padding: .25em .5em .25em .5em; }
</style>
</head>
<body onload="">
<p><b>Geocoder Demo</b></p>
<p>
<form method="GET">
<label>Enter an address:</label> <input type="text" name="q" value="<%= params[:q] %>" size="80">
<input type="hidden" name="f" value="html" />
<input type="submit" value="Geocode" />
</form>
</p>
<% unless @records.nil? %>
<table>
<tr>
<td>Match</td>
<td>Lat</td>
<td>Lon</td>
<td>#</td>
<td>Qual</td>
<td>Dir</td>
<td>Type</td>
<td>Street</td>
<td>Type</td>
<td>Dir</td>
<td>Qual</td>
<td>City</td>
<td>St</td>
<td>ZIP</td>
<td> </td>
</tr>
<% for record in @records %>
<tr>
<td><%= format("%.2f", record[:score]*100) %>%</td>
<td><%= record[:lat].to_s %></td>
<td><%= record[:lon].to_s %></td>
<td><%= record[:prefix] if record[:prefix] %><%= record[:number] %></td>
<td><%= record[:pretyp] %></td>
<td><%= record[:predir] %></td>
<td><%= record[:prequal] %></td>
<td><%= record[:street] %></td>
<td><%= record[:suftyp] %></td>
<td><%= record[:sufdir] %></td>
<td><%= record[:sufqual] %></td>
<td><%= record[:city] %></td>
<td><%= record[:state] %></td>
<td><%= record[:zip] %></td>
<td><a href="http://maps.google.com/maps?q=<%=record[:lat]%>,<%=record[:lon]%>"
target="_blank">map</a></td>
</tr>
<% end %>
</table>
<% end %>
</body>
</html>
================================================
FILE: demos/cli.rb
================================================
require 'geocoder/us/database'
require 'pp'
db = Geocoder::US::Database.new("/mnt/tiger2010/geocoder.db", :debug=>true)
result = db.geocode(ARGV[0])
pp(result)
print "#{result[0][:lat]} N, #{-result[0][:lon]} W\n"
================================================
FILE: demos/demo/app/ext/geocodewrap.rb
================================================
require 'rubygems'
require 'geocoder/us/database'
require 'logger'
module Sinatra
module GeocodeWrap
attr_accessor :db
def self.registered(app)
options = {:cache_size => 100000}
@@db = Geocoder::US::Database.new("/Users/katechapman/usgeocode.db", options)
stats = Logger.new("geocoderstats.log", 10, 1024000)
app.get '/' do
unless params[:address].nil?
begin
@records = @@db.geocode params[:address]
stats.debug "Geocoded: 1, Failed: 0, Geocoded At: " << DateTime.now.to_s
rescue Exception => e
stats.debug "Geocoded: 1, Failed: 1, Geocoded At: " << DateTime.now.to_s
puts e.message
end
end
case params[:format]
when /xml/
builder :index
when /atom/
builder :atom
when /json/
@records.to_json
else
erb :index
end
end
app.post '/batch' do
failed_codes = 0
total_codes = 0
puts Time.now
if params[:uploaded_csv].nil?
csv_file = request.env["rack.input"].read
csv = FasterCSV.parse(csv_file, :row_sep => "*", :col_sep => "|")
else
FileUtils.mkdir_p('uploads/')
FileUtils.mv(params[:uploaded_csv][:tempfile].path, "uploads/#{params[:uploaded_csv][:filename]}")
csv_file = open("uploads/#{params[:uploaded_csv][:filename]}")
@filename = params[:uploaded_csv][:filename].gsub(/\.csv/,"")
csv = FasterCSV.parse(csv_file)
end
headers = csv[0]
@records = csv.collect do |record|
total_codes += 1
next if record == headers
begin
result = @@db.geocode record[1]
if result.empty?
result[0] = {:lon => nil, :lat => nil, :precision => 'unmatched', :score => 0}
failed_codes += 1
end
result.first.merge(headers[0] => record[0])
rescue Exception => e
failed_codes += 1
puts e.message
next
end
end.compact
puts Time.now
stats.debug "Geocoded: " << total_codes.to_s << ", Failed: " << failed_codes.to_s << ",Geocoded At: " << DateTime.now.to_s
case params[:format]
when /xml/
builder :index
when /atom/
builder :atom
when /json/
@records.to_json
else
erb :index
end
end
end
end
register GeocodeWrap
end
================================================
FILE: demos/demo/app/views/index.builder
================================================
xml.locations do
unless @records.nil?
@records.each do |record|
xml.location do
xml.score format("%.2f", record[:score]*100)
%w{lat lon number prefix pretyp predir prequal street suftyp sufdir sufqual city state zip}.each do |field|
xml.tag! field, record[field.to_sym]
end
end
end
end
end
================================================
FILE: demos/demo/app/views/index.erb
================================================
<!DOCTYPE html
PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">
<head>
<style type="text/css">
html {font-family: Arial, sans-serif;}
table {font-size: 8pt; border-collapse: collapse;}
td { border: 1px solid black; padding: .25em .5em .25em .5em; }
</style>
</head>
<body onload="">
<p><b>Geocoder Demo</b></p>
<p>
<form>
<label>Enter an address:</label> <input type="text" name="address" value="<%= params[:address] %>" size="80">
<input type="submit" value="Geocode" />
</form>
<form action="batch" method="POST" enctype="multipart/form-data" accept-charset="utf-8">
<label>Upload a CSV:</label> <input type="file" name="uploaded_csv" id="uploaded_csv">
<input type="submit" value="Batch Geocode" />
</form>
</p>
<% unless @records.nil? %>
<table>
<tr>
<td>Match</td>
<td>Precision</td>
<td>Lat</td>
<td>Lon</td>
<td>#</td>
<td>Qual</td>
<td>Dir</td>
<td>Type</td>
<td>Street</td>
<td>Type</td>
<td>Dir</td>
<td>Qual</td>
<td>City</td>
<td>St</td>
<td>ZIP</td>
<td> </td>
</tr>
<% for record in @records %>
<tr>
<td><%= format("%.2f", record[:score]*100) %>%</td>
<td><%= record[:precision].to_s %></td>
<td><%= record[:lat].to_s %></td>
<td><%= record[:lon].to_s %></td>
<td><%= record[:prefix] if record[:prefix] %><%= record[:number] %></td>
<td><%= record[:pretyp] %></td>
<td><%= record[:predir] %></td>
<td><%= record[:prequal] %></td>
<td><%= record[:street] %></td>
<td><%= record[:suftyp] %></td>
<td><%= record[:sufdir] %></td>
<td><%= record[:sufqual] %></td>
<td><%= record[:city] %></td>
<td><%= record[:state] %></td>
<td><%= record[:zip] %></td>
<td><a href="http://maps.google.com/maps?q=<%=record[:lat]%>,<%=record[:lon]%>"
target="_blank">map</a></td>
</tr>
<% end %>
</table>
<% end %>
<% unless @filename.nil? %>
<a href="/link.atom?filename=<%= @filename %>">Atom Feed</a>
<% end %>
</body>
</html>
================================================
FILE: demos/demo/config/bootstraps.rb
================================================
require 'rubygems'
module BootStraps
class Framework
def initialize
@methods = {}
end
def apply_settings!(app)
@methods.each_pair do |method, calls|
calls.each do |arg_set|
app.send(method, *arg_set)
end
end
end
def method_missing(method, *args)
@methods[method] ||= []
@methods[method] << args
end
end
class DataStore
def connect_action(&block)
@connect_action = block
end
#TODO raise UndefinedConnectAction
def connect
@connect_action.call if @connect_action
end
end
class Configuration
attr_accessor :db, :global, :default_env, :vendor_dir, :lib_paths, :framework, :vendored
attr_reader :gems
def initialize
@framework = Framework.new
@gems = {}
@global = {}
@default_env = 'production'
@vendor_dir = File.join(root, 'vendor')
@lib_paths = []
@vendored = false
end
def env
ENV['RACK_ENV'] ||= default_env
end
def env=(val)
ENV['RACK_ENV'] = val
end
def root
File.join(File.expand_path(File.dirname(__FILE__)), "..")
end
def gem(*args)
gem = args.first
ver = args.last
@gems[gem] = ver
#its concievable that vendored could be changed mid config
use_vendor if vendored
Kernel.send(:gem, *args)
require gem
end
private
def use_vendor
Gem.clear_paths
prepend_gem_path!(File.join(root, 'vendor'))
end
def prepend_gem_path!(path)
ENV['GEM_PATH'] = path
end
end
class Initializer
@@config = Configuration.new
class << self
def configure
unless @@config.frozen?
yield @@config
@@config.freeze
end
end
def config
@@config
end
def boot!
require File.join(@@config.root, 'config', 'geoenvironment.rb')
require_libs
end
private
def require_libs
[
subdir_expansion('lib'),
subdir_expansion(File.join('app','ext'))
].each do |p|
require_all(p)
end
end
def require_all(path)
Dir[path].each { |f| require f }
end
def subdir_expansion(subdir)
File.join(@@config.root, subdir, '**', '*.rb')
end
end
end
end
BootStraps::Initializer.boot!
Straps = BootStraps::Initializer.config
================================================
FILE: demos/demo/config/geoenvironment.rb
================================================
BootStraps::Initializer.configure do |config|
#Use the vendor directory
config.vendored = true
config.default_env = 'production'
config.gem 'sinatra'
config.gem 'fastercsv'
config.gem 'json'
config.framework.set :root, config.root
config.framework.set :environment, config.env
config.framework.set :raise_errors, true
config.framework.set :views, File.join('app','views')
config.framework.set :server, 'mongrel'
config.framework.set :static, true
config.framework.set :logging, true
config.framework.set :port, 4567
config.framework.set :lock, false
end
================================================
FILE: demos/demo/config.ru
================================================
require 'rubygems'
require 'sinatra'
Sinatra::Application.default_options.merge!(
:run => false,
:env => ENV['RACK_ENV']
)
require 'geocom_geocode'
run GeocomGeocode::GeocodeServer
================================================
FILE: demos/demo/geocoder_helper.rb
================================================
require 'rubygems'
require 'geocoder/us/database'
require 'fastercsv'
require 'json'
def initialize
end
================================================
FILE: demos/demo/geocom_geocode.rb
================================================
require 'config/bootstraps'
module GeocomGeocode
class GeocodeServer < Sinatra::Base
register Sinatra::GeocodeWrap
configure do
Straps.framework.apply_settings!(self)
end
end
end
================================================
FILE: demos/demo/main.rb
================================================
require 'geocom_geocode'
GeocomGeocode::GeocodeServer.run!
================================================
FILE: demos/demo/rakefile.rb
================================================
require 'rake'
task :boot_env do
require 'config/bootstraps';
end
namespace :db do
task :migrate => :connect do
ActiveRecord::Base.logger = Logger.new(STDOUT)
ActiveRecord::Migration.verbose = true
ActiveRecord::Migrator.migrate('db/migrate/', nil)
end
task :connect => :boot_env do
BootStraps::Initializer.config.db.connect
end
end
================================================
FILE: demos/demo/tmp/restart.txt
================================================
================================================
FILE: demos/parse.rb
================================================
require 'geocoder/us/address'
require 'pp'
pp(Geocoder::US::Address.new(ARGV[0]))
================================================
FILE: demos/simpledemo/views/index.builder
================================================
xml.locations do
unless @records.nil?
@records.each do |record|
xml.location do
xml.score format("%.2f", record[:score]*100)
%w{lat lon number prefix pretyp predir prequal street suftyp sufdir sufqual city state zip}.each do |field|
xml.tag! field, record[field.to_sym]
end
end
end
end
end
================================================
FILE: demos/simpledemo/views/index.erb
================================================
<!DOCTYPE html
PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">
<head>
<style type="text/css">
html {font-family: Arial, sans-serif;}
table {font-size: 8pt; border-collapse: collapse;}
td { border: 1px solid black; padding: .25em .5em .25em .5em; }
</style>
</head>
<body onload="">
<p><b>Geocoder Demo</b></p>
<p>
<form>
<label>Enter an address:</label> <input type="text" name="address" value="<%= params[:address] %>" size="80">
<input type="submit" value="Geocode" />
</form>
<form action="batch" method="POST" enctype="multipart/form-data" accept-charset="utf-8">
<label>Upload a CSV:</label> <input type="file" name="uploaded_csv" id="uploaded_csv">
<input type="submit" value="Batch Geocode" />
</form>
</p>
<% unless @records.nil? %>
<table>
<tr>
<td>Match</td>
<td>Lat</td>
<td>Lon</td>
<td>#</td>
<td>Qual</td>
<td>Dir</td>
<td>Type</td>
<td>Street</td>
<td>Type</td>
<td>Dir</td>
<td>Qual</td>
<td>City</td>
<td>St</td>
<td>ZIP</td>
<td> </td>
</tr>
<% for record in @records %>
<tr>
<td><%= format("%.2f", record[:score]*100) %>%</td>
<td><%= record[:lat].to_s %></td>
<td><%= record[:lon].to_s %></td>
<td><%= record[:prefix] if record[:prefix] %><%= record[:number] %></td>
<td><%= record[:pretyp] %></td>
<td><%= record[:predir] %></td>
<td><%= record[:prequal] %></td>
<td><%= record[:street] %></td>
<td><%= record[:suftyp] %></td>
<td><%= record[:sufdir] %></td>
<td><%= record[:sufqual] %></td>
<td><%= record[:city] %></td>
<td><%= record[:state] %></td>
<td><%= record[:zip] %></td>
<td><a href="http://maps.google.com/maps?q=<%=record[:lat]%>,<%=record[:lon]%>"
target="_blank">map</a></td>
</tr>
<% end %>
</table>
<% end %>
<% unless @filename.nil? %>
<a href="/link.atom?filename=<%= @filename %>">Atom Feed</a>
<% end %>
</body>
</html>
================================================
FILE: demos/simpledemo/ws.rb
================================================
require 'rubygems'
require 'sinatra'
require 'geocoder/us/database'
require 'fastercsv'
require 'json'
set :port, 8080
@@db = Geocoder::US::Database.new("/fortiusone/geocoder/geocoder.db")
get '/' do
unless params[:address].nil?
@records = @@db.geocode params[:address]
end
case params[:format]
when /xml/
builder :index
when /atom/
builder :atom
else
erb :index
end
end
require 'open-uri'
get '/link.:format' do
if(params.include?(:url))
csv_file = params[:url]
else
csv_file = "uploads/#{params[:filename]}.csv"
end
csv = FasterCSV.parse(open(csv_file))
headers = csv[0]
@records = csv.collect do |record|
next if record == headers
begin
(@@db.geocode record[1]).first
rescue Exception => e
puts e.message
next
end
end.compact
case params[:format]
when /atom/
builder :atom
when /xml/
builder :index
else
erb :index
end
end
post '/batch' do
csv_file = request.env["rack.input"].read
csv = FasterCSV.parse(csv_file, :row_sep => "*", :col_sep => "|")
headers = csv[0]
@records = csv.collect do |record|
next if record == headers
begin
(@@db.geocode record[1]).first.merge(headers[0] => record[0])
rescue Exception => e
puts e.message
next
end
end.compact
case params[:format]
when /xml/
builder :index
when /atom/
builder :atom
when /json/
@records.to_json
else
erb :index
end
end
================================================
FILE: doc/Makefile
================================================
all: lookup.html parsing.html
%.html: %.rst voidspace.css
rst2html --stylesheet-path=voidspace.css --no-compact-lists $< > $@
clean:
rm -f *.html
================================================
FILE: doc/html4css1.css
================================================
/*
:Author: David Goodger
:Contact: goodger@users.sourceforge.net
:Date: $Date: 2005-12-18 01:56:14 +0100 (Sun, 18 Dec 2005) $
:Revision: $Revision: 4224 $
:Copyright: This stylesheet has been placed in the public domain.
Default cascading style sheet for the HTML output of Docutils.
See http://docutils.sf.net/docs/howto/html-stylesheets.html for how to
customize this style sheet.
*/
/* used to remove borders from tables and images */
.borderless, table.borderless td, table.borderless th {
border: 0 }
table.borderless td, table.borderless th {
/* Override padding for "table.docutils td" with "! important".
The right padding separates the table cells. */
padding: 0 0.5em 0 0 ! important }
.first {
/* Override more specific margin styles with "! important". */
margin-top: 0 ! important }
.last, .with-subtitle {
margin-bottom: 0 ! important }
.hidden {
display: none }
a.toc-backref {
text-decoration: none ;
color: black }
blockquote.epigraph {
margin: 2em 5em ; }
dl.docutils dd {
margin-bottom: 0.5em }
/* Uncomment (and remove this text!) to get bold-faced definition list terms
dl.docutils dt {
font-weight: bold }
*/
div.abstract {
margin: 2em 5em }
div.abstract p.topic-title {
font-weight: bold ;
text-align: center }
div.admonition, div.attention, div.caution, div.danger, div.error,
div.hint, div.important, div.note, div.tip, div.warning {
margin: 2em ;
border: medium outset ;
padding: 1em }
div.admonition p.admonition-title, div.hint p.admonition-title,
div.important p.admonition-title, div.note p.admonition-title,
div.tip p.admonition-title {
font-weight: bold ;
font-family: sans-serif }
div.attention p.admonition-title, div.caution p.admonition-title,
div.danger p.admonition-title, div.error p.admonition-title,
div.warning p.admonition-title {
color: red ;
font-weight: bold ;
font-family: sans-serif }
/* Uncomment (and remove this text!) to get reduced vertical space in
compound paragraphs.
div.compound .compound-first, div.compound .compound-middle {
margin-bottom: 0.5em }
div.compound .compound-last, div.compound .compound-middle {
margin-top: 0.5em }
*/
div.dedication {
margin: 2em 5em ;
text-align: center ;
font-style: italic }
div.dedication p.topic-title {
font-weight: bold ;
font-style: normal }
div.figure {
margin-left: 2em ;
margin-right: 2em }
div.footer, div.header {
clear: both;
font-size: smaller }
div.line-block {
display: block ;
margin-top: 1em ;
margin-bottom: 1em }
div.line-block div.line-block {
margin-top: 0 ;
margin-bottom: 0 ;
margin-left: 1.5em }
div.sidebar {
margin-left: 1em ;
border: medium outset ;
padding: 1em ;
background-color: #ffffee ;
width: 40% ;
float: right ;
clear: right }
div.sidebar p.rubric {
font-family: sans-serif ;
font-size: medium }
div.system-messages {
margin: 5em }
div.system-messages h1 {
color: red }
div.system-message {
border: medium outset ;
padding: 1em }
div.system-message p.system-message-title {
color: red ;
font-weight: bold }
div.topic {
margin: 2em }
h1.section-subtitle, h2.section-subtitle, h3.section-subtitle,
h4.section-subtitle, h5.section-subtitle, h6.section-subtitle {
margin-top: 0.4em }
h1.title {
text-align: center }
h2.subtitle {
text-align: center }
hr.docutils {
width: 75% }
img.align-left {
clear: left }
img.align-right {
clear: right }
ol.simple, ul.simple {
margin-bottom: 1em }
ol.arabic {
list-style: decimal }
ol.loweralpha {
list-style: lower-alpha }
ol.upperalpha {
list-style: upper-alpha }
ol.lowerroman {
list-style: lower-roman }
ol.upperroman {
list-style: upper-roman }
p.attribution {
text-align: right ;
margin-left: 50% }
p.caption {
font-style: italic }
p.credits {
font-style: italic ;
font-size: smaller }
p.label {
white-space: nowrap }
p.rubric {
font-weight: bold ;
font-size: larger ;
color: maroon ;
text-align: center }
p.sidebar-title {
font-family: sans-serif ;
font-weight: bold ;
font-size: larger }
p.sidebar-subtitle {
font-family: sans-serif ;
font-weight: bold }
p.topic-title {
font-weight: bold }
pre.address {
margin-bottom: 0 ;
margin-top: 0 ;
font-family: serif ;
font-size: 100% }
pre.literal-block, pre.doctest-block {
margin-left: 2em ;
margin-right: 2em ;
background-color: #eeeeee }
span.classifier {
font-family: sans-serif ;
font-style: oblique }
span.classifier-delimiter {
font-family: sans-serif ;
font-weight: bold }
span.interpreted {
font-family: sans-serif }
span.option {
white-space: nowrap }
span.pre {
white-space: pre }
span.problematic {
color: red }
span.section-subtitle {
/* font-size relative to parent (h1..h6 element) */
font-size: 80% }
table.citation {
border-left: solid 1px gray;
margin-left: 1px }
table.docinfo {
margin: 2em 4em }
table.docutils {
margin-top: 0.5em ;
margin-bottom: 0.5em }
table.footnote {
border-left: solid 1px black;
margin-left: 1px }
table.docutils td, table.docutils th,
table.docinfo td, table.docinfo th {
padding-left: 0.5em ;
padding-right: 0.5em ;
vertical-align: top }
table.docutils th.field-name, table.docinfo th.docinfo-name {
font-weight: bold ;
text-align: left ;
white-space: nowrap ;
padding-left: 0 }
h1 tt.docutils, h2 tt.docutils, h3 tt.docutils,
h4 tt.docutils, h5 tt.docutils, h6 tt.docutils {
font-size: 100% }
tt.docutils {
background-color: #eeeeee }
ul.auto-toc {
list-style-type: none }
================================================
FILE: doc/lookup.rst
================================================
.. _lookup:
===================================
Geocoder.us Address Lookup Strategy
===================================
:Author: Schuyler Erle
:Contact: schuyler at geocoder dot us
:Created: 2009/03/13
:Edited: 2009/03/14
Definitions
-----------
Edge
Database representation of a street segment, consisting of a linestring
geometry and an edge ID. Edges relate to many ranges and many features
through its ID.
Feature
Database representation of a named street, consisting of street name
and modifier elements, a reference ZIP code, and a primary/alternate flag.
Range
Database representation of a range of address numbers on a given
street, consisting of range start and end numbers, an optional prefix
ending with a non-numeric character, and a delivery ZIP code for that
range.
Place
Database representation of a ZIP code, consisting of a city name,
state abbreviation, a ZIP code, and a primary/alternate flag.
Address record
A set consisting of exactly one edge, one feature, and one range, related
through the edge ID.
Address query
An ordered set of {Number Prefix, Number, Directional Prefix, Type Prefix,
Qualifier Prefix, Street Name, Qualifier Suffix, Type Suffix, Directional
Suffix, City, State, ZIP}. All of the elements are optional except Number and
Street Name. Either ZIP or City must also be present. The State element
and all of the prefix and suffix elements are assumed to be normalized to
standard postal abbreviations.
Address string
A string including some or all of the elements of an address.
Address Lookup Strategy
-----------------------
1. Given a an address query, initialize an empty set of candidate places,
and an empty set of candidate address records.
#. If a ZIP was given, look up `the place from the ZIP`_, and add the
place, if any, to the candidate place set.
#. If a city was given, look up all `the places matching the metaphone hash
of the city name`_, and add them, if any, to the candidate place set.
#. Generate a unique set of ZIPs from the set of candidate places, since a ZIP
may have one or more names associated with it.
#. Generate `a list of candidate address records`_ by fetching all the street
features matching the metaphone hash of the street name and one of the ZIPs
in the query set, along with the ranges matching the edge ID of each
feature, where the given number is in the range. The edge does not
need to be fetched yet.
#. If the look up generates no results, optionally generate `more candidate
records`_ by looking up all the street features matching the metaphone hash
of the street name, along with the ranges matching the edge ID of each
feature, where the given number is in the range. This may be a very time
consuming database query, because some street names are quite common.
#. Score each of the candidate records as follows:
a. Score one point for every provided element of the address query that it
matches exactly.
#. Optionally, compute the scaled Damerau-Levenshtein distance (or
alternately the simple Levenshtein distance) between each provided
element of the address query and the corresponding element in the
candidate. Score one minus the scaled distance, which yields a fraction
of a point.
#. Score one point if the parity of starting range number matches the parity
of the queried address number.
#. Note that the maximum possible score is equal to the number of provided
elements in the address query. Divide the score by the maximum possible.
This is the confidence value of the candidate.
#. Sort the candidate address records by confidence. Retain only the records
that share the highest confidence as candidates.
#. Fetch `the edges and primary feature names`_ matching the edge IDs of
the remaining candidate address records.
#. For each remaining candidate record:
a. Replace the candidate record feature elements with those of the
primary feature name for that edge.
#. Fetch `all of the ranges for the edge ID`_ of the candidate, sorted by
starting number.
#. Compute the sum of the differences of the starting and ending house
number for each range. This is the total number width of the edge.
#. Take the difference between the candidate starting number and the lowest
starting number, add the difference between the queried number and the
candidate starting number, and divide by the total number width. This is
the interpolation distance.
#. Optionally, find the local UTM zone and project the edge into it.
#. Find the point along the line at the interpolation distance.
#. If the edge was projected, unproject the point.
#. Assign the point as the geocoded location of the query to the candidate
record.
#. Construct a set of result ZIPs from the remaining candidates, and look up
`the primary name and state for each ZIP`_ in the set. Assign the matching
primary city and state to each candidate.
#. Return the set of candidate records as the result of the query.
SQL Statements
--------------
the place from the ZIP
~~~~~~~~~~~~~~~~~~~~~~~
::
SELECT * FROM place WHERE zip = '...';
the places matching the metaphone hash of the city name
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
::
SELECT * FROM place WHERE city_phone = metaphone('...');
a list of candidate address records
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
::
SELECT feature.*, range.* FROM feature, range
WHERE name_phone = metaphone('...') AND feature.zip IN (...)
AND range.tlid = feature.tlid
AND fromhn <= ... AND tohn >= ...;
more candidate records
~~~~~~~~~~~~~~~~~~~~~~
::
SELECT feature.*, range.* FROM feature, range
WHERE name_phone = metaphone('...')
AND range.tlid = feature.tlid
AND fromhn <= ... AND tohn >= ...;
the edges and primary feature names
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
::
SELECT feature.*, edge.* FROM feature, edge
WHERE feature.tlid = ... AND paflag = 'P'
AND edge.tlid = feature.tlid;
-- or
SELECT feature.*, edge.* FROM feature, edge
WHERE feature.tlid IN (...)
AND paflag = 'P'
AND edge.tlid = feature.tlid;
all of the ranges for the edge ID
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
::
SELECT * FROM range WHERE range.tlid = ...;
-- or
SELECT * FROM range WHERE range.tlid IN (...);
the primary name and state for each ZIP
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
::
SELECT * FROM place WHERE zip IN (...) AND paflag = 'P';
= 30 =
================================================
FILE: doc/parsing.rst
================================================
.. _parsing:
====================================
Geocoder.us Address Parsing Strategy
====================================
:Author: Schuyler Erle
:Contact: schuyler at geocoder dot us
:Created: 2009/03/18
:Edited: 2009/03/18
Structured address components
-----------------------------
Unless otherwise labeled as "required", all components of a structured address
are optional.
prenum
The alphanumeric prefix portion of a house or building number. (e.g. "32-"
in "32-20 Jackson St".
number
The house or building number component. Required.
sufnum
The alphanumeric suffix portion of a house or building number. (e.g. "23B
Baker St")
fraction
The fractional portion of a house or building number. (e.g. "23 1/2 Baker
St")
predir
The prefixed street directional component. (e.g. "N", "SW")
prequal
The prefixed street qualifier component. (e.g. "Old", "Business")
pretyp
The prefixed street type component. (e.g. "US Hwy")
street
The main portion of the street name. Required.
suftyp
The suffixed street type component. (e.g. "Rd", "Ave")
sufqual
The suffixed street qualifier component.
sufdir
The suffixed street directional component.
unittyp
The unit type, if any. (e.g. "Fl", "Apt", "Ste")
unit
The unit identifer, if any.
city
The name of the city or locale.
state
The two letter postal state code.
zip
The zero padded, five digit ZIP postal code.
plus4
The zero padded, four digit ZIP+4 postal extension.
Parsing Strategy
----------------
Each component will have a regular expression, and a maximum
count. Components are ordered from first to last.
Those components drawn from finite lists - directionals, qualifiers,
types, and states - will have regular expressions composed of the union of
the corresponding list.
A *parse* will consist of a component state, a penalty count, a list of
component strings and a counter for each component.
1. Initialize an input stack, consisting of a single blank parse.
#. Split the address string on whitespace into tokens.
#. For each token:
A. For each component:
i. Test the token against the regular expression.
#. If the regexp matches, add the component name to a list of matching
components.
#. Initialize an empty output stack.
#. For each parse in the input stack:
i. Copy the current parse, increment the penalty count on the new parse,
and add it to the output stack.
#. For each matching component for the current token:
a. If the component state for this parse is later than the
matching component, continue to the next matching component.
#. If the component count for this parse state is equal to the
maximum count for the component, continue to the next matching
component.
#. Otherwise, copy the parse state, and append the token to the
component string, with a leading space, if necessary.
#. Increment the matching component counter for the current parse.
#. Set the component state of the current parse to the matching
component.
#. Push the new parse on to the output stack.
#. Replace the input stack with the output stack.
#. Post-process number prefix/suffixes and ZIP+4 extensions.
#. Score each parse by the number of components with non-empty strings,
minus the penalty count of the parse.
#. Return the sorted list of parsed string lists.
================================================
FILE: doc/voidspace.css
================================================
/*
:Authors: Ian Bicking, Michael Foord
:Contact: fuzzyman@voidspace.org.uk
:Date: 2005/08/26
:Version: 0.1.0
:Copyright: This stylesheet has been placed in the public domain.
:Modified By: Schuyler Erle, for geocoder.us, 2008-03-14
Stylesheet for Docutils.
Based on ``blue_box.css`` by Ian Bicking
and ``html4css1.css`` revision 1.46.
*/
@import url(html4css1.css);
/* changes made by SDE */
body {
font-family: Arial, sans-serif;
margin-left: 10%;
margin-right: 10%;
}
p { text-align: justify; }
dt { font-style: italic; }
/* end changes */
em, i {
/* Typically serif fonts have much nicer italics */
font-family: Times New Roman, Times, serif;
}
a.target {
color: blue;
}
a.target {
color: blue;
}
a.toc-backref {
text-decoration: none;
color: black;
}
a.toc-backref:hover {
background-color: inherit;
}
a:hover {
background-color: #cccccc;
}
div.attention, div.caution, div.danger, div.error, div.hint,
div.important, div.note, div.tip, div.warning {
background-color: #cccccc;
padding: 3px;
width: 80%;
}
div.admonition p.admonition-title, div.hint p.admonition-title,
div.important p.admonition-title, div.note p.admonition-title,
div.tip p.admonition-title {
text-align: center;
background-color: #999999;
display: block;
margin: 0;
}
div.attention p.admonition-title, div.caution p.admonition-title,
div.danger p.admonition-title, div.error p.admonition-title,
div.warning p.admonition-title {
color: #cc0000;
font-family: sans-serif;
text-align: center;
background-color: #999999;
display: block;
margin: 0;
}
h1, h2, h3, h4, h5, h6 {
font-family: Helvetica, Arial, sans-serif;
border: thin solid black;
/* This makes the borders rounded on Mozilla, which pleases me */
-moz-border-radius: 8px;
padding: 4px;
}
h1 {
background-color: #444499;
color: #ffffff;
border: medium solid black;
}
h1 a.toc-backref, h2 a.toc-backref {
color: #ffffff;
}
h2 {
background-color: #666666;
color: #ffffff;
border: medium solid black;
}
h3, h4, h5, h6 {
background-color: #cccccc;
color: #000000;
}
h3 a.toc-backref, h4 a.toc-backref, h5 a.toc-backref,
h6 a.toc-backref {
color: #000000;
}
h1.title {
text-align: center;
background-color: #444499;
color: #eeeeee;
border: thick solid black;
-moz-border-radius: 20px;
}
table.footnote {
padding-left: 0.5ex;
}
table.citation {
padding-left: 0.5ex
}
pre.literal-block, pre.doctest-block {
border: thin black solid;
padding: 5px;
}
.image img { border-style : solid;
border-width : 2px;
}
h1 tt, h2 tt, h3 tt, h4 tt, h5 tt, h6 tt {
font-size: 100%;
}
code, tt {
color: #000066;
}
================================================
FILE: gemspec
================================================
Gem::Specification.new do |s|
s.name = 'Geocoder-US'
s.version = "2.0.4"
s.author = "Schuyler Erle"
s.email = 'geocoder@entropyfree.com'
s.description = "US address geocoding based on TIGER/Line."
s.summary = "US address geocoding based on TIGER/Line."
s.homepage = "http://geocoder.us/"
s.files = ["lib/geocoder/us.rb"] + Dir["lib/geocoder/us/*"] + Dir["tests/*"]
s.require_path = "lib"
s.test_files = "test/run.rb"
s.add_dependency 'sqlite3-ruby', '~>1.3.1'
s.has_rdoc = true
s.extra_rdoc_files = ["README.rdoc"]
end
================================================
FILE: lib/geocoder/us/address.rb
================================================
require 'geocoder/us/constants'
module Geocoder::US
# Defines the matching of parsed address tokens.
Match = {
# FIXME: shouldn't have to anchor :number and :zip at start/end
:number => /^(\d+\W|[a-z]+)?(\d+)([a-z]?)\b/io,
:street => /(?:\b(?:\d+\w*|[a-z'-]+)\s*)+/io,
:city => /(?:\b[a-z'-]+\s*)+/io,
:state => Regexp.new(State.regexp.source + "\s*$", Regexp::IGNORECASE),
:zip => /(\d{5})(?:-\d{4})?\s*$/o,
:at => /\s(at|@|and|&)\s/io,
:po_box => /\b[P|p]*(OST|ost)*\.*\s*[O|o|0]*(ffice|FFICE)*\.*\s*[B|b][O|o|0][X|x]\b/
}
# The Address class takes a US street address or place name and
# constructs a list of possible structured parses of the address
# string.
class Address
attr_accessor :text
attr_accessor :prenum, :number, :sufnum
attr_accessor :street
attr_accessor :city
attr_accessor :state
attr_accessor :zip, :plus4
# Takes an address or place name string as its sole argument.
def initialize(text)
raise ArgumentError, "no text provided" unless text and !text.empty?
if text.class == Hash
@text = ""
assign_text_to_address text
else
@text = clean text
parse
end
end
# Removes any characters that aren't strictly part of an address string.
def clean(value)
value.strip \
.gsub(/[^a-z0-9 ,'&@\/-]+/io, "") \
.gsub(/\s+/o, " ")
end
def assign_text_to_address(text)
if text[:address]
@text = clean text[:address]
parse
else
@street = []
@prenum = text[:prenum]
@sufnum = text[:sufnum]
if !text[:street].nil?
@street = text[:street].scan(Match[:street])
end
@number = ""
if !@street.nil?
if text[:number].nil?
@street.map! { |single_street|
single_street.downcase!
@number = single_street.scan(Match[:number])[0].to_s
single_street.sub! @number, ""
single_street.sub! /^\s*,?\s*/o, ""
}
else
@number = text[:number].to_s
end
@street = expand_streets(@street)
street_parts
end
@city = []
if !text[:city].nil?
@city.push(text[:city])
else
@city.push("")
end
if !text[:region].nil?
# @state = []
@state = text[:region]
if @state.length > 2
# full_state = @state.strip # special case: New York
@state = State[@state]
end
elsif !text[:country].nil?
@state = text[:country]
elsif !text[:state].nil?
@state = text[:state]
end
@zip = text[:postal_code]
@plus4 = text[:plus4]
if !@zip
@zip = @plus4 = ""
end
end
end
# Expands a token into a list of possible strings based on
# the Geocoder::US::Name_Abbr constant, and expands numerals and
# number words into their possible equivalents.
def expand_numbers (string)
if /\b\d+(?:st|nd|rd|th)?\b/o.match string
match = $&
num = $&.to_i
elsif Ordinals.regexp.match string
num = Ordinals[$&]
match = $&
elsif Cardinals.regexp.match string
num = Cardinals[$&]
match = $&
end
strings = []
if num and num < 100
[num.to_s, Ordinals[num], Cardinals[num]].each {|replace|
strings << string.sub(match, replace)
}
else
strings << string
end
strings
end
def parse_zip(regex_match, text)
idx = text.rindex(regex_match)
text[idx...idx+regex_match.length] = ""
text.sub! /\s*,?\s*$/o, ""
@zip, @plus4 = @zip.map {|s|s.strip}
text
end
def parse_state(regex_match, text)
idx = text.rindex(regex_match)
text[idx...idx+regex_match.length] = ""
text.sub! /\s*,?\s*$/o, ""
@full_state = @state[0].strip # special case: New York
@state = State[@full_state]
text
end
def parse_number(regex_match, text)
# FIXME: What if this string appears twice?
idx = text.index(regex_match)
text[idx...idx+regex_match.length] = ""
text.sub! /^\s*,?\s*/o, ""
@prenum, @number, @sufnum = @number.map {|s| s and s.strip}
text
end
def parse
text = @text.clone.downcase
@zip = text.scan(Match[:zip])[-1]
if @zip
text = parse_zip($&, text)
else
@zip = @plus4 = ""
end
@state = text.scan(Match[:state])[-1]
if @state
text = parse_state($&, text)
else
@full_state = ""
@state = ""
end
@number = text.scan(Match[:number])[0]
# FIXME: 230 Fish And Game Rd, Hudson NY 12534
if @number # and not intersection?
text = parse_number($&, text)
else
@prenum = @number = @sufnum = ""
end
# FIXME: special case: Name_Abbr gets a bit aggressive
# about replacing St with Saint. exceptional case:
# Sault Ste. Marie
# FIXME: PO Box should geocode to ZIP
@street = text.scan(Match[:street])
@street = expand_streets(@street)
# SPECIAL CASE: 1600 Pennsylvania 20050
@street << @full_state if @street.empty? and @state.downcase != @full_state.downcase
@city = text.scan(Match[:city])
if !@city.empty?
@city = [@city[-1].strip]
add = @city.map {|item| item.gsub(Name_Abbr.regexp) {|m| Name_Abbr[m]}}
@city |= add
@city.map! {|s| s.downcase}
@city.uniq!
else
@city = []
end
# SPECIAL CASE: no city, but a state with the same name. e.g. "New York"
@city << @full_state if @state.downcase != @full_state.downcase
end
def expand_streets(street)
if !street.empty? && !street[0].nil?
street.map! {|s|s.strip}
add = street.map {|item| item.gsub(Name_Abbr.regexp) {|m| Name_Abbr[m]}}
street |= add
add = street.map {|item| item.gsub(Std_Abbr.regexp) {|m| Std_Abbr[m]}}
street |= add
street.map! {|item| expand_numbers(item)}
street.flatten!
street.map! {|s| s.downcase}
street.uniq!
else
street = []
end
street
end
def street_parts
strings = []
# Get all the substrings delimited by whitespace
@street.each {|string|
tokens = string.split(" ")
strings |= (0...tokens.length).map {|i|
(i...tokens.length).map {|j| tokens[i..j].join(" ")}}.flatten
}
strings = remove_noise_words(strings)
# Try a simpler case of adding the @number in case everything is an abbr.
strings += [@number] if strings.all? {|s| Std_Abbr.key? s or Name_Abbr.key? s}
strings.uniq
end
def remove_noise_words(strings)
# Don't return strings that consist solely of abbreviations.
# NOTE: Is this a micro-optimization that has edge cases that will break?
# Answer: Yes, it breaks on simple things like "Prairie St" or "Front St"
prefix = Regexp.new("^" + Prefix_Type.regexp.source + "\s*", Regexp::IGNORECASE)
suffix = Regexp.new("\s*" + Suffix_Type.regexp.source + "$", Regexp::IGNORECASE)
predxn = Regexp.new("^" + Directional.regexp.source + "\s*", Regexp::IGNORECASE)
sufdxn = Regexp.new("\s*" + Directional.regexp.source + "$", Regexp::IGNORECASE)
good_strings = strings.map {|s|
s = s.clone
s.gsub!(predxn, "")
s.gsub!(sufdxn, "")
s.gsub!(prefix, "")
s.gsub!(suffix, "")
s
}
good_strings.reject! {|s| s.empty?}
strings = good_strings if !good_strings.empty? {|s| not Std_Abbr.key?(s) and not Name_Abbr.key?(s)}
strings
end
def city_parts
strings = []
@city.map {|string|
tokens = string.split(" ")
strings |= (0...tokens.length).to_a.reverse.map {|i|
(i...tokens.length).map {|j| tokens[i..j].join(" ")}}.flatten
}
# Don't return strings that consist solely of abbreviations.
# NOTE: Is this a micro-optimization that has edge cases that will break?
# Answer: Yes, it breaks on "Prairie"
good_strings = strings.reject {|s| Std_Abbr.key? s}
strings = good_strings if !good_strings.empty?
strings.uniq
end
def city= (strings)
# NOTE: This will still fail on: 100 Broome St, 33333 (if 33333 is
# Broome, MT or what)
strings = expand_streets(strings) # fix for "Mountain View" -> "Mountain Vw"
match = Regexp.new('\s*\b(?:' + strings.join("|") + ')\b\s*$', Regexp::IGNORECASE)
# only remove city from street strings if address was parsed
unless @text == ""
@street = @street.map {|string| string.gsub(match, '')}.select {|s|!s.empty?}
end
end
def po_box?
Match[:po_box].match @text
end
def intersection?
Match[:at].match @text
end
end
end
================================================
FILE: lib/geocoder/us/constants.rb
================================================
# coding: utf-8
require 'set'
require 'geocoder/us/numbers'
module Geocoder
end
module Geocoder::US
class Map < Hash
# The Map class provides a two-way mapping between postal abbreviations
# and their fully written equivalents.
#attr_accessor :partial
attr_accessor :regexp
def self.[] (*items)
hash = super(*items)
#hash.build_partial
hash.build_match
hash.keys.each {|k| hash[k.downcase] = hash.fetch(k)}
hash.values.each {|v| hash[v.downcase] = v}
hash.freeze
end
# The build_partial method constructs a hash of case-insensitive,
# whitespace-delimited prefixes to keys and values in the two-way Map.
def build_partial
@partial = Set.new()
[keys, values].flatten.each {|item|
@partial << item.downcase
item.downcase.split.each {|token| @partial << token}
}
end
def build_match
@regexp = Regexp.new(
'\b(' + [keys,values].flatten.join("|") + ')\b',
Regexp::IGNORECASE)
end
# The partial? method returns true if the key is a prefix of some
# key in the Map.
def partial? (key)
@partial.member? key.downcase
end
def key? (key)
super(key.downcase)
end
def [] (key)
super(key.downcase)
end
end
# The Directional constant maps compass direction words in English and
# Spanish to their 1- or 2- letter abbreviations. See 2008 TIGER/Line
# technical documentation Appendix C for more details.
Directional = Map[
"North" => "N",
"South" => "S",
"East" => "E",
"West" => "W",
"Northeast" => "NE",
"Northwest" => "NW",
"Southeast" => "SE",
"Southwest" => "SW",
"Norte" => "N",
"Sur" => "S",
"Este" => "E",
"Oeste" => "O",
"Noreste" => "NE",
"Noroeste" => "NO",
"Sudeste" => "SE",
"Sudoeste" => "SO"
]
# The Prefix_Qualifier constant maps feature prefix qualifiers to their
# abbreviations. See 2008 TIGER/Line technical documentation Appendix D.
Prefix_Qualifier = Map[
"Alternate" => "Alt",
"Business" => "Bus",
"Bypass" => "Byp",
"Extended" => "Exd",
"Historic" => "Hst",
"Loop" => "Lp",
"Old" => "Old",
"Private" => "Pvt",
"Public" => "Pub",
"Spur" => "Spr",
]
# The Suffix_Qualifier constant maps feature suffix qualifiers to their
# abbreviations. See 2008 TIGER/Line technical documentation Appendix D.
Suffix_Qualifier = Map[
"Access" => "Acc",
"Alternate" => "Alt",
"Business" => "Bus",
"Bypass" => "Byp",
"Connector" => "Con",
"Extended" => "Exd",
"Extension" => "Exn",
"Loop" => "Lp",
"Private" => "Pvt",
"Public" => "Pub",
"Scenic" => "Scn",
"Spur" => "Spr",
"Ramp" => "Rmp",
"Underpass" => "Unp",
"Overpass" => "Ovp",
]
# The Prefix_Canonical constant maps canonical TIGER/Line street type
# prefixes to their abbreviations. This list is the subset of the list from
# 2008 TIGER/Line technical documentation Appendix E that was extracted from
# a TIGER/Line database import.
Prefix_Canonical = {
"Arcade" => "Arc",
"Autopista" => "Autopista",
"Avenida" => "Ave",
"Avenue" => "Ave",
"Boulevard" => "Blvd",
"Bulevar" => "Bulevar",
"Bureau of Indian Affairs Highway" => "BIA Hwy",
"Bureau of Indian Affairs Road" => "BIA Rd",
"Bureau of Indian Affairs Route" => "BIA Rte",
"Bureau of Land Management Road" => "BLM Rd",
"Bypass" => "Byp",
"Calle" => "Cll",
"Calleja" => "Calleja",
"Callejón" => "Callejón",
"Caminito" => "Cmt",
"Camino" => "Cam",
"Carretera" => "Carr",
"Cerrada" => "Cer",
"Círculo" => "Cír",
"Commons" => "Cmns",
"Corte" => "Corte",
"County Highway" => "Co Hwy",
"County Lane" => "Co Ln",
"County Road" => "Co Rd",
"County Route" => "Co Rte",
"County State Aid Highway" => "Co St Aid Hwy",
"County Trunk Highway" => "Co Trunk Hwy",
"County Trunk Road" => "Co Trunk Rd",
"Court" => "Ct",
"Delta Road" => "Delta Rd",
"District of Columbia Highway" => "DC Hwy",
"Driveway" => "Driveway",
"Entrada" => "Ent",
"Expreso" => "Expreso",
"Expressway" => "Expy",
"Farm Road" => "Farm Rd",
"Farm-to-Market Road" => "FM",
"Fire Control Road" => "Fire Cntrl Rd",
"Fire District Road" => "Fire Dist Rd",
"Fire Lane" => "Fire Ln",
"Fire Road" => "Fire Rd",
"Fire Route" => "Fire Rte",
"Fire Trail" => "Fire Trl",
"Forest Highway" => "Forest Hwy",
"Forest Road" => "Forest Rd",
"Forest Route" => "Forest Rte",
"Forest Service Road" => "FS Rd",
"Highway" => "Hwy",
"Indian Route" => "Indian Rte",
"Indian Service Route" => "Indian Svc Rte",
"Interstate Highway" => "I-",
"Lane" => "Ln",
"Logging Road" => "Logging Rd",
"Loop" => "Loop",
"National Forest Development Road" => "Nat For Dev Rd",
"Navajo Service Route" => "Navajo Svc Rte",
"Parish Road" => "Parish Rd",
"Pasaje" => "Pasaje",
"Paseo" => "Pso",
"Passage" => "Psge",
"Placita" => "Pla",
"Plaza" => "Plz",
"Point" => "Pt",
"Puente" => "Puente",
"Ranch Road" => "Ranch Rd",
"Ranch to Market Road" => "RM",
"Reservation Highway" => "Resvn Hwy",
"Road" => "Rd",
"Route" => "Rte",
"Row" => "Row",
"Rue" => "Rue",
"Ruta" => "Ruta",
"Sector" => "Sec",
"Sendero" => "Sendero",
"Service Road" => "Svc Rd",
"Skyway" => "Skwy",
"Square" => "Sq",
"State Forest Service Road" => "St FS Rd",
"State Highway" => "State Hwy",
"State Loop" => "State Loop",
"State Road" => "State Rd",
"State Route" => "State Rte",
"State Spur" => "State Spur",
"State Trunk Highway" => "St Trunk Hwy",
"Terrace" => "Ter",
"Town Highway" => "Town Hwy",
"Town Road" => "Town Rd",
"Township Highway" => "Twp Hwy",
"Township Road" => "Twp Rd",
"Trail" => "Trl",
"Tribal Road" => "Tribal Rd",
"Tunnel" => "Tunl",
"US Forest Service Highway" => "USFS Hwy",
"US Forest Service Road" => "USFS Rd",
"US Highway" => "US Hwy",
"US Route" => "US Rte",
"Vereda" => "Ver",
"Via" => "Via",
"Vista" => "Vis",
}
# The Prefix_Alternate constant maps alternate prefix street types to
# their canonical abbreviations. This list was merged in from the USPS
# list at http://www.usps.com/ncsc/lookups/abbr_suffix.txt.
Prefix_Alternate = {
"Av" => "Ave",
"Aven" => "Ave",
"Avenu" => "Ave",
"Avenue" => "Ave",
"Avn" => "Ave",
"Avnue" => "Ave",
"Boul" => "Blvd",
"Boulv" => "Blvd",
"Bypa" => "Byp",
"Bypas" => "Byp",
"Byps" => "Byp",
"Crt" => "Ct",
"Exp" => "Expy",
"Expr" => "Expy",
"Express" => "Expy",
"Expw" => "Expy",
"Highwy" => "Hwy",
"Hiway" => "Hwy",
"Hiwy" => "Hwy",
"Hway" => "Hwy",
#"La" => "Ln", # causes problems with Spanglish place names
"Lanes" => "Ln",
"Loops" => "Loop",
"Plza" => "Plz",
"Sqr" => "Sq",
"Sqre" => "Sq",
"Squ" => "Sq",
"Terr" => "Ter",
"Tr" => "Trl",
"Trails" => "Trl",
"Trls" => "Trl",
"Tunel" => "Tunl",
"Tunls" => "Tunl",
"Tunnels" => "Tunl",
"Tunnl" => "Tunl",
"Vdct" => "Via",
"Viadct" => "Via",
"Viaduct" => "Via",
"Vist" => "Vis",
"Vst" => "Vis",
"Vsta" => "Vis"
}
# The Prefix_Type constant merges the canonical prefix type abbreviations
# with their USPS accepted alternates.
Prefix_Type = Map[ Prefix_Canonical.merge(Prefix_Alternate) ]
# The Suffix_Canonical constant maps canonical TIGER/Line street type
# suffixes to their abbreviations. This list is the subset of the list from
# 2008 TIGER/Line technical documentation Appendix E that was extracted from
# a TIGER/Line database import.
Suffix_Canonical = {
"Alley" => "Aly",
"Arcade" => "Arc",
"Avenida" => "Ave",
"Avenue" => "Ave",
"Beltway" => "Beltway",
"Boulevard" => "Blvd",
"Bridge" => "Brg",
"Bypass" => "Byp",
"Causeway" => "Cswy",
"Circle" => "Cir",
"Common" => "Cmn",
"Commons" => "Cmns",
"Corners" => "Cors",
"Court" => "Ct",
"Courts" => "Cts",
"Crescent" => "Cres",
"Crest" => "Crst",
"Crossing" => "Xing",
"Cutoff" => "Cutoff",
"Drive" => "Dr",
"Driveway" => "Driveway",
"Esplanade" => "Esplanade",
"Estates" => "Ests",
"Expressway" => "Expy",
"Forest Highway" => "Forest Hwy",
"Fork" => "Frk",
"Four-Wheel Drive Trail" => "4WD Trl",
"Freeway" => "Fwy",
"Grade" => "Grade",
"Heights" => "Hts",
"Highway" => "Hwy",
"Jeep Trail" => "Jeep Trl",
"Landing" => "Lndg",
"Lane" => "Ln",
"Logging Road" => "Logging Rd",
"Loop" => "Loop",
"Motorway" => "Mtwy",
"Oval" => "Oval",
"Overpass" => "Opas",
"Parkway" => "Pkwy",
"Pass" => "Pass",
"Passage" => "Psge",
"Path" => "Path",
"Pike" => "Pike",
"Place" => "Pl",
"Plaza" => "Plz",
"Point" => "Pt",
"Pointe" => "Pointe",
"Promenade" => "Promenade",
"Railroad" => "RR",
"Railway" => "Rlwy",
"Ramp" => "Ramp",
"River" => "Riv",
"Road" => "Rd",
"Roadway" => "Roadway",
"Route" => "Rte",
"Row" => "Row",
"Rue" => "Rue",
"Service Road" => "Svc Rd",
"Skyway" => "Skwy",
"Spur" => "Spur",
"Square" => "Sq",
"Stravenue" => "Stra",
"Street" => "St",
"Strip" => "Strip",
"Terrace" => "Ter",
"Thoroughfare" => "Thoroughfare",
"Tollway" => "Tollway",
"Trace" => "Trce",
"Trafficway" => "Trfy",
"Trail" => "Trl",
"Trolley" => "Trolley",
"Truck Trail" => "Truck Trl",
"Tunnel" => "Tunl",
"Turnpike" => "Tpke",
"Viaduct" => "Viaduct",
"View" => "Vw",
"Vista" => "Vis",
"Walk" => "Walk",
"Walkway" => "Walkway",
"Way" => "Way",
}
# The Suffix_Alternate constant maps alternate suffix street types to
# their canonical abbreviations. This list was merged in from the USPS
# list at http://www.usps.com/ncsc/lookups/abbr_suffix.txt.
Suffix_Alternate = {
"Allee" => "Aly",
"Ally" => "Aly",
"Av" => "Ave",
"Aven" => "Ave",
"Avenu" => "Ave",
"Avenue" => "Ave",
"Avn" => "Ave",
"Avnue" => "Ave",
"Boul" => "Blvd",
"Boulv" => "Blvd",
"Brdge" => "Brg",
"Bypa" => "Byp",
"Bypas" => "Byp",
"Byps" => "Byp",
"Causway" => "Cswy",
"Circ" => "Cir",
"Circl" => "Cir",
"Crcl" => "Cir",
"Crcle" => "Cir",
"Crecent" => "Cres",
"Cresent" => "Cres",
"Crscnt" => "Cres",
"Crsent" => "Cres",
"Crsnt" => "Cres",
"Crssing" => "Xing",
"Crssng" => "Xing",
"Crt" => "Ct",
"Driv" => "Dr",
"Drv" => "Dr",
"Exp" => "Expy",
"Expr" => "Expy",
"Express" => "Expy",
"Expw" => "Expy",
"Freewy" => "Fwy",
"Frway" => "Fwy",
"Frwy" => "Fwy",
"Height" => "Hts",
"Hgts" => "Hts",
"Highwy" => "Hwy",
"Hiway" => "Hwy",
"Hiwy" => "Hwy",
"Ht" => "Hts",
"Hway" => "Hwy",
"La" => "Ln",
"Lanes" => "Ln",
"Lndng" => "Lndg",
"Loops" => "Loop",
"Ovl" => "Oval",
"Parkways" => "Pkwy",
"Parkwy" => "Pkwy",
"Paths" => "Path",
"Pikes" => "Pike",
"Pkway" => "Pkwy",
"Pkwys" => "Pkwy",
"Pky" => "Pkwy",
"Plza" => "Plz",
"Rivr" => "Riv",
"Rvr" => "Riv",
"Spurs" => "Spur",
"Sqr" => "Sq",
"Sqre" => "Sq",
"Squ" => "Sq",
"Str" => "St",
"Strav" => "Stra",
"Strave" => "Stra",
"Straven" => "Stra",
"Stravn" => "Stra",
"Strt" => "St",
"Strvn" => "Stra",
"Strvnue" => "Stra",
"Terr" => "Ter",
"Tpk" => "Tpke",
"Tr" => "Trl",
"Traces" => "Trce",
"Trails" => "Trl",
"Trls" => "Trl",
"Trnpk" => "Tpke",
"Trpk" => "Tpke",
"Tunel" => "Tunl",
"Tunls" => "Tunl",
"Tunnels" => "Tunl",
"Tunnl" => "Tunl",
"Turnpk" => "Tpke",
"Vist" => "Vis",
"Vst" => "Vis",
"Vsta" => "Vis",
"Walks" => "Walk",
"Wy" => "Way",
}
# The Suffix_Type constant merges the canonical suffix type abbreviations
# with their USPS accepted alternates.
Suffix_Type = Map[ Suffix_Canonical.merge(Suffix_Alternate) ]
# The Unit_Type constant lists acceptable USPS unit type abbreviations
# from http://www.usps.com/ncsc/lookups/abbr_sud.txt.
Unit_Type = Map[
"Apartment" => "Apt",
"Basement" => "Bsmt",
"Building" => "Bldg",
"Department"=> "Dept",
"Floor" => "Fl",
"Front" => "Frnt",
"Hangar" => "Hngr",
"Lobby" => "Lbby",
"Lot" => "Lot",
"Lower" => "Lowr",
"Office" => "Ofc",
"Penthouse" => "Ph",
"Pier" => "Pier",
"Rear" => "Rear",
"Room" => "Rm",
"Side" => "Side",
"Slip" => "Slip",
"Space" => "Spc",
"Stop" => "Stop",
"Suite" => "Ste",
"Trailer" => "Trlr",
"Unit" => "Unit",
"Upper" => "Uppr",
]
Std_Abbr = Map[
[Directional, Prefix_Qualifier, Suffix_Qualifier,
Prefix_Type, Suffix_Type].inject({}) {|x,y|x.merge y}
]
# The Name_Abbr constant maps common toponym abbreviations to their
# full word equivalents. This list was constructed partly by hand, and
# partly by matching USPS alternate abbreviations with feature names
# found in the TIGER/Line dataset.
Name_Abbr = Map[
"Av" => "Avenue",
"Ave" => "Avenue",
"Blvd" => "Boulevard",
"Bot" => "Bottom",
"Boul" => "Boulevard",
"Boulv" => "Boulevard",
"Br" => "Branch",
"Brg" => "Bridge",
"Canyn" => "Canyon",
"Cen" => "Center",
"Cent" => "Center",
"Cir" => "Circle",
"Circ" => "Circle",
"Ck" => "Creek",
"Cnter" => "Center",
"Cntr" => "Center",
"Cnyn" => "Canyon",
"Cor" => "Corner",
"Cors" => "Corners",
"Cp" => "Camp",
"Cr" => "Creek",
"Crcl" => "Circle",
"Crcle" => "Circle",
"Cres" => "Crescent",
"Crscnt" => "Crescent",
"Ct" => "Court",
"Ctr" => "Center",
"Cts" => "Courts",
"Cyn" => "Canyon",
"Div" => "Divide",
"Dr" => "Drive",
"Dv" => "Divide",
"Est" => "Estate",
"Ests" => "Estates",
"Ext" => "Extension",
"Extn" => "Extension",
"Extnsn" => "Extension",
"Forests" => "Forest",
"Forg" => "Forge",
"Frg" => "Forge",
"Ft" => "Fort",
"Gatewy" => "Gateway",
"Gdn" => "Garden",
"Gdns" => "Gardens",
"Gtwy" => "Gateway",
"Harb" => "Harbor",
"Hbr" => "Harbor",
"Height" => "Heights",
"Hgts" => "Heights",
"Highwy" => "Highway",
"Hiway" => "Highway",
"Hiwy" => "Highway",
"Holws" => "Hollow",
"Ht" => "Heights",
"Hway" => "Highway",
"Hwy" => "Highway",
"Is" => "Island",
"Iss" => "Islands",
"Jct" => "Junction",
"Jction" => "Junction",
"Jctn" => "Junction",
"Junctn" => "Junction",
"Juncton" => "Junction",
"Ldg" => "Lodge",
"Lgt" => "Light",
"Lndg" => "Landing",
"Lodg" => "Lodge",
"Loops" => "Loop",
"Mt" => "Mount",
"Mtin" => "Mountain",
"Mtn" => "Mountain",
"Orch" => "Orchard",
"Parkwy" => "Parkway",
"Pk" => "Park",
"Pkway" => "Parkway",
"Pkwy" => "Parkway",
"Pky" => "Parkway",
"Pl" => "Place",
"Pnes" => "Pines",
"Pr" => "Prairie",
"Prr" => "Prairie",
"Pt" => "Point",
"Pts" => "Points",
"Rdg" => "Ridge",
"Riv" => "River",
"Rnchs" => "Ranch",
"Spg" => "Spring",
"Spgs" => "Springs",
"Spng" => "Spring",
"Spngs" => "Springs",
"Sq" => "Square",
"Squ" => "Square",
# "St" => "Saint",
"Sta" => "Station",
"Statn" => "Station",
"Ste" => "Sainte",
"Stn" => "Station",
"Str" => "Street",
"Ter" => "Terrace",
"Terr" => "Terrace",
"Tpk" => "Turnpike",
"Tpke" => "Turnpike",
"Tr" => "Trail",
"Trls" => "Trail",
"Trpk" => "Turnpike",
"Tunls" => "Tunnel",
"Un" => "Union",
"Vill" => "Village",
"Villag" => "Village",
"Villg" => "Village",
"Vis" => "Vista",
"Vlg" => "Village",
"Vlgs" => "Villages",
"Wls" => "Wells",
"Wy" => "Way",
"Xing" => "Crossing",
]
# The State constant maps US state and territory names to their 2-letter
# USPS abbreviations.
State = Map[
"Alabama" => "AL",
"Alaska" => "AK",
"American Samoa" => "AS",
"Arizona" => "AZ",
"Arkansas" => "AR",
"California" => "CA",
"Colorado" => "CO",
"Connecticut" => "CT",
"Delaware" => "DE",
"District of Columbia" => "DC",
"Federated States of Micronesia" => "FM",
"Florida" => "FL",
"Georgia" => "GA",
"Guam" => "GU",
"Hawaii" => "HI",
"Idaho" => "ID",
"Illinois" => "IL",
"Indiana" => "IN",
"Iowa" => "IA",
"Kansas" => "KS",
"Kentucky" => "KY",
"Louisiana" => "LA",
"Maine" => "ME",
"Marshall Islands" => "MH",
"Maryland" => "MD",
"Massachusetts" => "MA",
"Michigan" => "MI",
"Minnesota" => "MN",
"Mississippi" => "MS",
"Missouri" => "MO",
"Montana" => "MT",
"Nebraska" => "NE",
"Nevada" => "NV",
"New Hampshire" => "NH",
"New Jersey" => "NJ",
"New Mexico" => "NM",
"New York" => "NY",
"North Carolina" => "NC",
"North Dakota" => "ND",
"Northern Mariana Islands" => "MP",
"Ohio" => "OH",
"Oklahoma" => "OK",
"Oregon" => "OR",
"Palau" => "PW",
"Pennsylvania" => "PA",
"Puerto Rico" => "PR",
"Rhode Island" => "RI",
"South Carolina" => "SC",
"South Dakota" => "SD",
"Tennessee" => "TN",
"Texas" => "TX",
"Utah" => "UT",
"Vermont" => "VT",
"Virgin Islands" => "VI",
"Virginia" => "VA",
"Washington" => "WA",
"West Virginia" => "WV",
"Wisconsin" => "WI",
"Wyoming" => "WY"
]
end
================================================
FILE: lib/geocoder/us/database.rb
================================================
# require 'rubygems'
require 'sqlite3'
# require 'text'
# require 'levenshtein'
require 'set'
require 'pp'
require 'time'
require 'thread'
require 'geocoder/us/address'
require 'geocoder/us/metaphone'
module Geocoder
end
module Geocoder::US
# Provides an interface to a Geocoder::US database.
class Database
Street_Weight = 3.0
Number_Weight = 2.0
Parity_Weight = 1.25
City_Weight = 1.0
@@mutex = Mutex.new
# Takes the path of an SQLite 3 database prepared for Geocoder::US
# as the sole mandatory argument. The helper argument points to the
# Geocoder::US SQLite plugin; the module looks for this in the same
# directory as database.rb by default. The cache_size argument is
# measured in kilobytes and is used to set the SQLite cache size; larger
# values will trade memory for speed in long-running processes.
# dbtype option is used when your datasbase encodes it's geometry blogs according to two formats
# the first. default, is in a series of little-endian 4-byte ints. The second is
# | 1 byte Type | 4 byte SRID | 4 byte element count| 8 byte double coordinates. Use option value 2 for this
# second type
def initialize (filename, options = {})
defaults = {:debug => false, :cache_size => 50000,
:helper => "sqlite3.so", :threadsafe => false,
:create => false, :dbtype => 1}
options = defaults.merge options
raise ArgumentError, "can't find database #{filename}" \
unless options[:create] or File.exists? filename
@db = SQLite3::Database.new( filename )
@st = {}
@dbtype = options[:dbtype]
@debug = options[:debug]
@threadsafe = options[:threadsafe]
tune options[:helper], options[:cache_size]
end
def synchronize
if not @threadsafe
@@mutex.synchronize { yield }
else
yield
end
end
#private
# Load the SQLite extension and tune the database settings.
# q.v. http://web.utk.edu/~jplyon/sqlite/SQLite_optimization_FAQ.html
def tune (helper, cache_size)
if File.expand_path(helper) != helper
helper = File.join(File.dirname(__FILE__), helper)
end
synchronize do
# @db.create_function("levenshtein", 2) do |func, word1, word2|
# test1, test2 = [word1, word2].map {|w|
# w.to_s.gsub(/\W/o, "").downcase
# }
# dist = Levenshtein.distance(test1, test2)
# result = dist.to_f / [test1.length, test2.length].max
# func.set_result result
# end
# @db.create_function("rb_metaphone", 2) do |func, string, len|
# test = string.to_s.gsub(/\W/o, "")
# if test =~ /^(\d+)/o
# mph = $1
# elsif test =~ /^([wy])$/io
# mph = $1
# else
# mph = Text::Metaphone.metaphone test
# end
# func.result = mph[0...len.to_i]
# end
# @db.create_function("nondigit_prefix", 1) do |func, string|
# string.to_s =~ /^(.*\D)?(\d+)$/o
# func.result = ($1 || "")
# end
# @db.create_function("digit_suffix", 1) do |func, string|
# string.to_s =~ /^(.*\D)?(\d+)$/o
# func.result = ($2 || "")
# end
@db.enable_load_extension(1)
@db.load_extension(helper)
@db.enable_load_extension(0)
@db.cache_size = cache_size
@db.temp_store = "memory"
@db.synchronous = "off"
end
end
# Return a cached SQLite statement object, preparing it first if
# it's not already in the cache.
def prepare (sql)
$stderr.print "SQL : #{sql}\n" if @debug
st = nil
synchronize do
# don't even bother cache SQL anymore, it seems to be messing things up
#@st[sql] = @db.prepare sql if not @st[sql] or @st[sql].closed?
st = @db.prepare sql
end
# return @st[sql]
return st
end
def flush_statements
@st = {}
end
# Generate enough SQL placeholders for a list of objects.
def placeholders_for (list)
(["?"] * list.length).join(",")
end
# Generate enough SQL placeholders for a list of objects.
def metaphone_placeholders_for (list)
(["metaphone(?,5)"] * list.length).join(",")
end
# Execute an SQL statement, bind a list of parameters, and
# return the result as a list of hashes.
def execute (sql, *params)
st = prepare(sql)
begin
execute_statement st, *params
ensure
st.close
end
end
# Execute an SQLite statement object, bind the parameters,
# map the column names to symbols, and return the rows
# as a list of hashes.
def execute_statement (st, *params)
if @debug
start = Time.now
$stderr.print "EXEC: #{params.inspect}\n" if !params.empty?
end
rows = []
synchronize do
result = st.execute(*params)
columns = result.columns.map {|c| c.to_sym}
result.each {|row|
rows << Hash[*(columns.zip(row).flatten)]}
end
if @debug
runtime = format("%.3f", Time.now - start)
$stderr.print "ROWS: #{rows.length} (#{runtime}s)\n"
end
rows.reverse!
end
def places_by_zip (city, zip)
execute("SELECT *, levenshtein(?, city) AS city_score
FROM place WHERE zip = ? order by priority desc;", city, zip)
end
# Query the place table for by city, optional state, and zip.
# The metaphone index on the place table is used to match
# city names.
def places_by_city (city, tokens, state)
if city.nil?
city = ""
end
if state.nil? or state.empty?
and_state = ""
args = [city] + tokens.clone
else
and_state = "AND state = ?"
args = [city] + tokens.clone + [state]
end
metaphones = metaphone_placeholders_for tokens
execute("SELECT *, levenshtein(?, city) AS city_score
FROM place WHERE city_phone IN (#{metaphones}) #{and_state} order by priority desc;", *args)
end
# Generate an SQL query and set of parameters against the feature and range
# tables for a street name and optional building number. The SQL is
# used by candidate_records and more_candidate_records to filter results
# by ZIP code.
def features_by_street (street, tokens)
metaphones = (["metaphone(?,5)"] * tokens.length).join(",")
sql = "
SELECT feature.*, levenshtein(?, street) AS street_score
FROM feature
WHERE street_phone IN (#{metaphones})"
params = [street] + tokens
return [sql, params]
end
# Query the feature and range tables for a set of ranges, given a
# building number, street name, and list of candidate ZIP codes.
# The metaphone and ZIP code indexes on the feature table are
# used to match results.
def features_by_street_and_zip (street, tokens, zips)
sql, params = features_by_street(street, tokens)
in_list = placeholders_for zips
sql += " AND feature.zip IN (#{in_list})"
params += zips
execute sql, *params
end
# Query the feature and range tables for a set of ranges, given a
# building number, street name, and list of candidate ZIP codes.
# The ZIP codes are reduced to a set of 3-digit prefixes, broadening
# the search area.
def more_features_by_street_and_zip (street, tokens, zips)
sql, params = features_by_street(street, tokens)
if !zips.empty? and !zips[0].nil?
#puts "zip results 2"
zip3s = zips.map {|z| z[0..2]+'%'}.to_set.to_a
like_list = zip3s.map {|z| "feature.zip LIKE ?"}.join(" OR ")
sql += " AND (#{like_list})"
params += zip3s
end
execute sql, *params
end
def ranges_by_feature (fids, number, prenum)
in_list = placeholders_for fids
limit = 4 * fids.length
sql = "
SELECT feature_edge.fid AS fid, range.*
FROM feature_edge, range
WHERE fid IN (#{in_list})
AND feature_edge.tlid = range.tlid"
params = fids.clone
unless prenum.nil?
sql += " AND prenum = ?"
params += [prenum]
end
sql += "
ORDER BY min(abs(fromhn - ?), abs(tohn - ?))
LIMIT #{limit};"
params += [number, number]
execute sql, *params
end
# Query the edge table for a list of edges matching a list of edge IDs.
def edges (edge_ids)
in_list = placeholders_for edge_ids
sql = "SELECT edge.* FROM edge WHERE edge.tlid IN (#{in_list})"
execute sql, *edge_ids
end
# Query the range table for all ranges associated with the given
# list of edge IDs.
def range_ends (edge_ids)
in_list = placeholders_for edge_ids
sql = "SELECT tlid, side,
min(fromhn) > min(tohn) AS flipped,
min(fromhn) AS from0, max(tohn) AS to0,
min(tohn) AS from1, max(fromhn) AS to1
FROM range WHERE tlid IN (#{in_list})
GROUP BY tlid, side;"
execute(sql, *edge_ids).map {|r|
if r[:flipped].to_i == 1
r[:flipped] = true
r[:fromhn], r[:tohn] = r[:from1], r[:to1]
else
r[:flipped] = false
r[:fromhn], r[:tohn] = r[:from0], r[:to0]
end
[:from0, :to0, :from1, :to1].each {|k| r.delete k}
r
}
end
def intersections_by_fid (fids)
temp_db = "temp_" + rand(1<<32).to_s
temp_table = "intersection_" + rand(1<<32).to_s
execute "ATTACH DATABASE ':memory:' as #{temp_db};"
begin
# flush_statements # the CREATE/DROP TABLE invalidates prepared statements
in_list = placeholders_for fids
sql = "
CREATE TABLE #{temp_db}.#{temp_table} AS
SELECT fid, substr(geometry,1,8) AS point
FROM feature_edge, edge
WHERE feature_edge.tlid = edge.tlid
AND fid IN (#{in_list})
UNION
SELECT fid, substr(geometry,length(geometry)-7,8) AS point
FROM feature_edge, edge
WHERE feature_edge.tlid = edge.tlid
AND fid IN (#{in_list});
CREATE INDEX #{temp_db}.#{temp_table}_pt_idx ON #{temp_table} (point);"
execute sql, *(fids + fids)
# the a.fid < b.fid inequality guarantees consistent ordering of street
# names in the output
sql = "
SELECT a.fid AS fid1, b.fid AS fid2, a.point
FROM #{temp_table} a, #{temp_table} b,
feature f1, feature f2
WHERE a.point = b.point AND a.fid < b.fid
AND f1.fid = a.fid AND f2.fid = b.fid
AND f1.zip = f2.zip
AND f1.paflag = 'P' AND f2.paflag = 'P';"
return execute sql
ensure
# flush_statements # the CREATE/DROP TABLE invalidates prepared statements
execute "DETACH DATABASE #{temp_db};"
end
end
# Query the place table for notional "primary" place names for each of a
# list of ZIP codes. Since the place table shipped with this code is
# bespoke, and constructed from a variety of public domain sources,
# the primary name for a ZIP is not always the "right" one.
def primary_places (zips)
in_list = placeholders_for zips
sql = "SELECT * FROM place WHERE zip IN (#{in_list}) order by priority desc;"
execute sql, *zips
end
# Given a list of rows, find the unique values for a given key.
def unique_values (rows, key)
rows.map {|r| r[key]}.to_set.to_a
end
# Convert a list of rows into a hash keyed by the given keys.
def rows_to_h (rows, *keys)
hash = {}
rows.each {|row| (hash[row.values_at(*keys)] ||= []) << row; }
hash
end
# Merge the values in the list of rows given in src into the
# list of rows in dest, matching rows on the given list of keys.
# May generate more than one row in dest for each input dest row.
def merge_rows! (dest, src, *keys)
src = rows_to_h src, *keys
dest.map! {|row|
vals = row.values_at(*keys)
if src.key? vals
src[vals].map {|row2| row.merge row2}
else
[row]
end
}
dest.flatten!
end
def find_candidates (address)
places = []
candidates = []
city = address.city.sort {|a,b|a.length <=> b.length}[0]
if(!address.zip.empty? && !address.zip.nil?)
places = places_by_zip city, address.zip
end
places = places_by_city city, address.city_parts, address.state if places.empty?
return [] if places.empty?
# setting city will remove city from street, so save off before
address.city = unique_values places, :city
return places if address.street.empty?
zips = unique_values places, :zip
street = address.street.sort {|a,b|a.length <=> b.length}[0]
candidates = features_by_street_and_zip street, address.street_parts, zips
if candidates.empty?
candidates = more_features_by_street_and_zip street, address.street_parts, zips
end
merge_rows! candidates, places, :zip
candidates
end
# Given a query hash and a list of candidates, assign :number
# and :precision values to each candidate. If the query building
# number is inside the candidate range, set the number on the result
# and set the precision to :range; otherwise, find the closest
# corner and set precision to :street.
def assign_number! (hn, candidates)
hn = 0 unless hn
for candidate in candidates
fromhn, tohn = candidate[:fromhn].to_i, candidate[:tohn].to_i
if (hn >= fromhn and hn <= tohn) or (hn <= fromhn and hn >= tohn)
candidate[:number] = hn.to_s
candidate[:precision] = :range
else
candidate[:number] = ((hn - fromhn).abs < (hn - tohn).abs ?
candidate[:fromhn] : candidate[:tohn]).to_s
candidate[:precision] = :street
end
end
end
def add_ranges! (address, candidates)
number = address.number.to_i
fids = unique_values candidates, :fid
ranges = ranges_by_feature fids, number, address.prenum
ranges = ranges_by_feature fids, number, nil unless !ranges.empty?
merge_rows! candidates, ranges, :fid
assign_number! number, candidates
end
def merge_edges! (candidates)
edge_ids = unique_values candidates, :tlid
records = edges edge_ids
merge_rows! candidates, records, :tlid
candidates.reject! {|record| record[:tlid].nil?}
edge_ids
end
def extend_ranges! (candidates)
edge_ids = merge_edges! candidates
full_ranges = range_ends edge_ids
merge_rows! candidates, full_ranges, :tlid, :side
end
# Score a list of candidates. For each candidate:
# * For each item in the query:
# ** if the query item is blank but the candidate is not, score 0.15;
# otherwise, if both are blank, score 1.0.
# ** If both items are set, compute the scaled Levenshtein-Damerau distance
# between them, and add that value (between 0.0 and 1.0) to the score.
# * Add 0.5 to the score for each numbered end of the range that matches
# the parity of the query number.
# * Add 1.0 if the query number is in the candidate range, otherwise
# add a fractional value for the notional distance between the
# closest candidate corner and the query.
# * Finally, divide the score by the total number of comparisons.
# The result should be between 0.0 and 1.0, with 1.0 indicating a
# perfect match.
def score_candidates! (address, candidates)
for candidate in candidates
candidate[:components] = {}
compare = [:prenum, :state, :zip]
denominator = compare.length + Street_Weight + City_Weight
street_score = (1.0 - candidate[:street_score].to_f) * Street_Weight
candidate[:components][:street] = street_score
city_score = (1.0 - candidate[:city_score].to_f) * City_Weight
candidate[:components][:city] = city_score
score = street_score + city_score
compare.each {|key|
src = address.send(key); src = src ? src.downcase : ""
dest = candidate[key]; dest = dest ? dest.downcase : ""
item_score = (src == dest) ? 1 : 0
candidate[:components][key] = item_score
score += item_score
}
if address.number and !address.number.empty?
parity = subscore = 0.0
fromhn, tohn, assigned, hn = [
candidate[:fromhn],
candidate[:tohn],
candidate[:number],
address.number].map {|s|s.to_i}
if candidate[:precision] == :range
subscore += Number_Weight
elsif assigned > 0
# only credit number subscore if assigned
subscore += Number_Weight/(assigned - hn).abs.to_f
end
candidate[:components][:number] = subscore
if hn > 0 and assigned > 0
# only credit parity if a number was given *and* assigned
parity += Parity_Weight/2.0 if fromhn % 2 == hn % 2
parity += Parity_Weight/2.0 if tohn % 2 == hn % 2
end
candidate[:components][:parity] = parity
score += subscore + parity
denominator += Number_Weight + Parity_Weight
end
candidate[:components][:total] = score.to_f
candidate[:components][:denominator] = denominator
candidate[:score] = score.to_f / denominator
end
end
# Find the candidates in a list of candidates that are tied for the
# top score and prune the remainder from the list.
def best_candidates! (candidates)
candidates.sort! {|a,b| b[:score] <=> a[:score]}
#candidates.reverse_each {|c| print "#{c[:number]} #{c[:state]} #{c[:city]} #{c[:raw_score]} #{c[:number_score]} #{c[:street_score]} #{c[:city_score]}\n" }
candidates.delete_if {|record| record[:score] < candidates[0][:score]}
end
# Compute the fractional interpolation distance for a query number along an
# edge, given all of the ranges for the same side of that edge.
def interpolation_distance (candidate)
fromhn, tohn, number = candidate.values_at(:fromhn, :tohn, :number).map{|x| x.to_i}
$stderr.print "NUM : #{fromhn} < #{number} < #{tohn} (flipped? #{candidate[:flipped]})\n" if @debug
# don't need this anymore since range_ends was improved...
fromhn, tohn = tohn, fromhn if fromhn > tohn
if fromhn > number
0.0
elsif tohn < number
1.0
elsif tohn == fromhn
# this is not supposed to happen, per Census Bureau rules, but apparently it does anyway.
0.0
else
(number - fromhn) / (tohn - fromhn).to_f
end
end
# Unpack an array of little-endian 4-byte ints, and convert them into
# signed floats by dividing by 10^6, inverting the process used by the
# compress_wkb_line() function in the SQLite helper extension.
def unpack_geometry (geom)
points = []
if !geom.nil?
if @dbtype == 2
# For special case?
#| 1 byte Type | 4 byte SRID | 4 byte element count| 8 byte double coordinates *
info = geom.unpack('CVVD*')
coords = info.slice(3, info.length)
else
#old format
coords = geom.unpack "V*" # little-endian 4-byte long ints
## now map them into signed floats
coords.map! {|i| ( i > (1 << 31) ? i - (1 << 32) : i ) / 1_000_000.0}
end
points << [coords.shift, coords.shift] until coords.empty?
end
points
end
# Calculate the longitude scaling for the average of two latitudes.
def scale_lon (lat1,lat2)
# an approximation in place of lookup.rst (10e) and (10g)
# = scale longitude distances by the cosine of the latitude
# (or, actually, the mean of two latitudes)
# -- is this even necessary?
Math.cos((lat1+lat2) / 2 * Math::PI / 180)
end
# Simple Euclidean distances between two 2-D coordinate pairs, scaled
# along the longitudinal axis by scale_lon.
def distance (a, b)
dx = (b[0] - a[0]) * scale_lon(a[1], b[1])
dy = (b[1] - a[1])
Math.sqrt(dx ** 2 + dy ** 2)
end
def street_side_offset (b, p1, p2)
# Find a point (x2, y2) that is at a distance b from a line A=(x0, y0)-(x1, y1)
# along a tangent B passing through (x1,y1)-(x2,y2).
# Let a = the length of line A
a = Math.sqrt((p2[0]-p1[0])**2.0 + (p2[1]-p1[1])**2.0)
# theta is the angle between the line A and the x axis
theta = Math.atan2(p2[1]-p1[1], p2[0]-p1[0])
# Now lines A and B form a right triangle where the third vertex is (x2, y2).
# Let c = the length of the hypotenuse line C=(x0,y0)-(x2,y2)
c = Math.sqrt(a**2.0 + b**2.0)
# Now alpha is the angle between lines A and C.
alpha = Math.atan2(b, a)
# Therefore the difference between theta and alpha is the angle between C and
# the x axis. Since we know the length of C, the lengths of the two lines
# parallel to the axes that form a right triangle C, and hence the
# coordinates (x2, y2), fall out.
return [p1[0] + c * Math.cos(theta - alpha), p1[1] + c * Math.sin(theta - alpha)]
end
# Find an interpolated point along a list of linestring vertices
# proportional to the given fractional distance along the line.
# Offset is in degrees and defaults to ~8 meters.
def interpolate (points, fraction, side, offset=0.000075)
$stderr.print "POINTS: #{points.inspect}" if @debug
return points[0] if fraction == 0.0
return points[-1] if fraction == 1.0
total = 0.0
(1...points.length).each {|n| total += distance(points[n-1], points[n])}
target = total * fraction
for n in 1...points.length
next if points[n-1] == points[n] # because otherwise step==0 and dx/dy==NaN
step = distance(points[n-1], points[n])
if step < target
target -= step
else
scale = scale_lon(points[n][1], points[n-1][1])
dx = (points[n][0] - points[n-1][0]) * (target/step) * scale
dy = (points[n][1] - points[n-1][1]) * (target/step)
found = [points[n-1][0]+dx, points[n-1][1]+dy]
return street_side_offset(offset*side, points[n-1], found)
end
end
# in a pathological case, points[n-1] == points[n] for n==-1
# so *sigh* just forget interpolating and return points[-1]
return points[-1]
end
# Find and replace the city, state, and county information
# in a list of candidates with the primary place information
# for the ZIP codes in the candidate list.
def canonicalize_places! (candidates)
zips_used = unique_values(candidates, :zip)
pri_places = rows_to_h primary_places(zips_used), :zip
candidates.map! {|record|
current_places = pri_places[[record[:zip]]]
# FIXME: this should never happen!
return [] unless current_places
top_priority = current_places.map{|p| p[:priority]}.min
current_places.select {|p| p[:priority] == top_priority}.map {|p|
record.merge({
:city => p[:city],
:state => p[:state],
:fips_county => p[:fips_county]
})
}
}
candidates.flatten!
end
# Clean up a candidate record by formatting the score, replacing nil
# values with empty strings, and deleting artifacts from database
# queries.
def clean_record! (record)
record[:score] = format("%.3f", record[:score]).to_f \
unless record[:score].nil?
record.keys.each {|k| record[k] = "" if record[k].nil? } # clean up nils
record.delete :components unless @debug
record.delete_if {|k,v| k.is_a? Fixnum or
[:geometry, :side, :tlid, :fid, :fid1, :fid2, :street_phone,
:city_phone, :fromhn, :tohn, :paflag, :flipped, :street_score,
:city_score, :priority, :fips_class, :fips_place, :status].include? k}
end
def best_places (address, places, canonicalize=false)
return [] unless !places.empty?
score_candidates! address, places
best_candidates! places
canonicalize_places! places if canonicalize
# uniqify places
by_name = rows_to_h(places, :city, :state)
if !by_name.nil?
begin
by_name.values.each {|v|
v.sort! {|a,b|
a[:zip] <=> b[:zip]
}}
rescue
end
places = by_name.map {|k,v| v[0]}
places.each {|record| clean_record! record}
places.each {|record|
record[:precision] = (record[:zip] == address.zip ? :zip : :city)
}
end
places
end
# Given an Address object, return a list of possible geocodes by place
# name. If canonicalize is true, attempt to return the "primary" postal
# place name for the given city, state, or ZIP.
def geocode_place (address, canonicalize=false)
places = []
places = places_by_zip address.text, address.zip if !address.zip.empty? or !address.zip.nil?
places = places_by_city address.text, address.city_parts, address.state if places.empty?
best_places address, places, canonicalize
end
def geocode_intersection (address, canonical_place=false)
candidates = find_candidates address
return [] if candidates.empty?
return best_places(address, candidates, canonical_place) if candidates[0][:street].nil?
features = rows_to_h candidates, :fid
intersects = intersections_by_fid features.keys.flatten
intersects.map! {|record|
feat1, feat2 = record.values_at(:fid1, :fid2).map {|k| features[[k]][0]}
record.merge! feat1
record[:street1] = record.delete(:street)
record[:street2] = feat2[:street]
record[:lon], record[:lat] = unpack_geometry(record.delete(:point))[0]
record[:precision] = :intersection
record[:street_score] = (feat1[:street_score].to_f + feat2[:street_score].to_f)/2
record
}
#pp(intersects)
score_candidates! address, intersects
best_candidates! intersects
by_point = rows_to_h(intersects, :lon, :lat)
candidates = by_point.values.map {|records| records[0]}
canonicalize_places! candidates if canonical_place
candidates.each {|record| clean_record! record}
candidates
end
# Given an Address object, return a list of possible geocodes by address
# range interpolation. If canonicalize is true, attempt to return the
# "primary" street and place names, if they are different from the ones
# given.
def geocode_address (address, canonical_place=false)
candidates = find_candidates address
return [] if candidates.empty?
return best_places(address, candidates, canonical_place) if candidates[0][:street].nil?
score_candidates! address, candidates
best_candidates! candidates
#candidates.sort {|a,b| b[:score] <=> a[:score]}.each {|candidate|
add_ranges! address, candidates
score_candidates! address, candidates
#pp candidates.sort {|a,b| b[:score] <=> a[:score]}
best_candidates! candidates
# sometimes multiple fids match the same tlid
by_tlid = rows_to_h candidates, :tlid
candidates = by_tlid.values.map {|records| records[0]}
# if no number is assigned in the query, only return one
# result for each street/zip combo
if !address.number.empty?
extend_ranges! candidates
else
by_street = rows_to_h candidates, :street, :zip
candidates = by_street.values.map {|records| records[0]}
merge_edges! candidates
end
candidates.map {|record|
dist = interpolation_distance record
points = unpack_geometry record[:geometry]
side = (record[:side] == "R" ? 1 : -1)
if record[:flipped]
side *= -1
points.reverse!
end
$stderr.print "DIST: #{dist} FLIPPED: #{record[:flipped]} SIDE: #{side}\n" if @debug
found = interpolate points, dist, side
record[:lon], record[:lat] = found.map {|x| format("%.6f", x).to_f}
}
canonicalize_places! candidates if canonical_place
candidates.each {|record| clean_record! record}
candidates
end
public
# Geocode a given address or place name string. The max_penalty and cutoff
# arguments are passed to the Address parse functions. If canonicalize is
# true, attempt to return the "primary" street and place names, if they are
# different from the ones given.
#
# Returns possible candidate matches as a list of hashes.
#
# * The :lat and :lon values of each hash store the range-interpolated
# address coordinates as latitude and longitude in the WGS84 spheroid.
# * The :precision value may be one of :city, :zip, :street, or :range, in
# order of increasing precision.
# * The :score value will be a float between 0.0 and 1.0 representing
# the approximate "goodness" of the candidate match.
# * The other values in the hash will represent various structured
# components of the address and place name.
def geocode (info_to_geocode, canonical_place=false)
address = Address.new info_to_geocode
$stderr.print "ADDR: #{address.inspect}\n" if @debug
return [] if address.city.empty? and address.zip.empty?
results = []
start_time = Time.now if @debug
if address.po_box? and !address.zip.empty?
results = geocode_place address, canonical_place
end
if address.intersection? and !address.street.empty? and address.number.empty?
results = geocode_intersection address, canonical_place
end
if results.empty? and !address.street.empty?
results = geocode_address address, canonical_place
end
if results.empty?
results = geocode_place address, canonical_place
end
if @debug
runtime = format("%.3f", Time.now - start_time)
$stderr.print "DONE: #{runtime}s\n"
end
results
end
end
end
================================================
FILE: lib/geocoder/us/metaphone.rb
================================================
module Text # :nodoc:
module Metaphone
module Rules # :nodoc:all
# Metaphone rules. These are simply applied in order.
#
STANDARD = [
# Regexp, replacement
[ /([bcdfhjklmnpqrstvwxyz])\1+/,
'\1' ], # Remove doubled consonants except g.
# [PHP] remove c from regexp.
[ /^ae/, 'E' ],
[ /^[gkp]n/, 'N' ],
[ /^wr/, 'R' ],
[ /^x/, 'S' ],
[ /^wh/, 'W' ],
[ /mb$/, 'M' ], # [PHP] remove $ from regexp.
[ /(?!^)sch/, 'SK' ],
[ /th/, '0' ],
[ /t?ch|sh/, 'X' ],
[ /c(?=ia)/, 'X' ],
[ /[st](?=i[ao])/, 'X' ],
[ /s?c(?=[iey])/, 'S' ],
[ /[cq]/, 'K' ],
[ /dg(?=[iey])/, 'J' ],
[ /d/, 'T' ],
[ /g(?=h[^aeiou])/, '' ],
[ /gn(ed)?/, 'N' ],
[ /([^g]|^)g(?=[iey])/,
'\1J' ],
[ /g+/, 'K' ],
[ /ph/, 'F' ],
[ /([aeiou])h(?=\b|[^aeiou])/,
'\1' ],
[ /[wy](?![aeiou])/, '' ],
[ /z/, 'S' ],
[ /v/, 'F' ],
[ /(?!^)[aeiou]+/, '' ],
]
# The rules for the 'buggy' alternate implementation used by PHP etc.
#
BUGGY = STANDARD.dup
BUGGY[0] = [ /([bdfhjklmnpqrstvwxyz])\1+/, '\1' ]
BUGGY[6] = [ /mb/, 'M' ]
end
# Returns the Metaphone representation of a string. If the string contains
# multiple words, each word in turn is converted into its Metaphone
# representation. Note that only the letters A-Z are supported, so any
# language-specific processing should be done beforehand.
#
# If the :buggy option is set, alternate 'buggy' rules are used.
#
def metaphone(str, options={})
return str.strip.split(/\s+/).map { |w| metaphone_word(w, options) }.join(' ')
end
private
def metaphone_word(w, options={})
# Normalise case and remove non-ASCII
s = w.downcase.gsub(/[^a-z]/, '')
# Apply the Metaphone rules
rules = options[:buggy] ? Rules::BUGGY : Rules::STANDARD
rules.each { |rx, rep| s.gsub!(rx, rep) }
return s.upcase
end
extend self
end
end
================================================
FILE: lib/geocoder/us/numbers.rb
================================================
module Geocoder
end
module Geocoder::US
# The NumberMap class provides a means for mapping ordinal
# and cardinal number words to digits and back.
class NumberMap < Hash
attr_accessor :regexp
def self.[] (array)
nmap = self.new({})
array.each {|item| nmap << item }
nmap.build_match
nmap
end
def initialize (array)
@count = 0
end
def build_match
@regexp = Regexp.new(
'\b(' + keys.flatten.join("|") + ')\b',
Regexp::IGNORECASE)
end
def clean (key)
key.is_a?(String) ? key.downcase.gsub(/\W/o, "") : key
end
def <<(item)
store clean(item), @count
store @count, item
@count += 1
end
def [] (key)
super(clean(key))
end
end
# The Cardinals constant maps digits to cardinal number words and back.
Cardinals = NumberMap[%w[
zero one two three four five six seven eight nine ten
eleven twelve thirteen fourteen fifteen sixteen seventeen
eighteen nineteen
]]
Cardinal_Tens = %w[ twenty thirty forty fifty sixty seventy eighty ninety ]
Cardinal_Tens.each {|tens|
Cardinals << tens
(1..9).each {|n| Cardinals << tens + "-" + Cardinals[n]}
}
# The Ordinals constant maps digits to ordinal number words and back.
Ordinals = NumberMap[%w[
zeroth first second third fourth fifth sixth seventh eighth ninth
tenth eleventh twelfth thirteenth fourteenth fifteenth sixteenth
seventeenth eighteenth nineteenth
]]
Cardinal_Tens.each {|tens|
Ordinals << tens.gsub("y","ieth")
(1..9).each {|n| Ordinals << tens + "-" + Ordinals[n]}
}
end
================================================
FILE: lib/geocoder/us/rest.rb
================================================
require 'rubygems'
require 'sinatra'
require 'geocoder/us/database'
require 'json'
@@db = Geocoder::US::Database.new(ENV["GEOCODER_DB"] || ARGV[0])
set :port, 8081
get '/geocode.?:format?' do
if params[:q]
results = @@db.geocode params[:q].gsub(/\s+(and|at)\s+/i,' ')
@features = []
results.each do |result|
coords = [result.delete(:lon), result.delete(:lat)]
result.keys.each do |key|
if result[key].is_a? String
result[key] = result[key].unpack("C*").pack("U*") # utf8
end
end
@features << {
:type => "Feature",
:properties => result,
:geometry => {
:type => "Point",
:coordinates => coords
}
}
end
case params[:format]
when /json/
begin
{
:type => "FeatureCollection",
:address => params[:q],
:features => @features
}.to_json
rescue JSON::GeneratorError
{
:type => "FeatureCollection",
:error => "JSON::GeneratorError",
:features => []
}.to_json
end
else
haml :index
end
else
status 400
"parameter 'q' is missing"
end
end
get '/health' do
"All is well."
end
def radius_for_precision(precision)
case precision
when /range/
50
else
200
end
end
__END__
@@ layout
%html
%head
%link(rel="stylesheet" href="http://leaflet.cloudmade.com/dist/leaflet.css")
%script(src="http://leaflet.cloudmade.com/dist/leaflet.js")
%body
= yield
@@ index
%div#map(style="height:400px")
%div
%h2 Features
%table{:border => "1", :cellspacing => "0", :cellpadding => "4"}
%tr
- @features.first[:properties].each do |key,property|
%th= key
- @features.each do |feature|
%tr
- feature[:properties].each do |key,property|
%td= property
%script
var features = []
- @features.each do |feature|
= "features.push(#{feature.to_json})"
:javascript
var map = new L.Map('map');
var cloudmadeUrl = 'http://acetate.geoiq.com/tiles/acetate/{z}/{x}/{y}.png',
cloudmadeAttrib = 'Map data © 2011 OpenStreetMap contributors, Style © 2011 GeoIQ',
cloudmade = new L.TileLayer(cloudmadeUrl, {maxZoom: 18, attribution: cloudmadeAttrib});
var center = new L.LatLng(#{@features.first[:geometry][:coordinates][1]}, #{@features.first[:geometry][:coordinates][0]});
map.setView(center, 13).addLayer(cloudmade);
circleOptions = {
color: 'red',
fillColor: '#f03',
fillOpacity: 0.5
};
var circleLocation, circle;
for(var i=0; i<features.length;i++) {
circleOptions.fillOpacity = features[i].properties.score;
circleLocation = new L.LatLng(features[i].geometry.coordinates[1],features[i].geometry.coordinates[0]),
circle = new L.Circle(circleLocation, 200, circleOptions);
map.addLayer(circle);
}
================================================
FILE: lib/geocoder/us.rb
================================================
require "geocoder/us/database"
require "geocoder/us/address"
# Imports the Geocoder::US::Database and Geocoder::US::Address
# modules.
#
# General usage is as follows:
#
# >> require 'geocoder/us'
# >> db = Geocoder::US::Database.new("/opt/tiger/geocoder.db")
# >> p db.geocode("1600 Pennsylvania Av, Washington DC")
#
# [{:pretyp=>"", :street=>"Pennsylvania", :sufdir=>"NW", :zip=>"20502",
# :lon=>-77.037528, :number=>"1600", :fips_county=>"11001", :predir=>"",
# :precision=>:range, :city=>"Washington", :lat=>38.898746, :suftyp=>"Ave",
# :state=>"DC", :prequal=>"", :sufqual=>"", :score=>0.906, :prenum=>""}]
#
# See Geocoder::US::Database and README.txt for more details.
module Geocoder::US
VERSION = "2.0.0"
end
================================================
FILE: navteq/README
================================================
The navteq_import script in this directory is designed to be used with Navteq's
local_streets layer. It works basically like tiger_import, except that you
provide either a list of .zip files containing the local_streets.* files on the
command line, or via standard input.
================================================
FILE: navteq/convert.sql
================================================
BEGIN;
CREATE INDEX navteq_link_id on local_streets (link_id);
CREATE TEMPORARY TABLE linezip AS
SELECT DISTINCT tlid, zip FROM (
SELECT link_id AS tlid, r_postcode AS zip FROM local_streets
WHERE addr_type IS NOT NULL AND st_name IS NOT NULL
AND r_postcode IS NOT NULL
UNION
SELECT link_id AS tlid, l_postcode AS zip FROM local_streets
WHERE addr_type IS NOT NULL AND st_name IS NOT NULL
AND l_postcode IS NOT NULL
) AS whatever;
INSERT INTO feature
SELECT l.tlid, st_nm_base, metaphone(st_nm_base,5), st_nm_pref, st_typ_bef,
NULL, st_nm_suff, st_typ_aft, NULL, 'P', zip
FROM linezip l, local_streets f
WHERE l.tlid=f.link_id AND st_name IS NOT NULL;
INSERT OR IGNORE INTO edge
SELECT l.tlid, compress_wkb_line(the_geom) FROM
(SELECT DISTINCT tlid FROM linezip) AS l, local_streets f
WHERE l.tlid=f.link_id AND st_name IS NOT NULL;
INSERT INTO range
SELECT link_id, digit_suffix(l_refaddr), digit_suffix(l_nrefaddr),
nondigit_prefix(l_refaddr), l_postcode, 'L'
FROM linezip l, local_streets f
WHERE l.tlid=f.link_id AND l_refaddr IS NOT NULL
UNION
SELECT link_id, digit_suffix(r_refaddr), digit_suffix(r_nrefaddr),
nondigit_prefix(r_refaddr), r_postcode, 'R'
FROM linezip l, local_streets f
WHERE l.tlid=f.link_id AND r_refaddr IS NOT NULL;
END;
================================================
FILE: navteq/navteq_import
================================================
#!/bin/bash
TMP="/tmp/navteq-import.$$"
SHPS="local_streets"
DBFS=""
BASE=$(dirname $0)
PATH=$PATH:$BASE/../bin
SQL="$BASE/../sql"
HELPER_LIB="$BASE/../lib/geocoder/us/sqlite3.so"
DATABASE=$1
shift
mkdir -p $TMP || exit 1
[ ! -r $DATABASE ] && cat ${SQL}/create.sql ${SQL}/place.sql | sqlite3 $DATABASE
if [ x"$1" = x"" ]; then
cat
else
ls $@
fi | while read county; do
echo "--- $county"
if [ -r ${county%.zip}.zip ]; then
unzip -q $(ls ${county}.zip) -d $TMP
else
cp ${county%.*}.* $TMP
fi
(echo ".load $HELPER_LIB" && \
cat ${BASE}/prepare.sql && \
for file in $SHPS; do
shp2sqlite -aS $(ls ${TMP}/${file}.shp) ${file}
done && \
for file in $DBFS; do
shp2sqlite -an $(ls ${TMP}/${file}.dbf) ${file}
done && \
cat ${BASE}/convert.sql) | sqlite3 $DATABASE
rm -f $TMP/*
done 2>&1 | tee import-$$.log
rm -rf $TMP
================================================
FILE: navteq/prepare.sql
================================================
PRAGMA temp_store=MEMORY;
PRAGMA journal_mode=MEMORY;
PRAGMA synchronous=OFF;
PRAGMA cache_size=250000;
PRAGMA count_changes=0;
BEGIN;
CREATE TABLE "local_streets" (gid integer PRIMARY KEY,
"the_geom" blob,
"link_id" integer,
"st_name" varchar(80),
"feat_id" integer,
"st_langcd" varchar(3),
"num_stnmes" integer,
"st_nm_pref" varchar(2),
"st_typ_bef" varchar(30),
"st_nm_base" varchar(35),
"st_nm_suff" varchar(2),
"st_typ_aft" varchar(30),
"st_typ_att" varchar(1),
"addr_type" varchar(1),
"l_refaddr" varchar(10),
"l_nrefaddr" varchar(10),
"l_addrsch" varchar(1),
"l_addrform" varchar(1),
"r_refaddr" varchar(10),
"r_nrefaddr" varchar(10),
"r_addrsch" varchar(1),
"r_addrform" varchar(1),
"ref_in_id" integer,
"nref_in_id" integer,
"n_shapepnt" integer,
"func_class" varchar(1),
"speed_cat" varchar(1),
"fr_spd_lim" integer,
"to_spd_lim" integer,
"to_lanes" integer,
"from_lanes" integer,
"enh_geom" varchar(1),
"lane_cat" varchar(1),
"divider" varchar(1),
"dir_travel" varchar(1),
"l_area_id" integer,
"r_area_id" integer,
"l_postcode" varchar(11),
"r_postcode" varchar(11),
"l_numzones" integer,
"r_numzones" integer,
"num_ad_rng" integer,
"ar_auto" varchar(1),
"ar_bus" varchar(1),
"ar_taxis" varchar(1),
"ar_carpool" varchar(1),
"ar_pedest" varchar(1),
"ar_trucks" varchar(1),
"ar_traff" varchar(1),
"ar_deliv" varchar(1),
"ar_emerveh" varchar(1),
"paved" varchar(1),
"private" varchar(1),
"frontage" varchar(1),
"bridge" varchar(1),
"tunnel" varchar(1),
"ramp" varchar(1),
"tollway" varchar(1),
"poiaccess" varchar(1),
"contracc" varchar(1),
"roundabout" varchar(1),
"interinter" varchar(1),
"undeftraff" varchar(1),
"ferry_type" varchar(1),
"multidigit" varchar(1),
"maxattr" varchar(1),
"spectrfig" varchar(1),
"indescrib" varchar(1),
"manoeuvre" varchar(1),
"dividerleg" varchar(1),
"inprocdata" varchar(1),
"full_geom" varchar(1),
"urban" varchar(1),
"route_type" varchar(1),
"dironsign" varchar(1),
"explicatbl" varchar(1),
"nameonrdsn" varchar(1),
"postalname" varchar(1),
"stalename" varchar(1),
"vanityname" varchar(1),
"junctionnm" varchar(1),
"exitname" varchar(1),
"scenic_rt" varchar(1),
"scenic_nm" varchar(1));
--SELECT AddGeometryColumn('','local_streets','the_geom','-1','MULTILINESTRING',2);
END;
================================================
FILE: setup.rb
================================================
#
# setup.rb
#
# Copyright (c) 2000-2005 Minero Aoki
#
# This program is free software.
# You can distribute/modify this program under the terms of
# the GNU LGPL, Lesser General Public License version 2.1.
#
unless Enumerable.method_defined?(:map) # Ruby 1.4.6
module Enumerable
alias map collect
end
end
unless File.respond_to?(:read) # Ruby 1.6
def File.read(fname)
open(fname) {|f|
return f.read
}
end
end
unless Errno.const_defined?(:ENOTEMPTY) # Windows?
module Errno
class ENOTEMPTY
# We do not raise this exception, implementation is not needed.
end
end
end
def File.binread(fname)
open(fname, 'rb') {|f|
return f.read
}
end
# for corrupted Windows' stat(2)
def File.dir?(path)
File.directory?((path[-1,1] == '/') ? path : path + '/')
end
class ConfigTable
include Enumerable
def initialize(rbconfig)
@rbconfig = rbconfig
@items = []
@table = {}
# options
@install_prefix = nil
@config_opt = nil
@verbose = true
@no_harm = false
end
attr_accessor :install_prefix
attr_accessor :config_opt
attr_writer :verbose
def verbose?
@verbose
end
attr_writer :no_harm
def no_harm?
@no_harm
end
def [](key)
lookup(key).resolve(self)
end
def []=(key, val)
lookup(key).set val
end
def names
@items.map {|i| i.name }
end
def each(&block)
@items.each(&block)
end
def key?(name)
@table.key?(name)
end
def lookup(name)
@table[name] or setup_rb_error "no such config item: #{name}"
end
def add(item)
@items.push item
@table[item.name] = item
end
def remove(name)
item = lookup(name)
@items.delete_if {|i| i.name == name }
@table.delete_if {|name, i| i.name == name }
item
end
def load_script(path, inst = nil)
if File.file?(path)
MetaConfigEnvironment.new(self, inst).instance_eval File.read(path), path
end
end
def savefile
'.config'
end
def load_savefile
begin
File.foreach(savefile()) do |line|
k, v = *line.split(/=/, 2)
self[k] = v.strip
end
rescue Errno::ENOENT
setup_rb_error $!.message + "\n#{File.basename($0)} config first"
end
end
def save
@items.each {|i| i.value }
File.open(savefile(), 'w') {|f|
@items.each do |i|
f.printf "%s=%s\n", i.name, i.value if i.value? and i.value
end
}
end
def load_standard_entries
standard_entries(@rbconfig).each do |ent|
add ent
end
end
def standard_entries(rbconfig)
c = rbconfig
rubypath = File.join(c['bindir'], c['ruby_install_name'] + c['EXEEXT'])
major = c['MAJOR'].to_i
minor = c['MINOR'].to_i
teeny = c['TEENY'].to_i
version = "#{major}.#{minor}"
# ruby ver. >= 1.4.4?
newpath_p = ((major >= 2) or
((major == 1) and
((minor >= 5) or
((minor == 4) and (teeny >= 4)))))
if c['rubylibdir']
# V > 1.6.3
libruby = "#{c['prefix']}/lib/ruby"
librubyver = c['rubylibdir']
librubyverarch = c['archdir']
siteruby = c['sitedir']
siterubyver = c['sitelibdir']
siterubyverarch = c['sitearchdir']
elsif newpath_p
# 1.4.4 <= V <= 1.6.3
libruby = "#{c['prefix']}/lib/ruby"
librubyver = "#{c['prefix']}/lib/ruby/#{version}"
librubyverarch = "#{c['prefix']}/lib/ruby/#{version}/#{c['arch']}"
siteruby = c['sitedir']
siterubyver = "$siteruby/#{version}"
siterubyverarch = "$siterubyver/#{c['arch']}"
else
# V < 1.4.4
libruby = "#{c['prefix']}/lib/ruby"
librubyver = "#{c['prefix']}/lib/ruby/#{version}"
librubyverarch = "#{c['prefix']}/lib/ruby/#{version}/#{c['arch']}"
siteruby = "#{c['prefix']}/lib/ruby/#{version}/site_ruby"
siterubyver = siteruby
siterubyverarch = "$siterubyver/#{c['arch']}"
end
parameterize = lambda {|path|
path.sub(/\A#{Regexp.quote(c['prefix'])}/, '$prefix')
}
if arg = c['configure_args'].split.detect {|arg| /--with-make-prog=/ =~ arg }
makeprog = arg.sub(/'/, '').split(/=/, 2)[1]
else
makeprog = 'make'
end
[
ExecItem.new('installdirs', 'std/site/home',
'std: install under libruby; site: install under site_ruby; home: install under $HOME')\
{|val, table|
case val
when 'std'
table['rbdir'] = '$librubyver'
table['sodir'] = '$librubyverarch'
when 'site'
table['rbdir'] = '$siterubyver'
table['sodir'] = '$siterubyverarch'
when 'home'
setup_rb_error '$HOME was not set' unless ENV['HOME']
table['prefix'] = ENV['HOME']
table['rbdir'] = '$libdir/ruby'
table['sodir'] = '$libdir/ruby'
end
},
PathItem.new('prefix', 'path', c['prefix'],
'path prefix of target environment'),
PathItem.new('bindir', 'path', parameterize.call(c['bindir']),
'the directory for commands'),
PathItem.new('libdir', 'path', parameterize.call(c['libdir']),
'the directory for libraries'),
PathItem.new('datadir', 'path', parameterize.call(c['datadir']),
'the directory for shared data'),
PathItem.new('mandir', 'path', parameterize.call(c['mandir']),
'the directory for man pages'),
PathItem.new('sysconfdir', 'path', parameterize.call(c['sysconfdir']),
'the directory for system configuration files'),
PathItem.new('localstatedir', 'path', parameterize.call(c['localstatedir']),
'the directory for local state data'),
PathItem.new('libruby', 'path', libruby,
'the directory for ruby libraries'),
PathItem.new('librubyver', 'path', librubyver,
'the directory for standard ruby libraries'),
PathItem.new('librubyverarch', 'path', librubyverarch,
'the directory for standard ruby extensions'),
PathItem.new('siteruby', 'path', siteruby,
'the directory for version-independent aux ruby libraries'),
PathItem.new('siterubyver', 'path', siterubyver,
'the directory for aux ruby libraries'),
PathItem.new('siterubyverarch', 'path', siterubyverarch,
'the directory for aux ruby binaries'),
PathItem.new('rbdir', 'path', '$siterubyver',
'the directory for ruby scripts'),
PathItem.new('sodir', 'path', '$siterubyverarch',
'the directory for ruby extentions'),
PathItem.new('rubypath', 'path', rubypath,
'the path to set to #! line'),
ProgramItem.new('rubyprog', 'name', rubypath,
'the ruby program using for installation'),
ProgramItem.new('makeprog', 'name', makeprog,
'the make program to compile ruby extentions'),
SelectItem.new('shebang', 'all/ruby/never', 'ruby',
'shebang line (#!) editing mode'),
BoolItem.new('without-ext', 'yes/no', 'no',
'does not compile/install ruby extentions')
]
end
private :standard_entries
def load_multipackage_entries
multipackage_entries().each do |ent|
add ent
end
end
def multipackage_entries
[
PackageSelectionItem.new('with', 'name,name...', '', 'ALL',
'package names that you want to install'),
PackageSelectionItem.new('without', 'name,name...', '', 'NONE',
'package names that you do not want to install')
]
end
private :multipackage_entries
ALIASES = {
'std-ruby' => 'librubyver',
'stdruby' => 'librubyver',
'rubylibdir' => 'librubyver',
'archdir' => 'librubyverarch',
'site-ruby-common' => 'siteruby', # For backward compatibility
'site-ruby' => 'siterubyver', # For backward compatibility
'bin-dir' => 'bindir',
'bin-dir' => 'bindir',
'rb-dir' => 'rbdir',
'so-dir' => 'sodir',
'data-dir' => 'datadir',
'ruby-path' => 'rubypath',
'ruby-prog' => 'rubyprog',
'ruby' => 'rubyprog',
'make-prog' => 'makeprog',
'make' => 'makeprog'
}
def fixup
ALIASES.each do |ali, name|
@table[ali] = @table[name]
end
@items.freeze
@table.freeze
@options_re = /\A--(#{@table.keys.join('|')})(?:=(.*))?\z/
end
def parse_opt(opt)
m = @options_re.match(opt) or setup_rb_error "config: unknown option #{opt}"
m.to_a[1,2]
end
def dllext
@rbconfig['DLEXT']
end
def value_config?(name)
lookup(name).value?
end
class Item
def initialize(name, template, default, desc)
@name = name.freeze
@template = template
@value = default
@default = default
@description = desc
end
attr_reader :name
attr_reader :description
attr_accessor :default
alias help_default default
def help_opt
"--#{@name}=#{@template}"
end
def value?
true
end
def value
@value
end
def resolve(table)
@value.gsub(%r<\$([^/]+)>) { table[$1] }
end
def set(val)
@value = check(val)
end
private
def check(val)
setup_rb_error "config: --#{name} requires argument" unless val
val
end
end
class BoolItem < Item
def config_type
'bool'
end
def help_opt
"--#{@name}"
end
private
def check(val)
return 'yes' unless val
case val
when /\Ay(es)?\z/i, /\At(rue)?\z/i then 'yes'
when /\An(o)?\z/i, /\Af(alse)\z/i then 'no'
else
setup_rb_error "config: --#{@name} accepts only yes/no for argument"
end
end
end
class PathItem < Item
def config_type
'path'
end
private
def check(path)
setup_rb_error "config: --#{@name} requires argument" unless path
path[0,1] == '$' ? path : File.expand_path(path)
end
end
class ProgramItem < Item
def config_type
'program'
end
end
class SelectItem < Item
def initialize(name, selection, default, desc)
super
@ok = selection.split('/')
end
def config_type
'select'
end
private
def check(val)
unless @ok.include?(val.strip)
setup_rb_error "config: use --#{@name}=#{@template} (#{val})"
end
val.strip
end
end
class ExecItem < Item
def initialize(name, selection, desc, &block)
super name, selection, nil, desc
@ok = selection.split('/')
@action = block
end
def config_type
'exec'
end
def value?
false
end
def resolve(table)
setup_rb_error "$#{name()} wrongly used as option value"
end
undef set
def evaluate(val, table)
v = val.strip.downcase
unless @ok.include?(v)
setup_rb_error "invalid option --#{@name}=#{val} (use #{@template})"
end
@action.call v, table
end
end
class PackageSelectionItem < Item
def initialize(name, template, default, help_default, desc)
super name, template, default, desc
@help_default = help_default
end
attr_reader :help_default
def config_type
'package'
end
private
def check(val)
unless File.dir?("packages/#{val}")
setup_rb_error "config: no such package: #{val}"
end
val
end
end
class MetaConfigEnvironment
def initialize(config, installer)
@config = config
@installer = installer
end
def config_names
@config.names
end
def config?(name)
@config.key?(name)
end
def bool_config?(name)
@config.lookup(name).config_type == 'bool'
end
def path_config?(name)
@config.lookup(name).config_type == 'path'
end
def value_config?(name)
@config.lookup(name).config_type != 'exec'
end
def add_config(item)
@config.add item
end
def add_bool_config(name, default, desc)
@config.add BoolItem.new(name, 'yes/no', default ? 'yes' : 'no', desc)
end
def add_path_config(name, default, desc)
@config.add PathItem.new(name, 'path', default, desc)
end
def set_config_default(name, default)
@config.lookup(name).default = default
end
def remove_config(name)
@config.remove(name)
end
# For only multipackage
def packages
raise '[setup.rb fatal] multi-package metaconfig API packages() called for single-package; contact application package vendor' unless @installer
@installer.packages
end
# For only multipackage
def declare_packages(list)
raise '[setup.rb fatal] multi-package metaconfig API declare_packages() called for single-package; contact application package vendor' unless @installer
@installer.packages = list
end
end
end # class ConfigTable
# This module requires: #verbose?, #no_harm?
module FileOperations
def mkdir_p(dirname, prefix = nil)
dirname = prefix + File.expand_path(dirname) if prefix
$stderr.puts "mkdir -p #{dirname}" if verbose?
return if no_harm?
# Does not check '/', it's too abnormal.
dirs = File.expand_path(dirname).split(%r<(?=/)>)
if /\A[a-z]:\z/i =~ dirs[0]
disk = dirs.shift
dirs[0] = disk + dirs[0]
end
dirs.each_index do |idx|
path = dirs[0..idx].join('')
Dir.mkdir path unless File.dir?(path)
end
end
def rm_f(path)
$stderr.puts "rm -f #{path}" if verbose?
return if no_harm?
force_remove_file path
end
def rm_rf(path)
$stderr.puts "rm -rf #{path}" if verbose?
return if no_harm?
remove_tree path
end
def remove_tree(path)
if File.symlink?(path)
remove_file path
elsif File.dir?(path)
remove_tree0 path
else
force_remove_file path
end
end
def remove_tree0(path)
Dir.foreach(path) do |ent|
next if ent == '.'
next if ent == '..'
entpath = "#{path}/#{ent}"
if File.symlink?(entpath)
remove_file entpath
elsif File.dir?(entpath)
remove_tree0 entpath
else
force_remove_file entpath
end
end
begin
Dir.rmdir path
rescue Errno::ENOTEMPTY
# directory may not be empty
end
end
def move_file(src, dest)
force_remove_file dest
begin
File.rename src, dest
rescue
File.open(dest, 'wb') {|f|
f.write File.binread(src)
}
File.chmod File.stat(src).mode, dest
File.unlink src
end
end
def force_remove_file(path)
begin
remove_file path
rescue
end
end
def remove_file(path)
File.chmod 0777, path
File.unlink path
end
def install(from, dest, mode, prefix = nil)
$stderr.puts "install #{from} #{dest}" if verbose?
return if no_harm?
realdest = prefix ? prefix + File.expand_path(dest) : dest
realdest = File.join(realdest, File.basename(from)) if File.dir?(realdest)
str = File.binread(from)
if diff?(str, realdest)
verbose_off {
rm_f realdest if File.exist?(realdest)
}
File.open(realdest, 'wb') {|f|
f.write str
}
File.chmod mode, realdest
File.open("#{objdir_root()}/InstalledFiles", 'a') {|f|
if prefix
f.puts realdest.sub(prefix, '')
else
f.puts realdest
end
}
end
end
def diff?(new_content, path)
return true unless File.exist?(path)
new_content != File.binread(path)
end
def command(*args)
$stderr.puts args.join(' ') if verbose?
system(*args) or raise RuntimeError,
"system(#{args.map{|a| a.inspect }.join(' ')}) failed"
end
def ruby(*args)
command config('rubyprog'), *args
end
def make(task = nil)
command(*[config('makeprog'), task].compact)
end
def extdir?(dir)
File.exist?("#{dir}/MANIFEST") or File.exist?("#{dir}/extconf.rb")
end
def files_of(dir)
Dir.open(dir) {|d|
return d.select {|ent| File.file?("#{dir}/#{ent}") }
}
end
DIR_REJECT = %w( . .. CVS SCCS RCS CVS.adm .svn )
def directories_of(dir)
Dir.open(dir) {|d|
return d.select {|ent| File.dir?("#{dir}/#{ent}") } - DIR_REJECT
}
end
end
# This module requires: #srcdir_root, #objdir_root, #relpath
module HookScriptAPI
def get_config(key)
@config[key]
end
alias config get_config
# obsolete: use metaconfig to change configuration
def set_config(key, val)
@config[key] = val
end
#
# srcdir/objdir (works only in the package directory)
#
def curr_srcdir
"#{srcdir_root()}/#{relpath()}"
end
def curr_objdir
"#{objdir_root()}/#{relpath()}"
end
def srcfile(path)
"#{curr_srcdir()}/#{path}"
end
def srcexist?(path)
File.exist?(srcfile(path))
end
def srcdirectory?(path)
File.dir?(srcfile(path))
end
def srcfile?(path)
File.file?(srcfile(path))
end
def srcentries(path = '.')
Dir.open("#{curr_srcdir()}/#{path}") {|d|
return d.to_a - %w(. ..)
}
end
def srcfiles(path = '.')
srcentries(path).select {|fname|
File.file?(File.join(curr_srcdir(), path, fname))
}
end
def srcdirectories(path = '.')
srcentries(path).select {|fname|
File.dir?(File.join(curr_srcdir(), path, fname))
}
end
end
class ToplevelInstaller
Version = '3.4.1'
Copyright = 'Copyright (c) 2000-2005 Minero Aoki'
TASKS = [
[ 'all', 'do config, setup, then install' ],
[ 'config', 'saves your configurations' ],
[ 'show', 'shows current configuration' ],
[ 'setup', 'compiles ruby extentions and others' ],
[ 'install', 'installs files' ],
[ 'test', 'run all tests in test/' ],
[ 'clean', "does `make clean' for each extention" ],
[ 'distclean',"does `make distclean' for each extention" ]
]
def ToplevelInstaller.invoke
config = ConfigTable.new(load_rbconfig())
config.load_standard_entries
config.load_multipackage_entries if multipackage?
config.fixup
klass = (multipackage?() ? ToplevelInstallerMulti : ToplevelInstaller)
klass.new(File.dirname($0), config).invoke
end
def ToplevelInstaller.multipackage?
File.dir?(File.dirname($0) + '/packages')
end
def ToplevelInstaller.load_rbconfig
if arg = ARGV.detect {|arg| /\A--rbconfig=/ =~ arg }
ARGV.delete(arg)
load File.expand_path(arg.split(/=/, 2)[1])
$".push 'rbconfig.rb'
else
require 'rbconfig'
end
::Config::CONFIG
end
def initialize(ardir_root, config)
@ardir = File.expand_path(ardir_root)
@config = config
# cache
@valid_task_re = nil
end
def config(key)
@config[key]
end
def inspect
"#<#{self.class} #{__id__()}>"
end
def invoke
run_metaconfigs
case task = parsearg_global()
when nil, 'all'
parsearg_config
init_installers
exec_config
exec_setup
exec_install
else
case task
when 'config', 'test'
;
when 'clean', 'distclean'
@config.load_savefile if File.exist?(@config.savefile)
else
@config.load_savefile
end
__send__ "parsearg_#{task}"
init_installers
__send__ "exec_#{task}"
end
end
def run_metaconfigs
@config.load_script "#{@ardir}/metaconfig"
end
def init_installers
@installer = Installer.new(@config, @ardir, File.expand_path('.'))
end
#
# Hook Script API bases
#
def srcdir_root
@ardir
end
def objdir_root
'.'
end
def relpath
'.'
end
#
# Option Parsing
#
def parsearg_global
while arg = ARGV.shift
case arg
when /\A\w+\z/
setup_rb_error "invalid task: #{arg}" unless valid_task?(arg)
return arg
when '-q', '--quiet'
@config.verbose = false
when '--verbose'
@config.verbose = true
when '--help'
print_usage $stdout
exit 0
when '--version'
puts "#{File.basename($0)} version #{Version}"
exit 0
when '--copyright'
puts Copyright
exit 0
else
setup_rb_error "unknown global option '#{arg}'"
end
end
nil
end
def valid_task?(t)
valid_task_re() =~ t
end
def valid_task_re
@valid_task_re ||= /\A(?:#{TASKS.map {|task,desc| task }.join('|')})\z/
end
def parsearg_no_options
unless ARGV.empty?
task = caller(0).first.slice(%r<`parsearg_(\w+)'>, 1)
setup_rb_error "#{task}: unknown options: #{ARGV.join(' ')}"
end
end
alias parsearg_show parsearg_no_options
alias parsearg_setup parsearg_no_options
alias parsearg_test parsearg_no_options
alias parsearg_clean parsearg_no_options
alias parsearg_distclean parsearg_no_options
def parsearg_config
evalopt = []
set = []
@config.config_opt = []
while i = ARGV.shift
if /\A--?\z/ =~ i
@config.config_opt = ARGV.dup
break
end
name, value = *@config.parse_opt(i)
if @config.value_config?(name)
@config[name] = value
else
evalopt.push [name, value]
end
set.push name
end
evalopt.each do |name, value|
@config.lookup(name).evaluate value, @config
end
# Check if configuration is valid
set.each do |n|
@config[n] if @config.value_config?(n)
end
end
def parsearg_install
@config.no_harm = false
@config.install_prefix = ''
while a = ARGV.shift
case a
when '--no-harm'
@config.no_harm = true
when /\A--prefix=/
path = a.split(/=/, 2)[1]
path = File.expand_path(path) unless path[0,1] == '/'
@config.install_prefix = path
else
setup_rb_error "install: unknown option #{a}"
end
end
end
def print_usage(out)
out.puts 'Typical Installation Procedure:'
out.puts " $ ruby #{File.basename $0} config"
out.puts " $ ruby #{File.basename $0} setup"
out.puts " # ruby #{File.basename $0} install (may require root privilege)"
out.puts
out.puts 'Detailed Usage:'
out.puts " ruby #{File.basename $0} <global option>"
out.puts " ruby #{File.basename $0} [<global options>] <task> [<task options>]"
fmt = " %-24s %s\n"
out.puts
out.puts 'Global options:'
out.printf fmt, '-q,--quiet', 'suppress message outputs'
out.printf fmt, ' --verbose', 'output messages verbosely'
out.printf fmt, ' --help', 'print this message'
out.printf fmt, ' --version', 'print version and quit'
out.printf fmt, ' --copyright', 'print copyright and quit'
out.puts
out.puts 'Tasks:'
TASKS.each do |name, desc|
out.printf fmt, name, desc
end
fmt = " %-24s %s [%s]\n"
out.puts
out.puts 'Options for CONFIG or ALL:'
@config.each do |item|
out.printf fmt, item.help_opt, item.description, item.help_default
end
out.printf fmt, '--rbconfig=path', 'rbconfig.rb to load',"running ruby's"
out.puts
out.puts 'Options for INSTALL:'
out.printf fmt, '--no-harm', 'only display what to do if given', 'off'
out.printf fmt, '--prefix=path', 'install path prefix', ''
out.puts
end
#
# Task Handlers
#
def exec_config
@installer.exec_config
@config.save # must be final
end
def exec_setup
@installer.exec_setup
end
def exec_install
@installer.exec_install
end
def exec_test
@installer.exec_test
end
def exec_show
@config.each do |i|
printf "%-20s %s\n", i.name, i.value if i.value?
end
end
def exec_clean
@installer.exec_clean
end
def exec_distclean
@installer.exec_distclean
end
end # class ToplevelInstaller
class ToplevelInstallerMulti < ToplevelInstaller
include FileOperations
def initialize(ardir_root, config)
super
@packages = directories_of("#{@ardir}/packages")
raise 'no package exists' if @packages.empty?
@root_installer = Installer.new(@config, @ardir, File.expand_path('.'))
end
def run_metaconfigs
@config.load_script "#{@ardir}/metaconfig", self
@packages.each do |name|
@config.load_script "#{@ardir}/packages/#{name}/metaconfig"
end
end
attr_reader :packages
def packages=(list)
raise 'package list is empty' if list.empty?
list.each do |name|
raise "directory packages/#{name} does not exist"\
unless File.dir?("#{@ardir}/packages/#{name}")
end
@packages = list
end
def init_installers
@installers = {}
@packages.each do |pack|
@installers[pack] = Installer.new(@config,
"#{@ardir}/packages/#{pack}",
"packages/#{pack}")
end
with = extract_selection(config('with'))
without = extract_selection(config('without'))
@selected = @installers.keys.select {|name|
(with.empty? or with.include?(name)) \
and not without.include?(name)
}
end
def extract_selection(list)
a = list.split(/,/)
a.each do |name|
setup_rb_error "no such package: #{name}" unless @installers.key?(name)
end
a
end
def print_usage(f)
super
f.puts 'Inluded packages:'
f.puts ' ' + @packages.sort.join(' ')
f.puts
end
#
# Task Handlers
#
def exec_config
run_hook 'pre-config'
each_selected_installers {|inst| inst.exec_config }
run_hook 'post-config'
@config.save # must be final
end
def exec_setup
run_hook 'pre-setup'
each_selected_installers {|inst| inst.exec_setup }
run_hook 'post-setup'
end
def exec_install
run_hook 'pre-install'
each_selected_installers {|inst| inst.exec_install }
run_hook 'post-install'
end
def exec_test
run_hook 'pre-test'
each_selected_installers {|inst| inst.exec_test }
run_hook 'post-test'
end
def exec_clean
rm_f @config.savefile
run_hook 'pre-clean'
each_selected_installers {|inst| inst.exec_clean }
run_hook 'post-clean'
end
def exec_distclean
rm_f @config.savefile
run_hook 'pre-distclean'
each_selected_installers {|inst| inst.exec_distclean }
run_hook 'post-distclean'
end
#
# lib
#
def each_selected_installers
Dir.mkdir 'packages' unless File.dir?('packages')
@selected.each do |pack|
$stderr.puts
gitextract_g0bu7lk9/
├── .gitignore
├── History.txt
├── LICENSE.txt
├── Makefile
├── Manifest.txt
├── README.rdoc
├── REST.rdoc
├── TODO.txt
├── bin/
│ └── rebuild_metaphones
├── build/
│ ├── build_indexes
│ ├── rebuild_cluster
│ ├── sql/
│ │ ├── cluster.sql
│ │ ├── convert.sql
│ │ ├── create.sql
│ │ ├── index.sql
│ │ ├── place.sql
│ │ └── setup.sql
│ ├── tiger2009_import
│ └── tiger_import
├── conf/
│ ├── geocoder-us/
│ │ ├── geocoder.ru
│ │ └── unicorn.rb
│ └── init/
│ └── geocoder-us.conf
├── debian/
│ ├── README.Debian
│ ├── changelog
│ ├── compat
│ ├── control
│ ├── copyright
│ ├── default
│ ├── docs
│ ├── geocoder-us.postinst
│ ├── geocoder-us.prerm
│ ├── rules
│ └── source/
│ └── format
├── demos/
│ ├── api/
│ │ ├── server.rb
│ │ └── views/
│ │ └── index.erb
│ ├── cli.rb
│ ├── demo/
│ │ ├── app/
│ │ │ ├── ext/
│ │ │ │ └── geocodewrap.rb
│ │ │ └── views/
│ │ │ ├── index.builder
│ │ │ └── index.erb
│ │ ├── config/
│ │ │ ├── bootstraps.rb
│ │ │ └── geoenvironment.rb
│ │ ├── config.ru
│ │ ├── geocoder_helper.rb
│ │ ├── geocom_geocode.rb
│ │ ├── main.rb
│ │ ├── rakefile.rb
│ │ └── tmp/
│ │ └── restart.txt
│ ├── parse.rb
│ └── simpledemo/
│ ├── views/
│ │ ├── index.builder
│ │ └── index.erb
│ └── ws.rb
├── doc/
│ ├── Makefile
│ ├── html4css1.css
│ ├── lookup.rst
│ ├── parsing.rst
│ └── voidspace.css
├── gemspec
├── lib/
│ └── geocoder/
│ ├── us/
│ │ ├── address.rb
│ │ ├── constants.rb
│ │ ├── database.rb
│ │ ├── metaphone.rb
│ │ ├── numbers.rb
│ │ └── rest.rb
│ └── us.rb
├── navteq/
│ ├── README
│ ├── convert.sql
│ ├── navteq_import
│ └── prepare.sql
├── setup.rb
├── src/
│ ├── Makefile
│ ├── README
│ ├── liblwgeom/
│ │ ├── Makefile
│ │ ├── box2d.c
│ │ ├── lex.yy.c
│ │ ├── liblwgeom.h
│ │ ├── lwalgorithm.c
│ │ ├── lwalgorithm.h
│ │ ├── lwcircstring.c
│ │ ├── lwcollection.c
│ │ ├── lwcompound.c
│ │ ├── lwcurvepoly.c
│ │ ├── lwgeom.c
│ │ ├── lwgeom_api.c
│ │ ├── lwgparse.c
│ │ ├── lwgunparse.c
│ │ ├── lwline.c
│ │ ├── lwmcurve.c
│ │ ├── lwmline.c
│ │ ├── lwmpoint.c
│ │ ├── lwmpoly.c
│ │ ├── lwmsurface.c
│ │ ├── lwpoint.c
│ │ ├── lwpoly.c
│ │ ├── lwsegmentize.c
│ │ ├── lwutil.c
│ │ ├── measures.c
│ │ ├── postgis_config.h
│ │ ├── ptarray.c
│ │ ├── vsprintf.c
│ │ ├── wktparse.h
│ │ ├── wktparse.lex
│ │ ├── wktparse.tab.c
│ │ ├── wktparse.tab.h
│ │ └── wktparse.y
│ ├── libsqlite3_geocoder/
│ │ ├── Makefile
│ │ ├── Makefile.nix
│ │ ├── Makefile.redhat
│ │ ├── extension.c
│ │ ├── extension.h
│ │ ├── levenshtein.c
│ │ ├── metaphon.c
│ │ ├── util.c
│ │ └── wkb_compress.c
│ ├── metaphone/
│ │ ├── Makefile
│ │ ├── README
│ │ ├── extension.c
│ │ └── metaphon.c
│ └── shp2sqlite/
│ ├── Makefile
│ ├── Makefile.macosx
│ ├── Makefile.nix
│ ├── Makefile.redhat
│ ├── dbfopen.c
│ ├── getopt.c
│ ├── getopt.h
│ ├── shapefil.h
│ ├── shp2sqlite.c
│ └── shpopen.c
└── test/
├── address.rb
├── benchmark.rb
├── constants.rb
├── data/
│ ├── address-sample.csv
│ ├── db-test.csv
│ └── locations.csv
├── database.rb
├── generate.rb
├── numbers.rb
└── run.rb
SYMBOL INDEX (1019 symbols across 64 files)
FILE: build/sql/convert.sql
type featnames_tlid (line 3) | CREATE INDEX featnames_tlid ON tiger_featnames (tlid)
type addr_tlid (line 4) | CREATE INDEX addr_tlid ON tiger_addr (tlid)
type edges_tlid (line 5) | CREATE INDEX edges_tlid ON tiger_edges (tlid)
type linezip (line 9) | CREATE TEMPORARY TABLE linezip AS
type linezip_tlid (line 20) | CREATE INDEX linezip_tlid ON linezip (tlid)
type feature_bin (line 29) | CREATE TEMPORARY TABLE feature_bin (
type feature_bin_idx (line 46) | CREATE INDEX feature_bin_idx ON feature_bin (street, zip)
FILE: build/sql/create.sql
type place (line 3) | CREATE TABLE place(
type edge (line 16) | CREATE TABLE edge (
type feature (line 20) | CREATE TABLE feature (
type feature_edge (line 27) | CREATE TABLE feature_edge (
type range (line 31) | CREATE TABLE range (
FILE: build/sql/index.sql
type place_city_phone_state_idx (line 8) | CREATE INDEX place_city_phone_state_idx ON place (city_phone, state)
type place_zip_priority_idx (line 9) | CREATE INDEX place_zip_priority_idx ON place (zip, priority)
type feature_street_phone_zip_idx (line 10) | CREATE INDEX feature_street_phone_zip_idx ON feature (street_phone, zip)
type feature_edge_fid_idx (line 11) | CREATE INDEX feature_edge_fid_idx ON feature_edge (fid)
type range_tlid_idx (line 12) | CREATE INDEX range_tlid_idx ON range (tlid)
FILE: build/sql/setup.sql
type "tiger_edges" (line 12) | CREATE TEMPORARY TABLE "tiger_edges" (
type "tiger_featnames" (line 47) | CREATE TEMPORARY TABLE "tiger_featnames" (
type "tiger_addr" (line 66) | CREATE TEMPORARY TABLE "tiger_addr" (
FILE: demos/demo/app/ext/geocodewrap.rb
type Sinatra (line 5) | module Sinatra
type GeocodeWrap (line 6) | module GeocodeWrap
function registered (line 8) | def self.registered(app)
FILE: demos/demo/config/bootstraps.rb
type BootStraps (line 3) | module BootStraps
class Framework (line 5) | class Framework
method initialize (line 7) | def initialize
method apply_settings! (line 11) | def apply_settings!(app)
method method_missing (line 19) | def method_missing(method, *args)
class DataStore (line 26) | class DataStore
method connect_action (line 27) | def connect_action(&block)
method connect (line 32) | def connect
class Configuration (line 37) | class Configuration
method initialize (line 41) | def initialize
method env (line 51) | def env
method env= (line 55) | def env=(val)
method root (line 59) | def root
method gem (line 63) | def gem(*args)
method use_vendor (line 76) | def use_vendor
method prepend_gem_path! (line 81) | def prepend_gem_path!(path)
class Initializer (line 86) | class Initializer
method configure (line 89) | def configure
method config (line 96) | def config
method boot! (line 100) | def boot!
method require_libs (line 107) | def require_libs
method require_all (line 116) | def require_all(path)
method subdir_expansion (line 120) | def subdir_expansion(subdir)
FILE: demos/demo/geocoder_helper.rb
function initialize (line 8) | def initialize
FILE: demos/demo/geocom_geocode.rb
type GeocomGeocode (line 3) | module GeocomGeocode
class GeocodeServer (line 4) | class GeocodeServer < Sinatra::Base
FILE: lib/geocoder/us.rb
type Geocoder::US (line 19) | module Geocoder::US
FILE: lib/geocoder/us/address.rb
type Geocoder::US (line 3) | module Geocoder::US
class Address (line 19) | class Address
method initialize (line 28) | def initialize(text)
method clean (line 40) | def clean(value)
method assign_text_to_address (line 47) | def assign_text_to_address(text)
method expand_numbers (line 103) | def expand_numbers (string)
method parse_zip (line 125) | def parse_zip(regex_match, text)
method parse_state (line 133) | def parse_state(regex_match, text)
method parse_number (line 142) | def parse_number(regex_match, text)
method parse (line 151) | def parse
method expand_streets (line 202) | def expand_streets(street)
method street_parts (line 219) | def street_parts
method remove_noise_words (line 234) | def remove_noise_words(strings)
method city_parts (line 255) | def city_parts
method city= (line 270) | def city= (strings)
method po_box? (line 281) | def po_box?
method intersection? (line 285) | def intersection?
FILE: lib/geocoder/us/constants.rb
type Geocoder (line 5) | module Geocoder
type Geocoder::US (line 8) | module Geocoder::US
class Map (line 9) | class Map < Hash
method [] (line 14) | def self.[] (*items)
method build_partial (line 24) | def build_partial
method build_match (line 31) | def build_match
method partial? (line 38) | def partial? (key)
method key? (line 41) | def key? (key)
method [] (line 44) | def [] (key)
FILE: lib/geocoder/us/database.rb
type Geocoder (line 14) | module Geocoder
type Geocoder::US (line 17) | module Geocoder::US
class Database (line 19) | class Database
method initialize (line 36) | def initialize (filename, options = {})
method synchronize (line 51) | def synchronize
method tune (line 63) | def tune (helper, cache_size)
method prepare (line 106) | def prepare (sql)
method flush_statements (line 118) | def flush_statements
method placeholders_for (line 123) | def placeholders_for (list)
method metaphone_placeholders_for (line 128) | def metaphone_placeholders_for (list)
method execute (line 134) | def execute (sql, *params)
method execute_statement (line 146) | def execute_statement (st, *params)
method places_by_zip (line 166) | def places_by_zip (city, zip)
method places_by_city (line 174) | def places_by_city (city, tokens, state)
method features_by_street (line 194) | def features_by_street (street, tokens)
method features_by_street_and_zip (line 208) | def features_by_street_and_zip (street, tokens, zips)
method more_features_by_street_and_zip (line 220) | def more_features_by_street_and_zip (street, tokens, zips)
method ranges_by_feature (line 232) | def ranges_by_feature (fids, number, prenum)
method edges (line 253) | def edges (edge_ids)
method range_ends (line 261) | def range_ends (edge_ids)
method intersections_by_fid (line 282) | def intersections_by_fid (fids)
method primary_places (line 323) | def primary_places (zips)
method unique_values (line 330) | def unique_values (rows, key)
method rows_to_h (line 335) | def rows_to_h (rows, *keys)
method merge_rows! (line 344) | def merge_rows! (dest, src, *keys)
method find_candidates (line 357) | def find_candidates (address)
method assign_number! (line 389) | def assign_number! (hn, candidates)
method add_ranges! (line 404) | def add_ranges! (address, candidates)
method merge_edges! (line 413) | def merge_edges! (candidates)
method extend_ranges! (line 421) | def extend_ranges! (candidates)
method score_candidates! (line 441) | def score_candidates! (address, candidates)
method best_candidates! (line 492) | def best_candidates! (candidates)
method interpolation_distance (line 500) | def interpolation_distance (candidate)
method unpack_geometry (line 520) | def unpack_geometry (geom)
method scale_lon (line 540) | def scale_lon (lat1,lat2)
method distance (line 550) | def distance (a, b)
method street_side_offset (line 556) | def street_side_offset (b, p1, p2)
method interpolate (line 578) | def interpolate (points, fraction, side, offset=0.000075)
method canonicalize_places! (line 606) | def canonicalize_places! (candidates)
method clean_record! (line 628) | def clean_record! (record)
method best_places (line 639) | def best_places (address, places, canonicalize=false)
method geocode_place (line 669) | def geocode_place (address, canonicalize=false)
method geocode_intersection (line 676) | def geocode_intersection (address, canonical_place=false)
method geocode_address (line 710) | def geocode_address (address, canonical_place=false)
method geocode (line 773) | def geocode (info_to_geocode, canonical_place=false)
FILE: lib/geocoder/us/metaphone.rb
type Text (line 1) | module Text # :nodoc:
type Metaphone (line 2) | module Metaphone
type Rules (line 4) | module Rules # :nodoc:all
function metaphone (line 56) | def metaphone(str, options={})
function metaphone_word (line 62) | def metaphone_word(w, options={})
FILE: lib/geocoder/us/numbers.rb
type Geocoder (line 1) | module Geocoder
type Geocoder::US (line 4) | module Geocoder::US
class NumberMap (line 7) | class NumberMap < Hash
method [] (line 9) | def self.[] (array)
method initialize (line 15) | def initialize (array)
method build_match (line 18) | def build_match
method clean (line 23) | def clean (key)
method << (line 26) | def <<(item)
method [] (line 31) | def [] (key)
FILE: lib/geocoder/us/rest.rb
function radius_for_precision (line 57) | def radius_for_precision(precision)
FILE: navteq/convert.sql
type navteq_link_id (line 2) | CREATE INDEX navteq_link_id on local_streets (link_id)
type linezip (line 4) | CREATE TEMPORARY TABLE linezip AS
FILE: navteq/prepare.sql
type "local_streets" (line 7) | CREATE TABLE "local_streets" (gid integer PRIMARY KEY,
FILE: setup.rb
type Enumerable (line 12) | module Enumerable
function read (line 18) | def File.read(fname)
type Errno (line 26) | module Errno
class ENOTEMPTY (line 27) | class ENOTEMPTY
function binread (line 33) | def File.binread(fname)
function dir? (line 40) | def File.dir?(path)
class ConfigTable (line 45) | class ConfigTable
method initialize (line 49) | def initialize(rbconfig)
method verbose? (line 65) | def verbose?
method no_harm? (line 71) | def no_harm?
method [] (line 75) | def [](key)
method []= (line 79) | def []=(key, val)
method names (line 83) | def names
method each (line 87) | def each(&block)
method key? (line 91) | def key?(name)
method lookup (line 95) | def lookup(name)
method add (line 99) | def add(item)
method remove (line 104) | def remove(name)
method load_script (line 111) | def load_script(path, inst = nil)
method savefile (line 117) | def savefile
method load_savefile (line 121) | def load_savefile
method save (line 132) | def save
method load_standard_entries (line 141) | def load_standard_entries
method standard_entries (line 147) | def standard_entries(rbconfig)
method load_multipackage_entries (line 260) | def load_multipackage_entries
method multipackage_entries (line 266) | def multipackage_entries
method fixup (line 295) | def fixup
method parse_opt (line 304) | def parse_opt(opt)
method dllext (line 309) | def dllext
method value_config? (line 313) | def value_config?(name)
class Item (line 317) | class Item
method initialize (line 318) | def initialize(name, template, default, desc)
method help_opt (line 332) | def help_opt
method value? (line 336) | def value?
method value (line 340) | def value
method resolve (line 344) | def resolve(table)
method set (line 348) | def set(val)
method check (line 354) | def check(val)
class BoolItem (line 360) | class BoolItem < Item
method config_type (line 361) | def config_type
method help_opt (line 365) | def help_opt
method check (line 371) | def check(val)
class PathItem (line 382) | class PathItem < Item
method config_type (line 383) | def config_type
method check (line 389) | def check(path)
class ProgramItem (line 395) | class ProgramItem < Item
method config_type (line 396) | def config_type
class SelectItem (line 401) | class SelectItem < Item
method initialize (line 402) | def initialize(name, selection, default, desc)
method config_type (line 407) | def config_type
method check (line 413) | def check(val)
class ExecItem (line 421) | class ExecItem < Item
method initialize (line 422) | def initialize(name, selection, desc, &block)
method config_type (line 428) | def config_type
method value? (line 432) | def value?
method resolve (line 436) | def resolve(table)
method evaluate (line 442) | def evaluate(val, table)
class PackageSelectionItem (line 451) | class PackageSelectionItem < Item
method initialize (line 452) | def initialize(name, template, default, help_default, desc)
method config_type (line 459) | def config_type
method check (line 465) | def check(val)
class MetaConfigEnvironment (line 473) | class MetaConfigEnvironment
method initialize (line 474) | def initialize(config, installer)
method config_names (line 479) | def config_names
method config? (line 483) | def config?(name)
method bool_config? (line 487) | def bool_config?(name)
method path_config? (line 491) | def path_config?(name)
method value_config? (line 495) | def value_config?(name)
method add_config (line 499) | def add_config(item)
method add_bool_config (line 503) | def add_bool_config(name, default, desc)
method add_path_config (line 507) | def add_path_config(name, default, desc)
method set_config_default (line 511) | def set_config_default(name, default)
method remove_config (line 515) | def remove_config(name)
method packages (line 520) | def packages
method declare_packages (line 526) | def declare_packages(list)
type FileOperations (line 536) | module FileOperations
function mkdir_p (line 538) | def mkdir_p(dirname, prefix = nil)
function rm_f (line 555) | def rm_f(path)
function rm_rf (line 561) | def rm_rf(path)
function remove_tree (line 567) | def remove_tree(path)
function remove_tree0 (line 577) | def remove_tree0(path)
function move_file (line 597) | def move_file(src, dest)
function force_remove_file (line 610) | def force_remove_file(path)
function remove_file (line 617) | def remove_file(path)
function install (line 622) | def install(from, dest, mode, prefix = nil)
function diff? (line 648) | def diff?(new_content, path)
function command (line 653) | def command(*args)
function ruby (line 659) | def ruby(*args)
function make (line 663) | def make(task = nil)
function extdir? (line 667) | def extdir?(dir)
function files_of (line 671) | def files_of(dir)
function directories_of (line 679) | def directories_of(dir)
type HookScriptAPI (line 689) | module HookScriptAPI
function get_config (line 691) | def get_config(key)
function set_config (line 698) | def set_config(key, val)
function curr_srcdir (line 706) | def curr_srcdir
function curr_objdir (line 710) | def curr_objdir
function srcfile (line 714) | def srcfile(path)
function srcexist? (line 718) | def srcexist?(path)
function srcdirectory? (line 722) | def srcdirectory?(path)
function srcfile? (line 726) | def srcfile?(path)
function srcentries (line 730) | def srcentries(path = '.')
function srcfiles (line 736) | def srcfiles(path = '.')
function srcdirectories (line 742) | def srcdirectories(path = '.')
class ToplevelInstaller (line 751) | class ToplevelInstaller
method invoke (line 767) | def ToplevelInstaller.invoke
method multipackage? (line 776) | def ToplevelInstaller.multipackage?
method load_rbconfig (line 780) | def ToplevelInstaller.load_rbconfig
method initialize (line 791) | def initialize(ardir_root, config)
method config (line 798) | def config(key)
method inspect (line 802) | def inspect
method invoke (line 806) | def invoke
method run_metaconfigs (line 830) | def run_metaconfigs
method init_installers (line 834) | def init_installers
method srcdir_root (line 842) | def srcdir_root
method objdir_root (line 846) | def objdir_root
method relpath (line 850) | def relpath
method parsearg_global (line 858) | def parsearg_global
method valid_task? (line 884) | def valid_task?(t)
method valid_task_re (line 888) | def valid_task_re
method parsearg_no_options (line 892) | def parsearg_no_options
method parsearg_config (line 905) | def parsearg_config
method parsearg_install (line 931) | def parsearg_install
method print_usage (line 948) | def print_usage(out)
method exec_config (line 990) | def exec_config
method exec_setup (line 995) | def exec_setup
method exec_install (line 999) | def exec_install
method exec_test (line 1003) | def exec_test
method exec_show (line 1007) | def exec_show
method exec_clean (line 1013) | def exec_clean
method exec_distclean (line 1017) | def exec_distclean
class ToplevelInstallerMulti (line 1024) | class ToplevelInstallerMulti < ToplevelInstaller
method initialize (line 1028) | def initialize(ardir_root, config)
method run_metaconfigs (line 1035) | def run_metaconfigs
method packages= (line 1044) | def packages=(list)
method init_installers (line 1053) | def init_installers
method extract_selection (line 1068) | def extract_selection(list)
method print_usage (line 1076) | def print_usage(f)
method exec_config (line 1087) | def exec_config
method exec_setup (line 1094) | def exec_setup
method exec_install (line 1100) | def exec_install
method exec_test (line 1106) | def exec_test
method exec_clean (line 1112) | def exec_clean
method exec_distclean (line 1119) | def exec_distclean
method each_selected_installers (line 1130) | def each_selected_installers
method run_hook (line 1141) | def run_hook(id)
method verbose? (line 1146) | def verbose?
method no_harm? (line 1151) | def no_harm?
class Installer (line 1158) | class Installer
method initialize (line 1165) | def initialize(config, srcroot, objroot)
method inspect (line 1172) | def inspect
method noop (line 1176) | def noop(rel)
method srcdir_root (line 1183) | def srcdir_root
method objdir_root (line 1187) | def objdir_root
method relpath (line 1191) | def relpath
method verbose? (line 1200) | def verbose?
method no_harm? (line 1205) | def no_harm?
method verbose_off (line 1209) | def verbose_off
method exec_config (line 1222) | def exec_config
method config_dir_ext (line 1229) | def config_dir_ext(rel)
method extconf (line 1237) | def extconf
method exec_setup (line 1245) | def exec_setup
method setup_dir_bin (line 1249) | def setup_dir_bin(rel)
method setup_dir_ext (line 1257) | def setup_dir_ext(rel)
method update_shebang_line (line 1265) | def update_shebang_line(path)
method new_shebang (line 1287) | def new_shebang(old)
method open_atomic_writer (line 1298) | def open_atomic_writer(path, &block)
class Shebang (line 1308) | class Shebang
method load (line 1309) | def Shebang.load(path)
method parse (line 1318) | def Shebang.parse(line)
method initialize (line 1323) | def initialize(cmd, args = [])
method to_s (line 1331) | def to_s
method exec_install (line 1340) | def exec_install
method install_dir_bin (line 1345) | def install_dir_bin(rel)
method install_dir_lib (line 1349) | def install_dir_lib(rel)
method install_dir_ext (line 1353) | def install_dir_ext(rel)
method install_dir_data (line 1360) | def install_dir_data(rel)
method install_dir_conf (line 1364) | def install_dir_conf(rel)
method install_dir_man (line 1370) | def install_dir_man(rel)
method install_files (line 1374) | def install_files(list, dest, mode)
method libfiles (line 1381) | def libfiles
method rubyextentions (line 1385) | def rubyextentions(dir)
method targetfiles (line 1393) | def targetfiles
method mapdir (line 1397) | def mapdir(ents)
method existfiles (line 1415) | def existfiles
method hookfiles (line 1419) | def hookfiles
method glob_select (line 1425) | def glob_select(pat, ents)
method glob_reject (line 1430) | def glob_reject(pats, ents)
method globs2re (line 1442) | def globs2re(pats)
method exec_test (line 1454) | def exec_test
method exec_clean (line 1474) | def exec_clean
method clean_dir_ext (line 1486) | def clean_dir_ext(rel)
method exec_distclean (line 1495) | def exec_distclean
method distclean_dir_ext (line 1504) | def distclean_dir_ext(rel)
method exec_task_traverse (line 1517) | def exec_task_traverse(task)
method traverse (line 1529) | def traverse(task, rel, mid)
method dive_into (line 1540) | def dive_into(rel)
method run_hook (line 1555) | def run_hook(id)
class SetupError (line 1570) | class SetupError < StandardError; end
function setup_rb_error (line 1572) | def setup_rb_error(msg)
FILE: src/liblwgeom/box2d.c
function expand_box2d (line 17) | void
function box2d_same (line 33) | char
function BOX2DFLOAT4 (line 48) | BOX2DFLOAT4 *
FILE: src/liblwgeom/lex.yy.c
type flex_int8_t (line 63) | typedef int8_t flex_int8_t;
type flex_uint8_t (line 64) | typedef uint8_t flex_uint8_t;
type flex_int16_t (line 65) | typedef int16_t flex_int16_t;
type flex_uint16_t (line 66) | typedef uint16_t flex_uint16_t;
type flex_int32_t (line 67) | typedef int32_t flex_int32_t;
type flex_uint32_t (line 68) | typedef uint32_t flex_uint32_t;
type flex_int8_t (line 70) | typedef signed char flex_int8_t;
type flex_int16_t (line 71) | typedef short int flex_int16_t;
type flex_int32_t (line 72) | typedef int flex_int32_t;
type flex_uint8_t (line 73) | typedef unsigned char flex_uint8_t;
type flex_uint16_t (line 74) | typedef unsigned short int flex_uint16_t;
type flex_uint32_t (line 75) | typedef unsigned int flex_uint32_t;
type yy_buffer_state (line 173) | struct yy_buffer_state
type yy_size_t (line 204) | typedef size_t yy_size_t;
type yy_buffer_state (line 209) | struct yy_buffer_state
type YY_CHAR (line 355) | typedef char YY_CHAR;
type yy_state_type (line 359) | typedef int yy_state_type;
type yy_trans_info (line 3279) | struct yy_trans_info
function init_parser (line 3362) | void init_parser(const char *src) { BEGIN(0);buf_state = lwg_parse_yy_sc...
function close_parser (line 3363) | void close_parser() { lwg_parse_yy_delete_buffer(buf_state); }
function lwg_parse_yywrap (line 3364) | int lwg_parse_yywrap(void){ return 1; }
function YY_RESTORE_YY_MORE_OFFSET (line 3813) | YY_RESTORE_YY_MORE_OFFSET
function yy_get_next_buffer (line 3942) | static int yy_get_next_buffer (void)
function yy_state_type (line 4076) | static yy_state_type yy_get_previous_state (void)
function yy_state_type (line 4106) | static yy_state_type yy_try_NUL_trans (yy_state_type yy_current_state )
function yyunput (line 4126) | static void yyunput (int c, register char * yy_bp )
function input (line 4167) | static int input (void)
function lwg_parse_yyrestart (line 4242) | void lwg_parse_yyrestart (FILE * input_file )
function lwg_parse_yy_switch_to_buffer (line 4259) | void lwg_parse_yy_switch_to_buffer (YY_BUFFER_STATE new_buffer )
function lwg_parse_yy_load_buffer_state (line 4290) | static void lwg_parse_yy_load_buffer_state (void)
function YY_BUFFER_STATE (line 4304) | YY_BUFFER_STATE lwg_parse_yy_create_buffer (FILE * file, int size )
function lwg_parse_yy_delete_buffer (line 4332) | void lwg_parse_yy_delete_buffer (YY_BUFFER_STATE b )
function lwg_parse_yy_init_buffer (line 4355) | static void lwg_parse_yy_init_buffer (YY_BUFFER_STATE b, FILE * file )
function lwg_parse_yy_flush_buffer (line 4383) | void lwg_parse_yy_flush_buffer (YY_BUFFER_STATE b )
function lwg_parse_yypush_buffer_state (line 4412) | void lwg_parse_yypush_buffer_state (YY_BUFFER_STATE new_buffer )
function lwg_parse_yypop_buffer_state (line 4442) | void lwg_parse_yypop_buffer_state (void)
function lwg_parse_yyensure_buffer_stack (line 4461) | static void lwg_parse_yyensure_buffer_stack (void)
function YY_BUFFER_STATE (line 4510) | YY_BUFFER_STATE lwg_parse_yy_scan_buffer (char * base, yy_size_t size )
function YY_BUFFER_STATE (line 4547) | YY_BUFFER_STATE lwg_parse_yy_scan_string (yyconst char * yystr )
function YY_BUFFER_STATE (line 4560) | YY_BUFFER_STATE lwg_parse_yy_scan_bytes (yyconst char * yybytes, int _...
function yy_fatal_error (line 4594) | static void yy_fatal_error (yyconst char* msg )
function lwg_parse_yyget_lineno (line 4622) | int lwg_parse_yyget_lineno (void)
function FILE (line 4631) | FILE *lwg_parse_yyget_in (void)
function FILE (line 4639) | FILE *lwg_parse_yyget_out (void)
function lwg_parse_yyget_leng (line 4647) | int lwg_parse_yyget_leng (void)
function lwg_parse_yyset_lineno (line 4665) | void lwg_parse_yyset_lineno (int line_number )
function lwg_parse_yyset_in (line 4677) | void lwg_parse_yyset_in (FILE * in_str )
function lwg_parse_yyset_out (line 4682) | void lwg_parse_yyset_out (FILE * out_str )
function lwg_parse_yyget_debug (line 4687) | int lwg_parse_yyget_debug (void)
function lwg_parse_yyset_debug (line 4692) | void lwg_parse_yyset_debug (int bdebug )
function yy_init_globals (line 4697) | static int yy_init_globals (void)
function lwg_parse_yylex_destroy (line 4726) | int lwg_parse_yylex_destroy (void)
function yy_flex_strncpy (line 4752) | static void yy_flex_strncpy (char* s1, yyconst char * s2, int n )
function yy_flex_strlen (line 4761) | static int yy_flex_strlen (yyconst char * s )
function lwg_parse_yyfree (line 4788) | void lwg_parse_yyfree (void * ptr )
FILE: src/liblwgeom/liblwgeom.h
type uint32 (line 69) | typedef unsigned int uint32;
type int32 (line 70) | typedef int int32;
type uchar (line 158) | typedef unsigned char uchar;
type BOX2DFLOAT4 (line 160) | typedef struct
type BOX3D (line 168) | typedef struct
type CHIP (line 174) | typedef struct chiptag
type SPHEROID (line 225) | typedef struct
type POINT3DZ (line 244) | typedef struct { double x,y,z; } POINT3DZ;
type POINT3D (line 245) | typedef struct { double x,y,z; } POINT3D;
type POINT3DM (line 246) | typedef struct { double x,y,m; } POINT3DM;
type POINT2D (line 253) | typedef struct
type POINT4D (line 259) | typedef struct
type POINTARRAY (line 275) | typedef struct
type DYNPTARRAY (line 292) | typedef struct {
type LWGEOM (line 322) | typedef struct
type LWPOINT (line 331) | typedef struct
type LWLINE (line 340) | typedef struct
type LWPOLY (line 349) | typedef struct
type LWMPOINT (line 359) | typedef struct
type LWMLINE (line 369) | typedef struct
type LWMPOLY (line 379) | typedef struct
type LWCOLLECTION (line 389) | typedef struct
type LWCIRCSTRING (line 399) | typedef struct
type LWCOMPOUND (line 408) | typedef struct
type LWCURVEPOLY (line 418) | typedef struct
type LWMCURVE (line 428) | typedef struct
type LWMSURFACE (line 438) | typedef struct
type PG_LWGEOM (line 639) | typedef struct {
type LWGEOM_INSPECTED (line 890) | typedef struct
type LWGEOM_PARSER_RESULT (line 1324) | typedef struct struct_lwgeom_parser_result
type LWGEOM_UNPARSER_RESULT (line 1347) | typedef struct struct_lwgeom_unparser_result
FILE: src/liblwgeom/lwalgorithm.c
function lw_segment_side (line 23) | double lw_segment_side(POINT2D *p1, POINT2D *p2, POINT2D *q)
function lw_segment_envelope_intersects (line 28) | int lw_segment_envelope_intersects(POINT2D p1, POINT2D p2, POINT2D q1, P...
function lw_segment_intersects (line 61) | int lw_segment_intersects(POINT2D *p1, POINT2D *p2, POINT2D *q1, POINT2D...
function lwline_crossing_direction (line 137) | int lwline_crossing_direction(LWLINE *l1, LWLINE *l2)
function lwpoint_get_ordinate (line 301) | double lwpoint_get_ordinate(const POINT4D *p, int ordinate)
function lwpoint_set_ordinate (line 325) | void lwpoint_set_ordinate(POINT4D *p, int ordinate, double value)
function lwpoint_interpolate (line 359) | int lwpoint_interpolate(const POINT4D *p1, const POINT4D *p2, POINT4D *p...
function LWCOLLECTION (line 394) | LWCOLLECTION *lwmline_clip_to_ordinate_range(LWMLINE *mline, int ordinat...
function LWCOLLECTION (line 474) | LWCOLLECTION *lwline_clip_to_ordinate_range(LWLINE *line, int ordinate, ...
function lwgeom_geohash_precision (line 803) | int lwgeom_geohash_precision(BOX3D bbox, BOX3D *bounds)
FILE: src/liblwgeom/lwalgorithm.h
type CG_SEGMENT_INTERSECTION_TYPE (line 16) | enum CG_SEGMENT_INTERSECTION_TYPE {
type CG_LINE_CROSS_TYPE (line 31) | enum CG_LINE_CROSS_TYPE {
FILE: src/liblwgeom/lwcircstring.c
function LWCIRCSTRING (line 42) | LWCIRCSTRING *
function LWCIRCSTRING (line 77) | LWCIRCSTRING *
function uchar (line 143) | uchar *
function lwcircstring_serialize_buf (line 168) | void lwcircstring_serialize_buf(LWCIRCSTRING *curve, uchar *buf, size_t ...
function lwcircstring_serialize_size (line 238) | size_t
function BOX3D (line 256) | BOX3D *
function BOX3D (line 448) | BOX3D *
function lwcircstring_compute_box2d_p (line 486) | int
function lwcircstring_free (line 497) | void lwcircstring_free(LWCIRCSTRING *curve)
function lwgeom_size_circstring (line 504) | size_t
function printLWCIRCSTRING (line 541) | void printLWCIRCSTRING(LWCIRCSTRING *curve)
function LWCIRCSTRING (line 551) | LWCIRCSTRING *
function LWGEOM (line 566) | LWGEOM *
function lwcircstring_reverse (line 612) | void lwcircstring_reverse(LWCIRCSTRING *curve)
function LWCIRCSTRING (line 620) | LWCIRCSTRING *
function lwcircstring_same (line 628) | char
function LWCIRCSTRING (line 638) | LWCIRCSTRING *
function LWCIRCSTRING (line 689) | LWCIRCSTRING *
function LWCIRCSTRING (line 724) | LWCIRCSTRING *
function LWCIRCSTRING (line 738) | LWCIRCSTRING *
function lwcircstring_setPoint4d (line 753) | void
FILE: src/liblwgeom/lwcollection.c
function lwcollection_release (line 21) | void
function LWCOLLECTION (line 28) | LWCOLLECTION *
function LWCOLLECTION (line 74) | LWCOLLECTION *
function LWCOLLECTION (line 91) | LWCOLLECTION *
function LWGEOM (line 134) | LWGEOM *
function lwcollection_serialize_size (line 141) | size_t
function lwcollection_serialize_buf (line 170) | void
function lwcollection_compute_box2d_p (line 223) | int
function LWCOLLECTION (line 244) | LWCOLLECTION *
function LWGEOM (line 273) | LWGEOM *
function LWCOLLECTION (line 316) | LWCOLLECTION *
function lwcollection_same (line 333) | char
function lwcollection_ngeoms (line 365) | int lwcollection_ngeoms(const LWCOLLECTION *col)
function lwcollection_free (line 447) | void lwcollection_free(LWCOLLECTION *col)
function BOX3D (line 491) | BOX3D *lwcollection_compute_box3d(LWCOLLECTION *col)
FILE: src/liblwgeom/lwcompound.c
function LWCOMPOUND (line 18) | LWCOMPOUND *
function LWGEOM (line 72) | LWGEOM *
FILE: src/liblwgeom/lwcurvepoly.c
function LWCURVEPOLY (line 21) | LWCURVEPOLY *
function LWGEOM (line 77) | LWGEOM *
FILE: src/liblwgeom/lwgeom.c
function LWGEOM (line 21) | LWGEOM *
function lwgeom_serialize_size (line 62) | size_t
function lwgeom_serialize_buf (line 95) | void
function uchar (line 135) | uchar *
function lwgeom_forceRHR (line 156) | void
function lwgeom_reverse (line 178) | void
function BOX3D (line 202) | BOX3D *lwgeom_compute_box3d(const LWGEOM *lwgeom)
function lwgeom_compute_box2d_p (line 230) | int
function BOX2DFLOAT4 (line 261) | BOX2DFLOAT4 *
function LWPOINT (line 272) | LWPOINT *
function LWLINE (line 280) | LWLINE *
function LWCIRCSTRING (line 288) | LWCIRCSTRING *
function LWPOLY (line 296) | LWPOLY *
function LWCOLLECTION (line 304) | LWCOLLECTION *
function LWMPOINT (line 313) | LWMPOINT *
function LWMLINE (line 321) | LWMLINE *
function LWMPOLY (line 329) | LWMPOLY *
function LWGEOM (line 337) | LWGEOM *lwmpoly_as_lwgeom(LWMPOLY *obj) { return (LWGEOM *)obj; }
function LWGEOM (line 338) | LWGEOM *lwmline_as_lwgeom(LWMLINE *obj) { return (LWGEOM *)obj; }
function LWGEOM (line 339) | LWGEOM *lwmpoint_as_lwgeom(LWMPOINT *obj) { return (LWGEOM *)obj; }
function LWGEOM (line 340) | LWGEOM *lwcollection_as_lwgeom(LWCOLLECTION *obj) { return (LWGEOM *)obj; }
function LWGEOM (line 341) | LWGEOM *lwpoly_as_lwgeom(LWPOLY *obj) { return (LWGEOM *)obj; }
function LWGEOM (line 342) | LWGEOM *lwline_as_lwgeom(LWLINE *obj) { return (LWGEOM *)obj; }
function LWGEOM (line 343) | LWGEOM *lwpoint_as_lwgeom(LWPOINT *obj) { return (LWGEOM *)obj; }
function lwgeom_release (line 345) | void
function LWGEOM (line 382) | LWGEOM *
function LWGEOM (line 419) | LWGEOM *
function uchar (line 523) | uchar *
function LWGEOM (line 548) | LWGEOM *
function LWGEOM (line 580) | LWGEOM *
function serialized_lwgeom_from_ewkt (line 606) | int
function serialized_lwgeom_to_ewkt (line 621) | int
function serialized_lwgeom_from_hexwkb (line 634) | int
function serialized_lwgeom_to_hexwkb (line 649) | int
function serialized_lwgeom_to_ewkb (line 662) | int
function lwgeom_same (line 680) | char
function lwgeom_changed (line 739) | void
function lwgeom_drop_bbox (line 747) | void
function lwgeom_add_bbox (line 760) | void
function lwgeom_dropSRID (line 768) | void
function LWGEOM (line 775) | LWGEOM *
function lwgeom_longitude_shift (line 797) | void
function lwgeom_contains_subgeoms (line 837) | int
function lwgeom_free (line 858) | void lwgeom_free(LWGEOM *lwgeom) {
FILE: src/liblwgeom/lwgeom_api.c
type int32_tt (line 31) | typedef int int32_tt;
type u_int32_tt (line 32) | typedef unsigned int u_int32_tt;
type ieee_float_shape_type (line 34) | typedef union
function nextafterf_custom (line 60) | float
function nextDown_f (line 106) | float nextDown_f(double d)
function nextUp_f (line 121) | float
function nextDown_d (line 137) | double
function nextUp_d (line 152) | double
function BOX2DFLOAT4 (line 169) | BOX2DFLOAT4 *
function box3d_to_box2df_p (line 196) | int
function BOX3D (line 222) | BOX3D
function box2df_to_box3d_p (line 247) | void
function BOX3D (line 269) | BOX3D *
function box3d_union_p (line 329) | int
function BOX2DFLOAT4 (line 392) | BOX2DFLOAT4 *
function getbox2d_p (line 403) | int
function POINT4D (line 450) | POINT4D
function getPoint4d_p (line 465) | int
function POINT3DZ (line 525) | POINT3DZ
function POINT3DM (line 539) | POINT3DM
function getPoint3dz_p (line 553) | int
function getPoint3dm_p (line 603) | int
function POINT2D (line 668) | POINT2D
function getPoint2d_p (line 682) | int
function setPoint4d (line 707) | void
function uchar (line 736) | uchar *
function POINTARRAY (line 769) | POINTARRAY *
function pointArray_ptsize (line 794) | int
function lwgeom_hasSRID (line 809) | char
function lwgeom_ndims (line 816) | int
function lwgeom_hasM (line 823) | int lwgeom_hasM(uchar type)
function lwgeom_hasZ (line 829) | int lwgeom_hasZ(uchar type)
function lwgeom_getType (line 836) | int
function uchar (line 846) | uchar
function uchar (line 856) | uchar
function lwgeom_hasBBOX (line 869) | char
function uint32 (line 880) | uint32
function int32 (line 890) | int32
function LWGEOM_INSPECTED (line 917) | LWGEOM_INSPECTED *
function LWPOINT (line 999) | LWPOINT *
function LWPOINT (line 1032) | LWPOINT *lwgeom_getpoint_inspected(LWGEOM_INSPECTED *inspected, int geom...
function LWLINE (line 1054) | LWLINE *
function LWLINE (line 1084) | LWLINE *
function LWPOLY (line 1106) | LWPOLY *
function LWPOLY (line 1136) | LWPOLY *
function LWCIRCSTRING (line 1158) | LWCIRCSTRING *
function LWGEOM (line 1178) | LWGEOM *lwgeom_getgeom_inspected(LWGEOM_INSPECTED *inspected, int geom_n...
function uchar (line 1211) | uchar *
function uchar (line 1223) | uchar *
function uchar (line 1245) | uchar
function uchar (line 1257) | uchar
function lwgeom_getnumgeometries (line 1271) | int
function lwgeom_getnumgeometries_inspected (line 1302) | int
function uchar (line 1316) | uchar *
function uchar (line 1419) | uchar *
function lwgeom_empty_length (line 1447) | size_t
function lwgeom_constructempty_buf (line 1459) | void
function lwgeom_size (line 1490) | size_t
function lwgeom_size_subgeom (line 1586) | size_t
function lwgeom_size_inspected (line 1598) | int
function compute_serialized_box3d_p (line 1604) | int
function compute_serialized_box2d_p (line 1626) | int
function BOX3D (line 1643) | BOX3D *
function lwinspected_release (line 1763) | void
function printBOX3D (line 1781) | void printBOX3D(BOX3D *box)
function printPA (line 1787) | void printPA(POINTARRAY *pa)
function printBYTES (line 1822) | void printBYTES(uchar *a, int n)
function printMULTI (line 1839) | void
function printType (line 1877) | void
function lwgeom_getsrid (line 1887) | int
function ptarray_isccw (line 1903) | char
function BOX2DFLOAT4 (line 1927) | BOX2DFLOAT4 *
function box2d_union_p (line 1969) | int
function uchar (line 2023) | uchar
function deparse_hex (line 2158) | void
function interpolate_point4d (line 2187) | void
FILE: src/liblwgeom/lwgparse.c
type int4 (line 29) | typedef uint32_t int4;
type tuple (line 31) | typedef struct tag_tuple tuple;
type tag_outputstate (line 33) | struct tag_outputstate{
type output_state (line 37) | typedef struct tag_outputstate output_state;
type tag_tuple (line 51) | struct tag_tuple{
function set_srid (line 220) | void
function tuple (line 232) | tuple *
function free_tuple (line 269) | void
function inc_num (line 286) | void
function alloc_stack_tuple (line 295) | void
function pop (line 311) | void
function popc (line 317) | void
function check_dims (line 349) | void
function WRITE_INT4 (line 379) | void
function WRITE_DOUBLES (line 405) | void
function empty_stack (line 425) | void
function alloc_lwgeom (line 431) | void
function write_point_2 (line 457) | void
function write_point_3 (line 463) | void
function write_point_4 (line 469) | void
function write_point_2i (line 475) | void
function write_point_3i (line 481) | void
function write_point_4i (line 487) | void
function alloc_point_2d (line 493) | void
function alloc_point_3d (line 513) | void
function alloc_point_4d (line 534) | void
function write_type (line 556) | void
function write_count (line 586) | void
function write_type_count (line 593) | void
function alloc_point (line 600) | void
function alloc_linestring (line 615) | void
function alloc_linestring_closed (line 630) | void alloc_linestring_closed(void)
function alloc_circularstring (line 638) | void
function alloc_circularstring_closed (line 649) | void alloc_circularstring_closed(void)
function alloc_polygon (line 657) | void
function alloc_curvepolygon (line 673) | void
function alloc_compoundcurve (line 684) | void
function alloc_multipoint (line 692) | void
function alloc_multilinestring (line 700) | void
function alloc_multicurve (line 708) | void
function alloc_multipolygon (line 716) | void
function alloc_multisurface (line 724) | void
function alloc_geomertycollection (line 732) | void
function alloc_counter (line 740) | void
function alloc_empty (line 748) | void
function make_serialized_lwgeom (line 779) | void
function lwg_parse_yynotice (line 805) | void
function lwg_parse_yyerror (line 811) | int
function uchar (line 844) | uchar
function uchar (line 862) | uchar
function read_wkb_bytes (line 872) | void
function int4 (line 884) | int4
function read_wkb_double (line 892) | double
function read_wkb_point (line 907) | void
function read_wkb_polygon (line 955) | void
function read_wkb_linestring (line 977) | void
function read_wkb_circstring (line 990) | void
function read_wkb_ordinate_array (line 1002) | void
function read_collection (line 1018) | void
function parse_wkb (line 1033) | void
function alloc_wkb (line 1156) | void
function parse_it (line 1167) | int
function parse_lwg (line 1196) | int
function parse_lwgi (line 1203) | int
function set_zm (line 1210) | void
FILE: src/liblwgeom/lwgunparse.c
type int4 (line 27) | typedef uint32_t int4;
type uchar (line 28) | typedef uchar* (*outfunc)(uchar*,int);
type uchar (line 29) | typedef uchar* (*outwkbfunc)(uchar*);
function ensure (line 134) | void
function to_end (line 149) | void
function write_str (line 157) | void
function write_double (line 165) | void
function write_int (line 175) | void
function int4 (line 182) | int4
function read_double (line 216) | double read_double(uchar** geom){
function uchar (line 231) | uchar *
function uchar (line 244) | uchar *
function uchar (line 254) | uchar *
function uchar (line 275) | uchar *
function uchar (line 303) | uchar *
function uchar (line 360) | uchar *
function uchar (line 394) | uchar *
function uchar (line 403) | uchar *output_multipoint(uchar* geom,int suppress){
function uchar (line 420) | uchar *output_compound(uchar* geom, int suppress) {
function uchar (line 440) | uchar *output_multisurface(uchar* geom, int suppress) {
function uchar (line 463) | uchar *
function unparse_WKT (line 618) | int
function write_wkb_hex_flip_bytes (line 653) | void
function write_wkb_hex_bytes (line 671) | void
function write_wkb_bin_flip_bytes (line 689) | void
function write_wkb_bin_bytes (line 706) | void
function uchar (line 722) | uchar *
function write_wkb_int (line 735) | void
function uchar (line 741) | uchar *
function uchar (line 754) | uchar *
function uchar (line 774) | uchar *
function uchar (line 825) | uchar *
function uchar (line 834) | uchar *
function uchar (line 858) | uchar *
function unparse_WKB (line 948) | int
FILE: src/liblwgeom/lwline.c
function LWLINE (line 26) | LWLINE *
function LWLINE (line 55) | LWLINE *
function uchar (line 118) | uchar *
function lwline_serialize_buf (line 144) | void
function BOX3D (line 214) | BOX3D *
function lwline_serialize_size (line 226) | size_t
function lwline_free (line 244) | void lwline_free (LWLINE *line)
function lwgeom_size_line (line 254) | size_t
function printLWLINE (line 293) | void printLWLINE(LWLINE *line)
function lwline_compute_box2d_p (line 302) | int
function LWLINE (line 309) | LWLINE *
function LWGEOM (line 327) | LWGEOM *
function lwline_release (line 373) | void
function lwline_reverse (line 379) | void
function LWLINE (line 385) | LWLINE *
function lwline_same (line 393) | char
function LWLINE (line 403) | LWLINE *
function LWLINE (line 455) | LWLINE *
function LWLINE (line 490) | LWLINE *
function LWLINE (line 505) | LWLINE *
function lwline_setPoint4d (line 521) | void
FILE: src/liblwgeom/lwmcurve.c
function LWMCURVE (line 18) | LWMCURVE *
function LWGEOM (line 86) | LWGEOM *
FILE: src/liblwgeom/lwmline.c
function lwmline_release (line 18) | void
function LWMLINE (line 24) | LWMLINE *
function LWGEOM (line 77) | LWGEOM *
function lwmline_free (line 118) | void lwmline_free(LWMLINE *mline)
FILE: src/liblwgeom/lwmpoint.c
function lwmpoint_release (line 18) | void
function LWMPOINT (line 25) | LWMPOINT *
function LWGEOM (line 77) | LWGEOM *
function lwmpoint_free (line 118) | void lwmpoint_free(LWMPOINT *mpt)
FILE: src/liblwgeom/lwmpoly.c
function lwmpoly_release (line 19) | void
function LWMPOLY (line 26) | LWMPOLY *
function LWGEOM (line 80) | LWGEOM *
function lwmpoly_free (line 121) | void lwmpoly_free(LWMPOLY *mpoly)
FILE: src/liblwgeom/lwmsurface.c
function LWMSURFACE (line 19) | LWMSURFACE *
function LWGEOM (line 89) | LWGEOM *
FILE: src/liblwgeom/lwpoint.c
function uchar (line 23) | uchar *
function lwpoint_serialize_buf (line 47) | void
function BOX3D (line 95) | BOX3D *
function lwpoint_getPoint2d_p (line 116) | int
function lwpoint_getPoint3dz_p (line 123) | int
function lwpoint_getPoint3dm_p (line 128) | int
function lwpoint_getPoint4d_p (line 133) | int
function lwpoint_serialize_size (line 140) | size_t
function LWPOINT (line 161) | LWPOINT *
function LWPOINT (line 178) | LWPOINT *
function LWPOINT (line 192) | LWPOINT *
function LWPOINT (line 207) | LWPOINT *
function LWPOINT (line 222) | LWPOINT *
function LWPOINT (line 244) | LWPOINT *
function lwpoint_free (line 303) | void lwpoint_free(LWPOINT *pt)
function printLWPOINT (line 310) | void printLWPOINT(LWPOINT *point)
function lwpoint_compute_box2d_p (line 320) | int
function LWPOINT (line 327) | LWPOINT *
function LWGEOM (line 345) | LWGEOM *
function lwgeom_size_point (line 391) | size_t
function lwpoint_release (line 427) | void
function lwpoint_same (line 435) | char
FILE: src/liblwgeom/lwpoly.c
function LWPOLY (line 26) | LWPOLY *
function LWPOLY (line 68) | LWPOLY *
function uchar (line 148) | uchar *
function lwpoly_serialize_buf (line 173) | void
function BOX3D (line 241) | BOX3D *
function lwgeom_size_poly (line 254) | size_t
function lwpoly_serialize_size (line 332) | size_t
function lwpoly_free (line 357) | void lwpoly_free (LWPOLY *poly)
function printLWPOLY (line 376) | void printLWPOLY(LWPOLY *poly)
function lwpoly_compute_box2d_p (line 391) | int
function LWPOLY (line 410) | LWPOLY *
function LWGEOM (line 427) | LWGEOM *
function lwpoly_forceRHR (line 473) | void
function lwpoly_release (line 492) | void
function lwpoly_reverse (line 498) | void
function LWPOLY (line 507) | LWPOLY *
function lwpoly_same (line 526) | char
function LWPOLY (line 547) | LWPOLY *
FILE: src/liblwgeom/lwsegmentize.c
function uint32 (line 49) | uint32
function lwcircle_center (line 86) | double
function interpolate_arc (line 132) | double
function POINTARRAY (line 149) | POINTARRAY *
function LWLINE (line 283) | LWLINE *
function LWLINE (line 333) | LWLINE *
function LWPOLY (line 383) | LWPOLY *
function LWMLINE (line 421) | LWMLINE *
function LWMPOLY (line 455) | LWMPOLY *
function LWCOLLECTION (line 491) | LWCOLLECTION *
function LWGEOM (line 528) | LWGEOM *
function LWGEOM (line 560) | LWGEOM *
function LWGEOM (line 732) | LWGEOM *
function LWGEOM (line 932) | LWGEOM *
function LWGEOM (line 940) | LWGEOM *
function LWGEOM (line 970) | LWGEOM *
function LWGEOM (line 999) | LWGEOM *
function LWGEOM (line 1027) | LWGEOM *
FILE: src/liblwgeom/lwutil.c
function lwnotice (line 50) | void
function lwerror (line 63) | void
function init_freeor (line 94) | void
function init_noticereporter (line 110) | void
function init_errorreporter (line 118) | void
function default_freeor (line 142) | void
function default_noticereporter (line 155) | void
function default_errorreporter (line 173) | void
function lwgeom_install_default_allocators (line 198) | void lwgeom_install_default_allocators(void)
function lwfree (line 234) | void
function trim_trailing_zeros (line 244) | void
function getMachineEndian (line 351) | char
function errorIfSRIDMismatch (line 362) | void
FILE: src/liblwgeom/measures.c
function pt_in_ring_2d (line 28) | int pt_in_ring_2d(POINT2D *p, POINTARRAY *ring)
function distance2d_pt_pt (line 84) | double distance2d_pt_pt(POINT2D *p1, POINT2D *p2)
function distance2d_pt_seg (line 98) | double distance2d_pt_seg(POINT2D *p, POINT2D *A, POINT2D *B)
function distance2d_seg_seg (line 146) | double distance2d_seg_seg(POINT2D *A, POINT2D *B, POINT2D *C, POINT2D *D)
function distance2d_pt_ptarray (line 229) | double distance2d_pt_ptarray(POINT2D *p, POINTARRAY *pa)
function distance2d_ptarray_ptarray (line 254) | double distance2d_ptarray_ptarray(POINTARRAY *l1, POINTARRAY *l2)
function pt_in_poly_2d (line 303) | int pt_in_poly_2d(POINT2D *p, LWPOLY *poly)
function distance2d_ptarray_poly (line 330) | double distance2d_ptarray_poly(POINTARRAY *pa, LWPOLY *poly)
function distance2d_point_point (line 382) | double distance2d_point_point(LWPOINT *point1, LWPOINT *point2)
function distance2d_point_line (line 393) | double distance2d_point_line(LWPOINT *point, LWLINE *line)
function distance2d_line_line (line 401) | double distance2d_line_line(LWLINE *line1, LWLINE *line2)
function distance2d_point_poly (line 413) | double distance2d_point_poly(LWPOINT *point, LWPOLY *poly)
function distance2d_poly_poly (line 459) | double distance2d_poly_poly(LWPOLY *poly1, LWPOLY *poly2)
function distance2d_line_poly (line 505) | double distance2d_line_poly(LWLINE *line, LWPOLY *poly)
function lwgeom_pointarray_length2d (line 512) | double lwgeom_pointarray_length2d(POINTARRAY *pts)
function lwgeom_pointarray_length (line 534) | double
function lwgeom_curvepolygon_area (line 562) | double
function lwgeom_polygon_area (line 573) | double
function lwgeom_polygon_perimeter (line 616) | double lwgeom_polygon_perimeter(LWPOLY *poly)
function lwgeom_polygon_perimeter2d (line 633) | double lwgeom_polygon_perimeter2d(LWPOLY *poly)
function lwgeom_mindistance2d_recursive (line 646) | double
function lwgeom_mindistance2d_recursive_tolerance (line 652) | double
function lwgeom_pt_inside_circle (line 796) | int
function azimuth_pt_pt (line 813) | int
FILE: src/liblwgeom/ptarray.c
function POINTARRAY (line 19) | POINTARRAY *
function ptarray_free (line 40) | void ptarray_free(POINTARRAY *pa)
function ptarray_reverse (line 51) | void
function ptarray_compute_box2d_p (line 77) | int
function BOX2DFLOAT4 (line 111) | BOX2DFLOAT4 *
function POINTARRAY (line 147) | POINTARRAY *
function ptarray_same (line 222) | char
function POINTARRAY (line 248) | POINTARRAY *
function POINTARRAY (line 306) | POINTARRAY *
function POINTARRAY (line 350) | POINTARRAY *
function ptarray_isclosed2d (line 368) | int
function BOX3D (line 380) | BOX3D *
function ptarray_compute_box3d_p (line 400) | int
function POINTARRAY (line 451) | POINTARRAY *
function closest_point_on_segment (line 654) | void
function ptarray_locate_point (line 696) | double
function ptarray_longitude_shift (line 770) | void
function DYNPTARRAY (line 784) | DYNPTARRAY *
function dynptarray_addPoint4d (line 815) | int
FILE: src/liblwgeom/vsprintf.c
function va_dcl (line 47) | va_dcl
function lw_vasprintf (line 141) | int
FILE: src/liblwgeom/wktparse.h
type uchar (line 16) | typedef unsigned char uchar;
type SERIALIZED_LWGEOM (line 18) | typedef struct serialized_lwgeom {
type LWGEOM_PARSER_RESULT (line 23) | typedef struct struct_lwgeom_parser_result
type LWGEOM_UNPARSER_RESULT (line 32) | typedef struct struct_lwgeom_unparser_result
FILE: src/liblwgeom/wktparse.tab.c
type yytokentype (line 76) | enum yytokentype {
type YYLTYPE (line 195) | typedef struct YYLTYPE
type YYTYPE_UINT8 (line 219) | typedef YYTYPE_UINT8 yytype_uint8;
type yytype_uint8 (line 221) | typedef unsigned char yytype_uint8;
type YYTYPE_INT8 (line 225) | typedef YYTYPE_INT8 yytype_int8;
type yytype_int8 (line 228) | typedef signed char yytype_int8;
type yytype_int8 (line 230) | typedef short int yytype_int8;
type YYTYPE_UINT16 (line 234) | typedef YYTYPE_UINT16 yytype_uint16;
type yytype_uint16 (line 236) | typedef unsigned short int yytype_uint16;
type YYTYPE_INT16 (line 240) | typedef YYTYPE_INT16 yytype_int16;
type yytype_int16 (line 242) | typedef short int yytype_int16;
function YYID (line 289) | static int
function yy_symbol_value_print (line 973) | static void
function yy_symbol_print (line 1007) | static void
function yy_stack_print (line 1036) | static void
function yy_reduce_print (line 1064) | static void
function YYSIZE_T (line 1134) | static YYSIZE_T
function YYSIZE_T (line 1183) | static YYSIZE_T
function YYSIZE_T (line 1230) | static YYSIZE_T
function yydestruct (line 1343) | static void
FILE: src/liblwgeom/wktparse.tab.h
type yytokentype (line 41) | enum yytokentype {
type YYLTYPE (line 133) | typedef struct YYLTYPE
FILE: src/libsqlite3_geocoder/extension.c
function sqlite3_metaphone (line 11) | static void
function sqlite3_levenshtein (line 30) | static void
function sqlite3_digit_suffix (line 44) | static void
function sqlite3_nondigit_prefix (line 57) | static void
function sqlite3_compress_wkb_line (line 71) | static void
function sqlite3_uncompress_wkb_line (line 88) | static void
function sqlite3_extension_init (line 101) | int sqlite3_extension_init (sqlite3 * db, char **pzErrMsg,
FILE: src/libsqlite3_geocoder/levenshtein.c
function levenshtein_distance (line 10) | double levenshtein_distance (const unsigned char *s1, const unsigned cha...
function main (line 36) | int main (int argc, char **argv) {
FILE: src/libsqlite3_geocoder/metaphon.c
function metaphone (line 43) | int metaphone(const char *Word, char *Metaph, int max_phones) {
FILE: src/libsqlite3_geocoder/util.c
function address_metaphone (line 4) | int address_metaphone(const char *input, char *output, int max_phones) {
function rindex_nondigit (line 17) | signed int rindex_nondigit (const char *string) {
function digit_suffix (line 24) | signed int digit_suffix (const char *input, char *output) {
function nondigit_prefix (line 30) | signed int nondigit_prefix (const char *input, char *output) {
FILE: src/libsqlite3_geocoder/wkb_compress.c
function compress_wkb_line (line 4) | uint32_t compress_wkb_line (void *dest, const void *src, uint32_t len) {
function uncompress_wkb_line (line 16) | uint32_t uncompress_wkb_line (void *dest, const void *src, uint32_t len) {
function main (line 33) | int main (int argc, char *argv) {
FILE: src/metaphone/extension.c
function sqlite3_metaphone (line 8) | static void
function sqlite3_extension_init (line 27) | int sqlite3_extension_init (sqlite3 * db, char **pzErrMsg,
FILE: src/metaphone/metaphon.c
function metaphone (line 42) | int metaphone(const char *Word, char *Metaph, int max_phones)
FILE: src/shp2sqlite/dbfopen.c
function DBFWriteHeader (line 246) | static void DBFWriteHeader(DBFHandle psDBF)
function DBFFlushRecord (line 304) | static void DBFFlushRecord( DBFHandle psDBF )
function DBFUpdateHeader (line 325) | void SHPAPI_CALL
function DBFHandle (line 356) | DBFHandle SHPAPI_CALL
function DBFClose (line 493) | void SHPAPI_CALL
function DBFHandle (line 558) | DBFHandle SHPAPI_CALL
function DBFAddField (line 635) | int SHPAPI_CALL
function DBFReadSetup (line 740) | int DBFReadSetup(DBFHandle psDBF, int hEntity)
function DBFReadDeleted (line 788) | int DBFReadDeleted(DBFHandle psDBF, int hEntity)
function DBFReadIntegerAttribute (line 888) | int SHPAPI_CALL
function DBFReadDoubleAttribute (line 908) | double SHPAPI_CALL
function SHPAPI_CALL1 (line 928) | const char SHPAPI_CALL1(*)
function DBFGetFieldCount (line 994) | int SHPAPI_CALL
function DBFGetRecordCount (line 1007) | int SHPAPI_CALL
function DBFFieldType (line 1020) | DBFFieldType SHPAPI_CALL
function DBFWriteAttribute (line 1070) | static int DBFWriteAttribute(DBFHandle psDBF, int hEntity, int iField,
function DBFWriteAttributeDirectly (line 1240) | int DBFWriteAttributeDirectly(DBFHandle psDBF, int hEntity, int iField,
function DBFWriteDoubleAttribute (line 1315) | int SHPAPI_CALL
function DBFWriteIntegerAttribute (line 1329) | int SHPAPI_CALL
function DBFWriteStringAttribute (line 1345) | int SHPAPI_CALL
function DBFWriteNULLAttribute (line 1359) | int SHPAPI_CALL
function DBFWriteLogicalAttribute (line 1372) | int SHPAPI_CALL
function DBFWriteTuple (line 1386) | int SHPAPI_CALL
function SHPAPI_CALL1 (line 1448) | const char SHPAPI_CALL1(*)
function DBFHandle (line 1494) | DBFHandle SHPAPI_CALL
function DBFGetNativeFieldType (line 1540) | char SHPAPI_CALL
function str_to_upper (line 1554) | static void str_to_upper (char *string)
function DBFGetFieldIndex (line 1574) | int SHPAPI_CALL
FILE: src/shp2sqlite/getopt.c
function my_strlen (line 121) | static int
function exchange (line 181) | static void
function _pgis_getopt_internal (line 266) | int
function pgis_getopt (line 600) | int
function pgis_getopt_long (line 612) | int
function main (line 630) | int
FILE: src/shp2sqlite/getopt.h
type option (line 77) | struct option
type option (line 105) | struct option
type option (line 108) | struct option
type option (line 113) | struct option
FILE: src/shp2sqlite/shapefil.h
type SHPInfo (line 198) | typedef struct
type SHPInfo (line 221) | typedef SHPInfo * SHPHandle;
type SHPObject (line 258) | typedef struct
function SHPObject (line 296) | SHPObject SHPAPI_CALL1(*)
FILE: src/shp2sqlite/shp2sqlite.c
type Point (line 54) | typedef struct {double x, y, z, m;} Point;
type Ring (line 56) | typedef struct Ring {
function lwgeom_init_allocators (line 138) | void lwgeom_init_allocators()
function PIP (line 276) | int
function Insert_attributes (line 298) | int
function main (line 393) | int
function LowerCase (line 461) | void
function Cleanup (line 468) | void
function OpenShape (line 474) | void
function CreateTable (line 525) | void
function CreateIndex (line 638) | void
function LoadData (line 654) | void
function usage (line 790) | void
function InsertLineString (line 822) | void
function FindPolygons (line 926) | int
function ReleasePolygons (line 1057) | void
function InsertPolygon (line 1079) | void
function InsertPoint (line 1223) | void
function OutputGeometry (line 1309) | void
function ParseCmdline (line 1362) | int
function SetPgType (line 1505) | void
function DropTable (line 1632) | void
function GetFieldsSpec (line 1665) | void
FILE: src/shp2sqlite/shpopen.c
type uchar (line 184) | typedef unsigned char uchar;
type int32 (line 187) | typedef long int32;
type int32 (line 189) | typedef int int32;
function SwapWord (line 212) | static void SwapWord( int length, void * wordP )
function SHPWriteHeader (line 249) | void SHPWriteHeader( SHPHandle psSHP )
function SHPHandle (line 358) | SHPHandle SHPAPI_CALL
function SHPClose (line 570) | void SHPAPI_CALL
function SHPGetInfo (line 603) | void SHPAPI_CALL
function SHPHandle (line 632) | SHPHandle SHPAPI_CALL
function _SHPSetBounds (line 740) | static void _SHPSetBounds( uchar * pabyRec, SHPObject * psShape )
function SHPComputeExtents (line 764) | void SHPAPI_CALL
function SHPObject (line 802) | SHPObject SHPAPI_CALL1(*)
function SHPObject (line 912) | SHPObject SHPAPI_CALL1(*)
function SHPObject (line 1300) | SHPObject SHPAPI_CALL1(*)
function SHPDestroyObject (line 1718) | void SHPAPI_CALL
function SHPRewindObject (line 1749) | int SHPAPI_CALL
FILE: test/address.rb
class TestAddress (line 9) | class TestAddress < Test::Unit::TestCase
method test_new (line 10) | def test_new
method test_clean (line 14) | def test_clean
method test_expand_numbers (line 26) | def test_expand_numbers
method test_city_parse (line 33) | def test_city_parse
method test_po_box (line 62) | def test_po_box
method test_parse (line 69) | def test_parse
method test_skip_parse (line 175) | def test_skip_parse
method test_states_abbreviated_in_skip_parse (line 190) | def test_states_abbreviated_in_skip_parse
method test_address_hash (line 202) | def test_address_hash
method test_partial_address (line 213) | def test_partial_address
method test_country_parse (line 225) | def test_country_parse
FILE: test/constants.rb
class TestConstants (line 8) | class TestConstants < Test::Unit::TestCase
method initialize (line 9) | def initialize (*args)
method test_class_constructor (line 17) | def test_class_constructor
method test_key (line 21) | def test_key
method test_fetch (line 29) | def test_fetch
method test_constants (line 47) | def test_constants
FILE: test/database.rb
type Geocoder::US (line 10) | module Geocoder::US
class TestDatabase (line 15) | class TestDatabase < Test::Unit::TestCase
method get_db (line 16) | def get_db
method setup (line 24) | def setup
method test_load (line 31) | def test_load
method test_zip (line 36) | def test_zip
method test_place (line 61) | def test_place
method test_sample (line 86) | def test_sample
method test_city_with_street_type_in_name (line 104) | def test_city_with_street_type_in_name
method test_should_get_street_number_correctly (line 111) | def test_should_get_street_number_correctly
method test_should_geocode_with_hash (line 116) | def test_should_geocode_with_hash
method test_should_work_with_partial_hash (line 122) | def test_should_work_with_partial_hash
method test_weird_edge_case_explosion (line 127) | def test_weird_edge_case_explosion
method test_city_state_together (line 133) | def test_city_state_together
method test_state_street_together (line 138) | def test_state_street_together
method test_intersection (line 143) | def test_intersection
FILE: test/numbers.rb
class TestAddress (line 8) | class TestAddress < Test::Unit::TestCase
method test_number_to_cardinal (line 9) | def test_number_to_cardinal
method test_cardinal_to_number (line 16) | def test_cardinal_to_number
method test_number_to_ordinal (line 26) | def test_number_to_ordinal
method test_ordinal_to_number (line 36) | def test_ordinal_to_number
Condensed preview — 137 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (1,200K chars).
[
{
"path": ".gitignore",
"chars": 104,
"preview": "*.o\n*.so\n*.gem\npkg/\nbin/shp2sqlite\nsrc/shp2sqlite/shp2sqlite\nsrc/liblwgeom/liblwgeom.a\ndoc/*.html\n*.log\n"
},
{
"path": "History.txt",
"chars": 62,
"preview": "=== 1.0.0 / 2009-06-02\n\n* 1 major enhancement\n\n * Birthday!\n\n"
},
{
"path": "LICENSE.txt",
"chars": 7637,
"preview": "\t\t GNU LESSER GENERAL PUBLIC LICENSE\n Version 3, 29 June 2007\n\n Copyright (C) 2007 Free Software"
},
{
"path": "Makefile",
"chars": 190,
"preview": "all:\n\tmake -C src install\n\tgem build gemspec\n\ntest: all\n\truby -Ilib tests/run.rb\n\ninstall: all\n\t# gem install *.gem\n\ncle"
},
{
"path": "Manifest.txt",
"chars": 356,
"preview": "History.txt\nManifest.txt\nREADME.rdoc\nRakefile\nlib/geocoder/us/database.rb\nlib/geocoder/us/numbers.rb\nlib/geocoder/us/add"
},
{
"path": "README.rdoc",
"chars": 9802,
"preview": "= Geocoder::US\n\nGeocoder::US 2.0 is a software package designed to geocode US street\naddresses. Although it is primaril"
},
{
"path": "REST.rdoc",
"chars": 3382,
"preview": "GET /1.0/geocode/address.json\n\nThe geocode/address endpoint returns the interpolated latitude and longitude of\na US stre"
},
{
"path": "TODO.txt",
"chars": 352,
"preview": "1. Check interpolate measure: scale longitude or not?\n5. Intersections...\n - import ALL linestrings (even those with "
},
{
"path": "bin/rebuild_metaphones",
"chars": 718,
"preview": "#!/usr/bin/ruby\n\nrequire 'rubygems'\nrequire 'sqlite3'\nrequire 'text'\n\nif(ARGV.length < 1)\nprint \"Missing SQLite file par"
},
{
"path": "build/build_indexes",
"chars": 148,
"preview": "#!/bin/bash\n\nBASE=$(dirname $0)\nPATH=$PATH:$BASE/bin\nSQL=\"$BASE/sql\"\n\n# Just run the SQL that constructs the indexes.\nsq"
},
{
"path": "build/rebuild_cluster",
"chars": 626,
"preview": "#!/bin/bash\n\nBASE=$(dirname $0)\nPATH=$PATH:$BASE/bin\nSQL=\"$BASE/sql\"\n\nOLD_DB=$1\nDATABASE=${OLD_DB}.$$\n\n[ -r $DATABASE ] "
},
{
"path": "build/sql/cluster.sql",
"chars": 662,
"preview": ".echo on\n-- turn off various pragmas to make SQLite faster\nPRAGMA temp_store=MEMORY;\nPRAGMA journal_mode=OFF;\nPRAGMA syn"
},
{
"path": "build/sql/convert.sql",
"chars": 2696,
"preview": "BEGIN;\n-- start by indexing the temporary tables created from the input data.\nCREATE INDEX featnames_tlid ON tiger_featn"
},
{
"path": "build/sql/create.sql",
"chars": 939,
"preview": "-- initialize the database tables.\n-- 'place' contains the gazetteer of place names.\nCREATE TABLE place(\n zip CHAR(5),\n"
},
{
"path": "build/sql/index.sql",
"chars": 512,
"preview": ".echo on\nPRAGMA temp_store=MEMORY;\nPRAGMA journal_mode=MEMORY;\nPRAGMA synchronous=OFF;\nPRAGMA cache_size=500000;\nPRAGMA "
},
{
"path": "build/sql/setup.sql",
"chars": 1878,
"preview": "-- create temporary tables to hold the TIGER/Line data before it's\n-- transformed and loaded into the permanent tables"
},
{
"path": "build/tiger2009_import",
"chars": 1765,
"preview": "#!/bin/bash\n\nTMP=\"/tmp/tiger-import.$$\"\nSHPS=\"edges\"\nDBFS=\"featnames addr\"\nBASE=$(dirname $0)\nPATH=$PATH:$BASE\nSQL=\"$BAS"
},
{
"path": "build/tiger_import",
"chars": 2115,
"preview": "#!/bin/bash\n\nTMP=\"/tmp/tiger-import.$$\"\nSHPS=\"edges\"\nDBFS=\"featnames addr\"\nBASE=$(dirname $0)\nPATH=$PATH:$BASE\nSQL=\"$BAS"
},
{
"path": "conf/geocoder-us/geocoder.ru",
"chars": 92,
"preview": "require 'sinatra'\ndisable :run, :reload\nrequire 'geocoder/us/rest'\nrun Sinatra::Application\n"
},
{
"path": "conf/geocoder-us/unicorn.rb",
"chars": 467,
"preview": "worker_processes 4\nuser \"www-data\", \"www-data\"\nlisten \"/var/run/geocoder-us/unicorn.sock\", :backlog => 64\npid \"/var/run/"
},
{
"path": "conf/init/geocoder-us.conf",
"chars": 221,
"preview": "description\t\"geocoder.us\"\n\nstart on runlevel [2345]\nstop on runlevel [!2345]\n\nrespawn\nexpect daemon\nscript\n . /etc/de"
},
{
"path": "debian/README.Debian",
"chars": 2738,
"preview": "geocoder-us for Debian\n----------------------\n\nThe Geocoder::US package is a Ruby library that uses a database built fro"
},
{
"path": "debian/changelog",
"chars": 13518,
"preview": "geocoder-us (2.0.1pre-1sg66) lucid; urgency=low\n\n [ Schuyler Erle ]\n * Remove \"la\" from feature type affixes, because "
},
{
"path": "debian/compat",
"chars": 2,
"preview": "7\n"
},
{
"path": "debian/control",
"chars": 553,
"preview": "Source: geocoder-us\nSection: ruby\nPriority: extra\nMaintainer: SimpleGeo Nerds <nerds@simplegeo.com>\nUploaders: Schuyler "
},
{
"path": "debian/copyright",
"chars": 1099,
"preview": "This work was packaged for Debian by:\n\n Schuyler Erle <schuyler@simplegeo.com> on Sat, 07 Aug 2010 00:51:40 +0000\n\nIt"
},
{
"path": "debian/default",
"chars": 241,
"preview": "# Defaults for geocoder-us upstart job\n# sourced by /etc/init/geocoder-us.conf\n# installed at /etc/default/geocoder-us b"
},
{
"path": "debian/docs",
"chars": 55,
"preview": "History.txt\nManifest.txt\nREADME.rdoc\nTODO.txt\nTODO.txt\n"
},
{
"path": "debian/geocoder-us.postinst",
"chars": 1166,
"preview": "#!/bin/sh\n# postinst script for #PACKAGE#\n#\n# see: dh_installdeb(1)\n\nset -e\n\n# summary of how this script can be called:"
},
{
"path": "debian/geocoder-us.prerm",
"chars": 364,
"preview": "#!/bin/sh\n\nset -e\n\ncase \"$1\" in\n remove|deconfigure)\n stop geocoder-us || true\n ;;\n upgrade)\n "
},
{
"path": "debian/rules",
"chars": 494,
"preview": "#!/usr/bin/make -f\n\ninclude /usr/share/cdbs/1/rules/debhelper.mk\ninclude /usr/share/ruby-pkg-tools/1/class/ruby-setup-rb"
},
{
"path": "debian/source/format",
"chars": 12,
"preview": "3.0 (quilt)\n"
},
{
"path": "demos/api/server.rb",
"chars": 411,
"preview": "require 'rubygems'\nrequire 'sinatra'\nrequire 'geocoder/us/database'\nrequire 'json'\n\nset :port, 8080\n@@db = Geocoder::US:"
},
{
"path": "demos/api/views/index.erb",
"chars": 1896,
"preview": "<!DOCTYPE html\n PUBLIC \"-//W3C//DTD XHTML 1.0 Strict//EN\"\n \"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd\""
},
{
"path": "demos/cli.rb",
"chars": 215,
"preview": "require 'geocoder/us/database'\nrequire 'pp'\n\ndb = Geocoder::US::Database.new(\"/mnt/tiger2010/geocoder.db\", :debug=>true)"
},
{
"path": "demos/demo/app/ext/geocodewrap.rb",
"chars": 2700,
"preview": "require 'rubygems'\nrequire 'geocoder/us/database'\nrequire 'logger'\n\nmodule Sinatra\n module GeocodeWrap\n attr_accesso"
},
{
"path": "demos/demo/app/views/index.builder",
"chars": 349,
"preview": "xml.locations do\n unless @records.nil?\n @records.each do |record|\n xml.location do\n xml.score format(\"%."
},
{
"path": "demos/demo/app/views/index.erb",
"chars": 2275,
"preview": "<!DOCTYPE html\n PUBLIC \"-//W3C//DTD XHTML 1.0 Strict//EN\"\n \"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd\""
},
{
"path": "demos/demo/config/bootstraps.rb",
"chars": 2476,
"preview": "require 'rubygems'\n\nmodule BootStraps\n\n class Framework \n\n def initialize \n @methods = {}\n end\n \n def "
},
{
"path": "demos/demo/config/geoenvironment.rb",
"chars": 602,
"preview": "\nBootStraps::Initializer.configure do |config|\n \n #Use the vendor directory\n config.vendored = true\n config.default_"
},
{
"path": "demos/demo/config.ru",
"chars": 187,
"preview": "require 'rubygems'\nrequire 'sinatra'\n\n\nSinatra::Application.default_options.merge!(\n :run => false,\n :env => ENV['RACK_E"
},
{
"path": "demos/demo/geocoder_helper.rb",
"chars": 110,
"preview": "require 'rubygems'\nrequire 'geocoder/us/database'\nrequire 'fastercsv'\nrequire 'json'\n\n\n\ndef initialize\n\n\n\nend\n"
},
{
"path": "demos/demo/geocom_geocode.rb",
"chars": 205,
"preview": "require 'config/bootstraps'\n\nmodule GeocomGeocode\n class GeocodeServer < Sinatra::Base\n register Sinatra::GeocodeWra"
},
{
"path": "demos/demo/main.rb",
"chars": 60,
"preview": "require 'geocom_geocode'\n\nGeocomGeocode::GeocodeServer.run!\n"
},
{
"path": "demos/demo/rakefile.rb",
"chars": 365,
"preview": "require 'rake'\n\ntask :boot_env do \n require 'config/bootstraps'; \nend\n\nnamespace :db do\n task :migrate => :connect do\n"
},
{
"path": "demos/demo/tmp/restart.txt",
"chars": 0,
"preview": ""
},
{
"path": "demos/parse.rb",
"chars": 83,
"preview": "require 'geocoder/us/address'\nrequire 'pp'\n\npp(Geocoder::US::Address.new(ARGV[0]))\n"
},
{
"path": "demos/simpledemo/views/index.builder",
"chars": 349,
"preview": "xml.locations do\n unless @records.nil?\n @records.each do |record|\n xml.location do\n xml.score format(\"%."
},
{
"path": "demos/simpledemo/views/index.erb",
"chars": 2204,
"preview": "<!DOCTYPE html\n PUBLIC \"-//W3C//DTD XHTML 1.0 Strict//EN\"\n \"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd\""
},
{
"path": "demos/simpledemo/ws.rb",
"chars": 1475,
"preview": "require 'rubygems'\nrequire 'sinatra'\nrequire 'geocoder/us/database'\nrequire 'fastercsv'\nrequire 'json'\n\nset :port, 8080\n"
},
{
"path": "doc/Makefile",
"chars": 150,
"preview": "all: lookup.html parsing.html\n\n%.html: %.rst voidspace.css\n\trst2html --stylesheet-path=voidspace.css --no-compact-lists "
},
{
"path": "doc/html4css1.css",
"chars": 5550,
"preview": "/*\n:Author: David Goodger\n:Contact: goodger@users.sourceforge.net\n:Date: $Date: 2005-12-18 01:56:14 +0100 (Sun, 18 Dec 2"
},
{
"path": "doc/lookup.rst",
"chars": 6637,
"preview": ".. _lookup:\n\n===================================\nGeocoder.us Address Lookup Strategy\n==================================="
},
{
"path": "doc/parsing.rst",
"chars": 3502,
"preview": ".. _parsing:\n\n====================================\nGeocoder.us Address Parsing Strategy\n================================"
},
{
"path": "doc/voidspace.css",
"chars": 2666,
"preview": "/*\n:Authors: Ian Bicking, Michael Foord\n:Contact: fuzzyman@voidspace.org.uk\n:Date: 2005/08/26 \n:Version: 0.1.0\n:Copyrigh"
},
{
"path": "gemspec",
"chars": 634,
"preview": "Gem::Specification.new do |s|\n s.name = 'Geocoder-US'\n s.version = \"2.0.4\"\n s.author = \"S"
},
{
"path": "lib/geocoder/us/address.rb",
"chars": 9136,
"preview": "require 'geocoder/us/constants'\n\nmodule Geocoder::US\n # Defines the matching of parsed address tokens.\n Match = {\n "
},
{
"path": "lib/geocoder/us/constants.rb",
"chars": 22330,
"preview": "# coding: utf-8\nrequire 'set'\nrequire 'geocoder/us/numbers'\n\nmodule Geocoder\nend\n\nmodule Geocoder::US\n class Map < Hash"
},
{
"path": "lib/geocoder/us/database.rb",
"chars": 30545,
"preview": "# require 'rubygems'\nrequire 'sqlite3'\n# require 'text'\n# require 'levenshtein'\n\nrequire 'set'\nrequire 'pp'\nrequire 'tim"
},
{
"path": "lib/geocoder/us/metaphone.rb",
"chars": 2280,
"preview": "module Text # :nodoc:\nmodule Metaphone\n\n module Rules # :nodoc:all\n \n # Metaphone rules. These are simply applie"
},
{
"path": "lib/geocoder/us/numbers.rb",
"chars": 1624,
"preview": "module Geocoder\nend\n\nmodule Geocoder::US\n # The NumberMap class provides a means for mapping ordinal\n # and cardinal n"
},
{
"path": "lib/geocoder/us/rest.rb",
"chars": 2905,
"preview": "require 'rubygems'\nrequire 'sinatra'\nrequire 'geocoder/us/database'\nrequire 'json'\n\n@@db = Geocoder::US::Database.new(EN"
},
{
"path": "lib/geocoder/us.rb",
"chars": 735,
"preview": "require \"geocoder/us/database\"\nrequire \"geocoder/us/address\"\n\n# Imports the Geocoder::US::Database and Geocoder::US::Add"
},
{
"path": "navteq/README",
"chars": 272,
"preview": "The navteq_import script in this directory is designed to be used with Navteq's\nlocal_streets layer. It works basically "
},
{
"path": "navteq/convert.sql",
"chars": 1429,
"preview": "BEGIN;\nCREATE INDEX navteq_link_id on local_streets (link_id);\n\nCREATE TEMPORARY TABLE linezip AS\n SELECT DISTINCT tl"
},
{
"path": "navteq/navteq_import",
"chars": 909,
"preview": "#!/bin/bash\n\nTMP=\"/tmp/navteq-import.$$\"\nSHPS=\"local_streets\"\nDBFS=\"\"\nBASE=$(dirname $0)\nPATH=$PATH:$BASE/../bin\nSQL=\"$B"
},
{
"path": "navteq/prepare.sql",
"chars": 2221,
"preview": "PRAGMA temp_store=MEMORY;\nPRAGMA journal_mode=MEMORY;\nPRAGMA synchronous=OFF;\nPRAGMA cache_size=250000;\nPRAGMA count_cha"
},
{
"path": "setup.rb",
"chars": 36162,
"preview": "#\n# setup.rb\n#\n# Copyright (c) 2000-2005 Minero Aoki\n#\n# This program is free software.\n# You can distribute/modify this"
},
{
"path": "src/Makefile",
"chars": 289,
"preview": "all:\n\t$(MAKE) -C libsqlite3_geocoder\n\t$(MAKE)\t-C liblwgeom\n\t$(MAKE) -C shp2sqlite\n\nclean:\n\t$(MAKE) -C libsqlite3_geocode"
},
{
"path": "src/README",
"chars": 461,
"preview": "What's in this directory\n------------------------\n\nshp2sqlite/\n A fork of shp2pgsql that generates SQLite 3 compatibl"
},
{
"path": "src/liblwgeom/Makefile",
"chars": 1525,
"preview": "# **********************************************************************\n# * $Id: Makefile.in \n# *\n# * PostGIS - Spatial"
},
{
"path": "src/liblwgeom/box2d.c",
"chars": 1077,
"preview": "#include <stdio.h>\n#include <stdlib.h>\n#include <string.h>\n#include <math.h>\n\n#include \"liblwgeom.h\"\n\n#ifndef EPSILON\n#d"
},
{
"path": "src/liblwgeom/lex.yy.c",
"chars": 191051,
"preview": "#line 2 \"lex.yy.c\"\n\n#line 4 \"lex.yy.c\"\n\n#define YY_INT_ALIGNED short int\n\n/* A lexical scanner generated by flex */\n\n#d"
},
{
"path": "src/liblwgeom/liblwgeom.h",
"chars": 48407,
"preview": "/**********************************************************************\n * $Id: liblwgeom.h 3812 2009-03-09 14:36:15Z pr"
},
{
"path": "src/liblwgeom/lwalgorithm.c",
"chars": 24072,
"preview": "/**********************************************************************\n * $Id: lwalgorithm.c 3812 2009-03-09 14:36:15Z "
},
{
"path": "src/liblwgeom/lwalgorithm.h",
"chars": 1865,
"preview": "/**********************************************************************\n * $Id: lwalgorithm.h 3688 2009-02-11 21:48:13Z "
},
{
"path": "src/liblwgeom/lwcircstring.c",
"chars": 23376,
"preview": "/**********************************************************************\n * $Id: lwcircstring.c 3639 2009-02-04 00:28:37Z"
},
{
"path": "src/liblwgeom/lwcollection.c",
"chars": 12313,
"preview": "/**********************************************************************\n * $Id: lwcollection.c 3812 2009-03-09 14:36:15Z"
},
{
"path": "src/liblwgeom/lwcompound.c",
"chars": 3830,
"preview": "/**********************************************************************\n * $Id: lwcompound.c 3639 2009-02-04 00:28:37Z p"
},
{
"path": "src/liblwgeom/lwcurvepoly.c",
"chars": 2796,
"preview": "/**********************************************************************\n * $Id: lwcurvepoly.c 3639 2009-02-04 00:28:37Z "
},
{
"path": "src/liblwgeom/lwgeom.c",
"chars": 21237,
"preview": "/**********************************************************************\n * $Id: lwgeom.c 3812 2009-03-09 14:36:15Z prams"
},
{
"path": "src/liblwgeom/lwgeom_api.c",
"chars": 45761,
"preview": "\n#include <math.h>\n#include <float.h>\n#include <string.h>\n#include <stdio.h>\n#include <errno.h>\n\n#include \"liblwgeom.h\"\n"
},
{
"path": "src/liblwgeom/lwgparse.c",
"chars": 24776,
"preview": "/*\n * Written by Ralph Mason ralph.mason<at>telogis.com\n *\n * Copyright Telogis 2004\n * www.telogis.com\n *\n */\n#include "
},
{
"path": "src/liblwgeom/lwgunparse.c",
"chars": 26185,
"preview": "/*\n * Written by Ralph Mason ralph.mason<at>telogis.com\n *\n * Copyright Telogis 2004\n * www.telogis.com\n *\n * $Id: lwgun"
},
{
"path": "src/liblwgeom/lwline.c",
"chars": 11596,
"preview": "/**********************************************************************\n * $Id: lwline.c 3639 2009-02-04 00:28:37Z prams"
},
{
"path": "src/liblwgeom/lwmcurve.c",
"chars": 3947,
"preview": "/**********************************************************************\n * $Id: lwmcurve.c 3639 2009-02-04 00:28:37Z pra"
},
{
"path": "src/liblwgeom/lwmline.c",
"chars": 2948,
"preview": "/**********************************************************************\n * $Id: lwmline.c 3639 2009-02-04 00:28:37Z pram"
},
{
"path": "src/liblwgeom/lwmpoint.c",
"chars": 2955,
"preview": "/**********************************************************************\n * $Id: lwmpoint.c 3639 2009-02-04 00:28:37Z pra"
},
{
"path": "src/liblwgeom/lwmpoly.c",
"chars": 3003,
"preview": "/**********************************************************************\n * $Id: lwmpoly.c 3639 2009-02-04 00:28:37Z pram"
},
{
"path": "src/liblwgeom/lwmsurface.c",
"chars": 4071,
"preview": "/**********************************************************************\n * $Id: lwmsurface.c 3639 2009-02-04 00:28:37Z p"
},
{
"path": "src/liblwgeom/lwpoint.c",
"chars": 9358,
"preview": "/**********************************************************************\n * $Id: lwpoint.c 3812 2009-03-09 14:36:15Z pram"
},
{
"path": "src/liblwgeom/lwpoly.c",
"chars": 12212,
"preview": "/**********************************************************************\n * $Id: lwpoly.c 3639 2009-02-04 00:28:37Z prams"
},
{
"path": "src/liblwgeom/lwsegmentize.c",
"chars": 36286,
"preview": "/**********************************************************************\n * $Id: lwsegmentize.c 3807 2009-03-08 21:15:00Z"
},
{
"path": "src/liblwgeom/lwutil.c",
"chars": 6779,
"preview": "#include <stdio.h>\n#include <stdlib.h>\n#include <stdarg.h>\n#include <string.h>\n\n\n/* Global variables */\n#include \"liblwg"
},
{
"path": "src/liblwgeom/measures.c",
"chars": 19151,
"preview": "/**********************************************************************\n * $Id: measures.c 3639 2009-02-04 00:28:37Z pra"
},
{
"path": "src/liblwgeom/postgis_config.h",
"chars": 2502,
"preview": "/* postgis_config.h. Generated from postgis_config.h.in by configure. */\n/* postgis_config.h.in. Generated from confi"
},
{
"path": "src/liblwgeom/ptarray.c",
"chars": 17480,
"preview": "/**********************************************************************\n * $Id: ptarray.c 3639 2009-02-04 00:28:37Z pram"
},
{
"path": "src/liblwgeom/vsprintf.c",
"chars": 3780,
"preview": "/* Like vsprintf but provides a pointer to malloc'd storage, which must\n be freed by the caller.\n Copyright (C) 1994"
},
{
"path": "src/liblwgeom/wktparse.h",
"chars": 3126,
"preview": "/*\n * Written by Ralph Mason ralph.mason<at>telogis.com\n *\n * Copyright Telogis 2004\n * www.telogis.com\n *\n */\n\n#ifndef "
},
{
"path": "src/liblwgeom/wktparse.lex",
"chars": 2965,
"preview": "/*\n * Written by Ralph Mason ralph.mason<at>telogis.com\n *\n * Copyright Telogis 2004\n * www.telogis.com\n *\n */\n\n%x vals_"
},
{
"path": "src/liblwgeom/wktparse.tab.c",
"chars": 69447,
"preview": "/* A Bison parser, made by GNU Bison 2.3. */\n\n/* Skeleton implementation for Bison's Yacc-like parsers in C\n\n Copyrig"
},
{
"path": "src/liblwgeom/wktparse.tab.h",
"chars": 4050,
"preview": "/* A Bison parser, made by GNU Bison 2.3. */\n\n/* Skeleton interface for Bison's Yacc-like parsers in C\n\n Copyright (C"
},
{
"path": "src/liblwgeom/wktparse.y",
"chars": 7623,
"preview": "/*\n * Written by Ralph Mason ralph.mason<at>telogis.com\n *\n * Copyright Telogis 2004\n * www.telogis.com\n *\n */\n\n%{\n#incl"
},
{
"path": "src/libsqlite3_geocoder/Makefile",
"chars": 552,
"preview": "all: libsqlite3_geocoder.so\nCC=gcc -fPIC \n\n#extension.o:\n#\t$(CC) -lm -lsqlite3 -shared $^ -o $@\n\nlibsqlite3_geocoder.so:"
},
{
"path": "src/libsqlite3_geocoder/Makefile.nix",
"chars": 360,
"preview": "all: libsqlite3_geocoder.so\n\nlibsqlite3_geocoder.so: extension.o wkb_compress.o util.o metaphon.o levenshtein.o\n\t$(CC) -"
},
{
"path": "src/libsqlite3_geocoder/Makefile.redhat",
"chars": 384,
"preview": "all: libsqlite3_geocoder.so\nCFLAGS=-fPIC\nlibsqlite3_geocoder.so: extension.o wkb_compress.o util.o metaphon.o levenshtei"
},
{
"path": "src/libsqlite3_geocoder/extension.c",
"chars": 4501,
"preview": "# include <sqlite3ext.h>\n# include <stdio.h>\n# include <string.h>\n# include <assert.h>\n# include <math.h>\n\n# include \"ex"
},
{
"path": "src/libsqlite3_geocoder/extension.h",
"chars": 480,
"preview": "#ifndef SQLITE3_GEOCODER\n#define SQLITE3_GEOCODER\n\n#include <stdint.h>\n\nint metaphone(const char *Word, char *Metaph, in"
},
{
"path": "src/libsqlite3_geocoder/levenshtein.c",
"chars": 1434,
"preview": "# include <string.h>\n# define STRLEN_MAX 256\n# define min(x, y) ((x) < (y) ? (x) : (y))\n# define max(x, y) ((x) > (y) ? "
},
{
"path": "src/libsqlite3_geocoder/metaphon.c",
"chars": 8758,
"preview": "/* +++Customized by SDE for sqlite3 use 09-Mar-2009 */\n/* +++File obtained from http://www.shedai.net/c/new/METAPHON.C *"
},
{
"path": "src/libsqlite3_geocoder/util.c",
"chars": 941,
"preview": "# include <string.h>\n# include <ctype.h>\n\nint address_metaphone(const char *input, char *output, int max_phones) {\n c"
},
{
"path": "src/libsqlite3_geocoder/wkb_compress.c",
"chars": 1481,
"preview": "#include <stdint.h>\n#include <string.h>\n\nuint32_t compress_wkb_line (void *dest, const void *src, uint32_t len) {\n ui"
},
{
"path": "src/metaphone/Makefile",
"chars": 110,
"preview": "all: metaphone.so\n\nmetaphone.so: extension.o metaphon.o\n\t$(CC) -fPIC -shared $^ -o $@\n\nclean:\n\trm -f *.o *.so\n"
},
{
"path": "src/metaphone/README",
"chars": 1557,
"preview": "= SQLite 3 Metaphone extension =\n\n * This library implements the Metaphone algorithm, originally developed by\n Laurenc"
},
{
"path": "src/metaphone/extension.c",
"chars": 1174,
"preview": "# include <sqlite3.h>\n# include <sqlite3ext.h>\n# include <stdio.h>\n# include <string.h>\n\nstatic SQLITE_EXTENSION_INIT1;\n"
},
{
"path": "src/metaphone/metaphon.c",
"chars": 8098,
"preview": "/* +++Customized by SDE for sqlite3 use 09-Mar-2009 */\n/* +++File obtained from http://www.shedai.net/c/new/METAPHON.C *"
},
{
"path": "src/shp2sqlite/Makefile",
"chars": 913,
"preview": "# **********************************************************************\n# * $Id: Makefile.in\n# *\n# * PostGIS - Spatial "
},
{
"path": "src/shp2sqlite/Makefile.macosx",
"chars": 1136,
"preview": "# **********************************************************************\n# * $Id: Makefile.in\n# *\n# * PostGIS - Spatial "
},
{
"path": "src/shp2sqlite/Makefile.nix",
"chars": 888,
"preview": "# **********************************************************************\n# * $Id: Makefile.in\n# *\n# * PostGIS - Spatial "
},
{
"path": "src/shp2sqlite/Makefile.redhat",
"chars": 894,
"preview": "# **********************************************************************\n# * $Id: Makefile.in\n# *\n# * PostGIS - Spatial "
},
{
"path": "src/shp2sqlite/dbfopen.c",
"chars": 55545,
"preview": "/******************************************************************************\n * $Id: dbfopen.c 3750 2009-02-19 21:12:"
},
{
"path": "src/shp2sqlite/getopt.c",
"chars": 19292,
"preview": "\n/* This version of `getopt' appears to the caller like standard Unix `getopt'\n but it behaves differently for the use"
},
{
"path": "src/shp2sqlite/getopt.h",
"chars": 4416,
"preview": "/* Declarations for getopt.\n Copyright (C) 1989, 1990, 1991, 1992, 1993 Free Software Foundation, Inc.\n\n This progra"
},
{
"path": "src/shp2sqlite/shapefil.h",
"chars": 16915,
"preview": "#ifndef _SHAPEFILE_H_INCLUDED\n#define _SHAPEFILE_H_INCLUDED\n\n/**********************************************************"
},
{
"path": "src/shp2sqlite/shp2sqlite.c",
"chars": 47027,
"preview": "/**********************************************************************\n * $Id: shp2pgsql.c 3623 2009-02-03 07:20:16Z pr"
},
{
"path": "src/shp2sqlite/shpopen.c",
"chars": 70082,
"preview": "/******************************************************************************\n * $Id: shpopen.c 2785 2008-05-27 15:08:"
},
{
"path": "test/address.rb",
"chars": 7759,
"preview": "$LOAD_PATH.unshift '../lib'\n\nrequire 'test/unit'\nrequire 'set'\nrequire 'geocoder/us/address'\n\ninclude Geocoder::US\n\nclas"
},
{
"path": "test/benchmark.rb",
"chars": 525,
"preview": "#!/usr/bin/ruby\n\nrequire 'test/unit'\nrequire 'geocoder/us/database'\nrequire 'benchmark'\ninclude Benchmark # we "
},
{
"path": "test/constants.rb",
"chars": 1592,
"preview": "$LOAD_PATH.unshift '../lib'\n\nrequire 'test/unit'\nrequire 'geocoder/us/constants'\n\ninclude Geocoder::US\n\nclass TestConsta"
},
{
"path": "test/data/address-sample.csv",
"chars": 4758,
"preview": "address,number,predir,prequal,pretyp,street,suftyp,sufqual,sufdir,unittyp,unit,city,state,zip,lon,lat,count,comment\n\"93 "
},
{
"path": "test/data/db-test.csv",
"chars": 6318,
"preview": "address,number,street,city,state,zip,lon,lat,count,comment\n\"93 NORTH 9TH STREET, BROOKLYN NY 11211\",93,N 9th St,Brooklyn"
},
{
"path": "test/data/locations.csv",
"chars": 151,
"preview": "name,address\n\"Home\",\"2026 21st St. N, Arlington, VA 22201\"\n\"Work\",\"2200 Wilson Blvd., Arlington, VA 22201\"\n\"RTI\",\"1506 N"
},
{
"path": "test/database.rb",
"chars": 5543,
"preview": "$LOAD_PATH.unshift '../lib'\n\nrequire 'test/unit'\nrequire 'geocoder/us/database'\nrequire 'csv'\n\nBase = File.dirname(__FIL"
},
{
"path": "test/generate.rb",
"chars": 1048,
"preview": "#!/usr/bin/ruby\n\nrequire 'test/unit'\nrequire 'geocoder/us/database'\nrequire 'fastercsv'\n\ndb = Geocoder::US::Database.new"
},
{
"path": "test/numbers.rb",
"chars": 1416,
"preview": "$LOAD_PATH.unshift '../lib'\n\nrequire 'test/unit'\nrequire 'geocoder/us/numbers'\n\ninclude Geocoder::US\n\nclass TestAddress "
},
{
"path": "test/run.rb",
"chars": 238,
"preview": "#!/usr/bin/ruby \n\n$LOAD_PATH << File.dirname(__FILE__)\n$LOAD_PATH << File.dirname(__FILE__) + \"/../lib\"\nputs \"Loadpath=#"
}
]
// ... and 1 more files (download for full content)
About this extraction
This page contains the full source code of the geocommons/geocoder GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 137 files (12.4 MB), approximately 373.0k tokens, and a symbol index with 1019 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.