Full Code of cardmagic/contacts for AI

Repository: cardmagic/contacts
Branch: master
Commit: e706105e24f0
Files: 28
Total size: 43.7 KB

Directory structure:
gitextract_qyvd4b8y/

├── .gitignore
├── .travis.yml
├── Gemfile
├── LICENSE
├── README.md
├── Rakefile
├── contacts.gemspec
├── cruise_config.rb
├── examples/
│   └── grab_contacts.rb
├── geminstaller.yml
├── lib/
│   ├── contacts/
│   │   ├── aol.rb
│   │   ├── base.rb
│   │   ├── gmail.rb
│   │   ├── hotmail.rb
│   │   ├── json_picker.rb
│   │   ├── mailru.rb
│   │   ├── plaxo.rb
│   │   └── yahoo.rb
│   └── contacts.rb
└── test/
    ├── example_accounts.yml
    ├── test_helper.rb
    ├── test_suite.rb
    └── unit/
        ├── aol_contact_importer_test.rb
        ├── gmail_contact_importer_test.rb
        ├── hotmail_contact_importer_test.rb
        ├── mailru_contact_importer_test.rb
        ├── test_accounts_test.rb
        └── yahoo_csv_contact_importer_test.rb

================================================
FILE CONTENTS
================================================

================================================
FILE: .gitignore
================================================
.DS_Store
pkg/*
test/accounts.yml
nbproject/


================================================
FILE: .travis.yml
================================================
language: ruby
rvm:
  - 1.9.3
  - 1.9.2
  - jruby-18mode
  - jruby-19mode
  - rbx-18mode
  - rbx-19mode
  - ruby-head
  - jruby-head
  - 1.8.7
  - ree



================================================
FILE: Gemfile
================================================
source 'https://rubygems.org'
gem 'json', ">= 1.1.1"
gem 'gdata_19', '1.1.5'
gem 'rspec', :require => 'spec'
gem 'rake'
gem 'hpricot'


================================================
FILE: LICENSE
================================================
Copyright (c) 2006, Lucas Carlson, MOG
All rights reserved.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
Neither the name of the Lucas Carlson nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.



================================================
FILE: README.md
================================================
Welcome to Contacts
===================

Contacts is a universal interface to grab contact list information from various providers including Hotmail, AOL, Gmail, Plaxo and Yahoo.

Download
--------

* gem install contacts
* http://github.com/cardmagic/contacts
* git clone git://github.com/cardmagic/contacts.git

Background
----------

For a long time, the only way to get a list of contacts from your free online email accounts was with proprietary PHP scripts that would cost you $50. The act of grabbing that list is a simple matter of screen scrapping and this library gives you all the functionality you need. Thanks to the generosity of the highly popular Rails website MOG (http://mog.com) for allowing this library to be released open-source to the world. It is easy to extend this library to add new free email providers, so please contact the author if you would like to help.

Usage
-----

<pre>
 <code>
   Contacts::Hotmail.new(login, password).contacts # => [["name", "foo@bar.com"], ["another name", "bow@wow.com"]]
   Contacts::Yahoo.new(login, password).contacts
   Contacts::Gmail.new(login, password).contacts
   Contacts.new(:gmail, login, password).contacts
   Contacts.new(:hotmail, login, password).contacts
   Contacts.new(:yahoo, login, password).contacts
   Contacts.guess(login, password).contacts
 </code>
</pre>

Notice there are three ways to use this library so that you can limit the use as much as you would like in your particular application. The Contacts.guess method will automatically concatenate all the address book contacts from each of the successful logins in the case that a username password works across multiple services.

Captcha error
-------------

If there are too many failed attempts with the gmail login info, Google will raise a captcha response. To integrate the captcha handling, pass in the token and response via:
<pre><code>
  Contacts::Gmail.new(login, password, :captcha_token => params[:captcha_token], :captcha_response => params[:captcha_response]).contacts
</code></pre>

Examples
--------

See the examples/ directory.

Authors
-------

> Lucas Carlson from MOG (mailto:lucas@rufy.com) - http://mog.com

Contributors
------------

* Britt Selvitelle from Twitter (mailto:anotherbritt@gmail.com) - http://twitter.com
* Tony Targonski from GigPark (mailto:tony@gigpark.com) - http://gigpark.com
* Waheed Barghouthi from Watwet (mailto:waheed.barghouthi@gmail.com) - http://watwet.com
* Glenn Sidney from Glenn Fu (mailto:glenn@glennfu.com) - http://glennfu.com
* Brian McQuay from Onomojo (mailto:brian@onomojo.com) - http://onomojo.com
* Adam Hunter (mailto:adamhunter@me.com) - http://adamhunter.me/
* Glenn Ford (mailto:glenn@glennfu.com) - http://www.glennfu.com/
* Leonardo Wong (mailto:mac@boy.name)
* Rusty Burchfield
* justintv

This library is released under the terms of the BSD.



================================================
FILE: Rakefile
================================================
require 'rubygems'
require 'bundler/setup'
require 'rake'
require 'rake/testtask'
require 'rake/rdoctask'
require 'rake/gempackagetask'
require 'rake/contrib/rubyforgepublisher'
require './lib/contacts'

PKG_VERSION = Contacts::VERSION

PKG_FILES = FileList[
    "lib/**/*", "bin/*", "test/**/*", "[A-Z]*", "Rakefile", "doc/**/*", "examples/**/*"
] - ["test/accounts.yml"]

desc "Default Task"
task :default => [ :test ]

# Run the unit tests
desc "Run all unit tests"
Rake::TestTask.new("test") { |t|
  t.libs << "lib"
  t.pattern = 'test/*/*_test.rb'
  t.verbose = true
}

# Make a console, useful when working on tests
desc "Generate a test console"
task :console do
   verbose( false ) { sh "irb -I lib/ -r 'contacts'" }
end

# Genereate the RDoc documentation
desc "Create documentation"
Rake::RDocTask.new("doc") { |rdoc|
  rdoc.title = "Contact List - ridiculously easy contact list information from various providers including Yahoo, Gmail, and Hotmail"
  rdoc.rdoc_dir = 'doc'
  rdoc.rdoc_files.include('README')
  rdoc.rdoc_files.include('lib/**/*.rb')
}

# Genereate the package
spec = Gem::Specification.new do |s|

  #### Basic information.

  s.name = 'adamhunter-contacts'
  s.version = PKG_VERSION
  s.summary = <<-EOF
   Ridiculously easy contact list information from various providers including Yahoo, Gmail, and Hotmail
  EOF
  s.description = <<-EOF
   Ridiculously easy contact list information from various providers including Yahoo, Gmail, and Hotmail
  EOF

  #### Which files are to be included in this gem?  Everything!  (Except CVS directories.)

  s.files = PKG_FILES

  #### Load-time details: library and application (you will need one or both).

  s.require_path = 'lib'
  s.autorequire = 'contacts'

  s.add_dependency('json', '>= 0.4.1')
  s.add_dependency('gdata', '= 1.1.1')
  s.requirements << "A json parser, the gdata ruby gem"

  #### Documentation and testing.

  s.has_rdoc = true

  #### Author and project details.

  s.author = "Lucas Carlson"
  s.email = "lucas@rufy.com"
  s.homepage = "http://rubyforge.org/projects/contacts"
end

Rake::GemPackageTask.new(spec) do |pkg|
  pkg.need_zip = true
  pkg.need_tar = true
end

desc "Report code statistics (KLOCs, etc) from the application"
task :stats do
  require 'code_statistics'
  CodeStatistics.new(
    ["Library", "lib"],
    ["Units", "test"]
  ).to_s
end


================================================
FILE: contacts.gemspec
================================================
Gem::Specification.new do |s|
  s.name = "contacts"
  s.version = "1.2.4"
  s.date = "2010-07-06"
  s.summary = "A universal interface to grab contact list information from various providers including Yahoo, AOL, Gmail, Hotmail, and Plaxo."
  s.email = "lucas@rufy.com"
  s.homepage = "http://github.com/cardmagic/contacts"
  s.description = "A universal interface to grab contact list information from various providers including Yahoo, AOL, Gmail, Hotmail, and Plaxo."
  s.has_rdoc = false
  s.authors = ["Lucas Carlson"]
  s.files = ["LICENSE", "Rakefile", "README", "examples/grab_contacts.rb", "lib/contacts.rb", "lib/contacts/base.rb", "lib/contacts/json_picker.rb", "lib/contacts/gmail.rb", "lib/contacts/aol.rb", "lib/contacts/hotmail.rb", "lib/contacts/plaxo.rb", "lib/contacts/yahoo.rb"]
  s.add_dependency("json", ">= 1.1.1")
  s.add_dependency('gdata', '1.1.2')
end


================================================
FILE: cruise_config.rb
================================================
# Project-specific configuration for CruiseControl.rb

Project.configure do |project|

  # Send email notifications about broken and fixed builds to email1@your.site, email2@your.site (default: send to nobody)
  # if building this on your own CI box, please remove!
   project.email_notifier.emails = ['opensource@pivotallabs.com']

  # Set email 'from' field to john@doe.com:
  # project.email_notifier.from = 'john@doe.com'

  # Build the project by invoking rake task 'custom'
  # project.rake_task = 'custom'

  # Build the project by invoking shell script "build_my_app.sh". Keep in mind that when the script is invoked, current working directory is
  # [cruise]/projects/your_project/work, so if you do not keep build_my_app.sh in version control, it should be '../build_my_app.sh' instead
  # project.build_command = 'build_my_app.sh'

  # Ping Subversion for new revisions every 5 minutes (default: 30 seconds)
  # project.scheduler.polling_interval = 5.minutes

end

================================================
FILE: examples/grab_contacts.rb
================================================
require File.dirname(__FILE__)+"/../lib/contacts"

login = ARGV[0]
password = ARGV[1]

Contacts::Gmail.new(login, password).contacts

Contacts.new(:gmail, login, password).contacts

Contacts.new("gmail", login, password).contacts

Contacts.guess(login, password).contacts

================================================
FILE: geminstaller.yml
================================================
---
defaults:
  install_options: --no-ri --no-rdoc
gems:
  - name: json
    version: >= 1.1.1
  - name: gdata
    version: >= 1.1.1


================================================
FILE: lib/contacts/aol.rb
================================================
class Contacts
  require 'hpricot'
  require 'csv'
  class Aol < Base
    URL                 = "http://www.aol.com/"
    LOGIN_URL           = "https://my.screenname.aol.com/_cqr/login/login.psp"
    LOGIN_REFERER_URL   = "http://webmail.aol.com/"
    LOGIN_REFERER_PATH = "sitedomain=sns.webmail.aol.com&lang=en&locale=us&authLev=0&uitype=mini&loginId=&redirType=js&xchk=false"
    AOL_NUM = "29970-343" # this seems to change each time they change the protocol
    
    CONTACT_LIST_URL    = "http://webmail.aol.com/#{AOL_NUM}/aim-2/en-us/Lite/ContactList.aspx?folder=Inbox&showUserFolders=False"
    CONTACT_LIST_CSV_URL = "http://webmail.aol.com/#{AOL_NUM}/aim-2/en-us/Lite/ABExport.aspx?command=all"
    PROTOCOL_ERROR      = "AOL has changed its protocols, please upgrade this library first. If that does not work, dive into the code and submit a patch at http://github.com/cardmagic/contacts"
    
    def real_connect
      if login.strip =~ /^(.+)@aol\.com$/ # strip off the @aol.com for AOL logins
        login = $1
      end

      postdata = {
        "loginId" => login,
        "password" => password,
        "rememberMe" => "on",
        "_sns_fg_color_" => "",
        "_sns_err_color_" => "",
        "_sns_link_color_" => "",
        "_sns_width_" => "",
        "_sns_height_" => "",
        "offerId" => "mail-second-en-us",
        "_sns_bg_color_" => "",
        "sitedomain" => "sns.webmail.aol.com",
        "regPromoCode" => "",
        "mcState" => "initialized",
        "uitype" => "std",
        "siteId" => "",
        "lang" => "en",
        "locale" => "us",
        "authLev" => "0",
        "siteState" => "",
        "isSiteStateEncoded" => "false",
        "use_aam" => "0",
        "seamless" => "novl",
        "aolsubmit" => CGI.escape("Sign In"),
        "idType" => "SN",
        "usrd" => "",
        "doSSL" => "",
        "redirType" => "",
        "xchk" => "false"
      }
 
      # Get this cookie and stick it in the form to confirm to Aol that your cookies work
      data, resp, cookies, forward = get(URL)
      postdata["stips"] = cookie_hash_from_string(cookies)["stips"]
      postdata["tst"] = cookie_hash_from_string(cookies)["tst"]
 
      data, resp, cookies, forward, old_url = get(LOGIN_REFERER_URL, cookies) + [URL]
      until forward.nil?
        data, resp, cookies, forward, old_url = get(forward, cookies, old_url) + [forward]
      end
      
      data, resp, cookies, forward, old_url = get("#{LOGIN_URL}?#{LOGIN_REFERER_PATH}", cookies) + [LOGIN_REFERER_URL]
      until forward.nil?
        data, resp, cookies, forward, old_url = get(forward, cookies, old_url) + [forward]
      end
 
      doc = Hpricot(data)
      (doc/:input).each do |input|
        postdata["usrd"] = input.attributes["value"] if input.attributes["name"] == "usrd"
      end
      # parse data for <input name="usrd" value="2726212" type="hidden"> and add it to the postdata
 
      postdata["SNS_SC"] = cookie_hash_from_string(cookies)["SNS_SC"]
      postdata["SNS_LDC"] = cookie_hash_from_string(cookies)["SNS_LDC"]
      postdata["LTState"] = cookie_hash_from_string(cookies)["LTState"]
      # raise data.inspect
      
      data, resp, cookies, forward, old_url = post(LOGIN_URL, h_to_query_string(postdata), cookies, LOGIN_REFERER_URL) + [LOGIN_REFERER_URL]
      
      until forward.nil?
        data, resp, cookies, forward, old_url = get(forward, cookies, old_url) + [forward]
      end
      
      if data.index("Invalid Username or Password. Please try again.")
        raise AuthenticationError, "Username and password do not match"
      elsif data.index("Required field must not be blank")
        raise AuthenticationError, "Login and password must not be blank"
      elsif data.index("errormsg_0_logincaptcha")
        raise AuthenticationError, "Captcha error"
      elsif data.index("Invalid request")
        raise ConnectionError, PROTOCOL_ERROR
      elsif cookies == ""
        raise ConnectionError, PROTOCOL_ERROR
      end
 
      @cookies = cookies
    end
 
    def contacts
      postdata = {
        "file" => 'contacts',
        "fileType" => 'csv'
      }
 
      return @contacts if @contacts
      if connected?
        data, resp, cookies, forward, old_url = get(CONTACT_LIST_URL, @cookies, CONTACT_LIST_URL) + [CONTACT_LIST_URL]
 
        until forward.nil?
          data, resp, cookies, forward, old_url = get(forward, cookies, old_url) + [forward]
        end
        
        if resp.code_type != Net::HTTPOK
          raise ConnectionError, self.class.const_get(:PROTOCOL_ERROR)
        end
 
        # parse data and grab <input name="user" value="8QzMPIAKs2" type="hidden">
        doc = Hpricot(data)
        (doc/:input).each do |input|
          postdata["user"] = input.attributes["value"] if input.attributes["name"] == "user"
        end
        
        data, resp, cookies, forward, old_url = get(CONTACT_LIST_CSV_URL, @cookies, CONTACT_LIST_URL) + [CONTACT_LIST_URL]
 
        until forward.nil?
          data, resp, cookies, forward, old_url = get(forward, cookies, old_url) + [forward]
        end
        
        if data.include?("error.gif")
          raise AuthenticationError, "Account invalid"
        end
        
        parse data
      end
    end
  private
    
    def parse(data, options={})
      data = CSV::Reader.parse(data)
      col_names = data.shift
      @contacts = data.map do |person|
        ["#{person[0]} #{person[1]}", person[4]] if person[4] && !person[4].empty?
      end.compact
    end    
 
    def h_to_query_string(hash)
      u = ERB::Util.method(:u)
      hash.map { |k, v|
        u.call(k) + "=" + u.call(v)
      }.join("&")
    end
  end
 
  TYPES[:aol] = Aol
end

================================================
FILE: lib/contacts/base.rb
================================================
require "cgi"
require "net/http"
require "net/https"
require "uri"
require "zlib"
require "stringio"
require "thread"
require "erb"

class Contacts
  TYPES = {}
  VERSION = "1.2.4"
  
  class Base
    def initialize(login, password, options={})
      @login = login
      @password = password
      @captcha_token = options[:captcha_token]
      @captcha_response = options[:captcha_response]
      @connections = {}
      connect
    end
    
    def connect
      raise AuthenticationError, "Login and password must not be nil, login: #{@login.inspect}, password: #{@password.inspect}" if @login.nil? || @login.empty? || @password.nil? || @password.empty?
      real_connect
    end
    
    def connected?
      @cookies && !@cookies.empty?
    end

    def contacts(options = {})
      return @contacts if @contacts
      if connected?
        url = URI.parse(contact_list_url)
        http = open_http(url)
        resp, data = http.get("#{url.path}?#{url.query}",
          "Cookie" => @cookies
        )
        
        if resp.code_type != Net::HTTPOK
          raise ConnectionError, self.class.const_get(:PROTOCOL_ERROR)
        end
        
        parse(data, options)
      end
    end
    
    def login
      @attempt ||= 0
      @attempt += 1
            
      if @attempt == 1
        @login
      else
        if @login.include?("@#{domain}")
          @login.sub("@#{domain}","")
        else
          "#{@login}@#{domain}"
        end
      end
    end
    
    def password
      @password
    end

    def skip_gzip?
      false
    end
    
  private
  
    def domain
      @d ||= URI.parse(self.class.const_get(:URL)).host.sub(/^www\./,'')
    end

    def contact_list_url
      self.class.const_get(:CONTACT_LIST_URL)
    end

    def address_book_url
      self.class.const_get(:ADDRESS_BOOK_URL)
    end

    def open_http(url)
      c = @connections[Thread.current.object_id] ||= {}
      http = c["#{url.host}:#{url.port}"]
      unless http
        http = Net::HTTP.new(url.host, url.port)
        if url.port == 443
          http.use_ssl = true
          http.verify_mode = OpenSSL::SSL::VERIFY_NONE
        end
        c["#{url.host}:#{url.port}"] = http
      end
      http.start unless http.started?
      http
    end
    
    def cookie_hash_from_string(cookie_string)
      cookie_string.split(";").map{|i|i.split("=", 2).map{|j|j.strip}}.inject({}){|h,i|h[i[0]]=i[1];h}
    end
    
    def parse_cookies(data, existing="")
      return existing if data.nil?

      cookies = cookie_hash_from_string(existing)
      
      data.gsub!(/ ?[\w]+=EXPIRED;/,'')
      data.gsub!(/ ?expires=(.*?, .*?)[;,$]/i, ';')
      data.gsub!(/ ?(domain|path)=[\S]*?[;,$]/i,';')
      data.gsub!(/[,;]?\s*(secure|httponly)/i,'')
      data.gsub!(/(;\s*){2,}/,', ')
      data.gsub!(/(,\s*){2,}/,', ')
      data.sub!(/^,\s*/,'')
      data.sub!(/\s*,$/,'')
      
      data.split(", ").map{|t|t.to_s.split(";").first}.each do |data|
        k, v = data.split("=", 2).map{|j|j.strip}
        if cookies[k] && v.empty?
          cookies.delete(k)
        elsif v && !v.empty?
          cookies[k] = v
        end
      end
      
      cookies.map{|k,v| "#{k}=#{v}"}.join("; ")
    end
    
    def remove_cookie(cookie, cookies)
      parse_cookies("#{cookie}=", cookies)
    end
    
    def post(url, postdata, cookies="", referer="")
      url = URI.parse(url)
      http = open_http(url)
      http_header = { "User-Agent" => "Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1) Gecko/20061010 Firefox/2.0",
        "Accept-Encoding" => "gzip",
        "Cookie" => cookies,
        "Referer" => referer,
        "Content-Type" => 'application/x-www-form-urlencoded'
      }
      http_header.reject!{|k, v| k == 'Accept-Encoding'} if skip_gzip?
      resp, data = http.post(url.path, postdata, http_header)
      data = uncompress(resp, data)
      cookies = parse_cookies(resp.response['set-cookie'], cookies)
      forward = resp.response['Location']
      forward ||= (data =~ /<meta.*?url='([^']+)'/ ? CGI.unescapeHTML($1) : nil)
	if (not forward.nil?) && URI.parse(forward).host.nil?
		forward = url.scheme.to_s + "://" + url.host.to_s + forward
	end
      return data, resp, cookies, forward
    end
    
    def get(url, cookies="", referer="")
      url = URI.parse(url)
      attempt = 0
      begin
        http = open_http(url)
        resp, data = http.get("#{url.path}?#{url.query}",
          "User-Agent" => "Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1) Gecko/20061010 Firefox/2.0",
          "Accept-Encoding" => "gzip",
          "Cookie" => cookies,
          "Referer" => referer
        )
      rescue EOFError => err
        attempt += 1
        retry if attempt == 1
      end
      data = uncompress(resp, data)
      cookies = parse_cookies(resp.response['set-cookie'], cookies)
      forward = resp.response['Location']
	  if (not forward.nil?) && URI.parse(forward).host.nil?
		forward = url.scheme.to_s + "://" + url.host.to_s + forward
	  end
      return data, resp, cookies, forward
    end
    
    def uncompress(resp, data)
      case resp.response['content-encoding']
      when 'gzip'
        gz = Zlib::GzipReader.new(StringIO.new(data))
        data = gz.read
        gz.close
        resp.response['content-encoding'] = nil
      # FIXME: Not sure what Hotmail was feeding me with their 'deflate',
      #        but the headers definitely were not right
      when 'deflate'
        data = Zlib::Inflate.inflate(data)
        resp.response['content-encoding'] = nil
      end

      data
    end
  end
  
  class ContactsError < StandardError
  end
  
  class AuthenticationError < ContactsError
  end

  class ConnectionError < ContactsError
  end
  
  class TypeNotFound < ContactsError
  end
  
  def self.new(type, login, password, options={})
    if TYPES.include?(type.to_s.intern)
      TYPES[type.to_s.intern].new(login, password, options)
    else
      raise TypeNotFound, "#{type.inspect} is not a valid type, please choose one of the following: #{TYPES.keys.inspect}"
    end
  end
  
  def self.guess(login, password, options={})
    TYPES.inject([]) do |a, t|
      begin
        a + t[1].new(login, password, options).contacts
      rescue AuthenticationError
        a
      end
    end.uniq
  end
end


================================================
FILE: lib/contacts/gmail.rb
================================================
require 'gdata'

class Contacts
  class Gmail < Base
    
    CONTACTS_SCOPE = 'http://www.google.com/m8/feeds/'
    CONTACTS_FEED = CONTACTS_SCOPE + 'contacts/default/full/?max-results=1000'
    
    def contacts
      return @contacts if @contacts
    end
    
    def real_connect
      @client = GData::Client::Contacts.new
      @client.clientlogin(@login, @password, @captcha_token, @captcha_response)
      
      feed = @client.get(CONTACTS_FEED).to_xml
      
      @contacts = feed.elements.to_a('entry').collect do |entry|
        title, email = entry.elements['title'].text, nil
        entry.elements.each('gd:email') do |e|
          email = e.attribute('address').value if e.attribute('primary')
        end
        [title, email] unless email.nil?
      end
      @contacts.compact!
    rescue GData::Client::AuthorizationError => e
      raise AuthenticationError, "Username or password are incorrect"
    end
    
    private
    
    TYPES[:gmail] = Gmail
  end
end

================================================
FILE: lib/contacts/hotmail.rb
================================================
class Contacts
  class Hotmail < Base
    URL                 = "https://login.live.com/login.srf?id=2"
    OLD_CONTACT_LIST_URL = "http://%s/cgi-bin/addresses"
    NEW_CONTACT_LIST_URL = "http://%s/mail/GetContacts.aspx"
    CONTACT_LIST_URL = "http://mpeople.live.com/default.aspx?pg=0" 
    COMPOSE_URL         = "http://%s/cgi-bin/compose?"
    PROTOCOL_ERROR      = "Hotmail has changed its protocols, please upgrade this library first. If that does not work, report this error at http://rubyforge.org/forum/?group_id=2693"
    PWDPAD = "IfYouAreReadingThisYouHaveTooMuchFreeTime"
    MAX_HTTP_THREADS    = 8
    
    def real_connect
      data, resp, cookies, forward = get(URL)
      old_url = URL
      until forward.nil?
        data, resp, cookies, forward, old_url = get(forward, cookies, old_url) + [forward]
      end
      
      postdata =  "PPSX=%s&PwdPad=%s&login=%s&passwd=%s&LoginOptions=2&PPFT=%s" % [
        CGI.escape(data.split("><").grep(/PPSX/).first[/=\S+$/][2..-3]),
        PWDPAD[0...(PWDPAD.length-@password.length)],
        CGI.escape(login),
        CGI.escape(password),
        CGI.escape(data.split("><").grep(/PPFT/).first[/=\S+$/][2..-3])
      ]
      
      form_url = data.split("><").grep(/form/).first.split[5][8..-2]
      data, resp, cookies, forward = post(form_url, postdata, cookies)
      
      old_url = form_url
      until cookies =~ /; PPAuth=/ || forward.nil?
        data, resp, cookies, forward, old_url = get(forward, cookies, old_url) + [forward]
      end
      
      if data.index("The e-mail address or password is incorrect")
        raise AuthenticationError, "Username and password do not match"
      elsif data != ""
        raise AuthenticationError, "Required field must not be blank"
      elsif cookies == ""
        raise ConnectionError, PROTOCOL_ERROR
      end
            
      data, resp, cookies, forward = get("http://mail.live.com/mail", cookies)
      until forward.nil?
        data, resp, cookies, forward, old_url = get(forward, cookies, old_url) + [forward]
      end
      
      
      @domain = URI.parse(old_url).host
      @cookies = cookies
    rescue AuthenticationError => m
      if @attempt == 1
        retry
      else
        raise m
      end
    end
    
    def contacts(options = {})
      if connected?
        @contacts = []
        build_contacts = []
        go = true
        index = 0
        
        while(go) do
          go = false
          
          data, resp, cookies, forward = get( get_contact_list_url(index), @cookies )

          email_match_text_beginning = Regexp.escape("http://m.mail.live.com/?rru=compose&amp;to=")
          email_match_text_end = Regexp.escape("&amp;ru=")
          
          raw_html = resp.body.split("
").grep(/(?:e|dn)lk[0-9]+/)
          raw_html.inject(-1) do |memo, row|
            c_info = row.match(/(e|dn)lk([0-9])+/)
            
            # Same contact, or different?
            build_contacts << [] if memo != c_info[2]
            
            # Grab info
            case c_info[1]
              when "e" # Email
                build_contacts.last[1] = row.match(/#{email_match_text_beginning}(.*)#{email_match_text_end}/)[1]
              when "dn" # Name
                build_contacts.last[0] = row.match(/<a[^>]*>(.+)<\/a>/)[1]
            end
            
            # Set memo to contact id
            c_info[2]
          end
          
          go = resp.body.include?("ContactList_next")
          index += 1
        end
        
        build_contacts.each do |contact|
          unless contact[1].nil?
            # Only return contacts with email addresses
            contact[1] = CGI::unescape CGI::unescape(contact[1])
            @contacts << contact
          end
        end
        
        return @contacts
      end
    end
    
    def get_contact_list_url(index) 
      "http://mpeople.live.com/default.aspx?pg=#{index}"
    end
    
    private
    
    TYPES[:hotmail] = Hotmail
  end
end

================================================
FILE: lib/contacts/json_picker.rb
================================================
if !Object.const_defined?('ActiveSupport')
  require 'json'
end

class Contacts
  def self.parse_json( string )
    if Object.const_defined?('ActiveSupport') and
       ActiveSupport.const_defined?('JSON')
      ActiveSupport::JSON.decode( string )
    elsif Object.const_defined?('JSON')
      JSON.parse( string )
    else
      raise 'Contacts requires JSON or Rails (with ActiveSupport::JSON)'
    end
  end
end

================================================
FILE: lib/contacts/mailru.rb
================================================
require 'csv'

class Contacts
  class Mailru < Base
    LOGIN_URL = "https://auth.mail.ru/cgi-bin/auth"
    ADDRESS_BOOK_URL = "http://win.mail.ru/cgi-bin/abexport/addressbook.csv"

    attr_accessor :cookies

    def real_connect
      username = login
      
      postdata =  "Login=%s&Domain=%s&Password=%s" % [
        CGI.escape(username),
        CGI.escape(domain_param(username)),
        CGI.escape(password)
      ]

      data, resp, self.cookies, forward = post(LOGIN_URL, postdata, "")

      if data.index("fail=1")
        raise AuthenticationError, "Username and password do not match"
      elsif cookies == "" or data == ""
        raise ConnectionError, PROTOCOL_ERROR
      end

      data, resp, cookies, forward = get(login_token_link(data), login_cookies.join(';'))
    end

    def contacts
      postdata = "confirm=1&abtype=6"
      data, resp, cookies, forward = post(ADDRESS_BOOK_URL, postdata, login_cookies.join(';'))

      @contacts = []
      CSV.parse(data) do |row|
        @contacts << [row[0], row[4]] unless header_row?(row)
      end

      @contacts
    end

    def skip_gzip?
      true
    end

    private
    def login_token_link(data)
      data.match(/url=(.+)\">/)[1]
    end

    def login_cookies
      self.cookies.split(';').collect{|c| c if (c.include?('t=') or c.include?('Mpop='))}.compact.collect{|c| c.strip}
    end

    def header_row?(row)
      row[0] == 'AB-Name'
    end

    def domain_param(login)
      login.include?('@') ?
        login.match(/.+@(.+)/)[1] :
        'mail.ru'
    end

  end

  TYPES[:mailru] = Mailru
end

================================================
FILE: lib/contacts/plaxo.rb
================================================
require 'rexml/document'

class Contacts
  class Plaxo < Base
    URL                 = "http://www.plaxo.com/"
    LOGIN_URL           = "https://www.plaxo.com/signin"
    ADDRESS_BOOK_URL    = "http://www.plaxo.com/po3/?module=ab&operation=viewFull&mode=normal"
    CONTACT_LIST_URL    = "http://www.plaxo.com/axis/soap/contact?_action=getContacts&_format=xml"
    PROTOCOL_ERROR      = "Plaxo has changed its protocols, please upgrade this library first. If that does not work, dive into the code and submit a patch at http://github.com/cardmagic/contacts"
    
    def real_connect
      
    end # real_connect
    
    def contacts
      getdata = "&authInfo.authByEmail.email=%s" % CGI.escape(login)
      getdata += "&authInfo.authByEmail.password=%s" % CGI.escape(password)
      data, resp, cookies, forward = get(CONTACT_LIST_URL + getdata)
      
      if resp.code_type != Net::HTTPOK
        raise ConnectionError, PROTOCOL_ERROR
      end
      
      parse data
    end # contacts
    
  private
    def parse(data, options={})
      doc = REXML::Document.new(data)
      code = doc.elements['//response/code'].text
      
      if code == '401'
        raise AuthenticationError, "Username and password do not match"
      elsif code == '200'
        @contacts = []
        doc.elements.each('//contact') do |cont|
          name = if cont.elements['fullName']
            cont.elements['fullName'].text
          elsif cont.elements['displayName']
            cont.elements['displayName'].text
          end
          email = if cont.elements['email1']
            cont.elements['email1'].text
          end
          if name || email
            @contacts << [name, email]
          end
        end
        @contacts
      else
        raise ConnectionError, PROTOCOL_ERROR
      end
      
    end # parse

  end # Plaxo
  
  TYPES[:plaxo] = Plaxo
  
end # Contacts


# sample contacts responses
=begin
Bad email
=========
<?xml version="1.0" encoding="utf-8" ?>
<ns1:GetContactsResponse xmlns:ns1="Plaxo">
  <response>
    <code>401</code>
    <subCode>1</subCode>
    <message>User not found.</message>
  </response>
</ns1:GetContactsResponse>


Bad password
============
<?xml version="1.0" encoding="utf-8" ?>
<ns1:GetContactsResponse xmlns:ns1="Plaxo">
  <response>
    <code>401</code>
    <subCode>4</subCode>
    <message>Bad password or security token.</message>
  </response>
</ns1:GetContactsResponse>


Success
=======
<?xml version="1.0" encoding="utf-8" ?>
<ns1:GetContactsResponse xmlns:ns1="Plaxo">

  <response>
    <code>200</code>
    <message>OK</message>
    <userId>77311236242</userId>
  </response>
  
  <contacts>

    <contact>
      <itemId>61312569</itemId>
      <displayName>Joe Blow1</displayName>
      <fullName>Joe Blow1</fullName>
      <firstName>Joe</firstName>
      <lastName>Blow1</lastName>
      <homeEmail1>joeblow1@mailinator.com</homeEmail1>
      <email1>joeblow1@mailinator.com</email1>
      <folderId>5291351</folderId>
    </contact>
    
    <contact>
      <itemId>61313159</itemId>
      <displayName>Joe Blow2</displayName>
      <fullName>Joe Blow2</fullName>
      <firstName>Joe</firstName>
      <lastName>Blow2</lastName>
      <homeEmail1>joeblow2@mailinator.com</homeEmail1>
      <email1>joeblow2@mailinator.com</email1>
      <folderId>5291351</folderId>
    </contact>
    
  </contacts>
  
  <totalCount>2</totalCount>
  <editCounter>3</editCounter>
  
</ns1:GetContactsResponse>
=end

================================================
FILE: lib/contacts/yahoo.rb
================================================
class Contacts
  class Yahoo < Base
    URL                 = "http://mail.yahoo.com/"
    LOGIN_URL           = "https://login.yahoo.com/config/login"
    ADDRESS_BOOK_URL    = "http://address.mail.yahoo.com/?.rand=430244936"
    CONTACT_LIST_URL    = "http://address.mail.yahoo.com/?_src=&_crumb=crumb&sortfield=3&bucket=1&scroll=1&VPC=social_list&.r=time"
    PROTOCOL_ERROR      = "Yahoo has changed its protocols, please upgrade this library first. If that does not work, dive into the code and submit a patch at http://github.com/cardmagic/contacts"

    def real_connect
      postdata =  ".tries=2&.src=ym&.md5=&.hash=&.js=&.last=&promo=&.intl=us&.bypass="
      postdata += "&.partner=&.u=4eo6isd23l8r3&.v=0&.challenge=gsMsEcoZP7km3N3NeI4mX"
      postdata += "kGB7zMV&.yplus=&.emailCode=&pkg=&stepid=&.ev=&hasMsgr=1&.chkP=Y&."
      postdata += "done=#{CGI.escape(URL)}&login=#{CGI.escape(login)}&passwd=#{CGI.escape(password)}"

      data, resp, cookies, forward = post(LOGIN_URL, postdata)

      if data.index("Invalid ID or password") || data.index("This ID is not yet taken")
        raise AuthenticationError, "Username and password do not match"
      elsif data.index("Sign in") && data.index("to Yahoo!")
        raise AuthenticationError, "Required field must not be blank"
      elsif !data.match(/uncompressed\/chunked/)
        raise ConnectionError, PROTOCOL_ERROR
      elsif cookies == ""
        raise ConnectionError, PROTOCOL_ERROR
      end

      data, resp, cookies, forward = get(forward, cookies, LOGIN_URL)

      if resp.code_type != Net::HTTPOK
        raise ConnectionError, PROTOCOL_ERROR
      end

      @cookies = cookies
    end

    def contacts
      return @contacts if @contacts
      @contacts = []

      if connected?
        # first, get the addressbook site with the new crumb parameter
        url = URI.parse(address_book_url)
        http = open_http(url)
        resp, data = http.get("#{url.path}?#{url.query}",
          "Cookie" => @cookies
        )

        if resp.code_type != Net::HTTPOK
          raise ConnectionError, self.class.const_get(:PROTOCOL_ERROR)
        end

        crumb = data.to_s[/dotCrumb:   '(.*?)'/][13...-1]

        # now proceed with the new ".crumb" parameter to get the csv data
        url = URI.parse(contact_list_url.sub("_crumb=crumb","_crumb=#{crumb}").sub("time", Time.now.to_f.to_s.sub(".","")[0...-2]))
        http = open_http(url)
        resp, more_data = http.get("#{url.path}?#{url.query}",
          "Cookie" => @cookies,
          "X-Requested-With" => "XMLHttpRequest",
          "Referer" => address_book_url
        )

        if resp.code_type != Net::HTTPOK
        raise ConnectionError, self.class.const_get(:PROTOCOL_ERROR)
        end

        if more_data =~ /"TotalABContacts":(\d+)/
          total = $1.to_i
          ((total / 50.0).ceil).times do |i|
            # now proceed with the new ".crumb" parameter to get the csv data
            url = URI.parse(contact_list_url.sub("bucket=1","bucket=#{i}").sub("_crumb=crumb","_crumb=#{crumb}").sub("time", Time.now.to_f.to_s.sub(".","")[0...-2]))
            http = open_http(url)
            resp, more_data = http.get("#{url.path}?#{url.query}",
              "Cookie" => @cookies,
              "X-Requested-With" => "XMLHttpRequest",
              "Referer" => address_book_url
            )

            if resp.code_type != Net::HTTPOK
            raise ConnectionError, self.class.const_get(:PROTOCOL_ERROR)
            end

            parse more_data
          end
        end

        @contacts
      end
    end

  private

    def parse(data, options={})
      @contacts ||= []
      @contacts += Contacts.parse_json(data)["response"]["ResultSet"]["Contacts"].to_a.select{|contact|!contact["email"].to_s.empty?}.map do |contact|
        name = contact["contactName"].split(",")
        [[name.pop, name.join(",")].join(" ").strip, contact["email"]]
      end if data =~ /^\{"response":/
      @contacts
    end

  end

  TYPES[:yahoo] = Yahoo
end


================================================
FILE: lib/contacts.rb
================================================
$:.unshift(File.dirname(__FILE__)+"/contacts/")

require 'rubygems'
require 'bundler/setup'

require 'base'
require 'json_picker'
require 'gmail'
require 'hotmail'
require 'yahoo'
require 'plaxo'
require 'aol'
require 'mailru'



================================================
FILE: test/example_accounts.yml
================================================
gmail:
  username: <changeme>
  password: <changeme>
  contacts:
    -
      name: "FirstName1 LastName1"
      email_address: "firstname1@example.com"
    -
      name: "FirstName2 LastName2"
      email_address: "firstname2@example.com"
yahoo:
  username: <changeme>
  password: <changeme>
  contacts:
    -
      name: "FirstName1 LastName1"
      email_address: "firstname1@example.com"
    -
      name: "FirstName2 LastName2"
      email_address: "firstname2@example.com"
hotmail:
  username: <changeme>
  password: <changeme>
  contacts:
    -
      name: "FirstName1 LastName1"
      email_address: "firstname1@example.com"
    -
      name: "FirstName2 LastName2"
      email_address: "firstname2@example.com"
aol:
  username: <changeme>
  password: <changeme>
  contacts:
    -
      name: "FirstName1 LastName1"
      email_address: "firstname1@example.com"
    -
      name: "FirstName2 LastName2"
      email_address: "firstname2@example.com"
mailru:
  username: <changeme>
  password: <changeme>
  contacts:
    -
      name: "FirstName1 LastName1"
      email_address: "firstname1@example.com"
    -
      name: "FirstName2 LastName2"
      email_address: "firstname2@example.com"


================================================
FILE: test/test_helper.rb
================================================
dir = File.dirname(__FILE__)
$LOAD_PATH.unshift(dir + "/../lib/")
require 'test/unit'
require 'contacts'
require 'yaml'

class ContactImporterTestCase < Test::Unit::TestCase
  # Add more helper methods to be used by all tests here...
  def default_test
    assert true
  end
end

class TestAccounts
  def self.[](type)
    load[type]
  end
  
  def self.load(file = File.dirname(__FILE__) + "/accounts.yml")
    raise "/test/accounts.yml file not found, please create, see /test/example_accounts.yml for information" unless File.exist?(file)
    
    accounts = {}
    YAML::load(File.open(file)).each do |type, contents|
      contacts = contents["contacts"].collect {|contact| [contact["name"], contact["email_address"]]}
      accounts[type.to_sym] = Account.new(type.to_sym, contents["username"], contents["password"], contacts)
    end
    accounts
  end

  Account = Struct.new :type, :username, :password, :contacts
end


================================================
FILE: test/test_suite.rb
================================================
dir = File.dirname(__FILE__)
Dir["#{dir}/**/*_test.rb"].each do |file|
  require file
end

================================================
FILE: test/unit/aol_contact_importer_test.rb
================================================
dir = File.dirname(__FILE__)
require "#{dir}/../test_helper"
require 'contacts'

class AolContactImporterTest < ContactImporterTestCase
  def setup
    super
    @account = TestAccounts[:aol]
  end

  def test_successful_login
    Contacts.new(:aol, @account.username, @account.password)
  end

  def test_importer_fails_with_invalid_password
    assert_raise(Contacts::AuthenticationError) do
      Contacts.new(:aol, @account.username, "wrong_password")
    end
  end

  def test_importer_fails_with_blank_password
    assert_raise(Contacts::AuthenticationError) do
      Contacts.new(:aol, @account.username, "")
    end
  end

  def test_importer_fails_with_blank_username
    assert_raise(Contacts::AuthenticationError) do
      Contacts.new(:aol, "", @account.password)
    end
  end

  def test_fetch_contacts
    contacts = Contacts.new(:aol, @account.username, @account.password).contacts
    @account.contacts.each do |contact|
      assert contacts.include?(contact), "Could not find: #{contact.inspect} in #{contacts.inspect}"
    end
  end
end

================================================
FILE: test/unit/gmail_contact_importer_test.rb
================================================
dir = File.dirname(__FILE__)
require "#{dir}/../test_helper"
require 'contacts'

class GmailContactImporterTest < ContactImporterTestCase
  def setup
    super
    @account = TestAccounts[:gmail]
  end

  def test_successful_login
    Contacts.new(:gmail, @account.username, @account.password)
  end

  def test_importer_fails_with_invalid_password
    assert_raise(Contacts::AuthenticationError) do
      Contacts.new(:gmail, @account.username, "wrong_password")
    end
  end

  def test_importer_fails_with_blank_password
    assert_raise(Contacts::AuthenticationError) do
      Contacts.new(:gmail, @account.username, "")
    end
  end

  def test_importer_fails_with_blank_username
    assert_raise(Contacts::AuthenticationError) do
      Contacts.new(:gmail, "", @account.password)
    end
  end

  def test_fetch_contacts
    contacts = Contacts.new(:gmail, @account.username, @account.password).contacts
    @account.contacts.each do |contact|
      assert contacts.include?(contact), "Could not find: #{contact.inspect} in #{contacts.inspect}"
    end
  end
end

================================================
FILE: test/unit/hotmail_contact_importer_test.rb
================================================
dir = File.dirname(__FILE__)
require "#{dir}/../test_helper"
require 'contacts'

class HotmailContactImporterTest < ContactImporterTestCase
  def setup
    super
    @account = TestAccounts[:hotmail]
  end

  def test_successful_login
    Contacts.new(:hotmail, @account.username, @account.password)
  end

  def test_importer_fails_with_invalid_password
    assert_raise(Contacts::AuthenticationError) do
      Contacts.new(:hotmail, @account.username,"wrong_password")
    end
  end

  def test_fetch_contacts
    contacts = Contacts.new(:hotmail, @account.username, @account.password).contacts
    @account.contacts.each do |contact|
      assert contacts.include?(contact), "Could not find: #{contact.inspect} in #{contacts.inspect}"
    end
  end
  
  def test_importer_fails_with_invalid_msn_password
    assert_raise(Contacts::AuthenticationError) do
      Contacts.new(:hotmail, "test@msn.com","wrong_password")
    end
  end
  
  # Since the hotmail scraper doesn't read names, test email
  def test_fetch_email
    contacts = Contacts.new(:hotmail, @account.username, @account.password).contacts
    @account.contacts.each do |contact|
      assert contacts.any?{|book_contact| book_contact.last == contact.last }, "Could not find: #{contact.inspect} in #{contacts.inspect}"
    end
  end
end


================================================
FILE: test/unit/mailru_contact_importer_test.rb
================================================
dir = File.dirname(__FILE__)
require "#{dir}/../test_helper"
require 'contacts'

class MailruContactImporterTest < ContactImporterTestCase
  def setup
    super
    @account = TestAccounts[:mailru]
  end

  def test_successful_login
    Contacts.new(:mailru, @account.username, @account.password)
  end

  def test_importer_fails_with_invalid_password
    assert_raise(Contacts::AuthenticationError) do
      Contacts.new(:mailru, @account.username, "wrong_password")
    end
  end

  def test_importer_fails_with_blank_password
    assert_raise(Contacts::AuthenticationError) do
      Contacts.new(:mailru, @account.username, "")
    end
  end

  def test_importer_fails_with_blank_username
    assert_raise(Contacts::AuthenticationError) do
      Contacts.new(:mailru, "", @account.password)
    end
  end

  def test_fetch_contacts
    contacts = Contacts.new(:mailru, @account.username, @account.password).contacts
    @account.contacts.each do |contact|
      assert contacts.include?(contact), "Could not find: #{contact.inspect} in #{contacts.inspect}"
    end
  end
end

================================================
FILE: test/unit/test_accounts_test.rb
================================================
dir = File.dirname(__FILE__)
require "#{dir}/../test_helper"

class TestAccountsTest < ContactImporterTestCase
  def test_test_accounts_loads_data_from_example_accounts_file
    account = TestAccounts.load(File.dirname(__FILE__) + "/../example_accounts.yml")[:gmail]
    
    assert_equal :gmail, account.type
    assert_equal "<changeme>", account.username
    assert_equal "<changeme>", account.password
    assert_equal [["FirstName1 LastName1", "firstname1@example.com"], ["FirstName2 LastName2", "firstname2@example.com"]], account.contacts
  end
  
  def test_test_accounts_blows_up_if_file_doesnt_exist
    assert_raise(RuntimeError) do
      TestAccounts.load("file_that_does_not_exist.yml")
    end
  end
  
  def test_we_can_load_from_account_file
    assert_not_nil TestAccounts[:gmail].username
  end
end

================================================
FILE: test/unit/yahoo_csv_contact_importer_test.rb
================================================
dir = File.dirname(__FILE__)
require "#{dir}/../test_helper"
require 'contacts'

class YahooContactImporterTest < ContactImporterTestCase
  def setup
    super
    @account = TestAccounts[:yahoo]
  end

  def test_a_successful_login
    Contacts.new(:yahoo, @account.username, @account.password)
  end

  def test_importer_fails_with_invalid_password
    assert_raise(Contacts::AuthenticationError) do
      Contacts.new(:yahoo, @account.username, "wrong_password")
    end
    # run the "successful" login test to ensure we reset yahoo's failed login lockout counter
    # See http://www.pivotaltracker.com/story/show/138210
    # yahoo needs some time to unset the failed login state, apparently... 
    # ...1 sec and 5 secs still failed sporadically
    sleep 10
    assert_nothing_raised do
      Contacts.new(:yahoo, @account.username, @account.password)
    end
  end

  def test_a_fetch_contacts
    contacts = Contacts.new(:yahoo, @account.username, @account.password).contacts
    @account.contacts.each do |contact|
      assert contacts.include?(contact), "Could not find: #{contact.inspect} in #{contacts.inspect}"
    end
  end
end