Showing preview only (453K chars total). Download the full file or copy to clipboard to get everything.
Repository: ankane/searchkick
Branch: master
Commit: 1009d03107a2
Files: 114
Total size: 425.2 KB
Directory structure:
gitextract_mcchxu51/
├── .github/
│ ├── ISSUE_TEMPLATE/
│ │ ├── bug_report.md
│ │ ├── config.yml
│ │ └── feature_request.md
│ ├── pull_request_template.md
│ └── workflows/
│ └── build.yml
├── .gitignore
├── CHANGELOG.md
├── Gemfile
├── LICENSE.txt
├── README.md
├── Rakefile
├── benchmark/
│ ├── Gemfile
│ ├── index.rb
│ ├── relation.rb
│ └── search.rb
├── examples/
│ ├── Gemfile
│ ├── hybrid.rb
│ └── semantic.rb
├── gemfiles/
│ ├── activerecord72.gemfile
│ ├── activerecord80.gemfile
│ ├── mongoid8.gemfile
│ ├── mongoid9.gemfile
│ ├── opensearch2.gemfile
│ └── opensearch3.gemfile
├── lib/
│ ├── searchkick/
│ │ ├── bulk_reindex_job.rb
│ │ ├── controller_runtime.rb
│ │ ├── hash_wrapper.rb
│ │ ├── index.rb
│ │ ├── index_cache.rb
│ │ ├── index_options.rb
│ │ ├── indexer.rb
│ │ ├── log_subscriber.rb
│ │ ├── middleware.rb
│ │ ├── model.rb
│ │ ├── multi_search.rb
│ │ ├── process_batch_job.rb
│ │ ├── process_queue_job.rb
│ │ ├── query.rb
│ │ ├── railtie.rb
│ │ ├── record_data.rb
│ │ ├── record_indexer.rb
│ │ ├── reindex_queue.rb
│ │ ├── reindex_v2_job.rb
│ │ ├── relation.rb
│ │ ├── relation_indexer.rb
│ │ ├── reranking.rb
│ │ ├── results.rb
│ │ ├── script.rb
│ │ ├── version.rb
│ │ └── where.rb
│ ├── searchkick.rb
│ └── tasks/
│ └── searchkick.rake
├── searchkick.gemspec
└── test/
├── aggs_test.rb
├── boost_test.rb
├── callbacks_test.rb
├── conversions_test.rb
├── default_scope_test.rb
├── exclude_test.rb
├── geo_shape_test.rb
├── highlight_test.rb
├── hybrid_test.rb
├── index_cache_test.rb
├── index_options_test.rb
├── index_test.rb
├── inheritance_test.rb
├── knn_test.rb
├── language_test.rb
├── load_test.rb
├── log_subscriber_test.rb
├── marshal_test.rb
├── match_test.rb
├── misspellings_test.rb
├── models/
│ ├── animal.rb
│ ├── artist.rb
│ ├── band.rb
│ ├── product.rb
│ ├── region.rb
│ ├── sku.rb
│ ├── song.rb
│ ├── speaker.rb
│ └── store.rb
├── multi_indices_test.rb
├── multi_search_test.rb
├── multi_tenancy_test.rb
├── notifications_test.rb
├── order_test.rb
├── pagination_test.rb
├── parameters_test.rb
├── partial_match_test.rb
├── partial_reindex_test.rb
├── query_test.rb
├── reindex_test.rb
├── reindex_v2_job_test.rb
├── relation_test.rb
├── results_test.rb
├── routing_test.rb
├── scroll_test.rb
├── search_synonyms_test.rb
├── search_test.rb
├── select_test.rb
├── should_index_test.rb
├── similar_test.rb
├── suggest_test.rb
├── support/
│ ├── activerecord.rb
│ ├── apartment.rb
│ ├── helpers.rb
│ ├── kaminari.yml
│ ├── mongoid.rb
│ └── redis.rb
├── synonyms_test.rb
├── test_helper.rb
├── unscope_test.rb
└── where_test.rb
================================================
FILE CONTENTS
================================================
================================================
FILE: .github/ISSUE_TEMPLATE/bug_report.md
================================================
---
name: Bug report
about: Create a report to help us improve
title: ''
labels: bug report
assignees: ''
---
**First**
Search existing issues to see if it’s been reported and make sure you’re on the latest version.
**Describe the bug**
A clear and concise description of the bug.
**To reproduce**
Use this code to reproduce when possible:
```ruby
require "bundler/inline"
gemfile do
source "https://rubygems.org"
gem "activerecord", require: "active_record"
gem "activejob", require: "active_job"
gem "sqlite3"
gem "searchkick", git: "https://github.com/ankane/searchkick.git"
# uncomment one
# gem "elasticsearch"
# gem "opensearch-ruby"
end
puts "Searchkick version: #{Searchkick::VERSION}"
puts "Server version: #{Searchkick.server_version}"
ActiveRecord::Base.establish_connection adapter: "sqlite3", database: ":memory:"
ActiveJob::Base.queue_adapter = :inline
ActiveRecord::Schema.define do
create_table :products do |t|
t.string :name
end
end
class Product < ActiveRecord::Base
searchkick
end
Product.reindex
Product.create!(name: "Test")
Product.search_index.refresh
p Product.search("test", fields: [:name]).response
```
**Additional context**
Add any other context.
================================================
FILE: .github/ISSUE_TEMPLATE/config.yml
================================================
blank_issues_enabled: false
contact_links:
- name: Help
url: https://stackoverflow.com/questions/tagged/searchkick
about: Ask and answer questions here
================================================
FILE: .github/ISSUE_TEMPLATE/feature_request.md
================================================
---
name: Feature request
about: Suggest an idea for this project
title: ''
labels: feature request
assignees: ''
---
**First**
Search existing issues to see if it’s been discussed.
**Is your feature request related to a problem? Please describe.**
A clear and concise description of the problem.
**Describe the solution you'd like**
A clear and concise description of your idea.
**Additional context**
Add any other context.
================================================
FILE: .github/pull_request_template.md
================================================
Thanks for contributing. You’re awesome! A few things to keep in mind:
- Keep changes to a minimum
- Follow the existing style
- Add one or more tests if possible
Finally, replace all this with a description of the changes.
================================================
FILE: .github/workflows/build.yml
================================================
name: build
on: [push, pull_request]
jobs:
build:
strategy:
fail-fast: false
matrix:
include:
- ruby: "4.0"
gemfile: Gemfile
elasticsearch: 9
- ruby: 3.3
gemfile: gemfiles/activerecord80.gemfile
elasticsearch: 9.0.0
- ruby: 3.2
gemfile: gemfiles/activerecord72.gemfile
elasticsearch: 8
- ruby: 3.4
gemfile: gemfiles/opensearch3.gemfile
opensearch: 3
- ruby: 3.3
gemfile: gemfiles/opensearch2.gemfile
opensearch: 2
- ruby: 3.4
gemfile: gemfiles/mongoid9.gemfile
elasticsearch: 9
mongodb: true
- ruby: 3.2
gemfile: gemfiles/mongoid8.gemfile
# TODO fix plugin installation for earlier versions
elasticsearch: 8.5.0
mongodb: true
runs-on: ubuntu-latest
env:
BUNDLE_GEMFILE: ${{ matrix.gemfile }}
steps:
- uses: actions/checkout@v6
- uses: ruby/setup-ruby@v1
with:
ruby-version: ${{ matrix.ruby }}
bundler-cache: true
- run: bundle update
- uses: actions/cache@v5
if: ${{ matrix.elasticsearch }}
with:
path: ~/elasticsearch
key: ${{ runner.os }}-elasticsearch-${{ matrix.elasticsearch }}
- uses: ankane/setup-elasticsearch@v1
if: ${{ matrix.elasticsearch }}
with:
elasticsearch-version: ${{ matrix.elasticsearch }}
plugins: |
analysis-kuromoji
analysis-smartcn
analysis-stempel
analysis-ukrainian
- uses: actions/cache@v5
if: ${{ matrix.opensearch }}
with:
path: ~/opensearch
key: ${{ runner.os }}-opensearch-${{ matrix.opensearch }}
- uses: ankane/setup-opensearch@v1
if: ${{ matrix.opensearch }}
with:
opensearch-version: ${{ matrix.opensearch }}
plugins: |
analysis-kuromoji
analysis-smartcn
analysis-stempel
analysis-ukrainian
- uses: ankane/setup-mongodb@v1
if: ${{ matrix.mongodb }}
- run: |
sudo apt-get update
sudo apt-get install redis-server
sudo systemctl start redis-server
- run: bundle exec rake test
================================================
FILE: .gitignore
================================================
*.gem
*.rbc
.bundle
.config
.yardoc
*.lock
InstalledFiles
_yardoc
coverage
doc/
lib/bundler/man
pkg
rdoc
spec/reports
test/tmp
test/version_tmp
tmp
*.log
.DS_Store
.ruby-*
.idea/
*.sqlite3
================================================
FILE: CHANGELOG.md
================================================
## 6.1.1 (unreleased)
- Fixed smart aggs behavior with `_and`
## 6.1.0 (2026-02-18)
- Added `per` method
- Fixed error with `aggs` method and non-hash arguments
- Fixed smart aggs behavior when multiple `where` calls
## 6.0.3 (2026-01-06)
- Fixed `inspect` method for `Relation`
## 6.0.2 (2025-10-24)
- Fixed `as_json` method for `HashWrapper`
## 6.0.1 (2025-10-24)
- Fixed `to_json` method for `HashWrapper`
## 6.0.0 (2025-10-19)
- Added new query builder API (similar to Active Record)
- Added `conversions_v2` option
- Added `job_options` option
- Added `parent_job` option
- Added `opaque_id` option
- Added `callback_options` option
- Added `ignore_missing` option for partial reindex
- Added support for `exists: false`
- Added `quantization` to `knn` option for Elasticsearch
- Changed async reindex to use ranges for numeric primary keys with Active Record
- Fixed error with `case_sensitive` option and synonyms
- Removed default quantization for `knn` option for Elasticsearch 8.14+
- Removed `results` method (use `to_a` instead)
- Removed `execute` option and method (no longer needed)
- Removed `options` method (use individual methods instead)
- Removed dependency on Hashie
- Deprecated `conversions` option in favor of `conversions_v2`
- Dropped support for Elasticsearch 7 and OpenSearch 1
- Dropped support for Active Record < 7.2
- Dropped support for Redis < 6.2
## 5.5.2 (2025-05-20)
- Fixed `scope` option for partial reindex
## 5.5.1 (2025-04-24)
- Added support for `elasticsearch` 9 gem
## 5.5.0 (2025-04-03)
- Added `m` and `ef_construction` to `knn` index option
- Added `ef_search` to `knn` search option
- Fixed exact cosine distance for OpenSearch 2.19+
- Dropped support for Ruby < 3.2 and Active Record < 7.1
- Dropped support for Mongoid < 8
## 5.4.0 (2024-09-04)
- Added `knn` option
- Added `rrf` method
- Added experimental support for scripting to `where` option
- Added warning for `exists` with non-`true` values
- Added warning for full reindex and `:queue` mode
- Fixed `per_page` method when paginating beyond `max_result_window`
- Dropped support for Ruby < 3.1
## 5.3.1 (2023-11-28)
- Fixed error with misspellings below and failed queries
## 5.3.0 (2023-07-02)
- Fixed error with `cutoff_frequency`
- Dropped support for Ruby < 3 and Active Record < 6.1
- Dropped support for Mongoid < 7
## 5.2.4 (2023-05-11)
- Fixed error with non-string routing and `:async` mode
## 5.2.3 (2023-04-12)
- Fixed error with missing records and multiple models
## 5.2.2 (2023-04-01)
- Fixed `total_docs` method
- Fixed deprecation warning with Active Support 7.1
## 5.2.1 (2023-02-21)
- Added support for `redis-client` gem
## 5.2.0 (2023-02-08)
- Added model name to warning about missing records
- Fixed unnecessary data loading when reindexing relations with `:async` and `:queue` modes
## 5.1.2 (2023-01-29)
- Fixed error with missing point in time
## 5.1.1 (2022-12-05)
- Added support for strings for `offset` and `per_page`
## 5.1.0 (2022-10-12)
- Added support for fractional search timeout
- Fixed search timeout with `elasticsearch` 8+ and `opensearch-ruby` gems
- Fixed search timeout not applying to `multi_search`
## 5.0.5 (2022-10-09)
- Added `model` method to `Searchkick::Relation`
- Fixed deprecation warning with `redis` gem
- Fixed `respond_to?` method on relation loading relation
- Fixed `Relation loaded` error for non-mutating methods on relation
## 5.0.4 (2022-06-16)
- Added `max_result_window` option
- Improved error message for unsupported versions of Elasticsearch
## 5.0.3 (2022-03-13)
- Fixed context for index name for inherited models
## 5.0.2 (2022-03-03)
- Fixed index name for inherited models
## 5.0.1 (2022-02-27)
- Prefer `mode: :async` over `async: true` for full reindex
- Fixed instance method overriding with concerns
## 5.0.0 (2022-02-21)
- Searches now use lazy loading (similar to Active Record)
- Added `unscope` option to better support working with default scopes
- Added support for `:async` and `:queue` modes for `reindex` on relation
- Added basic protection from unfiltered parameters to `where` option
- Added `models` option to `similar` method
- Changed async full reindex to fetch ids instead of using ranges for numeric primary keys with Active Record
- Changed `searchkick_index_options` to return symbol keys (instead of mix of strings and symbols)
- Changed non-anchored regular expressions to match expected results (previously warned)
- Changed record reindex to return `true` to match model and relation reindex
- Updated async reindex job to call `search_import` for nested associations
- Fixed removing records when `should_index?` is `false` when `reindex` called on relation
- Fixed issue with `merge_mappings` for fields that use `searchkick` options
- Raise error when `search` called on relations
- Raise `ArgumentError` (instead of warning) for invalid regular expression modifiers
- Raise `ArgumentError` instead of `RuntimeError` for unknown operators
- Removed mapping of `id` to `_id` with `order` option (not supported in Elasticsearch 8)
- Removed `wordnet` option (no longer worked)
- Removed dependency on `elasticsearch` gem (can use `elasticsearch` or `opensearch-ruby`)
- Dropped support for Elasticsearch 6
- Dropped support for Ruby < 2.6 and Active Record < 5.2
- Dropped support for NoBrainer and Cequel
- Dropped support for `faraday_middleware-aws-signers-v4` (use `faraday_middleware-aws-sigv4` instead)
## 4.6.3 (2021-11-19)
- Added support for reloadable synonyms for OpenSearch
- Added experimental support for `opensearch-ruby` gem
- Removed `elasticsearch-xpack` dependency for reloadable synonyms
## 4.6.2 (2021-11-15)
- Added support for beginless ranges to `where` option
- Fixed `like` and `ilike` with `+` character
- Fixed warning about accessing system indices when no model or index specified
## 4.6.1 (2021-09-25)
- Added `ilike` operator for Elasticsearch 7.10+
- Fixed missing methods with `multi_search`
## 4.6.0 (2021-08-22)
- Added support for case-insensitive regular expressions with Elasticsearch 7.10+
- Added support for `OPENSEARCH_URL`
- Fixed error with `debug` option
## 4.5.2 (2021-08-05)
- Fixed error with reindex queue
- Fixed error with `model_name` method with multiple models
- Fixed error with `debug` option with elasticsearch-ruby 7.14
## 4.5.1 (2021-08-03)
- Improved performance of reindex queue
## 4.5.0 (2021-06-07)
- Added experimental support for OpenSearch
- Added support for synonyms in Japanese
## 4.4.4 (2021-03-12)
- Fixed `too_long_frame_exception` with `scroll` method
- Fixed multi-word emoji tokenization
## 4.4.3 (2021-02-25)
- Added support for Hunspell
- Fixed warning about accessing system indices
## 4.4.2 (2020-11-23)
- Added `missing_records` method to results
- Fixed issue with `like` and special characters
## 4.4.1 (2020-06-24)
- Added `stem_exclusion` and `stemmer_override` options
- Added `with_score` method to search results
- Improved error message for `reload_synonyms` with non-OSS version of Elasticsearch
- Improved output for reindex rake task
## 4.4.0 (2020-06-17)
- Added support for reloadable, multi-word, search time synonyms
- Fixed another deprecation warning in Ruby 2.7
## 4.3.1 (2020-05-13)
- Fixed error with `exclude` in certain cases for Elasticsearch 7.7
## 4.3.0 (2020-02-19)
- Fixed `like` queries with `"` character
- Better error when invalid parameters passed to `where`
## 4.2.1 (2020-01-27)
- Fixed deprecation warnings with Elasticsearch
- Fixed deprecation warnings in Ruby 2.7
## 4.2.0 (2019-12-18)
- Added safety check for multiple `Model.reindex`
- Added `deep_paging` option
- Added request parameters to search notifications and curl representation
- Removed curl from search notifications to prevent confusion
## 4.1.1 (2019-11-19)
- Added `chinese2` and `korean2` languages
- Improved performance of async full reindex
- Fixed `searchkick:reindex:all` rake task for Rails 6
## 4.1.0 (2019-08-01)
- Added `like` operator
- Added `exists` operator
- Added warnings for certain regular expressions
- Fixed anchored regular expressions
## 4.0.2 (2019-06-04)
- Added block form of `scroll`
- Added `clear_scroll` method
- Fixed custom mappings
## 4.0.1 (2019-05-30)
- Added support for scroll API
- Made type optional for custom mapping for Elasticsearch 6
- Fixed error when suggestions empty
- Fixed `models` option with inheritance
## 4.0.0 (2019-04-11)
- Added support for Elasticsearch 7
- Added `models` option
Breaking changes
- Removed support for Elasticsearch 5
- Removed support for multi-word synonyms (they no longer work with shingles)
- Removed support for Active Record < 5
## 3.1.3 (2019-04-11)
- Added support for endless ranges
- Added support for routing to `similar` method
- Added `prefix` to `where`
- Fixed error with elasticsearch-ruby 6.3
- Fixed error with some language stemmers and Elasticsearch 6.5
- Fixed issue with misspellings below and body block
## 3.1.2 (2018-09-27)
- Improved performance of indices boost
- Fixed deletes with routing and `async` callbacks
- Fixed deletes with routing and `queue` callbacks
- Fixed deprecation warnings
- Fixed field misspellings for older partial match format
## 3.1.1 (2018-08-09)
- Added per-field misspellings
- Added `case_sensitive` option
- Added `stem` option
- Added `total_entries` option
- Fixed `exclude` option with match all
- Fixed `with_highlights` method
## 3.1.0 (2018-05-12)
- Added `:inline` as alias for `true` for `callbacks` and `mode` options
- Friendlier error message for bad mapping with partial matches
- Warn when records in search index do not exist in database
- Easier merging for `merge_mapping`
- Fixed `with_hit` and `with_highlights` when records in search index do not exist in database
- Fixed error with highlights and match all
## 3.0.3 (2018-04-22)
- Added support for pagination with `body` option
- Added `boost_by_recency` option
- Fixed "Model Search Data" output for `debug` option
- Fixed `reindex_status` error
- Fixed error with optional operators in Ruby regexp
- Fixed deprecation warnings for Elasticsearch 6.2+
## 3.0.2 (2018-03-26)
- Added support for Korean and Vietnamese
- Fixed `Unsupported argument type: Symbol` for async partial reindex
- Fixed infinite recursion with multi search and misspellings below
- Do not raise an error when `id` is indexed
## 3.0.1 (2018-03-14)
- Added `scope` option for partial reindex
- Added support for Japanese, Polish, and Ukrainian
## 3.0.0 (2018-03-03)
- Added support for Chinese
- No longer requires fields to query for Elasticsearch 6
- Results can be marshaled by default (unless using `highlight` option)
Breaking changes
- Removed support for Elasticsearch 2
- Removed support for Active Record < 4.2 and Mongoid < 5
- Types are no longer used
- The `_all` field is disabled by default in Elasticsearch 5
- Conversions are not stemmed by default
- An `ArgumentError` is raised instead of a warning when options are incompatible with the `body` option
- Removed `log` option from `boost_by`
- Removed `Model.enable_search_callbacks`, `Model.disable_search_callbacks`, and `Model.search_callbacks?`
- Removed `reindex_async` method, as `reindex` now defaults to callbacks mode specified on the model
- Removed `async` option from `record.reindex`
- Removed `search_hit` method - use `with_hit` instead
- Removed `each_with_hit` - use `with_hit.each` instead
- Removed `with_details` - use `with_highlights` instead
- Bumped default `limit` to 10,000
## 2.5.0 (2018-02-15)
- Try requests 3 times before raising error
- Better exception when trying to access results for failed multi-search query
- More efficient aggregations with `where` clauses
- Added support for `faraday_middleware-aws-sigv4`
- Added `credentials` option to `aws_credentials`
- Added `modifier` option to `boost_by`
- Added `scope_results` option
- Added `factor` option to `boost_by_distance`
## 2.4.0 (2017-11-14)
- Fixed `similar` for Elasticsearch 6
- Added `inheritance` option
- Added `_type` option
- Fixed `Must specify fields to search` error when searching `*`
## 2.3.2 (2017-09-08)
- Added `_all` and `default_fields` options
- Added global `index_prefix` option
- Added `wait` option to async reindex
- Added `model_includes` option
- Added `missing` option for `boost_by`
- Raise error for `reindex_status` when Redis not configured
- Warn when incompatible options used with `body` option
- Fixed bug where `routing` and `type` options were silently ignored with `body` option
- Fixed `reindex(async: true)` for non-numeric primary keys in Postgres
## 2.3.1 (2017-07-06)
- Added support for `reindex(async: true)` for non-numeric primary keys
- Added `conversions_term` option
- Added support for passing fields to `suggest` option
- Fixed `page_view_entries` for Kaminari
## 2.3.0 (2017-05-06)
- Fixed analyzer on dynamically mapped fields
- Fixed error with `similar` method and `_all` field
- Throw error when fields are needed
- Added `queue_name` option
- No longer require synonyms to be lowercase
## 2.2.1 (2017-04-16)
- Added `avg`, `cardinality`, `max`, `min`, and `sum` aggregations
- Added `load: {dumpable: true}` option
- Added `index_suffix` option
- Accept string for `exclude` option
## 2.2.0 (2017-03-19)
- Fixed bug with text values longer than 256 characters and `_all` field - see [#850](https://github.com/ankane/searchkick/issues/850)
- Fixed issue with `_all` field in `searchable`
- Fixed `exclude` option with `word_start`
## 2.1.1 (2017-01-17)
- Fixed duplicate notifications
- Added support for `connection_pool`
- Added `exclude` option
## 2.1.0 (2017-01-15)
- Background reindexing and queues are officially supported
- Log updates and deletes
## 2.0.4 (2017-01-15)
- Added support for queuing updates [experimental]
- Added `refresh_interval` option to `reindex`
- Prefer `search_index` over `searchkick_index`
## 2.0.3 (2017-01-12)
- Added `async` option to `reindex` [experimental]
- Added `misspellings?` method to results
## 2.0.2 (2017-01-08)
- Added `retain` option to `reindex`
- Added support for attributes in highlight tags
- Fixed potentially silent errors in reindex job
- Improved syntax for `boost_by_distance`
## 2.0.1 (2016-12-30)
- Added `search_hit` and `search_highlights` methods to models
- Improved reindex performance
## 2.0.0 (2016-12-28)
- Added support for `reindex` on associations
Breaking changes
- Removed support for Elasticsearch 1 as it reaches [end of life](https://www.elastic.co/support/eol)
- Removed facets, legacy options, and legacy methods
- Invalid options now throw an `ArgumentError`
- The `query` and `json` options have been removed in favor of `body`
- The `include` option has been removed in favor of `includes`
- The `personalize` option has been removed in favor of `boost_where`
- The `partial` option has been removed in favor of `operator`
- Renamed `select_v2` to `select` (legacy `select` no longer available)
- The `_all` field is disabled if `searchable` option is used (for performance)
- The `partial_reindex(:method_name)` method has been replaced with `reindex(:method_name)`
- The `unsearchable` and `only_analyzed` options have been removed in favor of `searchable` and `filterable`
- `load: false` no longer returns an array in Elasticsearch 2
## 1.5.1 (2016-12-28)
- Added `client_options`
- Added `refresh` option to `reindex` method
- Improved syntax for partial reindex
## 1.5.0 (2016-12-23)
- Added support for geo shape indexing and queries
- Added `_and`, `_or`, `_not` to `where` option
## 1.4.2 (2016-12-21)
- Added support for directional synonyms
- Easier AWS setup
- Fixed `total_docs` method for ES 5+
- Fixed exception on update errors
## 1.4.1 (2016-12-11)
- Added `partial_reindex` method
- Added `debug` option to `search` method
- Added `profile` option
## 1.4.0 (2016-10-26)
- Official support for Elasticsearch 5
- Boost exact matches for partial matching
- Added `searchkick_debug` method
- Added `geo_polygon` filter
## 1.3.6 (2016-10-08)
- Fixed `Job adapter not found` error
## 1.3.5 (2016-09-27)
- Added support for Elasticsearch 5.0 beta
- Added `request_params` option
- Added `filterable` option
## 1.3.4 (2016-08-23)
- Added `resume` option to reindex
- Added search timeout to payload
## 1.3.3 (2016-08-02)
- Fix for namespaced models (broken in 1.3.2)
## 1.3.2 (2016-08-01)
- Added `body_options` option
- Added `date_histogram` aggregation
- Added `indices_boost` option
- Added support for multiple conversions
## 1.3.1 (2016-07-10)
- Fixed error with Ruby 2.0
- Fixed error with indexing large fields
## 1.3.0 (2016-05-04)
- Added support for Elasticsearch 5.0 alpha
- Added support for phrase matches
- Added support for procs for `index_prefix` option
## 1.2.1 (2016-02-15)
- Added `multi_search` method
- Added support for routing for Elasticsearch 2
- Added support for `search_document_id` and `search_document_type` in models
- Fixed error with instrumentation for searching multiple models
- Fixed instrumentation for bulk updates
## 1.2.0 (2016-02-03)
- Fixed deprecation warnings with `alias_method_chain`
- Added `analyzed_only` option for large text fields
- Added `encoder` option to highlight
- Fixed issue in `similar` method with `per_page` option
- Added basic support for multiple models
## 1.1.2 (2015-12-18)
- Added bulk updates with `callbacks` method
- Added `bulk_delete` method
- Added `search_timeout` option
- Fixed bug with new location format for `boost_by_distance`
## 1.1.1 (2015-12-14)
- Added support for `{lat: lat, lon: lon}` as preferred format for locations
## 1.1.0 (2015-12-08)
- Added `below` option to misspellings to improve performance
- Fixed synonyms for `word_*` partial matches
- Added `searchable` option
- Added `similarity` option
- Added `match` option
- Added `word` option
- Added highlighted fields to `load: false`
## 1.0.3 (2015-11-27)
- Added support for Elasticsearch 2.1
## 1.0.2 (2015-11-15)
- Throw `Searchkick::ImportError` for errors when importing records
- Errors now inherit from `Searchkick::Error`
- Added `order` option to aggregations
- Added `mapping` method
## 1.0.1 (2015-11-05)
- Added aggregations method to get raw response
- Use `execute: false` for lazy loading
- Return nil when no aggs
- Added emoji search
## 1.0.0 (2015-10-30)
- Added support for Elasticsearch 2.0
- Added support for aggregations
- Added ability to use misspellings for partial matches
- Added `fragment_size` option for highlight
- Added `took` method to results
Breaking changes
- Raise `Searchkick::DangerousOperation` error when calling reindex with scope
- Enabled misspellings by default for partial matches
- Enabled transpositions by default for misspellings
## 0.9.1 (2015-08-31)
- `and` now matches `&`
- Added `transpositions` option to misspellings
- Added `boost_mode` and `log` options to `boost_by`
- Added `prefix_length` option to `misspellings`
- Added ability to set env
## 0.9.0 (2015-06-07)
- Much better performance for where queries if no facets
- Added basic support for regex
- Added support for routing
- Made `Searchkick.disable_callbacks` thread-safe
## 0.8.7 (2015-02-14)
- Fixed Mongoid import
## 0.8.6 (2015-02-10)
- Added support for NoBrainer
- Added `stem_conversions: false` option
- Added support for multiple `boost_where` values on the same field
- Added support for array of values for `boost_where`
- Fixed suggestions with partial match boost
- Fixed redefining existing instance methods in models
## 0.8.5 (2014-11-11)
- Added support for Elasticsearch 1.4
- Added `unsearchable` option
- Added `select: true` option
- Added `body` option
## 0.8.4 (2014-11-05)
- Added `boost_by_distance`
- More flexible highlight options
- Better `env` logic
## 0.8.3 (2014-09-20)
- Added support for Active Job
- Added `timeout` setting
- Fixed import with no records
## 0.8.2 (2014-08-18)
- Added `async` to `callbacks` option
- Added `wordnet` option
- Added `edit_distance` option to eventually replace `distance` option
- Catch misspelling of `misspellings` option
- Improved logging
## 0.8.1 (2014-08-16)
- Added `search_method_name` option
- Fixed `order` for array of hashes
- Added support for Mongoid 2
## 0.8.0 (2014-07-12)
- Added support for Elasticsearch 1.2
## 0.7.9 (2014-06-30)
- Added `tokens` method
- Added `json` option
- Added exact matches
- Added `prev_page` for Kaminari pagination
- Added `import` option to reindex
## 0.7.8 (2014-06-22)
- Added `boost_by` and `boost_where` options
- Added ability to boost fields - `name^10`
- Added `select` option for `load: false`
## 0.7.7 (2014-06-10)
- Added support for automatic failover
- Fixed `operator` option (and default) for partial matches
## 0.7.6 (2014-05-20)
- Added `stats` option to facets
- Added `padding` option
## 0.7.5 (2014-05-13)
- Do not throw errors when index becomes out of sync with database
- Added custom exception types
- Fixed `offset` and `offset_value`
## 0.7.4 (2014-05-06)
- Fixed reindex with inheritance
## 0.7.3 (2014-04-30)
- Fixed multi-index searches
- Fixed suggestions for partial matches
- Added `offset` and `length` for improved pagination
## 0.7.2 (2014-04-24)
- Added smart facets
- Added more fields to `load: false` result
- Fixed logging for multi-index searches
- Added `first_page?` and `last_page?` for improved Kaminari support
## 0.7.1 (2014-04-12)
- Fixed huge issue w/ zero-downtime reindexing on 0.90
## 0.7.0 (2014-04-10)
- Added support for Elasticsearch 1.1
- Dropped support for Elasticsearch below 0.90.4 (unfortunate side effect of above)
## 0.6.3 (2014-04-08)
- Removed patron since no support for Windows
- Added error if `searchkick` is called multiple times
## 0.6.2 (2014-04-05)
- Added logging
- Fixed index_name option
- Added ability to use proc as the index name
## 0.6.1 (2014-03-24)
- Fixed huge issue w/ zero-downtime reindexing on 0.90 and elasticsearch-ruby 1.0
- Restore load: false behavior
- Restore total_entries method
## 0.6.0 (2014-03-22)
- Moved to elasticsearch-ruby
- Added support for modifying the query and viewing the response
- Added support for page_entries_info method
## 0.5.3 (2014-02-24)
- Fixed bug w/ word_* queries
## 0.5.2 (2014-02-12)
- Use after_commit hook for Active Record to prevent data inconsistencies
## 0.5.1 (2014-02-12)
- Replaced stop words with common terms query
- Added language option
- Fixed bug with empty array in where clause
- Fixed bug with MongoDB integer _id
- Fixed reindex bug when callbacks disabled
## 0.5.0 (2014-01-20)
- Better control over partial matches
- Added merge_mappings option
- Added batch_size option
- Fixed bug with nil where clauses
## 0.4.2 (2013-12-29)
- Added `should_index?` method to control which records are indexed
- Added ability to temporarily disable callbacks
- Added custom mappings
## 0.4.1 (2013-12-19)
- Fixed issue w/ inheritance mapping
## 0.4.0 (2013-12-11)
- Added support for Mongoid 4
- Added support for multiple locations
## 0.3.5 (2013-12-08)
- Added facet ranges
- Added all operator
## 0.3.4 (2013-11-22)
- Added highlighting
- Added :distance option to misspellings
- Fixed issue w/ BigDecimal serialization
## 0.3.3 (2013-11-04)
- Better error messages
- Added where: {field: nil} queries
## 0.3.2 (2013-11-02)
- Added support for single table inheritance
- Removed Tire::Model::Search
## 0.3.1 (2013-11-02)
- Added index_prefix option
- Fixed ES issue with incorrect facet counts
- Added option to turn off special characters
## 0.3.0 (2013-11-02)
- Fixed reversed coordinates
- Added bounded by a box queries
- Expanded `or` queries
## 0.2.8 (2013-09-30)
- Added option to disable callbacks
- Fixed bug with facets with Elasticsearch 0.90.5
## 0.2.7 (2013-09-23)
- Added limit to facet
- Improved similar items
## 0.2.6 (2013-09-10)
- Added option to disable misspellings
## 0.2.5 (2013-08-30)
- Added geospartial searches
- Create alias before importing document if no alias exists
- Fixed exception when :per_page option is a string
- Check `RAILS_ENV` if `RACK_ENV` is not set
## 0.2.4 (2013-08-20)
- Use `to_hash` instead of `as_json` for default `search_data` method
- Works for Mongoid 1.3
- Use one shard in test environment for consistent scores
## 0.2.3 (2013-08-16)
- Setup Travis
- Clean old indices before reindex
- Search for `*` returns all results
- Fixed pagination
- Added `similar` method
## 0.2.2 (2013-08-11)
- Clean old indices after reindex
- More expansions for fuzzy queries
## 0.2.1 (2013-08-11)
- Added Rails logger
- Only fetch ids when `load: true`
## 0.2.0 (2013-08-10)
- Added autocomplete
- Added “Did you mean” suggestions
- Added personalized searches
## 0.1.4 (2013-08-03)
- Bug fix
## 0.1.3 (2013-08-03)
- Changed edit distance to one for misspellings
- Raise errors when indexing fails
- Fixed pagination
- Fixed :include option
## 0.1.2 (2013-07-30)
- Use conversions by default
## 0.1.1 (2013-07-29)
- Renamed `_source` to `search_data`
- Renamed `searchkick_import` to `search_import`
## 0.1.0 (2013-07-28)
- Added `_source` method
- Added `index_name` option
## 0.0.2 (2013-07-17)
- Added `conversions` option
## 0.0.1 (2013-07-14)
- First release
================================================
FILE: Gemfile
================================================
source "https://rubygems.org"
gemspec
gem "rake"
gem "minitest"
gem "sqlite3", platform: :ruby
gem "sqlite3-ffi", platform: :jruby
gem "activerecord", "~> 8.1.0"
gem "actionpack", "~> 8.1.0"
gem "activejob", "~> 8.1.0", require: "active_job"
gem "elasticsearch", "~> 9"
gem "redis-client"
gem "connection_pool"
gem "kaminari"
gem "gemoji-parser"
gem "parallel_tests"
gem "typhoeus", platform: :mri
gem "cgi" # for elasticsearch
================================================
FILE: LICENSE.txt
================================================
Copyright (c) 2013-2026 Andrew Kane
MIT License
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
================================================
FILE: README.md
================================================
# Searchkick
:rocket: Intelligent search made easy
**Searchkick learns what your users are looking for.** As more people search, it gets smarter and the results get better. It’s friendly for developers - and magical for your users.
Searchkick handles:
- stemming - `tomatoes` matches `tomato`
- special characters - `jalapeno` matches `jalapeño`
- extra whitespace - `dishwasher` matches `dish washer`
- misspellings - `zuchini` matches `zucchini`
- custom synonyms - `pop` matches `soda`
Plus:
- query like SQL - no need to learn a new query language
- reindex without downtime
- easily personalize results for each user
- autocomplete
- “Did you mean” suggestions
- supports many languages
- works with Active Record and Mongoid
Check out [Searchjoy](https://github.com/ankane/searchjoy) for analytics and [Autosuggest](https://github.com/ankane/autosuggest) for query suggestions
:tangerine: Battle-tested at [Instacart](https://www.instacart.com/opensource)
[](https://github.com/ankane/searchkick/actions)
## Contents
- [Getting Started](#getting-started)
- [Querying](#querying)
- [Indexing](#indexing)
- [Intelligent Search](#intelligent-search)
- [Instant Search / Autocomplete](#instant-search--autocomplete)
- [Aggregations](#aggregations)
- [Testing](#testing)
- [Deployment](#deployment)
- [Performance](#performance)
- [Advanced Search](#advanced)
- [Reference](#reference)
- [Contributing](#contributing)
Searchkick 6.0 was recently released! See [how to upgrade](#upgrading)
## Getting Started
Install [Elasticsearch](https://www.elastic.co/downloads/elasticsearch) or [OpenSearch](https://opensearch.org/downloads.html). For Homebrew, use:
```sh
brew install opensearch
brew services start opensearch
```
Add these lines to your application’s Gemfile:
```ruby
gem "searchkick"
gem "elasticsearch" # select one
gem "opensearch-ruby" # select one
```
The latest version works with Elasticsearch 8 and 9 and OpenSearch 2 and 3. For Elasticsearch 7 and OpenSearch 1, use version 5.5.2 and [this readme](https://github.com/ankane/searchkick/blob/v5.5.2/README.md).
Add `searchkick` to models you want to search.
```ruby
class Product < ApplicationRecord
searchkick
end
```
Add data to the search index.
```ruby
Product.reindex
```
And to query, use:
```ruby
products = Product.search("apples")
products.each do |product|
puts product.name
end
```
Searchkick supports the complete [Elasticsearch Search API](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-search.html) and [OpenSearch Search API](https://opensearch.org/docs/latest/opensearch/rest-api/search/). As your search becomes more advanced, we recommend you use the [search server DSL](#advanced) for maximum flexibility.
## Querying
Query like SQL
```ruby
Product.search("apples").where(in_stock: true).limit(10).offset(50)
```
Search specific fields
```ruby
fields(:name, :brand)
```
Where
```ruby
where(store_id: 1, expires_at: Time.now..)
```
[These types of filters are supported](#filtering)
Order
```ruby
order(_score: :desc) # most relevant first - default
```
[All of these sort options are supported](https://www.elastic.co/guide/en/elasticsearch/reference/current/sort-search-results.html)
Limit / offset
```ruby
limit(20).offset(40)
```
Select
```ruby
select(:name)
```
[These source filtering options are supported](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-fields.html#source-filtering)
### Results
Searches return a `Searchkick::Relation` object. This responds like an array to most methods.
```ruby
results = Product.search("milk")
results.size
results.any?
results.each { |result| ... }
```
By default, ids are fetched from the search server and records are fetched from your database. To fetch everything from the search server, use:
```ruby
Product.search("apples").load(false)
```
Get total results
```ruby
results.total_count
```
Get the time the search took (in milliseconds)
```ruby
results.took
```
Get the full response from the search server
```ruby
results.response
```
**Note:** By default, Elasticsearch and OpenSearch [limit paging](#deep-paging) to the first 10,000 results for performance. This applies to the total count as well.
### Filtering
Equal
```ruby
where(store_id: 1)
```
Not equal
```ruby
where.not(store_id: 2)
```
Greater than (`gt`), less than (`lt`), greater than or equal (`gte`), less than or equal (`lte`)
```ruby
where(expires_at: {gt: Time.now})
```
Range
```ruby
where(orders_count: 1..10)
```
In
```ruby
where(aisle_id: [25, 30])
```
Not in
```ruby
where.not(aisle_id: [25, 30])
```
Contains all
```ruby
where(user_ids: {all: [1, 3]})
```
Like
```ruby
where(category: {like: "%frozen%"})
```
Case-insensitive like
```ruby
where(category: {ilike: "%frozen%"})
```
Regular expression
```ruby
where(category: /frozen .+/)
```
Prefix
```ruby
where(category: {prefix: "frozen"})
```
Exists
```ruby
where(store_id: {exists: true})
```
Combine filters with OR
```ruby
where(_or: [{in_stock: true}, {backordered: true}])
```
### Boosting
Boost important fields
```ruby
fields("title^10", "description")
```
Boost by the value of a field (field must be numeric)
```ruby
boost_by(:orders_count) # give popular documents a little boost
boost_by(orders_count: {factor: 10}) # default factor is 1
```
Boost matching documents
```ruby
boost_where(user_id: 1)
boost_where(user_id: {value: 1, factor: 100}) # default factor is 1000
boost_where(user_id: [{value: 1, factor: 100}, {value: 2, factor: 200}])
```
Boost by recency
```ruby
boost_by_recency(created_at: {scale: "7d", decay: 0.5})
```
You can also boost by:
- [Conversions](#intelligent-search)
- [Distance](#boost-by-distance)
### Get Everything
Use a `*` for the query.
```ruby
Product.search("*")
```
### Pagination
Plays nicely with kaminari and will_paginate.
```ruby
# controller
@products = Product.search("milk").page(params[:page]).per_page(20)
```
View with kaminari
```erb
<%= paginate @products %>
```
View with will_paginate
```erb
<%= will_paginate @products %>
```
### Partial Matches
By default, results must match all words in the query.
```ruby
Product.search("fresh honey") # fresh AND honey
```
To change this, use:
```ruby
Product.search("fresh honey").operator("or") # fresh OR honey
```
By default, results must match the entire word - `back` will not match `backpack`. You can change this behavior with:
```ruby
class Product < ApplicationRecord
searchkick word_start: [:name]
end
```
And to search (after you reindex):
```ruby
Product.search("back").fields(:name).match(:word_start)
```
Available options are:
Option | Matches | Example
--- | --- | ---
`:word` | entire word | `apple` matches `apple`
`:word_start` | start of word | `app` matches `apple`
`:word_middle` | any part of word | `ppl` matches `apple`
`:word_end` | end of word | `ple` matches `apple`
`:text_start` | start of text | `gre` matches `green apple`, `app` does not match
`:text_middle` | any part of text | `een app` matches `green apple`
`:text_end` | end of text | `ple` matches `green apple`, `een` does not match
The default is `:word`. The most matches will happen with `:word_middle`.
To specify different matching for different fields, use:
```ruby
Product.search(query).fields({name: :word_start}, {brand: :word_middle})
```
### Exact Matches
To match a field exactly (case-sensitive), use:
```ruby
Product.search(query).fields({name: :exact})
```
### Phrase Matches
To only match the exact order, use:
```ruby
Product.search("fresh honey").match(:phrase)
```
### Stemming and Language
Searchkick stems words by default for better matching. `apple` and `apples` both stem to `appl`, so searches for either term will have the same matches.
Searchkick defaults to English for stemming. To change this, use:
```ruby
class Product < ApplicationRecord
searchkick language: "german"
end
```
See the [list of languages](https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-stemmer-tokenfilter.html#analysis-stemmer-tokenfilter-configure-parms). A few languages require plugins:
- `chinese` - [analysis-ik plugin](https://github.com/medcl/elasticsearch-analysis-ik)
- `chinese2` - [analysis-smartcn plugin](https://www.elastic.co/guide/en/elasticsearch/plugins/current/analysis-smartcn.html)
- `japanese` - [analysis-kuromoji plugin](https://www.elastic.co/guide/en/elasticsearch/plugins/current/analysis-kuromoji.html)
- `korean` - [analysis-openkoreantext plugin](https://github.com/open-korean-text/elasticsearch-analysis-openkoreantext)
- `korean2` - [analysis-nori plugin](https://www.elastic.co/guide/en/elasticsearch/plugins/current/analysis-nori.html)
- `polish` - [analysis-stempel plugin](https://www.elastic.co/guide/en/elasticsearch/plugins/current/analysis-stempel.html)
- `ukrainian` - [analysis-ukrainian plugin](https://www.elastic.co/guide/en/elasticsearch/plugins/7.4/analysis-ukrainian.html)
- `vietnamese` - [analysis-vietnamese plugin](https://github.com/duydo/elasticsearch-analysis-vietnamese)
You can also use a Hunspell dictionary for stemming.
```ruby
class Product < ApplicationRecord
searchkick stemmer: {type: "hunspell", locale: "en_US"}
end
```
Disable stemming with:
```ruby
class Image < ApplicationRecord
searchkick stem: false
end
```
Exclude certain words from stemming with:
```ruby
class Image < ApplicationRecord
searchkick stem_exclusion: ["apples"]
end
```
Or change how words are stemmed:
```ruby
class Image < ApplicationRecord
searchkick stemmer_override: ["apples => other"]
end
```
### Synonyms
```ruby
class Product < ApplicationRecord
searchkick search_synonyms: [["pop", "soda"], ["burger", "hamburger"]]
end
```
Call `Product.reindex` after changing synonyms. Synonyms are applied at search time before stemming, and can be a single word or multiple words.
For directional synonyms, use:
```ruby
search_synonyms: ["lightbulb => halogenlamp"]
```
### Dynamic Synonyms
The above approach works well when your synonym list is static, but in practice, this is often not the case. When you analyze search conversions, you often want to add new synonyms without a full reindex. We recommend placing synonyms in a file on the search server (in the `config` directory). This allows you to reload synonyms without reindexing.
```txt
pop, soda
burger, hamburger
```
Then use:
```ruby
class Product < ApplicationRecord
searchkick search_synonyms: "synonyms.txt"
end
```
And reload with:
```ruby
Product.search_index.reload_synonyms
```
### Misspellings
By default, Searchkick handles misspelled queries by returning results with an [edit distance](https://en.wikipedia.org/wiki/Levenshtein_distance) of one.
You can change this with:
```ruby
Product.search("zucini").misspellings(edit_distance: 2) # zucchini
```
To prevent poor precision and improve performance for correctly spelled queries (which should be a majority for most applications), Searchkick can first perform a search without misspellings, and if there are too few results, perform another with them.
```ruby
Product.search("zuchini").misspellings(below: 5)
```
If there are fewer than 5 results, a 2nd search is performed with misspellings enabled. The result of this query is returned.
Turn off misspellings with:
```ruby
Product.search("zuchini").misspellings(false) # no zucchini
```
Specify which fields can include misspellings with:
```ruby
Product.search("zucini").fields(:name, :color).misspellings(fields: [:name])
```
> When doing this, you must also specify fields to search
### Bad Matches
If a user searches `butter`, they may also get results for `peanut butter`. To prevent this, use:
```ruby
Product.search("butter").exclude("peanut butter")
```
You can map queries and terms to exclude with:
```ruby
exclude_queries = {
"butter" => ["peanut butter"],
"cream" => ["ice cream", "whipped cream"]
}
Product.search(query).exclude(exclude_queries[query])
```
You can demote results by boosting by a factor less than one:
```ruby
Product.search("butter").boost_where(category: {value: "pantry", factor: 0.5})
```
### Emoji
Search :ice_cream::cake: and get `ice cream cake`!
Add this line to your application’s Gemfile:
```ruby
gem "gemoji-parser"
```
And use:
```ruby
Product.search("🍨🍰").emoji
```
## Indexing
Control what data is indexed with the `search_data` method. Call `Product.reindex` after changing this method.
```ruby
class Product < ApplicationRecord
belongs_to :department
def search_data
{
name: name,
department_name: department.name,
on_sale: sale_price.present?
}
end
end
```
Searchkick uses `find_in_batches` to import documents. To eager load associations, use the `search_import` scope.
```ruby
class Product < ApplicationRecord
scope :search_import, -> { includes(:department) }
end
```
By default, all records are indexed. To control which records are indexed, use the `should_index?` method.
```ruby
class Product < ApplicationRecord
def should_index?
active # only index active records
end
end
```
If a reindex is interrupted, you can resume it with:
```ruby
Product.reindex(resume: true)
```
For large data sets, try [parallel reindexing](#parallel-reindexing).
### To Reindex, or Not to Reindex
#### Reindex
- when you install or upgrade searchkick
- change the `search_data` method
- change the `searchkick` method
#### No need to reindex
- app starts
### Strategies
There are four strategies for keeping the index synced with your database.
1. Inline (default)
Anytime a record is inserted, updated, or deleted
2. Asynchronous
Use background jobs for better performance
```ruby
class Product < ApplicationRecord
searchkick callbacks: :async
end
```
Jobs are added to a queue named `searchkick`.
3. Queuing
Push ids of records that need updated to a queue and reindex in the background in batches. This is more performant than the asynchronous method, which updates records individually. See [how to set up](#queuing).
4. Manual
Turn off automatic syncing
```ruby
class Product < ApplicationRecord
searchkick callbacks: false
end
```
And reindex a record or relation manually.
```ruby
product.reindex
# or
store.products.reindex(mode: :async)
```
You can also do bulk updates.
```ruby
Searchkick.callbacks(:bulk) do
Product.find_each(&:update_fields)
end
```
Or temporarily skip updates.
```ruby
Searchkick.callbacks(false) do
Product.find_each(&:update_fields)
end
```
Or override the model’s strategy.
```ruby
product.reindex(mode: :async) # :inline or :queue
```
### Associations
Data is **not** automatically synced when an association is updated. If this is desired, add a callback to reindex:
```ruby
class Image < ApplicationRecord
belongs_to :product
after_commit :reindex_product
def reindex_product
product.reindex
end
end
```
### Default Scopes
If you have a default scope that filters records, use the `should_index?` method to exclude them from indexing:
```ruby
class Product < ApplicationRecord
default_scope { where(deleted_at: nil) }
def should_index?
deleted_at.nil?
end
end
```
If you want to index and search filtered records, set:
```ruby
class Product < ApplicationRecord
searchkick unscope: true
end
```
## Intelligent Search
The best starting point to improve your search **by far** is to track searches and conversions. [Searchjoy](https://github.com/ankane/searchjoy) makes it easy.
```ruby
Product.search("apple").track(user_id: current_user.id)
```
[See the docs](https://github.com/ankane/searchjoy) for how to install and use. Focus on top searches with a low conversion rate.
Searchkick can then use the conversion data to learn what users are looking for. If a user searches for “ice cream” and adds Ben & Jerry’s Chunky Monkey to the cart (our conversion metric at Instacart), that item gets a little more weight for similar searches. This can make a huge difference on the quality of your search.
Add conversion data with:
```ruby
class Product < ApplicationRecord
has_many :conversions, class_name: "Searchjoy::Conversion", as: :convertable
has_many :searches, class_name: "Searchjoy::Search", through: :conversions
searchkick conversions_v2: [:conversions] # name of field
def search_data
{
name: name,
conversions: searches.group(:query).distinct.count(:user_id)
# {"ice cream" => 234, "chocolate" => 67, "cream" => 2}
}
end
end
```
Reindex and set up a cron job to add new conversions daily. For zero downtime deployment, temporarily set `conversions_v2(false)` in your search calls until the data is reindexed.
### Performant Conversions
A performant way to do conversions is to cache them to prevent N+1 queries. For Postgres, create a migration with:
```ruby
add_column :products, :search_conversions, :jsonb
```
For MySQL, use `:json`, and for others, use `:text` with a [JSON serializer](https://api.rubyonrails.org/classes/ActiveRecord/AttributeMethods/Serialization/ClassMethods.html).
Next, update your model. Create a separate method for conversion data so you can use [partial reindexing](#partial-reindexing).
```ruby
class Product < ApplicationRecord
searchkick conversions_v2: [:conversions]
def search_data
{
name: name,
category: category
}.merge(conversions_data)
end
def conversions_data
{
conversions: search_conversions || {}
}
end
end
```
Deploy and reindex your data. For zero downtime deployment, temporarily set `conversions_v2(false)` in your search calls until the data is reindexed.
```ruby
Product.reindex
```
Then, create a job to update the conversions column and reindex records with new conversions. Here’s one you can use for Searchjoy:
```ruby
class UpdateConversionsJob < ApplicationJob
def perform(class_name, since: nil, update: true, reindex: true)
model = Searchkick.load_model(class_name)
# get records that have a recent conversion
recently_converted_ids =
Searchjoy::Conversion.where(convertable_type: class_name, created_at: since..)
.order(:convertable_id).distinct.pluck(:convertable_id)
# split into batches
recently_converted_ids.in_groups_of(1000, false) do |ids|
if update
# fetch conversions
conversions =
Searchjoy::Conversion.where(convertable_id: ids, convertable_type: class_name)
.joins(:search).where.not(searchjoy_searches: {user_id: nil})
.group(:convertable_id, :query).distinct.count(:user_id)
# group by record
conversions_by_record = {}
conversions.each do |(id, query), count|
(conversions_by_record[id] ||= {})[query] = count
end
# update conversions column
model.transaction do
conversions_by_record.each do |id, conversions|
model.where(id: id).update_all(search_conversions: conversions)
end
end
end
if reindex
# reindex conversions data
model.where(id: ids).reindex(:conversions_data, ignore_missing: true)
end
end
end
end
```
Run the job:
```ruby
UpdateConversionsJob.perform_now("Product")
```
And set it up to run daily.
```ruby
UpdateConversionsJob.perform_later("Product", since: 1.day.ago)
```
## Personalized Results
Order results differently for each user. For example, show a user’s previously purchased products before other results.
```ruby
class Product < ApplicationRecord
def search_data
{
name: name,
orderer_ids: orders.pluck(:user_id) # boost this product for these users
}
end
end
```
Reindex and search with:
```ruby
Product.search("milk").boost_where(orderer_ids: current_user.id)
```
## Instant Search / Autocomplete
Autocomplete predicts what a user will type, making the search experience faster and easier.
**Note:** To autocomplete on search terms rather than results, check out [Autosuggest](https://github.com/ankane/autosuggest).
**Note 2:** If you only have a few thousand records, don’t use Searchkick for autocomplete. It’s *much* faster to load all records into JavaScript and autocomplete there (eliminates network requests).
First, specify which fields use this feature. This is necessary since autocomplete can increase the index size significantly, but don’t worry - this gives you blazing fast queries.
```ruby
class Movie < ApplicationRecord
searchkick word_start: [:title, :director]
end
```
Reindex and search with:
```ruby
Movie.search("jurassic pa").fields(:title).match(:word_start)
```
Use a front-end library like [typeahead.js](https://twitter.github.io/typeahead.js/) to show the results.
#### Here’s how to make it work with Rails
First, add a route and controller action.
```ruby
class MoviesController < ApplicationController
def autocomplete
render json: Movie.search(params[:query]).fields("title^5", "director")
.match(:word_start).limit(10).load(false).misspellings(below: 5).map(&:title)
end
end
```
**Note:** Use `load(false)` and `misspellings(below: n)` (or `misspellings(false)`) for best performance.
Then add the search box and JavaScript code to a view.
```html
<input type="text" id="query" name="query" />
<script src="jquery.js"></script>
<script src="typeahead.bundle.js"></script>
<script>
var movies = new Bloodhound({
datumTokenizer: Bloodhound.tokenizers.whitespace,
queryTokenizer: Bloodhound.tokenizers.whitespace,
remote: {
url: '/movies/autocomplete?query=%QUERY',
wildcard: '%QUERY'
}
});
$('#query').typeahead(null, {
source: movies
});
</script>
```
## Suggestions
```ruby
class Product < ApplicationRecord
searchkick suggest: [:name] # fields to generate suggestions
end
```
Reindex and search with:
```ruby
products = Product.search("peantu butta").suggest
products.suggestions # ["peanut butter"]
```
## Aggregations
[Aggregations](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations.html) provide aggregated search data.
```ruby
products = Product.search("chuck taylor").aggs(:product_type, :gender, :brand)
products.aggs
```
By default, `where` conditions apply to aggregations.
```ruby
Product.search("wingtips").where(color: "brandy").aggs(:size)
# aggregations for brandy wingtips are returned
```
Change this with:
```ruby
Product.search("wingtips").where(color: "brandy").aggs(:size).smart_aggs(false)
# aggregations for all wingtips are returned
```
Set `where` conditions for each aggregation separately with:
```ruby
Product.search("wingtips").aggs(size: {where: {color: "brandy"}})
```
Limit
```ruby
Product.search("apples").aggs(store_id: {limit: 10})
```
Order
```ruby
Product.search("wingtips").aggs(color: {order: {"_key" => "asc"}}) # alphabetically
```
[All of these options are supported](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-terms-aggregation.html#search-aggregations-bucket-terms-aggregation-order)
Ranges
```ruby
price_ranges = [{to: 20}, {from: 20, to: 50}, {from: 50}]
Product.search("*").aggs(price: {ranges: price_ranges})
```
Minimum document count
```ruby
Product.search("apples").aggs(store_id: {min_doc_count: 2})
```
Script support
```ruby
Product.search("*").aggs(color: {script: {source: "'Color: ' + _value"}})
```
Date histogram
```ruby
Product.search("pear").aggs(products_per_year: {date_histogram: {field: :created_at, interval: :year}})
```
For other aggregation types, including sub-aggregations, use `body_options`:
```ruby
Product.search("orange").body_options(aggs: {price: {histogram: {field: :price, interval: 10}}})
```
## Highlight
Specify which fields to index with highlighting.
```ruby
class Band < ApplicationRecord
searchkick highlight: [:name]
end
```
Highlight the search query in the results.
```ruby
bands = Band.search("cinema").highlight
```
View the highlighted fields with:
```ruby
bands.with_highlights.each do |band, highlights|
highlights[:name] # "Two Door <em>Cinema</em> Club"
end
```
To change the tag, use:
```ruby
Band.search("cinema").highlight(tag: "<strong>")
```
To highlight and search different fields, use:
```ruby
Band.search("cinema").fields(:name).highlight(fields: [:description])
```
By default, the entire field is highlighted. To get small snippets instead, use:
```ruby
bands = Band.search("cinema").highlight(fragment_size: 20)
bands.with_highlights(multiple: true).each do |band, highlights|
highlights[:name].join(" and ")
end
```
Additional options can be specified for each field:
```ruby
Band.search("cinema").fields(:name).highlight(fields: {name: {fragment_size: 200}})
```
You can find available highlight options in the [Elasticsearch](https://www.elastic.co/guide/en/elasticsearch/reference/current/highlighting.html) or [OpenSearch](https://opensearch.org/docs/latest/search-plugins/searching-data/highlight/) reference.
## Similar Items
Find similar items
```ruby
product = Product.first
product.similar.fields(:name).where(size: "12 oz")
```
## Geospatial Searches
```ruby
class Restaurant < ApplicationRecord
searchkick locations: [:location]
def search_data
attributes.merge(location: {lat: latitude, lon: longitude})
end
end
```
Reindex and search with:
```ruby
Restaurant.search("pizza").where(location: {near: {lat: 37, lon: -114}, within: "100mi"}) # or 160km
```
Bounded by a box
```ruby
Restaurant.search("sushi").where(location: {top_left: {lat: 38, lon: -123}, bottom_right: {lat: 37, lon: -122}})
```
**Note:** `top_right` and `bottom_left` also work
Bounded by a polygon
```ruby
Restaurant.search("dessert").where(location: {geo_polygon: {points: [{lat: 38, lon: -123}, {lat: 39, lon: -123}, {lat: 37, lon: 122}]}})
```
### Boost By Distance
Boost results by distance - closer results are boosted more
```ruby
Restaurant.search("noodles").boost_by_distance(location: {origin: {lat: 37, lon: -122}})
```
Also supports [additional options](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-function-score-query.html#function-decay)
```ruby
Restaurant.search("wings").boost_by_distance(location: {origin: {lat: 37, lon: -122}, function: "linear", scale: "30mi", decay: 0.5})
```
### Geo Shapes
You can also index and search geo shapes.
```ruby
class Restaurant < ApplicationRecord
searchkick geo_shape: [:bounds]
def search_data
attributes.merge(
bounds: {
type: "envelope",
coordinates: [{lat: 4, lon: 1}, {lat: 2, lon: 3}]
}
)
end
end
```
See the [Elasticsearch documentation](https://www.elastic.co/guide/en/elasticsearch/reference/current/geo-shape.html) for details.
Find shapes intersecting with the query shape
```ruby
Restaurant.search("soup").where(bounds: {geo_shape: {type: "polygon", coordinates: [[{lat: 38, lon: -123}, ...]]}})
```
Falling entirely within the query shape
```ruby
Restaurant.search("salad").where(bounds: {geo_shape: {type: "circle", relation: "within", coordinates: {lat: 38, lon: -123}, radius: "1km"}})
```
Not touching the query shape
```ruby
Restaurant.search("burger").where(bounds: {geo_shape: {type: "envelope", relation: "disjoint", coordinates: [{lat: 38, lon: -123}, {lat: 37, lon: -122}]}})
```
## Inheritance
Searchkick supports single table inheritance.
```ruby
class Dog < Animal
end
```
In your parent model, set:
```ruby
class Animal < ApplicationRecord
searchkick inheritance: true
end
```
The parent and child model can both reindex.
```ruby
Animal.reindex
Dog.reindex # equivalent, all animals reindexed
```
And to search, use:
```ruby
Animal.search("*") # all animals
Dog.search("*") # just dogs
Animal.search("*").type(Cat, Dog) # just cats and dogs
```
**Notes:**
1. The `suggest` option retrieves suggestions from the parent at the moment.
```ruby
Dog.search("airbudd").suggest # suggestions for all animals
```
2. This relies on a `type` field that is automatically added to the indexed document. Be wary of defining your own `type` field in `search_data`, as it will take precedence.
## Debugging Queries
To help with debugging queries, you can use:
```ruby
Product.search("soap").debug
```
This prints useful info to `stdout`.
See how the search server scores your queries with:
```ruby
Product.search("soap").explain.response
```
See how the search server tokenizes your queries with:
```ruby
Product.search_index.tokens("Dish Washer Soap", analyzer: "searchkick_index")
# ["dish", "dishwash", "washer", "washersoap", "soap"]
Product.search_index.tokens("dishwasher soap", analyzer: "searchkick_search")
# ["dishwashersoap"] - no match
Product.search_index.tokens("dishwasher soap", analyzer: "searchkick_search2")
# ["dishwash", "soap"] - match!!
```
Partial matches
```ruby
Product.search_index.tokens("San Diego", analyzer: "searchkick_word_start_index")
# ["s", "sa", "san", "d", "di", "die", "dieg", "diego"]
Product.search_index.tokens("dieg", analyzer: "searchkick_word_search")
# ["dieg"] - match!!
```
See the [complete list of analyzers](lib/searchkick/index_options.rb#L36).
## Testing
As you iterate on your search, it’s a good idea to add tests.
For performance, only enable Searchkick callbacks for the tests that need it.
### Rails
Add to your `test/test_helper.rb`:
```ruby
module ActiveSupport
class TestCase
parallelize_setup do |worker|
Searchkick.index_suffix = worker
# reindex models for parallel tests
Product.reindex
end
end
end
# reindex models for non-parallel tests
Product.reindex
# and disable callbacks
Searchkick.disable_callbacks
```
And use:
```ruby
class ProductTest < ActiveSupport::TestCase
setup do
Searchkick.enable_callbacks
end
teardown do
Searchkick.disable_callbacks
end
test "search" do
Product.create!(name: "Apple")
Product.search_index.refresh
assert_equal ["Apple"], Product.search("apple").map(&:name)
end
end
```
### Minitest
Add to your `test/test_helper.rb`:
```ruby
# reindex models
Product.reindex
# and disable callbacks
Searchkick.disable_callbacks
```
And use:
```ruby
class ProductTest < Minitest::Test
def setup
Searchkick.enable_callbacks
end
def teardown
Searchkick.disable_callbacks
end
def test_search
Product.create!(name: "Apple")
Product.search_index.refresh
assert_equal ["Apple"], Product.search("apple").map(&:name)
end
end
```
### RSpec
Add to your `spec/spec_helper.rb`:
```ruby
RSpec.configure do |config|
config.before(:suite) do
# reindex models
Product.reindex
# and disable callbacks
Searchkick.disable_callbacks
end
config.around(:each, search: true) do |example|
Searchkick.callbacks(nil) do
example.run
end
end
end
```
And use:
```ruby
describe Product, search: true do
it "searches" do
Product.create!(name: "Apple")
Product.search_index.refresh
assert_equal ["Apple"], Product.search("apple").map(&:name)
end
end
```
### Factory Bot
Define a trait for each model:
```ruby
FactoryBot.define do
factory :product do
trait :reindex do
after(:create) do |product, _|
product.reindex(refresh: true)
end
end
end
end
```
And use:
```ruby
FactoryBot.create(:product, :reindex)
```
### GitHub Actions
Check out [setup-elasticsearch](https://github.com/ankane/setup-elasticsearch) for an easy way to install Elasticsearch:
```yml
- uses: ankane/setup-elasticsearch@v1
```
And [setup-opensearch](https://github.com/ankane/setup-opensearch) for an easy way to install OpenSearch:
```yml
- uses: ankane/setup-opensearch@v1
```
## Deployment
For the search server, Searchkick uses `ENV["ELASTICSEARCH_URL"]` for Elasticsearch and `ENV["OPENSEARCH_URL"]` for OpenSearch. This defaults to `http://localhost:9200`.
- [Elastic Cloud](#elastic-cloud)
- [Amazon OpenSearch Service](#amazon-opensearch-service)
- [Heroku](#heroku)
- [Self-Hosted and Other](#self-hosted-and-other)
### Elastic Cloud
Create an initializer `config/initializers/elasticsearch.rb` with:
```ruby
ENV["ELASTICSEARCH_URL"] = "https://user:password@host:port"
```
Then deploy and reindex:
```sh
rake searchkick:reindex:all
```
### Amazon OpenSearch Service
Create an initializer `config/initializers/opensearch.rb` with:
```ruby
ENV["OPENSEARCH_URL"] = "https://es-domain-1234.us-east-1.es.amazonaws.com:443"
```
To use signed requests, include in your Gemfile:
```ruby
gem "faraday_middleware-aws-sigv4"
```
and add to your initializer:
```ruby
Searchkick.aws_credentials = {
access_key_id: ENV["AWS_ACCESS_KEY_ID"],
secret_access_key: ENV["AWS_SECRET_ACCESS_KEY"],
region: "us-east-1"
}
```
Then deploy and reindex:
```sh
rake searchkick:reindex:all
```
### Heroku
Choose an add-on: [Bonsai](https://elements.heroku.com/addons/bonsai), [SearchBox](https://elements.heroku.com/addons/searchbox), or [Elastic Cloud](https://elements.heroku.com/addons/foundelasticsearch).
For Elasticsearch on Bonsai:
```sh
heroku addons:create bonsai
heroku config:set ELASTICSEARCH_URL=`heroku config:get BONSAI_URL`
```
For OpenSearch on Bonsai:
```sh
heroku addons:create bonsai --engine=opensearch
heroku config:set OPENSEARCH_URL=`heroku config:get BONSAI_URL`
```
For SearchBox:
```sh
heroku addons:create searchbox:starter
heroku config:set ELASTICSEARCH_URL=`heroku config:get SEARCHBOX_URL`
```
For Elastic Cloud (previously Found):
```sh
heroku addons:create foundelasticsearch
heroku addons:open foundelasticsearch
```
Visit the Shield page and reset your password. You’ll need to add the username and password to your url. Get the existing url with:
```sh
heroku config:get FOUNDELASTICSEARCH_URL
```
And add `elastic:password@` right after `https://` and add port `9243` at the end:
```sh
heroku config:set ELASTICSEARCH_URL=https://elastic:password@12345.us-east-1.aws.found.io:9243
```
Then deploy and reindex:
```sh
heroku run rake searchkick:reindex:all
```
### Self-Hosted and Other
Create an initializer with:
```ruby
ENV["ELASTICSEARCH_URL"] = "https://user:password@host:port"
# or
ENV["OPENSEARCH_URL"] = "https://user:password@host:port"
```
Then deploy and reindex:
```sh
rake searchkick:reindex:all
```
### Data Protection
We recommend encrypting data at rest and in transit (even inside your own network). This is especially important if you send [personal data](https://en.wikipedia.org/wiki/Personally_identifiable_information) of your users to the search server.
Bonsai, Elastic Cloud, and Amazon OpenSearch Service all support encryption at rest and HTTPS.
### Automatic Failover
Create an initializer with multiple hosts:
```ruby
ENV["ELASTICSEARCH_URL"] = "https://user:password@host1,https://user:password@host2"
# or
ENV["OPENSEARCH_URL"] = "https://user:password@host1,https://user:password@host2"
```
### Client Options
Create an initializer with:
```ruby
Searchkick.client_options[:reload_connections] = true
```
See the docs for [Elasticsearch](https://www.elastic.co/guide/en/elasticsearch/client/ruby-api/current/advanced-config.html) or [Opensearch](https://rubydoc.info/gems/opensearch-transport#configuration) for a complete list of options.
### Lograge
Add the following to `config/environments/production.rb`:
```ruby
config.lograge.custom_options = lambda do |event|
options = {}
options[:search] = event.payload[:searchkick_runtime] if event.payload[:searchkick_runtime].to_f > 0
options
end
```
See [Production Rails](https://github.com/ankane/production_rails) for other good practices.
## Performance
### Persistent HTTP Connections
Significantly increase performance with persistent HTTP connections. Add [Typhoeus](https://github.com/typhoeus/typhoeus) to your Gemfile and it’ll automatically be used.
```ruby
gem "typhoeus"
```
To reduce log noise, create an initializer with:
```ruby
Ethon.logger = Logger.new(nil)
```
### Searchable Fields
By default, all string fields are searchable (can be used in `fields` option). Speed up indexing and reduce index size by only making some fields searchable.
```ruby
class Product < ApplicationRecord
searchkick searchable: [:name]
end
```
### Filterable Fields
By default, all string fields are filterable (can be used in `where` option). Speed up indexing and reduce index size by only making some fields filterable.
```ruby
class Product < ApplicationRecord
searchkick filterable: [:brand]
end
```
**Note:** Non-string fields are always filterable and should not be passed to this option.
### Parallel Reindexing
For large data sets, you can use background jobs to parallelize reindexing.
```ruby
Product.reindex(mode: :async)
# {index_name: "products_production_20250111210018065"}
```
Once the jobs complete, promote the new index with:
```ruby
Product.search_index.promote(index_name)
```
You can optionally track the status with Redis:
```ruby
Searchkick.redis = Redis.new
```
And use:
```ruby
Searchkick.reindex_status(index_name)
```
You can also have Searchkick wait for reindexing to complete
```ruby
Product.reindex(mode: :async, wait: true)
```
You can use your background job framework to control concurrency. For Solid Queue, create an initializer with:
```ruby
module SearchkickBulkReindexConcurrency
extend ActiveSupport::Concern
included do
limits_concurrency to: 3, key: ""
end
end
Rails.application.config.after_initialize do
Searchkick::BulkReindexJob.include(SearchkickBulkReindexConcurrency)
end
```
This will allow only 3 jobs to run at once.
### Refresh Interval
You can specify a longer refresh interval while reindexing to increase performance.
```ruby
Product.reindex(mode: :async, refresh_interval: "30s")
```
**Note:** This only makes a noticeable difference with parallel reindexing.
When promoting, have it restored to the value in your mapping (defaults to `1s`).
```ruby
Product.search_index.promote(index_name, update_refresh_interval: true)
```
### Queuing
Push ids of records needing reindexing to a queue and reindex in bulk for better performance. First, set up Redis in an initializer. We recommend using [connection_pool](https://github.com/mperham/connection_pool).
```ruby
Searchkick.redis = ConnectionPool.new { Redis.new }
```
And ask your models to queue updates.
```ruby
class Product < ApplicationRecord
searchkick callbacks: :queue
end
```
Then, set up a background job to run.
```ruby
Searchkick::ProcessQueueJob.perform_later(class_name: "Product")
```
You can check the queue length with:
```ruby
Product.search_index.reindex_queue.length
```
For more tips, check out [Keeping Elasticsearch in Sync](https://www.elastic.co/blog/found-keeping-elasticsearch-in-sync).
### Routing
Searchkick supports [routing](https://www.elastic.co/blog/customizing-your-document-routing), which can significantly speed up searches.
```ruby
class Business < ApplicationRecord
searchkick routing: true
def search_routing
city_id
end
end
```
Reindex and search with:
```ruby
Business.search("ice cream").routing(params[:city_id])
```
### Partial Reindexing
Reindex a subset of attributes to reduce time spent generating search data and cut down on network traffic.
```ruby
class Product < ApplicationRecord
def search_data
{
name: name,
category: category
}.merge(prices_data)
end
def prices_data
{
price: price,
sale_price: sale_price
}
end
end
```
And use:
```ruby
Product.reindex(:prices_data)
```
Ignore errors for missing documents with:
```ruby
Product.reindex(:prices_data, ignore_missing: true)
```
## Advanced
Searchkick makes it easy to use the Elasticsearch or OpenSearch DSL on its own.
### Advanced Mapping
Create a custom mapping:
```ruby
class Product < ApplicationRecord
searchkick mappings: {
properties: {
name: {type: "keyword"}
}
}
end
```
**Note:** If you use a custom mapping, you'll need to use [custom searching](#advanced-search) as well.
To keep the mappings and settings generated by Searchkick, use:
```ruby
class Product < ApplicationRecord
searchkick merge_mappings: true, mappings: {...}
end
```
### Advanced Search
And use the `body` option to search:
```ruby
products = Product.search.body(query: {match: {name: "milk"}})
```
View the response with:
```ruby
products.response
```
To modify the query generated by Searchkick, use:
```ruby
products = Product.search("milk").body_options(min_score: 1)
```
or
```ruby
products =
Product.search("apples") do |body|
body[:min_score] = 1
end
```
### Client
To access the `Elasticsearch::Client` or `OpenSearch::Client` directly, use:
```ruby
Searchkick.client
```
## Multi Search
To batch search requests for performance, use:
```ruby
products = Product.search("snacks")
coupons = Coupon.search("snacks")
Searchkick.multi_search([products, coupons])
```
Then use `products` and `coupons` as typical results.
**Note:** Errors are not raised as with single requests. Use the `error` method on each query to check for errors.
## Multiple Models
Search across multiple models with:
```ruby
Searchkick.search("milk").models(Product, Category)
```
Boost specific models with:
```ruby
indices_boost(Category => 2, Product => 1)
```
## Multi-Tenancy
Check out [this great post](https://www.tiagoamaro.com.br/2014/12/11/multi-tenancy-with-searchkick/) on the [Apartment](https://github.com/influitive/apartment) gem. Follow a similar pattern if you use another gem.
## Scroll API
Searchkick also supports the [scroll API](https://www.elastic.co/guide/en/elasticsearch/reference/current/paginate-search-results.html#scroll-search-results). Scrolling is not intended for real time user requests, but rather for processing large amounts of data.
```ruby
Product.search("*").scroll("1m") do |batch|
# process batch ...
end
```
You can also scroll batches manually.
```ruby
products = Product.search("*").scroll("1m")
while products.any?
# process batch ...
products = products.scroll
end
products.clear_scroll
```
## Deep Paging
By default, Elasticsearch and OpenSearch limit paging to the first 10,000 results. [Here’s why](https://www.elastic.co/guide/en/elasticsearch/guide/current/pagination.html). We don’t recommend changing this, but if you really need all results, you can use:
```ruby
class Product < ApplicationRecord
searchkick deep_paging: true
end
```
If you just need an accurate total count, you can instead use:
```ruby
Product.search("pears").body_options(track_total_hits: true)
```
## Nested Data
To query nested data, use dot notation.
```ruby
Product.search("san").fields("store.city").where("store.zip_code" => 12345)
```
## Nearest Neighbor Search
*Available for Elasticsearch 8.6+ and OpenSearch 2.4+*
```ruby
class Product < ApplicationRecord
searchkick knn: {embedding: {dimensions: 3, distance: "cosine"}}
end
```
Also supports `euclidean` and `inner_product`
Reindex and search with:
```ruby
Product.search.knn(field: :embedding, vector: [1, 2, 3]).limit(10)
```
### HNSW Options
Nearest neighbor search uses [HNSW](https://en.wikipedia.org/wiki/Hierarchical_navigable_small_world) for indexing.
Specify `m` and `ef_construction`
```ruby
class Product < ApplicationRecord
searchkick knn: {embedding: {dimensions: 3, distance: "cosine", m: 16, ef_construction: 100}}
end
```
Specify `ef_search`
```ruby
Product.search.knn(field: :embedding, vector: [1, 2, 3], ef_search: 40).limit(10)
```
## Semantic Search
First, add [nearest neighbor search](#nearest-neighbor-search) to your model
```ruby
class Product < ApplicationRecord
searchkick knn: {embedding: {dimensions: 768, distance: "cosine"}}
end
```
Generate an embedding for each record (you can use an external service or a library like [Informers](https://github.com/ankane/informers))
```ruby
embed = Informers.pipeline("embedding", "Snowflake/snowflake-arctic-embed-m-v1.5")
embed_options = {model_output: "sentence_embedding", pooling: "none"} # specific to embedding model
Product.find_each do |product|
embedding = embed.(product.name, **embed_options)
product.update!(embedding: embedding)
end
```
For search, generate an embedding for the query (the query prefix is specific to the [embedding model](https://huggingface.co/Snowflake/snowflake-arctic-embed-m-v1.5))
```ruby
query_prefix = "Represent this sentence for searching relevant passages: "
query_embedding = embed.(query_prefix + query, **embed_options)
```
And perform nearest neighbor search
```ruby
Product.search.knn(field: :embedding, vector: query_embedding).limit(20)
```
See a [full example](examples/semantic.rb)
## Hybrid Search
Perform keyword search and semantic search in parallel
```ruby
keyword_search = Product.search(query).limit(20)
semantic_search = Product.search.knn(field: :embedding, vector: query_embedding).limit(20)
Searchkick.multi_search([keyword_search, semantic_search])
```
To combine the results, use Reciprocal Rank Fusion (RRF)
```ruby
Searchkick::Reranking.rrf(keyword_search, semantic_search).first(5)
```
Or a reranking model
```ruby
rerank = Informers.pipeline("reranking", "mixedbread-ai/mxbai-rerank-xsmall-v1")
results = (keyword_search.to_a + semantic_search.to_a).uniq
rerank.(query, results.map(&:name)).first(5).map { |v| results[v[:doc_id]] }
```
See a [full example](examples/hybrid.rb)
## Reference
Reindex one record
```ruby
product = Product.find(1)
product.reindex
```
Reindex multiple records
```ruby
Product.where(store_id: 1).reindex
```
Reindex associations
```ruby
store.products.reindex
```
Remove old indices
```ruby
Product.search_index.clean_indices
```
Use custom settings
```ruby
class Product < ApplicationRecord
searchkick settings: {number_of_shards: 3}
end
```
Use a different index name
```ruby
class Product < ApplicationRecord
searchkick index_name: "products_v2"
end
```
Use a dynamic index name
```ruby
class Product < ApplicationRecord
searchkick index_name: -> { "#{name.tableize}-#{I18n.locale}" }
end
```
Prefix the index name
```ruby
class Product < ApplicationRecord
searchkick index_prefix: "datakick"
end
```
For all models
```ruby
Searchkick.index_prefix = "datakick"
```
Use a different term for boosting by conversions
```ruby
Product.search("banana").conversions_v2(term: "organic banana")
```
Define multiple conversion fields
```ruby
class Product < ApplicationRecord
has_many :searches, class_name: "Searchjoy::Search"
searchkick conversions_v2: ["unique_conversions", "total_conversions"]
def search_data
{
name: name,
unique_conversions: searches.group(:query).distinct.count(:user_id),
total_conversions: searches.group(:query).count
}
end
end
```
And specify which to use
```ruby
Product.search("banana") # boost by both fields (default)
Product.search("banana").conversions_v2("total_conversions") # only boost by total_conversions
Product.search("banana").conversions_v2(false) # no conversion boosting
```
Change timeout
```ruby
Searchkick.timeout = 15 # defaults to 10
```
Set a lower timeout for searches
```ruby
Searchkick.search_timeout = 3
```
Change the search method name
```ruby
Searchkick.search_method_name = :lookup
```
Change the queue name
```ruby
Searchkick.queue_name = :search_reindex # defaults to :searchkick
```
Change the queue name or priority for a model
```ruby
class Product < ApplicationRecord
searchkick job_options: {queue: "critical", priority: 10}
end
```
Change the queue name or priority for a specific call
```ruby
Product.reindex(mode: :async, job_options: {queue: "critical", priority: 10})
```
Change the parent job
```ruby
Searchkick.parent_job = "ApplicationJob" # defaults to "ActiveJob::Base"
```
Eager load associations
```ruby
Product.search("milk").includes(:brand, :stores)
```
Eager load different associations by model
```ruby
Searchkick.search("*").models(Product, Store).model_includes(Product => [:store], Store => [:product])
```
Run additional scopes on results
```ruby
Product.search("milk").scope_results(->(r) { r.with_attached_images })
```
Set opaque id for slow logs
```ruby
Product.search("milk").opaque_id("some-id")
# or
Searchkick.multi_search(searches, opaque_id: "some-id")
```
Specify default fields to search
```ruby
class Product < ApplicationRecord
searchkick default_fields: [:name]
end
```
Turn off special characters
```ruby
class Product < ApplicationRecord
# A will not match Ä
searchkick special_characters: false
end
```
Turn on stemming for conversions
```ruby
class Product < ApplicationRecord
searchkick stem_conversions: true
end
```
Make search case-sensitive
```ruby
class Product < ApplicationRecord
searchkick case_sensitive: true
end
```
**Note:** If misspellings are enabled (default), results with a single character case difference will match. Turn off misspellings if this is not desired.
Change import batch size
```ruby
class Product < ApplicationRecord
searchkick batch_size: 200 # defaults to 1000
end
```
Create index without importing
```ruby
Product.reindex(import: false)
```
Use a different id
```ruby
class Product < ApplicationRecord
def search_document_id
custom_id
end
end
```
Add [request parameters](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-search.html#search-search-api-query-params) like `search_type`
```ruby
Product.search("carrots").request_params(search_type: "dfs_query_then_fetch")
```
Set options across all models
```ruby
Searchkick.model_options = {
batch_size: 200
}
```
Reindex conditionally
```ruby
class Product < ApplicationRecord
searchkick callback_options: {if: :search_data_changed?}
def search_data_changed?
previous_changes.include?("name")
end
end
```
Reindex all models - Rails only
```sh
rake searchkick:reindex:all
```
Turn on misspellings after a certain number of characters
```ruby
Product.search("api").misspellings(prefix_length: 2) # api, apt, no ahi
```
BigDecimal values are indexed as floats by default so they can be used for boosting. Convert them to strings to keep full precision.
```ruby
class Product < ApplicationRecord
def search_data
{
units: units.to_s("F")
}
end
end
```
## Gotchas
### Consistency
Elasticsearch and OpenSearch are eventually consistent, meaning it can take up to a second for a change to reflect in search. You can use the `refresh` method to have it show up immediately.
```ruby
product.save!
Product.search_index.refresh
```
### Inconsistent Scores
Due to the distributed nature of Elasticsearch and OpenSearch, you can get incorrect results when the number of documents in the index is low. You can [read more about it here](https://www.elastic.co/blog/understanding-query-then-fetch-vs-dfs-query-then-fetch). To fix this, do:
```ruby
class Product < ApplicationRecord
searchkick settings: {number_of_shards: 1}
end
```
For convenience, this is set by default in the test environment.
## Upgrading
### 6.0
Searchkick 6 brings a new query builder API:
```ruby
Product.search("apples").where(in_stock: true).limit(10).offset(50)
```
All existing options can be used as methods, or you can continue to use the existing API.
This release also significantly improves the performance of searches when using conversions. To upgrade conversions without downtime, add `conversions_v2` to your model and an additional field to `search_data`:
```ruby
class Product < ApplicationRecord
searchkick conversions: [:conversions], conversions_v2: [:conversions_v2]
def search_data
conversions = searches.group(:query).distinct.count(:user_id)
{
conversions: conversions,
conversions_v2: conversions
}
end
end
```
Reindex, then remove `conversions`:
```ruby
class Product < ApplicationRecord
searchkick conversions_v2: [:conversions_v2]
def search_data
{
conversions_v2: searches.group(:query).distinct.count(:user_id)
}
end
end
```
Other improvements include the option to ignore errors for missing documents with partial reindexing and more customization for background jobs. Check out the [changelog](https://github.com/ankane/searchkick/blob/master/CHANGELOG.md) for the full list of changes.
## History
View the [changelog](https://github.com/ankane/searchkick/blob/master/CHANGELOG.md)
## Thanks
Thanks to Karel Minarik for [Elasticsearch Ruby](https://github.com/elasticsearch/elasticsearch-ruby) and [Tire](https://github.com/karmi/retire), Jaroslav Kalistsuk for [zero downtime reindexing](https://gist.github.com/jarosan/3124884), and Alex Leschenko for [Elasticsearch autocomplete](https://github.com/leschenko/elasticsearch_autocomplete).
## Contributing
Everyone is encouraged to help improve this project. Here are a few ways you can help:
- [Report bugs](https://github.com/ankane/searchkick/issues)
- Fix bugs and [submit pull requests](https://github.com/ankane/searchkick/pulls)
- Write, clarify, or fix documentation
- Suggest or add new features
To get started with development:
```sh
git clone https://github.com/ankane/searchkick.git
cd searchkick
bundle install
bundle exec rake test
```
Feel free to open an issue to get feedback on your idea before spending too much time on it.
================================================
FILE: Rakefile
================================================
require "bundler/gem_tasks"
require "rake/testtask"
Rake::TestTask.new do |t|
t.pattern = "test/**/*_test.rb"
end
task default: :test
# to test in parallel, uncomment and run:
# rake parallel:test
# require "parallel_tests/tasks"
================================================
FILE: benchmark/Gemfile
================================================
source "https://rubygems.org"
gemspec path: "../"
gem "sqlite3"
gem "pg"
gem "activerecord", "~> 8.0.0"
gem "activejob"
gem "elasticsearch"
# gem "opensearch-ruby"
gem "redis"
gem "sidekiq"
# performance
gem "typhoeus"
gem "oj"
gem "json"
# profiling
gem "ruby-prof"
gem "allocation_stats"
gem "get_process_mem"
gem "memory_profiler"
# gem "allocation_tracer"
gem "benchmark-ips"
================================================
FILE: benchmark/index.rb
================================================
require "bundler/setup"
Bundler.require(:default)
require "active_record"
require "active_job"
require "benchmark"
require "active_support/notifications"
ActiveSupport::Notifications.subscribe "request.searchkick" do |*args|
event = ActiveSupport::Notifications::Event.new(*args)
# puts "Import: #{event.duration.round}ms"
end
# ActiveJob::Base.queue_adapter = :sidekiq
class SearchSerializer
def dump(object)
JSON.generate(object)
end
end
# Elasticsearch::API.settings[:serializer] = SearchSerializer.new
# OpenSearch::API.settings[:serializer] = SearchSerializer.new
Searchkick.redis = Redis.new
ActiveRecord.default_timezone = :utc
ActiveRecord::Base.time_zone_aware_attributes = true
ActiveRecord::Base.establish_connection adapter: "sqlite3", database: "/tmp/searchkick"
# ActiveRecord::Base.establish_connection "postgresql://localhost/searchkick_bench"
# ActiveRecord::Base.logger = Logger.new(STDOUT)
ActiveJob::Base.logger = nil
class Product < ActiveRecord::Base
searchkick batch_size: 1000
def search_data
{
name: name,
color: color,
store_id: store_id
}
end
end
if ENV["SETUP"]
total_docs = 100000
ActiveRecord::Schema.define do
create_table :products, force: :cascade do |t|
t.string :name
t.string :color
t.integer :store_id
end
end
records = []
total_docs.times do |i|
records << {
name: "Product #{i}",
color: ["red", "blue"].sample,
store_id: rand(10)
}
end
Product.insert_all(records)
puts "Imported"
end
result = nil
report = nil
stats = nil
Product.searchkick_index.delete rescue nil
GC.start
GC.disable
start_mem = GetProcessMem.new.mb
time =
Benchmark.realtime do
# result = RubyProf::Profile.profile do
# report = MemoryProfiler.report do
# stats = AllocationStats.trace do
reindex = Product.reindex #(async: true)
# p reindex
# end
# 60.times do |i|
# if reindex.is_a?(Hash)
# docs = Searchkick::Index.new(reindex[:index_name]).total_docs
# else
# docs = Product.searchkick_index.total_docs
# end
# puts "#{i}: #{docs}"
# if docs == total_docs
# break
# end
# p Searchkick.reindex_status(reindex[:index_name]) if reindex.is_a?(Hash)
# sleep(1)
# # Product.searchkick_index.refresh
# end
end
puts "Time: #{time.round(1)}s"
if result
printer = RubyProf::GraphPrinter.new(result)
printer.print(STDOUT, min_percent: 5)
end
if report
puts report.pretty_print
end
if stats
puts result.allocations(alias_paths: true).group_by(:sourcefile, :class).to_text
end
================================================
FILE: benchmark/relation.rb
================================================
require "bundler/setup"
Bundler.require(:default)
require "active_record"
class Product < ActiveRecord::Base
searchkick
end
Product.all # initial Active Record allocations
stats = AllocationStats.trace do
Product.search("apples").where(store_id: 1).where(in_stock: true).order(:name).limit(10).offset(50)
end
puts stats.allocations(alias_paths: true).to_text
================================================
FILE: benchmark/search.rb
================================================
require "bundler/setup"
Bundler.require(:default)
require "active_record"
require "benchmark/ips"
ActiveRecord.default_timezone = :utc
ActiveRecord::Base.time_zone_aware_attributes = true
ActiveRecord::Base.establish_connection adapter: "sqlite3", database: "/tmp/searchkick"
class Product < ActiveRecord::Base
searchkick batch_size: 1000
def search_data
{
name: name,
color: color,
store_id: store_id
}
end
end
if ENV["SETUP"]
total_docs = 1000000
ActiveRecord::Schema.define do
create_table :products, force: :cascade do |t|
t.string :name
t.string :color
t.integer :store_id
end
end
records = []
total_docs.times do |i|
records << {
name: "Product #{i}",
color: ["red", "blue"].sample,
store_id: rand(10)
}
end
Product.insert_all(records)
puts "Imported"
Product.reindex
puts "Reindexed"
end
query = Product.search("product", fields: [:name], where: {color: "red", store_id: 5}, limit: 10000, load: false)
pp query.body.as_json
puts
Benchmark.ips do |x|
x.report { query.dup.load }
end
================================================
FILE: examples/Gemfile
================================================
source "https://rubygems.org"
gemspec path: ".."
gem "activerecord"
gem "elasticsearch"
gem "informers"
gem "opensearch-ruby"
gem "sqlite3"
================================================
FILE: examples/hybrid.rb
================================================
require "bundler/setup"
require "active_record"
require "elasticsearch" # or "opensearch-ruby"
require "informers"
require "searchkick"
ActiveRecord::Base.establish_connection adapter: "sqlite3", database: ":memory:"
ActiveRecord::Schema.verbose = false
ActiveRecord::Schema.define do
create_table :products do |t|
t.string :name
t.json :embedding
end
end
class Product < ActiveRecord::Base
searchkick knn: {embedding: {dimensions: 768, distance: "cosine"}}
end
Product.reindex
Product.create!(name: "Breakfast cereal")
Product.create!(name: "Ice cream")
Product.create!(name: "Eggs")
embed = Informers.pipeline("embedding", "Snowflake/snowflake-arctic-embed-m-v1.5")
embed_options = {model_output: "sentence_embedding", pooling: "none"} # specific to embedding model
Product.find_each do |product|
embedding = embed.(product.name, **embed_options)
product.update!(embedding: embedding)
end
Product.search_index.refresh
query = "breakfast"
keyword_search = Product.search(query, limit: 20)
# the query prefix is specific to the embedding model (https://huggingface.co/Snowflake/snowflake-arctic-embed-m-v1.5)
query_prefix = "Represent this sentence for searching relevant passages: "
query_embedding = embed.(query_prefix + query, **embed_options)
semantic_search = Product.search(knn: {field: :embedding, vector: query_embedding}, limit: 20)
Searchkick.multi_search([keyword_search, semantic_search])
# to combine the results, use Reciprocal Rank Fusion (RRF)
p Searchkick::Reranking.rrf(keyword_search, semantic_search).first(5).map { |v| v[:result].name }
# or a reranking model
rerank = Informers.pipeline("reranking", "mixedbread-ai/mxbai-rerank-xsmall-v1")
results = (keyword_search.to_a + semantic_search.to_a).uniq
p rerank.(query, results.map(&:name)).first(5).map { |v| results[v[:doc_id]] }.map(&:name)
================================================
FILE: examples/semantic.rb
================================================
require "bundler/setup"
require "active_record"
require "elasticsearch" # or "opensearch-ruby"
require "informers"
require "searchkick"
ActiveRecord::Base.establish_connection adapter: "sqlite3", database: ":memory:"
ActiveRecord::Schema.verbose = false
ActiveRecord::Schema.define do
create_table :products do |t|
t.string :name
t.json :embedding
end
end
class Product < ActiveRecord::Base
searchkick knn: {embedding: {dimensions: 768, distance: "cosine"}}
end
Product.reindex
Product.create!(name: "Cereal")
Product.create!(name: "Ice cream")
Product.create!(name: "Eggs")
embed = Informers.pipeline("embedding", "Snowflake/snowflake-arctic-embed-m-v1.5")
embed_options = {model_output: "sentence_embedding", pooling: "none"} # specific to embedding model
Product.find_each do |product|
embedding = embed.(product.name, **embed_options)
product.update!(embedding: embedding)
end
Product.search_index.refresh
query = "breakfast"
# the query prefix is specific to the embedding model (https://huggingface.co/Snowflake/snowflake-arctic-embed-m-v1.5)
query_prefix = "Represent this sentence for searching relevant passages: "
query_embedding = embed.(query_prefix + query, **embed_options)
pp Product.search(knn: {field: :embedding, vector: query_embedding}, limit: 20).map(&:name)
================================================
FILE: gemfiles/activerecord72.gemfile
================================================
source "https://rubygems.org"
gemspec path: ".."
gem "rake"
gem "minitest"
gem "sqlite3"
gem "activerecord", "~> 7.2.0"
gem "actionpack", "~> 7.2.0"
gem "activejob", "~> 7.2.0", require: "active_job"
gem "elasticsearch", "~> 8"
gem "redis-client"
gem "connection_pool"
gem "kaminari"
gem "gemoji-parser"
================================================
FILE: gemfiles/activerecord80.gemfile
================================================
source "https://rubygems.org"
gemspec path: ".."
gem "rake"
gem "minitest"
gem "sqlite3"
gem "activerecord", "~> 8.0.0"
gem "actionpack", "~> 8.0.0"
gem "activejob", "~> 8.0.0", require: "active_job"
gem "elasticsearch", "~> 9"
gem "redis-client"
gem "connection_pool"
gem "kaminari"
gem "gemoji-parser"
================================================
FILE: gemfiles/mongoid8.gemfile
================================================
source "https://rubygems.org"
gemspec path: ".."
gem "rake"
gem "minitest"
gem "mongoid", "~> 8"
gem "activejob", require: "active_job"
gem "redis"
gem "elasticsearch", "~> 8"
gem "actionpack"
gem "kaminari"
gem "gemoji-parser"
gem "ostruct" # for mongoid
================================================
FILE: gemfiles/mongoid9.gemfile
================================================
source "https://rubygems.org"
gemspec path: ".."
gem "rake"
gem "minitest"
gem "mongoid", "~> 9"
gem "activejob", require: "active_job"
gem "redis"
gem "elasticsearch", "~> 9"
gem "actionpack"
gem "kaminari"
gem "gemoji-parser"
gem "ostruct" # for mongoid
================================================
FILE: gemfiles/opensearch2.gemfile
================================================
source "https://rubygems.org"
gemspec path: ".."
gem "rake"
gem "minitest"
gem "sqlite3"
gem "activerecord", "~> 7.2.0"
gem "actionpack", "~> 7.2.0"
gem "activejob", "~> 7.2.0", require: "active_job"
gem "opensearch-ruby", "~> 2"
gem "redis-client"
gem "connection_pool"
gem "kaminari"
gem "gemoji-parser"
gem "parallel_tests"
gem "typhoeus"
================================================
FILE: gemfiles/opensearch3.gemfile
================================================
source "https://rubygems.org"
gemspec path: ".."
gem "rake"
gem "minitest"
gem "sqlite3"
gem "activerecord", "~> 8.0.0"
gem "actionpack", "~> 8.0.0"
gem "activejob", "~> 8.0.0", require: "active_job"
gem "opensearch-ruby", "~> 3"
gem "redis-client"
gem "connection_pool"
gem "kaminari"
gem "gemoji-parser"
gem "parallel_tests"
gem "typhoeus"
================================================
FILE: lib/searchkick/bulk_reindex_job.rb
================================================
module Searchkick
class BulkReindexJob < Searchkick.parent_job.constantize
queue_as { Searchkick.queue_name }
def perform(class_name:, record_ids: nil, index_name: nil, method_name: nil, batch_id: nil, min_id: nil, max_id: nil, ignore_missing: nil)
model = Searchkick.load_model(class_name)
index = model.searchkick_index(name: index_name)
record_ids ||= min_id..max_id
relation = Searchkick.scope(model)
relation = Searchkick.load_records(relation, record_ids)
relation = relation.search_import if relation.respond_to?(:search_import)
RecordIndexer.new(index).reindex(relation, mode: :inline, method_name: method_name, ignore_missing: ignore_missing, full: false)
RelationIndexer.new(index).batch_completed(batch_id) if batch_id
end
end
end
================================================
FILE: lib/searchkick/controller_runtime.rb
================================================
# based on https://gist.github.com/mnutt/566725
module Searchkick
module ControllerRuntime
extend ActiveSupport::Concern
protected
attr_internal :searchkick_runtime
def process_action(action, *args)
# We also need to reset the runtime before each action
# because of queries in middleware or in cases we are streaming
# and it won't be cleaned up by the method below.
Searchkick::LogSubscriber.reset_runtime
super
end
def cleanup_view_runtime
searchkick_rt_before_render = Searchkick::LogSubscriber.reset_runtime
runtime = super
searchkick_rt_after_render = Searchkick::LogSubscriber.reset_runtime
self.searchkick_runtime = searchkick_rt_before_render + searchkick_rt_after_render
runtime - searchkick_rt_after_render
end
def append_info_to_payload(payload)
super
payload[:searchkick_runtime] = (searchkick_runtime || 0) + Searchkick::LogSubscriber.reset_runtime
end
module ClassMethods
def log_process_action(payload)
messages = super
runtime = payload[:searchkick_runtime]
messages << ("Searchkick: %.1fms" % runtime.to_f) if runtime.to_f > 0
messages
end
end
end
end
================================================
FILE: lib/searchkick/hash_wrapper.rb
================================================
module Searchkick
class HashWrapper
def initialize(attributes)
@attributes = attributes
end
def [](name)
@attributes[name.to_s]
end
def to_h
@attributes
end
def as_json(...)
@attributes.as_json(...)
end
def to_json(...)
@attributes.to_json(...)
end
def method_missing(name, ...)
if @attributes.key?(name.to_s)
self[name]
else
super
end
end
def respond_to_missing?(name, ...)
@attributes.key?(name.to_s) || super
end
def inspect
attributes = @attributes.reject { |k, v| k[0] == "_" }.map { |k, v| "#{k}: #{v.inspect}" }
attributes.unshift(attributes.pop) # move id to start
"#<#{self.class.name} #{attributes.join(", ")}>"
end
end
end
================================================
FILE: lib/searchkick/index.rb
================================================
module Searchkick
class Index
attr_reader :name, :options
def initialize(name, options = {})
@name = name
@options = options
@klass_document_type = {} # cache
end
def index_options
IndexOptions.new(self).index_options
end
def create(body = {})
client.indices.create index: name, body: body
end
def delete
if alias_exists?
# can't call delete directly on aliases in ES 6
indices = client.indices.get_alias(name: name).keys
client.indices.delete index: indices
else
client.indices.delete index: name
end
end
def exists?
client.indices.exists index: name
end
def refresh
client.indices.refresh index: name
end
def alias_exists?
client.indices.exists_alias name: name
end
# call to_h for consistent results between elasticsearch gem 7 and 8
# could do for all API calls, but just do for ones where return value is focus for now
def mapping
client.indices.get_mapping(index: name).to_h
end
# call to_h for consistent results between elasticsearch gem 7 and 8
def settings
client.indices.get_settings(index: name).to_h
end
def refresh_interval
index_settings["refresh_interval"]
end
def update_settings(settings)
client.indices.put_settings index: name, body: settings
end
def tokens(text, options = {})
client.indices.analyze(body: {text: text}.merge(options), index: name)["tokens"].map { |t| t["token"] }
end
def total_docs
response =
client.search(
index: name,
body: {
query: {match_all: {}},
size: 0,
track_total_hits: true
}
)
Results.new(nil, response).total_count
end
def promote(new_name, update_refresh_interval: false)
if update_refresh_interval
new_index = Index.new(new_name, @options)
settings = options[:settings] || {}
refresh_interval = (settings[:index] && settings[:index][:refresh_interval]) || "1s"
new_index.update_settings(index: {refresh_interval: refresh_interval})
end
old_indices =
begin
client.indices.get_alias(name: name).keys
rescue => e
raise e unless Searchkick.not_found_error?(e)
{}
end
actions = old_indices.map { |old_name| {remove: {index: old_name, alias: name}} } + [{add: {index: new_name, alias: name}}]
client.indices.update_aliases body: {actions: actions}
end
alias_method :swap, :promote
def retrieve(record)
record_data = RecordData.new(self, record).record_data
# remove underscore
get_options = record_data.to_h { |k, v| [k.to_s.delete_prefix("_").to_sym, v] }
client.get(get_options)["_source"]
end
def all_indices(unaliased: false)
indices =
begin
if client.indices.respond_to?(:get_alias)
client.indices.get_alias(index: "#{name}*")
else
client.indices.get_aliases
end
rescue => e
raise e unless Searchkick.not_found_error?(e)
{}
end
indices = indices.select { |_k, v| v.empty? || v["aliases"].empty? } if unaliased
indices.select { |k, _v| k =~ /\A#{Regexp.escape(name)}_\d{14,17}\z/ }.keys
end
# remove old indices that start w/ index_name
def clean_indices
indices = all_indices(unaliased: true)
indices.each do |index|
Index.new(index).delete
end
indices
end
def store(record)
notify(record, "Store") do
queue_index([record])
end
end
def remove(record)
notify(record, "Remove") do
queue_delete([record])
end
end
def update_record(record, method_name)
notify(record, "Update") do
queue_update([record], method_name)
end
end
def bulk_delete(records)
return if records.empty?
notify_bulk(records, "Delete") do
queue_delete(records)
end
end
def bulk_index(records)
return if records.empty?
notify_bulk(records, "Import") do
queue_index(records)
end
end
alias_method :import, :bulk_index
def bulk_update(records, method_name, ignore_missing: nil)
return if records.empty?
notify_bulk(records, "Update") do
queue_update(records, method_name, ignore_missing: ignore_missing)
end
end
def search_id(record)
RecordData.new(self, record).search_id
end
def document_type(record)
RecordData.new(self, record).document_type
end
def similar_record(record, **options)
options[:per_page] ||= 10
options[:similar] = [RecordData.new(self, record).record_data]
options[:models] ||= [record.class] unless options.key?(:model)
Searchkick.search("*", **options)
end
def reload_synonyms
if Searchkick.opensearch?
client.transport.perform_request "POST", "_plugins/_refresh_search_analyzers/#{CGI.escape(name)}"
else
begin
client.transport.perform_request("GET", "#{CGI.escape(name)}/_reload_search_analyzers")
rescue => e
raise Error, "Requires non-OSS version of Elasticsearch" if Searchkick.not_allowed_error?(e)
raise e
end
end
end
# queue
def reindex_queue
ReindexQueue.new(name)
end
# reindex
# note: this is designed to be used internally
# so it does not check object matches index class
def reindex(object, method_name: nil, ignore_missing: nil, full: false, **options)
if @options[:job_options]
options[:job_options] = (@options[:job_options] || {}).merge(options[:job_options] || {})
end
if object.is_a?(Array)
# note: purposefully skip full
return reindex_records(object, method_name: method_name, ignore_missing: ignore_missing, **options)
end
if !object.respond_to?(:searchkick_klass)
raise Error, "Cannot reindex object"
end
scoped = Searchkick.relation?(object)
# call searchkick_klass for inheritance
relation = scoped ? object.all : Searchkick.scope(object.searchkick_klass).all
refresh = options.fetch(:refresh, !scoped)
options.delete(:refresh)
if method_name || (scoped && !full)
mode = options.delete(:mode) || :inline
scope = options.delete(:scope)
job_options = options.delete(:job_options)
raise ArgumentError, "unsupported keywords: #{options.keys.map(&:inspect).join(", ")}" if options.any?
# import only
import_scope(relation, method_name: method_name, mode: mode, scope: scope, ignore_missing: ignore_missing, job_options: job_options)
self.refresh if refresh
true
else
async = options.delete(:async)
if async
if async.is_a?(Hash) && async[:wait]
Searchkick.warn "async option is deprecated - use mode: :async, wait: true instead"
options[:wait] = true unless options.key?(:wait)
else
Searchkick.warn "async option is deprecated - use mode: :async instead"
end
options[:mode] ||= :async
end
full_reindex(relation, **options)
end
end
def create_index(index_options: nil)
index_options ||= self.index_options
index = Index.new("#{name}_#{Time.now.strftime('%Y%m%d%H%M%S%L')}", @options)
index.create(index_options)
index
end
def import_scope(relation, **options)
relation_indexer.reindex(relation, **options)
end
def batches_left
relation_indexer.batches_left
end
# private
def klass_document_type(klass, ignore_type = false)
@klass_document_type[[klass, ignore_type]] ||= begin
if !ignore_type && klass.searchkick_klass.searchkick_options[:_type]
type = klass.searchkick_klass.searchkick_options[:_type]
type = type.call if type.respond_to?(:call)
type
else
klass.model_name.to_s.underscore
end
end
end
# private
def conversions_fields
@conversions_fields ||= begin
conversions = Array(options[:conversions])
conversions.map(&:to_s) + conversions.map(&:to_sym)
end
end
# private
def conversions_v2_fields
@conversions_v2_fields ||= Array(options[:conversions_v2]).map(&:to_s)
end
# private
def suggest_fields
@suggest_fields ||= Array(options[:suggest]).map(&:to_s)
end
# private
def locations_fields
@locations_fields ||= begin
locations = Array(options[:locations])
locations.map(&:to_s) + locations.map(&:to_sym)
end
end
# private
def uuid
index_settings["uuid"]
end
protected
def client
Searchkick.client
end
def queue_index(records)
Searchkick.indexer.queue(records.map { |r| RecordData.new(self, r).index_data })
end
def queue_delete(records)
Searchkick.indexer.queue(records.reject { |r| r.id.blank? }.map { |r| RecordData.new(self, r).delete_data })
end
def queue_update(records, method_name, ignore_missing:)
items = records.map { |r| RecordData.new(self, r).update_data(method_name) }
items.each { |i| i.instance_variable_set(:@ignore_missing, true) } if ignore_missing
Searchkick.indexer.queue(items)
end
def relation_indexer
@relation_indexer ||= RelationIndexer.new(self)
end
def index_settings
settings.values.first["settings"]["index"]
end
def import_before_promotion(index, relation, **import_options)
index.import_scope(relation, **import_options)
end
def reindex_records(object, mode: nil, refresh: false, **options)
mode ||= Searchkick.callbacks_value || @options[:callbacks] || :inline
mode = :inline if mode == :bulk
result = RecordIndexer.new(self).reindex(object, mode: mode, full: false, **options)
self.refresh if refresh
result
end
# https://gist.github.com/jarosan/3124884
# https://www.elastic.co/blog/changing-mapping-with-zero-downtime/
def full_reindex(relation, import: true, resume: false, retain: false, mode: nil, refresh_interval: nil, scope: nil, wait: nil, job_options: nil)
raise ArgumentError, "wait only available in :async mode" if !wait.nil? && mode != :async
raise ArgumentError, "Full reindex does not support :queue mode - use :async mode instead" if mode == :queue
if resume
index_name = all_indices.sort.last
raise Error, "No index to resume" unless index_name
index = Index.new(index_name, @options)
else
clean_indices unless retain
index_options = relation.searchkick_index_options
index_options.deep_merge!(settings: {index: {refresh_interval: refresh_interval}}) if refresh_interval
index = create_index(index_options: index_options)
end
import_options = {
mode: (mode || :inline),
full: true,
resume: resume,
scope: scope,
job_options: job_options
}
uuid = index.uuid
# check if alias exists
alias_exists = alias_exists?
if alias_exists
import_before_promotion(index, relation, **import_options) if import
# get existing indices to remove
unless mode == :async
check_uuid(uuid, index.uuid)
promote(index.name, update_refresh_interval: !refresh_interval.nil?)
clean_indices unless retain
end
else
delete if exists?
promote(index.name, update_refresh_interval: !refresh_interval.nil?)
# import after promotion
index.import_scope(relation, **import_options) if import
end
if mode == :async
if wait
puts "Created index: #{index.name}"
puts "Jobs queued. Waiting..."
loop do
sleep 3
status = Searchkick.reindex_status(index.name)
break if status[:completed]
puts "Batches left: #{status[:batches_left]}"
end
# already promoted if alias didn't exist
if alias_exists
puts "Jobs complete. Promoting..."
check_uuid(uuid, index.uuid)
promote(index.name, update_refresh_interval: !refresh_interval.nil?)
end
clean_indices unless retain
puts "SUCCESS!"
end
{index_name: index.name}
else
index.refresh
true
end
rescue => e
if Searchkick.transport_error?(e) && (e.message.include?("No handler for type [text]") || e.message.include?("class java.util.ArrayList cannot be cast to class java.util.Map"))
raise UnsupportedVersionError
end
raise e
end
# safety check
# still a chance for race condition since its called before promotion
# ideal is for user to disable automatic index creation
# https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-index_.html#index-creation
def check_uuid(old_uuid, new_uuid)
if old_uuid != new_uuid
raise Error, "Safety check failed - only run one Model.reindex per model at a time"
end
end
def notify(record, name)
if Searchkick.callbacks_value == :bulk
yield
else
name = "#{record.class.searchkick_klass.name} #{name}" if record && record.class.searchkick_klass
event = {
name: name,
id: search_id(record)
}
ActiveSupport::Notifications.instrument("request.searchkick", event) do
yield
end
end
end
def notify_bulk(records, name)
if Searchkick.callbacks_value == :bulk
yield
else
event = {
name: "#{records.first.class.searchkick_klass.name} #{name}",
count: records.size
}
ActiveSupport::Notifications.instrument("request.searchkick", event) do
yield
end
end
end
end
end
================================================
FILE: lib/searchkick/index_cache.rb
================================================
module Searchkick
class IndexCache
def initialize(max_size: 20)
@data = {}
@mutex = Mutex.new
@max_size = max_size
end
# probably a better pattern for this
# but keep it simple
def fetch(name)
# thread-safe in MRI without mutex
# due to how context switching works
@mutex.synchronize do
if @data.key?(name)
@data[name]
else
@data.clear if @data.size >= @max_size
@data[name] = yield
end
end
end
def clear
@mutex.synchronize do
@data.clear
end
end
end
end
================================================
FILE: lib/searchkick/index_options.rb
================================================
module Searchkick
class IndexOptions
attr_reader :options
def initialize(index)
@options = index.options
end
def index_options
# mortal symbols are garbage collected in Ruby 2.2+
custom_settings = (options[:settings] || {}).deep_symbolize_keys
custom_mappings = (options[:mappings] || {}).deep_symbolize_keys
if options[:mappings] && !options[:merge_mappings]
settings = custom_settings
mappings = custom_mappings
else
settings = generate_settings.deep_symbolize_keys.deep_merge(custom_settings)
mappings = generate_mappings.deep_symbolize_keys.deep_merge(custom_mappings)
end
set_deep_paging(settings) if options[:deep_paging] || options[:max_result_window]
{
settings: settings,
mappings: mappings
}
end
def generate_settings
language = options[:language]
language = language.call if language.respond_to?(:call)
settings = {
analysis: {
analyzer: {
searchkick_keyword: {
type: "custom",
tokenizer: "keyword",
filter: ["lowercase"] + (options[:stem_conversions] ? ["searchkick_stemmer"] : [])
},
default_analyzer => {
type: "custom",
# character filters -> tokenizer -> token filters
# https://www.elastic.co/guide/en/elasticsearch/guide/current/analysis-intro.html
char_filter: ["ampersand"],
tokenizer: "standard",
# synonym should come last, after stemming and shingle
# shingle must come before searchkick_stemmer
filter: ["lowercase", "asciifolding", "searchkick_index_shingle", "searchkick_stemmer"]
},
searchkick_search: {
type: "custom",
char_filter: ["ampersand"],
tokenizer: "standard",
filter: ["lowercase", "asciifolding", "searchkick_search_shingle", "searchkick_stemmer"]
},
searchkick_search2: {
type: "custom",
char_filter: ["ampersand"],
tokenizer: "standard",
filter: ["lowercase", "asciifolding", "searchkick_stemmer"]
},
# https://github.com/leschenko/elasticsearch_autocomplete/blob/master/lib/elasticsearch_autocomplete/analyzers.rb
searchkick_autocomplete_search: {
type: "custom",
tokenizer: "keyword",
filter: ["lowercase", "asciifolding"]
},
searchkick_word_search: {
type: "custom",
tokenizer: "standard",
filter: ["lowercase", "asciifolding"]
},
searchkick_suggest_index: {
type: "custom",
tokenizer: "standard",
filter: ["lowercase", "asciifolding", "searchkick_suggest_shingle"]
},
searchkick_text_start_index: {
type: "custom",
tokenizer: "keyword",
filter: ["lowercase", "asciifolding", "searchkick_edge_ngram"]
},
searchkick_text_middle_index: {
type: "custom",
tokenizer: "keyword",
filter: ["lowercase", "asciifolding", "searchkick_ngram"]
},
searchkick_text_end_index: {
type: "custom",
tokenizer: "keyword",
filter: ["lowercase", "asciifolding", "reverse", "searchkick_edge_ngram", "reverse"]
},
searchkick_word_start_index: {
type: "custom",
tokenizer: "standard",
filter: ["lowercase", "asciifolding", "searchkick_edge_ngram"]
},
searchkick_word_middle_index: {
type: "custom",
tokenizer: "standard",
filter: ["lowercase", "asciifolding", "searchkick_ngram"]
},
searchkick_word_end_index: {
type: "custom",
tokenizer: "standard",
filter: ["lowercase", "asciifolding", "reverse", "searchkick_edge_ngram", "reverse"]
}
},
filter: {
searchkick_index_shingle: {
type: "shingle",
token_separator: ""
},
# lucky find https://web.archiveorange.com/archive/v/AAfXfQ17f57FcRINsof7
searchkick_search_shingle: {
type: "shingle",
token_separator: "",
output_unigrams: false,
output_unigrams_if_no_shingles: true
},
searchkick_suggest_shingle: {
type: "shingle",
max_shingle_size: 5
},
searchkick_edge_ngram: {
type: "edge_ngram",
min_gram: 1,
max_gram: 50
},
searchkick_ngram: {
type: "ngram",
min_gram: 1,
max_gram: 50
},
searchkick_stemmer: {
# use stemmer if language is lowercase, snowball otherwise
type: language == language.to_s.downcase ? "stemmer" : "snowball",
language: language || "English"
}
},
char_filter: {
# https://www.elastic.co/guide/en/elasticsearch/guide/current/custom-analyzers.html
# &_to_and
ampersand: {
type: "mapping",
mappings: ["&=> and "]
}
}
}
}
raise ArgumentError, "Can't pass both language and stemmer" if options[:stemmer] && language
update_language(settings, language)
update_stemming(settings)
if Searchkick.env == "test"
settings[:number_of_shards] = 1
settings[:number_of_replicas] = 0
end
if options[:similarity]
settings[:similarity] = {default: {type: options[:similarity]}}
end
settings[:index] = {
max_ngram_diff: 49,
max_shingle_diff: 4
}
if options[:knn]
unless Searchkick.knn_support?
if Searchkick.opensearch?
raise Error, "knn requires OpenSearch 2.4+"
else
raise Error, "knn requires Elasticsearch 8.6+"
end
end
if Searchkick.opensearch? && options[:knn].any? { |_, v| !v[:distance].nil? }
# only enable if doing approximate search
settings[:index][:knn] = true
end
end
add_synonyms(settings)
add_search_synonyms(settings)
if options[:special_characters] == false
settings[:analysis][:analyzer].each_value do |analyzer_settings|
analyzer_settings[:filter].reject! { |f| f == "asciifolding" }
end
end
if options[:case_sensitive]
settings[:analysis][:analyzer].each do |_, analyzer|
analyzer[:filter].delete("lowercase")
end
end
settings
end
def update_language(settings, language)
case language
when "chinese"
settings[:analysis][:analyzer].merge!(
default_analyzer => {
type: "ik_smart"
},
searchkick_search: {
type: "ik_smart"
},
searchkick_search2: {
type: "ik_max_word"
}
)
when "chinese2", "smartcn"
settings[:analysis][:analyzer].merge!(
default_analyzer => {
type: "smartcn"
},
searchkick_search: {
type: "smartcn"
},
searchkick_search2: {
type: "smartcn"
}
)
when "japanese", "japanese2"
analyzer = {
type: "custom",
tokenizer: "kuromoji_tokenizer",
filter: [
"kuromoji_baseform",
"kuromoji_part_of_speech",
"cjk_width",
"ja_stop",
"searchkick_stemmer",
"lowercase"
]
}
settings[:analysis][:analyzer].merge!(
default_analyzer => analyzer.deep_dup,
searchkick_search: analyzer.deep_dup,
searchkick_search2: analyzer.deep_dup
)
settings[:analysis][:filter][:searchkick_stemmer] = {
type: "kuromoji_stemmer"
}
when "korean"
settings[:analysis][:analyzer].merge!(
default_analyzer => {
type: "openkoreantext-analyzer"
},
searchkick_search: {
type: "openkoreantext-analyzer"
},
searchkick_search2: {
type: "openkoreantext-analyzer"
}
)
when "korean2"
settings[:analysis][:analyzer].merge!(
default_analyzer => {
type: "nori"
},
searchkick_search: {
type: "nori"
},
searchkick_search2: {
type: "nori"
}
)
when "vietnamese"
settings[:analysis][:analyzer].merge!(
default_analyzer => {
type: "vi_analyzer"
},
searchkick_search: {
type: "vi_analyzer"
},
searchkick_search2: {
type: "vi_analyzer"
}
)
when "polish", "ukrainian"
settings[:analysis][:analyzer].merge!(
default_analyzer => {
type: language
},
searchkick_search: {
type: language
},
searchkick_search2: {
type: language
}
)
end
end
def update_stemming(settings)
if options[:stemmer]
stemmer = options[:stemmer]
# could also support snowball and stemmer
case stemmer[:type]
when "hunspell"
# supports all token filter options
settings[:analysis][:filter][:searchkick_stemmer] = stemmer
else
raise ArgumentError, "Unknown stemmer: #{stemmer[:type]}"
end
end
stem = options[:stem]
# language analyzer used
stem = false if settings[:analysis][:analyzer][default_analyzer][:type] != "custom"
if stem == false
settings[:analysis][:filter].delete(:searchkick_stemmer)
settings[:analysis][:analyzer].each do |_, analyzer|
analyzer[:filter].delete("searchkick_stemmer") if analyzer[:filter]
end
end
if options[:stemmer_override]
stemmer_override = {
type: "stemmer_override"
}
if options[:stemmer_override].is_a?(String)
stemmer_override[:rules_path] = options[:stemmer_override]
else
stemmer_override[:rules] = options[:stemmer_override]
end
settings[:analysis][:filter][:searchkick_stemmer_override] = stemmer_override
settings[:analysis][:analyzer].each do |_, analyzer|
stemmer_index = analyzer[:filter].index("searchkick_stemmer") if analyzer[:filter]
analyzer[:filter].insert(stemmer_index, "searchkick_stemmer_override") if stemmer_index
end
end
if options[:stem_exclusion]
settings[:analysis][:filter][:searchkick_stem_exclusion] = {
type: "keyword_marker",
keywords: options[:stem_exclusion]
}
settings[:analysis][:analyzer].each do |_, analyzer|
stemmer_index = analyzer[:filter].index("searchkick_stemmer") if analyzer[:filter]
analyzer[:filter].insert(stemmer_index, "searchkick_stem_exclusion") if stemmer_index
end
end
end
def generate_mappings
mapping = {}
keyword_mapping = {type: "keyword"}
keyword_mapping[:ignore_above] = options[:ignore_above] || 30000
# conversions
Array(options[:conversions]).each do |conversions_field|
mapping[conversions_field] = {
type: "nested",
properties: {
query: {type: default_type, analyzer: "searchkick_keyword"},
count: {type: "integer"}
}
}
end
Array(options[:conversions_v2]).each do |conversions_field|
mapping[conversions_field] = {
type: "rank_features"
}
end
if (Array(options[:conversions_v2]).map(&:to_s) & Array(options[:conversions]).map(&:to_s)).any?
raise ArgumentError, "Must have separate conversions fields"
end
mapping_options =
[:suggest, :word, :text_start, :text_middle, :text_end, :word_start, :word_middle, :word_end, :highlight, :searchable, :filterable]
.to_h { |type| [type, (options[type] || []).map(&:to_s)] }
word = options[:word] != false && (!options[:match] || options[:match] == :word)
mapping_options[:searchable].delete("_all")
analyzed_field_options = {type: default_type, index: true, analyzer: default_analyzer.to_s}
mapping_options.values.flatten.uniq.each do |field|
fields = {}
if options.key?(:filterable) && !mapping_options[:filterable].include?(field)
fields[field] = {type: default_type, index: false}
else
fields[field] = keyword_mapping
end
if !options[:searchable] || mapping_options[:searchable].include?(field)
if word
fields[:analyzed] = analyzed_field_options
if mapping_options[:highlight].include?(field)
fields[:analyzed][:term_vector] = "with_positions_offsets"
end
end
mapping_options.except(:highlight, :searchable, :filterable, :word).each do |type, f|
if options[:match] == type || f.include?(field)
fields[type] = {type: default_type, index: true, analyzer: "searchkick_#{type}_index"}
end
end
end
mapping[field] = fields[field].merge(fields: fields.except(field))
end
(options[:locations] || []).map(&:to_s).each do |field|
mapping[field] = {
type: "geo_point"
}
end
options[:geo_shape] = options[:geo_shape].product([{}]).to_h if options[:geo_shape].is_a?(Array)
(options[:geo_shape] || {}).each do |field, shape_options|
mapping[field] = shape_options.merge(type: "geo_shape")
end
(options[:knn] || []).each do |field, knn_options|
distance = knn_options[:distance]
quantization = knn_options[:quantization]
if Searchkick.opensearch?
if distance.nil?
# avoid server crash if method not specified
raise ArgumentError, "Must specify a distance for OpenSearch"
end
vector_options = {
type: "knn_vector",
dimension: knn_options[:dimensions]
}
if !distance.nil?
space_type =
case distance
when "cosine"
"cosinesimil"
when "euclidean"
"l2"
when "inner_product"
"innerproduct"
else
raise ArgumentError, "Unknown distance: #{distance}"
end
if !quantization.nil?
raise ArgumentError, "Quantization not supported yet for OpenSearch"
end
vector_options[:method] = {
name: "hnsw",
space_type: space_type,
engine: "lucene",
parameters: knn_options.slice(:m, :ef_construction)
}
end
mapping[field.to_s] = vector_options
else
vector_options = {
type: "dense_vector",
dims: knn_options[:dimensions],
index: !distance.nil?
}
if !distance.nil?
vector_options[:similarity] =
case distance
when "cosine"
"cosine"
when "euclidean"
"l2_norm"
when "inner_product"
"max_inner_product"
else
raise ArgumentError, "Unknown distance: #{distance}"
end
type =
case quantization
when "int8", "int4", "bbq"
"#{quantization}_hnsw"
when nil
"hnsw"
else
raise ArgumentError, "Unknown quantization: #{quantization}"
end
vector_index_options = knn_options.slice(:m, :ef_construction)
vector_options[:index_options] = {type: type}.merge(vector_index_options)
end
mapping[field.to_s] = vector_options
end
end
if options[:inheritance]
mapping[:type] = keyword_mapping
end
routing = {}
if options[:routing]
routing = {required: true}
unless options[:routing] == true
routing[:path] = options[:routing].to_s
end
end
dynamic_fields = {
# analyzed field must be the default field for include_in_all
# https://www.elastic.co/guide/reference/mapping/multi-field-type/
# however, we can include the not_analyzed field in _all
# and the _all index analyzer will take care of it
"{name}" => keyword_mapping
}
if options.key?(:filterable)
dynamic_fields["{name}"] = {type: default_type, index: false}
end
unless options[:searchable]
if options[:match] && options[:match] != :word
dynamic_fields[options[:match]] = {type: default_type, index: true, analyzer: "searchkick_#{options[:match]}_index"}
end
if word
dynamic_fields[:analyzed] = analyzed_field_options
end
end
# https://www.elastic.co/guide/reference/mapping/multi-field-type/
multi_field = dynamic_fields["{name}"].merge(fields: dynamic_fields.except("{name}"))
mappings = {
properties: mapping,
_routing: routing,
# https://gist.github.com/kimchy/2898285
dynamic_templates: [
{
string_template: {
match: "*",
match_mapping_type: "string",
mapping: multi_field
}
}
]
}
mappings
end
def add_synonyms(settings)
synonyms = options[:synonyms] || []
synonyms = synonyms.call if synonyms.respond_to?(:call)
if synonyms.any?
settings[:analysis][:filter][:searchkick_synonym] = {
type: "synonym",
# only remove a single space from synonyms so three-word synonyms will fail noisily instead of silently
synonyms: synonyms.select { |s| s.size > 1 }.map { |s| s.is_a?(Array) ? s.map { |s2| s2.sub(/\s+/, "") }.join(",") : s }.map(&:downcase)
}
# choosing a place for the synonym filter when stemming is not easy
# https://groups.google.com/forum/#!topic/elasticsearch/p7qcQlgHdB8
# TODO use a snowball stemmer on synonyms when creating the token filter
# https://discuss.elastic.co/t/synonym-multi-words-search/10964
# I find the following approach effective if you are doing multi-word synonyms (synonym phrases):
# - Only apply the synonym expansion at index time
# - Don't have the synonym filter applied search
# - Use directional synonyms where appropriate. You want to make sure that you're not injecting terms that are too general.
settings[:analysis][:analyzer][default_analyzer][:filter].insert(2, "searchkick_synonym")
%w(word_start word_middle word_end).each do |type|
settings[:analysis][:analyzer]["searchkick_#{type}_index".to_sym][:filter].insert(2, "searchkick_synonym")
end
end
end
def add_search_synonyms(settings)
search_synonyms = options[:search_synonyms] || []
search_synonyms = search_synonyms.call if search_synonyms.respond_to?(:call)
if search_synonyms.is_a?(String) || search_synonyms.any?
if search_synonyms.is_a?(String)
synonym_graph = {
type: "synonym_graph",
synonyms_path: search_synonyms,
updateable: true
}
else
synonym_graph = {
type: "synonym_graph",
# TODO confirm this is correct
synonyms: search_synonyms.select { |s| s.size > 1 }.map { |s| s.is_a?(Array) ? s.join(",") : s }.map(&:downcase)
}
end
settings[:analysis][:filter][:searchkick_synonym_graph] = synonym_graph
if ["japanese", "japanese2"].include?(options[:language])
[:searchkick_search, :searchkick_search2].each do |analyzer|
settings[:analysis][:analyzer][analyzer][:filter].insert(4, "searchkick_synonym_graph")
end
else
[:searchkick_search2, :searchkick_word_search].each do |analyzer|
unless settings[:analysis][:analyzer][analyzer].key?(:filter)
raise Error, "Search synonyms are not supported yet for language"
end
settings[:analysis][:analyzer][analyzer][:filter].insert(2, "searchkick_synonym_graph")
end
end
end
end
def set_deep_paging(settings)
if !settings.dig(:index, :max_result_window) && !settings[:"index.max_result_window"]
settings[:index] ||= {}
settings[:index][:max_result_window] = options[:max_result_window] || 1_000_000_000
end
end
def index_type
@index_type ||= begin
index_type = options[:_type]
index_type = index_type.call if index_type.respond_to?(:call)
index_type
end
end
def default_type
"text"
end
def default_analyzer
:searchkick_index
end
end
end
================================================
FILE: lib/searchkick/indexer.rb
================================================
# thread-local (technically fiber-local) indexer
# used to aggregate bulk callbacks across models
module Searchkick
class Indexer
attr_reader :queued_items
def initialize
@queued_items = []
end
def queue(items)
@queued_items.concat(items)
perform unless Searchkick.callbacks_value == :bulk
end
def perform
items = @queued_items
@queued_items = []
return if items.empty?
response = Searchkick.client.bulk(body: items)
if response["errors"]
# note: delete does not set error when item not found
first_with_error = response["items"].map do |item|
(item["index"] || item["delete"] || item["update"])
end.find.with_index { |item, i| item["error"] && !ignore_missing?(items[i], item["error"]) }
if first_with_error
raise ImportError, "#{first_with_error["error"]} on item with id '#{first_with_error["_id"]}'"
end
end
# maybe return response in future
nil
end
private
def ignore_missing?(item, error)
error["type"] == "document_missing_exception" && item.instance_variable_defined?(:@ignore_missing)
end
end
end
================================================
FILE: lib/searchkick/log_subscriber.rb
================================================
# based on https://gist.github.com/mnutt/566725
module Searchkick
class LogSubscriber < ActiveSupport::LogSubscriber
def self.runtime=(value)
Thread.current[:searchkick_runtime] = value
end
def self.runtime
Thread.current[:searchkick_runtime] ||= 0
end
def self.reset_runtime
rt = runtime
self.runtime = 0
rt
end
def search(event)
self.class.runtime += event.duration
return unless logger.debug?
payload = event.payload
name = "#{payload[:name]} (#{event.duration.round(1)}ms)"
index = payload[:query][:index].is_a?(Array) ? payload[:query][:index].join(",") : payload[:query][:index]
type = payload[:query][:type]
request_params = payload[:query].except(:index, :type, :body, :opaque_id)
params = []
request_params.each do |k, v|
params << "#{CGI.escape(k.to_s)}=#{CGI.escape(v.to_s)}"
end
debug " #{color(name, YELLOW, bold: true)} #{index}#{type ? "/#{type.join(',')}" : ''}/_search#{params.any? ? '?' + params.join('&') : nil} #{payload[:query][:body].to_json}"
end
def request(event)
self.class.runtime += event.duration
return unless logger.debug?
payload = event.payload
name = "#{payload[:name]} (#{event.duration.round(1)}ms)"
debug " #{color(name, YELLOW, bold: true)} #{payload.except(:name).to_json}"
end
def multi_search(event)
self.class.runtime += event.duration
return unless logger.debug?
payload = event.payload
name = "#{payload[:name]} (#{event.duration.round(1)}ms)"
debug " #{color(name, YELLOW, bold: true)} _msearch #{payload[:body]}"
end
end
end
================================================
FILE: lib/searchkick/middleware.rb
================================================
require "faraday"
module Searchkick
class Middleware < Faraday::Middleware
def call(env)
path = env[:url].path.to_s
if path.end_with?("/_search")
env[:request][:timeout] = Searchkick.search_timeout
elsif path.end_with?("/_msearch")
# assume no concurrent searches for timeout for now
searches = env[:request_body].count("\n") / 2
# do not allow timeout to exceed Searchkick.timeout
timeout = [Searchkick.search_timeout * searches, Searchkick.timeout].min
env[:request][:timeout] = timeout
end
@app.call(env)
end
end
end
================================================
FILE: lib/searchkick/model.rb
================================================
module Searchkick
module Model
def searchkick(**options)
options = Searchkick.model_options.deep_merge(options)
if options[:conversions]
Searchkick.warn("The `conversions` option is deprecated in favor of `conversions_v2`, which provides much better search performance. Upgrade to `conversions_v2` or rename `conversions` to `conversions_v1`")
end
if options.key?(:conversions_v1)
options[:conversions] = options.delete(:conversions_v1)
end
unknown_keywords = options.keys - [:_all, :_type, :batch_size, :callbacks, :callback_options, :case_sensitive, :conversions, :conversions_v2, :deep_paging, :default_fields,
:filterable, :geo_shape, :highlight, :ignore_above, :index_name, :index_prefix, :inheritance, :job_options, :knn, :language,
:locations, :mappings, :match, :max_result_window, :merge_mappings, :routing, :searchable, :search_synonyms, :settings, :similarity,
:special_characters, :stem, :stemmer, :stem_conversions, :stem_exclusion, :stemmer_override, :suggest, :synonyms, :text_end,
:text_middle, :text_start, :unscope, :word, :word_end, :word_middle, :word_start]
raise ArgumentError, "unknown keywords: #{unknown_keywords.join(", ")}" if unknown_keywords.any?
raise "Only call searchkick once per model" if respond_to?(:searchkick_index)
Searchkick.models << self
options[:_type] ||= -> { searchkick_index.klass_document_type(self, true) }
options[:class_name] = model_name.name
callbacks = options.key?(:callbacks) ? options[:callbacks] : :inline
unless [:inline, true, false, :async, :queue].include?(callbacks)
raise ArgumentError, "Invalid value for callbacks"
end
callback_options = (options[:callback_options] || {}).dup
callback_options[:if] = [-> { Searchkick.callbacks?(default: callbacks) }, callback_options[:if]].compact.flatten(1)
base = self
mod = Module.new
include(mod)
mod.module_eval do
def reindex(method_name = nil, mode: nil, refresh: false, ignore_missing: nil, job_options: nil)
self.class.searchkick_index.reindex([self], method_name: method_name, mode: mode, refresh: refresh, ignore_missing: ignore_missing, job_options: job_options, single: true)
end unless base.method_defined?(:reindex)
def similar(**options)
self.class.searchkick_index.similar_record(self, **options)
end unless base.method_defined?(:similar)
def search_data
data = respond_to?(:to_hash) ? to_hash : serializable_hash
data.delete("id")
data.delete("_id")
data.delete("_type")
data
end unless base.method_defined?(:search_data)
def should_index?
true
end unless base.method_defined?(:should_index?)
end
class_eval do
cattr_reader :searchkick_options, :searchkick_klass, instance_reader: false
class_variable_set :@@searchkick_options, options.dup
class_variable_set :@@searchkick_klass, self
class_variable_set :@@searchkick_index_cache, Searchkick::IndexCache.new
class << self
def searchkick_search(term = "*", **options, &block)
if Searchkick.relation?(self)
raise Searchkick::Error, "search must be called on model, not relation"
end
Searchkick.search(term, model: self, **options, &block)
end
alias_method Searchkick.search_method_name, :searchkick_search if Searchkick.search_method_name
def searchkick_index(name: nil)
index_name = name || searchkick_klass.searchkick_index_name
index_name = index_name.call if index_name.respond_to?(:call)
index_cache = class_variable_get(:@@searchkick_index_cache)
index_cache.fetch(index_name) { Searchkick::Index.new(index_name, searchkick_options) }
end
alias_method :search_index, :searchkick_index unless method_defined?(:search_index)
def searchkick_reindex(method_name = nil, **options)
searchkick_index.reindex(self, method_name: method_name, **options)
end
alias_method :reindex, :searchkick_reindex unless method_defined?(:reindex)
def searchkick_index_options
searchkick_index.index_options
end
def searchkick_index_name
@searchkick_index_name ||= begin
options = class_variable_get(:@@searchkick_options)
if options[:index_name]
options[:index_name]
elsif options[:index_prefix].respond_to?(:call)
-> { [options[:index_prefix].call, model_name.plural, Searchkick.env, Searchkick.index_suffix].compact.join("_") }
else
[options.key?(:index_prefix) ? options[:index_prefix] : Searchkick.index_prefix, model_name.plural, Searchkick.env, Searchkick.index_suffix].compact.join("_")
end
end
end
end
# always add callbacks, even when callbacks is false
# so Model.callbacks block can be used
if respond_to?(:after_commit)
after_commit :reindex, **callback_options
elsif respond_to?(:after_save)
after_save :reindex, **callback_options
after_destroy :reindex, **callback_options
end
end
end
end
end
================================================
FILE: lib/searchkick/multi_search.rb
================================================
module Searchkick
class MultiSearch
attr_reader :queries
def initialize(queries, opaque_id: nil)
@queries = queries
@opaque_id = opaque_id
end
def perform
if queries.any?
perform_search(queries)
end
end
private
def perform_search(search_queries, perform_retry: true)
params = {
body: search_queries.flat_map { |q| [q.params.except(:body), q.body] }
}
params[:opaque_id] = @opaque_id if @opaque_id
responses = client.msearch(params)["responses"]
retry_queries = []
search_queries.each_with_index do |query, i|
if perform_retry && query.retry_misspellings?(responses[i])
query.send(:prepare) # okay, since we don't want to expose this method outside Searchkick
retry_queries << query
else
query.handle_response(responses[i])
end
end
if retry_queries.any?
perform_search(retry_queries, perform_retry: false)
end
search_queries
end
def client
Searchkick.client
end
end
end
================================================
FILE: lib/searchkick/process_batch_job.rb
================================================
module Searchkick
class ProcessBatchJob < Searchkick.parent_job.constantize
queue_as { Searchkick.queue_name }
def perform(class_name:, record_ids:, index_name: nil)
model = Searchkick.load_model(class_name)
index = model.searchkick_index(name: index_name)
items =
record_ids.map do |r|
parts = r.split(/(?<!\|)\|(?!\|)/, 2)
.map { |v| v.gsub("||", "|") }
{id: parts[0], routing: parts[1]}
end
relation = Searchkick.scope(model)
RecordIndexer.new(index).reindex_items(relation, items, method_name: nil, ignore_missing: nil)
end
end
end
================================================
FILE: lib/searchkick/process_queue_job.rb
================================================
module Searchkick
class ProcessQueueJob < Searchkick.parent_job.constantize
queue_as { Searchkick.queue_name }
def perform(class_name:, index_name: nil, inline: false, job_options: nil)
model = Searchkick.load_model(class_name)
index = model.searchkick_index(name: index_name)
limit = model.searchkick_options[:batch_size] || 1000
job_options = (model.searchkick_options[:job_options] || {}).merge(job_options || {})
loop do
record_ids = index.reindex_queue.reserve(limit: limit)
if record_ids.any?
batch_options = {
class_name: class_name,
record_ids: record_ids.uniq,
index_name: index_name
}
if inline
# use new.perform to avoid excessive logging
Searchkick::ProcessBatchJob.new.perform(**batch_options)
else
Searchkick::ProcessBatchJob.set(job_options).perform_later(**batch_options)
end
# TODO when moving to reliable queuing, mark as complete
end
break unless record_ids.size == limit
end
end
end
end
================================================
FILE: lib/searchkick/query.rb
================================================
module Searchkick
class Query
include Enumerable
extend Forwardable
@@metric_aggs = [:avg, :cardinality, :max, :min, :sum]
attr_reader :klass, :term, :options
attr_accessor :body
def_delegators :execute, :map, :each, :any?, :empty?, :size, :length, :slice, :[], :to_ary,
:results, :suggestions, :each_with_hit, :with_details, :aggregations, :aggs,
:took, :error, :model_name, :entry_name, :total_count, :total_entries,
:current_page, :per_page, :limit_value, :padding, :total_pages, :num_pages,
:offset_value, :offset, :previous_page, :prev_page, :next_page, :first_page?, :last_page?,
:out_of_range?, :hits, :response, :to_a, :first, :scroll, :highlights, :with_highlights,
:with_score, :misspellings?, :scroll_id, :clear_scroll, :missing_records, :with_hit
def initialize(klass, term = "*", **options)
if options[:conversions]
Searchkick.warn("The `conversions` option is deprecated in favor of `conversions_v2`, which provides much better search performance. Upgrade to `conversions_v2` or rename `conversions` to `conversions_v1`")
end
if options.key?(:conversions_v1)
options[:conversions] = options.delete(:conversions_v1)
end
unknown_keywords = options.keys - [:aggs, :block, :body, :body_options, :boost,
:boost_by, :boost_by_distance, :boost_by_recency, :boost_where, :conversions, :conversions_v2, :conversions_term, :debug, :emoji, :exclude, :explain,
:fields, :highlight, :includes, :index_name, :indices_boost, :knn, :limit, :load,
:match, :misspellings, :models, :model_includes, :offset, :opaque_id, :operator, :order, :padding, :page, :per_page, :profile,
:request_params, :routing, :scope_results, :scroll, :select, :similar, :smart_aggs, :suggest, :total_entries, :track, :type, :where]
raise ArgumentError, "unknown keywords: #{unknown_keywords.join(", ")}" if unknown_keywords.any?
term = term.to_s
if options[:emoji]
term = EmojiParser.parse_unicode(term) { |e| " #{e.name.tr('_', ' ')} " }.strip
end
@klass = klass
@term = term
@options = options
@match_suffix = options[:match] || searchkick_options[:match] || "analyzed"
# prevent Ruby warnings
@type = nil
@routing = nil
@misspellings = false
@misspellings_below = nil
@highlighted_fields = nil
@index_mapping = nil
prepare
end
def searchkick_index
klass ? klass.searchkick_index : nil
end
def searchkick_options
klass ? klass.searchkick_options : {}
end
def searchkick_klass
klass ? klass.searchkick_klass : nil
end
def params
if options[:models]
@index_mapping = {}
Array(options[:models]).each do |model|
# there can be multiple models per index name due to inheritance - see #1259
(@index_mapping[model.searchkick_index.name] ||= []) << model
end
end
index =
if options[:index_name]
Array(options[:index_name]).map { |v| v.respond_to?(:searchkick_index) ? v.searchkick_index.name : v }.join(",")
elsif options[:models]
@index_mapping.keys.join(",")
elsif searchkick_index
searchkick_index.name
else
# fixes warning about accessing system indices
"*,-.*"
end
params = {
index: index,
body: body
}
params[:type] = @type if @type
params[:routing] = @routing if @routing
params[:scroll] = @scroll if @scroll
params[:opaque_id] = @opaque_id if @opaque_id
params.merge!(options[:request_params]) if options[:request_params]
params
end
def execute
@execute ||= begin
begin
response = execute_search
if retry_misspellings?(response)
prepare
response = execute_search
end
rescue => e
handle_error(e)
end
handle_response(response)
end
end
def handle_response(response)
opts = {
page: @page,
per_page: @per_page,
padding: @padding,
load: @load,
includes: options[:includes],
model_includes: options[:model_includes],
json: !@json.nil?,
match_suffix: @match_suffix,
highlight: options[:highlight],
highlighted_fields: @highlighted_fields || [],
misspellings: @misspellings,
term: term,
scope_results: options[:scope_results],
total_entries: options[:total_entries],
index_mapping: @index_mapping,
suggest: options[:suggest],
scroll: options[:scroll],
opaque_id: options[:opaque_id]
}
if options[:debug]
server = Searchkick.opensearch? ? "OpenSearch" : "Elasticsearch"
puts "Searchkick #{Searchkick::VERSION}"
puts "#{server} #{Searchkick.server_version}"
puts
puts "Model Options"
pp searchkick_options
puts
puts "Search Options"
pp options
puts
if searchkick_index
puts "Record Data"
begin
pp klass.limit(3).map { |r| RecordData.new(searchkick_index, r).index_data }
rescue => e
puts "#{e.class.name}: #{e.message}"
end
puts
puts "Mapping"
puts JSON.pretty_generate(searchkick_index.mapping)
puts
puts "Settings"
puts JSON.pretty_generate(searchkick_index.settings)
puts
end
puts "Query"
puts JSON.pretty_generate(params[:body])
puts
puts "Results"
puts JSON.pretty_generate(response.to_h)
end
# set execute for multi search
@execute = Results.new(searchkick_klass, response, opts)
end
def retry_misspellings?(response)
@misspellings_below && response["error"].nil? && Results.new(searchkick_klass, response).total_count < @misspellings_below
end
private
def handle_error(e)
status_code = e.message[1..3].to_i
if status_code == 404
if e.message.include?("No search context found for id")
raise MissingIndexError, "No search context found for id"
else
raise MissingIndexError, "Index missing - run #{reindex_command}"
end
elsif status_code == 500 && (
e.message.include?("IllegalArgumentException[minimumSimilarity >= 1]") ||
e.message.include?("No query registered for [multi_match]") ||
e.message.include?("[match] query does not support [cutoff_frequency]") ||
e.message.include?("No query registered for [function_score]")
)
raise UnsupportedVersionError
elsif status_code == 400
if (
e.message.include?("bool query does not support [filter]") ||
e.message.include?("[bool] filter does not support [filter]")
)
raise UnsupportedVersionError
elsif e.message.match?(/analyzer \[searchkick_.+\] not found/)
raise InvalidQueryError, "Bad mapping - run #{reindex_command}"
else
raise InvalidQueryError, e.message
end
else
raise e
end
end
def reindex_command
searchkick_klass ? "#{searchkick_klass.name}.reindex" : "reindex"
end
def execute_search
name = searchkick_klass ? "#{searchkick_klass.name} Search" : "Search"
event = {
name: name,
query: params
}
ActiveSupport::Notifications.instrument("search.searchkick", event) do
Searchkick.client.search(params)
end
end
def prepare
boost_fields, fields = set_fields
operator = options[:operator] || "and"
# pagination
page = [options[:page].to_i, 1].max
# maybe use index.max_result_window in the future
default_limit = searchkick_options[:deep_paging] ? 1_000_000_000 : 10_000
per_page = (options[:limit] || options[:per_page] || default_limit).to_i
padding = [options[:padding].to_i, 0].max
offset = (options[:offset] || (page - 1) * per_page + padding).to_i
scroll = options[:scroll]
opaque_id = options[:opaque_id]
max_result_window = searchkick_options[:max_result_window]
original_per_page = per_page
if max_result_window
offset = max_result_window if offset > max_result_window
per_page = max_result_window - offset if offset + per_page > max_result_window
end
# model and eager loading
load = options[:load].nil? ? true : options[:load]
all = term == "*"
@json = options[:body]
if @json
ignored_options = options.keys & [:aggs, :boost,
:boost_by, :boost_by_distance, :boost_by_recency, :boost_where, :conversions, :conversions_term, :exclude, :explain,
:fields, :highlight, :indices_boost, :match, :misspellings, :operator, :order,
:profile, :select, :smart_aggs, :suggest, :where]
raise ArgumentError, "Options incompatible with body option: #{ignored_options.join(", ")}" if ignored_options.any?
payload = @json
else
must_not = []
should = []
if options[:similar]
like = options[:similar] == true ? term : options[:similar]
query = {
more_like_this: {
like: like,
min_doc_freq: 1,
min_term_freq: 1,
analyzer: "searchkick_search2"
}
}
if fields.all? { |f| f.start_with?("*.") }
raise ArgumentError, "Must specify fields to search"
end
if fields != ["_all"]
query[:more_like_this][:fields] = fields
end
elsif all && !options[:exclude]
query = {
match_all: {}
}
else
queries = []
misspellings =
if options.key?(:misspellings)
options[:misspellings]
else
true
end
if misspellings.is_a?(Hash) && misspellings[:below] && !@misspellings_below
@misspellings_below = misspellings[:below].to_i
misspellings = false
end
if misspellings != false
edit_distance = (misspellings.is_a?(Hash) && (misspellings[:edit_distance] || misspellings[:distance])) || 1
transpositions =
if misspellings.is_a?(Hash) && misspellings.key?(:transpositions)
{fuzzy_transpositions: misspellings[:transpositions]}
else
{fuzzy_transpositions: true}
end
prefix_length = (misspellings.is_a?(Hash) && misspellings[:prefix_length]) || 0
default_max_expansions = @misspellings_below ? 20 : 3
max_expansions = (misspellings.is_a?(Hash) && misspellings[:max_expansions]) || default_max_expansions
misspellings_fields = misspellings.is_a?(Hash) && misspellings.key?(:fields) && misspellings[:fields].map(&:to_s)
if misspellings_fields
missing_fields = misspellings_fields - fields.map { |f| base_field(f) }
if missing_fields.any?
raise ArgumentError, "All fields in per-field misspellings must also be specified in fields option"
end
end
@misspellings = true
else
@misspellings = false
end
fields.each do |field|
queries_to_add = []
qs = []
factor = boost_fields[field] || 1
shared_options = {
query: term,
boost: 10 * factor
}
match_type =
if field.end_with?(".phrase")
field =
if field == "_all.phrase"
"_all"
else
field.sub(/\.phrase\z/, ".analyzed")
end
:match_phrase
else
:match
end
shared_options[:operator] = operator if match_type == :match
exclude_analyzer = nil
exclude_field = field
field_misspellings = misspellings && (!misspellings_fields || misspellings_fields.include?(base_field(field)))
if field == "_all" || field.end_with?(".analyzed")
qs << shared_options.merge(analyzer: "searchkick_search")
# searchkick_search and searchkick_search2 are the same for some languages
unless %w(japanese japanese2 korean polish ukrainian vietnamese).include?(searchkick_options[:language])
qs << shared_options.merge(analyzer: "searchkick_search2")
end
exclude_analyzer = "searchkick_search2"
elsif field.end_with?(".exact")
f = field.split(".")[0..-2].join(".")
queries_to_add << {match: {f => shared_options.merge(analyzer: "keyword")}}
exclude_field = f
exclude_analyzer = "keyword"
else
analyzer = field.match?(/\.word_(start|middle|end)\z/) ? "searchkick_word_search" : "searchkick_autocomplete_search"
qs << shared_options.merge(analyzer: analyzer)
exclude_analyzer = analyzer
end
if field_misspellings != false && match_type == :match
qs.concat(qs.map { |q| q.except(:cutoff_frequency).merge(fuzziness: edit_distance, prefix_length: prefix_length, max_expansions: max_expansions, boost: factor).merge(transpositions) })
end
if field.start_with?("*.")
q2 = qs.map { |q| {multi_match: q.merge(fields: [field], type: match_type == :match_phrase ? "phrase" : "best_fields")} }
else
q2 = qs.map { |q| {match_type => {field => q}} }
end
# boost exact matches more
if field =~ /\.word_(start|middle|end)\z/ && searchkick_options[:word] != false
queries_to_add << {
bool: {
must: {
bool: {
should: q2
}
},
should: {match_type => {field.sub(/\.word_(start|middle|end)\z/, ".analyzed") => qs.first}}
}
}
else
queries_to_add.concat(q2)
end
queries << queries_to_add
if options[:exclude]
must_not.concat(set_exclude(exclude_field, exclude_analyzer))
end
end
# all + exclude option
if all
query = {
match_all: {}
}
should = []
else
# higher score for matching more fields
payload = {
bool: {
should: queries.map { |qs| {dis_max: {queries: qs}} }
}
}
should.concat(set_conversions)
should.concat(set_conversions_v2)
end
query = payload
end
payload = {}
# type when inheritance
where = ensure_permitted(options[:where] || {}).dup
if searchkick_options[:inheritance] && (options[:type] || (klass != searchkick_klass && searchkick_index))
where[:type] = [options[:type] || klass].flatten.map { |v| searchkick_index.klass_document_type(v, true) }
end
models = Array(options[:models])
if models.any? { |m| m != m.searchkick_klass }
index_type_or =
models.map do |m|
v = {_index: m.searchkick_index.name}
v[:type] = m.searchkick_index.klass_document_type(m, true) if m != m.searchkick_klass
v
end
where[:or] = Array(where[:or]) + [index_type_or]
end
# start everything as efficient filters
# move to post_filters as aggs demand
filters = where_filters(where)
post_filters = []
# aggregations
set_aggregations(payload, filters, post_filters) if options[:aggs]
# post filters
set_post_filters(payload, post_filters) if post_filters.any?
custom_filters = []
multiply_filters = []
set_boost_by(multiply_filters, custom_filters)
set_boost_where(custom_filters)
set_boost_by_distance(custom_filters) if options[:boost_by_distance]
set_boost_by_recency(custom_filters) if options[:boost_by_recency]
payload[:query] = build_query(query, filters, should, must_not, custom_filters, multiply_filters)
payload[:explain] = options[:explain] if options[:explain]
payload[:profile] = options[:profile] if options[:profile]
# order
set_order(payload) if options[:order]
# indices_boost
set_boost_by_indices(payload)
# suggestions
set_suggestions(payload, options[:suggest]) if options[:suggest]
# highlight
set_highlights(payload, fields) if options[:highlight]
# timeout shortly after client times out
payload[:timeout] ||= "#{((Searchkick.search_timeout + 1) * 1000).round}ms"
# An empty array will cause only the _id and _type for each hit to be returned
# https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-source-filtering.html
if options[:select]
if options[:select] == []
# intuitively [] makes sense to return no fields, but ES by default returns all fields
payload[:_source] = false
else
payload[:_source] = options[:select]
end
elsif load
payload[:_source] = false
end
end
# knn
set_knn(payload, options[:knn], per_page, offset) if options[:knn]
# pagination
pagination_options = options[:page] || options[:limit] || options[:per_page] || options[:offset] || options[:padding]
if !options[:body] || pagination_options
payload[:size] = per_page
payload[:from] = offset if offset > 0
end
# type
if !searchkick_options[:inheritance] && (options[:type] || (klass != searchkick_klass && searchkick_index))
@type = [options[:type] || klass].flatten.map { |v| searchkick_index.klass_document_type(v) }
end
# routing
@routing = options[:routing] if options[:routing]
if track_total_hits?
payload[:track_total_hits] = true
end
# merge more body options
payload = payload.deep_merge(options[:body_options]) if options[:body_options]
# run block
options[:block].call(payload) if options[:block]
# scroll optimization when iterating over all docs
# https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-scroll.html
if options[:scroll] && payload[:query] == {match_all: {}}
payload[:sort] ||= ["_doc"]
end
@body = payload
@page = page
@per_page = original_per_page
@padding = padding
@load = load
@scroll = scroll
@opaque_id = opaque_id
end
def set_fields
boost_fields = {}
fields = options[:fields] || searchkick_options[:default_fields] || searchkick_options[:searchable]
all = searchkick_options.key?(:_all) ? searchkick_options[:_all] : false
default_match = options[:match] || searchkick_options[:match] || :word
fields =
if fields
fields.map do |value|
k, v = value.is_a?(Hash) ? value.to_a.first : [value, default_match]
k2, boost = k.to_s.split("^", 2)
field = "#{k2}.#{v == :word ? 'analyzed' : v}"
boost_fields[field] = boost.to_f if boost
field
end
elsif all && default_match == :word
["_all"]
elsif all && default_match == :phrase
["_all.phrase"]
elsif term != "*" && default_match == :exact
raise ArgumentError, "Must specify fields to search"
else
[default_match == :word ? "*.analyzed" : "*.#{default_match}"]
end
[boost_fields, fields]
end
def build_query(query, filters, should, must_not, custom_filters, multiply_filters)
if filters.any? || must_not.any? || should.any?
bool = {}
bool[:must] = query if query
bool[:filter] = filters if filters.any? # where
bool[:must_not] = must_not if must_not.any? # exclude
bool[:should] = should if should.any? # conversions
query = {bool: bool}
end
if custom_filters.any?
query = {
function_score: {
functions: custom_filters,
query: query,
score_mode: "sum"
}
}
end
if multiply_filters.any?
query = {
function_score: {
functions: multiply_filters,
query: query,
score_mode: "multiply"
}
}
end
query
end
def set_conversions
conversions_fields = Array(options[:conversions] || searchkick_options[:conversions]).map(&:to_s)
if conversions_fields.present? && options[:conversions] != false
conversions_fields.map do |conversions_field|
{
nested: {
path: conversions_field,
score_mode: "sum",
query: {
function_score: {
boost_mode: "replace",
query: {
match: {
"#{conversions_field}.query" => options[:conversions_term] || term
}
},
field_value_factor: {
field: "#{conversions_field}.count"
}
}
}
}
}
end
else
[]
end
end
def set_conversions_v2
conversions_v2 = options[:conversions_v2]
return [] if conversions_v2.nil? && !searchkick_options[:conversions_v2]
return [] if conversions_v2 == false
# disable if searchkick_options[:conversions] to make it easy to upgrade without downtime
return [] if conversions_v2.nil? && searchkick_options[:conversions]
unless conversions_v2.is_a?(Hash)
conversions_v2 = {field: conversions_v2}
end
conversions_fields =
case conversions_v2[:field]
when true, nil
Array(searchkick_options[:conversions_v2]).map(&:to_s)
else
[conversions_v2[:field].to_s]
end
conversions_term = (conversions_v2[:term] || options[:conversions_term] || term).to_s
unless searchkick_options[:case_sensitive]
conversions_term = conversions_term.downcase
end
conversions_term = conversions_term.gsub(".", "*")
conversions_fields.map do |conversions_field|
{
rank_feature: {
field: "#{conversions_field}.#{conversions_term}",
linear: {},
boost: conversions_v2[:factor] || 1
}
}
end
end
def set_exclude(field, analyzer)
Array(options[:exclude]).map do |phrase|
{
multi_match: {
fields: [field],
query: phrase,
analyzer: analyzer,
type: "phrase"
}
}
end
end
def set_boost_by_distance(custom_filters)
boost_by_distance = options[:boost_by_distance] || {}
# legacy format
if boost_by_distance[:field]
boost_by_distance = {boost_by_distance[:field] => boost_by_distance.except(:field)}
end
boost_by_distance.each do |field, attributes|
attributes = {function: :gauss, scale: "5mi"}.merge(attributes)
unless attributes[:origin]
raise ArgumentError, "boost_by_distance requires :origin"
end
function_params = attributes.except(:factor, :function)
function_params[:origin] = location_value(function_params[:origin])
custom_filters << {
weight: attributes[:factor] || 1,
attributes[:function] => {
field => function_params
}
}
end
end
def set_boost_by_recency(custom_filters)
options[:boost_by_recency].each do |field, attributes|
attributes = {function: :gauss, origin: Time.now}.merge(attributes)
custom_filters << {
weight: attributes[:factor] || 1,
attributes[:function] => {
field => attributes.except(:factor, :function)
}
}
end
end
def set_boost_by(multiply_filters, custom_filters)
boost_by = options[:boost_by] || {}
if boost_by.is_a?(Array)
boost_by = boost_by.to_h { |f| [f, {factor: 1}] }
elsif boost_by.is_a?(Hash)
multiply_by, boost_by = boost_by.transform_values(&:dup).partition { |_, v| v.delete(:boost_mode) == "multiply" }.map(&:to_h)
end
boost_by[options[:boost]] = {factor: 1} if options[:boost]
custom_filters.concat boost_filters(boost_by, modifier: "ln2p")
multiply_filters.concat boost_filters(multiply_by || {})
end
def set_boost_where(custom_filters)
boost_where = options[:boost_where] || {}
boost_where.each do |field, value|
if value.is_a?(Array) && value.first.is_a?(Hash)
value.each do |value_factor|
custom_filters << custom_filter(field, value_factor[:value], value_factor[:factor])
end
elsif value.is_a?(Hash)
custom_filters << custom_filter(field, value[:value], value[:factor])
else
factor = 1000
custom_filters << custom_filter(field, value, factor)
end
end
end
def set_boost_by_indices(payload)
return unless options[:indices_boost]
indices_boost = options[:indices_boost].map do |key, boost|
index = key.respond_to?(:searchkick_index) ? key.searchkick_index.name : key
{index => boost}
end
payload[:indices_boost] = indices_boost
end
def set_suggestions(payload, suggest)
suggest_fields = nil
if suggest.is_a?(Array)
suggest_fields = suggest
else
suggest_fields = (searchkick_options[:suggest] || []).map(&:to_s)
# intersection
if options[:fields]
suggest_fields &= options[:fields].map { |v| (v.is_a?(Hash) ? v.keys.first : v).to_s.split("^", 2).first }
end
end
if suggest_fields.any?
payload[:suggest] = {text: term}
suggest_fields.each do |field|
payload[:suggest][field] = {
phrase: {
field: "#{field}.suggest"
}
}
end
else
raise ArgumentError, "Must pass fields to suggest option"
end
end
def set_highlights(payload, fields)
payload[:highlight] = {
fields: fields.to_h { |f| [f, {}] },
fragment_size: 0
}
if options[:highlight].is_a?(Hash)
if (tag = options[:highlight][:tag])
payload[:highlight][:pre_tags] = [tag]
payload[:highlight][:post_tags] = [tag.to_s.gsub(/\A<(\w+).+/, "</\\1>")]
end
if (fragment_size = options[:highlight][:fragment_size])
payload[:highlight][:fragment_size] = fragment_size
end
if (encoder = options[:highlight][:encoder])
payload[:highlight][:encoder] = encoder
end
highlight_fields = options[:highlight][:fields]
if highlight_fields
payload[:highlight][:fields] = {}
highlight_fields.each do |name, opts|
payload[:highlight][:fields]["#{name}.#{@match_suffix}"] = opts || {}
end
end
end
@highlighted_fields = payload[:highlight][:fields].keys
end
def set_aggregations(payload, filters, post_filters)
aggs = options[:aggs]
payload[:aggs] = {}
aggs = aggs.to_h { |f| [f, {}] } if aggs.is_a?(Array) # convert to more advanced syntax
aggs.each do |field, agg_options|
size = agg_options[:limit] ? agg_options[:limit] : 1_000
shared_agg_options = agg_options.except(:limit, :field, :ranges, :date_ranges, :where)
if agg_options[:ranges]
payload[:aggs][field] = {
range: {
field: agg_options[:field] || field,
ranges: agg_options[:ranges]
}.merge(shared_agg_options)
}
elsif agg_options[:date_ranges]
payload[:aggs][field] = {
date_range: {
field: agg_options[:field] || field,
ranges: agg_options[:date_ranges]
}.merge(shared_agg_options)
}
elsif (histogram = agg_options[:date_histogram])
payload[:aggs][field] = {
date_histogram: histogram
}.merge(shared_agg_options)
elsif (metric = @@metric_aggs.find { |k| agg_options.has_key?(k) })
payload[:aggs][field] = {
metric => {
field: agg_options[metric][:field] || field
}
}.merge(shared_agg_options)
else
payload[:aggs][field] = {
terms: {
field: agg_options[:field] || field,
size: size
}.merge(shared_agg_options)
}
end
agg_where = ensure_permitted(agg_options[:where] || {})
if options[:smart_aggs] != false && options[:where]
where = ensure_permitted(options[:where])
where_without_field = where.reject { |k| k == field }
# where_without_field = where_without_field(where, field.to_s)
if where_without_field.any?
if agg_where.any?
agg_where = where.merge(agg_where)
# agg_where = combine_agg_where(agg_where, where_without_field)
else
agg_where = where_without_field
end
end
end
agg_filters = where_filters(agg_where)
# only do one level comparison for simplicity
filters.select! do |filter|
if agg_filters.include?(filter)
true
else
post_filters << filter
false
end
end
if agg_filters.any?
payload[:aggs][field] = {
filter: {
bool: {
must: agg_filters
}
},
aggs: {
field => payload[:aggs][field]
}
}
end
end
end
def where_without_field(where, field)
result = {}
where.each do |f, v|
case f
when :_and
r = v.map { |v2| where_without_field(v2, field) }.reject(&:empty?)
result[f] = r unless r.empty?
when :_or
r = v.map { |v2| where_without_field(v2, field) }
result[f] = r unless r.any?(&:empty?)
when :or
r = v.map { |v2| v2.map { |v3| where_without_field(v3, field) }.reject { |v2| v2.any?(&:empty?) } }
result[f] = r unless r.empty?
when :_not
r = where_without_field(v, field)
result[f] = r unless r.empty?
when :_script
result[f] = v
else
if f.to_s != field
result[f] = v
end
end
end
result
end
def combine_agg_where(agg_where, where)
result = agg_where.dup
field_keys = result.except(:_and, :_or, :or, :_not, :_script).transform_keys(&:to_s)
where.each do |f, v|
case f
when :_and, :_or, :or, :_not, :_script
if result.key?(f)
# combine with _and if needed
result[:_and] ||= []
result[:_and] += [{f => v}]
else
result[f] = v
end
else
result[f] = v unless field_keys.include?(f.to_s)
end
end
result
end
def set_knn(payload, knn, per_page, offset)
if term != "*"
raise ArgumentError, "Use Searchkick.multi_search for hybrid search"
end
field = knn[:field]
field_options = searchkick_options.dig(:knn, field.to_sym) || searchkick_options.dig(:knn, field.to_s) || {}
vector = knn[:vector]
distance = knn[:distance] || field_options[:distance]
exact = knn[:exact]
exact = field_options[:distance].nil? || distance != field_options[:distance] if exact.nil?
k = per_page + offset
ef_search = knn[:ef_search]
filter = payload.delete(:query)
if distance.nil?
raise ArgumentError, "distance required"
elsif !exact && distance != field_options[:distance]
raise ArgumentError, "distance must match searchkick options for approximate search"
end
if Searchkick.opensearch?
if exact
# https://opensearch.org/docs/latest/search-plugins/knn/knn-score-script/#spaces
space_type =
case distance
when "cosine"
"cosinesimil"
when "euclidean"
"l2"
when "taxicab"
"l1"
when "inner_product"
"innerproduct"
when "chebyshev"
"linf"
else
raise ArgumentError, "Unknown distance: #{distance}"
end
payload[:query] = {
script_score: {
query: {
bool: {
must: [filter, {exists: {field: field}}]
}
},
script: {
source: "knn_score",
lang: "knn",
params: {
field: field,
query_value: vector,
space_type: space_type
}
},
boost: distance == "cosine" && Searchkick.server_below?("2.19.0") ? 0.5 : 1.0
}
}
else
if ef_search && Searchkick.server_below?("2.16.0")
raise Error, "ef_search requires OpenSearch 2.16+"
end
payload[:query] = {
knn: {
field.to_sym => {
vector: vector,
k: k,
filter: filter
}.merge(ef_search ? {method_parameters: {ef_search: ef_search}} : {})
}
}
end
else
if exact
# prevent incorrect distances/results with Elasticsearch 9.0.0-rc1
if !Searchkick.server_below?("9.0.0") && field_options[:distance] == "cosine" && distance != "cosine"
raise ArgumentError, "distance must match searchkick options"
end
# https://github.com/elastic/elasticsearch/blob/main/docs/reference/vectors/vector-functions.asciidoc
source =
case distance
when "cosine"
"(cosineSimilarity(params.query_vector, params.field) + 1.0) * 0.5"
when "euclidean"
"double l2 = l2norm(params.query_vector, params.field); 1 / (1 + l2 * l2)"
when "taxicab"
"1 / (1 + l1norm(params.query_vector, params.field))"
when "inner_product"
"double dot = dotProduct(params.query_vector, params.field); dot > 0 ? dot + 1 : 1 / (1 - dot)"
else
raise ArgumentError, "Unknown distance: #{distance}"
end
payload[:query] = {
script_score: {
query: {
bool: {
must: [filter, {exists: {field: field}}]
}
},
script: {
source: source,
params: {
field: field,
query_vector: vector
}
}
}
}
else
payload[:knn] = {
field: field,
query_vector: vector,
k: k,
filter: filter
}.merge(ef_search ? {num_candidates: ef_search} : {})
end
end
end
def set_post_filters(payload, post_filters)
payload[:post_filter] = {
bool: {
filter: post_filters
}
}
end
def set_order(payload)
value = options[:order]
payload[:sort] = value.is_a?(Enumerable) ? value : {value => :asc}
end
# provides *very* basic protection from unfiltered parameters
# this is not meant to be comprehensive and may be expanded in the future
def ensure_permitted(obj)
obj.to_h
end
def where_filters(where)
filters = []
(where || {}).each do |field, value|
field = :_id if field.to_s == "id"
# update smart aggs when adding new symbol
if field == :or
value.each do |or_clause|
filters << {bool: {should: or_clause.map { |or_statement| {bool: {filter: where_filters(or_statement)}} }}}
end
elsif field == :_or
filters << {bool: {should: value.map { |or_statement| {bool: {filter: where_filters(or_statement)}} }}}
elsif field == :_not
filters << {bool: {must_not: where_filters(value)}}
elsif field == :_and
filters << {bool: {must: value.map { |or_statement| {bool: {filter: where_filters(or_statement)}} }}}
elsif field == :_script
unless value.is_a?(Script)
raise TypeError, "expected Searchkick::Script"
end
filters << {script: {script: {source: value.source, lang: value.lang, params: value.params}}}
else
# expand ranges
if value.is_a?(Range)
value = expand_range(value)
end
value = {in: value} if value.is_a?(Array)
if value.is_a?(Hash)
value.each do |op, op_value|
case op
when :within, :bottom_right, :bottom_left
# do nothing
when :near
filters << {
geo_distance: {
field => location_value(op_value),
distance: value[:within] || "50mi"
}
}
when :geo_polygon
filters << {
geo_polygon: {
field => op_value
}
}
when :geo_shape
shape = op_value.except(:relation)
shape[:coordinates] = coordinate_array(shape[:coordinates]) if shape[:coordinates]
filters << {
geo_shape: {
field => {
relation: op_value[:relation] || "intersects",
shape: shape
}
}
}
when :top_left
filters << {
geo_bounding_box: {
field => {
top_left: location_value(op_value),
bottom_right: location_value(value[:bottom_right])
}
}
}
when :top_right
filters << {
geo_bounding_box: {
field => {
top_right: location_value(op_value),
bottom_left: location_value(value[:bottom_left])
}
}
}
when :like, :ilike
# based on Postgres
# https://www.postgresql.org/docs/current/functions-matching.html
# % matches zero or more characters
# _ matches one character
# \ is escape character
# escape Lucene reserved characters
# https://www.elastic.co/guide/en/elasticsearch/reference/current/regexp-syntax.html#regexp-optional-operators
reserved = %w(\\ . ? + * | { } [ ] ( ) ")
regex = op_value.dup
reserved.each do |v|
regex.gsub!(v, "\\\\" + v)
end
regex = regex.gsub(/(?<!\\)%/, ".*").gsub(/(?<!\\)_/, ".").gsub("\\%", "%").gsub("\\_", "_")
if op == :ilike
filters << {regexp: {field => {value: regex, flags: "NONE", case_insensitive: true}}}
else
filters << {regexp: {field => {value: regex, flags: "NONE"}}}
end
when :prefix
filters << {prefix: {field => {value: op_value}}}
when :regexp # support for regexp queries without using a regexp ruby object
filters << {regexp: {field => {value: op_value}}}
when :not, :_not # not equal
filters << {bool: {must_not: term_filters(field, op_value)}}
when :all
op_value.each do |val|
filters << term_filters(field, val)
end
when :in
filters << term_filters(field, op_value)
when :exists
case op_value
when true
filters << {exists: {field: field}}
when false
filters << {bool: {must_not: {exists: {field: field}}}}
else
raise ArgumentError, "Passing a value other than true or false to exists is not supported"
end
else
range_query =
case op
when :gt
{gt: op_value}
when :gte
{gte: op_value}
when :lt
{lt: op_value}
when :lte
{lte: op_value}
else
raise ArgumentError, "Unknown where operator: #{op.inspect}"
end
# issue 132
if (existing = filters.find { |f| f[:range] && f[:range][field] })
existing[:range][field].merge!(range_query)
else
filters << {range: {field => range_query}}
end
end
end
else
filters << term_filters(field, value)
end
end
end
filters
end
def term_filters(field, value)
if value.is_a?(Array) # in query
if value.any?(&:nil?)
{bool: {should: [term_filters(field, nil), term_filters(field, value.compact)]}}
else
{terms: {field => value}}
end
elsif value.nil?
{bool: {must_not: {exists: {field: field}}}}
elsif value.is_a?(Regexp)
source = value.source
# TODO handle other regexp options
# TODO handle other anchor characters, like ^, $, \Z
if source.start_with?("\\A")
source = source[2..-1]
else
source = ".*#{source}"
end
if source.end_with?("\\z")
source = source[0..-3]
else
source = "#{source}.*"
end
{regexp: {field => {value: source, flags: "NONE", case_insensitive: value.casefold?}}}
else
# TODO add this for other values
if value.as_json.is_a?(Enumerable)
# query will fail, but this is better
# same message as Active Record
raise TypeError, "can't cast #{value.class.name}"
end
{term: {field => {value: value}}}
end
end
def custom_filter(field, value, factor)
{
filter: where_filters(field => value),
weight: factor
}
end
def boost_filter(field, factor: 1, modifier: nil, missing: nil)
script_score = {
field_value_factor: {
field: field,
factor: factor.to_f,
modifier: modifier
}
}
if missing
script_score[:field_value_factor][:missing] = missing.to_f
else
script_score[:filter] = {
exists: {
field: field
}
}
end
script_score
end
def boost_filters(boost_by, modifier: nil)
boost_by.map do |field, value|
boost_filter(field, modifier: modifier, **value)
end
end
# Recursively descend through nesting of arrays until we reach either a lat/lon object or an array of numbers,
# eventually returning the same structure with all values transformed to [lon, lat].
#
def coordinate_array(value)
if value.is_a?(Hash)
[value[:lon], value[:lat]]
elsif value.is_a?(Array) and !value[0].is_a?(Numeric)
value.map { |a| coordinate_array(a) }
else
value
end
end
def location_value(value)
if value.is_a?(Array)
value.map(&:to_f).reverse
else
value
end
end
def expand_range(range)
expanded = {}
expanded[:gte] = range.begin if range.begin
if range.end && !(range.end.respond_to?(:infinite?) && range.end.infinite?)
expanded[range.exclude_end? ? :lt : :lte] = range.end
end
expanded
end
def base_field(k)
k.sub(/\.(analyzed|word_start|word_middle|word_end|text_start|text_middle|text_end|exact)\z/, "")
end
def track_total_hits?
searchkick_options[:deep_paging] || body_options[:track_total_hits]
end
def body_options
options[:body_options] || {}
end
end
end
================================================
FILE: lib/searchkick/railtie.rb
================================================
module Searchkick
class Railtie < Rails::Railtie
rake_tasks do
load "tasks/searchkick.rake"
end
end
end
================================================
FILE: lib/searchkick/record_data.rb
================================================
module Searchkick
class RecordData
TYPE_KEYS = ["type", :type]
attr_reader :index, :record
def initialize(index, record)
@index = index
@record = record
end
def index_data
data = record_data
data[:data] = search_data
{index: data}
end
def update_data(method_name)
data = record_data
data[:data] = {doc: search_data(method_name)}
{update: data}
end
def delete_data
{delete: record_data}
end
# custom id can be useful for load: false
def search_id
id = record.respond_to?(:search_document_id) ? record.search_document_id : record.id
id.is_a?(Numeric) ? id : id.to_s
end
def document_type(ignore_type = false)
index.klass_document_type(record.class, ignore_type)
end
def record_data
data = {
_index: index.name,
_id: search_id
}
data[:routing] = record.search_routing if record.respond_to?(:search_routing)
data
end
private
def search_data(method_name = nil)
partial_reindex = !method_name.nil?
source = record.send(method_name || :search_data)
# conversions
index.conversions_fields.each do |conversions_field|
if source[conversions_field]
source[conversions_field] = source[conversions_field].map { |k, v| {query: k, count: v} }
end
end
index.conversions_v2_fields.each do |conversions_field|
key = source.key?(conversions_field) ? conversions_field : conversions_field.to_sym
if !partial_reindex || source[key]
if index.options[:case_sensitive]
source[key] =
(source[key] || {}).reduce(Hash.new(0)) do |memo, (k, v)|
memo[k.to_s.gsub(".", "*")] += v
memo
end
else
source[key] =
(source[key] || {}).reduce(Hash.new(0)) do |memo, (k, v)|
memo[k.to_s.downcase.gsub(".", "*")] += v
memo
end
end
end
end
# hack to prevent generator field doesn't exist error
if !partial_reindex
index.suggest_fields.each do |field|
if !source.key?(field) && !source.key?(field.to_sym)
source[field] = nil
end
end
end
# locations
index.locations_fields.each do |field|
if source[field]
if !source[field].is_a?(Hash) && (source[field].first.is_a?(Array) || source[field].first.is_a?(Hash))
# multiple locations
source[field] = source[field].map { |a| location_value(a) }
else
source[field] = location_value(source[field])
end
end
end
if index.options[:inheritance]
if !TYPE_KEYS.any? { |tk| source.key?(tk) }
source[:type] = document_type(true)
gitextract_mcchxu51/
├── .github/
│ ├── ISSUE_TEMPLATE/
│ │ ├── bug_report.md
│ │ ├── config.yml
│ │ └── feature_request.md
│ ├── pull_request_template.md
│ └── workflows/
│ └── build.yml
├── .gitignore
├── CHANGELOG.md
├── Gemfile
├── LICENSE.txt
├── README.md
├── Rakefile
├── benchmark/
│ ├── Gemfile
│ ├── index.rb
│ ├── relation.rb
│ └── search.rb
├── examples/
│ ├── Gemfile
│ ├── hybrid.rb
│ └── semantic.rb
├── gemfiles/
│ ├── activerecord72.gemfile
│ ├── activerecord80.gemfile
│ ├── mongoid8.gemfile
│ ├── mongoid9.gemfile
│ ├── opensearch2.gemfile
│ └── opensearch3.gemfile
├── lib/
│ ├── searchkick/
│ │ ├── bulk_reindex_job.rb
│ │ ├── controller_runtime.rb
│ │ ├── hash_wrapper.rb
│ │ ├── index.rb
│ │ ├── index_cache.rb
│ │ ├── index_options.rb
│ │ ├── indexer.rb
│ │ ├── log_subscriber.rb
│ │ ├── middleware.rb
│ │ ├── model.rb
│ │ ├── multi_search.rb
│ │ ├── process_batch_job.rb
│ │ ├── process_queue_job.rb
│ │ ├── query.rb
│ │ ├── railtie.rb
│ │ ├── record_data.rb
│ │ ├── record_indexer.rb
│ │ ├── reindex_queue.rb
│ │ ├── reindex_v2_job.rb
│ │ ├── relation.rb
│ │ ├── relation_indexer.rb
│ │ ├── reranking.rb
│ │ ├── results.rb
│ │ ├── script.rb
│ │ ├── version.rb
│ │ └── where.rb
│ ├── searchkick.rb
│ └── tasks/
│ └── searchkick.rake
├── searchkick.gemspec
└── test/
├── aggs_test.rb
├── boost_test.rb
├── callbacks_test.rb
├── conversions_test.rb
├── default_scope_test.rb
├── exclude_test.rb
├── geo_shape_test.rb
├── highlight_test.rb
├── hybrid_test.rb
├── index_cache_test.rb
├── index_options_test.rb
├── index_test.rb
├── inheritance_test.rb
├── knn_test.rb
├── language_test.rb
├── load_test.rb
├── log_subscriber_test.rb
├── marshal_test.rb
├── match_test.rb
├── misspellings_test.rb
├── models/
│ ├── animal.rb
│ ├── artist.rb
│ ├── band.rb
│ ├── product.rb
│ ├── region.rb
│ ├── sku.rb
│ ├── song.rb
│ ├── speaker.rb
│ └── store.rb
├── multi_indices_test.rb
├── multi_search_test.rb
├── multi_tenancy_test.rb
├── notifications_test.rb
├── order_test.rb
├── pagination_test.rb
├── parameters_test.rb
├── partial_match_test.rb
├── partial_reindex_test.rb
├── query_test.rb
├── reindex_test.rb
├── reindex_v2_job_test.rb
├── relation_test.rb
├── results_test.rb
├── routing_test.rb
├── scroll_test.rb
├── search_synonyms_test.rb
├── search_test.rb
├── select_test.rb
├── should_index_test.rb
├── similar_test.rb
├── suggest_test.rb
├── support/
│ ├── activerecord.rb
│ ├── apartment.rb
│ ├── helpers.rb
│ ├── kaminari.yml
│ ├── mongoid.rb
│ └── redis.rb
├── synonyms_test.rb
├── test_helper.rb
├── unscope_test.rb
└── where_test.rb
SYMBOL INDEX (1131 symbols across 91 files)
FILE: benchmark/index.rb
class SearchSerializer (line 15) | class SearchSerializer
method dump (line 16) | def dump(object)
class Product (line 34) | class Product < ActiveRecord::Base
method search_data (line 37) | def search_data
FILE: benchmark/relation.rb
class Product (line 5) | class Product < ActiveRecord::Base
FILE: benchmark/search.rb
class Product (line 10) | class Product < ActiveRecord::Base
method search_data (line 13) | def search_data
FILE: examples/hybrid.rb
class Product (line 16) | class Product < ActiveRecord::Base
FILE: examples/semantic.rb
class Product (line 16) | class Product < ActiveRecord::Base
FILE: lib/searchkick.rb
type Searchkick (line 38) | module Searchkick
class Error (line 49) | class Error < StandardError; end
class MissingIndexError (line 50) | class MissingIndexError < Error; end
class UnsupportedVersionError (line 51) | class UnsupportedVersionError < Error
method message (line 52) | def message
class InvalidQueryError (line 56) | class InvalidQueryError < Error; end
class DangerousOperation (line 57) | class DangerousOperation < Error; end
class ImportError (line 58) | class ImportError < Error; end
function client (line 73) | def self.client
function env (line 112) | def self.env
function search_timeout (line 116) | def self.search_timeout
function server_info (line 121) | def self.server_info
function server_version (line 125) | def self.server_version
function opensearch? (line 129) | def self.opensearch?
function server_below? (line 136) | def self.server_below?(version)
function knn_support? (line 141) | def self.knn_support?
function search (line 149) | def self.search(term = "*", model: nil, **options, &block)
function multi_search (line 178) | def self.multi_search(queries, opaque_id: nil)
function script (line 194) | def self.script(source, **options)
function enable_callbacks (line 200) | def self.enable_callbacks
function disable_callbacks (line 204) | def self.disable_callbacks
function callbacks? (line 208) | def self.callbacks?(default: true)
function callbacks (line 217) | def self.callbacks(value = nil, message: nil)
function aws_credentials= (line 244) | def self.aws_credentials=(creds)
function reindex_status (line 251) | def self.reindex_status(index_name)
function with_redis (line 261) | def self.with_redis
function warn (line 273) | def self.warn(message)
function load_records (line 278) | def self.load_records(relation, ids)
function load_model (line 295) | def self.load_model(class_name, allow_child: false)
function indexer (line 311) | def self.indexer
function callbacks_value (line 316) | def self.callbacks_value
function callbacks_value= (line 321) | def self.callbacks_value=(value)
function signer_middleware_aws_params (line 326) | def self.signer_middleware_aws_params
function relation? (line 334) | def self.relation?(klass)
function scope (line 343) | def self.scope(model)
function not_found_error? (line 355) | def self.not_found_error?(e)
function transport_error? (line 362) | def self.transport_error?(e)
function not_allowed_error? (line 369) | def self.not_allowed_error?(e)
FILE: lib/searchkick/bulk_reindex_job.rb
type Searchkick (line 1) | module Searchkick
class BulkReindexJob (line 2) | class BulkReindexJob < Searchkick.parent_job.constantize
method perform (line 5) | def perform(class_name:, record_ids: nil, index_name: nil, method_na...
FILE: lib/searchkick/controller_runtime.rb
type Searchkick (line 2) | module Searchkick
type ControllerRuntime (line 3) | module ControllerRuntime
function process_action (line 10) | def process_action(action, *args)
function cleanup_view_runtime (line 18) | def cleanup_view_runtime
function append_info_to_payload (line 26) | def append_info_to_payload(payload)
type ClassMethods (line 31) | module ClassMethods
function log_process_action (line 32) | def log_process_action(payload)
FILE: lib/searchkick/hash_wrapper.rb
type Searchkick (line 1) | module Searchkick
class HashWrapper (line 2) | class HashWrapper
method initialize (line 3) | def initialize(attributes)
method [] (line 7) | def [](name)
method to_h (line 11) | def to_h
method as_json (line 15) | def as_json(...)
method to_json (line 19) | def to_json(...)
method method_missing (line 23) | def method_missing(name, ...)
method respond_to_missing? (line 31) | def respond_to_missing?(name, ...)
method inspect (line 35) | def inspect
FILE: lib/searchkick/index.rb
type Searchkick (line 1) | module Searchkick
class Index (line 2) | class Index
method initialize (line 5) | def initialize(name, options = {})
method index_options (line 11) | def index_options
method create (line 15) | def create(body = {})
method delete (line 19) | def delete
method exists? (line 29) | def exists?
method refresh (line 33) | def refresh
method alias_exists? (line 37) | def alias_exists?
method mapping (line 43) | def mapping
method settings (line 48) | def settings
method refresh_interval (line 52) | def refresh_interval
method update_settings (line 56) | def update_settings(settings)
method tokens (line 60) | def tokens(text, options = {})
method total_docs (line 64) | def total_docs
method promote (line 78) | def promote(new_name, update_refresh_interval: false)
method retrieve (line 98) | def retrieve(record)
method all_indices (line 107) | def all_indices(unaliased: false)
method clean_indices (line 124) | def clean_indices
method store (line 132) | def store(record)
method remove (line 138) | def remove(record)
method update_record (line 144) | def update_record(record, method_name)
method bulk_delete (line 150) | def bulk_delete(records)
method bulk_index (line 158) | def bulk_index(records)
method bulk_update (line 167) | def bulk_update(records, method_name, ignore_missing: nil)
method search_id (line 175) | def search_id(record)
method document_type (line 179) | def document_type(record)
method similar_record (line 183) | def similar_record(record, **options)
method reload_synonyms (line 191) | def reload_synonyms
method reindex_queue (line 206) | def reindex_queue
method reindex (line 214) | def reindex(object, method_name: nil, ignore_missing: nil, full: fal...
method create_index (line 261) | def create_index(index_options: nil)
method import_scope (line 268) | def import_scope(relation, **options)
method batches_left (line 272) | def batches_left
method klass_document_type (line 277) | def klass_document_type(klass, ignore_type = false)
method conversions_fields (line 290) | def conversions_fields
method conversions_v2_fields (line 298) | def conversions_v2_fields
method suggest_fields (line 303) | def suggest_fields
method locations_fields (line 308) | def locations_fields
method uuid (line 316) | def uuid
method client (line 322) | def client
method queue_index (line 326) | def queue_index(records)
method queue_delete (line 330) | def queue_delete(records)
method queue_update (line 334) | def queue_update(records, method_name, ignore_missing:)
method relation_indexer (line 340) | def relation_indexer
method index_settings (line 344) | def index_settings
method import_before_promotion (line 348) | def import_before_promotion(index, relation, **import_options)
method reindex_records (line 352) | def reindex_records(object, mode: nil, refresh: false, **options)
method full_reindex (line 363) | def full_reindex(relation, import: true, resume: false, retain: fals...
method check_uuid (line 445) | def check_uuid(old_uuid, new_uuid)
method notify (line 451) | def notify(record, name)
method notify_bulk (line 466) | def notify_bulk(records, name)
FILE: lib/searchkick/index_cache.rb
type Searchkick (line 1) | module Searchkick
class IndexCache (line 2) | class IndexCache
method initialize (line 3) | def initialize(max_size: 20)
method fetch (line 11) | def fetch(name)
method clear (line 24) | def clear
FILE: lib/searchkick/index_options.rb
type Searchkick (line 1) | module Searchkick
class IndexOptions (line 2) | class IndexOptions
method initialize (line 5) | def initialize(index)
method index_options (line 9) | def index_options
method generate_settings (line 30) | def generate_settings
method update_language (line 205) | def update_language(settings, language)
method update_stemming (line 303) | def update_stemming(settings)
method generate_mappings (line 358) | def generate_mappings
method add_synonyms (line 568) | def add_synonyms(settings)
method add_search_synonyms (line 594) | def add_search_synonyms(settings)
method set_deep_paging (line 629) | def set_deep_paging(settings)
method index_type (line 636) | def index_type
method default_type (line 644) | def default_type
method default_analyzer (line 648) | def default_analyzer
FILE: lib/searchkick/indexer.rb
type Searchkick (line 3) | module Searchkick
class Indexer (line 4) | class Indexer
method initialize (line 7) | def initialize
method queue (line 11) | def queue(items)
method perform (line 16) | def perform
method ignore_missing? (line 39) | def ignore_missing?(item, error)
FILE: lib/searchkick/log_subscriber.rb
type Searchkick (line 2) | module Searchkick
class LogSubscriber (line 3) | class LogSubscriber < ActiveSupport::LogSubscriber
method runtime= (line 4) | def self.runtime=(value)
method runtime (line 8) | def self.runtime
method reset_runtime (line 12) | def self.reset_runtime
method search (line 18) | def search(event)
method request (line 37) | def request(event)
method multi_search (line 47) | def multi_search(event)
FILE: lib/searchkick/middleware.rb
type Searchkick (line 3) | module Searchkick
class Middleware (line 4) | class Middleware < Faraday::Middleware
method call (line 5) | def call(env)
FILE: lib/searchkick/model.rb
type Searchkick (line 1) | module Searchkick
type Model (line 2) | module Model
function searchkick (line 3) | def searchkick(**options)
FILE: lib/searchkick/multi_search.rb
type Searchkick (line 1) | module Searchkick
class MultiSearch (line 2) | class MultiSearch
method initialize (line 5) | def initialize(queries, opaque_id: nil)
method perform (line 10) | def perform
method perform_search (line 18) | def perform_search(search_queries, perform_retry: true)
method client (line 42) | def client
FILE: lib/searchkick/process_batch_job.rb
type Searchkick (line 1) | module Searchkick
class ProcessBatchJob (line 2) | class ProcessBatchJob < Searchkick.parent_job.constantize
method perform (line 5) | def perform(class_name:, record_ids:, index_name: nil)
FILE: lib/searchkick/process_queue_job.rb
type Searchkick (line 1) | module Searchkick
class ProcessQueueJob (line 2) | class ProcessQueueJob < Searchkick.parent_job.constantize
method perform (line 5) | def perform(class_name:, index_name: nil, inline: false, job_options...
FILE: lib/searchkick/query.rb
type Searchkick (line 1) | module Searchkick
class Query (line 2) | class Query
method initialize (line 19) | def initialize(klass, term = "*", **options)
method searchkick_index (line 57) | def searchkick_index
method searchkick_options (line 61) | def searchkick_options
method searchkick_klass (line 65) | def searchkick_klass
method params (line 69) | def params
method execute (line 102) | def execute
method handle_response (line 117) | def handle_response(response)
method retry_misspellings? (line 183) | def retry_misspellings?(response)
method handle_error (line 189) | def handle_error(e)
method reindex_command (line 222) | def reindex_command
method execute_search (line 226) | def execute_search
method prepare (line 237) | def prepare
method set_fields (line 557) | def set_fields
method build_query (line 583) | def build_query(query, filters, should, must_not, custom_filters, mu...
method set_conversions (line 616) | def set_conversions
method set_conversions_v2 (line 645) | def set_conversions_v2
method set_exclude (line 682) | def set_exclude(field, analyzer)
method set_boost_by_distance (line 695) | def set_boost_by_distance(custom_filters)
method set_boost_by_recency (line 720) | def set_boost_by_recency(custom_filters)
method set_boost_by (line 733) | def set_boost_by(multiply_filters, custom_filters)
method set_boost_where (line 746) | def set_boost_where(custom_filters)
method set_boost_by_indices (line 762) | def set_boost_by_indices(payload)
method set_suggestions (line 773) | def set_suggestions(payload, suggest)
method set_highlights (line 801) | def set_highlights(payload, fields)
method set_aggregations (line 833) | def set_aggregations(payload, filters, post_filters)
method where_without_field (line 916) | def where_without_field(where, field)
method combine_agg_where (line 943) | def combine_agg_where(agg_where, where)
method set_knn (line 963) | def set_knn(payload, knn, per_page, offset)
method set_post_filters (line 1086) | def set_post_filters(payload, post_filters)
method set_order (line 1094) | def set_order(payload)
method ensure_permitted (line 1101) | def ensure_permitted(obj)
method where_filters (line 1105) | def where_filters(where)
method term_filters (line 1253) | def term_filters(field, value)
method custom_filter (line 1293) | def custom_filter(field, value, factor)
method boost_filter (line 1300) | def boost_filter(field, factor: 1, modifier: nil, missing: nil)
method boost_filters (line 1322) | def boost_filters(boost_by, modifier: nil)
method coordinate_array (line 1331) | def coordinate_array(value)
method location_value (line 1341) | def location_value(value)
method expand_range (line 1349) | def expand_range(range)
method base_field (line 1360) | def base_field(k)
method track_total_hits? (line 1364) | def track_total_hits?
method body_options (line 1368) | def body_options
FILE: lib/searchkick/railtie.rb
type Searchkick (line 1) | module Searchkick
class Railtie (line 2) | class Railtie < Rails::Railtie
FILE: lib/searchkick/record_data.rb
type Searchkick (line 1) | module Searchkick
class RecordData (line 2) | class RecordData
method initialize (line 7) | def initialize(index, record)
method index_data (line 12) | def index_data
method update_data (line 18) | def update_data(method_name)
method delete_data (line 24) | def delete_data
method search_id (line 29) | def search_id
method document_type (line 34) | def document_type(ignore_type = false)
method record_data (line 38) | def record_data
method search_data (line 49) | def search_data(method_name = nil)
method location_value (line 112) | def location_value(value)
method cast_big_decimal (line 125) | def cast_big_decimal(obj)
FILE: lib/searchkick/record_indexer.rb
type Searchkick (line 1) | module Searchkick
class RecordIndexer (line 2) | class RecordIndexer
method initialize (line 5) | def initialize(index)
method reindex (line 9) | def reindex(records, mode:, method_name:, ignore_missing:, full: fal...
method reindex_items (line 73) | def reindex_items(klass, items, method_name:, ignore_missing:, singl...
method index_record? (line 94) | def index_record?(record)
method import_inline (line 99) | def import_inline(index_records, delete_records, method_name:, ignor...
method maybe_bulk (line 117) | def maybe_bulk(index_records, delete_records, method_name, single)
method construct_record (line 149) | def construct_record(klass, id, routing)
method with_retries (line 160) | def with_retries
FILE: lib/searchkick/reindex_queue.rb
type Searchkick (line 1) | module Searchkick
class ReindexQueue (line 2) | class ReindexQueue
method initialize (line 5) | def initialize(name)
method push (line 12) | def push(record_ids)
method push_records (line 16) | def push_records(records)
method reserve (line 35) | def reserve(limit: 1000)
method clear (line 39) | def clear
method length (line 43) | def length
method redis_key (line 49) | def redis_key
method escape (line 53) | def escape(value)
FILE: lib/searchkick/reindex_v2_job.rb
type Searchkick (line 1) | module Searchkick
class ReindexV2Job (line 2) | class ReindexV2Job < Searchkick.parent_job.constantize
method perform (line 5) | def perform(class_name, id, method_name = nil, routing: nil, index_n...
FILE: lib/searchkick/relation.rb
type Searchkick (line 1) | module Searchkick
class Relation (line 2) | class Relation
method initialize (line 14) | def initialize(model, term = "*", **options)
method inspect (line 24) | def inspect
method aggs (line 30) | def aggs(*args, **kwargs)
method aggs! (line 38) | def aggs!(*args, **kwargs)
method body (line 53) | def body(value = NO_DEFAULT_VALUE)
method body! (line 61) | def body!(value)
method body_options (line 67) | def body_options(value)
method body_options! (line 71) | def body_options!(value)
method boost (line 77) | def boost(value)
method boost! (line 81) | def boost!(value)
method boost_by (line 87) | def boost_by(value)
method boost_by! (line 91) | def boost_by!(value)
method boost_by_distance (line 102) | def boost_by_distance(value)
method boost_by_distance! (line 106) | def boost_by_distance!(value)
method boost_by_recency (line 114) | def boost_by_recency(value)
method boost_by_recency! (line 118) | def boost_by_recency!(value)
method boost_where (line 124) | def boost_where(value)
method boost_where! (line 128) | def boost_where!(value)
method conversions (line 135) | def conversions(value)
method conversions! (line 139) | def conversions!(value)
method conversions_v1 (line 145) | def conversions_v1(value)
method conversions_v1! (line 149) | def conversions_v1!(value)
method conversions_v2 (line 155) | def conversions_v2(value)
method conversions_v2! (line 159) | def conversions_v2!(value)
method conversions_term (line 165) | def conversions_term(value)
method conversions_term! (line 169) | def conversions_term!(value)
method debug (line 175) | def debug(value = true)
method debug! (line 179) | def debug!(value = true)
method emoji (line 185) | def emoji(value = true)
method emoji! (line 189) | def emoji!(value = true)
method exclude (line 195) | def exclude(*values)
method exclude! (line 199) | def exclude!(*values)
method explain (line 205) | def explain(value = true)
method explain! (line 209) | def explain!(value = true)
method fields (line 215) | def fields(*values)
method fields! (line 219) | def fields!(*values)
method highlight (line 225) | def highlight(value)
method highlight! (line 229) | def highlight!(value)
method includes (line 235) | def includes(*values)
method includes! (line 239) | def includes!(*values)
method index_name (line 245) | def index_name(*values)
method index_name! (line 249) | def index_name!(*values)
method indices_boost (line 260) | def indices_boost(value)
method indices_boost! (line 264) | def indices_boost!(value)
method knn (line 270) | def knn(value)
method knn! (line 274) | def knn!(value)
method limit (line 280) | def limit(value)
method limit! (line 284) | def limit!(value)
method load (line 290) | def load(value = NO_DEFAULT_VALUE)
method load! (line 299) | def load!(value)
method match (line 305) | def match(value)
method match! (line 309) | def match!(value)
method misspellings (line 315) | def misspellings(value)
method misspellings! (line 319) | def misspellings!(value)
method models (line 325) | def models(*values)
method models! (line 329) | def models!(*values)
method model_includes (line 335) | def model_includes(*values)
method model_includes! (line 339) | def model_includes!(*values)
method offset (line 345) | def offset(value = NO_DEFAULT_VALUE)
method offset! (line 353) | def offset!(value)
method opaque_id (line 359) | def opaque_id(value)
method opaque_id! (line 363) | def opaque_id!(value)
method operator (line 369) | def operator(value)
method operator! (line 373) | def operator!(value)
method order (line 379) | def order(*values)
method order! (line 383) | def order!(*values)
method padding (line 389) | def padding(value = NO_DEFAULT_VALUE)
method padding! (line 397) | def padding!(value)
method page (line 403) | def page(value)
method page! (line 407) | def page!(value)
method per_page (line 413) | def per_page(value = NO_DEFAULT_VALUE)
method per (line 421) | def per(value)
method per_page! (line 425) | def per_page!(value)
method profile (line 432) | def profile(value = true)
method profile! (line 436) | def profile!(value = true)
method request_params (line 442) | def request_params(value)
method request_params! (line 446) | def request_params!(value)
method routing (line 452) | def routing(value)
method routing! (line 456) | def routing!(value)
method scope_results (line 462) | def scope_results(value)
method scope_results! (line 466) | def scope_results!(value)
method scroll (line 472) | def scroll(value = NO_DEFAULT_VALUE, &block)
method scroll! (line 482) | def scroll!(value)
method select (line 488) | def select(*values, &block)
method select! (line 496) | def select!(*values)
method similar (line 502) | def similar(value = true)
method similar! (line 506) | def similar!(value = true)
method smart_aggs (line 512) | def smart_aggs(value)
method smart_aggs! (line 516) | def smart_aggs!(value)
method suggest (line 522) | def suggest(value = true)
method suggest! (line 526) | def suggest!(value = true)
method total_entries (line 532) | def total_entries(value = NO_DEFAULT_VALUE)
method total_entries! (line 540) | def total_entries!(value)
method track (line 546) | def track(value = true)
method track! (line 550) | def track!(value = true)
method type (line 556) | def type(*values)
method type! (line 560) | def type!(*values)
method where (line 566) | def where(value = NO_DEFAULT_VALUE)
method where! (line 574) | def where!(value)
method first (line 592) | def first(value = NO_DEFAULT_VALUE)
method pluck (line 612) | def pluck(*keys)
method reorder (line 620) | def reorder(*values)
method reorder! (line 624) | def reorder!(*values)
method reselect (line 630) | def reselect(*values)
method reselect! (line 634) | def reselect!(*values)
method rewhere (line 640) | def rewhere(value)
method rewhere! (line 644) | def rewhere!(value)
method only (line 650) | def only(*keys)
method except (line 654) | def except(*keys)
method loaded? (line 658) | def loaded?
method respond_to_missing? (line 664) | def respond_to_missing?(...)
method to_yaml (line 678) | def to_yaml
method private_execute (line 684) | def private_execute
method query (line 688) | def query
method check_loaded (line 692) | def check_loaded
method ensure_permitted (line 701) | def ensure_permitted(obj)
method initialize_copy (line 705) | def initialize_copy(other)
method concat_option (line 712) | def concat_option(key, value)
method merge_option (line 720) | def merge_option(key, value)
FILE: lib/searchkick/relation_indexer.rb
type Searchkick (line 1) | module Searchkick
class RelationIndexer (line 2) | class RelationIndexer
method initialize (line 5) | def initialize(index)
method reindex (line 9) | def reindex(relation, mode:, method_name: nil, ignore_missing: nil, ...
method batches_left (line 51) | def batches_left
method batch_completed (line 55) | def batch_completed(batch_id)
method resume_relation (line 61) | def resume_relation(relation)
method in_batches (line 72) | def in_batches(relation)
method each_batch (line 115) | def each_batch(relation, batch_size:)
method batch_size (line 129) | def batch_size
method full_reindex_async (line 133) | def full_reindex_async(relation, job_options: nil)
method batch_job (line 168) | def batch_job(class_name, batch_id, job_options, **options)
method batches_key (line 180) | def batches_key
FILE: lib/searchkick/reranking.rb
type Searchkick (line 1) | module Searchkick
type Reranking (line 2) | module Reranking
function rrf (line 3) | def self.rrf(first_ranking, *rankings, k: 60)
FILE: lib/searchkick/results.rb
type Searchkick (line 1) | module Searchkick
class Results (line 2) | class Results
method initialize (line 10) | def initialize(klass, response, options = {})
method with_hit (line 16) | def with_hit
method missing_records (line 24) | def missing_records
method suggestions (line 28) | def suggestions
method aggregations (line 38) | def aggregations
method aggs (line 42) | def aggs
method took (line 57) | def took
method error (line 61) | def error
method model_name (line 65) | def model_name
method entry_name (line 73) | def entry_name(options = {})
method total_count (line 83) | def total_count
method current_page (line 94) | def current_page
method per_page (line 98) | def per_page
method padding (line 103) | def padding
method total_pages (line 107) | def total_pages
method offset_value (line 112) | def offset_value
method previous_page (line 117) | def previous_page
method next_page (line 122) | def next_page
method first_page? (line 126) | def first_page?
method last_page? (line 130) | def last_page?
method out_of_range? (line 134) | def out_of_range?
method hits (line 138) | def hits
method highlights (line 146) | def highlights(multiple: false)
method with_highlights (line 152) | def with_highlights(multiple: false)
method with_score (line 160) | def with_score
method misspellings? (line 168) | def misspellings?
method scroll_id (line 172) | def scroll_id
method scroll (line 176) | def scroll
method clear_scroll (line 206) | def clear_scroll
method results (line 221) | def results
method with_hit_and_missing_records (line 225) | def with_hit_and_missing_records
method build_hits (line 309) | def build_hits
method results_query (line 318) | def results_query(records, hits)
method combine_includes (line 337) | def combine_includes(result, inc)
method base_field (line 347) | def base_field(k)
method hit_highlights (line 351) | def hit_highlights(hit, multiple: false)
FILE: lib/searchkick/script.rb
type Searchkick (line 1) | module Searchkick
class Script (line 2) | class Script
method initialize (line 5) | def initialize(source, lang: "painless", params: {})
FILE: lib/searchkick/version.rb
type Searchkick (line 1) | module Searchkick
FILE: lib/searchkick/where.rb
type Searchkick (line 1) | module Searchkick
class Where (line 2) | class Where
method initialize (line 3) | def initialize(relation)
method not (line 7) | def not(value)
FILE: test/aggs_test.rb
class AggsTest (line 3) | class AggsTest < Minitest::Test
method setup (line 4) | def setup
method test_single (line 14) | def test_single
method test_multiple (line 19) | def test_multiple
method test_multiple_where (line 26) | def test_multiple_where
method test_none (line 32) | def test_none
method test_where (line 36) | def test_where
method test_field (line 43) | def test_field
method test_min_doc_count (line 49) | def test_min_doc_count
method test_script (line 53) | def test_script
method test_order (line 58) | def test_order
method test_limit (line 63) | def test_limit
method test_ranges (line 70) | def test_ranges
method test_date_ranges (line 81) | def test_date_ranges
method test_group_by_date (line 89) | def test_group_by_date
method test_time_zone (line 96) | def test_time_zone
method test_avg (line 118) | def test_avg
method test_cardinality (line 123) | def test_cardinality
method test_min_max (line 128) | def test_min_max
method test_sum (line 134) | def test_sum
method test_body_options (line 139) | def test_body_options
method test_smart_aggs (line 144) | def test_smart_aggs
method test_smart_aggs_overlap (line 164) | def test_smart_aggs_overlap
method test_smart_aggs_agg_where (line 206) | def test_smart_aggs_agg_where
method test_smart_aggs_agg_where_overlap (line 235) | def test_smart_aggs_agg_where_overlap
method test_smart_aggs_relation (line 251) | def test_smart_aggs_relation
method assert_aggs (line 264) | def assert_aggs(expected, options)
method agg_buckets (line 273) | def agg_buckets(relation)
FILE: test/boost_test.rb
class BoostTest (line 3) | class BoostTest < Minitest::Test
method test_boost (line 6) | def test_boost
method test_boost_zero (line 15) | def test_boost_zero
method test_fields (line 24) | def test_fields
method test_fields_decimal (line 32) | def test_fields_decimal
method test_fields_word_start (line 40) | def test_fields_word_start
method test_fields_apostrophes (line 49) | def test_fields_apostrophes
method test_boost_by (line 56) | def test_boost_by
method test_boost_by_missing (line 66) | def test_boost_by_missing
method test_boost_by_boost_mode_multiply (line 74) | def test_boost_by_boost_mode_multiply
method test_boost_where (line 83) | def test_boost_where
method test_boost_where_negative_boost (line 97) | def test_boost_where_negative_boost
method test_boost_by_recency (line 106) | def test_boost_by_recency
method test_boost_by_recency_origin (line 115) | def test_boost_by_recency_origin
method test_boost_by_distance (line 124) | def test_boost_by_distance
method test_boost_by_distance_hash (line 133) | def test_boost_by_distance_hash
method test_boost_by_distance_v2 (line 142) | def test_boost_by_distance_v2
method test_boost_by_distance_v2_hash (line 151) | def test_boost_by_distance_v2_hash
method test_boost_by_distance_v2_factor (line 160) | def test_boost_by_distance_v2_factor
method test_boost_by_indices (line 170) | def test_boost_by_indices
FILE: test/callbacks_test.rb
class CallbacksTest (line 3) | class CallbacksTest < Minitest::Test
method test_false (line 4) | def test_false
method test_bulk (line 11) | def test_bulk
method test_async (line 19) | def test_async
method test_queue (line 27) | def test_queue
method test_record_async (line 67) | def test_record_async
method test_relation_async (line 79) | def test_relation_async
method test_disable_callbacks (line 87) | def test_disable_callbacks
FILE: test/conversions_test.rb
class ConversionsTest (line 3) | class ConversionsTest < Minitest::Test
method setup (line 4) | def setup
method test_v1 (line 9) | def test_v1
method test_v1_case (line 20) | def test_v1_case
method test_v1_case_sensitive (line 28) | def test_v1_case_sensitive
method test_v1_term (line 40) | def test_v1_term
method test_v1_weight (line 49) | def test_v1_weight
method test_v1_multiple_conversions (line 58) | def test_v1_multiple_conversions
method test_v1_multiple_conversions_with_boost_term (line 73) | def test_v1_multiple_conversions_with_boost_term
method test_v2 (line 85) | def test_v2
method test_v2_case (line 96) | def test_v2_case
method test_v2_case_sensitive (line 104) | def test_v2_case_sensitive
method test_v2_term (line 116) | def test_v2_term
method test_v2_weight (line 126) | def test_v2_weight
method test_v2_space (line 135) | def test_v2_space
method test_v2_dot (line 144) | def test_v2_dot
method test_v2_unicode (line 153) | def test_v2_unicode
method test_v2_score (line 162) | def test_v2_score
method test_v2_factor (line 173) | def test_v2_factor
method test_v2_no_tokenization (line 187) | def test_v2_no_tokenization
method test_v2_max_conversions (line 196) | def test_v2_max_conversions
method test_v2_max_length (line 206) | def test_v2_max_length
method test_v2_zero (line 214) | def test_v2_zero
method test_v2_partial_reindex (line 221) | def test_v2_partial_reindex
FILE: test/default_scope_test.rb
class DefaultScopeTest (line 3) | class DefaultScopeTest < Minitest::Test
method setup (line 4) | def setup
method test_reindex (line 8) | def test_reindex
method test_search (line 18) | def test_search
method default_model (line 28) | def default_model
FILE: test/exclude_test.rb
class ExcludeTest (line 3) | class ExcludeTest < Minitest::Test
method test_butter (line 4) | def test_butter
method test_butter_word_start (line 9) | def test_butter_word_start
method test_butter_exact (line 14) | def test_butter_exact
method test_same_exact (line 19) | def test_same_exact
method test_egg_word_start (line 24) | def test_egg_word_start
method test_string (line 29) | def test_string
method test_match_all (line 34) | def test_match_all
method test_match_all_fields (line 39) | def test_match_all_fields
FILE: test/geo_shape_test.rb
class GeoShapeTest (line 3) | class GeoShapeTest < Minitest::Test
method setup (line 4) | def setup
method test_envelope (line 34) | def test_envelope
method test_polygon (line 47) | def test_polygon
method test_multipolygon (line 60) | def test_multipolygon
method test_disjoint (line 76) | def test_disjoint
method test_within (line 90) | def test_within
method test_search_match (line 104) | def test_search_match
method test_search_no_match (line 117) | def test_search_no_match
method test_latlon (line 130) | def test_latlon
method default_model (line 143) | def default_model
FILE: test/highlight_test.rb
class HighlightTest (line 3) | class HighlightTest < Minitest::Test
method test_basic (line 4) | def test_basic
method test_with_highlights (line 9) | def test_with_highlights
method test_tag (line 14) | def test_tag
method test_tag_class (line 20) | def test_tag_class
method test_very_long (line 25) | def test_very_long
method test_multiple_fields (line 30) | def test_multiple_fields
method test_fields (line 37) | def test_fields
method test_field_options (line 44) | def test_field_options
method test_multiple_words (line 50) | def test_multiple_words
method test_encoder (line 55) | def test_encoder
method test_word_middle (line 60) | def test_word_middle
method test_body (line 65) | def test_body
method test_multiple_highlights (line 85) | def test_multiple_highlights
method test_search_highlights_method (line 96) | def test_search_highlights_method
method test_match_all (line 101) | def test_match_all
method test_match_all_load_false (line 106) | def test_match_all_load_false
method test_match_all_search_highlights (line 111) | def test_match_all_search_highlights
FILE: test/hybrid_test.rb
class HybridTest (line 3) | class HybridTest < Minitest::Test
method setup (line 4) | def setup
method test_search (line 9) | def test_search
method test_multi_search (line 16) | def test_multi_search
FILE: test/index_cache_test.rb
class IndexCacheTest (line 3) | class IndexCacheTest < Minitest::Test
method setup (line 4) | def setup
method test_default (line 8) | def test_default
method test_max_size (line 15) | def test_max_size
method test_thread_safe (line 22) | def test_thread_safe
method test_thread_safe_max_size (line 30) | def test_thread_safe_max_size
method object_ids (line 36) | def object_ids(count)
method with_threads (line 40) | def with_threads
FILE: test/index_options_test.rb
class IndexOptionsTest (line 3) | class IndexOptionsTest < Minitest::Test
method setup (line 4) | def setup
method test_case_sensitive (line 8) | def test_case_sensitive
method test_no_stemming (line 15) | def test_no_stemming
method test_no_stem_exclusion (line 22) | def test_no_stem_exclusion
method test_stem_exclusion (line 32) | def test_stem_exclusion
method test_no_stemmer_override (line 42) | def test_no_stemmer_override
method test_stemmer_override (line 52) | def test_stemmer_override
method test_special_characters (line 62) | def test_special_characters
method test_index_name (line 69) | def test_index_name
method test_index_name_callable (line 75) | def test_index_name_callable
method test_index_prefix (line 81) | def test_index_prefix
method test_index_prefix_callable (line 87) | def test_index_prefix_callable
method default_model (line 93) | def default_model
FILE: test/index_test.rb
class IndexTest (line 3) | class IndexTest < Minitest::Test
method setup (line 4) | def setup
method test_tokens (line 9) | def test_tokens
method test_tokens_analyzer (line 13) | def test_tokens_analyzer
method test_total_docs (line 17) | def test_total_docs
method test_clean_indices (line 22) | def test_clean_indices
method test_clean_indices_old_format (line 41) | def test_clean_indices_old_format
method test_retain (line 51) | def test_retain
method test_mappings (line 58) | def test_mappings
method test_settings (line 66) | def test_settings
method test_remove_blank_id (line 70) | def test_remove_blank_id
method test_store_response (line 79) | def test_store_response
method test_bulk_index_response (line 85) | def test_bulk_index_response
method test_filterable (line 92) | def test_filterable
method test_filterable_non_string (line 100) | def test_filterable_non_string
method test_large_value (line 105) | def test_large_value
method test_very_large_value (line 114) | def test_very_large_value
method test_bulk_import_raises_error (line 126) | def test_bulk_import_raises_error
FILE: test/inheritance_test.rb
class InheritanceTest (line 3) | class InheritanceTest < Minitest::Test
method setup (line 4) | def setup
method test_child_reindex (line 9) | def test_child_reindex
method test_child_index_name (line 15) | def test_child_index_name
method test_child_search (line 19) | def test_child_search
method test_parent_search (line 25) | def test_parent_search
method test_force_one_type (line 31) | def test_force_one_type
method test_force_multiple_types (line 37) | def test_force_multiple_types
method test_child_autocomplete (line 44) | def test_child_autocomplete
method test_parent_autocomplete (line 50) | def test_parent_autocomplete
method test_parent_suggest (line 62) | def test_parent_suggest
method test_reindex (line 68) | def test_reindex
method test_child_models_option (line 75) | def test_child_models_option
method test_missing_records (line 84) | def test_missing_records
method test_inherited_and_non_inherited_models (line 101) | def test_inherited_and_non_inherited_models
method test_multiple_indices (line 113) | def test_multiple_indices
method test_index_name_model (line 120) | def test_index_name_model
method test_index_name_string (line 125) | def test_index_name_string
method test_similar (line 133) | def test_similar
FILE: test/knn_test.rb
class KnnTest (line 3) | class KnnTest < Minitest::Test
method setup (line 4) | def setup
method test_basic (line 12) | def test_basic
method test_basic_exact (line 21) | def test_basic_exact
method test_where (line 30) | def test_where
method test_where_exact (line 40) | def test_where_exact
method test_pagination (line 50) | def test_pagination
method test_pagination_exact (line 61) | def test_pagination_exact
method test_euclidean (line 72) | def test_euclidean
method test_euclidean_exact (line 81) | def test_euclidean_exact
method test_taxicab_exact (line 90) | def test_taxicab_exact
method test_chebyshev_exact (line 99) | def test_chebyshev_exact
method test_inner_product (line 110) | def test_inner_product
method test_inner_product_exact (line 121) | def test_inner_product_exact
method test_unindexed (line 130) | def test_unindexed
method test_explain (line 163) | def test_explain
method test_ef_search (line 201) | def test_ef_search
method assert_approx (line 210) | def assert_approx(approx, field, distance, **knn_options)
FILE: test/language_test.rb
class LanguageTest (line 3) | class LanguageTest < Minitest::Test
method setup (line 4) | def setup
method test_chinese (line 10) | def test_chinese
method test_chinese2 (line 22) | def test_chinese2
method test_japanese (line 32) | def test_japanese
method test_japanese_search_synonyms (line 42) | def test_japanese_search_synonyms
method test_korean (line 51) | def test_korean
method test_korean2 (line 63) | def test_korean2
method test_polish (line 75) | def test_polish
method test_ukrainian (line 83) | def test_ukrainian
method test_vietnamese (line 91) | def test_vietnamese
method test_stemmer_hunspell (line 102) | def test_stemmer_hunspell
method test_stemmer_unknown_type (line 111) | def test_stemmer_unknown_type
method test_stemmer_language (line 119) | def test_stemmer_language
method assert_language_search (line 129) | def assert_language_search(term, expected)
method default_model (line 133) | def default_model
FILE: test/load_test.rb
class LoadTest (line 3) | class LoadTest < Minitest::Test
method test_default (line 4) | def test_default
method test_false (line 37) | def test_false
method test_false_methods (line 63) | def test_false_methods
method test_false_with_includes (line 68) | def test_false_with_includes
method test_false_nested_object (line 73) | def test_false_nested_object
FILE: test/log_subscriber_test.rb
class LogSubscriberTest (line 3) | class LogSubscriberTest < Minitest::Test
method test_create (line 4) | def test_create
method test_update (line 11) | def test_update
method test_destroy (line 19) | def test_destroy
method test_bulk (line 27) | def test_bulk
method test_reindex (line 37) | def test_reindex
method test_reindex_relation (line 46) | def test_reindex_relation
method test_search (line 55) | def test_search
method test_multi_search (line 65) | def test_multi_search
method create_products (line 77) | def create_products
method capture_logs (line 85) | def capture_logs
FILE: test/marshal_test.rb
class MarshalTest (line 3) | class MarshalTest < Minitest::Test
method test_marshal (line 4) | def test_marshal
method test_marshal_highlights (line 9) | def test_marshal_highlights
FILE: test/match_test.rb
class MatchTest (line 3) | class MatchTest < Minitest::Test
method test_match (line 6) | def test_match
method test_case (line 11) | def test_case
method test_cheese_space_in_index (line 16) | def test_cheese_space_in_index
method test_middle_token (line 26) | def test_middle_token
method test_middle_token_wine (line 31) | def test_middle_token_wine
method test_percent (line 36) | def test_percent
method test_jalapenos (line 43) | def test_jalapenos
method test_swedish (line 48) | def test_swedish
method test_stemming (line 55) | def test_stemming
method test_stemming_tokens (line 61) | def test_stemming_tokens
method test_misspelling_sriracha (line 68) | def test_misspelling_sriracha
method test_misspelling_multiple (line 73) | def test_misspelling_multiple
method test_short_word (line 78) | def test_short_word
method test_edit_distance_two (line 83) | def test_edit_distance_two
method test_edit_distance_one (line 90) | def test_edit_distance_one
method test_edit_distance_long_word (line 97) | def test_edit_distance_long_word
method test_misspelling_tabasco (line 103) | def test_misspelling_tabasco
method test_misspelling_zucchini (line 108) | def test_misspelling_zucchini
method test_misspelling_ziploc (line 113) | def test_misspelling_ziploc
method test_misspelling_zucchini_transposition (line 118) | def test_misspelling_zucchini_transposition
method test_misspelling_lasagna (line 128) | def test_misspelling_lasagna
method test_misspelling_lasagna_pasta (line 136) | def test_misspelling_lasagna_pasta
method test_misspellings_word_start (line 143) | def test_misspellings_word_start
method test_spaces_in_field (line 150) | def test_spaces_in_field
method test_spaces_in_query (line 155) | def test_spaces_in_query
method test_spaces_three_words (line 160) | def test_spaces_three_words
method test_spaces_stemming (line 165) | def test_spaces_stemming
method test_all (line 172) | def test_all
method test_no_arguments (line 177) | def test_no_arguments
method test_no_term (line 182) | def test_no_term
method test_to_be_or_not_to_be (line 187) | def test_to_be_or_not_to_be
method test_apostrophe (line 192) | def test_apostrophe
method test_apostrophe_search (line 197) | def test_apostrophe_search
method test_ampersand_index (line 202) | def test_ampersand_index
method test_ampersand_search (line 207) | def test_ampersand_search
method test_phrase (line 212) | def test_phrase
method test_phrase_again (line 217) | def test_phrase_again
method test_phrase_order (line 222) | def test_phrase_order
method test_dynamic_fields (line 227) | def test_dynamic_fields
method test_unsearchable (line 233) | def test_unsearchable
method test_unsearchable_where (line 241) | def test_unsearchable_where
method test_emoji (line 248) | def test_emoji
method test_emoji_multiple (line 253) | def test_emoji_multiple
method test_operator (line 261) | def test_operator
method test_operator_scoring (line 270) | def test_operator_scoring
method test_fields_operator (line 277) | def test_fields_operator
method test_fields (line 288) | def test_fields
method test_non_existent_field (line 296) | def test_non_existent_field
method test_fields_both_match (line 301) | def test_fields_both_match
FILE: test/misspellings_test.rb
class MisspellingsTest (line 3) | class MisspellingsTest < Minitest::Test
method test_false (line 4) | def test_false
method test_distance (line 9) | def test_distance
method test_prefix_length (line 14) | def test_prefix_length
method test_prefix_length_operator (line 20) | def test_prefix_length_operator
method test_fields_operator (line 26) | def test_fields_operator
method test_below_unmet (line 37) | def test_below_unmet
method test_below_unmet_result (line 42) | def test_below_unmet_result
method test_below_met (line 47) | def test_below_met
method test_below_met_result (line 52) | def test_below_met_result
method test_field_correct_spelling_still_works (line 57) | def test_field_correct_spelling_still_works
method test_field_enabled (line 63) | def test_field_enabled
method test_field_disabled (line 69) | def test_field_disabled
method test_field_with_transpositions (line 75) | def test_field_with_transpositions
method test_field_with_edit_distance (line 80) | def test_field_with_edit_distance
method test_field_multiple (line 85) | def test_field_multiple
method test_field_requires_explicit_search_fields (line 94) | def test_field_requires_explicit_search_fields
method test_field_word_start (line 101) | def test_field_word_start
method assert_misspellings (line 108) | def assert_misspellings(term, expected, misspellings = {}, model = def...
FILE: test/models/animal.rb
class Animal (line 1) | class Animal
FILE: test/models/artist.rb
class Artist (line 1) | class Artist
method should_index? (line 4) | def should_index?
FILE: test/models/band.rb
class Band (line 1) | class Band
FILE: test/models/product.rb
class Product (line 1) | class Product
method search_data (line 37) | def search_data
method should_index? (line 51) | def should_index?
method search_name (line 55) | def search_name
FILE: test/models/region.rb
class Region (line 1) | class Region
method search_data (line 7) | def search_data
FILE: test/models/sku.rb
class Sku (line 1) | class Sku
FILE: test/models/song.rb
class Song (line 1) | class Song
method search_routing (line 4) | def search_routing
FILE: test/models/speaker.rb
class Speaker (line 1) | class Speaker
method search_data (line 17) | def search_data
FILE: test/models/store.rb
class Store (line 1) | class Store
method search_document_id (line 13) | def search_document_id
method search_routing (line 17) | def search_routing
FILE: test/multi_indices_test.rb
class MultiIndicesTest (line 3) | class MultiIndicesTest < Minitest::Test
method setup (line 4) | def setup
method test_basic (line 9) | def test_basic
method test_index_name (line 15) | def test_index_name
method test_models_and_index_name (line 24) | def test_models_and_index_name
method test_model_with_another_model (line 36) | def test_model_with_another_model
method test_model_with_another_model_in_index_name (line 43) | def test_model_with_another_model_in_index_name
method test_no_models_or_index_name (line 51) | def test_no_models_or_index_name
method test_no_models_or_index_name_load_false (line 60) | def test_no_models_or_index_name_load_false
method assert_search_multi (line 67) | def assert_search_multi(term, expected, options = {})
FILE: test/multi_search_test.rb
class MultiSearchTest (line 3) | class MultiSearchTest < Minitest::Test
method test_basic (line 4) | def test_basic
method test_methods (line 14) | def test_methods
method test_error (line 20) | def test_error
method test_misspellings_below_unmet (line 29) | def test_misspellings_below_unmet
method test_misspellings_below_error (line 36) | def test_misspellings_below_error
method test_query_error (line 42) | def test_query_error
FILE: test/multi_tenancy_test.rb
class MultiTenancyTest (line 3) | class MultiTenancyTest < Minitest::Test
method setup (line 4) | def setup
method test_basic (line 8) | def test_basic
method teardown (line 19) | def teardown
method default_model (line 23) | def default_model
FILE: test/notifications_test.rb
class NotificationsTest (line 3) | class NotificationsTest < Minitest::Test
method test_search (line 4) | def test_search
method capture_notifications (line 17) | def capture_notifications
FILE: test/order_test.rb
class OrderTest (line 3) | class OrderTest < Minitest::Test
method test_hash (line 4) | def test_hash
method test_string (line 10) | def test_string
method test_multiple (line 16) | def test_multiple
method test_unmapped_type (line 29) | def test_unmapped_type
method test_array (line 35) | def test_array
method test_script (line 41) | def test_script
FILE: test/pagination_test.rb
class PaginationTest (line 3) | class PaginationTest < Minitest::Test
method test_limit (line 4) | def test_limit
method test_no_limit (line 10) | def test_no_limit
method test_offset (line 16) | def test_offset
method test_pagination (line 22) | def test_pagination
method test_relation (line 48) | def test_relation
method test_per (line 74) | def test_per
method test_body (line 79) | def test_body
method test_nil_page (line 105) | def test_nil_page
method test_strings (line 113) | def test_strings
method test_total_entries (line 123) | def test_total_entries
method test_kaminari (line 128) | def test_kaminari
method test_deep_paging (line 146) | def test_deep_paging
method test_no_deep_paging (line 152) | def test_no_deep_paging
method test_max_result_window (line 160) | def test_max_result_window
method test_search_after (line 170) | def test_search_after
method test_pit (line 189) | def test_pit
method pit_supported? (line 237) | def pit_supported?
FILE: test/parameters_test.rb
class ParametersTest (line 3) | class ParametersTest < Minitest::Test
method setup (line 4) | def setup
method test_options (line 9) | def test_options
method test_where (line 16) | def test_where
method test_where_relation (line 23) | def test_where_relation
method test_rewhere_relation (line 30) | def test_rewhere_relation
method test_where_permitted (line 37) | def test_where_permitted
method test_where_permitted_relation (line 43) | def test_where_permitted_relation
method test_rewhere_permitted_relation (line 49) | def test_rewhere_permitted_relation
method test_where_value (line 55) | def test_where_value
method test_where_value_relation (line 61) | def test_where_value_relation
method test_rewhere_value_relation (line 67) | def test_rewhere_value_relation
method test_where_hash (line 73) | def test_where_hash
method test_where_hash_relation (line 82) | def test_where_hash_relation
method test_rewhere_hash_relation (line 91) | def test_rewhere_hash_relation
method test_aggs_where (line 99) | def test_aggs_where
method test_aggs_where_smart_aggs_false (line 106) | def test_aggs_where_smart_aggs_false
FILE: test/partial_match_test.rb
class PartialMatchTest (line 3) | class PartialMatchTest < Minitest::Test
method test_autocomplete (line 4) | def test_autocomplete
method test_autocomplete_two_words (line 9) | def test_autocomplete_two_words
method test_autocomplete_fields (line 14) | def test_autocomplete_fields
method test_text_start (line 19) | def test_text_start
method test_text_middle (line 25) | def test_text_middle
method test_text_end (line 33) | def test_text_end
method test_word_start (line 39) | def test_word_start
method test_word_middle (line 44) | def test_word_middle
method test_word_end (line 49) | def test_word_end
method test_word_start_multiple_words (line 54) | def test_word_start_multiple_words
method test_word_start_exact (line 59) | def test_word_start_exact
method test_word_start_exact_martin (line 64) | def test_word_start_exact_martin
method test_exact (line 71) | def test_exact
method test_exact_case (line 76) | def test_exact_case
FILE: test/partial_reindex_test.rb
class PartialReindexTest (line 3) | class PartialReindexTest < Minitest::Test
method test_record_inline (line 4) | def test_record_inline
method test_record_async (line 19) | def test_record_async
method test_record_queue (line 37) | def test_record_queue
method test_record_missing_inline (line 45) | def test_record_missing_inline
method test_record_ignore_missing_inline (line 57) | def test_record_ignore_missing_inline
method test_record_missing_async (line 69) | def test_record_missing_async
method test_record_ignore_missing_async (line 83) | def test_record_ignore_missing_async
method test_relation_inline (line 94) | def test_relation_inline
method test_relation_async (line 112) | def test_relation_async
method test_relation_queue (line 129) | def test_relation_queue
method test_relation_missing_inline (line 137) | def test_relation_missing_inline
method test_relation_ignore_missing_inline (line 149) | def test_relation_ignore_missing_inline
method test_relation_missing_async (line 158) | def test_relation_missing_async
method test_relation_ignore_missing_async (line 172) | def test_relation_ignore_missing_async
FILE: test/query_test.rb
class QueryTest (line 3) | class QueryTest < Minitest::Test
method test_basic (line 4) | def test_basic
method test_with_uneffective_min_score (line 10) | def test_with_uneffective_min_score
method test_default_timeout (line 15) | def test_default_timeout
method test_timeout_override (line 19) | def test_timeout_override
method test_request_params (line 23) | def test_request_params
method test_opaque_id (line 27) | def test_opaque_id
method test_debug (line 38) | def test_debug
method test_big_decimal (line 46) | def test_big_decimal
method test_body_options_should_merge_into_body (line 55) | def test_body_options_should_merge_into_body
method test_nested_search (line 62) | def test_nested_search
method test_includes (line 70) | def test_includes
method test_model_includes (line 78) | def test_model_includes
method test_scope_results (line 94) | def test_scope_results
method test_scope_results_relation (line 103) | def test_scope_results_relation
method set_search_slow_log (line 114) | def set_search_slow_log(value)
FILE: test/reindex_test.rb
class ReindexTest (line 3) | class ReindexTest < Minitest::Test
method test_record_inline (line 4) | def test_record_inline
method test_record_destroyed (line 12) | def test_record_destroyed
method test_record_async (line 21) | def test_record_async
method test_record_async_job_options (line 32) | def test_record_async_job_options
method test_record_queue (line 39) | def test_record_queue
method test_process_queue_job_options (line 57) | def test_process_queue_job_options
method test_record_index (line 65) | def test_record_index
method test_relation_inline (line 73) | def test_relation_inline
method test_relation_associations (line 80) | def test_relation_associations
method test_relation_scoping (line 88) | def test_relation_scoping
method test_relation_scoping_restored (line 101) | def test_relation_scoping_restored
method test_relation_should_index (line 118) | def test_relation_should_index
method test_relation_async (line 128) | def test_relation_async
method test_relation_async_should_index (line 138) | def test_relation_async_should_index
method test_relation_async_routing (line 150) | def test_relation_async_routing
method test_relation_async_job_options (line 159) | def test_relation_async_job_options
method test_relation_queue (line 166) | def test_relation_queue
method test_relation_queue_all (line 184) | def test_relation_queue_all
method test_relation_queue_routing (line 202) | def test_relation_queue_routing
method test_relation_index (line 218) | def test_relation_index
method test_full_async (line 225) | def test_full_async
method test_full_async_should_index (line 245) | def test_full_async_should_index
method test_full_async_wait (line 259) | def test_full_async_wait
method test_full_async_job_options (line 271) | def test_full_async_job_options
method test_full_async_non_integer_pk (line 279) | def test_full_async_non_integer_pk
method test_full_queue (line 296) | def test_full_queue
method test_full_refresh_interval (line 303) | def test_full_refresh_interval
method test_full_resume (line 314) | def test_full_resume
method test_full_refresh (line 327) | def test_full_refresh
method test_full_partial_async (line 331) | def test_full_partial_async
method test_wait_not_async (line 337) | def test_wait_not_async
method test_object_index (line 344) | def test_object_index
method test_transaction (line 351) | def test_transaction
method test_both_paths (line 361) | def test_both_paths
FILE: test/reindex_v2_job_test.rb
class ReindexV2JobTest (line 3) | class ReindexV2JobTest < Minitest::Test
method test_create (line 4) | def test_create
method test_destroy (line 13) | def test_destroy
FILE: test/relation_test.rb
class RelationTest (line 3) | class RelationTest < Minitest::Test
method test_loaded (line 4) | def test_loaded
method test_mutating (line 19) | def test_mutating
method test_non_mutating (line 26) | def test_non_mutating
method test_load (line 33) | def test_load
method test_clone (line 40) | def test_clone
method test_only (line 46) | def test_only
method test_except (line 50) | def test_except
method test_first (line 54) | def test_first
method test_first_loaded (line 62) | def test_first_loaded
method test_pluck (line 70) | def test_pluck
method test_model (line 76) | def test_model
method test_klass (line 81) | def test_klass
method test_respond_to (line 86) | def test_respond_to
method test_inspect (line 95) | def test_inspect
method test_to_yaml (line 120) | def test_to_yaml
FILE: test/results_test.rb
class ResultsTest (line 3) | class ResultsTest < Minitest::Test
method test_array_methods (line 4) | def test_array_methods
method test_with_hit (line 20) | def test_with_hit
method test_with_score (line 34) | def test_with_score
method test_model_name_with_model (line 48) | def test_model_name_with_model
method test_model_name_without_model (line 54) | def test_model_name_without_model
FILE: test/routing_test.rb
class RoutingTest (line 3) | class RoutingTest < Minitest::Test
method test_query (line 4) | def test_query
method test_mappings (line 9) | def test_mappings
method test_correct_node (line 14) | def test_correct_node
method test_incorrect_node (line 19) | def test_incorrect_node
method test_async (line 24) | def test_async
method test_queue (line 31) | def test_queue
FILE: test/scroll_test.rb
class ScrollTest (line 3) | class ScrollTest < Minitest::Test
method test_works (line 4) | def test_works
method test_body (line 28) | def test_body
method test_all (line 52) | def test_all
method test_all_relation (line 57) | def test_all_relation
method test_no_option (line 62) | def test_no_option
method test_block (line 70) | def test_block
method test_block_relation (line 80) | def test_block_relation
FILE: test/search_synonyms_test.rb
class SearchSynonymsTest (line 3) | class SearchSynonymsTest < Minitest::Test
method setup (line 4) | def setup
method test_bleach (line 9) | def test_bleach
method test_burger_buns (line 14) | def test_burger_buns
method test_bandaids (line 19) | def test_bandaids
method test_reverse (line 24) | def test_reverse
method test_not_stemmed (line 29) | def test_not_stemmed
method test_word_start (line 35) | def test_word_start
method test_directional (line 40) | def test_directional
method test_case (line 48) | def test_case
method test_multiple_words (line 53) | def test_multiple_words
method test_multiple_words_expanded (line 60) | def test_multiple_words_expanded
method test_reload_synonyms (line 67) | def test_reload_synonyms
method test_reload_synonyms_better (line 71) | def test_reload_synonyms_better
method write_synonyms (line 90) | def write_synonyms(contents)
method default_model (line 94) | def default_model
FILE: test/search_test.rb
class SearchTest (line 3) | class SearchTest < Minitest::Test
method test_search_relation (line 4) | def test_search_relation
method test_unscoped (line 11) | def test_unscoped
method test_body (line 30) | def test_body
method test_body_incompatible_options (line 35) | def test_body_incompatible_options
method test_block (line 41) | def test_block
method test_missing_records (line 50) | def test_missing_records
method test_bad_mapping (line 65) | def test_bad_mapping
method test_missing_index (line 74) | def test_missing_index
method test_invalid_body (line 78) | def test_invalid_body
FILE: test/select_test.rb
class SelectTest (line 3) | class SelectTest < Minitest::Test
method test_basic (line 4) | def test_basic
method test_relation (line 12) | def test_relation
method test_block (line 20) | def test_block
method test_block_arguments (line 25) | def test_block_arguments
method test_multiple (line 32) | def test_multiple
method test_reselect (line 40) | def test_reselect
method test_array (line 47) | def test_array
method test_single_field (line 53) | def test_single_field
method test_all (line 61) | def test_all
method test_none (line 68) | def test_none
method test_includes (line 76) | def test_includes
method test_excludes (line 84) | def test_excludes
method test_include_and_excludes (line 92) | def test_include_and_excludes
FILE: test/should_index_test.rb
class ShouldIndexTest (line 3) | class ShouldIndexTest < Minitest::Test
method test_basic (line 4) | def test_basic
method test_default_true (line 9) | def test_default_true
method test_change_to_true (line 13) | def test_change_to_true
method test_change_to_false (line 23) | def test_change_to_false
method test_bulk (line 33) | def test_bulk
FILE: test/similar_test.rb
class SimilarTest (line 3) | class SimilarTest < Minitest::Test
method test_similar (line 4) | def test_similar
method test_fields (line 9) | def test_fields
method test_order (line 15) | def test_order
method test_limit (line 20) | def test_limit
method test_per_page (line 27) | def test_per_page
method test_routing (line 34) | def test_routing
FILE: test/suggest_test.rb
class SuggestTest (line 3) | class SuggestTest < Minitest::Test
method setup (line 4) | def setup
method test_basic (line 9) | def test_basic
method test_perfect (line 14) | def test_perfect
method test_phrase (line 19) | def test_phrase
method test_empty (line 24) | def test_empty
method test_without_option (line 28) | def test_without_option
method test_multiple_fields (line 33) | def test_multiple_fields
method test_multiple_fields_highest_score_first (line 41) | def test_multiple_fields_highest_score_first
method test_multiple_fields_same_value (line 48) | def test_multiple_fields_same_value
method test_fields_option (line 55) | def test_fields_option
method test_fields_option_multiple (line 62) | def test_fields_option_multiple
method test_fields_partial_match (line 69) | def test_fields_partial_match
method test_fields_partial_match_boost (line 74) | def test_fields_partial_match_boost
method test_multiple_models (line 79) | def test_multiple_models
method test_multiple_models_no_fields (line 85) | def test_multiple_models_no_fields
method test_star (line 90) | def test_star
method assert_suggest (line 96) | def assert_suggest(term, expected, options = {})
method assert_suggest_all (line 106) | def assert_suggest_all(term, expected, options = {})
FILE: test/support/activerecord.rb
class Product (line 76) | class Product < ActiveRecord::Base
class Store (line 85) | class Store < ActiveRecord::Base
class Region (line 89) | class Region < ActiveRecord::Base
class Speaker (line 92) | class Speaker < ActiveRecord::Base
class Animal (line 95) | class Animal < ActiveRecord::Base
class Dog (line 98) | class Dog < Animal
class Cat (line 101) | class Cat < Animal
class Sku (line 104) | class Sku < ActiveRecord::Base
class Song (line 107) | class Song < ActiveRecord::Base
class Band (line 110) | class Band < ActiveRecord::Base
class Artist (line 114) | class Artist < ActiveRecord::Base
FILE: test/support/apartment.rb
type Rails (line 1) | module Rails
function env (line 2) | def self.env
class Tenant (line 14) | class Tenant < ActiveRecord::Base
FILE: test/support/helpers.rb
class Minitest::Test (line 1) | class Minitest::Test
method setup (line 4) | def setup
method setup_animal (line 12) | def setup_animal
method setup_region (line 16) | def setup_region
method setup_speaker (line 20) | def setup_speaker
method setup_model (line 24) | def setup_model(model)
method store (line 34) | def store(documents, model = default_model, reindex: true)
method store_names (line 53) | def store_names(names, model = default_model, reindex: true)
method assert_search (line 58) | def assert_search(term, expected, options = {}, model = default_model)
method assert_search_relation (line 63) | def assert_search_relation(expected, relation)
method assert_order (line 67) | def assert_order(term, expected, options = {}, model = default_model)
method assert_order_relation (line 72) | def assert_order_relation(expected, relation)
method assert_equal_scores (line 76) | def assert_equal_scores(term, options = {}, model = default_model)
method assert_first (line 80) | def assert_first(term, expected, options = {}, model = default_model)
method assert_warns (line 84) | def assert_warns(message)
method build_relation (line 91) | def build_relation(model, term, **options)
method with_options (line 99) | def with_options(options, model = default_model)
method with_callbacks (line 113) | def with_callbacks(value, &block)
method with_transaction (line 121) | def with_transaction(model, &block)
method activerecord? (line 129) | def activerecord?
method mongoid? (line 133) | def mongoid?
method default_model (line 137) | def default_model
method ci? (line 141) | def ci?
method tagged_logger (line 146) | def tagged_logger
FILE: test/support/mongoid.rb
class Product (line 8) | class Product
class Store (line 30) | class Store
class Region (line 37) | class Region
class Speaker (line 44) | class Speaker
class Animal (line 50) | class Animal
class Dog (line 56) | class Dog < Animal
class Cat (line 59) | class Cat < Animal
class Sku (line 62) | class Sku
class Song (line 68) | class Song
class Band (line 74) | class Band
class Artist (line 83) | class Artist
FILE: test/support/redis.rb
type RedisInstrumentation (line 13) | module RedisInstrumentation
function call (line 14) | def call(command, redis_config)
function call_pipelined (line 19) | def call_pipelined(commands, redis_config)
FILE: test/synonyms_test.rb
class SynonymsTest (line 3) | class SynonymsTest < Minitest::Test
method test_bleach (line 4) | def test_bleach
method test_burger_buns (line 9) | def test_burger_buns
method test_bandaids (line 14) | def test_bandaids
method test_reverse (line 19) | def test_reverse
method test_stemmed (line 24) | def test_stemmed
method test_word_start (line 29) | def test_word_start
method test_directional (line 34) | def test_directional
method test_case (line 42) | def test_case
FILE: test/unscope_test.rb
class UnscopeTest (line 3) | class UnscopeTest < Minitest::Test
method setup (line 4) | def setup
method test_reindex (line 10) | def test_reindex
method test_relation_async (line 18) | def test_relation_async
method create_records (line 29) | def create_records
method default_model (line 37) | def default_model
FILE: test/where_test.rb
class WhereTest (line 3) | class WhereTest < Minitest::Test
method test_where (line 4) | def test_where
method test_relation (line 79) | def test_relation
method test_string_operators (line 107) | def test_string_operators
method test_unknown_operator (line 114) | def test_unknown_operator
method test_regexp (line 121) | def test_regexp
method test_alternate_regexp (line 126) | def test_alternate_regexp
method test_special_regexp (line 131) | def test_special_regexp
method test_regexp_not_anchored (line 136) | def test_regexp_not_anchored
method test_regexp_anchored (line 144) | def test_regexp_anchored
method test_regexp_case (line 153) | def test_regexp_case
method test_prefix (line 159) | def test_prefix
method test_exists (line 164) | def test_exists
method test_like (line 177) | def test_like
method test_like_escape (line 187) | def test_like_escape
method test_like_special_characters (line 192) | def test_like_special_characters
method test_like_optional_operators (line 209) | def test_like_optional_operators
method test_ilike (line 216) | def test_ilike
method test_ilike_escape (line 226) | def test_ilike_escape
method test_ilike_special_characters (line 231) | def test_ilike_special_characters
method test_ilike_optional_operators (line 236) | def test_ilike_optional_operators
method test_script (line 243) | def test_script
method test_script_string (line 253) | def test_script_string
method test_string (line 260) | def test_string
method test_nil (line 267) | def test_nil
method test_id (line 275) | def test_id
method test_empty (line 281) | def test_empty
method test_empty_array (line 286) | def test_empty_array
method test_range_array (line 293) | def test_range_array
method test_range_array_again (line 302) | def test_range_array_again
method test_near (line 310) | def test_near
method test_near_hash (line 318) | def test_near_hash
method test_near_within (line 326) | def test_near_within
method test_near_within_hash (line 335) | def test_near_within_hash
method test_geo_polygon (line 344) | def test_geo_polygon
method test_top_left_bottom_right (line 362) | def test_top_left_bottom_right
method test_top_left_bottom_right_hash (line 370) | def test_top_left_bottom_right_hash
method test_top_right_bottom_left (line 378) | def test_top_right_bottom_left
method test_top_right_bottom_left_hash (line 386) | def test_top_right_bottom_left_hash
method test_multiple_locations (line 394) | def test_multiple_locations
method test_multiple_locations_with_term_filter (line 402) | def test_multiple_locations_with_term_filter
method test_multiple_locations_hash (line 411) | def test_multiple_locations_hash
method test_nested (line 419) | def test_nested
Condensed preview — 114 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (465K chars).
[
{
"path": ".github/ISSUE_TEMPLATE/bug_report.md",
"chars": 1216,
"preview": "---\nname: Bug report\nabout: Create a report to help us improve\ntitle: ''\nlabels: bug report\nassignees: ''\n\n---\n\n**First*"
},
{
"path": ".github/ISSUE_TEMPLATE/config.yml",
"chars": 162,
"preview": "blank_issues_enabled: false\ncontact_links:\n - name: Help\n url: https://stackoverflow.com/questions/tagged/searchkick"
},
{
"path": ".github/ISSUE_TEMPLATE/feature_request.md",
"chars": 431,
"preview": "---\nname: Feature request\nabout: Suggest an idea for this project\ntitle: ''\nlabels: feature request\nassignees: ''\n\n---\n\n"
},
{
"path": ".github/pull_request_template.md",
"chars": 226,
"preview": "Thanks for contributing. You’re awesome! A few things to keep in mind:\n\n- Keep changes to a minimum\n- Follow the existin"
},
{
"path": ".github/workflows/build.yml",
"chars": 2398,
"preview": "name: build\non: [push, pull_request]\njobs:\n build:\n strategy:\n fail-fast: false\n matrix:\n include:\n"
},
{
"path": ".gitignore",
"chars": 189,
"preview": "*.gem\n*.rbc\n.bundle\n.config\n.yardoc\n*.lock\nInstalledFiles\n_yardoc\ncoverage\ndoc/\nlib/bundler/man\npkg\nrdoc\nspec/reports\nte"
},
{
"path": "CHANGELOG.md",
"chars": 25309,
"preview": "## 6.1.1 (unreleased)\n\n- Fixed smart aggs behavior with `_and`\n\n## 6.1.0 (2026-02-18)\n\n- Added `per` method\n- Fixed erro"
},
{
"path": "Gemfile",
"chars": 430,
"preview": "source \"https://rubygems.org\"\n\ngemspec\n\ngem \"rake\"\ngem \"minitest\"\ngem \"sqlite3\", platform: :ruby\ngem \"sqlite3-ffi\", plat"
},
{
"path": "LICENSE.txt",
"chars": 1073,
"preview": "Copyright (c) 2013-2026 Andrew Kane\n\nMIT License\n\nPermission is hereby granted, free of charge, to any person obtaining\n"
},
{
"path": "README.md",
"chars": 53923,
"preview": "# Searchkick\n\n:rocket: Intelligent search made easy\n\n**Searchkick learns what your users are looking for.** As more peop"
},
{
"path": "Rakefile",
"chars": 235,
"preview": "require \"bundler/gem_tasks\"\nrequire \"rake/testtask\"\n\nRake::TestTask.new do |t|\n t.pattern = \"test/**/*_test.rb\"\nend\n\nta"
},
{
"path": "benchmark/Gemfile",
"chars": 384,
"preview": "source \"https://rubygems.org\"\n\ngemspec path: \"../\"\n\ngem \"sqlite3\"\ngem \"pg\"\ngem \"activerecord\", \"~> 8.0.0\"\ngem \"activejob"
},
{
"path": "benchmark/index.rb",
"chars": 2632,
"preview": "require \"bundler/setup\"\nBundler.require(:default)\nrequire \"active_record\"\nrequire \"active_job\"\nrequire \"benchmark\"\nrequi"
},
{
"path": "benchmark/relation.rb",
"chars": 366,
"preview": "require \"bundler/setup\"\nBundler.require(:default)\nrequire \"active_record\"\n\nclass Product < ActiveRecord::Base\n searchki"
},
{
"path": "benchmark/search.rb",
"chars": 1106,
"preview": "require \"bundler/setup\"\nBundler.require(:default)\nrequire \"active_record\"\nrequire \"benchmark/ips\"\n\nActiveRecord.default_"
},
{
"path": "examples/Gemfile",
"chars": 142,
"preview": "source \"https://rubygems.org\"\n\ngemspec path: \"..\"\n\ngem \"activerecord\"\ngem \"elasticsearch\"\ngem \"informers\"\ngem \"opensearc"
},
{
"path": "examples/hybrid.rb",
"chars": 1844,
"preview": "require \"bundler/setup\"\nrequire \"active_record\"\nrequire \"elasticsearch\" # or \"opensearch-ruby\"\nrequire \"informers\"\nrequi"
},
{
"path": "examples/semantic.rb",
"chars": 1307,
"preview": "require \"bundler/setup\"\nrequire \"active_record\"\nrequire \"elasticsearch\" # or \"opensearch-ruby\"\nrequire \"informers\"\nrequi"
},
{
"path": "gemfiles/activerecord72.gemfile",
"chars": 306,
"preview": "source \"https://rubygems.org\"\n\ngemspec path: \"..\"\n\ngem \"rake\"\ngem \"minitest\"\ngem \"sqlite3\"\ngem \"activerecord\", \"~> 7.2.0"
},
{
"path": "gemfiles/activerecord80.gemfile",
"chars": 306,
"preview": "source \"https://rubygems.org\"\n\ngemspec path: \"..\"\n\ngem \"rake\"\ngem \"minitest\"\ngem \"sqlite3\"\ngem \"activerecord\", \"~> 8.0.0"
},
{
"path": "gemfiles/mongoid8.gemfile",
"chars": 258,
"preview": "source \"https://rubygems.org\"\n\ngemspec path: \"..\"\n\ngem \"rake\"\ngem \"minitest\"\ngem \"mongoid\", \"~> 8\"\ngem \"activejob\", requ"
},
{
"path": "gemfiles/mongoid9.gemfile",
"chars": 258,
"preview": "source \"https://rubygems.org\"\n\ngemspec path: \"..\"\n\ngem \"rake\"\ngem \"minitest\"\ngem \"mongoid\", \"~> 9\"\ngem \"activejob\", requ"
},
{
"path": "gemfiles/opensearch2.gemfile",
"chars": 344,
"preview": "source \"https://rubygems.org\"\n\ngemspec path: \"..\"\n\ngem \"rake\"\ngem \"minitest\"\ngem \"sqlite3\"\ngem \"activerecord\", \"~> 7.2.0"
},
{
"path": "gemfiles/opensearch3.gemfile",
"chars": 344,
"preview": "source \"https://rubygems.org\"\n\ngemspec path: \"..\"\n\ngem \"rake\"\ngem \"minitest\"\ngem \"sqlite3\"\ngem \"activerecord\", \"~> 8.0.0"
},
{
"path": "lib/searchkick/bulk_reindex_job.rb",
"chars": 810,
"preview": "module Searchkick\n class BulkReindexJob < Searchkick.parent_job.constantize\n queue_as { Searchkick.queue_name }\n\n "
},
{
"path": "lib/searchkick/controller_runtime.rb",
"chars": 1240,
"preview": "# based on https://gist.github.com/mnutt/566725\nmodule Searchkick\n module ControllerRuntime\n extend ActiveSupport::C"
},
{
"path": "lib/searchkick/hash_wrapper.rb",
"chars": 797,
"preview": "module Searchkick\n class HashWrapper\n def initialize(attributes)\n @attributes = attributes\n end\n\n def []("
},
{
"path": "lib/searchkick/index.rb",
"chars": 14153,
"preview": "module Searchkick\n class Index\n attr_reader :name, :options\n\n def initialize(name, options = {})\n @name = na"
},
{
"path": "lib/searchkick/index_cache.rb",
"chars": 609,
"preview": "module Searchkick\n class IndexCache\n def initialize(max_size: 20)\n @data = {}\n @mutex = Mutex.new\n @m"
},
{
"path": "lib/searchkick/index_options.rb",
"chars": 21686,
"preview": "module Searchkick\n class IndexOptions\n attr_reader :options\n\n def initialize(index)\n @options = index.option"
},
{
"path": "lib/searchkick/indexer.rb",
"chars": 1190,
"preview": "# thread-local (technically fiber-local) indexer\n# used to aggregate bulk callbacks across models\nmodule Searchkick\n cl"
},
{
"path": "lib/searchkick/log_subscriber.rb",
"chars": 1706,
"preview": "# based on https://gist.github.com/mnutt/566725\nmodule Searchkick\n class LogSubscriber < ActiveSupport::LogSubscriber\n "
},
{
"path": "lib/searchkick/middleware.rb",
"chars": 611,
"preview": "require \"faraday\"\n\nmodule Searchkick\n class Middleware < Faraday::Middleware\n def call(env)\n path = env[:url].p"
},
{
"path": "lib/searchkick/model.rb",
"chars": 5442,
"preview": "module Searchkick\n module Model\n def searchkick(**options)\n options = Searchkick.model_options.deep_merge(optio"
},
{
"path": "lib/searchkick/multi_search.rb",
"chars": 1089,
"preview": "module Searchkick\n class MultiSearch\n attr_reader :queries\n\n def initialize(queries, opaque_id: nil)\n @queri"
},
{
"path": "lib/searchkick/process_batch_job.rb",
"chars": 633,
"preview": "module Searchkick\n class ProcessBatchJob < Searchkick.parent_job.constantize\n queue_as { Searchkick.queue_name }\n\n "
},
{
"path": "lib/searchkick/process_queue_job.rb",
"chars": 1127,
"preview": "module Searchkick\n class ProcessQueueJob < Searchkick.parent_job.constantize\n queue_as { Searchkick.queue_name }\n\n "
},
{
"path": "lib/searchkick/query.rb",
"chars": 45443,
"preview": "module Searchkick\n class Query\n include Enumerable\n extend Forwardable\n\n @@metric_aggs = [:avg, :cardinality, "
},
{
"path": "lib/searchkick/railtie.rb",
"chars": 122,
"preview": "module Searchkick\n class Railtie < Rails::Railtie\n rake_tasks do\n load \"tasks/searchkick.rake\"\n end\n end\nen"
},
{
"path": "lib/searchkick/record_data.rb",
"chars": 3771,
"preview": "module Searchkick\n class RecordData\n TYPE_KEYS = [\"type\", :type]\n\n attr_reader :index, :record\n\n def initializ"
},
{
"path": "lib/searchkick/record_indexer.rb",
"chars": 5150,
"preview": "module Searchkick\n class RecordIndexer\n attr_reader :index\n\n def initialize(index)\n @index = index\n end\n\n"
},
{
"path": "lib/searchkick/reindex_queue.rb",
"chars": 1289,
"preview": "module Searchkick\n class ReindexQueue\n attr_reader :name\n\n def initialize(name)\n @name = name\n\n raise E"
},
{
"path": "lib/searchkick/reindex_v2_job.rb",
"chars": 798,
"preview": "module Searchkick\n class ReindexV2Job < Searchkick.parent_job.constantize\n queue_as { Searchkick.queue_name }\n\n d"
},
{
"path": "lib/searchkick/relation.rb",
"chars": 13910,
"preview": "module Searchkick\n class Relation\n NO_DEFAULT_VALUE = Object.new\n\n # note: modifying body directly is not support"
},
{
"path": "lib/searchkick/relation_indexer.rb",
"chars": 5840,
"preview": "module Searchkick\n class RelationIndexer\n attr_reader :index\n\n def initialize(index)\n @index = index\n end"
},
{
"path": "lib/searchkick/reranking.rb",
"chars": 637,
"preview": "module Searchkick\n module Reranking\n def self.rrf(first_ranking, *rankings, k: 60)\n rankings.unshift(first_rank"
},
{
"path": "lib/searchkick/results.rb",
"chars": 9617,
"preview": "module Searchkick\n class Results\n include Enumerable\n extend Forwardable\n\n attr_reader :response\n\n def_dele"
},
{
"path": "lib/searchkick/script.rb",
"chars": 214,
"preview": "module Searchkick\n class Script\n attr_reader :source, :lang, :params\n\n def initialize(source, lang: \"painless\", p"
},
{
"path": "lib/searchkick/version.rb",
"chars": 42,
"preview": "module Searchkick\n VERSION = \"6.1.0\"\nend\n"
},
{
"path": "lib/searchkick/where.rb",
"chars": 169,
"preview": "module Searchkick\n class Where\n def initialize(relation)\n @relation = relation\n end\n\n def not(value)\n "
},
{
"path": "lib/searchkick.rb",
"chars": 11137,
"preview": "# dependencies\nrequire \"active_support\"\nrequire \"active_support/core_ext/hash/deep_merge\"\nrequire \"active_support/core_e"
},
{
"path": "lib/tasks/searchkick.rake",
"chars": 980,
"preview": "namespace :searchkick do\n desc \"reindex a model (specify CLASS)\"\n task reindex: :environment do\n class_name = ENV[\""
},
{
"path": "searchkick.gemspec",
"chars": 601,
"preview": "require_relative \"lib/searchkick/version\"\n\nGem::Specification.new do |spec|\n spec.name = \"searchkick\"\n spec.v"
},
{
"path": "test/aggs_test.rb",
"chars": 14911,
"preview": "require_relative \"test_helper\"\n\nclass AggsTest < Minitest::Test\n def setup\n super\n store [\n {name: \"Product "
},
{
"path": "test/boost_test.rb",
"chars": 6557,
"preview": "require_relative \"test_helper\"\n\nclass BoostTest < Minitest::Test\n # global boost\n\n def test_boost\n store [\n {n"
},
{
"path": "test/callbacks_test.rb",
"chars": 2687,
"preview": "require_relative \"test_helper\"\n\nclass CallbacksTest < Minitest::Test\n def test_false\n Searchkick.callbacks(false) do"
},
{
"path": "test/conversions_test.rb",
"chars": 8751,
"preview": "require_relative \"test_helper\"\n\nclass ConversionsTest < Minitest::Test\n def setup\n super\n setup_speaker\n end\n\n "
},
{
"path": "test/default_scope_test.rb",
"chars": 590,
"preview": "require_relative \"test_helper\"\n\nclass DefaultScopeTest < Minitest::Test\n def setup\n setup_model(Band)\n end\n\n def t"
},
{
"path": "test/exclude_test.rb",
"chars": 1339,
"preview": "require_relative \"test_helper\"\n\nclass ExcludeTest < Minitest::Test\n def test_butter\n store_names [\"Butter Tub\", \"Pea"
},
{
"path": "test/geo_shape_test.rb",
"chars": 3046,
"preview": "require_relative \"test_helper\"\n\nclass GeoShapeTest < Minitest::Test\n def setup\n setup_region\n store [\n {\n "
},
{
"path": "test/highlight_test.rb",
"chars": 4462,
"preview": "require_relative \"test_helper\"\n\nclass HighlightTest < Minitest::Test\n def test_basic\n store_names [\"Two Door Cinema "
},
{
"path": "test/hybrid_test.rb",
"chars": 1160,
"preview": "require_relative \"test_helper\"\n\nclass HybridTest < Minitest::Test\n def setup\n skip unless Searchkick.knn_support?\n "
},
{
"path": "test/index_cache_test.rb",
"chars": 1244,
"preview": "require_relative \"test_helper\"\n\nclass IndexCacheTest < Minitest::Test\n def setup\n Product.class_variable_get(:@@sear"
},
{
"path": "test/index_options_test.rb",
"chars": 3189,
"preview": "require_relative \"test_helper\"\n\nclass IndexOptionsTest < Minitest::Test\n def setup\n Song.destroy_all\n end\n\n def te"
},
{
"path": "test/index_test.rb",
"chars": 4607,
"preview": "require_relative \"test_helper\"\n\nclass IndexTest < Minitest::Test\n def setup\n super\n setup_region\n end\n\n def tes"
},
{
"path": "test/inheritance_test.rb",
"chars": 4663,
"preview": "require_relative \"test_helper\"\n\nclass InheritanceTest < Minitest::Test\n def setup\n super\n setup_animal\n end\n\n d"
},
{
"path": "test/knn_test.rb",
"chars": 8722,
"preview": "require_relative \"test_helper\"\n\nclass KnnTest < Minitest::Test\n def setup\n skip unless Searchkick.knn_support?\n s"
},
{
"path": "test/language_test.rb",
"chars": 4078,
"preview": "require_relative \"test_helper\"\n\nclass LanguageTest < Minitest::Test\n def setup\n skip \"Requires plugin\" unless ci? ||"
},
{
"path": "test/load_test.rb",
"chars": 3180,
"preview": "require_relative \"test_helper\"\n\nclass LoadTest < Minitest::Test\n def test_default\n store_names [\"Product A\"]\n pro"
},
{
"path": "test/log_subscriber_test.rb",
"chars": 2246,
"preview": "require_relative \"test_helper\"\n\nclass LogSubscriberTest < Minitest::Test\n def test_create\n output = capture_logs do\n"
},
{
"path": "test/marshal_test.rb",
"chars": 340,
"preview": "require_relative \"test_helper\"\n\nclass MarshalTest < Minitest::Test\n def test_marshal\n store_names [\"Product A\"]\n "
},
{
"path": "test/match_test.rb",
"chars": 8891,
"preview": "require_relative \"test_helper\"\n\nclass MatchTest < Minitest::Test\n # exact\n\n def test_match\n store_names [\"Whole Mil"
},
{
"path": "test/misspellings_test.rb",
"chars": 3742,
"preview": "require_relative \"test_helper\"\n\nclass MisspellingsTest < Minitest::Test\n def test_false\n store_names [\"abc\", \"abd\", "
},
{
"path": "test/models/animal.rb",
"chars": 101,
"preview": "class Animal\n searchkick \\\n inheritance: true,\n text_start: [:name],\n suggest: [:name]\nend\n"
},
{
"path": "test/models/artist.rb",
"chars": 88,
"preview": "class Artist\n searchkick unscope: true\n\n def should_index?\n should_index\n end\nend\n"
},
{
"path": "test/models/band.rb",
"chars": 28,
"preview": "class Band\n searchkick\nend\n"
},
{
"path": "test/models/product.rb",
"chars": 1671,
"preview": "class Product\n searchkick \\\n synonyms: [\n [\"clorox\", \"bleach\"],\n [\"burger\", \"hamburger\"],\n [\"bandaid\""
},
{
"path": "test/models/region.rb",
"chars": 188,
"preview": "class Region\n searchkick \\\n geo_shape: [:territory]\n\n attr_accessor :territory\n\n def search_data\n {\n name:"
},
{
"path": "test/models/sku.rb",
"chars": 45,
"preview": "class Sku\n searchkick callbacks: :async\nend\n"
},
{
"path": "test/models/song.rb",
"chars": 65,
"preview": "class Song\n searchkick\n\n def search_routing\n name\n end\nend\n"
},
{
"path": "test/models/speaker.rb",
"chars": 602,
"preview": "class Speaker\n searchkick \\\n conversions_v1: [\"conversions_a\", \"conversions_b\"],\n search_synonyms: [\n [\"clor"
},
{
"path": "test/models/store.rb",
"chars": 246,
"preview": "class Store\n mappings = {\n properties: {\n name: {type: \"text\"}\n }\n }\n\n searchkick \\\n routing: true,\n "
},
{
"path": "test/multi_indices_test.rb",
"chars": 2285,
"preview": "require_relative \"test_helper\"\n\nclass MultiIndicesTest < Minitest::Test\n def setup\n super\n setup_speaker\n end\n\n "
},
{
"path": "test/multi_search_test.rb",
"chars": 1490,
"preview": "require_relative \"test_helper\"\n\nclass MultiSearchTest < Minitest::Test\n def test_basic\n store_names [\"Product A\"]\n "
},
{
"path": "test/multi_tenancy_test.rb",
"chars": 605,
"preview": "require_relative \"test_helper\"\n\nclass MultiTenancyTest < Minitest::Test\n def setup\n skip unless defined?(Apartment)\n"
},
{
"path": "test/notifications_test.rb",
"chars": 644,
"preview": "require_relative \"test_helper\"\n\nclass NotificationsTest < Minitest::Test\n def test_search\n Product.searchkick_index."
},
{
"path": "test/order_test.rb",
"chars": 2398,
"preview": "require_relative \"test_helper\"\n\nclass OrderTest < Minitest::Test\n def test_hash\n store_names [\"Product A\", \"Product "
},
{
"path": "test/pagination_test.rb",
"chars": 8741,
"preview": "require_relative \"test_helper\"\n\nclass PaginationTest < Minitest::Test\n def test_limit\n store_names [\"Product A\", \"Pr"
},
{
"path": "test/parameters_test.rb",
"chars": 4036,
"preview": "require_relative \"test_helper\"\n\nclass ParametersTest < Minitest::Test\n def setup\n require \"action_controller\"\n su"
},
{
"path": "test/partial_match_test.rb",
"chars": 2884,
"preview": "require_relative \"test_helper\"\n\nclass PartialMatchTest < Minitest::Test\n def test_autocomplete\n store_names [\"Hummus"
},
{
"path": "test/partial_reindex_test.rb",
"chars": 4836,
"preview": "require_relative \"test_helper\"\n\nclass PartialReindexTest < Minitest::Test\n def test_record_inline\n store [{name: \"Hi"
},
{
"path": "test/query_test.rb",
"chars": 3432,
"preview": "require_relative \"test_helper\"\n\nclass QueryTest < Minitest::Test\n def test_basic\n store_names [\"Milk\", \"Apple\"]\n "
},
{
"path": "test/reindex_test.rb",
"chars": 10682,
"preview": "require_relative \"test_helper\"\n\nclass ReindexTest < Minitest::Test\n def test_record_inline\n store_names [\"Product A\""
},
{
"path": "test/reindex_v2_job_test.rb",
"chars": 715,
"preview": "require_relative \"test_helper\"\n\nclass ReindexV2JobTest < Minitest::Test\n def test_create\n product = Searchkick.callb"
},
{
"path": "test/relation_test.rb",
"chars": 3707,
"preview": "require_relative \"test_helper\"\n\nclass RelationTest < Minitest::Test\n def test_loaded\n Product.searchkick_index.refre"
},
{
"path": "test/results_test.rb",
"chars": 1677,
"preview": "require_relative \"test_helper\"\n\nclass ResultsTest < Minitest::Test\n def test_array_methods\n store_names [\"Product A\""
},
{
"path": "test/routing_test.rb",
"chars": 1017,
"preview": "require_relative \"test_helper\"\n\nclass RoutingTest < Minitest::Test\n def test_query\n query = Store.search(\"Dollar Tre"
},
{
"path": "test/scroll_test.rb",
"chars": 2864,
"preview": "require_relative \"test_helper\"\n\nclass ScrollTest < Minitest::Test\n def test_works\n store_names [\"Product A\", \"Produc"
},
{
"path": "test/search_synonyms_test.rb",
"chars": 2453,
"preview": "require_relative \"test_helper\"\n\nclass SearchSynonymsTest < Minitest::Test\n def setup\n super\n setup_speaker\n end\n"
},
{
"path": "test/search_test.rb",
"chars": 2399,
"preview": "require_relative \"test_helper\"\n\nclass SearchTest < Minitest::Test\n def test_search_relation\n error = assert_raises(S"
},
{
"path": "test/select_test.rb",
"chars": 3890,
"preview": "require_relative \"test_helper\"\n\nclass SelectTest < Minitest::Test\n def test_basic\n store [{name: \"Product A\", store_"
},
{
"path": "test/should_index_test.rb",
"chars": 1013,
"preview": "require_relative \"test_helper\"\n\nclass ShouldIndexTest < Minitest::Test\n def test_basic\n store_names [\"INDEX\", \"DO NO"
},
{
"path": "test/similar_test.rb",
"chars": 1712,
"preview": "require_relative \"test_helper\"\n\nclass SimilarTest < Minitest::Test\n def test_similar\n store_names [\"Annie's Naturals"
},
{
"path": "test/suggest_test.rb",
"chars": 3105,
"preview": "require_relative \"test_helper\"\n\nclass SuggestTest < Minitest::Test\n def setup\n super\n Product.reindex\n end\n\n de"
},
{
"path": "test/support/activerecord.rb",
"chars": 2217,
"preview": "require \"active_record\"\n\n# for debugging\nActiveRecord::Base.logger = $logger\n\n# rails does this in activerecord/lib/acti"
},
{
"path": "test/support/apartment.rb",
"chars": 768,
"preview": "module Rails\n def self.env\n ENV[\"RACK_ENV\"]\n end\nend\n\ntenants = [\"tenant1\", \"tenant2\"]\nApartment.configure do |conf"
},
{
"path": "test/support/helpers.rb",
"chars": 3446,
"preview": "class Minitest::Test\n include ActiveJob::TestHelper\n\n def setup\n [Product, Store].each do |model|\n setup_model"
},
{
"path": "test/support/kaminari.yml",
"chars": 620,
"preview": "en:\n views:\n pagination:\n first: \"« First\"\n last: \"Last »\"\n previous: \"‹ Prev\"\n "
},
{
"path": "test/support/mongoid.rb",
"chars": 1557,
"preview": "Mongoid.logger = $logger\nMongo::Logger.logger = $logger if defined?(Mongo::Logger)\n\nMongoid.configure do |config|\n conf"
},
{
"path": "test/support/redis.rb",
"chars": 565,
"preview": "options = {}\noptions[:logger] = $logger if !defined?(RedisClient)\n\nSearchkick.redis =\n if !defined?(Redis)\n RedisCli"
},
{
"path": "test/synonyms_test.rb",
"chars": 1213,
"preview": "require_relative \"test_helper\"\n\nclass SynonymsTest < Minitest::Test\n def test_bleach\n store_names [\"Clorox Bleach\", "
},
{
"path": "test/test_helper.rb",
"chars": 1334,
"preview": "require \"bundler/setup\"\nBundler.require(:default)\nrequire \"minitest/autorun\"\nrequire \"active_support/notifications\"\n\nENV"
},
{
"path": "test/unscope_test.rb",
"chars": 803,
"preview": "require_relative \"test_helper\"\n\nclass UnscopeTest < Minitest::Test\n def setup\n @@once ||= Artist.reindex\n\n Artist"
},
{
"path": "test/where_test.rb",
"chars": 18815,
"preview": "require_relative \"test_helper\"\n\nclass WhereTest < Minitest::Test\n def test_where\n now = Time.now\n store [\n {"
}
]
About this extraction
This page contains the full source code of the ankane/searchkick GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 114 files (425.2 KB), approximately 117.6k tokens, and a symbol index with 1131 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.