[ { "path": ".github/CONTRIBUTING.md", "content": "## How to contribute to Tile38\n\nBefore getting starting with contributing, please know that we currently use [Tile38 Slack](https://tile38.com/slack) channel for casual questions and user chat.\n\n### Did you find a bug?\n\n- **Do not open up a GitHub issue if the bug is a security vulnerability in Tile38**. Sensitive security-related issues should be reported to [security@tile38.com](mailto:security@tile38.com).\n\n- **Ensure the bug was not already reported** by searching on GitHub under [Issues](https://github.com/tidwall/tile38/issues).\n- If you're unable to find an open issue addressing the problem, [open a new one](https://github.com/tidwall/tile38/issues/new). Be sure to include a **title and clear description**, as much relevant information as possible, and a **code sample** or an **executable test case** demonstrating the expected behavior that is not occurring.\n\n### Did you fix whitespace, format code, or make a purely cosmetic patch?\n\nChanges that are cosmetic in nature and do not add anything substantial to the stability, functionality, or testability of Tile38 will generally not be accepted.\n\n### Do you intend to add a new feature or change an existing one?\n\n- New features will probably not be approved without prior discussion. If you need a specialized feature, make sure to express your willingness to fund the work and maintenance.\n- Please do not open a pull request without filing an issue and/or discussing it with a maintainer beforehand.\n\nThanks!\n" }, { "path": ".github/ISSUE_TEMPLATE/bug_report.md", "content": "---\nname: Bug report\nabout: Create a report to help us improve\ntitle: ''\nlabels: ''\nassignees: ''\n\n---\n\n**Describe the bug**\nA clear and concise description of what the bug is.\n\n**To Reproduce**\nSteps to reproduce the behavior:\n\n**Expected behavior**\nA clear and concise description of what you expected to happen.\n\n**Logs**\nIf applicable, provide logs, panics, system messages to help explain your problem.\n\n**Operating System (please complete the following information):**\n - OS: [e.g. Linux / Windows / Mac OS]\n - CPU: [e.g. amd64 / arm64 / Apple Silicon / Intel]\n - Version: [e.g. 1.19.0]\n - Container: [e.g. Docker / None]\n\n**Additional context**\nAdd any other context about the problem here.\n" }, { "path": ".github/ISSUE_TEMPLATE/config.yml", "content": "blank_issues_enabled: true\ncontact_links:\n - name: Community Support\n url: https://tile38.com/slack/\n about: Please ask and answer questions here.\n - name: Documenation Issues\n url: https://github.com/tile38/tile38.github.io/issues\n about: Please documenation related issues here.\n \n" }, { "path": ".github/PULL_REQUEST_TEMPLATE.md", "content": "Please do not open a pull request without first filing an issue and/or discussing the feature directly with the project maintainer.\n\n### Please ensure you adhere to every item in this list\n\n- [ ] This PR was pre-approved by the project maintainer\n- [ ] I have self-reviewed the code\n- [ ] I have added all necessary tests\n\n### Describe your changes\n\nPlease provide detailed description of the changes.\n\n### Issue number and link\n\nPull request require a prior issue with discussion. \nInclude the issue number of link here.\n" }, { "path": ".github/workflows/main.yml", "content": "name: Go\n\non:\n push:\n branches: [master]\n pull_request:\n branches: [master]\n\njobs:\n build:\n name: Build\n runs-on: ubuntu-latest\n steps:\n - name: Set up Go 1.x\n uses: actions/setup-go@v2\n with:\n go-version: ^1.25\n\n - name: Check out code\n uses: actions/checkout@v2\n with:\n fetch-depth: 0\n\n - name: Test\n run: make test\n\n - name: Package\n run: make package\n\n - name: Docker push\n env:\n DOCKER_LOGIN: tidwall\n DOCKER_USER: tile38\n DOCKER_PASSWORD: ${{ secrets.DOCKER_PASSWORD }}\n run: ./scripts/docker-push.sh\n" }, { "path": ".gitignore", "content": ".DS_Store\ntile38-*\n!cmd/tile38-*\n*.test\ndata*/\ncoverage.out\npackages/\n\n# Ignore IDE folders\n.idea/\n.vscode/\n" }, { "path": "CHANGELOG.md", "content": "# Change Log\nAll notable changes to this project will be documented in this file.\nThis project adheres to [Semantic Versioning](http://semver.org/).\n\n## [1.37.0] = 2026-01-06\n### Added\n- #791: Add regexp to where expressions using '=~' (@TomDeVito)\n\n### Fixed\n- #793: Avoid NaN points and rects for insertions and searches (@krkeshav)\n- #789: Fix LineString feature encoding to use all points (@metaxasa)\n- 45496a0: Ensure strict resp clients for pubsub\n- f6e6fae: Ignore -o json flag for HELLO command\n\n### Updated\n- 653e7a0: Bumped github.com/eclipse/paho.mqtt.golang\n- 2a86b4b: Bumped golang.org/x/crypto\n\n## [1.36.5] = 2025-10-05\n### Added\n- 5462de9: Add -o flag to auto set client output to json or resp #779 (@huangpeizhi2018)\n\n### Fixed\n- 8c27bd7: Fix 'outside' detect not firing when 'cross' is present\n\n## [1.36.4] = 2025-10-03\n### Fixed\n- 8c9f56c: Fix leader hanging on sigterm #783 (@dobiadi)\n\n## [1.36.3] = 2025-09-26\n### Fixed\n- 684ad73: Fix panic, move pubq init cond #782 (@ayaIbrahimm)\n- cde0ef7: Do not throw Lua nil access error during scans\n\n## [1.36.2] = 2025-09-04\n### Fixed\n- 653aea6: Add GeoJSON \"properties\" member for Lua filtering\n- 70c244f: Make the String.match() function case insensitive\n\n## [1.36.1] = 2025-08-28\n### Updated\n- 53bed30: Update lock strategy for faster writes \n- 40f58b0: Kafka Endpoint Improvement #778 (@ifiok)\n- 82f4d24: Smaller vector tile sizes for polygons\n- 5b21c24: Auto clip geometries at low zoom levels\n\n## [1.36.0] = 2025-07-14\n### Added\n- 35bde95: Support for Cloudflare Queues endpoint #773 (@tobilg)\n- a2afb21: Vector tile support (Mapbox vector tiles) #775\n\n### Updated\n- 56c70a1: Bumped github.com/golang-jwt/jwt/v4\n- 86b698d: Updated Go dependencies\n- f95fcab: Go 1.25\n\n## [1.35.0] = 2025-06-16\n### Added\n- 4638279: Added NATS Jetstream acks, user credentials, and tls #770 (@VeryStrongFingers)\n\n### Updated\n- 3fe57a7; Use atomics for HEALTHZ command to remove contention.\n- 3085316: Expose the 'this' property to WHERE clauses.\n\n## [1.34.4] = 2025-05-08\n### Fixed\n- 977bf25: Fix issue with some startup flags not being read (@salilgupta1)\n\n### Updated\n- 86333cd: Bump golang.org/x/net #767\n\n## [1.34.3] = 2025-04-16\n### Fixed\n- dd98481: Fix channel message delay #763 (@txtsd)\n\n## [1.34.2] = 2025-04-01\n### Updated\n- a80eaf2: Upgrade to Go 1.24\n\n### Fixed\n- 556390e: Fix equality tests with WHERE clause for nested values\n- 42e17a1: Fix channel test that sometimes stalls\n\n### Security\n- 0ecf097: Bump golang.org/x/net #765\n\n## [1.34.1] = 2025-01-13\n### Security\n- 927f382: CVE-2024-45338 golang.org/x/net #762 (@tduong2049)\n- 07389d8: CVE-2024-45337 golang.org/x/crypto #760\n\n## [1.34.0] = 2024-12-09\n### Added \n- 459b3e6: Added fifo support for SQS webhooks #759 (@crankycookie)\n\n### Updated\n- bed590b: Upgrade to Alpine 3.20 #757 (@tduong2049)\n- 2b09508: Upgrade to Go 1.23\n\n## [1.33.4] = 2024-11-05\n### Fixed\n- aa1caa6: Use zero for undefined fields in where expressions #754 (@unendingblue)\n\n## [1.33.3] = 2024-09-29\n### Fixed\n- 2b080f4: Include field to INFO replication command output #752 (@Kilowhisky)\n\n## [1.33.2] = 2024-08-02\n### Fixed\n- 2e3eaa7: Remove extra quote in ROLE command with JSON output #749 (@Kilowhisky)\n\n## [1.33.1] = 2024-07-02\n### Fixed\n- 193bce1: Fix followers not receiving channel messages #468 (@hibooboo2, @trendstate, @DoisKoh)\n\n## [1.33.0] = 2024-05-03\n## Added\n- #726: Add EXIST and FEXIST command (@Kilowhisky)\n\n## Fixed\n- #738: Add support for CORS in http requests (@Kilowhisky)\n- #741: FSET transforms field names to lowercase (@unendingblue, @iwpnd)\n- #736: Fix field floating point parsing misrepresentation (@Kilowhisky)\n\n## Updated\n- #733: golang.org/x/net\n- #724: google.golang.org/protobuf\n\n## [1.32.2] = 2024-02-14\n### Fixed\n- #714: Fix crash when mixing z-coord dimensionality in a geometry (@prathik)\n- #717: Metric expired_keys never incremented (@undeadcat)\n- Updated Go runtime to 1.22\n\n## [1.32.1] = 2023-11-20\n### Fixed\n- #711: Updated dependencies to address security vulnerabilities (@hcmf-wice)\n- #706: Add support for 'none' authentication for kafka while still allowing SSL (@Kilowhisky)\n- #702: Fix AWS SQS base domain parsing for China region (@LLluma)\n\n## [1.32.0] = 2023-07-31\n### Added\n- #686: Support the ROLE command (@Kilowhisky)\n\n### Fixed\n- #698: Allow AUTH while loading data (@Kilowhisky)\n- #694: Allow PING in pubsub (@Kilowhisky)\n- #692: Properly support replica_announce properties (@Kilowhisky)\n- #691: HEALTHZ should not be AUTH protected (@Kilowhisky)\n- #685: Heap size not coming down after objects are removed (@Mukund2900, @iwpnd)\n- 0144ca6: Fix missing lock\n\n## [1.31.0] = 2023-05-09\n### Added\n- #682: Enables cross platform building and pushing of docker images (arm64/amd64) (@eelcocramer)\n- #680: Add hostname, port, output, and password env variables to tile38-cli (@ptsilva)\n\n### Fixed\n- #606: Only create AMQP queue and bindings for non-topic exchanges (@pacaj2am, @uwer)\n- #672: Add graceful shutdown on SIGTERM (@dmitri-zganiaiko)\n\n### Updated\n- e9a0500: Upgrade to Go 1.20\n- 05b2fb9: Security updates\n\n## [1.30.2] = 2022-12-29\n### Fixed\n- #668: Fixed fields not persisting (@DucPhan2997)\n\n## [1.30.1] = 2022-12-14\n### Fixed\n- a8c92a0: Speed up leader/follower replication\n- e60ea70: Fix field names converting to lowercase\n\n## [1.30.0] = 2022-11-22\n### Added\n- bdc80a7: Add WHERE expressions ([more info](https://tile38.com/topics/filter-expressions))\n- f24c251: Allow for multiple MATCH patterns\n- #652: Allow WHERE for geofence detection\n- #657: Add distance to NEARBY IDS response (@iwpnd)\n- #663: Lua Sanitization (@program--)\n\n### Fixed\n- 023433a: Fix server hang on shared address\n- #655: fix: allow host ca sets for SASL and TLS connections (@iwpnd)\n\n### Updated\n- 7f2ce23: Upgrade to Go 1.19\n- cbfb271: Updated data structures to use Go generics\n\n## [1.29.2] = 2022-11-11\n### Fixed\n- #664: Fix bad line in inner ring response\n\n## [1.29.1] = 2022-09-21\n### Fixed\n- fe180dc: Fix follower not authenticating after aofshink\n\n## [1.29.0] = 2022-07-14\n### Added\n- b883f35: Add pending_events stat\n- #643: Expose config and INFO response for replia-priorty (@rave-eserating)\n\n### Fixed\n- 8e61f81: Fixed test on Apple silicon\n\n## [1.28.0] = 2022-04-12\n### Added\n- 10f8564: Added option to \"not found\" for DEL\n- #633: Added \"clear\" command in the tile38-cli (@CaioDallaqua)\n- #634: Added -x flag to tile38-cli (@sign0)\n\n### Fixed\n- #636: Workaround for the RESP3 Java lettuce client (@rave-eserating)\n- a1cc8e6: Fix eof error for incomplete commands (Theresa D)\n\n### Updated\n- fcdb469: Security updates\n- #638: Upgrade alpine in Dockerfile (@bb)\n- a124738: Upgrade to Go 1.18\n- 38ea913: Upgrade prometheous client\n- 45fde6a: Upgraded nats dependencies\n\n## [1.27.1] = 2021-01-04\n### Fix\n- b6833a2: Auto assign server_id for bootstrapped config files\n\n## [1.27.0] = 2021-12-28\n### Added\n- #629: JSON logging (@iwpnd)\n- 241117c: BUFFER option for WITHIN and INTERSECTS, see #79\n\n## [1.26.4] = 2021-10-25\n### Hotfix\n- a7592f7: Bump version to match changelog\n\n## [1.26.3] = 2021-10-25\n### Updated\n- a47443a: Upgrade tidwall modules\n\n## [1.26.2] = 2021-10-22\n### Added\n- #625: Azure EventHub hook support\n\n### Changed\n- 11cea4d: Removed vendor directory\n\n## [1.26.1] = 2021-10-01\n### Updated\n- 9e552c3: Allow some basic client commands before AOF data loads\n\n## [1.26.0] = 2021-09-29\n### Added\n- #623: Added SECTOR type to spatial searches (@iwpnd, @gmonk)\n\n### Fixed\n- #624: AOFSHRINK causes panic on server (@saques)\n\n## [1.25.5] = 2021-09-26\n### Fixed\n- 8ebcbeb: Fixed Z not matching on where clause for Feature/Point. (@tomquas)\n\n## [1.25.4] = 2021-09-14\n### Added\n- a737a78: Add unix socket support\n\n### Updated\n- 8829b8f: Change hooks collection type from hashmap to btree\n- 83094b2: Update hook expiration logic\n- c686b87: Return hook ttl with HOOKS request\n- 06a92d8: Increase the precision of TIMEOUT\n- Upgrade to Go 1.17.1\n\n## [1.25.3] = 2021-08-23\n### Fixed\n- #621: Fixed a memory leak (@Morgiflute)\n\n### Updated\n- Update B-tree library\n- Upgrade to Go 1.17\n\n## [1.25.2] = 2021-08-10\n### Fixed\n- #620: Fixed kafka authentication methods\n\n### Updated\n- Upgraded various dependencies\n\n## [1.25.1] = 2021-07-22\n### Fixed\n- #618: Fixed NEARBY with SPARSE returning too many results. (@nesjett)\n\n## [1.25.0] = 2021-07-12\n### Added\n- #504: Added TLS support for Nats webhook provider.\n- #552: Add CLIPBY subcommand to INTERSECTS/WITHIN. (@rshura)\n- #561: Added geofence webhook for GCP Pubsub. (@mscno)\n- #615: Add SASL to Kafka provider. (@mathieux51, @iwpnd)\n\n### Updated\n- #551: Optimize field value access. (@mpoindexter)\n- #554: Improved kNN using geodesic algorithm for NEARBY command. (@mpoindexter)\n\n### Fixed\n- #611: Close follower files before finishing aofshrink. (@mzbrau)\n- #613: Fix Memory Leak in Kafka Producer. (@iwpnd)\n- #616: Fixed expiration logic issue. (@Neuintown)\n\n## [1.24.3] = 2021-06-09\n### Fixed\n- af43d5a: Hotfix. Fixed invalid healthz output.\n\n## [1.24.2] = 2021-06-07\n### Updated\n- b610633: Update Go to 1.16\n\n## [1.24.1] = 2021-06-07\n### Added\n- #609: Added HEALTHZ command (@iwpnd, @stevelacy)\n\n## [1.24.0] = 2021-05-19\n### Added\n- #604: Added Prometheus metrics (@oliver006)\n\n### Fixed\n- #605: Remove deprecated threads flag (@cep-ter) \n\n## [1.23.0] = 2021-04-01\n### Updated\n- #598: Added TLS Config to Kafka (@iwpnd)\n- #599: Include \"distance\" to output when user specifically requests (@iwpnd)\n- #597: Allow for all command types for roaming event (@johnpmayer)\n- 31a0fbd: Upgraded dependencies and moved to Go 1.16\n\n### Fixed\n- #600: Fix invalid queue.db error (@lokisisland)\n- #603: Fix tile38-cli output showing protocol size when piping (@bb)\n\n## [1.22.6] = 2021-02-07\n### Updated\n- 72dfaae: Updated various dependencies\n- 016f397: Updated btree library, optimization \n- 4f8bc05: Updated rtree library, optimization\n\n### Fixed\n- 6092f73: Better handle connection errors in tile38-cli\n\n## [1.22.5] = 2020-11-09\n### Fixed\n- 9ce2033: Fixed fields being shuffled after AOFSHRINK\n\n## [1.22.4] = 2020-11-07\n### Updated\n- 1a7d8d6: Added ENV var for 500 http errors\n\n## [1.22.3] = 2020-10-28\n### Updated\n- #583: Optimization for non-\"cross\" based geofence detection (@cliedeman)\n- 79bee85: Replaced the underlying B-tree structure.\n\n## [1.22.2] = 2020-10-07\n### Fixed\n- #230: Fix trailing zeros in AOF at startup\n\n## [1.22.1] = 2020-09-22\n### Updated\n- 9a34a37: Updated Go version to 1.15\n- b1dc463: Updated outdated dependencies (40 in total)\n\n### Added\n- #578 Fix \"cross\" detection not firing in some cases (@feichler-or)\n\n## [1.22.0] = 2020-08-12\n### Added\n- #571 Added MONITOR command (@tomquas)\n\n### Fixed\n- #566: Fixed crash in fenceMatchRoam causing an index out of range panic (@larsw)\n- #569: Fixed wrong order for fields with SCAN (@ipsusila)\n- #573: Fixed crash with geohash precision above 12 (@superloach)\n- 68e2b6d: Updated Kafka client to support (@LeonardoBonacci)\n\n## [1.21.1] = 2020-06-04\n### Fixed\n- #564: Fix OUTPUT client command requiring authentication. (@LeonardoBonacci)\n\n## [1.20.0] = 2020-05-20\n### Updated\n- #534: Avoid sorting fields for each written object. (@rshura)\n- #544: Match geometry indexing to server config\n- b3dc025: Optimize point in ring\n- 3718cd7: Added priority option for AMQP endpoints\n\n### Fixed\n- #538: DEL geofence notifications are missing the \"key\" field\n- #539: Fixed issue with some features not working with WITHIN (@rshura)\n- #540: Fix a concurrent write/read on the server conn map (@mpoindexter)\n- #543: Fix clipping empty rings (@rshura)\n- #558: Fixed clip test (@mmcloughlin)\n- #562: Crashes under go1.14 runtime\n- ff48054: Fixed a missing faraway event for roaming geofences\n- 5162ac5: Stable sort roam notifications\n\n## [1.19.5] = 2020-02-11\n### Fixed\n- c567512: Fix packages not vendoring on build\n\n## [1.19.4] = 2020-02-10\n### Fixed\n- #529: Fix linestring features behave diffrent with CIRCLE (@spierepf)\n\n## [1.19.3] = 2019-12-11\n### Fixed\n- #513: Fix tile38-cli from freezing with non-quoted geojson (@duartejc)\n\n## [1.19.2] = 2019-11-28\n### Fixed\n- 6f3716a: Fix false negative for intersecting rings (@thomascoquet)\n\n## [1.19.1] = 2019-11-18\n### Updated\n- cfc65a1: Refactored repo, moved to Go modules, updated vendor dependencies.\n\n### Fixed\n- 9d27533: Fix infinite loop on tile38-cli connection failure.\n- #509: Fixed panic on AOFSHRINK. (@jordanferenz)\n\n## [1.19.0] = 2019-11-02\n### Added\n- #464: Add area expressions TEST command. (@rshura)\n\n### Fixed\n- #493: Fix invalid JSON when JSET strings that look like numbers. (@spierepf, @JordanArmstrong)\n- #499: Fix invalid PubSub format when output is set to JSON. (@dmvass)\n- #500: Fix Tile38-cli not propertly handling quotes. (@vthorsell)\n- #502: Fix excessive memory usage for objects with TTLs. commit 23b016d. (@FreakyBytes)\n- #503: Fix fprintf type error in stats_cpu.go for non-linux/darwin builds. (@JordanArmstrong)\n\n### Changed\n- #505: Update Travi-ci to use Go 1.13.x\n\n## [1.18.0] = 2019-10-09\n### Updated\n- 639f6e2: Updated the spatial index (R-tree) implementation.\n\n### Fixed\n- b092cea: Fixed MQTT blocking on publish/wait.\n- #496: Fixed MQTT client ID uniqueness. (@neterror)\n- #497: Fixed data race on webhook map with TTLs. (@belek)\n- #498: Fixed JSET cancels objects TTL expiry value. (@belek)\n\n## [1.17.6] - 2019-08-22\n### Fixed\n- 3d96b17: Fixed periodic stop-the-world pauses for systems with large heaps.\n\n## [1.17.5] - 2019-08-22\n### Fixed\n- #489: Fixed nearby count always one (@jkarjala)\n\n## [1.17.4] - 2019-08-09\n### Fixed\n- #486: Fixed data condition on connections map (@saltatory)\n\n## [1.17.3] - 2019-08-03\n### Fixed\n- #483: Fixed lua pool pruning (@rshura)\n- f7888c1: Fixed malformed json for chans command\n\n## [1.17.2] - 2019-06-28\n### Fixed\n- #422: Fixes NEARBY command distance normalization issue (@TrivikrAm-Pamarthi, @melbania)\n\n## [1.17.1] - 2019-05-04\n### Fixed\n- #448: Fixed missing commands for unsubscribing from active channel (@githubfr)\n- #454: Fixed colored output for fatalf (@olevole)\n- #453: Fixed nearby json field results showing wrong data (@melbania)\n\n## [1.17.0] - 2019-04-26\n### Added\n- #446: Added timeouts to allow prepending commands with a TIMEOUT option. (@rshura)\n\n### Fixed\n- #440: Fixed crash with fence ROAM (@githubfr)\n\n### Changed\n- 3ae5927: Removed experimental evio option\n\n## [1.16.4] - 2019-03-19\n### Fixed\n- e1a7145: Hotfix. Do not ignore SIGHUP. Use the `--nohup` flag or `nohup` command.\n\n## [1.16.3] - 2019-03-19\n### Fixed\n- #437: Fixed clients blocking while webook sending. (@tesujiro)\n\n### Added\n- #430: Support more SQS credential providers. (@tobilg)\n- #435: Added pprof flags for optional memory and cpu diagnostics.\n- e47540b: Added auth flag to tile38-benchmark.\n- 5335aec: Allow for standard SQS URLs. (@tobilg)\n\n## [1.16.2] - 2019-03-12\n### Fixed\n- #432: Ignore SIGHUP signals. (@abhit011)\n- #433: Fixed nearby inaccuracy with geofence. (@stcktrce)\n- #429: Memory optimization, recycle AOF buffer.\n- 95a5556: Added periodic yielding to iterators. (@rshura)\n\n## [1.16.1] - 2019-03-01\n### Fixed\n- #421: Nearby with MATCH is returning invalid results (@nithinkota)\n\n## [1.16.0] - 2019-02-25\n### Fixed\n- #415: Fixed overlapping geofences sending notifcation to wrong endpoint. (@belek, @s32x)\n- #412: Allow SERVER command for Lua scripts. (@1995parham)\n- #410: Allow slashes in MQTT Topics (@pstuifzand)\n- #409: Fixed bug in polygon clipping. (@rshura)\n- 30f903b: Require properties member for geojson features. (@rshura)\n\n### Added\n- #409: Added TEST command for executing WITHIN and INTERSECTS on two objects. (@rshura)\n- #407: Allow 201 & 202 status code on webhooks. (@s32x)\n- #404: Adding more replication data to INFO response. (@s32x)\n\n## [1.15.0] - 2019-01-16\n### Fixed\n- #403: JSON Output for INFO and CLIENT (@s32x)\n- #401: Fixing KEYS command (@s32x)\n- #398: Ensuring channel publish order (@s32x)\n- d7d0baa: Fix roam fence missing\n\n### Added\n- #402: Adding ARM and ARM64 packages (@s32x)\n- #399: Add RequireValid and update geojson dependency (@stevelacy)\n- #396: Add distance_to function to the tile38 namespace in lua. (@rshura)\n- #395: Add RENAME and RENAMENX commands. (@rshura)\n\n## [1.14.4] - 2018-12-03\n### Fixed\n- #394: Hotfix MultiPolygon intersect failure. (@contra)\n- #392: Fix TLS certs missing in Docker. (@vziukas, @s32x)\n\n### Added\n- Add extended server stats with SERVER EXT. (@s32x)\n- Add Kafka key to match notication key. (@Joey92)\n- Add optimized spatial index for fences\n\n## [1.14.3] - 2018-11-20\n### Fixed\n- Hotfix SCRIPT LOAD not executing from cli. (@rshura)\n\n## [1.14.2] - 2018-11-15\n### Fixed\n- #386: Fix version not being set at build. (@stevelacy)\n\n## [1.14.1] - 2018-11-15\n### Fixed\n- #385: Add `version` to SERVER command response (@stevelacy)\n- Hotfix replica sync needs flushing (@rshura)\n- Fixed a bug where some AOF commands where corrupted during reload\n\n## [1.14.0] - 2018-11-11\n### Added\n- INTERSECT/WITHIN optimization that may drastically improve searching polygons that have lots of points.\n- Faster responses for write operations such as SET/DEL\n- NEARBY now always returns objects from nearest to farthest (@rshura)\n- kNN haversine distance optimization (@rshura)\n- Evio networking beta using the \"-evio yes\" and \"-threads num\" flags\n\n### Fixed\n- #369: Fix poly in hole query\n\n## [1.13.0] - 2018-08-29\n### Added\n- eef5f3c: Add geofence notifications over pub/sub channels\n- 3a6f366: Add NODWELL keyword to roaming geofences\n- #343: Add Nats endpoints (@lennycampino)\n- #340: Add MQTT tls/cert options (@tobilg)\n- #314: Add CLIP subcommand to INTERSECTS (@rshura)\n\n### Changed\n- 3ae26e3: Updated B-tree implementation\n- 1d78a41: Updated R-tree implementation\n\n## [1.12.3] - 2018-06-16\n### Fixed\n- #316: Fix AMQP and AMQPS webhook endpoints to support namespaces (@DeadWisdom)\n- #318: Fix aofshrink crash on windows (@abhit011)\n- #326: Fix sporadic kNN results when TTL is used (@pmaseberg)\n\n## [1.12.2] - 2018-05-10\n### Fixed\n- #313: Hotfix intersect returning incorrect results (@stevelacy)\n\n## [1.12.1] - 2018-04-30\n### Fixed\n- #300: Fix pdelhooks not persisting (@tobilg)\n- #293: Fix kafka lockup issue (@Joey92)\n- #301: Fix AMQP uri custom params not working (@tobilg)\n- #302: Fix tile with zoom level over 63 panics (@rshura)\n- b99cd39: Fix Sync hook msg ttl with server time\n\n## [1.12.0] - 2018-04-12\n### Added\n- 11b42c0: Option to disable AOF or to use a custom path: #220 #223 #297 (@sign0, @umpc, @fmr683, @zhangfeng158)\n- #296: Add Meta data to hooks command (@tobilg)\n\n### Changed\n- 11b42c0: Updated help menu and show more options\n\n### Fixed\n- #295: Intersects returning nothing in some cases (@fils)\n- #294: HTTP requests stopped working (@zhangfeng158)\n- 0aa04a1: Lotsa package not vendored\n\n## [1.11.1] - 2018-03-16\n### Added\n- #272: Preserve Docker image tag history (@gechr)\n- 9428b84: Added cpu and threads to SERVER stats\n\n### Fixed\n- #281: Linestring intersection failure (@contra)\n- #280: Filter id match before kNN results (@sweco-semtne)\n- #269: Safe atomic ints for arm32 (@gmonk63)\n- #267: Optimization for multiploygons intersect queries (@contra)\n\n## [1.11.0] - 2018-03-05\n### Added\n- #221: Add WHEREEVAL clause to scan/search commands (@rshura)\n\n### Fixed\n- #254: Add maxmemory protection to FSET (@rshura)\n- #258: Clear expires on reset (@zycbobby)\n- #268: Avoid bbox intersect for non-bbox objects (@contra)\n\n## [1.10.1] - 2018-01-17\n### Fixed\n- #244: Fix issue with points not being detected inside MultiPolygons (@fazlul3003)\n- #245: Precalculate and store bboxes for complex objects (@huangpeizhi)\n- #246: Fix server crash when receiving zero arg commands (@behrad)\n\n## [1.10.0] - 2017-12-18\n### Added\n- #221: Sqs endpoint (@lennycampino)\n- #226: Lua scripting (@rshura) \n- #231: Allow setting multiple fields in a single fset command (@rshura)\n- #235: Add json library (encode/decode methods) to lua. (@rshura)\n- 26d0083: Update vendoring to use golang/dep \n- c8ed7ca: Add WHEREIN command (@rshura)\n- d817814: Optimized network pipelining\n\n### Fixed\n- #237: Flush to file periodically (@rshura)\n- #241: Point match on interior hole (@genesor)\n- 920dc3a: Use atomic ints/bools\n- 730502d: Set keepalive default to 300 seconds\n- 1084c60: Apply limit on top of cursor (@rshura)\n\n## [1.9.1] - 2017-08-16\n### Added\n- cd05708: Spatial index optimizations\n- #208: Debug message for failed webhook notifications (@karnivas)\n- #201: New ECHO command (@yorkxiao)\n- #183: Include tile38-cli in Docker image (@jchamberlain)\n- #121: Allow reads for disconnected followers (@octete)\n\n### Fixed\n- 3fae3f7: Allow cursors for kNN queries\n- #211: Crash when shrinking AOF on Windows (@icewukong)\n- #203: Lifted LIMIT restriction all queries and COUNT keyword (@yorkxiao, @FX-HAO)\n- #207: Send empty results for queries on nonexistent keys (@FX-HAO)\n- #195: Added kNN overscan ordering (@rshura)\n- #199: Apply LIMIT after WHERE clause (@rshura)\n- #199: Require Go 1.7 (@rshura)\n- #198: Omit fields for Resp when NOFIELDS is used (@rshura)\n\n## [1.9.0] - 2017-04-13\n### Added\n- #159: AMQP/RabbitMQ webhook support (@m1ome, @paavalan)\n- #152: Kafka webhook support (@m1ome)\n- #141: Add distances to Geofence notifications\n- #54: New benchmark tool (@literadix, @Lars-Meijer, @m1ome)\n- #20: Ability to specify pidfile via args (@olevole)\n\n### Fixed\n- b1c76d7: tile38-cli auto doesn't auto reconnect\n- #156: Use redis-style TTL implementation (@Lars-Meijer, @m1ome)\n- #150: Live \"inside\" fence event not triggering for new object (@phulst)\n\n## [1.8.0] - 2017-02-21\n### Added\n- #145: TCP Keepalives option (@UriHendler)\n- #136: K nearest neighbors for NEARBY command (@m1ome, @tomquas, @joernroeder)\n- #139: Added CLIENT command (@UriHendler)\n- #133: AutoGC config option (@m1ome, @amorskoy)\n\n### Fixed\n- #147: Leaking http hook connections (@mkabischev)\n- #143: Duplicate data in hook data (@mkabischev)\n\n## [1.7.5] - 2017-01-13\n### Added\n- Performance bump for all SET commands, ~10% faster\n- Lower memory footprint for large datasets\n- #112: Added distance to NEARBY command (@m1ome, @auselen)\n- #123: Redis endpoint for webhooks (@m1ome)\n- #128: Allow disabling HTTP & WebSocket transport (@m1ome)\n\n### Fixed\n- #116: Missing response in TTL json command (@phulst)\n- #117: Error in command documentation (@juanpabloaj)\n- #118: Unexpected EOF bug with websockets (@m1ome)\n- #122: Disque typo timeout handling (@m1ome)\n- #127: 3d object searches with 2d geojson area (@damariei)\n\n## [1.7.0] - 2016-12-29\n### Added\n- #104: PDEL command - Selete objects that match a pattern (@GameFreedom)\n- #99: COMMAND keyword for masking geofences by command type (@amorskoy)\n- #96: SCAN keyword for roaming geofences\n- fba34a9: JSET, JGET, JDEL commands\n\n### Fixed\n- #107: Memory leak (@amorskoy)\n- #98: Output json fix\n\n## [1.6.0] - 2016-12-11\n### Added\n- #87: Fencing event grouping (@huangpeizhi)\n\n### Fixed\n- #91: Wrong winding order for CirclePolygon function (@antonioromano)\n- #73: Corruption for AOFSHRINK (@huangpeizhi)\n- #71: Lower memory usage. About 25% savings (@thisisaaronland, @umpc)\n- Polygon raycast bug. tidwall/poly#1 (@drewlesueur)\n- Added black-box testing\n\n## [1.5.4] - 2016-11-17\n### Fixed\n- #84: Hotfix - roaming fence deadlock (@tomquas)\n\n## [1.5.3] - 2016-11-16\n### Added\n- #4: Official docker support (@gordysc)\n\n### Fixed\n- #77: NX/XX bug (@damariei)\n- #76: Match on prefix star (@GameFreedom, @icewukong)\n- #82: Allow for precise search for strings (@GameFreedom)\n- #83: Faster congruent modulo for points (@icewukong, @umpc)\n\n## [1.5.2] - 2016-10-20\n### Fixed\n- #70: Invalid results for INTERSECTS query (@thisisaaronland)\n\n## [1.5.1] - 2016-10-19\n### Fixed\n- #67: Call the EXPIRE command hangs the server (@PapaStifflera)\n- #64: Missing points in 'Nearby' queries (@umpc)\n\n## [1.5.0] - 2016-10-03\n### Added\n- #61: Optimized queries on 3d objects (@damariei)\n- #60: Added [NX|XX] keywords to SET command (@damariei)\n- #29: Generalized hook interface (@jeremytregunna)\n- GRPC geofence hook support \n\n### Fixed\n- #62: Potential Replace Bug Corrupting the Index (@umpc)\n- #57: CRLF codes in info after bump from 1.3.0 to 1.4.2 (@olevole)\n\n## [1.4.2] - 2016-08-26\n### Fixed\n- #49. Allow fragmented pipeline requests (@owaaa)\n- #51: Allow multispace delim in native proto (@huangpeizhi)\n- #50: MATCH with slashes (@huangpeizhi)\n- #43: Linestring nearby search correction (@owaaa)\n\n## [1.4.1] - 2016-08-26\n### Added\n- #34: Added \"BOUNDS key\" command (@icewukong)\n\n### Fixed\n- #38: Allow for nginx support (@GameFreedom)\n- #39: Reset requirepass (@GameFreedom)\n\n## [1.3.0] - 2016-07-22\n### Added\n- New EXPIRE, PERSISTS, TTL commands. New EX keyword to SET command\n- Support for plain strings using `SET ... STRING value.` syntax\n- New SEARCH command for finding strings\n- Scans can now order descending\n\n### Fixed\n- #28: fix windows cli issue (@zhangkaizhao)\n\n## [1.2.0] - 2016-05-24\n### Added\n- #17: Roaming Geofences for NEARBY command (@ElectroCamel, @davidxv)\n- #15: maxmemory config setting (@jrots)\n\n## [1.1.4] - 2016-04-19\n### Fixed\n- #12: Issue where a newline was being added to HTTP POST requests (@davidxv)\n- #13: OBJECT keyword not accepted for WITHIN command (@ray93)\n- Panic on missing key for search requests\n\n## [1.1.2] - 2016-04-12\n### Fixed\n- A glob suffix wildcard can result in extra hits\n- The native live geofence sometimes fails connections\n\n## [1.1.0] - 2016-04-02\n### Added\n- Resp client support. All major programming languages now supported\n- Added WITHFIELDS option to GET\n- Added OUTPUT command to allow for outputing JSON when using RESP\n- Added DETECT option to geofences\n\n### Changed\n- New AOF file structure.\n- Quicker and safer AOFSHRINK.\n\n### Deprecation Warning\n- Native protocol support is being deprecated in a future release in favor of RESP\n" }, { "path": "CONTRIBUTING.md", "content": "# Contributing\n\nThanks for your interest in improving Tile38!\n\n## Issues\n- **Found a bug?** Please open an issue with a clear description and precise steps to reproduce.\n\n## Pull Requests\n- **Scope.** PRs are welcome for bug fixes and small, low-risk changes.\n- **Please coordinate first.** For large changes, such as new features, please file an issue and/or discuss it with a maintainer beforehand.\n- **Quality.** Keep PRs focused, include tests (where applicable), and update docs.\n\n## When Not to use a Pull Request\n\nDo NOT submit a Pull Request for:\n\n- Security-related changes.\n- Works in progress.\n- Changes that require design discussion, or are likely to be controversial.\n- Changes needing specialized or cross-subsystem review.\n- Large refactors or mechanical tree-wide changes.\n- Changes generated by AI tools.\n\n## AI Assisted Contributions\n\nWe are not accepting AI assisted code into Tile38 at this time. \nThis policy may change in the future. \nPlease reach out if you have any questions or concerns regarding AI and Tile38.\n\n### Contributor Responsibilities\n\n- Monitor your Pull Request and respond to review feedback promptly.\n- Pull Requests may be closed if there is no response for **one month**.\n\nBy submitting a contribution, you confirm you have the right to do so and grant Tile38 LLC the rights needed to use, modify, and redistribute your contribution as part of the project.\n\nIf you have any questions or concerns regarding licensing, please contact us at licensing@tile38.com. \nFor all other questions, contributing@tile38.com.\n" }, { "path": "Dockerfile", "content": "FROM alpine:3.20\n\nARG VERSION\nARG TARGETOS\nARG TARGETARCH\n\nRUN apk add --no-cache ca-certificates\n\nADD packages/tile38-$VERSION-$TARGETOS-$TARGETARCH/tile38-server /usr/local/bin\nADD packages/tile38-$VERSION-$TARGETOS-$TARGETARCH/tile38-cli /usr/local/bin\nADD packages/tile38-$VERSION-$TARGETOS-$TARGETARCH/tile38-benchmark /usr/local/bin\n\nRUN addgroup -S tile38 && \\\n adduser -S -G tile38 tile38 && \\\n mkdir /data && chown tile38:tile38 /data\n\nVOLUME /data\n\nEXPOSE 9851\nCMD [\"tile38-server\", \"-d\", \"/data\"]\n" }, { "path": "LICENSE", "content": "Copyright (c) 2016 Josh Baker\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in\nall copies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN\nTHE SOFTWARE.\n" }, { "path": "Makefile", "content": "all: tile38-server tile38-cli tile38-benchmark tile38-luamemtest\n\n.PHONY: tile38-server\ntile38-server:\n\t@./scripts/build.sh tile38-server\n\n.PHONY: tile38-cli\ntile38-cli:\n\t@./scripts/build.sh tile38-cli\n\n.PHONY: tile38-benchmark\ntile38-benchmark:\n\t@./scripts/build.sh tile38-benchmark\n\n.PHONY: tile38-luamemtest\ntile38-luamemtest:\n\t@./scripts/build.sh tile38-luamemtest\n\ntest: all\n\t@./scripts/test.sh\n\npackage:\n\t@rm -rf packages/\n\t@scripts/package.sh Windows windows amd64\n\t@scripts/package.sh Mac darwin amd64\n\t@scripts/package.sh Linux linux amd64\n\t@scripts/package.sh FreeBSD freebsd amd64\n\t@scripts/package.sh ARM linux arm\n\t@scripts/package.sh ARM64 linux arm64\n\nclean:\n\trm -rf tile38-server tile38-cli tile38-benchmark tile38-luamemtest \n\ndistclean: clean\n\trm -rf packages/\n\ninstall: all\n\tcp tile38-server /usr/local/bin\n\tcp tile38-cli /usr/local/bin\n\tcp tile38-benchmark /usr/local/bin\n\nuninstall: \n\trm -f /usr/local/bin/tile38-server\n\trm -f /usr/local/bin/tile38-cli\n\trm -f /usr/local/bin/tile38-benchmark\n" }, { "path": "README.md", "content": "

\n\n \n \n \"Tile38\"\n\n

\n

\n\"Slack\"\n\"Discord\"\n

\n\n\n\n\nTile38 is an open source (MIT licensed), in-memory geolocation data store, spatial index, and realtime geofencing server.\nIt supports a variety of object types including lat/lon points, bounding boxes, XYZ tiles, Geohashes, and GeoJSON. \n\n

\nThis README is quick start document. You can find detailed documentation at https://tile38.com.

\n\"Nearby\"\n\"Within\"\n\"Intersects\"\n\"Geofencing\"\n\"Roaming\n

\n\n## Features\n\n- Spatial index with [search](#searching) methods such as Nearby, Within, and Intersects.\n- Realtime [geofencing](#geofencing) through [webhooks](https://tile38.com/commands/sethook) or [pub/sub channels](#pubsub-channels).\n- Object types of [lat/lon](#latlon-point), [bbox](#bounding-box), [Geohash](#geohash), [GeoJSON](#geojson), [QuadKey](#quadkey), and [XYZ tile](#xyz-tile).\n- Support for lots of [Clients Libraries](#tile38-client-libraries) written in many different languages.\n- Variety of protocols, including [http](#http) (curl), [websockets](#websockets), [telnet](#telnet), and the [Redis RESP](https://redis.io/topics/protocol).\n- Server responses are [RESP](https://redis.io/topics/protocol) or [JSON](https://www.json.org).\n- Full [command line interface](#cli).\n- Leader / follower [replication](#replication).\n- In-memory database that persists on disk.\n\n## Components\n- tile38-server: The server\n- tile38-cli: Command line interface tool\n- tile38-benchmark: Server benchmark tool\n\n## Getting Started\n\n### Getting Tile38\n\nPerhaps the easiest way to get the latest Tile38 is to use one of the pre-built release binaries which are available for OSX, Linux, FreeBSD, and Windows. Instructions for using these binaries are on the GitHub [releases page](https://github.com/tidwall/tile38/releases).\n\n### Docker \n\nTo run the latest stable version of Tile38:\n\n```\ndocker pull tile38/tile38\ndocker run -p 9851:9851 tile38/tile38\n```\n\nVisit the [Tile38 hub page](https://hub.docker.com/r/tile38/tile38/) for more information.\n\n### Homebrew (macOS)\n\nInstall Tile38 using [Homebrew](https://brew.sh/)\n\n```sh\nbrew install tile38\ntile38-server\n```\n\n### Building Tile38 \n\nTile38 can be compiled and used on Linux, OSX, Windows, FreeBSD, and probably others since the codebase is 100% Go. We support both 32 bit and 64 bit systems. [Go](https://golang.org/dl/) must be installed on the build machine.\n\nTo build everything simply:\n```\n$ make\n```\n\nTo test:\n```\n$ make test\n```\n\n### Running \nFor command line options invoke:\n```\n$ ./tile38-server -h\n```\n\nTo run a single server:\n\n```\n$ ./tile38-server\n\n# The tile38 shell connects to localhost:9851\n$ ./tile38-cli\n> help\n```\n\n#### Prometheus Metrics\nTile38 can natively export Prometheus metrics by setting the `--metrics-addr` command line flag (disabled by default). This example exposes the HTTP metrics server on port 4321:\n```\n# start server and enable Prometheus metrics, listen on local interface only\n./tile38-server --metrics-addr=127.0.0.1:4321\n\n# access metrics\ncurl http://127.0.0.1:4321/metrics\n```\nIf you need to access the `/metrics` endpoint from a different host you'll have to set the flag accordingly, e.g. set it to `0.0.0.0:<>` to listen on all interfaces.\n\nUse the [redis_exporter](https://github.com/oliver006/redis_exporter) for more advanced use cases like extracting key values or running a lua script.\n\n\n## Playing with Tile38\n\nBasic operations:\n```\n$ ./tile38-cli\n\n# add a couple of points named 'truck1' and 'truck2' to a collection named 'fleet'.\n> set fleet truck1 point 33.5123 -112.2693 # on the Loop 101 in Phoenix\n> set fleet truck2 point 33.4626 -112.1695 # on the I-10 in Phoenix\n\n# search the 'fleet' collection.\n> scan fleet # returns both trucks in 'fleet'\n> nearby fleet point 33.462 -112.268 6000 # search 6 kilometers around a point. returns one truck.\n\n# key value operations\n> get fleet truck1 # returns 'truck1'\n> del fleet truck2 # deletes 'truck2'\n> drop fleet # removes all \n```\n\nTile38 has a ton of [great commands](https://tile38.com/commands).\n\n## Fields\nFields are extra data that belongs to an object. A field is always a double precision floating point. There is no limit to the number of fields that an object can have. \n\nTo set a field when setting an object:\n```\n> set fleet truck1 field speed 90 point 33.5123 -112.2693 \n> set fleet truck1 field speed 90 field age 21 point 33.5123 -112.2693\n```\n\nTo set a field when an object already exists:\n```\n> fset fleet truck1 speed 90\n```\n\nTo get a field when an object already exists:\n```\n> fget fleet truck1 speed\n```\n\n## Searching\n\nTile38 has support to search for objects and points that are within or intersects other objects. All object types can be searched including Polygons, MultiPolygons, GeometryCollections, etc.\n\n\"Search\n\n#### Within \nWITHIN searches a collection for objects that are fully contained inside a specified bounding area.\n
\n\n\"Search\n\n#### Intersects\nINTERSECTS searches a collection for objects that intersect a specified bounding area.\n
\n\n\"Search\n\n#### Nearby\nNEARBY searches a collection for objects that intersect a specified radius.\n
\n\n### Search options\n**WHERE** - This option allows for filtering out results based on [field](#fields) values. For example
```nearby fleet where speed 70 +inf point 33.462 -112.268 6000``` will return only the objects in the 'fleet' collection that are within the 6 km radius **and** have a field named `speed` that is greater than `70`.

Multiple WHEREs are concatenated as **and** clauses. ```WHERE speed 70 +inf WHERE age -inf 24``` would be interpreted as *speed is over 70 and age is less than 24.*

The default value for a field is always `0`. Thus if you do a WHERE on the field `speed` and an object does not have that field set, the server will pretend that the object does and that the value is Zero.

WHERE expressions support the `=~` operator for regex matching. It uses Go's re2 regular expression engine to match string values in fields or GeoJSON properties within objects. For example, `WHERE properties.name =~ 'truck.*'` filters objects where the 'name' property matches the pattern, and `WHERE field_name =~ 'value.*'` works similarly for fields.\n\n**MATCH** - MATCH is similar to WHERE except that it works on the object id instead of fields.
```nearby fleet match truck* point 33.462 -112.268 6000``` will return only the objects in the 'fleet' collection that are within the 6 km radius **and** have an object id that starts with `truck`. There can be multiple MATCH options in a single search. The MATCH value is a simple [glob pattern](https://en.wikipedia.org/wiki/Glob_(programming)).\n\n**CURSOR** - CURSOR is used to iterate though many objects from the search results. An iteration begins when the CURSOR is set to Zero or not included with the request, and completes when the cursor returned by the server is Zero.\n\n**NOFIELDS** - NOFIELDS tells the server that you do not want field values returned with the search results.\n\n**LIMIT** - LIMIT can be used to limit the number of objects returned for a single search request.\n\n\n## Geofencing\n\n\"Geofence\nA geofence is a virtual boundary that can detect when an object enters or exits the area. This boundary can be a radius, bounding box, or a polygon. Tile38 can turn any standard search into a geofence monitor by adding the FENCE keyword to the search. \n\n*Tile38 also allows for [Webhooks](https://tile38.com/commands/sethook) to be assigned to Geofences.*\n\n
\n\nA simple example:\n```\n> nearby fleet fence point 33.462 -112.268 6000\n```\nThis command opens a geofence that monitors the 'fleet' collection. The server will respond with:\n```\n{\"ok\":true,\"live\":true}\n```\nAnd the connection will be kept open. If any object enters or exits the 6 km radius around `33.462,-112.268` the server will respond in realtime with a message such as:\n\n```\n{\"command\":\"set\",\"detect\":\"enter\",\"id\":\"truck02\",\"object\":{\"type\":\"Point\",\"coordinates\":[-112.2695,33.4626]}}\n```\n\nThe server will notify the client if the `command` is `del | set | drop`. \n\n- `del` notifies the client that an object has been deleted from the collection that is being fenced.\n- `drop` notifies the client that the entire collection is dropped.\n- `set` notifies the client that an object has been added or updated, and when it's position is detected by the fence.\n\nThe `detect` may be one of the following values.\n\n- `inside` is when an object is inside the specified area.\n- `outside` is when an object is outside the specified area.\n- `enter` is when an object that **was not** previously in the fence has entered the area.\n- `exit` is when an object that **was** previously in the fence has exited the area.\n- `cross` is when an object that **was not** previously in the fence has entered **and** exited the area.\n\nThese can be used when establishing a geofence, to pre-filter responses. For instance, to limit responses to `enter` and `exit` detections:\n\n```\n> nearby fleet fence detect enter,exit point 33.462 -112.268 6000\n```\n\n### Pub/sub channels\n\nTile38 supports delivering geofence notifications over pub/sub channels. \n\nTo create a static geofence that sends notifications when a bus is within 200 meters of a point and sends to the `busstop` channel:\n\n```\n> setchan busstop nearby buses fence point 33.5123 -112.2693 200\n```\n\nSubscribe on the `busstop` channel:\n\n```\n> subscribe busstop\n```\n\n## Object types\n\nAll object types except for XYZ Tiles and QuadKeys can be stored in a collection. XYZ Tiles and QuadKeys are reserved for the SEARCH keyword only.\n\n#### Lat/lon point\nThe most basic object type is a point that is composed of a latitude and a longitude. There is an optional `z` member that may be used for auxiliary data such as elevation or a timestamp.\n```\nset fleet truck1 point 33.5123 -112.2693 # plain lat/lon\nset fleet truck1 point 33.5123 -112.2693 225 # lat/lon with z member\n```\n\n#### Bounding box\nA bounding box consists of two points. The first being the southwestern most point and the second is the northeastern most point.\n```\nset fleet truck1 bounds 30 -110 40 -100\n```\n#### Geohash\nA [geohash](https://en.wikipedia.org/wiki/Geohash) is a string representation of a point. With the length of the string indicating the precision of the point. \n```\nset fleet truck1 hash 9tbnthxzr # this would be equivalent to 'point 33.5123 -112.2693'\n```\n\n#### GeoJSON\n[GeoJSON](https://tools.ietf.org/html/rfc7946) is an industry standard format for representing a variety of object types including a point, multipoint, linestring, multilinestring, polygon, multipolygon, geometrycollection, feature, and featurecollection.\n\n* All ignored members will not persist.\n\n**Important to note that all coordinates are in Longitude, Latitude order.**\n\n```\nset city tempe object {\"type\":\"Polygon\",\"coordinates\":[[[0,0],[10,10],[10,0],[0,0]]]}\n```\n\n#### XYZ Tile\nAn XYZ tile is rectangle bounding area on earth that is represented by an X, Y coordinate and a Z (zoom) level.\nCheck out [maptiler.org](http://www.maptiler.org/google-maps-coordinates-tile-bounds-projection/) for an interactive example.\n\n#### QuadKey\nA QuadKey used the same coordinate system as an XYZ tile except that the string representation is a string characters composed of 0, 1, 2, or 3. For a detailed explanation checkout [The Bing Maps Tile System](https://msdn.microsoft.com/en-us/library/bb259689.aspx).\n\n## Network protocols\n\nIt's recommended to use a [client library](#tile38-client-libraries) or the [Tile38 CLI](#running), but there are times when only HTTP is available or when you need to test from a remote terminal. In those cases we provide an HTTP and telnet options.\n\n#### HTTP\nOne of the simplest ways to call a tile38 command is to use HTTP. From the command line you can use [curl](https://curl.haxx.se/). For example:\n\n```\n# call with request in the body\ncurl --data \"set fleet truck3 point 33.4762 -112.10923\" localhost:9851\n\n# call with request in the url path\ncurl localhost:9851/set+fleet+truck3+point+33.4762+-112.10923\n```\n\n#### Websockets\nWebsockets can be used when you need to Geofence and keep the connection alive. It works just like the HTTP example above, with the exception that the connection stays alive and the data is sent from the server as text websocket messages.\n\n#### Telnet\nThere is the option to use a plain telnet connection. The default output through telnet is [RESP](https://redis.io/topics/protocol).\n\n```\ntelnet localhost 9851\nset fleet truck3 point 33.4762 -112.10923\n+OK\n\n```\n\nThe server will respond in [JSON](https://json.org) or [RESP](https://redis.io/topics/protocol) depending on which protocol is used when initiating the first command.\n\n- HTTP and Websockets use JSON. \n- Telnet and RESP clients use RESP.\n\n## Tile38 Client Libraries\n\nThe following clients are built specifically for Tile38. \nClients that support most Tile38 features are marked with a ⭐️.\n\n- ⭐️ Go: [xjem/t38c](https://github.com/xjem/t38c)\n- ⭐️ Node.js: [node-tile38](https://github.com/phulst/node-tile38) ([example code](https://github.com/tidwall/tile38/wiki/Node.js-example-(node-tile38)))\n- ⭐️ Python: [pyle38](https://github.com/iwpnd/pyle38)\n- ⭐️ TypeScript: [tile38-ts](https://github.com/iwpnd/tile38-ts)\n- Go: [cjkreklow/t38c](https://github.com/cjkreklow/t38c)\n- Python: [pytile38](https://github.com/mitghi/pytile38)\n- Rust: [nazar](https://github.com/younisshah/nazar)\n- Swift: [Talon](https://github.com/mikekinney/Talon)\n- Java: [tile38-client-java](https://github.com/jamshidrostami/tile38-client-java)\n- Java: [tile38-client](https://github.com/HkMoyun/tile38-client)\n\n## Redis Client Libraries\n\nTile38 uses the [Redis RESP](https://redis.io/topics/protocol) protocol natively. \nTherefore most clients that support basic Redis commands will also support Tile38.\n\n- C: [hiredis](https://github.com/redis/hiredis)\n- C#: [StackExchange.Redis](https://github.com/StackExchange/StackExchange.Redis)\n- C++: [redox](https://github.com/hmartiro/redox)\n- Clojure: [carmine](https://github.com/ptaoussanis/carmine)\n- Common Lisp: [CL-Redis](https://github.com/vseloved/cl-redis)\n- Erlang: [Eredis](https://github.com/wooga/eredis)\n- Go: [go-redis](https://github.com/go-redis/redis) ([example code](https://github.com/tidwall/tile38/wiki/Go-example-(go-redis)))\n- Go: [redigo](https://github.com/gomodule/redigo) ([example code](https://github.com/tidwall/tile38/wiki/Go-example-(redigo)))\n- Haskell: [hedis](https://github.com/informatikr/hedis)\n- Java: [lettuce](https://github.com/mp911de/lettuce) ([example code](https://github.com/tidwall/tile38/wiki/Java-example-(lettuce)))\n- Node.js: [node_redis](https://github.com/NodeRedis/node_redis) ([example code](https://github.com/tidwall/tile38/wiki/Node.js-example-(node-redis)))\n- Perl: [perl-redis](https://github.com/PerlRedis/perl-redis)\n- PHP: [tinyredisclient](https://github.com/ptrofimov/tinyredisclient) ([example code](https://github.com/tidwall/tile38/wiki/PHP-example-(tinyredisclient)))\n- PHP: [phpredis](https://github.com/phpredis/phpredis)\n- Python: [redis-py](https://github.com/andymccurdy/redis-py) ([example code](https://github.com/tidwall/tile38/wiki/Python-example))\n- Ruby: [redic](https://github.com/amakawa/redic) ([example code](https://github.com/tidwall/tile38/wiki/Ruby-example-(redic)))\n- Ruby: [redis-rb](https://github.com/redis/redis-rb) ([example code](https://github.com/tidwall/tile38/wiki/Ruby-example-(redis-rb)))\n- Rust: [redis-rs](https://github.com/mitsuhiko/redis-rs)\n- Scala: [scala-redis](https://github.com/debasishg/scala-redis)\n- Swift: [Redbird](https://github.com/czechboy0/Redbird)\n\n## Contact\n\nJosh Baker [@tidwall](https://twitter.com/tidwall)\n\n## License\n\nTile38 source code is available under the MIT [License](/LICENSE).\n" }, { "path": "cmd/tile38-benchmark/az/az.go", "content": "package az\n\n// JSON is GeoJSON\nvar JSON = `\n{ \"type\": \"MultiPolygon\", \"coordinates\": [ [ [ [ -114.635458, 34.876902 ], [ -114.636768000000103, 34.885705 ], [ -114.636725, 34.889107 ], [ -114.635425, 34.895192 ], [ -114.63185, 34.903942 ], [ -114.630877, 34.907263 ], [ -114.630552, 34.911852 ], [ -114.631537, 34.916153 ], [ -114.633237, 34.92123 ], [ -114.633253, 34.924608 ], [ -114.632196, 34.930628 ], [ -114.629753, 34.938684 ], [ -114.629811, 34.94481 ], [ -114.631681, 34.95131 ], [ -114.634274, 34.956662 ], [ -114.634953, 34.958918 ], [ -114.635237000000103, 34.965149 ], [ -114.634607, 34.96906 ], [ -114.629907, 34.980791 ], [ -114.629129, 34.986132 ], [ -114.629443, 34.991825 ], [ -114.630244000000104, 34.99464 ], [ -114.631807, 34.998632 ], [ -114.632665, 34.999806 ], [ -114.635570000000101, 35.005933 ], [ -114.637071, 35.010371 ], [ -114.637769, 35.014948 ], [ -114.63819, 35.022069 ], [ -114.637524, 35.027053 ], [ -114.633715, 35.035602 ], [ -114.629027, 35.042531 ], [ -114.625799, 35.045834 ], [ -114.615902, 35.05272 ], [ -114.610701000000105, 35.055458 ], [ -114.606694, 35.058941 ], [ -114.604715, 35.061744 ], [ -114.603619, 35.064226 ], [ -114.602908, 35.068588 ], [ -114.603175, 35.070445 ], [ -114.604736, 35.07483 ], [ -114.607701, 35.078533 ], [ -114.613132, 35.083097 ], [ -114.61842, 35.086539 ], [ -114.622517, 35.088703 ], [ -114.632053, 35.092559 ], [ -114.63937, 35.094733 ], [ -114.642831, 35.096503 ], [ -114.646579, 35.10082 ], [ -114.646764, 35.101868 ], [ -114.645152, 35.104995 ], [ -114.644354, 35.105903 ], [ -114.641116, 35.108401 ], [ -114.637432000000103, 35.112489 ], [ -114.632282, 35.117088 ], [ -114.628427, 35.118943 ], [ -114.623761, 35.120602 ], [ -114.628993, 35.119411 ], [ -114.624954000000102, 35.120742 ], [ -114.618697, 35.121749 ], [ -114.604007, 35.121252 ], [ -114.60274, 35.121666 ], [ -114.597794, 35.121735 ], [ -114.589787, 35.123522 ], [ -114.584877, 35.125194 ], [ -114.579882, 35.127506 ], [ -114.578263, 35.12881 ], [ -114.577146, 35.130982 ], [ -114.574411, 35.13495 ], [ -114.572597, 35.139557 ], [ -114.573706, 35.142698 ], [ -114.573879, 35.145351 ], [ -114.569529, 35.162317 ], [ -114.56876, 35.172195 ], [ -114.569214, 35.17289 ], [ -114.568989, 35.175085 ], [ -114.569258, 35.183424 ], [ -114.569653, 35.186267 ], [ -114.571404, 35.191026 ], [ -114.572084, 35.200794 ], [ -114.574037, 35.20379 ], [ -114.574233, 35.205481 ], [ -114.574958, 35.206714 ], [ -114.578581, 35.208113 ], [ -114.579535, 35.208911 ], [ -114.579897000000102, 35.21097 ], [ -114.580312, 35.220095 ], [ -114.583523, 35.230348 ], [ -114.58248, 35.233173 ], [ -114.582842, 35.238703 ], [ -114.584993, 35.242717 ], [ -114.586053, 35.248891 ], [ -114.585714, 35.253145 ], [ -114.585768, 35.257743 ], [ -114.586604, 35.262386 ], [ -114.587497, 35.265473 ], [ -114.590513, 35.272334 ], [ -114.593247, 35.284361 ], [ -114.595705, 35.289939 ], [ -114.596682, 35.294557 ], [ -114.597268, 35.299565 ], [ -114.59721, 35.303223 ], [ -114.595163, 35.321883 ], [ -114.595553, 35.326547 ], [ -114.599771, 35.34111 ], [ -114.604607, 35.355239 ], [ -114.606173, 35.359651 ], [ -114.611206, 35.370119 ], [ -114.617698, 35.380131 ], [ -114.618257, 35.382646 ], [ -114.618984, 35.389391 ], [ -114.620887, 35.396867 ], [ -114.621783, 35.39945 ], [ -114.625702, 35.407976 ], [ -114.626765, 35.409644 ], [ -114.629061000000107, 35.411175 ], [ -114.65208, 35.430134 ], [ -114.653817, 35.432853 ], [ -114.654295, 35.436854 ], [ -114.658105, 35.441835 ], [ -114.661747, 35.444735 ], [ -114.662896, 35.446449 ], [ -114.663934, 35.449466 ], [ -114.664215, 35.451707 ], [ -114.663880000000105, 35.454657 ], [ -114.664217, 35.455845 ], [ -114.665142, 35.457331 ], [ -114.666151, 35.458198 ], [ -114.667217, 35.46037 ], [ -114.666769000000102, 35.462085 ], [ -114.665790000000101, 35.463915 ], [ -114.665651, 35.466911 ], [ -114.665988, 35.467985 ], [ -114.667389, 35.469904 ], [ -114.67235, 35.47374 ], [ -114.673164, 35.474814 ], [ -114.673585, 35.475843 ], [ -114.673473000000101, 35.476849 ], [ -114.672074, 35.479709 ], [ -114.671794, 35.480806 ], [ -114.671907, 35.482087 ], [ -114.673534, 35.485675 ], [ -114.676815000000104, 35.489787 ], [ -114.6767040000001, 35.491845 ], [ -114.676257, 35.493103 ], [ -114.677743, 35.495182 ], [ -114.678642, 35.497628 ], [ -114.678587, 35.499846 ], [ -114.678892000000104, 35.501276 ], [ -114.67748, 35.510948 ], [ -114.677143, 35.512945 ], [ -114.675685, 35.51563 ], [ -114.672767, 35.518428 ], [ -114.66954, 35.52079 ], [ -114.668586, 35.521225 ], [ -114.666565, 35.520993 ], [ -114.664601000000104, 35.521519 ], [ -114.663983000000101, 35.522161 ], [ -114.661682, 35.526682 ], [ -114.659886, 35.527919 ], [ -114.657753, 35.530741 ], [ -114.657163, 35.532301 ], [ -114.65677, 35.534964 ], [ -114.657809, 35.536963 ], [ -114.660335, 35.540433 ], [ -114.661457, 35.544062 ], [ -114.66157, 35.545692 ], [ -114.66112, 35.549021 ], [ -114.661963, 35.552604 ], [ -114.661963, 35.555887 ], [ -114.663451, 35.559884 ], [ -114.663535, 35.560963 ], [ -114.662805, 35.564268 ], [ -114.6639, 35.56629 ], [ -114.664433000000102, 35.568426 ], [ -114.666231, 35.571642 ], [ -114.668393, 35.574331 ], [ -114.670022, 35.575596 ], [ -114.671567, 35.576217 ], [ -114.674881, 35.578379 ], [ -114.675751, 35.579459 ], [ -114.675667, 35.580033 ], [ -114.670191, 35.583471 ], [ -114.664209, 35.585944 ], [ -114.660558, 35.586583 ], [ -114.659238, 35.587477 ], [ -114.654518, 35.596609 ], [ -114.653900000000107, 35.598491 ], [ -114.653731, 35.600373 ], [ -114.654489, 35.605173 ], [ -114.653618, 35.607192 ], [ -114.653534, 35.609672 ], [ -114.653927, 35.611739 ], [ -114.655219, 35.614059 ], [ -114.657241, 35.617046 ], [ -114.659461, 35.619552 ], [ -114.660641, 35.620334 ], [ -114.663647, 35.620773 ], [ -114.665389, 35.621556 ], [ -114.666682, 35.623073 ], [ -114.668087, 35.627115 ], [ -114.669015000000101, 35.628861 ], [ -114.672134, 35.633365 ], [ -114.675001, 35.638304 ], [ -114.677615, 35.641774 ], [ -114.679415000000105, 35.643429 ], [ -114.686133, 35.647522 ], [ -114.689001, 35.65028 ], [ -114.689507, 35.651429 ], [ -114.689226, 35.652898 ], [ -114.690494000000101, 35.662657 ], [ -114.690214, 35.665159 ], [ -114.686055, 35.670642 ], [ -114.682317, 35.677825 ], [ -114.680827, 35.682255 ], [ -114.680631000000105, 35.684046 ], [ -114.680997000000104, 35.685929 ], [ -114.682657000000106, 35.688571 ], [ -114.691263, 35.693125 ], [ -114.696214, 35.69655 ], [ -114.701416, 35.701084 ], [ -114.703608, 35.703922 ], [ -114.704501, 35.705993 ], [ -114.704959, 35.706366 ], [ -114.704842, 35.706744 ], [ -114.705597, 35.708274 ], [ -114.705347000000103, 35.708344 ], [ -114.705447, 35.711757 ], [ -114.699405, 35.726929 ], [ -114.697859, 35.731657 ], [ -114.69654, 35.738934 ], [ -114.6964, 35.742653 ], [ -114.696655, 35.746143 ], [ -114.697585, 35.748417 ], [ -114.697726, 35.750966 ], [ -114.696854, 35.752756 ], [ -114.696546, 35.754638 ], [ -114.694267, 35.756633 ], [ -114.694717, 35.757897 ], [ -114.69742, 35.760677 ], [ -114.700266, 35.766879 ], [ -114.701027, 35.76968 ], [ -114.70117, 35.774112 ], [ -114.699036000000106, 35.788046 ], [ -114.699318000000105, 35.79048 ], [ -114.703178, 35.794685 ], [ -114.705827, 35.798889 ], [ -114.71149, 35.80438 ], [ -114.712026, 35.805529 ], [ -114.710534, 35.807525 ], [ -114.709324, 35.81005 ], [ -114.70634, 35.812022 ], [ -114.703665, 35.814614 ], [ -114.700654, 35.822004 ], [ -114.697276, 35.826776 ], [ -114.69553, 35.829897 ], [ -114.695277000000104, 35.831091 ], [ -114.695249, 35.832285 ], [ -114.695757, 35.833387 ], [ -114.701478, 35.839316 ], [ -114.702293, 35.840792 ], [ -114.702339, 35.842151 ], [ -114.703527, 35.841845 ], [ -114.704173, 35.842669 ], [ -114.704203000000106, 35.844274 ], [ -114.706288, 35.846218 ], [ -114.706532, 35.849027 ], [ -114.705856, 35.850508 ], [ -114.703599, 35.852595 ], [ -114.701904, 35.853223 ], [ -114.696581, 35.853727 ], [ -114.69437, 35.854463 ], [ -114.693446, 35.855125 ], [ -114.691456, 35.858661 ], [ -114.6877980000001, 35.860728 ], [ -114.68205, 35.86295 ], [ -114.678186, 35.863311 ], [ -114.672289, 35.865011 ], [ -114.66968, 35.865036 ], [ -114.667471000000106, 35.867061 ], [ -114.662623, 35.869213 ], [ -114.661636, 35.870545 ], [ -114.661636, 35.871233 ], [ -114.663214, 35.873692 ], [ -114.668145, 35.875201 ], [ -114.672009, 35.878018 ], [ -114.678972, 35.88551 ], [ -114.693602, 35.895311 ], [ -114.69454, 35.896587 ], [ -114.696064, 35.896464 ], [ -114.694928, 35.897594 ], [ -114.696132000000105, 35.898662 ], [ -114.697558, 35.89936 ], [ -114.700258000000105, 35.901757 ], [ -114.700769, 35.903064 ], [ -114.703538, 35.906707 ], [ -114.705119, 35.907637 ], [ -114.705991, 35.908598 ], [ -114.7057140000001, 35.909316 ], [ -114.706767, 35.90895 ], [ -114.708112, 35.909933 ], [ -114.709187, 35.916827 ], [ -114.707784, 35.916993 ], [ -114.707398, 35.918057 ], [ -114.70788, 35.919207 ], [ -114.707329000000101, 35.926177 ], [ -114.707603, 35.92795 ], [ -114.712965, 35.932159 ], [ -114.712756, 35.932639 ], [ -114.713413, 35.9319 ], [ -114.713312, 35.933844 ], [ -114.729762, 35.959895 ], [ -114.728496000000106, 35.960395 ], [ -114.728666, 35.961757 ], [ -114.730090000000104, 35.962691 ], [ -114.732456, 35.965891 ], [ -114.736195, 35.969421 ], [ -114.740536, 35.975545 ], [ -114.743494, 35.983553 ], [ -114.743638, 35.985785 ], [ -114.743117, 35.987387 ], [ -114.740043, 35.990534 ], [ -114.739318, 35.991804 ], [ -114.740544, 35.994853 ], [ -114.740815, 35.997464 ], [ -114.741536, 35.99969 ], [ -114.741679, 36.002283 ], [ -114.743163, 36.006722 ], [ -114.743005, 36.00845 ], [ -114.740866, 36.012928 ], [ -114.738555, 36.015223 ], [ -114.728874000000104, 36.021387 ], [ -114.723324, 36.026588 ], [ -114.722214, 36.027964 ], [ -114.722096, 36.028952 ], [ -114.722742, 36.030286 ], [ -114.723673, 36.03123 ], [ -114.727602, 36.033099 ], [ -114.730563, 36.036207 ], [ -114.733417000000102, 36.037913 ], [ -114.735739, 36.038033 ], [ -114.740018, 36.037467 ], [ -114.741262000000106, 36.038044 ], [ -114.742105, 36.039792 ], [ -114.742661, 36.042573 ], [ -114.742479, 36.045697 ], [ -114.741677, 36.047877 ], [ -114.735701, 36.053393 ], [ -114.73508, 36.054435 ], [ -114.735285, 36.056648 ], [ -114.736023000000102, 36.059063 ], [ -114.74006, 36.062437 ], [ -114.7422, 36.067833 ], [ -114.742138, 36.068676 ], [ -114.743542, 36.071037 ], [ -114.748891, 36.074981 ], [ -114.75057, 36.08033 ], [ -114.754032, 36.083093 ], [ -114.754681, 36.085052 ], [ -114.754508, 36.086171 ], [ -114.752836, 36.089393 ], [ -114.750095, 36.092275 ], [ -114.748913000000101, 36.095183 ], [ -114.737497000000104, 36.103102 ], [ -114.734857, 36.104426 ], [ -114.718257, 36.107164 ], [ -114.709269, 36.107396 ], [ -114.706091, 36.108239 ], [ -114.703737, 36.108348 ], [ -114.696981, 36.110297 ], [ -114.693655000000106, 36.112482 ], [ -114.691631, 36.112535 ], [ -114.6880740000001, 36.111457 ], [ -114.684426000000101, 36.109472 ], [ -114.681847000000104, 36.109192 ], [ -114.679775000000106, 36.109874 ], [ -114.678375, 36.110815 ], [ -114.675106, 36.114111 ], [ -114.671867, 36.115964 ], [ -114.664343000000102, 36.1163 ], [ -114.662144, 36.117742 ], [ -114.660448, 36.119999 ], [ -114.658131, 36.124127 ], [ -114.655512, 36.126187 ], [ -114.645728, 36.131995 ], [ -114.641976, 36.13373 ], [ -114.640125, 36.135126 ], [ -114.636862, 36.135552 ], [ -114.635809, 36.13617 ], [ -114.630474, 36.142218 ], [ -114.628462, 36.141822 ], [ -114.627079, 36.140761 ], [ -114.623837, 36.137144 ], [ -114.620605, 36.131759 ], [ -114.618429, 36.130328 ], [ -114.615455, 36.129653 ], [ -114.61324, 36.130266 ], [ -114.609288, 36.132229 ], [ -114.596474, 36.141537 ], [ -114.5930350000001, 36.142674 ], [ -114.589828, 36.143192 ], [ -114.583716, 36.14556 ], [ -114.580707, 36.145987 ], [ -114.578828, 36.147175 ], [ -114.57706, 36.148845 ], [ -114.57109, 36.151099 ], [ -114.561173, 36.150921 ], [ -114.556162, 36.15247 ], [ -114.548742000000104, 36.150697 ], [ -114.543232, 36.151871 ], [ -114.539233, 36.151764 ], [ -114.534478, 36.15023 ], [ -114.532924, 36.149282 ], [ -114.532308, 36.14804 ], [ -114.531091, 36.147644 ], [ -114.52621, 36.148177 ], [ -114.51428, 36.150795 ], [ -114.511218, 36.150576 ], [ -114.508104000000102, 36.149713 ], [ -114.501049, 36.144516 ], [ -114.500236, 36.143226 ], [ -114.499992, 36.141594 ], [ -114.500339, 36.1407 ], [ -114.50108, 36.14006 ], [ -114.50515, 36.138078 ], [ -114.507175, 36.13634 ], [ -114.50921, 36.133247 ], [ -114.508467, 36.129913 ], [ -114.507201, 36.128484 ], [ -114.504715, 36.127188 ], [ -114.501798, 36.126556 ], [ -114.498849, 36.126612 ], [ -114.487635, 36.128656 ], [ -114.483827, 36.12972 ], [ -114.478248, 36.132683 ], [ -114.468674, 36.138889 ], [ -114.465579, 36.139496 ], [ -114.4626, 36.139644 ], [ -114.458945, 36.139214 ], [ -114.456487, 36.138032 ], [ -114.45511, 36.136372 ], [ -114.453798, 36.133586 ], [ -114.451331, 36.129831 ], [ -114.447135, 36.126022 ], [ -114.445042, 36.125346 ], [ -114.443736, 36.125593 ], [ -114.435507, 36.130057 ], [ -114.423114, 36.13735 ], [ -114.418193, 36.142771 ], [ -114.415253, 36.145123 ], [ -114.412491000000102, 36.146511 ], [ -114.40914, 36.147 ], [ -114.405624, 36.146983 ], [ -114.398373, 36.145799 ], [ -114.381479, 36.141349 ], [ -114.379976, 36.141388 ], [ -114.375278, 36.143592 ], [ -114.373745, 36.143722 ], [ -114.370181, 36.142624 ], [ -114.368551, 36.140892 ], [ -114.367381, 36.13852 ], [ -114.365529, 36.136306 ], [ -114.364499, 36.134072 ], [ -114.358968, 36.127795 ], [ -114.348592, 36.121147 ], [ -114.3451, 36.118556 ], [ -114.342601, 36.115878 ], [ -114.34095, 36.113457 ], [ -114.338815, 36.111309 ], [ -114.337264, 36.110428 ], [ -114.334632, 36.106784 ], [ -114.333587, 36.106342 ], [ -114.328801, 36.105902 ], [ -114.325814, 36.103933 ], [ -114.325539, 36.102989 ], [ -114.323458000000102, 36.101186 ], [ -114.320866, 36.096463 ], [ -114.316983, 36.093409 ], [ -114.313086, 36.088816 ], [ -114.306939, 36.082487 ], [ -114.304171, 36.07558 ], [ -114.304384, 36.074019 ], [ -114.305853, 36.071478 ], [ -114.307485, 36.069672 ], [ -114.31242, 36.066117 ], [ -114.3136, 36.064148 ], [ -114.314328, 36.062016 ], [ -114.314427, 36.060523 ], [ -114.313591, 36.059048 ], [ -114.311904, 36.057661 ], [ -114.308624, 36.056976 ], [ -114.300971000000104, 36.05746 ], [ -114.298593, 36.057263 ], [ -114.295941, 36.056168 ], [ -114.293435000000102, 36.0545 ], [ -114.290867, 36.050511 ], [ -114.287992, 36.04907 ], [ -114.284006, 36.048242 ], [ -114.279637, 36.046103 ], [ -114.278166, 36.045819 ], [ -114.273911, 36.046529 ], [ -114.272299, 36.046289 ], [ -114.270862, 36.045523 ], [ -114.269548, 36.043769 ], [ -114.268896, 36.04094 ], [ -114.26922, 36.036807 ], [ -114.268586, 36.035034 ], [ -114.26438, 36.027911 ], [ -114.262388, 36.026107 ], [ -114.259518, 36.024206 ], [ -114.251633, 36.019886 ], [ -114.248419, 36.018556 ], [ -114.246111, 36.017164 ], [ -114.243865, 36.015266 ], [ -114.240439, 36.015245 ], [ -114.238154, 36.014473 ], [ -114.236892, 36.013247 ], [ -114.233443, 36.012835 ], [ -114.231854, 36.013147 ], [ -114.228015, 36.014731 ], [ -114.226459, 36.014606 ], [ -114.224798, 36.013699 ], [ -114.218759, 36.014511 ], [ -114.216609, 36.014336 ], [ -114.214679, 36.014806 ], [ -114.213549, 36.014615 ], [ -114.211932, 36.014834 ], [ -114.206052, 36.016634 ], [ -114.204156, 36.016575 ], [ -114.201227, 36.017751 ], [ -114.200066, 36.017743 ], [ -114.191221, 36.020019 ], [ -114.185860000000105, 36.022266 ], [ -114.179438, 36.024313 ], [ -114.176304, 36.026129 ], [ -114.174683, 36.02667 ], [ -114.164402, 36.026852 ], [ -114.161237, 36.026279 ], [ -114.157344, 36.024966 ], [ -114.1534, 36.02317 ], [ -114.15139, 36.023133 ], [ -114.150225, 36.023515 ], [ -114.145907, 36.027229 ], [ -114.145637, 36.028559 ], [ -114.145672, 36.03297 ], [ -114.144666, 36.034272 ], [ -114.143153, 36.035295 ], [ -114.13826, 36.03719 ], [ -114.137112, 36.038491 ], [ -114.135721, 36.041238 ], [ -114.134841, 36.043873 ], [ -114.134824, 36.045343 ], [ -114.135927, 36.050358 ], [ -114.136206, 36.053232 ], [ -114.1352, 36.056946 ], [ -114.133389, 36.061665 ], [ -114.129768, 36.068484 ], [ -114.125891, 36.072935 ], [ -114.124019, 36.075563 ], [ -114.121186, 36.082755 ], [ -114.119648, 36.085822 ], [ -114.112297, 36.09405 ], [ -114.111998, 36.09491 ], [ -114.1119, 36.095845 ], [ -114.115208, 36.099878 ], [ -114.11707, 36.101177 ], [ -114.119329, 36.10193 ], [ -114.121033, 36.103885 ], [ -114.121779, 36.105699 ], [ -114.12167, 36.108294 ], [ -114.120865, 36.11085 ], [ -114.118497, 36.1139 ], [ -114.116061, 36.115471 ], [ -114.108381, 36.119154 ], [ -114.107419, 36.119401 ], [ -114.100433, 36.119359 ], [ -114.097707, 36.120213 ], [ -114.096994, 36.120823 ], [ -114.092753, 36.132356 ], [ -114.092366, 36.135331 ], [ -114.091701, 36.137303 ], [ -114.089279, 36.140326 ], [ -114.087899, 36.142923 ], [ -114.081234, 36.150208 ], [ -114.07945, 36.154625 ], [ -114.078832, 36.157434 ], [ -114.075641, 36.162523 ], [ -114.071652, 36.170921 ], [ -114.066798, 36.179087 ], [ -114.058662, 36.187835 ], [ -114.052743, 36.190919 ], [ -114.049484, 36.192134 ], [ -114.043944, 36.19335 ], [ -114.043849, 36.245114 ], [ -114.045518, 36.27439 ], [ -114.045559, 36.288837 ], [ -114.045033, 36.30305 ], [ -114.044345, 36.310234 ], [ -114.044051, 36.317628 ], [ -114.044776, 36.331969 ], [ -114.044702, 36.346298 ], [ -114.043034, 36.38587 ], [ -114.042843000000104, 36.448175 ], [ -114.043133, 36.469716 ], [ -114.044816, 36.491343 ], [ -114.045647, 36.521095 ], [ -114.04632, 36.564615 ], [ -114.049935, 36.709521 ], [ -114.049973, 36.738672 ], [ -114.050327, 36.752899 ], [ -114.049879, 36.781909 ], [ -114.050502, 36.895232 ], [ -114.049995, 36.957769 ], [ -114.050600000000102, 37.000396 ], [ -114.0008, 37.000448 ], [ -113.96266, 36.999973 ], [ -113.052912, 36.999983 ], [ -112.875756, 37.000533 ], [ -112.538546, 37.000652 ], [ -112.529846, 37.000899 ], [ -112.36102, 37.001114 ], [ -112.36037, 37.000912 ], [ -112.359329, 37.001117 ], [ -112.125741, 37.001237 ], [ -112.000735, 37.000959 ], [ -111.62572, 37.001401 ], [ -111.616249, 37.001647 ], [ -111.406146, 37.001481 ], [ -111.405895, 37.001702 ], [ -111.313211, 37.000894 ], [ -111.312169, 37.001193 ], [ -111.305843, 37.000776 ], [ -111.278221, 37.000467 ], [ -111.254853, 37.001076 ], [ -111.133718, 37.000779 ], [ -111.081493, 37.002261 ], [ -111.052354, 37.00246 ], [ -111.00182, 37.002293 ], [ -110.625691, 37.003725 ], [ -110.625605, 37.003416 ], [ -110.599512, 37.003448 ], [ -110.509004, 37.003985 ], [ -110.50069, 37.00426 ], [ -110.490908, 37.003566 ], [ -110.478446, 36.999996 ], [ -110.47729, 36.999997 ], [ -110.47019, 36.997997 ], [ -110.023043, 36.998601 ], [ -110.000876, 36.998502 ], [ -110.000677, 36.997968 ], [ -109.969958, 36.997949 ], [ -109.938511, 36.998491 ], [ -109.750669, 36.99816 ], [ -109.743284, 36.998453 ], [ -109.625658, 36.998308 ], [ -109.495338, 36.999105 ], [ -109.362565, 36.999304 ], [ -109.125691, 36.999389 ], [ -109.045223, 36.999084 ], [ -109.045554, 36.645013 ], [ -109.04539, 36.503241 ], [ -109.045946, 36.375002 ], [ -109.045637, 36.374625 ], [ -109.045744, 36.257214 ], [ -109.046024, 36.247197 ], [ -109.045877, 36.188719 ], [ -109.046183, 36.181751 ], [ -109.045726, 36.116908 ], [ -109.045767, 36.033679 ], [ -109.046124, 35.990618 ], [ -109.046009, 35.875012 ], [ -109.046423, 35.624911 ], [ -109.046181, 35.614569 ], [ -109.046795, 35.379918 ], [ -109.046084, 35.249986 ], [ -109.046256, 35.125041 ], [ -109.045842, 34.966076 ], [ -109.046136, 34.875006 ], [ -109.046072, 34.828566 ], [ -109.045626, 34.814226 ], [ -109.046104, 34.799981 ], [ -109.045363, 34.785406 ], [ -109.046087, 34.770963 ], [ -109.046175, 34.520102 ], [ -109.046561, 34.379479 ], [ -109.046337, 34.283639 ], [ -109.046664, 34.250046 ], [ -109.04696, 34.068968 ], [ -109.047006, 34.00005 ], [ -109.046426, 33.875052 ], [ -109.046869, 33.844183 ], [ -109.047145, 33.74001 ], [ -109.046662, 33.625055 ], [ -109.046825, 33.469389 ], [ -109.047309, 33.462131 ], [ -109.046928, 33.4428 ], [ -109.047304, 33.439442 ], [ -109.047298, 33.409774 ], [ -109.046564, 33.375059 ], [ -109.047045, 33.36928 ], [ -109.046827, 33.365271 ], [ -109.047104, 33.27046 ], [ -109.04747, 33.250168 ], [ -109.047122, 33.2408 ], [ -109.047324, 33.18408 ], [ -109.047208, 33.107377 ], [ -109.046905, 33.091931 ], [ -109.047513, 33.059137 ], [ -109.047382, 33.000311 ], [ -109.04711, 32.99225 ], [ -109.047117, 32.777569 ], [ -109.047518, 32.749997 ], [ -109.047796, 32.68263 ], [ -109.047912, 32.500261 ], [ -109.047629, 32.413987 ], [ -109.048323, 32.070887 ], [ -109.048731, 32.028174 ], [ -109.048465, 32.000089 ], [ -109.048738, 31.876905 ], [ -109.049048, 31.870689 ], [ -109.049298, 31.796742 ], [ -109.04899, 31.721922 ], [ -109.049311, 31.544932 ], [ -109.050173, 31.480004 ], [ -109.049934, 31.437907 ], [ -109.050044, 31.332502 ], [ -109.1256, 31.332685 ], [ -109.271744, 31.333942 ], [ -109.49449, 31.334125 ], [ -109.500621, 31.333911 ], [ -109.875628, 31.33405 ], [ -110.000613, 31.333145 ], [ -110.140512, 31.333965 ], [ -110.375635, 31.332896 ], [ -110.460172, 31.332827 ], [ -110.68143, 31.33309 ], [ -110.750638, 31.333636 ], [ -110.795467, 31.33363 ], [ -110.94232, 31.332833 ], [ -111.000643, 31.332177 ], [ -111.074825, 31.332239 ], [ -111.125646, 31.348978 ], [ -111.129451, 31.349979 ], [ -111.357436, 31.423346 ], [ -111.500659, 31.468862 ], [ -111.560194, 31.488138 ], [ -111.659998, 31.519448 ], [ -111.738873, 31.544718 ], [ -111.875674, 31.587657 ], [ -111.979304, 31.620648 ], [ -112.200717, 31.690033 ], [ -112.365328, 31.741078 ], [ -112.375759, 31.743987 ], [ -112.399254, 31.751638 ], [ -112.433246, 31.762162 ], [ -112.737399, 31.855527 ], [ -112.800213, 31.87507 ], [ -112.834233, 31.885137 ], [ -112.871505, 31.896838 ], [ -113.125961, 31.97278 ], [ -113.21163, 32.000061 ], [ -113.211365, 32.000061 ], [ -113.217307, 32.002106 ], [ -113.250731, 32.012405 ], [ -113.493196, 32.088943 ], [ -113.750756, 32.169005 ], [ -113.78168, 32.179034 ], [ -114.250775, 32.323909 ], [ -114.625785, 32.43789 ], [ -114.790245, 32.487505 ], [ -114.813613, 32.494276 ], [ -114.813991000000101, 32.497231 ], [ -114.812316, 32.500054 ], [ -114.813402, 32.501764 ], [ -114.813753000000105, 32.50426 ], [ -114.815185, 32.506023 ], [ -114.81651, 32.506963 ], [ -114.816591000000102, 32.507696 ], [ -114.815591, 32.508612 ], [ -114.814321000000106, 32.509023 ], [ -114.812942, 32.509116 ], [ -114.810159, 32.508383 ], [ -114.807726, 32.508726 ], [ -114.804076, 32.510375 ], [ -114.802833, 32.511749 ], [ -114.802211, 32.513191 ], [ -114.802238, 32.515206 ], [ -114.80367, 32.516374 ], [ -114.807753, 32.516925 ], [ -114.809672, 32.517567 ], [ -114.810374, 32.518391 ], [ -114.809969, 32.520291 ], [ -114.810482, 32.521758 ], [ -114.810969, 32.522444 ], [ -114.812888, 32.52359 ], [ -114.813348000000104, 32.524186 ], [ -114.812645, 32.525399 ], [ -114.811293, 32.526429 ], [ -114.810563, 32.527666 ], [ -114.808617, 32.529017 ], [ -114.80640000000011, 32.531191 ], [ -114.804858, 32.533689 ], [ -114.802559, 32.535521 ], [ -114.802181, 32.536414 ], [ -114.802018, 32.53946 ], [ -114.80237, 32.540078 ], [ -114.804776000000103, 32.541659 ], [ -114.805966, 32.545346 ], [ -114.8058300000001, 32.546354 ], [ -114.803883, 32.548001 ], [ -114.795635, 32.550956 ], [ -114.793769, 32.552329 ], [ -114.792065, 32.555009 ], [ -114.791551, 32.557023 ], [ -114.791523, 32.558602 ], [ -114.792955000000106, 32.562085 ], [ -114.792088, 32.568497 ], [ -114.792358000000107, 32.569091 ], [ -114.793224, 32.569459 ], [ -114.794684, 32.568703 ], [ -114.795253, 32.56662 ], [ -114.79766, 32.564444 ], [ -114.801311, 32.562865 ], [ -114.803664, 32.560689 ], [ -114.806830000000105, 32.55888 ], [ -114.808885, 32.558467 ], [ -114.810318, 32.558628 ], [ -114.812914, 32.560049 ], [ -114.813995, 32.562201 ], [ -114.814212, 32.56369 ], [ -114.813968, 32.566209 ], [ -114.812995, 32.568706 ], [ -114.81148, 32.569781 ], [ -114.804421, 32.572941 ], [ -114.803474, 32.573628 ], [ -114.801877, 32.576009 ], [ -114.801471, 32.578255 ], [ -114.80193, 32.579194 ], [ -114.803879, 32.580889 ], [ -114.803987, 32.582652 ], [ -114.802823, 32.585079 ], [ -114.800441, 32.588079 ], [ -114.799737, 32.592177 ], [ -114.799683, 32.593621 ], [ -114.801251, 32.596232 ], [ -114.801548, 32.598591 ], [ -114.802361, 32.59937 ], [ -114.805932, 32.600721 ], [ -114.8069050000001, 32.60143 ], [ -114.808041, 32.603172 ], [ -114.807879, 32.605416 ], [ -114.809042, 32.608806 ], [ -114.808906, 32.612951 ], [ -114.809555, 32.616203 ], [ -114.808662, 32.619157 ], [ -114.80739, 32.621332 ], [ -114.806821, 32.621721 ], [ -114.799302, 32.625115 ], [ -114.797564, 32.624578 ], [ -114.794102, 32.622475 ], [ -114.792640000000105, 32.621948 ], [ -114.791179, 32.621833 ], [ -114.787715, 32.623573 ], [ -114.782573, 32.624304 ], [ -114.781896, 32.624702 ], [ -114.781766, 32.625613 ], [ -114.782518, 32.628625 ], [ -114.782235, 32.630215 ], [ -114.779215, 32.633578 ], [ -114.77457, 32.63593 ], [ -114.771978, 32.637954 ], [ -114.768199, 32.639874 ], [ -114.764382, 32.642666 ], [ -114.76331, 32.644616 ], [ -114.763512, 32.645995 ], [ -114.764917, 32.648079 ], [ -114.76495, 32.649391 ], [ -114.75831, 32.655178 ], [ -114.751079, 32.659789 ], [ -114.749480000000105, 32.66178 ], [ -114.748000000000104, 32.664184 ], [ -114.748183, 32.665098 ], [ -114.747817, 32.667777 ], [ -114.746383, 32.669853 ], [ -114.745344, 32.67219 ], [ -114.7449, 32.677231 ], [ -114.744349, 32.678935 ], [ -114.740541, 32.684196 ], [ -114.739405, 32.686385 ], [ -114.730453, 32.698844 ], [ -114.72981, 32.700282 ], [ -114.72974, 32.703121 ], [ -114.730086, 32.704298 ], [ -114.728408, 32.706648 ], [ -114.726974, 32.707875 ], [ -114.72534, 32.710369 ], [ -114.72241, 32.713597 ], [ -114.719938, 32.71829 ], [ -114.717695, 32.721547 ], [ -114.715788, 32.727758 ], [ -114.714522, 32.73039 ], [ -114.712629, 32.732678 ], [ -114.710615, 32.733936 ], [ -114.709074, 32.735456 ], [ -114.706114, 32.740986 ], [ -114.70294, 32.744793 ], [ -114.701582000000101, 32.745632 ], [ -114.699247, 32.745098 ], [ -114.695387, 32.742244 ], [ -114.691801, 32.740147 ], [ -114.689282, 32.737927 ], [ -114.688230000000104, 32.73753 ], [ -114.682614, 32.737348 ], [ -114.672025, 32.734951 ], [ -114.665921, 32.734028 ], [ -114.654247, 32.73357 ], [ -114.645353, 32.732139 ], [ -114.635006000000104, 32.731372 ], [ -114.629299, 32.729908 ], [ -114.617479, 32.728243 ], [ -114.61567, 32.728454 ], [ -114.61587, 32.729717 ], [ -114.615501, 32.730044 ], [ -114.615504, 32.731449 ], [ -114.614786, 32.732846 ], [ -114.614787000000106, 32.734076 ], [ -114.615112, 32.734515 ], [ -114.581784, 32.734946 ], [ -114.581736, 32.74232 ], [ -114.564508, 32.742274 ], [ -114.564447, 32.749554 ], [ -114.539224, 32.749812 ], [ -114.539092, 32.756949 ], [ -114.526856, 32.757094 ], [ -114.528443, 32.767276 ], [ -114.529264, 32.769484 ], [ -114.531831, 32.774264 ], [ -114.532432, 32.776922 ], [ -114.532426, 32.778644 ], [ -114.531746, 32.782503 ], [ -114.531669, 32.791185 ], [ -114.529633000000103, 32.795477 ], [ -114.522031, 32.801675 ], [ -114.520385, 32.803576 ], [ -114.520363000000103, 32.804385 ], [ -114.519758, 32.805676 ], [ -114.515389, 32.811439 ], [ -114.510327, 32.816488 ], [ -114.494116, 32.823287 ], [ -114.475892, 32.838693 ], [ -114.468971, 32.845155 ], [ -114.465711, 32.873681 ], [ -114.465172, 32.885295 ], [ -114.463307, 32.899116 ], [ -114.462929, 32.907944 ], [ -114.46365, 32.911682 ], [ -114.464448, 32.913128 ], [ -114.473713, 32.920594 ], [ -114.47664, 32.923628 ], [ -114.477952000000101, 32.925706 ], [ -114.479005, 32.928291 ], [ -114.480783, 32.933678 ], [ -114.480925, 32.936276 ], [ -114.48074, 32.937027 ], [ -114.478456, 32.940555 ], [ -114.474042, 32.94515 ], [ -114.470768, 32.949424 ], [ -114.468536, 32.953922 ], [ -114.467624, 32.956663 ], [ -114.467274, 32.960172 ], [ -114.467367, 32.965384 ], [ -114.468379, 32.970745 ], [ -114.468995, 32.972239 ], [ -114.470511, 32.973858 ], [ -114.472606, 32.974654 ], [ -114.475171, 32.975154 ], [ -114.477308, 32.975023 ], [ -114.479477, 32.974189 ], [ -114.480831, 32.973362 ], [ -114.481315, 32.972064 ], [ -114.484806, 32.971339 ], [ -114.488625, 32.969946 ], [ -114.490129, 32.969884 ], [ -114.492184, 32.971021 ], [ -114.492938, 32.971781 ], [ -114.494212, 32.974262 ], [ -114.495712, 32.980075 ], [ -114.496798, 32.986534 ], [ -114.497052, 32.990206 ], [ -114.49941, 33.00004 ], [ -114.499797, 33.003905 ], [ -114.50287, 33.011154 ], [ -114.506129, 33.017009 ], [ -114.507956, 33.019708 ], [ -114.511343, 33.023455 ], [ -114.5149, 33.026524 ], [ -114.52013, 33.029984 ], [ -114.523578, 33.03096 ], [ -114.538459, 33.033422 ], [ -114.553189, 33.033974 ], [ -114.56085, 33.035285 ], [ -114.5648, 33.035077 ], [ -114.571653, 33.036624 ], [ -114.575161, 33.036541 ], [ -114.578287, 33.035375 ], [ -114.581404, 33.032545 ], [ -114.584765, 33.02823 ], [ -114.586982, 33.026944 ], [ -114.589778, 33.026228 ], [ -114.598093, 33.025384 ], [ -114.601014, 33.02541 ], [ -114.611584, 33.026221 ], [ -114.618788, 33.027202 ], [ -114.625787, 33.029435 ], [ -114.628294, 33.03105 ], [ -114.629732, 33.032546 ], [ -114.63419, 33.039024 ], [ -114.639552, 33.04529 ], [ -114.641621, 33.046894 ], [ -114.64482, 33.048644 ], [ -114.645979, 33.048902 ], [ -114.647049, 33.048416 ], [ -114.649001, 33.046762 ], [ -114.650999, 33.044131 ], [ -114.655038000000104, 33.037106 ], [ -114.657827, 33.033824 ], [ -114.659832, 33.032664 ], [ -114.662317, 33.03267 ], [ -114.66506, 33.033906 ], [ -114.670803, 33.037983 ], [ -114.673659, 33.041896 ], [ -114.67483, 33.045507 ], [ -114.675103000000107, 33.04753 ], [ -114.674295, 33.057169 ], [ -114.679114, 33.061966 ], [ -114.686991, 33.070968 ], [ -114.68912, 33.076121 ], [ -114.689307, 33.079179 ], [ -114.688597000000101, 33.082869 ], [ -114.68902, 33.084035 ], [ -114.692548000000102, 33.085786 ], [ -114.694628, 33.086226 ], [ -114.701165000000103, 33.086368 ], [ -114.70473, 33.087051 ], [ -114.706488, 33.08816 ], [ -114.707819, 33.091102 ], [ -114.708133000000103, 33.094022 ], [ -114.707896000000105, 33.097431 ], [ -114.706175, 33.105334 ], [ -114.703682, 33.113768 ], [ -114.696914, 33.131119 ], [ -114.694858, 33.13346 ], [ -114.690246, 33.137724 ], [ -114.687405, 33.141983 ], [ -114.684907, 33.147823 ], [ -114.682759, 33.154808 ], [ -114.679945, 33.159059 ], [ -114.67935, 33.162433 ], [ -114.68089, 33.169074 ], [ -114.680237, 33.169637 ], [ -114.679115, 33.174608 ], [ -114.675830000000104, 33.18152 ], [ -114.6753590000001, 33.185488 ], [ -114.675189, 33.188178 ], [ -114.678163, 33.199488 ], [ -114.678749, 33.203448 ], [ -114.676072, 33.210835 ], [ -114.673715, 33.219245 ], [ -114.673626, 33.223121 ], [ -114.674479000000105, 33.225504 ], [ -114.678097, 33.2303 ], [ -114.682731, 33.234918 ], [ -114.689421, 33.24525 ], [ -114.689541, 33.246428 ], [ -114.688205, 33.247965 ], [ -114.683253, 33.250034 ], [ -114.67766, 33.254426 ], [ -114.674491000000103, 33.255597 ], [ -114.672924, 33.257042 ], [ -114.672088, 33.258499 ], [ -114.672401, 33.260469 ], [ -114.677032, 33.270169 ], [ -114.680507, 33.273576 ], [ -114.684363000000104, 33.276023 ], [ -114.688599, 33.277861 ], [ -114.694449, 33.279785 ], [ -114.702873, 33.281916 ], [ -114.711197, 33.283341 ], [ -114.717875, 33.285156 ], [ -114.72167, 33.286982 ], [ -114.723259, 33.288079 ], [ -114.731223, 33.302433 ], [ -114.731222000000102, 33.304039 ], [ -114.729904000000104, 33.305745 ], [ -114.726484, 33.308273 ], [ -114.724665000000101, 33.310097 ], [ -114.723623000000103, 33.312109 ], [ -114.71861, 33.315761 ], [ -114.710627, 33.3205 ], [ -114.70787, 33.323316 ], [ -114.705186, 33.327709 ], [ -114.700938, 33.337014 ], [ -114.69935, 33.345692 ], [ -114.699124, 33.349258 ], [ -114.698035, 33.352442 ], [ -114.69817, 33.356575 ], [ -114.699056, 33.361148 ], [ -114.701959, 33.367134 ], [ -114.704201, 33.371238 ], [ -114.706722, 33.37503 ], [ -114.707348, 33.376627 ], [ -114.707485000000105, 33.378375 ], [ -114.707009, 33.380633 ], [ -114.707309, 33.38254 ], [ -114.708407, 33.384142 ], [ -114.713602, 33.388256 ], [ -114.72425, 33.40042 ], [ -114.725292, 33.402341 ], [ -114.725535, 33.404055 ], [ -114.725282, 33.405048 ], [ -114.723829, 33.406531 ], [ -114.722201, 33.407384 ], [ -114.720065000000105, 33.407891 ], [ -114.710878, 33.407254 ], [ -114.701788, 33.408377 ], [ -114.697708, 33.410942 ], [ -114.696805, 33.412087 ], [ -114.696507, 33.414063 ], [ -114.695658, 33.415128 ], [ -114.68795, 33.417934 ], [ -114.673691, 33.419157 ], [ -114.658254, 33.413021 ], [ -114.656735, 33.412813 ], [ -114.652828, 33.412923 ], [ -114.64954, 33.413633 ], [ -114.643302, 33.416746 ], [ -114.635183, 33.422725 ], [ -114.633262, 33.425024 ], [ -114.630903, 33.426754 ], [ -114.62964, 33.428137 ], [ -114.627479, 33.432307 ], [ -114.622283, 33.447558 ], [ -114.622519, 33.450879 ], [ -114.623395000000102, 33.45449 ], [ -114.622918, 33.456561 ], [ -114.618354, 33.462708 ], [ -114.614331, 33.467315 ], [ -114.6137820000001, 33.469049 ], [ -114.612472, 33.470768 ], [ -114.607843000000102, 33.474834 ], [ -114.603396000000103, 33.480631 ], [ -114.601694, 33.481396 ], [ -114.599712, 33.484316 ], [ -114.597283000000104, 33.490653 ], [ -114.593721, 33.495932 ], [ -114.592369, 33.498675 ], [ -114.589246, 33.501813 ], [ -114.580468, 33.506465 ], [ -114.573757, 33.507543 ], [ -114.569533, 33.509219 ], [ -114.560963, 33.516739 ], [ -114.560552, 33.518272 ], [ -114.560835, 33.524334 ], [ -114.560098, 33.526663 ], [ -114.559507, 33.530724 ], [ -114.558898, 33.531819 ], [ -114.542011, 33.542481 ], [ -114.531802, 33.547862 ], [ -114.530401, 33.550099 ], [ -114.525997000000103, 33.551457 ], [ -114.524599, 33.552231 ], [ -114.524215, 33.553068 ], [ -114.52822, 33.559318 ], [ -114.531613, 33.561702 ], [ -114.532333, 33.562879 ], [ -114.533192, 33.565823 ], [ -114.535965000000104, 33.569154 ], [ -114.536784, 33.570959 ], [ -114.537801, 33.575555 ], [ -114.538983, 33.576792 ], [ -114.5403, 33.580615 ], [ -114.540652, 33.582872 ], [ -114.540111, 33.588354 ], [ -114.540664, 33.589789 ], [ -114.540617, 33.591412 ], [ -114.537493, 33.594895 ], [ -114.536777, 33.596394 ], [ -114.531051, 33.604482 ], [ -114.529186, 33.60665 ], [ -114.526782, 33.608831 ], [ -114.523994, 33.60999 ], [ -114.522071, 33.611277 ], [ -114.521845, 33.612544 ], [ -114.522367, 33.614172 ], [ -114.527378, 33.617828 ], [ -114.528578, 33.619994 ], [ -114.52908, 33.621711 ], [ -114.531215, 33.623913 ], [ -114.531034, 33.628213 ], [ -114.530311, 33.629037 ], [ -114.52637, 33.630259 ], [ -114.523802, 33.6347 ], [ -114.525394, 33.640669 ], [ -114.529549000000102, 33.643861 ], [ -114.533215, 33.648443 ], [ -114.533194, 33.65166 ], [ -114.532164, 33.653194 ], [ -114.530583, 33.654461 ], [ -114.525163, 33.655939 ], [ -114.518337, 33.655927 ], [ -114.514559000000105, 33.658014 ], [ -114.514057, 33.660179 ], [ -114.515336, 33.662033 ], [ -114.517112, 33.662877 ], [ -114.520671, 33.662681 ], [ -114.526439, 33.66388 ], [ -114.530267, 33.666821 ], [ -114.532123, 33.669702 ], [ -114.531523, 33.675108 ], [ -114.530348, 33.679245 ], [ -114.527782, 33.682684 ], [ -114.523959, 33.685879 ], [ -114.519113, 33.688473 ], [ -114.512409, 33.691282 ], [ -114.507996, 33.692018 ], [ -114.504993, 33.693022 ], [ -114.502899, 33.694255 ], [ -114.496489, 33.696901 ], [ -114.495719, 33.698454 ], [ -114.495537, 33.701506 ], [ -114.494407, 33.705395 ], [ -114.494197, 33.707922 ], [ -114.494901, 33.71443 ], [ -114.496565, 33.719155 ], [ -114.498133, 33.720634 ], [ -114.500788, 33.722204 ], [ -114.502661, 33.724584 ], [ -114.504176, 33.728055 ], [ -114.506799, 33.730518 ], [ -114.510265, 33.732146 ], [ -114.512348000000102, 33.734214 ], [ -114.510777, 33.737574 ], [ -114.508206, 33.741587 ], [ -114.506, 33.746344 ], [ -114.504483, 33.750998 ], [ -114.50434, 33.756381 ], [ -114.504863, 33.760465 ], [ -114.507089, 33.76793 ], [ -114.516734, 33.788345 ], [ -114.518942, 33.797302 ], [ -114.521555, 33.801982 ], [ -114.524682, 33.808961 ], [ -114.527188, 33.812639 ], [ -114.52805, 33.814963 ], [ -114.527886, 33.815617 ], [ -114.527161, 33.816191 ], [ -114.522714, 33.818979 ], [ -114.520733, 33.822031 ], [ -114.51997, 33.825381 ], [ -114.520465, 33.827778 ], [ -114.523409, 33.835323 ], [ -114.525539, 33.838614 ], [ -114.529597, 33.848063 ], [ -114.530607, 33.85544 ], [ -114.529883, 33.857563 ], [ -114.527069, 33.859429 ], [ -114.525666, 33.860003 ], [ -114.524292, 33.860133 ], [ -114.52287, 33.859965 ], [ -114.518998, 33.858563 ], [ -114.516811000000104, 33.85812 ], [ -114.514673, 33.858638 ], [ -114.511346, 33.86157 ], [ -114.506635, 33.863484 ], [ -114.505638, 33.864276 ], [ -114.503887, 33.865754 ], [ -114.503104, 33.867166 ], [ -114.503017, 33.867998 ], [ -114.503860000000103, 33.871234 ], [ -114.503395, 33.875018 ], [ -114.50434, 33.876882 ], [ -114.510138, 33.880777 ], [ -114.51866, 33.888263 ], [ -114.522768, 33.892583 ], [ -114.524813, 33.895684 ], [ -114.525872, 33.901008 ], [ -114.52569, 33.901428 ], [ -114.524289, 33.901587 ], [ -114.516344, 33.897918 ], [ -114.513715, 33.897959 ], [ -114.510944, 33.899099 ], [ -114.508708, 33.90064 ], [ -114.507988, 33.901813 ], [ -114.50792, 33.903807 ], [ -114.508558, 33.906098 ], [ -114.511511, 33.911092 ], [ -114.514503, 33.914214 ], [ -114.518434, 33.917518 ], [ -114.523393, 33.921221 ], [ -114.525361, 33.922272 ], [ -114.528385, 33.923674 ], [ -114.531107, 33.924633 ], [ -114.534146, 33.925187 ], [ -114.534951, 33.9257 ], [ -114.535853, 33.928103 ], [ -114.535478, 33.934651 ], [ -114.530566, 33.943629 ], [ -114.52868, 33.947817 ], [ -114.526353, 33.950917 ], [ -114.522002, 33.955623 ], [ -114.51586, 33.958106 ], [ -114.51497, 33.958149 ], [ -114.511231, 33.95704 ], [ -114.509568, 33.957264 ], [ -114.499883, 33.961789 ], [ -114.496042, 33.96589 ], [ -114.490398, 33.97062 ], [ -114.488459, 33.972832 ], [ -114.484784, 33.975519 ], [ -114.483097, 33.977745 ], [ -114.482333, 33.980181 ], [ -114.481455, 33.981261 ], [ -114.475907, 33.984424 ], [ -114.471138, 33.98804 ], [ -114.467932, 33.992877 ], [ -114.466187, 33.993465 ], [ -114.461133, 33.993541 ], [ -114.46012, 33.993888 ], [ -114.459258, 33.994711 ], [ -114.458028, 33.997158 ], [ -114.458026000000103, 33.99782 ], [ -114.459184, 34.000016 ], [ -114.460689, 34.001128 ], [ -114.46628, 34.003885 ], [ -114.46731, 34.00519 ], [ -114.467404, 34.00745 ], [ -114.465867000000102, 34.010987 ], [ -114.464525, 34.011982 ], [ -114.463336, 34.012259 ], [ -114.454807, 34.010968 ], [ -114.450206, 34.012574 ], [ -114.446815, 34.01421 ], [ -114.443821, 34.016176 ], [ -114.44054, 34.019329 ], [ -114.438266, 34.022609 ], [ -114.436171, 34.028083 ], [ -114.434949, 34.037784 ], [ -114.435816, 34.04373 ], [ -114.438602, 34.050205 ], [ -114.439406, 34.05381 ], [ -114.43934, 34.057893 ], [ -114.437683, 34.071937 ], [ -114.435907, 34.077491 ], [ -114.434181, 34.087379 ], [ -114.43338, 34.088413 ], [ -114.428026, 34.092787 ], [ -114.426168, 34.097042 ], [ -114.422899, 34.099661 ], [ -114.420499, 34.103466 ], [ -114.415908, 34.107636 ], [ -114.41168, 34.110031 ], [ -114.405916, 34.111468 ], [ -114.401336, 34.111638 ], [ -114.390565, 34.110084 ], [ -114.379223, 34.11599 ], [ -114.369292000000101, 34.117519 ], [ -114.366517, 34.118577 ], [ -114.360402, 34.123577 ], [ -114.359389, 34.125016 ], [ -114.358358, 34.127617 ], [ -114.356372, 34.130428 ], [ -114.35303, 34.13312 ], [ -114.350478, 34.134107 ], [ -114.348051, 34.134457 ], [ -114.336112, 34.134034 ], [ -114.324576, 34.136759 ], [ -114.320777, 34.138635 ], [ -114.312206, 34.144776 ], [ -114.307802, 34.150574 ], [ -114.298168, 34.160321 ], [ -114.292806, 34.166725 ], [ -114.287294, 34.170529 ], [ -114.275267, 34.172149 ], [ -114.26846, 34.170177 ], [ -114.257034, 34.172837 ], [ -114.253444000000101, 34.174129 ], [ -114.244421, 34.179403 ], [ -114.240712000000102, 34.183232 ], [ -114.229715, 34.186928 ], [ -114.227034, 34.188866 ], [ -114.225814, 34.191238 ], [ -114.224941, 34.193896 ], [ -114.225075, 34.196814 ], [ -114.22579, 34.199236 ], [ -114.225861, 34.201774 ], [ -114.225194, 34.203642 ], [ -114.223384, 34.205136 ], [ -114.215454, 34.208956 ], [ -114.211761, 34.211539 ], [ -114.208253, 34.215505 ], [ -114.190876, 34.230858 ], [ -114.17805, 34.239969 ], [ -114.176403, 34.241512 ], [ -114.175948, 34.242695 ], [ -114.175906, 34.245587 ], [ -114.174597, 34.247303 ], [ -114.166536, 34.249647 ], [ -114.166124, 34.250015 ], [ -114.164476, 34.251667 ], [ -114.163867, 34.253349 ], [ -114.163959, 34.255377 ], [ -114.165335, 34.258486 ], [ -114.165249, 34.259125 ], [ -114.164648, 34.259699 ], [ -114.156853, 34.258415 ], [ -114.153346, 34.258289 ], [ -114.147159, 34.259564 ], [ -114.144779, 34.259623 ], [ -114.13545, 34.257886 ], [ -114.133264, 34.258462 ], [ -114.131489000000101, 34.260387 ], [ -114.131211, 34.26273 ], [ -114.134768, 34.268965 ], [ -114.136671, 34.274377 ], [ -114.137045, 34.277018 ], [ -114.13605, 34.280833 ], [ -114.136677, 34.283936 ], [ -114.138365, 34.288564 ], [ -114.139534, 34.295844 ], [ -114.139187, 34.298074 ], [ -114.138167, 34.300936 ], [ -114.138282, 34.30323 ], [ -114.14093, 34.305919 ], [ -114.157206, 34.317862 ], [ -114.157939, 34.320277 ], [ -114.164249, 34.330816 ], [ -114.168807, 34.339513 ], [ -114.172845, 34.344979 ], [ -114.176909, 34.349306 ], [ -114.181145, 34.352186 ], [ -114.185556, 34.354386 ], [ -114.191094, 34.356125 ], [ -114.19648, 34.359187 ], [ -114.199482, 34.361373 ], [ -114.213774, 34.36246 ], [ -114.226107, 34.365916 ], [ -114.229686, 34.368908 ], [ -114.233065, 34.375013 ], [ -114.234275, 34.376662 ], [ -114.245261, 34.385659 ], [ -114.248649, 34.388113 ], [ -114.252739, 34.3901 ], [ -114.25822, 34.395046 ], [ -114.262909, 34.400373 ], [ -114.264317000000105, 34.401329 ], [ -114.267521, 34.402486 ], [ -114.272184, 34.402961 ], [ -114.280108, 34.403147 ], [ -114.282261, 34.403641 ], [ -114.286802, 34.40534 ], [ -114.288663, 34.406623 ], [ -114.290219, 34.408291 ], [ -114.291751, 34.411104 ], [ -114.291903, 34.416231 ], [ -114.292226, 34.417606 ], [ -114.294836, 34.421389 ], [ -114.301016000000104, 34.426807 ], [ -114.308659, 34.430485 ], [ -114.312251, 34.432726 ], [ -114.319054, 34.435831 ], [ -114.32613, 34.437251 ], [ -114.32688, 34.438048 ], [ -114.330669, 34.445295 ], [ -114.332991, 34.448082 ], [ -114.335372, 34.450038 ], [ -114.339627, 34.451435 ], [ -114.342615, 34.451442 ], [ -114.348974, 34.450166 ], [ -114.356025, 34.449744 ], [ -114.363404000000102, 34.447773 ], [ -114.373719, 34.446938 ], [ -114.375789, 34.447798 ], [ -114.378852, 34.450376 ], [ -114.382985, 34.453971 ], [ -114.386699, 34.457911 ], [ -114.387407, 34.460492 ], [ -114.387187, 34.462021 ], [ -114.383525, 34.470405 ], [ -114.381701, 34.47604 ], [ -114.381555, 34.477883 ], [ -114.383038, 34.488903 ], [ -114.382358, 34.495758 ], [ -114.381402, 34.499245 ], [ -114.378124, 34.507288 ], [ -114.378223, 34.516521 ], [ -114.380838, 34.529724 ], [ -114.389603, 34.542982 ], [ -114.398847, 34.559149 ], [ -114.405228, 34.569637 ], [ -114.422382, 34.580711 ], [ -114.435671, 34.593841 ], [ -114.43681, 34.596074 ], [ -114.436363, 34.596797 ], [ -114.427502, 34.599227 ], [ -114.425338, 34.600842 ], [ -114.424326, 34.602338 ], [ -114.424202, 34.610453 ], [ -114.428648, 34.614641 ], [ -114.438739, 34.621455 ], [ -114.439495, 34.625858 ], [ -114.441398, 34.630171 ], [ -114.441525, 34.631529 ], [ -114.440685, 34.634739 ], [ -114.440294, 34.63824 ], [ -114.440519, 34.640066 ], [ -114.441465, 34.64253 ], [ -114.444276, 34.646542 ], [ -114.445664, 34.647542 ], [ -114.449549, 34.651423 ], [ -114.457985, 34.657113 ], [ -114.45821, 34.657994 ], [ -114.457702, 34.659328 ], [ -114.457185, 34.659992 ], [ -114.451785, 34.663891 ], [ -114.450614, 34.665793 ], [ -114.450506, 34.666836 ], [ -114.451532, 34.668605 ], [ -114.454305, 34.671234 ], [ -114.45491, 34.673092 ], [ -114.454881, 34.675639 ], [ -114.455536, 34.677335 ], [ -114.458163, 34.681161 ], [ -114.462178, 34.6858 ], [ -114.463633, 34.68794 ], [ -114.465246, 34.691202 ], [ -114.46809, 34.701786 ], [ -114.46813, 34.704445 ], [ -114.46862, 34.707573 ], [ -114.470477, 34.711368 ], [ -114.47162, 34.712966 ], [ -114.473682, 34.713964 ], [ -114.477297, 34.714514 ], [ -114.482779, 34.714511 ], [ -114.487508, 34.716626 ], [ -114.489287, 34.720155 ], [ -114.490971, 34.724848 ], [ -114.492017, 34.725702 ], [ -114.495858, 34.727956 ], [ -114.499007, 34.729096 ], [ -114.500795, 34.730418 ], [ -114.510292, 34.733582 ], [ -114.514178, 34.735288 ], [ -114.516619, 34.736745 ], [ -114.521048, 34.741173 ], [ -114.522619, 34.74373 ], [ -114.525611, 34.747005 ], [ -114.529079, 34.750006 ], [ -114.529615, 34.750822 ], [ -114.540306, 34.757109 ], [ -114.546884, 34.761802 ], [ -114.552682, 34.766871 ], [ -114.558653, 34.773852 ], [ -114.563979, 34.782597 ], [ -114.565184, 34.785976 ], [ -114.569383, 34.791568 ], [ -114.571010000000101, 34.794294 ], [ -114.574474, 34.804214 ], [ -114.574694, 34.807471 ], [ -114.576452, 34.8153 ], [ -114.578681000000103, 34.820977 ], [ -114.581126, 34.826115 ], [ -114.586842, 34.835672 ], [ -114.592339, 34.841153 ], [ -114.600653, 34.847361 ], [ -114.604255, 34.849573 ], [ -114.619878, 34.856873 ], [ -114.623939, 34.859738 ], [ -114.628276, 34.863596 ], [ -114.630682000000107, 34.866352 ], [ -114.633051, 34.869971 ], [ -114.634382, 34.87289 ], [ -114.635176, 34.875003 ], [ -114.635458, 34.876902 ] ] ] ] }\n`\n" }, { "path": "cmd/tile38-benchmark/main.go", "content": "package main\n\nimport (\n\t\"fmt\"\n\t\"log\"\n\t\"math\"\n\t\"math/rand\"\n\t\"net\"\n\t\"os\"\n\t\"strconv\"\n\t\"strings\"\n\t\"sync\"\n\t\"sync/atomic\"\n\t\"time\"\n\n\t\"github.com/tidwall/redbench\"\n\t\"github.com/tidwall/redcon\"\n\t\"github.com/tidwall/tile38/cmd/tile38-benchmark/az\"\n\t\"github.com/tidwall/tile38/core\"\n)\n\nvar (\n\thostname = \"127.0.0.1\"\n\tport = 9851\n\tauth = \"\"\n\tclients = 50\n\trequests = 100000\n\tquiet = false\n\tpipeline = 1\n\tcsv = false\n\tjson = false\n\tallTests = \"PING,SET,GET,INTERSECTS,WITHIN,NEARBY,EVAL\"\n\ttests = allTests\n\tredis = false\n)\n\nvar addr string\n\nfunc showHelp() bool {\n\tgitsha := \"\"\n\tif core.GitSHA == \"\" || core.GitSHA == \"0000000\" {\n\t\tgitsha = \"\"\n\t} else {\n\t\tgitsha = \" (git:\" + core.GitSHA + \")\"\n\t}\n\tfmt.Fprintf(os.Stdout, \"tile38-benchmark %s%s\\n\\n\", core.Version, gitsha)\n\tfmt.Fprintf(os.Stdout, \"Usage: tile38-benchmark [-h ] [-p ] [-c ] [-n ]\\n\")\n\n\tfmt.Fprintf(os.Stdout, \" -h Server hostname (default: %s)\\n\", hostname)\n\tfmt.Fprintf(os.Stdout, \" -p Server port (default: %d)\\n\", port)\n\tfmt.Fprintf(os.Stdout, \" -a Password for Tile38 Auth\\n\")\n\tfmt.Fprintf(os.Stdout, \" -c Number of parallel connections (default %d)\\n\", clients)\n\tfmt.Fprintf(os.Stdout, \" -n Total number or requests (default %d)\\n\", requests)\n\tfmt.Fprintf(os.Stdout, \" -q Quiet. Just show query/sec values\\n\")\n\tfmt.Fprintf(os.Stdout, \" -P Pipeline requests. Default 1 (no pipeline).\\n\")\n\tfmt.Fprintf(os.Stdout, \" -t Only run the comma separated list of tests. The test\\n\")\n\tfmt.Fprintf(os.Stdout, \" names are the same as the ones produced as output.\\n\")\n\tfmt.Fprintf(os.Stdout, \" --csv Output in CSV format.\\n\")\n\tfmt.Fprintf(os.Stdout, \" --json Request JSON responses (default is RESP output)\\n\")\n\tfmt.Fprintf(os.Stdout, \" --redis Runs against a Redis server\\n\")\n\tfmt.Fprintf(os.Stdout, \"\\n\")\n\treturn false\n}\n\nfunc parseArgs() bool {\n\tdefer func() {\n\t\tif v := recover(); v != nil {\n\t\t\tif v, ok := v.(string); ok && v == \"bad arg\" {\n\t\t\t\tshowHelp()\n\t\t\t}\n\t\t}\n\t}()\n\n\targs := os.Args[1:]\n\treadArg := func(arg string) string {\n\t\tif len(args) == 0 {\n\t\t\tpanic(\"bad arg\")\n\t\t}\n\t\tvar narg = args[0]\n\t\targs = args[1:]\n\t\treturn narg\n\t}\n\treadIntArg := func(arg string) int {\n\t\tn, err := strconv.ParseUint(readArg(arg), 10, 64)\n\t\tif err != nil {\n\t\t\tpanic(\"bad arg\")\n\t\t}\n\t\treturn int(n)\n\t}\n\tbadArg := func(arg string) bool {\n\t\tfmt.Fprintf(os.Stderr, \"Unrecognized option or bad number of args for: '%s'\\n\", arg)\n\t\treturn false\n\t}\n\n\tfor len(args) > 0 {\n\t\targ := readArg(\"\")\n\t\tif arg == \"--help\" || arg == \"-?\" {\n\t\t\treturn showHelp()\n\t\t}\n\t\tif !strings.HasPrefix(arg, \"-\") {\n\t\t\targs = append([]string{arg}, args...)\n\t\t\tbreak\n\t\t}\n\t\tswitch arg {\n\t\tdefault:\n\t\t\treturn badArg(arg)\n\t\tcase \"-h\":\n\t\t\thostname = readArg(arg)\n\t\tcase \"-p\":\n\t\t\tport = readIntArg(arg)\n\t\tcase \"-a\":\n\t\t\tauth = readArg(arg)\n\t\tcase \"-c\":\n\t\t\tclients = readIntArg(arg)\n\t\t\tif clients <= 0 {\n\t\t\t\tclients = 1\n\t\t\t}\n\t\tcase \"-n\":\n\t\t\trequests = readIntArg(arg)\n\t\t\tif requests <= 0 {\n\t\t\t\trequests = 0\n\t\t\t}\n\t\tcase \"-q\":\n\t\t\tquiet = true\n\t\tcase \"-P\":\n\t\t\tpipeline = readIntArg(arg)\n\t\t\tif pipeline <= 0 {\n\t\t\t\tpipeline = 1\n\t\t\t}\n\t\tcase \"-t\":\n\t\t\ttests = readArg(arg)\n\t\tcase \"--csv\":\n\t\t\tcsv = true\n\t\tcase \"--json\":\n\t\t\tjson = true\n\t\tcase \"--redis\":\n\t\t\tredis = true\n\t\t}\n\t}\n\treturn true\n}\n\nfunc fillOpts() *redbench.Options {\n\topts := *redbench.DefaultOptions\n\topts.CSV = csv\n\topts.Clients = clients\n\topts.Pipeline = pipeline\n\topts.Quiet = quiet\n\topts.Requests = requests\n\topts.Stderr = os.Stderr\n\topts.Stdout = os.Stdout\n\treturn &opts\n}\n\nfunc randPoint() (lat, lon float64) {\n\treturn rand.Float64()*180 - 90, rand.Float64()*360 - 180\n}\n\nfunc isValidRect(minlat, minlon, maxlat, maxlon float64) bool {\n\treturn minlat > -90 && maxlat < 90 && minlon > -180 && maxlon < 180\n}\n\nfunc randRect(meters float64) (minlat, minlon, maxlat, maxlon float64) {\n\tfor {\n\t\tlat, lon := randPoint()\n\t\tmaxlat, _ = destinationPoint(lat, lon, meters, 0)\n\t\t_, maxlon = destinationPoint(lat, lon, meters, 90)\n\t\tminlat, _ = destinationPoint(lat, lon, meters, 180)\n\t\t_, minlon = destinationPoint(lat, lon, meters, 270)\n\t\tif isValidRect(minlat, minlon, maxlat, maxlon) {\n\t\t\treturn\n\t\t}\n\t}\n}\n\nfunc prepFn(conn net.Conn) bool {\n\tvar resp [64]byte\n\tconn.Write([]byte(\"CONFIG GET requirepass\\r\\n\"))\n\tn, err := conn.Read(resp[:])\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tif string(resp[:n]) == \"-ERR authentication required\\r\\n\" {\n\t\tif auth == \"\" {\n\t\t\tlog.Fatal(\"invalid auth\")\n\t\t} else {\n\t\t\tcmd := redcon.AppendArray(nil, 2)\n\t\t\tcmd = redcon.AppendBulkString(cmd, \"AUTH\")\n\t\t\tcmd = redcon.AppendBulkString(cmd, auth)\n\t\t\tconn.Write(cmd)\n\t\t\tn, err := conn.Read(resp[:])\n\t\t\tif err != nil || string(resp[:n]) != \"+OK\\r\\n\" {\n\t\t\t\tlog.Fatal(\"invalid auth\")\n\t\t\t}\n\t\t}\n\t} else if auth != \"\" {\n\t\tlog.Fatal(\"invalid auth\")\n\t}\n\tif json {\n\t\tconn.Write([]byte(\"output json\\r\\n\"))\n\t\tconn.Read(make([]byte, 64))\n\t}\n\treturn true\n}\nfunc main() {\n\trand.Seed(time.Now().UnixNano())\n\tif !parseArgs() {\n\t\treturn\n\t}\n\topts := fillOpts()\n\taddr = fmt.Sprintf(\"%s:%d\", hostname, port)\n\n\ttestsArr := strings.Split(allTests, \",\")\n\tvar subtract bool\n\tvar add bool\n\tfor _, test := range strings.Split(tests, \",\") {\n\t\tif strings.HasPrefix(test, \"-\") {\n\t\t\tif add {\n\t\t\t\tos.Stderr.Write([]byte(\"test flag cannot mix add and subtract\\n\"))\n\t\t\t\tos.Exit(1)\n\t\t\t}\n\t\t\tsubtract = true\n\t\t\tfor i := range testsArr {\n\t\t\t\tif strings.EqualFold(testsArr[i], test[1:]) {\n\t\t\t\t\ttestsArr = append(testsArr[:i], testsArr[i+1:]...)\n\t\t\t\t\tbreak\n\t\t\t\t}\n\t\t\t}\n\t\t} else if subtract {\n\t\t\tadd = true\n\t\t\tos.Stderr.Write([]byte(\"test flag cannot mix add and subtract\\n\"))\n\t\t\tos.Exit(1)\n\t\t}\n\t}\n\tif !subtract {\n\t\ttestsArr = strings.Split(tests, \",\")\n\t}\n\n\tfor _, test := range testsArr {\n\t\tswitch strings.ToUpper(strings.TrimSpace(test)) {\n\t\tcase \"PING\":\n\t\t\tredbench.Bench(\"PING\", addr, opts, prepFn,\n\t\t\t\tfunc(buf []byte) []byte {\n\t\t\t\t\treturn redbench.AppendCommand(buf, \"PING\")\n\t\t\t\t},\n\t\t\t)\n\t\tcase \"GEOADD\":\n\t\t\t//GEOADD key longitude latitude member\n\t\t\tif redis {\n\t\t\t\tvar i int64\n\t\t\t\tredbench.Bench(\"GEOADD\", addr, opts, prepFn,\n\t\t\t\t\tfunc(buf []byte) []byte {\n\t\t\t\t\t\ti := atomic.AddInt64(&i, 1)\n\t\t\t\t\t\tlat, lon := randPoint()\n\t\t\t\t\t\treturn redbench.AppendCommand(buf, \"GEOADD\", \"key:bench\",\n\t\t\t\t\t\t\tstrconv.FormatFloat(lat, 'f', 5, 64),\n\t\t\t\t\t\t\tstrconv.FormatFloat(lon, 'f', 5, 64),\n\t\t\t\t\t\t\t\"id:\"+strconv.FormatInt(i, 10),\n\t\t\t\t\t\t)\n\t\t\t\t\t},\n\t\t\t\t)\n\t\t\t}\n\n\t\tcase \"SET\", \"SET-POINT\", \"SET-RECT\", \"SET-LINE\", \"SET-STRING\":\n\t\t\tif redis {\n\t\t\t\tredbench.Bench(\"SET\", addr, opts, prepFn,\n\t\t\t\t\tfunc(buf []byte) []byte {\n\t\t\t\t\t\treturn redbench.AppendCommand(buf, \"SET\", \"key:__rand_int__\", \"xxx\")\n\t\t\t\t\t},\n\t\t\t\t)\n\t\t\t} else {\n\t\t\t\tvar i int64\n\t\t\t\tswitch strings.ToUpper(strings.TrimSpace(test)) {\n\t\t\t\tcase \"SET\", \"SET-POINT\":\n\t\t\t\t\tredbench.Bench(\"SET (point)\", addr, opts, prepFn,\n\t\t\t\t\t\tfunc(buf []byte) []byte {\n\t\t\t\t\t\t\ti := atomic.AddInt64(&i, 1)\n\t\t\t\t\t\t\tlat, lon := randPoint()\n\t\t\t\t\t\t\treturn redbench.AppendCommand(buf, \"SET\", \"key:bench\", \"id:\"+strconv.FormatInt(i, 10), \"POINT\",\n\t\t\t\t\t\t\t\tstrconv.FormatFloat(lat, 'f', 5, 64),\n\t\t\t\t\t\t\t\tstrconv.FormatFloat(lon, 'f', 5, 64),\n\t\t\t\t\t\t\t)\n\t\t\t\t\t\t},\n\t\t\t\t\t)\n\t\t\t\t}\n\t\t\t\tswitch strings.ToUpper(strings.TrimSpace(test)) {\n\t\t\t\tcase \"SET\", \"SET-RECT\":\n\t\t\t\t\tredbench.Bench(\"SET (rect)\", addr, opts, prepFn,\n\t\t\t\t\t\tfunc(buf []byte) []byte {\n\t\t\t\t\t\t\ti := atomic.AddInt64(&i, 1)\n\t\t\t\t\t\t\tminlat, minlon, maxlat, maxlon := randRect(10000)\n\t\t\t\t\t\t\treturn redbench.AppendCommand(buf, \"SET\", \"key:bench\", \"id:\"+strconv.FormatInt(i, 10), \"BOUNDS\",\n\t\t\t\t\t\t\t\tstrconv.FormatFloat(minlat, 'f', 5, 64),\n\t\t\t\t\t\t\t\tstrconv.FormatFloat(minlon, 'f', 5, 64),\n\t\t\t\t\t\t\t\tstrconv.FormatFloat(maxlat, 'f', 5, 64),\n\t\t\t\t\t\t\t\tstrconv.FormatFloat(maxlon, 'f', 5, 64),\n\t\t\t\t\t\t\t)\n\t\t\t\t\t\t},\n\t\t\t\t\t)\n\t\t\t\t}\n\t\t\t\tswitch strings.ToUpper(strings.TrimSpace(test)) {\n\t\t\t\tcase \"SET\", \"SET-LINE\":\n\t\t\t\t\tredbench.Bench(\"SET (line)\", addr, opts, prepFn,\n\t\t\t\t\t\tfunc(buf []byte) []byte {\n\t\t\t\t\t\t\ti := atomic.AddInt64(&i, 1)\n\t\t\t\t\t\t\talat, alon, blat, blon := randRect(10000)\n\t\t\t\t\t\t\tif rand.Int()%2 == 0 {\n\t\t\t\t\t\t\t\talat, alon, blat, blon = blat, blon, alat, alon\n\t\t\t\t\t\t\t}\n\t\t\t\t\t\t\tlinestring := fmt.Sprintf(`{\"type\":\"LineString\",\"coordinates\":[[%f,%f],[%f,%f]]}`, alon, alat, blon, blat)\n\t\t\t\t\t\t\treturn redbench.AppendCommand(buf, \"SET\", \"key:bench\", \"id:\"+strconv.FormatInt(i, 10), \"OBJECT\",\n\t\t\t\t\t\t\t\tlinestring,\n\t\t\t\t\t\t\t)\n\t\t\t\t\t\t},\n\t\t\t\t\t)\n\t\t\t\t}\n\t\t\t\tswitch strings.ToUpper(strings.TrimSpace(test)) {\n\t\t\t\tcase \"SET\", \"SET-STRING\":\n\t\t\t\t\tredbench.Bench(\"SET (string)\", addr, opts, prepFn,\n\t\t\t\t\t\tfunc(buf []byte) []byte {\n\t\t\t\t\t\t\ti := atomic.AddInt64(&i, 1)\n\t\t\t\t\t\t\treturn redbench.AppendCommand(buf, \"SET\", \"key:bench\", \"id:\"+strconv.FormatInt(i, 10), \"STRING\", \"xxx\")\n\t\t\t\t\t\t},\n\t\t\t\t\t)\n\t\t\t\t}\n\t\t\t}\n\t\tcase \"GET\":\n\t\t\tif redis {\n\t\t\t\tredbench.Bench(\"GET\", addr, opts, prepFn,\n\t\t\t\t\tfunc(buf []byte) []byte {\n\t\t\t\t\t\treturn redbench.AppendCommand(buf, \"GET\", \"key:__rand_int__\")\n\t\t\t\t\t},\n\t\t\t\t)\n\t\t\t} else {\n\t\t\t\tvar i int64\n\t\t\t\tredbench.Bench(\"GET (point)\", addr, opts, prepFn,\n\t\t\t\t\tfunc(buf []byte) []byte {\n\t\t\t\t\t\ti := atomic.AddInt64(&i, 1)\n\t\t\t\t\t\treturn redbench.AppendCommand(buf, \"GET\", \"key:bench\", \"id:\"+strconv.FormatInt(i, 10), \"POINT\")\n\t\t\t\t\t},\n\t\t\t\t)\n\t\t\t\tredbench.Bench(\"GET (rect)\", addr, opts, prepFn,\n\t\t\t\t\tfunc(buf []byte) []byte {\n\t\t\t\t\t\ti := atomic.AddInt64(&i, 1)\n\t\t\t\t\t\treturn redbench.AppendCommand(buf, \"GET\", \"key:bench\", \"id:\"+strconv.FormatInt(i, 10), \"BOUNDS\")\n\t\t\t\t\t},\n\t\t\t\t)\n\t\t\t\tredbench.Bench(\"GET (string)\", addr, opts, prepFn,\n\t\t\t\t\tfunc(buf []byte) []byte {\n\t\t\t\t\t\ti := atomic.AddInt64(&i, 1)\n\t\t\t\t\t\treturn redbench.AppendCommand(buf, \"GET\", \"key:bench\", \"id:\"+strconv.FormatInt(i, 10), \"OBJECT\")\n\t\t\t\t\t},\n\t\t\t\t)\n\t\t\t}\n\t\tcase \"INTERSECTS\",\n\t\t\t\"INTERSECTS-BOUNDS\", \"INTERSECTS-BOUNDS-1000\", \"INTERSECTS-BOUNDS-10000\", \"INTERSECTS-BOUNDS-100000\",\n\t\t\t\"INTERSECTS-CIRCLE\", \"INTERSECTS-CIRCLE-1000\", \"INTERSECTS-CIRCLE-10000\", \"INTERSECTS-CIRCLE-100000\",\n\t\t\t\"INTERSECTS-AZ\":\n\t\t\tif redis {\n\t\t\t\tbreak\n\t\t\t}\n\t\t\tswitch strings.ToUpper(strings.TrimSpace(test)) {\n\t\t\tcase \"INTERSECTS\", \"INTERSECTS-CIRCLE\", \"INTERSECTS-CIRCLE-1000\":\n\t\t\t\tredbench.Bench(\"INTERSECTS (intersects-circle 1km)\", addr, opts, prepFn,\n\t\t\t\t\tfunc(buf []byte) []byte {\n\t\t\t\t\t\tlat, lon := randPoint()\n\t\t\t\t\t\treturn redbench.AppendCommand(buf,\n\t\t\t\t\t\t\t\"INTERSECTS\", \"key:bench\", \"COUNT\", \"CIRCLE\",\n\t\t\t\t\t\t\tstrconv.FormatFloat(lat, 'f', 5, 64),\n\t\t\t\t\t\t\tstrconv.FormatFloat(lon, 'f', 5, 64),\n\t\t\t\t\t\t\t\"1000\")\n\t\t\t\t\t},\n\t\t\t\t)\n\t\t\t}\n\t\t\tswitch strings.ToUpper(strings.TrimSpace(test)) {\n\t\t\tcase \"INTERSECTS\", \"INTERSECTS-CIRCLE\", \"INTERSECTS-CIRCLE-10000\":\n\t\t\t\tredbench.Bench(\"INTERSECTS (intersects-circle 10km)\", addr, opts, prepFn,\n\t\t\t\t\tfunc(buf []byte) []byte {\n\t\t\t\t\t\tlat, lon := randPoint()\n\t\t\t\t\t\treturn redbench.AppendCommand(buf,\n\t\t\t\t\t\t\t\"INTERSECTS\", \"key:bench\", \"COUNT\", \"CIRCLE\",\n\t\t\t\t\t\t\tstrconv.FormatFloat(lat, 'f', 5, 64),\n\t\t\t\t\t\t\tstrconv.FormatFloat(lon, 'f', 5, 64),\n\t\t\t\t\t\t\t\"10000\")\n\t\t\t\t\t},\n\t\t\t\t)\n\t\t\t}\n\t\t\tswitch strings.ToUpper(strings.TrimSpace(test)) {\n\t\t\tcase \"INTERSECTS\", \"INTERSECTS-CIRCLE\", \"INTERSECTS-CIRCLE-100000\":\n\t\t\t\tredbench.Bench(\"INTERSECTS (intersects-circle 100km)\", addr, opts, prepFn,\n\t\t\t\t\tfunc(buf []byte) []byte {\n\t\t\t\t\t\tlat, lon := randPoint()\n\t\t\t\t\t\treturn redbench.AppendCommand(buf,\n\t\t\t\t\t\t\t\"INTERSECTS\", \"key:bench\", \"COUNT\", \"CIRCLE\",\n\t\t\t\t\t\t\tstrconv.FormatFloat(lat, 'f', 5, 64),\n\t\t\t\t\t\t\tstrconv.FormatFloat(lon, 'f', 5, 64),\n\t\t\t\t\t\t\t\"100000\")\n\t\t\t\t\t},\n\t\t\t\t)\n\t\t\t}\n\t\t\t// INTERSECTS-BOUNDS\n\t\t\tswitch strings.ToUpper(strings.TrimSpace(test)) {\n\t\t\tcase \"INTERSECTS\", \"INTERSECTS-BOUNDS\", \"INTERSECTS-BOUNDS-1000\":\n\t\t\t\tminlat, minlon, maxlat, maxlon := randRect(1000)\n\t\t\t\tredbench.Bench(\"INTERSECTS (intersects-bounds 1km)\", addr, opts, prepFn,\n\t\t\t\t\tfunc(buf []byte) []byte {\n\t\t\t\t\t\treturn redbench.AppendCommand(buf,\n\t\t\t\t\t\t\t\"INTERSECTS\", \"key:bench\", \"COUNT\", \"BOUNDS\",\n\t\t\t\t\t\t\tstrconv.FormatFloat(minlat, 'f', 5, 64),\n\t\t\t\t\t\t\tstrconv.FormatFloat(minlon, 'f', 5, 64),\n\t\t\t\t\t\t\tstrconv.FormatFloat(maxlat, 'f', 5, 64),\n\t\t\t\t\t\t\tstrconv.FormatFloat(maxlon, 'f', 5, 64))\n\t\t\t\t\t},\n\t\t\t\t)\n\t\t\t}\n\t\t\tswitch strings.ToUpper(strings.TrimSpace(test)) {\n\t\t\tcase \"INTERSECTS\", \"INTERSECTS-BOUNDS\", \"INTERSECTS-BOUNDS-10000\":\n\t\t\t\tminlat, minlon, maxlat, maxlon := randRect(10000)\n\t\t\t\tredbench.Bench(\"INTERSECTS (intersects-bounds 10km)\", addr, opts, prepFn,\n\t\t\t\t\tfunc(buf []byte) []byte {\n\t\t\t\t\t\treturn redbench.AppendCommand(buf,\n\t\t\t\t\t\t\t\"INTERSECTS\", \"key:bench\", \"COUNT\", \"BOUNDS\",\n\t\t\t\t\t\t\tstrconv.FormatFloat(minlat, 'f', 5, 64),\n\t\t\t\t\t\t\tstrconv.FormatFloat(minlon, 'f', 5, 64),\n\t\t\t\t\t\t\tstrconv.FormatFloat(maxlat, 'f', 5, 64),\n\t\t\t\t\t\t\tstrconv.FormatFloat(maxlon, 'f', 5, 64))\n\t\t\t\t\t},\n\t\t\t\t)\n\t\t\t}\n\t\t\tswitch strings.ToUpper(strings.TrimSpace(test)) {\n\t\t\tcase \"INTERSECTS\", \"INTERSECTS-BOUNDS\", \"INTERSECTS-BOUNDS-100000\":\n\t\t\t\tminlat, minlon, maxlat, maxlon := randRect(10000)\n\t\t\t\tredbench.Bench(\"INTERSECTS (intersects-bounds 100km)\", addr, opts, prepFn,\n\t\t\t\t\tfunc(buf []byte) []byte {\n\t\t\t\t\t\treturn redbench.AppendCommand(buf,\n\t\t\t\t\t\t\t\"INTERSECTS\", \"key:bench\", \"COUNT\", \"BOUNDS\",\n\t\t\t\t\t\t\tstrconv.FormatFloat(minlat, 'f', 5, 64),\n\t\t\t\t\t\t\tstrconv.FormatFloat(minlon, 'f', 5, 64),\n\t\t\t\t\t\t\tstrconv.FormatFloat(maxlat, 'f', 5, 64),\n\t\t\t\t\t\t\tstrconv.FormatFloat(maxlon, 'f', 5, 64))\n\t\t\t\t\t},\n\t\t\t\t)\n\t\t\t}\n\n\t\t\tswitch strings.ToUpper(strings.TrimSpace(test)) {\n\t\t\tcase \"INTERSECTS\", \"INTERSECTS-AZ\":\n\t\t\t\tvar mu sync.Mutex\n\t\t\t\tvar loaded bool\n\t\t\t\tredbench.Bench(\"INTERSECTS (intersects-az limit 5)\", addr, opts, func(conn net.Conn) bool {\n\t\t\t\t\tfunc() {\n\t\t\t\t\t\tmu.Lock()\n\t\t\t\t\t\tdefer mu.Unlock()\n\t\t\t\t\t\tif loaded {\n\t\t\t\t\t\t\treturn\n\t\t\t\t\t\t}\n\t\t\t\t\t\tloaded = true\n\t\t\t\t\t\tp := make([]byte, 0xFF)\n\t\t\t\t\t\tconn.Write([]byte(\"GET keys:bench:geo az point\\r\\n\"))\n\t\t\t\t\t\tn, err := conn.Read(p)\n\t\t\t\t\t\tif err != nil {\n\t\t\t\t\t\t\tpanic(err)\n\t\t\t\t\t\t}\n\t\t\t\t\t\tif string(p[:n]) != \"$-1\\r\\n\" {\n\t\t\t\t\t\t\treturn\n\t\t\t\t\t\t}\n\t\t\t\t\t\targs := []string{\"SET\", \"key:bench:geo\", \"az\", \"object\", az.JSON}\n\t\t\t\t\t\tout := redcon.AppendArray(nil, len(args))\n\t\t\t\t\t\tfor _, arg := range args {\n\t\t\t\t\t\t\tout = redcon.AppendBulkString(out, arg)\n\t\t\t\t\t\t}\n\t\t\t\t\t\tconn.Write(out)\n\t\t\t\t\t\tn, err = conn.Read(p)\n\t\t\t\t\t\tif err != nil {\n\t\t\t\t\t\t\tpanic(err)\n\t\t\t\t\t\t}\n\t\t\t\t\t\tif string(p[:n]) != \"+OK\\r\\n\" {\n\t\t\t\t\t\t\tpanic(\"expected OK\")\n\t\t\t\t\t\t}\n\t\t\t\t\t}()\n\t\t\t\t\treturn prepFn(conn)\n\t\t\t\t},\n\t\t\t\t\tfunc(buf []byte) []byte {\n\t\t\t\t\t\targs := []string{\"INTERSECTS\", \"key:bench\", \"LIMIT\", \"5\",\n\t\t\t\t\t\t\t\"COUNT\", \"GET\", \"key:bench:geo\", \"az\"}\n\t\t\t\t\t\treturn redbench.AppendCommand(buf, args...)\n\t\t\t\t\t},\n\t\t\t\t)\n\t\t\t}\n\n\t\tcase \"WITHIN\",\n\t\t\t\"WITHIN-RECT\", \"WITHIN-RECT-1000\", \"WITHIN-RECT-10000\", \"WITHIN-RECT-100000\",\n\t\t\t\"WITHIN-CIRCLE\", \"WITHIN-CIRCLE-1000\", \"WITHIN-CIRCLE-10000\", \"WITHIN-CIRCLE-100000\":\n\t\t\tif redis {\n\t\t\t\tbreak\n\t\t\t}\n\t\t\tswitch strings.ToUpper(strings.TrimSpace(test)) {\n\t\t\tcase \"WITHIN\", \"WITHIN-CIRCLE\", \"WITHIN-CIRCLE-1000\":\n\t\t\t\tredbench.Bench(\"WITHIN (within-circle 1km)\", addr, opts, prepFn,\n\t\t\t\t\tfunc(buf []byte) []byte {\n\t\t\t\t\t\tlat, lon := randPoint()\n\t\t\t\t\t\treturn redbench.AppendCommand(buf,\n\t\t\t\t\t\t\t\"WITHIN\", \"key:bench\", \"COUNT\", \"CIRCLE\",\n\t\t\t\t\t\t\tstrconv.FormatFloat(lat, 'f', 5, 64),\n\t\t\t\t\t\t\tstrconv.FormatFloat(lon, 'f', 5, 64),\n\t\t\t\t\t\t\t\"1000\")\n\t\t\t\t\t},\n\t\t\t\t)\n\t\t\t}\n\t\t\tswitch strings.ToUpper(strings.TrimSpace(test)) {\n\t\t\tcase \"WITHIN\", \"WITHIN-CIRCLE\", \"WITHIN-CIRCLE-10000\":\n\t\t\t\tredbench.Bench(\"WITHIN (within-circle 10km)\", addr, opts, prepFn,\n\t\t\t\t\tfunc(buf []byte) []byte {\n\t\t\t\t\t\tlat, lon := randPoint()\n\t\t\t\t\t\treturn redbench.AppendCommand(buf,\n\t\t\t\t\t\t\t\"WITHIN\", \"key:bench\", \"COUNT\", \"CIRCLE\",\n\t\t\t\t\t\t\tstrconv.FormatFloat(lat, 'f', 5, 64),\n\t\t\t\t\t\t\tstrconv.FormatFloat(lon, 'f', 5, 64),\n\t\t\t\t\t\t\t\"10000\")\n\t\t\t\t\t},\n\t\t\t\t)\n\t\t\t}\n\t\t\tswitch strings.ToUpper(strings.TrimSpace(test)) {\n\t\t\tcase \"WITHIN\", \"WITHIN-CIRCLE\", \"WITHIN-CIRCLE-100000\":\n\t\t\t\tredbench.Bench(\"WITHIN (within-circle 100km)\", addr, opts, prepFn,\n\t\t\t\t\tfunc(buf []byte) []byte {\n\t\t\t\t\t\tlat, lon := randPoint()\n\t\t\t\t\t\treturn redbench.AppendCommand(buf,\n\t\t\t\t\t\t\t\"WITHIN\", \"key:bench\", \"COUNT\", \"CIRCLE\",\n\t\t\t\t\t\t\tstrconv.FormatFloat(lat, 'f', 5, 64),\n\t\t\t\t\t\t\tstrconv.FormatFloat(lon, 'f', 5, 64),\n\t\t\t\t\t\t\t\"100000\")\n\t\t\t\t\t},\n\t\t\t\t)\n\t\t\t}\n\t\t\t// WITHIN-BOUNDS\n\t\t\tswitch strings.ToUpper(strings.TrimSpace(test)) {\n\t\t\tcase \"WITHIN\", \"WITHIN-BOUNDS\", \"WITHIN-BOUNDS-1000\":\n\t\t\t\tminlat, minlon, maxlat, maxlon := randRect(1000)\n\t\t\t\tredbench.Bench(\"WITHIN (within-bounds 1km)\", addr, opts, prepFn,\n\t\t\t\t\tfunc(buf []byte) []byte {\n\t\t\t\t\t\treturn redbench.AppendCommand(buf,\n\t\t\t\t\t\t\t\"WITHIN\", \"key:bench\", \"COUNT\", \"BOUNDS\",\n\t\t\t\t\t\t\tstrconv.FormatFloat(minlat, 'f', 5, 64),\n\t\t\t\t\t\t\tstrconv.FormatFloat(minlon, 'f', 5, 64),\n\t\t\t\t\t\t\tstrconv.FormatFloat(maxlat, 'f', 5, 64),\n\t\t\t\t\t\t\tstrconv.FormatFloat(maxlon, 'f', 5, 64))\n\t\t\t\t\t},\n\t\t\t\t)\n\t\t\t}\n\t\t\tswitch strings.ToUpper(strings.TrimSpace(test)) {\n\t\t\tcase \"WITHIN\", \"WITHIN-BOUNDS\", \"WITHIN-BOUNDS-10000\":\n\t\t\t\tminlat, minlon, maxlat, maxlon := randRect(10000)\n\t\t\t\tredbench.Bench(\"WITHIN (within-bounds 10km)\", addr, opts, prepFn,\n\t\t\t\t\tfunc(buf []byte) []byte {\n\t\t\t\t\t\treturn redbench.AppendCommand(buf,\n\t\t\t\t\t\t\t\"WITHIN\", \"key:bench\", \"COUNT\", \"BOUNDS\",\n\t\t\t\t\t\t\tstrconv.FormatFloat(minlat, 'f', 5, 64),\n\t\t\t\t\t\t\tstrconv.FormatFloat(minlon, 'f', 5, 64),\n\t\t\t\t\t\t\tstrconv.FormatFloat(maxlat, 'f', 5, 64),\n\t\t\t\t\t\t\tstrconv.FormatFloat(maxlon, 'f', 5, 64))\n\t\t\t\t\t},\n\t\t\t\t)\n\t\t\t}\n\t\t\tswitch strings.ToUpper(strings.TrimSpace(test)) {\n\t\t\tcase \"WITHIN\", \"WITHIN-BOUNDS\", \"WITHIN-BOUNDS-100000\":\n\t\t\t\tminlat, minlon, maxlat, maxlon := randRect(10000)\n\t\t\t\tredbench.Bench(\"WITHIN (within-bounds 100km)\", addr, opts, prepFn,\n\t\t\t\t\tfunc(buf []byte) []byte {\n\t\t\t\t\t\treturn redbench.AppendCommand(buf,\n\t\t\t\t\t\t\t\"WITHIN\", \"key:bench\", \"COUNT\", \"BOUNDS\",\n\t\t\t\t\t\t\tstrconv.FormatFloat(minlat, 'f', 5, 64),\n\t\t\t\t\t\t\tstrconv.FormatFloat(minlon, 'f', 5, 64),\n\t\t\t\t\t\t\tstrconv.FormatFloat(maxlat, 'f', 5, 64),\n\t\t\t\t\t\t\tstrconv.FormatFloat(maxlon, 'f', 5, 64))\n\t\t\t\t\t},\n\t\t\t\t)\n\t\t\t}\n\t\tcase \"NEARBY\",\n\t\t\t\"NEARBY-KNN\", \"NEARBY-KNN-1\", \"NEARBY-KNN-10\", \"NEARBY-KNN-100\",\n\t\t\t\"NEARBY-POINT\", \"NEARBY-POINT-1000\", \"NEARBY-POINT-10000\", \"NEARBY-POINT-100000\":\n\t\t\tif redis {\n\t\t\t\tbreak\n\t\t\t}\n\t\t\tswitch strings.ToUpper(strings.TrimSpace(test)) {\n\t\t\tcase \"NEARBY\", \"NEARBY-KNN\", \"NEARBY-KNN-1\":\n\t\t\t\tredbench.Bench(\"NEARBY (limit 1)\", addr, opts, prepFn,\n\t\t\t\t\tfunc(buf []byte) []byte {\n\t\t\t\t\t\tlat, lon := randPoint()\n\t\t\t\t\t\treturn redbench.AppendCommand(buf,\n\t\t\t\t\t\t\t\"NEARBY\", \"key:bench\", \"LIMIT\", \"1\", \"COUNT\", \"POINT\",\n\t\t\t\t\t\t\tstrconv.FormatFloat(lat, 'f', 5, 64),\n\t\t\t\t\t\t\tstrconv.FormatFloat(lon, 'f', 5, 64),\n\t\t\t\t\t\t)\n\t\t\t\t\t},\n\t\t\t\t)\n\t\t\t}\n\t\t\tswitch strings.ToUpper(strings.TrimSpace(test)) {\n\t\t\tcase \"NEARBY\", \"NEARBY-KNN\", \"NEARBY-KNN-10\":\n\t\t\t\tredbench.Bench(\"NEARBY (limit 10)\", addr, opts, prepFn,\n\t\t\t\t\tfunc(buf []byte) []byte {\n\t\t\t\t\t\tlat, lon := randPoint()\n\t\t\t\t\t\treturn redbench.AppendCommand(buf,\n\t\t\t\t\t\t\t\"NEARBY\", \"key:bench\", \"LIMIT\", \"10\", \"COUNT\", \"POINT\",\n\t\t\t\t\t\t\tstrconv.FormatFloat(lat, 'f', 5, 64),\n\t\t\t\t\t\t\tstrconv.FormatFloat(lon, 'f', 5, 64),\n\t\t\t\t\t\t)\n\t\t\t\t\t},\n\t\t\t\t)\n\t\t\t}\n\t\t\tswitch strings.ToUpper(strings.TrimSpace(test)) {\n\t\t\tcase \"NEARBY\", \"NEARBY-KNN\", \"NEARBY-KNN-100\":\n\t\t\t\tredbench.Bench(\"NEARBY (limit 100)\", addr, opts, prepFn,\n\t\t\t\t\tfunc(buf []byte) []byte {\n\t\t\t\t\t\tlat, lon := randPoint()\n\t\t\t\t\t\treturn redbench.AppendCommand(buf,\n\t\t\t\t\t\t\t\"NEARBY\", \"key:bench\", \"LIMIT\", \"100\", \"COUNT\", \"POINT\",\n\t\t\t\t\t\t\tstrconv.FormatFloat(lat, 'f', 5, 64),\n\t\t\t\t\t\t\tstrconv.FormatFloat(lon, 'f', 5, 64),\n\t\t\t\t\t\t)\n\t\t\t\t\t},\n\t\t\t\t)\n\t\t\t}\n\t\t\tswitch strings.ToUpper(strings.TrimSpace(test)) {\n\t\t\tcase \"NEARBY\", \"NEARBY-POINT\", \"NEARBY-POINT-1000\":\n\t\t\t\tredbench.Bench(\"NEARBY (point 1km)\", addr, opts, prepFn,\n\t\t\t\t\tfunc(buf []byte) []byte {\n\t\t\t\t\t\tlat, lon := randPoint()\n\t\t\t\t\t\treturn redbench.AppendCommand(buf,\n\t\t\t\t\t\t\t\"NEARBY\", \"key:bench\", \"COUNT\", \"POINT\",\n\t\t\t\t\t\t\tstrconv.FormatFloat(lat, 'f', 5, 64),\n\t\t\t\t\t\t\tstrconv.FormatFloat(lon, 'f', 5, 64),\n\t\t\t\t\t\t\t\"1000\",\n\t\t\t\t\t\t)\n\t\t\t\t\t},\n\t\t\t\t)\n\t\t\t}\n\t\t\tswitch strings.ToUpper(strings.TrimSpace(test)) {\n\t\t\tcase \"NEARBY\", \"NEARBY-POINT\", \"NEARBY-POINT-10000\":\n\t\t\t\tredbench.Bench(\"NEARBY (point 10km)\", addr, opts, prepFn,\n\t\t\t\t\tfunc(buf []byte) []byte {\n\t\t\t\t\t\tlat, lon := randPoint()\n\t\t\t\t\t\treturn redbench.AppendCommand(buf,\n\t\t\t\t\t\t\t\"NEARBY\", \"key:bench\", \"COUNT\", \"POINT\",\n\t\t\t\t\t\t\tstrconv.FormatFloat(lat, 'f', 5, 64),\n\t\t\t\t\t\t\tstrconv.FormatFloat(lon, 'f', 5, 64),\n\t\t\t\t\t\t\t\"10000\",\n\t\t\t\t\t\t)\n\t\t\t\t\t},\n\t\t\t\t)\n\t\t\t}\n\t\t\tswitch strings.ToUpper(strings.TrimSpace(test)) {\n\t\t\tcase \"NEARBY\", \"NEARBY-POINT\", \"NEARBY-POINT-100000\":\n\t\t\t\tredbench.Bench(\"NEARBY (point 100km)\", addr, opts, prepFn,\n\t\t\t\t\tfunc(buf []byte) []byte {\n\t\t\t\t\t\tlat, lon := randPoint()\n\t\t\t\t\t\treturn redbench.AppendCommand(buf,\n\t\t\t\t\t\t\t\"NEARBY\", \"key:bench\", \"COUNT\", \"POINT\",\n\t\t\t\t\t\t\tstrconv.FormatFloat(lat, 'f', 5, 64),\n\t\t\t\t\t\t\tstrconv.FormatFloat(lon, 'f', 5, 64),\n\t\t\t\t\t\t\t\"100000\",\n\t\t\t\t\t\t)\n\t\t\t\t\t},\n\t\t\t\t)\n\t\t\t}\n\t\tcase \"EVAL\":\n\t\t\tif !redis {\n\t\t\t\tvar i int64\n\t\t\t\tgetScript := \"return tile38.call('GET', KEYS[1], ARGV[1], 'point')\"\n\t\t\t\tget4Script :=\n\t\t\t\t\t\"local a = tile38.call('GET', KEYS[1], ARGV[1], 'point');\" +\n\t\t\t\t\t\t\"local b = tile38.call('GET', KEYS[1], ARGV[2], 'point');\" +\n\t\t\t\t\t\t\"local c = tile38.call('GET', KEYS[1], ARGV[3], 'point');\" +\n\t\t\t\t\t\t\"local d = tile38.call('GET', KEYS[1], ARGV[4], 'point');\" +\n\t\t\t\t\t\t\"return d\"\n\n\t\t\t\tsetScript := \"return tile38.call('SET', KEYS[1], ARGV[1], 'point', ARGV[2], ARGV[3])\"\n\t\t\t\tif !opts.Quiet {\n\t\t\t\t\tfmt.Println(\"Scripts to run:\")\n\t\t\t\t\tfmt.Println(\"GET SCRIPT: \" + getScript)\n\t\t\t\t\tfmt.Println(\"GET FOUR SCRIPT: \" + get4Script)\n\t\t\t\t\tfmt.Println(\"SET SCRIPT: \" + setScript)\n\t\t\t\t}\n\n\t\t\t\tredbench.Bench(\"EVAL (set point)\", addr, opts, prepFn,\n\t\t\t\t\tfunc(buf []byte) []byte {\n\t\t\t\t\t\ti := atomic.AddInt64(&i, 1)\n\t\t\t\t\t\tlat, lon := randPoint()\n\t\t\t\t\t\treturn redbench.AppendCommand(buf, \"EVAL\", setScript, \"1\",\n\t\t\t\t\t\t\t\"key:bench\",\n\t\t\t\t\t\t\t\"id:\"+strconv.FormatInt(i, 10),\n\t\t\t\t\t\t\tstrconv.FormatFloat(lat, 'f', 5, 64),\n\t\t\t\t\t\t\tstrconv.FormatFloat(lon, 'f', 5, 64),\n\t\t\t\t\t\t)\n\t\t\t\t\t},\n\t\t\t\t)\n\t\t\t\tredbench.Bench(\"EVALNA (set point)\", addr, opts, prepFn,\n\t\t\t\t\tfunc(buf []byte) []byte {\n\t\t\t\t\t\ti := atomic.AddInt64(&i, 1)\n\t\t\t\t\t\tlat, lon := randPoint()\n\t\t\t\t\t\treturn redbench.AppendCommand(buf, \"EVALNA\", setScript, \"1\",\n\t\t\t\t\t\t\t\"key:bench\",\n\t\t\t\t\t\t\t\"id:\"+strconv.FormatInt(i, 10),\n\t\t\t\t\t\t\tstrconv.FormatFloat(lat, 'f', 5, 64),\n\t\t\t\t\t\t\tstrconv.FormatFloat(lon, 'f', 5, 64),\n\t\t\t\t\t\t)\n\t\t\t\t\t},\n\t\t\t\t)\n\t\t\t\tredbench.Bench(\"EVALRO (get point)\", addr, opts, prepFn,\n\t\t\t\t\tfunc(buf []byte) []byte {\n\t\t\t\t\t\ti := atomic.AddInt64(&i, 1)\n\t\t\t\t\t\treturn redbench.AppendCommand(buf, \"EVALRO\", getScript, \"1\", \"key:bench\", \"id:\"+strconv.FormatInt(i, 10))\n\t\t\t\t\t},\n\t\t\t\t)\n\t\t\t\tredbench.Bench(\"EVALRO (get 4 points)\", addr, opts, prepFn,\n\t\t\t\t\tfunc(buf []byte) []byte {\n\t\t\t\t\t\ti := atomic.AddInt64(&i, 1)\n\t\t\t\t\t\treturn redbench.AppendCommand(buf, \"EVALRO\", get4Script, \"1\",\n\t\t\t\t\t\t\t\"key:bench\",\n\t\t\t\t\t\t\t\"id:\"+strconv.FormatInt(i, 10),\n\t\t\t\t\t\t\t\"id:\"+strconv.FormatInt(i+1, 10),\n\t\t\t\t\t\t\t\"id:\"+strconv.FormatInt(i+2, 10),\n\t\t\t\t\t\t\t\"id:\"+strconv.FormatInt(i+3, 10),\n\t\t\t\t\t\t)\n\t\t\t\t\t},\n\t\t\t\t)\n\t\t\t\tredbench.Bench(\"EVALNA (get point)\", addr, opts, prepFn,\n\t\t\t\t\tfunc(buf []byte) []byte {\n\t\t\t\t\t\ti := atomic.AddInt64(&i, 1)\n\t\t\t\t\t\treturn redbench.AppendCommand(buf, \"EVALNA\", getScript, \"1\", \"key:bench\", \"id:\"+strconv.FormatInt(i, 10))\n\t\t\t\t\t},\n\t\t\t\t)\n\t\t\t}\n\t\t}\n\t}\n}\n\nconst earthRadius = 6371e3\n\nfunc toRadians(deg float64) float64 { return deg * math.Pi / 180 }\nfunc toDegrees(rad float64) float64 { return rad * 180 / math.Pi }\n\n// destinationPoint return the destination from a point based on a distance and bearing.\nfunc destinationPoint(lat, lon, meters, bearingDegrees float64) (destLat, destLon float64) {\n\t// see http://williams.best.vwh.net/avform.htm#LL\n\tδ := meters / earthRadius // angular distance in radians\n\tθ := toRadians(bearingDegrees)\n\tφ1 := toRadians(lat)\n\tλ1 := toRadians(lon)\n\tφ2 := math.Asin(math.Sin(φ1)*math.Cos(δ) + math.Cos(φ1)*math.Sin(δ)*math.Cos(θ))\n\tλ2 := λ1 + math.Atan2(math.Sin(θ)*math.Sin(δ)*math.Cos(φ1), math.Cos(δ)-math.Sin(φ1)*math.Sin(φ2))\n\tλ2 = math.Mod(λ2+3*math.Pi, 2*math.Pi) - math.Pi // normalise to -180..+180°\n\treturn toDegrees(φ2), toDegrees(λ2)\n}\n" }, { "path": "cmd/tile38-cli/main.go", "content": "package main\n\nimport (\n\t\"bufio\"\n\t\"bytes\"\n\t\"encoding/json\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"net\"\n\t\"os\"\n\t\"os/exec\"\n\t\"path/filepath\"\n\t\"runtime\"\n\t\"sort\"\n\t\"strconv\"\n\t\"strings\"\n\n\t\"github.com/peterh/liner\"\n\t\"github.com/tidwall/gjson\"\n\t\"github.com/tidwall/resp\"\n\t\"github.com/tidwall/tile38/core\"\n)\n\nfunc getEnv(name string, defaultValue string) string {\n\tval, exists := os.LookupEnv(name)\n\tif !exists {\n\t\treturn defaultValue\n\t}\n\treturn val\n}\n\nfunc userHomeDir() string {\n\tif runtime.GOOS == \"windows\" {\n\t\thome := os.Getenv(\"HOMEDRIVE\") + os.Getenv(\"HOMEPATH\")\n\t\tif home == \"\" {\n\t\t\thome = os.Getenv(\"USERPROFILE\")\n\t\t}\n\t\treturn home\n\t}\n\treturn os.Getenv(\"HOME\")\n}\n\nvar (\n\thistoryFile = filepath.Join(userHomeDir(), \".liner_example_history\")\n)\n\ntype connError struct {\n\tOK bool `json:\"ok\"`\n\tErr string `json:\"err\"`\n}\n\nvar (\n\thostname = \"127.0.0.1\"\n\toutput = \"json\"\n\tport = 9851\n\toneCommand string\n\traw bool\n\tnoprompt bool\n\ttty bool\n\tstdin bool\n)\n\nfunc showHelp() bool {\n\n\tgitsha := \"\"\n\tif core.GitSHA == \"\" || core.GitSHA == \"0000000\" {\n\t\tgitsha = \"\"\n\t} else {\n\t\tgitsha = \" (git:\" + core.GitSHA + \")\"\n\t}\n\tfmt.Fprintf(os.Stdout, \"tile38-cli %s%s\\n\\n\", core.Version, gitsha)\n\tfmt.Fprintf(os.Stdout, \"Usage: tile38-cli [OPTIONS] [cmd [arg [arg ...]]]\\n\")\n\tfmt.Fprintf(os.Stdout, \" --raw Use raw formatting for replies\\n\")\n\tfmt.Fprintf(os.Stdout, \" --noprompt Do not display a prompt\\n\")\n\tfmt.Fprintf(os.Stdout, \" --resp Use RESP output formatting (default is JSON output)\\n\")\n\tfmt.Fprintf(os.Stdout, \" --json Use JSON output formatting (default is JSON output)\\n\")\n\tfmt.Fprintf(os.Stdout, \" -h Server hostname (default: %s)\\n\", hostname)\n\tfmt.Fprintf(os.Stdout, \" -p Server port (default: %d)\\n\", port)\n\tfmt.Fprintf(os.Stdout, \" -x Read last argument from STDIN.\\n\")\n\tfmt.Fprintf(os.Stdout, \"\\n\")\n\treturn false\n}\n\nfunc parseArgs() bool {\n\tdefer func() {\n\t\tif v := recover(); v != nil {\n\t\t\tif v, ok := v.(string); ok && v == \"bad arg\" {\n\t\t\t\tshowHelp()\n\t\t\t}\n\t\t}\n\t}()\n\n\thostname = getEnv(\"TILE38_HOSTNAME\", hostname)\n\toutput = getEnv(\"TILE38_OUTPUT\", output)\n\tportStr := getEnv(\"TILE38_PORT\", \"\")\n\n\tif portStr != \"\" {\n\t\ttempPort, err := strconv.Atoi(portStr)\n\t\tif err == nil {\n\t\t\tport = tempPort\n\t\t}\n\t}\n\n\targs := os.Args[1:]\n\treadArg := func(arg string) string {\n\t\tif len(args) == 0 {\n\t\t\tpanic(\"bad arg\")\n\t\t}\n\t\tvar narg = args[0]\n\t\targs = args[1:]\n\t\treturn narg\n\t}\n\tbadArg := func(arg string) bool {\n\t\tfmt.Fprintf(os.Stderr, \"Unrecognized option or bad number of args for: '%s'\\n\", arg)\n\t\treturn false\n\t}\n\n\tfor len(args) > 0 {\n\t\targ := readArg(\"\")\n\t\tif arg == \"--help\" || arg == \"-?\" {\n\t\t\treturn showHelp()\n\t\t}\n\t\tif !strings.HasPrefix(arg, \"-\") {\n\t\t\targs = append([]string{arg}, args...)\n\t\t\tbreak\n\t\t}\n\t\tswitch arg {\n\t\tdefault:\n\t\t\treturn badArg(arg)\n\t\tcase \"--raw\":\n\t\t\traw = true\n\t\tcase \"--tty\":\n\t\t\ttty = true\n\t\tcase \"--noprompt\":\n\t\t\tnoprompt = true\n\t\tcase \"--resp\":\n\t\t\toutput = \"resp\"\n\t\tcase \"--json\":\n\t\t\toutput = \"json\"\n\t\tcase \"-x\":\n\t\t\tstdin = true\n\t\tcase \"-h\":\n\t\t\thostname = readArg(arg)\n\t\tcase \"-p\":\n\t\t\tn, err := strconv.ParseUint(readArg(arg), 10, 16)\n\t\t\tif err != nil {\n\t\t\t\treturn badArg(arg)\n\t\t\t}\n\t\t\tport = int(n)\n\t\t}\n\t}\n\toneCommand = strings.Join(args, \" \")\n\tif stdin {\n\t\tdata, err := io.ReadAll(os.Stdin)\n\t\tif err != nil {\n\t\t\tprintln(err)\n\t\t}\n\t\tif !gjson.ValidBytes(data) {\n\t\t\tfmt.Fprintf(os.Stderr, \"Invalid STDIN: Not JSON\\n\")\n\t\t\treturn false\n\t\t}\n\t\targ := strings.Replace(string(data), \"\\r\", \"\", -1)\n\t\targ = strings.Replace(arg, \"\\n\", \"\", -1)\n\t\targ = strings.Replace(arg, \"'\", \"\\\\'\", -1)\n\t\toneCommand += \" '\" + arg + \"'\"\n\t}\n\treturn true\n}\n\nfunc refusedErrorString(addr string) string {\n\treturn fmt.Sprintf(\"Could not connect to Tile38 at %s: Connection refused\", addr)\n}\n\nvar groupsM = make(map[string][]string)\n\nfunc jsonOK(msg []byte) bool {\n\treturn gjson.GetBytes(msg, \"ok\").Bool()\n}\n\nfunc main() {\n\tif !parseArgs() {\n\t\treturn\n\t}\n\n\tif len(oneCommand) > 0 && strings.Split(strings.ToLower(oneCommand), \" \")[0] == \"help\" {\n\t\tshowHelp()\n\t\treturn\n\t}\n\n\taddr := fmt.Sprintf(\"%s:%d\", hostname, port)\n\tvar conn *client\n\tconnDial := func() {\n\t\tvar err error\n\t\tconn, err = clientDial(\"tcp\", addr)\n\t\tif err != nil {\n\t\t\tif _, ok := err.(net.Error); ok {\n\t\t\t\tfmt.Fprintln(os.Stderr, refusedErrorString(addr))\n\t\t\t} else {\n\t\t\t\tfmt.Fprintln(os.Stderr, err.Error())\n\t\t\t\tos.Exit(1)\n\t\t\t}\n\t\t\tif oneCommand != \"\" {\n\t\t\t\tos.Exit(1)\n\t\t\t}\n\t\t} else if _, err := conn.Do(\"output \" + output); err != nil {\n\t\t\tfmt.Fprintln(os.Stderr, err.Error())\n\t\t\tos.Exit(1)\n\t\t}\n\t}\n\tconnDial()\n\tmonitor := false\n\tlivemode := false\n\taof := false\n\tdefer func() {\n\t\tif livemode {\n\t\t\tvar err error\n\t\t\tif aof {\n\t\t\t\t_, err = io.Copy(os.Stdout, conn.Reader())\n\t\t\t\tfmt.Fprintln(os.Stderr, \"\")\n\t\t\t} else {\n\t\t\t\tvar msg []byte\n\t\t\t\tfor {\n\t\t\t\t\tmsg, err = conn.readLiveResp()\n\t\t\t\t\tif err != nil {\n\t\t\t\t\t\tbreak\n\t\t\t\t\t}\n\t\t\t\t\tif !raw {\n\t\t\t\t\t\tif output == \"resp\" {\n\t\t\t\t\t\t\tmsg = convert2termresp(msg)\n\t\t\t\t\t\t} else {\n\t\t\t\t\t\t\tmsg = convert2termjson(msg)\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t\tfmt.Fprintln(os.Stderr, string(msg))\n\t\t\t\t}\n\t\t\t}\n\t\t\tif err != nil && err != io.EOF {\n\t\t\t\tfmt.Fprintln(os.Stderr, err.Error())\n\t\t\t}\n\t\t}\n\t}()\n\n\tline := liner.NewLiner()\n\tdefer line.Close()\n\n\tvar commands []string\n\tfor name, command := range core.Commands {\n\t\tcommands = append(commands, name)\n\t\tgroupsM[command.Group] = append(groupsM[command.Group], name)\n\t}\n\tsort.Strings(commands)\n\tvar groups []string\n\tfor group, arr := range groupsM {\n\t\tgroups = append(groups, \"@\"+group)\n\t\tsort.Strings(arr)\n\t\tgroupsM[group] = arr\n\t}\n\tsort.Strings(groups)\n\n\tline.SetMultiLineMode(false)\n\tline.SetCtrlCAborts(true)\n\tif !(noprompt && tty) {\n\t\tline.SetCompleter(func(line string) (c []string) {\n\t\t\tif strings.HasPrefix(strings.ToLower(line), \"help \") {\n\t\t\t\tvar nitems []string\n\t\t\t\tnline := strings.TrimSpace(line[5:])\n\t\t\t\tif nline == \"\" || nline[0] == '@' {\n\t\t\t\t\tfor _, n := range groups {\n\t\t\t\t\t\tif strings.HasPrefix(strings.ToLower(n), strings.ToLower(nline)) {\n\t\t\t\t\t\t\tnitems = append(nitems, line[:len(line)-len(nline)]+strings.ToLower(n))\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t} else {\n\t\t\t\t\tfor _, n := range commands {\n\t\t\t\t\t\tif strings.HasPrefix(strings.ToLower(n), strings.ToLower(nline)) {\n\t\t\t\t\t\t\tnitems = append(nitems, line[:len(line)-len(nline)]+strings.ToUpper(n))\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t\tfor _, n := range nitems {\n\t\t\t\t\tif strings.HasPrefix(strings.ToLower(n), strings.ToLower(line)) {\n\t\t\t\t\t\tc = append(c, n)\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\tfor _, n := range commands {\n\t\t\t\t\tif strings.HasPrefix(strings.ToLower(n), strings.ToLower(line)) {\n\t\t\t\t\t\tc = append(c, n)\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t\treturn\n\t\t})\n\t}\n\tif f, err := os.Open(historyFile); err == nil {\n\t\tline.ReadHistory(f)\n\t\tf.Close()\n\t}\n\tdefer func() {\n\t\tif f, err := os.Create(historyFile); err != nil {\n\t\t\tfmt.Fprintln(os.Stderr, err.Error())\n\t\t} else {\n\t\t\tline.WriteHistory(f)\n\t\t\tf.Close()\n\t\t}\n\t}()\n\n\tpassword := getEnv(\"TILE38_PASSWORD\", \"\")\n\n\tif conn != nil && password != \"\" {\n\t\tconn.Do(fmt.Sprintf(\"auth %s\", password))\n\t}\n\n\tfor {\n\n\t\tvar command string\n\t\tvar err error\n\t\tif oneCommand == \"\" {\n\t\t\tif raw || noprompt {\n\t\t\t\tcommand, err = line.Prompt(\"\")\n\t\t\t} else {\n\t\t\t\tif conn == nil {\n\t\t\t\t\tcommand, err = line.Prompt(\"not connected> \")\n\t\t\t\t} else {\n\t\t\t\t\tcommand, err = line.Prompt(addr + \"> \")\n\t\t\t\t}\n\t\t\t}\n\n\t\t} else {\n\t\t\tcommand = oneCommand\n\t\t}\n\t\tif err == nil {\n\t\t\tnohist := strings.HasPrefix(command, \" \")\n\t\t\tcommand = strings.TrimSpace(command)\n\t\t\tif command == \"\" {\n\t\t\t\tif conn != nil {\n\t\t\t\t\t_, err := conn.Do(\"pInG\")\n\t\t\t\t\tif err != nil {\n\t\t\t\t\t\tif err != io.EOF && !strings.Contains(err.Error(), \"broken pipe\") {\n\t\t\t\t\t\t\tfmt.Fprintln(os.Stderr, err.Error())\n\t\t\t\t\t\t} else {\n\t\t\t\t\t\t\tfmt.Fprintln(os.Stderr, refusedErrorString(addr))\n\t\t\t\t\t\t}\n\t\t\t\t\t\tconn.wr.Close()\n\t\t\t\t\t\tconn = nil\n\t\t\t\t\t\tcontinue\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\tif !nohist {\n\t\t\t\t\tline.AppendHistory(command)\n\t\t\t\t}\n\t\t\t\tif strings.ToLower(command) == \"exit\" {\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t\tif strings.ToLower(command) == \"quit\" {\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t\tif strings.ToLower(command) == \"clear\" {\n\t\t\t\t\tclearScreen()\n\t\t\t\t\tcontinue\n\t\t\t\t}\n\t\t\t\tif strings.ToLower(command) == \"help\" || strings.HasPrefix(strings.ToLower(command), \"help\") {\n\t\t\t\t\terr = help(strings.TrimSpace(command[4:]))\n\t\t\t\t\tif err != nil {\n\t\t\t\t\t\treturn\n\t\t\t\t\t}\n\t\t\t\t\tcontinue\n\t\t\t\t}\n\t\t\t\taof = strings.HasPrefix(strings.ToLower(command), \"aof \")\n\t\t\ttryAgain:\n\t\t\t\tif conn == nil {\n\t\t\t\t\tconnDial()\n\t\t\t\t\tif conn == nil {\n\t\t\t\t\t\tcontinue\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t\tmsg, err := conn.Do(command)\n\t\t\t\tif err != nil {\n\t\t\t\t\tif err != io.EOF {\n\t\t\t\t\t\tfmt.Fprintln(os.Stderr, err.Error())\n\t\t\t\t\t}\n\t\t\t\t\tconn.wr.Close()\n\t\t\t\t\tconn = nil\n\t\t\t\t\tgoto tryAgain\n\t\t\t\t}\n\t\t\t\tswitch strings.ToLower(command) {\n\t\t\t\tcase \"output resp\":\n\t\t\t\t\tif string(msg) == \"+OK\\r\\n\" {\n\t\t\t\t\t\toutput = \"resp\"\n\t\t\t\t\t}\n\t\t\t\tcase \"output json\":\n\t\t\t\t\tif jsonOK(msg) {\n\t\t\t\t\t\toutput = \"json\"\n\t\t\t\t\t}\n\t\t\t\tcase \"monitor\":\n\t\t\t\t\tmonitor = true\n\t\t\t\t\tlivemode = true\n\t\t\t\t\toutput = \"resp\"\n\t\t\t\t}\n\t\t\t\tif output == \"resp\" &&\n\t\t\t\t\t(strings.HasPrefix(string(msg), \"*3\\r\\n$10\\r\\npsubscribe\\r\\n\") ||\n\t\t\t\t\t\tstrings.HasPrefix(string(msg), \"*3\\r\\n$9\\r\\nsubscribe\\r\\n\")) {\n\t\t\t\t\tlivemode = true\n\t\t\t\t}\n\t\t\t\tif !raw {\n\t\t\t\t\tif output == \"resp\" {\n\t\t\t\t\t\tmsg = convert2termresp(msg)\n\t\t\t\t\t} else {\n\t\t\t\t\t\tmsg = convert2termjson(msg)\n\t\t\t\t\t}\n\t\t\t\t}\n\n\t\t\t\tif !livemode && output == \"json\" {\n\t\t\t\t\tif gjson.GetBytes(msg, \"command\").String() == \"psubscribe\" ||\n\t\t\t\t\t\tgjson.GetBytes(msg, \"command\").String() == \"subscribe\" ||\n\t\t\t\t\t\tstring(msg) == liveJSON {\n\t\t\t\t\t\tlivemode = true\n\t\t\t\t\t}\n\t\t\t\t}\n\n\t\t\t\tmustOutput := true\n\t\t\t\tif !monitor && oneCommand == \"\" && output == \"json\" && !jsonOK(msg) {\n\t\t\t\t\tvar cerr connError\n\t\t\t\t\tif err := json.Unmarshal(msg, &cerr); err == nil {\n\t\t\t\t\t\tfmt.Fprintln(os.Stderr, \"(error) \"+cerr.Err)\n\t\t\t\t\t\tmustOutput = false\n\t\t\t\t\t}\n\t\t\t\t} else if livemode {\n\t\t\t\t\tfmt.Fprintln(os.Stderr, string(msg))\n\t\t\t\t\tbreak // break out of prompt and just feed data to screen\n\t\t\t\t}\n\t\t\t\tif mustOutput {\n\t\t\t\t\tfmt.Fprintln(os.Stdout, string(msg))\n\t\t\t\t}\n\t\t\t}\n\t\t} else if err == liner.ErrPromptAborted {\n\t\t\treturn\n\t\t} else if err == io.EOF {\n\t\t\tos.Exit(0)\n\t\t} else {\n\t\t\tfmt.Fprintf(os.Stderr, \"Error reading line: %s\", err.Error())\n\t\t}\n\t\tif oneCommand != \"\" {\n\t\t\treturn\n\t\t}\n\t}\n}\n\nfunc convert2termresp(msg []byte) []byte {\n\trd := resp.NewReader(bytes.NewBuffer(msg))\n\tout := \"\"\n\tfor {\n\t\tv, _, err := rd.ReadValue()\n\t\tif err != nil {\n\t\t\tbreak\n\t\t}\n\t\tout += convert2termrespval(v, 0)\n\t}\n\treturn []byte(strings.TrimSpace(out))\n}\n\nfunc convert2termjson(msg []byte) []byte {\n\tif msg[0] == '{' {\n\t\treturn msg\n\t}\n\treturn bytes.TrimSpace(msg[bytes.IndexByte(msg, '\\n')+1:])\n}\n\nfunc convert2termrespval(v resp.Value, spaces int) string {\n\tswitch v.Type() {\n\tdefault:\n\t\treturn v.String()\n\tcase resp.BulkString:\n\t\tif v.IsNull() {\n\t\t\treturn \"(nil)\"\n\t\t}\n\t\treturn \"\\\"\" + v.String() + \"\\\"\"\n\tcase resp.Integer:\n\t\treturn \"(integer) \" + v.String()\n\tcase resp.Error:\n\t\treturn \"(error) \" + v.String()\n\tcase resp.Array:\n\t\tarr := v.Array()\n\t\tif len(arr) == 0 {\n\t\t\treturn \"(empty list or set)\"\n\t\t}\n\t\tout := \"\"\n\t\tnspaces := spaces + numlen(len(arr))\n\t\tfor i, v := range arr {\n\t\t\tif i > 0 {\n\t\t\t\tout += strings.Repeat(\" \", spaces)\n\t\t\t}\n\t\t\tiout := strings.TrimSpace(convert2termrespval(v, nspaces+2))\n\t\t\tout += fmt.Sprintf(\"%d) %s\\n\", i+1, iout)\n\t\t}\n\t\treturn out\n\t}\n}\n\nfunc numlen(n int) int {\n\tl := 1\n\tif n < 0 {\n\t\tl++\n\t\tn = n * -1\n\t}\n\tfor i := 0; i < 1000; i++ {\n\t\tif n < 10 {\n\t\t\tbreak\n\t\t}\n\t\tl++\n\t\tn = n / 10\n\t}\n\treturn l\n}\n\nfunc help(arg string) error {\n\tvar groupsA []string\n\tfor group := range groupsM {\n\t\tgroupsA = append(groupsA, \"@\"+group)\n\t}\n\tgroups := \"Groups: \" + strings.Join(groupsA, \", \") + \"\\n\"\n\n\tif arg == \"\" {\n\t\tfmt.Fprintf(os.Stderr, \"tile38-cli %s (git:%s)\\n\", core.Version, core.GitSHA)\n\t\tfmt.Fprintf(os.Stderr, `Type: \"help @\" to get a list of commands in `+\"\\n\")\n\t\tfmt.Fprintf(os.Stderr, ` \"help \" for help on `+\"\\n\")\n\t\tif !(noprompt && tty) {\n\t\t\tfmt.Fprintf(os.Stderr, ` \"help \" to get a list of possible help topics`+\"\\n\")\n\t\t}\n\t\tfmt.Fprintf(os.Stderr, ` \"quit\" to exit`+\"\\n\")\n\t\tif noprompt && tty {\n\t\t\tfmt.Fprint(os.Stderr, groups)\n\t\t}\n\t\treturn nil\n\t}\n\tshowGroups := false\n\tfound := false\n\tif strings.HasPrefix(arg, \"@\") {\n\t\tfor _, command := range groupsM[arg[1:]] {\n\t\t\tfmt.Fprintf(os.Stderr, \"%s\\n\", core.Commands[command].TermOutput(\" \"))\n\t\t\tfound = true\n\t\t}\n\t\tif !found {\n\t\t\tshowGroups = true\n\t\t}\n\t} else {\n\t\tif command, ok := core.Commands[strings.ToUpper(arg)]; ok {\n\t\t\tfmt.Fprintf(os.Stderr, \"%s\\n\", command.TermOutput(\" \"))\n\t\t\tfound = true\n\t\t}\n\t}\n\tif showGroups {\n\t\tif noprompt && tty {\n\t\t\tfmt.Fprint(os.Stderr, groups)\n\t\t}\n\t} else if !found {\n\t\tif noprompt && tty {\n\t\t\thelp(\"\")\n\t\t}\n\t}\n\treturn nil\n}\n\nconst liveJSON = `{\"ok\":true,\"live\":true}`\n\ntype client struct {\n\twr net.Conn\n\trd *bufio.Reader\n}\n\nfunc clientDial(network, addr string) (*client, error) {\n\tconn, err := net.Dial(network, addr)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn &client{wr: conn, rd: bufio.NewReader(conn)}, nil\n}\n\nfunc (c *client) Do(command string) ([]byte, error) {\n\t_, err := c.wr.Write(plainToCompat(command))\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn c.readResp()\n}\n\nfunc (c *client) readResp() ([]byte, error) {\n\tch, err := c.rd.Peek(1)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tswitch ch[0] {\n\tcase ':', '+', '-', '{':\n\t\treturn c.readLine()\n\tcase '$':\n\t\treturn c.readBulk()\n\tcase '*':\n\t\treturn c.readArray()\n\tdefault:\n\t\treturn nil, fmt.Errorf(\"invalid response character '%c\", ch[0])\n\t}\n}\n\nfunc (c *client) readArray() ([]byte, error) {\n\tout, err := c.readLine()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tn, err := strconv.ParseUint(string(bytes.TrimSpace(out[1:])), 10, 64)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tfor i := 0; i < int(n); i++ {\n\t\tresp, err := c.readResp()\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\tout = append(out, resp...)\n\t}\n\treturn out, nil\n}\n\nfunc (c *client) readBulk() ([]byte, error) {\n\tline, err := c.readLine()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tx, err := strconv.ParseInt(string(bytes.TrimSpace(line[1:])), 10, 64)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tif x < 0 {\n\t\treturn line, nil\n\t}\n\tout := make([]byte, len(line)+int(x)+2)\n\tif _, err := io.ReadFull(c.rd, out[len(line):]); err != nil {\n\t\treturn nil, err\n\t}\n\tif !bytes.HasSuffix(out, []byte{'\\r', '\\n'}) {\n\t\treturn nil, errors.New(\"invalid response\")\n\t}\n\tcopy(out, line)\n\treturn out, nil\n}\n\nfunc (c *client) readLine() ([]byte, error) {\n\tline, err := c.rd.ReadBytes('\\r')\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tch, err := c.rd.ReadByte()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tif ch != '\\n' {\n\t\treturn nil, errors.New(\"invalid response\")\n\t}\n\treturn append(line, '\\n'), nil\n}\n\nfunc (c *client) Reader() io.Reader {\n\treturn c.rd\n}\n\nfunc (c *client) readLiveResp() (message []byte, err error) {\n\treturn c.readResp()\n}\n\n// plainToCompat converts a plain message like \"SET fleet truck1 ...\" into a\n// Tile38 compatible blob.\nfunc plainToCompat(message string) []byte {\n\tvar args []string\n\t// search for the beginning of the first argument\n\tfor i := 0; i < len(message); i++ {\n\t\tif message[i] != ' ' {\n\t\t\t// first argument found\n\t\t\tif message[i] == '\"' || message[i] == '\\'' {\n\t\t\t\t// using a string caps\n\t\t\t\ts := i\n\t\t\t\tcap := message[i]\n\t\t\t\tfor ; i < len(message); i++ {\n\t\t\t\t\tif message[i] == cap {\n\t\t\t\t\t\tif message[i-1] == '\\\\' {\n\t\t\t\t\t\t\tcontinue\n\t\t\t\t\t\t}\n\t\t\t\t\t\tif i == len(message)-1 || message[i+1] == ' ' {\n\t\t\t\t\t\t\targs = append(args, message[s:i+1])\n\t\t\t\t\t\t\ti++\n\t\t\t\t\t\t\tbreak\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\t// using plain string, terminated by a space\n\t\t\t\ts := i\n\t\t\t\tvar quotes bool\n\t\t\t\tfor ; i < len(message); i++ {\n\t\t\t\t\tif message[i] == '\"' || message[i] == '\\'' {\n\t\t\t\t\t\tquotes = true\n\t\t\t\t\t}\n\t\t\t\t\tif i == len(message)-1 || message[i+1] == ' ' {\n\t\t\t\t\t\targ := message[s : i+1]\n\t\t\t\t\t\tif quotes {\n\t\t\t\t\t\t\targ = strconv.Quote(arg)\n\t\t\t\t\t\t}\n\t\t\t\t\t\targs = append(args, arg)\n\t\t\t\t\t\ti++\n\t\t\t\t\t\tbreak\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\treturn []byte(strings.Join(args, \" \") + \"\\r\\n\")\n}\n\nfunc clearScreen() {\n\tvar cmd *exec.Cmd\n\tif runtime.GOOS == \"windows\" {\n\t\tcmd = exec.Command(\"cmd\", \"/c\", \"cls\")\n\t} else {\n\t\tcmd = exec.Command(\"clear\")\n\t}\n\tcmd.Stdout = os.Stdout\n\tcmd.Run()\n}\n" }, { "path": "cmd/tile38-luamemtest/main.go", "content": "package main\n\nimport (\n\t\"crypto/sha1\"\n\t\"encoding/hex\"\n\t\"errors\"\n\t\"fmt\"\n\t\"math\"\n\t\"runtime\"\n\t\"runtime/debug\"\n\n\t\"strings\"\n\n\t\"github.com/tidwall/resp\"\n\tlua \"github.com/yuin/gopher-lua\"\n)\n\nfunc Sha1Sum(s string) string {\n\th := sha1.New()\n\th.Write([]byte(s))\n\treturn hex.EncodeToString(h.Sum(nil))\n}\n\n// Convert lua LValue to RESP value\nfunc ConvertToResp(val lua.LValue) resp.Value {\n\tswitch val.Type() {\n\tcase lua.LTNil:\n\t\treturn resp.NullValue()\n\tcase lua.LTBool:\n\t\tif val == lua.LTrue {\n\t\t\treturn resp.IntegerValue(1)\n\t\t} else {\n\t\t\treturn resp.NullValue()\n\t\t}\n\tcase lua.LTNumber:\n\t\tif float := float64(val.(lua.LNumber)); math.IsNaN(float) || math.IsInf(float, 0) {\n\t\t\treturn resp.FloatValue(float)\n\t\t} else {\n\t\t\treturn resp.IntegerValue(int(math.Floor(float)))\n\t\t}\n\tcase lua.LTString:\n\t\treturn resp.StringValue(val.String())\n\tcase lua.LTTable:\n\t\tvar values []resp.Value\n\t\tvar specialValues []resp.Value\n\t\tvar cb func(lk lua.LValue, lv lua.LValue)\n\t\ttbl := val.(*lua.LTable)\n\n\t\tif tbl.Len() != 0 { // list\n\t\t\tcb = func(lk lua.LValue, lv lua.LValue) {\n\t\t\t\tvalues = append(values, ConvertToResp(lv))\n\t\t\t}\n\t\t} else { // map\n\t\t\tcb = func(lk lua.LValue, lv lua.LValue) {\n\t\t\t\tif lk.Type() == lua.LTString {\n\t\t\t\t\tlks := lk.String()\n\t\t\t\t\tswitch lks {\n\t\t\t\t\tcase \"ok\":\n\t\t\t\t\t\tspecialValues = append(specialValues, resp.SimpleStringValue(lv.String()))\n\t\t\t\t\tcase \"err\":\n\t\t\t\t\t\tspecialValues = append(specialValues, resp.ErrorValue(errors.New(lv.String())))\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t\tvalues = append(values, resp.ArrayValue(\n\t\t\t\t\t[]resp.Value{ConvertToResp(lk), ConvertToResp(lv)}))\n\t\t\t}\n\t\t}\n\t\ttbl.ForEach(cb)\n\t\tif len(values) == 1 && len(specialValues) == 1 {\n\t\t\treturn specialValues[0]\n\t\t}\n\t\treturn resp.ArrayValue(values)\n\t}\n\treturn resp.ErrorValue(errors.New(\"Unsupported lua type: \" + val.Type().String()))\n}\n\n// Convert RESP value to lua LValue\nfunc ConvertToLua(L *lua.LState, val resp.Value) lua.LValue {\n\tif val.IsNull() {\n\t\treturn lua.LFalse\n\t}\n\tswitch val.Type() {\n\tcase resp.Integer:\n\t\treturn lua.LNumber(val.Integer())\n\tcase resp.BulkString:\n\t\treturn lua.LString(val.String())\n\tcase resp.Error:\n\t\ttbl := L.CreateTable(0, 1)\n\t\ttbl.RawSetString(\"err\", lua.LString(val.String()))\n\t\treturn tbl\n\tcase resp.SimpleString:\n\t\ttbl := L.CreateTable(0, 1)\n\t\ttbl.RawSetString(\"ok\", lua.LString(val.String()))\n\t\treturn tbl\n\tcase resp.Array:\n\t\ttbl := L.CreateTable(len(val.Array()), 0)\n\t\tfor _, item := range val.Array() {\n\t\t\ttbl.Append(ConvertToLua(L, item))\n\t\t}\n\t\treturn tbl\n\t}\n\treturn lua.LString(\"ERR: unknown RESP type: \" + val.Type().String())\n}\n\nfunc luaTile38Call(evalcmd string, cmd string, args ...string) (resp.Value, error) {\n\tvar values []resp.Value\n\tvalues = append(values, resp.StringValue(\"RUNNING:\"))\n\tvalues = append(values, resp.StringValue(evalcmd))\n\tvalues = append(values, resp.StringValue(cmd))\n\tfor _, arg := range args {\n\t\tvalues = append(values, resp.StringValue(arg))\n\t}\n\n\treturn resp.ArrayValue(values), nil\n}\n\nfunc NewLuaState() *lua.LState {\n\tL := lua.NewState()\n\n\tget_args := func(ls *lua.LState) (evalCmd string, args []string) {\n\t\tevalCmd = ls.GetGlobal(\"EVAL_CMD\").String()\n\t\t//log.Debugf(\"EVAL_CMD %s\\n\", evalCmd)\n\n\t\t// Trying to work with unknown number of args.\n\t\t// When we see empty arg we call it enough.\n\t\tfor i := 1; ; i++ {\n\t\t\tif arg := ls.ToString(i); arg == \"\" {\n\t\t\t\tbreak\n\t\t\t} else {\n\t\t\t\targs = append(args, arg)\n\t\t\t}\n\t\t}\n\t\treturn\n\t}\n\tcall := func(ls *lua.LState) int {\n\t\tevalCmd, args := get_args(ls)\n\t\tif res, err := luaTile38Call(evalCmd, args[0], args[1:]...); err != nil {\n\t\t\t//log.Debugf(\"RES type: %s value: %s ERR %s\\n\", res.Type(), res.String(), err);\n\t\t\tls.RaiseError(\"ERR %s\", err.Error())\n\t\t\treturn 0\n\t\t} else {\n\t\t\t//log.Debugf(\"RES type: %s value: %s\\n\", res.Type(), res.String());\n\t\t\tls.Push(ConvertToLua(ls, res))\n\t\t\treturn 1\n\t\t}\n\t}\n\tpcall := func(ls *lua.LState) int {\n\t\tevalCmd, args := get_args(ls)\n\t\tif res, err := luaTile38Call(evalCmd, args[0], args[1:]...); err != nil {\n\t\t\t//log.Debugf(\"RES type: %s value: %s ERR %s\\n\", res.Type(), res.String(), err);\n\t\t\tls.Push(ConvertToLua(ls, resp.ErrorValue(err)))\n\t\t\treturn 1\n\t\t} else {\n\t\t\t//log.Debugf(\"RES type: %s value: %s\\n\", res.Type(), res.String());\n\t\t\tls.Push(ConvertToLua(ls, res))\n\t\t\treturn 1\n\t\t}\n\t}\n\terror_reply := func(ls *lua.LState) int {\n\t\ttbl := L.CreateTable(0, 1)\n\t\ttbl.RawSetString(\"err\", lua.LString(ls.ToString(1)))\n\t\tls.Push(tbl)\n\t\treturn 1\n\t}\n\tstatus_reply := func(ls *lua.LState) int {\n\t\ttbl := L.CreateTable(0, 1)\n\t\ttbl.RawSetString(\"ok\", lua.LString(ls.ToString(1)))\n\t\tls.Push(tbl)\n\t\treturn 1\n\t}\n\tsha1hex := func(ls *lua.LState) int {\n\t\tsha_sum := Sha1Sum(ls.ToString(1))\n\t\tls.Push(lua.LString(sha_sum))\n\t\treturn 1\n\t}\n\tvar exports = map[string]lua.LGFunction{\n\t\t\"call\": call,\n\t\t\"pcall\": pcall,\n\t\t\"error_reply\": error_reply,\n\t\t\"status_reply\": status_reply,\n\t\t\"sha1hex\": sha1hex,\n\t}\n\tL.SetGlobal(\"tile38\", L.SetFuncs(L.NewTable(), exports))\n\treturn L\n}\n\nfunc makeSafeErr(err error) error {\n\treturn errors.New(strings.Replace(err.Error(), \"\\n\", `\\n`, -1))\n}\n\nfunc runLuaFunc(luaState *lua.LState, script string, name string) resp.Value {\n\tluaState.SetGlobal(\"EVAL_CMD\", lua.LString(\"FAKE_EVAL\"))\n\tfn, err := luaState.Load(strings.NewReader(script), name)\n\tif err != nil {\n\t\treturn resp.ErrorValue(makeSafeErr(err))\n\t}\n\tluaState.Push(fn)\n\tif err := luaState.PCall(0, 1, nil); err != nil {\n\t\treturn resp.ErrorValue(makeSafeErr(err))\n\t}\n\tret := luaState.Get(-1) // returned value\n\tluaState.Pop(1)\n\tluaState.SetGlobal(\"EVAL_CMD\", lua.LNil)\n\treturn ConvertToResp(ret)\n}\n\nfunc runMany(luaState *lua.LState, start int, num int) int {\n\tfmt.Printf(\"\\nRunning %d lua calls... \", num)\n\tfor i := 0; i < num; i++ {\n\t\tscript := fmt.Sprintf(\"return tile38.call('foo', 'bar', %d)\", i)\n\t\tname := fmt.Sprintf(\"f_%020d\", i)\n\t\tret := runLuaFunc(luaState, script, name)\n\t\tif ret.Type() == resp.Error {\n\t\t\tpanic(ret.String())\n\t\t}\n\t}\n\tfmt.Printf(\"done.\\n\")\n\treturn start + num\n}\n\nfunc printMemStats() {\n\tvar mem runtime.MemStats\n\truntime.GC()\n\tdebug.FreeOSMemory()\n\truntime.GC()\n\tdebug.FreeOSMemory()\n\truntime.GC()\n\tdebug.FreeOSMemory()\n\truntime.GC()\n\tdebug.FreeOSMemory()\n\truntime.ReadMemStats(&mem)\n\tfmt.Printf(\"MemStats: Alloc %d, HeapAlloc %d, HeapSys %d, GCSys %d, HeapObjects %d.\\n\",\n\t\tmem.Alloc, mem.HeapAlloc, mem.HeapSys, mem.GCSys, mem.HeapObjects)\n}\n\nfunc testLua() {\n\tvar luaState *lua.LState\n\tstart := 12345\n\tluaState = NewLuaState()\n\n\tprintMemStats()\n\n\tfmt.Printf(\"\\nRunning single call as a test\\n\")\n\tret := runLuaFunc(luaState, \"return tile38.call('fake_cmd', 'a', 'b')\", \"test_call\")\n\tfmt.Printf(\"Result: %s\\n\", ret.String())\n\n\tprintMemStats()\n\n\tstart = runMany(luaState, start, 100)\n\tprintMemStats()\n\n\tstart = runMany(luaState, start, 100)\n\tprintMemStats()\n\n\tstart = runMany(luaState, start, 100)\n\tprintMemStats()\n\n\tstart = runMany(luaState, start, 100)\n\tprintMemStats()\n\n\tstart = runMany(luaState, start, 1000)\n\tprintMemStats()\n\n\tstart = runMany(luaState, start, 10000)\n\tprintMemStats()\n\n\tstart = runMany(luaState, start, 1000)\n\tprintMemStats()\n\n\tstart = runMany(luaState, start, 100)\n\tprintMemStats()\n\n\t_ = runMany(luaState, start, 1000)\n\tprintMemStats()\n\n\tluaState.Close()\n}\n\nfunc main() {\n\tfmt.Printf(\"Starting memtest.\\n\")\n\ttestLua()\n}\n" }, { "path": "cmd/tile38-server/main.go", "content": "package main\n\nimport (\n\t\"flag\"\n\t\"fmt\"\n\t\"io\"\n\t\"net\"\n\t\"net/http\"\n\t_ \"net/http/pprof\"\n\t\"os\"\n\t\"os/signal\"\n\t\"path/filepath\"\n\t\"runtime\"\n\t\"runtime/pprof\"\n\t\"strconv\"\n\t\"strings\"\n\t\"sync\"\n\t\"syscall\"\n\n\t\"github.com/tidwall/gjson\"\n\t\"github.com/tidwall/tile38/core\"\n\t\"github.com/tidwall/tile38/internal/hservice\"\n\t\"github.com/tidwall/tile38/internal/log\"\n\t\"github.com/tidwall/tile38/internal/server\"\n\n\t\"golang.org/x/net/context\"\n\t\"google.golang.org/grpc\"\n)\n\n// TODO: Set to false in 2.*\nvar httpTransport = true\n\n////////////////////////////////////////////////////////////////////////////////\n//\n// Fire up a webhook test server by using the --webhook-http-consumer-port\n// for example\n// $ ./tile38-server --webhook-http-consumer-port 9999\n//\n// The create hooks like such...\n// SETHOOK myhook http://localhost:9999/myhook NEARBY mykey FENCE POINT 33.5 -115.5 1000\n//\n////////////////////////////////////////////////////////////////////////////////\n//\n// Memory profiling - start the server with the -pprofport flag\n//\n// $ ./tile38-server -pprofport 6060\n//\n// Then, at any point, from a different terminal execute:\n// $ go tool pprof -svg http://localhost:6060/debug/pprof/heap > out.svg\n//\n// Load the SVG into a web browser to visualize the memory usage\n//\n////////////////////////////////////////////////////////////////////////////////\n\ntype hserver struct{}\n\nfunc (s *hserver) Send(ctx context.Context, in *hservice.MessageRequest) (*hservice.MessageReply, error) {\n\treturn &hservice.MessageReply{Ok: true}, nil\n}\n\nfunc main() {\n\tgitsha := \" (\" + core.GitSHA + \")\"\n\tif gitsha == \" (0000000)\" {\n\t\tgitsha = \"\"\n\t}\n\tversionLine := `tile38-server version: ` + core.Version + gitsha\n\n\toutput := os.Stderr\n\tflag.Usage = func() {\n\t\tfmt.Fprintf(output,\n\t\t\t\"%s\", versionLine+`\n\nUsage: tile38-server [-p port]\n\nBasic Options:\n -h hostname : listening host\n -p port : listening port (default: 9851)\n -d path : data directory (default: data)\n -s socket : listen on unix socket file\n -l encoding : set log encoding to json or text (default: text) \n -o output : auto set client output to json or resp (default: resp) \n -q : no logging. totally silent output\n -v : enable verbose logging\n -vv : enable very verbose logging\n\nAdvanced Options: \n --pidfile path : file that contains the pid\n --appendonly yes/no : AOF persistence (default: yes)\n --appendfilename path : AOF path (default: data/appendonly.aof)\n --queuefilename path : Event queue path (default:data/queue.db)\n --http-transport yes/no : HTTP transport (default: yes)\n --protected-mode yes/no : protected mode (default: yes)\n --nohup : do not exit on SIGHUP\n --spinlock : use a spinlock. For very write-heavy workloads\n\nDeveloper Options:\n --dev : enable developer mode\n --webhook-http-consumer-port port : Start a test HTTP webhook server\n --webhook-grpc-consumer-port port : Start a test GRPC webhook server\n\n`,\n\t\t)\n\t}\n\n\tif len(os.Args) == 3 && os.Args[1] == \"--webhook-http-consumer-port\" {\n\t\tlog.SetOutput(os.Stderr)\n\t\tport, err := strconv.ParseUint(os.Args[2], 10, 16)\n\t\tif err != nil {\n\t\t\tlog.Fatal(err)\n\t\t}\n\t\thttp.HandleFunc(\"/\", func(w http.ResponseWriter, r *http.Request) {\n\t\t\tdata, err := io.ReadAll(r.Body)\n\t\t\tif err != nil {\n\t\t\t\tlog.Fatal(err)\n\t\t\t}\n\t\t\tlog.HTTPf(\"http: %s : %s\", r.URL.Path, string(data))\n\t\t})\n\t\tlog.Infof(\"webhook server http://localhost:%d/\", port)\n\t\tif err := http.ListenAndServe(fmt.Sprintf(\":%d\", port), nil); err != nil {\n\t\t\tlog.Fatal(err)\n\t\t}\n\t\treturn\n\t}\n\n\tif len(os.Args) == 3 && os.Args[1] == \"--webhook-grpc-consumer-port\" {\n\t\tlog.SetOutput(os.Stderr)\n\t\tport, err := strconv.ParseUint(os.Args[2], 10, 16)\n\t\tif err != nil {\n\t\t\tlog.Fatal(err)\n\t\t}\n\n\t\tlis, err := net.Listen(\"tcp\", fmt.Sprintf(\":%d\", port))\n\t\tif err != nil {\n\t\t\tlog.Fatal(err)\n\t\t}\n\t\ts := grpc.NewServer()\n\t\thservice.RegisterHookServiceServer(s, &hserver{})\n\t\tlog.Infof(\"webhook server grpc://localhost:%d/\", port)\n\t\tif err := s.Serve(lis); err != nil {\n\t\t\tlog.Fatal(err)\n\t\t}\n\t\treturn\n\t}\n\n\tvar (\n\t\tnohup bool\n\t\tshowEvioDisabled bool\n\t\tshowThreadsDisabled bool\n\t\tspinlock bool\n\t)\n\n\tvar (\n\t\t// use to be in core/options.go\n\n\t\t// DevMode puts application in to dev mode\n\t\tdevMode = false\n\n\t\t// ShowDebugMessages allows for log.Debug to print to console.\n\t\tshowDebugMessages = false\n\n\t\t// ProtectedMode forces Tile38 to default in protected mode.\n\t\tprotectedMode = \"no\"\n\n\t\t// AppendOnly allows for disabling the appendonly file.\n\t\tappendOnly = true\n\n\t\t// AppendFileName allows for custom appendonly file path\n\t\tappendFileName = \"\"\n\n\t\t// QueueFileName allows for custom queue.db file path\n\t\tqueueFileName = \"\"\n\n\t\t// ClientOutput for auto assigning the output for client.\n\t\tclientOutput = \"\"\n\t)\n\n\t// parse non standard args.\n\tnargs := []string{os.Args[0]}\n\tfor i := 1; i < len(os.Args); i++ {\n\t\tswitch os.Args[i] {\n\t\tcase \"--help\":\n\t\t\toutput = os.Stdout\n\t\t\tflag.Usage()\n\t\t\treturn\n\t\tcase \"--version\":\n\t\t\tfmt.Fprintf(os.Stdout, \"%s\\n\", versionLine)\n\t\t\treturn\n\t\tcase \"--protected-mode\", \"-protected-mode\":\n\t\t\ti++\n\t\t\tif i < len(os.Args) {\n\t\t\t\tswitch strings.ToLower(os.Args[i]) {\n\t\t\t\tcase \"no\":\n\t\t\t\t\tprotectedMode = \"no\"\n\t\t\t\t\tcontinue\n\t\t\t\tcase \"yes\":\n\t\t\t\t\tprotectedMode = \"yes\"\n\t\t\t\t\tcontinue\n\t\t\t\t}\n\t\t\t}\n\t\t\tfmt.Fprintf(os.Stderr, \"protected-mode must be 'yes' or 'no'\\n\")\n\t\t\tos.Exit(1)\n\t\tcase \"--dev\", \"-dev\":\n\t\t\tdevMode = true\n\t\t\tcontinue\n\t\tcase \"--nohup\", \"-nohup\":\n\t\t\tnohup = true\n\t\t\tcontinue\n\t\tcase \"--spinlock\", \"-spinlock\":\n\t\t\tspinlock = true\n\t\t\tcontinue\n\t\tcase \"--appendonly\", \"-appendonly\":\n\t\t\ti++\n\t\t\tif i < len(os.Args) {\n\t\t\t\tswitch strings.ToLower(os.Args[i]) {\n\t\t\t\tcase \"no\":\n\t\t\t\t\tappendOnly = false\n\t\t\t\t\tcontinue\n\t\t\t\tcase \"yes\":\n\t\t\t\t\tappendOnly = true\n\t\t\t\t\tcontinue\n\t\t\t\t}\n\t\t\t}\n\t\t\tfmt.Fprintf(os.Stderr, \"appendonly must be 'yes' or 'no'\\n\")\n\t\t\tos.Exit(1)\n\t\tcase \"--appendfilename\", \"-appendfilename\":\n\t\t\ti++\n\t\t\tif i == len(os.Args) || os.Args[i] == \"\" {\n\t\t\t\tfmt.Fprintf(os.Stderr, \"appendfilename must have a value\\n\")\n\t\t\t\tos.Exit(1)\n\t\t\t}\n\t\t\tappendFileName = os.Args[i]\n\t\tcase \"--queuefilename\", \"-queuefilename\":\n\t\t\ti++\n\t\t\tif i == len(os.Args) || os.Args[i] == \"\" {\n\t\t\t\tfmt.Fprintf(os.Stderr, \"queuefilename must have a value\\n\")\n\t\t\t\tos.Exit(1)\n\t\t\t}\n\t\t\tqueueFileName = os.Args[i]\n\t\tcase \"-o\":\n\t\t\ti++\n\t\t\tif i < len(os.Args) {\n\t\t\t\tswitch strings.ToLower(os.Args[i]) {\n\t\t\t\tcase \"resp\", \"json\":\n\t\t\t\t\tclientOutput = strings.ToLower(os.Args[i])\n\t\t\t\t\tcontinue\n\t\t\t\t}\n\t\t\t}\n\t\t\tfmt.Fprintf(os.Stderr, \"output must be 'resp' or 'json'\\n\")\n\t\t\tos.Exit(1)\n\t\tcase \"--http-transport\", \"-http-transport\":\n\t\t\ti++\n\t\t\tif i < len(os.Args) {\n\t\t\t\tswitch strings.ToLower(os.Args[i]) {\n\t\t\t\tcase \"1\", \"true\", \"yes\":\n\t\t\t\t\thttpTransport = true\n\t\t\t\t\tcontinue\n\t\t\t\tcase \"0\", \"false\", \"no\":\n\t\t\t\t\thttpTransport = false\n\t\t\t\t\tcontinue\n\t\t\t\t}\n\t\t\t}\n\t\t\tfmt.Fprintf(os.Stderr, \"http-transport must be 'yes' or 'no'\\n\")\n\t\t\tos.Exit(1)\n\t\tcase \"--threads\", \"-threads\":\n\t\t\ti++\n\t\t\tif i < len(os.Args) {\n\t\t\t\t_, err := strconv.ParseUint(os.Args[i], 10, 16)\n\t\t\t\tif err != nil {\n\t\t\t\t\tfmt.Fprintf(os.Stderr, \"threads must be a valid number\\n\")\n\t\t\t\t\tos.Exit(1)\n\t\t\t\t}\n\t\t\t\tshowThreadsDisabled = true\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tfmt.Fprintf(os.Stderr, \"threads must be a valid number \\n\")\n\t\t\tos.Exit(1)\n\t\tcase \"--evio\", \"-evio\":\n\t\t\ti++\n\t\t\tif i < len(os.Args) {\n\t\t\t\tswitch strings.ToLower(os.Args[i]) {\n\t\t\t\tcase \"no\", \"yes\":\n\t\t\t\t\tshowEvioDisabled = true\n\t\t\t\t\tcontinue\n\t\t\t\t}\n\t\t\t}\n\t\t\tfmt.Fprintf(os.Stderr, \"evio must be 'yes' or 'no'\\n\")\n\t\t\tos.Exit(1)\n\t\tdefault:\n\t\t\tnargs = append(nargs, os.Args[i])\n\t\t}\n\t}\n\tos.Args = nargs\n\n\tmetricsAddr := flag.String(\"metrics-addr\", \"\", \"The listening addr for Prometheus metrics.\")\n\n\tvar (\n\t\tdir string\n\t\tport int\n\t\thost string\n\t\tunixSocket string\n\t\tverbose bool\n\t\tveryVerbose bool\n\t\tlogEncoding string\n\t\tquiet bool\n\t\tpidfile string\n\t\tcpuprofile string\n\t\tmemprofile string\n\t\tpprofport int\n\t)\n\n\tflag.IntVar(&port, \"p\", 9851, \"The listening port\")\n\tflag.StringVar(&pidfile, \"pidfile\", \"\", \"A file that contains the pid\")\n\tflag.StringVar(&host, \"h\", \"\", \"The listening host\")\n\tflag.StringVar(&unixSocket, \"s\", \"\", \"Listen on a unix socket\")\n\tflag.StringVar(&dir, \"d\", \"data\", \"The data directory\")\n\tflag.StringVar(&logEncoding, \"l\", \"text\", \"The log encoding json or text (default: text)\")\n\tflag.BoolVar(&verbose, \"v\", false, \"Enable verbose logging\")\n\tflag.BoolVar(&quiet, \"q\", false, \"Quiet logging. Totally silent\")\n\tflag.BoolVar(&veryVerbose, \"vv\", false, \"Enable very verbose logging\")\n\tflag.IntVar(&pprofport, \"pprofport\", 0, \"pprofport http at port\")\n\tflag.StringVar(&cpuprofile, \"cpuprofile\", \"\", \"write cpu profile to `file`\")\n\tflag.StringVar(&memprofile, \"memprofile\", \"\", \"write memory profile to `file`\")\n\tflag.Parse()\n\n\tif logEncoding == \"json\" {\n\t\tlog.SetLogJSON(true)\n\t\tdata, _ := os.ReadFile(filepath.Join(dir, \"config\"))\n\t\tif gjson.GetBytes(data, \"logconfig.encoding\").String() == \"json\" {\n\t\t\tc := gjson.GetBytes(data, \"logconfig\").String()\n\t\t\tlog.Build(c)\n\t\t} else {\n\t\t\tlog.Build(\"\")\n\t\t}\n\t}\n\n\tvar logw io.Writer = os.Stderr\n\tif quiet {\n\t\tlogw = io.Discard\n\t}\n\n\tlog.SetOutput(logw)\n\n\tif quiet {\n\t\tlog.SetLevel(0)\n\t} else if veryVerbose {\n\t\tlog.SetLevel(3)\n\t} else if verbose {\n\t\tlog.SetLevel(2)\n\t} else {\n\t\tlog.SetLevel(1)\n\t}\n\n\tshowDebugMessages = veryVerbose\n\n\thostd := \"\"\n\tif host != \"\" {\n\t\thostd = \"Addr: \" + host + \", \"\n\t}\n\n\t// pprof\n\tif cpuprofile != \"\" {\n\t\tlog.Debugf(\"cpuprofile active\")\n\t\tf, err := os.Create(cpuprofile)\n\t\tif err != nil {\n\t\t\tlog.Fatal(\"could not create CPU profile: \", err)\n\t\t}\n\t\tif err := pprof.StartCPUProfile(f); err != nil {\n\t\t\tlog.Fatal(\"could not start CPU profile: \", err)\n\t\t}\n\t}\n\tif memprofile != \"\" {\n\t\tlog.Debug(\"memprofile active\")\n\t}\n\n\tvar pprofcleanedup bool\n\tvar pprofcleanupMu sync.Mutex\n\tpprofcleanup := func() {\n\t\tpprofcleanupMu.Lock()\n\t\tdefer pprofcleanupMu.Unlock()\n\t\tif pprofcleanedup {\n\t\t\treturn\n\t\t}\n\t\t// cleanup code\n\t\tif cpuprofile != \"\" {\n\t\t\tpprof.StopCPUProfile()\n\t\t}\n\t\tif memprofile != \"\" {\n\t\t\tf, err := os.Create(memprofile)\n\t\t\tif err != nil {\n\t\t\t\tlog.Fatal(\"could not create memory profile: \", err)\n\t\t\t}\n\t\t\truntime.GC() // get up-to-date statistics\n\t\t\tif err := pprof.WriteHeapProfile(f); err != nil {\n\t\t\t\tlog.Fatal(\"could not write memory profile: \", err)\n\t\t\t}\n\t\t\tf.Close()\n\t\t}\n\t\tpprofcleanedup = true\n\t}\n\tdefer pprofcleanup()\n\n\tif pprofport != 0 {\n\t\tlog.Debugf(\"pprof http at port %d\", pprofport)\n\t\tgo func() {\n\t\t\tlog.Fatal(http.ListenAndServe(fmt.Sprintf(\":%d\", pprofport), nil))\n\t\t}()\n\t}\n\n\tif unixSocket != \"\" {\n\t\tport = 0\n\t}\n\n\t// pid file\n\tvar pidferr error\n\tvar pidcleanedup bool\n\tvar pidcleanupMu sync.Mutex\n\tpidcleanup := func() {\n\t\tif pidfile != \"\" {\n\t\t\tpidcleanupMu.Lock()\n\t\t\tdefer pidcleanupMu.Unlock()\n\t\t\tif pidcleanedup {\n\t\t\t\treturn\n\t\t\t}\n\t\t\t// cleanup code\n\t\t\tif pidfile != \"\" {\n\t\t\t\tos.Remove(pidfile)\n\t\t\t}\n\t\t\tpidcleanedup = true\n\t\t}\n\t}\n\tdefer pidcleanup()\n\tif pidfile != \"\" {\n\t\tos.WriteFile(pidfile, []byte(fmt.Sprintf(\"%d\\n\", os.Getpid())), 0666)\n\t}\n\n\tc := make(chan os.Signal, 1)\n\tshutdown := make(chan bool, 1)\n\n\tsignal.Notify(c, syscall.SIGHUP, syscall.SIGINT, syscall.SIGTERM, syscall.SIGQUIT)\n\tgo func() {\n\t\tfor s := range c {\n\t\t\tif s == syscall.SIGHUP && nohup {\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tlog.Warnf(\"signal: %v\", s)\n\t\t\tpidcleanup()\n\t\t\tpprofcleanup()\n\t\t\tswitch {\n\t\t\tdefault:\n\t\t\t\tos.Exit(-1)\n\t\t\tcase s == syscall.SIGHUP:\n\t\t\t\tos.Exit(1)\n\t\t\tcase s == syscall.SIGINT:\n\t\t\t\tos.Exit(2)\n\t\t\tcase s == syscall.SIGQUIT:\n\t\t\t\tos.Exit(3)\n\t\t\tcase s == syscall.SIGTERM:\n\t\t\t\tshutdown <- true\n\t\t\t}\n\t\t}\n\t}()\n\n\tvar saddr string\n\tif unixSocket != \"\" {\n\t\tsaddr = fmt.Sprintf(\"Socket: %s\", unixSocket)\n\t} else {\n\t\tsaddr = fmt.Sprintf(\"Port: %d\", port)\n\t}\n\n\tif log.LogJSON() {\n\t\tlog.Printf(`Tile38 %s%s %d bit (%s/%s) %s%s, PID: %d. Visit tile38.com/sponsor to support the project`,\n\t\t\tcore.Version, gitsha, strconv.IntSize, runtime.GOARCH, runtime.GOOS, hostd, saddr, os.Getpid())\n\t} else {\n\t\tfmt.Fprintf(logw, `\n _____ _ _ ___ ___\n |_ _|_| |___|_ | . | Tile38 %s%s %d bit (%s/%s)\n | | | | | -_|_ | . | %s%s, PID: %d\n |_| |_|_|___|___|___| tile38.com\n\n`, core.Version, gitsha, strconv.IntSize, runtime.GOARCH, runtime.GOOS, hostd,\n\t\t\tsaddr, os.Getpid())\n\t}\n\n\tif pidferr != nil {\n\t\tlog.Warnf(\"pidfile: %v\", pidferr)\n\t}\n\tif showEvioDisabled {\n\t\tlog.Warnf(\"evio is not currently supported\")\n\t}\n\tif showThreadsDisabled {\n\t\tlog.Warnf(\"thread flag is deprecated use GOMAXPROCS to set number of threads instead\")\n\t}\n\topts := server.Options{\n\t\tHost: host,\n\t\tPort: port,\n\t\tDir: dir,\n\t\tUseHTTP: httpTransport,\n\t\tMetricsAddr: *metricsAddr,\n\t\tUnixSocketPath: unixSocket,\n\t\tDevMode: devMode,\n\t\tShowDebugMessages: showDebugMessages,\n\t\tProtectedMode: protectedMode,\n\t\tAppendOnly: appendOnly,\n\t\tAppendFileName: appendFileName,\n\t\tQueueFileName: queueFileName,\n\t\tShutdown: shutdown,\n\t\tSpinlock: spinlock,\n\t\tClientOutput: clientOutput,\n\t}\n\tif err := server.Serve(opts); err != nil {\n\t\tlog.Fatal(err)\n\t}\n}\n" }, { "path": "core/commands.go", "content": "//go:build ignore\n\npackage core\n\nimport (\n\t\"encoding/json\"\n\t\"strings\"\n)\n\nconst (\n\tclear = \"\\x1b[0m\"\n\tbright = \"\\x1b[1m\"\n\tgray = \"\\x1b[90m\"\n\tyellow = \"\\x1b[33m\"\n)\n\n// Command represents a Tile38 command.\ntype Command struct {\n\tName string `json:\"-\"`\n\tSummary string `json:\"summary\"`\n\tComplexity string `json:\"complexity\"`\n\tArguments []Argument `json:\"arguments\"`\n\tSince string `json:\"since\"`\n\tGroup string `json:\"group\"`\n\tDevOnly bool `json:\"dev\"`\n}\n\n// String returns a string representation of the command.\nfunc (c Command) String() string {\n\tvar s = c.Name\n\tfor _, arg := range c.Arguments {\n\t\ts += \" \" + arg.String()\n\t}\n\treturn s\n}\n\n// TermOutput returns a string representation of the command suitable for displaying in a terminal.\nfunc (c Command) TermOutput(indent string) string {\n\tline := c.String()\n\tvar line1 string\n\tif strings.HasPrefix(line, c.Name) {\n\t\tline1 = bright + c.Name + clear + gray + line[len(c.Name):] + clear\n\t} else {\n\t\tline1 = bright + strings.Replace(c.String(), \" \", \" \"+clear+gray, 1) + clear\n\t}\n\tline2 := yellow + \"summary: \" + clear + c.Summary\n\t//line3 := yellow + \"since: \" + clear + c.Since\n\treturn indent + line1 + \"\\n\" + indent + line2 + \"\\n\" //+ indent + line3 + \"\\n\"\n}\n\n// EnumArg represents a enum arguments.\ntype EnumArg struct {\n\tName string `json:\"name\"`\n\tArguments []Argument `json:\"arguments\"`\n}\n\n// String returns a string representation of an EnumArg.\nfunc (a EnumArg) String() string {\n\tvar s = a.Name\n\tfor _, arg := range a.Arguments {\n\t\ts += \" \" + arg.String()\n\t}\n\treturn s\n}\n\n// Argument represents a command argument.\ntype Argument struct {\n\tCommand string `json:\"command\"`\n\tNameAny interface{} `json:\"name\"`\n\tTypeAny interface{} `json:\"type\"`\n\tOptional bool `json:\"optional\"`\n\tMultiple bool `json:\"multiple\"`\n\tVariadic bool `json:\"variadic\"`\n\tEnum []string `json:\"enum\"`\n\tEnumArgs []EnumArg `json:\"enumargs\"`\n}\n\n// String returns a string representation of an Argument.\nfunc (a Argument) String() string {\n\tvar s string\n\tif a.Command != \"\" {\n\t\ts += \" \" + a.Command\n\t}\n\tif len(a.EnumArgs) > 0 {\n\t\teargs := \"\"\n\t\tfor _, arg := range a.EnumArgs {\n\t\t\tv := arg.String()\n\t\t\tif strings.Contains(v, \" \") {\n\t\t\t\tv = \"(\" + v + \")\"\n\t\t\t}\n\t\t\teargs += v + \"|\"\n\t\t}\n\t\tif len(eargs) > 0 {\n\t\t\teargs = eargs[:len(eargs)-1]\n\t\t}\n\t\ts += \" \" + eargs\n\t} else if len(a.Enum) > 0 {\n\t\ts += \" \" + strings.Join(a.Enum, \"|\")\n\t} else {\n\t\tnames, _ := a.NameTypes()\n\t\tsubs := \"\"\n\t\tfor _, name := range names {\n\t\t\tsubs += \" \" + name\n\t\t}\n\t\tsubs = strings.TrimSpace(subs)\n\t\ts += \" \" + subs\n\t\tif a.Variadic {\n\t\t\tif len(names) == 0 {\n\t\t\t\ts += \" [\" + subs + \" ...]\"\n\t\t\t} else {\n\t\t\t\ts += \" [\" + names[len(names)-1] + \" ...]\"\n\t\t\t}\n\t\t}\n\t\tif a.Multiple {\n\t\t\ts += \" ...\"\n\t\t}\n\t}\n\ts = strings.TrimSpace(s)\n\tif a.Optional {\n\t\ts = \"[\" + s + \"]\"\n\t}\n\treturn s\n}\n\nfunc parseAnyStringArray(any interface{}) []string {\n\tif str, ok := any.(string); ok {\n\t\treturn []string{str}\n\t} else if any, ok := any.([]interface{}); ok {\n\t\tarr := []string{}\n\t\tfor _, any := range any {\n\t\t\tif str, ok := any.(string); ok {\n\t\t\t\tarr = append(arr, str)\n\t\t\t}\n\t\t}\n\t\treturn arr\n\t}\n\treturn []string{}\n}\n\n// NameTypes returns the types and names of an argument as separate arrays.\nfunc (a Argument) NameTypes() (names, types []string) {\n\tnames = parseAnyStringArray(a.NameAny)\n\ttypes = parseAnyStringArray(a.TypeAny)\n\tif len(types) > len(names) {\n\t\ttypes = types[:len(names)]\n\t} else {\n\t\tfor len(types) < len(names) {\n\t\t\ttypes = append(types, \"\")\n\t\t}\n\t}\n\treturn\n}\n\n// Commands is a map of all of the commands.\nvar Commands = func() map[string]Command {\n\tvar commands map[string]Command\n\tif err := json.Unmarshal([]byte(commandsJSON), &commands); err != nil {\n\t\tpanic(err.Error())\n\t}\n\tfor name, command := range commands {\n\t\tcommand.Name = strings.ToUpper(name)\n\t\tcommands[name] = command\n\t}\n\treturn commands\n}()\n\nvar commandsJSON = `{{.CommandsJSON}}`\n" }, { "path": "core/commands.json", "content": "{\n \"SET\": {\n \"summary\": \"Sets the value of an id\",\n \"complexity\": \"O(1)\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\"\n },\n {\n \"name\": \"id\",\n \"type\": \"string\"\n },\n {\n \"command\": \"FIELD\",\n \"name\": [\"name\", \"value\"],\n \"type\": [\"string\", \"double\"],\n \"optional\": true,\n \"multiple\": true\n },\n {\n \"command\": \"EX\",\n \"name\": [\"seconds\"],\n \"type\": [\"double\"],\n \"optional\": true,\n \"multiple\": false\n },\n {\n \"name\": \"type\",\n \"optional\": true,\n \"enumargs\": [\n {\n \"name\": \"NX\"\n },\n {\n \"name\": \"XX\"\n }\n ]\n },\n {\n \"name\": \"value\",\n \"enumargs\": [\n {\n \"name\": \"OBJECT\",\n \"arguments\": [\n {\n \"name\": \"geojson\",\n \"type\": \"geojson\"\n }\n ]\n },\n {\n \"name\": \"POINT\",\n \"arguments\": [\n {\n \"name\": \"lat\",\n \"type\": \"double\"\n },\n {\n \"name\": \"lon\",\n \"type\": \"double\"\n },\n {\n \"name\": \"z\",\n \"type\": \"double\",\n \"optional\": true\n }\n ]\n },\n {\n \"name\": \"BOUNDS\",\n \"arguments\": [\n {\n \"name\": \"minlat\",\n \"type\": \"double\"\n },\n {\n \"name\": \"minlon\",\n \"type\": \"double\"\n },\n {\n \"name\": \"maxlat\",\n \"type\": \"double\"\n },\n {\n \"name\": \"maxlon\",\n \"type\": \"double\"\n }\n ]\n },\n {\n \"name\": \"HASH\",\n \"arguments\": [\n {\n \"name\": \"geohash\",\n \"type\": \"geohash\"\n }\n ]\n },\n {\n \"name\": \"STRING\",\n \"arguments\": [\n {\n \"name\": \"value\",\n \"type\": \"string\"\n }\n ]\n }\n ]\n }\n ],\n \"since\": \"1.0.0\",\n \"group\": \"keys\"\n },\n \"EXPIRE\": {\n \"summary\": \"Set a timeout on an id\",\n \"complexity\": \"O(1)\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\"\n },\n {\n \"name\": \"id\",\n \"type\": \"string\"\n },\n {\n \"name\": \"seconds\",\n \"type\": \"double\"\n }\n ],\n \"since\": \"1.0.0\",\n \"group\": \"keys\"\n },\n \"TTL\": {\n \"summary\": \"Get a timeout on an id\",\n \"complexity\": \"O(1)\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\"\n },\n {\n \"name\": \"id\",\n \"type\": \"string\"\n }\n ],\n \"since\": \"1.0.0\",\n \"group\": \"keys\"\n },\n \"EXISTS\": {\n \"summary\": \"Checks to see if a id exists\",\n \"complexity\": \"O(1)\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\"\n },\n {\n \"name\": \"id\",\n \"type\": \"string\"\n }\n ],\n \"since\": \"1.33.0\",\n \"group\": \"keys\"\n },\n \"FEXISTS\": {\n \"summary\": \"Checks to see if a field exists on a id\",\n \"complexity\": \"O(1)\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\"\n },\n {\n \"name\": \"id\",\n \"type\": \"string\"\n },\n {\n \"name\": \"field\",\n \"type\": \"string\"\n }\n ],\n \"since\": \"1.33.0\",\n \"group\": \"keys\"\n },\n \"PERSIST\": {\n \"summary\": \"Remove the existing timeout on an id\",\n \"complexity\": \"O(1)\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\"\n },\n {\n \"name\": \"id\",\n \"type\": \"string\"\n }\n ],\n \"since\": \"1.0.0\",\n \"group\": \"keys\"\n },\n \"FSET\": {\n \"summary\": \"Set the value for one or more fields of an id\",\n \"complexity\": \"O(1)\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\"\n },\n {\n \"name\": \"id\",\n \"type\": \"string\"\n },\n {\n \"command\": \"XX\",\n \"name\": [],\n \"type\": [],\n \"optional\": true\n },\n {\n \"name\": [\"field\", \"value\"],\n \"type\": [\"string\", \"double\"]\n },\n {\n \"name\": [\"field\", \"value\"],\n \"type\": [\"string\", \"double\"],\n \"multiple\": true,\n \"optional\": true\n }\n ],\n \"since\": \"1.0.0\",\n \"group\": \"keys\"\n },\n \"FGET\": {\n \"summary\": \"Gets the value for the field of an id\",\n \"complexity\": \"O(1)\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\"\n },\n {\n \"name\": \"id\",\n \"type\": \"string\"\n },\n {\n \"name\": \"field\",\n \"type\": \"string\"\n }\n ],\n \"since\": \"1.33.0\",\n \"group\": \"keys\"\n },\n \"BOUNDS\": {\n \"summary\": \"Get the combined bounds of all the objects in a key\",\n \"complexity\": \"O(1)\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\"\n }\n ],\n \"since\": \"1.3.0\",\n \"group\": \"keys\"\n },\n \"GET\": {\n \"summary\": \"Get the object of an id\",\n \"complexity\": \"O(1)\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\"\n },\n {\n \"name\": \"id\",\n \"type\": \"string\"\n },\n {\n \"command\": \"WITHFIELDS\",\n \"name\": [],\n \"type\": [],\n \"optional\": true\n },\n {\n \"name\": \"type\",\n \"optional\": true,\n \"enumargs\": [\n {\n \"name\": \"OBJECT\"\n },\n {\n \"name\": \"POINT\"\n },\n {\n \"name\": \"BOUNDS\"\n },\n {\n \"name\": \"HASH\",\n \"arguments\": [\n {\n \"name\": \"geohash\",\n \"type\": \"geohash\"\n }\n ]\n }\n ]\n }\n ],\n \"since\": \"1.0.0\",\n \"group\": \"keys\"\n },\n \"DEL\": {\n \"summary\": \"Delete an id from a key\",\n \"complexity\": \"O(1)\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\"\n },\n {\n \"name\": \"id\",\n \"type\": \"string\"\n },\n {\n \"command\": \"ERRON404\",\n \"name\": [],\n \"type\": [],\n \"optional\": true\n }\n ],\n \"since\": \"1.0.0\",\n \"group\": \"keys\"\n },\n \"DROP\": {\n \"summary\": \"Remove a key from the database\",\n \"complexity\": \"O(1)\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\"\n }\n ],\n \"since\": \"1.0.0\",\n \"group\": \"keys\"\n },\n \"RENAME\": {\n \"summary\": \"Rename a key to be stored under a different name.\",\n \"complexity\": \"O(1)\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\"\n },\n {\n \"name\": \"newkey\",\n \"type\": \"string\"\n }\n ],\n \"since\": \"1.14.5\",\n \"group\": \"keys\"\n },\n \"RENAMENX\": {\n \"summary\": \"Rename a key to be stored under a different name, if a new key does not exist.\",\n \"complexity\": \"O(1)\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\"\n },\n {\n \"name\": \"newkey\",\n \"type\": \"string\"\n }\n ],\n \"since\": \"1.14.5\",\n \"group\": \"keys\"\n },\n \"KEYS\": {\n \"summary\": \"Finds all keys matching the given pattern\",\n \"complexity\": \"O(N) where N is the number of keys in the database\",\n \"arguments\": [\n {\n \"name\": \"pattern\",\n \"type\": \"pattern\"\n }\n ],\n \"since\": \"1.0.0\",\n \"group\": \"keys\"\n },\n \"STATS\": {\n \"summary\": \"Show stats for one or more keys\",\n \"complexity\": \"O(N) where N is the number of keys being requested\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\",\n \"variadic\": true\n }\n ],\n \"since\": \"1.0.0\",\n \"group\": \"keys\"\n },\n \"SEARCH\": {\n \"summary\": \"Search for string values in a key\",\n \"complexity\": \"O(N) where N is the number of values in the key\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\"\n },\n {\n \"command\": \"CURSOR\",\n \"name\": \"start\",\n \"type\": \"integer\",\n \"optional\": true\n },\n {\n \"command\": \"LIMIT\",\n \"name\": \"count\",\n \"type\": \"integer\",\n \"optional\": true\n },\n {\n \"command\": \"MATCH\",\n \"name\": \"pattern\",\n \"type\": \"pattern\",\n \"optional\": true\n },\n {\n \"name\": \"order\",\n \"optional\": true,\n \"enumargs\": [\n {\n \"name\": \"ASC\"\n },\n {\n \"name\": \"DESC\"\n }\n ]\n },\n {\n \"command\": \"WHERE\",\n \"name\": [\"field\", \"min\", \"max\"],\n \"type\": [\"string\", \"double\", \"double\"],\n \"optional\": true,\n \"multiple\": true\n },\n {\n \"command\": \"WHEREIN\",\n \"name\": [\"field\", \"count\", \"value\"],\n \"type\": [\"string\", \"integer\", \"double\"],\n \"optional\": true,\n \"multiple\": true,\n \"variadic\": true\n },\n {\n \"command\": \"WHEREEVAL\",\n \"name\": [\"script\", \"numargs\", \"arg\"],\n \"type\": [\"string\", \"integer\", \"string\"],\n \"optional\": true,\n \"multiple\": true,\n \"variadic\": true\n },\n {\n \"command\": \"WHEREEVALSHA\",\n \"name\": [\"sha1\", \"numargs\", \"arg\"],\n \"type\": [\"string\", \"integer\", \"string\"],\n \"optional\": true,\n \"multiple\": true,\n \"variadic\": true\n },\n {\n \"command\": \"NOFIELDS\",\n \"name\": [],\n \"type\": [],\n \"optional\": true\n },\n {\n \"name\": \"type\",\n \"optional\": true,\n \"enumargs\": [\n {\n \"name\": \"COUNT\"\n },\n {\n \"name\": \"IDS\"\n }\n ]\n }\n ],\n \"since\": \"1.4.2\",\n \"group\": \"search\"\n },\n \"SCAN\": {\n \"summary\": \"Incrementally iterate though a key\",\n \"complexity\": \"O(N) where N is the number of ids in the key\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\"\n },\n {\n \"command\": \"CURSOR\",\n \"name\": \"start\",\n \"type\": \"integer\",\n \"optional\": true\n },\n {\n \"command\": \"LIMIT\",\n \"name\": \"count\",\n \"type\": \"integer\",\n \"optional\": true\n },\n {\n \"command\": \"MATCH\",\n \"name\": \"pattern\",\n \"type\": \"pattern\",\n \"optional\": true\n },\n {\n \"name\": \"order\",\n \"optional\": true,\n \"enumargs\": [\n {\n \"name\": \"ASC\"\n },\n {\n \"name\": \"DESC\"\n }\n ]\n },\n {\n \"command\": \"WHERE\",\n \"name\": [\"field\", \"min\", \"max\"],\n \"type\": [\"string\", \"double\", \"double\"],\n \"optional\": true,\n \"multiple\": true\n },\n {\n \"command\": \"WHEREIN\",\n \"name\": [\"field\", \"count\", \"value\"],\n \"type\": [\"string\", \"integer\", \"double\"],\n \"optional\": true,\n \"multiple\": true,\n \"variadic\": true\n },\n {\n \"command\": \"WHEREEVAL\",\n \"name\": [\"script\", \"numargs\", \"arg\"],\n \"type\": [\"string\", \"integer\", \"string\"],\n \"optional\": true,\n \"multiple\": true,\n \"variadic\": true\n },\n {\n \"command\": \"WHEREEVALSHA\",\n \"name\": [\"sha1\", \"numargs\", \"arg\"],\n \"type\": [\"string\", \"integer\", \"string\"],\n \"optional\": true,\n \"multiple\": true,\n \"variadic\": true\n },\n {\n \"command\": \"NOFIELDS\",\n \"name\": [],\n \"type\": [],\n \"optional\": true\n },\n {\n \"name\": \"type\",\n \"optional\": true,\n \"enumargs\": [\n {\n \"name\": \"COUNT\"\n },\n {\n \"name\": \"IDS\"\n },\n {\n \"name\": \"OBJECTS\"\n },\n {\n \"name\": \"POINTS\"\n },\n {\n \"name\": \"BOUNDS\"\n },\n {\n \"name\": \"HASHES\",\n \"arguments\": [\n {\n \"name\": \"precision\",\n \"type\": \"integer\"\n }\n ]\n }\n ]\n }\n ],\n \"since\": \"1.0.0\",\n \"group\": \"search\"\n },\n \"NEARBY\": {\n \"summary\": \"Searches for ids that are nearby a point\",\n \"complexity\": \"O(log(N)) where N is the number of ids in the area\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\"\n },\n {\n \"command\": \"CURSOR\",\n \"name\": \"start\",\n \"type\": \"integer\",\n \"optional\": true\n },\n {\n \"command\": \"LIMIT\",\n \"name\": \"count\",\n \"type\": \"integer\",\n \"optional\": true\n },\n {\n \"command\": \"MATCH\",\n \"name\": \"pattern\",\n \"type\": \"pattern\",\n \"optional\": true\n },\n {\n \"command\": \"DISTANCE\",\n \"name\": [],\n \"type\": [],\n \"optional\": true\n },\n {\n \"command\": \"WHERE\",\n \"name\": [\"field\", \"min\", \"max\"],\n \"type\": [\"string\", \"double\", \"double\"],\n \"optional\": true,\n \"multiple\": true\n },\n {\n \"command\": \"WHEREIN\",\n \"name\": [\"field\", \"count\", \"value\"],\n \"type\": [\"string\", \"integer\", \"double\"],\n \"optional\": true,\n \"multiple\": true,\n \"variadic\": true\n },\n {\n \"command\": \"WHEREEVAL\",\n \"name\": [\"script\", \"numargs\", \"arg\"],\n \"type\": [\"string\", \"integer\", \"string\"],\n \"optional\": true,\n \"multiple\": true,\n \"variadic\": true\n },\n {\n \"command\": \"WHEREEVALSHA\",\n \"name\": [\"sha1\", \"numargs\", \"arg\"],\n \"type\": [\"string\", \"integer\", \"string\"],\n \"optional\": true,\n \"multiple\": true,\n \"variadic\": true\n },\n {\n \"command\": \"NOFIELDS\",\n \"name\": [],\n \"type\": [],\n \"optional\": true\n },\n {\n \"command\": \"FENCE\",\n \"name\": [],\n \"type\": [],\n \"optional\": true\n },\n {\n \"command\": \"DETECT\",\n \"name\": [\"what\"],\n \"type\": [\"string\"],\n \"optional\": true\n },\n {\n \"command\": \"COMMANDS\",\n \"name\": [\"which\"],\n \"type\": [\"string\"],\n \"optional\": true\n },\n {\n \"name\": \"type\",\n \"optional\": true,\n \"enumargs\": [\n {\n \"name\": \"COUNT\"\n },\n {\n \"name\": \"IDS\"\n },\n {\n \"name\": \"OBJECTS\"\n },\n {\n \"name\": \"POINTS\"\n },\n {\n \"name\": \"BOUNDS\"\n },\n {\n \"name\": \"HASHES\",\n \"arguments\": [\n {\n \"name\": \"precision\",\n \"type\": \"integer\"\n }\n ]\n }\n ]\n },\n {\n \"name\": \"area\",\n \"enumargs\": [\n {\n \"name\": \"POINT\",\n \"arguments\": [\n {\n \"name\": \"lat\",\n \"type\": \"double\"\n },\n {\n \"name\": \"lon\",\n \"type\": \"double\"\n },\n {\n \"name\": \"meters\",\n \"type\": \"double\"\n }\n ]\n },\n {\n \"name\": \"ROAM\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\"\n },\n {\n \"name\": \"pattern\",\n \"type\": \"pattern\"\n },\n {\n \"name\": \"meters\",\n \"type\": \"double\"\n }\n ]\n }\n ]\n }\n ],\n \"since\": \"1.0.0\",\n \"group\": \"search\"\n },\n \"WITHIN\": {\n \"summary\": \"Searches for ids that completely within the area\",\n \"complexity\": \"O(log(N)) where N is the number of ids in the area\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\"\n },\n {\n \"command\": \"CURSOR\",\n \"name\": \"start\",\n \"type\": \"integer\",\n \"optional\": true\n },\n {\n \"command\": \"LIMIT\",\n \"name\": \"count\",\n \"type\": \"integer\",\n \"optional\": true\n },\n {\n \"command\": \"MATCH\",\n \"name\": \"pattern\",\n \"type\": \"pattern\",\n \"optional\": true\n },\n {\n \"command\": \"WHERE\",\n \"name\": [\"field\", \"min\", \"max\"],\n \"type\": [\"string\", \"double\", \"double\"],\n \"optional\": true,\n \"multiple\": true\n },\n {\n \"command\": \"WHEREIN\",\n \"name\": [\"field\", \"count\", \"value\"],\n \"type\": [\"string\", \"integer\", \"double\"],\n \"optional\": true,\n \"multiple\": true,\n \"variadic\": true\n },\n {\n \"command\": \"WHEREEVAL\",\n \"name\": [\"script\", \"numargs\", \"arg\"],\n \"type\": [\"string\", \"integer\", \"string\"],\n \"optional\": true,\n \"multiple\": true,\n \"variadic\": true\n },\n {\n \"command\": \"WHEREEVALSHA\",\n \"name\": [\"sha1\", \"numargs\", \"arg\"],\n \"type\": [\"string\", \"integer\", \"string\"],\n \"optional\": true,\n \"multiple\": true,\n \"variadic\": true\n },\n {\n \"command\": \"NOFIELDS\",\n \"name\": [],\n \"type\": [],\n \"optional\": true\n },\n {\n \"command\": \"FENCE\",\n \"name\": [],\n \"type\": [],\n \"optional\": true\n },\n {\n \"command\": \"DETECT\",\n \"name\": [\"what\"],\n \"type\": [\"string\"],\n \"optional\": true\n },\n {\n \"command\": \"COMMANDS\",\n \"name\": [\"which\"],\n \"type\": [\"string\"],\n \"optional\": true\n },\n {\n \"name\": \"type\",\n \"optional\": true,\n \"enumargs\": [\n {\n \"name\": \"COUNT\"\n },\n {\n \"name\": \"IDS\"\n },\n {\n \"name\": \"OBJECTS\"\n },\n {\n \"name\": \"POINTS\"\n },\n {\n \"name\": \"BOUNDS\"\n },\n {\n \"name\": \"HASHES\",\n \"arguments\": [\n {\n \"name\": \"precision\",\n \"type\": \"integer\"\n }\n ]\n }\n ]\n },\n {\n \"name\": \"area\",\n \"enumargs\": [\n {\n \"name\": \"GET\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\"\n },\n {\n \"name\": \"id\",\n \"type\": \"string\"\n }\n ]\n },\n {\n \"name\": \"BOUNDS\",\n \"arguments\": [\n {\n \"name\": \"minlat\",\n \"type\": \"double\"\n },\n {\n \"name\": \"minlon\",\n \"type\": \"double\"\n },\n {\n \"name\": \"maxlat\",\n \"type\": \"double\"\n },\n {\n \"name\": \"maxlon\",\n \"type\": \"double\"\n }\n ]\n },\n {\n \"name\": \"OBJECT\",\n \"arguments\": [\n {\n \"name\": \"geojson\",\n \"type\": \"geojson\"\n }\n ]\n },\n {\n \"name\": \"CIRCLE\",\n \"arguments\": [\n {\n \"name\": \"lat\",\n \"type\": \"double\"\n },\n {\n \"name\": \"lon\",\n \"type\": \"double\"\n },\n {\n \"name\": \"meters\",\n \"type\": \"double\"\n }\n ]\n },\n {\n \"name\": \"TILE\",\n \"arguments\": [\n {\n \"name\": \"x\",\n \"type\": \"double\"\n },\n {\n \"name\": \"y\",\n \"type\": \"double\"\n },\n {\n \"name\": \"z\",\n \"type\": \"double\"\n }\n ]\n },\n {\n \"name\": \"QUADKEY\",\n \"arguments\": [\n {\n \"name\": \"quadkey\",\n \"type\": \"string\"\n }\n ]\n },\n {\n \"name\": \"HASH\",\n \"arguments\": [\n {\n \"name\": \"geohash\",\n \"type\": \"geohash\"\n }\n ]\n }, \n {\n \"name\": \"SECTOR\",\n \"arguments\": [\n {\n \"name\": \"lat\",\n \"type\": \"double\"\n },\n {\n \"name\": \"lon\",\n \"type\": \"double\"\n },\n {\n \"name\": \"radius\",\n \"type\": \"double\"\n },\n {\n \"name\": \"startBearing\",\n \"type\": \"double\"\n },\n {\n \"name\": \"endBearing\",\n \"type\": \"double\"\n }\n ]\n }\n ]\n }\n ],\n \"since\": \"1.0.0\",\n \"group\": \"search\"\n },\n \"INTERSECTS\": {\n \"summary\": \"Searches for ids that intersect an area\",\n \"complexity\": \"O(log(N)) where N is the number of ids in the area\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\"\n },\n {\n \"command\": \"CURSOR\",\n \"name\": \"start\",\n \"type\": \"integer\",\n \"optional\": true\n },\n {\n \"command\": \"LIMIT\",\n \"name\": \"count\",\n \"type\": \"integer\",\n \"optional\": true\n },\n {\n \"command\": \"MATCH\",\n \"name\": \"pattern\",\n \"type\": \"pattern\",\n \"optional\": true\n },\n {\n \"command\": \"WHERE\",\n \"name\": [\"field\", \"min\", \"max\"],\n \"type\": [\"string\", \"double\", \"double\"],\n \"optional\": true,\n \"multiple\": true\n },\n {\n \"command\": \"WHEREIN\",\n \"name\": [\"field\", \"count\", \"value\"],\n \"type\": [\"string\", \"integer\", \"double\"],\n \"optional\": true,\n \"multiple\": true,\n \"variadic\": true\n },\n {\n \"command\": \"WHEREEVAL\",\n \"name\": [\"script\", \"numargs\", \"arg\"],\n \"type\": [\"string\", \"integer\", \"string\"],\n \"optional\": true,\n \"multiple\": true,\n \"variadic\": true\n },\n {\n \"command\": \"WHEREEVALSHA\",\n \"name\": [\"sha1\", \"numargs\", \"arg\"],\n \"type\": [\"string\", \"integer\", \"string\"],\n \"optional\": true,\n \"multiple\": true,\n \"variadic\": true\n },\n {\n \"command\": \"CLIP\",\n \"name\": [],\n \"type\": [],\n \"optional\": true\n },\n {\n \"command\": \"NOFIELDS\",\n \"name\": [],\n \"type\": [],\n \"optional\": true\n },\n {\n \"command\": \"FENCE\",\n \"name\": [],\n \"type\": [],\n \"optional\": true\n },\n {\n \"command\": \"DETECT\",\n \"name\": [\"what\"],\n \"type\": [\"string\"],\n \"optional\": true\n },\n {\n \"command\": \"COMMANDS\",\n \"name\": [\"which\"],\n \"type\": [\"string\"],\n \"optional\": true\n },\n {\n \"name\": \"type\",\n \"optional\": true,\n \"enumargs\": [\n {\n \"name\": \"COUNT\"\n },\n {\n \"name\": \"IDS\"\n },\n {\n \"name\": \"OBJECTS\"\n },\n {\n \"name\": \"POINTS\"\n },\n {\n \"name\": \"BOUNDS\"\n },\n {\n \"name\": \"HASHES\",\n \"arguments\": [\n {\n \"name\": \"precision\",\n \"type\": \"integer\"\n }\n ]\n }\n ]\n },\n {\n \"name\": \"area\",\n \"enumargs\": [\n {\n \"name\": \"GET\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\"\n },\n {\n \"name\": \"id\",\n \"type\": \"string\"\n }\n ]\n },\n {\n \"name\": \"BOUNDS\",\n \"arguments\": [\n {\n \"name\": \"minlat\",\n \"type\": \"double\"\n },\n {\n \"name\": \"minlon\",\n \"type\": \"double\"\n },\n {\n \"name\": \"maxlat\",\n \"type\": \"double\"\n },\n {\n \"name\": \"maxlon\",\n \"type\": \"double\"\n }\n ]\n },\n {\n \"name\": \"OBJECT\",\n \"arguments\": [\n {\n \"name\": \"geojson\",\n \"type\": \"geojson\"\n }\n ]\n },\n {\n \"name\": \"CIRCLE\",\n \"arguments\": [\n {\n \"name\": \"lat\",\n \"type\": \"double\"\n },\n {\n \"name\": \"lon\",\n \"type\": \"double\"\n },\n {\n \"name\": \"meters\",\n \"type\": \"double\"\n }\n ]\n },\n {\n \"name\": \"TILE\",\n \"arguments\": [\n {\n \"name\": \"x\",\n \"type\": \"double\"\n },\n {\n \"name\": \"y\",\n \"type\": \"double\"\n },\n {\n \"name\": \"z\",\n \"type\": \"double\"\n }\n ]\n },\n {\n \"name\": \"QUADKEY\",\n \"arguments\": [\n {\n \"name\": \"quadkey\",\n \"type\": \"string\"\n }\n ]\n },\n {\n \"name\": \"HASH\",\n \"arguments\": [\n {\n \"name\": \"geohash\",\n \"type\": \"geohash\"\n }\n ]\n },\n {\n \"name\": \"SECTOR\",\n \"arguments\": [\n {\n \"name\": \"lat\",\n \"type\": \"double\"\n },\n {\n \"name\": \"lon\",\n \"type\": \"double\"\n },\n {\n \"name\": \"radius\",\n \"type\": \"double\"\n },\n {\n \"name\": \"startBearing\",\n \"type\": \"double\"\n },\n {\n \"name\": \"endBearing\",\n \"type\": \"double\"\n }\n ]\n }\n ]\n }\n ],\n \"since\": \"1.0.0\",\n \"group\": \"search\"\n },\n \"CONFIG GET\": {\n \"summary\": \"Get the value of a configuration parameter\",\n \"arguments\": [\n {\n \"name\": \"parameter\",\n \"type\": \"string\"\n }\n ],\n \"group\": \"server\"\n },\n \"CONFIG SET\": {\n \"summary\": \"Set a configuration parameter to the given value\",\n \"arguments\": [\n {\n \"name\": \"parameter\",\n \"type\": \"string\"\n },\n {\n \"name\": \"value\",\n \"type\": \"string\",\n \"optional\": true\n }\n ],\n \"group\": \"server\"\n },\n \"CONFIG REWRITE\": {\n \"summary\": \"Rewrite the configuration file with the in memory configuration\",\n \"arguments\": [],\n \"group\": \"server\"\n },\n \"SERVER\": {\n \"summary\": \"Show server stats and details\",\n \"complexity\": \"O(1)\",\n \"arguments\": [],\n \"since\": \"1.0.0\",\n \"group\": \"server\"\n },\n \"GC\": {\n \"summary\": \"Forces a garbage collection\",\n \"complexity\": \"O(1)\",\n \"arguments\": [],\n \"since\": \"1.0.0\",\n \"group\": \"server\"\n },\n \"READONLY\": {\n \"summary\": \"Turns on or off readonly mode\",\n \"complexity\": \"O(1)\",\n \"arguments\": [\n {\n \"enum\": [\"yes\", \"no\"]\n }\n ],\n \"since\": \"1.0.0\",\n \"group\": \"server\"\n },\n \"FLUSHDB\": {\n \"summary\": \"Removes all keys\",\n \"complexity\": \"O(1)\",\n \"arguments\": [],\n \"since\": \"1.0.0\",\n \"group\": \"server\"\n },\n \"FOLLOW\": {\n \"summary\": \"Follows a leader host\",\n \"complexity\": \"O(1)\",\n \"arguments\": [\n {\n \"name\": \"host\",\n \"type\": \"string\"\n },\n {\n \"name\": \"port\",\n \"type\": \"integer\"\n }\n ],\n \"since\": \"1.0.0\",\n \"group\": \"replication\"\n },\n \"AOF\": {\n \"summary\": \"Downloads the AOF starting from pos and keeps the connection alive\",\n \"complexity\": \"O(1)\",\n \"arguments\": [\n {\n \"name\": \"pos\",\n \"type\": \"integer\"\n }\n ],\n \"since\": \"1.0.0\",\n \"group\": \"replication\"\n },\n \"AOFMD5\": {\n \"summary\": \"Performs a checksum on a portion of the aof\",\n \"complexity\": \"O(1)\",\n \"arguments\": [\n {\n \"name\": \"pos\",\n \"type\": \"integer\"\n },\n {\n \"name\": \"size\",\n \"type\": \"integer\"\n }\n ],\n \"since\": \"1.0.0\",\n \"group\": \"replication\"\n },\n \"AOFSHRINK\": {\n \"summary\": \"Shrinks the aof in the background\",\n \"group\": \"replication\"\n },\n \"PING\": {\n \"summary\": \"Ping the server\",\n \"group\": \"connection\"\n },\n \"QUIT\": {\n \"summary\": \"Close the connection\",\n \"group\": \"connection\"\n },\n \"AUTH\": {\n \"summary\": \"Authenticate to the server\",\n \"arguments\": [\n {\n \"name\": \"password\",\n \"type\": \"string\"\n }\n ],\n \"group\": \"connection\"\n },\n \"OUTPUT\": {\n \"summary\": \"Gets or sets the output format for the current connection.\",\n \"arguments\": [\n {\n \"name\": \"format\",\n \"optional\": true,\n \"enumargs\": [\n {\n \"name\": \"json\"\n },\n {\n \"name\": \"resp\"\n }\n ]\n }\n ],\n \"group\": \"connection\"\n },\n \"TIMEOUT\": {\n \"summary\": \"Runs the following command with the timeout\",\n \"arguments\": [\n {\n \"name\": \"seconds\",\n \"type\": \"double\"\n },\n {\n \"name\": \"COMMAND\",\n \"type\": \"string\"\n },\n {\n \"command\": \"arg\",\n \"type\": \"string\",\n \"multiple\": true,\n \"optional\": true\n }\n ],\n \"group\": \"connection\"\n },\n \"SETHOOK\": {\n \"summary\": \"Creates a webhook which points to geofenced search\",\n \"arguments\": [\n {\n \"name\": \"name\",\n \"type\": \"string\"\n },\n {\n \"name\": \"endpoint\",\n \"type\": \"string\"\n },\n {\n \"command\": \"META\",\n \"name\": [\"name\", \"value\"],\n \"type\": [\"string\", \"string\"],\n \"optional\": true,\n \"multiple\": true\n },\n {\n \"command\": \"EX\",\n \"name\": [\"seconds\"],\n \"type\": [\"double\"],\n \"optional\": true,\n \"multiple\": false\n },\n {\n \"enum\": [\"NEARBY\", \"WITHIN\", \"INTERSECTS\"]\n },\n {\n \"name\": \"key\",\n \"type\": \"string\"\n },\n {\n \"command\": \"FENCE\",\n \"name\": [],\n \"type\": []\n },\n {\n \"command\": \"DETECT\",\n \"name\": [\"what\"],\n \"type\": [\"string\"],\n \"optional\": true\n },\n {\n \"command\": \"COMMANDS\",\n \"name\": [\"which\"],\n \"type\": [\"string\"],\n \"optional\": true\n },\n {\n \"name\": \"param\",\n \"type\": \"string\",\n \"variadic\": true\n }\n ],\n \"group\": \"webhook\"\n },\n \"DELHOOK\": {\n \"summary\": \"Removes a webhook\",\n \"arguments\": [\n {\n \"name\": \"name\",\n \"type\": \"string\"\n }\n ],\n \"group\": \"webhook\"\n },\n \"HOOKS\": {\n \"summary\": \"Finds all hooks matching a pattern\",\n \"arguments\": [\n {\n \"name\": \"pattern\",\n \"type\": \"pattern\"\n }\n ],\n \"group\": \"webhook\"\n },\n \"PDELHOOK\": {\n \"summary\": \"Removes all hooks matching a pattern\",\n \"arguments\": [\n {\n \"name\": \"pattern\",\n \"type\": \"pattern\"\n }\n ],\n \"group\": \"webhook\"\n },\n\n \"SETCHAN\": {\n \"summary\": \"Creates a pubsub channel which points to geofenced search\",\n \"arguments\": [\n {\n \"name\": \"name\",\n \"type\": \"string\"\n },\n {\n \"command\": \"META\",\n \"name\": [\"name\", \"value\"],\n \"type\": [\"string\", \"string\"],\n \"optional\": true,\n \"multiple\": true\n },\n {\n \"command\": \"EX\",\n \"name\": [\"seconds\"],\n \"type\": [\"double\"],\n \"optional\": true,\n \"multiple\": false\n },\n {\n \"enum\": [\"NEARBY\", \"WITHIN\", \"INTERSECTS\"]\n },\n {\n \"name\": \"key\",\n \"type\": \"string\"\n },\n {\n \"command\": \"FENCE\",\n \"name\": [],\n \"type\": []\n },\n {\n \"command\": \"DETECT\",\n \"name\": [\"what\"],\n \"type\": [\"string\"],\n \"optional\": true\n },\n {\n \"command\": \"COMMANDS\",\n \"name\": [\"which\"],\n \"type\": [\"string\"],\n \"optional\": true\n },\n {\n \"name\": \"param\",\n \"type\": \"string\",\n \"variadic\": true\n }\n ],\n \"group\": \"pubsub\"\n },\n \"DELCHAN\": {\n \"summary\": \"Removes a channel\",\n \"arguments\": [\n {\n \"name\": \"name\",\n \"type\": \"string\"\n }\n ],\n \"group\": \"pubsub\"\n },\n \"CHANS\": {\n \"summary\": \"Finds all channels matching a pattern\",\n \"arguments\": [\n {\n \"name\": \"pattern\",\n \"type\": \"pattern\"\n }\n ],\n \"group\": \"pubsub\"\n },\n \"PDELCHAN\": {\n \"summary\": \"Removes all channels matching a pattern\",\n \"arguments\": [\n {\n \"name\": \"pattern\",\n \"type\": \"pattern\"\n }\n ],\n \"group\": \"pubsub\"\n },\n \"SUBSCRIBE\": {\n \"summary\": \"Subscribe to a geofence channel\",\n \"arguments\": [\n {\n \"name\": \"channel\",\n \"type\": \"string\",\n \"variadic\": true\n }\n ],\n \"group\": \"pubsub\"\n },\n \"PSUBSCRIBE\": {\n \"summary\": \"Subscribes the client to the given patterns\",\n \"arguments\": [\n {\n \"name\": \"pattern\",\n \"type\": \"pattern\",\n \"variadic\": true\n }\n ],\n \"group\": \"pubsub\"\n },\n \"PDEL\": {\n \"summary\": \"Removes all objects matching a pattern\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\"\n },\n {\n \"name\": \"pattern\",\n \"type\": \"pattern\"\n }\n ],\n \"group\": \"keys\"\n },\n \"JGET\": {\n \"summary\": \"Get a value from a JSON document\",\n \"complexity\": \"O(1)\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\"\n },\n {\n \"name\": \"id\",\n \"type\": \"string\"\n },\n {\n \"name\": \"path\",\n \"type\": \"string\"\n },\n {\n \"command\": \"RAW\",\n \"name\": [],\n \"type\": [],\n \"optional\": true\n }\n ],\n \"group\": \"keys\"\n },\n \"JSET\": {\n \"summary\": \"Set a value in a JSON document\",\n \"complexity\": \"O(1)\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\"\n },\n {\n \"name\": \"id\",\n \"type\": \"string\"\n },\n {\n \"name\": \"path\",\n \"type\": \"string\"\n },\n {\n \"name\": \"value\",\n \"type\": \"string\"\n },\n {\n \"name\": [],\n \"optional\": true,\n \"enumargs\": [\n {\n \"name\": \"RAW\"\n },\n {\n \"name\": \"STR\"\n }\n ]\n }\n ],\n \"group\": \"keys\"\n },\n \"JDEL\": {\n \"summary\": \"Delete a value from a JSON document\",\n \"complexity\": \"O(1)\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\"\n },\n {\n \"name\": \"id\",\n \"type\": \"string\"\n },\n {\n \"name\": \"path\",\n \"type\": \"string\"\n }\n ],\n \"group\": \"keys\"\n },\n \"EVAL\": {\n \"summary\": \"Evaluates a Lua script\",\n \"complexity\": \"Depends on the evaluated script\",\n \"arguments\": [\n {\n \"name\": \"script\",\n \"type\": \"string\"\n },\n {\n \"name\": \"numkeys\",\n \"type\": \"integer\"\n },\n {\n \"name\": \"key\",\n \"type\": \"string\",\n \"optional\": true,\n \"multiple\": true\n },\n {\n \"name\": \"arg\",\n \"type\": \"string\",\n \"optional\": true,\n \"multiple\": true\n }\n ],\n \"since\": \"1.10.0\",\n \"group\": \"scripting\"\n },\n \"EVALSHA\": {\n \"summary\": \"Evaluates a Lua script cached on the server by its SHA1 digest\",\n \"complexity\": \"Depends on the evaluated script\",\n \"arguments\": [\n {\n \"name\": \"sha1\",\n \"type\": \"string\"\n },\n {\n \"name\": \"numkeys\",\n \"type\": \"integer\"\n },\n {\n \"name\": \"key\",\n \"type\": \"string\",\n \"optional\": true,\n \"multiple\": true\n },\n {\n \"name\": \"arg\",\n \"type\": \"string\",\n \"optional\": true,\n \"multiple\": true\n }\n ],\n \"since\": \"1.10.0\",\n \"group\": \"scripting\"\n },\n \"EVALRO\": {\n \"summary\": \"Evaluates a read-only Lua script\",\n \"complexity\": \"Depends on the evaluated script\",\n \"arguments\": [\n {\n \"name\": \"script\",\n \"type\": \"string\"\n },\n {\n \"name\": \"numkeys\",\n \"type\": \"integer\"\n },\n {\n \"name\": \"key\",\n \"type\": \"string\",\n \"optional\": true,\n \"multiple\": true\n },\n {\n \"name\": \"arg\",\n \"type\": \"string\",\n \"optional\": true,\n \"multiple\": true\n }\n ],\n \"since\": \"1.10.0\",\n \"group\": \"scripting\"\n },\n \"EVALROSHA\": {\n \"summary\": \"Evaluates a read-only Lua script cached on the server by its SHA1 digest\",\n \"complexity\": \"Depends on the evaluated script\",\n \"arguments\": [\n {\n \"name\": \"script\",\n \"type\": \"string\"\n },\n {\n \"name\": \"numkeys\",\n \"type\": \"integer\"\n },\n {\n \"name\": \"key\",\n \"type\": \"string\",\n \"optional\": true,\n \"multiple\": true\n },\n {\n \"name\": \"arg\",\n \"type\": \"string\",\n \"optional\": true,\n \"multiple\": true\n }\n ],\n \"since\": \"1.10.0\",\n \"group\": \"scripting\"\n },\n \"EVALNA\": {\n \"summary\": \"Evaluates a Lua script in a non-atomic fashion\",\n \"complexity\": \"Depends on the evaluated script\",\n \"arguments\": [\n {\n \"name\": \"script\",\n \"type\": \"string\"\n },\n {\n \"name\": \"numkeys\",\n \"type\": \"integer\"\n },\n {\n \"name\": \"key\",\n \"type\": \"string\",\n \"optional\": true,\n \"multiple\": true\n },\n {\n \"name\": \"arg\",\n \"type\": \"string\",\n \"optional\": true,\n \"multiple\": true\n }\n ],\n \"since\": \"1.10.0\",\n \"group\": \"scripting\"\n },\n \"EVALNASHA\": {\n \"summary\": \"Evaluates, in a non-atomic fashion, a Lua script cached on the server by its SHA1 digest\",\n \"complexity\": \"Depends on the evaluated script\",\n \"arguments\": [\n {\n \"name\": \"sha1\",\n \"type\": \"string\"\n },\n {\n \"name\": \"numkeys\",\n \"type\": \"integer\"\n },\n {\n \"name\": \"key\",\n \"type\": \"string\",\n \"optional\": true,\n \"multiple\": true\n },\n {\n \"name\": \"arg\",\n \"type\": \"string\",\n \"optional\": true,\n \"multiple\": true\n }\n ],\n \"since\": \"1.10.0\",\n \"group\": \"scripting\"\n },\n \"SCRIPT EXISTS\": {\n \"summary\": \"Returns information about the existence of the scripts in server cache\",\n \"complexity\": \"O(N) where N is the number of provided sha1 arguments\",\n \"arguments\": [\n {\n \"name\": \"sha1\",\n \"type\": \"string\",\n \"multiple\": true\n }\n ],\n \"since\": \"1.10.0\",\n \"group\": \"scripting\"\n },\n \"SCRIPT LOAD\": {\n \"summary\": \"Loads the compiled version of a script into the server cache, without executing\",\n \"complexity\": \"O(N) where N is the number of bytes in the script\",\n \"arguments\": [\n {\n \"name\": \"script\",\n \"type\": \"string\"\n }\n ],\n \"since\": \"1.10.0\",\n \"group\": \"scripting\"\n },\n \"SCRIPT FLUSH\": {\n \"summary\": \"Flushes the server cache of Lua scripts\",\n \"complexity\": \"O(1)\",\n \"since\": \"1.10.0\",\n \"group\": \"scripting\"\n },\n \"TEST\": {\n \"summary\": \"Performs spatial test\",\n \"complexity\": \"One test per command, complexity depends on the test\",\n \"arguments\": [\n {\n \"name\": \"area1\",\n \"enumargs\": [\n {\n \"name\": \"POINT\",\n \"arguments\": [\n {\n \"name\": \"lat\",\n \"type\": \"double\"\n },\n {\n \"name\": \"lon\",\n \"type\": \"double\"\n }\n ]\n },\n {\n \"name\": \"GET\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\"\n },\n {\n \"name\": \"id\",\n \"type\": \"string\"\n }\n ]\n },\n {\n \"name\": \"BOUNDS\",\n \"arguments\": [\n {\n \"name\": \"minlat\",\n \"type\": \"double\"\n },\n {\n \"name\": \"minlon\",\n \"type\": \"double\"\n },\n {\n \"name\": \"maxlat\",\n \"type\": \"double\"\n },\n {\n \"name\": \"maxlon\",\n \"type\": \"double\"\n }\n ]\n },\n {\n \"name\": \"OBJECT\",\n \"arguments\": [\n {\n \"name\": \"geojson\",\n \"type\": \"geojson\"\n }\n ]\n },\n {\n \"name\": \"CIRCLE\",\n \"arguments\": [\n {\n \"name\": \"lat\",\n \"type\": \"double\"\n },\n {\n \"name\": \"lon\",\n \"type\": \"double\"\n },\n {\n \"name\": \"meters\",\n \"type\": \"double\"\n }\n ]\n },\n {\n \"name\": \"TILE\",\n \"arguments\": [\n {\n \"name\": \"x\",\n \"type\": \"double\"\n },\n {\n \"name\": \"y\",\n \"type\": \"double\"\n },\n {\n \"name\": \"z\",\n \"type\": \"double\"\n }\n ]\n },\n {\n \"name\": \"QUADKEY\",\n \"arguments\": [\n {\n \"name\": \"quadkey\",\n \"type\": \"string\"\n }\n ]\n },\n {\n \"name\": \"HASH\",\n \"arguments\": [\n {\n \"name\": \"geohash\",\n \"type\": \"geohash\"\n }\n ]\n }\n ]\n },\n {\n \"name\": \"test\",\n \"enumargs\": [\n {\n \"name\": \"INTERSECTS\"\n },\n {\n \"name\": \"WITHIN\"\n }\n ]\n },\n {\n \"command\": \"CLIP\",\n \"name\": [],\n \"type\": [],\n \"optional\": true\n },\n {\n \"name\": \"area2\",\n \"enumargs\": [\n {\n \"name\": \"POINT\",\n \"arguments\": [\n {\n \"name\": \"lat\",\n \"type\": \"double\"\n },\n {\n \"name\": \"lon\",\n \"type\": \"double\"\n }\n ]\n },\n {\n \"name\": \"GET\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\"\n },\n {\n \"name\": \"id\",\n \"type\": \"string\"\n }\n ]\n },\n {\n \"name\": \"BOUNDS\",\n \"arguments\": [\n {\n \"name\": \"minlat\",\n \"type\": \"double\"\n },\n {\n \"name\": \"minlon\",\n \"type\": \"double\"\n },\n {\n \"name\": \"maxlat\",\n \"type\": \"double\"\n },\n {\n \"name\": \"maxlon\",\n \"type\": \"double\"\n }\n ]\n },\n {\n \"name\": \"OBJECT\",\n \"arguments\": [\n {\n \"name\": \"geojson\",\n \"type\": \"geojson\"\n }\n ]\n },\n {\n \"name\": \"CIRCLE\",\n \"arguments\": [\n {\n \"name\": \"lat\",\n \"type\": \"double\"\n },\n {\n \"name\": \"lon\",\n \"type\": \"double\"\n },\n {\n \"name\": \"meters\",\n \"type\": \"double\"\n }\n ]\n },\n {\n \"name\": \"TILE\",\n \"arguments\": [\n {\n \"name\": \"x\",\n \"type\": \"double\"\n },\n {\n \"name\": \"y\",\n \"type\": \"double\"\n },\n {\n \"name\": \"z\",\n \"type\": \"double\"\n }\n ]\n },\n {\n \"name\": \"QUADKEY\",\n \"arguments\": [\n {\n \"name\": \"quadkey\",\n \"type\": \"string\"\n }\n ]\n },\n {\n \"name\": \"HASH\",\n \"arguments\": [\n {\n \"name\": \"geohash\",\n \"type\": \"geohash\"\n }\n ]\n }\n ]\n }\n ],\n \"since\": \"1.16.0\",\n \"group\": \"tests\"\n }\n}\n" }, { "path": "core/commands_gen.go", "content": "// This file was autogenerated. DO NOT EDIT.\n\npackage core\n\nimport (\n\t\"encoding/json\"\n\t\"strings\"\n)\n\nconst (\n\tclear = \"\\x1b[0m\"\n\tbright = \"\\x1b[1m\"\n\tgray = \"\\x1b[90m\"\n\tyellow = \"\\x1b[33m\"\n)\n\n// Command represents a Tile38 command.\ntype Command struct {\n\tName string `json:\"-\"`\n\tSummary string `json:\"summary\"`\n\tComplexity string `json:\"complexity\"`\n\tArguments []Argument `json:\"arguments\"`\n\tSince string `json:\"since\"`\n\tGroup string `json:\"group\"`\n\tDevOnly bool `json:\"dev\"`\n}\n\n// String returns a string representation of the command.\nfunc (c Command) String() string {\n\tvar s = c.Name\n\tfor _, arg := range c.Arguments {\n\t\ts += \" \" + arg.String()\n\t}\n\treturn s\n}\n\n// TermOutput returns a string representation of the command suitable for displaying in a terminal.\nfunc (c Command) TermOutput(indent string) string {\n\tline := c.String()\n\tvar line1 string\n\tif strings.HasPrefix(line, c.Name) {\n\t\tline1 = bright + c.Name + clear + gray + line[len(c.Name):] + clear\n\t} else {\n\t\tline1 = bright + strings.Replace(c.String(), \" \", \" \"+clear+gray, 1) + clear\n\t}\n\tline2 := yellow + \"summary: \" + clear + c.Summary\n\t//line3 := yellow + \"since: \" + clear + c.Since\n\treturn indent + line1 + \"\\n\" + indent + line2 + \"\\n\" //+ indent + line3 + \"\\n\"\n}\n\n// EnumArg represents a enum arguments.\ntype EnumArg struct {\n\tName string `json:\"name\"`\n\tArguments []Argument `json:\"arguments\"`\n}\n\n// String returns a string representation of an EnumArg.\nfunc (a EnumArg) String() string {\n\tvar s = a.Name\n\tfor _, arg := range a.Arguments {\n\t\ts += \" \" + arg.String()\n\t}\n\treturn s\n}\n\n// Argument represents a command argument.\ntype Argument struct {\n\tCommand string `json:\"command\"`\n\tNameAny interface{} `json:\"name\"`\n\tTypeAny interface{} `json:\"type\"`\n\tOptional bool `json:\"optional\"`\n\tMultiple bool `json:\"multiple\"`\n\tVariadic bool `json:\"variadic\"`\n\tEnum []string `json:\"enum\"`\n\tEnumArgs []EnumArg `json:\"enumargs\"`\n}\n\n// String returns a string representation of an Argument.\nfunc (a Argument) String() string {\n\tvar s string\n\tif a.Command != \"\" {\n\t\ts += \" \" + a.Command\n\t}\n\tif len(a.EnumArgs) > 0 {\n\t\teargs := \"\"\n\t\tfor _, arg := range a.EnumArgs {\n\t\t\tv := arg.String()\n\t\t\tif strings.Contains(v, \" \") {\n\t\t\t\tv = \"(\" + v + \")\"\n\t\t\t}\n\t\t\teargs += v + \"|\"\n\t\t}\n\t\tif len(eargs) > 0 {\n\t\t\teargs = eargs[:len(eargs)-1]\n\t\t}\n\t\ts += \" \" + eargs\n\t} else if len(a.Enum) > 0 {\n\t\ts += \" \" + strings.Join(a.Enum, \"|\")\n\t} else {\n\t\tnames, _ := a.NameTypes()\n\t\tsubs := \"\"\n\t\tfor _, name := range names {\n\t\t\tsubs += \" \" + name\n\t\t}\n\t\tsubs = strings.TrimSpace(subs)\n\t\ts += \" \" + subs\n\t\tif a.Variadic {\n\t\t\tif len(names) == 0 {\n\t\t\t\ts += \" [\" + subs + \" ...]\"\n\t\t\t} else {\n\t\t\t\ts += \" [\" + names[len(names)-1] + \" ...]\"\n\t\t\t}\n\t\t}\n\t\tif a.Multiple {\n\t\t\ts += \" ...\"\n\t\t}\n\t}\n\ts = strings.TrimSpace(s)\n\tif a.Optional {\n\t\ts = \"[\" + s + \"]\"\n\t}\n\treturn s\n}\n\nfunc parseAnyStringArray(any interface{}) []string {\n\tif str, ok := any.(string); ok {\n\t\treturn []string{str}\n\t} else if any, ok := any.([]interface{}); ok {\n\t\tarr := []string{}\n\t\tfor _, any := range any {\n\t\t\tif str, ok := any.(string); ok {\n\t\t\t\tarr = append(arr, str)\n\t\t\t}\n\t\t}\n\t\treturn arr\n\t}\n\treturn []string{}\n}\n\n// NameTypes returns the types and names of an argument as separate arrays.\nfunc (a Argument) NameTypes() (names, types []string) {\n\tnames = parseAnyStringArray(a.NameAny)\n\ttypes = parseAnyStringArray(a.TypeAny)\n\tif len(types) > len(names) {\n\t\ttypes = types[:len(names)]\n\t} else {\n\t\tfor len(types) < len(names) {\n\t\t\ttypes = append(types, \"\")\n\t\t}\n\t}\n\treturn\n}\n\n// Commands is a map of all of the commands.\nvar Commands = func() map[string]Command {\n\tvar commands map[string]Command\n\tif err := json.Unmarshal([]byte(commandsJSON), &commands); err != nil {\n\t\tpanic(err.Error())\n\t}\n\tfor name, command := range commands {\n\t\tcommand.Name = strings.ToUpper(name)\n\t\tcommands[name] = command\n\t}\n\treturn commands\n}()\n\nvar commandsJSON = `{\n \"SET\": {\n \"summary\": \"Sets the value of an id\",\n \"complexity\": \"O(1)\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\"\n },\n {\n \"name\": \"id\",\n \"type\": \"string\"\n },\n {\n \"command\": \"FIELD\",\n \"name\": [\"name\", \"value\"],\n \"type\": [\"string\", \"double\"],\n \"optional\": true,\n \"multiple\": true\n },\n {\n \"command\": \"EX\",\n \"name\": [\"seconds\"],\n \"type\": [\"double\"],\n \"optional\": true,\n \"multiple\": false\n },\n {\n \"name\": \"type\",\n \"optional\": true,\n \"enumargs\": [\n {\n \"name\": \"NX\"\n },\n {\n \"name\": \"XX\"\n }\n ]\n },\n {\n \"name\": \"value\",\n \"enumargs\": [\n {\n \"name\": \"OBJECT\",\n \"arguments\": [\n {\n \"name\": \"geojson\",\n \"type\": \"geojson\"\n }\n ]\n },\n {\n \"name\": \"POINT\",\n \"arguments\": [\n {\n \"name\": \"lat\",\n \"type\": \"double\"\n },\n {\n \"name\": \"lon\",\n \"type\": \"double\"\n },\n {\n \"name\": \"z\",\n \"type\": \"double\",\n \"optional\": true\n }\n ]\n },\n {\n \"name\": \"BOUNDS\",\n \"arguments\": [\n {\n \"name\": \"minlat\",\n \"type\": \"double\"\n },\n {\n \"name\": \"minlon\",\n \"type\": \"double\"\n },\n {\n \"name\": \"maxlat\",\n \"type\": \"double\"\n },\n {\n \"name\": \"maxlon\",\n \"type\": \"double\"\n }\n ]\n },\n {\n \"name\": \"HASH\",\n \"arguments\": [\n {\n \"name\": \"geohash\",\n \"type\": \"geohash\"\n }\n ]\n },\n {\n \"name\": \"STRING\",\n \"arguments\": [\n {\n \"name\": \"value\",\n \"type\": \"string\"\n }\n ]\n }\n ]\n }\n ],\n \"since\": \"1.0.0\",\n \"group\": \"keys\"\n },\n \"EXPIRE\": {\n \"summary\": \"Set a timeout on an id\",\n \"complexity\": \"O(1)\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\"\n },\n {\n \"name\": \"id\",\n \"type\": \"string\"\n },\n {\n \"name\": \"seconds\",\n \"type\": \"double\"\n }\n ],\n \"since\": \"1.0.0\",\n \"group\": \"keys\"\n },\n \"TTL\": {\n \"summary\": \"Get a timeout on an id\",\n \"complexity\": \"O(1)\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\"\n },\n {\n \"name\": \"id\",\n \"type\": \"string\"\n }\n ],\n \"since\": \"1.0.0\",\n \"group\": \"keys\"\n },\n \"EXISTS\": {\n \"summary\": \"Checks to see if a id exists\",\n \"complexity\": \"O(1)\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\"\n },\n {\n \"name\": \"id\",\n \"type\": \"string\"\n }\n ],\n \"since\": \"1.33.0\",\n \"group\": \"keys\"\n },\n \"FEXISTS\": {\n \"summary\": \"Checks to see if a field exists on a id\",\n \"complexity\": \"O(1)\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\"\n },\n {\n \"name\": \"id\",\n \"type\": \"string\"\n },\n {\n \"name\": \"field\",\n \"type\": \"string\"\n }\n ],\n \"since\": \"1.33.0\",\n \"group\": \"keys\"\n },\n \"PERSIST\": {\n \"summary\": \"Remove the existing timeout on an id\",\n \"complexity\": \"O(1)\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\"\n },\n {\n \"name\": \"id\",\n \"type\": \"string\"\n }\n ],\n \"since\": \"1.0.0\",\n \"group\": \"keys\"\n },\n \"FSET\": {\n \"summary\": \"Set the value for one or more fields of an id\",\n \"complexity\": \"O(1)\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\"\n },\n {\n \"name\": \"id\",\n \"type\": \"string\"\n },\n {\n \"command\": \"XX\",\n \"name\": [],\n \"type\": [],\n \"optional\": true\n },\n {\n \"name\": [\"field\", \"value\"],\n \"type\": [\"string\", \"double\"]\n },\n {\n \"name\": [\"field\", \"value\"],\n \"type\": [\"string\", \"double\"],\n \"multiple\": true,\n \"optional\": true\n }\n ],\n \"since\": \"1.0.0\",\n \"group\": \"keys\"\n },\n \"FGET\": {\n \"summary\": \"Gets the value for the field of an id\",\n \"complexity\": \"O(1)\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\"\n },\n {\n \"name\": \"id\",\n \"type\": \"string\"\n },\n {\n \"name\": \"field\",\n \"type\": \"string\"\n }\n ],\n \"since\": \"1.33.0\",\n \"group\": \"keys\"\n },\n \"BOUNDS\": {\n \"summary\": \"Get the combined bounds of all the objects in a key\",\n \"complexity\": \"O(1)\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\"\n }\n ],\n \"since\": \"1.3.0\",\n \"group\": \"keys\"\n },\n \"GET\": {\n \"summary\": \"Get the object of an id\",\n \"complexity\": \"O(1)\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\"\n },\n {\n \"name\": \"id\",\n \"type\": \"string\"\n },\n {\n \"command\": \"WITHFIELDS\",\n \"name\": [],\n \"type\": [],\n \"optional\": true\n },\n {\n \"name\": \"type\",\n \"optional\": true,\n \"enumargs\": [\n {\n \"name\": \"OBJECT\"\n },\n {\n \"name\": \"POINT\"\n },\n {\n \"name\": \"BOUNDS\"\n },\n {\n \"name\": \"HASH\",\n \"arguments\": [\n {\n \"name\": \"geohash\",\n \"type\": \"geohash\"\n }\n ]\n }\n ]\n }\n ],\n \"since\": \"1.0.0\",\n \"group\": \"keys\"\n },\n \"DEL\": {\n \"summary\": \"Delete an id from a key\",\n \"complexity\": \"O(1)\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\"\n },\n {\n \"name\": \"id\",\n \"type\": \"string\"\n },\n {\n \"command\": \"ERRON404\",\n \"name\": [],\n \"type\": [],\n \"optional\": true\n }\n ],\n \"since\": \"1.0.0\",\n \"group\": \"keys\"\n },\n \"DROP\": {\n \"summary\": \"Remove a key from the database\",\n \"complexity\": \"O(1)\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\"\n }\n ],\n \"since\": \"1.0.0\",\n \"group\": \"keys\"\n },\n \"RENAME\": {\n \"summary\": \"Rename a key to be stored under a different name.\",\n \"complexity\": \"O(1)\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\"\n },\n {\n \"name\": \"newkey\",\n \"type\": \"string\"\n }\n ],\n \"since\": \"1.14.5\",\n \"group\": \"keys\"\n },\n \"RENAMENX\": {\n \"summary\": \"Rename a key to be stored under a different name, if a new key does not exist.\",\n \"complexity\": \"O(1)\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\"\n },\n {\n \"name\": \"newkey\",\n \"type\": \"string\"\n }\n ],\n \"since\": \"1.14.5\",\n \"group\": \"keys\"\n },\n \"KEYS\": {\n \"summary\": \"Finds all keys matching the given pattern\",\n \"complexity\": \"O(N) where N is the number of keys in the database\",\n \"arguments\": [\n {\n \"name\": \"pattern\",\n \"type\": \"pattern\"\n }\n ],\n \"since\": \"1.0.0\",\n \"group\": \"keys\"\n },\n \"STATS\": {\n \"summary\": \"Show stats for one or more keys\",\n \"complexity\": \"O(N) where N is the number of keys being requested\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\",\n \"variadic\": true\n }\n ],\n \"since\": \"1.0.0\",\n \"group\": \"keys\"\n },\n \"SEARCH\": {\n \"summary\": \"Search for string values in a key\",\n \"complexity\": \"O(N) where N is the number of values in the key\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\"\n },\n {\n \"command\": \"CURSOR\",\n \"name\": \"start\",\n \"type\": \"integer\",\n \"optional\": true\n },\n {\n \"command\": \"LIMIT\",\n \"name\": \"count\",\n \"type\": \"integer\",\n \"optional\": true\n },\n {\n \"command\": \"MATCH\",\n \"name\": \"pattern\",\n \"type\": \"pattern\",\n \"optional\": true\n },\n {\n \"name\": \"order\",\n \"optional\": true,\n \"enumargs\": [\n {\n \"name\": \"ASC\"\n },\n {\n \"name\": \"DESC\"\n }\n ]\n },\n {\n \"command\": \"WHERE\",\n \"name\": [\"field\", \"min\", \"max\"],\n \"type\": [\"string\", \"double\", \"double\"],\n \"optional\": true,\n \"multiple\": true\n },\n {\n \"command\": \"WHEREIN\",\n \"name\": [\"field\", \"count\", \"value\"],\n \"type\": [\"string\", \"integer\", \"double\"],\n \"optional\": true,\n \"multiple\": true,\n \"variadic\": true\n },\n {\n \"command\": \"WHEREEVAL\",\n \"name\": [\"script\", \"numargs\", \"arg\"],\n \"type\": [\"string\", \"integer\", \"string\"],\n \"optional\": true,\n \"multiple\": true,\n \"variadic\": true\n },\n {\n \"command\": \"WHEREEVALSHA\",\n \"name\": [\"sha1\", \"numargs\", \"arg\"],\n \"type\": [\"string\", \"integer\", \"string\"],\n \"optional\": true,\n \"multiple\": true,\n \"variadic\": true\n },\n {\n \"command\": \"NOFIELDS\",\n \"name\": [],\n \"type\": [],\n \"optional\": true\n },\n {\n \"name\": \"type\",\n \"optional\": true,\n \"enumargs\": [\n {\n \"name\": \"COUNT\"\n },\n {\n \"name\": \"IDS\"\n }\n ]\n }\n ],\n \"since\": \"1.4.2\",\n \"group\": \"search\"\n },\n \"SCAN\": {\n \"summary\": \"Incrementally iterate though a key\",\n \"complexity\": \"O(N) where N is the number of ids in the key\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\"\n },\n {\n \"command\": \"CURSOR\",\n \"name\": \"start\",\n \"type\": \"integer\",\n \"optional\": true\n },\n {\n \"command\": \"LIMIT\",\n \"name\": \"count\",\n \"type\": \"integer\",\n \"optional\": true\n },\n {\n \"command\": \"MATCH\",\n \"name\": \"pattern\",\n \"type\": \"pattern\",\n \"optional\": true\n },\n {\n \"name\": \"order\",\n \"optional\": true,\n \"enumargs\": [\n {\n \"name\": \"ASC\"\n },\n {\n \"name\": \"DESC\"\n }\n ]\n },\n {\n \"command\": \"WHERE\",\n \"name\": [\"field\", \"min\", \"max\"],\n \"type\": [\"string\", \"double\", \"double\"],\n \"optional\": true,\n \"multiple\": true\n },\n {\n \"command\": \"WHEREIN\",\n \"name\": [\"field\", \"count\", \"value\"],\n \"type\": [\"string\", \"integer\", \"double\"],\n \"optional\": true,\n \"multiple\": true,\n \"variadic\": true\n },\n {\n \"command\": \"WHEREEVAL\",\n \"name\": [\"script\", \"numargs\", \"arg\"],\n \"type\": [\"string\", \"integer\", \"string\"],\n \"optional\": true,\n \"multiple\": true,\n \"variadic\": true\n },\n {\n \"command\": \"WHEREEVALSHA\",\n \"name\": [\"sha1\", \"numargs\", \"arg\"],\n \"type\": [\"string\", \"integer\", \"string\"],\n \"optional\": true,\n \"multiple\": true,\n \"variadic\": true\n },\n {\n \"command\": \"NOFIELDS\",\n \"name\": [],\n \"type\": [],\n \"optional\": true\n },\n {\n \"name\": \"type\",\n \"optional\": true,\n \"enumargs\": [\n {\n \"name\": \"COUNT\"\n },\n {\n \"name\": \"IDS\"\n },\n {\n \"name\": \"OBJECTS\"\n },\n {\n \"name\": \"POINTS\"\n },\n {\n \"name\": \"BOUNDS\"\n },\n {\n \"name\": \"HASHES\",\n \"arguments\": [\n {\n \"name\": \"precision\",\n \"type\": \"integer\"\n }\n ]\n }\n ]\n }\n ],\n \"since\": \"1.0.0\",\n \"group\": \"search\"\n },\n \"NEARBY\": {\n \"summary\": \"Searches for ids that are nearby a point\",\n \"complexity\": \"O(log(N)) where N is the number of ids in the area\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\"\n },\n {\n \"command\": \"CURSOR\",\n \"name\": \"start\",\n \"type\": \"integer\",\n \"optional\": true\n },\n {\n \"command\": \"LIMIT\",\n \"name\": \"count\",\n \"type\": \"integer\",\n \"optional\": true\n },\n {\n \"command\": \"MATCH\",\n \"name\": \"pattern\",\n \"type\": \"pattern\",\n \"optional\": true\n },\n {\n \"command\": \"DISTANCE\",\n \"name\": [],\n \"type\": [],\n \"optional\": true\n },\n {\n \"command\": \"WHERE\",\n \"name\": [\"field\", \"min\", \"max\"],\n \"type\": [\"string\", \"double\", \"double\"],\n \"optional\": true,\n \"multiple\": true\n },\n {\n \"command\": \"WHEREIN\",\n \"name\": [\"field\", \"count\", \"value\"],\n \"type\": [\"string\", \"integer\", \"double\"],\n \"optional\": true,\n \"multiple\": true,\n \"variadic\": true\n },\n {\n \"command\": \"WHEREEVAL\",\n \"name\": [\"script\", \"numargs\", \"arg\"],\n \"type\": [\"string\", \"integer\", \"string\"],\n \"optional\": true,\n \"multiple\": true,\n \"variadic\": true\n },\n {\n \"command\": \"WHEREEVALSHA\",\n \"name\": [\"sha1\", \"numargs\", \"arg\"],\n \"type\": [\"string\", \"integer\", \"string\"],\n \"optional\": true,\n \"multiple\": true,\n \"variadic\": true\n },\n {\n \"command\": \"NOFIELDS\",\n \"name\": [],\n \"type\": [],\n \"optional\": true\n },\n {\n \"command\": \"FENCE\",\n \"name\": [],\n \"type\": [],\n \"optional\": true\n },\n {\n \"command\": \"DETECT\",\n \"name\": [\"what\"],\n \"type\": [\"string\"],\n \"optional\": true\n },\n {\n \"command\": \"COMMANDS\",\n \"name\": [\"which\"],\n \"type\": [\"string\"],\n \"optional\": true\n },\n {\n \"name\": \"type\",\n \"optional\": true,\n \"enumargs\": [\n {\n \"name\": \"COUNT\"\n },\n {\n \"name\": \"IDS\"\n },\n {\n \"name\": \"OBJECTS\"\n },\n {\n \"name\": \"POINTS\"\n },\n {\n \"name\": \"BOUNDS\"\n },\n {\n \"name\": \"HASHES\",\n \"arguments\": [\n {\n \"name\": \"precision\",\n \"type\": \"integer\"\n }\n ]\n }\n ]\n },\n {\n \"name\": \"area\",\n \"enumargs\": [\n {\n \"name\": \"POINT\",\n \"arguments\": [\n {\n \"name\": \"lat\",\n \"type\": \"double\"\n },\n {\n \"name\": \"lon\",\n \"type\": \"double\"\n },\n {\n \"name\": \"meters\",\n \"type\": \"double\"\n }\n ]\n },\n {\n \"name\": \"ROAM\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\"\n },\n {\n \"name\": \"pattern\",\n \"type\": \"pattern\"\n },\n {\n \"name\": \"meters\",\n \"type\": \"double\"\n }\n ]\n }\n ]\n }\n ],\n \"since\": \"1.0.0\",\n \"group\": \"search\"\n },\n \"WITHIN\": {\n \"summary\": \"Searches for ids that completely within the area\",\n \"complexity\": \"O(log(N)) where N is the number of ids in the area\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\"\n },\n {\n \"command\": \"CURSOR\",\n \"name\": \"start\",\n \"type\": \"integer\",\n \"optional\": true\n },\n {\n \"command\": \"LIMIT\",\n \"name\": \"count\",\n \"type\": \"integer\",\n \"optional\": true\n },\n {\n \"command\": \"MATCH\",\n \"name\": \"pattern\",\n \"type\": \"pattern\",\n \"optional\": true\n },\n {\n \"command\": \"WHERE\",\n \"name\": [\"field\", \"min\", \"max\"],\n \"type\": [\"string\", \"double\", \"double\"],\n \"optional\": true,\n \"multiple\": true\n },\n {\n \"command\": \"WHEREIN\",\n \"name\": [\"field\", \"count\", \"value\"],\n \"type\": [\"string\", \"integer\", \"double\"],\n \"optional\": true,\n \"multiple\": true,\n \"variadic\": true\n },\n {\n \"command\": \"WHEREEVAL\",\n \"name\": [\"script\", \"numargs\", \"arg\"],\n \"type\": [\"string\", \"integer\", \"string\"],\n \"optional\": true,\n \"multiple\": true,\n \"variadic\": true\n },\n {\n \"command\": \"WHEREEVALSHA\",\n \"name\": [\"sha1\", \"numargs\", \"arg\"],\n \"type\": [\"string\", \"integer\", \"string\"],\n \"optional\": true,\n \"multiple\": true,\n \"variadic\": true\n },\n {\n \"command\": \"NOFIELDS\",\n \"name\": [],\n \"type\": [],\n \"optional\": true\n },\n {\n \"command\": \"FENCE\",\n \"name\": [],\n \"type\": [],\n \"optional\": true\n },\n {\n \"command\": \"DETECT\",\n \"name\": [\"what\"],\n \"type\": [\"string\"],\n \"optional\": true\n },\n {\n \"command\": \"COMMANDS\",\n \"name\": [\"which\"],\n \"type\": [\"string\"],\n \"optional\": true\n },\n {\n \"name\": \"type\",\n \"optional\": true,\n \"enumargs\": [\n {\n \"name\": \"COUNT\"\n },\n {\n \"name\": \"IDS\"\n },\n {\n \"name\": \"OBJECTS\"\n },\n {\n \"name\": \"POINTS\"\n },\n {\n \"name\": \"BOUNDS\"\n },\n {\n \"name\": \"HASHES\",\n \"arguments\": [\n {\n \"name\": \"precision\",\n \"type\": \"integer\"\n }\n ]\n }\n ]\n },\n {\n \"name\": \"area\",\n \"enumargs\": [\n {\n \"name\": \"GET\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\"\n },\n {\n \"name\": \"id\",\n \"type\": \"string\"\n }\n ]\n },\n {\n \"name\": \"BOUNDS\",\n \"arguments\": [\n {\n \"name\": \"minlat\",\n \"type\": \"double\"\n },\n {\n \"name\": \"minlon\",\n \"type\": \"double\"\n },\n {\n \"name\": \"maxlat\",\n \"type\": \"double\"\n },\n {\n \"name\": \"maxlon\",\n \"type\": \"double\"\n }\n ]\n },\n {\n \"name\": \"OBJECT\",\n \"arguments\": [\n {\n \"name\": \"geojson\",\n \"type\": \"geojson\"\n }\n ]\n },\n {\n \"name\": \"CIRCLE\",\n \"arguments\": [\n {\n \"name\": \"lat\",\n \"type\": \"double\"\n },\n {\n \"name\": \"lon\",\n \"type\": \"double\"\n },\n {\n \"name\": \"meters\",\n \"type\": \"double\"\n }\n ]\n },\n {\n \"name\": \"TILE\",\n \"arguments\": [\n {\n \"name\": \"x\",\n \"type\": \"double\"\n },\n {\n \"name\": \"y\",\n \"type\": \"double\"\n },\n {\n \"name\": \"z\",\n \"type\": \"double\"\n }\n ]\n },\n {\n \"name\": \"QUADKEY\",\n \"arguments\": [\n {\n \"name\": \"quadkey\",\n \"type\": \"string\"\n }\n ]\n },\n {\n \"name\": \"HASH\",\n \"arguments\": [\n {\n \"name\": \"geohash\",\n \"type\": \"geohash\"\n }\n ]\n }, \n {\n \"name\": \"SECTOR\",\n \"arguments\": [\n {\n \"name\": \"lat\",\n \"type\": \"double\"\n },\n {\n \"name\": \"lon\",\n \"type\": \"double\"\n },\n {\n \"name\": \"radius\",\n \"type\": \"double\"\n },\n {\n \"name\": \"startBearing\",\n \"type\": \"double\"\n },\n {\n \"name\": \"endBearing\",\n \"type\": \"double\"\n }\n ]\n }\n ]\n }\n ],\n \"since\": \"1.0.0\",\n \"group\": \"search\"\n },\n \"INTERSECTS\": {\n \"summary\": \"Searches for ids that intersect an area\",\n \"complexity\": \"O(log(N)) where N is the number of ids in the area\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\"\n },\n {\n \"command\": \"CURSOR\",\n \"name\": \"start\",\n \"type\": \"integer\",\n \"optional\": true\n },\n {\n \"command\": \"LIMIT\",\n \"name\": \"count\",\n \"type\": \"integer\",\n \"optional\": true\n },\n {\n \"command\": \"MATCH\",\n \"name\": \"pattern\",\n \"type\": \"pattern\",\n \"optional\": true\n },\n {\n \"command\": \"WHERE\",\n \"name\": [\"field\", \"min\", \"max\"],\n \"type\": [\"string\", \"double\", \"double\"],\n \"optional\": true,\n \"multiple\": true\n },\n {\n \"command\": \"WHEREIN\",\n \"name\": [\"field\", \"count\", \"value\"],\n \"type\": [\"string\", \"integer\", \"double\"],\n \"optional\": true,\n \"multiple\": true,\n \"variadic\": true\n },\n {\n \"command\": \"WHEREEVAL\",\n \"name\": [\"script\", \"numargs\", \"arg\"],\n \"type\": [\"string\", \"integer\", \"string\"],\n \"optional\": true,\n \"multiple\": true,\n \"variadic\": true\n },\n {\n \"command\": \"WHEREEVALSHA\",\n \"name\": [\"sha1\", \"numargs\", \"arg\"],\n \"type\": [\"string\", \"integer\", \"string\"],\n \"optional\": true,\n \"multiple\": true,\n \"variadic\": true\n },\n {\n \"command\": \"CLIP\",\n \"name\": [],\n \"type\": [],\n \"optional\": true\n },\n {\n \"command\": \"NOFIELDS\",\n \"name\": [],\n \"type\": [],\n \"optional\": true\n },\n {\n \"command\": \"FENCE\",\n \"name\": [],\n \"type\": [],\n \"optional\": true\n },\n {\n \"command\": \"DETECT\",\n \"name\": [\"what\"],\n \"type\": [\"string\"],\n \"optional\": true\n },\n {\n \"command\": \"COMMANDS\",\n \"name\": [\"which\"],\n \"type\": [\"string\"],\n \"optional\": true\n },\n {\n \"name\": \"type\",\n \"optional\": true,\n \"enumargs\": [\n {\n \"name\": \"COUNT\"\n },\n {\n \"name\": \"IDS\"\n },\n {\n \"name\": \"OBJECTS\"\n },\n {\n \"name\": \"POINTS\"\n },\n {\n \"name\": \"BOUNDS\"\n },\n {\n \"name\": \"HASHES\",\n \"arguments\": [\n {\n \"name\": \"precision\",\n \"type\": \"integer\"\n }\n ]\n }\n ]\n },\n {\n \"name\": \"area\",\n \"enumargs\": [\n {\n \"name\": \"GET\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\"\n },\n {\n \"name\": \"id\",\n \"type\": \"string\"\n }\n ]\n },\n {\n \"name\": \"BOUNDS\",\n \"arguments\": [\n {\n \"name\": \"minlat\",\n \"type\": \"double\"\n },\n {\n \"name\": \"minlon\",\n \"type\": \"double\"\n },\n {\n \"name\": \"maxlat\",\n \"type\": \"double\"\n },\n {\n \"name\": \"maxlon\",\n \"type\": \"double\"\n }\n ]\n },\n {\n \"name\": \"OBJECT\",\n \"arguments\": [\n {\n \"name\": \"geojson\",\n \"type\": \"geojson\"\n }\n ]\n },\n {\n \"name\": \"CIRCLE\",\n \"arguments\": [\n {\n \"name\": \"lat\",\n \"type\": \"double\"\n },\n {\n \"name\": \"lon\",\n \"type\": \"double\"\n },\n {\n \"name\": \"meters\",\n \"type\": \"double\"\n }\n ]\n },\n {\n \"name\": \"TILE\",\n \"arguments\": [\n {\n \"name\": \"x\",\n \"type\": \"double\"\n },\n {\n \"name\": \"y\",\n \"type\": \"double\"\n },\n {\n \"name\": \"z\",\n \"type\": \"double\"\n }\n ]\n },\n {\n \"name\": \"QUADKEY\",\n \"arguments\": [\n {\n \"name\": \"quadkey\",\n \"type\": \"string\"\n }\n ]\n },\n {\n \"name\": \"HASH\",\n \"arguments\": [\n {\n \"name\": \"geohash\",\n \"type\": \"geohash\"\n }\n ]\n },\n {\n \"name\": \"SECTOR\",\n \"arguments\": [\n {\n \"name\": \"lat\",\n \"type\": \"double\"\n },\n {\n \"name\": \"lon\",\n \"type\": \"double\"\n },\n {\n \"name\": \"radius\",\n \"type\": \"double\"\n },\n {\n \"name\": \"startBearing\",\n \"type\": \"double\"\n },\n {\n \"name\": \"endBearing\",\n \"type\": \"double\"\n }\n ]\n }\n ]\n }\n ],\n \"since\": \"1.0.0\",\n \"group\": \"search\"\n },\n \"CONFIG GET\": {\n \"summary\": \"Get the value of a configuration parameter\",\n \"arguments\": [\n {\n \"name\": \"parameter\",\n \"type\": \"string\"\n }\n ],\n \"group\": \"server\"\n },\n \"CONFIG SET\": {\n \"summary\": \"Set a configuration parameter to the given value\",\n \"arguments\": [\n {\n \"name\": \"parameter\",\n \"type\": \"string\"\n },\n {\n \"name\": \"value\",\n \"type\": \"string\",\n \"optional\": true\n }\n ],\n \"group\": \"server\"\n },\n \"CONFIG REWRITE\": {\n \"summary\": \"Rewrite the configuration file with the in memory configuration\",\n \"arguments\": [],\n \"group\": \"server\"\n },\n \"SERVER\": {\n \"summary\": \"Show server stats and details\",\n \"complexity\": \"O(1)\",\n \"arguments\": [],\n \"since\": \"1.0.0\",\n \"group\": \"server\"\n },\n \"GC\": {\n \"summary\": \"Forces a garbage collection\",\n \"complexity\": \"O(1)\",\n \"arguments\": [],\n \"since\": \"1.0.0\",\n \"group\": \"server\"\n },\n \"READONLY\": {\n \"summary\": \"Turns on or off readonly mode\",\n \"complexity\": \"O(1)\",\n \"arguments\": [\n {\n \"enum\": [\"yes\", \"no\"]\n }\n ],\n \"since\": \"1.0.0\",\n \"group\": \"server\"\n },\n \"FLUSHDB\": {\n \"summary\": \"Removes all keys\",\n \"complexity\": \"O(1)\",\n \"arguments\": [],\n \"since\": \"1.0.0\",\n \"group\": \"server\"\n },\n \"FOLLOW\": {\n \"summary\": \"Follows a leader host\",\n \"complexity\": \"O(1)\",\n \"arguments\": [\n {\n \"name\": \"host\",\n \"type\": \"string\"\n },\n {\n \"name\": \"port\",\n \"type\": \"integer\"\n }\n ],\n \"since\": \"1.0.0\",\n \"group\": \"replication\"\n },\n \"AOF\": {\n \"summary\": \"Downloads the AOF starting from pos and keeps the connection alive\",\n \"complexity\": \"O(1)\",\n \"arguments\": [\n {\n \"name\": \"pos\",\n \"type\": \"integer\"\n }\n ],\n \"since\": \"1.0.0\",\n \"group\": \"replication\"\n },\n \"AOFMD5\": {\n \"summary\": \"Performs a checksum on a portion of the aof\",\n \"complexity\": \"O(1)\",\n \"arguments\": [\n {\n \"name\": \"pos\",\n \"type\": \"integer\"\n },\n {\n \"name\": \"size\",\n \"type\": \"integer\"\n }\n ],\n \"since\": \"1.0.0\",\n \"group\": \"replication\"\n },\n \"AOFSHRINK\": {\n \"summary\": \"Shrinks the aof in the background\",\n \"group\": \"replication\"\n },\n \"PING\": {\n \"summary\": \"Ping the server\",\n \"group\": \"connection\"\n },\n \"QUIT\": {\n \"summary\": \"Close the connection\",\n \"group\": \"connection\"\n },\n \"AUTH\": {\n \"summary\": \"Authenticate to the server\",\n \"arguments\": [\n {\n \"name\": \"password\",\n \"type\": \"string\"\n }\n ],\n \"group\": \"connection\"\n },\n \"OUTPUT\": {\n \"summary\": \"Gets or sets the output format for the current connection.\",\n \"arguments\": [\n {\n \"name\": \"format\",\n \"optional\": true,\n \"enumargs\": [\n {\n \"name\": \"json\"\n },\n {\n \"name\": \"resp\"\n }\n ]\n }\n ],\n \"group\": \"connection\"\n },\n \"TIMEOUT\": {\n \"summary\": \"Runs the following command with the timeout\",\n \"arguments\": [\n {\n \"name\": \"seconds\",\n \"type\": \"double\"\n },\n {\n \"name\": \"COMMAND\",\n \"type\": \"string\"\n },\n {\n \"command\": \"arg\",\n \"type\": \"string\",\n \"multiple\": true,\n \"optional\": true\n }\n ],\n \"group\": \"connection\"\n },\n \"SETHOOK\": {\n \"summary\": \"Creates a webhook which points to geofenced search\",\n \"arguments\": [\n {\n \"name\": \"name\",\n \"type\": \"string\"\n },\n {\n \"name\": \"endpoint\",\n \"type\": \"string\"\n },\n {\n \"command\": \"META\",\n \"name\": [\"name\", \"value\"],\n \"type\": [\"string\", \"string\"],\n \"optional\": true,\n \"multiple\": true\n },\n {\n \"command\": \"EX\",\n \"name\": [\"seconds\"],\n \"type\": [\"double\"],\n \"optional\": true,\n \"multiple\": false\n },\n {\n \"enum\": [\"NEARBY\", \"WITHIN\", \"INTERSECTS\"]\n },\n {\n \"name\": \"key\",\n \"type\": \"string\"\n },\n {\n \"command\": \"FENCE\",\n \"name\": [],\n \"type\": []\n },\n {\n \"command\": \"DETECT\",\n \"name\": [\"what\"],\n \"type\": [\"string\"],\n \"optional\": true\n },\n {\n \"command\": \"COMMANDS\",\n \"name\": [\"which\"],\n \"type\": [\"string\"],\n \"optional\": true\n },\n {\n \"name\": \"param\",\n \"type\": \"string\",\n \"variadic\": true\n }\n ],\n \"group\": \"webhook\"\n },\n \"DELHOOK\": {\n \"summary\": \"Removes a webhook\",\n \"arguments\": [\n {\n \"name\": \"name\",\n \"type\": \"string\"\n }\n ],\n \"group\": \"webhook\"\n },\n \"HOOKS\": {\n \"summary\": \"Finds all hooks matching a pattern\",\n \"arguments\": [\n {\n \"name\": \"pattern\",\n \"type\": \"pattern\"\n }\n ],\n \"group\": \"webhook\"\n },\n \"PDELHOOK\": {\n \"summary\": \"Removes all hooks matching a pattern\",\n \"arguments\": [\n {\n \"name\": \"pattern\",\n \"type\": \"pattern\"\n }\n ],\n \"group\": \"webhook\"\n },\n\n \"SETCHAN\": {\n \"summary\": \"Creates a pubsub channel which points to geofenced search\",\n \"arguments\": [\n {\n \"name\": \"name\",\n \"type\": \"string\"\n },\n {\n \"command\": \"META\",\n \"name\": [\"name\", \"value\"],\n \"type\": [\"string\", \"string\"],\n \"optional\": true,\n \"multiple\": true\n },\n {\n \"command\": \"EX\",\n \"name\": [\"seconds\"],\n \"type\": [\"double\"],\n \"optional\": true,\n \"multiple\": false\n },\n {\n \"enum\": [\"NEARBY\", \"WITHIN\", \"INTERSECTS\"]\n },\n {\n \"name\": \"key\",\n \"type\": \"string\"\n },\n {\n \"command\": \"FENCE\",\n \"name\": [],\n \"type\": []\n },\n {\n \"command\": \"DETECT\",\n \"name\": [\"what\"],\n \"type\": [\"string\"],\n \"optional\": true\n },\n {\n \"command\": \"COMMANDS\",\n \"name\": [\"which\"],\n \"type\": [\"string\"],\n \"optional\": true\n },\n {\n \"name\": \"param\",\n \"type\": \"string\",\n \"variadic\": true\n }\n ],\n \"group\": \"pubsub\"\n },\n \"DELCHAN\": {\n \"summary\": \"Removes a channel\",\n \"arguments\": [\n {\n \"name\": \"name\",\n \"type\": \"string\"\n }\n ],\n \"group\": \"pubsub\"\n },\n \"CHANS\": {\n \"summary\": \"Finds all channels matching a pattern\",\n \"arguments\": [\n {\n \"name\": \"pattern\",\n \"type\": \"pattern\"\n }\n ],\n \"group\": \"pubsub\"\n },\n \"PDELCHAN\": {\n \"summary\": \"Removes all channels matching a pattern\",\n \"arguments\": [\n {\n \"name\": \"pattern\",\n \"type\": \"pattern\"\n }\n ],\n \"group\": \"pubsub\"\n },\n \"SUBSCRIBE\": {\n \"summary\": \"Subscribe to a geofence channel\",\n \"arguments\": [\n {\n \"name\": \"channel\",\n \"type\": \"string\",\n \"variadic\": true\n }\n ],\n \"group\": \"pubsub\"\n },\n \"PSUBSCRIBE\": {\n \"summary\": \"Subscribes the client to the given patterns\",\n \"arguments\": [\n {\n \"name\": \"pattern\",\n \"type\": \"pattern\",\n \"variadic\": true\n }\n ],\n \"group\": \"pubsub\"\n },\n \"PDEL\": {\n \"summary\": \"Removes all objects matching a pattern\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\"\n },\n {\n \"name\": \"pattern\",\n \"type\": \"pattern\"\n }\n ],\n \"group\": \"keys\"\n },\n \"JGET\": {\n \"summary\": \"Get a value from a JSON document\",\n \"complexity\": \"O(1)\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\"\n },\n {\n \"name\": \"id\",\n \"type\": \"string\"\n },\n {\n \"name\": \"path\",\n \"type\": \"string\"\n },\n {\n \"command\": \"RAW\",\n \"name\": [],\n \"type\": [],\n \"optional\": true\n }\n ],\n \"group\": \"keys\"\n },\n \"JSET\": {\n \"summary\": \"Set a value in a JSON document\",\n \"complexity\": \"O(1)\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\"\n },\n {\n \"name\": \"id\",\n \"type\": \"string\"\n },\n {\n \"name\": \"path\",\n \"type\": \"string\"\n },\n {\n \"name\": \"value\",\n \"type\": \"string\"\n },\n {\n \"name\": [],\n \"optional\": true,\n \"enumargs\": [\n {\n \"name\": \"RAW\"\n },\n {\n \"name\": \"STR\"\n }\n ]\n }\n ],\n \"group\": \"keys\"\n },\n \"JDEL\": {\n \"summary\": \"Delete a value from a JSON document\",\n \"complexity\": \"O(1)\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\"\n },\n {\n \"name\": \"id\",\n \"type\": \"string\"\n },\n {\n \"name\": \"path\",\n \"type\": \"string\"\n }\n ],\n \"group\": \"keys\"\n },\n \"EVAL\": {\n \"summary\": \"Evaluates a Lua script\",\n \"complexity\": \"Depends on the evaluated script\",\n \"arguments\": [\n {\n \"name\": \"script\",\n \"type\": \"string\"\n },\n {\n \"name\": \"numkeys\",\n \"type\": \"integer\"\n },\n {\n \"name\": \"key\",\n \"type\": \"string\",\n \"optional\": true,\n \"multiple\": true\n },\n {\n \"name\": \"arg\",\n \"type\": \"string\",\n \"optional\": true,\n \"multiple\": true\n }\n ],\n \"since\": \"1.10.0\",\n \"group\": \"scripting\"\n },\n \"EVALSHA\": {\n \"summary\": \"Evaluates a Lua script cached on the server by its SHA1 digest\",\n \"complexity\": \"Depends on the evaluated script\",\n \"arguments\": [\n {\n \"name\": \"sha1\",\n \"type\": \"string\"\n },\n {\n \"name\": \"numkeys\",\n \"type\": \"integer\"\n },\n {\n \"name\": \"key\",\n \"type\": \"string\",\n \"optional\": true,\n \"multiple\": true\n },\n {\n \"name\": \"arg\",\n \"type\": \"string\",\n \"optional\": true,\n \"multiple\": true\n }\n ],\n \"since\": \"1.10.0\",\n \"group\": \"scripting\"\n },\n \"EVALRO\": {\n \"summary\": \"Evaluates a read-only Lua script\",\n \"complexity\": \"Depends on the evaluated script\",\n \"arguments\": [\n {\n \"name\": \"script\",\n \"type\": \"string\"\n },\n {\n \"name\": \"numkeys\",\n \"type\": \"integer\"\n },\n {\n \"name\": \"key\",\n \"type\": \"string\",\n \"optional\": true,\n \"multiple\": true\n },\n {\n \"name\": \"arg\",\n \"type\": \"string\",\n \"optional\": true,\n \"multiple\": true\n }\n ],\n \"since\": \"1.10.0\",\n \"group\": \"scripting\"\n },\n \"EVALROSHA\": {\n \"summary\": \"Evaluates a read-only Lua script cached on the server by its SHA1 digest\",\n \"complexity\": \"Depends on the evaluated script\",\n \"arguments\": [\n {\n \"name\": \"script\",\n \"type\": \"string\"\n },\n {\n \"name\": \"numkeys\",\n \"type\": \"integer\"\n },\n {\n \"name\": \"key\",\n \"type\": \"string\",\n \"optional\": true,\n \"multiple\": true\n },\n {\n \"name\": \"arg\",\n \"type\": \"string\",\n \"optional\": true,\n \"multiple\": true\n }\n ],\n \"since\": \"1.10.0\",\n \"group\": \"scripting\"\n },\n \"EVALNA\": {\n \"summary\": \"Evaluates a Lua script in a non-atomic fashion\",\n \"complexity\": \"Depends on the evaluated script\",\n \"arguments\": [\n {\n \"name\": \"script\",\n \"type\": \"string\"\n },\n {\n \"name\": \"numkeys\",\n \"type\": \"integer\"\n },\n {\n \"name\": \"key\",\n \"type\": \"string\",\n \"optional\": true,\n \"multiple\": true\n },\n {\n \"name\": \"arg\",\n \"type\": \"string\",\n \"optional\": true,\n \"multiple\": true\n }\n ],\n \"since\": \"1.10.0\",\n \"group\": \"scripting\"\n },\n \"EVALNASHA\": {\n \"summary\": \"Evaluates, in a non-atomic fashion, a Lua script cached on the server by its SHA1 digest\",\n \"complexity\": \"Depends on the evaluated script\",\n \"arguments\": [\n {\n \"name\": \"sha1\",\n \"type\": \"string\"\n },\n {\n \"name\": \"numkeys\",\n \"type\": \"integer\"\n },\n {\n \"name\": \"key\",\n \"type\": \"string\",\n \"optional\": true,\n \"multiple\": true\n },\n {\n \"name\": \"arg\",\n \"type\": \"string\",\n \"optional\": true,\n \"multiple\": true\n }\n ],\n \"since\": \"1.10.0\",\n \"group\": \"scripting\"\n },\n \"SCRIPT EXISTS\": {\n \"summary\": \"Returns information about the existence of the scripts in server cache\",\n \"complexity\": \"O(N) where N is the number of provided sha1 arguments\",\n \"arguments\": [\n {\n \"name\": \"sha1\",\n \"type\": \"string\",\n \"multiple\": true\n }\n ],\n \"since\": \"1.10.0\",\n \"group\": \"scripting\"\n },\n \"SCRIPT LOAD\": {\n \"summary\": \"Loads the compiled version of a script into the server cache, without executing\",\n \"complexity\": \"O(N) where N is the number of bytes in the script\",\n \"arguments\": [\n {\n \"name\": \"script\",\n \"type\": \"string\"\n }\n ],\n \"since\": \"1.10.0\",\n \"group\": \"scripting\"\n },\n \"SCRIPT FLUSH\": {\n \"summary\": \"Flushes the server cache of Lua scripts\",\n \"complexity\": \"O(1)\",\n \"since\": \"1.10.0\",\n \"group\": \"scripting\"\n },\n \"TEST\": {\n \"summary\": \"Performs spatial test\",\n \"complexity\": \"One test per command, complexity depends on the test\",\n \"arguments\": [\n {\n \"name\": \"area1\",\n \"enumargs\": [\n {\n \"name\": \"POINT\",\n \"arguments\": [\n {\n \"name\": \"lat\",\n \"type\": \"double\"\n },\n {\n \"name\": \"lon\",\n \"type\": \"double\"\n }\n ]\n },\n {\n \"name\": \"GET\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\"\n },\n {\n \"name\": \"id\",\n \"type\": \"string\"\n }\n ]\n },\n {\n \"name\": \"BOUNDS\",\n \"arguments\": [\n {\n \"name\": \"minlat\",\n \"type\": \"double\"\n },\n {\n \"name\": \"minlon\",\n \"type\": \"double\"\n },\n {\n \"name\": \"maxlat\",\n \"type\": \"double\"\n },\n {\n \"name\": \"maxlon\",\n \"type\": \"double\"\n }\n ]\n },\n {\n \"name\": \"OBJECT\",\n \"arguments\": [\n {\n \"name\": \"geojson\",\n \"type\": \"geojson\"\n }\n ]\n },\n {\n \"name\": \"CIRCLE\",\n \"arguments\": [\n {\n \"name\": \"lat\",\n \"type\": \"double\"\n },\n {\n \"name\": \"lon\",\n \"type\": \"double\"\n },\n {\n \"name\": \"meters\",\n \"type\": \"double\"\n }\n ]\n },\n {\n \"name\": \"TILE\",\n \"arguments\": [\n {\n \"name\": \"x\",\n \"type\": \"double\"\n },\n {\n \"name\": \"y\",\n \"type\": \"double\"\n },\n {\n \"name\": \"z\",\n \"type\": \"double\"\n }\n ]\n },\n {\n \"name\": \"QUADKEY\",\n \"arguments\": [\n {\n \"name\": \"quadkey\",\n \"type\": \"string\"\n }\n ]\n },\n {\n \"name\": \"HASH\",\n \"arguments\": [\n {\n \"name\": \"geohash\",\n \"type\": \"geohash\"\n }\n ]\n }\n ]\n },\n {\n \"name\": \"test\",\n \"enumargs\": [\n {\n \"name\": \"INTERSECTS\"\n },\n {\n \"name\": \"WITHIN\"\n }\n ]\n },\n {\n \"command\": \"CLIP\",\n \"name\": [],\n \"type\": [],\n \"optional\": true\n },\n {\n \"name\": \"area2\",\n \"enumargs\": [\n {\n \"name\": \"POINT\",\n \"arguments\": [\n {\n \"name\": \"lat\",\n \"type\": \"double\"\n },\n {\n \"name\": \"lon\",\n \"type\": \"double\"\n }\n ]\n },\n {\n \"name\": \"GET\",\n \"arguments\": [\n {\n \"name\": \"key\",\n \"type\": \"string\"\n },\n {\n \"name\": \"id\",\n \"type\": \"string\"\n }\n ]\n },\n {\n \"name\": \"BOUNDS\",\n \"arguments\": [\n {\n \"name\": \"minlat\",\n \"type\": \"double\"\n },\n {\n \"name\": \"minlon\",\n \"type\": \"double\"\n },\n {\n \"name\": \"maxlat\",\n \"type\": \"double\"\n },\n {\n \"name\": \"maxlon\",\n \"type\": \"double\"\n }\n ]\n },\n {\n \"name\": \"OBJECT\",\n \"arguments\": [\n {\n \"name\": \"geojson\",\n \"type\": \"geojson\"\n }\n ]\n },\n {\n \"name\": \"CIRCLE\",\n \"arguments\": [\n {\n \"name\": \"lat\",\n \"type\": \"double\"\n },\n {\n \"name\": \"lon\",\n \"type\": \"double\"\n },\n {\n \"name\": \"meters\",\n \"type\": \"double\"\n }\n ]\n },\n {\n \"name\": \"TILE\",\n \"arguments\": [\n {\n \"name\": \"x\",\n \"type\": \"double\"\n },\n {\n \"name\": \"y\",\n \"type\": \"double\"\n },\n {\n \"name\": \"z\",\n \"type\": \"double\"\n }\n ]\n },\n {\n \"name\": \"QUADKEY\",\n \"arguments\": [\n {\n \"name\": \"quadkey\",\n \"type\": \"string\"\n }\n ]\n },\n {\n \"name\": \"HASH\",\n \"arguments\": [\n {\n \"name\": \"geohash\",\n \"type\": \"geohash\"\n }\n ]\n }\n ]\n }\n ],\n \"since\": \"1.16.0\",\n \"group\": \"tests\"\n }\n}`\n" }, { "path": "core/commands_test.go", "content": "package core\n\nimport (\n\t\"fmt\"\n\t\"sort\"\n\t\"testing\"\n)\n\nfunc TestCommands(t *testing.T) {\n\tvar names []string\n\tfor name := range Commands {\n\t\tnames = append(names, name)\n\t}\n\tsort.Strings(names)\n\tfor _, name := range names {\n\t\tcmd := Commands[name]\n\t\tif cmd.Group == \"server\" {\n\t\t\tfmt.Printf(\"%v\\n\", cmd.String())\n\t\t}\n\t}\n\n}\n" }, { "path": "core/gen.sh", "content": "#!/bin/sh\n\nset -e\n\ncd $(dirname $0)\nexport CommandsJSON=\"$(cat commands.json)\"\n\n# replace out the json\nperl -pe '\n while (($i = index($_, \"{{.CommandsJSON}}\")) != -1) {\n substr($_, $i, length(\"{{.CommandsJSON}}\")) = $ENV{\"CommandsJSON\"};\n }\n' commands.go > commands_gen.go\n\n# remove the ignore\nsed -i -e 's/\\/\\/go:build ignore/\\/\\/ This file was autogenerated. DO NOT EDIT./g' commands_gen.go\nrm -rf commands_gen.go-e\n" }, { "path": "core/version.go", "content": "package core\n\n// Build variables\nvar (\n\tVersion = \"0.0.0\" // Placeholder for the version\n\tBuildTime = \"\" // Placeholder for the build time\n\tGitSHA = \"0000000\" // Placeholder for the git sha\n)\n" }, { "path": "go.mod", "content": "module github.com/tidwall/tile38\n\ngo 1.24.0\n\nrequire (\n\tgithub.com/tidwall/assert v0.1.0\n\tgithub.com/tidwall/btree v1.8.1\n\tgithub.com/tidwall/buntdb v1.3.2\n\tgithub.com/tidwall/expr v0.14.0\n\tgithub.com/tidwall/geojson v1.4.6\n\tgithub.com/tidwall/gjson v1.18.0\n\tgithub.com/tidwall/hashmap v1.8.1\n\tgithub.com/tidwall/limiter v0.4.0\n\tgithub.com/tidwall/match v1.2.0\n\tgithub.com/tidwall/mvt v0.2.1\n\tgithub.com/tidwall/pretty v1.2.1\n\tgithub.com/tidwall/redbench v0.1.0\n\tgithub.com/tidwall/redcon v1.6.2\n\tgithub.com/tidwall/resp v0.1.1\n\tgithub.com/tidwall/rtree v1.10.0\n\tgithub.com/tidwall/sjson v1.2.5\n\tgithub.com/tidwall/tinylru v1.2.1\n)\n\nrequire (\n\tcloud.google.com/go/pubsub v1.50.0\n\tgithub.com/Azure/azure-event-hubs-go/v3 v3.6.2\n\tgithub.com/IBM/sarama v1.46.0\n\tgithub.com/aws/aws-sdk-go v1.55.8\n\tgithub.com/cloudflare/cloudflare-go/v4 v4.6.0\n\tgithub.com/eclipse/paho.mqtt.golang v1.5.1\n\tgithub.com/golang/protobuf v1.5.4\n\tgithub.com/gomodule/redigo v1.9.2\n\tgithub.com/iwpnd/sectr v0.1.2\n\tgithub.com/mmcloughlin/geohash v0.10.0\n\tgithub.com/nats-io/nats.go v1.44.0\n\tgithub.com/peterh/liner v1.2.2\n\tgithub.com/prometheus/client_golang v1.23.0\n\tgithub.com/streadway/amqp v1.1.0\n\tgithub.com/xdg-go/scram v1.1.2\n\tgithub.com/yuin/gopher-lua v1.1.1\n\tgo.uber.org/atomic v1.11.0\n\tgo.uber.org/zap v1.27.0\n\tgolang.org/x/net v0.47.0\n\tgolang.org/x/term v0.37.0\n\tgoogle.golang.org/api v0.246.0\n\tgoogle.golang.org/grpc v1.74.2\n\tlayeh.com/gopher-json v0.0.0-20201124131017-552bb3c4c3bf\n)\n\nrequire (\n\tcloud.google.com/go v0.121.4 // indirect\n\tcloud.google.com/go/auth v0.16.3 // indirect\n\tcloud.google.com/go/auth/oauth2adapt v0.2.8 // indirect\n\tcloud.google.com/go/compute/metadata v0.7.0 // indirect\n\tcloud.google.com/go/iam v1.5.2 // indirect\n\tcloud.google.com/go/pubsub/v2 v2.0.0 // indirect\n\tgithub.com/Azure/azure-amqp-common-go/v4 v4.2.0 // indirect\n\tgithub.com/Azure/azure-sdk-for-go v65.0.0+incompatible // indirect\n\tgithub.com/Azure/go-amqp v1.0.0 // indirect\n\tgithub.com/Azure/go-autorest v14.2.0+incompatible // indirect\n\tgithub.com/Azure/go-autorest/autorest v0.11.28 // indirect\n\tgithub.com/Azure/go-autorest/autorest/adal v0.9.21 // indirect\n\tgithub.com/Azure/go-autorest/autorest/date v0.3.0 // indirect\n\tgithub.com/Azure/go-autorest/autorest/to v0.4.0 // indirect\n\tgithub.com/Azure/go-autorest/autorest/validation v0.3.1 // indirect\n\tgithub.com/Azure/go-autorest/logger v0.2.1 // indirect\n\tgithub.com/Azure/go-autorest/tracing v0.6.0 // indirect\n\tgithub.com/beorn7/perks v1.0.1 // indirect\n\tgithub.com/cespare/xxhash/v2 v2.3.0 // indirect\n\tgithub.com/davecgh/go-spew v1.1.1 // indirect\n\tgithub.com/devigned/tab v0.1.1 // indirect\n\tgithub.com/eapache/go-resiliency v1.7.0 // indirect\n\tgithub.com/eapache/go-xerial-snappy v0.0.0-20230731223053-c322873962e3 // indirect\n\tgithub.com/eapache/queue v1.1.0 // indirect\n\tgithub.com/felixge/httpsnoop v1.0.4 // indirect\n\tgithub.com/go-logr/logr v1.4.3 // indirect\n\tgithub.com/go-logr/stdr v1.2.2 // indirect\n\tgithub.com/golang-jwt/jwt/v4 v4.5.2 // indirect\n\tgithub.com/golang/snappy v0.0.4 // indirect\n\tgithub.com/google/s2a-go v0.1.9 // indirect\n\tgithub.com/google/uuid v1.6.0 // indirect\n\tgithub.com/googleapis/enterprise-certificate-proxy v0.3.6 // indirect\n\tgithub.com/googleapis/gax-go/v2 v2.15.0 // indirect\n\tgithub.com/gorilla/websocket v1.5.3 // indirect\n\tgithub.com/hashicorp/go-uuid v1.0.3 // indirect\n\tgithub.com/jcmturner/aescts/v2 v2.0.0 // indirect\n\tgithub.com/jcmturner/dnsutils/v2 v2.0.0 // indirect\n\tgithub.com/jcmturner/gofork v1.7.6 // indirect\n\tgithub.com/jcmturner/gokrb5/v8 v8.4.4 // indirect\n\tgithub.com/jcmturner/rpc/v2 v2.0.3 // indirect\n\tgithub.com/jmespath/go-jmespath v0.4.0 // indirect\n\tgithub.com/jpillora/backoff v1.0.0 // indirect\n\tgithub.com/klauspost/compress v1.18.0 // indirect\n\tgithub.com/klauspost/cpuid/v2 v2.0.9 // indirect\n\tgithub.com/mattn/go-runewidth v0.0.3 // indirect\n\tgithub.com/mitchellh/mapstructure v1.5.0 // indirect\n\tgithub.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect\n\tgithub.com/nats-io/nkeys v0.4.11 // indirect\n\tgithub.com/nats-io/nuid v1.0.1 // indirect\n\tgithub.com/pierrec/lz4/v4 v4.1.22 // indirect\n\tgithub.com/prometheus/client_model v0.6.2 // indirect\n\tgithub.com/prometheus/common v0.65.0 // indirect\n\tgithub.com/prometheus/procfs v0.16.1 // indirect\n\tgithub.com/rcrowley/go-metrics v0.0.0-20250401214520-65e299d6c5c9 // indirect\n\tgithub.com/tidwall/conv v0.1.0 // indirect\n\tgithub.com/tidwall/geoindex v1.7.0 // indirect\n\tgithub.com/tidwall/grect v0.1.4 // indirect\n\tgithub.com/tidwall/rtred v0.1.2 // indirect\n\tgithub.com/tidwall/tinyqueue v0.1.1 // indirect\n\tgithub.com/xdg-go/pbkdf2 v1.0.0 // indirect\n\tgithub.com/xdg-go/stringprep v1.0.4 // indirect\n\tgithub.com/zeebo/xxh3 v1.0.2 // indirect\n\tgo.opencensus.io v0.24.0 // indirect\n\tgo.opentelemetry.io/auto/sdk v1.1.0 // indirect\n\tgo.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.61.0 // indirect\n\tgo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.61.0 // indirect\n\tgo.opentelemetry.io/otel v1.36.0 // indirect\n\tgo.opentelemetry.io/otel/metric v1.36.0 // indirect\n\tgo.opentelemetry.io/otel/trace v1.36.0 // indirect\n\tgo.uber.org/multierr v1.10.0 // indirect\n\tgolang.org/x/crypto v0.45.0 // indirect\n\tgolang.org/x/oauth2 v0.30.0 // indirect\n\tgolang.org/x/sync v0.18.0 // indirect\n\tgolang.org/x/sys v0.38.0 // indirect\n\tgolang.org/x/text v0.31.0 // indirect\n\tgolang.org/x/time v0.12.0 // indirect\n\tgoogle.golang.org/genproto v0.0.0-20250603155806-513f23925822 // indirect\n\tgoogle.golang.org/genproto/googleapis/api v0.0.0-20250721164621-a45f3dfb1074 // indirect\n\tgoogle.golang.org/genproto/googleapis/rpc v0.0.0-20250728155136-f173205681a0 // indirect\n\tgoogle.golang.org/protobuf v1.36.6 // indirect\n)\n" }, { "path": "go.sum", "content": "cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=\ncloud.google.com/go v0.121.4 h1:cVvUiY0sX0xwyxPwdSU2KsF9knOVmtRyAMt8xou0iTs=\ncloud.google.com/go v0.121.4/go.mod h1:XEBchUiHFJbz4lKBZwYBDHV/rSyfFktk737TLDU089s=\ncloud.google.com/go/auth v0.16.3 h1:kabzoQ9/bobUmnseYnBO6qQG7q4a/CffFRlJSxv2wCc=\ncloud.google.com/go/auth v0.16.3/go.mod h1:NucRGjaXfzP1ltpcQ7On/VTZ0H4kWB5Jy+Y9Dnm76fA=\ncloud.google.com/go/auth/oauth2adapt v0.2.8 h1:keo8NaayQZ6wimpNSmW5OPc283g65QNIiLpZnkHRbnc=\ncloud.google.com/go/auth/oauth2adapt v0.2.8/go.mod h1:XQ9y31RkqZCcwJWNSx2Xvric3RrU88hAYYbjDWYDL+c=\ncloud.google.com/go/compute/metadata v0.7.0 h1:PBWF+iiAerVNe8UCHxdOt6eHLVc3ydFeOCw78U8ytSU=\ncloud.google.com/go/compute/metadata v0.7.0/go.mod h1:j5MvL9PprKL39t166CoB1uVHfQMs4tFQZZcKwksXUjo=\ncloud.google.com/go/iam v1.5.2 h1:qgFRAGEmd8z6dJ/qyEchAuL9jpswyODjA2lS+w234g8=\ncloud.google.com/go/iam v1.5.2/go.mod h1:SE1vg0N81zQqLzQEwxL2WI6yhetBdbNQuTvIKCSkUHE=\ncloud.google.com/go/kms v1.22.0 h1:dBRIj7+GDeeEvatJeTB19oYZNV0aj6wEqSIT/7gLqtk=\ncloud.google.com/go/kms v1.22.0/go.mod h1:U7mf8Sva5jpOb4bxYZdtw/9zsbIjrklYwPcvMk34AL8=\ncloud.google.com/go/longrunning v0.6.7 h1:IGtfDWHhQCgCjwQjV9iiLnUta9LBCo8R9QmAFsS/PrE=\ncloud.google.com/go/longrunning v0.6.7/go.mod h1:EAFV3IZAKmM56TyiE6VAP3VoTzhZzySwI/YI1s/nRsY=\ncloud.google.com/go/pubsub v1.50.0 h1:hnYpOIxVlgVD1Z8LN7est4DQZK3K6tvZNurZjIVjUe0=\ncloud.google.com/go/pubsub v1.50.0/go.mod h1:Di2Y+nqXBpIS+dXUEJPQzLh8PbIQZMLE9IVUFhf2zmM=\ncloud.google.com/go/pubsub/v2 v2.0.0 h1:0qS6mRJ41gD1lNmM/vdm6bR7DQu6coQcVwD+VPf0Bz0=\ncloud.google.com/go/pubsub/v2 v2.0.0/go.mod h1:0aztFxNzVQIRSZ8vUr79uH2bS3jwLebwK6q1sgEub+E=\ngithub.com/Azure/azure-amqp-common-go/v4 v4.2.0 h1:q/jLx1KJ8xeI8XGfkOWMN9XrXzAfVTkyvCxPvHCjd2I=\ngithub.com/Azure/azure-amqp-common-go/v4 v4.2.0/go.mod h1:GD3m/WPPma+621UaU6KNjKEo5Hl09z86viKwQjTpV0Q=\ngithub.com/Azure/azure-event-hubs-go/v3 v3.6.2 h1:7rNj1/iqS/i3mUKokA2n2eMYO72TB7lO7OmpbKoakKY=\ngithub.com/Azure/azure-event-hubs-go/v3 v3.6.2/go.mod h1:n+ocYr9j2JCLYqUqz9eI+lx/TEAtL/g6rZzyTFSuIpc=\ngithub.com/Azure/azure-sdk-for-go v65.0.0+incompatible h1:HzKLt3kIwMm4KeJYTdx9EbjRYTySD/t8i1Ee/W5EGXw=\ngithub.com/Azure/azure-sdk-for-go v65.0.0+incompatible/go.mod h1:9XXNKU+eRnpl9moKnB4QOLf1HestfXbmab5FXxiDBjc=\ngithub.com/Azure/go-amqp v1.0.0 h1:QfCugi1M+4F2JDTRgVnRw7PYXLXZ9hmqk3+9+oJh3OA=\ngithub.com/Azure/go-amqp v1.0.0/go.mod h1:+bg0x3ce5+Q3ahCEXnCsGG3ETpDQe3MEVnOuT2ywPwc=\ngithub.com/Azure/go-autorest v14.2.0+incompatible h1:V5VMDjClD3GiElqLWO7mz2MxNAK/vTfRHdAubSIPRgs=\ngithub.com/Azure/go-autorest v14.2.0+incompatible/go.mod h1:r+4oMnoxhatjLLJ6zxSWATqVooLgysK6ZNox3g/xq24=\ngithub.com/Azure/go-autorest/autorest v0.11.28 h1:ndAExarwr5Y+GaHE6VCaY1kyS/HwwGGyuimVhWsHOEM=\ngithub.com/Azure/go-autorest/autorest v0.11.28/go.mod h1:MrkzG3Y3AH668QyF9KRk5neJnGgmhQ6krbhR8Q5eMvA=\ngithub.com/Azure/go-autorest/autorest/adal v0.9.18/go.mod h1:XVVeme+LZwABT8K5Lc3hA4nAe8LDBVle26gTrguhhPQ=\ngithub.com/Azure/go-autorest/autorest/adal v0.9.21 h1:jjQnVFXPfekaqb8vIsv2G1lxshoW+oGv4MDlhRtnYZk=\ngithub.com/Azure/go-autorest/autorest/adal v0.9.21/go.mod h1:zua7mBUaCc5YnSLKYgGJR/w5ePdMDA6H56upLsHzA9U=\ngithub.com/Azure/go-autorest/autorest/azure/auth v0.4.2 h1:iM6UAvjR97ZIeR93qTcwpKNMpV+/FTWjwEbuPD495Tk=\ngithub.com/Azure/go-autorest/autorest/azure/auth v0.4.2/go.mod h1:90gmfKdlmKgfjUpnCEpOJzsUEjrWDSLwHIG73tSXddM=\ngithub.com/Azure/go-autorest/autorest/azure/cli v0.3.1 h1:LXl088ZQlP0SBppGFsRZonW6hSvwgL5gRByMbvUbx8U=\ngithub.com/Azure/go-autorest/autorest/azure/cli v0.3.1/go.mod h1:ZG5p860J94/0kI9mNJVoIoLgXcirM2gF5i2kWloofxw=\ngithub.com/Azure/go-autorest/autorest/date v0.3.0 h1:7gUk1U5M/CQbp9WoqinNzJar+8KY+LPI6wiWrP/myHw=\ngithub.com/Azure/go-autorest/autorest/date v0.3.0/go.mod h1:BI0uouVdmngYNUzGWeSYnokU+TrmwEsOqdt8Y6sso74=\ngithub.com/Azure/go-autorest/autorest/mocks v0.4.1/go.mod h1:LTp+uSrOhSkaKrUy935gNZuuIPPVsHlr9DSOxSayd+k=\ngithub.com/Azure/go-autorest/autorest/mocks v0.4.2 h1:PGN4EDXnuQbojHbU0UWoNvmu9AGVwYHG9/fkDYhtAfw=\ngithub.com/Azure/go-autorest/autorest/mocks v0.4.2/go.mod h1:Vy7OitM9Kei0i1Oj+LvyAWMXJHeKH1MVlzFugfVrmyU=\ngithub.com/Azure/go-autorest/autorest/to v0.4.0 h1:oXVqrxakqqV1UZdSazDOPOLvOIz+XA683u8EctwboHk=\ngithub.com/Azure/go-autorest/autorest/to v0.4.0/go.mod h1:fE8iZBn7LQR7zH/9XU2NcPR4o9jEImooCeWJcYV/zLE=\ngithub.com/Azure/go-autorest/autorest/validation v0.3.1 h1:AgyqjAd94fwNAoTjl/WQXg4VvFeRFpO+UhNyRXqF1ac=\ngithub.com/Azure/go-autorest/autorest/validation v0.3.1/go.mod h1:yhLgjC0Wda5DYXl6JAsWyUe4KVNffhoDhG0zVzUMo3E=\ngithub.com/Azure/go-autorest/logger v0.2.1 h1:IG7i4p/mDa2Ce4TRyAO8IHnVhAVF3RFU+ZtXWSmf4Tg=\ngithub.com/Azure/go-autorest/logger v0.2.1/go.mod h1:T9E3cAhj2VqvPOtCYAvby9aBXkZmbF5NWuPV8+WeEW8=\ngithub.com/Azure/go-autorest/tracing v0.6.0 h1:TYi4+3m5t6K48TGI9AUdb+IzbnSxvnvUMfuitfgcfuo=\ngithub.com/Azure/go-autorest/tracing v0.6.0/go.mod h1:+vhtPC754Xsa23ID7GlGsrdKBpUA79WCAKPPZVC2DeU=\ngithub.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=\ngithub.com/IBM/sarama v1.46.0 h1:+YTM1fNd6WKMchlnLKRUB5Z0qD4M8YbvwIIPLvJD53s=\ngithub.com/IBM/sarama v1.46.0/go.mod h1:0lOcuQziJ1/mBGHkdp5uYrltqQuKQKM5O5FOWUQVVvo=\ngithub.com/aws/aws-sdk-go v1.55.8 h1:JRmEUbU52aJQZ2AjX4q4Wu7t4uZjOu71uyNmaWlUkJQ=\ngithub.com/aws/aws-sdk-go v1.55.8/go.mod h1:ZkViS9AqA6otK+JBBNH2++sx1sgxrPKcSzPPvQkUtXk=\ngithub.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=\ngithub.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=\ngithub.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=\ngithub.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=\ngithub.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=\ngithub.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=\ngithub.com/cloudflare/cloudflare-go/v4 v4.6.0 h1:ZaWwXjHFR5NoY8UEf4QFY0g3KTi72kqqEXpajV610/o=\ngithub.com/cloudflare/cloudflare-go/v4 v4.6.0/go.mod h1:XcYpLe7Mf6FN87kXzEWVnJ6z+vskW/k6eUqgqfhFE9k=\ngithub.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc=\ngithub.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=\ngithub.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=\ngithub.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=\ngithub.com/devigned/tab v0.1.1 h1:3mD6Kb1mUOYeLpJvTVSDwSg5ZsfSxfvxGRTxRsJsITA=\ngithub.com/devigned/tab v0.1.1/go.mod h1:XG9mPq0dFghrYvoBF3xdRrJzSTX1b7IQrvaL9mzjeJY=\ngithub.com/dimchansky/utfbom v1.1.0 h1:FcM3g+nofKgUteL8dm/UpdRXNC9KmADgTpLKsu0TRo4=\ngithub.com/dimchansky/utfbom v1.1.0/go.mod h1:rO41eb7gLfo8SF1jd9F8HplJm1Fewwi4mQvIirEdv+8=\ngithub.com/eapache/go-resiliency v1.7.0 h1:n3NRTnBn5N0Cbi/IeOHuQn9s2UwVUH7Ga0ZWcP+9JTA=\ngithub.com/eapache/go-resiliency v1.7.0/go.mod h1:5yPzW0MIvSe0JDsv0v+DvcjEv2FyD6iZYSs1ZI+iQho=\ngithub.com/eapache/go-xerial-snappy v0.0.0-20230731223053-c322873962e3 h1:Oy0F4ALJ04o5Qqpdz8XLIpNA3WM/iSIXqxtqo7UGVws=\ngithub.com/eapache/go-xerial-snappy v0.0.0-20230731223053-c322873962e3/go.mod h1:YvSRo5mw33fLEx1+DlK6L2VV43tJt5Eyel9n9XBcR+0=\ngithub.com/eapache/queue v1.1.0 h1:YOEu7KNc61ntiQlcEeUIoDTJ2o8mQznoNvUhiigpIqc=\ngithub.com/eapache/queue v1.1.0/go.mod h1:6eCeP0CKFpHLu8blIFXhExK/dRa7WDZfr6jVFPTqq+I=\ngithub.com/eclipse/paho.mqtt.golang v1.5.1 h1:/VSOv3oDLlpqR2Epjn1Q7b2bSTplJIeV2ISgCl2W7nE=\ngithub.com/eclipse/paho.mqtt.golang v1.5.1/go.mod h1:1/yJCneuyOoCOzKSsOTUc0AJfpsItBGWvYpBLimhArU=\ngithub.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=\ngithub.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=\ngithub.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98=\ngithub.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c=\ngithub.com/felixge/httpsnoop v1.0.4 h1:NFTV2Zj1bL4mc9sqWACXbQFVBBg2W3GPvqp8/ESS2Wg=\ngithub.com/felixge/httpsnoop v1.0.4/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U=\ngithub.com/fortytw2/leaktest v1.3.0 h1:u8491cBMTQ8ft8aeV+adlcytMZylmA5nnwwkRZjI8vw=\ngithub.com/fortytw2/leaktest v1.3.0/go.mod h1:jDsjWgpAGjm2CA7WthBh/CdZYEPF31XHquHwclZch5g=\ngithub.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A=\ngithub.com/go-logr/logr v1.4.3 h1:CjnDlHq8ikf6E492q6eKboGOC0T8CDaOvkHCIg8idEI=\ngithub.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=\ngithub.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=\ngithub.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=\ngithub.com/golang-jwt/jwt/v4 v4.0.0/go.mod h1:/xlHOz8bRuivTWchD4jCa+NbatV+wEUSzwAxVc6locg=\ngithub.com/golang-jwt/jwt/v4 v4.2.0/go.mod h1:/xlHOz8bRuivTWchD4jCa+NbatV+wEUSzwAxVc6locg=\ngithub.com/golang-jwt/jwt/v4 v4.5.2 h1:YtQM7lnr8iZ+j5q71MGKkNw9Mn7AjHM68uc9g5fXeUI=\ngithub.com/golang-jwt/jwt/v4 v4.5.2/go.mod h1:m21LjoU+eqJr34lmDMbreY2eSTRJ1cv77w39/MY0Ch0=\ngithub.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=\ngithub.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=\ngithub.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da h1:oI5xCqsCo564l8iNU+DwB5epxmsaqB+rhGL0m5jtYqE=\ngithub.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=\ngithub.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=\ngithub.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=\ngithub.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=\ngithub.com/golang/protobuf v1.4.0-rc.1/go.mod h1:ceaxUfeHdC40wWswd/P6IGgMaK3YpKi5j83Wpe3EHw8=\ngithub.com/golang/protobuf v1.4.0-rc.1.0.20200221234624-67d41d38c208/go.mod h1:xKAWHe0F5eneWXFV3EuXVDTCmh+JuBKY0li0aMyXATA=\ngithub.com/golang/protobuf v1.4.0-rc.2/go.mod h1:LlEzMj4AhA7rCAGe4KMBDvJI+AwstrUpVNzEA03Pprs=\ngithub.com/golang/protobuf v1.4.0-rc.4.0.20200313231945-b860323f09d0/go.mod h1:WU3c8KckQ9AFe+yFwt9sWVRKCVIyN9cPHBJSNnbL67w=\ngithub.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvqG2KuDX0=\ngithub.com/golang/protobuf v1.4.1/go.mod h1:U8fpvMrcmy5pZrNK1lt4xCsGvpyWQ/VVv6QDs8UjoX8=\ngithub.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=\ngithub.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek=\ngithub.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps=\ngithub.com/golang/snappy v0.0.4 h1:yAGX7huGHXlcLOEtBnF4w7FQwA26wojNCwOYAEhLjQM=\ngithub.com/golang/snappy v0.0.4/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=\ngithub.com/gomodule/redigo v1.9.2 h1:HrutZBLhSIU8abiSfW8pj8mPhOyMYjZT/wcA4/L9L9s=\ngithub.com/gomodule/redigo v1.9.2/go.mod h1:KsU3hiK/Ay8U42qpaJk+kuNa3C+spxapWpM+ywhcgtw=\ngithub.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=\ngithub.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=\ngithub.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=\ngithub.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=\ngithub.com/google/go-cmp v0.5.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=\ngithub.com/google/go-cmp v0.5.3/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=\ngithub.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=\ngithub.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=\ngithub.com/google/s2a-go v0.1.9 h1:LGD7gtMgezd8a/Xak7mEWL0PjoTQFvpRudN895yqKW0=\ngithub.com/google/s2a-go v0.1.9/go.mod h1:YA0Ei2ZQL3acow2O62kdp9UlnvMmU7kA6Eutn0dXayM=\ngithub.com/google/uuid v1.1.2/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=\ngithub.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=\ngithub.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=\ngithub.com/googleapis/enterprise-certificate-proxy v0.3.6 h1:GW/XbdyBFQ8Qe+YAmFU9uHLo7OnF5tL52HFAgMmyrf4=\ngithub.com/googleapis/enterprise-certificate-proxy v0.3.6/go.mod h1:MkHOF77EYAE7qfSuSS9PU6g4Nt4e11cnsDUowfwewLA=\ngithub.com/googleapis/gax-go/v2 v2.15.0 h1:SyjDc1mGgZU5LncH8gimWo9lW1DtIfPibOG81vgd/bo=\ngithub.com/googleapis/gax-go/v2 v2.15.0/go.mod h1:zVVkkxAQHa1RQpg9z2AUCMnKhi0Qld9rcmyfL1OZhoc=\ngithub.com/gorilla/securecookie v1.1.1/go.mod h1:ra0sb63/xPlUeL+yeDciTfxMRAA+MP+HVt/4epWDjd4=\ngithub.com/gorilla/sessions v1.2.1/go.mod h1:dk2InVEVJ0sfLlnXv9EAgkf6ecYs/i80K/zI+bUmuGM=\ngithub.com/gorilla/websocket v1.5.3 h1:saDtZ6Pbx/0u+bgYQ3q96pZgCzfhKXGPqt7kZ72aNNg=\ngithub.com/gorilla/websocket v1.5.3/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE=\ngithub.com/hashicorp/go-uuid v1.0.2/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro=\ngithub.com/hashicorp/go-uuid v1.0.3 h1:2gKiV6YVmrJ1i2CKKa9obLvRieoRGviZFL26PcT/Co8=\ngithub.com/hashicorp/go-uuid v1.0.3/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro=\ngithub.com/iwpnd/sectr v0.1.2 h1:FauaPRn5C2tC42HTF7gM3FJZXvGXWc6jabBbIxzTMag=\ngithub.com/iwpnd/sectr v0.1.2/go.mod h1:Dm6YXDJCRx1NTfMX/1RMIkGfVp2ORjCY/cQGfbknz4c=\ngithub.com/jcmturner/aescts/v2 v2.0.0 h1:9YKLH6ey7H4eDBXW8khjYslgyqG2xZikXP0EQFKrle8=\ngithub.com/jcmturner/aescts/v2 v2.0.0/go.mod h1:AiaICIRyfYg35RUkr8yESTqvSy7csK90qZ5xfvvsoNs=\ngithub.com/jcmturner/dnsutils/v2 v2.0.0 h1:lltnkeZGL0wILNvrNiVCR6Ro5PGU/SeBvVO/8c/iPbo=\ngithub.com/jcmturner/dnsutils/v2 v2.0.0/go.mod h1:b0TnjGOvI/n42bZa+hmXL+kFJZsFT7G4t3HTlQ184QM=\ngithub.com/jcmturner/gofork v1.7.6 h1:QH0l3hzAU1tfT3rZCnW5zXl+orbkNMMRGJfdJjHVETg=\ngithub.com/jcmturner/gofork v1.7.6/go.mod h1:1622LH6i/EZqLloHfE7IeZ0uEJwMSUyQ/nDd82IeqRo=\ngithub.com/jcmturner/goidentity/v6 v6.0.1 h1:VKnZd2oEIMorCTsFBnJWbExfNN7yZr3EhJAxwOkZg6o=\ngithub.com/jcmturner/goidentity/v6 v6.0.1/go.mod h1:X1YW3bgtvwAXju7V3LCIMpY0Gbxyjn/mY9zx4tFonSg=\ngithub.com/jcmturner/gokrb5/v8 v8.4.4 h1:x1Sv4HaTpepFkXbt2IkL29DXRf8sOfZXo8eRKh687T8=\ngithub.com/jcmturner/gokrb5/v8 v8.4.4/go.mod h1:1btQEpgT6k+unzCwX1KdWMEwPPkkgBtP+F6aCACiMrs=\ngithub.com/jcmturner/rpc/v2 v2.0.3 h1:7FXXj8Ti1IaVFpSAziCZWNzbNuZmnvw/i6CqLNdWfZY=\ngithub.com/jcmturner/rpc/v2 v2.0.3/go.mod h1:VUJYCIDm3PVOEHw8sgt091/20OJjskO/YJki3ELg/Hc=\ngithub.com/jmespath/go-jmespath v0.4.0 h1:BEgLn5cpjn8UN1mAw4NjwDrS35OdebyEtFe+9YPoQUg=\ngithub.com/jmespath/go-jmespath v0.4.0/go.mod h1:T8mJZnbsbmF+m6zOOFylbeCJqk5+pHWvzYPziyZiYoo=\ngithub.com/jmespath/go-jmespath/internal/testify v1.5.1 h1:shLQSRRSCCPj3f2gpwzGwWFoC7ycTf1rcQZHOlsJ6N8=\ngithub.com/jmespath/go-jmespath/internal/testify v1.5.1/go.mod h1:L3OGu8Wl2/fWfCI6z80xFu9LTZmf1ZRjMHUOPmWr69U=\ngithub.com/joho/godotenv v1.3.0 h1:Zjp+RcGpHhGlrMbJzXTrZZPrWj+1vfm90La1wgB6Bhc=\ngithub.com/joho/godotenv v1.3.0/go.mod h1:7hK45KPybAkOC6peb+G5yklZfMxEjkZhHbwpqxOKXbg=\ngithub.com/jpillora/backoff v1.0.0 h1:uvFg412JmmHBHw7iwprIxkPMI+sGQ4kzOWsMeHnm2EA=\ngithub.com/jpillora/backoff v1.0.0/go.mod h1:J/6gKK9jxlEcS3zixgDgUAsiuZ7yrSoa/FX5e0EB2j4=\ngithub.com/klauspost/compress v1.18.0 h1:c/Cqfb0r+Yi+JtIEq73FWXVkRonBlf0CRNYc8Zttxdo=\ngithub.com/klauspost/compress v1.18.0/go.mod h1:2Pp+KzxcywXVXMr50+X0Q/Lsb43OQHYWRCY2AiWywWQ=\ngithub.com/klauspost/cpuid/v2 v2.0.9 h1:lgaqFMSdTdQYdZ04uHyN2d/eKdOMyi2YLSvlQIBFYa4=\ngithub.com/klauspost/cpuid/v2 v2.0.9/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg=\ngithub.com/konsorten/go-windows-terminal-sequences v1.0.1 h1:mweAR1A6xJ3oS2pRaGiHgQ4OO8tzTaLawm8vnODuwDk=\ngithub.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=\ngithub.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc=\ngithub.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw=\ngithub.com/mattn/go-runewidth v0.0.3 h1:a+kO+98RDGEfo6asOGMmpodZq4FNtnGP54yps8BzLR4=\ngithub.com/mattn/go-runewidth v0.0.3/go.mod h1:LwmH8dsx7+W8Uxz3IHJYH5QSwggIsqBzpuz5H//U1FU=\ngithub.com/mitchellh/go-homedir v1.1.0 h1:lukF9ziXFxDFPkA1vsr5zpc1XuPDn/wFntq5mG+4E0Y=\ngithub.com/mitchellh/go-homedir v1.1.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0=\ngithub.com/mitchellh/mapstructure v1.5.0 h1:jeMsZIYE/09sWLaz43PL7Gy6RuMjD2eJVyuac5Z2hdY=\ngithub.com/mitchellh/mapstructure v1.5.0/go.mod h1:bFUtVrKA4DC2yAKiSyO/QUcy7e+RRV2QTWOzhPopBRo=\ngithub.com/mmcloughlin/geohash v0.10.0 h1:9w1HchfDfdeLc+jFEf/04D27KP7E2QmpDu52wPbJWRE=\ngithub.com/mmcloughlin/geohash v0.10.0/go.mod h1:oNZxQo5yWJh0eMQEP/8hwQuVx9Z9tjwFUqcTB1SmG0c=\ngithub.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA=\ngithub.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=\ngithub.com/nats-io/nats.go v1.44.0 h1:ECKVrDLdh/kDPV1g0gAQ+2+m2KprqZK5O/eJAyAnH2M=\ngithub.com/nats-io/nats.go v1.44.0/go.mod h1:iRWIPokVIFbVijxuMQq4y9ttaBTMe0SFdlZfMDd+33g=\ngithub.com/nats-io/nkeys v0.4.11 h1:q44qGV008kYd9W1b1nEBkNzvnWxtRSQ7A8BoqRrcfa0=\ngithub.com/nats-io/nkeys v0.4.11/go.mod h1:szDimtgmfOi9n25JpfIdGw12tZFYXqhGxjhVxsatHVE=\ngithub.com/nats-io/nuid v1.0.1 h1:5iA8DT8V7q8WK2EScv2padNa/rTESc1KdnPw4TC2paw=\ngithub.com/nats-io/nuid v1.0.1/go.mod h1:19wcPz3Ph3q0Jbyiqsd0kePYG7A95tJPxeL+1OSON2c=\ngithub.com/peterh/liner v1.2.2 h1:aJ4AOodmL+JxOZZEL2u9iJf8omNRpqHc/EbrK+3mAXw=\ngithub.com/peterh/liner v1.2.2/go.mod h1:xFwJyiKIXJZUKItq5dGHZSTBRAuG/CpeNpWLyiNRNwI=\ngithub.com/pierrec/lz4/v4 v4.1.22 h1:cKFw6uJDK+/gfw5BcDL0JL5aBsAFdsIT18eRtLj7VIU=\ngithub.com/pierrec/lz4/v4 v4.1.22/go.mod h1:gZWDp/Ze/IJXGXf23ltt2EXimqmTUXEy0GFuRQyBid4=\ngithub.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=\ngithub.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=\ngithub.com/prometheus/client_golang v1.23.0 h1:ust4zpdl9r4trLY/gSjlm07PuiBq2ynaXXlptpfy8Uc=\ngithub.com/prometheus/client_golang v1.23.0/go.mod h1:i/o0R9ByOnHX0McrTMTyhYvKE4haaf2mW08I+jGAjEE=\ngithub.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=\ngithub.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk=\ngithub.com/prometheus/client_model v0.6.2/go.mod h1:y3m2F6Gdpfy6Ut/GBsUqTWZqCUvMVzSfMLjcu6wAwpE=\ngithub.com/prometheus/common v0.65.0 h1:QDwzd+G1twt//Kwj/Ww6E9FQq1iVMmODnILtW1t2VzE=\ngithub.com/prometheus/common v0.65.0/go.mod h1:0gZns+BLRQ3V6NdaerOhMbwwRbNh9hkGINtQAsP5GS8=\ngithub.com/prometheus/procfs v0.16.1 h1:hZ15bTNuirocR6u0JZ6BAHHmwS1p8B4P6MRqxtzMyRg=\ngithub.com/prometheus/procfs v0.16.1/go.mod h1:teAbpZRB1iIAJYREa1LsoWUXykVXA1KlTmWl8x/U+Is=\ngithub.com/rcrowley/go-metrics v0.0.0-20250401214520-65e299d6c5c9 h1:bsUq1dX0N8AOIL7EB/X911+m4EHsnWEHeJ0c+3TTBrg=\ngithub.com/rcrowley/go-metrics v0.0.0-20250401214520-65e299d6c5c9/go.mod h1:bCqnVzQkZxMG4s8nGwiZ5l3QUCyqpo9Y+/ZMZ9VjZe4=\ngithub.com/sirupsen/logrus v1.2.0 h1:juTguoYk5qI21pwyTXY3B3Y5cOTH3ZUyZCg1v/mihuo=\ngithub.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo=\ngithub.com/streadway/amqp v1.1.0 h1:py12iX8XSyI7aN/3dUT8DFIDJazNJsVJdxNVEpnQTZM=\ngithub.com/streadway/amqp v1.1.0/go.mod h1:WYSrTEYHOXHd0nwFeUXAe2G2hRnQT+deZJJf88uS9Bg=\ngithub.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=\ngithub.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=\ngithub.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=\ngithub.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=\ngithub.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=\ngithub.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=\ngithub.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=\ngithub.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=\ngithub.com/stretchr/testify v1.11.0 h1:ib4sjIrwZKxE5u/Japgo/7SJV3PvgjGiRNAvTVGqQl8=\ngithub.com/stretchr/testify v1.11.0/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=\ngithub.com/tidwall/assert v0.1.0 h1:aWcKyRBUAdLoVebxo95N7+YZVTFF/ASTr7BN4sLP6XI=\ngithub.com/tidwall/assert v0.1.0/go.mod h1:QLYtGyeqse53vuELQheYl9dngGCJQ+mTtlxcktb+Kj8=\ngithub.com/tidwall/btree v1.1.0/go.mod h1:TzIRzen6yHbibdSfK6t8QimqbUnoxUSrZfeW7Uob0q4=\ngithub.com/tidwall/btree v1.8.1 h1:27ehoXvm5AG/g+1VxLS1SD3vRhp/H7LuEfwNvddEdmA=\ngithub.com/tidwall/btree v1.8.1/go.mod h1:jBbTdUWhSZClZWoDg54VnvV7/54modSOzDN7VXftj1A=\ngithub.com/tidwall/buntdb v1.3.2 h1:qd+IpdEGs0pZci37G4jF51+fSKlkuUTMXuHhXL1AkKg=\ngithub.com/tidwall/buntdb v1.3.2/go.mod h1:lZZrZUWzlyDJKlLQ6DKAy53LnG7m5kHyrEHvvcDmBpU=\ngithub.com/tidwall/cities v0.1.0 h1:CVNkmMf7NEC9Bvokf5GoSsArHCKRMTgLuubRTHnH0mE=\ngithub.com/tidwall/cities v0.1.0/go.mod h1:lV/HDp2gCcRcHJWqgt6Di54GiDrTZwh1aG2ZUPNbqa4=\ngithub.com/tidwall/conv v0.1.0 h1:hieA69RrgNAIpirLIymxM1+v0TmuijwU4w9qfeyApHo=\ngithub.com/tidwall/conv v0.1.0/go.mod h1:jvlaM2rq1CHtNznWb3A5i+tkG6k6XYeMUNwPfyEGxIo=\ngithub.com/tidwall/expr v0.14.0 h1:tb2MDhay/Qtorm+UXv3DXdCpMlpsE8Phg1bWhXMM3Dw=\ngithub.com/tidwall/expr v0.14.0/go.mod h1:R5XZxQS2HA/teLqU67CqLZX78FyfvcFoDBDne6VRlaA=\ngithub.com/tidwall/geoindex v1.4.4/go.mod h1:rvVVNEFfkJVWGUdEfU8QaoOg/9zFX0h9ofWzA60mz1I=\ngithub.com/tidwall/geoindex v1.7.0 h1:jtk41sfgwIt8MEDyC3xyKSj75iXXf6rjReJGDNPtR5o=\ngithub.com/tidwall/geoindex v1.7.0/go.mod h1:rvVVNEFfkJVWGUdEfU8QaoOg/9zFX0h9ofWzA60mz1I=\ngithub.com/tidwall/geojson v1.4.6 h1:HpEGer4tc5ieFn8Ts8aTG9fo+hgFJkqfql4O9cgphmg=\ngithub.com/tidwall/geojson v1.4.6/go.mod h1:1cn3UWfSYCJOq53NZoQ9rirdw89+DM0vw+ZOAVvuReg=\ngithub.com/tidwall/gjson v1.12.1/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=\ngithub.com/tidwall/gjson v1.14.2/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=\ngithub.com/tidwall/gjson v1.18.0 h1:FIDeeyB800efLX89e5a8Y0BNH+LOngJyGrIWxG2FKQY=\ngithub.com/tidwall/gjson v1.18.0/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=\ngithub.com/tidwall/grect v0.1.4 h1:dA3oIgNgWdSspFzn1kS4S/RDpZFLrIxAZOdJKjYapOg=\ngithub.com/tidwall/grect v0.1.4/go.mod h1:9FBsaYRaR0Tcy4UwefBX/UDcDcDy9V5jUcxHzv2jd5Q=\ngithub.com/tidwall/hashmap v1.8.1 h1:hXNzBfSJ2Jwvt0lbkWD59O/r3OfatSIcbuWT0VKEVns=\ngithub.com/tidwall/hashmap v1.8.1/go.mod h1:v+0qJrJn7l+l2dB8+fAFpC62p2G0SMP2Teu8ejkebg8=\ngithub.com/tidwall/limiter v0.4.0 h1:nj+7mS6aMDRzp15QTVDrgkun0def5/PfB4ogs5NlIVQ=\ngithub.com/tidwall/limiter v0.4.0/go.mod h1:n+qBGuSOgAvgcq1xUvo+mXWg8oBLQC8wkkheN9KZou0=\ngithub.com/tidwall/lotsa v1.0.2 h1:dNVBH5MErdaQ/xd9s769R31/n2dXavsQ0Yf4TMEHHw8=\ngithub.com/tidwall/lotsa v1.0.2/go.mod h1:X6NiU+4yHA3fE3Puvpnn1XMDrFZrE9JO2/w+UMuqgR8=\ngithub.com/tidwall/match v1.1.1/go.mod h1:eRSPERbgtNPcGhD8UCthc6PmLEQXEWd3PRB5JTxsfmM=\ngithub.com/tidwall/match v1.2.0 h1:0pt8FlkOwjN2fPt4bIl4BoNxb98gGHN2ObFEDkrfZnM=\ngithub.com/tidwall/match v1.2.0/go.mod h1:eRSPERbgtNPcGhD8UCthc6PmLEQXEWd3PRB5JTxsfmM=\ngithub.com/tidwall/mvt v0.2.1 h1:pVoD4/INMeXk8VF8yfMOzn534W4u/0vhSnqyi+/FfG4=\ngithub.com/tidwall/mvt v0.2.1/go.mod h1:RLOVf5y8yfDR4vyA636MiY0kBJeP/i7RxxpZzygREI8=\ngithub.com/tidwall/pretty v1.2.0/go.mod h1:ITEVvHYasfjBbM0u2Pg8T2nJnzm8xPwvNhhsoaGGjNU=\ngithub.com/tidwall/pretty v1.2.1 h1:qjsOFOWWQl+N3RsoF5/ssm1pHmJJwhjlSbZ51I6wMl4=\ngithub.com/tidwall/pretty v1.2.1/go.mod h1:ITEVvHYasfjBbM0u2Pg8T2nJnzm8xPwvNhhsoaGGjNU=\ngithub.com/tidwall/redbench v0.1.0 h1:UZYUMhwMMObQRq5xU4SA3lmlJRztXzqtushDii+AmPo=\ngithub.com/tidwall/redbench v0.1.0/go.mod h1:zxcRGCq/JcqV48YjK9WxBNJL7JSpMzbLXaHvMcnanKQ=\ngithub.com/tidwall/redcon v1.6.2 h1:5qfvrrybgtO85jnhSravmkZyC0D+7WstbfCs3MmPhow=\ngithub.com/tidwall/redcon v1.6.2/go.mod h1:p5Wbsgeyi2VSTBWOcA5vRXrOb9arFTcU2+ZzFjqV75Y=\ngithub.com/tidwall/resp v0.1.1 h1:Ly20wkhqKTmDUPlyM1S7pWo5kk0tDu8OoC/vFArXmwE=\ngithub.com/tidwall/resp v0.1.1/go.mod h1:3/FrruOBAxPTPtundW0VXgmsQ4ZBA0Aw714lVYgwFa0=\ngithub.com/tidwall/rtred v0.1.2 h1:exmoQtOLvDoO8ud++6LwVsAMTu0KPzLTUrMln8u1yu8=\ngithub.com/tidwall/rtred v0.1.2/go.mod h1:hd69WNXQ5RP9vHd7dqekAz+RIdtfBogmglkZSRxCHFQ=\ngithub.com/tidwall/rtree v1.3.1/go.mod h1:S+JSsqPTI8LfWA4xHBo5eXzie8WJLVFeppAutSegl6M=\ngithub.com/tidwall/rtree v1.10.0 h1:+EcI8fboEaW1L3/9oW/6AMoQ8HiEIHyR7bQOGnmz4Mg=\ngithub.com/tidwall/rtree v1.10.0/go.mod h1:iDJQ9NBRtbfKkzZu02za+mIlaP+bjYPnunbSNidpbCQ=\ngithub.com/tidwall/sjson v1.2.4/go.mod h1:098SZ494YoMWPmMO6ct4dcFnqxwj9r/gF0Etp19pSNM=\ngithub.com/tidwall/sjson v1.2.5 h1:kLy8mja+1c9jlljvWTlSazM7cKDRfJuR/bOJhcY5NcY=\ngithub.com/tidwall/sjson v1.2.5/go.mod h1:Fvgq9kS/6ociJEDnK0Fk1cpYF4FIW6ZF7LAe+6jwd28=\ngithub.com/tidwall/tinylru v1.2.1 h1:VgBr72c2IEr+V+pCdkPZUwiQ0KJknnWIYbhxAVkYfQk=\ngithub.com/tidwall/tinylru v1.2.1/go.mod h1:9bQnEduwB6inr2Y7AkBP7JPgCkyrhTV/ZpX0oOOpBI4=\ngithub.com/tidwall/tinyqueue v0.1.1 h1:SpNEvEggbpyN5DIReaJ2/1ndroY8iyEGxPYxoSaymYE=\ngithub.com/tidwall/tinyqueue v0.1.1/go.mod h1:O/QNHwrnjqr6IHItYrzoHAKYhBkLI67Q096fQP5zMYw=\ngithub.com/xdg-go/pbkdf2 v1.0.0 h1:Su7DPu48wXMwC3bs7MCNG+z4FhcyEuz5dlvchbq0B0c=\ngithub.com/xdg-go/pbkdf2 v1.0.0/go.mod h1:jrpuAogTd400dnrH08LKmI/xc1MbPOebTwRqcT5RDeI=\ngithub.com/xdg-go/scram v1.1.2 h1:FHX5I5B4i4hKRVRBCFRxq1iQRej7WO3hhBuJf+UUySY=\ngithub.com/xdg-go/scram v1.1.2/go.mod h1:RT/sEzTbU5y00aCK8UOx6R7YryM0iF1N2MOmC3kKLN4=\ngithub.com/xdg-go/stringprep v1.0.4 h1:XLI/Ng3O1Atzq0oBs3TWm+5ZVgkq2aqdlvP9JtoZ6c8=\ngithub.com/xdg-go/stringprep v1.0.4/go.mod h1:mPGuuIYwz7CmR2bT9j4GbQqutWS1zV24gijq1dTyGkM=\ngithub.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=\ngithub.com/yuin/gopher-lua v1.1.1 h1:kYKnWBjvbNP4XLT3+bPEwAXJx262OhaHDWDVOPjL46M=\ngithub.com/yuin/gopher-lua v1.1.1/go.mod h1:GBR0iDaNXjAgGg9zfCvksxSRnQx76gclCIb7kdAd1Pw=\ngithub.com/zeebo/assert v1.3.0 h1:g7C04CbJuIDKNPFHmsk4hwZDO5O+kntRxzaUoNXj+IQ=\ngithub.com/zeebo/assert v1.3.0/go.mod h1:Pq9JiuJQpG8JLJdtkwrJESF0Foym2/D9XMU5ciN/wJ0=\ngithub.com/zeebo/xxh3 v1.0.2 h1:xZmwmqxHZA8AI603jOQ0tMqmBr9lPeFwGg6d+xy9DC0=\ngithub.com/zeebo/xxh3 v1.0.2/go.mod h1:5NWz9Sef7zIDm2JHfFlcQvNekmcEl9ekUZQQKCYaDcA=\ngo.einride.tech/aip v0.73.0 h1:bPo4oqBo2ZQeBKo4ZzLb1kxYXTY1ysJhpvQyfuGzvps=\ngo.einride.tech/aip v0.73.0/go.mod h1:Mj7rFbmXEgw0dq1dqJ7JGMvYCZZVxmGOR3S4ZcV5LvQ=\ngo.opencensus.io v0.24.0 h1:y73uSU6J157QMP2kn2r30vwW1A2W2WFwSCGnAVxeaD0=\ngo.opencensus.io v0.24.0/go.mod h1:vNK8G9p7aAivkbmorf4v+7Hgx+Zs0yY+0fOtgBfjQKo=\ngo.opentelemetry.io/auto/sdk v1.1.0 h1:cH53jehLUN6UFLY71z+NDOiNJqDdPRaXzTel0sJySYA=\ngo.opentelemetry.io/auto/sdk v1.1.0/go.mod h1:3wSPjt5PWp2RhlCcmmOial7AvC4DQqZb7a7wCow3W8A=\ngo.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.61.0 h1:q4XOmH/0opmeuJtPsbFNivyl7bCt7yRBbeEm2sC/XtQ=\ngo.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.61.0/go.mod h1:snMWehoOh2wsEwnvvwtDyFCxVeDAODenXHtn5vzrKjo=\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.61.0 h1:F7Jx+6hwnZ41NSFTO5q4LYDtJRXBf2PD0rNBkeB/lus=\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.61.0/go.mod h1:UHB22Z8QsdRDrnAtX4PntOl36ajSxcdUMt1sF7Y6E7Q=\ngo.opentelemetry.io/otel v1.36.0 h1:UumtzIklRBY6cI/lllNZlALOF5nNIzJVb16APdvgTXg=\ngo.opentelemetry.io/otel v1.36.0/go.mod h1:/TcFMXYjyRNh8khOAO9ybYkqaDBb/70aVwkNML4pP8E=\ngo.opentelemetry.io/otel/metric v1.36.0 h1:MoWPKVhQvJ+eeXWHFBOPoBOi20jh6Iq2CcCREuTYufE=\ngo.opentelemetry.io/otel/metric v1.36.0/go.mod h1:zC7Ks+yeyJt4xig9DEw9kuUFe5C3zLbVjV2PzT6qzbs=\ngo.opentelemetry.io/otel/sdk v1.36.0 h1:b6SYIuLRs88ztox4EyrvRti80uXIFy+Sqzoh9kFULbs=\ngo.opentelemetry.io/otel/sdk v1.36.0/go.mod h1:+lC+mTgD+MUWfjJubi2vvXWcVxyr9rmlshZni72pXeY=\ngo.opentelemetry.io/otel/sdk/metric v1.36.0 h1:r0ntwwGosWGaa0CrSt8cuNuTcccMXERFwHX4dThiPis=\ngo.opentelemetry.io/otel/sdk/metric v1.36.0/go.mod h1:qTNOhFDfKRwX0yXOqJYegL5WRaW376QbB7P4Pb0qva4=\ngo.opentelemetry.io/otel/trace v1.36.0 h1:ahxWNuqZjpdiFAyrIoQ4GIiAIhxAunQR6MUoKrsNd4w=\ngo.opentelemetry.io/otel/trace v1.36.0/go.mod h1:gQ+OnDZzrybY4k4seLzPAWNwVBBVlF2szhehOBB/tGA=\ngo.uber.org/atomic v1.11.0 h1:ZvwS0R+56ePWxUNi+Atn9dWONBPp/AUETXlHW0DxSjE=\ngo.uber.org/atomic v1.11.0/go.mod h1:LUxbIzbOniOlMKjJjyPfpl4v+PKK2cNJn91OQbhoJI0=\ngo.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=\ngo.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE=\ngo.uber.org/multierr v1.10.0 h1:S0h4aNzvfcFsC3dRF1jLoaov7oRaKqRGC/pUEJ2yvPQ=\ngo.uber.org/multierr v1.10.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y=\ngo.uber.org/zap v1.27.0 h1:aJMhYGrd5QSmlpLMr2MftRKl7t8J8PTZPA732ud/XR8=\ngo.uber.org/zap v1.27.0/go.mod h1:GB2qFLM7cTU87MWRP2mPIjqfIDnGu+VIO4V/SdhGo2E=\ngolang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=\ngolang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=\ngolang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=\ngolang.org/x/crypto v0.0.0-20220722155217-630584e8d5aa/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=\ngolang.org/x/crypto v0.6.0/go.mod h1:OFC/31mSvZgRz0V1QTNCzfAI1aIRzbiufJtkMIlEp58=\ngolang.org/x/crypto v0.45.0 h1:jMBrvKuj23MTlT0bQEOBcAE0mjg8mK9RXFhRH6nyF3Q=\ngolang.org/x/crypto v0.45.0/go.mod h1:XTGrrkGJve7CYK7J8PEww4aY7gM3qMCElcJQ8n8JdX4=\ngolang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=\ngolang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=\ngolang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU=\ngolang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=\ngolang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=\ngolang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=\ngolang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=\ngolang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=\ngolang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=\ngolang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=\ngolang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=\ngolang.org/x/net v0.0.0-20200114155413-6afb5195e5aa/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=\ngolang.org/x/net v0.0.0-20201110031124-69a78807bb2b/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=\ngolang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=\ngolang.org/x/net v0.0.0-20211112202133-69e39bad7dc2/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=\ngolang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=\ngolang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=\ngolang.org/x/net v0.7.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=\ngolang.org/x/net v0.47.0 h1:Mx+4dIFzqraBXUugkia1OOvlD6LemFo1ALMHjrXDOhY=\ngolang.org/x/net v0.47.0/go.mod h1:/jNxtkgq5yWUGYkaZGqo27cfGZ1c5Nen03aYrrKpVRU=\ngolang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=\ngolang.org/x/oauth2 v0.30.0 h1:dnDm7JmhM45NNpd8FDDeLhK6FwqbOf4MLCM9zb1BOHI=\ngolang.org/x/oauth2 v0.30.0/go.mod h1:B++QgG3ZKulg6sRPGD/mqlHQs5rB3Ml9erfeDY7xKlU=\ngolang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.18.0 h1:kr88TuHDroi+UVf+0hZnirlk8o8T+4MrK6mr60WkH/I=\ngolang.org/x/sync v0.18.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI=\ngolang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=\ngolang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=\ngolang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.0.0-20211117180635-dee7805ff2e1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=\ngolang.org/x/sys v0.38.0 h1:3yZWxaJjBmCWXqhN1qh02AkOnCQ1poK6oF+a7xWL6Gc=\ngolang.org/x/sys v0.38.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=\ngolang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=\ngolang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=\ngolang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k=\ngolang.org/x/term v0.37.0 h1:8EGAD0qCmHYZg6J17DvsMy9/wJ7/D/4pV/wfnld5lTU=\ngolang.org/x/term v0.37.0/go.mod h1:5pB4lxRNYYVZuTLmy8oR2BH8dflOR+IbTYFD8fi3254=\ngolang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=\ngolang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=\ngolang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=\ngolang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=\ngolang.org/x/text v0.3.8/go.mod h1:E6s5w1FMmriuDzIBO73fBruAKo1PCIq6d2Q6DHfQ8WQ=\ngolang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=\ngolang.org/x/text v0.31.0 h1:aC8ghyu4JhP8VojJ2lEHBnochRno1sgL6nEi9WGFGMM=\ngolang.org/x/text v0.31.0/go.mod h1:tKRAlv61yKIjGGHX/4tP1LTbc13YSec1pxVEWXzfoeM=\ngolang.org/x/time v0.12.0 h1:ScB/8o8olJvc+CQPWrK3fPZNfh7qgwCrY0zJmoEQLSE=\ngolang.org/x/time v0.12.0/go.mod h1:CDIdPxbZBQxdj6cxyCIdrNogrJKMJ7pr37NYpMcMDSg=\ngolang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=\ngolang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=\ngolang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY=\ngolang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=\ngolang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=\ngolang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=\ngolang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=\ngolang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=\ngolang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=\ngoogle.golang.org/api v0.246.0 h1:H0ODDs5PnMZVZAEtdLMn2Ul2eQi7QNjqM2DIFp8TlTM=\ngoogle.golang.org/api v0.246.0/go.mod h1:dMVhVcylamkirHdzEBAIQWUCgqY885ivNeZYd7VAVr8=\ngoogle.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=\ngoogle.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=\ngoogle.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=\ngoogle.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc=\ngoogle.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013/go.mod h1:NbSheEEYHJ7i3ixzK3sjbqSGDJWnxyFXZblF3eUsNvo=\ngoogle.golang.org/genproto v0.0.0-20250603155806-513f23925822 h1:rHWScKit0gvAPuOnu87KpaYtjK5zBMLcULh7gxkCXu4=\ngoogle.golang.org/genproto v0.0.0-20250603155806-513f23925822/go.mod h1:HubltRL7rMh0LfnQPkMH4NPDFEWp0jw3vixw7jEM53s=\ngoogle.golang.org/genproto/googleapis/api v0.0.0-20250721164621-a45f3dfb1074 h1:mVXdvnmR3S3BQOqHECm9NGMjYiRtEvDYcqAqedTXY6s=\ngoogle.golang.org/genproto/googleapis/api v0.0.0-20250721164621-a45f3dfb1074/go.mod h1:vYFwMYFbmA8vl6Z/krj/h7+U/AqpHknwJX4Uqgfyc7I=\ngoogle.golang.org/genproto/googleapis/rpc v0.0.0-20250728155136-f173205681a0 h1:MAKi5q709QWfnkkpNQ0M12hYJ1+e8qYVDyowc4U1XZM=\ngoogle.golang.org/genproto/googleapis/rpc v0.0.0-20250728155136-f173205681a0/go.mod h1:qQ0YXyHHx3XkvlzUtpXDkS29lDSafHMZBAZDc03LQ3A=\ngoogle.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=\ngoogle.golang.org/grpc v1.23.0/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg=\ngoogle.golang.org/grpc v1.25.1/go.mod h1:c3i+UQWmh7LiEpx4sFZnkU36qjEYZ0imhYfXVyQciAY=\ngoogle.golang.org/grpc v1.27.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=\ngoogle.golang.org/grpc v1.33.2/go.mod h1:JMHMWHQWaTccqQQlmk3MJZS+GWXOdAesneDmEnv2fbc=\ngoogle.golang.org/grpc v1.74.2 h1:WoosgB65DlWVC9FqI82dGsZhWFNBSLjQ84bjROOpMu4=\ngoogle.golang.org/grpc v1.74.2/go.mod h1:CtQ+BGjaAIXHs/5YS3i473GqwBBa1zGQNevxdeBEXrM=\ngoogle.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8=\ngoogle.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0=\ngoogle.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM=\ngoogle.golang.org/protobuf v1.20.1-0.20200309200217-e05f789c0967/go.mod h1:A+miEFZTKqfCUM6K7xSMQL9OKL/b6hQv+e19PK+JZNE=\ngoogle.golang.org/protobuf v1.21.0/go.mod h1:47Nbq4nVaFHyn7ilMalzfO3qCViNmqZ2kzikPIcrTAo=\ngoogle.golang.org/protobuf v1.22.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=\ngoogle.golang.org/protobuf v1.23.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=\ngoogle.golang.org/protobuf v1.23.1-0.20200526195155-81db48ad09cc/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=\ngoogle.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlbajtzgsN7c=\ngoogle.golang.org/protobuf v1.36.6 h1:z1NpPI8ku2WgiWnf+t9wTPsn6eP1L7ksHUlkfLvd9xY=\ngoogle.golang.org/protobuf v1.36.6/go.mod h1:jduwjTPXsFjZGTmRluh+L6NjiWu7pchiJ2/5YcXBHnY=\ngopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=\ngopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=\ngopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=\ngopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY=\ngopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=\ngopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=\ngopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=\ngopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=\nhonnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=\nhonnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=\nlayeh.com/gopher-json v0.0.0-20201124131017-552bb3c4c3bf h1:rRz0YsF7VXj9fXRF6yQgFI7DzST+hsI3TeFSGupntu0=\nlayeh.com/gopher-json v0.0.0-20201124131017-552bb3c4c3bf/go.mod h1:ivKkcY8Zxw5ba0jldhZCYYQfGdb2K6u9tbYK1AwMIBc=\n" }, { "path": "internal/bing/bing.go", "content": "// https://msdn.microsoft.com/en-us/library/bb259689.aspx\n\npackage bing\n\nimport \"math\"\n\nconst (\n\t// EarthRadius is the radius of the earth\n\tEarthRadius = 6378137.0\n\t// MinLatitude is the min lat\n\tMinLatitude = -85.05112878\n\t// MaxLatitude is the max lat\n\tMaxLatitude = 85.05112878\n\t// MinLongitude is the min lon\n\tMinLongitude = -180.0\n\t// MaxLongitude is the max lon\n\tMaxLongitude = 180.0\n\t// TileSize is the size of a tile\n\tTileSize = 256\n\t// MaxLevelOfDetail is the max level of detail\n\tMaxLevelOfDetail = 38\n)\n\n// Clips a number to the specified minimum and maximum values.\n// Param 'n' is the number to clip.\n// Param 'minValue' is the minimum allowable value.\n// Param 'maxValue' is the maximum allowable value.\n// Returns the clipped value.\nfunc clip(n, minValue, maxValue float64) float64 {\n\tif n < minValue {\n\t\treturn minValue\n\t}\n\tif n > maxValue {\n\t\treturn maxValue\n\t}\n\treturn n\n}\n\n// MapSize determines the map width and height (in pixels) at a specified level of detail.\n// Param 'levelOfDetail' is the level of detail, from 1 (lowest detail) to N (highest detail).\n// Returns the map width and height in pixels.\nfunc MapSize(levelOfDetail uint64) uint64 {\n\treturn TileSize << levelOfDetail\n}\n\n// // Determines the ground resolution (in meters per pixel) at a specified latitude and level of detail.\n// // Param 'latitude' is the Latitude (in degrees) at which to measure the ground resolution.\n// // Param 'levelOfDetail' is the Level of detail, from 1 (lowest detail) to N (highest detail).\n// // Returns the ground resolution, in meters per pixel.\n// func GroundResolution(latitude float64, levelOfDetail uint64) float64 {\n// \tlatitude = clip(latitude, MinLatitude, MaxLatitude)\n// \treturn math.Cos(latitude*math.Pi/180) * 2 * math.Pi * EarthRadius / float64(MapSize(levelOfDetail))\n// }\n\n// // Determines the map scale at a specified latitude, level of detail, and screen resolution.\n// // Param 'latitude' is the latitude (in degrees) at which to measure the map scale.\n// // Param 'levelOfDetail' is the level of detail, from 1 (lowest detail) to N (highest detail).\n// // Param 'screenDpi' is the resolution of the screen, in dots per inch.\n// // Returns the map scale, expressed as the denominator N of the ratio 1 : N.\n// func MapScale(latitude float64, levelOfDetail, screenDpi uint64) float64 {\n// \treturn GroundResolution(latitude, levelOfDetail) * float64(screenDpi) / 0.0254\n// }\n\n// LatLongToPixelXY converts a point from latitude/longitude WGS-84 coordinates (in degrees) into pixel XY coordinates at a specified level of detail.\n// Param 'latitude' is the latitude of the point, in degrees.\n// Param 'longitude' is the longitude of the point, in degrees.\n// Param 'levelOfDetail' is the level of detail, from 1 (lowest detail) to N (highest detail).\n// Return value 'pixelX' is the output parameter receiving the X coordinate in pixels.\n// Return value 'pixelY' is the output parameter receiving the Y coordinate in pixels.\nfunc LatLongToPixelXY(latitude, longitude float64, levelOfDetail uint64) (pixelX, pixelY int64) {\n\tlatitude = clip(latitude, MinLatitude, MaxLatitude)\n\tlongitude = clip(longitude, MinLongitude, MaxLongitude)\n\tx := (longitude + 180) / 360\n\tsinLatitude := math.Sin(latitude * math.Pi / 180)\n\ty := 0.5 - math.Log((1+sinLatitude)/(1-sinLatitude))/(4*math.Pi)\n\tmapSize := float64(MapSize(levelOfDetail))\n\tpixelX = int64(clip(x*mapSize+0.5, 0, mapSize-1))\n\tpixelY = int64(clip(y*mapSize+0.5, 0, mapSize-1))\n\treturn\n}\n\n// PixelXYToLatLong converts a pixel from pixel XY coordinates at a specified level of detail into latitude/longitude WGS-84 coordinates (in degrees).\n// Param 'pixelX' is the X coordinate of the point, in pixels.\n// Param 'pixelY' is the Y coordinates of the point, in pixels.\n// Param 'levelOfDetail' is the level of detail, from 1 (lowest detail) to N (highest detail).\n// Return value 'latitude' is the output parameter receiving the latitude in degrees.\n// Return value 'longitude' is the output parameter receiving the longitude in degrees.\nfunc PixelXYToLatLong(pixelX, pixelY int64, levelOfDetail uint64) (latitude, longitude float64) {\n\tmapSize := float64(MapSize(levelOfDetail))\n\tx := (clip(float64(pixelX), 0, mapSize-1) / mapSize) - 0.5\n\ty := 0.5 - (clip(float64(pixelY), 0, mapSize-1) / mapSize)\n\tlatitude = 90 - 360*math.Atan(math.Exp(-y*2*math.Pi))/math.Pi\n\tlongitude = 360 * x\n\treturn\n}\n\n// PixelXYToTileXY converts pixel XY coordinates into tile XY coordinates of the tile containing the specified pixel.\n// Param 'pixelX' is the pixel X coordinate.\n// Param 'pixelY' is the pixel Y coordinate.\n// Return value 'tileX' is the output parameter receiving the tile X coordinate.\n// Return value 'tileY' is the output parameter receiving the tile Y coordinate.\nfunc PixelXYToTileXY(pixelX, pixelY int64) (tileX, tileY int64) {\n\treturn pixelX >> 8, pixelY >> 8\n}\n\n// TileXYToPixelXY converts tile XY coordinates into pixel XY coordinates of the upper-left pixel of the specified tile.\n// Param 'tileX' is the tile X coordinate.\n// Param 'tileY' is the tile Y coordinate.\n// Return value 'pixelX' is the output parameter receiving the pixel X coordinate.\n// Return value 'pixelY' is the output parameter receiving the pixel Y coordinate.\nfunc TileXYToPixelXY(tileX, tileY int64) (pixelX, pixelY int64) {\n\treturn tileX << 8, tileY << 8\n}\n\n// TileXYToQuadKey converts tile XY coordinates into a QuadKey at a specified level of detail.\n// Param 'tileX' is the tile X coordinate.\n// Param 'tileY' is the tile Y coordinate.\n// Param 'levelOfDetail' is the Level of detail, from 1 (lowest detail) to N (highest detail).\n// Returns a string containing the QuadKey.\nfunc TileXYToQuadKey(tileX, tileY int64, levelOfDetail uint64) string {\n\tquadKey := make([]byte, levelOfDetail)\n\tfor i, j := levelOfDetail, 0; i > 0; i, j = i-1, j+1 {\n\t\tmask := int64(1 << (i - 1))\n\t\tif (tileX & mask) != 0 {\n\t\t\tif (tileY & mask) != 0 {\n\t\t\t\tquadKey[j] = '3'\n\t\t\t} else {\n\t\t\t\tquadKey[j] = '1'\n\t\t\t}\n\t\t} else if (tileY & mask) != 0 {\n\t\t\tquadKey[j] = '2'\n\t\t} else {\n\t\t\tquadKey[j] = '0'\n\t\t}\n\t}\n\treturn string(quadKey)\n}\n\n// QuadKeyToTileXY converts a QuadKey into tile XY coordinates.\n// Param 'quadKey' is the quadKey of the tile.\n// Return value 'tileX' is the output parameter receiving the tile X coordinate.\n// Return value 'tileY is the output parameter receiving the tile Y coordinate.\n// Return value 'levelOfDetail' is the output parameter receiving the level of detail.\nfunc QuadKeyToTileXY(quadKey string) (tileX, tileY int64, levelOfDetail uint64) {\n\tlevelOfDetail = uint64(len(quadKey))\n\tfor i := levelOfDetail; i > 0; i-- {\n\t\tmask := int64(1 << (i - 1))\n\t\tswitch quadKey[levelOfDetail-i] {\n\t\tcase '0':\n\t\tcase '1':\n\t\t\ttileX |= mask\n\t\tcase '2':\n\t\t\ttileY |= mask\n\t\tcase '3':\n\t\t\ttileX |= mask\n\t\t\ttileY |= mask\n\t\tdefault:\n\t\t\tpanic(\"Invalid QuadKey digit sequence.\")\n\t\t}\n\t}\n\treturn\n}\n" }, { "path": "internal/bing/bing_test.go", "content": "package bing\n\nimport (\n\t\"math/rand\"\n\t\"testing\"\n\t\"time\"\n)\n\nfunc TestLevelFuzz(t *testing.T) {\n\trand.Seed(time.Now().UnixNano())\n\tfor i := 0; i < 10000; i++ {\n\t\tlevel := (rand.Int() % MaxLevelOfDetail) + 1\n\t\tquad := \"\"\n\t\tfor j := 0; j < level; j++ {\n\t\t\tquad += string(byte(rand.Int()%4) + '0')\n\t\t}\n\t\ttileX, tileY, levelOfDetail := QuadKeyToTileXY(quad)\n\t\tif levelOfDetail != uint64(len(quad)) {\n\t\t\tt.Fatalf(\"[%d,%d] levelOfDetail == %d, expect %d\", i, level, levelOfDetail, len(quad))\n\t\t}\n\t\tpixelX, pixelY := TileXYToPixelXY(tileX, tileY)\n\t\tlatitude, longitude := PixelXYToLatLong(pixelX, pixelY, levelOfDetail)\n\t\tpixelX2, pixelY2 := LatLongToPixelXY(latitude, longitude, levelOfDetail)\n\t\tif pixelX2 != pixelX {\n\t\t\tt.Fatalf(\"[%d,%d] pixelX2 == %d, expect %d\", i, level, pixelX2, pixelX)\n\t\t}\n\t\tif pixelY2 != pixelY {\n\t\t\tt.Fatalf(\"[%d,%d] pixelY2 == %d, expect %d\", i, level, pixelY2, pixelY)\n\t\t}\n\t\ttileX2, tileY2 := PixelXYToTileXY(pixelX2, pixelY2)\n\t\tif tileX2 != tileX {\n\t\t\tt.Fatalf(\"[%d,%d] tileX2 == %d, expect %d\", i, level, tileX2, tileX)\n\t\t}\n\t\tif tileY2 != tileY {\n\t\t\tt.Fatalf(\"[%d,%d] tileY2 == %d, expect %d\", i, level, tileY2, tileY)\n\t\t}\n\t\tquad2 := TileXYToQuadKey(tileX2, tileY2, levelOfDetail)\n\t\tif quad2 != quad {\n\t\t\tt.Fatalf(\"[%d,%d] quad2 == %s, expect %s\", i, level, quad2, quad)\n\t\t}\n\t}\n}\n\nfunc TestInvalidQuadKeyFuzz(t *testing.T) {\n\trand.Seed(time.Now().UnixNano())\n\tfor i := 0; i < 10000; i++ {\n\t\tfunc() {\n\t\t\tdefer func() {\n\t\t\t\tvar s string\n\t\t\t\tif v := recover(); v != nil {\n\t\t\t\t\ts = v.(string)\n\t\t\t\t}\n\t\t\t\tif s != \"Invalid QuadKey digit sequence.\" {\n\t\t\t\t\tt.Fatalf(\"s == '%s', expect '%s\", s, \"Invalid QuadKey digit sequence.\")\n\t\t\t\t}\n\t\t\t}()\n\t\t\tlevel := (rand.Int() % MaxLevelOfDetail) + 1\n\n\t\t\tvalid := true\n\t\t\tquad := \"\"\n\t\t\tfor valid {\n\t\t\t\tquad = \"\"\n\t\t\t\tfor j := 0; j < level; j++ {\n\t\t\t\t\tc := byte(rand.Int()%5) + '0'\n\t\t\t\t\tquad += string(c)\n\t\t\t\t\tif c < '0' || c > '3' {\n\t\t\t\t\t\tvalid = false\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t\tQuadKeyToTileXY(quad)\n\t\t}()\n\t}\n}\n\nfunc TestLatLonClippingFuzz(t *testing.T) {\n\trand.Seed(time.Now().UnixNano())\n\tfor i := 0; i < 10000; i++ {\n\t\tlat := clip(rand.Float64()*180.0-90.0, MinLatitude, MaxLatitude)\n\t\tlon := clip(rand.Float64()*380.0-180.0, MinLongitude, MaxLongitude)\n\t\tif lat < MinLatitude {\n\t\t\tt.Fatalf(\"lat == %f, expect < %f\", lat, MinLatitude)\n\t\t}\n\t\tif lat > MaxLatitude {\n\t\t\tt.Fatalf(\"lat == %f, expect > %f\", lat, MaxLatitude)\n\t\t}\n\t\tif lon < MinLongitude {\n\t\t\tt.Fatalf(\"lon == %f, expect < %f\", lon, MinLongitude)\n\t\t}\n\t\tif lon > MaxLongitude {\n\t\t\tt.Fatalf(\"lon == %f, expect > %f\", lon, MaxLongitude)\n\t\t}\n\t}\n}\n\nfunc TestIssue302(t *testing.T) {\n\t// Requesting tile with zoom level > 63 crashes the server #302\n\tfor z := uint64(0); z < 256; z++ {\n\t\ttileX, tileY := PixelXYToTileXY(LatLongToPixelXY(33, -115, z))\n\t\tTileXYToBounds(tileX, tileY, z)\n\t}\n}\n" }, { "path": "internal/bing/ext.go", "content": "package bing\n\nimport \"errors\"\n\n// LatLongToQuad iterates through all of the quads parts until levelOfDetail is reached.\nfunc LatLongToQuad(latitude, longitude float64, levelOfDetail uint64, iterator func(part int) bool) {\n\tpixelX, pixelY := LatLongToPixelXY(latitude, longitude, levelOfDetail)\n\ttileX, tileY := PixelXYToTileXY(pixelX, pixelY)\n\tfor i := levelOfDetail; i > 0; i-- {\n\t\tif !iterator(partForTileXY(tileX, tileY, i)) {\n\t\t\tbreak\n\t\t}\n\t}\n}\n\nfunc partForTileXY(tileX, tileY int64, levelOfDetail uint64) int {\n\tmask := int64(1 << (levelOfDetail - 1))\n\tif (tileX & mask) != 0 {\n\t\tif (tileY & mask) != 0 {\n\t\t\treturn 3\n\t\t}\n\t\treturn 1\n\t} else if (tileY & mask) != 0 {\n\t\treturn 2\n\t}\n\treturn 0\n}\n\n// TileXYToBounds returns the bounds around a tile.\nfunc TileXYToBounds(tileX, tileY int64, levelOfDetail uint64) (minLat, minLon, maxLat, maxLon float64) {\n\tsize := int64(1 << levelOfDetail)\n\tpixelX, pixelY := TileXYToPixelXY(tileX, tileY)\n\tmaxLat, minLon = PixelXYToLatLong(pixelX, pixelY, levelOfDetail)\n\tpixelX, pixelY = TileXYToPixelXY(tileX+1, tileY+1)\n\tminLat, maxLon = PixelXYToLatLong(pixelX, pixelY, levelOfDetail)\n\tif size == 0 || tileX%size == 0 {\n\t\tminLon = MinLongitude\n\t}\n\tif size == 0 || tileX%size == size-1 {\n\t\tmaxLon = MaxLongitude\n\t}\n\tif tileY <= 0 {\n\t\tmaxLat = MaxLatitude\n\t}\n\tif tileY >= size-1 {\n\t\tminLat = MinLatitude\n\t}\n\treturn\n}\n\n// QuadKeyToBounds converts a quadkey to bounds\nfunc QuadKeyToBounds(quadkey string) (minLat, minLon, maxLat, maxLon float64, err error) {\n\tfor i := 0; i < len(quadkey); i++ {\n\t\tswitch quadkey[i] {\n\t\tcase '0', '1', '2', '3':\n\t\tdefault:\n\t\t\terr = errors.New(\"invalid quadkey\")\n\t\t\treturn\n\t\t}\n\t}\n\tminLat, minLon, maxLat, maxLon = TileXYToBounds(QuadKeyToTileXY(quadkey))\n\treturn\n}\n" }, { "path": "internal/bing/ext_test.go", "content": "package bing\n\nimport (\n\t\"math/rand\"\n\t\"testing\"\n\t\"time\"\n)\n\nfunc TestIteratorFuzz(t *testing.T) {\n\trand.Seed(time.Now().UnixNano())\n\tfor i := 0; i < 10000; i++ {\n\t\tlatitude := rand.Float64()*180.0 - 90.0\n\t\tlongitude := rand.Float64()*380.0 - 180.0\n\t\tlevelOfDetail := uint64((rand.Int() % MaxLevelOfDetail) + 1)\n\t\tpixelX, pixelY := LatLongToPixelXY(latitude, longitude, levelOfDetail)\n\t\ttileX, tileY := PixelXYToTileXY(pixelX, pixelY)\n\t\tquad1 := TileXYToQuadKey(tileX, tileY, levelOfDetail)\n\t\tl := rand.Int() % len(quad1)\n\t\ti := 0\n\t\tquad2 := \"\"\n\t\tLatLongToQuad(latitude, longitude, levelOfDetail, func(part int) bool {\n\t\t\tif i == l {\n\t\t\t\treturn false\n\t\t\t}\n\t\t\tquad2 += string(byte(part) + '0')\n\t\t\ti++\n\t\t\treturn true\n\t\t})\n\t\tif quad2 != quad1[:l] {\n\t\t\tt.Fatalf(\"[%d,%d] quad2 == %s, expect %s\", i, levelOfDetail, quad2, quad1[:l])\n\t\t}\n\t}\n}\n\nfunc TestExt(t *testing.T) {\n\t// tileX, tileY, levelOfDetail := int64(0), int64(0), uint64(0)\n\t// parts := strings.Split(os.Getenv(\"TEST_TILE\"), \",\")\n\t// if len(parts) == 3 {\n\t// \ttileX, _ = strconv.ParseInt(parts[0], 10, 64)\n\t// \ttileY, _ = strconv.ParseInt(parts[1], 10, 64)\n\t// \tlevelOfDetail, _ = strconv.ParseUint(parts[2], 10, 64)\n\t// }\n\t// minLat, minLon, maxLat, maxLon := TileXYToBounds(tileX, tileY, levelOfDetail)\n\t// fmt.Printf(\"\\x1b[32m== Tile Boundaries ==\\x1b[0m\\n\")\n\t// fmt.Printf(\"\\x1b[31m%d,%d,%d\\x1b[0m\\n\", tileX, tileY, levelOfDetail)\n\t// fmt.Printf(\"\\x1b[31mWGS84 datum (longitude/latitude):\\x1b[0m\\n\")\n\t// fmt.Printf(\"%v %v\\n%v %v\\n\\n\", minLon, minLat, maxLon, maxLat)\n\n\t//fmt.Printf(\"\\x1b[32m\\x1b[0m\\n%v %v\\n%v %v\\n\\n\", minLon, minLat, maxLon, maxLat)\n\t// minLat, minLon, maxLat, maxLon = TileXYToBounds(1, 0, 1)\n\t// fmt.Printf(\"\\x1b[32m1,0\\x1b[0m\\n%v %v\\n%v %v\\n\\n\", minLon, minLat, maxLon, maxLat)\n\t// minLat, minLon, maxLat, maxLon = TileXYToBounds(0, 1, 1)\n\t// fmt.Printf(\"\\x1b[32m0,1\\x1b[0m\\n%v %v\\n%v %v\\n\\n\", minLon, minLat, maxLon, maxLat)\n\t// minLat, minLon, maxLat, maxLon = TileXYToBounds(1, 1, 1)\n\t// fmt.Printf(\"\\x1b[32m1,1\\x1b[0m\\n%v %v\\n%v %v\\n\\n\", minLon, minLat, maxLon, maxLat)\n\n\t// minLat, minLon, maxLat, maxLon = TileXYToBounds(1, 0, 1)\n\t// fmt.Printf(\"1,0: %f,%f %f,%f\\n\", minLat, minLon, maxLat, maxLon)\n\t// minLat, minLon, maxLat, maxLon = TileXYToBounds(0, 1, 1)\n\t// fmt.Printf(\"0,1: %f,%f %f,%f\\n\", minLat, minLon, maxLat, maxLon)\n\t// minLat, minLon, maxLat, maxLon = TileXYToBounds(1, 1, 1)\n\t// fmt.Printf(\"1,1: %f,%f %f,%f\\n\", minLat, minLon, maxLat, maxLon)\n}\n" }, { "path": "internal/buffer/buffer.go", "content": "package buffer\n\nimport (\n\t\"errors\"\n\t\"math\"\n\n\t\"github.com/tidwall/geojson\"\n\t\"github.com/tidwall/geojson/geo\"\n\t\"github.com/tidwall/geojson/geometry\"\n\t\"github.com/tidwall/gjson\"\n)\n\n// TODO: detect of pole and antimeridian crossing and generate\n// valid multigeometries\n\nconst bufferSteps = 15\n\n// Simple performs a very simple buffer operation on a geojson object.\nfunc Simple(g geojson.Object, meters float64) (geojson.Object, error) {\n\tif meters <= 0 {\n\t\treturn g, nil\n\t}\n\tif math.IsInf(meters, 0) || math.IsNaN(meters) {\n\t\treturn g, errors.New(\"invalid meters\")\n\t}\n\tswitch g := g.(type) {\n\tcase *geojson.Point:\n\t\treturn bufferSimplePoint(g.Base(), meters), nil\n\tcase *geojson.SimplePoint:\n\t\treturn bufferSimplePoint(g.Base(), meters), nil\n\tcase *geojson.MultiPoint:\n\t\treturn bufferSimpleGeometries(g.Base(), meters)\n\tcase *geojson.LineString:\n\t\treturn bufferSimpleLineString(g, meters)\n\tcase *geojson.MultiLineString:\n\t\treturn bufferSimpleGeometries(g.Base(), meters)\n\tcase *geojson.Polygon:\n\t\treturn bufferSimplePolygon(g, meters)\n\tcase *geojson.MultiPolygon:\n\t\treturn bufferSimpleGeometries(g.Base(), meters)\n\tcase *geojson.FeatureCollection:\n\t\treturn bufferSimpleFeatures(g.Base(), meters)\n\tcase *geojson.Feature:\n\t\tbg, err := Simple(g.Base(), meters)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\treturn geojson.NewFeature(bg, g.Members()), nil\n\tcase *geojson.Circle:\n\t\treturn Simple(g.Polygon(), meters)\n\tcase nil:\n\t\treturn nil, errors.New(\"cannot buffer nil object\")\n\tdefault:\n\t\ttyp := gjson.Get(g.JSON(), \"type\").String()\n\t\treturn nil, errors.New(\"cannot buffer \" + typ + \" type\")\n\t}\n}\n\nfunc bufferSimplePoint(p geometry.Point, meters float64) *geojson.Polygon {\n\tmeters = geo.NormalizeDistance(meters)\n\tpoints := make([]geometry.Point, 0, bufferSteps+1)\n\n\t// calc the four corners\n\tmaxY, _ := geo.DestinationPoint(p.Y, p.X, meters, 0)\n\t_, maxX := geo.DestinationPoint(p.Y, p.X, meters, 90)\n\tminY, _ := geo.DestinationPoint(p.Y, p.X, meters, 180)\n\t_, minX := geo.DestinationPoint(p.Y, p.X, meters, 270)\n\n\t// use the half width of the lat and lon\n\tlons := (maxX - minX) / 2\n\tlats := (maxY - minY) / 2\n\n\t// generate the circle polygon\n\tfor th := 0.0; th <= 360.0; th += 360.0 / float64(bufferSteps) {\n\t\tradians := (math.Pi / 180) * th\n\t\tx := p.X + lons*math.Cos(radians)\n\t\ty := p.Y + lats*math.Sin(radians)\n\t\tpoints = append(points, geometry.Point{X: x, Y: y})\n\t}\n\t// add last connecting point, make a total of steps+1\n\tpoints = append(points, points[0])\n\tpoly := geojson.NewPolygon(\n\t\tgeometry.NewPoly(points, nil, &geometry.IndexOptions{\n\t\t\tKind: geometry.None,\n\t\t}),\n\t)\n\treturn poly\n}\n\nfunc bufferSimpleGeometries(objs []geojson.Object, meters float64,\n) (*geojson.GeometryCollection, error) {\n\tgeoms := make([]geojson.Object, len(objs))\n\tfor i := 0; i < len(objs); i++ {\n\t\tg, err := Simple(objs[i], meters)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\tgeoms[i] = g\n\t}\n\treturn geojson.NewGeometryCollection(geoms), nil\n}\n\nfunc bufferSimpleFeatures(objs []geojson.Object, meters float64,\n) (*geojson.FeatureCollection, error) {\n\tgeoms := make([]geojson.Object, len(objs))\n\tfor i := 0; i < len(objs); i++ {\n\t\tg, err := Simple(objs[i], meters)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\tgeoms[i] = g\n\t}\n\treturn geojson.NewFeatureCollection(geoms), nil\n}\n\n// appendBufferSimpleSeries buffers a series and appends its parts to dst\nfunc appendBufferSimpleSeries(dst []geojson.Object, s geometry.Series, meters float64) []geojson.Object {\n\tnsegs := s.NumSegments()\n\tfor i := 0; i < nsegs; i++ {\n\t\tdst = appendSimpleBufferSegment(dst, s.SegmentAt(i), meters, i == 0)\n\t}\n\treturn dst\n}\n\n// appendSimpleBufferSegment buffers a segment and appends its parts to dst\nfunc appendSimpleBufferSegment(dst []geojson.Object, seg geometry.Segment,\n\tmeters float64, first bool,\n) []geojson.Object {\n\tif first {\n\t\t// endcap A\n\t\tdst = append(dst, bufferSimplePoint(seg.A, meters))\n\t}\n\t// line polygon\n\tbear1 := geo.BearingTo(seg.A.Y, seg.A.X, seg.B.Y, seg.B.X)\n\tlat1, lon1 := geo.DestinationPoint(seg.A.Y, seg.A.X, meters, bear1-90)\n\tlat2, lon2 := geo.DestinationPoint(seg.A.Y, seg.A.X, meters, bear1+90)\n\tbear2 := geo.BearingTo(seg.B.Y, seg.B.X, seg.A.Y, seg.A.X)\n\tlat3, lon3 := geo.DestinationPoint(seg.B.Y, seg.B.X, meters, bear2-90)\n\tlat4, lon4 := geo.DestinationPoint(seg.B.Y, seg.B.X, meters, bear2+90)\n\tdst = append(dst, geojson.NewPolygon(\n\t\tgeometry.NewPoly([]geometry.Point{\n\t\t\t{X: lon1, Y: lat1},\n\t\t\t{X: lon2, Y: lat2},\n\t\t\t{X: lon3, Y: lat3},\n\t\t\t{X: lon4, Y: lat4},\n\t\t\t{X: lon1, Y: lat1},\n\t\t}, nil, nil)))\n\t// endcap B\n\tdst = append(dst, bufferSimplePoint(seg.B, meters))\n\treturn dst\n}\n\nfunc bufferSimplePolygon(p *geojson.Polygon, meters float64,\n) (*geojson.GeometryCollection, error) {\n\tvar geoms []geojson.Object\n\tb := p.Base()\n\tgeoms = appendBufferSimpleSeries(geoms, b.Exterior, meters)\n\tfor _, hole := range b.Holes {\n\t\tgeoms = appendBufferSimpleSeries(geoms, hole, meters)\n\t}\n\tgeoms = append(geoms, p)\n\treturn geojson.NewGeometryCollection(geoms), nil\n}\n\nfunc bufferSimpleLineString(l *geojson.LineString, meters float64,\n) (*geojson.GeometryCollection, error) {\n\tgeoms := appendBufferSimpleSeries(nil, l.Base(), meters)\n\treturn geojson.NewGeometryCollection(geoms), nil\n}\n" }, { "path": "internal/buffer/buffer_test.go", "content": "package buffer\n\nimport (\n\t\"testing\"\n\n\t\"github.com/tidwall/geojson\"\n\t\"github.com/tidwall/geojson/geometry\"\n)\n\nconst lineString = `{\"type\":\"LineString\",\"coordinates\":[\n\t[-116.40289306640624,34.125447565116126],\n\t[-116.36444091796875,34.14818102254435],\n\t[-116.0980224609375,34.15045403191448],\n\t[-115.74920654296874,34.127721186043985],\n\t[-115.54870605468749,34.075412438417395],\n\t[-115.5267333984375,34.11407854333859],\n\t[-115.21911621093749,34.048108084909835],\n\t[-115.25207519531249,33.8339199536547],\n\t[-115.40588378906249,33.71748624018193]\n]}`\n\nvar lineInPoints = []geometry.Point{\n\t{X: -115.64363479614258, Y: 34.108251327293296},\n\t{X: -115.54355621337892, Y: 34.07199987534163},\n\t{X: -115.21482467651367, Y: 34.051237154976164},\n\t{X: -115.4110336303711, Y: 33.715201644740844},\n\t{X: -116.40701293945311, Y: 34.12345809664606},\n}\n\nfunc TestBufferLineString(t *testing.T) {\n\tg, err := geojson.Parse(lineString, nil)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tg2, err := Simple(g, 1000)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tfor _, pt := range lineInPoints {\n\t\tok := g2.Contains(geojson.NewPoint(pt))\n\t\tif !ok {\n\t\t\tt.Fatalf(\"!ok\")\n\t\t}\n\t}\n}\n\nconst polygon = `{\"type\": \"Polygon\",\"coordinates\":[\n\t[\n\t\t[116.46881103515624,34.277644878733824],\n\t\t[115.87280273437499,34.20953080048952],\n\t\t[115.70251464843749,34.397844946449865],\n\t\t[115.9881591796875,34.61286625296406],\n\t\t[116.46881103515624,34.277644878733824]\n\t],\n\t[\n\t\t[115.90438842773436,34.38651267795365],\n\t\t[116.05270385742188,34.35023911062779],\n\t\t[115.99914550781249,34.44655621402982],\n\t\t[115.90438842773436,34.38651267795365]\n\t]\n]}`\n\nvar polyInPoints = []geometry.Point{\n\t{X: 115.95837593078612, Y: 34.59887847065301},\n\t{X: 115.98755836486816, Y: 34.61879975173954},\n\t{X: 115.98833084106445, Y: 34.59795999847678},\n\t{X: 116.04536533355714, Y: 34.58082509817638},\n\t{X: 116.47567749023438, Y: 34.27651009584797},\n\t{X: 116.42005920410155, Y: 34.32018817684490},\n\t{X: 116.33216857910156, Y: 34.25948651450623},\n\t{X: 115.89340209960939, Y: 34.24132422972854},\n\t{X: 115.95588684082033, Y: 34.42786803680155},\n\t{X: 115.97236633300783, Y: 34.42107129982385},\n\t{X: 115.99639892578125, Y: 34.43579686485573},\n\t{X: 116.04652404785155, Y: 34.35364042469895},\n\t{X: 115.92155456542967, Y: 34.38877925439021},\n\t{X: 115.96755981445311, Y: 34.37687904351907},\n\t{X: 115.88859558105467, Y: 34.42956713470528},\n\t{X: 115.97511291503906, Y: 34.36327673174518},\n\t{X: 115.69564819335938, Y: 34.39784494644986},\n\t{X: 115.87005615234375, Y: 34.20385213966983},\n\t{X: 115.76980590820312, Y: 34.31678550602221},\n}\nvar polyOutPoints = []geometry.Point{\n\t{X: 115.68534851074217, Y: 34.40917568058836},\n\t{X: 115.98953247070312, Y: 34.63038297923298},\n\t{X: 115.98541259765624, Y: 34.39671178864245},\n\t{X: 116.31500244140626, Y: 34.22145474280257},\n\t{X: 115.85426330566406, Y: 34.18510984477340},\n}\n\nfunc TestBufferPolygon(t *testing.T) {\n\tg, err := geojson.Parse(polygon, nil)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tg2, err := Simple(g, 1000)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tfor _, pt := range polyInPoints {\n\t\tok := g2.Contains(geojson.NewPoint(pt))\n\t\tif !ok {\n\t\t\tt.Fatalf(\"!ok\")\n\t\t}\n\t}\n\tfor _, pt := range polyOutPoints {\n\t\tok := g2.Contains(geojson.NewPoint(pt))\n\t\tif ok {\n\t\t\tt.Fatalf(\"ok\")\n\t\t}\n\t}\n}\n" }, { "path": "internal/clip/clip.go", "content": "package clip\n\nimport (\n\t\"github.com/tidwall/geojson\"\n\t\"github.com/tidwall/geojson/geometry\"\n)\n\n// Clip clips the contents of a geojson object and return\nfunc Clip(\n\tobj geojson.Object, clipper geojson.Object, opts *geometry.IndexOptions,\n) (clipped geojson.Object) {\n\tswitch obj := obj.(type) {\n\tcase *geojson.Point:\n\t\treturn clipPoint(obj, clipper, opts)\n\tcase *geojson.Rect:\n\t\treturn clipRect(obj, clipper, opts)\n\tcase *geojson.LineString:\n\t\treturn clipLineString(obj, clipper, opts)\n\tcase *geojson.Polygon:\n\t\treturn clipPolygon(obj, clipper, opts)\n\tcase *geojson.Feature:\n\t\treturn clipFeature(obj, clipper, opts)\n\tcase geojson.Collection:\n\t\treturn clipCollection(obj, clipper, opts)\n\t}\n\treturn obj\n}\n\n// clipSegment is Cohen-Sutherland Line Clipping\n// https://www.cs.helsinki.fi/group/goa/viewing/leikkaus/lineClip.html\nfunc clipSegment(seg geometry.Segment, rect geometry.Rect) (\n\tres geometry.Segment, rejected bool,\n) {\n\tstartCode := getCode(rect, seg.A)\n\tendCode := getCode(rect, seg.B)\n\tif (startCode | endCode) == 0 {\n\t\t// trivially accept\n\t\tres = seg\n\t} else if (startCode & endCode) != 0 {\n\t\t// trivially reject\n\t\trejected = true\n\t} else if startCode != 0 {\n\t\t// start is outside. get new start.\n\t\tnewStart := intersect(rect, startCode, seg.A, seg.B)\n\t\tres, rejected =\n\t\t\tclipSegment(geometry.Segment{A: newStart, B: seg.B}, rect)\n\t} else {\n\t\t// end is outside. get new end.\n\t\tnewEnd := intersect(rect, endCode, seg.A, seg.B)\n\t\tres, rejected = clipSegment(geometry.Segment{A: seg.A, B: newEnd}, rect)\n\t}\n\treturn\n}\n\n// clipRing is Sutherland-Hodgman Polygon Clipping\n// https://www.cs.helsinki.fi/group/goa/viewing/leikkaus/intro2.html\nfunc clipRing(ring []geometry.Point, bbox geometry.Rect) (\n\tresRing []geometry.Point,\n) {\n\tif len(ring) < 4 {\n\t\t// under 4 elements this is not a polygon ring!\n\t\treturn\n\t}\n\tvar edge uint8\n\tvar inside, prevInside bool\n\tvar prev geometry.Point\n\tfor edge = 1; edge <= 8; edge *= 2 {\n\t\tprev = ring[len(ring)-2]\n\t\tprevInside = (getCode(bbox, prev) & edge) == 0\n\t\tfor _, p := range ring {\n\t\t\tinside = (getCode(bbox, p) & edge) == 0\n\t\t\tif prevInside && inside {\n\t\t\t\t// Staying inside\n\t\t\t\tresRing = append(resRing, p)\n\t\t\t} else if prevInside && !inside {\n\t\t\t\t// Leaving\n\t\t\t\tresRing = append(resRing, intersect(bbox, edge, prev, p))\n\t\t\t} else if !prevInside && inside {\n\t\t\t\t// Entering\n\t\t\t\tresRing = append(resRing, intersect(bbox, edge, prev, p))\n\t\t\t\tresRing = append(resRing, p)\n\t\t\t} /* else {\n\t\t\t\t// Stay outside\n\t\t\t} */\n\t\t\tprev, prevInside = p, inside\n\t\t}\n\t\tif len(resRing) > 0 && resRing[0] != resRing[len(resRing)-1] {\n\t\t\tresRing = append(resRing, resRing[0])\n\t\t}\n\t\tring, resRing = resRing, []geometry.Point{}\n\t\tif len(ring) == 0 {\n\t\t\tbreak\n\t\t}\n\t}\n\tresRing = ring\n\treturn\n}\n\nfunc getCode(bbox geometry.Rect, point geometry.Point) (code uint8) {\n\tcode = 0\n\n\tif point.X < bbox.Min.X {\n\t\tcode |= 1 // left\n\t} else if point.X > bbox.Max.X {\n\t\tcode |= 2 // right\n\t}\n\n\tif point.Y < bbox.Min.Y {\n\t\tcode |= 4 // bottom\n\t} else if point.Y > bbox.Max.Y {\n\t\tcode |= 8 // top\n\t}\n\n\treturn\n}\n\nfunc intersect(bbox geometry.Rect, code uint8, start, end geometry.Point) (\n\tnew geometry.Point,\n) {\n\tif (code & 8) != 0 { // top\n\t\tnew = geometry.Point{\n\t\t\tX: start.X + (end.X-start.X)*(bbox.Max.Y-start.Y)/(end.Y-start.Y),\n\t\t\tY: bbox.Max.Y,\n\t\t}\n\t} else if (code & 4) != 0 { // bottom\n\t\tnew = geometry.Point{\n\t\t\tX: start.X + (end.X-start.X)*(bbox.Min.Y-start.Y)/(end.Y-start.Y),\n\t\t\tY: bbox.Min.Y,\n\t\t}\n\t} else if (code & 2) != 0 { //right\n\t\tnew = geometry.Point{\n\t\t\tX: bbox.Max.X,\n\t\t\tY: start.Y + (end.Y-start.Y)*(bbox.Max.X-start.X)/(end.X-start.X),\n\t\t}\n\t} else if (code & 1) != 0 { // left\n\t\tnew = geometry.Point{\n\t\t\tX: bbox.Min.X,\n\t\t\tY: start.Y + (end.Y-start.Y)*(bbox.Min.X-start.X)/(end.X-start.X),\n\t\t}\n\t} /* else {\n\t\t// should not call intersect with the zero code\n\t} */\n\n\treturn\n}\n" }, { "path": "internal/clip/clip_test.go", "content": "package clip\n\nimport (\n\t\"testing\"\n\n\t\"github.com/tidwall/geojson\"\n\t\"github.com/tidwall/geojson/geometry\"\n)\n\nfunc LO(points []geometry.Point) *geojson.LineString {\n\treturn geojson.NewLineString(geometry.NewLine(points, nil))\n}\n\nfunc RO(minX, minY, maxX, maxY float64) *geojson.Rect {\n\treturn geojson.NewRect(geometry.Rect{\n\t\tMin: geometry.Point{X: minX, Y: minY},\n\t\tMax: geometry.Point{X: maxX, Y: maxY},\n\t})\n}\n\nfunc PPO(exterior []geometry.Point, holes [][]geometry.Point) *geojson.Polygon {\n\treturn geojson.NewPolygon(geometry.NewPoly(exterior, holes, nil))\n}\n\nfunc TestClipLineStringSimple(t *testing.T) {\n\tls := LO([]geometry.Point{\n\t\t{X: 1, Y: 1},\n\t\t{X: 2, Y: 2},\n\t\t{X: 3, Y: 1}})\n\tclipped := Clip(ls, RO(1.5, 0.5, 2.5, 1.8), nil)\n\tcl, ok := clipped.(*geojson.MultiLineString)\n\tif !ok {\n\t\tt.Fatal(\"wrong type\")\n\t}\n\tif len(cl.Children()) != 2 {\n\t\tt.Fatal(\"result must have two parts in MultiString\")\n\t}\n}\n\nfunc TestClipPolygonSimple(t *testing.T) {\n\texterior := []geometry.Point{\n\t\t{X: 2, Y: 2},\n\t\t{X: 1, Y: 2},\n\t\t{X: 1.5, Y: 1.5},\n\t\t{X: 1, Y: 1},\n\t\t{X: 2, Y: 1},\n\t\t{X: 2, Y: 2},\n\t}\n\tholes := [][]geometry.Point{\n\t\t{\n\t\t\t{X: 1.9, Y: 1.9},\n\t\t\t{X: 1.2, Y: 1.9},\n\t\t\t{X: 1.45, Y: 1.65},\n\t\t\t{X: 1.9, Y: 1.5},\n\t\t\t{X: 1.9, Y: 1.9},\n\t\t},\n\t}\n\tpolygon := PPO(exterior, holes)\n\tclipped := Clip(polygon, RO(1.3, 1.3, 1.4, 2.15), nil)\n\tcp, ok := clipped.(*geojson.Polygon)\n\tif !ok {\n\t\tt.Fatal(\"wrong type\")\n\t}\n\tif cp.Base().Exterior.Empty() {\n\t\tt.Fatal(\"Empty result.\")\n\t}\n\tif len(cp.Base().Holes) != 1 {\n\t\tt.Fatal(\"result must be a two-ring Polygon\")\n\t}\n}\n\nfunc TestClipPolygon2(t *testing.T) {\n\texterior := []geometry.Point{\n\t\t{X: 2, Y: 2},\n\t\t{X: 1, Y: 2},\n\t\t{X: 1.5, Y: 1.5},\n\t\t{X: 1, Y: 1},\n\t\t{X: 2, Y: 1},\n\t\t{X: 2, Y: 2},\n\t}\n\tholes := [][]geometry.Point{\n\t\t{\n\t\t\t{X: 1.9, Y: 1.9},\n\t\t\t{X: 1.2, Y: 1.9},\n\t\t\t{X: 1.45, Y: 1.65},\n\t\t\t{X: 1.9, Y: 1.5},\n\t\t\t{X: 1.9, Y: 1.9},\n\t\t},\n\t}\n\tpolygon := PPO(exterior, holes)\n\tclipped := Clip(polygon, RO(1.1, 0.8, 1.15, 2.1), nil)\n\tcp, ok := clipped.(*geojson.Polygon)\n\tif !ok {\n\t\tt.Fatal(\"wrong type\")\n\t}\n\tif cp.Base().Exterior.Empty() {\n\t\tt.Fatal(\"Empty result.\")\n\t}\n\tif len(cp.Base().Holes) != 0 {\n\t\tt.Fatal(\"result must be a single-ring Polygon\")\n\t}\n}\n\n// func TestClipLineString(t *testing.T) {\n// \tfeaturesJSON := `\n// \t\t{\"type\": \"FeatureCollection\",\"features\": [\n// \t\t\t{\"type\": \"Feature\",\"properties\":{},\"geometry\": {\"type\": \"LineString\",\"coordinates\": [[-71.46537780761717,42.594290856363344],[-71.37714385986328,42.600861802789524],[-71.37508392333984,42.538156868495555],[-71.43756866455078,42.535374141307415],[-71.44683837890625,42.466018925787495],[-71.334228515625,42.465005871175755],[-71.32736206054688,42.52424199254517]]}},\n// \t\t\t{\"type\": \"Feature\",\"properties\":{},\"geometry\": {\"type\": \"Polygon\",\"coordinates\": [[[-71.49284362792969,42.527784255084676],[-71.35791778564453,42.527784255084676],[-71.35791778564453,42.61096959812047],[-71.49284362792969,42.61096959812047],[-71.49284362792969,42.527784255084676]]]}},\n// \t\t\t{\"type\": \"Feature\",\"properties\":{},\"geometry\": {\"type\": \"Polygon\",\"coordinates\": [[[-71.47396087646484,42.48247876554176],[-71.30744934082031,42.48247876554176],[-71.30744934082031,42.576596402826894],[-71.47396087646484,42.576596402826894],[-71.47396087646484,42.48247876554176]]]}},\n// \t\t\t{\"type\": \"Feature\",\"properties\":{},\"geometry\": {\"type\": \"Polygon\",\"coordinates\": [[[-71.33491516113281,42.613496290695196],[-71.29920959472656,42.613496290695196],[-71.29920959472656,42.643556064374536],[-71.33491516113281,42.643556064374536],[-71.33491516113281,42.613496290695196]]]}},\n// \t\t\t{\"type\": \"Feature\",\"properties\":{},\"geometry\": {\"type\": \"Polygon\",\"coordinates\": [[[-71.37130737304686,42.530061317794775],[-71.3287353515625,42.530061317794775],[-71.3287353515625,42.60414701616359],[-71.37130737304686,42.60414701616359],[-71.37130737304686,42.530061317794775]]]}},\n// \t\t\t{\"type\": \"Feature\",\"properties\":{},\"geometry\": {\"type\": \"Polygon\",\"coordinates\": [[[-71.52889251708984,42.564460160624115],[-71.45713806152342,42.54043355305221],[-71.53266906738281,42.49969365675931],[-71.36547088623047,42.508552415528634],[-71.43962860107422,42.58999409368092],[-71.52889251708984,42.564460160624115]]]}},\n// \t\t\t{\"type\": \"Feature\",\"properties\": {},\"geometry\": {\"type\": \"Point\",\"coordinates\": [-71.33079528808594,42.55940269610327]}},\n// \t\t\t{\"type\": \"Feature\",\"properties\": {},\"geometry\": {\"type\": \"Point\",\"coordinates\": [-71.27208709716797,42.53107331902133]}}\n// \t\t]}\n// \t`\n// \trectJSON := `{\"type\": \"Feature\",\"properties\": {},\"geometry\": {\"type\": \"Polygon\",\"coordinates\": [[[-71.44065856933594,42.51740991900762],[-71.29131317138672,42.51740991900762],[-71.29131317138672,42.62663343969058],[-71.44065856933594,42.62663343969058],[-71.44065856933594,42.51740991900762]]]}}`\n// \tfeatures := expectJSON(t, featuresJSON, nil)\n// \trect := expectJSON(t, rectJSON, nil)\n// \tclipped := features.Clipped(rect)\n// \tprintln(clipped.String())\n\n// }\n" }, { "path": "internal/clip/collection.go", "content": "package clip\n\nimport (\n\t\"github.com/tidwall/geojson\"\n\t\"github.com/tidwall/geojson/geometry\"\n)\n\nfunc clipCollection(\n\tcollection geojson.Collection, clipper geojson.Object,\n\topts *geometry.IndexOptions,\n) geojson.Object {\n\tvar features []geojson.Object\n\tfor _, feature := range collection.Children() {\n\t\tfeature = Clip(feature, clipper, opts)\n\t\tif feature.Empty() {\n\t\t\tcontinue\n\t\t}\n\t\tif _, ok := feature.(*geojson.Feature); !ok {\n\t\t\tfeature = geojson.NewFeature(feature, \"\")\n\t\t}\n\t\tfeatures = append(features, feature)\n\t}\n\treturn geojson.NewFeatureCollection(features)\n}\n" }, { "path": "internal/clip/feature.go", "content": "package clip\n\nimport (\n\t\"github.com/tidwall/geojson\"\n\t\"github.com/tidwall/geojson/geometry\"\n)\n\nfunc clipFeature(\n\tfeature *geojson.Feature, clipper geojson.Object,\n\topts *geometry.IndexOptions,\n) geojson.Object {\n\tnewFeature := Clip(feature.Base(), clipper, opts)\n\tif _, ok := newFeature.(*geojson.Feature); !ok {\n\t\tnewFeature = geojson.NewFeature(newFeature, feature.Members())\n\t}\n\treturn newFeature\n}\n" }, { "path": "internal/clip/linestring.go", "content": "package clip\n\nimport (\n\t\"github.com/tidwall/geojson\"\n\t\"github.com/tidwall/geojson/geometry\"\n)\n\nfunc clipLineString(\n\tlineString *geojson.LineString, clipper geojson.Object,\n\topts *geometry.IndexOptions,\n) geojson.Object {\n\tbbox := clipper.Rect()\n\tvar newPoints [][]geometry.Point\n\tvar clipped geometry.Segment\n\tvar rejected bool\n\tvar line []geometry.Point\n\tbase := lineString.Base()\n\tnSegments := base.NumSegments()\n\tfor i := 0; i < nSegments; i++ {\n\t\tclipped, rejected = clipSegment(base.SegmentAt(i), bbox)\n\t\tif rejected {\n\t\t\tcontinue\n\t\t}\n\t\tif len(line) > 0 && line[len(line)-1] != clipped.A {\n\t\t\tnewPoints = append(newPoints, line)\n\t\t\tline = []geometry.Point{clipped.A}\n\t\t} else if len(line) == 0 {\n\t\t\tline = append(line, clipped.A)\n\t\t}\n\t\tline = append(line, clipped.B)\n\t}\n\tif len(line) > 0 {\n\t\tnewPoints = append(newPoints, line)\n\t}\n\tvar children []*geometry.Line\n\tfor _, points := range newPoints {\n\t\tchildren = append(children,\n\t\t\tgeometry.NewLine(points, opts))\n\t}\n\tif len(children) == 1 {\n\t\treturn geojson.NewLineString(children[0])\n\t}\n\treturn geojson.NewMultiLineString(children)\n}\n" }, { "path": "internal/clip/point.go", "content": "package clip\n\nimport (\n\t\"github.com/tidwall/geojson\"\n\t\"github.com/tidwall/geojson/geometry\"\n)\n\nfunc clipPoint(\n\tpoint *geojson.Point, clipper geojson.Object, opts *geometry.IndexOptions,\n) geojson.Object {\n\tif point.IntersectsRect(clipper.Rect()) {\n\t\treturn point\n\t}\n\treturn geojson.NewMultiPoint(nil)\n}\n" }, { "path": "internal/clip/polygon.go", "content": "package clip\n\nimport (\n\t\"github.com/tidwall/geojson\"\n\t\"github.com/tidwall/geojson/geometry\"\n)\n\nfunc clipPolygon(\n\tpolygon *geojson.Polygon, clipper geojson.Object,\n\topts *geometry.IndexOptions,\n) geojson.Object {\n\trect := clipper.Rect()\n\tvar newPoints [][]geometry.Point\n\tbase := polygon.Base()\n\trings := []geometry.Ring{base.Exterior}\n\trings = append(rings, base.Holes...)\n\tfor _, ring := range rings {\n\t\tringPoints := make([]geometry.Point, ring.NumPoints())\n\t\tfor i := 0; i < len(ringPoints); i++ {\n\t\t\tringPoints[i] = ring.PointAt(i)\n\t\t}\n\t\tif clippedRing := clipRing(ringPoints, rect); len(clippedRing) > 0 {\n\t\t\tnewPoints = append(newPoints, clippedRing)\n\t\t}\n\t}\n\n\tvar exterior []geometry.Point\n\tvar holes [][]geometry.Point\n\tif len(newPoints) > 0 {\n\t\texterior = newPoints[0]\n\t}\n\tif len(newPoints) > 1 {\n\t\tholes = newPoints[1:]\n\t}\n\tnewPoly := geojson.NewPolygon(\n\t\tgeometry.NewPoly(exterior, holes, opts),\n\t)\n\tif newPoly.Empty() {\n\t\treturn geojson.NewMultiPolygon(nil)\n\t}\n\treturn newPoly\n}\n" }, { "path": "internal/clip/rect.go", "content": "package clip\n\nimport (\n\t\"github.com/tidwall/geojson\"\n\t\"github.com/tidwall/geojson/geometry\"\n)\n\nfunc clipRect(\n\trect *geojson.Rect, clipper geojson.Object, opts *geometry.IndexOptions,\n) geojson.Object {\n\tbase := rect.Base()\n\tpoints := make([]geometry.Point, base.NumPoints())\n\tfor i := 0; i < len(points); i++ {\n\t\tpoints[i] = base.PointAt(i)\n\t}\n\tpoly := geometry.NewPoly(points, nil, opts)\n\tgPoly := geojson.NewPolygon(poly)\n\treturn Clip(gPoly, clipper, opts)\n}\n" }, { "path": "internal/collection/collection.go", "content": "package collection\n\nimport (\n\t\"math\"\n\t\"runtime\"\n\n\t\"github.com/tidwall/btree\"\n\t\"github.com/tidwall/geojson\"\n\t\"github.com/tidwall/geojson/geometry\"\n\t\"github.com/tidwall/rtree\"\n\t\"github.com/tidwall/tile38/internal/deadline\"\n\t\"github.com/tidwall/tile38/internal/field\"\n\t\"github.com/tidwall/tile38/internal/object\"\n)\n\n// yieldStep forces the iterator to yield goroutine every 256 steps.\nconst yieldStep = 256\n\n// Cursor allows for quickly paging through Scan, Within, Intersects, and Nearby\ntype Cursor interface {\n\tOffset() uint64\n\tStep(count uint64)\n}\n\nfunc byID(a, b *object.Object) bool {\n\treturn a.ID() < b.ID()\n}\n\nfunc byValue(a, b *object.Object) bool {\n\tvalue1 := a.String()\n\tvalue2 := b.String()\n\tif value1 < value2 {\n\t\treturn true\n\t}\n\tif value1 > value2 {\n\t\treturn false\n\t}\n\t// the values match so we'll compare IDs, which are always unique.\n\treturn byID(a, b)\n}\n\nfunc byExpires(a, b *object.Object) bool {\n\tif a.Expires() < b.Expires() {\n\t\treturn true\n\t}\n\tif a.Expires() > b.Expires() {\n\t\treturn false\n\t}\n\t// the values match so we'll compare IDs, which are always unique.\n\treturn byID(a, b)\n}\n\n// Collection represents a collection of geojson objects.\ntype Collection struct {\n\tobjs btree.Map[string, *object.Object] // sorted by id\n\tspatial rtree.RTreeGN[float32, *object.Object] // geospatially indexed\n\tvalues *btree.BTreeG[*object.Object] // sorted by value+id\n\texpires *btree.BTreeG[*object.Object] // sorted by ex+id\n\tweight int\n\tpoints int\n\tobjects int // geometry count\n\tnobjects int // non-geometry count\n}\n\nvar optsNoLock = btree.Options{NoLocks: true}\n\n// New creates an empty collection\nfunc New() *Collection {\n\tcol := &Collection{\n\t\tvalues: btree.NewBTreeGOptions(byValue, optsNoLock),\n\t\texpires: btree.NewBTreeGOptions(byExpires, optsNoLock),\n\t}\n\treturn col\n}\n\n// Count returns the number of objects in collection.\nfunc (c *Collection) Count() int {\n\treturn c.objects + c.nobjects\n}\n\n// StringCount returns the number of string values.\nfunc (c *Collection) StringCount() int {\n\treturn c.nobjects\n}\n\n// PointCount returns the number of points (lat/lon coordinates) in collection.\nfunc (c *Collection) PointCount() int {\n\treturn c.points\n}\n\n// TotalWeight calculates the in-memory cost of the collection in bytes.\nfunc (c *Collection) TotalWeight() int {\n\treturn c.weight\n}\n\n// Bounds returns the bounds of all the items in the collection.\nfunc (c *Collection) Bounds() (minX, minY, maxX, maxY float64) {\n\t_, _, left := c.spatial.LeftMost()\n\t_, _, bottom := c.spatial.BottomMost()\n\t_, _, right := c.spatial.RightMost()\n\t_, _, top := c.spatial.TopMost()\n\tif left == nil {\n\t\treturn\n\t}\n\treturn left.Rect().Min.X, bottom.Rect().Min.Y,\n\t\tright.Rect().Max.X, top.Rect().Max.Y\n}\n\nfunc (c *Collection) indexDelete(item *object.Object) {\n\tif !item.Geo().Empty() {\n\t\tc.spatial.Delete(rtreeItem(item))\n\t}\n}\n\nfunc (c *Collection) indexInsert(item *object.Object) {\n\tif !item.Geo().Empty() {\n\t\tc.spatial.Insert(rtreeItem(item))\n\t}\n}\n\nconst dRNDTOWARDS = (1.0 - 1.0/8388608.0) /* Round towards zero */\nconst dRNDAWAY = (1.0 + 1.0/8388608.0) /* Round away from zero */\n\nfunc rtreeValueDown(d float64) float32 {\n\tf := float32(d)\n\tif float64(f) > d {\n\t\tif d < 0 {\n\t\t\tf = float32(d * dRNDAWAY)\n\t\t} else {\n\t\t\tf = float32(d * dRNDTOWARDS)\n\t\t}\n\t}\n\treturn f\n}\nfunc rtreeValueUp(d float64) float32 {\n\tf := float32(d)\n\tif float64(f) < d {\n\t\tif d < 0 {\n\t\t\tf = float32(d * dRNDTOWARDS)\n\t\t} else {\n\t\t\tf = float32(d * dRNDAWAY)\n\t\t}\n\t}\n\treturn f\n}\n\nfunc rtreeItem(item *object.Object) (min, max [2]float32, data *object.Object) {\n\tmin, max = rtreeRect(item.Rect())\n\treturn min, max, item\n}\n\nfunc rtreeRect(rect geometry.Rect) (min, max [2]float32) {\n\treturn [2]float32{\n\t\t\trtreeValueDown(rect.Min.X),\n\t\t\trtreeValueDown(rect.Min.Y),\n\t\t}, [2]float32{\n\t\t\trtreeValueUp(rect.Max.X),\n\t\t\trtreeValueUp(rect.Max.Y),\n\t\t}\n}\n\n// Set adds or replaces an object in the collection and returns the fields\n// array.\nfunc (c *Collection) Set(obj *object.Object) (prev *object.Object) {\n\tprev, _ = c.objs.Set(obj.ID(), obj)\n\tc.setFill(prev, obj)\n\treturn prev\n}\n\nfunc (c *Collection) setFill(prev, obj *object.Object) {\n\tif prev != nil {\n\t\tif prev.IsSpatial() {\n\t\t\tc.indexDelete(prev)\n\t\t\tc.objects--\n\t\t} else {\n\t\t\tc.values.Delete(prev)\n\t\t\tc.nobjects--\n\t\t}\n\t\tif prev.Expires() != 0 {\n\t\t\tc.expires.Delete(prev)\n\t\t}\n\t\tc.points -= prev.Geo().NumPoints()\n\t\tc.weight -= prev.Weight()\n\t}\n\tif obj.IsSpatial() {\n\t\tc.indexInsert(obj)\n\t\tc.objects++\n\t} else {\n\t\tc.values.Set(obj)\n\t\tc.nobjects++\n\t}\n\tif obj.Expires() != 0 {\n\t\tc.expires.Set(obj)\n\t}\n\tc.points += obj.Geo().NumPoints()\n\tc.weight += obj.Weight()\n}\n\n// Delete removes an object and returns it.\n// If the object does not exist then the 'ok' return value will be false.\nfunc (c *Collection) Delete(id string) (prev *object.Object) {\n\tprev, _ = c.objs.Delete(id)\n\tif prev == nil {\n\t\treturn nil\n\t}\n\tif prev.IsSpatial() {\n\t\tif !prev.Geo().Empty() {\n\t\t\tc.indexDelete(prev)\n\t\t}\n\t\tc.objects--\n\t} else {\n\t\tc.values.Delete(prev)\n\t\tc.nobjects--\n\t}\n\tif prev.Expires() != 0 {\n\t\tc.expires.Delete(prev)\n\t}\n\tc.points -= prev.Geo().NumPoints()\n\tc.weight -= prev.Weight()\n\treturn prev\n}\n\n// Get returns an object.\n// If the object does not exist then the 'ok' return value will be false.\nfunc (c *Collection) Get(id string) *object.Object {\n\tobj, _ := c.objs.Get(id)\n\treturn obj\n}\n\n// Scan iterates though the collection ids.\nfunc (c *Collection) Scan(\n\tdesc bool,\n\tcursor Cursor,\n\tdeadline *deadline.Deadline,\n\titerator func(obj *object.Object) bool,\n) bool {\n\tvar keepon = true\n\tvar count uint64\n\tvar offset uint64\n\tif cursor != nil {\n\t\toffset = cursor.Offset()\n\t\tcursor.Step(offset)\n\t}\n\titer := func(_ string, obj *object.Object) bool {\n\t\tcount++\n\t\tif count <= offset {\n\t\t\treturn true\n\t\t}\n\t\tnextStep(count, cursor, deadline)\n\t\tkeepon = iterator(obj)\n\t\treturn keepon\n\t}\n\tif desc {\n\t\tc.objs.Reverse(iter)\n\t} else {\n\t\tc.objs.Scan(iter)\n\t}\n\treturn keepon\n}\n\n// ScanRange iterates though the collection starting with specified id.\nfunc (c *Collection) ScanRange(\n\tstart, end string,\n\tdesc bool,\n\tcursor Cursor,\n\tdeadline *deadline.Deadline,\n\titerator func(o *object.Object) bool,\n) bool {\n\tvar keepon = true\n\tvar count uint64\n\tvar offset uint64\n\tif cursor != nil {\n\t\toffset = cursor.Offset()\n\t\tcursor.Step(offset)\n\t}\n\titer := func(_ string, o *object.Object) bool {\n\t\tcount++\n\t\tif count <= offset {\n\t\t\treturn true\n\t\t}\n\t\tnextStep(count, cursor, deadline)\n\t\tif !desc {\n\t\t\tif o.ID() >= end {\n\t\t\t\treturn false\n\t\t\t}\n\t\t} else {\n\t\t\tif o.ID() <= end {\n\t\t\t\treturn false\n\t\t\t}\n\t\t}\n\t\tkeepon = iterator(o)\n\t\treturn keepon\n\t}\n\n\tif desc {\n\t\tc.objs.Descend(start, iter)\n\t} else {\n\t\tc.objs.Ascend(start, iter)\n\t}\n\treturn keepon\n}\n\n// SearchValues iterates though the collection values.\nfunc (c *Collection) SearchValues(\n\tdesc bool,\n\tcursor Cursor,\n\tdeadline *deadline.Deadline,\n\titerator func(o *object.Object) bool,\n) bool {\n\tvar keepon = true\n\tvar count uint64\n\tvar offset uint64\n\tif cursor != nil {\n\t\toffset = cursor.Offset()\n\t\tcursor.Step(offset)\n\t}\n\titer := func(o *object.Object) bool {\n\t\tcount++\n\t\tif count <= offset {\n\t\t\treturn true\n\t\t}\n\t\tnextStep(count, cursor, deadline)\n\t\tkeepon = iterator(o)\n\t\treturn keepon\n\t}\n\tif desc {\n\t\tc.values.Reverse(iter)\n\t} else {\n\t\tc.values.Scan(iter)\n\t}\n\treturn keepon\n}\n\n// SearchValuesRange iterates though the collection values.\nfunc (c *Collection) SearchValuesRange(start, end string, desc bool,\n\tcursor Cursor,\n\tdeadline *deadline.Deadline,\n\titerator func(o *object.Object) bool,\n) bool {\n\tvar keepon = true\n\tvar count uint64\n\tvar offset uint64\n\tif cursor != nil {\n\t\toffset = cursor.Offset()\n\t\tcursor.Step(offset)\n\t}\n\titer := func(o *object.Object) bool {\n\t\tcount++\n\t\tif count <= offset {\n\t\t\treturn true\n\t\t}\n\t\tnextStep(count, cursor, deadline)\n\t\tkeepon = iterator(o)\n\t\treturn keepon\n\t}\n\n\tpstart := object.New(\"\", String(start), 0, field.List{})\n\tpend := object.New(\"\", String(end), 0, field.List{})\n\tif desc {\n\t\t// descend range\n\t\tc.values.Descend(pstart, func(item *object.Object) bool {\n\t\t\treturn bGT(c.values, item, pend) && iter(item)\n\t\t})\n\t} else {\n\t\tc.values.Ascend(pstart, func(item *object.Object) bool {\n\t\t\treturn bLT(c.values, item, pend) && iter(item)\n\t\t})\n\t}\n\treturn keepon\n}\n\nfunc bLT(tr *btree.BTreeG[*object.Object], a, b *object.Object) bool { return tr.Less(a, b) }\nfunc bGT(tr *btree.BTreeG[*object.Object], a, b *object.Object) bool { return tr.Less(b, a) }\n\n// ScanGreaterOrEqual iterates though the collection starting with specified id.\nfunc (c *Collection) ScanGreaterOrEqual(id string, desc bool,\n\tcursor Cursor,\n\tdeadline *deadline.Deadline,\n\titerator func(o *object.Object) bool,\n) bool {\n\tvar keepon = true\n\tvar count uint64\n\tvar offset uint64\n\tif cursor != nil {\n\t\toffset = cursor.Offset()\n\t\tcursor.Step(offset)\n\t}\n\titer := func(_ string, o *object.Object) bool {\n\t\tcount++\n\t\tif count <= offset {\n\t\t\treturn true\n\t\t}\n\t\tnextStep(count, cursor, deadline)\n\t\tkeepon = iterator(o)\n\t\treturn keepon\n\t}\n\tif desc {\n\t\tc.objs.Descend(id, iter)\n\t} else {\n\t\tc.objs.Ascend(id, iter)\n\t}\n\treturn keepon\n}\n\nfunc (c *Collection) geoSearch(\n\trect geometry.Rect,\n\titer func(o *object.Object) bool,\n) bool {\n\talive := true\n\tmin, max := rtreeRect(rect)\n\n\t// avoid search if NaN present as it results in full search\n\t// https://github.com/tidwall/tile38/issues/793\n\tif math.IsNaN(float64(min[0])) && math.IsNaN(float64(min[1])) &&\n\t\tmath.IsNaN(float64(max[0])) && math.IsNaN(float64(max[1])) {\n\t\treturn alive\n\t}\n\n\tc.spatial.Search(\n\t\tmin, max,\n\t\tfunc(_, _ [2]float32, o *object.Object) bool {\n\t\t\talive = iter(o)\n\t\t\treturn alive\n\t\t},\n\t)\n\treturn alive\n}\n\nfunc (c *Collection) geoSparse(\n\tobj geojson.Object, sparse uint8,\n\titer func(o *object.Object) (match, ok bool),\n) bool {\n\tmatches := make(map[string]bool)\n\talive := true\n\tc.geoSparseInner(obj.Rect(), sparse, func(o *object.Object) (match, ok bool) {\n\t\tok = true\n\t\tif !matches[o.ID()] {\n\t\t\tmatch, ok = iter(o)\n\t\t\tif match {\n\t\t\t\tmatches[o.ID()] = true\n\t\t\t}\n\t\t}\n\t\treturn match, ok\n\t})\n\treturn alive\n}\nfunc (c *Collection) geoSparseInner(\n\trect geometry.Rect, sparse uint8,\n\titer func(o *object.Object) (match, ok bool),\n) bool {\n\tif sparse > 0 {\n\t\tw := rect.Max.X - rect.Min.X\n\t\th := rect.Max.Y - rect.Min.Y\n\t\tquads := [4]geometry.Rect{\n\t\t\t{\n\t\t\t\tMin: geometry.Point{X: rect.Min.X, Y: rect.Min.Y + h/2},\n\t\t\t\tMax: geometry.Point{X: rect.Min.X + w/2, Y: rect.Max.Y},\n\t\t\t},\n\t\t\t{\n\t\t\t\tMin: geometry.Point{X: rect.Min.X + w/2, Y: rect.Min.Y + h/2},\n\t\t\t\tMax: geometry.Point{X: rect.Max.X, Y: rect.Max.Y},\n\t\t\t},\n\t\t\t{\n\t\t\t\tMin: geometry.Point{X: rect.Min.X, Y: rect.Min.Y},\n\t\t\t\tMax: geometry.Point{X: rect.Min.X + w/2, Y: rect.Min.Y + h/2},\n\t\t\t},\n\t\t\t{\n\t\t\t\tMin: geometry.Point{X: rect.Min.X + w/2, Y: rect.Min.Y},\n\t\t\t\tMax: geometry.Point{X: rect.Max.X, Y: rect.Min.Y + h/2},\n\t\t\t},\n\t\t}\n\t\tfor _, quad := range quads {\n\t\t\tif !c.geoSparseInner(quad, sparse-1, iter) {\n\t\t\t\treturn false\n\t\t\t}\n\t\t}\n\t\treturn true\n\t}\n\talive := true\n\tc.geoSearch(rect, func(o *object.Object) bool {\n\t\tmatch, ok := iter(o)\n\t\tif !ok {\n\t\t\talive = false\n\t\t\treturn false\n\t\t}\n\t\treturn !match\n\t})\n\treturn alive\n}\n\n// Within returns all object that are fully contained within an object or\n// bounding box. Set obj to nil in order to use the bounding box.\nfunc (c *Collection) Within(\n\tobj geojson.Object,\n\tsparse uint8,\n\tcursor Cursor,\n\tdeadline *deadline.Deadline,\n\titer func(o *object.Object) bool,\n) bool {\n\tvar count uint64\n\tvar offset uint64\n\tif cursor != nil {\n\t\toffset = cursor.Offset()\n\t\tcursor.Step(offset)\n\t}\n\tif sparse > 0 {\n\t\treturn c.geoSparse(obj, sparse, func(o *object.Object) (match, ok bool) {\n\t\t\tcount++\n\t\t\tif count <= offset {\n\t\t\t\treturn false, true\n\t\t\t}\n\t\t\tnextStep(count, cursor, deadline)\n\t\t\tif match = o.Geo().Within(obj); match {\n\t\t\t\tok = iter(o)\n\t\t\t}\n\t\t\treturn match, ok\n\t\t})\n\t}\n\treturn c.geoSearch(obj.Rect(), func(o *object.Object) bool {\n\t\tcount++\n\t\tif count <= offset {\n\t\t\treturn true\n\t\t}\n\t\tnextStep(count, cursor, deadline)\n\t\tif o.Geo().Within(obj) {\n\t\t\treturn iter(o)\n\t\t}\n\t\treturn true\n\t})\n}\n\n// Intersects returns all object that are intersect an object or bounding box.\n// Set obj to nil in order to use the bounding box.\nfunc (c *Collection) Intersects(\n\tgobj geojson.Object,\n\tsparse uint8,\n\tcursor Cursor,\n\tdeadline *deadline.Deadline,\n\titer func(o *object.Object) bool,\n) bool {\n\tvar count uint64\n\tvar offset uint64\n\tif cursor != nil {\n\t\toffset = cursor.Offset()\n\t\tcursor.Step(offset)\n\t}\n\tif sparse > 0 {\n\t\treturn c.geoSparse(gobj, sparse, func(o *object.Object) (match, ok bool) {\n\t\t\tcount++\n\t\t\tif count <= offset {\n\t\t\t\treturn false, true\n\t\t\t}\n\t\t\tnextStep(count, cursor, deadline)\n\t\t\tif match = o.Geo().Intersects(gobj); match {\n\t\t\t\tok = iter(o)\n\t\t\t}\n\t\t\treturn match, ok\n\t\t})\n\t}\n\treturn c.geoSearch(gobj.Rect(), func(o *object.Object) bool {\n\t\tcount++\n\t\tif count <= offset {\n\t\t\treturn true\n\t\t}\n\t\tnextStep(count, cursor, deadline)\n\t\tif o.Geo().Intersects(gobj) {\n\t\t\treturn iter(o)\n\t\t}\n\t\treturn true\n\t},\n\t)\n}\n\n// Nearby returns the nearest neighbors\nfunc (c *Collection) Nearby(\n\ttarget geojson.Object,\n\tcursor Cursor,\n\tdeadline *deadline.Deadline,\n\titer func(o *object.Object, dist float64) bool,\n) bool {\n\talive := true\n\tcenter := target.Center()\n\tvar count uint64\n\tvar offset uint64\n\tif cursor != nil {\n\t\toffset = cursor.Offset()\n\t\tcursor.Step(offset)\n\t}\n\tdistFn := geodeticDistAlgo([2]float64{center.X, center.Y})\n\tc.spatial.Nearby(\n\t\tfunc(min, max [2]float32, data *object.Object, item bool) float64 {\n\t\t\treturn distFn(\n\t\t\t\t[2]float64{float64(min[0]), float64(min[1])},\n\t\t\t\t[2]float64{float64(max[0]), float64(max[1])},\n\t\t\t\tdata, item,\n\t\t\t)\n\t\t},\n\t\tfunc(_, _ [2]float32, o *object.Object, dist float64) bool {\n\t\t\tcount++\n\t\t\tif count <= offset {\n\t\t\t\treturn true\n\t\t\t}\n\t\t\tnextStep(count, cursor, deadline)\n\t\t\talive = iter(o, dist)\n\t\t\treturn alive\n\t\t},\n\t)\n\treturn alive\n}\n\nfunc nextStep(step uint64, cursor Cursor, deadline *deadline.Deadline) {\n\tif step&(yieldStep-1) == (yieldStep - 1) {\n\t\truntime.Gosched()\n\t\tdeadline.Check()\n\t}\n\tif cursor != nil {\n\t\tcursor.Step(1)\n\t}\n}\n\n// ScanExpires returns a list of all objects that have expired.\nfunc (c *Collection) ScanExpires(iter func(o *object.Object) bool) {\n\tc.expires.Scan(iter)\n}\n" }, { "path": "internal/collection/collection_test.go", "content": "package collection\n\nimport (\n\t\"fmt\"\n\t\"math/rand\"\n\t\"reflect\"\n\t\"strconv\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/tidwall/geojson\"\n\t\"github.com/tidwall/geojson/geometry\"\n\t\"github.com/tidwall/gjson\"\n\t\"github.com/tidwall/tile38/internal/field\"\n\t\"github.com/tidwall/tile38/internal/object\"\n)\n\nfunc PO(x, y float64) *geojson.Point {\n\treturn geojson.NewPoint(geometry.Point{X: x, Y: y})\n}\n\nfunc init() {\n\tseed := time.Now().UnixNano()\n\tprintln(seed)\n\trand.Seed(seed)\n}\n\nfunc expect(t testing.TB, expect bool) {\n\tt.Helper()\n\tif !expect {\n\t\tt.Fatal(\"not what you expected\")\n\t}\n}\n\nfunc bounds(c *Collection) geometry.Rect {\n\tminX, minY, maxX, maxY := c.Bounds()\n\treturn geometry.Rect{\n\t\tMin: geometry.Point{X: minX, Y: minY},\n\t\tMax: geometry.Point{X: maxX, Y: maxY},\n\t}\n}\n\nfunc TestCollectionNewCollection(t *testing.T) {\n\tconst numItems = 10000\n\tobjs := make(map[string]geojson.Object)\n\tc := New()\n\tfor i := 0; i < numItems; i++ {\n\t\tid := strconv.FormatInt(int64(i), 10)\n\t\tobj := PO(rand.Float64()*360-180, rand.Float64()*180-90)\n\t\tobjs[id] = obj\n\t\tc.Set(object.New(id, obj, 0, field.List{}))\n\t}\n\tcount := 0\n\tbbox := geometry.Rect{\n\t\tMin: geometry.Point{X: -180, Y: -90},\n\t\tMax: geometry.Point{X: 180, Y: 90},\n\t}\n\tc.geoSearch(bbox, func(o *object.Object) bool {\n\t\tcount++\n\t\treturn true\n\t})\n\tif count != len(objs) {\n\t\tt.Fatalf(\"count = %d, expect %d\", count, len(objs))\n\t}\n\tcount = c.Count()\n\tif count != len(objs) {\n\t\tt.Fatalf(\"c.Count() = %d, expect %d\", count, len(objs))\n\t}\n\ttestCollectionVerifyContents(t, c, objs)\n}\n\nfunc toFields(fNames, fValues []string) field.List {\n\tvar fields field.List\n\tfor i := 0; i < len(fNames); i++ {\n\t\tfields = fields.Set(field.Make(fNames[i], fValues[i]))\n\t}\n\treturn fields\n}\n\nfunc TestCollectionSet(t *testing.T) {\n\tt.Run(\"AddString\", func(t *testing.T) {\n\t\tc := New()\n\t\tstr1 := String(\"hello\")\n\t\told := c.Set(object.New(\"str\", str1, 0, field.List{}))\n\t\texpect(t, old == nil)\n\t})\n\tt.Run(\"UpdateString\", func(t *testing.T) {\n\t\tc := New()\n\t\tstr1 := String(\"hello\")\n\t\tstr2 := String(\"world\")\n\t\told := c.Set(object.New(\"str\", str1, 0, field.List{}))\n\t\texpect(t, old == nil)\n\t\told = c.Set(object.New(\"str\", str2, 0, field.List{}))\n\t\texpect(t, old.Geo() == str1)\n\t})\n\tt.Run(\"AddPoint\", func(t *testing.T) {\n\t\tc := New()\n\t\tpoint1 := PO(-112.1, 33.1)\n\t\told := c.Set(object.New(\"point\", point1, 0, field.List{}))\n\t\texpect(t, old == nil)\n\t})\n\tt.Run(\"UpdatePoint\", func(t *testing.T) {\n\t\tc := New()\n\t\tpoint1 := PO(-112.1, 33.1)\n\t\tpoint2 := PO(-112.2, 33.2)\n\t\told := c.Set(object.New(\"point\", point1, 0, field.List{}))\n\t\texpect(t, old == nil)\n\t\told = c.Set(object.New(\"point\", point2, 0, field.List{}))\n\t\texpect(t, old.Geo().Center() == point1.Base())\n\t})\n\tt.Run(\"Fields\", func(t *testing.T) {\n\t\tc := New()\n\t\tstr1 := String(\"hello\")\n\n\t\tfNames := []string{\"a\", \"b\", \"c\"}\n\t\tfValues := []string{\"1\", \"2\", \"3\"}\n\t\tfields1 := toFields(fNames, fValues)\n\t\told := c.Set(object.New(\"str\", str1, 0, fields1))\n\t\texpect(t, old == nil)\n\n\t\tstr2 := String(\"hello\")\n\n\t\tfNames = []string{\"d\", \"e\", \"f\"}\n\t\tfValues = []string{\"4\", \"5\", \"6\"}\n\t\tfields2 := toFields(fNames, fValues)\n\n\t\told = c.Set(object.New(\"str\", str2, 0, fields2))\n\t\texpect(t, old.Geo() == str1)\n\t\texpect(t, reflect.DeepEqual(old.Fields(), fields1))\n\n\t\tfNames = []string{\"a\", \"b\", \"c\", \"d\", \"e\", \"f\"}\n\t\tfValues = []string{\"7\", \"8\", \"9\", \"10\", \"11\", \"12\"}\n\t\tfields3 := toFields(fNames, fValues)\n\t\told = c.Set(object.New(\"str\", str1, 0, fields3))\n\t\texpect(t, old.Geo() == str2)\n\t\texpect(t, reflect.DeepEqual(old.Fields(), fields2))\n\t})\n\tt.Run(\"Delete\", func(t *testing.T) {\n\t\tc := New()\n\n\t\tc.Set(object.New(\"1\", String(\"1\"), 0, field.List{}))\n\t\tc.Set(object.New(\"2\", String(\"2\"), 0, field.List{}))\n\t\tc.Set(object.New(\"3\", PO(1, 2), 0, field.List{}))\n\n\t\texpect(t, c.Count() == 3)\n\t\texpect(t, c.StringCount() == 2)\n\t\texpect(t, c.PointCount() == 1)\n\t\texpect(t, bounds(c) == geometry.Rect{\n\t\t\tMin: geometry.Point{X: 1, Y: 2},\n\t\t\tMax: geometry.Point{X: 1, Y: 2}})\n\t\tvar prev *object.Object\n\n\t\tprev = c.Delete(\"2\")\n\t\texpect(t, prev.Geo().String() == \"2\")\n\t\texpect(t, c.Count() == 2)\n\t\texpect(t, c.StringCount() == 1)\n\t\texpect(t, c.PointCount() == 1)\n\n\t\tprev = c.Delete(\"1\")\n\t\texpect(t, prev.Geo().String() == \"1\")\n\t\texpect(t, c.Count() == 1)\n\t\texpect(t, c.StringCount() == 0)\n\t\texpect(t, c.PointCount() == 1)\n\n\t\tprev = c.Delete(\"3\")\n\t\texpect(t, prev.Geo().String() == `{\"type\":\"Point\",\"coordinates\":[1,2]}`)\n\t\texpect(t, c.Count() == 0)\n\t\texpect(t, c.StringCount() == 0)\n\t\texpect(t, c.PointCount() == 0)\n\t\tprev = c.Delete(\"3\")\n\t\texpect(t, prev == nil)\n\t\texpect(t, c.Count() == 0)\n\t\texpect(t, bounds(c) == geometry.Rect{})\n\t\texpect(t, c.Get(\"3\") == nil)\n\t})\n}\n\nfunc fieldValueAt(fields field.List, index int) string {\n\tif index < 0 || index >= fields.Len() {\n\t\tpanic(\"out of bounds\")\n\t}\n\tvar retval string\n\tvar i int\n\tfields.Scan(func(f field.Field) bool {\n\t\tif i == index {\n\t\t\tretval = f.Value().Data()\n\t\t}\n\t\ti++\n\t\treturn true\n\t})\n\treturn retval\n}\n\nfunc TestCollectionScan(t *testing.T) {\n\tN := 256\n\tc := New()\n\tfor _, i := range rand.Perm(N) {\n\t\tid := fmt.Sprintf(\"%04d\", i)\n\t\tc.Set(object.New(id, String(id), 0, makeFields(\n\t\t\tfield.Make(\"ex\", id),\n\t\t)))\n\t}\n\tvar n int\n\tvar prevID string\n\tc.Scan(false, nil, nil, func(o *object.Object) bool {\n\t\tif n > 0 {\n\t\t\texpect(t, o.ID() > prevID)\n\t\t}\n\t\texpect(t, o.ID() == fieldValueAt(o.Fields(), 0))\n\t\tn++\n\t\tprevID = o.ID()\n\t\treturn true\n\t})\n\texpect(t, n == c.Count())\n\tn = 0\n\tc.Scan(true, nil, nil, func(o *object.Object) bool {\n\t\tif n > 0 {\n\t\t\texpect(t, o.ID() < prevID)\n\t\t}\n\t\texpect(t, o.ID() == fieldValueAt(o.Fields(), 0))\n\t\tn++\n\t\tprevID = o.ID()\n\t\treturn true\n\t})\n\texpect(t, n == c.Count())\n\n\tn = 0\n\tc.ScanRange(\"0060\", \"0070\", false, nil, nil,\n\t\tfunc(o *object.Object) bool {\n\t\t\tif n > 0 {\n\t\t\t\texpect(t, o.ID() > prevID)\n\t\t\t}\n\t\t\texpect(t, o.ID() == fieldValueAt(o.Fields(), 0))\n\t\t\tn++\n\t\t\tprevID = o.ID()\n\t\t\treturn true\n\t\t})\n\texpect(t, n == 10)\n\n\tn = 0\n\tc.ScanRange(\"0070\", \"0060\", true, nil, nil,\n\t\tfunc(o *object.Object) bool {\n\t\t\tif n > 0 {\n\t\t\t\texpect(t, o.ID() < prevID)\n\t\t\t}\n\t\t\texpect(t, o.ID() == fieldValueAt(o.Fields(), 0))\n\t\t\tn++\n\t\t\tprevID = o.ID()\n\t\t\treturn true\n\t\t})\n\texpect(t, n == 10)\n\n\tn = 0\n\tc.ScanGreaterOrEqual(\"0070\", true, nil, nil,\n\t\tfunc(o *object.Object) bool {\n\t\t\tif n > 0 {\n\t\t\t\texpect(t, o.ID() < prevID)\n\t\t\t}\n\t\t\texpect(t, o.ID() == fieldValueAt(o.Fields(), 0))\n\t\t\tn++\n\t\t\tprevID = o.ID()\n\t\t\treturn true\n\t\t})\n\texpect(t, n == 71)\n\n\tn = 0\n\tc.ScanGreaterOrEqual(\"0070\", false, nil, nil,\n\t\tfunc(o *object.Object) bool {\n\t\t\tif n > 0 {\n\t\t\t\texpect(t, o.ID() > prevID)\n\t\t\t}\n\t\t\texpect(t, o.ID() == fieldValueAt(o.Fields(), 0))\n\t\t\tn++\n\t\t\tprevID = o.ID()\n\t\t\treturn true\n\t\t})\n\texpect(t, n == c.Count()-70)\n\n}\n\nfunc makeFields(entries ...field.Field) field.List {\n\tvar fields field.List\n\tfor _, f := range entries {\n\t\tfields = fields.Set(f)\n\t}\n\treturn fields\n}\n\nfunc TestCollectionSearch(t *testing.T) {\n\tN := 256\n\tc := New()\n\tfor i, j := range rand.Perm(N) {\n\t\tid := fmt.Sprintf(\"%04d\", j)\n\t\tex := fmt.Sprintf(\"%04d\", i)\n\t\tc.Set(object.New(id, String(ex),\n\t\t\t0, makeFields(\n\t\t\t\tfield.Make(\"i\", ex),\n\t\t\t\tfield.Make(\"j\", id),\n\t\t\t)))\n\t}\n\tvar n int\n\tvar prevValue string\n\tc.SearchValues(false, nil, nil, func(o *object.Object) bool {\n\t\tif n > 0 {\n\t\t\texpect(t, o.Geo().String() > prevValue)\n\t\t}\n\t\texpect(t, o.ID() == fieldValueAt(o.Fields(), 1))\n\t\tn++\n\t\tprevValue = o.Geo().String()\n\t\treturn true\n\t})\n\texpect(t, n == c.Count())\n\tn = 0\n\tc.SearchValues(true, nil, nil, func(o *object.Object) bool {\n\t\tif n > 0 {\n\t\t\texpect(t, o.Geo().String() < prevValue)\n\t\t}\n\t\texpect(t, o.ID() == fieldValueAt(o.Fields(), 1))\n\t\tn++\n\t\tprevValue = o.Geo().String()\n\t\treturn true\n\t})\n\texpect(t, n == c.Count())\n\n\tn = 0\n\tc.SearchValuesRange(\"0060\", \"0070\", false, nil, nil,\n\t\tfunc(o *object.Object) bool {\n\t\t\tif n > 0 {\n\t\t\t\texpect(t, o.Geo().String() > prevValue)\n\t\t\t}\n\t\t\texpect(t, o.ID() == fieldValueAt(o.Fields(), 1))\n\t\t\tn++\n\t\t\tprevValue = o.Geo().String()\n\t\t\treturn true\n\t\t})\n\texpect(t, n == 10)\n\n\tn = 0\n\tc.SearchValuesRange(\"0070\", \"0060\", true, nil, nil,\n\t\tfunc(o *object.Object) bool {\n\t\t\tif n > 0 {\n\t\t\t\texpect(t, o.Geo().String() < prevValue)\n\t\t\t}\n\t\t\texpect(t, o.ID() == fieldValueAt(o.Fields(), 1))\n\t\t\tn++\n\t\t\tprevValue = o.Geo().String()\n\t\t\treturn true\n\t\t})\n\texpect(t, n == 10)\n}\n\nfunc TestCollectionWeight(t *testing.T) {\n\tc := New()\n\tc.Set(object.New(\"1\", String(\"1\"), 0, field.List{}))\n\texpect(t, c.TotalWeight() > 0)\n\tc.Delete(\"1\")\n\texpect(t, c.TotalWeight() == 0)\n\tc.Set(object.New(\"1\", String(\"1\"), 0,\n\t\ttoFields(\n\t\t\t[]string{\"a\", \"b\", \"c\"},\n\t\t\t[]string{\"1\", \"2\", \"3\"},\n\t\t),\n\t))\n\texpect(t, c.TotalWeight() > 0)\n\tc.Delete(\"1\")\n\texpect(t, c.TotalWeight() == 0)\n\tc.Set(object.New(\"1\", String(\"1\"), 0,\n\t\ttoFields(\n\t\t\t[]string{\"a\", \"b\", \"c\"},\n\t\t\t[]string{\"1\", \"2\", \"3\"},\n\t\t),\n\t))\n\tc.Set(object.New(\"2\", String(\"2\"), 0,\n\t\ttoFields(\n\t\t\t[]string{\"d\", \"e\", \"f\"},\n\t\t\t[]string{\"4\", \"5\", \"6\"},\n\t\t),\n\t))\n\tc.Set(object.New(\"1\", String(\"1\"), 0,\n\t\ttoFields(\n\t\t\t[]string{\"d\", \"e\", \"f\"},\n\t\t\t[]string{\"4\", \"5\", \"6\"},\n\t\t),\n\t))\n\tc.Delete(\"1\")\n\tc.Delete(\"2\")\n\texpect(t, c.TotalWeight() == 0)\n}\n\nfunc TestSpatialSearch(t *testing.T) {\n\tjson := `\n\t\t{\"type\":\"FeatureCollection\",\"features\":[\n\t\t\t{\"type\":\"Feature\",\"id\":\"p1\",\"properties\":{\"marker-color\":\"#962d28\",\"stroke\":\"#962d28\",\"fill\":\"#962d28\"},\"geometry\":{\"type\":\"Point\",\"coordinates\":[-71.4743041992187,42.51867517417283]}},\n\t\t\t{\"type\":\"Feature\",\"id\":\"p2\",\"properties\":{\"marker-color\":\"#962d28\",\"stroke\":\"#962d28\",\"fill\":\"#962d28\"},\"geometry\":{\"type\":\"Point\",\"coordinates\":[-71.4056396484375,42.50197174319114]}},\n\t\t\t{\"type\":\"Feature\",\"id\":\"p3\",\"properties\":{\"marker-color\":\"#962d28\",\"stroke\":\"#962d28\",\"fill\":\"#962d28\"},\"geometry\":{\"type\":\"Point\",\"coordinates\":[-71.4619445800781,42.49437779897246]}},\n\t\t\t{\"type\":\"Feature\",\"id\":\"p4\",\"properties\":{\"marker-color\":\"#962d28\",\"stroke\":\"#962d28\",\"fill\":\"#962d28\"},\"geometry\":{\"type\":\"Point\",\"coordinates\":[-71.4337921142578,42.53891577257117]}},\n\t\t\t{\"type\":\"Feature\",\"id\":\"r1\",\"properties\":{\"marker-color\":\"#962d28\",\"stroke\":\"#962d28\",\"fill\":\"#962d28\"},\"geometry\":{\"type\":\"Polygon\",\"coordinates\":[[[-71.4279556274414,42.48804880765346],[-71.37439727783203,42.48804880765346],[-71.37439727783203,42.52322988064187],[-71.4279556274414,42.52322988064187],[-71.4279556274414,42.48804880765346]]]}},\n\t\t\t{\"type\":\"Feature\",\"id\":\"r2\",\"properties\":{\"marker-color\":\"#962d28\",\"stroke\":\"#962d28\",\"fill\":\"#962d28\"},\"geometry\":{\"type\":\"Polygon\",\"coordinates\":[[[-71.4825439453125,42.53588010092859],[-71.45027160644531,42.53588010092859],[-71.45027160644531,42.55839115400447],[-71.4825439453125,42.55839115400447],[-71.4825439453125,42.53588010092859]]]}},\n\t\t\t{\"type\":\"Feature\",\"id\":\"r3\",\"properties\":{\"marker-color\":\"#962d28\",\"stroke\":\"#962d28\",\"fill\":\"#962d28\"},\"geometry\":{\"type\":\"Polygon\",\"coordinates\": [[[-71.4111328125,42.53512115995963],[-71.3833236694336,42.53512115995963],[-71.3833236694336,42.54953946116446],[-71.4111328125,42.54953946116446],[-71.4111328125,42.53512115995963]]]}},\n\t\t\t{\"type\":\"Feature\",\"id\":\"q1\",\"properties\":{},\"geometry\":{\"type\":\"Polygon\",\"coordinates\":[[[-71.55258178710938,42.51361399979923],[-71.42074584960938,42.51361399979923],[-71.42074584960938,42.59100512331456],[-71.55258178710938,42.59100512331456],[-71.55258178710938,42.51361399979923]]]}},\n\t\t\t{\"type\":\"Feature\",\"id\":\"q2\",\"properties\":{},\"geometry\":{\"type\":\"Polygon\",\"coordinates\":[[[-71.52992248535156,42.48121277771616],[-71.36375427246092,42.48121277771616],[-71.36375427246092,42.57786045892046],[-71.52992248535156,42.57786045892046],[-71.52992248535156,42.48121277771616]]]}},\n\t\t\t{\"type\":\"Feature\",\"id\":\"q3\",\"properties\":{},\"geometry\":{\"type\":\"Polygon\",\"coordinates\":[[[-71.49490356445312,42.56673588590953],[-71.52236938476562,42.47462922809497],[-71.42898559570312,42.464499337722344],[-71.43241882324219,42.522217752342236],[-71.37954711914061,42.56420729713456],[-71.49490356445312,42.56673588590953]]]}},\n\t\t\t{\"type\":\"Feature\",\"id\":\"q4\",\"properties\":{},\"geometry\":{\"type\":\"Point\",\"coordinates\": [-71.46366119384766,42.54043355305221]}}\n\t\t]}\n\t`\n\tp1, _ := geojson.Parse(gjson.Get(json, `features.#[id==\"p1\"]`).Raw, nil)\n\tp2, _ := geojson.Parse(gjson.Get(json, `features.#[id==\"p2\"]`).Raw, nil)\n\tp3, _ := geojson.Parse(gjson.Get(json, `features.#[id==\"p3\"]`).Raw, nil)\n\tp4, _ := geojson.Parse(gjson.Get(json, `features.#[id==\"p4\"]`).Raw, nil)\n\tr1, _ := geojson.Parse(gjson.Get(json, `features.#[id==\"r1\"]`).Raw, nil)\n\tr2, _ := geojson.Parse(gjson.Get(json, `features.#[id==\"r2\"]`).Raw, nil)\n\tr3, _ := geojson.Parse(gjson.Get(json, `features.#[id==\"r3\"]`).Raw, nil)\n\tq1, _ := geojson.Parse(gjson.Get(json, `features.#[id==\"q1\"]`).Raw, nil)\n\tq2, _ := geojson.Parse(gjson.Get(json, `features.#[id==\"q2\"]`).Raw, nil)\n\tq3, _ := geojson.Parse(gjson.Get(json, `features.#[id==\"q3\"]`).Raw, nil)\n\tq4, _ := geojson.Parse(gjson.Get(json, `features.#[id==\"q4\"]`).Raw, nil)\n\n\tc := New()\n\tc.Set(object.New(\"p1\", p1, 0, field.List{}))\n\tc.Set(object.New(\"p2\", p2, 0, field.List{}))\n\tc.Set(object.New(\"p3\", p3, 0, field.List{}))\n\tc.Set(object.New(\"p4\", p4, 0, field.List{}))\n\tc.Set(object.New(\"r1\", r1, 0, field.List{}))\n\tc.Set(object.New(\"r2\", r2, 0, field.List{}))\n\tc.Set(object.New(\"r3\", r3, 0, field.List{}))\n\n\tvar n int\n\n\tn = 0\n\tc.Within(q1, 0, nil, nil, func(o *object.Object) bool {\n\t\tn++\n\t\treturn true\n\t})\n\texpect(t, n == 3)\n\n\tn = 0\n\tc.Within(q2, 0, nil, nil, func(o *object.Object) bool {\n\t\tn++\n\t\treturn true\n\t})\n\texpect(t, n == 7)\n\n\tn = 0\n\tc.Within(q3, 0, nil, nil, func(o *object.Object) bool {\n\t\tn++\n\t\treturn true\n\t})\n\texpect(t, n == 4)\n\n\tn = 0\n\tc.Intersects(q1, 0, nil, nil, func(o *object.Object) bool {\n\t\tn++\n\t\treturn true\n\t})\n\texpect(t, n == 4)\n\n\tn = 0\n\tc.Intersects(q2, 0, nil, nil, func(o *object.Object) bool {\n\t\tn++\n\t\treturn true\n\t})\n\texpect(t, n == 7)\n\n\tn = 0\n\tc.Intersects(q3, 0, nil, nil, func(o *object.Object) bool {\n\t\tn++\n\t\treturn true\n\t})\n\texpect(t, n == 5)\n\n\tn = 0\n\tc.Intersects(q3, 0, nil, nil, func(o *object.Object) bool {\n\t\tn++\n\t\treturn n <= 1\n\t})\n\texpect(t, n == 2)\n\n\tvar items []geojson.Object\n\texitems := []geojson.Object{\n\t\tr2, p4, p1, r1, r3, p3, p2,\n\t}\n\n\tlastDist := float64(-1)\n\tdistsMonotonic := true\n\tc.Nearby(q4, nil, nil, func(o *object.Object, dist float64) bool {\n\t\tif dist < lastDist {\n\t\t\tdistsMonotonic = false\n\t\t}\n\t\titems = append(items, o.Geo())\n\t\treturn true\n\t})\n\texpect(t, len(items) == 7)\n\texpect(t, distsMonotonic)\n\texpect(t, reflect.DeepEqual(items, exitems))\n}\n\nfunc TestCollectionSparse(t *testing.T) {\n\trect := geojson.NewRect(geometry.Rect{\n\t\tMin: geometry.Point{X: -71.598930, Y: 42.4586739},\n\t\tMax: geometry.Point{X: -71.37302, Y: 42.607937},\n\t})\n\tN := 10000\n\tc := New()\n\tr := rect.Rect()\n\tfor i := 0; i < N; i++ {\n\t\tx := (r.Max.X-r.Min.X)*rand.Float64() + r.Min.X\n\t\ty := (r.Max.Y-r.Min.Y)*rand.Float64() + r.Min.Y\n\t\tpoint := PO(x, y)\n\t\tc.Set(object.New(fmt.Sprintf(\"%d\", i), point, 0, field.List{}))\n\t}\n\tvar n int\n\tn = 0\n\tc.Within(rect, 1, nil, nil, func(o *object.Object) bool {\n\t\tn++\n\t\treturn true\n\t})\n\texpect(t, n == 4)\n\n\tn = 0\n\tc.Within(rect, 2, nil, nil, func(o *object.Object) bool {\n\t\tn++\n\t\treturn true\n\t})\n\texpect(t, n == 16)\n\n\tn = 0\n\tc.Within(rect, 3, nil, nil, func(o *object.Object) bool {\n\t\tn++\n\t\treturn true\n\t})\n\texpect(t, n == 64)\n\n\tn = 0\n\tc.Within(rect, 3, nil, nil, func(o *object.Object) bool {\n\t\tn++\n\t\treturn n <= 30\n\t})\n\texpect(t, n == 31)\n\n\tn = 0\n\tc.Intersects(rect, 3, nil, nil, func(o *object.Object) bool {\n\t\tn++\n\t\treturn true\n\t})\n\texpect(t, n == 64)\n\n\tn = 0\n\tc.Intersects(rect, 3, nil, nil, func(o *object.Object) bool {\n\t\tn++\n\t\treturn n <= 30\n\t})\n\texpect(t, n == 31)\n\n}\n\nfunc testCollectionVerifyContents(t *testing.T, c *Collection, objs map[string]geojson.Object) {\n\tfor id, o2 := range objs {\n\t\to := c.Get(id)\n\t\tif o == nil {\n\t\t\tt.Fatalf(\"ok[%s] = false, expect true\", id)\n\t\t}\n\t\tj1 := string(o.Geo().AppendJSON(nil))\n\t\tj2 := string(o2.AppendJSON(nil))\n\t\tif j1 != j2 {\n\t\t\tt.Fatalf(\"j1 == %s, expect %s\", j1, j2)\n\t\t}\n\t}\n}\n\nfunc TestManyCollections(t *testing.T) {\n\tcolsM := make(map[string]*Collection)\n\tcols := 100\n\tobjs := 1000\n\tk := 0\n\tfor i := 0; i < cols; i++ {\n\t\tkey := strconv.FormatInt(int64(i), 10)\n\t\tfor j := 0; j < objs; j++ {\n\t\t\tid := strconv.FormatInt(int64(j), 10)\n\t\t\tp := geometry.Point{\n\t\t\t\tX: rand.Float64()*360 - 180,\n\t\t\t\tY: rand.Float64()*180 - 90,\n\t\t\t}\n\t\t\tobj := geojson.Object(PO(p.X, p.Y))\n\t\t\tcol, ok := colsM[key]\n\t\t\tif !ok {\n\t\t\t\tcol = New()\n\t\t\t\tcolsM[key] = col\n\t\t\t}\n\t\t\tcol.Set(object.New(id, obj, 0, field.List{}))\n\t\t\tk++\n\t\t}\n\t}\n\n\tcol := colsM[\"13\"]\n\t//println(col.Count())\n\tbbox := geometry.Rect{\n\t\tMin: geometry.Point{X: -180, Y: 30},\n\t\tMax: geometry.Point{X: 34, Y: 100},\n\t}\n\tcol.geoSearch(bbox, func(o *object.Object) bool {\n\t\t//println(id)\n\t\treturn true\n\t})\n}\n\ntype testPointItem struct {\n\tid string\n\tobject geojson.Object\n\tfields field.List\n}\n\nfunc makeBenchFields(nFields int) field.List {\n\tvar fields field.List\n\tfor i := 0; i < nFields; i++ {\n\t\tkey := fmt.Sprintf(\"%d\", i)\n\t\tval := key\n\t\tfields = fields.Set(field.Make(key, val))\n\t}\n\treturn fields\n}\n\nfunc BenchmarkInsert_Fields(t *testing.B) {\n\tbenchmarkInsert(t, 1)\n}\n\nfunc BenchmarkInsert_NoFields(t *testing.B) {\n\tbenchmarkInsert(t, 0)\n}\n\nfunc benchmarkInsert(t *testing.B, nFields int) {\n\trand.Seed(time.Now().UnixNano())\n\titems := make([]testPointItem, t.N)\n\tfor i := 0; i < t.N; i++ {\n\t\titems[i] = testPointItem{\n\t\t\tfmt.Sprintf(\"%d\", i),\n\t\t\tPO(rand.Float64()*360-180, rand.Float64()*180-90),\n\t\t\tmakeBenchFields(nFields),\n\t\t}\n\t}\n\tcol := New()\n\tt.ResetTimer()\n\tfor i := 0; i < t.N; i++ {\n\t\tcol.Set(object.New(items[i].id, items[i].object, 0, items[i].fields))\n\t}\n}\n\nfunc BenchmarkReplace_Fields(t *testing.B) {\n\tbenchmarkReplace(t, 1)\n}\n\nfunc BenchmarkReplace_NoFields(t *testing.B) {\n\tbenchmarkReplace(t, 0)\n}\n\nfunc benchmarkReplace(t *testing.B, nFields int) {\n\trand.Seed(time.Now().UnixNano())\n\titems := make([]testPointItem, t.N)\n\tfor i := 0; i < t.N; i++ {\n\t\titems[i] = testPointItem{\n\t\t\tfmt.Sprintf(\"%d\", i),\n\t\t\tPO(rand.Float64()*360-180, rand.Float64()*180-90),\n\t\t\tmakeBenchFields(nFields),\n\t\t}\n\t}\n\tcol := New()\n\tfor i := 0; i < t.N; i++ {\n\t\tcol.Set(object.New(items[i].id, items[i].object, 0, items[i].fields))\n\t}\n\tt.ResetTimer()\n\tfor _, i := range rand.Perm(t.N) {\n\t\to := col.Set(object.New(items[i].id, items[i].object, 0, field.List{}))\n\t\tif o.Geo() != items[i].object {\n\t\t\tt.Fatal(\"shoot!\")\n\t\t}\n\t}\n}\n\nfunc BenchmarkGet_Fields(t *testing.B) {\n\tbenchmarkGet(t, 1)\n}\n\nfunc BenchmarkGet_NoFields(t *testing.B) {\n\tbenchmarkGet(t, 0)\n}\n\nfunc benchmarkGet(t *testing.B, nFields int) {\n\trand.Seed(time.Now().UnixNano())\n\titems := make([]testPointItem, t.N)\n\tfor i := 0; i < t.N; i++ {\n\t\titems[i] = testPointItem{\n\t\t\tfmt.Sprintf(\"%d\", i),\n\t\t\tPO(rand.Float64()*360-180, rand.Float64()*180-90),\n\t\t\tmakeBenchFields(nFields),\n\t\t}\n\t}\n\tcol := New()\n\tfor i := 0; i < t.N; i++ {\n\t\tcol.Set(object.New(items[i].id, items[i].object, 0, items[i].fields))\n\t}\n\tt.ResetTimer()\n\tfor _, i := range rand.Perm(t.N) {\n\t\to := col.Get(items[i].id)\n\t\tif o.Geo() != items[i].object {\n\t\t\tt.Fatal(\"shoot!\")\n\t\t}\n\t}\n}\n\nfunc BenchmarkRemove_Fields(t *testing.B) {\n\tbenchmarkRemove(t, 1)\n}\n\nfunc BenchmarkRemove_NoFields(t *testing.B) {\n\tbenchmarkRemove(t, 0)\n}\n\nfunc benchmarkRemove(t *testing.B, nFields int) {\n\trand.Seed(time.Now().UnixNano())\n\titems := make([]testPointItem, t.N)\n\tfor i := 0; i < t.N; i++ {\n\t\titems[i] = testPointItem{\n\t\t\tfmt.Sprintf(\"%d\", i),\n\t\t\tPO(rand.Float64()*360-180, rand.Float64()*180-90),\n\t\t\tmakeBenchFields(nFields),\n\t\t}\n\t}\n\tcol := New()\n\tfor i := 0; i < t.N; i++ {\n\t\tcol.Set(object.New(items[i].id, items[i].object, 0, items[i].fields))\n\t}\n\tt.ResetTimer()\n\tfor _, i := range rand.Perm(t.N) {\n\t\tprev := col.Delete(items[i].id)\n\t\tif prev.Geo() != items[i].object {\n\t\t\tt.Fatal(\"shoot!\")\n\t\t}\n\t}\n}\n\nfunc BenchmarkScan_Fields(t *testing.B) {\n\tbenchmarkScan(t, 1)\n}\n\nfunc BenchmarkScan_NoFields(t *testing.B) {\n\tbenchmarkScan(t, 0)\n}\n\nfunc benchmarkScan(t *testing.B, nFields int) {\n\trand.Seed(time.Now().UnixNano())\n\titems := make([]testPointItem, t.N)\n\tfor i := 0; i < t.N; i++ {\n\t\titems[i] = testPointItem{\n\t\t\tfmt.Sprintf(\"%d\", i),\n\t\t\tPO(rand.Float64()*360-180, rand.Float64()*180-90),\n\t\t\tmakeBenchFields(nFields),\n\t\t}\n\t}\n\tcol := New()\n\tfor i := 0; i < t.N; i++ {\n\t\tcol.Set(object.New(items[i].id, items[i].object, 0, items[i].fields))\n\t}\n\tt.ResetTimer()\n\tfor i := 0; i < t.N; i++ {\n\t\tvar scanIteration int\n\t\tcol.Scan(true, nil, nil, func(o *object.Object) bool {\n\t\t\tscanIteration++\n\t\t\treturn scanIteration <= 500\n\t\t})\n\t}\n}\n" }, { "path": "internal/collection/geodesic.go", "content": "package collection\n\nimport (\n\t\"math\"\n\n\t\"github.com/tidwall/tile38/internal/object\"\n)\n\nfunc geodeticDistAlgo(center [2]float64) (\n\talgo func(min, max [2]float64, obj *object.Object, item bool) (dist float64),\n) {\n\tconst earthRadius = 6371e3\n\treturn func(min, max [2]float64, obj *object.Object, item bool) (dist float64) {\n\t\tif item {\n\t\t\tr := obj.Rect()\n\t\t\tmin[0] = r.Min.X\n\t\t\tmin[1] = r.Min.Y\n\t\t\tmax[0] = r.Max.X\n\t\t\tmax[1] = r.Max.Y\n\t\t}\n\t\treturn earthRadius * pointRectDistGeodeticDeg(\n\t\t\tcenter[1], center[0],\n\t\t\tmin[1], min[0],\n\t\t\tmax[1], max[0],\n\t\t)\n\t}\n}\n\nfunc pointRectDistGeodeticDeg(pLat, pLng, minLat, minLng, maxLat, maxLng float64) float64 {\n\tresult := pointRectDistGeodeticRad(\n\t\tpLat*math.Pi/180, pLng*math.Pi/180,\n\t\tminLat*math.Pi/180, minLng*math.Pi/180,\n\t\tmaxLat*math.Pi/180, maxLng*math.Pi/180,\n\t)\n\treturn result\n}\n\nfunc pointRectDistGeodeticRad(φq, λq, φl, λl, φh, λh float64) float64 {\n\t// Algorithm from:\n\t// Schubert, E., Zimek, A., & Kriegel, H.-P. (2013).\n\t// Geodetic Distance Queries on R-Trees for Indexing Geographic Data.\n\t// Lecture Notes in Computer Science, 146–164.\n\t// doi:10.1007/978-3-642-40235-7_9\n\tconst (\n\t\ttwoΠ = 2 * math.Pi\n\t\thalfΠ = math.Pi / 2\n\t)\n\n\t// distance on the unit sphere computed using Haversine formula\n\tdistRad := func(φa, λa, φb, λb float64) float64 {\n\t\tif φa == φb && λa == λb {\n\t\t\treturn 0\n\t\t}\n\n\t\tΔφ := φa - φb\n\t\tΔλ := λa - λb\n\t\tsinΔφ := math.Sin(Δφ / 2)\n\t\tsinΔλ := math.Sin(Δλ / 2)\n\t\tcosφa := math.Cos(φa)\n\t\tcosφb := math.Cos(φb)\n\n\t\treturn 2 * math.Asin(math.Sqrt(sinΔφ*sinΔφ+sinΔλ*sinΔλ*cosφa*cosφb))\n\t}\n\n\t// Simple case, point or invalid rect\n\tif φl >= φh && λl >= λh {\n\t\treturn distRad(φl, λl, φq, λq)\n\t}\n\n\tif λl <= λq && λq <= λh {\n\t\t// q is between the bounding meridians of r\n\t\t// hence, q is north, south or within r\n\t\tif φl <= φq && φq <= φh { // Inside\n\t\t\treturn 0\n\t\t}\n\n\t\tif φq < φl { // South\n\t\t\treturn φl - φq\n\t\t}\n\n\t\treturn φq - φh // North\n\t}\n\n\t// determine if q is closer to the east or west edge of r to select edge for\n\t// tests below\n\tΔλe := λl - λq\n\tΔλw := λq - λh\n\tif Δλe < 0 {\n\t\tΔλe += twoΠ\n\t}\n\tif Δλw < 0 {\n\t\tΔλw += twoΠ\n\t}\n\tvar Δλ float64 // distance to closest edge\n\tvar λedge float64 // longitude of closest edge\n\tif Δλe <= Δλw {\n\t\tΔλ = Δλe\n\t\tλedge = λl\n\t} else {\n\t\tΔλ = Δλw\n\t\tλedge = λh\n\t}\n\n\tsinΔλ, cosΔλ := math.Sincos(Δλ)\n\ttanφq := math.Tan(φq)\n\n\tif Δλ >= halfΠ {\n\t\t// If Δλ > 90 degrees (1/2 pi in radians) we're in one of the corners\n\t\t// (NW/SW or NE/SE depending on the edge selected). Compare against the\n\t\t// center line to decide which case we fall into\n\t\tφmid := (φh + φl) / 2\n\t\tif tanφq >= math.Tan(φmid)*cosΔλ {\n\t\t\treturn distRad(φq, λq, φh, λedge) // North corner\n\t\t}\n\t\treturn distRad(φq, λq, φl, λedge) // South corner\n\t}\n\n\tif tanφq >= math.Tan(φh)*cosΔλ {\n\t\treturn distRad(φq, λq, φh, λedge) // North corner\n\t}\n\n\tif tanφq <= math.Tan(φl)*cosΔλ {\n\t\treturn distRad(φq, λq, φl, λedge) // South corner\n\t}\n\n\t// We're to the East or West of the rect, compute distance using cross-track\n\t// Note that this is a simplification of the cross track distance formula\n\t// valid since the track in question is a meridian.\n\treturn math.Asin(math.Cos(φq) * sinΔλ)\n}\n" }, { "path": "internal/collection/string.go", "content": "package collection\n\nimport (\n\t\"encoding/json\"\n\n\t\"github.com/tidwall/geojson\"\n\t\"github.com/tidwall/geojson/geometry\"\n)\n\ntype String string\n\nvar _ geojson.Object = String(\"\")\n\nfunc (s String) Spatial() geojson.Spatial {\n\treturn geojson.EmptySpatial{}\n}\n\nfunc (s String) ForEach(iter func(geom geojson.Object) bool) bool {\n\treturn iter(s)\n}\n\nfunc (s String) Empty() bool {\n\treturn true\n}\n\nfunc (s String) Valid() bool {\n\treturn false\n}\n\nfunc (s String) Rect() geometry.Rect {\n\treturn geometry.Rect{}\n}\n\nfunc (s String) Center() geometry.Point {\n\treturn geometry.Point{}\n}\n\nfunc (s String) AppendJSON(dst []byte) []byte {\n\tdata, _ := json.Marshal(string(s))\n\treturn append(dst, data...)\n}\n\nfunc (s String) String() string {\n\treturn string(s)\n}\n\nfunc (s String) JSON() string {\n\treturn string(s.AppendJSON(nil))\n}\n\nfunc (s String) MarshalJSON() ([]byte, error) {\n\treturn s.AppendJSON(nil), nil\n}\n\nfunc (s String) Within(obj geojson.Object) bool {\n\treturn false\n}\n\nfunc (s String) Contains(obj geojson.Object) bool {\n\treturn false\n}\n\nfunc (s String) Intersects(obj geojson.Object) bool {\n\treturn false\n}\n\nfunc (s String) NumPoints() int {\n\treturn 0\n}\n\nfunc (s String) Distance(obj geojson.Object) float64 {\n\treturn 0\n}\n\nfunc (s String) Members() string {\n\treturn \"\"\n}\n" }, { "path": "internal/deadline/deadline.go", "content": "package deadline\n\nimport \"time\"\n\n// Deadline allows for commands to expire when they run too long\ntype Deadline struct {\n\tunixNano int64\n\thit bool\n}\n\n// New returns a new deadline object\nfunc New(dl time.Time) *Deadline {\n\treturn &Deadline{unixNano: dl.UnixNano()}\n}\n\n// Check the deadline and panic when reached\n//\n//go:noinline\nfunc (dl *Deadline) Check() {\n\tif dl == nil || dl.unixNano == 0 {\n\t\treturn\n\t}\n\tif !dl.hit && time.Now().UnixNano() > dl.unixNano {\n\t\tdl.hit = true\n\t\tpanic(\"deadline\")\n\t}\n}\n\n// Hit returns true if the deadline has been hit\nfunc (dl *Deadline) Hit() bool {\n\treturn dl.hit\n}\n\n// GetDeadlineTime returns the time object for the deadline, and an\n// \"empty\" boolean\nfunc (dl *Deadline) GetDeadlineTime() time.Time {\n\treturn time.Unix(0, dl.unixNano)\n}\n" }, { "path": "internal/endpoint/amqp.go", "content": "package endpoint\n\nimport (\n\t\"fmt\"\n\t\"net\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/streadway/amqp\"\n)\n\nconst amqpExpiresAfter = time.Second * 30\n\n// AMQPConn is an endpoint connection\ntype AMQPConn struct {\n\tmu sync.Mutex\n\tep Endpoint\n\tconn *amqp.Connection\n\tchannel *amqp.Channel\n\tex bool\n\tt time.Time\n}\n\n// Expired returns true if the connection has expired\nfunc (conn *AMQPConn) Expired() bool {\n\tconn.mu.Lock()\n\tdefer conn.mu.Unlock()\n\tif !conn.ex {\n\t\tif time.Since(conn.t) > amqpExpiresAfter {\n\t\t\tconn.close()\n\t\t\tconn.ex = true\n\t\t}\n\t}\n\treturn conn.ex\n}\n\n// ExpireNow forces the connection to expire\nfunc (conn *AMQPConn) ExpireNow() {\n\tconn.mu.Lock()\n\tdefer conn.mu.Unlock()\n\tconn.close()\n\tconn.ex = true\n}\n\nfunc (conn *AMQPConn) close() {\n\tif conn.conn != nil {\n\t\tconn.conn.Close()\n\t\tconn.conn = nil\n\t\tconn.channel = nil\n\t}\n}\n\n// Send sends a message\nfunc (conn *AMQPConn) Send(msg string) error {\n\tconn.mu.Lock()\n\tdefer conn.mu.Unlock()\n\n\tif conn.ex {\n\t\treturn errExpired\n\t}\n\tconn.t = time.Now()\n\n\tif conn.conn == nil {\n\t\tprefix := \"amqp://\"\n\t\tif conn.ep.AMQP.SSL {\n\t\t\tprefix = \"amqps://\"\n\t\t}\n\n\t\tvar cfg amqp.Config\n\t\tcfg.Dial = func(network, addr string) (net.Conn, error) {\n\t\t\treturn net.DialTimeout(network, addr, time.Second)\n\t\t}\n\t\tc, err := amqp.DialConfig(fmt.Sprintf(\"%s%s\", prefix, conn.ep.AMQP.URI), cfg)\n\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\tchannel, err := c.Channel()\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\t// Declare new exchange\n\t\tif err := channel.ExchangeDeclare(\n\t\t\tconn.ep.AMQP.QueueName,\n\t\t\tconn.ep.AMQP.Type,\n\t\t\tconn.ep.AMQP.Durable,\n\t\t\tconn.ep.AMQP.AutoDelete,\n\t\t\tconn.ep.AMQP.Internal,\n\t\t\tconn.ep.AMQP.NoWait,\n\t\t\tnil,\n\t\t); err != nil {\n\t\t\treturn err\n\t\t}\n\t\tif conn.ep.AMQP.Type != \"topic\" {\n\t\t\t// Create queue if queue don't exists\n\t\t\tif _, err := channel.QueueDeclare(\n\t\t\t\tconn.ep.AMQP.QueueName,\n\t\t\t\tconn.ep.AMQP.Durable,\n\t\t\t\tconn.ep.AMQP.AutoDelete,\n\t\t\t\tfalse,\n\t\t\t\tconn.ep.AMQP.NoWait,\n\t\t\t\tnil,\n\t\t\t); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\t// Binding exchange to queue\n\t\t\tif err := channel.QueueBind(\n\t\t\t\tconn.ep.AMQP.QueueName,\n\t\t\t\tconn.ep.AMQP.RouteKey,\n\t\t\t\tconn.ep.AMQP.QueueName,\n\t\t\t\tconn.ep.AMQP.NoWait,\n\t\t\t\tnil,\n\t\t\t); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\t\tconn.conn = c\n\t\tconn.channel = channel\n\t}\n\n\treturn conn.channel.Publish(\n\t\tconn.ep.AMQP.QueueName,\n\t\tconn.ep.AMQP.RouteKey,\n\t\tconn.ep.AMQP.Mandatory,\n\t\tconn.ep.AMQP.Immediate,\n\t\tamqp.Publishing{\n\t\t\tHeaders: amqp.Table{},\n\t\t\tContentType: \"application/json\",\n\t\t\tContentEncoding: \"\",\n\t\t\tBody: []byte(msg),\n\t\t\tDeliveryMode: conn.ep.AMQP.DeliveryMode,\n\t\t\tPriority: conn.ep.AMQP.Priority,\n\t\t},\n\t)\n}\n\nfunc newAMQPConn(ep Endpoint) *AMQPConn {\n\treturn &AMQPConn{\n\t\tep: ep,\n\t\tt: time.Now(),\n\t}\n}\n" }, { "path": "internal/endpoint/cfqueue.go", "content": "package endpoint\n\nimport (\n\t\"context\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/cloudflare/cloudflare-go/v4\"\n\t\"github.com/cloudflare/cloudflare-go/v4/option\"\n\t\"github.com/cloudflare/cloudflare-go/v4/queues\"\n)\n\nconst cfqueueExpiresAfter = time.Second * 30\n\n// CFQueueConn is an endpoint connection\ntype CFQueueConn struct {\n\tmu sync.Mutex\n\tep Endpoint\n\tclient *cloudflare.Client\n\tex bool\n\tt time.Time\n}\n\n// Expired returns true if the connection has expired\nfunc (conn *CFQueueConn) Expired() bool {\n\tconn.mu.Lock()\n\tdefer conn.mu.Unlock()\n\tif !conn.ex {\n\t\tif time.Since(conn.t) > cfqueueExpiresAfter {\n\t\t\tconn.close()\n\t\t\tconn.ex = true\n\t\t}\n\t}\n\treturn conn.ex\n}\n\n// ExpireNow forces the connection to expire\nfunc (conn *CFQueueConn) ExpireNow() {\n\tconn.mu.Lock()\n\tdefer conn.mu.Unlock()\n\tconn.close()\n\tconn.ex = true\n}\n\nfunc (conn *CFQueueConn) close() {\n\tif conn.client != nil {\n\t\tconn.client = nil\n\t}\n}\n\n// Send sends a message\nfunc (conn *CFQueueConn) Send(msg string) error {\n\tconn.mu.Lock()\n\tdefer conn.mu.Unlock()\n\n\tif conn.ex {\n\t\treturn errExpired\n\t}\n\tconn.t = time.Now()\n\n\t// Initialize client if not already done\n\tif conn.client == nil {\n\t\tconn.client = cloudflare.NewClient(\n\t\t\toption.WithAPIToken(conn.ep.CFQueue.APIToken),\n\t\t)\n\t}\n\n\t// Push message to CF Queue\n\t_, err := conn.client.Queues.Messages.Push(\n\t\tcontext.TODO(),\n\t\tconn.ep.CFQueue.QueueID,\n\t\tqueues.MessagePushParams{\n\t\t\tAccountID: cloudflare.String(conn.ep.CFQueue.AccountID),\n\t\t\tBody: queues.MessagePushParamsBodyMqQueueMessageText{\n\t\t\t\tBody: cloudflare.String(msg),\n\t\t\t\tContentType: cloudflare.F(queues.MessagePushParamsBodyMqQueueMessageTextContentTypeText),\n\t\t\t},\n\t\t},\n\t)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\treturn nil\n}\n\nfunc newCFQueueConn(ep Endpoint) *CFQueueConn {\n\treturn &CFQueueConn{\n\t\tep: ep,\n\t\tt: time.Now(),\n\t}\n}\n" }, { "path": "internal/endpoint/disque.go", "content": "package endpoint\n\nimport (\n\t\"fmt\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/gomodule/redigo/redis\"\n\t\"github.com/tidwall/tile38/internal/log\"\n)\n\nconst disqueExpiresAfter = time.Second * 30\n\n// DisqueConn is an endpoint connection\ntype DisqueConn struct {\n\tmu sync.Mutex\n\tep Endpoint\n\tex bool\n\tt time.Time\n\tconn redis.Conn\n}\n\nfunc newDisqueConn(ep Endpoint) *DisqueConn {\n\treturn &DisqueConn{\n\t\tep: ep,\n\t\tt: time.Now(),\n\t}\n}\n\n// Expired returns true if the connection has expired\nfunc (conn *DisqueConn) Expired() bool {\n\tconn.mu.Lock()\n\tdefer conn.mu.Unlock()\n\tif !conn.ex {\n\t\tif time.Since(conn.t) > disqueExpiresAfter {\n\t\t\tconn.close()\n\t\t\tconn.ex = true\n\t\t}\n\t}\n\treturn conn.ex\n}\n\n// ExpireNow forces the connection to expire\nfunc (conn *DisqueConn) ExpireNow() {\n\tconn.mu.Lock()\n\tdefer conn.mu.Unlock()\n\tconn.close()\n\tconn.ex = true\n}\n\nfunc (conn *DisqueConn) close() {\n\tif conn.conn != nil {\n\t\tconn.conn.Close()\n\t\tconn.conn = nil\n\t}\n}\n\n// Send sends a message\nfunc (conn *DisqueConn) Send(msg string) error {\n\tconn.mu.Lock()\n\tdefer conn.mu.Unlock()\n\tif conn.ex {\n\t\treturn errExpired\n\t}\n\tconn.t = time.Now()\n\tif conn.conn == nil {\n\t\taddr := fmt.Sprintf(\"%s:%d\", conn.ep.Disque.Host, conn.ep.Disque.Port)\n\t\tvar err error\n\t\tconn.conn, err = redis.Dial(\"tcp\", addr)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\tvar args []interface{}\n\targs = append(args, conn.ep.Disque.QueueName, msg, 0)\n\tif conn.ep.Disque.Options.Replicate > 0 {\n\t\targs = append(args, \"REPLICATE\", conn.ep.Disque.Options.Replicate)\n\t}\n\n\treply, err := redis.String(conn.conn.Do(\"ADDJOB\", args...))\n\tif err != nil {\n\t\tconn.close()\n\t\treturn err\n\t}\n\tlog.Debugf(\"Disque: ADDJOB '%s'\", reply)\n\treturn nil\n}\n" }, { "path": "internal/endpoint/endpoint.go", "content": "package endpoint\n\nimport (\n\t\"errors\"\n\t\"net/url\"\n\t\"strconv\"\n\t\"strings\"\n\t\"sync\"\n\t\"sync/atomic\"\n\t\"time\"\n\n\t\"github.com/streadway/amqp\"\n)\n\nvar errExpired = errors.New(\"expired\")\n\n// Protocol is the type of protocol that the endpoint represents.\ntype Protocol string\n\nconst (\n\t// Local protocol\n\tLocal = Protocol(\"local\")\n\t// HTTP protocol\n\tHTTP = Protocol(\"http\")\n\t// Disque protocol\n\tDisque = Protocol(\"disque\")\n\t// GRPC protocol\n\tGRPC = Protocol(\"grpc\")\n\t// Redis protocol\n\tRedis = Protocol(\"redis\")\n\t// Kafka protocol\n\tKafka = Protocol(\"kafka\")\n\t// MQTT protocol\n\tMQTT = Protocol(\"mqtt\")\n\t// AMQP protocol\n\tAMQP = Protocol(\"amqp\")\n\t// SQS protocol\n\tSQS = Protocol(\"sqs\")\n\t// Google Cloud Pubsub protocol\n\tPubSub = Protocol(\"pubsub\")\n\t// NATS protocol\n\tNATS = Protocol(\"nats\")\n\t// EventHub protocol\n\tEventHub = Protocol(\"sb\")\n\t// CFQueue protocol\n\tCFQueue = Protocol(\"cf-queue\")\n)\n\n// Endpoint represents an endpoint.\ntype Endpoint struct {\n\tProtocol Protocol\n\tOriginal string\n\tGRPC struct {\n\t\tHost string\n\t\tPort int\n\t}\n\tDisque struct {\n\t\tHost string\n\t\tPort int\n\t\tQueueName string\n\t\tOptions struct {\n\t\t\tReplicate int\n\t\t}\n\t}\n\tRedis struct {\n\t\tHost string\n\t\tPort int\n\t\tChannel string\n\t}\n\tKafka struct {\n\t\tHost string\n\t\tPort int\n\t\tTopicName string\n\t\tAuth string\n\t\tSSL bool\n\t\tSASLSHA256 bool\n\t\tSASLSHA512 bool\n\t\tCACertFile string\n\t\tCertFile string\n\t\tKeyFile string\n\t}\n\tAMQP struct {\n\t\tURI string\n\t\tSSL bool\n\t\tQueueName string\n\t\tRouteKey string\n\t\tType string\n\t\tDurable bool\n\t\tAutoDelete bool\n\t\tInternal bool\n\t\tNoWait bool\n\t\tMandatory bool\n\t\tImmediate bool\n\t\tDeliveryMode uint8\n\t\tPriority uint8\n\t}\n\tMQTT struct {\n\t\tHost string\n\t\tPort int\n\t\tQueueName string\n\t\tQos byte\n\t\tRetained bool\n\t\tCACertFile string\n\t\tCertFile string\n\t\tKeyFile string\n\t}\n\tPubSub struct {\n\t\tProject string\n\t\tTopic string\n\t\tCredPath string\n\t}\n\tSQS struct {\n\t\tPlainURL string\n\t\tQueueID string\n\t\tRegion string\n\t\tCredPath string\n\t\tCredProfile string\n\t\tQueueName string\n\t\tCreateQueue bool\n\t}\n\tNATS struct {\n\t\tHost string\n\t\tPort int\n\t\tUser string\n\t\tPass string\n\t\tTopic string\n\t\tToken string\n\t\tTLS bool\n\t\tTLSCert string\n\t\tTLSKey string\n\t\tSecure bool\n\t\t// Jetstream indicates publishing via jetstream acknowledgements.\n\t\tJetstream bool\n\t\tUserCredentialPath string\n\t}\n\tEventHub struct {\n\t\tConnectionString string\n\t}\n\tCFQueue struct {\n\t\tAccountID string\n\t\tQueueID string\n\t\tAPIToken string\n\t}\n\tLocal struct {\n\t\tChannel string\n\t}\n}\n\n// Conn is an endpoint connection\ntype Conn interface {\n\tExpireNow()\n\tExpired() bool\n\tSend(val string) error\n}\n\n// Manager manages all endpoints\ntype Manager struct {\n\tmu sync.RWMutex\n\tconns map[string]Conn\n\tpublisher LocalPublisher\n\tshutdown atomic.Bool // atomic bool\n\twg sync.WaitGroup // run wait group\n}\n\n// NewManager returns a new manager\nfunc NewManager(publisher LocalPublisher) *Manager {\n\tepc := &Manager{\n\t\tconns: make(map[string]Conn),\n\t\tpublisher: publisher,\n\t}\n\tepc.wg.Add(1)\n\tgo epc.run()\n\treturn epc\n}\n\nfunc (epc *Manager) Shutdown() {\n\tdefer epc.wg.Wait()\n\tepc.shutdown.Store(true)\n\t// expire the connections\n\tepc.mu.Lock()\n\tdefer epc.mu.Unlock()\n\tfor _, conn := range epc.conns {\n\t\tconn.ExpireNow()\n\t}\n}\n\n// Run starts the managing of endpoints\nfunc (epc *Manager) run() {\n\tdefer epc.wg.Done()\n\tfor {\n\t\tif epc.shutdown.Load() {\n\t\t\treturn\n\t\t}\n\t\ttime.Sleep(time.Second)\n\t\tfunc() {\n\t\t\tepc.mu.Lock()\n\t\t\tdefer epc.mu.Unlock()\n\t\t\tfor endpoint, conn := range epc.conns {\n\t\t\t\tif conn.Expired() {\n\t\t\t\t\tdelete(epc.conns, endpoint)\n\t\t\t\t}\n\t\t\t}\n\t\t}()\n\t}\n}\n\n// Validate an endpoint url\nfunc (epc *Manager) Validate(url string) error {\n\t_, err := parseEndpoint(url)\n\treturn err\n}\n\n// Send send a message to an endpoint\nfunc (epc *Manager) Send(endpoint, msg string) error {\n\tfor {\n\t\tepc.mu.Lock()\n\t\tconn, exists := epc.conns[endpoint]\n\t\tif !exists || conn.Expired() {\n\t\t\tep, err := parseEndpoint(endpoint)\n\t\t\tif err != nil {\n\t\t\t\tepc.mu.Unlock()\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tswitch ep.Protocol {\n\t\t\tdefault:\n\t\t\t\treturn errors.New(\"invalid protocol\")\n\t\t\tcase HTTP:\n\t\t\t\tconn = newHTTPConn(ep)\n\t\t\tcase Disque:\n\t\t\t\tconn = newDisqueConn(ep)\n\t\t\tcase GRPC:\n\t\t\t\tconn = newGRPCConn(ep)\n\t\t\tcase Redis:\n\t\t\t\tconn = newRedisConn(ep)\n\t\t\tcase Kafka:\n\t\t\t\tconn = newKafkaConn(ep)\n\t\t\tcase MQTT:\n\t\t\t\tconn = newMQTTConn(ep)\n\t\t\tcase AMQP:\n\t\t\t\tconn = newAMQPConn(ep)\n\t\t\tcase PubSub:\n\t\t\t\tconn = newPubSubConn(ep)\n\t\t\tcase SQS:\n\t\t\t\tconn = newSQSConn(ep)\n\t\t\tcase NATS:\n\t\t\t\tconn = newNATSConn(ep)\n\t\t\tcase Local:\n\t\t\t\tconn = newLocalConn(ep, epc.publisher)\n\t\t\tcase EventHub:\n\t\t\t\tconn = newEventHubConn(ep)\n\t\t\tcase CFQueue:\n\t\t\t\tconn = newCFQueueConn(ep)\n\t\t\t}\n\t\t\tepc.conns[endpoint] = conn\n\t\t}\n\t\tepc.mu.Unlock()\n\t\terr := conn.Send(msg)\n\t\tif err != nil {\n\t\t\tif err == errExpired {\n\t\t\t\t// it's possible that the connection has expired in-between\n\t\t\t\t// the last conn.Expired() check and now. If so, we should\n\t\t\t\t// just try the send again.\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\treturn err\n\t\t}\n\t\treturn nil\n\t}\n}\n\nfunc parseEndpoint(s string) (Endpoint, error) {\n\tvar endpoint Endpoint\n\tendpoint.Original = s\n\tswitch {\n\tdefault:\n\t\treturn endpoint, errors.New(\"unknown scheme\")\n\tcase strings.HasPrefix(s, \"local:\"):\n\t\tendpoint.Protocol = Local\n\tcase strings.HasPrefix(s, \"http:\"):\n\t\tendpoint.Protocol = HTTP\n\tcase strings.HasPrefix(s, \"https:\"):\n\t\tif probeSQS(s) {\n\t\t\tendpoint.SQS.PlainURL = s\n\t\t\tendpoint.Protocol = SQS\n\t\t} else {\n\t\t\tendpoint.Protocol = HTTP\n\t\t}\n\tcase strings.HasPrefix(s, \"disque:\"):\n\t\tendpoint.Protocol = Disque\n\tcase strings.HasPrefix(s, \"grpc:\"):\n\t\tendpoint.Protocol = GRPC\n\tcase strings.HasPrefix(s, \"redis:\"):\n\t\tendpoint.Protocol = Redis\n\tcase strings.HasPrefix(s, \"kafka:\"):\n\t\tendpoint.Protocol = Kafka\n\tcase strings.HasPrefix(s, \"amqp:\"):\n\t\tendpoint.Protocol = AMQP\n\tcase strings.HasPrefix(s, \"amqps:\"):\n\t\tendpoint.Protocol = AMQP\n\tcase strings.HasPrefix(s, \"mqtt:\"):\n\t\tendpoint.Protocol = MQTT\n\tcase strings.HasPrefix(s, \"pubsub:\"):\n\t\tendpoint.Protocol = PubSub\n\tcase strings.HasPrefix(s, \"sqs:\"):\n\t\tendpoint.Protocol = SQS\n\tcase strings.HasPrefix(s, \"nats:\"):\n\t\tendpoint.Protocol = NATS\n\tcase strings.HasPrefix(s, \"Endpoint=\"):\n\t\tendpoint.Protocol = EventHub\n\tcase strings.HasPrefix(s, \"cf-queue:\"):\n\t\tendpoint.Protocol = CFQueue\n\t}\n\n\ts = s[strings.Index(s, \":\")+1:]\n\tif !strings.HasPrefix(s, \"//\") {\n\t\treturn endpoint, errors.New(\"missing the two slashes\")\n\t}\n\n\tsqp := strings.Split(s[2:], \"?\")\n\tsp := strings.Split(sqp[0], \"/\")\n\ts = sp[0]\n\tif s == \"\" {\n\t\tif endpoint.Protocol == Local {\n\t\t\treturn endpoint, errors.New(\"missing channel\")\n\t\t}\n\t\treturn endpoint, errors.New(\"missing host\")\n\t}\n\n\t// Local PubSub channel\n\t// local://\n\tif endpoint.Protocol == Local {\n\t\tendpoint.Local.Channel = s\n\t}\n\tif endpoint.Protocol == GRPC {\n\t\tdp := strings.Split(s, \":\")\n\t\tswitch len(dp) {\n\t\tdefault:\n\t\t\treturn endpoint, errors.New(\"invalid grpc url\")\n\t\tcase 1:\n\t\t\tendpoint.GRPC.Host = dp[0]\n\t\t\tendpoint.GRPC.Port = 80\n\t\tcase 2:\n\t\t\tendpoint.GRPC.Host = dp[0]\n\t\t\tn, err := strconv.ParseUint(dp[1], 10, 16)\n\t\t\tif err != nil {\n\t\t\t\treturn endpoint, errors.New(\"invalid grpc url\")\n\t\t\t}\n\t\t\tendpoint.GRPC.Port = int(n)\n\t\t}\n\t}\n\n\tif endpoint.Protocol == Redis {\n\t\tdp := strings.Split(s, \":\")\n\t\tswitch len(dp) {\n\t\tdefault:\n\t\t\treturn endpoint, errors.New(\"invalid redis url\")\n\t\tcase 1:\n\t\t\tendpoint.Redis.Host = dp[0]\n\t\t\tendpoint.Redis.Port = 6379\n\t\tcase 2:\n\t\t\tendpoint.Redis.Host = dp[0]\n\t\t\tn, err := strconv.ParseUint(dp[1], 10, 16)\n\t\t\tif err != nil {\n\t\t\t\treturn endpoint, errors.New(\"invalid redis url port\")\n\t\t\t}\n\t\t\tendpoint.Redis.Port = int(n)\n\t\t}\n\n\t\tif len(sp) > 1 {\n\t\t\tvar err error\n\t\t\tendpoint.Redis.Channel, err = url.QueryUnescape(sp[1])\n\t\t\tif err != nil {\n\t\t\t\treturn endpoint, errors.New(\"invalid redis channel name\")\n\t\t\t}\n\t\t}\n\t}\n\n\tif endpoint.Protocol == Disque {\n\t\tdp := strings.Split(s, \":\")\n\t\tswitch len(dp) {\n\t\tdefault:\n\t\t\treturn endpoint, errors.New(\"invalid disque url\")\n\t\tcase 1:\n\t\t\tendpoint.Disque.Host = dp[0]\n\t\t\tendpoint.Disque.Port = 7711\n\t\tcase 2:\n\t\t\tendpoint.Disque.Host = dp[0]\n\t\t\tn, err := strconv.ParseUint(dp[1], 10, 16)\n\t\t\tif err != nil {\n\t\t\t\treturn endpoint, errors.New(\"invalid disque url\")\n\t\t\t}\n\t\t\tendpoint.Disque.Port = int(n)\n\t\t}\n\t\tif len(sp) > 1 {\n\t\t\tvar err error\n\t\t\tendpoint.Disque.QueueName, err = url.QueryUnescape(sp[1])\n\t\t\tif err != nil {\n\t\t\t\treturn endpoint, errors.New(\"invalid disque queue name\")\n\t\t\t}\n\t\t}\n\t\tif len(sqp) > 1 {\n\t\t\tm, err := url.ParseQuery(sqp[1])\n\t\t\tif err != nil {\n\t\t\t\treturn endpoint, errors.New(\"invalid disque url\")\n\t\t\t}\n\t\t\tfor key, val := range m {\n\t\t\t\tif len(val) == 0 {\n\t\t\t\t\tcontinue\n\t\t\t\t}\n\t\t\t\tswitch key {\n\t\t\t\tcase \"replicate\":\n\t\t\t\t\tn, err := strconv.ParseUint(val[0], 10, 8)\n\t\t\t\t\tif err != nil {\n\t\t\t\t\t\treturn endpoint, errors.New(\"invalid disque replicate value\")\n\t\t\t\t\t}\n\t\t\t\t\tendpoint.Disque.Options.Replicate = int(n)\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t\tif endpoint.Disque.QueueName == \"\" {\n\t\t\treturn endpoint, errors.New(\"missing disque queue name\")\n\t\t}\n\t}\n\n\tif endpoint.Protocol == Kafka {\n\t\t// Parsing connection from URL string\n\t\thp := strings.Split(s, \":\")\n\t\tswitch len(hp) {\n\t\tdefault:\n\t\t\treturn endpoint, errors.New(\"invalid kafka url\")\n\t\tcase 1:\n\t\t\tendpoint.Kafka.Host = hp[0]\n\t\t\tendpoint.Kafka.Port = 9092\n\t\tcase 2:\n\t\t\tn, err := strconv.ParseUint(hp[1], 10, 16)\n\t\t\tif err != nil {\n\t\t\t\treturn endpoint, errors.New(\"invalid kafka url port\")\n\t\t\t}\n\n\t\t\tendpoint.Kafka.Host = hp[0]\n\t\t\tendpoint.Kafka.Port = int(n)\n\t\t}\n\n\t\t// Parsing Kafka queue name\n\t\tif len(sp) > 1 {\n\t\t\tvar err error\n\t\t\tendpoint.Kafka.TopicName, err = url.QueryUnescape(sp[1])\n\t\t\tif err != nil {\n\t\t\t\treturn endpoint, errors.New(\"invalid kafka topic name\")\n\t\t\t}\n\t\t}\n\n\t\t// Throw error if we not provide any queue name\n\t\tif endpoint.Kafka.TopicName == \"\" {\n\t\t\treturn endpoint, errors.New(\"missing kafka topic name\")\n\t\t}\n\n\t\t// Parsing additional params\n\t\tif len(sqp) > 1 {\n\t\t\tm, err := url.ParseQuery(sqp[1])\n\t\t\tif err != nil {\n\t\t\t\treturn endpoint, errors.New(\"invalid kafka url\")\n\t\t\t}\n\t\t\tfor key, val := range m {\n\t\t\t\tif len(val) == 0 {\n\t\t\t\t\tcontinue\n\t\t\t\t}\n\t\t\t\tswitch key {\n\t\t\t\tcase \"auth\":\n\t\t\t\t\tendpoint.Kafka.Auth = val[0]\n\t\t\t\tcase \"ssl\":\n\t\t\t\t\tendpoint.Kafka.SSL, _ = strconv.ParseBool(val[0])\n\t\t\t\tcase \"cacert\":\n\t\t\t\t\tendpoint.Kafka.CACertFile = val[0]\n\t\t\t\tcase \"cert\":\n\t\t\t\t\tendpoint.Kafka.CertFile = val[0]\n\t\t\t\tcase \"key\":\n\t\t\t\t\tendpoint.Kafka.KeyFile = val[0]\n\t\t\t\tcase \"sha256\":\n\t\t\t\t\tendpoint.Kafka.SASLSHA256, _ = strconv.ParseBool(val[0])\n\t\t\t\tcase \"sha512\":\n\t\t\t\t\tendpoint.Kafka.SASLSHA512, _ = strconv.ParseBool(val[0])\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\n\tif endpoint.Protocol == MQTT {\n\t\t// Parsing connection from URL string\n\t\thp := strings.Split(s, \":\")\n\t\tswitch len(hp) {\n\t\tdefault:\n\t\t\treturn endpoint, errors.New(\"invalid MQTT url\")\n\t\tcase 1:\n\t\t\tendpoint.MQTT.Host = hp[0]\n\t\t\tendpoint.MQTT.Port = 1883\n\t\tcase 2:\n\t\t\tn, err := strconv.ParseUint(hp[1], 10, 16)\n\t\t\tif err != nil {\n\t\t\t\treturn endpoint, errors.New(\"invalid MQTT url port\")\n\t\t\t}\n\n\t\t\tendpoint.MQTT.Host = hp[0]\n\t\t\tendpoint.MQTT.Port = int(n)\n\t\t}\n\n\t\t// Parsing MQTT queue name\n\t\tif len(sp) > 1 {\n\t\t\tvar err error\n\t\t\tvar parts []string\n\t\t\tfor _, part := range sp[1:] {\n\t\t\t\tpart, err = url.QueryUnescape(part)\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn endpoint, errors.New(\"invalid MQTT topic name\")\n\t\t\t\t}\n\t\t\t\tparts = append(parts, part)\n\t\t\t}\n\t\t\tendpoint.MQTT.QueueName = strings.Join(parts, \"/\")\n\t\t}\n\n\t\t// Parsing additional params\n\t\tif len(sqp) > 1 {\n\t\t\tm, err := url.ParseQuery(sqp[1])\n\t\t\tif err != nil {\n\t\t\t\treturn endpoint, errors.New(\"invalid MQTT url\")\n\t\t\t}\n\t\t\tfor key, val := range m {\n\t\t\t\tif len(val) == 0 {\n\t\t\t\t\tcontinue\n\t\t\t\t}\n\t\t\t\tswitch key {\n\t\t\t\tcase \"qos\":\n\t\t\t\t\tn, err := strconv.ParseUint(val[0], 10, 8)\n\t\t\t\t\tif err != nil {\n\t\t\t\t\t\treturn endpoint, errors.New(\"invalid MQTT qos value\")\n\t\t\t\t\t}\n\t\t\t\t\tendpoint.MQTT.Qos = byte(n)\n\t\t\t\tcase \"retained\":\n\t\t\t\t\tn, err := strconv.ParseUint(val[0], 10, 8)\n\t\t\t\t\tif err != nil {\n\t\t\t\t\t\treturn endpoint, errors.New(\"invalid MQTT retained value\")\n\t\t\t\t\t}\n\n\t\t\t\t\tif n != 1 && n != 0 {\n\t\t\t\t\t\treturn endpoint, errors.New(\"invalid MQTT retained, should be [0, 1]\")\n\t\t\t\t\t}\n\n\t\t\t\t\tif n == 1 {\n\t\t\t\t\t\tendpoint.MQTT.Retained = true\n\t\t\t\t\t}\n\t\t\t\tcase \"cacert\":\n\t\t\t\t\tendpoint.MQTT.CACertFile = val[0]\n\t\t\t\tcase \"cert\":\n\t\t\t\t\tendpoint.MQTT.CertFile = val[0]\n\t\t\t\tcase \"key\":\n\t\t\t\t\tendpoint.MQTT.KeyFile = val[0]\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\n\t\t// Throw error if we not provide any queue name\n\t\tif endpoint.MQTT.QueueName == \"\" {\n\t\t\treturn endpoint, errors.New(\"missing MQTT topic name\")\n\t\t}\n\t}\n\t// Basic SQS connection strings in HOOKS interface\n\t// sqs://://?params=value\n\t//\n\t// params are:\n\t//\n\t// credpath - path where aws credentials are located\n\t// credprofile - credential profile\n\tif endpoint.Protocol == SQS {\n\t\tif endpoint.SQS.PlainURL == \"\" {\n\t\t\t// Parsing connection from URL string\n\t\t\thp := strings.Split(s, \":\")\n\t\t\tswitch len(hp) {\n\t\t\tdefault:\n\t\t\t\treturn endpoint, errors.New(\"invalid SQS url\")\n\t\t\tcase 2:\n\t\t\t\tendpoint.SQS.Region = hp[0]\n\t\t\t\tendpoint.SQS.QueueID = hp[1]\n\t\t\t}\n\n\t\t\t// Parsing SQS queue name\n\t\t\tif len(sp) > 1 {\n\t\t\t\tvar err error\n\t\t\t\tendpoint.SQS.QueueName, err = url.QueryUnescape(sp[1])\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn endpoint, errors.New(\"invalid SQS queue name\")\n\t\t\t\t}\n\t\t\t}\n\t\t\t// Throw error if we not provide any queue name\n\t\t\tif endpoint.SQS.QueueName == \"\" {\n\t\t\t\treturn endpoint, errors.New(\"missing SQS queue name\")\n\t\t\t}\n\t\t}\n\n\t\t// Parsing additional params\n\t\tif len(sqp) > 1 {\n\t\t\tm, err := url.ParseQuery(sqp[1])\n\t\t\tif err != nil {\n\t\t\t\treturn endpoint, errors.New(\"invalid SQS url\")\n\t\t\t}\n\t\t\tfor key, val := range m {\n\t\t\t\tif len(val) == 0 {\n\t\t\t\t\tcontinue\n\t\t\t\t}\n\t\t\t\tswitch key {\n\t\t\t\tcase \"credpath\":\n\t\t\t\t\tendpoint.SQS.CredPath = val[0]\n\t\t\t\tcase \"credprofile\":\n\t\t\t\t\tendpoint.SQS.CredProfile = val[0]\n\t\t\t\tcase \"createqueue\":\n\t\t\t\t\tswitch strings.ToLower(val[0]) {\n\t\t\t\t\tcase \"0\", \"false\":\n\t\t\t\t\tdefault:\n\t\t\t\t\t\tendpoint.SQS.CreateQueue = true\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\t// Basic Pubsub connection strings in HOOKS interface\n\t// pubsub://:?params=value\n\t//\n\t// params are:\n\t//\n\t// credpath - path where gcp credentials are located\n\tif endpoint.Protocol == PubSub {\n\t\tsplit := strings.Split(s, \":\")\n\t\tif len(split) != 2 {\n\t\t\treturn endpoint, errors.New(\"invalid PubSub format should be project/topic\")\n\t\t}\n\t\tendpoint.PubSub.Project = split[0]\n\t\tendpoint.PubSub.Topic = split[1]\n\n\t\tif len(sqp) > 1 {\n\t\t\tm, err := url.ParseQuery(sqp[1])\n\t\t\tif err != nil {\n\t\t\t\treturn endpoint, errors.New(\"invalid Pubsub url\")\n\t\t\t}\n\t\t\tfor key, val := range m {\n\t\t\t\tif len(val) == 0 {\n\t\t\t\t\tcontinue\n\t\t\t\t}\n\t\t\t\tswitch key {\n\t\t\t\tcase \"credpath\":\n\t\t\t\t\tendpoint.PubSub.CredPath = val[0]\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\n\t}\n\n\t// Basic AMQP connection strings in HOOKS interface\n\t// amqp://guest:guest@localhost:5672//?params=value\n\t// or amqp://guest:guest@localhost:5672///?params=value\n\t//\n\t// Default params are:\n\t//\n\t// Mandatory - false\n\t// Immeditate - false\n\t// Durable - true\n\t// Routing-Key - tile38\n\t//\n\t// - \"route\" - [string] routing key\n\t//\n\tif endpoint.Protocol == AMQP {\n\t\t// Bind connection information\n\t\tendpoint.AMQP.URI = s\n\t\tendpoint.AMQP.Type = \"direct\"\n\t\tendpoint.AMQP.Durable = true\n\t\tendpoint.AMQP.DeliveryMode = amqp.Transient\n\n\t\t// Fix incase of namespace, e.g. example.com/namespace/queue\n\t\t// but not example.com/queue/ - with an endslash.\n\t\tif len(sp) > 2 && len(sp[2]) > 0 {\n\t\t\tendpoint.AMQP.URI = endpoint.AMQP.URI + \"/\" + sp[1]\n\t\t\tsp = append([]string{endpoint.AMQP.URI}, sp[2:]...)\n\t\t}\n\n\t\t// Bind queue name with no namespace\n\t\tif len(sp) > 1 {\n\t\t\tvar err error\n\t\t\tendpoint.AMQP.QueueName, err = url.QueryUnescape(sp[1])\n\t\t\tif err != nil {\n\t\t\t\treturn endpoint, errors.New(\"invalid AMQP queue name\")\n\t\t\t}\n\t\t}\n\n\t\t// Parsing additional attributes\n\t\tif len(sqp) > 1 {\n\t\t\tm, err := url.ParseQuery(sqp[1])\n\t\t\tif err != nil {\n\t\t\t\treturn endpoint, errors.New(\"invalid AMQP url\")\n\t\t\t}\n\t\t\tfor key, val := range m {\n\t\t\t\tif len(val) == 0 {\n\t\t\t\t\tcontinue\n\t\t\t\t}\n\t\t\t\tswitch key {\n\t\t\t\tcase \"route\":\n\t\t\t\t\tendpoint.AMQP.RouteKey = val[0]\n\t\t\t\tcase \"type\":\n\t\t\t\t\tendpoint.AMQP.Type = val[0]\n\t\t\t\tcase \"durable\":\n\t\t\t\t\tendpoint.AMQP.Durable = queryBool(val[0])\n\t\t\t\tcase \"internal\":\n\t\t\t\t\tendpoint.AMQP.Internal = queryBool(val[0])\n\t\t\t\tcase \"no_wait\":\n\t\t\t\t\tendpoint.AMQP.NoWait = queryBool(val[0])\n\t\t\t\tcase \"auto_delete\":\n\t\t\t\t\tendpoint.AMQP.AutoDelete = queryBool(val[0])\n\t\t\t\tcase \"immediate\":\n\t\t\t\t\tendpoint.AMQP.Immediate = queryBool(val[0])\n\t\t\t\tcase \"mandatory\":\n\t\t\t\t\tendpoint.AMQP.Mandatory = queryBool(val[0])\n\t\t\t\tcase \"delivery_mode\":\n\t\t\t\t\tendpoint.AMQP.DeliveryMode = uint8(queryInt(val[0]))\n\t\t\t\tcase \"priority\":\n\t\t\t\t\tendpoint.AMQP.Priority = uint8(queryInt(val[0]))\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\n\t\tif strings.HasPrefix(endpoint.Original, \"amqps:\") {\n\t\t\tendpoint.AMQP.SSL = true\n\t\t}\n\n\t\tif endpoint.AMQP.QueueName == \"\" {\n\t\t\treturn endpoint, errors.New(\"missing AMQP queue name\")\n\t\t}\n\n\t\tif endpoint.AMQP.RouteKey == \"\" {\n\t\t\tendpoint.AMQP.RouteKey = \"tile38\"\n\t\t}\n\t}\n\n\t// Basic NATS connection strings in HOOKS interface\n\t// nats://://?params=value\n\t//\n\t// params are:\n\t//\n\t// user - username\n\t// pass - password\n\t// when user or pass is not set then login without password is used\n\tif endpoint.Protocol == NATS {\n\t\t// Parsing connection from URL string\n\t\thp := strings.Split(s, \":\")\n\t\tswitch len(hp) {\n\t\tdefault:\n\t\t\treturn endpoint, errors.New(\"invalid SQS url\")\n\t\tcase 2:\n\t\t\tendpoint.NATS.Host = hp[0]\n\t\t\tport, err := strconv.Atoi(hp[1])\n\t\t\tif err != nil {\n\t\t\t\tendpoint.NATS.Port = 4222 // default nats port\n\t\t\t} else {\n\t\t\t\tendpoint.NATS.Port = port\n\t\t\t}\n\t\t}\n\n\t\t// Parsing NATS topic name\n\t\tif len(sp) > 1 {\n\t\t\tvar err error\n\t\t\tendpoint.NATS.Topic, err = url.QueryUnescape(sp[1])\n\t\t\tif err != nil {\n\t\t\t\treturn endpoint, errors.New(\"invalid NATS topic name\")\n\t\t\t}\n\t\t}\n\n\t\t// Parsing additional params\n\t\tif len(sqp) > 1 {\n\t\t\tm, err := url.ParseQuery(sqp[1])\n\t\t\tif err != nil {\n\t\t\t\treturn endpoint, errors.New(\"invalid NATS url\")\n\t\t\t}\n\t\t\tfor key, val := range m {\n\t\t\t\tif len(val) == 0 {\n\t\t\t\t\tcontinue\n\t\t\t\t}\n\t\t\t\tswitch key {\n\t\t\t\tcase \"user\":\n\t\t\t\t\tendpoint.NATS.User = val[0]\n\t\t\t\tcase \"pass\":\n\t\t\t\t\tendpoint.NATS.Pass = val[0]\n\t\t\t\tcase \"token\":\n\t\t\t\t\tendpoint.NATS.Token = val[0]\n\t\t\t\tcase \"secure\":\n\t\t\t\t\tendpoint.NATS.Secure = queryBool(val[0])\n\t\t\t\tcase \"credential\":\n\t\t\t\t\tendpoint.NATS.UserCredentialPath = val[0]\n\t\t\t\tcase \"jetstream\":\n\t\t\t\t\tendpoint.NATS.Jetstream = queryBool(val[0])\n\t\t\t\tcase \"tls\":\n\t\t\t\t\tendpoint.NATS.TLS = queryBool(val[0])\n\t\t\t\tcase \"tlscert\":\n\t\t\t\t\tendpoint.NATS.TLSCert = val[0]\n\t\t\t\tcase \"tlskey\":\n\t\t\t\t\tendpoint.NATS.TLSKey = val[0]\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\n\tif endpoint.Protocol == EventHub {\n\t\tdp := strings.Split(endpoint.Original, \";\")\n\t\tif len(dp) != 4 {\n\t\t\treturn endpoint, errors.New(\"malformed EventHub connection string\")\n\t\t}\n\n\t\tsakn := strings.Split(dp[1], \"=\")\n\t\tif sakn[0] != \"SharedAccessKeyName\" {\n\t\t\treturn endpoint, errors.New(\"missing SharedAccessKeyName\")\n\t\t}\n\n\t\tsak := strings.Split(dp[2], \"=\")\n\t\tif sak[0] != \"SharedAccessKey\" {\n\t\t\treturn endpoint, errors.New(\"missing SharedAccessKey\")\n\t\t}\n\n\t\tep := strings.Split(dp[3], \"=\")\n\t\tif ep[0] != \"EntityPath\" {\n\t\t\treturn endpoint, errors.New(\"missing EntityPath\")\n\t\t}\n\n\t\tendpoint.EventHub.ConnectionString = endpoint.Original\n\t}\n\n\t// Basic CF Queue connection strings in HOOKS interface\n\t// cf-queue:///?token=\n\t//\n\t// params are:\n\t//\n\t// token - API token\n\tif endpoint.Protocol == CFQueue {\n\t\t// Parse account_id/queue_id from the path parts\n\t\tif len(sp) < 2 {\n\t\t\treturn endpoint, errors.New(\"invalid CF Queue format, should be account_id/queue_id\")\n\t\t}\n\t\tendpoint.CFQueue.AccountID = sp[0]\n\t\tendpoint.CFQueue.QueueID = sp[1]\n\n\t\t// Parse query parameters for API token\n\t\tif len(sqp) > 1 {\n\t\t\tm, err := url.ParseQuery(sqp[1])\n\t\t\tif err != nil {\n\t\t\t\treturn endpoint, errors.New(\"invalid CF Queue url\")\n\t\t\t}\n\t\t\tfor key, val := range m {\n\t\t\t\tif len(val) == 0 {\n\t\t\t\t\tcontinue\n\t\t\t\t}\n\t\t\t\tswitch key {\n\t\t\t\tcase \"token\":\n\t\t\t\t\tendpoint.CFQueue.APIToken = val[0]\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\n\t\tif endpoint.CFQueue.AccountID == \"\" {\n\t\t\treturn endpoint, errors.New(\"missing CF Queue account ID\")\n\t\t}\n\t\tif endpoint.CFQueue.QueueID == \"\" {\n\t\t\treturn endpoint, errors.New(\"missing CF Queue queue ID\")\n\t\t}\n\t\tif endpoint.CFQueue.APIToken == \"\" {\n\t\t\treturn endpoint, errors.New(\"missing CF Queue API token\")\n\t\t}\n\t}\n\n\treturn endpoint, nil\n}\n\nfunc queryInt(s string) int {\n\tx, _ := strconv.ParseInt(s, 10, 64)\n\treturn int(x)\n}\n\nfunc queryBool(s string) bool {\n\tif len(s) > 0 {\n\t\tif s[0] >= '1' && s[0] <= '9' {\n\t\t\treturn true\n\t\t}\n\t\tswitch s[0] {\n\t\tcase 'Y', 'y', 'T', 't':\n\t\t\treturn true\n\t\t}\n\t}\n\treturn false\n}\n" }, { "path": "internal/endpoint/eventHub.go", "content": "package endpoint\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"time\"\n\n\t\"github.com/tidwall/gjson\"\n\n\teventhub \"github.com/Azure/azure-event-hubs-go/v3\"\n)\n\nconst ()\n\n// HTTPConn is an endpoint connection\ntype EvenHubConn struct {\n\tep Endpoint\n}\n\nfunc newEventHubConn(ep Endpoint) *EvenHubConn {\n\treturn &EvenHubConn{\n\t\tep: ep,\n\t}\n}\n\n// Expired returns true if the connection has expired\nfunc (conn *EvenHubConn) Expired() bool {\n\treturn false\n}\n\n// ExpireNow forces the connection to expire\nfunc (conn *EvenHubConn) ExpireNow() {\n}\n\n// Send sends a message\nfunc (conn *EvenHubConn) Send(msg string) error {\n\thub, err := eventhub.NewHubFromConnectionString(conn.ep.EventHub.ConnectionString)\n\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tctx, cancel := context.WithTimeout(context.Background(), 20*time.Second)\n\tdefer cancel()\n\n\t// parse json again to get out info for our kafka key\n\tkey := gjson.Get(msg, \"key\")\n\tid := gjson.Get(msg, \"id\")\n\tkeyValue := fmt.Sprintf(\"%s-%s\", key.String(), id.String())\n\n\tevtHubMsg := eventhub.NewEventFromString(msg)\n\tevtHubMsg.PartitionKey = &keyValue\n\terr = hub.Send(ctx, evtHubMsg)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\treturn nil\n}\n" }, { "path": "internal/endpoint/grpc.go", "content": "package endpoint\n\nimport (\n\t\"errors\"\n\t\"fmt\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/tidwall/tile38/internal/hservice\"\n\t\"golang.org/x/net/context\"\n\t\"google.golang.org/grpc\"\n)\n\nconst grpcExpiresAfter = time.Second * 30\n\n// GRPCConn is an endpoint connection\ntype GRPCConn struct {\n\tmu sync.Mutex\n\tep Endpoint\n\tex bool\n\tt time.Time\n\tconn *grpc.ClientConn\n\tsconn hservice.HookServiceClient\n}\n\nfunc newGRPCConn(ep Endpoint) *GRPCConn {\n\treturn &GRPCConn{\n\t\tep: ep,\n\t\tt: time.Now(),\n\t}\n}\n\n// Expired returns true if the connection has expired\nfunc (conn *GRPCConn) Expired() bool {\n\tconn.mu.Lock()\n\tdefer conn.mu.Unlock()\n\tif !conn.ex {\n\t\tif time.Since(conn.t) > grpcExpiresAfter {\n\t\t\tconn.close()\n\t\t\tconn.ex = true\n\t\t}\n\t}\n\treturn conn.ex\n}\n\n// ExpireNow forces the connection to expire\nfunc (conn *GRPCConn) ExpireNow() {\n\tconn.mu.Lock()\n\tdefer conn.mu.Unlock()\n\tconn.close()\n\tconn.ex = true\n}\n\nfunc (conn *GRPCConn) close() {\n\tif conn.conn != nil {\n\t\tconn.conn.Close()\n\t\tconn.conn = nil\n\t}\n}\n\n// Send sends a message\nfunc (conn *GRPCConn) Send(msg string) error {\n\tconn.mu.Lock()\n\tdefer conn.mu.Unlock()\n\tif conn.ex {\n\t\treturn errExpired\n\t}\n\tconn.t = time.Now()\n\tif conn.conn == nil {\n\t\taddr := fmt.Sprintf(\"%s:%d\", conn.ep.GRPC.Host, conn.ep.GRPC.Port)\n\t\tvar err error\n\t\tconn.conn, err = grpc.Dial(addr, grpc.WithInsecure())\n\t\tif err != nil {\n\t\t\tconn.close()\n\t\t\treturn err\n\t\t}\n\t\tconn.sconn = hservice.NewHookServiceClient(conn.conn)\n\t}\n\tr, err := conn.sconn.Send(context.Background(), &hservice.MessageRequest{Value: msg})\n\tif err != nil {\n\t\tconn.close()\n\t\treturn err\n\t}\n\tif !r.Ok {\n\t\tconn.close()\n\t\treturn errors.New(\"invalid grpc reply\")\n\t}\n\treturn nil\n}\n" }, { "path": "internal/endpoint/http.go", "content": "package endpoint\n\nimport (\n\t\"bytes\"\n\t\"fmt\"\n\t\"io\"\n\t\"net/http\"\n\t\"time\"\n)\n\nconst (\n\thttpExpiresAfter = time.Second * 30\n\thttpRequestTimeout = time.Second * 5\n\thttpMaxIdleConnections = 20\n)\n\n// HTTPConn is an endpoint connection\ntype HTTPConn struct {\n\tep Endpoint\n\tclient *http.Client\n}\n\nfunc newHTTPConn(ep Endpoint) *HTTPConn {\n\treturn &HTTPConn{\n\t\tep: ep,\n\t\tclient: &http.Client{\n\t\t\tTransport: &http.Transport{\n\t\t\t\tMaxIdleConnsPerHost: httpMaxIdleConnections,\n\t\t\t\tIdleConnTimeout: httpExpiresAfter,\n\t\t\t},\n\t\t\tTimeout: httpRequestTimeout,\n\t\t},\n\t}\n}\n\n// Expired returns true if the connection has expired\nfunc (conn *HTTPConn) Expired() bool {\n\treturn false\n}\n\n// ExpireNow forces the connection to expire\nfunc (conn *HTTPConn) ExpireNow() {\n}\n\n// Send sends a message\nfunc (conn *HTTPConn) Send(msg string) error {\n\treq, err := http.NewRequest(\"POST\", conn.ep.Original, bytes.NewBufferString(msg))\n\tif err != nil {\n\t\treturn err\n\t}\n\n\treq.Header.Set(\"Content-Type\", \"application/json\")\n\tresp, err := conn.client.Do(req)\n\tif err != nil {\n\t\treturn err\n\t}\n\t// close the connection to reuse it\n\tdefer resp.Body.Close()\n\t// discard response\n\tif _, err := io.Copy(io.Discard, resp.Body); err != nil {\n\t\treturn err\n\t}\n\t// Only allow responses with status code 200, 201, and 202\n\tif resp.StatusCode != http.StatusOK &&\n\t\tresp.StatusCode != http.StatusCreated &&\n\t\tresp.StatusCode != http.StatusAccepted {\n\t\treturn fmt.Errorf(\"invalid status: %s\", resp.Status)\n\t}\n\treturn nil\n}\n" }, { "path": "internal/endpoint/kafka.go", "content": "package endpoint\n\nimport (\n\t\"crypto/tls\"\n\t\"crypto/x509\"\n\t\"errors\"\n\t\"fmt\"\n\t\"os\"\n\t\"sync\"\n\t\"time\"\n\n\tlg \"log\"\n\n\t\"github.com/IBM/sarama\"\n\t\"github.com/tidwall/gjson\"\n\t\"github.com/tidwall/tile38/internal/log\"\n)\n\nconst kafkaExpiresAfter = time.Second * 30\n\n// KafkaConn is an endpoint connection\ntype KafkaConn struct {\n\tmu sync.Mutex\n\tep Endpoint\n\tconn sarama.SyncProducer\n\tcfg *sarama.Config\n\tex bool\n\tt time.Time\n}\n\n// Expired returns true if the connection has expired\nfunc (conn *KafkaConn) Expired() bool {\n\tconn.mu.Lock()\n\tdefer conn.mu.Unlock()\n\tif !conn.ex {\n\t\tif time.Since(conn.t) > kafkaExpiresAfter {\n\t\t\tconn.close()\n\t\t\tconn.ex = true\n\t\t}\n\t}\n\treturn conn.ex\n}\n\n// ExpireNow forces the connection to expire\nfunc (conn *KafkaConn) ExpireNow() {\n\tconn.mu.Lock()\n\tdefer conn.mu.Unlock()\n\tconn.close()\n\tconn.ex = true\n}\n\nfunc (conn *KafkaConn) close() {\n\tif conn.conn != nil {\n\t\tconn.conn.Close()\n\t\tconn.conn = nil\n\t\tconn.cfg.MetricRegistry.UnregisterAll()\n\t\tconn.cfg = nil\n\t}\n}\n\n// Send sends a message\nfunc (conn *KafkaConn) Send(msg string) error {\n\tconn.mu.Lock()\n\tdefer conn.mu.Unlock()\n\n\tif conn.ex {\n\t\treturn errExpired\n\t}\n\tconn.t = time.Now()\n\n\tif log.Level() > 2 {\n\t\tsarama.Logger = lg.New(log.Output(), \"[sarama] \", 0)\n\t}\n\n\turi := fmt.Sprintf(\"%s:%d\", conn.ep.Kafka.Host, conn.ep.Kafka.Port)\n\tif conn.conn == nil {\n\t\tcfg := sarama.NewConfig()\n\n\t\tcfg.Net.DialTimeout = time.Second\n\t\tcfg.Net.ReadTimeout = time.Second * 5\n\t\tcfg.Net.WriteTimeout = time.Second * 5\n\t\t// Fix #333 : fix backward incompatibility introduced by sarama library\n\t\tcfg.Producer.Return.Successes = true\n\t\t// Sarama now sets the version based on the broker version in the release 1.46.0 as of August 25 2025.\n\t\t// There is no need to force this anymore as it is breaking the version check in Kafka 4.0 and above.\n\t\t// cfg.Version = sarama.V0_10_0_0\n\n\t\tswitch conn.ep.Kafka.Auth {\n\t\tcase \"sasl\":\n\t\t\t// This path allows to either provide a custom ca certificate\n\t\t\t// or, because RootCAs is nil, is using the hosts ca set\n\t\t\t// to verify the server certificate\n\t\t\tif conn.ep.Kafka.SSL {\n\t\t\t\ttlsConfig := tls.Config{}\n\n\t\t\t\tif conn.ep.Kafka.CACertFile != \"\" {\n\t\t\t\t\tcaCertPool, err := loadRootTLSCert(conn.ep.Kafka.CACertFile)\n\t\t\t\t\tif err != nil {\n\t\t\t\t\t\treturn err\n\t\t\t\t\t}\n\t\t\t\t\ttlsConfig.RootCAs = &caCertPool\n\t\t\t\t}\n\n\t\t\t\tcfg.Net.TLS.Enable = true\n\t\t\t\tcfg.Net.TLS.Config = &tlsConfig\n\t\t\t}\n\n\t\t\tcfg.Net.SASL.Enable = true\n\t\t\tcfg.Net.SASL.User = os.Getenv(\"KAFKA_USERNAME\")\n\t\t\tcfg.Net.SASL.Password = os.Getenv(\"KAFKA_PASSWORD\")\n\t\t\tcfg.Net.SASL.Handshake = true\n\t\t\tcfg.Net.SASL.Mechanism = sarama.SASLTypePlaintext\n\n\t\t\tif conn.ep.Kafka.SASLSHA256 {\n\t\t\t\tcfg.Net.SASL.SCRAMClientGeneratorFunc = func() sarama.SCRAMClient { return &XDGSCRAMClient{HashGeneratorFcn: SHA256} }\n\t\t\t\tcfg.Net.SASL.Mechanism = sarama.SASLTypeSCRAMSHA256\n\t\t\t}\n\t\t\tif conn.ep.Kafka.SASLSHA512 {\n\t\t\t\tcfg.Net.SASL.SCRAMClientGeneratorFunc = func() sarama.SCRAMClient { return &XDGSCRAMClient{HashGeneratorFcn: SHA512} }\n\t\t\t\tcfg.Net.SASL.Mechanism = sarama.SASLTypeSCRAMSHA512\n\t\t\t}\n\t\tcase \"tls\":\n\t\t\ttlsConfig := tls.Config{}\n\t\t\tcfg.Net.TLS.Enable = true\n\n\t\t\tcertificates, err := loadClientTLSCert(conn.ep.Kafka.KeyFile, conn.ep.Kafka.CertFile)\n\t\t\tif err != nil {\n\t\t\t\tcfg.MetricRegistry.UnregisterAll()\n\t\t\t\treturn err\n\t\t\t}\n\t\t\ttlsConfig.Certificates = certificates\n\n\t\t\t// This path allows to either provide a custom ca certificate\n\t\t\t// or, because RootCAs is nil, is using the hosts ca set\n\t\t\t// to verify server certificate\n\t\t\tif conn.ep.Kafka.CACertFile != \"\" {\n\t\t\t\tcaCertPool, err := loadRootTLSCert(conn.ep.Kafka.CACertFile)\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\t\t\t\ttlsConfig.RootCAs = &caCertPool\n\t\t\t}\n\n\t\t\tcfg.Net.TLS.Config = &tlsConfig\n\t\tcase \"none\":\n\t\t\t// This path allows to either provide a custom ca certificate\n\t\t\t// or, because RootCAs is nil, is using the hosts ca set\n\t\t\t// to verify the server certificate\n\t\t\tif conn.ep.Kafka.SSL {\n\t\t\t\ttlsConfig := tls.Config{}\n\n\t\t\t\tif conn.ep.Kafka.CACertFile != \"\" {\n\t\t\t\t\tcaCertPool, err := loadRootTLSCert(conn.ep.Kafka.CACertFile)\n\t\t\t\t\tif err != nil {\n\t\t\t\t\t\treturn err\n\t\t\t\t\t}\n\t\t\t\t\ttlsConfig.RootCAs = &caCertPool\n\t\t\t\t}\n\n\t\t\t\tcfg.Net.TLS.Enable = true\n\t\t\t\tcfg.Net.TLS.Config = &tlsConfig\n\t\t\t}\n\t\t}\n\n\t\tc, err := sarama.NewSyncProducer([]string{uri}, cfg)\n\t\tif err != nil {\n\t\t\tcfg.MetricRegistry.UnregisterAll()\n\t\t\treturn err\n\t\t}\n\n\t\tconn.conn = c\n\t\tconn.cfg = cfg\n\t}\n\n\t// parse json again to get out info for our kafka key\n\tkey := gjson.Get(msg, \"key\")\n\tid := gjson.Get(msg, \"id\")\n\tkeyValue := fmt.Sprintf(\"%s-%s\", key.String(), id.String())\n\n\tmessage := &sarama.ProducerMessage{\n\t\tTopic: conn.ep.Kafka.TopicName,\n\t\tKey: sarama.StringEncoder(keyValue),\n\t\tValue: sarama.StringEncoder(msg),\n\t}\n\n\t_, offset, err := conn.conn.SendMessage(message)\n\tif err != nil {\n\t\tconn.close()\n\t\treturn err\n\t}\n\n\tif offset < 0 {\n\t\tconn.close()\n\t\treturn errors.New(\"invalid kafka reply\")\n\t}\n\n\treturn nil\n}\n\nfunc newKafkaConn(ep Endpoint) *KafkaConn {\n\treturn &KafkaConn{\n\t\tep: ep,\n\t\tt: time.Now(),\n\t}\n}\n\nfunc loadClientTLSCert(KeyFile, CertFile string) ([]tls.Certificate, error) {\n\t// load client cert\n\tcert, err := tls.LoadX509KeyPair(CertFile, KeyFile)\n\n\tif err != nil {\n\t\treturn []tls.Certificate{cert}, err\n\t}\n\n\treturn []tls.Certificate{cert}, err\n}\n\nfunc loadRootTLSCert(CACertFile string) (x509.CertPool, error) {\n\t// Load CA cert\n\tcaCert, err := os.ReadFile(CACertFile)\n\n\tif err != nil {\n\t\treturn x509.CertPool{}, err\n\t}\n\n\tcaCertPool := x509.NewCertPool()\n\tcaCertPool.AppendCertsFromPEM(caCert)\n\treturn *caCertPool, err\n}\n" }, { "path": "internal/endpoint/local.go", "content": "package endpoint\n\n// LocalPublisher is used to publish local notifications\ntype LocalPublisher interface {\n\tPublish(channel string, message ...string) int\n}\n\n// LocalConn is an endpoint connection\ntype LocalConn struct {\n\tep Endpoint\n\tpublisher LocalPublisher\n}\n\nfunc newLocalConn(ep Endpoint, publisher LocalPublisher) *LocalConn {\n\treturn &LocalConn{\n\t\tep: ep,\n\t\tpublisher: publisher,\n\t}\n}\n\n// Expired returns true if the connection has expired\nfunc (conn *LocalConn) Expired() bool {\n\treturn false\n}\n\n// ExpireNow forces the connection to expire\nfunc (conn *LocalConn) ExpireNow() {\n}\n\n// Send sends a message\nfunc (conn *LocalConn) Send(msg string) error {\n\tconn.publisher.Publish(conn.ep.Local.Channel, msg)\n\treturn nil\n}\n" }, { "path": "internal/endpoint/mqtt.go", "content": "package endpoint\n\nimport (\n\t\"crypto/tls\"\n\t\"crypto/x509\"\n\t\"fmt\"\n\t\"math/rand\"\n\t\"os\"\n\t\"sync\"\n\t\"time\"\n\n\tpaho \"github.com/eclipse/paho.mqtt.golang\"\n\t\"github.com/tidwall/tile38/internal/log\"\n)\n\nconst (\n\tmqttExpiresAfter = time.Second * 30\n\tmqttPublishTimeout = time.Second * 5\n)\n\n// MQTTConn is an endpoint connection\ntype MQTTConn struct {\n\tmu sync.Mutex\n\tep Endpoint\n\tconn paho.Client\n\tex bool\n\tt time.Time\n}\n\n// Expired returns true if the connection has expired\nfunc (conn *MQTTConn) Expired() bool {\n\tconn.mu.Lock()\n\tdefer conn.mu.Unlock()\n\tif !conn.ex {\n\t\tif time.Since(conn.t) > mqttExpiresAfter {\n\t\t\tconn.close()\n\t\t\tconn.ex = true\n\t\t}\n\t}\n\treturn conn.ex\n}\n\n// ExpireNow forces the connection to expire\nfunc (conn *MQTTConn) ExpireNow() {\n\tconn.mu.Lock()\n\tdefer conn.mu.Unlock()\n\tconn.close()\n\tconn.ex = true\n}\n\nfunc (conn *MQTTConn) close() {\n\tif conn.conn != nil {\n\t\tif conn.conn.IsConnected() {\n\t\t\tconn.conn.Disconnect(250)\n\t\t}\n\t\tconn.conn = nil\n\t}\n}\n\n// Send sends a message\nfunc (conn *MQTTConn) Send(msg string) error {\n\tconn.mu.Lock()\n\tdefer conn.mu.Unlock()\n\n\tif conn.ex {\n\t\treturn errExpired\n\t}\n\tconn.t = time.Now()\n\n\tif conn.conn == nil {\n\t\turi := fmt.Sprintf(\"tcp://%s:%d\", conn.ep.MQTT.Host, conn.ep.MQTT.Port)\n\t\tops := paho.NewClientOptions()\n\t\tif conn.ep.MQTT.CertFile != \"\" || conn.ep.MQTT.KeyFile != \"\" ||\n\t\t\tconn.ep.MQTT.CACertFile != \"\" {\n\t\t\tvar config tls.Config\n\t\t\tif conn.ep.MQTT.CertFile != \"\" || conn.ep.MQTT.KeyFile != \"\" {\n\t\t\t\tcert, err := tls.LoadX509KeyPair(conn.ep.MQTT.CertFile,\n\t\t\t\t\tconn.ep.MQTT.KeyFile)\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\t\t\t\tconfig.Certificates = append(config.Certificates, cert)\n\t\t\t}\n\t\t\tif conn.ep.MQTT.CACertFile != \"\" {\n\t\t\t\t// Load CA cert\n\t\t\t\tcaCert, err := os.ReadFile(conn.ep.MQTT.CACertFile)\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\t\t\t\tcaCertPool := x509.NewCertPool()\n\t\t\t\tcaCertPool.AppendCertsFromPEM(caCert)\n\t\t\t\tconfig.RootCAs = caCertPool\n\t\t\t}\n\t\t\tops = ops.SetTLSConfig(&config)\n\t\t}\n\t\t//generate UUID for the client-id.\n\t\tb := make([]byte, 16)\n\t\t_, err := rand.Read(b)\n\t\tif err != nil {\n\t\t\tlog.Debugf(\"Failed to generate guid for the mqtt client. The endpoint will not work\")\n\t\t\treturn err\n\t\t}\n\t\tuuid := fmt.Sprintf(\"tile38-%x-%x-%x-%x-%x\", b[0:4], b[4:6], b[6:8], b[8:10], b[10:])\n\n\t\tops = ops.SetClientID(uuid).AddBroker(uri)\n\t\tc := paho.NewClient(ops)\n\n\t\tif token := c.Connect(); token.Wait() && token.Error() != nil {\n\t\t\treturn token.Error()\n\t\t}\n\n\t\tconn.conn = c\n\t}\n\n\tt := conn.conn.Publish(conn.ep.MQTT.QueueName, conn.ep.MQTT.Qos,\n\t\tconn.ep.MQTT.Retained, msg)\n\n\tif !t.WaitTimeout(mqttPublishTimeout) || t.Error() != nil {\n\t\tconn.close()\n\t\treturn t.Error()\n\t}\n\n\treturn nil\n}\n\nfunc newMQTTConn(ep Endpoint) *MQTTConn {\n\treturn &MQTTConn{\n\t\tep: ep,\n\t\tt: time.Now(),\n\t}\n}\n" }, { "path": "internal/endpoint/nats.go", "content": "package endpoint\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/nats-io/nats.go\"\n\t\"github.com/nats-io/nats.go/jetstream\"\n)\n\nconst natsExpiresAfter = time.Second * 30\n\n// NATSConn is an endpoint connection\ntype NATSConn struct {\n\tmu sync.Mutex\n\tep Endpoint\n\tex bool\n\tt time.Time\n\tconn *nats.Conn\n\tjs jetstream.JetStream\n}\n\nfunc newNATSConn(ep Endpoint) *NATSConn {\n\treturn &NATSConn{\n\t\tep: ep,\n\t\tt: time.Now(),\n\t}\n}\n\n// Expired returns true if the connection has expired\nfunc (conn *NATSConn) Expired() bool {\n\tconn.mu.Lock()\n\tdefer conn.mu.Unlock()\n\tif !conn.ex {\n\t\tif time.Since(conn.t) > natsExpiresAfter {\n\t\t\tconn.close()\n\t\t\tconn.ex = true\n\t\t}\n\t}\n\treturn conn.ex\n}\n\n// ExpireNow forces the connection to expire\nfunc (conn *NATSConn) ExpireNow() {\n\tconn.mu.Lock()\n\tdefer conn.mu.Unlock()\n\tconn.close()\n\tconn.ex = true\n}\n\nfunc (conn *NATSConn) close() {\n\tif conn.conn != nil {\n\t\tconn.conn.Close()\n\t\tconn.conn = nil\n\t}\n}\n\n// Send sends a message\nfunc (conn *NATSConn) Send(msg string) error {\n\tconn.mu.Lock()\n\tdefer conn.mu.Unlock()\n\tif conn.ex {\n\t\treturn errExpired\n\t}\n\tconn.t = time.Now()\n\tif conn.conn == nil {\n\t\taddr := fmt.Sprintf(\"%s:%d\", conn.ep.NATS.Host, conn.ep.NATS.Port)\n\t\tscheme := \"nats\" // 'nats://' by default\n\t\tvar err error\n\t\tvar opts []nats.Option\n\t\tif conn.ep.NATS.User != \"\" && conn.ep.NATS.Pass != \"\" {\n\t\t\topts = append(opts, nats.UserInfo(conn.ep.NATS.User, conn.ep.NATS.Pass))\n\t\t}\n\t\tif conn.ep.NATS.TLS {\n\t\t\topts = append(opts, nats.ClientCert(\n\t\t\t\tconn.ep.NATS.TLSCert, conn.ep.NATS.TLSKey,\n\t\t\t))\n\t\t}\n\t\tif conn.ep.NATS.Token != \"\" {\n\t\t\topts = append(opts, nats.Token(conn.ep.NATS.Token))\n\t\t}\n\t\tif conn.ep.NATS.UserCredentialPath != \"\" {\n\t\t\topts = append(opts, nats.UserCredentials(conn.ep.NATS.UserCredentialPath))\n\t\t}\n\n\t\tif conn.ep.NATS.Secure {\n\t\t\tscheme = \"tls\"\n\t\t}\n\n\t\taddr = fmt.Sprintf(\"%s://%s\", scheme, addr)\n\t\tconn.conn, err = nats.Connect(addr, opts...)\n\t\tif err != nil {\n\t\t\tconn.close()\n\t\t\treturn err\n\t\t}\n\n\t\tif conn.ep.NATS.Jetstream {\n\t\t\tconn.js, err = jetstream.New(conn.conn)\n\t\t\tif err != nil {\n\t\t\t\tconn.close()\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\t}\n\n\tif conn.js == nil {\n\t\treturn conn.publish(msg)\n\t}\n\n\treturn conn.publishJS(msg)\n}\n\n// publishJS will publish the message to the subject using core nats.\nfunc (conn *NATSConn) publish(msg string) error {\n\terr := conn.conn.Publish(conn.ep.NATS.Topic, []byte(msg))\n\tif err != nil {\n\t\tconn.close()\n\t\treturn err\n\t}\n\n\treturn nil\n}\n\n// publishJS will publish the message expecting a jetstream acknowledgement.\nfunc (conn *NATSConn) publishJS(msg string) error {\n\tctx, cancel := context.WithTimeout(context.Background(), 20*time.Second)\n\tdefer cancel()\n\n\t_, err := conn.js.Publish(ctx, conn.ep.NATS.Topic, []byte(msg))\n\tif err != nil {\n\t\tconn.close()\n\t\treturn err\n\t}\n\n\treturn nil\n}\n" }, { "path": "internal/endpoint/pubsub.go", "content": "package endpoint\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"sync\"\n\t\"time\"\n\n\t\"cloud.google.com/go/pubsub\"\n\t\"google.golang.org/api/option\"\n)\n\nconst pubsubExpiresAfter = time.Second * 30\n\n// SQSConn is an endpoint connection\ntype PubSubConn struct {\n\tmu sync.Mutex\n\tep Endpoint\n\tsvc *pubsub.Client\n\ttopic *pubsub.Topic\n\tex bool\n\tt time.Time\n}\n\nfunc (conn *PubSubConn) close() {\n\tif conn.svc != nil {\n\t\tconn.svc.Close()\n\t\tconn.svc = nil\n\t}\n}\n\n// Send sends a message\nfunc (conn *PubSubConn) Send(msg string) error {\n\tconn.mu.Lock()\n\tdefer conn.mu.Unlock()\n\n\tif conn.ex {\n\t\treturn errExpired\n\t}\n\n\tctx := context.Background()\n\n\tconn.t = time.Now()\n\n\tif conn.svc == nil {\n\t\tvar creds option.ClientOption\n\t\tvar svc *pubsub.Client\n\t\tvar err error\n\t\tcredPath := conn.ep.PubSub.CredPath\n\n\t\tif credPath != \"\" {\n\t\t\tcreds = option.WithCredentialsFile(credPath)\n\t\t\tsvc, err = pubsub.NewClient(ctx, conn.ep.PubSub.Project, creds)\n\t\t} else {\n\t\t\tsvc, err = pubsub.NewClient(ctx, conn.ep.PubSub.Project)\n\t\t}\n\n\t\tif err != nil {\n\t\t\tfmt.Println(err)\n\t\t\treturn err\n\t\t}\n\n\t\ttopic := svc.Topic(conn.ep.PubSub.Topic)\n\n\t\tconn.svc = svc\n\t\tconn.topic = topic\n\t}\n\n\t// Send message\n\tres := conn.topic.Publish(ctx, &pubsub.Message{\n\t\tData: []byte(msg),\n\t})\n\t_, err := res.Get(ctx)\n\tif err != nil {\n\t\tfmt.Println(err)\n\t\treturn err\n\t}\n\treturn nil\n}\n\nfunc (conn *PubSubConn) Expired() bool {\n\tconn.mu.Lock()\n\tdefer conn.mu.Unlock()\n\tif !conn.ex {\n\t\tif time.Since(conn.t) > pubsubExpiresAfter {\n\t\t\tconn.close()\n\t\t\tconn.ex = true\n\t\t}\n\t}\n\treturn conn.ex\n}\n\n// ExpireNow forces the connection to expire\nfunc (conn *PubSubConn) ExpireNow() {\n\tconn.mu.Lock()\n\tdefer conn.mu.Unlock()\n\tconn.close()\n\tconn.ex = true\n}\n\nfunc newPubSubConn(ep Endpoint) *PubSubConn {\n\treturn &PubSubConn{\n\t\tep: ep,\n\t\tt: time.Now(),\n\t}\n}\n" }, { "path": "internal/endpoint/redis.go", "content": "package endpoint\n\nimport (\n\t\"fmt\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/gomodule/redigo/redis\"\n)\n\nconst redisExpiresAfter = time.Second * 30\n\n// RedisConn is an endpoint connection\ntype RedisConn struct {\n\tmu sync.Mutex\n\tep Endpoint\n\tex bool\n\tt time.Time\n\tconn redis.Conn\n}\n\nfunc newRedisConn(ep Endpoint) *RedisConn {\n\treturn &RedisConn{\n\t\tep: ep,\n\t\tt: time.Now(),\n\t}\n}\n\n// Expired returns true if the connection has expired\nfunc (conn *RedisConn) Expired() bool {\n\tconn.mu.Lock()\n\tdefer conn.mu.Unlock()\n\tif !conn.ex {\n\t\tif time.Since(conn.t) > redisExpiresAfter {\n\t\t\tconn.close()\n\t\t\tconn.ex = true\n\t\t}\n\t}\n\treturn conn.ex\n}\n\n// ExpireNow forces the connection to expire\nfunc (conn *RedisConn) ExpireNow() {\n\tconn.mu.Lock()\n\tdefer conn.mu.Unlock()\n\tconn.close()\n\tconn.ex = true\n}\n\nfunc (conn *RedisConn) close() {\n\tif conn.conn != nil {\n\t\tconn.conn.Close()\n\t\tconn.conn = nil\n\t}\n}\n\n// Send sends a message\nfunc (conn *RedisConn) Send(msg string) error {\n\tconn.mu.Lock()\n\tdefer conn.mu.Unlock()\n\n\tif conn.ex {\n\t\treturn errExpired\n\t}\n\tconn.t = time.Now()\n\tif conn.conn == nil {\n\t\taddr := fmt.Sprintf(\"%s:%d\", conn.ep.Redis.Host, conn.ep.Redis.Port)\n\t\tvar err error\n\t\tconn.conn, err = redis.Dial(\"tcp\", addr)\n\t\tif err != nil {\n\t\t\tconn.close()\n\t\t\treturn err\n\t\t}\n\t}\n\t_, err := redis.Int(conn.conn.Do(\"PUBLISH\", conn.ep.Redis.Channel, msg))\n\tif err != nil {\n\t\tconn.close()\n\t\treturn err\n\t}\n\treturn nil\n}\n" }, { "path": "internal/endpoint/scram_client.go", "content": "package endpoint\n\nimport (\n\t\"crypto/sha256\"\n\t\"crypto/sha512\"\n\n\t\"github.com/xdg-go/scram\"\n)\n\nvar (\n\tSHA256 scram.HashGeneratorFcn = sha256.New\n\tSHA512 scram.HashGeneratorFcn = sha512.New\n)\n\ntype XDGSCRAMClient struct {\n\t*scram.Client\n\t*scram.ClientConversation\n\tscram.HashGeneratorFcn\n}\n\nfunc (x *XDGSCRAMClient) Begin(userName, password, authzID string) (err error) {\n\tx.Client, err = x.HashGeneratorFcn.NewClient(userName, password, authzID)\n\tif err != nil {\n\t\treturn err\n\t}\n\tx.ClientConversation = x.Client.NewConversation()\n\treturn nil\n}\n\nfunc (x *XDGSCRAMClient) Step(challenge string) (response string, err error) {\n\tresponse, err = x.ClientConversation.Step(challenge)\n\treturn\n}\n\nfunc (x *XDGSCRAMClient) Done() bool {\n\treturn x.ClientConversation.Done()\n}\n" }, { "path": "internal/endpoint/sqs.go", "content": "package endpoint\n\nimport (\n\t\"fmt\"\n\t\"strings\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/tidwall/gjson\"\n\n\t\"github.com/aws/aws-sdk-go/aws\"\n\t\"github.com/aws/aws-sdk-go/aws/credentials\"\n\t\"github.com/aws/aws-sdk-go/aws/session\"\n\t\"github.com/aws/aws-sdk-go/service/sqs\"\n\t\"github.com/tidwall/tile38/internal/log\"\n)\n\nconst sqsExpiresAfter = time.Second * 30\n\n// SQSConn is an endpoint connection\ntype SQSConn struct {\n\tmu sync.Mutex\n\tep Endpoint\n\tsession *session.Session\n\tsvc *sqs.SQS\n\tex bool\n\tt time.Time\n}\n\nfunc (conn *SQSConn) generateSQSURL() string {\n\tif conn.ep.SQS.PlainURL != \"\" {\n\t\treturn conn.ep.SQS.PlainURL\n\t}\n\treturn \"https://sqs.\" + conn.ep.SQS.Region + \".amazonaws.com/\" +\n\t\tconn.ep.SQS.QueueID + \"/\" + conn.ep.SQS.QueueName\n}\n\n// Expired returns true if the connection has expired\nfunc (conn *SQSConn) Expired() bool {\n\tconn.mu.Lock()\n\tdefer conn.mu.Unlock()\n\tif !conn.ex {\n\t\tif time.Since(conn.t) > sqsExpiresAfter {\n\t\t\tconn.close()\n\t\t\tconn.ex = true\n\t\t}\n\t}\n\treturn conn.ex\n}\n\n// ExpireNow forces the connection to expire\nfunc (conn *SQSConn) ExpireNow() {\n\tconn.mu.Lock()\n\tdefer conn.mu.Unlock()\n\tconn.close()\n\tconn.ex = true\n}\n\nfunc (conn *SQSConn) close() {\n\tif conn.svc != nil {\n\t\tconn.svc = nil\n\t\tconn.session = nil\n\t}\n}\n\n// Send sends a message\nfunc (conn *SQSConn) Send(msg string) error {\n\tconn.mu.Lock()\n\tdefer conn.mu.Unlock()\n\n\tif conn.ex {\n\t\treturn errExpired\n\t}\n\tconn.t = time.Now()\n\n\tif conn.svc == nil && conn.session == nil {\n\t\tvar creds *credentials.Credentials\n\t\tcredPath := conn.ep.SQS.CredPath\n\t\tif credPath != \"\" {\n\t\t\tcredProfile := conn.ep.SQS.CredProfile\n\t\t\tif credProfile == \"\" {\n\t\t\t\tcredProfile = \"default\"\n\t\t\t}\n\t\t\tcreds = credentials.NewSharedCredentials(credPath, credProfile)\n\t\t}\n\t\tvar region string\n\t\tif conn.ep.SQS.Region != \"\" {\n\t\t\tregion = conn.ep.SQS.Region\n\t\t} else {\n\t\t\tregion = sqsRegionFromPlainURL(conn.ep.SQS.PlainURL)\n\t\t}\n\t\tsess := session.Must(session.NewSession(&aws.Config{\n\t\t\tRegion: ®ion,\n\t\t\tCredentials: creds,\n\t\t\tCredentialsChainVerboseErrors: aws.Bool(log.Level() >= 3),\n\t\t\tMaxRetries: aws.Int(5),\n\t\t}))\n\t\tsvc := sqs.New(sess)\n\t\tif conn.ep.SQS.CreateQueue {\n\t\t\tsvc.CreateQueue(&sqs.CreateQueueInput{\n\t\t\t\tQueueName: aws.String(conn.ep.SQS.QueueName),\n\t\t\t\tAttributes: map[string]*string{\n\t\t\t\t\t\"DelaySeconds\": aws.String(\"60\"),\n\t\t\t\t\t\"MessageRetentionPeriod\": aws.String(\"86400\"),\n\t\t\t\t},\n\t\t\t})\n\t\t}\n\t\tconn.session = sess\n\t\tconn.svc = svc\n\t}\n\n\tqueueURL := conn.generateSQSURL()\n\t// Create message\n\tsendParams := &sqs.SendMessageInput{\n\t\tMessageBody: aws.String(msg),\n\t\tQueueUrl: aws.String(queueURL),\n\t}\n\tif isFifoQueue(queueURL) {\n\t\tkey := gjson.Get(msg, \"key\")\n\t\tid := gjson.Get(msg, \"id\")\n\t\tkeyValue := fmt.Sprintf(\"%s#%s\", key.String(), id.String())\n\t\tsendParams.MessageGroupId = aws.String(keyValue)\n\t}\n\t_, err := conn.svc.SendMessage(sendParams)\n\tif err != nil {\n\t\tfmt.Println(err)\n\t\treturn err\n\t}\n\n\treturn nil\n}\n\nfunc newSQSConn(ep Endpoint) *SQSConn {\n\treturn &SQSConn{\n\t\tep: ep,\n\t\tt: time.Now(),\n\t}\n}\n\nfunc probeSQS(s string) bool {\n\t// https://sqs.eu-central-1.amazonaws.com/123456789/myqueue\n\treturn strings.HasPrefix(s, \"https://sqs.\") &&\n\t\tstrings.Contains(s, \".amazonaws.com\")\n}\n\nfunc sqsRegionFromPlainURL(s string) string {\n\tparts := strings.Split(s, \"https://sqs.\")\n\tif len(parts) > 1 {\n\t\tparts = strings.Split(parts[1], \".amazonaws.com\")\n\t\tif len(parts) > 1 {\n\t\t\treturn parts[0]\n\t\t}\n\t}\n\treturn \"\"\n}\n\nfunc isFifoQueue(s string) bool {\n\treturn strings.HasSuffix(s, \".fifo\")\n}\n" }, { "path": "internal/field/field.go", "content": "package field\n\nimport (\n\t\"math\"\n\t\"strconv\"\n\t\"strings\"\n\n\t\"github.com/tidwall/gjson\"\n\t\"github.com/tidwall/pretty\"\n)\n\nvar ZeroValue = Value{kind: Number, data: \"0\", num: 0}\nvar ZeroField = Field{name: \"\", value: ZeroValue}\n\ntype Kind byte\n\nconst (\n\tNull = Kind(gjson.Null)\n\tFalse = Kind(gjson.False)\n\tNumber = Kind(gjson.Number)\n\tString = Kind(gjson.String)\n\tTrue = Kind(gjson.True)\n\tJSON = Kind(gjson.JSON)\n)\n\ntype Value struct {\n\tkind Kind\n\tdata string\n\tnum float64\n}\n\nfunc (v Value) IsZero() bool {\n\treturn (v.kind == Number && v.data == \"0\" && v.num == 0) || v == (Value{})\n}\n\nfunc (v Value) Equals(b Value) bool {\n\treturn !v.Less(b) && !b.Less(v)\n}\n\nfunc (v Value) Kind() Kind {\n\treturn v.kind\n}\n\nfunc (v Value) Data() string {\n\treturn v.data\n}\n\nfunc (v Value) Num() float64 {\n\treturn v.num\n}\n\nfunc (v Value) JSON() string {\n\tswitch v.Kind() {\n\tcase Number:\n\t\tswitch v.Data() {\n\t\tcase \"NaN\":\n\t\t\treturn `\"NaN\"`\n\t\tcase \"+Inf\":\n\t\t\treturn `\"+Inf\"`\n\t\tcase \"-Inf\":\n\t\t\treturn `\"-Inf\"`\n\t\tdefault:\n\t\t\treturn v.Data()\n\t\t}\n\tcase String:\n\t\treturn string(gjson.AppendJSONString(nil, v.Data()))\n\tcase True:\n\t\treturn \"true\"\n\tcase False:\n\t\treturn \"false\"\n\tcase Null:\n\t\tif v != (Value{}) {\n\t\t\treturn \"null\"\n\t\t}\n\tcase JSON:\n\t\treturn v.Data()\n\t}\n\treturn \"0\"\n}\n\nfunc stringLessInsensitive(a, b string) bool {\n\tfor i := 0; i < len(a) && i < len(b); i++ {\n\t\tif a[i] >= 'A' && a[i] <= 'Z' {\n\t\t\tif b[i] >= 'A' && b[i] <= 'Z' {\n\t\t\t\t// both are uppercase, do nothing\n\t\t\t\tif a[i] < b[i] {\n\t\t\t\t\treturn true\n\t\t\t\t} else if a[i] > b[i] {\n\t\t\t\t\treturn false\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\t// a is uppercase, convert a to lowercase\n\t\t\t\tif a[i]+32 < b[i] {\n\t\t\t\t\treturn true\n\t\t\t\t} else if a[i]+32 > b[i] {\n\t\t\t\t\treturn false\n\t\t\t\t}\n\t\t\t}\n\t\t} else if b[i] >= 'A' && b[i] <= 'Z' {\n\t\t\t// b is uppercase, convert b to lowercase\n\t\t\tif a[i] < b[i]+32 {\n\t\t\t\treturn true\n\t\t\t} else if a[i] > b[i]+32 {\n\t\t\t\treturn false\n\t\t\t}\n\t\t} else {\n\t\t\t// neither are uppercase\n\t\t\tif a[i] < b[i] {\n\t\t\t\treturn true\n\t\t\t} else if a[i] > b[i] {\n\t\t\t\treturn false\n\t\t\t}\n\t\t}\n\t}\n\treturn len(a) < len(b)\n}\n\n// Less return true if a value is less than another value.\n// The caseSensitive parameter is used when the value are Strings.\n// The order when comparing two different kinds is:\n//\n//\tNull < False < Number < String < True < JSON\n//\n// Pulled from github.com/tidwall/gjson\nfunc (v Value) LessCase(b Value, caseSensitive bool) bool {\n\tif v.kind < b.kind {\n\t\treturn true\n\t}\n\tif v.kind > b.kind {\n\t\treturn false\n\t}\n\tif v.kind == Number {\n\t\treturn v.num < b.num\n\t}\n\tif v.kind == String {\n\t\tif caseSensitive {\n\t\t\treturn v.data < b.data\n\t\t}\n\t\treturn stringLessInsensitive(v.data, b.data)\n\t}\n\treturn v.data < b.data\n}\n\n// Less return true if a value is less than another value.\n//\n//\tNull < False < Number < String < True < JSON\n//\n// Pulled from github.com/tidwall/gjson\nfunc (v Value) Less(b Value) bool {\n\treturn v.LessCase(b, false)\n}\n\ntype Field struct {\n\tname string\n\tvalue Value\n}\n\nfunc (f Field) Name() string {\n\treturn f.name\n}\n\nfunc (f Field) Value() Value {\n\treturn f.value\n}\n\nfunc (f Field) Weight() int {\n\treturn len(f.name) + 8 + len(f.value.data)\n}\n\nvar nan = math.NaN()\nvar pinf = math.Inf(+1)\nvar ninf = math.Inf(-1)\n\nfunc ValueOf(data string) Value {\n\tdata = strings.TrimSpace(data)\n\tnum, err := strconv.ParseFloat(data, 64)\n\tif err == nil {\n\t\tif math.IsInf(num, 0) {\n\t\t\tif math.IsInf(num, +1) {\n\t\t\t\treturn Value{kind: Number, data: \"+Inf\", num: pinf}\n\t\t\t} else {\n\t\t\t\treturn Value{kind: Number, data: \"-Inf\", num: ninf}\n\t\t\t}\n\t\t} else if math.IsNaN(num) {\n\t\t\treturn Value{kind: Number, data: \"NaN\", num: nan}\n\t\t}\n\t\t// Make sure that this is a JSON compatible number.\n\t\t// For example, \"000123\" and \"000_123\" both parse as floats but aren't\n\t\t// really Numbers that can be represents in JSON.\n\t\tif gjson.Valid(data) {\n\t\t\treturn Value{kind: Number, data: data, num: num}\n\t\t}\n\t} else if gjson.Valid(data) {\n\t\tdata = strings.TrimSpace(data)\n\t\tr := gjson.Parse(data)\n\t\tswitch r.Type {\n\t\tcase gjson.Null:\n\t\t\treturn Value{kind: Null, data: \"null\"}\n\t\tcase gjson.JSON:\n\t\t\treturn Value{kind: JSON, data: string(pretty.Ugly([]byte(data)))}\n\t\tcase gjson.True:\n\t\t\treturn Value{kind: True, data: \"true\"}\n\t\tcase gjson.False:\n\t\t\treturn Value{kind: False, data: \"false\"}\n\t\tcase gjson.Number:\n\t\t\t// Ignore. Numbers will always be picked up by the ParseFloat above.\n\t\tcase gjson.String:\n\t\t\t// Ignore. Strings fallthrough by default\n\t\t}\n\t\t// Extract String from JSON\n\t\tdata = r.String()\n\t}\n\t// Check if string is NaN, Inf(inity), +Inf(inity), -Inf(inity)\n\tif len(data) >= 3 && len(data) <= 9 {\n\t\tswitch data[0] {\n\t\tcase '-', '+', 'I', 'i', 'N', 'n':\n\t\t\tswitch strings.ToLower(data) {\n\t\t\tcase \"nan\":\n\t\t\t\treturn Value{kind: Number, data: \"NaN\", num: nan}\n\t\t\tcase \"inf\", \"+inf\", \"infinity\", \"+infinity\":\n\t\t\t\treturn Value{kind: Number, data: \"+Inf\", num: pinf}\n\t\t\tcase \"-inf\", \"-infinity\":\n\t\t\t\treturn Value{kind: Number, data: \"-Inf\", num: ninf}\n\t\t\t}\n\t\t}\n\t}\n\n\treturn Value{kind: String, data: data}\n}\n\nfunc Make(name, data string) Field {\n\treturn Field{\n\t\tstrings.TrimSpace(name),\n\t\tValueOf(data),\n\t}\n}\n" }, { "path": "internal/field/field_test.go", "content": "package field\n\nimport (\n\t\"testing\"\n\n\t\"github.com/tidwall/assert\"\n)\n\nfunc mLT(a, b Value) bool { return a.Less(b) }\nfunc mLTE(a, b Value) bool { return !mLT(b, a) }\nfunc mGT(a, b Value) bool { return mLT(b, a) }\nfunc mGTE(a, b Value) bool { return !mLT(a, b) }\nfunc mEQ(a, b Value) bool { return !mLT(a, b) && !mLT(b, a) }\n\nfunc TestOrder(t *testing.T) {\n\tassert.Assert(mLT(ValueOf(\"hello\"), ValueOf(\"jello\")))\n\tassert.Assert(mLT(ValueOf(\"hello\"), ValueOf(\"JELLO\")))\n\tassert.Assert(mLT(ValueOf(\"HELLO\"), ValueOf(\"JELLO\")))\n\tassert.Assert(mLT(ValueOf(\"HELLO\"), ValueOf(\"jello\")))\n\tassert.Assert(!mLT(ValueOf(\"hello\"), ValueOf(\"hello\")))\n\tassert.Assert(!mLT(ValueOf(\"jello\"), ValueOf(\"hello\")))\n\tassert.Assert(!mLT(ValueOf(\"Jello\"), ValueOf(\"Hello\")))\n\tassert.Assert(!mLT(ValueOf(\"Jello\"), ValueOf(\"hello\")))\n\tassert.Assert(!mLT(ValueOf(\"jello\"), ValueOf(\"Hello\")))\n\tassert.Assert(mGT(ValueOf(\"jello\"), ValueOf(\"hello\")))\n\tassert.Assert(!mGT(ValueOf(\"jello\"), ValueOf(\"jello\")))\n\tassert.Assert(!mGT(ValueOf(\"hello\"), ValueOf(\"jello\")))\n\tassert.Assert(mLTE(ValueOf(\"hello\"), ValueOf(\"jello\")))\n\tassert.Assert(mLTE(ValueOf(\"hello\"), ValueOf(\"hello\")))\n\tassert.Assert(mLTE(ValueOf(\"hello\"), ValueOf(\"HELLO\")))\n\tassert.Assert(!mLTE(ValueOf(\"jello\"), ValueOf(\"hello\")))\n\tassert.Assert(mGTE(ValueOf(\"jello\"), ValueOf(\"jello\")))\n\tassert.Assert(mGTE(ValueOf(\"jello\"), ValueOf(\"hello\")))\n\tassert.Assert(mGTE(ValueOf(\"jello\"), ValueOf(\"JELLO\")))\n\tassert.Assert(!mGTE(ValueOf(\"hello\"), ValueOf(\"jello\")))\n\tassert.Assert(mEQ(ValueOf(\"jello\"), ValueOf(\"jello\")))\n\tassert.Assert(mEQ(ValueOf(\"jello\"), ValueOf(\"JELLO\")))\n\tassert.Assert(!mEQ(ValueOf(\"jello\"), ValueOf(\"hello\")))\n}\n\nfunc TestLess(t *testing.T) {\n\tassert.Assert(mLT(ValueOf(\"null\"), ValueOf(\"false\")))\n\tassert.Assert(mLT(ValueOf(\"null\"), ValueOf(\"123\")))\n\tassert.Assert(mLT(ValueOf(\"null\"), ValueOf(\"hello\")))\n\tassert.Assert(mLT(ValueOf(\"null\"), ValueOf(\"true\")))\n\tassert.Assert(mLT(ValueOf(\"null\"), ValueOf(\"[]\")))\n\tassert.Assert(mLT(ValueOf(\"false\"), ValueOf(\"123\")))\n\tassert.Assert(mLT(ValueOf(\"false\"), ValueOf(\"hello\")))\n\tassert.Assert(mLT(ValueOf(\"false\"), ValueOf(\"true\")))\n\tassert.Assert(mLT(ValueOf(\"false\"), ValueOf(\"[]\")))\n\tassert.Assert(mLT(ValueOf(\"123\"), ValueOf(\"hello\")))\n\tassert.Assert(mLT(ValueOf(\"123\"), ValueOf(\"true\")))\n\tassert.Assert(mLT(ValueOf(\"123\"), ValueOf(\"[]\")))\n\tassert.Assert(mLT(ValueOf(\"hello\"), ValueOf(\"true\")))\n\tassert.Assert(mLT(ValueOf(\"hello\"), ValueOf(\"[]\")))\n\tassert.Assert(mLT(ValueOf(\"true\"), ValueOf(\"[]\")))\n\tassert.Assert(!mLT(ValueOf(\"false\"), ValueOf(\"null\")))\n\tassert.Assert(!mLT(ValueOf(\"123\"), ValueOf(\"null\")))\n\tassert.Assert(!mLT(ValueOf(\"hello\"), ValueOf(\"null\")))\n\tassert.Assert(!mLT(ValueOf(\"true\"), ValueOf(\"null\")))\n\tassert.Assert(!mLT(ValueOf(\"[]\"), ValueOf(\"null\")))\n\tassert.Assert(!mLT(ValueOf(\"123\"), ValueOf(\"false\")))\n\tassert.Assert(!mLT(ValueOf(\"hello\"), ValueOf(\"false\")))\n\tassert.Assert(!mLT(ValueOf(\"true\"), ValueOf(\"false\")))\n\tassert.Assert(!mLT(ValueOf(\"[]\"), ValueOf(\"false\")))\n\tassert.Assert(!mLT(ValueOf(\"hello\"), ValueOf(\"123\")))\n\tassert.Assert(!mLT(ValueOf(\"true\"), ValueOf(\"123\")))\n\tassert.Assert(!mLT(ValueOf(\"[]\"), ValueOf(\"123\")))\n\tassert.Assert(!mLT(ValueOf(\"true\"), ValueOf(\"hello\")))\n\tassert.Assert(!mLT(ValueOf(\"[]\"), ValueOf(\"hello\")))\n\tassert.Assert(!mLT(ValueOf(\"[]\"), ValueOf(\"true\")))\n\tassert.Assert(mLT(ValueOf(\"123\"), ValueOf(\"456\")))\n\tassert.Assert(mLT(ValueOf(\"[1]\"), ValueOf(\"[2]\")))\n}\n\nfunc TestLessCase(t *testing.T) {\n\tassert.Assert(ValueOf(\"A\").LessCase(ValueOf(\"B\"), true))\n\tassert.Assert(!ValueOf(\"A\").LessCase(ValueOf(\"A\"), true))\n\tassert.Assert(!ValueOf(\"B\").LessCase(ValueOf(\"A\"), true))\n}\n\nfunc TestVarious(t *testing.T) {\n\tassert.Assert(!ValueOf(\"A\").IsZero())\n\tassert.Assert(ValueOf(\"0\").IsZero())\n\tassert.Assert(Value{}.IsZero())\n\tassert.Assert(ZeroValue.IsZero())\n\tassert.Assert(ZeroValue.Equals(ZeroValue))\n\tassert.Assert(ZeroValue.Kind() == Number)\n\tassert.Assert(ValueOf(\"0\").Kind() == Number)\n\tassert.Assert(ValueOf(\"hello\").Kind() == String)\n\tassert.Assert(ValueOf(`\"hello\"`).Kind() == String)\n\tassert.Assert(ValueOf(`\"123\"`).Kind() == String)\n\tassert.Assert(ValueOf(`\"123\"`).Data() == `123`)\n\tassert.Assert(ValueOf(`\"123\"`).Num() == 0)\n}\n\nfunc TestJSON(t *testing.T) {\n\tassert.Assert(ValueOf(`A`).JSON() == `\"A\"`)\n\tassert.Assert(ValueOf(`\"A\"`).JSON() == `\"A\"`)\n\tassert.Assert(ValueOf(`123`).JSON() == `123`)\n\tassert.Assert(ValueOf(`{}`).JSON() == `{}`)\n\tassert.Assert(ValueOf(`{ }`).JSON() == `{}`)\n\tassert.Assert(ValueOf(` -Inf `).JSON() == `\"-Inf\"`)\n\tassert.Assert(ValueOf(` \"-Inf\" `).JSON() == `\"-Inf\"`)\n\tassert.Assert(ValueOf(`+Inf`).JSON() == `\"+Inf\"`)\n\tassert.Assert(ValueOf(`\"+Inf\"`).JSON() == `\"+Inf\"`)\n\tassert.Assert(ValueOf(`Inf`).JSON() == `\"+Inf\"`)\n\tassert.Assert(ValueOf(`\"Inf\"`).JSON() == `\"+Inf\"`)\n\tassert.Assert(ValueOf(`NaN`).JSON() == `\"NaN\"`)\n\tassert.Assert(ValueOf(`\"NaN\"`).JSON() == `\"NaN\"`)\n\tassert.Assert(ValueOf(`nan`).JSON() == `\"NaN\"`)\n\tassert.Assert(ValueOf(`infinity`).JSON() == `\"+Inf\"`)\n\tassert.Assert(ValueOf(` true `).JSON() == `true`)\n\tassert.Assert(ValueOf(` false `).JSON() == `false`)\n\tassert.Assert(ValueOf(` null `).JSON() == `null`)\n\tassert.Assert(Value{}.JSON() == `0`)\n\tassert.Assert(Value{}.JSON() == `0`)\n}\n\nfunc TestField(t *testing.T) {\n\tassert.Assert(Make(\"hello\", \"123\").Name() == \"hello\")\n\tassert.Assert(Make(\"HELLO\", \"123\").Name() == \"HELLO\")\n\tassert.Assert(Make(\"HELLO\", \"123\").Value().Num() == 123)\n\tassert.Assert(Make(\"HELLO\", \"123\").Value().JSON() == \"123\")\n\tassert.Assert(Make(\"HELLO\", \"123\").Value().Num() == 123)\n}\n\nfunc TestWeight(t *testing.T) {\n\tassert.Assert(Make(\"hello\", \"123\").Weight() == 16)\n}\n\nfunc TestNumber(t *testing.T) {\n\tassert.Assert(ValueOf(\"12\").Num() == 12)\n\tassert.Assert(ValueOf(\"012\").Num() == 0)\n}\n" }, { "path": "internal/field/list_binary.go", "content": "package field\n\nimport (\n\t\"encoding/binary\"\n\t\"strconv\"\n\t\"strings\"\n\t\"unsafe\"\n\n\t\"github.com/tidwall/gjson\"\n\t\"github.com/tidwall/pretty\"\n\t\"github.com/tidwall/tile38/internal/sstring\"\n)\n\n// binary format\n// (size,entry,[entry...])\n// size: uvarint -- size of the full byte slice, excluding itself.\n// entry: (name,value) -- one field entry\n// name: shared string num -- field name, string data, uses the shared library\n// size: uvarint -- number of bytes in data\n// value: (kind,vdata) -- field value\n// kind: byte -- value kind\n// vdata: (size,data) -- value data, string data\n\n// useSharedNames will results in smaller memory usage by sharing the names\n// of fields using the sstring package. Otherwise the names are embedded with\n// the list.\nconst useSharedNames = true\n\n// List of fields, ordered by Name.\ntype List struct {\n\tp *byte\n}\n\ntype bytes struct {\n\tp *byte\n\tl int\n\tc int\n}\n\nfunc ptob(p *byte) []byte {\n\tif p == nil {\n\t\treturn nil\n\t}\n\t// Get the size of the bytes (excluding the header)\n\tx, n := uvarint(*(*[]byte)(unsafe.Pointer(&bytes{p, 10, 10})))\n\t// Return the byte slice (excluding the header)\n\treturn (*(*[]byte)(unsafe.Pointer(&bytes{p, n + x, n + x})))[n:]\n}\n\nfunc btoa(b []byte) string {\n\treturn *(*string)(unsafe.Pointer(&b))\n}\n\n// uvarint is a slightly modified version of binary.Uvarint, and it's a little\n// faster. But it lacks overflow checks which are not needed for our use.\nfunc uvarint(buf []byte) (int, int) {\n\tvar x uint64\n\tfor i := 0; i < len(buf); i++ {\n\t\tb := buf[i]\n\t\tif b < 0x80 {\n\t\t\treturn int(x | uint64(b)<<(i*7)), i + 1\n\t\t}\n\t\tx |= uint64(b&0x7f) << (i * 7)\n\t}\n\treturn 0, 0\n}\n\nfunc datakind(kind Kind) bool {\n\tswitch kind {\n\tcase Number, String, JSON:\n\t\treturn true\n\t}\n\treturn false\n}\n\nfunc bfield(name string, kind Kind, data string) Field {\n\tvar num float64\n\tswitch kind {\n\tcase Number:\n\t\tnum, _ = strconv.ParseFloat(data, 64)\n\tcase Null:\n\t\tdata = \"null\"\n\tcase False:\n\t\tdata = \"false\"\n\tcase True:\n\t\tdata = \"true\"\n\t}\n\treturn Field{\n\t\tname: name,\n\t\tvalue: Value{\n\t\t\tkind: Kind(kind),\n\t\t\tdata: data,\n\t\t\tnum: num,\n\t\t},\n\t}\n}\n\n// Set a field in the list.\n// If the input field value is zero `f.Value().IsZero()` then the field is\n// deleted or removed from the list since lists cannot have Zero values.\n// Returns a newly allocated list the updated field.\n// The original (receiver) list is not modified.\nfunc (fields List) Set(field Field) List {\n\tb := ptob(fields.p)\n\tvar i int\n\tfor {\n\t\ts := i\n\t\t// read the name\n\t\tvar name string\n\t\tx, n := uvarint(b[i:])\n\t\tif n == 0 {\n\t\t\tbreak\n\t\t}\n\t\tif useSharedNames {\n\t\t\tname = sstring.Load(x)\n\t\t\ti += n\n\t\t} else {\n\t\t\tname = btoa(b[i+n : i+n+x])\n\t\t\ti += n + x\n\t\t}\n\t\tkind := Kind(b[i])\n\t\ti++\n\t\tvar data string\n\t\tif datakind(kind) {\n\t\t\tx, n = uvarint(b[i:])\n\t\t\tdata = btoa(b[i+n : i+n+x])\n\t\t\ti += n + x\n\t\t}\n\t\tif field.name < name {\n\t\t\t// insert before\n\t\t\ti = s\n\t\t\tbreak\n\t\t}\n\t\tif name == field.name {\n\t\t\tif field.Value().IsZero() {\n\t\t\t\t// delete\n\t\t\t\treturn List{delfield(b, s, i)}\n\t\t\t}\n\t\t\tprev := bfield(name, kind, data)\n\t\t\tif prev.Value().Equals(field.Value()) {\n\t\t\t\t// no change\n\t\t\t\treturn fields\n\t\t\t}\n\t\t\t// replace\n\t\t\treturn List{putfield(b, field, s, i)}\n\t\t}\n\t}\n\tif field.Value().IsZero() {\n\t\treturn fields\n\t}\n\t// insert after\n\treturn List{putfield(b, field, i, i)}\n}\n\nfunc delfield(b []byte, s, e int) *byte {\n\ttotallen := s + (len(b) - e)\n\tif totallen == 0 {\n\t\treturn nil\n\t}\n\tvar psz [10]byte\n\tpn := binary.PutUvarint(psz[:], uint64(totallen))\n\tplen := pn + totallen\n\tp := make([]byte, plen)\n\t// copy each component\n\ti := 0\n\n\t// -- header size\n\tcopy(p[i:], psz[:pn])\n\ti += pn\n\n\t// -- head entries\n\tcopy(p[i:], b[:s])\n\ti += s\n\n\t// -- tail entries\n\tcopy(p[i:], b[e:])\n\n\treturn &p[0]\n}\n\nfunc putfield(b []byte, f Field, s, e int) *byte {\n\tname := f.Name()\n\tvar namesz [10]byte\n\tvar namen int\n\tif useSharedNames {\n\t\tnum := sstring.Store(name)\n\t\tnamen = binary.PutUvarint(namesz[:], uint64(num))\n\t} else {\n\t\tnamen = binary.PutUvarint(namesz[:], uint64(len(name)))\n\t}\n\tvalue := f.Value()\n\tkind := value.Kind()\n\tisdatakind := datakind(kind)\n\tvar data string\n\tvar datasz [10]byte\n\tvar datan int\n\tif isdatakind {\n\t\tdata = value.Data()\n\t\tdatan = binary.PutUvarint(datasz[:], uint64(len(data)))\n\t}\n\tvar totallen int\n\tif useSharedNames {\n\t\ttotallen = s + namen + 1 + (len(b) - e)\n\t} else {\n\t\ttotallen = s + namen + len(name) + 1 + +(len(b) - e)\n\t}\n\tif isdatakind {\n\t\ttotallen += datan + len(data)\n\t}\n\tvar psz [10]byte\n\tpn := binary.PutUvarint(psz[:], uint64(totallen))\n\tplen := pn + totallen\n\tp := make([]byte, plen)\n\n\t// copy each component\n\ti := 0\n\n\t// -- header size\n\tcopy(p[i:], psz[:pn])\n\ti += pn\n\n\t// -- head entries\n\tcopy(p[i:], b[:s])\n\ti += s\n\n\t// -- name\n\tcopy(p[i:], namesz[:namen])\n\ti += namen\n\n\tif !useSharedNames {\n\t\tcopy(p[i:], name)\n\t\ti += len(name)\n\t}\n\n\t// -- kind\n\tp[i] = byte(kind)\n\ti++\n\n\tif isdatakind {\n\t\t// -- data\n\t\tcopy(p[i:], datasz[:datan])\n\t\ti += datan\n\n\t\tcopy(p[i:], data)\n\t\ti += len(data)\n\t}\n\n\t// -- tail entries\n\tcopy(p[i:], b[e:])\n\n\treturn &p[0]\n}\n\n// Get a field from the list. Or returns ZeroField if not found.\nfunc (fields List) Get(name string) Field {\n\tvar isj bool\n\tvar jname string\n\tvar jpath string\n\tdot := strings.IndexByte(name, '.')\n\tif dot != -1 {\n\t\tisj = true\n\t\tjname = name[:dot]\n\t\tjpath = name[dot+1:]\n\t}\n\tb := ptob(fields.p)\n\tvar i int\n\tfor {\n\t\t// read the fname\n\t\tvar fname string\n\t\tx, n := uvarint(b[i:])\n\t\tif n == 0 {\n\t\t\tbreak\n\t\t}\n\t\tif useSharedNames {\n\t\t\tfname = sstring.Load(x)\n\t\t\ti += n\n\t\t} else {\n\t\t\tfname = btoa(b[i+n : i+n+x])\n\t\t\ti += n + x\n\t\t}\n\t\tkind := Kind(b[i])\n\t\ti++\n\t\tvar data string\n\t\tif datakind(kind) {\n\t\t\tx, n = uvarint(b[i:])\n\t\t\tdata = btoa(b[i+n : i+n+x])\n\t\t\ti += n + x\n\t\t}\n\t\tif kind == JSON && isj {\n\t\t\tif jname < fname {\n\t\t\t\tbreak\n\t\t\t}\n\t\t\tif fname == jname {\n\t\t\t\tres := gjson.Get(data, jpath)\n\t\t\t\tif res.Exists() {\n\t\t\t\t\treturn bfield(name, Kind(res.Type), res.String())\n\t\t\t\t}\n\t\t\t}\n\t\t} else {\n\t\t\tif name < fname {\n\t\t\t\tbreak\n\t\t\t}\n\t\t\tif fname == name {\n\t\t\t\treturn bfield(name, kind, data)\n\t\t\t}\n\t\t}\n\t}\n\treturn ZeroField\n}\n\n// Scan each field in list\nfunc (fields List) Scan(iter func(field Field) bool) {\n\tb := ptob(fields.p)\n\tvar i int\n\tfor {\n\t\t// read the fname\n\t\tvar fname string\n\t\tx, n := uvarint(b[i:])\n\t\tif n == 0 {\n\t\t\tbreak\n\t\t}\n\t\tif useSharedNames {\n\t\t\tfname = sstring.Load(x)\n\t\t\ti += n\n\t\t} else {\n\t\t\tfname = btoa(b[i+n : i+n+x])\n\t\t\ti += n + x\n\t\t}\n\t\tkind := Kind(b[i])\n\t\ti++\n\t\tvar data string\n\t\tif datakind(kind) {\n\t\t\tx, n = uvarint(b[i:])\n\t\t\tdata = btoa(b[i+n : i+n+x])\n\t\t\ti += n + x\n\t\t}\n\t\tif !iter(bfield(fname, kind, data)) {\n\t\t\treturn\n\t\t}\n\t}\n}\n\n// Len return the number of fields in list.\nfunc (fields List) Len() int {\n\tvar count int\n\tb := ptob(fields.p)\n\tvar i int\n\tfor {\n\t\tx, n := uvarint(b[i:])\n\t\tif n == 0 {\n\t\t\tbreak\n\t\t}\n\t\tif useSharedNames {\n\t\t\ti += n\n\t\t} else {\n\t\t\ti += n + x\n\t\t}\n\t\tisdatakind := datakind(Kind(b[i]))\n\t\ti++\n\t\tif isdatakind {\n\t\t\tx, n = uvarint(b[i:])\n\t\t\ti += n + x\n\t\t}\n\t\tcount++\n\t}\n\treturn count\n}\n\n// Weight is the number of bytes of the list.\nfunc (fields List) Weight() int {\n\tif fields.p == nil {\n\t\treturn 0\n\t}\n\tx, n := uvarint(*(*[]byte)(unsafe.Pointer(&bytes{fields.p, 10, 10})))\n\treturn x + n\n}\n\n// MakeList returns a field list from an array of fields.\nfunc MakeList(fields []Field) List {\n\t// TODO: optimize to reduce allocations.\n\tvar list List\n\tfor _, f := range fields {\n\t\tlist = list.Set(f)\n\t}\n\treturn list\n}\n\nfunc (fields List) String() string {\n\tvar dst []byte\n\tdst = append(dst, '{')\n\tvar i int\n\tfields.Scan(func(f Field) bool {\n\t\tif i > 0 {\n\t\t\tdst = append(dst, ',')\n\t\t}\n\t\tdst = gjson.AppendJSONString(dst, f.Name())\n\t\tdst = append(dst, ':')\n\t\tdst = append(dst, f.Value().JSON()...)\n\t\ti++\n\t\treturn true\n\t})\n\tdst = append(dst, '}')\n\treturn string(pretty.UglyInPlace(dst))\n}\n" }, { "path": "internal/field/list_struct.go", "content": "//go:build exclude\n\npackage field\n\ntype List struct {\n\tentries []Field\n}\n\n// bsearch searches array for value.\nfunc (fields List) bsearch(name string) (index int, found bool) {\n\ti, j := 0, len(fields.entries)\n\tfor i < j {\n\t\th := i + (j-i)/2\n\t\tif name >= fields.entries[h].name {\n\t\t\ti = h + 1\n\t\t} else {\n\t\t\tj = h\n\t\t}\n\t}\n\tif i > 0 && fields.entries[i-1].name >= name {\n\t\treturn i - 1, true\n\t}\n\treturn i, false\n}\n\nfunc (fields List) Set(field Field) List {\n\tvar updated List\n\tindex, found := fields.bsearch(field.name)\n\tif found {\n\t\tif field.value.IsZero() {\n\t\t\t// delete\n\t\t\tif len(fields.entries) > 1 {\n\t\t\t\tupdated.entries = make([]Field, len(fields.entries)-1)\n\t\t\t\tcopy(updated.entries, fields.entries[:index])\n\t\t\t\tcopy(updated.entries[index:], fields.entries[index+1:])\n\t\t\t}\n\t\t} else if !fields.entries[index].value.Equals(field.value) {\n\t\t\t// update\n\t\t\tupdated.entries = make([]Field, len(fields.entries))\n\t\t\tcopy(updated.entries, fields.entries)\n\t\t\tupdated.entries[index].value = field.value\n\t\t} else {\n\t\t\t// nothing changes\n\t\t\tupdated = fields\n\t\t}\n\t\treturn updated\n\t}\n\tif field.Value().IsZero() {\n\t\treturn fields\n\t}\n\tupdated.entries = make([]Field, len(fields.entries)+1)\n\tcopy(updated.entries, fields.entries[:index])\n\tcopy(updated.entries[index+1:], fields.entries[index:])\n\tupdated.entries[index] = field\n\treturn updated\n}\n\nfunc (fields List) Get(name string) Field {\n\tindex, found := fields.bsearch(name)\n\tif !found {\n\t\treturn ZeroField\n\t}\n\treturn fields.entries[index]\n}\n\nfunc (fields List) Scan(iter func(field Field) bool) {\n\tfor _, f := range fields.entries {\n\t\tif !iter(f) {\n\t\t\treturn\n\t\t}\n\t}\n}\n\nfunc (fields List) Len() int {\n\treturn len(fields.entries)\n}\n\nfunc (fields List) Weight() int {\n\tvar weight int\n\tfor _, f := range fields.entries {\n\t\tweight += f.Weight()\n\t}\n\treturn weight\n}\n" }, { "path": "internal/field/list_test.go", "content": "package field\n\nimport (\n\t\"fmt\"\n\t\"math/rand\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/tidwall/assert\"\n\t\"github.com/tidwall/btree\"\n)\n\nfunc TestList(t *testing.T) {\n\tvar fields List\n\n\tfields = fields.Set(Make(\"hello\", \"123\"))\n\tassert.Assert(fields.Len() == 1)\n\t// println(fields.Weight())\n\t// assert.Assert(fields.Weight() == 16)\n\n\tfields = fields.Set(Make(\"jello\", \"456\"))\n\tassert.Assert(fields.Len() == 2)\n\t// assert.Assert(fields.Weight() == 32)\n\n\tvalue := fields.Get(\"jello\")\n\tassert.Assert(value.Value().Data() == \"456\")\n\tassert.Assert(value.Value().JSON() == \"456\")\n\tassert.Assert(value.Value().Num() == 456)\n\n\tvalue = fields.Get(\"nello\")\n\tassert.Assert(value.Name() == \"\")\n\tassert.Assert(value.Value().IsZero())\n\n\tfields = fields.Set(Make(\"jello\", \"789\"))\n\tassert.Assert(fields.Len() == 2)\n\t// assert.Assert(fields.Weight() == 32)\n\n\tfields = fields.Set(Make(\"nello\", \"0\"))\n\tassert.Assert(fields.Len() == 2)\n\t// assert.Assert(fields.Weight() == 32)\n\n\tfields = fields.Set(Make(\"jello\", \"789\"))\n\tassert.Assert(fields.Len() == 2)\n\t// assert.Assert(fields.Weight() == 32)\n\n\tfields = fields.Set(Make(\"jello\", \"0\"))\n\tassert.Assert(fields.Len() == 1)\n\t// assert.Assert(fields.Weight() == 16)\n\n\tfields = fields.Set(Make(\"nello\", \"012\"))\n\tfields = fields.Set(Make(\"hello\", \"456\"))\n\tfields = fields.Set(Make(\"fello\", \"123\"))\n\tfields = fields.Set(Make(\"jello\", \"789\"))\n\n\tvar names string\n\tvar datas string\n\tvar nums float64\n\tfields.Scan(func(f Field) bool {\n\t\tnames += f.Name()\n\t\tdatas += f.Value().Data()\n\t\tnums += f.Value().Num()\n\t\treturn true\n\t})\n\tassert.Assert(names == \"fellohellojellonello\")\n\tassert.Assert(datas == \"123456789012\")\n\tassert.Assert(nums == 1368)\n\n\tnames = \"\"\n\tdatas = \"\"\n\tnums = 0\n\tfields.Scan(func(f Field) bool {\n\t\tnames += f.Name()\n\t\tdatas += f.Value().Data()\n\t\tnums += f.Value().Num()\n\t\treturn false\n\t})\n\tassert.Assert(names == \"fello\")\n\tassert.Assert(datas == \"123\")\n\tassert.Assert(nums == 123)\n\n}\n\nfunc randStr(n int) string {\n\tb := make([]byte, n)\n\trand.Read(b)\n\tfor i := 0; i < n; i++ {\n\t\tb[i] = 'a' + b[i]%26\n\t}\n\treturn string(b)\n}\n\nfunc randVal(n int) string {\n\tswitch rand.Intn(10) {\n\tcase 0:\n\t\treturn \"null\"\n\tcase 1:\n\t\treturn \"true\"\n\tcase 2:\n\t\treturn \"false\"\n\tcase 3:\n\t\treturn `{\"a\":\"` + randStr(n) + `\"}`\n\tcase 4:\n\t\treturn `[\"` + randStr(n) + `\"]`\n\tcase 5:\n\t\treturn `\"` + randStr(n) + `\"`\n\tcase 6:\n\t\treturn randStr(n)\n\tdefault:\n\t\treturn fmt.Sprintf(\"%f\", rand.Float64()*360)\n\t}\n}\n\nfunc TestRandom(t *testing.T) {\n\tseed := time.Now().UnixNano()\n\t// seed = 1663607868546669000\n\trand.Seed(seed)\n\tstart := time.Now()\n\tvar total int\n\tfor time.Since(start) < time.Second*2 {\n\t\tN := rand.Intn(500)\n\t\tvar org []Field\n\t\tvar tr btree.Map[string, Field]\n\t\tvar fields List\n\t\tfor i := 0; i < N; i++ {\n\t\t\tname := randStr(rand.Intn(10))\n\t\t\tvalue := randVal(rand.Intn(10))\n\t\t\tfield := Make(name, value)\n\t\t\torg = append(org, field)\n\t\t\tfields = fields.Set(field)\n\t\t\tv := fields.Get(name)\n\t\t\t// println(name, v.Value().Data(), field.Value().Data())\n\t\t\tif !v.Value().Equals(field.Value()) {\n\t\t\t\tt.Fatalf(\"seed: %d, expected true\", seed)\n\t\t\t}\n\t\t\ttr.Set(name, field)\n\t\t\tif fields.Len() != tr.Len() {\n\t\t\t\tt.Fatalf(\"seed: %d, expected %d, got %d\",\n\t\t\t\t\tseed, tr.Len(), fields.Len())\n\t\t\t}\n\t\t}\n\t\tcomp := func() {\n\t\t\tvar all []Field\n\t\t\tfields.Scan(func(f Field) bool {\n\t\t\t\tall = append(all, f)\n\t\t\t\treturn true\n\t\t\t})\n\t\t\tif len(all) != fields.Len() {\n\t\t\t\tt.Fatalf(\"seed: %d, expected %d, got %d\",\n\t\t\t\t\tseed, fields.Len(), len(all))\n\t\t\t}\n\t\t\tif fields.Len() != tr.Len() {\n\t\t\t\tt.Fatalf(\"seed: %d, expected %d, got %d\",\n\t\t\t\t\tseed, tr.Len(), fields.Len())\n\t\t\t}\n\t\t\tvar i int\n\t\t\ttr.Scan(func(name string, f Field) bool {\n\t\t\t\tif name != f.Name() || all[i].Name() != f.Name() {\n\t\t\t\t\tt.Fatalf(\"seed: %d, out of order\", seed)\n\t\t\t\t}\n\t\t\t\ti++\n\t\t\t\treturn true\n\t\t\t})\n\t\t}\n\t\tcomp()\n\t\trand.Shuffle(len(org), func(i, j int) {\n\t\t\torg[i], org[j] = org[j], org[i]\n\t\t})\n\t\tfor _, f := range org {\n\t\t\tcomp()\n\t\t\ttr.Delete(f.Name())\n\t\t\tfields = fields.Set(Make(f.Name(), \"0\"))\n\t\t\tif fields.Len() != tr.Len() {\n\t\t\t\tt.Fatalf(\"seed: %d, expected %d, got %d\",\n\t\t\t\t\tseed, tr.Len(), fields.Len())\n\t\t\t}\n\t\t\tcomp()\n\t\t}\n\t\ttotal++\n\t}\n\n}\n\nfunc TestJSONGet(t *testing.T) {\n\n\tvar list List\n\tlist = list.Set(Make(\"hello\", \"world\"))\n\tlist = list.Set(Make(\"hello\", `\"world\"`))\n\tlist = list.Set(Make(\"jello\", \"planet\"))\n\tlist = list.Set(Make(\"telly\", `{\"a\":[1,2,3],\"b\":null,\"c\":true,\"d\":false}`))\n\tlist = list.Set(Make(\"belly\", `{\"a\":{\"b\":{\"c\":\"fancy\"}}}`))\n\tjson := list.String()\n\texp := `{\"belly\":{\"a\":{\"b\":{\"c\":\"fancy\"}}},\"hello\":\"world\",\"jello\":` +\n\t\t`\"planet\",\"telly\":{\"a\":[1,2,3],\"b\":null,\"c\":true,\"d\":false}}`\n\tif json != exp {\n\t\tt.Fatalf(\"expected '%s', got '%s'\", exp, json)\n\t}\n\tdata := list.Get(\"hello\").Value().Data()\n\tif data != \"world\" {\n\t\tt.Fatalf(\"expected '%s', got '%s'\", \"world\", data)\n\t}\n\tdata = list.Get(\"telly\").Value().Data()\n\tif data != `{\"a\":[1,2,3],\"b\":null,\"c\":true,\"d\":false}` {\n\t\tt.Fatalf(\"expected '%s', got '%s'\",\n\t\t\t`{\"a\":[1,2,3],\"b\":null,\"c\":true,\"d\":false}`, data)\n\t}\n\tdata = list.Get(\"belly\").Value().Data()\n\tif data != `{\"a\":{\"b\":{\"c\":\"fancy\"}}}` {\n\t\tt.Fatalf(\"expected '%s', got '%s'\",\n\t\t\t`{\"a\":{\"b\":{\"c\":\"fancy\"}}}`, data)\n\t}\n\tdata = list.Get(\"belly.a\").Value().Data()\n\tif data != `{\"b\":{\"c\":\"fancy\"}}` {\n\t\tt.Fatalf(\"expected '%s', got '%s'\",\n\t\t\t`{\"b\":{\"c\":\"fancy\"}}`, data)\n\t}\n\tdata = list.Get(\"belly.a.b\").Value().Data()\n\tif data != `{\"c\":\"fancy\"}` {\n\t\tt.Fatalf(\"expected '%s', got '%s'\",\n\t\t\t`{\"c\":\"fancy\"}`, data)\n\t}\n\tdata = list.Get(\"belly.a.b.c\").Value().Data()\n\tif data != `fancy` {\n\t\tt.Fatalf(\"expected '%s', got '%s'\",\n\t\t\t`fancy`, data)\n\t}\n\t// Tile38 defaults non-existent fields to zero.\n\tdata = list.Get(\"belly.a.b.c.d\").Value().Data()\n\tif data != `0` {\n\t\tt.Fatalf(\"expected '%s', got '%s'\",\n\t\t\t`0`, data)\n\t}\n}\n" }, { "path": "internal/glob/glob.go", "content": "package glob\n\nimport \"strings\"\n\n// Glob structure for simple string matching\ntype Glob struct {\n\tPattern string\n\tDesc bool\n\tLimits []string\n\tIsGlob bool\n}\n\n// Match returns true when string matches pattern. Returns an error when the\n// pattern is invalid.\nfunc Match(pattern, str string) (matched bool, err error) {\n\treturn wildcardMatch(pattern, str)\n}\n\n// IsGlob returns true when the pattern is a valid glob\nfunc IsGlob(pattern string) bool {\n\tfor i := 0; i < len(pattern); i++ {\n\t\tswitch pattern[i] {\n\t\tcase '[', '*', '?':\n\t\t\t_, err := Match(pattern, \"whatever\")\n\t\t\treturn err == nil\n\t\t}\n\t}\n\treturn false\n}\n\n// Parse returns a glob structure from the pattern.\nfunc Parse(pattern string, desc bool) *Glob {\n\tg := &Glob{Pattern: pattern, Desc: desc, Limits: []string{\"\", \"\"}}\n\tif strings.HasPrefix(pattern, \"*\") {\n\t\tg.IsGlob = true\n\t\treturn g\n\t}\n\tif pattern == \"\" {\n\t\tg.IsGlob = false\n\t\treturn g\n\t}\n\tn := 0\n\tisGlob := false\nouter:\n\tfor i := 0; i < len(pattern); i++ {\n\t\tswitch pattern[i] {\n\t\tcase '[', '*', '?':\n\t\t\t_, err := Match(pattern, \"whatever\")\n\t\t\tif err == nil {\n\t\t\t\tisGlob = true\n\t\t\t}\n\t\t\tbreak outer\n\t\t}\n\t\tn++\n\t}\n\tif n == 0 {\n\t\tg.Limits = []string{pattern, pattern}\n\t\tg.IsGlob = false\n\t\treturn g\n\t}\n\tvar a, b string\n\tif desc {\n\t\ta = pattern[:n]\n\t\tb = a\n\t\tif b[n-1] == 0x00 {\n\t\t\tfor len(b) > 0 && b[len(b)-1] == 0x00 {\n\t\t\t\tif len(b) > 1 {\n\t\t\t\t\tif b[len(b)-2] == 0x00 {\n\t\t\t\t\t\tb = b[:len(b)-1]\n\t\t\t\t\t} else {\n\t\t\t\t\t\tb = string(append([]byte(b[:len(b)-2]), b[len(b)-2]-1, 0xFF))\n\t\t\t\t\t}\n\t\t\t\t} else {\n\t\t\t\t\tb = \"\"\n\t\t\t\t}\n\t\t\t}\n\t\t} else {\n\t\t\tb = string(append([]byte(b[:n-1]), b[n-1]-1))\n\t\t}\n\t\tif a[n-1] == 0xFF {\n\t\t\ta = string(append([]byte(a), 0x00))\n\t\t} else {\n\t\t\ta = string(append([]byte(a[:n-1]), a[n-1]+1))\n\t\t}\n\t} else {\n\t\ta = pattern[:n]\n\t\tif a[n-1] == 0xFF {\n\t\t\tb = string(append([]byte(a), 0x00))\n\t\t} else {\n\t\t\tb = string(append([]byte(a[:n-1]), a[n-1]+1))\n\t\t}\n\t}\n\tg.Limits = []string{a, b}\n\tg.IsGlob = isGlob\n\treturn g\n}\n" }, { "path": "internal/glob/glob_test.go", "content": "package glob\n\nimport (\n\t\"math/rand\"\n\t\"testing\"\n\t\"time\"\n)\n\nfunc test(t *testing.T, pattern string, desc bool, limitsExpect []string, isGlobExpect bool) {\n\tg := Parse(pattern, desc)\n\tif g.IsGlob != isGlobExpect {\n\t\tt.Fatalf(\"pattern[%v] desc[%v] (isGlob=%v, expected=%v)\", pattern, desc, g.IsGlob, isGlobExpect)\n\t}\n\tif g.Limits[0] != limitsExpect[0] || g.Limits[1] != limitsExpect[1] {\n\t\tt.Fatalf(\"pattern[%v] desc[%v] (limits=%v, expected=%v)\", pattern, desc, g.Limits, limitsExpect)\n\t}\n\tif g.Pattern != pattern {\n\t\tt.Fatalf(\"pattern[%v] desc[%v] (pattern=%v, expected=%v)\", pattern, desc, g.Pattern, pattern)\n\t}\n\tif g.Desc != desc {\n\t\tt.Fatalf(\"pattern[%v] desc[%v] (desc=%v, expected=%v)\", pattern, desc, g.Desc, desc)\n\t}\n}\n\nfunc TestGlob(t *testing.T) {\n\ttest(t, \"*\", false, []string{\"\", \"\"}, true)\n\ttest(t, \"\", false, []string{\"\", \"\"}, false)\n\ttest(t, \"hello*\", false, []string{\"hello\", \"hellp\"}, true)\n\ttest(t, \"hello\", false, []string{\"hello\", \"hellp\"}, false)\n\ttest(t, \"\\xff*\", false, []string{\"\\xff\", \"\\xff\\x00\"}, true)\n\ttest(t, \"\\x00*\", false, []string{\"\\x00\", \"\\x01\"}, true)\n\ttest(t, \"\\xff\", false, []string{\"\\xff\", \"\\xff\\x00\"}, false)\n\n\ttest(t, \"*\", true, []string{\"\", \"\"}, true)\n\ttest(t, \"\", true, []string{\"\", \"\"}, false)\n\ttest(t, \"hello*\", true, []string{\"hellp\", \"helln\"}, true)\n\ttest(t, \"hello\", true, []string{\"hellp\", \"helln\"}, false)\n\ttest(t, \"a\\xff*\", true, []string{\"a\\xff\\x00\", \"a\\xfe\"}, true)\n\ttest(t, \"\\x00*\", true, []string{\"\\x01\", \"\"}, true)\n\ttest(t, \"\\x01*\", true, []string{\"\\x02\", \"\\x00\"}, true)\n\ttest(t, \"b\\x00*\", true, []string{\"b\\x01\", \"a\\xff\"}, true)\n\ttest(t, \"\\x00\\x00*\", true, []string{\"\\x00\\x01\", \"\"}, true)\n\ttest(t, \"\\x00\\x01\\x00*\", true, []string{\"\\x00\\x01\\x01\", \"\\x00\\x00\\xff\"}, true)\n}\n\nfunc testMatch(s, pattern string) bool {\n\tok, _ := Match(pattern, s)\n\treturn ok\n}\n\nfunc TestMatch(t *testing.T) {\n\tif !testMatch(\"hello world\", \"hello world\") {\n\t\tt.Fatal(\"fail\")\n\t}\n\tif testMatch(\"hello world\", \"jello world\") {\n\t\tt.Fatal(\"fail\")\n\t}\n\tif !testMatch(\"hello world\", \"hello*\") {\n\t\tt.Fatal(\"fail\")\n\t}\n\tif testMatch(\"hello world\", \"jello*\") {\n\t\tt.Fatal(\"fail\")\n\t}\n\tif !testMatch(\"hello world\", \"hello?world\") {\n\t\tt.Fatal(\"fail\")\n\t}\n\tif testMatch(\"hello world\", \"jello?world\") {\n\t\tt.Fatal(\"fail\")\n\t}\n\tif !testMatch(\"hello world\", \"he*o?world\") {\n\t\tt.Fatal(\"fail\")\n\t}\n\tif !testMatch(\"hello world\", \"he*o?wor*\") {\n\t\tt.Fatal(\"fail\")\n\t}\n\tif !testMatch(\"hello world\", \"he*o?*r*\") {\n\t\tt.Fatal(\"fail\")\n\t}\n\tif !testMatch(\"的情况下解析一个\", \"*\") {\n\t\tt.Fatal(\"fail\")\n\t}\n\tif !testMatch(\"的情况下解析一个\", \"*况下*\") {\n\t\tt.Fatal(\"fail\")\n\t}\n\tif !testMatch(\"的情况下解析一个\", \"*况?*\") {\n\t\tt.Fatal(\"fail\")\n\t}\n\tif !testMatch(\"的情况下解析一个\", \"的情况?解析一个\") {\n\t\tt.Fatal(\"fail\")\n\t}\n}\n\n// TestWildcardMatch - Tests validate the logic of wild card matching.\n// `WildcardMatch` supports '*' and '?' wildcards.\n// Sample usage: In resource matching for folder policy validation.\nfunc TestWildcardMatch(t *testing.T) {\n\ttestCases := []struct {\n\t\tpattern string\n\t\ttext string\n\t\tmatched bool\n\t}{\n\t\t// Test case - 1.\n\t\t// Test case with pattern containing key name with a prefix. Should accept the same text without a \"*\".\n\t\t{\n\t\t\tpattern: \"my-folder/oo*\",\n\t\t\ttext: \"my-folder/oo\",\n\t\t\tmatched: true,\n\t\t},\n\t\t// Test case - 2.\n\t\t// Test case with \"*\" at the end of the pattern.\n\t\t{\n\t\t\tpattern: \"my-folder/In*\",\n\t\t\ttext: \"my-folder/India/Karnataka/\",\n\t\t\tmatched: true,\n\t\t},\n\t\t// Test case - 3.\n\t\t// Test case with prefixes shuffled.\n\t\t// This should fail.\n\t\t{\n\t\t\tpattern: \"my-folder/In*\",\n\t\t\ttext: \"my-folder/Karnataka/India/\",\n\t\t\tmatched: false,\n\t\t},\n\t\t// Test case - 4.\n\t\t// Test case with text expanded to the wildcards in the pattern.\n\t\t{\n\t\t\tpattern: \"my-folder/In*/Ka*/Ban\",\n\t\t\ttext: \"my-folder/India/Karnataka/Ban\",\n\t\t\tmatched: true,\n\t\t},\n\t\t// Test case - 5.\n\t\t// Test case with the keyname part is repeated as prefix several times.\n\t\t// This is valid.\n\t\t{\n\t\t\tpattern: \"my-folder/In*/Ka*/Ban\",\n\t\t\ttext: \"my-folder/India/Karnataka/Ban/Ban/Ban/Ban/Ban\",\n\t\t\tmatched: true,\n\t\t},\n\t\t// Test case - 6.\n\t\t// Test case to validate that `*` can be expanded into multiple prefixes.\n\t\t{\n\t\t\tpattern: \"my-folder/In*/Ka*/Ban\",\n\t\t\ttext: \"my-folder/India/Karnataka/Area1/Area2/Area3/Ban\",\n\t\t\tmatched: true,\n\t\t},\n\t\t// Test case - 7.\n\t\t// Test case to validate that `*` can be expanded into multiple prefixes.\n\t\t{\n\t\t\tpattern: \"my-folder/In*/Ka*/Ban\",\n\t\t\ttext: \"my-folder/India/State1/State2/Karnataka/Area1/Area2/Area3/Ban\",\n\t\t\tmatched: true,\n\t\t},\n\t\t// Test case - 8.\n\t\t// Test case where the keyname part of the pattern is expanded in the text.\n\t\t{\n\t\t\tpattern: \"my-folder/In*/Ka*/Ban\",\n\t\t\ttext: \"my-folder/India/Karnataka/Bangalore\",\n\t\t\tmatched: false,\n\t\t},\n\t\t// Test case - 9.\n\t\t// Test case with prefixes and wildcard expanded for all \"*\".\n\t\t{\n\t\t\tpattern: \"my-folder/In*/Ka*/Ban*\",\n\t\t\ttext: \"my-folder/India/Karnataka/Bangalore\",\n\t\t\tmatched: true,\n\t\t},\n\t\t// Test case - 10.\n\t\t// Test case with keyname part being a wildcard in the pattern.\n\t\t{pattern: \"my-folder/*\",\n\t\t\ttext: \"my-folder/India\",\n\t\t\tmatched: true,\n\t\t},\n\t\t// Test case - 11.\n\t\t{\n\t\t\tpattern: \"my-folder/oo*\",\n\t\t\ttext: \"my-folder/odo\",\n\t\t\tmatched: false,\n\t\t},\n\n\t\t// Test case with pattern containing wildcard '?'.\n\t\t// Test case - 12.\n\t\t// \"my-folder?/\" matches \"my-folder1/\", \"my-folder2/\", \"my-folder3\" etc...\n\t\t// doesn't match \"myfolder/\".\n\t\t{\n\t\t\tpattern: \"my-folder?/abc*\",\n\t\t\ttext: \"myfolder/abc\",\n\t\t\tmatched: false,\n\t\t},\n\t\t// Test case - 13.\n\t\t{\n\t\t\tpattern: \"my-folder?/abc*\",\n\t\t\ttext: \"my-folder1/abc\",\n\t\t\tmatched: true,\n\t\t},\n\t\t// Test case - 14.\n\t\t{\n\t\t\tpattern: \"my-?-folder/abc*\",\n\t\t\ttext: \"my--folder/abc\",\n\t\t\tmatched: false,\n\t\t},\n\t\t// Test case - 15.\n\t\t{\n\t\t\tpattern: \"my-?-folder/abc*\",\n\t\t\ttext: \"my-1-folder/abc\",\n\t\t\tmatched: true,\n\t\t},\n\t\t// Test case - 16.\n\t\t{\n\t\t\tpattern: \"my-?-folder/abc*\",\n\t\t\ttext: \"my-k-folder/abc\",\n\t\t\tmatched: true,\n\t\t},\n\t\t// Test case - 17.\n\t\t{\n\t\t\tpattern: \"my??folder/abc*\",\n\t\t\ttext: \"myfolder/abc\",\n\t\t\tmatched: false,\n\t\t},\n\t\t// Test case - 18.\n\t\t{\n\t\t\tpattern: \"my??folder/abc*\",\n\t\t\ttext: \"my4afolder/abc\",\n\t\t\tmatched: true,\n\t\t},\n\t\t// Test case - 19.\n\t\t{\n\t\t\tpattern: \"my-folder?abc*\",\n\t\t\ttext: \"my-folder/abc\",\n\t\t\tmatched: true,\n\t\t},\n\t\t// Test case 20-21.\n\t\t// '?' matches '/' too. (works with s3).\n\t\t// This is because the namespace is considered flat.\n\t\t// \"abc?efg\" matches both \"abcdefg\" and \"abc/efg\".\n\t\t{\n\t\t\tpattern: \"my-folder/abc?efg\",\n\t\t\ttext: \"my-folder/abcdefg\",\n\t\t\tmatched: true,\n\t\t},\n\t\t{\n\t\t\tpattern: \"my-folder/abc?efg\",\n\t\t\ttext: \"my-folder/abc/efg\",\n\t\t\tmatched: true,\n\t\t},\n\t\t// Test case - 22.\n\t\t{\n\t\t\tpattern: \"my-folder/abc????\",\n\t\t\ttext: \"my-folder/abc\",\n\t\t\tmatched: false,\n\t\t},\n\t\t// Test case - 23.\n\t\t{\n\t\t\tpattern: \"my-folder/abc????\",\n\t\t\ttext: \"my-folder/abcde\",\n\t\t\tmatched: false,\n\t\t},\n\t\t// Test case - 24.\n\t\t{\n\t\t\tpattern: \"my-folder/abc????\",\n\t\t\ttext: \"my-folder/abcdefg\",\n\t\t\tmatched: true,\n\t\t},\n\t\t// Test case 25-26.\n\t\t// test case with no '*'.\n\t\t{\n\t\t\tpattern: \"my-folder/abc?\",\n\t\t\ttext: \"my-folder/abc\",\n\t\t\tmatched: false,\n\t\t},\n\t\t{\n\t\t\tpattern: \"my-folder/abc?\",\n\t\t\ttext: \"my-folder/abcd\",\n\t\t\tmatched: true,\n\t\t},\n\t\t{\n\t\t\tpattern: \"my-folder/abc?\",\n\t\t\ttext: \"my-folder/abcde\",\n\t\t\tmatched: false,\n\t\t},\n\t\t// Test case 27.\n\t\t{\n\t\t\tpattern: \"my-folder/mnop*?\",\n\t\t\ttext: \"my-folder/mnop\",\n\t\t\tmatched: false,\n\t\t},\n\t\t// Test case 28.\n\t\t{\n\t\t\tpattern: \"my-folder/mnop*?\",\n\t\t\ttext: \"my-folder/mnopqrst/mnopqr\",\n\t\t\tmatched: true,\n\t\t},\n\t\t// Test case 29.\n\t\t{\n\t\t\tpattern: \"my-folder/mnop*?\",\n\t\t\ttext: \"my-folder/mnopqrst/mnopqrs\",\n\t\t\tmatched: true,\n\t\t},\n\t\t// Test case 30.\n\t\t{\n\t\t\tpattern: \"my-folder/mnop*?\",\n\t\t\ttext: \"my-folder/mnop\",\n\t\t\tmatched: false,\n\t\t},\n\t\t// Test case 31.\n\t\t{\n\t\t\tpattern: \"my-folder/mnop*?\",\n\t\t\ttext: \"my-folder/mnopq\",\n\t\t\tmatched: true,\n\t\t},\n\t\t// Test case 32.\n\t\t{\n\t\t\tpattern: \"my-folder/mnop*?\",\n\t\t\ttext: \"my-folder/mnopqr\",\n\t\t\tmatched: true,\n\t\t},\n\t\t// Test case 33.\n\t\t{\n\t\t\tpattern: \"my-folder/mnop*?and\",\n\t\t\ttext: \"my-folder/mnopqand\",\n\t\t\tmatched: true,\n\t\t},\n\t\t// Test case 34.\n\t\t{\n\t\t\tpattern: \"my-folder/mnop*?and\",\n\t\t\ttext: \"my-folder/mnopand\",\n\t\t\tmatched: false,\n\t\t},\n\t\t// Test case 35.\n\t\t{\n\t\t\tpattern: \"my-folder/mnop*?and\",\n\t\t\ttext: \"my-folder/mnopqand\",\n\t\t\tmatched: true,\n\t\t},\n\t\t// Test case 36.\n\t\t{\n\t\t\tpattern: \"my-folder/mnop*?\",\n\t\t\ttext: \"my-folder/mn\",\n\t\t\tmatched: false,\n\t\t},\n\t\t// Test case 37.\n\t\t{\n\t\t\tpattern: \"my-folder/mnop*?\",\n\t\t\ttext: \"my-folder/mnopqrst/mnopqrs\",\n\t\t\tmatched: true,\n\t\t},\n\t\t// Test case 38.\n\t\t{\n\t\t\tpattern: \"my-folder/mnop*??\",\n\t\t\ttext: \"my-folder/mnopqrst\",\n\t\t\tmatched: true,\n\t\t},\n\t\t// Test case 39.\n\t\t{\n\t\t\tpattern: \"my-folder/mnop*qrst\",\n\t\t\ttext: \"my-folder/mnopabcdegqrst\",\n\t\t\tmatched: true,\n\t\t},\n\t\t// Test case 40.\n\t\t{\n\t\t\tpattern: \"my-folder/mnop*?and\",\n\t\t\ttext: \"my-folder/mnopqand\",\n\t\t\tmatched: true,\n\t\t},\n\t\t// Test case 41.\n\t\t{\n\t\t\tpattern: \"my-folder/mnop*?and\",\n\t\t\ttext: \"my-folder/mnopand\",\n\t\t\tmatched: false,\n\t\t},\n\t\t// Test case 42.\n\t\t{\n\t\t\tpattern: \"my-folder/mnop*?and?\",\n\t\t\ttext: \"my-folder/mnopqanda\",\n\t\t\tmatched: true,\n\t\t},\n\t\t// Test case 43.\n\t\t{\n\t\t\tpattern: \"my-folder/mnop*?and\",\n\t\t\ttext: \"my-folder/mnopqanda\",\n\t\t\tmatched: false,\n\t\t},\n\t\t// Test case 44.\n\n\t\t{\n\t\t\tpattern: \"my-?-folder/abc*\",\n\t\t\ttext: \"my-folder/mnopqanda\",\n\t\t\tmatched: false,\n\t\t},\n\t}\n\t// Iterating over the test cases, call the function under test and asert the output.\n\tfor i, testCase := range testCases {\n\t\tactualResult := testMatch(testCase.text, testCase.pattern)\n\t\tif testCase.matched != actualResult {\n\t\t\tt.Errorf(\"Test %d: Expected the result to be `%v`, but instead found it to be `%v`\", i+1, testCase.matched, actualResult)\n\t\t}\n\t}\n}\nfunc TestRandomInput(t *testing.T) {\n\trand.Seed(time.Now().UnixNano())\n\tb1 := make([]byte, 100)\n\tb2 := make([]byte, 100)\n\tfor i := 0; i < 1000000; i++ {\n\t\tif _, err := rand.Read(b1); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif _, err := rand.Read(b2); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\ttestMatch(string(b1), string(b2))\n\t}\n}\nfunc BenchmarkAscii(t *testing.B) {\n\tfor i := 0; i < t.N; i++ {\n\t\tif !testMatch(\"hello\", \"hello\") {\n\t\t\tt.Fatal(\"fail\")\n\t\t}\n\t}\n}\n\nfunc BenchmarkUnicode(t *testing.B) {\n\tfor i := 0; i < t.N; i++ {\n\t\tif !testMatch(\"h情llo\", \"h情llo\") {\n\t\t\tt.Fatal(\"fail\")\n\t\t}\n\t}\n}\n" }, { "path": "internal/glob/match.go", "content": "// Copyright 2010 The Go Authors. All rights reserved.\n// Use of this source code is governed by a BSD-style\n// license that can be found in the LICENSE file.\n\npackage glob\n\nimport (\n\t\"errors\"\n\t\"unicode/utf8\"\n)\n\n// ErrBadPattern indicates a globbing pattern was malformed.\nvar ErrBadPattern = errors.New(\"syntax error in pattern\")\n\n// Match reports whether name matches the shell file name pattern.\n// The pattern syntax is:\n//\n//\tpattern:\n//\t\t{ term }\n//\tterm:\n//\t\t'*' matches any sequence of non-Separator characters\n//\t\t'?' matches any single non-Separator character\n//\t\t'[' [ '^' ] { character-range } ']'\n//\t\t character class (must be non-empty)\n//\t\tc matches character c (c != '*', '?', '\\\\', '[')\n//\t\t'\\\\' c matches character c\n//\n//\tcharacter-range:\n//\t\tc matches character c (c != '\\\\', '-', ']')\n//\t\t'\\\\' c matches character c\n//\t\tlo '-' hi matches character c for lo <= c <= hi\n//\n// Match requires pattern to match all of name, not just a substring.\n// The only possible returned error is ErrBadPattern, when pattern\n// is malformed.\n//\n// On Windows, escaping is disabled. Instead, '\\\\' is treated as\n// path separator.\nfunc wildcardMatch(pattern, name string) (matched bool, err error) {\nPattern:\n\tfor len(pattern) > 0 {\n\t\tvar star bool\n\t\tvar chunk string\n\t\tstar, chunk, pattern = scanChunk(pattern)\n\t\tif star && chunk == \"\" {\n\t\t\t// Trailing * matches rest of string unless it has a /.\n\t\t\treturn true, nil\n\t\t}\n\t\t// Look for match at current position.\n\t\tt, ok, err := matchChunk(chunk, name)\n\t\t// if we're the last chunk, make sure we've exhausted the name\n\t\t// otherwise we'll give a false result even if we could still match\n\t\t// using the star\n\t\tif ok && (len(t) == 0 || len(pattern) > 0) {\n\t\t\tname = t\n\t\t\tcontinue\n\t\t}\n\t\tif err != nil {\n\t\t\treturn false, err\n\t\t}\n\t\tif star {\n\t\t\t// Look for match skipping i+1 bytes.\n\t\t\t// Cannot skip /.\n\t\t\tfor i := 0; i < len(name); i++ {\n\t\t\t\tt, ok, err := matchChunk(chunk, name[i+1:])\n\t\t\t\tif ok {\n\t\t\t\t\t// if we're the last chunk, make sure we exhausted the name\n\t\t\t\t\tif len(pattern) == 0 && len(t) > 0 {\n\t\t\t\t\t\tcontinue\n\t\t\t\t\t}\n\t\t\t\t\tname = t\n\t\t\t\t\tcontinue Pattern\n\t\t\t\t}\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn false, err\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t\treturn false, nil\n\t}\n\treturn len(name) == 0, nil\n}\n\n// scanChunk gets the next segment of pattern, which is a non-star string\n// possibly preceded by a star.\nfunc scanChunk(pattern string) (star bool, chunk, rest string) {\n\tfor len(pattern) > 0 && pattern[0] == '*' {\n\t\tpattern = pattern[1:]\n\t\tstar = true\n\t}\n\tinrange := false\n\tvar i int\nScan:\n\tfor i = 0; i < len(pattern); i++ {\n\t\tswitch pattern[i] {\n\t\tcase '\\\\':\n\t\t\t// error check handled in matchChunk: bad pattern.\n\t\t\tif i+1 < len(pattern) {\n\t\t\t\ti++\n\t\t\t}\n\t\tcase '[':\n\t\t\tinrange = true\n\t\tcase ']':\n\t\t\tinrange = false\n\t\tcase '*':\n\t\t\tif !inrange {\n\t\t\t\tbreak Scan\n\t\t\t}\n\t\t}\n\t}\n\treturn star, pattern[0:i], pattern[i:]\n}\n\n// matchChunk checks whether chunk matches the beginning of s.\n// If so, it returns the remainder of s (after the match).\n// Chunk is all single-character operators: literals, char classes, and ?.\nfunc matchChunk(chunk, s string) (rest string, ok bool, err error) {\n\tfor len(chunk) > 0 {\n\t\tif len(s) == 0 {\n\t\t\treturn\n\t\t}\n\t\tswitch chunk[0] {\n\t\tcase '[':\n\t\t\t// character class\n\t\t\tr, n := utf8.DecodeRuneInString(s)\n\t\t\ts = s[n:]\n\t\t\tchunk = chunk[1:]\n\t\t\t// We can't end right after '[', we're expecting at least\n\t\t\t// a closing bracket and possibly a caret.\n\t\t\tif len(chunk) == 0 {\n\t\t\t\terr = ErrBadPattern\n\t\t\t\treturn\n\t\t\t}\n\t\t\t// possibly negated\n\t\t\tnegated := chunk[0] == '^'\n\t\t\tif negated {\n\t\t\t\tchunk = chunk[1:]\n\t\t\t}\n\t\t\t// parse all ranges\n\t\t\tmatch := false\n\t\t\tnrange := 0\n\t\t\tfor {\n\t\t\t\tif len(chunk) > 0 && chunk[0] == ']' && nrange > 0 {\n\t\t\t\t\tchunk = chunk[1:]\n\t\t\t\t\tbreak\n\t\t\t\t}\n\t\t\t\tvar lo, hi rune\n\t\t\t\tif lo, chunk, err = getEsc(chunk); err != nil {\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t\thi = lo\n\t\t\t\tif chunk[0] == '-' {\n\t\t\t\t\tif hi, chunk, err = getEsc(chunk[1:]); err != nil {\n\t\t\t\t\t\treturn\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t\tif lo <= r && r <= hi {\n\t\t\t\t\tmatch = true\n\t\t\t\t}\n\t\t\t\tnrange++\n\t\t\t}\n\t\t\tif match == negated {\n\t\t\t\treturn\n\t\t\t}\n\n\t\tcase '?':\n\t\t\t_, n := utf8.DecodeRuneInString(s)\n\t\t\ts = s[n:]\n\t\t\tchunk = chunk[1:]\n\n\t\tcase '\\\\':\n\t\t\tchunk = chunk[1:]\n\t\t\tif len(chunk) == 0 {\n\t\t\t\terr = ErrBadPattern\n\t\t\t\treturn\n\t\t\t}\n\t\t\tfallthrough\n\n\t\tdefault:\n\t\t\tif chunk[0] != s[0] {\n\t\t\t\treturn\n\t\t\t}\n\t\t\ts = s[1:]\n\t\t\tchunk = chunk[1:]\n\t\t}\n\t}\n\treturn s, true, nil\n}\n\n// getEsc gets a possibly-escaped character from chunk, for a character class.\nfunc getEsc(chunk string) (r rune, nchunk string, err error) {\n\tif len(chunk) == 0 || chunk[0] == '-' || chunk[0] == ']' {\n\t\terr = ErrBadPattern\n\t\treturn\n\t}\n\tif chunk[0] == '\\\\' {\n\t\tchunk = chunk[1:]\n\t\tif len(chunk) == 0 {\n\t\t\terr = ErrBadPattern\n\t\t\treturn\n\t\t}\n\t}\n\tr, n := utf8.DecodeRuneInString(chunk)\n\tif r == utf8.RuneError && n == 1 {\n\t\terr = ErrBadPattern\n\t}\n\tnchunk = chunk[n:]\n\tif len(nchunk) == 0 {\n\t\terr = ErrBadPattern\n\t}\n\treturn\n}\n" }, { "path": "internal/hservice/gen.sh", "content": "#!/bin/bash\n\ncd $(dirname \"${BASH_SOURCE[0]}\")\nprotoc --go_out=plugins=grpc,import_path=hservice:. *.proto\n" }, { "path": "internal/hservice/hservice.pb.go", "content": "// Code generated by protoc-gen-go.\n// source: hservice.proto\n// DO NOT EDIT!\n\n/*\nPackage hservice is a generated protocol buffer package.\n\nIt is generated from these files:\n\thservice.proto\n\nIt has these top-level messages:\n\tMessageRequest\n\tMessageReply\n*/\npackage hservice\n\nimport proto \"github.com/golang/protobuf/proto\"\nimport fmt \"fmt\"\nimport math \"math\"\n\nimport (\n\tcontext \"golang.org/x/net/context\"\n\tgrpc \"google.golang.org/grpc\"\n)\n\n// Reference imports to suppress errors if they are not otherwise used.\nvar _ = proto.Marshal\nvar _ = fmt.Errorf\nvar _ = math.Inf\n\n// This is a compile-time assertion to ensure that this generated file\n// is compatible with the proto package it is being compiled against.\n// A compilation error at this line likely means your copy of the\n// proto package needs to be updated.\nconst _ = proto.ProtoPackageIsVersion2 // please upgrade the proto package\n\n// The request message containing the message value\ntype MessageRequest struct {\n\tValue string `protobuf:\"bytes,1,opt,name=value\" json:\"value,omitempty\"`\n}\n\nfunc (m *MessageRequest) Reset() { *m = MessageRequest{} }\nfunc (m *MessageRequest) String() string { return proto.CompactTextString(m) }\nfunc (*MessageRequest) ProtoMessage() {}\nfunc (*MessageRequest) Descriptor() ([]byte, []int) { return fileDescriptor0, []int{0} }\n\n// The response message containing an ok (true or false)\ntype MessageReply struct {\n\tOk bool `protobuf:\"varint,1,opt,name=ok\" json:\"ok,omitempty\"`\n}\n\nfunc (m *MessageReply) Reset() { *m = MessageReply{} }\nfunc (m *MessageReply) String() string { return proto.CompactTextString(m) }\nfunc (*MessageReply) ProtoMessage() {}\nfunc (*MessageReply) Descriptor() ([]byte, []int) { return fileDescriptor0, []int{1} }\n\nfunc init() {\n\tproto.RegisterType((*MessageRequest)(nil), \"hservice.MessageRequest\")\n\tproto.RegisterType((*MessageReply)(nil), \"hservice.MessageReply\")\n}\n\n// Reference imports to suppress errors if they are not otherwise used.\nvar _ context.Context\nvar _ grpc.ClientConn\n\n// This is a compile-time assertion to ensure that this generated file\n// is compatible with the grpc package it is being compiled against.\nconst _ = grpc.SupportPackageIsVersion3\n\n// Client API for HookService service\n\ntype HookServiceClient interface {\n\t// Sends a greeting\n\tSend(ctx context.Context, in *MessageRequest, opts ...grpc.CallOption) (*MessageReply, error)\n}\n\ntype hookServiceClient struct {\n\tcc *grpc.ClientConn\n}\n\nfunc NewHookServiceClient(cc *grpc.ClientConn) HookServiceClient {\n\treturn &hookServiceClient{cc}\n}\n\nfunc (c *hookServiceClient) Send(ctx context.Context, in *MessageRequest, opts ...grpc.CallOption) (*MessageReply, error) {\n\tout := new(MessageReply)\n\terr := grpc.Invoke(ctx, \"/hservice.HookService/Send\", in, out, c.cc, opts...)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn out, nil\n}\n\n// Server API for HookService service\n\ntype HookServiceServer interface {\n\t// Sends a greeting\n\tSend(context.Context, *MessageRequest) (*MessageReply, error)\n}\n\nfunc RegisterHookServiceServer(s *grpc.Server, srv HookServiceServer) {\n\ts.RegisterService(&_HookService_serviceDesc, srv)\n}\n\nfunc _HookService_Send_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {\n\tin := new(MessageRequest)\n\tif err := dec(in); err != nil {\n\t\treturn nil, err\n\t}\n\tif interceptor == nil {\n\t\treturn srv.(HookServiceServer).Send(ctx, in)\n\t}\n\tinfo := &grpc.UnaryServerInfo{\n\t\tServer: srv,\n\t\tFullMethod: \"/hservice.HookService/Send\",\n\t}\n\thandler := func(ctx context.Context, req interface{}) (interface{}, error) {\n\t\treturn srv.(HookServiceServer).Send(ctx, req.(*MessageRequest))\n\t}\n\treturn interceptor(ctx, in, info, handler)\n}\n\nvar _HookService_serviceDesc = grpc.ServiceDesc{\n\tServiceName: \"hservice.HookService\",\n\tHandlerType: (*HookServiceServer)(nil),\n\tMethods: []grpc.MethodDesc{\n\t\t{\n\t\t\tMethodName: \"Send\",\n\t\t\tHandler: _HookService_Send_Handler,\n\t\t},\n\t},\n\tStreams: []grpc.StreamDesc{},\n\tMetadata: fileDescriptor0,\n}\n\nfunc init() { proto.RegisterFile(\"hservice.proto\", fileDescriptor0) }\n\nvar fileDescriptor0 = []byte{\n\t// 168 bytes of a gzipped FileDescriptorProto\n\t0x1f, 0x8b, 0x08, 0x00, 0x00, 0x09, 0x6e, 0x88, 0x02, 0xff, 0xe2, 0xe2, 0xcb, 0x28, 0x4e, 0x2d,\n\t0x2a, 0xcb, 0x4c, 0x4e, 0xd5, 0x2b, 0x28, 0xca, 0x2f, 0xc9, 0x17, 0xe2, 0x80, 0xf1, 0x95, 0xd4,\n\t0xb8, 0xf8, 0x7c, 0x53, 0x8b, 0x8b, 0x13, 0xd3, 0x53, 0x83, 0x52, 0x0b, 0x4b, 0x53, 0x8b, 0x4b,\n\t0x84, 0x44, 0xb8, 0x58, 0xcb, 0x12, 0x73, 0x4a, 0x53, 0x25, 0x18, 0x15, 0x18, 0x35, 0x38, 0x83,\n\t0x20, 0x1c, 0x25, 0x39, 0x2e, 0x1e, 0xb8, 0xba, 0x82, 0x9c, 0x4a, 0x21, 0x3e, 0x2e, 0xa6, 0xfc,\n\t0x6c, 0xb0, 0x12, 0x8e, 0x20, 0xa6, 0xfc, 0x6c, 0x23, 0x4f, 0x2e, 0x6e, 0x8f, 0xfc, 0xfc, 0xec,\n\t0x60, 0x88, 0xb1, 0x42, 0x56, 0x5c, 0x2c, 0xc1, 0xa9, 0x79, 0x29, 0x42, 0x12, 0x7a, 0x70, 0x9b,\n\t0x51, 0xad, 0x91, 0x12, 0xc3, 0x22, 0x53, 0x90, 0x53, 0xa9, 0xc4, 0xe0, 0xa4, 0xc9, 0x25, 0x9c,\n\t0x9c, 0x9f, 0xab, 0x57, 0x92, 0x99, 0x93, 0x6a, 0x6c, 0x01, 0x57, 0xe5, 0x24, 0x80, 0x64, 0x7e,\n\t0x00, 0xc8, 0x17, 0x01, 0x8c, 0x49, 0x6c, 0x60, 0xef, 0x18, 0x03, 0x02, 0x00, 0x00, 0xff, 0xff,\n\t0x6d, 0xd0, 0x2b, 0x13, 0xe0, 0x00, 0x00, 0x00,\n}\n" }, { "path": "internal/hservice/hservice.proto", "content": "syntax = \"proto3\";\n\noption java_multiple_files = true;\noption java_package = \"com.tile38.hservice\";\noption java_outer_classname = \"HookServiceProto\";\n\npackage hservice;\n\n// The greeting service definition.\nservice HookService {\n // Sends a greeting\n rpc Send (MessageRequest) returns (MessageReply) {}\n}\n\n// The request message containing the message value\nmessage MessageRequest {\n string value = 1;\n}\n\n// The response message containing an ok (true or false)\nmessage MessageReply {\n bool ok = 1;\n}\n" }, { "path": "internal/log/log.go", "content": "package log\n\nimport (\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"io\"\n\t\"os\"\n\t\"sync\"\n\t\"sync/atomic\"\n\t\"time\"\n\n\t\"go.uber.org/zap\"\n\t\"golang.org/x/term\"\n)\n\nvar wmu sync.Mutex\nvar wr io.Writer\n\nvar zmu sync.Mutex\nvar zlogger *zap.SugaredLogger\n\nvar tty atomic.Bool\nvar ljson atomic.Bool\nvar llevel atomic.Int32\n\nfunc init() {\n\tSetOutput(os.Stderr)\n\tSetLevel(1)\n}\n\n// Level is the log level\n// 0: silent - do not log\n// 1: normal - show everything except debug and warn\n// 2: verbose - show everything except debug\n// 3: very verbose - show everything\nfunc SetLevel(level int) {\n\tif level < 0 {\n\t\tlevel = 0\n\t} else if level > 3 {\n\t\tlevel = 3\n\t}\n\tllevel.Store(int32(level))\n}\n\n// Level returns the log level\nfunc Level() int {\n\treturn int(llevel.Load())\n}\n\nfunc SetLogJSON(logJSON bool) {\n\tljson.Store(logJSON)\n}\n\nfunc LogJSON() bool {\n\treturn ljson.Load()\n}\n\n// SetOutput sets the output of the logger\nfunc SetOutput(w io.Writer) {\n\tf, ok := w.(*os.File)\n\ttty.Store(ok && term.IsTerminal(int(f.Fd())))\n\twmu.Lock()\n\twr = w\n\twmu.Unlock()\n}\n\n// Build a zap logger from default or custom config\nfunc Build(c string) error {\n\tvar zcfg zap.Config\n\tif c == \"\" {\n\t\tzcfg = zap.NewProductionConfig()\n\n\t\t// to be able to filter with Tile38 levels\n\t\tzcfg.Level.SetLevel(zap.DebugLevel)\n\t\t// disable caller because caller is always log.go\n\t\tzcfg.DisableCaller = true\n\n\t} else {\n\t\terr := json.Unmarshal([]byte(c), &zcfg)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\t// to be able to filter with Tile38 levels\n\t\tzcfg.Level.SetLevel(zap.DebugLevel)\n\t\t// disable caller because caller is always log.go\n\t\tzcfg.DisableCaller = true\n\t}\n\tcore, err := zcfg.Build()\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer core.Sync()\n\tzmu.Lock()\n\tzlogger = core.Sugar()\n\tzmu.Unlock()\n\treturn nil\n}\n\n// Set a zap logger\nfunc Set(sl *zap.SugaredLogger) {\n\tzmu.Lock()\n\tzlogger = sl\n\tzmu.Unlock()\n}\n\n// Get a zap logger\nfunc Get() *zap.SugaredLogger {\n\tzmu.Lock()\n\tsl := zlogger\n\tzmu.Unlock()\n\treturn sl\n}\n\n// Output returns the output writer\nfunc Output() io.Writer {\n\twmu.Lock()\n\tdefer wmu.Unlock()\n\treturn wr\n}\n\nfunc log(level int, tag, color string, formatted bool, format string, args ...interface{}) {\n\tif llevel.Load() < int32(level) {\n\t\treturn\n\t}\n\tvar msg string\n\tif formatted {\n\t\tmsg = fmt.Sprintf(format, args...)\n\t} else {\n\t\tmsg = fmt.Sprint(args...)\n\t}\n\tif ljson.Load() {\n\t\tzmu.Lock()\n\t\tdefer zmu.Unlock()\n\t\tswitch tag {\n\t\tcase \"ERRO\":\n\t\t\tzlogger.Error(msg)\n\t\tcase \"FATA\":\n\t\t\tzlogger.Fatal(msg)\n\t\tcase \"WARN\":\n\t\t\tzlogger.Warn(msg)\n\t\tcase \"DEBU\":\n\t\t\tzlogger.Debug(msg)\n\t\tdefault:\n\t\t\tzlogger.Info(msg)\n\t\t}\n\t\treturn\n\t}\n\ts := []byte(time.Now().Format(\"2006/01/02 15:04:05\"))\n\ts = append(s, ' ')\n\tif tty.Load() {\n\t\ts = append(s, color...)\n\t}\n\ts = append(s, '[')\n\ts = append(s, tag...)\n\ts = append(s, ']')\n\tif tty.Load() {\n\t\ts = append(s, \"\\x1b[0m\"...)\n\t}\n\ts = append(s, ' ')\n\ts = append(s, msg...)\n\tif s[len(s)-1] != '\\n' {\n\t\ts = append(s, '\\n')\n\t}\n\twmu.Lock()\n\twr.Write(s)\n\twmu.Unlock()\n}\n\nvar emptyFormat string\n\n// Infof ...\nfunc Infof(format string, args ...interface{}) {\n\tif llevel.Load() >= 1 {\n\t\tlog(1, \"INFO\", \"\\x1b[36m\", true, format, args...)\n\t}\n}\n\n// Info ...\nfunc Info(args ...interface{}) {\n\tif llevel.Load() >= 1 {\n\t\tlog(1, \"INFO\", \"\\x1b[36m\", false, emptyFormat, args...)\n\t}\n}\n\n// HTTPf ...\nfunc HTTPf(format string, args ...interface{}) {\n\tif llevel.Load() >= 1 {\n\t\tlog(1, \"HTTP\", \"\\x1b[1m\\x1b[30m\", true, format, args...)\n\t}\n}\n\n// HTTP ...\nfunc HTTP(args ...interface{}) {\n\tif llevel.Load() >= 1 {\n\t\tlog(1, \"HTTP\", \"\\x1b[1m\\x1b[30m\", false, emptyFormat, args...)\n\t}\n}\n\n// Errorf ...\nfunc Errorf(format string, args ...interface{}) {\n\tif llevel.Load() >= 1 {\n\t\tlog(1, \"ERRO\", \"\\x1b[1m\\x1b[31m\", true, format, args...)\n\t}\n}\n\n// Error ..\nfunc Error(args ...interface{}) {\n\tif llevel.Load() >= 1 {\n\t\tlog(1, \"ERRO\", \"\\x1b[1m\\x1b[31m\", false, emptyFormat, args...)\n\t}\n}\n\n// Warnf ...\nfunc Warnf(format string, args ...interface{}) {\n\tif llevel.Load() >= 1 {\n\t\tlog(2, \"WARN\", \"\\x1b[33m\", true, format, args...)\n\t}\n}\n\n// Warn ...\nfunc Warn(args ...interface{}) {\n\tif llevel.Load() >= 1 {\n\t\tlog(2, \"WARN\", \"\\x1b[33m\", false, emptyFormat, args...)\n\t}\n}\n\n// Debugf ...\nfunc Debugf(format string, args ...interface{}) {\n\tif llevel.Load() >= 3 {\n\t\tlog(3, \"DEBU\", \"\\x1b[35m\", true, format, args...)\n\t}\n}\n\n// Debug ...\nfunc Debug(args ...interface{}) {\n\tif llevel.Load() >= 3 {\n\t\tlog(3, \"DEBU\", \"\\x1b[35m\", false, emptyFormat, args...)\n\t}\n}\n\n// Printf ...\nfunc Printf(format string, args ...interface{}) {\n\tInfof(format, args...)\n}\n\n// Print ...\nfunc Print(args ...interface{}) {\n\tInfo(args...)\n}\n\n// Fatalf ...\nfunc Fatalf(format string, args ...interface{}) {\n\tlog(1, \"FATA\", \"\\x1b[31m\", true, format, args...)\n\tos.Exit(1)\n}\n\n// Fatal ...\nfunc Fatal(args ...interface{}) {\n\tlog(1, \"FATA\", \"\\x1b[31m\", false, emptyFormat, args...)\n\tos.Exit(1)\n}\n" }, { "path": "internal/log/log_test.go", "content": "package log\n\nimport (\n\t\"bytes\"\n\t\"io\"\n\t\"strings\"\n\t\"testing\"\n\n\t\"go.uber.org/zap\"\n\t\"go.uber.org/zap/zapcore\"\n\t\"go.uber.org/zap/zaptest/observer\"\n)\n\nfunc TestLog(t *testing.T) {\n\tf := &bytes.Buffer{}\n\tSetLogJSON(false)\n\tSetOutput(f)\n\tPrintf(\"hello %v\", \"everyone\")\n\tif !strings.HasSuffix(f.String(), \"hello everyone\\n\") {\n\t\tt.Fatal(\"fail\")\n\t}\n}\n\nfunc TestLogJSON(t *testing.T) {\n\n\tSetLogJSON(true)\n\tBuild(\"\")\n\n\ttype tcase struct {\n\t\tlevel int\n\t\tformat string\n\t\targs string\n\t\tops func(...interface{})\n\t\tfops func(string, ...interface{})\n\t\texpMsg string\n\t\texpLvl zapcore.Level\n\t}\n\n\tfn := func(tc tcase) func(*testing.T) {\n\t\treturn func(t *testing.T) {\n\t\t\tobservedZapCore, observedLogs := observer.New(zap.DebugLevel)\n\t\t\tSet(zap.New(observedZapCore).Sugar())\n\t\t\tSetLevel(tc.level)\n\n\t\t\tif tc.format != \"\" {\n\t\t\t\ttc.fops(tc.format, tc.args)\n\t\t\t} else {\n\t\t\t\ttc.ops(tc.args)\n\t\t\t}\n\n\t\t\tif observedLogs.Len() < 1 {\n\t\t\t\tt.Fatal(\"fail\")\n\t\t\t}\n\n\t\t\tallLogs := observedLogs.All()\n\n\t\t\tif allLogs[0].Message != tc.expMsg {\n\t\t\t\tt.Fatal(\"fail\")\n\t\t\t}\n\n\t\t\tif allLogs[0].Level != tc.expLvl {\n\t\t\t\tt.Fatal(\"fail\")\n\t\t\t}\n\t\t}\n\t}\n\n\ttests := map[string]tcase{\n\t\t\"Print\": {\n\t\t\tlevel: 1,\n\t\t\targs: \"Print json logger\",\n\t\t\tops: func(args ...interface{}) {\n\t\t\t\tPrint(args...)\n\t\t\t},\n\t\t\texpMsg: \"Print json logger\",\n\t\t\texpLvl: zapcore.InfoLevel,\n\t\t},\n\t\t\"Printf\": {\n\t\t\tlevel: 1,\n\t\t\tformat: \"Printf json %v\",\n\t\t\targs: \"logger\",\n\t\t\tfops: func(format string, args ...interface{}) {\n\t\t\t\tPrintf(format, args...)\n\t\t\t},\n\t\t\texpMsg: \"Printf json logger\",\n\t\t\texpLvl: zapcore.InfoLevel,\n\t\t},\n\t\t\"Info\": {\n\t\t\tlevel: 1,\n\t\t\targs: \"Info json logger\",\n\t\t\tops: func(args ...interface{}) {\n\t\t\t\tInfo(args...)\n\t\t\t},\n\t\t\texpMsg: \"Info json logger\",\n\t\t\texpLvl: zapcore.InfoLevel,\n\t\t},\n\t\t\"Infof\": {\n\t\t\tlevel: 1,\n\t\t\tformat: \"Infof json %v\",\n\t\t\targs: \"logger\",\n\t\t\tfops: func(format string, args ...interface{}) {\n\t\t\t\tInfof(format, args...)\n\t\t\t},\n\t\t\texpMsg: \"Infof json logger\",\n\t\t\texpLvl: zapcore.InfoLevel,\n\t\t},\n\t\t\"Debug\": {\n\t\t\tlevel: 3,\n\t\t\targs: \"Debug json logger\",\n\t\t\tops: func(args ...interface{}) {\n\t\t\t\tDebug(args...)\n\t\t\t},\n\t\t\texpMsg: \"Debug json logger\",\n\t\t\texpLvl: zapcore.DebugLevel,\n\t\t},\n\t\t\"Debugf\": {\n\t\t\tlevel: 3,\n\t\t\tformat: \"Debugf json %v\",\n\t\t\targs: \"logger\",\n\t\t\tfops: func(format string, args ...interface{}) {\n\t\t\t\tDebugf(format, args...)\n\t\t\t},\n\t\t\texpMsg: \"Debugf json logger\",\n\t\t\texpLvl: zapcore.DebugLevel,\n\t\t},\n\t\t\"Warn\": {\n\t\t\tlevel: 2,\n\t\t\targs: \"Warn json logger\",\n\t\t\tops: func(args ...interface{}) {\n\t\t\t\tWarn(args...)\n\t\t\t},\n\t\t\texpMsg: \"Warn json logger\",\n\t\t\texpLvl: zapcore.WarnLevel,\n\t\t},\n\t\t\"Warnf\": {\n\t\t\tlevel: 2,\n\t\t\tformat: \"Warnf json %v\",\n\t\t\targs: \"logger\",\n\t\t\tfops: func(format string, args ...interface{}) {\n\t\t\t\tWarnf(format, args...)\n\t\t\t},\n\t\t\texpMsg: \"Warnf json logger\",\n\t\t\texpLvl: zapcore.WarnLevel,\n\t\t},\n\t\t\"Error\": {\n\t\t\tlevel: 1,\n\t\t\targs: \"Error json logger\",\n\t\t\tops: func(args ...interface{}) {\n\t\t\t\tError(args...)\n\t\t\t},\n\t\t\texpMsg: \"Error json logger\",\n\t\t\texpLvl: zapcore.ErrorLevel,\n\t\t},\n\t\t\"Errorf\": {\n\t\t\tlevel: 1,\n\t\t\tformat: \"Errorf json %v\",\n\t\t\targs: \"logger\",\n\t\t\tfops: func(format string, args ...interface{}) {\n\t\t\t\tErrorf(format, args...)\n\t\t\t},\n\t\t\texpMsg: \"Errorf json logger\",\n\t\t\texpLvl: zapcore.ErrorLevel,\n\t\t},\n\t\t\"Http\": {\n\t\t\tlevel: 1,\n\t\t\targs: \"Http json logger\",\n\t\t\tops: func(args ...interface{}) {\n\t\t\t\tHTTP(args...)\n\t\t\t},\n\t\t\texpMsg: \"Http json logger\",\n\t\t\texpLvl: zapcore.InfoLevel,\n\t\t},\n\t\t\"Httpf\": {\n\t\t\tlevel: 1,\n\t\t\tformat: \"Httpf json %v\",\n\t\t\targs: \"logger\",\n\t\t\tfops: func(format string, args ...interface{}) {\n\t\t\t\tHTTPf(format, args...)\n\t\t\t},\n\t\t\texpMsg: \"Httpf json logger\",\n\t\t\texpLvl: zapcore.InfoLevel,\n\t\t},\n\t}\n\n\tfor name, tc := range tests {\n\t\tt.Run(name, fn(tc))\n\t}\n}\n\nfunc BenchmarkLogPrintf(t *testing.B) {\n\tSetLogJSON(false)\n\tSetLevel(1)\n\tSetOutput(io.Discard)\n\tt.ResetTimer()\n\tfor i := 0; i < t.N; i++ {\n\t\tPrintf(\"X %s\", \"Y\")\n\t}\n}\n\nfunc BenchmarkLogJSONPrintf(t *testing.B) {\n\tSetLogJSON(true)\n\tSetLevel(1)\n\n\tec := zap.NewProductionEncoderConfig()\n\tec.EncodeDuration = zapcore.NanosDurationEncoder\n\tec.EncodeTime = zapcore.EpochNanosTimeEncoder\n\tenc := zapcore.NewJSONEncoder(ec)\n\n\tlogger := zap.New(\n\t\tzapcore.NewCore(\n\t\t\tenc,\n\t\t\tzapcore.AddSync(io.Discard),\n\t\t\tzap.DebugLevel,\n\t\t)).Sugar()\n\n\tSet(logger)\n\tt.ResetTimer()\n\tfor i := 0; i < t.N; i++ {\n\t\tPrintf(\"X %s\", \"Y\")\n\t}\n}\n" }, { "path": "internal/object/object_binary.go", "content": "package object\n\nimport (\n\t\"encoding/binary\"\n\t\"unsafe\"\n\n\t\"github.com/tidwall/geojson\"\n\t\"github.com/tidwall/geojson/geometry\"\n\t\"github.com/tidwall/tile38/internal/field\"\n)\n\ntype pointObject struct {\n\tbase Object\n\tpt geojson.SimplePoint\n}\n\ntype geoObject struct {\n\tbase Object\n\tgeo geojson.Object\n}\n\nconst opoint = 1\nconst ogeo = 2\n\ntype Object struct {\n\thead string // tuple (kind,expires,id)\n\tfields field.List\n}\n\nfunc (o *Object) geo() geojson.Object {\n\tif o != nil {\n\t\tswitch o.head[0] {\n\t\tcase opoint:\n\t\t\treturn &(*pointObject)(unsafe.Pointer(o)).pt\n\t\tcase ogeo:\n\t\t\treturn (*geoObject)(unsafe.Pointer(o)).geo\n\t\t}\n\t}\n\treturn nil\n}\n\n// uvarint is a slightly modified version of binary.Uvarint, and it's a little\n// faster. But it lacks overflow checks which are not needed for our use.\nfunc uvarint(s string) (uint64, int) {\n\tvar x uint64\n\tfor i := 0; i < len(s); i++ {\n\t\tb := s[i]\n\t\tif b < 0x80 {\n\t\t\treturn x | uint64(b)<<(i*7), i + 1\n\t\t}\n\t\tx |= uint64(b&0x7f) << (i * 7)\n\t}\n\treturn 0, 0\n}\n\nfunc varint(s string) (int64, int) {\n\tux, n := uvarint(s)\n\tx := int64(ux >> 1)\n\tif ux&1 != 0 {\n\t\tx = ^x\n\t}\n\treturn x, n\n}\n\nfunc (o *Object) ID() string {\n\tif o.head[1] == 0 {\n\t\treturn o.head[2:]\n\t}\n\t_, n := varint(o.head[1:])\n\treturn o.head[1+n:]\n}\n\nfunc (o *Object) Fields() field.List {\n\treturn o.fields\n}\n\nfunc (o *Object) Expires() int64 {\n\tex, _ := varint(o.head[1:])\n\treturn ex\n}\n\nfunc (o *Object) Rect() geometry.Rect {\n\togeo := o.geo()\n\tif ogeo == nil {\n\t\treturn geometry.Rect{}\n\t}\n\treturn ogeo.Rect()\n}\n\nfunc (o *Object) Geo() geojson.Object {\n\treturn o.geo()\n}\n\nfunc (o *Object) String() string {\n\togeo := o.geo()\n\tif ogeo == nil {\n\t\treturn \"\"\n\t}\n\treturn ogeo.String()\n}\n\nfunc (o *Object) IsSpatial() bool {\n\t_, ok := o.geo().(geojson.Spatial)\n\treturn ok\n}\n\nfunc (o *Object) Weight() int {\n\tvar weight int\n\tweight += len(o.ID())\n\togeo := o.geo()\n\tif ogeo != nil {\n\t\tif o.IsSpatial() {\n\t\t\tweight += ogeo.NumPoints() * 16\n\t\t} else {\n\t\t\tweight += len(ogeo.String())\n\t\t}\n\t}\n\tweight += o.Fields().Weight()\n\treturn weight\n}\n\nfunc makeHead(kind byte, id string, expires int64) string {\n\tvar exb [20]byte\n\texn := 1\n\tif expires != 0 {\n\t\texn = binary.PutVarint(exb[:], expires)\n\t}\n\tn := 1 + exn + len(id)\n\thead := make([]byte, n)\n\thead[0] = kind\n\tcopy(head[1:], exb[:exn])\n\tcopy(head[1+exn:], id)\n\treturn *(*string)(unsafe.Pointer(&head))\n}\n\nfunc newPoint(id string, pt geometry.Point, expires int64, fields field.List,\n) *Object {\n\treturn (*Object)(unsafe.Pointer(&pointObject{\n\t\tObject{\n\t\t\thead: makeHead(opoint, id, expires),\n\t\t\tfields: fields,\n\t\t},\n\t\tgeojson.SimplePoint{Point: pt},\n\t}))\n}\nfunc newGeo(id string, geo geojson.Object, expires int64, fields field.List,\n) *Object {\n\treturn (*Object)(unsafe.Pointer(&geoObject{\n\t\tObject{\n\t\t\thead: makeHead(ogeo, id, expires),\n\t\t\tfields: fields,\n\t\t},\n\t\tgeo,\n\t}))\n}\n\nfunc New(id string, geo geojson.Object, expires int64, fields field.List,\n) *Object {\n\tswitch p := geo.(type) {\n\tcase *geojson.SimplePoint:\n\t\treturn newPoint(id, p.Base(), expires, fields)\n\tcase *geojson.Point:\n\t\tif p.IsSimple() {\n\t\t\treturn newPoint(id, p.Base(), expires, fields)\n\t\t}\n\t}\n\treturn newGeo(id, geo, expires, fields)\n}\n" }, { "path": "internal/object/object_struct.go", "content": "//go:build exclude\n\npackage object\n\nimport (\n\t\"github.com/tidwall/geojson\"\n\t\"github.com/tidwall/geojson/geometry\"\n\t\"github.com/tidwall/tile38/internal/field\"\n)\n\ntype Object struct {\n\tid string\n\tgeo geojson.Object\n\texpires int64 // unix nano expiration\n\tfields field.List\n}\n\nfunc (o *Object) ID() string {\n\tif o == nil {\n\t\treturn \"\"\n\t}\n\treturn o.id\n}\n\nfunc (o *Object) Fields() field.List {\n\tif o == nil {\n\t\treturn field.List{}\n\t}\n\treturn o.fields\n}\n\nfunc (o *Object) Expires() int64 {\n\tif o == nil {\n\t\treturn 0\n\t}\n\treturn o.expires\n}\n\nfunc (o *Object) Rect() geometry.Rect {\n\tif o == nil || o.geo == nil {\n\t\treturn geometry.Rect{}\n\t}\n\treturn o.geo.Rect()\n}\n\nfunc (o *Object) Geo() geojson.Object {\n\tif o == nil || o.geo == nil {\n\t\treturn nil\n\t}\n\treturn o.geo\n}\n\nfunc (o *Object) String() string {\n\tif o == nil || o.geo == nil {\n\t\treturn \"\"\n\t}\n\treturn o.geo.String()\n}\n\nfunc (o *Object) IsSpatial() bool {\n\t_, ok := o.geo.(geojson.Spatial)\n\treturn ok\n}\n\nfunc (o *Object) Weight() int {\n\tif o == nil {\n\t\treturn 0\n\t}\n\tvar weight int\n\tweight += len(o.ID())\n\tif o.IsSpatial() {\n\t\tweight += o.Geo().NumPoints() * 16\n\t} else {\n\t\tweight += len(o.Geo().String())\n\t}\n\tweight += o.Fields().Weight()\n\treturn weight\n}\n\nfunc New(id string, geo geojson.Object, expires int64, fields field.List,\n) *Object {\n\treturn &Object{\n\t\tid: id,\n\t\tgeo: geo,\n\t\texpires: expires,\n\t\tfields: fields,\n\t}\n}\n" }, { "path": "internal/object/object_test.go", "content": "package object\n\nimport (\n\t\"testing\"\n\n\t\"github.com/tidwall/assert\"\n\t\"github.com/tidwall/geojson\"\n\t\"github.com/tidwall/geojson/geometry\"\n\t\"github.com/tidwall/tile38/internal/field\"\n)\n\nfunc P(x, y float64) geojson.Object {\n\treturn geojson.NewSimplePoint(geometry.Point{X: 10, Y: 20})\n}\nfunc TestObject(t *testing.T) {\n\to := New(\"hello\", P(10, 20), 99, field.List{})\n\tassert.Assert(o.ID() == \"hello\")\n}\n" }, { "path": "internal/server/aof.go", "content": "package server\n\nimport (\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"math\"\n\t\"net\"\n\t\"os\"\n\t\"sort\"\n\t\"strconv\"\n\t\"strings\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/tidwall/buntdb\"\n\t\"github.com/tidwall/gjson\"\n\t\"github.com/tidwall/redcon\"\n\t\"github.com/tidwall/resp\"\n\t\"github.com/tidwall/tile38/internal/log\"\n)\n\ntype errAOFHook struct {\n\terr error\n}\n\nfunc (err errAOFHook) Error() string {\n\treturn fmt.Sprintf(\"hook: %v\", err.err)\n}\n\nfunc (s *Server) loadAOF() (err error) {\n\tfi, err := s.aof.Stat()\n\tif err != nil {\n\t\treturn err\n\t}\n\tstart := time.Now()\n\tvar count int\n\tdefer func() {\n\t\td := time.Since(start)\n\t\tps := float64(count) / (float64(d) / float64(time.Second))\n\t\tsuf := []string{\"bytes/s\", \"KB/s\", \"MB/s\", \"GB/s\", \"TB/s\"}\n\t\tbps := float64(fi.Size()) / (float64(d) / float64(time.Second))\n\t\tfor i := 0; bps > 1024 && len(suf) > 1; i++ {\n\t\t\tbps /= 1024\n\t\t\tsuf = suf[1:]\n\t\t}\n\t\tbyteSpeed := fmt.Sprintf(\"%.0f %s\", bps, suf[0])\n\t\tlog.Infof(\"AOF loaded %d commands: %.2fs, %.0f/s, %s\",\n\t\t\tcount, float64(d)/float64(time.Second), ps, byteSpeed)\n\t}()\n\tvar buf []byte\n\tvar args [][]byte\n\tvar packet [0xFFFF]byte\n\tfor {\n\t\tn, err := s.aof.Read(packet[:])\n\t\tif err != nil {\n\t\t\tif err != io.EOF {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tif len(buf) > 0 {\n\t\t\t\t// There was an incomplete command or other data at the end of\n\t\t\t\t// the AOF file. Attempt to recover the file by truncating the\n\t\t\t\t// file at the end position of the last complete command.\n\t\t\t\tlog.Warnf(\"Truncating %d bytes due to an incomplete command\\n\",\n\t\t\t\t\tlen(buf))\n\t\t\t\ts.aofsz -= len(buf)\n\t\t\t\tif err := s.aof.Truncate(int64(s.aofsz)); err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\t\t\t\tif _, err := s.aof.Seek(int64(s.aofsz), 0); err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\t\t\t}\n\t\t\treturn nil\n\t\t}\n\t\ts.aofsz += n\n\t\tdata := packet[:n]\n\t\tif len(buf) > 0 {\n\t\t\tdata = append(buf, data...)\n\t\t}\n\t\tvar complete bool\n\t\tfor {\n\t\t\tif len(data) > 0 && data[0] == 0 {\n\t\t\t\t// Zeros found in AOF file (issue #230).\n\t\t\t\t// Just ignore it and move the next byte.\n\t\t\t\tdata = data[1:]\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tcomplete, args, _, data, err = redcon.ReadNextCommand(data, args[:0])\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tif !complete {\n\t\t\t\tbreak\n\t\t\t}\n\t\t\tif len(args) > 0 {\n\t\t\t\tvar msg Message\n\t\t\t\tmsg.Args = msg.Args[:0]\n\t\t\t\tfor _, arg := range args {\n\t\t\t\t\tmsg.Args = append(msg.Args, string(arg))\n\t\t\t\t}\n\t\t\t\tif _, _, err := s.command(&msg, nil); err != nil {\n\t\t\t\t\tif commandErrIsFatal(err) {\n\t\t\t\t\t\treturn err\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t\tcount++\n\t\t\t}\n\t\t}\n\t\tif len(data) > 0 {\n\t\t\tbuf = append(buf[:0], data...)\n\t\t} else if len(buf) > 0 {\n\t\t\tbuf = buf[:0]\n\t\t}\n\t}\n}\n\nfunc commandErrIsFatal(err error) bool {\n\t// FSET (and other writable commands) may return errors that we need\n\t// to ignore during the loading process. These errors may occur (though unlikely)\n\t// due to the aof rewrite operation.\n\treturn !(err == errKeyNotFound || err == errIDNotFound)\n}\n\n// flushAOF flushes all aof buffer data to disk. Set sync to true to sync the\n// fsync the file.\nfunc (s *Server) flushAOF(sync bool) {\n\tif len(s.aofbuf) > 0 {\n\t\t_, err := s.aof.Write(s.aofbuf)\n\t\tif err != nil {\n\t\t\tpanic(err)\n\t\t}\n\t\t// send a broadcast to all sleeping followers\n\t\ts.fcond.Broadcast()\n\t\tif sync {\n\t\t\tif err := s.aof.Sync(); err != nil {\n\t\t\t\tpanic(err)\n\t\t\t}\n\t\t}\n\t\tif cap(s.aofbuf) > 1024*1024*32 {\n\t\t\ts.aofbuf = make([]byte, 0, 1024*1024*32)\n\t\t} else {\n\t\t\ts.aofbuf = s.aofbuf[:0]\n\t\t}\n\t}\n}\n\nfunc (s *Server) writeAOF(args []string, d *commandDetails) error {\n\tif d != nil && !d.updated {\n\t\t// just ignore writes if the command did not update\n\t\treturn nil\n\t}\n\n\tif s.shrinking {\n\t\tnargs := make([]string, len(args))\n\t\tcopy(nargs, args)\n\t\ts.shrinklog = append(s.shrinklog, nargs)\n\t}\n\n\tif s.aof != nil {\n\t\ts.aofdirty.Store(true) // prewrite optimization flag\n\t\tn := len(s.aofbuf)\n\t\ts.aofbuf = redcon.AppendArray(s.aofbuf, len(args))\n\t\tfor _, arg := range args {\n\t\t\ts.aofbuf = redcon.AppendBulkString(s.aofbuf, arg)\n\t\t}\n\t\ts.aofsz += len(s.aofbuf) - n\n\t}\n\n\t// process geofences\n\tif d != nil {\n\t\t// webhook geofences\n\t\tif s.config.followHost() == \"\" {\n\t\t\t// for leader only\n\t\t\tif d.parent {\n\t\t\t\t// queue children\n\t\t\t\tfor _, d := range d.children {\n\t\t\t\t\tif err := s.queueHooks(d); err != nil {\n\t\t\t\t\t\treturn err\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\t// queue parent\n\t\t\t\tif err := s.queueHooks(d); err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\n\t\t// live geofences\n\t\ts.lcond.L.Lock()\n\t\tif len(s.lives) > 0 {\n\t\t\tif d.parent {\n\t\t\t\t// queue children\n\t\t\t\ts.lstack = append(s.lstack, d.children...)\n\t\t\t} else {\n\t\t\t\t// queue parent\n\t\t\t\ts.lstack = append(s.lstack, d)\n\t\t\t}\n\t\t\ts.lcond.Broadcast()\n\t\t}\n\t\ts.lcond.L.Unlock()\n\t}\n\treturn nil\n}\n\nfunc (s *Server) getQueueCandidates(d *commandDetails) []*Hook {\n\tcandidates := make(map[*Hook]bool)\n\t// add the hooks with \"outside\" detection\n\ts.hooksOut.Ascend(nil, func(v interface{}) bool {\n\t\thook := v.(*Hook)\n\t\tif hook.Key == d.key {\n\t\t\tcandidates[hook] = true\n\t\t}\n\t\treturn true\n\t})\n\t// look for candidates that might \"cross\" geofences\n\tif d.old != nil && d.obj != nil && s.hookCross.Len() > 0 {\n\t\tr1, r2 := d.old.Rect(), d.obj.Rect()\n\t\ts.hookCross.Search(\n\t\t\t[2]float64{\n\t\t\t\tmath.Min(r1.Min.X, r2.Min.X),\n\t\t\t\tmath.Min(r1.Min.Y, r2.Min.Y),\n\t\t\t},\n\t\t\t[2]float64{\n\t\t\t\tmath.Max(r1.Max.X, r2.Max.X),\n\t\t\t\tmath.Max(r1.Max.Y, r2.Max.Y),\n\t\t\t},\n\t\t\tfunc(min, max [2]float64, value interface{}) bool {\n\t\t\t\thook := value.(*Hook)\n\t\t\t\tif hook.Key == d.key {\n\t\t\t\t\tcandidates[hook] = true\n\t\t\t\t}\n\t\t\t\treturn true\n\t\t\t})\n\t}\n\t// look for candidates that overlap the old object\n\tif d.old != nil {\n\t\tr1 := d.old.Rect()\n\t\ts.hookTree.Search(\n\t\t\t[2]float64{r1.Min.X, r1.Min.Y},\n\t\t\t[2]float64{r1.Max.X, r1.Max.Y},\n\t\t\tfunc(min, max [2]float64, value interface{}) bool {\n\t\t\t\thook := value.(*Hook)\n\t\t\t\tif hook.Key == d.key {\n\t\t\t\t\tcandidates[hook] = true\n\t\t\t\t}\n\t\t\t\treturn true\n\t\t\t})\n\t}\n\t// look for candidates that overlap the new object\n\tif d.obj != nil {\n\t\tr1 := d.obj.Rect()\n\t\ts.hookTree.Search(\n\t\t\t[2]float64{r1.Min.X, r1.Min.Y},\n\t\t\t[2]float64{r1.Max.X, r1.Max.Y},\n\t\t\tfunc(min, max [2]float64, value interface{}) bool {\n\t\t\t\thook := value.(*Hook)\n\t\t\t\tif hook.Key == d.key {\n\t\t\t\t\tcandidates[hook] = true\n\t\t\t\t}\n\t\t\t\treturn true\n\t\t\t})\n\t}\n\tif len(candidates) == 0 {\n\t\treturn nil\n\t}\n\t// return the candidates as a slice\n\tret := make([]*Hook, 0, len(candidates))\n\tfor hook := range candidates {\n\t\tret = append(ret, hook)\n\t}\n\treturn ret\n}\n\nfunc (s *Server) queueHooks(d *commandDetails) error {\n\t// Create the slices that will store all messages and hooks\n\tvar cmsgs, wmsgs []string\n\tvar whooks []*Hook\n\n\t// Compile a slice of potential hook recipients\n\tcandidates := s.getQueueCandidates(d)\n\tfor _, hook := range candidates {\n\t\t// Calculate all matching fence messages for all candidates and append\n\t\t// them to the appropriate message slice\n\t\tmsgs := FenceMatch(hook.Name, hook.ScanWriter, hook.Fence, hook.Metas, d)\n\t\tif len(msgs) > 0 {\n\t\t\tif hook.channel {\n\t\t\t\tcmsgs = append(cmsgs, msgs...)\n\t\t\t} else {\n\t\t\t\twmsgs = append(wmsgs, msgs...)\n\t\t\t\twhooks = append(whooks, hook)\n\t\t\t}\n\t\t}\n\t}\n\n\t// Return nil if there are no messages to be sent\n\tif len(cmsgs)+len(wmsgs) == 0 {\n\t\treturn nil\n\t}\n\n\t// Sort both message channel and webhook message slices\n\tif len(cmsgs) > 1 {\n\t\tsortMsgs(cmsgs)\n\t}\n\tif len(wmsgs) > 1 {\n\t\tsortMsgs(wmsgs)\n\t}\n\n\t// Publish all channel messages if any exist\n\tif len(cmsgs) > 0 {\n\t\tfor _, m := range cmsgs {\n\t\t\ts.Publish(gjson.Get(m, \"hook\").String(), m)\n\t\t}\n\t}\n\n\t// Queue the webhook messages in the buntdb database\n\terr := s.qdb.Update(func(tx *buntdb.Tx) error {\n\t\tfor _, msg := range wmsgs {\n\t\t\ts.qidx++ // increment the log id\n\t\t\tkey := hookLogPrefix + uint64ToString(s.qidx)\n\t\t\t_, _, err := tx.Set(key, msg, hookLogSetDefaults)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tlog.Debugf(\"queued hook: %d\", s.qidx)\n\t\t}\n\t\t_, _, err := tx.Set(\"hook:idx\", uint64ToString(s.qidx), nil)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\treturn nil\n\t})\n\tif err != nil {\n\t\treturn err\n\t}\n\t// all the messages have been queued.\n\t// notify the hooks\n\tfor _, hook := range whooks {\n\t\thook.Signal()\n\t}\n\treturn nil\n}\n\n// sortMsgs sorts passed notification messages by their detect and hook fields\nfunc sortMsgs(msgs []string) {\n\tsort.SliceStable(msgs, func(i, j int) bool {\n\t\tdetectI := msgDetectCode(gjson.Get(msgs[i], \"detect\").String())\n\t\tdetectJ := msgDetectCode(gjson.Get(msgs[j], \"detect\").String())\n\t\tif detectI < detectJ {\n\t\t\treturn true\n\t\t}\n\t\tif detectI > detectJ {\n\t\t\treturn false\n\t\t}\n\t\thookI := gjson.Get(msgs[i], \"hook\").String()\n\t\thookJ := gjson.Get(msgs[j], \"hook\").String()\n\t\treturn hookI < hookJ\n\t})\n}\n\n// msgDetectCode returns a weight value for the passed detect value\nfunc msgDetectCode(detect string) int {\n\tswitch detect {\n\tcase \"exit\":\n\t\treturn 1\n\tcase \"outside\":\n\t\treturn 2\n\tcase \"enter\":\n\t\treturn 3\n\tcase \"inside\":\n\t\treturn 4\n\tdefault:\n\t\treturn 0\n\t}\n}\n\n// Converts string to an integer\nfunc stringToUint64(s string) uint64 {\n\tn, _ := strconv.ParseUint(s, 10, 64)\n\treturn n\n}\n\n// Converts a uint to a string\nfunc uint64ToString(u uint64) string {\n\ts := strings.Repeat(\"0\", 20) + strconv.FormatUint(u, 10)\n\treturn s[len(s)-20:]\n}\n\ntype liveAOFSwitches struct {\n\tpos int64\n}\n\nfunc (s liveAOFSwitches) Error() string {\n\treturn goingLive\n}\n\n// AOFMD5 pos size\nfunc (s *Server) cmdAOFMD5(msg *Message) (resp.Value, error) {\n\tstart := time.Now()\n\n\t// >> Args\n\n\targs := msg.Args\n\tif len(args) != 3 {\n\t\treturn retrerr(errInvalidNumberOfArguments)\n\t}\n\tpos, err := strconv.ParseInt(args[1], 10, 64)\n\tif err != nil || pos < 0 {\n\t\treturn retrerr(errInvalidArgument(args[1]))\n\t}\n\tsize, err := strconv.ParseInt(args[2], 10, 64)\n\tif err != nil || size < 0 {\n\t\treturn retrerr(errInvalidArgument(args[2]))\n\t}\n\n\t// >> Operation\n\n\tsum, err := s.checksum(pos, size)\n\tif err != nil {\n\t\treturn retrerr(err)\n\t}\n\n\t// >> Response\n\n\tif msg.OutputType == JSON {\n\t\treturn resp.StringValue(fmt.Sprintf(\n\t\t\t`{\"ok\":true,\"md5\":\"%s\",\"elapsed\":\"%s\"}`,\n\t\t\tsum, time.Since(start))), nil\n\t}\n\treturn resp.SimpleStringValue(sum), nil\n}\n\n// AOF pos\nfunc (s *Server) cmdAOF(msg *Message) (resp.Value, error) {\n\tif s.aof == nil {\n\t\treturn retrerr(errors.New(\"aof disabled\"))\n\t}\n\n\t// >> Args\n\n\targs := msg.Args\n\tif len(args) != 2 {\n\t\treturn retrerr(errInvalidNumberOfArguments)\n\t}\n\n\tpos, err := strconv.ParseInt(args[1], 10, 64)\n\tif err != nil || pos < 0 {\n\t\treturn retrerr(errInvalidArgument(args[1]))\n\t}\n\n\t// >> Operation\n\n\tf, err := os.Open(s.aof.Name())\n\tif err != nil {\n\t\treturn retrerr(err)\n\t}\n\tdefer f.Close()\n\n\tn, err := f.Seek(0, 2)\n\tif err != nil {\n\t\treturn retrerr(err)\n\t}\n\n\tif n < pos {\n\t\treturn retrerr(errors.New(\n\t\t\t\"pos is too big, must be less that the aof_size of leader\"))\n\t}\n\n\t// >> Response\n\n\tvar ls liveAOFSwitches\n\tls.pos = pos\n\treturn NOMessage, ls\n}\n\nfunc (s *Server) liveAOF(pos int64, conn net.Conn, rd *PipelineReader, msg *Message) error {\n\ts.mu.RLock()\n\tf, err := os.Open(s.aof.Name())\n\ts.mu.RUnlock()\n\tif err != nil {\n\t\treturn err\n\t}\n\ts.mu.Lock()\n\ts.aofconnM[conn] = f\n\ts.mu.Unlock()\n\tdefer func() {\n\t\ts.mu.Lock()\n\t\tdelete(s.aofconnM, conn)\n\t\ts.mu.Unlock()\n\t\tconn.Close()\n\t\tf.Close()\n\t}()\n\n\tif _, err := conn.Write([]byte(\"+OK\\r\\n\")); err != nil {\n\t\treturn err\n\t}\n\tif _, err := f.Seek(pos, 0); err != nil {\n\t\treturn err\n\t}\n\tvar wg sync.WaitGroup\n\twg.Add(1)\n\tgo func() {\n\t\tdefer func() {\n\t\t\tf.Close()\n\t\t\tconn.Close()\n\t\t\twg.Done()\n\t\t}()\n\t\t// Any incoming message should end the connection\n\t\trd.ReadMessages()\n\t}()\n\t_, err = io.Copy(conn, f)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tb := make([]byte, 4096*2)\n\tfor {\n\t\tn, err := f.Read(b)\n\t\tif n > 0 {\n\t\t\tif _, err := conn.Write(b[:n]); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\t\tif err == io.EOF {\n\t\t\ts.fcond.L.Lock()\n\t\t\ts.fcond.Wait()\n\t\t\ts.fcond.L.Unlock()\n\t\t} else if err != nil {\n\t\t\tif errors.Is(err, os.ErrClosed) {\n\t\t\t\t// The live aof file can be closed when a client (follower) has\n\t\t\t\t// closed their connection or following an AOFSHRINK operation.\n\t\t\t\terr = nil\n\t\t\t}\n\t\t\treturn err\n\t\t}\n\t}\n}\n" }, { "path": "internal/server/aofmigrate.go", "content": "package server\n\nimport (\n\t\"bufio\"\n\t\"encoding/binary\"\n\t\"errors\"\n\t\"io\"\n\t\"os\"\n\t\"path\"\n\t\"time\"\n\n\t\"github.com/tidwall/resp\"\n\t\"github.com/tidwall/tile38/internal/log\"\n)\n\nvar errCorruptedAOF = errors.New(\"corrupted aof file\")\n\n// LegacyAOFReader represents the older AOF file reader.\ntype LegacyAOFReader struct {\n\tr io.Reader // reader\n\trerr error // read error\n\tchunk []byte // chunk buffer\n\tbuf []byte // main buffer\n\tl int // length of valid data in buffer\n\tp int // pointer\n}\n\n// ReadCommand reads an old command.\nfunc (rd *LegacyAOFReader) ReadCommand() ([]byte, error) {\n\tif rd.l >= 4 {\n\t\tsz1 := int(binary.LittleEndian.Uint32(rd.buf[rd.p:]))\n\t\tif rd.l >= sz1+9 {\n\t\t\t// we have enough data for a record\n\t\t\tsz2 := int(binary.LittleEndian.Uint32(rd.buf[rd.p+4+sz1:]))\n\t\t\tif sz2 != sz1 || rd.buf[rd.p+4+sz1+4] != 0 {\n\t\t\t\treturn nil, errCorruptedAOF\n\t\t\t}\n\t\t\tbuf := rd.buf[rd.p+4 : rd.p+4+sz1]\n\t\t\trd.p += sz1 + 9\n\t\t\trd.l -= sz1 + 9\n\t\t\treturn buf, nil\n\t\t}\n\t}\n\t// need more data\n\tif rd.rerr != nil {\n\t\tif rd.rerr == io.EOF {\n\t\t\trd.rerr = nil // we want to return EOF, but we want to be able to try again\n\t\t\tif rd.l != 0 {\n\t\t\t\treturn nil, io.ErrUnexpectedEOF\n\t\t\t}\n\t\t\treturn nil, io.EOF\n\t\t}\n\t\treturn nil, rd.rerr\n\t}\n\tif rd.p != 0 {\n\t\t// move p to the beginning\n\t\tcopy(rd.buf, rd.buf[rd.p:rd.p+rd.l])\n\t\trd.p = 0\n\t}\n\tvar n int\n\tn, rd.rerr = rd.r.Read(rd.chunk)\n\tif n > 0 {\n\t\tcbuf := rd.chunk[:n]\n\t\tif len(rd.buf)-rd.l < n {\n\t\t\tif len(rd.buf) == 0 {\n\t\t\t\trd.buf = make([]byte, len(cbuf))\n\t\t\t\tcopy(rd.buf, cbuf)\n\t\t\t} else {\n\t\t\t\tcopy(rd.buf[rd.l:], cbuf[:len(rd.buf)-rd.l])\n\t\t\t\trd.buf = append(rd.buf, cbuf[len(rd.buf)-rd.l:]...)\n\t\t\t}\n\t\t} else {\n\t\t\tcopy(rd.buf[rd.l:], cbuf)\n\t\t}\n\t\trd.l += n\n\t}\n\treturn rd.ReadCommand()\n}\n\n// NewLegacyAOFReader creates a new LegacyAOFReader.\nfunc NewLegacyAOFReader(r io.Reader) *LegacyAOFReader {\n\trd := &LegacyAOFReader{r: r, chunk: make([]byte, 0xFFFF)}\n\treturn rd\n}\n\nfunc (s *Server) migrateAOF() error {\n\t_, err := os.Stat(path.Join(s.dir, \"appendonly.aof\"))\n\tif err == nil {\n\t\treturn nil\n\t}\n\tif !os.IsNotExist(err) {\n\t\treturn err\n\t}\n\t_, err = os.Stat(path.Join(s.dir, \"aof\"))\n\tif err != nil {\n\t\tif os.IsNotExist(err) {\n\t\t\treturn nil\n\t\t}\n\t\treturn err\n\t}\n\tlog.Warn(\"Migrating aof to new format\")\n\tnewf, err := os.Create(path.Join(s.dir, \"migrate.aof\"))\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer newf.Close()\n\n\toldf, err := os.Open(path.Join(s.dir, \"aof\"))\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer oldf.Close()\n\tstart := time.Now()\n\tcount := 0\n\twr := bufio.NewWriter(newf)\n\trd := NewLegacyAOFReader(oldf)\n\tfor {\n\t\tcmdb, err := rd.ReadCommand()\n\t\tif err != nil {\n\t\t\tif err == io.EOF {\n\t\t\t\tbreak\n\t\t\t}\n\t\t\treturn err\n\t\t}\n\t\tline := string(cmdb)\n\t\tvar tok string\n\t\tvalues := make([]resp.Value, 0, 64)\n\t\tfor line != \"\" {\n\t\t\tline, tok = token(line)\n\t\t\tif len(tok) > 0 && tok[0] == '{' {\n\t\t\t\tif line != \"\" {\n\t\t\t\t\ttok = tok + \" \" + line\n\t\t\t\t\tline = \"\"\n\t\t\t\t}\n\t\t\t}\n\t\t\tvalues = append(values, resp.StringValue(tok))\n\t\t}\n\t\tdata, err := resp.ArrayValue(values).MarshalRESP()\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tif _, err := wr.Write(data); err != nil {\n\t\t\treturn err\n\t\t}\n\t\tif wr.Buffered() > 1024*1024 {\n\t\t\tif err := wr.Flush(); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\t\tcount++\n\t}\n\tif err := wr.Flush(); err != nil {\n\t\treturn err\n\t}\n\toldf.Close()\n\tnewf.Close()\n\tlog.Debugf(\"%d items: %.0f/sec\", count, float64(count)/(float64(time.Since(start))/float64(time.Second)))\n\treturn os.Rename(path.Join(s.dir, \"migrate.aof\"), path.Join(s.dir, \"appendonly.aof\"))\n}\n" }, { "path": "internal/server/aofshrink.go", "content": "package server\n\nimport (\n\t\"math\"\n\t\"os\"\n\t\"strconv\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/tidwall/btree\"\n\t\"github.com/tidwall/tile38/internal/collection\"\n\t\"github.com/tidwall/tile38/internal/field\"\n\t\"github.com/tidwall/tile38/internal/log\"\n\t\"github.com/tidwall/tile38/internal/object\"\n)\n\nconst maxkeys = 8\nconst maxids = 32\nconst maxchunk = 4 * 1024 * 1024\n\nfunc (s *Server) aofshrink() {\n\tstart := time.Now()\n\ts.mu.Lock()\n\tif s.aof == nil || s.shrinking {\n\t\ts.mu.Unlock()\n\t\treturn\n\t}\n\ts.shrinking = true\n\ts.shrinklog = nil\n\ts.mu.Unlock()\n\n\tdefer func() {\n\t\ts.mu.Lock()\n\t\ts.shrinking = false\n\t\ts.shrinklog = nil\n\t\ts.mu.Unlock()\n\t\tlog.Infof(\"aof shrink ended %v\", time.Since(start))\n\t}()\n\n\terr := func() error {\n\t\tf, err := os.Create(s.opts.AppendFileName + \"-shrink\")\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tdefer f.Close()\n\t\tvar aofbuf []byte\n\t\tvar values []string\n\t\tvar keys []string\n\t\tvar nextkey string\n\t\tvar keysdone bool\n\t\tfor {\n\t\t\tif len(keys) == 0 {\n\t\t\t\t// load more keys\n\t\t\t\tif keysdone {\n\t\t\t\t\tbreak\n\t\t\t\t}\n\t\t\t\tkeysdone = true\n\t\t\t\tfunc() {\n\t\t\t\t\ts.mu.Lock()\n\t\t\t\t\tdefer s.mu.Unlock()\n\t\t\t\t\ts.cols.Ascend(nextkey,\n\t\t\t\t\t\tfunc(key string, col *collection.Collection) bool {\n\t\t\t\t\t\t\tif len(keys) == maxkeys {\n\t\t\t\t\t\t\t\tkeysdone = false\n\t\t\t\t\t\t\t\tnextkey = key\n\t\t\t\t\t\t\t\treturn false\n\t\t\t\t\t\t\t}\n\t\t\t\t\t\t\tkeys = append(keys, key)\n\t\t\t\t\t\t\treturn true\n\t\t\t\t\t\t},\n\t\t\t\t\t)\n\t\t\t\t}()\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\tvar idsdone bool\n\t\t\tvar nextid string\n\t\t\tfor {\n\t\t\t\tif idsdone {\n\t\t\t\t\tkeys = keys[1:]\n\t\t\t\t\tbreak\n\t\t\t\t}\n\n\t\t\t\t// load more objects\n\t\t\t\tfunc() {\n\t\t\t\t\tidsdone = true\n\t\t\t\t\ts.mu.Lock()\n\t\t\t\t\tdefer s.mu.Unlock()\n\t\t\t\t\tcol, ok := s.cols.Get(keys[0])\n\t\t\t\t\tif !ok {\n\t\t\t\t\t\treturn\n\t\t\t\t\t}\n\t\t\t\t\tvar now = time.Now().UnixNano() // used for expiration\n\t\t\t\t\tvar count = 0 // the object count\n\t\t\t\t\tcol.ScanGreaterOrEqual(nextid, false, nil, nil,\n\t\t\t\t\t\tfunc(o *object.Object) bool {\n\t\t\t\t\t\t\tif count == maxids {\n\t\t\t\t\t\t\t\t// we reached the max number of ids for one batch\n\t\t\t\t\t\t\t\tnextid = o.ID()\n\t\t\t\t\t\t\t\tidsdone = false\n\t\t\t\t\t\t\t\treturn false\n\t\t\t\t\t\t\t}\n\t\t\t\t\t\t\t// here we fill the values array with a new command\n\t\t\t\t\t\t\tvalues = values[:0]\n\t\t\t\t\t\t\tvalues = append(values, \"set\")\n\t\t\t\t\t\t\tvalues = append(values, keys[0])\n\t\t\t\t\t\t\tvalues = append(values, o.ID())\n\t\t\t\t\t\t\to.Fields().Scan(func(f field.Field) bool {\n\t\t\t\t\t\t\t\tif !f.Value().IsZero() {\n\t\t\t\t\t\t\t\t\tvalues = append(values, \"field\")\n\t\t\t\t\t\t\t\t\tvalues = append(values, f.Name())\n\t\t\t\t\t\t\t\t\tvalues = append(values, f.Value().JSON())\n\t\t\t\t\t\t\t\t}\n\t\t\t\t\t\t\t\treturn true\n\t\t\t\t\t\t\t})\n\t\t\t\t\t\t\tif o.Expires() != 0 {\n\t\t\t\t\t\t\t\tttl := math.Floor(float64(o.Expires()-now)/float64(time.Second)*10) / 10\n\t\t\t\t\t\t\t\tif ttl < 0.1 {\n\t\t\t\t\t\t\t\t\t// always leave a little bit of ttl.\n\t\t\t\t\t\t\t\t\tttl = 0.1\n\t\t\t\t\t\t\t\t}\n\t\t\t\t\t\t\t\tvalues = append(values, \"ex\")\n\t\t\t\t\t\t\t\tvalues = append(values, strconv.FormatFloat(ttl, 'f', -1, 64))\n\t\t\t\t\t\t\t}\n\t\t\t\t\t\t\tif objIsSpatial(o.Geo()) {\n\t\t\t\t\t\t\t\tvalues = append(values, \"object\")\n\t\t\t\t\t\t\t\tvalues = append(values, string(o.Geo().AppendJSON(nil)))\n\t\t\t\t\t\t\t} else {\n\t\t\t\t\t\t\t\tvalues = append(values, \"string\")\n\t\t\t\t\t\t\t\tvalues = append(values, o.Geo().String())\n\t\t\t\t\t\t\t}\n\n\t\t\t\t\t\t\t// append the values to the aof buffer\n\t\t\t\t\t\t\taofbuf = append(aofbuf, '*')\n\t\t\t\t\t\t\taofbuf = append(aofbuf, strconv.FormatInt(int64(len(values)), 10)...)\n\t\t\t\t\t\t\taofbuf = append(aofbuf, '\\r', '\\n')\n\t\t\t\t\t\t\tfor _, value := range values {\n\t\t\t\t\t\t\t\taofbuf = append(aofbuf, '$')\n\t\t\t\t\t\t\t\taofbuf = append(aofbuf, strconv.FormatInt(int64(len(value)), 10)...)\n\t\t\t\t\t\t\t\taofbuf = append(aofbuf, '\\r', '\\n')\n\t\t\t\t\t\t\t\taofbuf = append(aofbuf, value...)\n\t\t\t\t\t\t\t\taofbuf = append(aofbuf, '\\r', '\\n')\n\t\t\t\t\t\t\t}\n\n\t\t\t\t\t\t\t// increment the object count\n\t\t\t\t\t\t\tcount++\n\t\t\t\t\t\t\treturn true\n\t\t\t\t\t\t},\n\t\t\t\t\t)\n\n\t\t\t\t}()\n\t\t\t\tif len(aofbuf) > maxchunk {\n\t\t\t\t\tif _, err := f.Write(aofbuf); err != nil {\n\t\t\t\t\t\treturn err\n\t\t\t\t\t}\n\t\t\t\t\taofbuf = aofbuf[:0]\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\n\t\t// load hooks\n\t\t// first load the names of the hooks\n\t\tvar hnames []string\n\t\tfunc() {\n\t\t\ts.mu.Lock()\n\t\t\tdefer s.mu.Unlock()\n\t\t\thnames = make([]string, 0, s.hooks.Len())\n\t\t\ts.hooks.Walk(func(v []interface{}) {\n\t\t\t\tfor _, v := range v {\n\t\t\t\t\thnames = append(hnames, v.(*Hook).Name)\n\t\t\t\t}\n\t\t\t})\n\t\t}()\n\t\tvar hookHint btree.PathHint\n\t\tfor _, name := range hnames {\n\t\t\tfunc() {\n\t\t\t\ts.mu.Lock()\n\t\t\t\tdefer s.mu.Unlock()\n\t\t\t\thook, _ := s.hooks.GetHint(&Hook{Name: name}, &hookHint).(*Hook)\n\t\t\t\tif hook == nil {\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t\thook.cond.L.Lock()\n\t\t\t\tdefer hook.cond.L.Unlock()\n\n\t\t\t\tvar values []string\n\t\t\t\tif hook.channel {\n\t\t\t\t\tvalues = append(values, \"setchan\", name)\n\t\t\t\t} else {\n\t\t\t\t\tvalues = append(values, \"sethook\", name,\n\t\t\t\t\t\tstrings.Join(hook.Endpoints, \",\"))\n\t\t\t\t}\n\t\t\t\tfor _, meta := range hook.Metas {\n\t\t\t\t\tvalues = append(values, \"meta\", meta.Name, meta.Value)\n\t\t\t\t}\n\t\t\t\tif !hook.expires.IsZero() {\n\t\t\t\t\tex := float64(time.Until(hook.expires)) / float64(time.Second)\n\t\t\t\t\tvalues = append(values, \"ex\",\n\t\t\t\t\t\tstrconv.FormatFloat(ex, 'f', 1, 64))\n\t\t\t\t}\n\t\t\t\tvalues = append(values, hook.Message.Args...)\n\t\t\t\t// append the values to the aof buffer\n\t\t\t\taofbuf = append(aofbuf, '*')\n\t\t\t\taofbuf = append(aofbuf, strconv.FormatInt(int64(len(values)), 10)...)\n\t\t\t\taofbuf = append(aofbuf, '\\r', '\\n')\n\t\t\t\tfor _, value := range values {\n\t\t\t\t\taofbuf = append(aofbuf, '$')\n\t\t\t\t\taofbuf = append(aofbuf, strconv.FormatInt(int64(len(value)), 10)...)\n\t\t\t\t\taofbuf = append(aofbuf, '\\r', '\\n')\n\t\t\t\t\taofbuf = append(aofbuf, value...)\n\t\t\t\t\taofbuf = append(aofbuf, '\\r', '\\n')\n\t\t\t\t}\n\t\t\t}()\n\t\t}\n\t\tif len(aofbuf) > 0 {\n\t\t\tif _, err := f.Write(aofbuf); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\taofbuf = aofbuf[:0]\n\t\t}\n\t\tif err := f.Sync(); err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\t// finally grab any new data that may have been written since\n\t\t// the aofshrink has started and swap out the files.\n\t\treturn func() error {\n\t\t\ts.mu.Lock()\n\t\t\tdefer s.mu.Unlock()\n\n\t\t\t// kill all followers connections and close their files. This\n\t\t\t// ensures that there is only one opened AOF at a time which is\n\t\t\t// what Windows requires in order to perform the Rename function\n\t\t\t// below.\n\t\t\tfor conn, f := range s.aofconnM {\n\t\t\t\tconn.Close()\n\t\t\t\tf.Close()\n\t\t\t}\n\n\t\t\t// send a broadcast to all sleeping followers\n\t\t\ts.fcond.Broadcast()\n\n\t\t\t// flush the aof buffer\n\t\t\ts.flushAOF(false)\n\n\t\t\taofbuf = aofbuf[:0]\n\t\t\tfor _, values := range s.shrinklog {\n\t\t\t\t// append the values to the aof buffer\n\t\t\t\taofbuf = append(aofbuf, '*')\n\t\t\t\taofbuf = append(aofbuf, strconv.FormatInt(int64(len(values)), 10)...)\n\t\t\t\taofbuf = append(aofbuf, '\\r', '\\n')\n\t\t\t\tfor _, value := range values {\n\t\t\t\t\taofbuf = append(aofbuf, '$')\n\t\t\t\t\taofbuf = append(aofbuf, strconv.FormatInt(int64(len(value)), 10)...)\n\t\t\t\t\taofbuf = append(aofbuf, '\\r', '\\n')\n\t\t\t\t\taofbuf = append(aofbuf, value...)\n\t\t\t\t\taofbuf = append(aofbuf, '\\r', '\\n')\n\t\t\t\t}\n\t\t\t}\n\t\t\tif _, err := f.Write(aofbuf); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tif err := f.Sync(); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\t// we now have a shrunken aof file that is fully in-sync with\n\t\t\t// the current dataset. let's swap out the on disk files and\n\t\t\t// point to the new file.\n\n\t\t\t// anything below this point is unrecoverable. just log and exit process\n\t\t\t// back up the live aof, just in case of fatal error\n\t\t\tif err := s.aof.Close(); err != nil {\n\t\t\t\tlog.Fatalf(\"shrink live aof close fatal operation: %v\", err)\n\t\t\t}\n\t\t\tif err := f.Close(); err != nil {\n\t\t\t\tlog.Fatalf(\"shrink new aof close fatal operation: %v\", err)\n\t\t\t}\n\t\t\tif err := os.Rename(s.opts.AppendFileName, s.opts.AppendFileName+\"-bak\"); err != nil {\n\t\t\t\tlog.Fatalf(\"shrink backup fatal operation: %v\", err)\n\t\t\t}\n\t\t\tif err := os.Rename(s.opts.AppendFileName+\"-shrink\", s.opts.AppendFileName); err != nil {\n\t\t\t\tlog.Fatalf(\"shrink rename fatal operation: %v\", err)\n\t\t\t}\n\t\t\ts.aof, err = os.OpenFile(s.opts.AppendFileName, os.O_CREATE|os.O_RDWR, 0600)\n\t\t\tif err != nil {\n\t\t\t\tlog.Fatalf(\"shrink openfile fatal operation: %v\", err)\n\t\t\t}\n\t\t\tvar n int64\n\t\t\tn, err = s.aof.Seek(0, 2)\n\t\t\tif err != nil {\n\t\t\t\tlog.Fatalf(\"shrink seek end fatal operation: %v\", err)\n\t\t\t}\n\t\t\ts.aofsz = int(n)\n\n\t\t\tos.Remove(s.opts.AppendFileName + \"-bak\") // ignore error\n\n\t\t\treturn nil\n\t\t}()\n\t}()\n\tif err != nil {\n\t\tlog.Errorf(\"aof shrink failed: %v\", err)\n\t\treturn\n\t}\n}\n" }, { "path": "internal/server/bson.go", "content": "package server\n\nimport (\n\t\"crypto/md5\"\n\t\"crypto/rand\"\n\t\"encoding/binary\"\n\t\"encoding/hex\"\n\t\"os\"\n\t\"sync/atomic\"\n\t\"time\"\n)\n\nfunc bsonID() string {\n\tb := make([]byte, 12)\n\tbinary.BigEndian.PutUint32(b, uint32(time.Now().Unix()))\n\tcopy(b[4:], bsonMachine)\n\tbinary.BigEndian.PutUint32(b[8:], atomic.AddUint32(&bsonCounter, 1))\n\tbinary.BigEndian.PutUint16(b[7:], bsonProcess)\n\treturn hex.EncodeToString(b)\n}\n\nvar (\n\tbsonProcess = uint16(os.Getpid())\n\tbsonMachine = func() []byte {\n\t\thost, _ := os.Hostname()\n\t\tb := make([]byte, 3)\n\t\tMust(rand.Read(b))\n\t\thost = Default(host, string(b))\n\t\thw := md5.New()\n\t\thw.Write([]byte(host))\n\t\treturn hw.Sum(nil)[:3]\n\t}()\n\tbsonCounter = func() uint32 {\n\t\tb := make([]byte, 4)\n\t\tMust(rand.Read(b))\n\t\treturn binary.BigEndian.Uint32(b)\n\t}()\n)\n" }, { "path": "internal/server/bson_test.go", "content": "package server\n\nimport \"testing\"\n\nfunc TestBSON(t *testing.T) {\n\tid := bsonID()\n\tif len(id) != 24 {\n\t\tt.Fail()\n\t}\n}\n" }, { "path": "internal/server/checksum.go", "content": "package server\n\nimport (\n\t\"crypto/md5\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"os\"\n\t\"time\"\n\n\t\"github.com/tidwall/resp\"\n\t\"github.com/tidwall/tile38/internal/log\"\n)\n\n// checksum performs a simple md5 checksum on the aof file\nfunc (s *Server) checksum(pos, size int64) (sum string, err error) {\n\tif pos+size > int64(s.aofsz) {\n\t\treturn \"\", io.EOF\n\t}\n\tvar f *os.File\n\tf, err = os.Open(s.aof.Name())\n\tif err != nil {\n\t\treturn\n\t}\n\tdefer f.Close()\n\tsumr := md5.New()\n\terr = func() error {\n\t\tif size == 0 {\n\t\t\tn, err := f.Seek(int64(s.aofsz), 0)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tif pos >= n {\n\t\t\t\treturn io.EOF\n\t\t\t}\n\t\t\treturn nil\n\t\t}\n\t\t_, err = f.Seek(pos, 0)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\t_, err = io.CopyN(sumr, f, size)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\treturn nil\n\t}()\n\tif err != nil {\n\t\tif err == io.ErrUnexpectedEOF {\n\t\t\terr = io.EOF\n\t\t}\n\t\treturn \"\", err\n\t}\n\treturn fmt.Sprintf(\"%x\", sumr.Sum(nil)), nil\n}\n\nfunc connAOFMD5(conn *RESPConn, pos, size int64) (sum string, err error) {\n\tv, err := conn.Do(\"aofmd5\", pos, size)\n\tif err != nil {\n\t\treturn \"\", err\n\t}\n\tif v.Error() != nil {\n\t\terrmsg := v.Error().Error()\n\t\tif errmsg == \"ERR EOF\" || errmsg == \"EOF\" {\n\t\t\treturn \"\", io.EOF\n\t\t}\n\t\treturn \"\", v.Error()\n\t}\n\tsum = v.String()\n\tif len(sum) != 32 {\n\t\treturn \"\", errors.New(\"checksum not ok\")\n\t}\n\treturn sum, nil\n}\n\nfunc (s *Server) matchChecksums(conn *RESPConn, pos, size int64) (match bool, err error) {\n\tsum, err := s.checksum(pos, size)\n\tif err != nil {\n\t\tif err == io.EOF {\n\t\t\treturn false, nil\n\t\t}\n\t\treturn false, err\n\t}\n\tcsum, err := connAOFMD5(conn, pos, size)\n\tif err != nil {\n\t\tif err == io.EOF {\n\t\t\treturn false, nil\n\t\t}\n\t\treturn false, err\n\t}\n\treturn csum == sum, nil\n}\n\n// getEndOfLastValuePositionInFile is a very slow operation because it reads the file\n// backwards on byte at a time. Eek. It seek+read, seek+read, etc.\nfunc getEndOfLastValuePositionInFile(fname string, startPos int64) (int64, error) {\n\tpos := startPos\n\tf, err := os.Open(fname)\n\tif err != nil {\n\t\treturn 0, err\n\t}\n\tdefer f.Close()\n\treadByte := func() (byte, error) {\n\t\tif pos <= 0 {\n\t\t\treturn 0, io.EOF\n\t\t}\n\t\tpos--\n\t\tif _, err := f.Seek(pos, 0); err != nil {\n\t\t\treturn 0, err\n\t\t}\n\t\tb := make([]byte, 1)\n\t\tif n, err := f.Read(b); err != nil {\n\t\t\treturn 0, err\n\t\t} else if n != 1 {\n\t\t\treturn 0, errors.New(\"invalid read\")\n\t\t}\n\t\treturn b[0], nil\n\t}\n\tfor {\n\t\tc, err := readByte()\n\t\tif err != nil {\n\t\t\treturn 0, err\n\t\t}\n\t\tif c == '*' {\n\t\t\tif _, err := f.Seek(pos, 0); err != nil {\n\t\t\t\treturn 0, err\n\t\t\t}\n\t\t\trd := resp.NewReader(f)\n\t\t\t_, telnet, n, err := rd.ReadMultiBulk()\n\t\t\tif err != nil || telnet {\n\t\t\t\tcontinue // keep reading backwards\n\t\t\t}\n\t\t\treturn pos + int64(n), nil\n\t\t}\n\t}\n}\n\n// followCheckSome is not a full checksum. It just \"checks some\" data.\n// We will do some various checksums on the leader until we find the correct position to start at.\nfunc (s *Server) followCheckSome(addr string, followc int, auth string,\n) (pos int64, err error) {\n\tif s.opts.ShowDebugMessages {\n\t\tlog.Debug(\"follow:\", addr, \":check some\")\n\t}\n\ts.mu.Lock()\n\tdefer s.mu.Unlock()\n\tif int(s.followc.Load()) != followc {\n\t\treturn 0, errNoLongerFollowing\n\t}\n\tif s.aofsz < checksumsz {\n\t\treturn 0, nil\n\t}\n\n\tconn, err := DialTimeout(addr, time.Second*2)\n\tif err != nil {\n\t\treturn 0, err\n\t}\n\tdefer conn.Close()\n\n\tif auth != \"\" {\n\t\tif err := s.followDoLeaderAuth(conn, auth); err != nil {\n\t\t\treturn 0, err\n\t\t}\n\t}\n\n\tmin := int64(0)\n\tmax := int64(s.aofsz) - checksumsz\n\tlimit := int64(s.aofsz)\n\tmatch, err := s.matchChecksums(conn, min, checksumsz)\n\tif err != nil {\n\t\treturn 0, err\n\t}\n\n\tif match {\n\t\tmin += checksumsz // bump up the min\n\t\tfor {\n\t\t\tif max < min || max+checksumsz > limit {\n\t\t\t\tpos = min\n\t\t\t\tbreak\n\t\t\t} else {\n\t\t\t\tmatch, err = s.matchChecksums(conn, max, checksumsz)\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn 0, err\n\t\t\t\t}\n\t\t\t\tif match {\n\t\t\t\t\tmin = max + checksumsz\n\t\t\t\t} else {\n\t\t\t\t\tlimit = max\n\t\t\t\t}\n\t\t\t\tmax = (limit-min)/2 - checksumsz/2 + min // multiply\n\t\t\t}\n\t\t}\n\t}\n\tfullpos := pos\n\tfname := s.aof.Name()\n\tif pos == 0 {\n\t\ts.aof.Close()\n\t\ts.aof, err = os.Create(fname)\n\t\tif err != nil {\n\t\t\tlog.Fatalf(\"could not recreate aof, possible data loss. %s\", err.Error())\n\t\t\treturn 0, err\n\t\t}\n\t\treturn 0, nil\n\t}\n\n\t// we want to truncate at a command location\n\t// search for nearest command\n\tpos, err = getEndOfLastValuePositionInFile(s.aof.Name(), fullpos)\n\tif err != nil {\n\t\treturn 0, err\n\t}\n\tif pos == fullpos {\n\t\tif s.opts.ShowDebugMessages {\n\t\t\tlog.Debug(\"follow: aof fully intact\")\n\t\t}\n\t\treturn pos, nil\n\t}\n\tlog.Warnf(\"truncating aof to %d\", pos)\n\t// any error below are fatal.\n\ts.aof.Close()\n\tif err := os.Truncate(fname, pos); err != nil {\n\t\tlog.Fatalf(\"could not truncate aof, possible data loss. %s\", err.Error())\n\t\treturn 0, err\n\t}\n\ts.aof, err = os.OpenFile(fname, os.O_CREATE|os.O_RDWR, 0600)\n\tif err != nil {\n\t\tlog.Fatalf(\"could not create aof, possible data loss. %s\", err.Error())\n\t\treturn 0, err\n\t}\n\t// reset the entire system.\n\tlog.Infof(\"reloading aof commands\")\n\ts.reset()\n\tif err := s.loadAOF(); err != nil {\n\t\tlog.Fatalf(\"could not reload aof, possible data loss. %s\", err.Error())\n\t\treturn 0, err\n\t}\n\tif int64(s.aofsz) != pos {\n\t\tlog.Fatalf(\"aof size mismatch during reload, possible data loss.\")\n\t\treturn 0, errors.New(\"?\")\n\t}\n\treturn pos, nil\n}\n" }, { "path": "internal/server/client.go", "content": "package server\n\nimport (\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"io\"\n\t\"sort\"\n\t\"strings\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/tidwall/resp\"\n)\n\n// Client is an remote connection into to Tile38\ntype Client struct {\n\tid int // unique id\n\treplPort int // the known replication port for follower connections\n\treplAddr string // the known replication addr for follower connections\n\tauthd bool // client has been authenticated\n\toutputType Type // Null, JSON, or RESP\n\tstrictRESP bool // client is in strict RESP mode\n\tremoteAddr string // original remote address\n\tin InputStream // input stream\n\tpr PipelineReader // command reader\n\tout []byte // output write buffer\n\n\tgoLiveErr error // error type used for going line\n\tgoLiveMsg *Message // last message for go live\n\n\tmu sync.Mutex // guard\n\tconn io.ReadWriteCloser // out-of-loop connection.\n\tname string // optional defined name\n\topened time.Time // when the client was created/opened, unix nano\n\tlast time.Time // last client request/response, unix nano\n\n\tcloser io.Closer // used to close the connection\n}\n\n// Write ...\nfunc (client *Client) Write(b []byte) (n int, err error) {\n\tclient.out = append(client.out, b...)\n\treturn len(b), nil\n}\n\n// CLIENT (LIST | KILL | GETNAME | SETNAME)\nfunc (s *Server) cmdCLIENT(msg *Message, client *Client) (resp.Value, error) {\n\tstart := time.Now()\n\n\targs := msg.Args\n\tif len(args) == 1 {\n\t\treturn retrerr(errInvalidNumberOfArguments)\n\t}\n\n\tswitch strings.ToLower(args[1]) {\n\tcase \"list\":\n\t\tif len(args) != 2 {\n\t\t\treturn retrerr(errInvalidNumberOfArguments)\n\t\t}\n\t\tvar list []*Client\n\t\ts.connsmu.RLock()\n\t\tfor _, cc := range s.conns {\n\t\t\tlist = append(list, cc)\n\t\t}\n\t\ts.connsmu.RUnlock()\n\t\tsort.Slice(list, func(i, j int) bool {\n\t\t\treturn list[i].id < list[j].id\n\t\t})\n\t\tnow := time.Now()\n\t\tvar buf []byte\n\t\tfor _, client := range list {\n\t\t\tclient.mu.Lock()\n\t\t\tbuf = append(buf,\n\t\t\t\tfmt.Sprintf(\"id=%d addr=%s name=%s age=%d idle=%d\\n\",\n\t\t\t\t\tclient.id,\n\t\t\t\t\tclient.remoteAddr,\n\t\t\t\t\tclient.name,\n\t\t\t\t\tnow.Sub(client.opened)/time.Second,\n\t\t\t\t\tnow.Sub(client.last)/time.Second,\n\t\t\t\t)...,\n\t\t\t)\n\t\t\tclient.mu.Unlock()\n\t\t}\n\t\tif msg.OutputType == JSON {\n\t\t\t// Create a map of all key/value info fields\n\t\t\tvar cmap []map[string]interface{}\n\t\t\tclients := strings.Split(string(buf), \"\\n\")\n\t\t\tfor _, client := range clients {\n\t\t\t\tclient = strings.TrimSpace(client)\n\t\t\t\tm := make(map[string]interface{})\n\t\t\t\tvar hasFields bool\n\t\t\t\tfor _, kv := range strings.Split(client, \" \") {\n\t\t\t\t\tkv = strings.TrimSpace(kv)\n\t\t\t\t\tif split := strings.SplitN(kv, \"=\", 2); len(split) == 2 {\n\t\t\t\t\t\thasFields = true\n\t\t\t\t\t\tm[split[0]] = tryParseType(split[1])\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t\tif hasFields {\n\t\t\t\t\tcmap = append(cmap, m)\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tdata, _ := json.Marshal(cmap)\n\t\t\treturn resp.StringValue(`{\"ok\":true,\"list\":` + string(data) +\n\t\t\t\t`,\"elapsed\":\"` + time.Since(start).String() + \"\\\"}\"), nil\n\t\t}\n\t\treturn resp.BytesValue(buf), nil\n\tcase \"getname\":\n\t\tif len(args) != 2 {\n\t\t\treturn retrerr(errInvalidNumberOfArguments)\n\t\t}\n\t\tclient.mu.Lock()\n\t\tname := client.name\n\t\tclient.mu.Unlock()\n\t\tif msg.OutputType == JSON {\n\t\t\treturn resp.StringValue(`{\"ok\":true,\"name\":` + jsonString(name) +\n\t\t\t\t`,\"elapsed\":\"` + time.Since(start).String() + \"\\\"}\"), nil\n\t\t}\n\t\treturn resp.StringValue(name), nil\n\tcase \"setname\":\n\t\tif len(args) != 3 {\n\t\t\treturn retrerr(errInvalidNumberOfArguments)\n\t\t}\n\t\tname := msg.Args[2]\n\t\tfor i := 0; i < len(name); i++ {\n\t\t\tif name[i] < '!' || name[i] > '~' {\n\t\t\t\treturn retrerr(clientErrorf(\n\t\t\t\t\t\"Client names cannot contain spaces, newlines or special characters.\",\n\t\t\t\t))\n\t\t\t}\n\t\t}\n\t\tclient.mu.Lock()\n\t\tclient.name = name\n\t\tclient.mu.Unlock()\n\t\tif msg.OutputType == JSON {\n\t\t\treturn resp.StringValue(`{\"ok\":true,\"elapsed\":\"` +\n\t\t\t\ttime.Since(start).String() + \"\\\"}\"), nil\n\t\t}\n\t\treturn resp.SimpleStringValue(\"OK\"), nil\n\tcase \"kill\":\n\t\tif len(args) < 3 {\n\t\t\treturn retrerr(errInvalidNumberOfArguments)\n\t\t}\n\t\tvar useAddr bool\n\t\tvar addr string\n\t\tvar useID bool\n\t\tvar id string\n\t\tfor i := 2; i < len(args); i++ {\n\t\t\tif useAddr || useID {\n\t\t\t\treturn retrerr(errInvalidNumberOfArguments)\n\t\t\t}\n\t\t\targ := args[i]\n\t\t\tif strings.Contains(arg, \":\") {\n\t\t\t\taddr = arg\n\t\t\t\tuseAddr = true\n\t\t\t} else {\n\t\t\t\tswitch strings.ToLower(arg) {\n\t\t\t\tcase \"addr\":\n\t\t\t\t\ti++\n\t\t\t\t\tif i == len(args) {\n\t\t\t\t\t\treturn retrerr(errInvalidNumberOfArguments)\n\t\t\t\t\t}\n\t\t\t\t\taddr = args[i]\n\t\t\t\t\tuseAddr = true\n\t\t\t\tcase \"id\":\n\t\t\t\t\ti++\n\t\t\t\t\tif i == len(args) {\n\t\t\t\t\t\treturn retrerr(errInvalidNumberOfArguments)\n\t\t\t\t\t}\n\t\t\t\t\tid = args[i]\n\t\t\t\t\tuseID = true\n\t\t\t\tdefault:\n\t\t\t\t\treturn retrerr(clientErrorf(\"No such client\"))\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t\tvar closing []io.Closer\n\t\ts.connsmu.RLock()\n\t\tfor _, cc := range s.conns {\n\t\t\tif useID && fmt.Sprintf(\"%d\", cc.id) == id {\n\t\t\t\tif cc.closer != nil {\n\t\t\t\t\tclosing = append(closing, cc.closer)\n\t\t\t\t}\n\t\t\t} else if useAddr {\n\t\t\t\tif cc.remoteAddr == addr {\n\t\t\t\t\tif cc.closer != nil {\n\t\t\t\t\t\tclosing = append(closing, cc.closer)\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t\ts.connsmu.RUnlock()\n\t\tif len(closing) == 0 {\n\t\t\treturn retrerr(clientErrorf(\"No such client\"))\n\t\t}\n\t\t// go func() {\n\t\t// close the connections behind the scene\n\t\tfor _, closer := range closing {\n\t\t\tcloser.Close()\n\t\t}\n\t\t// }()\n\t\tif msg.OutputType == JSON {\n\t\t\treturn resp.StringValue(`{\"ok\":true,\"elapsed\":\"` +\n\t\t\t\ttime.Since(start).String() + \"\\\"}\"), nil\n\t\t}\n\t\treturn resp.SimpleStringValue(\"OK\"), nil\n\tdefault:\n\t\treturn retrerr(clientErrorf(\n\t\t\t\"Syntax error, try CLIENT (LIST | KILL | GETNAME | SETNAME)\",\n\t\t))\n\t}\n}\n" }, { "path": "internal/server/config.go", "content": "package server\n\nimport (\n\t\"encoding/json\"\n\t\"os\"\n\t\"strconv\"\n\t\"strings\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/tidwall/gjson\"\n\t\"github.com/tidwall/resp\"\n\t\"github.com/tidwall/tile38/internal/glob\"\n)\n\nconst (\n\tdefaultKeepAlive = 300 // seconds\n\tdefaultProtectedMode = \"yes\"\n)\n\n// Config keys\nconst (\n\tFollowHost = \"follow_host\"\n\tFollowPort = \"follow_port\"\n\tFollowID = \"follow_id\"\n\tFollowPos = \"follow_pos\"\n\tReplicaPriority = \"replica-priority\"\n\tServerID = \"server_id\"\n\tReadOnly = \"read_only\"\n\tRequirePass = \"requirepass\"\n\tLeaderAuth = \"leaderauth\"\n\tProtectedMode = \"protected-mode\"\n\tMaxMemory = \"maxmemory\"\n\tAutoGC = \"autogc\"\n\tKeepAlive = \"keepalive\"\n\tLogConfig = \"logconfig\"\n\tAnnounceIP = \"replica_announce_ip\"\n\tAnnouncePort = \"replica_announce_port\"\n)\n\nvar validProperties = []string{RequirePass, LeaderAuth, ProtectedMode, MaxMemory, AutoGC, KeepAlive, LogConfig, ReplicaPriority, AnnouncePort, AnnounceIP}\n\n// Config is a tile38 config\ntype Config struct {\n\tpath string\n\n\tmu sync.RWMutex\n\n\t_followHost string\n\t_followPort int64\n\t_followID string\n\t_followPos int64\n\t_replicaPriority int64\n\t_serverID string\n\t_readOnly bool\n\n\t_requirePassP string\n\t_requirePass string\n\t_leaderAuthP string\n\t_leaderAuth string\n\t_protectedModeP string\n\t_protectedMode string\n\t_maxMemoryP string\n\t_maxMemory int64\n\t_autoGCP string\n\t_autoGC uint64\n\t_keepAliveP string\n\t_keepAlive int64\n\t_logConfigP interface{}\n\t_logConfig string\n\t_announceIPP string\n\t_announceIP string\n\t_announcePortP string\n\t_announcePort int64\n}\n\nfunc loadConfig(path string) (*Config, error) {\n\tvar json string\n\tdata, err := os.ReadFile(path)\n\tif err != nil {\n\t\tif !os.IsNotExist(err) {\n\t\t\treturn nil, err\n\t\t}\n\t} else {\n\t\tjson = string(data)\n\t}\n\n\tconfig := &Config{\n\t\tpath: path,\n\t\t_followHost: gjson.Get(json, FollowHost).String(),\n\t\t_followPort: gjson.Get(json, FollowPort).Int(),\n\t\t_followID: gjson.Get(json, FollowID).String(),\n\t\t_followPos: gjson.Get(json, FollowPos).Int(),\n\t\t_serverID: gjson.Get(json, ServerID).String(),\n\t\t_readOnly: gjson.Get(json, ReadOnly).Bool(),\n\t\t_requirePassP: gjson.Get(json, RequirePass).String(),\n\t\t_leaderAuthP: gjson.Get(json, LeaderAuth).String(),\n\t\t_protectedModeP: gjson.Get(json, ProtectedMode).String(),\n\t\t_maxMemoryP: gjson.Get(json, MaxMemory).String(),\n\t\t_autoGCP: gjson.Get(json, AutoGC).String(),\n\t\t_keepAliveP: gjson.Get(json, KeepAlive).String(),\n\t\t_logConfig: gjson.Get(json, LogConfig).String(),\n\t\t_announceIPP: gjson.Get(json, AnnounceIP).String(),\n\t\t_announcePortP: gjson.Get(json, AnnouncePort).String(),\n\t}\n\n\tif config._serverID == \"\" {\n\t\tconfig._serverID = randomKey(16)\n\t}\n\n\t// Need to be sure we look for existence vs not zero because zero is an intentional setting\n\t// anything less than zero will be considered default and will result in no slave_priority\n\t// being output when INFO is called.\n\tif gjson.Get(json, ReplicaPriority).Exists() {\n\t\tconfig._replicaPriority = gjson.Get(json, ReplicaPriority).Int()\n\t} else {\n\t\tconfig._replicaPriority = -1\n\t}\n\n\t// load properties\n\tif err := config.setProperty(RequirePass, config._requirePassP, true); err != nil {\n\t\treturn nil, err\n\t}\n\tif err := config.setProperty(LeaderAuth, config._leaderAuthP, true); err != nil {\n\t\treturn nil, err\n\t}\n\tif err := config.setProperty(ProtectedMode, config._protectedModeP, true); err != nil {\n\t\treturn nil, err\n\t}\n\tif err := config.setProperty(MaxMemory, config._maxMemoryP, true); err != nil {\n\t\treturn nil, err\n\t}\n\tif err := config.setProperty(AutoGC, config._autoGCP, true); err != nil {\n\t\treturn nil, err\n\t}\n\tif err := config.setProperty(KeepAlive, config._keepAliveP, true); err != nil {\n\t\treturn nil, err\n\t}\n\tif err := config.setProperty(LogConfig, config._logConfig, true); err != nil {\n\t\treturn nil, err\n\t}\n\tif err := config.setProperty(AnnounceIP, config._announceIPP, true); err != nil {\n\t\treturn nil, err\n\t}\n\tif err := config.setProperty(AnnouncePort, config._announcePortP, true); err != nil {\n\t\treturn nil, err\n\t}\n\tconfig.write(false)\n\treturn config, nil\n}\n\nfunc (config *Config) write(writeProperties bool) {\n\tconfig.mu.Lock()\n\tdefer config.mu.Unlock()\n\n\tif writeProperties {\n\t\t// save properties\n\t\tconfig._requirePassP = config._requirePass\n\t\tconfig._leaderAuthP = config._leaderAuth\n\t\tif config._protectedMode == defaultProtectedMode {\n\t\t\tconfig._protectedModeP = \"\"\n\t\t} else {\n\t\t\tconfig._protectedModeP = config._protectedMode\n\t\t}\n\t\tconfig._maxMemoryP = formatMemSize(config._maxMemory)\n\t\tif config._autoGC == 0 {\n\t\t\tconfig._autoGCP = \"\"\n\t\t} else {\n\t\t\tconfig._autoGCP = strconv.FormatUint(config._autoGC, 10)\n\t\t}\n\t\tif config._keepAlive == defaultKeepAlive {\n\t\t\tconfig._keepAliveP = \"\"\n\t\t} else {\n\t\t\tconfig._keepAliveP = strconv.FormatUint(uint64(config._keepAlive), 10)\n\t\t}\n\t\tif config._logConfig != \"\" {\n\t\t\tconfig._logConfigP = config._logConfig\n\t\t}\n\t\tif config._announceIP != \"\" {\n\t\t\tconfig._announceIPP = config._announceIP\n\t\t}\n\t\tif config._announcePort == 0 {\n\t\t\tconfig._announcePortP = \"\"\n\t\t} else {\n\t\t\tconfig._announcePortP = strconv.FormatUint(uint64(config._announcePort), 10)\n\t\t}\n\t}\n\n\tm := make(map[string]interface{})\n\tif config._followHost != \"\" {\n\t\tm[FollowHost] = config._followHost\n\t}\n\tif config._followPort != 0 {\n\t\tm[FollowPort] = config._followPort\n\t}\n\tif config._followID != \"\" {\n\t\tm[FollowID] = config._followID\n\t}\n\tif config._followPos != 0 {\n\t\tm[FollowPos] = config._followPos\n\t}\n\tif config._replicaPriority >= 0 {\n\t\tm[ReplicaPriority] = config._replicaPriority\n\t}\n\tif config._serverID != \"\" {\n\t\tm[ServerID] = config._serverID\n\t}\n\tif config._readOnly {\n\t\tm[ReadOnly] = config._readOnly\n\t}\n\tif config._requirePassP != \"\" {\n\t\tm[RequirePass] = config._requirePassP\n\t}\n\tif config._leaderAuthP != \"\" {\n\t\tm[LeaderAuth] = config._leaderAuthP\n\t}\n\tif config._protectedModeP != \"\" {\n\t\tm[ProtectedMode] = config._protectedModeP\n\t}\n\tif config._maxMemoryP != \"\" {\n\t\tm[MaxMemory] = config._maxMemoryP\n\t}\n\tif config._autoGCP != \"\" {\n\t\tm[AutoGC] = config._autoGCP\n\t}\n\tif config._keepAliveP != \"\" {\n\t\tm[KeepAlive] = config._keepAliveP\n\t}\n\tif config._logConfigP != \"\" {\n\t\tvar lcfg map[string]interface{}\n\t\tjson.Unmarshal([]byte(config._logConfig), &lcfg)\n\t\tif len(lcfg) > 0 {\n\t\t\tm[LogConfig] = lcfg\n\t\t}\n\t}\n\tif config._announceIPP != \"\" {\n\t\tm[AnnounceIP] = config._announceIPP\n\t}\n\tif config._announcePortP != \"\" {\n\t\tm[AnnouncePort] = config._announcePortP\n\t}\n\tdata, err := json.MarshalIndent(m, \"\", \"\\t\")\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\tdata = append(data, '\\n')\n\terr = os.WriteFile(config.path, data, 0600)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n}\n\nfunc parseMemSize(s string) (bytes int64, ok bool) {\n\tif s == \"\" {\n\t\treturn 0, true\n\t}\n\ts = strings.ToLower(s)\n\tvar n uint64\n\tvar sz int64\n\tvar err error\n\tif strings.HasSuffix(s, \"gb\") {\n\t\tn, err = strconv.ParseUint(s[:len(s)-2], 10, 64)\n\t\tsz = int64(n * 1024 * 1024 * 1024)\n\t} else if strings.HasSuffix(s, \"mb\") {\n\t\tn, err = strconv.ParseUint(s[:len(s)-2], 10, 64)\n\t\tsz = int64(n * 1024 * 1024)\n\t} else if strings.HasSuffix(s, \"kb\") {\n\t\tn, err = strconv.ParseUint(s[:len(s)-2], 10, 64)\n\t\tsz = int64(n * 1024)\n\t} else {\n\t\tn, err = strconv.ParseUint(s, 10, 64)\n\t\tsz = int64(n)\n\t}\n\tif err != nil {\n\t\treturn 0, false\n\t}\n\treturn sz, true\n}\n\nfunc formatMemSize(sz int64) string {\n\tif sz <= 0 {\n\t\treturn \"\"\n\t}\n\tif sz < 1024 {\n\t\treturn strconv.FormatInt(sz, 10)\n\t}\n\tsz /= 1024\n\tif sz < 1024 {\n\t\treturn strconv.FormatInt(sz, 10) + \"kb\"\n\t}\n\tsz /= 1024\n\tif sz < 1024 {\n\t\treturn strconv.FormatInt(sz, 10) + \"mb\"\n\t}\n\tsz /= 1024\n\treturn strconv.FormatInt(sz, 10) + \"gb\"\n}\n\nfunc (config *Config) setProperty(name, value string, fromLoad bool) error {\n\tconfig.mu.Lock()\n\tdefer config.mu.Unlock()\n\tvar invalid bool\n\tswitch name {\n\tdefault:\n\t\treturn clientErrorf(\"Unsupported CONFIG parameter: %s\", name)\n\tcase RequirePass:\n\t\tconfig._requirePass = value\n\tcase LeaderAuth:\n\t\tconfig._leaderAuth = value\n\tcase AutoGC:\n\t\tif value == \"\" {\n\t\t\tconfig._autoGC = 0\n\t\t} else {\n\t\t\tgc, err := strconv.ParseUint(value, 10, 64)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tconfig._autoGC = gc\n\t\t}\n\tcase MaxMemory:\n\t\tsz, ok := parseMemSize(value)\n\t\tif !ok {\n\t\t\treturn clientErrorf(\"Invalid argument '%s' for CONFIG SET '%s'\", value, name)\n\t\t}\n\t\tconfig._maxMemory = sz\n\tcase ProtectedMode:\n\t\tswitch strings.ToLower(value) {\n\t\tcase \"\":\n\t\t\tif fromLoad {\n\t\t\t\tconfig._protectedMode = defaultProtectedMode\n\t\t\t} else {\n\t\t\t\tinvalid = true\n\t\t\t}\n\t\tcase \"yes\", \"no\":\n\t\t\tconfig._protectedMode = strings.ToLower(value)\n\t\tdefault:\n\t\t\tinvalid = true\n\t\t}\n\tcase KeepAlive:\n\t\tif value == \"\" {\n\t\t\tconfig._keepAlive = defaultKeepAlive\n\t\t} else {\n\t\t\tkeepalive, err := strconv.ParseUint(value, 10, 64)\n\t\t\tif err != nil {\n\t\t\t\tinvalid = true\n\t\t\t} else {\n\t\t\t\tconfig._keepAlive = int64(keepalive)\n\t\t\t}\n\t\t}\n\tcase LogConfig:\n\t\tif value == \"\" {\n\t\t\tconfig._logConfig = \"\"\n\t\t} else {\n\t\t\tconfig._logConfig = value\n\t\t}\n\tcase ReplicaPriority:\n\t\treplicaPriority, err := strconv.ParseInt(value, 10, 64)\n\t\tif err != nil || replicaPriority < 0 {\n\t\t\tinvalid = true\n\t\t} else {\n\t\t\tconfig._replicaPriority = replicaPriority\n\t\t}\n\tcase AnnounceIP:\n\t\tconfig._announceIP = value\n\tcase AnnouncePort:\n\t\tif value == \"\" {\n\t\t\tconfig._announcePort = 0\n\t\t} else {\n\t\t\tannouncePort, err := strconv.ParseUint(value, 10, 64)\n\t\t\tif err != nil {\n\t\t\t\tinvalid = true\n\t\t\t} else {\n\t\t\t\tconfig._announcePort = int64(announcePort)\n\t\t\t}\n\t\t}\n\t}\n\n\tif invalid {\n\t\treturn clientErrorf(\"Invalid argument '%s' for CONFIG SET '%s'\", value, name)\n\t}\n\treturn nil\n}\n\nfunc (config *Config) getProperties(pattern string) map[string]interface{} {\n\tm := make(map[string]interface{})\n\tfor _, name := range validProperties {\n\t\tmatched, _ := glob.Match(pattern, name)\n\t\tif matched {\n\t\t\tm[name] = config.getProperty(name)\n\t\t}\n\t}\n\treturn m\n}\n\nfunc (config *Config) getProperty(name string) string {\n\tconfig.mu.RLock()\n\tdefer config.mu.RUnlock()\n\tswitch name {\n\tdefault:\n\t\treturn \"\"\n\tcase AutoGC:\n\t\treturn strconv.FormatUint(config._autoGC, 10)\n\tcase RequirePass:\n\t\treturn config._requirePass\n\tcase LeaderAuth:\n\t\treturn config._leaderAuth\n\tcase ProtectedMode:\n\t\treturn config._protectedMode\n\tcase MaxMemory:\n\t\treturn formatMemSize(config._maxMemory)\n\tcase KeepAlive:\n\t\treturn strconv.FormatUint(uint64(config._keepAlive), 10)\n\tcase LogConfig:\n\t\treturn config._logConfig\n\tcase ReplicaPriority:\n\t\tif config._replicaPriority < 0 {\n\t\t\treturn \"\"\n\t\t} else {\n\t\t\treturn strconv.FormatUint(uint64(config._replicaPriority), 10)\n\t\t}\n\tcase AnnounceIP:\n\t\treturn config._announceIP\n\tcase AnnouncePort:\n\t\treturn strconv.FormatUint(uint64(config._announcePort), 10)\n\t}\n}\n\nfunc (s *Server) cmdConfigGet(msg *Message) (res resp.Value, err error) {\n\tstart := time.Now()\n\tvs := msg.Args[1:]\n\tvar ok bool\n\tvar name string\n\n\tif vs, name, ok = tokenval(vs); !ok {\n\t\treturn NOMessage, errInvalidNumberOfArguments\n\t}\n\tif len(vs) != 0 {\n\t\treturn NOMessage, errInvalidNumberOfArguments\n\t}\n\tm := s.config.getProperties(name)\n\tswitch msg.OutputType {\n\tcase JSON:\n\t\tdata, err := json.Marshal(m)\n\t\tif err != nil {\n\t\t\treturn NOMessage, err\n\t\t}\n\t\tres = resp.StringValue(`{\"ok\":true,\"properties\":` + string(data) + `,\"elapsed\":\"` + time.Since(start).String() + \"\\\"}\")\n\tcase RESP:\n\t\tvals := respValuesSimpleMap(m)\n\t\tres = resp.ArrayValue(vals)\n\t}\n\treturn\n}\nfunc (s *Server) cmdConfigSet(msg *Message) (res resp.Value, err error) {\n\tstart := time.Now()\n\tvs := msg.Args[1:]\n\tvar ok bool\n\tvar name string\n\n\tif vs, name, ok = tokenval(vs); !ok {\n\t\treturn NOMessage, errInvalidNumberOfArguments\n\t}\n\tvar value string\n\tif vs, value, ok = tokenval(vs); !ok {\n\t\tif strings.ToLower(name) != RequirePass {\n\t\t\treturn NOMessage, errInvalidNumberOfArguments\n\t\t}\n\t}\n\tif len(vs) != 0 {\n\t\treturn NOMessage, errInvalidNumberOfArguments\n\t}\n\tif err := s.config.setProperty(name, value, false); err != nil {\n\t\treturn NOMessage, err\n\t}\n\tif name == MaxMemory {\n\t\ts.checkOutOfMemory()\n\t}\n\treturn OKMessage(msg, start), nil\n}\nfunc (s *Server) cmdConfigRewrite(msg *Message) (res resp.Value, err error) {\n\tstart := time.Now()\n\tvs := msg.Args[1:]\n\n\tif len(vs) != 0 {\n\t\treturn NOMessage, errInvalidNumberOfArguments\n\t}\n\ts.config.write(true)\n\treturn OKMessage(msg, start), nil\n}\n\nfunc (config *Config) followHost() string {\n\tconfig.mu.RLock()\n\tv := config._followHost\n\tconfig.mu.RUnlock()\n\treturn v\n}\nfunc (config *Config) followPort() int {\n\tconfig.mu.RLock()\n\tv := config._followPort\n\tconfig.mu.RUnlock()\n\treturn int(v)\n}\nfunc (config *Config) replicaPriority() int {\n\tconfig.mu.RLock()\n\tv := config._replicaPriority\n\tconfig.mu.RUnlock()\n\treturn int(v)\n}\nfunc (config *Config) serverID() string {\n\tconfig.mu.RLock()\n\tv := config._serverID\n\tconfig.mu.RUnlock()\n\treturn v\n}\nfunc (config *Config) readOnly() bool {\n\tconfig.mu.RLock()\n\tv := config._readOnly\n\tconfig.mu.RUnlock()\n\treturn v\n}\nfunc (config *Config) requirePass() string {\n\tconfig.mu.RLock()\n\tv := config._requirePass\n\tconfig.mu.RUnlock()\n\treturn v\n}\nfunc (config *Config) leaderAuth() string {\n\tconfig.mu.RLock()\n\tv := config._leaderAuth\n\tconfig.mu.RUnlock()\n\treturn v\n}\nfunc (config *Config) protectedMode() string {\n\tconfig.mu.RLock()\n\tv := config._protectedMode\n\tconfig.mu.RUnlock()\n\treturn v\n}\nfunc (config *Config) maxMemory() int {\n\tconfig.mu.RLock()\n\tv := config._maxMemory\n\tconfig.mu.RUnlock()\n\treturn int(v)\n}\nfunc (config *Config) autoGC() uint64 {\n\tconfig.mu.RLock()\n\tv := config._autoGC\n\tconfig.mu.RUnlock()\n\treturn v\n}\nfunc (config *Config) keepAlive() int64 {\n\tconfig.mu.RLock()\n\tv := config._keepAlive\n\tconfig.mu.RUnlock()\n\treturn v\n}\nfunc (config *Config) announceIP() string {\n\tconfig.mu.RLock()\n\tv := config._announceIP\n\tconfig.mu.RUnlock()\n\treturn v\n}\nfunc (config *Config) announcePort() int {\n\tconfig.mu.RLock()\n\tv := config._announcePort\n\tconfig.mu.RUnlock()\n\treturn int(v)\n}\nfunc (config *Config) setFollowHost(v string) {\n\tconfig.mu.Lock()\n\tconfig._followHost = v\n\tconfig.mu.Unlock()\n}\nfunc (config *Config) setFollowPort(v int) {\n\tconfig.mu.Lock()\n\tconfig._followPort = int64(v)\n\tconfig.mu.Unlock()\n}\nfunc (config *Config) setReadOnly(v bool) {\n\tconfig.mu.Lock()\n\tconfig._readOnly = v\n\tconfig.mu.Unlock()\n}\n" }, { "path": "internal/server/crud.go", "content": "package server\n\nimport (\n\t\"bytes\"\n\t\"math\"\n\t\"strconv\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/mmcloughlin/geohash\"\n\t\"github.com/tidwall/geojson\"\n\t\"github.com/tidwall/geojson/geometry\"\n\t\"github.com/tidwall/resp\"\n\t\"github.com/tidwall/tile38/internal/collection\"\n\t\"github.com/tidwall/tile38/internal/field\"\n\t\"github.com/tidwall/tile38/internal/glob\"\n\t\"github.com/tidwall/tile38/internal/object\"\n)\n\n// BOUNDS key\nfunc (s *Server) cmdBOUNDS(msg *Message) (resp.Value, error) {\n\tstart := time.Now()\n\n\t// >> Args\n\n\targs := msg.Args\n\tif len(args) != 2 {\n\t\treturn retrerr(errInvalidNumberOfArguments)\n\t}\n\tkey := args[1]\n\n\t// >> Operation\n\n\tcol, _ := s.cols.Get(key)\n\tif col == nil {\n\t\tif msg.OutputType == RESP {\n\t\t\treturn resp.NullValue(), nil\n\t\t}\n\t\treturn retrerr(errKeyNotFound)\n\t}\n\n\t// >> Response\n\n\tvals := make([]resp.Value, 0, 2)\n\tvar buf bytes.Buffer\n\tif msg.OutputType == JSON {\n\t\tbuf.WriteString(`{\"ok\":true`)\n\t}\n\tminX, minY, maxX, maxY := col.Bounds()\n\n\tbbox := geojson.NewRect(geometry.Rect{\n\t\tMin: geometry.Point{X: minX, Y: minY},\n\t\tMax: geometry.Point{X: maxX, Y: maxY},\n\t})\n\tif msg.OutputType == JSON {\n\t\tbuf.WriteString(`,\"bounds\":`)\n\t\tbuf.WriteString(string(bbox.AppendJSON(nil)))\n\t\tbuf.WriteString(`,\"elapsed\":\"` + time.Since(start).String() + \"\\\"}\")\n\t\treturn resp.StringValue(buf.String()), nil\n\t}\n\n\t// RESP\n\tvals = append(vals, resp.ArrayValue([]resp.Value{\n\t\tresp.ArrayValue([]resp.Value{\n\t\t\tresp.FloatValue(minX),\n\t\t\tresp.FloatValue(minY),\n\t\t}),\n\t\tresp.ArrayValue([]resp.Value{\n\t\t\tresp.FloatValue(maxX),\n\t\t\tresp.FloatValue(maxY),\n\t\t}),\n\t}))\n\treturn vals[0], nil\n}\n\n// TYPE key\n// undocumented return \"none\" or \"hash\"\nfunc (s *Server) cmdTYPE(msg *Message) (resp.Value, error) {\n\tstart := time.Now()\n\n\t// >> Args\n\n\targs := msg.Args\n\tif len(args) != 2 {\n\t\treturn retrerr(errInvalidNumberOfArguments)\n\t}\n\tkey := args[1]\n\n\t// >> Operation\n\n\tcol, _ := s.cols.Get(key)\n\tif col == nil {\n\t\tif msg.OutputType == RESP {\n\t\t\treturn resp.SimpleStringValue(\"none\"), nil\n\t\t}\n\t\treturn retrerr(errKeyNotFound)\n\t}\n\n\t// >> Response\n\n\ttyp := \"hash\"\n\n\tif msg.OutputType == JSON {\n\t\treturn resp.StringValue(`{\"ok\":true,\"type\":` + jsonString(typ) +\n\t\t\t`,\"elapsed\":\"` + time.Since(start).String() + \"\\\"}\"), nil\n\t}\n\treturn resp.SimpleStringValue(typ), nil\n}\n\n// GET key id [WITHFIELDS] [OBJECT|POINT|BOUNDS|(HASH geohash)]\nfunc (s *Server) cmdGET(msg *Message) (resp.Value, error) {\n\tstart := time.Now()\n\n\t// >> Args\n\n\targs := msg.Args\n\n\tif len(args) < 3 {\n\t\treturn retrerr(errInvalidNumberOfArguments)\n\t}\n\tkey, id := args[1], args[2]\n\n\twithfields := false\n\tkind := \"object\"\n\tvar precision int64\n\tfor i := 3; i < len(args); i++ {\n\t\tswitch strings.ToLower(args[i]) {\n\t\tcase \"withfields\":\n\t\t\twithfields = true\n\t\tcase \"object\":\n\t\t\tkind = \"object\"\n\t\tcase \"point\":\n\t\t\tkind = \"point\"\n\t\tcase \"bounds\":\n\t\t\tkind = \"bounds\"\n\t\tcase \"hash\":\n\t\t\tkind = \"hash\"\n\t\t\ti++\n\t\t\tif i == len(args) {\n\t\t\t\treturn retrerr(errInvalidNumberOfArguments)\n\t\t\t}\n\t\t\tvar err error\n\t\t\tprecision, err = strconv.ParseInt(args[i], 10, 64)\n\t\t\tif err != nil || precision < 1 || precision > 12 {\n\t\t\t\treturn retrerr(errInvalidArgument(args[i]))\n\t\t\t}\n\t\tdefault:\n\t\t\treturn retrerr(errInvalidNumberOfArguments)\n\t\t}\n\t}\n\n\t// >> Operation\n\n\tcol, _ := s.cols.Get(key)\n\tif col == nil {\n\t\tif msg.OutputType == RESP {\n\t\t\treturn resp.NullValue(), nil\n\t\t}\n\t\treturn retrerr(errKeyNotFound)\n\t}\n\to := col.Get(id)\n\tif o == nil {\n\t\tif msg.OutputType == RESP {\n\t\t\treturn resp.NullValue(), nil\n\t\t}\n\t\treturn retrerr(errIDNotFound)\n\t}\n\n\t// >> Response\n\n\toval := buildObjectResponse(msg, o, start, kind, precision, withfields, msg.OutputType == JSON)\n\n\treturn oval, nil\n}\n\nfunc buildObjectResponse(msg *Message, o *object.Object, start time.Time, kind string, precision int64, withfields, json bool) resp.Value {\n\tvals := make([]resp.Value, 0, 2)\n\tvar buf bytes.Buffer\n\tif msg.OutputType == JSON {\n\t\tbuf.WriteString(`{\"ok\":true`)\n\t}\n\tswitch kind {\n\tcase \"object\":\n\t\tif msg.OutputType == JSON {\n\t\t\tbuf.WriteString(`,\"object\":`)\n\t\t\tbuf.WriteString(string(o.Geo().AppendJSON(nil)))\n\t\t} else {\n\t\t\tvals = append(vals, resp.StringValue(o.Geo().String()))\n\t\t}\n\tcase \"point\":\n\t\tif msg.OutputType == JSON {\n\t\t\tbuf.WriteString(`,\"point\":`)\n\t\t\tbuf.Write(appendJSONSimplePoint(nil, o.Geo()))\n\t\t} else {\n\t\t\tpoint := o.Geo().Center()\n\t\t\tz := extractZCoordinate(o.Geo())\n\t\t\tif z != 0 {\n\t\t\t\tvals = append(vals, resp.ArrayValue([]resp.Value{\n\t\t\t\t\tresp.StringValue(strconv.FormatFloat(point.Y, 'f', -1, 64)),\n\t\t\t\t\tresp.StringValue(strconv.FormatFloat(point.X, 'f', -1, 64)),\n\t\t\t\t\tresp.StringValue(strconv.FormatFloat(z, 'f', -1, 64)),\n\t\t\t\t}))\n\t\t\t} else {\n\t\t\t\tvals = append(vals, resp.ArrayValue([]resp.Value{\n\t\t\t\t\tresp.StringValue(strconv.FormatFloat(point.Y, 'f', -1, 64)),\n\t\t\t\t\tresp.StringValue(strconv.FormatFloat(point.X, 'f', -1, 64)),\n\t\t\t\t}))\n\t\t\t}\n\t\t}\n\tcase \"hash\":\n\t\tif msg.OutputType == JSON {\n\t\t\tbuf.WriteString(`,\"hash\":`)\n\t\t}\n\t\tcenter := o.Geo().Center()\n\t\tp := geohash.EncodeWithPrecision(center.Y, center.X, uint(precision))\n\t\tif msg.OutputType == JSON {\n\t\t\tbuf.WriteString(`\"` + p + `\"`)\n\t\t} else {\n\t\t\tvals = append(vals, resp.StringValue(p))\n\t\t}\n\tcase \"bounds\":\n\t\tif msg.OutputType == JSON {\n\t\t\tbuf.WriteString(`,\"bounds\":`)\n\t\t\tbuf.Write(appendJSONSimpleBounds(nil, o.Geo()))\n\t\t} else {\n\t\t\tbbox := o.Rect()\n\t\t\tvals = append(vals, resp.ArrayValue([]resp.Value{\n\t\t\t\tresp.ArrayValue([]resp.Value{\n\t\t\t\t\tresp.FloatValue(bbox.Min.Y),\n\t\t\t\t\tresp.FloatValue(bbox.Min.X),\n\t\t\t\t}),\n\t\t\t\tresp.ArrayValue([]resp.Value{\n\t\t\t\t\tresp.FloatValue(bbox.Max.Y),\n\t\t\t\t\tresp.FloatValue(bbox.Max.X),\n\t\t\t\t}),\n\t\t\t}))\n\t\t}\n\t}\n\n\tif withfields {\n\t\tnfields := o.Fields().Len()\n\t\tif nfields > 0 {\n\t\t\tfvals := make([]resp.Value, 0, nfields*2)\n\t\t\tif msg.OutputType == JSON {\n\t\t\t\tbuf.WriteString(`,\"fields\":{`)\n\t\t\t}\n\t\t\tvar i int\n\t\t\to.Fields().Scan(func(f field.Field) bool {\n\t\t\t\tif json {\n\t\t\t\t\tif i > 0 {\n\t\t\t\t\t\tbuf.WriteString(`,`)\n\t\t\t\t\t}\n\t\t\t\t\tbuf.WriteString(jsonString(f.Name()) + \":\" +\n\t\t\t\t\t\tf.Value().JSON())\n\t\t\t\t} else {\n\t\t\t\t\tfvals = append(fvals, resp.StringValue(f.Name()),\n\t\t\t\t\t\tresp.StringValue(f.Value().Data()))\n\t\t\t\t}\n\t\t\t\ti++\n\t\t\t\treturn true\n\t\t\t})\n\t\t\tif json {\n\t\t\t\tbuf.WriteString(`}`)\n\t\t\t} else {\n\t\t\t\tvals = append(vals, resp.ArrayValue(fvals))\n\t\t\t}\n\t\t}\n\t}\n\tif json {\n\t\tbuf.WriteString(`,\"elapsed\":\"` + time.Since(start).String() + \"\\\"}\")\n\t\treturn resp.StringValue(buf.String())\n\t}\n\tvar oval resp.Value\n\tif withfields {\n\t\toval = resp.ArrayValue(vals)\n\t} else {\n\t\toval = vals[0]\n\t}\n\treturn oval\n}\n\n// DEL key id [ERRON404]\nfunc (s *Server) cmdDEL(msg *Message) (resp.Value, commandDetails, error) {\n\tstart := time.Now()\n\n\t// >> Args\n\n\targs := msg.Args\n\tif len(args) < 3 {\n\t\treturn retwerr(errInvalidNumberOfArguments)\n\t}\n\tkey := args[1]\n\tid := args[2]\n\terron404 := false\n\tfor i := 3; i < len(args); i++ {\n\t\tswitch strings.ToLower(args[i]) {\n\t\tcase \"erron404\":\n\t\t\terron404 = true\n\t\tdefault:\n\t\t\treturn retwerr(errInvalidArgument(args[i]))\n\t\t}\n\t}\n\n\t// >> Operation\n\n\tupdated := false\n\tvar old *object.Object\n\tcol, _ := s.cols.Get(key)\n\tif col != nil {\n\t\told = col.Delete(id)\n\t\tif old != nil {\n\t\t\tif col.Count() == 0 {\n\t\t\t\ts.cols.Delete(key)\n\t\t\t}\n\t\t\tupdated = true\n\t\t} else if erron404 {\n\t\t\treturn retwerr(errIDNotFound)\n\t\t}\n\t} else if erron404 {\n\t\treturn retwerr(errKeyNotFound)\n\t}\n\ts.groupDisconnectObject(key, id)\n\n\t// >> Response\n\n\tvar d commandDetails\n\n\td.command = \"del\"\n\td.key = key\n\td.obj = old\n\td.updated = updated\n\td.timestamp = time.Now()\n\n\tvar res resp.Value\n\n\tswitch msg.OutputType {\n\tcase JSON:\n\t\tres = resp.StringValue(`{\"ok\":true,\"elapsed\":\"` +\n\t\t\ttime.Since(start).String() + \"\\\"}\")\n\tcase RESP:\n\t\tif d.updated {\n\t\t\tres = resp.IntegerValue(1)\n\t\t} else {\n\t\t\tres = resp.IntegerValue(0)\n\t\t}\n\t}\n\treturn res, d, nil\n}\n\n// PDEL key pattern\nfunc (s *Server) cmdPDEL(msg *Message) (resp.Value, commandDetails, error) {\n\tstart := time.Now()\n\n\t// >> Args\n\n\targs := msg.Args\n\tif len(args) != 3 {\n\t\treturn retwerr(errInvalidNumberOfArguments)\n\t}\n\tkey := args[1]\n\tpattern := args[2]\n\n\t// >> Operation\n\n\tnow := time.Now()\n\tvar children []*commandDetails\n\tcol, _ := s.cols.Get(key)\n\tif col != nil {\n\t\tg := glob.Parse(pattern, false)\n\t\tvar ids []string\n\t\titer := func(o *object.Object) bool {\n\t\t\tif match, _ := glob.Match(pattern, o.ID()); match {\n\t\t\t\tids = append(ids, o.ID())\n\t\t\t}\n\t\t\treturn true\n\t\t}\n\t\tif g.Limits[0] == \"\" && g.Limits[1] == \"\" {\n\t\t\tcol.Scan(false, nil, msg.Deadline, iter)\n\t\t} else {\n\t\t\tcol.ScanRange(g.Limits[0], g.Limits[1],\n\t\t\t\tfalse, nil, msg.Deadline, iter)\n\t\t}\n\t\tfor _, id := range ids {\n\t\t\tobj := col.Delete(id)\n\t\t\tchildren = append(children, &commandDetails{\n\t\t\t\tcommand: \"del\",\n\t\t\t\tupdated: true,\n\t\t\t\ttimestamp: now,\n\t\t\t\tkey: key,\n\t\t\t\tobj: obj,\n\t\t\t})\n\t\t\ts.groupDisconnectObject(key, id)\n\t\t}\n\t\tif col.Count() == 0 {\n\t\t\ts.cols.Delete(key)\n\t\t}\n\t}\n\n\t// >> Response\n\n\tvar d commandDetails\n\tvar res resp.Value\n\n\td.command = \"pdel\"\n\td.children = children\n\td.key = key\n\td.pattern = pattern\n\td.updated = len(d.children) > 0\n\td.timestamp = now\n\td.parent = true\n\tswitch msg.OutputType {\n\tcase JSON:\n\t\tres = resp.StringValue(`{\"ok\":true,\"elapsed\":\"` +\n\t\t\ttime.Since(start).String() + \"\\\"}\")\n\tcase RESP:\n\t\tres = resp.IntegerValue(len(d.children))\n\t}\n\treturn res, d, nil\n}\n\nfunc (s *Server) cmdDROPop(key string) *collection.Collection {\n\tcol, _ := s.cols.Get(key)\n\tif col != nil {\n\t\ts.cols.Delete(key)\n\t}\n\ts.groupDisconnectCollection(key)\n\treturn col\n}\n\n// DROP key\nfunc (s *Server) cmdDROP(msg *Message) (resp.Value, commandDetails, error) {\n\tstart := time.Now()\n\n\t// >> Args\n\n\targs := msg.Args\n\tif len(args) != 2 {\n\t\treturn retwerr(errInvalidNumberOfArguments)\n\t}\n\tkey := args[1]\n\n\t// >> Operation\n\tcol := s.cmdDROPop(key)\n\n\t// >> Response\n\n\tvar res resp.Value\n\tvar d commandDetails\n\td.key = key\n\td.updated = col != nil\n\td.command = \"drop\"\n\td.timestamp = time.Now()\n\tswitch msg.OutputType {\n\tcase JSON:\n\t\tres = resp.StringValue(`{\"ok\":true,\"elapsed\":\"` +\n\t\t\ttime.Since(start).String() + \"\\\"}\")\n\tcase RESP:\n\t\tif d.updated {\n\t\t\tres = resp.IntegerValue(1)\n\t\t} else {\n\t\t\tres = resp.IntegerValue(0)\n\t\t}\n\t}\n\treturn res, d, nil\n}\n\n// RENAME key newkey\n// RENAMENX key newkey\nfunc (s *Server) cmdRENAME(msg *Message) (resp.Value, commandDetails, error) {\n\tstart := time.Now()\n\n\t// >> Args\n\n\targs := msg.Args\n\tif len(args) != 3 {\n\t\treturn retwerr(errInvalidNumberOfArguments)\n\t}\n\tnx := strings.ToLower(args[0]) == \"renamenx\"\n\tkey := args[1]\n\tnewKey := args[2]\n\n\t// >> Operation\n\n\tcol, _ := s.cols.Get(key)\n\tif col == nil {\n\t\treturn retwerr(errKeyNotFound)\n\t}\n\tvar hasHook, hasChannel bool\n\ts.hooks.Ascend(nil, func(v interface{}) bool {\n\t\th := v.(*Hook)\n\t\tif h.Key == key || h.Key == newKey {\n\t\t\tif h.channel {\n\t\t\t\thasChannel = true\n\t\t\t} else {\n\t\t\t\thasHook = true\n\t\t\t}\n\t\t}\n\t\treturn true\n\t})\n\tif hasHook {\n\t\treturn retwerr(errKeyHasHooksSet)\n\t}\n\tif hasChannel {\n\t\treturn retwerr(errKeyHasChannelsSet)\n\t}\n\tvar updated bool\n\tnewCol, _ := s.cols.Get(newKey)\n\tif newCol == nil {\n\t\tupdated = true\n\t} else if !nx {\n\t\ts.cols.Delete(newKey)\n\t\tupdated = true\n\t}\n\tif updated {\n\t\ts.cols.Delete(key)\n\t\ts.cols.Set(newKey, col)\n\t}\n\n\t// >> Response\n\n\tvar d commandDetails\n\tvar res resp.Value\n\n\td.command = \"rename\"\n\td.key = key\n\td.newKey = newKey\n\td.updated = updated\n\td.timestamp = time.Now()\n\n\tswitch msg.OutputType {\n\tcase JSON:\n\t\tres = resp.StringValue(`{\"ok\":true,\"elapsed\":\"` +\n\t\t\ttime.Since(start).String() + \"\\\"}\")\n\tcase RESP:\n\t\tif !nx {\n\t\t\tres = resp.SimpleStringValue(\"OK\")\n\t\t} else if d.updated {\n\t\t\tres = resp.IntegerValue(1)\n\t\t} else {\n\t\t\tres = resp.IntegerValue(0)\n\t\t}\n\t}\n\treturn res, d, nil\n}\n\n// FLUSHDB\nfunc (s *Server) cmdFLUSHDB(msg *Message) (resp.Value, commandDetails, error) {\n\tstart := time.Now()\n\n\t// >> Args\n\n\targs := msg.Args\n\n\tif len(args) != 1 {\n\t\treturn retwerr(errInvalidNumberOfArguments)\n\t}\n\n\t// >> Operation\n\n\t// clear the entire database\n\n\t// drop each collection\n\tkeys := s.cols.Keys()\n\tfor _, key := range keys {\n\t\ts.cmdDROPop(key)\n\t}\n\n\t// delete all channels\n\tvar names []string\n\ts.hooks.Ascend(nil, func(item any) bool {\n\t\thook := item.(*Hook)\n\t\tif hook.channel {\n\t\t\tnames = append(names, hook.Name)\n\t\t}\n\t\treturn true\n\t})\n\tfor _, name := range names {\n\t\ts.cmdDELHOOKop(name, true)\n\t}\n\n\t// delete all hooks\n\tnames = names[:0]\n\ts.hooks.Ascend(nil, func(item any) bool {\n\t\thook := item.(*Hook)\n\t\tif !hook.channel {\n\t\t\tnames = append(names, hook.Name)\n\t\t}\n\t\treturn true\n\t})\n\tfor _, name := range names {\n\t\ts.cmdDELHOOKop(name, false)\n\t}\n\n\ts.cols.Clear()\n\ts.groupHooks.Clear()\n\ts.groupObjects.Clear()\n\ts.hookExpires.Clear()\n\ts.hooks.Clear()\n\ts.hooksOut.Clear()\n\ts.hookTree.Clear()\n\ts.hookCross.Clear()\n\n\t// >> Response\n\n\tvar d commandDetails\n\td.command = \"flushdb\"\n\td.updated = true\n\td.timestamp = time.Now()\n\n\tvar res resp.Value\n\tif msg.OutputType == JSON {\n\t\tres = resp.StringValue(`{\"ok\":true,\"elapsed\":\"` +\n\t\t\ttime.Since(start).String() + \"\\\"}\")\n\t} else {\n\t\tres = resp.SimpleStringValue(\"OK\")\n\t}\n\treturn res, d, nil\n}\n\n// SET key id [FIELD name value ...] [EX seconds] [NX|XX]\n// (OBJECT geojson)|(POINT lat lon z)|(BOUNDS minlat minlon maxlat maxlon)|\n// (HASH geohash)|(STRING value)\nfunc (s *Server) cmdSET(msg *Message) (resp.Value, commandDetails, error) {\n\tstart := time.Now()\n\tif s.config.maxMemory() > 0 && s.outOfMemory.Load() {\n\t\treturn retwerr(errOOM)\n\t}\n\n\t// >> Args\n\n\tvar key string\n\tvar id string\n\tvar fields []field.Field\n\tvar ex int64\n\tvar xx bool\n\tvar nx bool\n\tvar ret bool\n\tvar withfields bool\n\tkind := \"object\"\n\tvar precision int64\n\tvar oobj geojson.Object\n\n\targs := msg.Args\n\tif len(args) < 3 {\n\t\treturn retwerr(errInvalidNumberOfArguments)\n\t}\n\n\tkey, id = args[1], args[2]\n\n\tfor i := 3; i < len(args); i++ {\n\t\tswitch strings.ToLower(args[i]) {\n\t\tcase \"field\":\n\t\t\tif i+2 >= len(args) {\n\t\t\t\treturn retwerr(errInvalidNumberOfArguments)\n\t\t\t}\n\t\t\tfkey := args[i+1]\n\t\t\tfval := args[i+2]\n\t\t\ti += 2\n\t\t\tif isReservedFieldName(fkey) {\n\t\t\t\treturn retwerr(errInvalidArgument(fkey))\n\t\t\t}\n\t\t\tfields = append(fields, field.Make(fkey, fval))\n\t\tcase \"ex\":\n\t\t\tif i+1 >= len(args) {\n\t\t\t\treturn retwerr(errInvalidNumberOfArguments)\n\t\t\t}\n\t\t\texval := args[i+1]\n\t\t\ti += 1\n\t\t\tx, err := strconv.ParseFloat(exval, 64)\n\t\t\tif err != nil {\n\t\t\t\treturn retwerr(errInvalidArgument(exval))\n\t\t\t}\n\t\t\tex = time.Now().UnixNano() + int64(float64(time.Second)*x)\n\t\tcase \"nx\":\n\t\t\tif xx {\n\t\t\t\treturn retwerr(errInvalidArgument(args[i]))\n\t\t\t}\n\t\t\tnx = true\n\t\tcase \"xx\":\n\t\t\tif nx {\n\t\t\t\treturn retwerr(errInvalidArgument(args[i]))\n\t\t\t}\n\t\t\txx = true\n\t\tcase \"return\":\n\t\t\tif ret {\n\t\t\t\treturn retwerr(errInvalidArgument(args[i]))\n\t\t\t}\n\t\t\tret = true\n\n\t\t\tfor j := i; j < i+3; j++ {\n\t\t\t\tif j >= len(args) {\n\t\t\t\t\tbreak\n\t\t\t\t}\n\t\t\t\tswitch strings.ToLower(args[j]) {\n\t\t\t\tcase \"withfields\":\n\t\t\t\t\twithfields = true\n\t\t\t\t\ti += 1\n\t\t\t\tcase \"object\":\n\t\t\t\t\tkind = \"object\"\n\t\t\t\t\ti += 1\n\t\t\t\tcase \"point\":\n\t\t\t\t\tkind = \"point\"\n\t\t\t\t\ti += 1\n\t\t\t\tcase \"bounds\":\n\t\t\t\t\tkind = \"bounds\"\n\t\t\t\t\ti += 1\n\t\t\t\tcase \"hash\":\n\t\t\t\t\tkind = \"hash\"\n\t\t\t\t\tj++\n\t\t\t\t\tif j == len(args) {\n\t\t\t\t\t\treturn retwerr(errInvalidNumberOfArguments)\n\t\t\t\t\t}\n\t\t\t\t\tvar err error\n\t\t\t\t\tprecision, err = strconv.ParseInt(args[j], 10, 64)\n\t\t\t\t\tif err != nil || precision < 1 || precision > 12 {\n\t\t\t\t\t\treturn retwerr(errInvalidArgument(args[j]))\n\t\t\t\t\t}\n\t\t\t\t\ti += 2\n\t\t\t\t}\n\t\t\t}\n\t\tcase \"string\":\n\t\t\tif i+1 >= len(args) {\n\t\t\t\treturn retwerr(errInvalidNumberOfArguments)\n\t\t\t}\n\t\t\tstr := args[i+1]\n\t\t\ti += 1\n\t\t\toobj = collection.String(str)\n\t\tcase \"point\":\n\t\t\tif i+2 >= len(args) {\n\t\t\t\treturn retwerr(errInvalidNumberOfArguments)\n\t\t\t}\n\t\t\tslat := args[i+1]\n\t\t\tslon := args[i+2]\n\t\t\ti += 2\n\t\t\tvar z float64\n\t\t\tvar hasZ bool\n\t\t\tif i+1 < len(args) {\n\t\t\t\t// probe for possible z coordinate\n\t\t\t\tvar err error\n\t\t\t\tz, err = strconv.ParseFloat(args[i+1], 64)\n\t\t\t\tif err == nil {\n\t\t\t\t\thasZ = true\n\t\t\t\t\ti++\n\t\t\t\t}\n\t\t\t}\n\t\t\ty, err := strconv.ParseFloat(slat, 64)\n\t\t\tif err != nil {\n\t\t\t\treturn retwerr(errInvalidArgument(slat))\n\t\t\t}\n\t\t\tx, err := strconv.ParseFloat(slon, 64)\n\t\t\tif err != nil {\n\t\t\t\treturn retwerr(errInvalidArgument(slon))\n\t\t\t}\n\t\t\tif !hasZ {\n\t\t\t\toobj = geojson.NewPoint(geometry.Point{X: x, Y: y})\n\t\t\t} else {\n\t\t\t\toobj = geojson.NewPointZ(geometry.Point{X: x, Y: y}, z)\n\t\t\t}\n\t\tcase \"bounds\":\n\t\t\tif i+4 >= len(args) {\n\t\t\t\treturn retwerr(errInvalidNumberOfArguments)\n\t\t\t}\n\t\t\tvar vals [4]float64\n\t\t\tfor j := 0; j < 4; j++ {\n\t\t\t\tvar err error\n\t\t\t\tvals[j], err = strconv.ParseFloat(args[i+1+j], 64)\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn retwerr(errInvalidArgument(args[i+1+j]))\n\t\t\t\t}\n\t\t\t}\n\t\t\ti += 4\n\t\t\toobj = geojson.NewRect(geometry.Rect{\n\t\t\t\tMin: geometry.Point{X: vals[1], Y: vals[0]},\n\t\t\t\tMax: geometry.Point{X: vals[3], Y: vals[2]},\n\t\t\t})\n\t\tcase \"hash\":\n\t\t\tif i+1 >= len(args) {\n\t\t\t\treturn retwerr(errInvalidNumberOfArguments)\n\t\t\t}\n\t\t\tshash := args[i+1]\n\t\t\ti += 1\n\t\t\tlat, lon := geohash.Decode(shash)\n\t\t\toobj = geojson.NewPoint(geometry.Point{X: lon, Y: lat})\n\t\tcase \"object\":\n\t\t\tif i+1 >= len(args) {\n\t\t\t\treturn retwerr(errInvalidNumberOfArguments)\n\t\t\t}\n\t\t\tjson := args[i+1]\n\t\t\ti += 1\n\t\t\tvar err error\n\t\t\toobj, err = geojson.Parse(json, &s.geomParseOpts)\n\t\t\tif err != nil {\n\t\t\t\treturn retwerr(err)\n\t\t\t}\n\t\tdefault:\n\t\t\treturn retwerr(errInvalidArgument(args[i]))\n\t\t}\n\t}\n\tif oobj == nil {\n\t\treturn retwerr(errInvalidNumberOfArguments)\n\t}\n\n\t// >> Operation\n\n\tnada := func() (resp.Value, commandDetails, error) {\n\t\t// exclude operation due to 'xx' or 'nx' match\n\t\tif msg.OutputType == JSON {\n\t\t\tif nx {\n\t\t\t\treturn retwerr(errIDAlreadyExists)\n\t\t\t} else {\n\t\t\t\treturn retwerr(errIDNotFound)\n\t\t\t}\n\t\t}\n\t\treturn resp.NullValue(), commandDetails{}, nil\n\t}\n\n\tcol, ok := s.cols.Get(key)\n\tif !ok {\n\t\tif xx {\n\t\t\treturn nada()\n\t\t}\n\t\tcol = collection.New()\n\t\ts.cols.Set(key, col)\n\t}\n\n\tif xx || nx {\n\t\tif col.Get(id) == nil {\n\t\t\tif xx {\n\t\t\t\treturn nada()\n\t\t\t}\n\t\t} else {\n\t\t\tif nx {\n\t\t\t\treturn nada()\n\t\t\t}\n\t\t}\n\t}\n\n\tvar flist field.List\n\tif old := col.Get(id); old != nil {\n\t\tflist = old.Fields()\n\t}\n\tfor _, f := range fields {\n\t\tflist = flist.Set(f)\n\t}\n\tobj := object.New(id, oobj, ex, flist)\n\told := col.Set(obj)\n\n\t// >> Response\n\n\tvar d commandDetails\n\td.command = \"set\"\n\td.key = key\n\td.obj = obj\n\td.old = old\n\td.updated = true // perhaps we should do a diff on the previous object?\n\td.timestamp = time.Now()\n\n\tif ret {\n\t\tres := buildObjectResponse(msg, obj, start, kind, precision, withfields, msg.OutputType == JSON)\n\t\treturn res, d, nil\n\t}\n\n\tvar res resp.Value\n\tswitch msg.OutputType {\n\tdefault:\n\tcase JSON:\n\t\tres = resp.StringValue(`{\"ok\":true,\"elapsed\":\"` + time.Since(start).String() + \"\\\"}\")\n\tcase RESP:\n\t\tres = resp.SimpleStringValue(\"OK\")\n\t}\n\treturn res, d, nil\n}\n\nfunc retwerr(err error) (resp.Value, commandDetails, error) {\n\treturn resp.Value{}, commandDetails{}, err\n}\nfunc retrerr(err error) (resp.Value, error) {\n\treturn resp.Value{}, err\n}\n\n// FSET key id [XX] field value [field value...]\nfunc (s *Server) cmdFSET(msg *Message) (resp.Value, commandDetails, error) {\n\tstart := time.Now()\n\tif s.config.maxMemory() > 0 && s.outOfMemory.Load() {\n\t\treturn retwerr(errOOM)\n\t}\n\n\t// >> Args\n\n\tvar id string\n\tvar key string\n\tvar xx bool\n\tvar ret bool\n\tvar withfields bool\n\tkind := \"object\"\n\tvar precision int64\n\n\tvar fields []field.Field // raw fields\n\n\targs := msg.Args\n\tif len(args) < 5 {\n\t\treturn retwerr(errInvalidNumberOfArguments)\n\t}\n\tkey, id = args[1], args[2]\n\tfor i := 3; i < len(args); i++ {\n\t\targ := args[i]\n\t\tswitch strings.ToLower(arg) {\n\t\tcase \"xx\":\n\t\t\txx = true\n\t\tcase \"return\":\n\t\t\tif ret {\n\t\t\t\treturn retwerr(errInvalidArgument(args[i]))\n\t\t\t}\n\t\t\tret = true\n\n\t\t\tfor j := i; j < i+3; j++ {\n\t\t\t\tif j >= len(args) {\n\t\t\t\t\tbreak\n\t\t\t\t}\n\t\t\t\tswitch strings.ToLower(args[j]) {\n\t\t\t\tcase \"withfields\":\n\t\t\t\t\twithfields = true\n\t\t\t\t\ti += 1\n\t\t\t\tcase \"object\":\n\t\t\t\t\tkind = \"object\"\n\t\t\t\t\ti += 1\n\t\t\t\tcase \"point\":\n\t\t\t\t\tkind = \"point\"\n\t\t\t\t\ti += 1\n\t\t\t\tcase \"bounds\":\n\t\t\t\t\tkind = \"bounds\"\n\t\t\t\t\ti += 1\n\t\t\t\tcase \"hash\":\n\t\t\t\t\tkind = \"hash\"\n\t\t\t\t\tj++\n\t\t\t\t\tif j == len(args) {\n\t\t\t\t\t\treturn retwerr(errInvalidNumberOfArguments)\n\t\t\t\t\t}\n\t\t\t\t\tvar err error\n\t\t\t\t\tprecision, err = strconv.ParseInt(args[j], 10, 64)\n\t\t\t\t\tif err != nil || precision < 1 || precision > 12 {\n\t\t\t\t\t\treturn retwerr(errInvalidArgument(args[j]))\n\t\t\t\t\t}\n\t\t\t\t\ti += 2\n\t\t\t\t}\n\t\t\t}\n\n\t\tdefault:\n\t\t\tfkey := arg\n\t\t\ti++\n\t\t\tif i == len(args) {\n\t\t\t\treturn retwerr(errInvalidNumberOfArguments)\n\t\t\t}\n\t\t\tif isReservedFieldName(fkey) {\n\t\t\t\treturn retwerr(errInvalidArgument(fkey))\n\t\t\t}\n\t\t\tfval := args[i]\n\t\t\tfields = append(fields, field.Make(fkey, fval))\n\t\t}\n\t}\n\n\t// >> Operation\n\n\tvar d commandDetails\n\tvar updateCount int\n\n\tcol, ok := s.cols.Get(key)\n\tif !ok {\n\t\treturn retwerr(errKeyNotFound)\n\t}\n\to := col.Get(id)\n\tok = o != nil\n\tif !(ok || xx) {\n\t\treturn retwerr(errIDNotFound)\n\t}\n\n\tif ok {\n\t\tofields := o.Fields()\n\t\tfor _, f := range fields {\n\t\t\tprev := ofields.Get(f.Name())\n\t\t\tif !prev.Value().Equals(f.Value()) {\n\t\t\t\tofields = ofields.Set(f)\n\t\t\t\tupdateCount++\n\t\t\t}\n\t\t}\n\t\tobj := object.New(id, o.Geo(), o.Expires(), ofields)\n\t\tcol.Set(obj)\n\t\td.command = \"fset\"\n\t\td.key = key\n\t\td.obj = obj\n\t\td.timestamp = time.Now()\n\t\td.updated = updateCount > 0\n\t}\n\n\t// >> Response\n\n\tvar res resp.Value\n\n\tif ret {\n\t\tres := buildObjectResponse(msg, d.obj, start, kind, precision, withfields, msg.OutputType == JSON)\n\t\treturn res, d, nil\n\t}\n\n\tswitch msg.OutputType {\n\tcase JSON:\n\t\tres = resp.StringValue(`{\"ok\":true,\"elapsed\":\"` +\n\t\t\ttime.Since(start).String() + \"\\\"}\")\n\tcase RESP:\n\t\tres = resp.IntegerValue(updateCount)\n\t}\n\n\treturn res, d, nil\n}\n\n// FGET key id field\nfunc (s *Server) cmdFGET(msg *Message) (resp.Value, error) {\n\tstart := time.Now()\n\n\t// >> Args\n\n\targs := msg.Args\n\n\tif len(args) < 4 {\n\t\treturn retrerr(errInvalidNumberOfArguments)\n\t}\n\tkey, id, field := args[1], args[2], args[3]\n\n\t// >> Operation\n\n\tcol, _ := s.cols.Get(key)\n\tif col == nil {\n\t\treturn retrerr(errKeyNotFound)\n\t}\n\to := col.Get(id)\n\tif o == nil {\n\t\treturn retrerr(errIDNotFound)\n\t}\n\tf := o.Fields().Get(field)\n\n\t// >> Response\n\n\tvar buf bytes.Buffer\n\tswitch msg.OutputType {\n\tcase JSON:\n\t\tbuf.WriteString(`{\"ok\":true`)\n\t\tbuf.WriteString(`,\"value\":` + f.Value().JSON())\n\t\tbuf.WriteString(`,\"elapsed\":\"` + time.Since(start).String() + \"\\\"}\")\n\t\treturn resp.StringValue(buf.String()), nil\n\tcase RESP:\n\t\treturn resp.StringValue(f.Value().Data()), nil\n\t}\n\treturn NOMessage, nil\n}\n\n// EXPIRE key id seconds\nfunc (s *Server) cmdEXPIRE(msg *Message) (resp.Value, commandDetails, error) {\n\tstart := time.Now()\n\n\t// >> Args\n\n\targs := msg.Args\n\tif len(args) != 4 {\n\t\treturn retwerr(errInvalidNumberOfArguments)\n\t}\n\tkey, id, svalue := args[1], args[2], args[3]\n\tvalue, err := strconv.ParseFloat(svalue, 64)\n\tif err != nil {\n\t\treturn retwerr(errInvalidArgument(svalue))\n\t}\n\n\t// >> Operation\n\n\tvar ok bool\n\tvar obj *object.Object\n\tcol, _ := s.cols.Get(key)\n\tif col != nil {\n\t\t// replace the expiration by getting the old object\n\t\tex := time.Now().Add(\n\t\t\ttime.Duration(float64(time.Second) * value)).UnixNano()\n\t\to := col.Get(id)\n\t\tok = o != nil\n\t\tif ok {\n\t\t\tobj = object.New(id, o.Geo(), ex, o.Fields())\n\t\t\tcol.Set(obj)\n\t\t}\n\t}\n\n\t// >> Response\n\n\tvar d commandDetails\n\tif ok {\n\t\td.key = key\n\t\td.obj = obj\n\t\td.command = \"expire\"\n\t\td.updated = true\n\t\td.timestamp = time.Now()\n\t}\n\tvar res resp.Value\n\tswitch msg.OutputType {\n\tcase JSON:\n\t\tif ok {\n\t\t\tres = resp.StringValue(`{\"ok\":true,\"elapsed\":\"` +\n\t\t\t\ttime.Since(start).String() + \"\\\"}\")\n\t\t} else if col == nil {\n\t\t\treturn retwerr(errKeyNotFound)\n\t\t} else {\n\t\t\treturn retwerr(errIDNotFound)\n\t\t}\n\tcase RESP:\n\t\tif ok {\n\t\t\tres = resp.IntegerValue(1)\n\t\t} else {\n\t\t\tres = resp.IntegerValue(0)\n\t\t}\n\t}\n\treturn res, d, nil\n}\n\n// PERSIST key id\nfunc (s *Server) cmdPERSIST(msg *Message) (resp.Value, commandDetails, error) {\n\tstart := time.Now()\n\n\t// >> Args\n\n\targs := msg.Args\n\tif len(args) != 3 {\n\t\treturn retwerr(errInvalidNumberOfArguments)\n\t}\n\tkey, id := args[1], args[2]\n\n\t// >> Operation\n\n\tcol, _ := s.cols.Get(key)\n\tif col == nil {\n\t\tif msg.OutputType == RESP {\n\t\t\treturn resp.IntegerValue(0), commandDetails{}, nil\n\t\t}\n\t\treturn retwerr(errKeyNotFound)\n\t}\n\to := col.Get(id)\n\tif o == nil {\n\t\tif msg.OutputType == RESP {\n\t\t\treturn resp.IntegerValue(0), commandDetails{}, nil\n\t\t}\n\t\treturn retwerr(errIDNotFound)\n\t}\n\n\tvar obj *object.Object\n\tvar cleared bool\n\tif o.Expires() != 0 {\n\t\tobj = object.New(id, o.Geo(), 0, o.Fields())\n\t\tcol.Set(obj)\n\t\tcleared = true\n\t}\n\n\t// >> Response\n\n\tvar res resp.Value\n\n\tvar d commandDetails\n\td.command = \"persist\"\n\td.key = key\n\td.obj = obj\n\td.updated = cleared\n\td.timestamp = time.Now()\n\n\tswitch msg.OutputType {\n\tcase JSON:\n\t\tres = resp.SimpleStringValue(`{\"ok\":true,\"elapsed\":\"` +\n\t\t\ttime.Since(start).String() + \"\\\"}\")\n\tcase RESP:\n\t\tif cleared {\n\t\t\tres = resp.IntegerValue(1)\n\t\t} else {\n\t\t\tres = resp.IntegerValue(0)\n\t\t}\n\t}\n\treturn res, d, nil\n}\n\n// TTL key id\nfunc (s *Server) cmdTTL(msg *Message) (resp.Value, error) {\n\tstart := time.Now()\n\n\t// >> Args\n\n\targs := msg.Args\n\tif len(args) != 3 {\n\t\treturn retrerr(errInvalidNumberOfArguments)\n\t}\n\tkey, id := args[1], args[2]\n\n\t// >> Operation\n\n\tcol, _ := s.cols.Get(key)\n\tif col == nil {\n\t\tif msg.OutputType == JSON {\n\t\t\treturn retrerr(errKeyNotFound)\n\t\t}\n\t\treturn resp.IntegerValue(-2), nil\n\t}\n\n\to := col.Get(id)\n\tif o == nil {\n\t\tif msg.OutputType == JSON {\n\t\t\treturn retrerr(errIDNotFound)\n\t\t}\n\t\treturn resp.IntegerValue(-2), nil\n\t}\n\n\tvar ttl float64\n\tif o.Expires() == 0 {\n\t\tttl = -1\n\t} else {\n\t\tnow := start.UnixNano()\n\t\tttl = math.Max(float64(o.Expires()-now)/float64(time.Second), 0)\n\t}\n\n\t// >> Response\n\n\tif msg.OutputType == JSON {\n\t\treturn resp.SimpleStringValue(\n\t\t\t`{\"ok\":true,\"ttl\":` + strconv.Itoa(int(ttl)) + `,\"elapsed\":\"` +\n\t\t\t\ttime.Since(start).String() + \"\\\"}\"), nil\n\t}\n\treturn resp.IntegerValue(int(ttl)), nil\n}\n\n// EXISTS key id\nfunc (s *Server) cmdEXISTS(msg *Message) (resp.Value, error) {\n\tstart := time.Now()\n\n\t// >> Args\n\n\targs := msg.Args\n\tif len(args) != 3 {\n\t\treturn retrerr(errInvalidNumberOfArguments)\n\t}\n\tkey, id := args[1], args[2]\n\n\t// >> Operation\n\n\tcol, _ := s.cols.Get(key)\n\tif col == nil {\n\t\treturn retrerr(errKeyNotFound)\n\t}\n\n\to := col.Get(id)\n\texists := o != nil\n\n\t// >> Response\n\n\tif msg.OutputType == JSON {\n\t\treturn resp.SimpleStringValue(\n\t\t\t`{\"ok\":true,\"exists\":` + strconv.FormatBool(exists) + `,\"elapsed\":\"` +\n\t\t\t\ttime.Since(start).String() + \"\\\"}\"), nil\n\t}\n\treturn resp.BoolValue(exists), nil\n}\n\n// FEXISTS key id field\nfunc (s *Server) cmdFEXISTS(msg *Message) (resp.Value, error) {\n\tstart := time.Now()\n\n\t// >> Args\n\n\targs := msg.Args\n\tif len(args) != 4 {\n\t\treturn retrerr(errInvalidNumberOfArguments)\n\t}\n\tkey, id, field := args[1], args[2], args[3]\n\n\t// >> Operation\n\n\tcol, _ := s.cols.Get(key)\n\tif col == nil {\n\t\treturn retrerr(errKeyNotFound)\n\t}\n\n\to := col.Get(id)\n\tif o == nil {\n\t\treturn retrerr(errIDNotFound)\n\t}\n\n\tf := o.Fields().Get(field)\n\texists := f.Name() != \"\"\n\n\t// >> Response\n\n\tif msg.OutputType == JSON {\n\t\treturn resp.SimpleStringValue(\n\t\t\t`{\"ok\":true,\"exists\":` + strconv.FormatBool(exists) + `,\"elapsed\":\"` +\n\t\t\t\ttime.Since(start).String() + \"\\\"}\"), nil\n\t}\n\treturn resp.BoolValue(exists), nil\n}\n" }, { "path": "internal/server/dev.go", "content": "package server\n\nimport (\n\t\"errors\"\n\t\"fmt\"\n\t\"math/rand\"\n\t\"strconv\"\n\t\"sync/atomic\"\n\t\"time\"\n\n\t\"github.com/tidwall/resp\"\n\t\"github.com/tidwall/tile38/internal/log\"\n)\n\n// MASSINSERT num_keys num_points [minx miny maxx maxy]\n\nfunc randMassInsertPosition(minLat, minLon, maxLat, maxLon float64) (float64, float64) {\n\tlat, lon := (rand.Float64()*(maxLat-minLat))+minLat, (rand.Float64()*(maxLon-minLon))+minLon\n\treturn lat, lon\n}\n\nfunc (s *Server) cmdMassInsert(msg *Message) (res resp.Value, err error) {\n\tstart := time.Now()\n\tvs := msg.Args[1:]\n\n\tminLat, minLon, maxLat, maxLon := -90.0, -180.0, 90.0, 180.0 //37.10776, -122.67145, 38.19502, -121.62775\n\n\tvar snumCols, snumPoints string\n\tvar cols, objs int\n\tvar ok bool\n\tif vs, snumCols, ok = tokenval(vs); !ok || snumCols == \"\" {\n\t\treturn NOMessage, errInvalidNumberOfArguments\n\t}\n\tif vs, snumPoints, ok = tokenval(vs); !ok || snumPoints == \"\" {\n\t\treturn NOMessage, errInvalidNumberOfArguments\n\t}\n\tif len(vs) != 0 {\n\t\tvar sminLat, sminLon, smaxLat, smaxLon string\n\t\tif vs, sminLat, ok = tokenval(vs); !ok || sminLat == \"\" {\n\t\t\treturn NOMessage, errInvalidNumberOfArguments\n\t\t}\n\t\tif vs, sminLon, ok = tokenval(vs); !ok || sminLon == \"\" {\n\t\t\treturn NOMessage, errInvalidNumberOfArguments\n\t\t}\n\t\tif vs, smaxLat, ok = tokenval(vs); !ok || smaxLat == \"\" {\n\t\t\treturn NOMessage, errInvalidNumberOfArguments\n\t\t}\n\t\tif vs, smaxLon, ok = tokenval(vs); !ok || smaxLon == \"\" {\n\t\t\treturn NOMessage, errInvalidNumberOfArguments\n\t\t}\n\t\tvar err error\n\t\tif minLat, err = strconv.ParseFloat(sminLat, 64); err != nil {\n\t\t\treturn NOMessage, err\n\t\t}\n\t\tif minLon, err = strconv.ParseFloat(sminLon, 64); err != nil {\n\t\t\treturn NOMessage, err\n\t\t}\n\t\tif maxLat, err = strconv.ParseFloat(smaxLat, 64); err != nil {\n\t\t\treturn NOMessage, err\n\t\t}\n\t\tif maxLon, err = strconv.ParseFloat(smaxLon, 64); err != nil {\n\t\t\treturn NOMessage, err\n\t\t}\n\t\tif len(vs) != 0 {\n\t\t\treturn NOMessage, errors.New(\"invalid number of arguments\")\n\t\t}\n\t}\n\tn, err := strconv.ParseUint(snumCols, 10, 64)\n\tif err != nil {\n\t\treturn NOMessage, errInvalidArgument(snumCols)\n\t}\n\tcols = int(n)\n\tn, err = strconv.ParseUint(snumPoints, 10, 64)\n\tif err != nil {\n\t\treturn NOMessage, errInvalidArgument(snumPoints)\n\t}\n\n\tdocmd := func(args []string) error {\n\t\ts.mu.Lock()\n\t\tdefer s.mu.Unlock()\n\t\tnmsg := *msg\n\t\tnmsg._command = \"\"\n\t\tnmsg.Args = args\n\t\t_, d, err := s.command(&nmsg, nil)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\treturn s.writeAOF(nmsg.Args, &d)\n\n\t}\n\trand.Seed(time.Now().UnixNano())\n\tobjs = int(n)\n\tvar k atomic.Uint64\n\tfor i := 0; i < cols; i++ {\n\t\tkey := \"mi:\" + strconv.FormatInt(int64(i), 10)\n\t\tfunc(key string) {\n\t\t\t// lock cycle\n\t\t\tfor j := 0; j < objs; j++ {\n\t\t\t\tid := strconv.FormatInt(int64(j), 10)\n\t\t\t\tvar values []string\n\t\t\t\tvalues = append(values, \"set\", key, id)\n\t\t\t\tfvals := []float64{\n\t\t\t\t\t1, // one\n\t\t\t\t\t0, // zero\n\t\t\t\t\t-1, // negOne\n\t\t\t\t\t14, // nibble\n\t\t\t\t\t20.5, // tinyDiv10\n\t\t\t\t\t120, // int8\n\t\t\t\t\t-120, // int8\n\t\t\t\t\t20000, // int16\n\t\t\t\t\t-20000, // int16\n\t\t\t\t\t214748300, // int32\n\t\t\t\t\t-214748300, // int32\n\t\t\t\t\t2014748300, // float64\n\t\t\t\t\t123.12312301, // float64\n\t\t\t\t}\n\t\t\t\tfor i, fval := range fvals {\n\t\t\t\t\tvalues = append(values, \"FIELD\",\n\t\t\t\t\t\tfmt.Sprintf(\"fname:%d\", i),\n\t\t\t\t\t\tstrconv.FormatFloat(fval, 'f', -1, 64))\n\t\t\t\t}\n\t\t\t\tif rand.Int()%2 == 0 {\n\t\t\t\t\tvalues = append(values, \"EX\", fmt.Sprint(rand.Intn(25)+5))\n\t\t\t\t}\n\n\t\t\t\tif j%8 == 0 {\n\t\t\t\t\tvalues = append(values, \"STRING\", fmt.Sprintf(\"str%v\", j))\n\t\t\t\t} else {\n\t\t\t\t\tlat, lon := randMassInsertPosition(minLat, minLon, maxLat, maxLon)\n\t\t\t\t\tvalues = append(values, \"POINT\",\n\t\t\t\t\t\tstrconv.FormatFloat(lat, 'f', -1, 64),\n\t\t\t\t\t\tstrconv.FormatFloat(lon, 'f', -1, 64),\n\t\t\t\t\t)\n\t\t\t\t}\n\t\t\t\terr := docmd(values)\n\t\t\t\tif err != nil {\n\t\t\t\t\tlog.Fatal(err)\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t\tk.Add(1)\n\t\t\t\tif j%1000 == 1000-1 {\n\t\t\t\t\tlog.Debugf(\"massinsert: %s %d/%d\", key, k.Load(), cols*objs)\n\t\t\t\t}\n\t\t\t}\n\t\t}(key)\n\t}\n\tlog.Infof(\"massinsert: done %d objects\", k.Load())\n\treturn OKMessage(msg, start), nil\n}\n\nfunc (s *Server) cmdSleep(msg *Message) (res resp.Value, err error) {\n\tstart := time.Now()\n\tif len(msg.Args) != 2 {\n\t\treturn NOMessage, errInvalidNumberOfArguments\n\t}\n\td, _ := strconv.ParseFloat(msg.Args[1], 64)\n\ttime.Sleep(time.Duration(float64(time.Second) * d))\n\treturn OKMessage(msg, start), nil\n}\n" }, { "path": "internal/server/expire.go", "content": "package server\n\nimport (\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/tidwall/tile38/internal/collection\"\n\t\"github.com/tidwall/tile38/internal/log\"\n\t\"github.com/tidwall/tile38/internal/object\"\n)\n\nconst bgExpireDelay = time.Second / 10\n\n// backgroundExpiring deletes expired items from the database.\n// It's executes every 1/10 of a second.\nfunc (s *Server) backgroundExpiring(wg *sync.WaitGroup) {\n\tdefer wg.Done()\n\ts.loopUntilServerStops(bgExpireDelay, func() {\n\t\ts.mu.LockLowPriority()\n\t\tdefer s.mu.Unlock()\n\t\tnow := time.Now()\n\t\ts.backgroundExpireObjects(now)\n\t\ts.backgroundExpireHooks(now)\n\t})\n}\n\nfunc (s *Server) backgroundExpireObjects(now time.Time) {\n\tnano := now.UnixNano()\n\tvar msgs []*Message\n\ts.cols.Scan(func(key string, col *collection.Collection) bool {\n\t\tcol.ScanExpires(func(o *object.Object) bool {\n\t\t\tif nano < o.Expires() {\n\t\t\t\treturn false\n\t\t\t}\n\t\t\ts.statsExpired.Add(1)\n\t\t\tmsgs = append(msgs, &Message{Args: []string{\"del\", key, o.ID()}})\n\t\t\treturn true\n\t\t})\n\t\treturn true\n\t})\n\tfor _, msg := range msgs {\n\t\t_, d, err := s.cmdDEL(msg)\n\t\tif err != nil {\n\t\t\tlog.Fatal(err)\n\t\t}\n\t\tif err := s.writeAOF(msg.Args, &d); err != nil {\n\t\t\tlog.Fatal(err)\n\t\t}\n\t}\n\tif len(msgs) > 0 {\n\t\tlog.Debugf(\"Expired %d objects\\n\", len(msgs))\n\t}\n}\n\nfunc (s *Server) backgroundExpireHooks(now time.Time) {\n\tvar msgs []*Message\n\ts.hookExpires.Ascend(nil, func(v interface{}) bool {\n\t\th := v.(*Hook)\n\t\tif h.expires.After(now) {\n\t\t\treturn false\n\t\t}\n\t\tmsg := &Message{}\n\t\tif h.channel {\n\t\t\tmsg.Args = []string{\"delchan\", h.Name}\n\t\t} else {\n\t\t\tmsg.Args = []string{\"delhook\", h.Name}\n\t\t}\n\t\tmsgs = append(msgs, msg)\n\t\treturn true\n\t})\n\n\tfor _, msg := range msgs {\n\t\t_, d, err := s.cmdDelHook(msg)\n\t\tif err != nil {\n\t\t\tlog.Fatal(err)\n\t\t}\n\t\tif err := s.writeAOF(msg.Args, &d); err != nil {\n\t\t\tlog.Fatal(err)\n\t\t}\n\t}\n\tif len(msgs) > 0 {\n\t\tlog.Debugf(\"Expired %d hooks\\n\", len(msgs))\n\t}\n}\n" }, { "path": "internal/server/expr.go", "content": "package server\n\nimport (\n\t\"fmt\"\n\t\"regexp\"\n\t\"sync\"\n\n\t\"github.com/tidwall/expr\"\n\t\"github.com/tidwall/geojson\"\n\t\"github.com/tidwall/gjson\"\n\t\"github.com/tidwall/match\"\n\t\"github.com/tidwall/tile38/internal/field\"\n\t\"github.com/tidwall/tile38/internal/log\"\n\t\"github.com/tidwall/tile38/internal/object\"\n\t\"github.com/tidwall/tinylru\"\n)\n\ntype exprPool struct {\n\tpool *sync.Pool\n\tregexCache tinylru.LRUG[string, *regexp.Regexp]\n}\n\nfunc typeForObject(o *object.Object) expr.Value {\n\tswitch o.Geo().(type) {\n\tcase *geojson.Point, *geojson.SimplePoint:\n\t\treturn expr.String(\"Point\")\n\tcase *geojson.LineString:\n\t\treturn expr.String(\"LineString\")\n\tcase *geojson.Polygon, *geojson.Circle, *geojson.Rect:\n\t\treturn expr.String(\"Polygon\")\n\tcase *geojson.MultiPoint:\n\t\treturn expr.String(\"MultiPoint\")\n\tcase *geojson.MultiLineString:\n\t\treturn expr.String(\"MultiLineString\")\n\tcase *geojson.MultiPolygon:\n\t\treturn expr.String(\"MultiPolygon\")\n\tcase *geojson.GeometryCollection:\n\t\treturn expr.String(\"GeometryCollection\")\n\tcase *geojson.Feature:\n\t\treturn expr.String(\"Feature\")\n\tcase *geojson.FeatureCollection:\n\t\treturn expr.String(\"FeatureCollection\")\n\tdefault:\n\t\treturn expr.Undefined\n\t}\n}\n\nfunc resultToValue(r gjson.Result) expr.Value {\n\tif !r.Exists() {\n\t\treturn expr.Undefined\n\t}\n\tswitch r.Type {\n\tcase gjson.String:\n\t\treturn expr.String(r.String())\n\tcase gjson.False:\n\t\treturn expr.Bool(false)\n\tcase gjson.True:\n\t\treturn expr.Bool(true)\n\tcase gjson.Number:\n\t\treturn expr.Number(r.Float())\n\tcase gjson.JSON:\n\t\treturn expr.Object(r)\n\tdefault:\n\t\treturn expr.Null\n\t}\n}\n\nfunc objExpr(o *object.Object, info expr.RefInfo) (expr.Value, error) {\n\tif r := gjson.Get(o.Geo().Members(), info.Ident); r.Exists() {\n\t\treturn resultToValue(r), nil\n\t}\n\tswitch info.Ident {\n\tcase \"id\":\n\t\treturn expr.String(o.ID()), nil\n\tcase \"type\":\n\t\treturn typeForObject(o), nil\n\tdefault:\n\t\tvar rf field.Field\n\t\tvar ok bool\n\t\to.Fields().Scan(func(f field.Field) bool {\n\t\t\tif f.Name() == info.Ident {\n\t\t\t\trf = f\n\t\t\t\tok = true\n\t\t\t\treturn false\n\t\t\t}\n\t\t\treturn true\n\t\t})\n\t\tif ok {\n\t\t\tr := gjson.Parse(rf.Value().JSON())\n\t\t\treturn resultToValue(r), nil\n\t\t}\n\t}\n\treturn expr.Number(0), nil\n}\n\nfunc newExprPool(s *Server) *exprPool {\n\tpool := &exprPool{}\n\n\text := expr.NewExtender(\n\t\t// ref\n\t\tfunc(info expr.RefInfo, ctx *expr.Context) (expr.Value, error) {\n\t\t\to := ctx.UserData.(*object.Object)\n\t\t\tif !info.Chain {\n\t\t\t\t// root\n\t\t\t\tif info.Ident == \"this\" {\n\t\t\t\t\treturn expr.Object(o), nil\n\t\t\t\t}\n\t\t\t\treturn objExpr(o, info)\n\t\t\t} else {\n\t\t\t\tswitch v := info.Value.Value().(type) {\n\t\t\t\tcase *object.Object:\n\t\t\t\t\treturn objExpr(o, info)\n\t\t\t\tcase gjson.Result:\n\t\t\t\t\treturn resultToValue(v.Get(info.Ident)), nil\n\t\t\t\tdefault:\n\t\t\t\t\t// object methods\n\t\t\t\t\tswitch info.Ident {\n\t\t\t\t\tcase \"match\":\n\t\t\t\t\t\treturn expr.Function(\"match\"), nil\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t\treturn expr.Undefined, nil\n\t\t\t}\n\t\t},\n\t\t// call\n\t\tfunc(info expr.CallInfo, ctx *expr.Context) (expr.Value, error) {\n\t\t\tif info.Chain {\n\t\t\t\tswitch info.Ident {\n\t\t\t\tcase \"match\":\n\t\t\t\t\tif info.Args.Len() < 0 {\n\t\t\t\t\t\treturn expr.Undefined, nil\n\t\t\t\t\t}\n\t\t\t\t\tt := match.MatchNoCase(info.Value.String(),\n\t\t\t\t\t\tinfo.Args.At(0).String())\n\t\t\t\t\treturn expr.Bool(t), nil\n\t\t\t\t}\n\t\t\t}\n\t\t\treturn expr.Undefined, nil\n\t\t},\n\t\t// op\n\t\tfunc(info expr.OpInfo, ctx *expr.Context) (expr.Value, error) {\n\t\t\tswitch info.Op {\n\t\t\tcase expr.OpRegex:\n\t\t\t\tfield := info.Left.String()\n\t\t\t\tpattern := info.Right.String()\n\t\t\t\tre, ok := pool.regexCache.Get(pattern)\n\t\t\t\tif !ok {\n\t\t\t\t\tvar err error\n\t\t\t\t\tre, err = regexp.Compile(pattern)\n\t\t\t\t\tif err != nil {\n\t\t\t\t\t\treturn expr.Undefined,\n\t\t\t\t\t\t\tfmt.Errorf(\"invalid regex pattern: %v\", err)\n\t\t\t\t\t}\n\t\t\t\t\tpool.regexCache.Set(pattern, re)\n\t\t\t\t}\n\t\t\t\treturn expr.Bool(re.MatchString(field)), nil\n\t\t\t}\n\t\t\treturn expr.Undefined, nil\n\t\t},\n\t)\n\n\tpool.pool = &sync.Pool{\n\t\tNew: func() any {\n\t\t\tctx := &expr.Context{\n\t\t\t\tExtender: ext,\n\t\t\t}\n\t\t\treturn ctx\n\t\t},\n\t}\n\n\treturn pool\n}\n\nfunc (p *exprPool) Get(o *object.Object) *expr.Context {\n\tctx := p.pool.Get().(*expr.Context)\n\tctx.UserData = o\n\tctx.NoCase = true\n\treturn ctx\n}\n\nfunc (p *exprPool) Put(ctx *expr.Context) {\n\tp.pool.Put(ctx)\n}\n\nfunc (where whereT) matchExpr(s *Server, o *object.Object) bool {\n\tctx := s.epool.Get(o)\n\tres, err := expr.Eval(where.name, ctx)\n\tif err != nil {\n\t\tlog.Debugf(\"%v\", err)\n\t}\n\ts.epool.Put(ctx)\n\treturn res.Bool()\n}\n" }, { "path": "internal/server/expression.go", "content": "package server\n\nimport (\n\t\"strings\"\n\n\t\"github.com/tidwall/geojson\"\n)\n\n// BinaryOp represents various operators for expressions\ntype BinaryOp byte\n\n// expression operator enum\nconst (\n\tNOOP BinaryOp = iota\n\tAND\n\tOR\n\ttokenAND = \"and\"\n\ttokenOR = \"or\"\n\ttokenNOT = \"not\"\n\ttokenLParen = \"(\"\n\ttokenRParen = \")\"\n)\n\n// areaExpression is (maybe negated) either an spatial object or operator +\n// children (other expressions).\ntype areaExpression struct {\n\tnegate bool\n\tobj geojson.Object\n\top BinaryOp\n\tchildren children\n}\n\ntype children []*areaExpression\n\n// String representation, helpful in logging.\nfunc (e *areaExpression) String() (res string) {\n\tif e.obj != nil {\n\t\tres = e.obj.String()\n\t} else {\n\t\tvar chStrings []string\n\t\tfor _, c := range e.children {\n\t\t\tchStrings = append(chStrings, c.String())\n\t\t}\n\t\tswitch e.op {\n\t\tcase NOOP:\n\t\t\tres = \"empty operator\"\n\t\tcase AND:\n\t\t\tres = \"(\" + strings.Join(chStrings, \" \"+tokenAND+\" \") + \")\"\n\t\tcase OR:\n\t\t\tres = \"(\" + strings.Join(chStrings, \" \"+tokenOR+\" \") + \")\"\n\t\tdefault:\n\t\t\tres = \"unknown operator\"\n\t\t}\n\t}\n\tif e.negate {\n\t\tres = tokenNOT + \" \" + res\n\t}\n\treturn\n}\n\n// Return boolean value modulo negate field of the expression.\nfunc (e *areaExpression) maybeNegate(val bool) bool {\n\tif e.negate {\n\t\treturn !val\n\t}\n\treturn val\n}\n\n// Methods for testing an areaExpression against the spatial object.\nfunc (e *areaExpression) testObject(\n\to geojson.Object,\n\tobjObjTest func(o1, o2 geojson.Object) bool,\n\texprObjTest func(ae *areaExpression, ob geojson.Object) bool,\n) bool {\n\tif e.obj != nil {\n\t\treturn objObjTest(e.obj, o)\n\t}\n\tswitch e.op {\n\tcase AND:\n\t\tfor _, c := range e.children {\n\t\t\tif !exprObjTest(c, o) {\n\t\t\t\treturn false\n\t\t\t}\n\t\t}\n\t\treturn true\n\tcase OR:\n\t\tfor _, c := range e.children {\n\t\t\tif exprObjTest(c, o) {\n\t\t\t\treturn true\n\t\t\t}\n\t\t}\n\t\treturn false\n\t}\n\treturn false\n}\n\nfunc (e *areaExpression) rawIntersects(o geojson.Object) bool {\n\treturn e.testObject(o, geojson.Object.Intersects, (*areaExpression).Intersects)\n}\n\nfunc (e *areaExpression) rawContains(o geojson.Object) bool {\n\treturn e.testObject(o, geojson.Object.Contains, (*areaExpression).Contains)\n}\n\nfunc (e *areaExpression) rawWithin(o geojson.Object) bool {\n\treturn e.testObject(o, geojson.Object.Within, (*areaExpression).Within)\n}\n\nfunc (e *areaExpression) Intersects(o geojson.Object) bool {\n\treturn e.maybeNegate(e.rawIntersects(o))\n}\n\nfunc (e *areaExpression) Contains(o geojson.Object) bool {\n\treturn e.maybeNegate(e.rawContains(o))\n}\n\nfunc (e *areaExpression) Within(o geojson.Object) bool {\n\treturn e.maybeNegate(e.rawWithin(o))\n}\n\n// Methods for testing an areaExpression against another areaExpression.\nfunc (e *areaExpression) testExpression(\n\tother *areaExpression,\n\texprObjTest func(ae *areaExpression, ob geojson.Object) bool,\n\trawExprExprTest func(ae1, ae2 *areaExpression) bool,\n\texprExprTest func(ae1, ae2 *areaExpression) bool,\n) bool {\n\tif other.negate {\n\t\toppositeExp := &areaExpression{negate: !e.negate, obj: e.obj, op: e.op, children: e.children}\n\t\tnonNegateOther := &areaExpression{obj: other.obj, op: other.op, children: other.children}\n\t\treturn exprExprTest(oppositeExp, nonNegateOther)\n\t}\n\tif other.obj != nil {\n\t\treturn exprObjTest(e, other.obj)\n\t}\n\tswitch other.op {\n\tcase AND:\n\t\tfor _, c := range other.children {\n\t\t\tif !rawExprExprTest(e, c) {\n\t\t\t\treturn false\n\t\t\t}\n\t\t}\n\t\treturn true\n\tcase OR:\n\t\tfor _, c := range other.children {\n\t\t\tif rawExprExprTest(e, c) {\n\t\t\t\treturn true\n\t\t\t}\n\t\t}\n\t\treturn false\n\t}\n\treturn false\n}\n\nfunc (e *areaExpression) rawIntersectsExpr(other *areaExpression) bool {\n\treturn e.testExpression(\n\t\tother,\n\t\t(*areaExpression).rawIntersects,\n\t\t(*areaExpression).rawIntersectsExpr,\n\t\t(*areaExpression).IntersectsExpr)\n}\n\nfunc (e *areaExpression) rawWithinExpr(other *areaExpression) bool {\n\treturn e.testExpression(\n\t\tother,\n\t\t(*areaExpression).rawWithin,\n\t\t(*areaExpression).rawWithinExpr,\n\t\t(*areaExpression).WithinExpr)\n}\n\nfunc (e *areaExpression) rawContainsExpr(other *areaExpression) bool {\n\treturn e.testExpression(\n\t\tother,\n\t\t(*areaExpression).rawContains,\n\t\t(*areaExpression).rawContainsExpr,\n\t\t(*areaExpression).ContainsExpr)\n}\n\nfunc (e *areaExpression) IntersectsExpr(other *areaExpression) bool {\n\treturn e.maybeNegate(e.rawIntersectsExpr(other))\n}\n\nfunc (e *areaExpression) WithinExpr(other *areaExpression) bool {\n\treturn e.maybeNegate(e.rawWithinExpr(other))\n}\n\nfunc (e *areaExpression) ContainsExpr(other *areaExpression) bool {\n\treturn e.maybeNegate(e.rawContainsExpr(other))\n}\n" }, { "path": "internal/server/fence.go", "content": "package server\n\nimport (\n\t\"math\"\n\t\"sort\"\n\t\"strconv\"\n\t\"time\"\n\n\t\"github.com/tidwall/geojson\"\n\t\"github.com/tidwall/geojson/geo\"\n\t\"github.com/tidwall/geojson/geometry\"\n\t\"github.com/tidwall/gjson\"\n\t\"github.com/tidwall/tile38/internal/field\"\n\t\"github.com/tidwall/tile38/internal/glob\"\n\t\"github.com/tidwall/tile38/internal/object\"\n)\n\n// FenceMatch executes a fence match returns back json messages for fence detection.\nfunc FenceMatch(hookName string, sw *scanWriter, fence *liveFenceSwitches, metas []FenceMeta, details *commandDetails) []string {\n\tmsgs := fenceMatch(hookName, sw, fence, metas, details)\n\tif len(fence.accept) == 0 {\n\t\treturn msgs\n\t}\n\tnmsgs := make([]string, 0, len(msgs))\n\tfor _, msg := range msgs {\n\t\tif fence.accept[gjson.Get(msg, \"command\").String()] {\n\t\t\tnmsgs = append(nmsgs, msg)\n\t\t}\n\t}\n\treturn nmsgs\n}\nfunc appendHookDetails(b []byte, hookName string, metas []FenceMeta) []byte {\n\tif len(hookName) > 0 {\n\t\tb = append(b, `,\"hook\":`...)\n\t\tb = appendJSONString(b, hookName)\n\t}\n\tif len(metas) > 0 {\n\t\tb = append(b, `,\"meta\":{`...)\n\t\tfor i, meta := range metas {\n\t\t\tif i > 0 {\n\t\t\t\tb = append(b, ',')\n\t\t\t}\n\t\t\tb = appendJSONString(b, meta.Name)\n\t\t\tb = append(b, ':')\n\t\t\tb = appendJSONString(b, meta.Value)\n\t\t}\n\t\tb = append(b, '}')\n\t}\n\treturn b\n}\n\nfunc objIsSpatial(obj geojson.Object) bool {\n\t_, ok := obj.(geojson.Spatial)\n\treturn ok\n}\n\nfunc hookJSONString(hookName string, metas []FenceMeta) string {\n\treturn string(appendHookDetails(nil, hookName, metas))\n}\n\nfunc multiGlobMatch(globs []string, s string) bool {\n\tif len(globs) == 0 || (len(globs) == 1 && globs[0] == \"*\") {\n\t\treturn true\n\t}\n\tfor _, pattern := range globs {\n\t\tmatch, _ := glob.Match(pattern, s)\n\t\tif match {\n\t\t\treturn true\n\t\t}\n\t}\n\treturn false\n}\n\nfunc fenceMatch(\n\thookName string, sw *scanWriter, fence *liveFenceSwitches,\n\tmetas []FenceMeta, details *commandDetails,\n) []string {\n\tif details.command == \"drop\" {\n\t\treturn []string{\n\t\t\t`{\"command\":\"drop\"` + hookJSONString(hookName, metas) +\n\t\t\t\t`,\"key\":` + jsonString(details.key) +\n\t\t\t\t`,\"time\":` + jsonTimeFormat(details.timestamp) + `}`,\n\t\t}\n\t}\n\tif details.obj == nil {\n\t\treturn nil\n\t}\n\tif !multiGlobMatch(fence.globs, details.obj.ID()) {\n\t\treturn nil\n\t}\n\tif !objIsSpatial(details.obj.Geo()) {\n\t\treturn nil\n\t}\n\tif details.command == \"fset\" {\n\t\tnofields := sw.nofields\n\t\tif nofields {\n\t\t\treturn nil\n\t\t}\n\t}\n\tif details.command == \"del\" {\n\t\treturn []string{\n\t\t\t`{\"command\":\"del\"` + hookJSONString(hookName, metas) +\n\t\t\t\t`,\"key\":` + jsonString(details.key) +\n\t\t\t\t`,\"id\":` + jsonString(details.obj.ID()) +\n\t\t\t\t`,\"time\":` + jsonTimeFormat(details.timestamp) + `}`,\n\t\t}\n\t}\n\tvar roamNearbys, roamFaraways []roamMatch\n\tvar detect = \"outside\"\n\tif fence != nil {\n\t\tif fence.roam.on {\n\t\t\tif details.command == \"set\" {\n\t\t\t\troamNearbys, roamFaraways =\n\t\t\t\t\tfenceMatchRoam(sw.s, fence, details.obj, details.old)\n\t\t\t\tif len(roamNearbys) == 0 && len(roamFaraways) == 0 {\n\t\t\t\t\treturn nil\n\t\t\t\t}\n\t\t\t}\n\t\t\tdetect = \"roam\"\n\t\t} else {\n\t\t\tvar nocross bool\n\t\t\t// not using roaming\n\t\t\tmatch1 := fenceMatchObject(fence, details.old)\n\t\t\tif match1 {\n\t\t\t\tmatch1, _, _ = sw.testObject(details.old)\n\t\t\t\tnocross = !match1\n\t\t\t}\n\t\t\tmatch2 := fenceMatchObject(fence, details.obj)\n\t\t\tif match2 {\n\t\t\t\tmatch2, _, _ = sw.testObject(details.obj)\n\t\t\t\tnocross = !match2\n\t\t\t}\n\t\t\tif match1 && match2 {\n\t\t\t\tdetect = \"inside\"\n\t\t\t} else if match1 && !match2 {\n\t\t\t\tdetect = \"exit\"\n\t\t\t} else if !match1 && match2 {\n\t\t\t\tdetect = \"enter\"\n\t\t\t\tif details.command == \"fset\" {\n\t\t\t\t\tdetect = \"inside\"\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\tif details.command != \"fset\" {\n\t\t\t\t\t// For cross detection, the object is outside the fence spatially,\n\t\t\t\t\t// so testObject wasn't called above. We need to check WHERE clause\n\t\t\t\t\t// before proceeding with cross detection.\n\t\t\t\t\tif match, _ := sw.fieldMatch(details.obj); !match {\n\t\t\t\t\t\treturn nil\n\t\t\t\t\t}\n\t\t\t\t\t// Maybe the old object and new object create a line that crosses the fence.\n\t\t\t\t\t// Must detect for that possibility.\n\t\t\t\t\tif !nocross && details.old != nil {\n\t\t\t\t\t\tls := geojson.NewLineString(geometry.NewLine(\n\t\t\t\t\t\t\t[]geometry.Point{\n\t\t\t\t\t\t\t\tdetails.old.Geo().Center(),\n\t\t\t\t\t\t\t\tdetails.obj.Geo().Center(),\n\t\t\t\t\t\t\t}, nil))\n\t\t\t\t\t\ttemp := false\n\t\t\t\t\t\tif fence.cmd == \"within\" {\n\t\t\t\t\t\t\t// because we are testing if the line croses the area we need to use\n\t\t\t\t\t\t\t// \"intersects\" instead of \"within\".\n\t\t\t\t\t\t\tfence.cmd = \"intersects\"\n\t\t\t\t\t\t\ttemp = true\n\t\t\t\t\t\t}\n\t\t\t\t\t\tlso := object.New(\"\", ls, 0, field.List{})\n\t\t\t\t\t\tif fenceMatchObject(fence, lso) {\n\t\t\t\t\t\t\tdetect = \"cross\"\n\t\t\t\t\t\t}\n\t\t\t\t\t\tif temp {\n\t\t\t\t\t\t\tfence.cmd = \"within\"\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\n\t// TODO: fields\n\t// if details.fmap == nil {\n\t// \treturn nil\n\t// }\n\tfor {\n\t\tif fence.detect != nil && !fence.detect[detect] {\n\t\t\tif detect == \"enter\" {\n\t\t\t\tdetect = \"inside\"\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tif detect == \"exit\" || detect == \"cross\" {\n\t\t\t\tdetect = \"outside\"\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\treturn nil\n\t\t}\n\t\tbreak\n\t}\n\tvar distance float64\n\tif fence.distance && fence.obj != nil {\n\t\tdistance = details.obj.Geo().Distance(fence.obj)\n\t}\n\n\tsw.fullFields = true\n\tsw.msg.OutputType = JSON\n\tsw.writeObject(ScanWriterParams{\n\t\tobj: details.obj,\n\t\tnoTest: true,\n\t\tdist: distance,\n\t\tdistOutput: fence.distance,\n\t})\n\n\tif sw.wr.Len() == 0 {\n\t\treturn nil\n\t}\n\n\tres := sw.wr.String()\n\tsw.wr.Reset()\n\tif len(res) > 0 && res[0] == ',' {\n\t\tres = res[1:]\n\t}\n\tif sw.output == outputIDs {\n\t\tres = `{\"id\":` + string(res) + `}`\n\t}\n\n\tvar group string\n\tif detect == \"enter\" {\n\t\tgroup = sw.s.groupConnect(hookName, details.key, details.obj.ID())\n\t} else if detect == \"cross\" {\n\t\tsw.s.groupDisconnect(hookName, details.key, details.obj.ID())\n\t\tgroup = sw.s.groupConnect(hookName, details.key, details.obj.ID())\n\t} else {\n\t\tgroup = sw.s.groupGet(hookName, details.key, details.obj.ID())\n\t\tif group == \"\" {\n\t\t\tgroup = sw.s.groupConnect(hookName, details.key, details.obj.ID())\n\t\t}\n\t}\n\tvar msgs []string\n\tif fence.detect == nil || fence.detect[detect] {\n\t\tif len(res) > 0 && res[0] == '{' {\n\t\t\tmsgs = append(msgs, makemsg(details.command, group, detect,\n\t\t\t\thookName, metas, details.key, details.timestamp, res[1:]))\n\t\t} else {\n\t\t\tmsgs = append(msgs, string(res))\n\t\t}\n\t}\n\tswitch detect {\n\tcase \"enter\":\n\t\tif fence.detect == nil || fence.detect[\"inside\"] {\n\t\t\tmsgs = append(msgs, makemsg(details.command, group, \"inside\", hookName, metas, details.key, details.timestamp, res[1:]))\n\t\t}\n\tcase \"exit\", \"cross\":\n\t\tif fence.detect == nil || fence.detect[\"outside\"] {\n\t\t\tmsgs = append(msgs, makemsg(details.command, group, \"outside\", hookName, metas, details.key, details.timestamp, res[1:]))\n\t\t}\n\tcase \"roam\":\n\t\tif len(msgs) > 0 {\n\t\t\tvar nmsgs []string\n\t\t\tfor _, msg := range msgs {\n\t\t\t\tcmd := gjson.Get(msg, \"command\")\n\t\t\t\tif cmd.Exists() && cmd.String() != \"set\" {\n\t\t\t\t\tnmsgs = append(nmsgs, msg)\n\t\t\t\t}\n\t\t\t}\n\t\t\tfor i := range roamNearbys {\n\t\t\t\tnmsg := extendRoamMessage(sw, fence,\n\t\t\t\t\t\"nearby\", msgs[0], roamNearbys[i])\n\t\t\t\tnmsgs = append(nmsgs, string(nmsg))\n\t\t\t}\n\t\t\tfor i := range roamFaraways {\n\t\t\t\tnmsg := extendRoamMessage(sw, fence,\n\t\t\t\t\t\"faraway\", msgs[0], roamFaraways[i])\n\t\t\t\tnmsgs = append(nmsgs, string(nmsg))\n\t\t\t}\n\t\t\tmsgs = nmsgs\n\t\t}\n\t}\n\treturn msgs\n}\n\nfunc extendRoamMessage(\n\tsw *scanWriter, fence *liveFenceSwitches,\n\tkind string, baseMsg string, match roamMatch,\n) string {\n\t// hack off the last '}'\n\tnmsg := []byte(baseMsg[:len(baseMsg)-1])\n\tnmsg = append(nmsg, `,\"`+kind+`\":{\"key\":`...)\n\tnmsg = appendJSONString(nmsg, fence.roam.key)\n\tnmsg = append(nmsg, `,\"id\":`...)\n\tnmsg = appendJSONString(nmsg, match.id)\n\tnmsg = append(nmsg, `,\"object\":`...)\n\tnmsg = match.obj.AppendJSON(nmsg)\n\tnmsg = append(nmsg, `,\"meters\":`...)\n\tnmsg = strconv.AppendFloat(nmsg,\n\t\tmath.Floor(match.meters*1000)/1000, 'f', -1, 64)\n\tif fence.roam.scan != \"\" {\n\t\tnmsg = append(nmsg, `,\"scan\":[`...)\n\t\tcol, _ := sw.s.cols.Get(fence.roam.key)\n\t\tif col != nil {\n\t\t\to := col.Get(match.id)\n\t\t\tif o != nil {\n\t\t\t\tnmsg = append(nmsg, `{\"id\":`...)\n\t\t\t\tnmsg = appendJSONString(nmsg, match.id)\n\t\t\t\tnmsg = append(nmsg, `,\"self\":true,\"object\":`...)\n\t\t\t\tnmsg = o.Geo().AppendJSON(nmsg)\n\t\t\t\tnmsg = append(nmsg, '}')\n\t\t\t}\n\t\t\tpattern := match.id + fence.roam.scan\n\t\t\titerator := func(o *object.Object) bool {\n\t\t\t\tif o.ID() == match.id {\n\t\t\t\t\treturn true\n\t\t\t\t}\n\t\t\t\tif matched, _ := glob.Match(pattern, o.ID()); matched {\n\t\t\t\t\tnmsg = append(nmsg, `,{\"id\":`...)\n\t\t\t\t\tnmsg = appendJSONString(nmsg, o.ID())\n\t\t\t\t\tnmsg = append(nmsg, `,\"object\":`...)\n\t\t\t\t\tnmsg = o.Geo().AppendJSON(nmsg)\n\t\t\t\t\tnmsg = append(nmsg, '}')\n\t\t\t\t}\n\t\t\t\treturn true\n\t\t\t}\n\t\t\tg := glob.Parse(pattern, false)\n\t\t\tif g.Limits[0] == \"\" && g.Limits[1] == \"\" {\n\t\t\t\tcol.Scan(false, nil, nil, iterator)\n\t\t\t} else {\n\t\t\t\tcol.ScanRange(g.Limits[0], g.Limits[1],\n\t\t\t\t\tfalse, nil, nil, iterator)\n\t\t\t}\n\t\t}\n\t\tnmsg = append(nmsg, ']')\n\t}\n\n\tnmsg = append(nmsg, '}')\n\n\t// re-add the last '}'\n\tnmsg = append(nmsg, '}')\n\treturn string(nmsg)\n}\n\nfunc makemsg(\n\tcommand, group, detect, hookName string,\n\tmetas []FenceMeta, key string, t time.Time, tail string,\n) string {\n\tvar buf []byte\n\tbuf = append(append(buf, `{\"command\":\"`...), command...)\n\tbuf = append(append(buf, `\",\"group\":\"`...), group...)\n\tbuf = append(append(buf, `\",\"detect\":\"`...), detect...)\n\tbuf = append(buf, '\"')\n\tbuf = appendHookDetails(buf, hookName, metas)\n\tbuf = appendJSONString(append(buf, `,\"key\":`...), key)\n\tbuf = appendJSONTimeFormat(append(buf, `,\"time\":`...), t)\n\tbuf = append(append(buf, ','), tail...)\n\treturn string(buf)\n}\n\nfunc fenceMatchObject(fence *liveFenceSwitches, o *object.Object) bool {\n\tif o == nil {\n\t\treturn false\n\t}\n\tif fence.roam.on {\n\t\t// we need to check this object against\n\t\treturn false\n\t}\n\tswitch fence.cmd {\n\tcase \"nearby\":\n\t\t// nearby is an INTERSECT on a Circle\n\t\treturn o.Geo().Intersects(fence.obj)\n\tcase \"within\":\n\t\treturn o.Geo().Within(fence.obj)\n\tcase \"intersects\":\n\t\treturn o.Geo().Intersects(fence.obj)\n\t}\n\treturn false\n}\n\nfunc fenceMatchNearbys(\n\ts *Server, fence *liveFenceSwitches,\n\tobj *object.Object,\n) (nearbys []roamMatch) {\n\tif obj == nil {\n\t\treturn nil\n\t}\n\tcol, _ := s.cols.Get(fence.roam.key)\n\tif col == nil {\n\t\treturn nil\n\t}\n\tcenter := obj.Geo().Center()\n\tminLat, minLon, maxLat, maxLon :=\n\t\tgeo.RectFromCenter(center.Y, center.X, fence.roam.meters)\n\trect := geometry.Rect{\n\t\tMin: geometry.Point{X: minLon, Y: minLat},\n\t\tMax: geometry.Point{X: maxLon, Y: maxLat},\n\t}\n\tcol.Intersects(geojson.NewRect(rect), 0, nil, nil,\n\t\tfunc(o *object.Object) bool {\n\t\t\tvar idMatch bool\n\t\t\tif o.ID() == obj.ID() {\n\t\t\t\treturn true // skip self\n\t\t\t}\n\t\t\tmeters := o.Geo().Distance(o.Geo())\n\t\t\tif meters > fence.roam.meters {\n\t\t\t\treturn true // skip outside radius\n\t\t\t}\n\t\t\tif fence.roam.pattern {\n\t\t\t\tidMatch, _ = glob.Match(fence.roam.id, o.ID())\n\t\t\t} else {\n\t\t\t\tidMatch = fence.roam.id == o.ID()\n\t\t\t}\n\t\t\tif !idMatch {\n\t\t\t\treturn true // skip non-id match\n\t\t\t}\n\t\t\tmatch := roamMatch{\n\t\t\t\tid: o.ID(),\n\t\t\t\tobj: o.Geo(),\n\t\t\t\tmeters: obj.Geo().Distance(o.Geo()),\n\t\t\t}\n\t\t\tnearbys = append(nearbys, match)\n\t\t\treturn true\n\t\t},\n\t)\n\treturn nearbys\n}\n\nfunc fenceMatchRoam(\n\ts *Server, fence *liveFenceSwitches,\n\tobj, old *object.Object,\n) (nearbys, faraways []roamMatch) {\n\toldNearbys := fenceMatchNearbys(s, fence, old)\n\tnewNearbys := fenceMatchNearbys(s, fence, obj)\n\t// Go through all matching objects in new-nearbys and old-nearbys.\n\tfor i := 0; i < len(oldNearbys); i++ {\n\t\tvar match bool\n\t\tvar j int\n\t\tfor ; j < len(newNearbys); j++ {\n\t\t\tif newNearbys[j].id == oldNearbys[i].id {\n\t\t\t\tmatch = true\n\t\t\t\tbreak\n\t\t\t}\n\t\t}\n\t\tif match {\n\t\t\t// dwelling, more from old-nearbys\n\t\t\toldNearbys[i] = oldNearbys[len(oldNearbys)-1]\n\t\t\toldNearbys = oldNearbys[:len(oldNearbys)-1]\n\t\t\ti--\n\t\t\tif fence.nodwell {\n\t\t\t\t// no dwelling allowed, remove from both lists\n\t\t\t\tnewNearbys[j] = newNearbys[len(newNearbys)-1]\n\t\t\t\tnewNearbys = newNearbys[:len(newNearbys)-1]\n\t\t\t}\n\t\t}\n\t}\n\tfaraways, nearbys = oldNearbys, newNearbys\n\t// ensure the faraways distances are to the new object\n\tfor i := 0; i < len(faraways); i++ {\n\t\tfaraways[i].meters = faraways[i].obj.Distance(obj.Geo())\n\t}\n\tsortRoamMatches(faraways)\n\tsortRoamMatches(nearbys)\n\treturn nearbys, faraways\n}\n\n// sortRoamMatches stable sorts roam matches\nfunc sortRoamMatches(matches []roamMatch) {\n\tsort.Slice(matches, func(i, j int) bool {\n\t\tif matches[i].meters < matches[j].meters {\n\t\t\treturn true\n\t\t}\n\t\tif matches[i].meters > matches[j].meters {\n\t\t\treturn false\n\t\t}\n\t\treturn matches[i].id < matches[j].id\n\t})\n}\n" }, { "path": "internal/server/follow.go", "content": "package server\n\nimport (\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"strconv\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/tidwall/resp\"\n\t\"github.com/tidwall/tile38/internal/log\"\n)\n\nvar errNoLongerFollowing = errors.New(\"no longer following\")\n\nconst checksumsz = 512 * 1024\n\nfunc (s *Server) cmdFollow(msg *Message) (res resp.Value, err error) {\n\tstart := time.Now()\n\tvs := msg.Args[1:]\n\tvar ok bool\n\tvar host, sport string\n\n\tif vs, host, ok = tokenval(vs); !ok || host == \"\" {\n\t\treturn NOMessage, errInvalidNumberOfArguments\n\t}\n\tif vs, sport, ok = tokenval(vs); !ok || sport == \"\" {\n\t\treturn NOMessage, errInvalidNumberOfArguments\n\t}\n\tif len(vs) != 0 {\n\t\treturn NOMessage, errInvalidNumberOfArguments\n\t}\n\thost = strings.ToLower(host)\n\tsport = strings.ToLower(sport)\n\tvar update bool\n\tif host == \"no\" && sport == \"one\" {\n\t\tupdate = s.config.followHost() != \"\" || s.config.followPort() != 0\n\t\ts.config.setFollowHost(\"\")\n\t\ts.config.setFollowPort(0)\n\t} else {\n\t\tn, err := strconv.ParseUint(sport, 10, 64)\n\t\tif err != nil {\n\t\t\treturn NOMessage, errInvalidArgument(sport)\n\t\t}\n\t\tport := int(n)\n\t\tupdate = s.config.followHost() != host || s.config.followPort() != port\n\t\tauth := s.config.leaderAuth()\n\t\tif update {\n\t\t\ts.mu.Unlock()\n\t\t\tconn, err := DialTimeout(fmt.Sprintf(\"%s:%d\", host, port), time.Second*2)\n\t\t\tif err != nil {\n\t\t\t\ts.mu.Lock()\n\t\t\t\treturn NOMessage, fmt.Errorf(\"cannot follow: %v\", err)\n\t\t\t}\n\t\t\tdefer conn.Close()\n\t\t\tif auth != \"\" {\n\t\t\t\tif err := s.followDoLeaderAuth(conn, auth); err != nil {\n\t\t\t\t\treturn NOMessage, fmt.Errorf(\"cannot follow: %v\", err)\n\t\t\t\t}\n\t\t\t}\n\t\t\tm, err := doServer(conn)\n\t\t\tif err != nil {\n\t\t\t\ts.mu.Lock()\n\t\t\t\treturn NOMessage, fmt.Errorf(\"cannot follow: %v\", err)\n\t\t\t}\n\t\t\tif m[\"id\"] == \"\" {\n\t\t\t\ts.mu.Lock()\n\t\t\t\treturn NOMessage, fmt.Errorf(\"cannot follow: invalid id\")\n\t\t\t}\n\t\t\tif m[\"id\"] == s.config.serverID() {\n\t\t\t\ts.mu.Lock()\n\t\t\t\treturn NOMessage, fmt.Errorf(\"cannot follow self\")\n\t\t\t}\n\t\t\tif m[\"following\"] != \"\" {\n\t\t\t\ts.mu.Lock()\n\t\t\t\treturn NOMessage, fmt.Errorf(\"cannot follow a follower\")\n\t\t\t}\n\t\t\ts.mu.Lock()\n\t\t}\n\t\ts.config.setFollowHost(host)\n\t\ts.config.setFollowPort(port)\n\t}\n\ts.config.write(false)\n\tif update {\n\t\ts.followc.Add(1)\n\t\tif s.config.followHost() != \"\" {\n\t\t\tlog.Infof(\"following new host '%s' '%s'.\", host, sport)\n\t\t\tgo s.follow(s.config.followHost(), s.config.followPort(),\n\t\t\t\tint(s.followc.Load()))\n\t\t} else {\n\t\t\tlog.Infof(\"following no one\")\n\t\t}\n\t}\n\treturn OKMessage(msg, start), nil\n}\n\n// cmdReplConf is a command handler that sets replication configuration info\nfunc (s *Server) cmdReplConf(msg *Message, client *Client) (res resp.Value, err error) {\n\tstart := time.Now()\n\tvs := msg.Args[1:]\n\tvar ok bool\n\tvar cmd, val string\n\n\t// Parse the message\n\tif vs, cmd, ok = tokenval(vs); !ok || cmd == \"\" {\n\t\treturn NOMessage, errInvalidNumberOfArguments\n\t}\n\tif _, val, ok = tokenval(vs); !ok || val == \"\" {\n\t\treturn NOMessage, errInvalidNumberOfArguments\n\t}\n\n\t// Switch on the command received\n\tswitch cmd {\n\tcase \"listening-port\":\n\t\t// Parse the port as an integer\n\t\tport, err := strconv.Atoi(val)\n\t\tif err != nil {\n\t\t\treturn NOMessage, errInvalidArgument(val)\n\t\t}\n\n\t\t// Apply the replication port to the client and return\n\t\ts.connsmu.RLock()\n\t\tdefer s.connsmu.RUnlock()\n\t\tfor _, c := range s.conns {\n\t\t\tif c.remoteAddr == client.remoteAddr {\n\t\t\t\tc.mu.Lock()\n\t\t\t\tc.replPort = port\n\t\t\t\tc.mu.Unlock()\n\t\t\t\treturn OKMessage(msg, start), nil\n\t\t\t}\n\t\t}\n\tcase \"ip-address\":\n\t\t// Apply the replication ip to the client and return\n\t\ts.connsmu.RLock()\n\t\tdefer s.connsmu.RUnlock()\n\t\tfor _, c := range s.conns {\n\t\t\tif c.remoteAddr == client.remoteAddr {\n\t\t\t\tc.mu.Lock()\n\t\t\t\tc.replAddr = val\n\t\t\t\tc.mu.Unlock()\n\t\t\t\treturn OKMessage(msg, start), nil\n\t\t\t}\n\t\t}\n\t}\n\treturn NOMessage, fmt.Errorf(\"cannot find follower\")\n}\n\nfunc doServer(conn *RESPConn) (map[string]string, error) {\n\tv, err := conn.Do(\"server\")\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tif v.Error() != nil {\n\t\treturn nil, v.Error()\n\t}\n\tarr := v.Array()\n\tm := make(map[string]string)\n\tfor i := 0; i < len(arr)/2; i++ {\n\t\tm[arr[i*2+0].String()] = arr[i*2+1].String()\n\t}\n\treturn m, err\n}\n\nfunc (s *Server) followHandleCommand(args []string, followc int, w io.Writer) (int, error) {\n\ts.mu.Lock()\n\tdefer s.mu.Unlock()\n\tif int(s.followc.Load()) != followc {\n\t\treturn s.aofsz, errNoLongerFollowing\n\t}\n\tmsg := &Message{Args: args}\n\t_, d, err := s.command(msg, nil)\n\tif err != nil {\n\t\tif commandErrIsFatal(err) {\n\t\t\treturn s.aofsz, err\n\t\t}\n\t}\n\tswitch msg.Command() {\n\tcase \"publish\":\n\t\t// Avoid writing these commands to the AOF\n\tdefault:\n\t\tif err := s.writeAOF(args, &d); err != nil {\n\t\t\treturn s.aofsz, err\n\t\t}\n\t}\n\tif len(s.aofbuf) > 10240 {\n\t\ts.flushAOF(false)\n\t}\n\treturn s.aofsz, nil\n}\n\nfunc (s *Server) followDoLeaderAuth(conn *RESPConn, auth string) error {\n\tv, err := conn.Do(\"auth\", auth)\n\tif err != nil {\n\t\treturn err\n\t}\n\tif v.Error() != nil {\n\t\treturn v.Error()\n\t}\n\tif v.String() != \"OK\" {\n\t\treturn errors.New(\"cannot follow: auth no ok\")\n\t}\n\treturn nil\n}\n\n// bit flags for the fcupflags server field\nconst (\n\tbitCaughtUpOnce int32 = 1 // follower caught up at least once in the past\n\tbitCaughtUp int32 = 2 // follower is fully caught up to leader\n)\n\nfunc (s *Server) setCaughtUp(caughtUp bool) {\n\tvar flags int32\n\tif caughtUp {\n\t\tflags = bitCaughtUp | bitCaughtUpOnce\n\t} else {\n\t\tflags = s.fcupflags.Load() & bitCaughtUpOnce\n\t}\n\ts.fcupflags.Store(flags)\n}\n\nfunc (s *Server) caughtUp() bool {\n\treturn (s.fcupflags.Load() & bitCaughtUp) == bitCaughtUp\n}\n\nfunc (s *Server) caughtUpOnce() bool {\n\treturn (s.fcupflags.Load() & bitCaughtUpOnce) == bitCaughtUpOnce\n}\n\nfunc (s *Server) followStep(host string, port int, followc int) error {\n\tif int(s.followc.Load()) != followc {\n\t\treturn errNoLongerFollowing\n\t}\n\ts.mu.Lock()\n\ts.faofsz = 0\n\ts.setCaughtUp(false)\n\tauth := s.config.leaderAuth()\n\ts.mu.Unlock()\n\taddr := fmt.Sprintf(\"%s:%d\", host, port)\n\n\t// check if we are following self\n\tconn, err := DialTimeout(addr, time.Second*2)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"cannot follow: %v\", err)\n\t}\n\tdefer conn.Close()\n\tif auth != \"\" {\n\t\tif err := s.followDoLeaderAuth(conn, auth); err != nil {\n\t\t\treturn fmt.Errorf(\"cannot follow: %v\", err)\n\t\t}\n\t}\n\tm, err := doServer(conn)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"cannot follow: %v\", err)\n\t}\n\n\tif m[\"id\"] == \"\" {\n\t\treturn fmt.Errorf(\"cannot follow: invalid id\")\n\t}\n\tif m[\"id\"] == s.config.serverID() {\n\t\treturn fmt.Errorf(\"cannot follow self\")\n\t}\n\tif m[\"following\"] != \"\" {\n\t\treturn fmt.Errorf(\"cannot follow a follower\")\n\t}\n\n\t// verify checksum\n\tpos, err := s.followCheckSome(addr, followc, auth)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// Send the replication port to the leader\n\tp := s.config.announcePort()\n\tif p == 0 {\n\t\tp = s.port\n\t}\n\tv, err := conn.Do(\"replconf\", \"listening-port\", p)\n\tif err != nil {\n\t\treturn err\n\t}\n\tif v.Error() != nil {\n\t\treturn v.Error()\n\t}\n\tif v.String() != \"OK\" {\n\t\treturn errors.New(\"invalid response to replconf request\")\n\t}\n\n\t// Send the replication ip to the leader\n\tip := s.config.announceIP()\n\tif ip != \"\" {\n\t\tv, err := conn.Do(\"replconf\", \"ip-address\", ip)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tif v.Error() != nil {\n\t\t\treturn v.Error()\n\t\t}\n\t\tif v.String() != \"OK\" {\n\t\t\treturn errors.New(\"invalid response to replconf request\")\n\t\t}\n\t}\n\tif s.opts.ShowDebugMessages {\n\t\tlog.Debug(\"follow:\", addr, \":replconf\")\n\t}\n\n\tv, err = conn.Do(\"aof\", pos)\n\tif err != nil {\n\t\treturn err\n\t}\n\tif v.Error() != nil {\n\t\treturn v.Error()\n\t}\n\tif v.String() != \"OK\" {\n\t\treturn errors.New(\"invalid response to aof live request\")\n\t}\n\tif s.opts.ShowDebugMessages {\n\t\tlog.Debug(\"follow:\", addr, \":read aof\")\n\t}\n\n\taofSize, err := strconv.ParseInt(m[\"aof_size\"], 10, 64)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\ts.mu.Lock()\n\ts.faofsz = int(aofSize)\n\ts.mu.Unlock()\n\n\tcaughtUp := pos >= aofSize\n\tif caughtUp {\n\t\ts.setCaughtUp(true)\n\t\tlog.Info(\"caught up\")\n\t}\n\n\tnullw := io.Discard\n\tfor {\n\t\tv, telnet, _, err := conn.rd.ReadMultiBulk()\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tvals := v.Array()\n\t\tif telnet || v.Type() != resp.Array {\n\t\t\treturn errors.New(\"invalid multibulk\")\n\t\t}\n\t\tsvals := make([]string, len(vals))\n\t\tfor i := 0; i < len(vals); i++ {\n\t\t\tsvals[i] = vals[i].String()\n\t\t}\n\n\t\taofsz, err := s.followHandleCommand(svals, followc, nullw)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\ts.mu.Lock()\n\t\ts.faofsz = aofsz\n\t\ts.mu.Unlock()\n\t\tif !caughtUp {\n\t\t\tif aofsz >= int(aofSize) {\n\t\t\t\tcaughtUp = true\n\t\t\t\ts.mu.Lock()\n\t\t\t\ts.flushAOF(false)\n\t\t\t\ts.setCaughtUp(true)\n\t\t\t\ts.mu.Unlock()\n\t\t\t\tlog.Info(\"caught up\")\n\t\t\t}\n\t\t}\n\n\t}\n}\n\nfunc (s *Server) follow(host string, port int, followc int) {\n\tfor {\n\t\terr := s.followStep(host, port, followc)\n\t\tif err == errNoLongerFollowing {\n\t\t\treturn\n\t\t}\n\t\tif err != nil && err != io.EOF {\n\t\t\tlog.Error(\"follow: \" + err.Error())\n\t\t}\n\t\ttime.Sleep(time.Second)\n\t}\n}\n" }, { "path": "internal/server/group.go", "content": "package server\n\nimport (\n\t\"github.com/tidwall/btree\"\n)\n\nfunc byGroupHook(va, vb interface{}) bool {\n\ta, b := va.(*groupItem), vb.(*groupItem)\n\tif a.hookName < b.hookName {\n\t\treturn true\n\t}\n\tif a.hookName > b.hookName {\n\t\treturn false\n\t}\n\tif a.colKey < b.colKey {\n\t\treturn true\n\t}\n\tif a.colKey > b.colKey {\n\t\treturn false\n\t}\n\treturn a.objID < b.objID\n}\n\nfunc byGroupObject(va, vb interface{}) bool {\n\ta, b := va.(*groupItem), vb.(*groupItem)\n\tif a.colKey < b.colKey {\n\t\treturn true\n\t}\n\tif a.colKey > b.colKey {\n\t\treturn false\n\t}\n\tif a.objID < b.objID {\n\t\treturn true\n\t}\n\tif a.objID > b.objID {\n\t\treturn false\n\t}\n\treturn a.hookName < b.hookName\n}\n\ntype groupItem struct {\n\thookName string\n\tcolKey string\n\tobjID string\n\tgroupID string\n}\n\nfunc newGroupItem(hookName, colKey, objID string) *groupItem {\n\tgroupID := bsonID()\n\tg := &groupItem{}\n\t// create a single string allocation\n\tustr := hookName + colKey + objID + groupID\n\tvar pos int\n\tg.hookName = ustr[pos : pos+len(hookName)]\n\tpos += len(hookName)\n\tg.colKey = ustr[pos : pos+len(colKey)]\n\tpos += len(colKey)\n\tg.objID = ustr[pos : pos+len(objID)]\n\tpos += len(objID)\n\tg.groupID = ustr[pos : pos+len(groupID)]\n\tpos += len(groupID)\n\treturn g\n}\n\nfunc (s *Server) groupConnect(hookName, colKey, objID string) (groupID string) {\n\tg := newGroupItem(hookName, colKey, objID)\n\ts.groupHooks.Set(g)\n\ts.groupObjects.Set(g)\n\treturn g.groupID\n}\n\nfunc (s *Server) groupDisconnect(hookName, colKey, objID string) {\n\tg := &groupItem{\n\t\thookName: hookName,\n\t\tcolKey: colKey,\n\t\tobjID: objID,\n\t}\n\ts.groupHooks.Delete(g)\n\ts.groupObjects.Delete(g)\n}\n\nfunc (s *Server) groupGet(hookName, colKey, objID string) (groupID string) {\n\tv := s.groupHooks.Get(&groupItem{\n\t\thookName: hookName,\n\t\tcolKey: colKey,\n\t\tobjID: objID,\n\t})\n\tif v != nil {\n\t\treturn v.(*groupItem).groupID\n\t}\n\treturn \"\"\n}\n\nfunc deleteGroups(s *Server, groups []*groupItem) {\n\tvar hhint btree.PathHint\n\tvar ohint btree.PathHint\n\tfor _, g := range groups {\n\t\ts.groupHooks.DeleteHint(g, &hhint)\n\t\ts.groupObjects.DeleteHint(g, &ohint)\n\t}\n}\n\n// groupDisconnectObject disconnects all hooks from provide object\nfunc (s *Server) groupDisconnectObject(colKey, objID string) {\n\tvar groups []*groupItem\n\ts.groupObjects.Ascend(&groupItem{colKey: colKey, objID: objID},\n\t\tfunc(v interface{}) bool {\n\t\t\tg := v.(*groupItem)\n\t\t\tif g.colKey != colKey || g.objID != objID {\n\t\t\t\treturn false\n\t\t\t}\n\t\t\tgroups = append(groups, g)\n\t\t\treturn true\n\t\t},\n\t)\n\tdeleteGroups(s, groups)\n}\n\n// groupDisconnectCollection disconnects all hooks from objects in provided\n// collection.\nfunc (s *Server) groupDisconnectCollection(colKey string) {\n\tvar groups []*groupItem\n\ts.groupObjects.Ascend(&groupItem{colKey: colKey},\n\t\tfunc(v interface{}) bool {\n\t\t\tg := v.(*groupItem)\n\t\t\tif g.colKey != colKey {\n\t\t\t\treturn false\n\t\t\t}\n\t\t\tgroups = append(groups, g)\n\t\t\treturn true\n\t\t},\n\t)\n\tdeleteGroups(s, groups)\n}\n\n// groupDisconnectHook disconnects all objects from provided hook.\nfunc (s *Server) groupDisconnectHook(hookName string) {\n\tvar groups []*groupItem\n\ts.groupHooks.Ascend(&groupItem{hookName: hookName},\n\t\tfunc(v interface{}) bool {\n\t\t\tg := v.(*groupItem)\n\t\t\tif g.hookName != hookName {\n\t\t\t\treturn false\n\t\t\t}\n\t\t\tgroups = append(groups, g)\n\t\t\treturn true\n\t\t},\n\t)\n\tdeleteGroups(s, groups)\n}\n" }, { "path": "internal/server/hooks.go", "content": "package server\n\nimport (\n\t\"bytes\"\n\t\"errors\"\n\t\"sort\"\n\t\"strconv\"\n\t\"strings\"\n\t\"sync\"\n\t\"sync/atomic\"\n\t\"time\"\n\n\t\"github.com/tidwall/buntdb\"\n\t\"github.com/tidwall/gjson\"\n\t\"github.com/tidwall/resp\"\n\t\"github.com/tidwall/tile38/internal/endpoint\"\n\t\"github.com/tidwall/tile38/internal/glob\"\n\t\"github.com/tidwall/tile38/internal/log\"\n)\n\nvar hookLogSetDefaults = &buntdb.SetOptions{\n\tExpires: true, // automatically delete after 30 seconds\n\tTTL: time.Second * 30,\n}\n\nfunc byHookName(a, b interface{}) bool {\n\treturn a.(*Hook).Name < b.(*Hook).Name\n}\n\nfunc (s *Server) cmdSetHook(msg *Message) (\n\tres resp.Value, d commandDetails, err error,\n) {\n\tchannel := msg.Command() == \"setchan\"\n\tstart := time.Now()\n\tvs := msg.Args[1:]\n\tvar name, urls, cmd string\n\tvar ok bool\n\tif vs, name, ok = tokenval(vs); !ok || name == \"\" {\n\t\treturn NOMessage, d, errInvalidNumberOfArguments\n\t}\n\tvar endpoints []string\n\tif channel {\n\t\tendpoints = []string{\"local://\" + name}\n\t} else {\n\t\tif vs, urls, ok = tokenval(vs); !ok || urls == \"\" {\n\t\t\treturn NOMessage, d, errInvalidNumberOfArguments\n\t\t}\n\t\tfor _, url := range strings.Split(urls, \",\") {\n\t\t\turl = strings.TrimSpace(url)\n\t\t\terr := s.epc.Validate(url)\n\t\t\tif err != nil {\n\t\t\t\tlog.Errorf(\"sethook: %v\", err)\n\t\t\t\treturn resp.SimpleStringValue(\"\"), d, errInvalidArgument(url)\n\t\t\t}\n\t\t\tendpoints = append(endpoints, url)\n\t\t}\n\t}\n\tvar commandvs []string\n\tvar cmdlc string\n\tvar types map[string]bool\n\tvar expires float64\n\tvar expiresSet bool\n\tmetaMap := make(map[string]string)\n\tfor {\n\t\tcommandvs = vs\n\t\tif vs, cmd, ok = tokenval(vs); !ok || cmd == \"\" {\n\t\t\treturn NOMessage, d, errInvalidNumberOfArguments\n\t\t}\n\t\tcmdlc = strings.ToLower(cmd)\n\t\tswitch cmdlc {\n\t\tdefault:\n\t\t\treturn NOMessage, d, errInvalidArgument(cmd)\n\t\tcase \"meta\":\n\t\t\tvar metakey string\n\t\t\tvar metaval string\n\t\t\tif vs, metakey, ok = tokenval(vs); !ok || metakey == \"\" {\n\t\t\t\treturn NOMessage, d, errInvalidNumberOfArguments\n\t\t\t}\n\t\t\tif vs, metaval, ok = tokenval(vs); !ok || metaval == \"\" {\n\t\t\t\treturn NOMessage, d, errInvalidNumberOfArguments\n\t\t\t}\n\t\t\tmetaMap[metakey] = metaval\n\t\t\tcontinue\n\t\tcase \"ex\":\n\t\t\tvar s string\n\t\t\tif vs, s, ok = tokenval(vs); !ok || s == \"\" {\n\t\t\t\treturn NOMessage, d, errInvalidNumberOfArguments\n\t\t\t}\n\t\t\tv, err := strconv.ParseFloat(s, 64)\n\t\t\tif err != nil {\n\t\t\t\treturn NOMessage, d, errInvalidArgument(s)\n\t\t\t}\n\t\t\texpires = v\n\t\t\texpiresSet = true\n\t\t\tcontinue\n\t\tcase \"nearby\":\n\t\t\ttypes = nearbyTypes\n\t\tcase \"within\", \"intersects\":\n\t\t\ttypes = withinOrIntersectsTypes\n\t\t}\n\t\tbreak\n\t}\n\targs, err := s.cmdSearchArgs(true, cmdlc, vs, types)\n\tif args.usingLua() {\n\t\tdefer args.Close()\n\t}\n\tif err != nil {\n\t\treturn NOMessage, d, err\n\t}\n\tif !args.fence {\n\t\treturn NOMessage, d, errors.New(\"missing FENCE argument\")\n\t}\n\targs.cmd = cmdlc\n\tcmsg := &Message{}\n\t*cmsg = *msg\n\tcmsg.Args = make([]string, len(commandvs))\n\tcopy(cmsg.Args, commandvs)\n\tmetas := make([]FenceMeta, 0, len(metaMap))\n\tfor key, val := range metaMap {\n\t\tmetas = append(metas, FenceMeta{key, val})\n\t}\n\tsort.Sort(hookMetaByName(metas))\n\n\thook := &Hook{\n\t\tKey: args.key,\n\t\tName: name,\n\t\tEndpoints: endpoints,\n\t\tFence: &args,\n\t\tMessage: cmsg,\n\t\tepm: s.epc,\n\t\tMetas: metas,\n\t\tchannel: channel,\n\t\tcond: sync.NewCond(&sync.Mutex{}),\n\t\tcounter: &s.statsTotalMsgsSent,\n\t}\n\tif expiresSet {\n\t\thook.expires =\n\t\t\ttime.Now().Add(time.Duration(expires * float64(time.Second)))\n\t}\n\tif !channel {\n\t\thook.db = s.qdb\n\t}\n\tvar wr bytes.Buffer\n\thook.ScanWriter, err = s.newScanWriter(\n\t\t&wr, cmsg, args.key, args.output, args.precision, args.globs, false,\n\t\targs.cursor, args.limit, args.wheres, args.whereins, args.whereevals,\n\t\targs.nofields, args.mvt, args.tileX, args.tileY, args.tileZ)\n\tif err != nil {\n\n\t\treturn NOMessage, d, err\n\t}\n\tprevHook, _ := s.hooks.Get(&Hook{Name: name}).(*Hook)\n\tif prevHook != nil {\n\t\tif prevHook.channel != channel {\n\t\t\treturn NOMessage, d,\n\t\t\t\terrors.New(\"hooks and channels cannot share the same name\")\n\t\t}\n\t\tif prevHook.Equals(hook) {\n\t\t\t// it was a match so we do nothing. But let's signal just\n\t\t\t// for good measure.\n\t\t\tprevHook.Signal()\n\t\t\tif !hook.expires.IsZero() {\n\t\t\t\ts.hookExpires.Set(hook)\n\t\t\t}\n\t\t\tswitch msg.OutputType {\n\t\t\tcase JSON:\n\t\t\t\treturn OKMessage(msg, start), d, nil\n\t\t\tcase RESP:\n\t\t\t\treturn resp.IntegerValue(0), d, nil\n\t\t\t}\n\t\t}\n\t\tprevHook.Close()\n\t\ts.hooks.Delete(prevHook)\n\t\ts.hooksOut.Delete(prevHook)\n\t\tif !prevHook.expires.IsZero() {\n\t\t\ts.hookExpires.Delete(prevHook)\n\t\t}\n\t\ts.groupDisconnectHook(name)\n\t}\n\n\td.updated = true\n\td.timestamp = time.Now()\n\n\ts.hooks.Set(hook)\n\tif hook.Fence.detect == nil || hook.Fence.detect[\"outside\"] {\n\t\ts.hooksOut.Set(hook)\n\t}\n\n\t// remove previous hook from spatial index\n\tif prevHook != nil && prevHook.Fence != nil && prevHook.Fence.obj != nil {\n\t\trect := prevHook.Fence.obj.Rect()\n\t\ts.hookTree.Delete(\n\t\t\t[2]float64{rect.Min.X, rect.Min.Y},\n\t\t\t[2]float64{rect.Max.X, rect.Max.Y},\n\t\t\tprevHook)\n\t\tif prevHook.Fence.detect[\"cross\"] {\n\t\t\ts.hookCross.Delete(\n\t\t\t\t[2]float64{rect.Min.X, rect.Min.Y},\n\t\t\t\t[2]float64{rect.Max.X, rect.Max.Y},\n\t\t\t\tprevHook)\n\t\t}\n\t}\n\t// add hook to spatial index\n\tif hook != nil && hook.Fence != nil && hook.Fence.obj != nil {\n\t\trect := hook.Fence.obj.Rect()\n\t\ts.hookTree.Insert(\n\t\t\t[2]float64{rect.Min.X, rect.Min.Y},\n\t\t\t[2]float64{rect.Max.X, rect.Max.Y},\n\t\t\thook)\n\t\tif hook.Fence.detect[\"cross\"] {\n\t\t\ts.hookCross.Insert(\n\t\t\t\t[2]float64{rect.Min.X, rect.Min.Y},\n\t\t\t\t[2]float64{rect.Max.X, rect.Max.Y},\n\t\t\t\thook)\n\t\t}\n\t}\n\n\thook.Open() // Opens a goroutine to notify the hook\n\tif !hook.expires.IsZero() {\n\t\ts.hookExpires.Set(hook)\n\t}\n\tswitch msg.OutputType {\n\tcase JSON:\n\t\treturn OKMessage(msg, start), d, nil\n\tcase RESP:\n\t\treturn resp.IntegerValue(1), d, nil\n\t}\n\treturn NOMessage, d, nil\n}\n\nfunc byHookExpires(a, b interface{}) bool {\n\tha := a.(*Hook)\n\thb := b.(*Hook)\n\tif ha.expires.Before(hb.expires) {\n\t\treturn true\n\t}\n\tif ha.expires.After(hb.expires) {\n\t\treturn false\n\t}\n\treturn ha.Name < hb.Name\n}\n\nfunc (s *Server) cmdDELHOOKop(name string, channel bool) (updated bool) {\n\thook, _ := s.hooks.Get(&Hook{Name: name}).(*Hook)\n\tif hook == nil || hook.channel != channel {\n\t\treturn false\n\t}\n\thook.Close()\n\t// remove hook from maps\n\ts.hooks.Delete(hook)\n\ts.hooksOut.Delete(hook)\n\tif !hook.expires.IsZero() {\n\t\ts.hookExpires.Delete(hook)\n\t}\n\t// remove any hook / object connections\n\ts.groupDisconnectHook(hook.Name)\n\t// remove hook from spatial index\n\tif hook.Fence != nil && hook.Fence.obj != nil {\n\t\trect := hook.Fence.obj.Rect()\n\t\ts.hookTree.Delete(\n\t\t\t[2]float64{rect.Min.X, rect.Min.Y},\n\t\t\t[2]float64{rect.Max.X, rect.Max.Y},\n\t\t\thook)\n\t\tif hook.Fence.detect[\"cross\"] {\n\t\t\ts.hookCross.Delete(\n\t\t\t\t[2]float64{rect.Min.X, rect.Min.Y},\n\t\t\t\t[2]float64{rect.Max.X, rect.Max.Y},\n\t\t\t\thook)\n\t\t}\n\t}\n\treturn true\n}\n\nfunc (s *Server) cmdDelHook(msg *Message) (\n\tres resp.Value, d commandDetails, err error,\n) {\n\tchannel := msg.Command() == \"delchan\"\n\tstart := time.Now()\n\tvs := msg.Args[1:]\n\n\tvar name string\n\tvar ok bool\n\tif vs, name, ok = tokenval(vs); !ok || name == \"\" {\n\t\treturn NOMessage, d, errInvalidNumberOfArguments\n\t}\n\tif len(vs) != 0 {\n\t\treturn NOMessage, d, errInvalidNumberOfArguments\n\t}\n\n\td.updated = s.cmdDELHOOKop(name, channel)\n\td.timestamp = time.Now()\n\n\tswitch msg.OutputType {\n\tcase JSON:\n\t\treturn OKMessage(msg, start), d, nil\n\tcase RESP:\n\t\tif d.updated {\n\t\t\treturn resp.IntegerValue(1), d, nil\n\t\t}\n\t\treturn resp.IntegerValue(0), d, nil\n\t}\n\treturn\n}\n\nfunc (s *Server) cmdPDelHook(msg *Message) (\n\tres resp.Value, d commandDetails, err error,\n) {\n\tchannel := msg.Command() == \"pdelchan\"\n\tstart := time.Now()\n\tvs := msg.Args[1:]\n\n\tvar pattern string\n\tvar ok bool\n\tif vs, pattern, ok = tokenval(vs); !ok || pattern == \"\" {\n\t\treturn NOMessage, d, errInvalidNumberOfArguments\n\t}\n\tif len(vs) != 0 {\n\t\treturn NOMessage, d, errInvalidNumberOfArguments\n\t}\n\n\tcount := 0\n\tvar hooks []*Hook\n\ts.forEachHookByPattern(pattern, channel, func(hook *Hook) bool {\n\t\thooks = append(hooks, hook)\n\t\treturn true\n\t})\n\n\tfor _, hook := range hooks {\n\t\tif hook.channel != channel {\n\t\t\tcontinue\n\t\t}\n\t\ts.cmdDELHOOKop(hook.Name, channel)\n\t\td.updated = true\n\t\tcount++\n\t}\n\td.timestamp = time.Now()\n\n\tswitch msg.OutputType {\n\tcase JSON:\n\t\treturn OKMessage(msg, start), d, nil\n\tcase RESP:\n\t\treturn resp.IntegerValue(count), d, nil\n\t}\n\treturn\n}\n\nfunc (s *Server) forEachHookByPattern(\n\tpattern string, channel bool, iter func(hook *Hook) bool,\n) {\n\tg := glob.Parse(pattern, false)\n\thasUpperLimit := g.Limits[1] != \"\"\n\ts.hooks.Ascend(&Hook{Name: g.Limits[0]}, func(v interface{}) bool {\n\t\thook := v.(*Hook)\n\t\tif hasUpperLimit && hook.Name > g.Limits[1] {\n\t\t\treturn false\n\t\t}\n\t\tif hook.channel == channel {\n\t\t\tmatch, _ := glob.Match(pattern, hook.Name)\n\t\t\tif match {\n\t\t\t\treturn iter(hook)\n\t\t\t}\n\t\t}\n\t\treturn true\n\t})\n}\n\nfunc (s *Server) cmdHooks(msg *Message) (\n\tres resp.Value, err error,\n) {\n\tchannel := msg.Command() == \"chans\"\n\tstart := time.Now()\n\tvs := msg.Args[1:]\n\n\tvar pattern string\n\tvar ok bool\n\n\tif vs, pattern, ok = tokenval(vs); !ok || pattern == \"\" {\n\t\treturn NOMessage, errInvalidNumberOfArguments\n\t}\n\tif len(vs) != 0 {\n\t\treturn NOMessage, errInvalidNumberOfArguments\n\t}\n\n\tswitch msg.OutputType {\n\tcase JSON:\n\t\tbuf := &bytes.Buffer{}\n\t\tbuf.WriteString(`{\"ok\":true,`)\n\t\tif channel {\n\t\t\tbuf.WriteString(`\"chans\":[`)\n\t\t} else {\n\t\t\tbuf.WriteString(`\"hooks\":[`)\n\t\t}\n\t\tvar i int\n\t\ts.forEachHookByPattern(pattern, channel, func(hook *Hook) bool {\n\t\t\tvar ttl = -1\n\t\t\tif !hook.expires.IsZero() {\n\t\t\t\tttl = int(hook.expires.Sub(start).Seconds())\n\t\t\t\tif ttl < 0 {\n\t\t\t\t\tttl = 0\n\t\t\t\t}\n\t\t\t}\n\t\t\tif i > 0 {\n\t\t\t\tbuf.WriteByte(',')\n\t\t\t}\n\t\t\tbuf.WriteString(`{`)\n\t\t\tbuf.WriteString(`\"name\":` + jsonString(hook.Name))\n\t\t\tbuf.WriteString(`,\"key\":` + jsonString(hook.Key))\n\t\t\tbuf.WriteString(`,\"ttl\":` + strconv.Itoa(ttl))\n\t\t\tif !channel {\n\t\t\t\tbuf.WriteString(`,\"endpoints\":[`)\n\t\t\t\tfor i, endpoint := range hook.Endpoints {\n\t\t\t\t\tif i > 0 {\n\t\t\t\t\t\tbuf.WriteByte(',')\n\t\t\t\t\t}\n\t\t\t\t\tbuf.WriteString(jsonString(endpoint))\n\t\t\t\t}\n\t\t\t\tbuf.WriteString(`]`)\n\t\t\t}\n\t\t\tbuf.WriteString(`,\"command\":[`)\n\t\t\tfor i, v := range hook.Message.Args {\n\t\t\t\tif i > 0 {\n\t\t\t\t\tbuf.WriteString(`,`)\n\t\t\t\t}\n\t\t\t\tbuf.WriteString(jsonString(v))\n\t\t\t}\n\t\t\tbuf.WriteString(`],\"meta\":{`)\n\t\t\tfor i, meta := range hook.Metas {\n\t\t\t\tif i > 0 {\n\t\t\t\t\tbuf.WriteString(`,`)\n\t\t\t\t}\n\t\t\t\tbuf.WriteString(jsonString(meta.Name))\n\t\t\t\tbuf.WriteString(`:`)\n\t\t\t\tbuf.WriteString(jsonString(meta.Value))\n\t\t\t}\n\t\t\tbuf.WriteString(`}}`)\n\t\t\ti++\n\t\t\treturn true\n\t\t})\n\t\tbuf.WriteString(`],\"elapsed\":\"` +\n\t\t\ttime.Since(start).String() + \"\\\"}\")\n\t\treturn resp.StringValue(buf.String()), nil\n\tcase RESP:\n\t\tvar vals []resp.Value\n\t\ts.forEachHookByPattern(pattern, channel, func(hook *Hook) bool {\n\t\t\tvar hvals []resp.Value\n\t\t\thvals = append(hvals, resp.StringValue(hook.Name))\n\t\t\thvals = append(hvals, resp.StringValue(hook.Key))\n\t\t\tvar evals []resp.Value\n\t\t\tfor _, endpoint := range hook.Endpoints {\n\t\t\t\tevals = append(evals, resp.StringValue(endpoint))\n\t\t\t}\n\t\t\thvals = append(hvals, resp.ArrayValue(evals))\n\t\t\tavals := make([]resp.Value, len(hook.Message.Args))\n\t\t\tfor i := 0; i < len(hook.Message.Args); i++ {\n\t\t\t\tavals[i] = resp.StringValue(hook.Message.Args[i])\n\t\t\t}\n\t\t\thvals = append(hvals, resp.ArrayValue(avals))\n\t\t\tvar metas []resp.Value\n\t\t\tfor _, meta := range hook.Metas {\n\t\t\t\tmetas = append(metas, resp.StringValue(meta.Name))\n\t\t\t\tmetas = append(metas, resp.StringValue(meta.Value))\n\t\t\t}\n\t\t\thvals = append(hvals, resp.ArrayValue(metas))\n\t\t\tvals = append(vals, resp.ArrayValue(hvals))\n\t\t\treturn true\n\t\t})\n\t\treturn resp.ArrayValue(vals), nil\n\t}\n\treturn resp.SimpleStringValue(\"\"), nil\n}\n\n// Hook represents a hook.\ntype Hook struct {\n\tcond *sync.Cond\n\tKey string\n\tName string\n\tEndpoints []string\n\tMessage *Message\n\tFence *liveFenceSwitches\n\tScanWriter *scanWriter\n\tMetas []FenceMeta\n\tdb *buntdb.DB\n\tchannel bool\n\tclosed bool\n\topened bool\n\tquery string\n\tepm *endpoint.Manager\n\texpires time.Time\n\tcounter *atomic.Int64 // counter that grows when a message was sent\n\tsig int\n}\n\n// Expires returns when the hook expires. Required by the expire.Item interface.\nfunc (h *Hook) Expires() time.Time {\n\treturn h.expires\n}\n\n// Equals returns true if two hooks are equal\nfunc (h *Hook) Equals(hook *Hook) bool {\n\tif h.Key != hook.Key ||\n\t\th.Name != hook.Name ||\n\t\tlen(h.Endpoints) != len(hook.Endpoints) ||\n\t\tlen(h.Metas) != len(hook.Metas) {\n\t\treturn false\n\t}\n\tif !h.expires.Equal(hook.expires) {\n\t\treturn false\n\t}\n\tfor i, endpoint := range h.Endpoints {\n\t\tif endpoint != hook.Endpoints[i] {\n\t\t\treturn false\n\t\t}\n\t}\n\tfor i, meta := range h.Metas {\n\t\tif meta.Name != hook.Metas[i].Name ||\n\t\t\tmeta.Value != hook.Metas[i].Value {\n\t\t\treturn false\n\t\t}\n\t}\n\tif len(h.Message.Args) != len(hook.Message.Args) {\n\t\treturn false\n\t}\n\tfor i := 0; i < len(h.Message.Args); i++ {\n\t\tif h.Message.Args[i] != hook.Message.Args[i] {\n\t\t\treturn false\n\t\t}\n\t}\n\treturn true\n}\n\n// FenceMeta is a meta key/value pair for fences\ntype FenceMeta struct {\n\tName, Value string\n}\n\ntype hookMetaByName []FenceMeta\n\nfunc (arr hookMetaByName) Len() int {\n\treturn len(arr)\n}\n\nfunc (arr hookMetaByName) Less(a, b int) bool {\n\treturn arr[a].Name < arr[b].Name\n}\n\nfunc (arr hookMetaByName) Swap(a, b int) {\n\tarr[a], arr[b] = arr[b], arr[a]\n}\n\n// Open is called when a hook is first created. It calls the manager\n// function in a goroutine\nfunc (h *Hook) Open() {\n\tif h.channel {\n\t\t// nothing to open for channels\n\t\treturn\n\t}\n\th.cond.L.Lock()\n\tdefer h.cond.L.Unlock()\n\tif h.opened {\n\t\treturn\n\t}\n\th.opened = true\n\th.query = `{\"hook\":` + jsonString(h.Name) + `}`\n\tgo h.manager()\n}\n\n// Close closed the hook and stop the manager function\nfunc (h *Hook) Close() {\n\tif h.channel {\n\t\t// nothing to close for channels\n\t\treturn\n\t}\n\th.cond.L.Lock()\n\tdefer h.cond.L.Unlock()\n\tif h.closed {\n\t\treturn\n\t}\n\th.closed = true\n\th.cond.Broadcast()\n}\n\n// Signal can be called at any point to wake up the hook and\n// notify the manager that there may be something new in the queue.\nfunc (h *Hook) Signal() {\n\tif h.channel {\n\t\t// nothing to signal for channels\n\t\treturn\n\t}\n\th.cond.L.Lock()\n\th.sig++\n\th.cond.Broadcast()\n\th.cond.L.Unlock()\n}\n\n// the manager is a forever loop that calls proc whenever there's a signal.\n// it ends when the \"closed\" flag is set.\nfunc (h *Hook) manager() {\n\t// lock the hook to waiting on signals\n\th.cond.L.Lock()\n\tdefer h.cond.L.Unlock()\n\tvar sig int\n\tfor {\n\t\tif h.closed {\n\t\t\t// the hook has closed, end manager\n\t\t\treturn\n\t\t}\n\t\tsig = h.sig\n\t\t// unlock/logk the hook and send outgoing messages\n\t\tif !func() bool {\n\t\t\th.cond.L.Unlock()\n\t\t\tdefer h.cond.L.Lock()\n\t\t\treturn h.proc()\n\t\t}() {\n\t\t\t// a send failed, try again in a moment\n\t\t\ttime.Sleep(time.Second / 2)\n\t\t\tcontinue\n\t\t}\n\t\tif sig != h.sig {\n\t\t\t// there was another incoming signal\n\t\t\tcontinue\n\t\t}\n\t\t// wait on signal\n\t\th.cond.Wait()\n\t}\n}\n\n// proc processes queued hook logs.\n// returning true will indicate that all log entries have been\n// successfully handled.\nfunc (h *Hook) proc() (ok bool) {\n\tvar keys, vals []string\n\tvar ttls []time.Duration\n\tstart := time.Now()\n\terr := h.db.Update(func(tx *buntdb.Tx) error {\n\t\t// get keys and vals\n\t\terr := tx.AscendGreaterOrEqual(\"hooks\",\n\t\t\th.query, func(key, val string) bool {\n\t\t\t\tif strings.HasPrefix(key, hookLogPrefix) {\n\t\t\t\t\t// Verify this hooks name matches the one in the notif\n\t\t\t\t\tif h.Name == gjson.Get(val, \"hook\").String() {\n\t\t\t\t\t\tkeys = append(keys, key)\n\t\t\t\t\t\tvals = append(vals, val)\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t\treturn true\n\t\t\t},\n\t\t)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\n\t\t// delete the keys\n\t\tfor _, key := range keys {\n\t\t\tttl, err := tx.TTL(key)\n\t\t\tif err != nil {\n\t\t\t\tif err != buntdb.ErrNotFound {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\t\t\t}\n\t\t\tttls = append(ttls, ttl)\n\t\t\t_, err = tx.Delete(key)\n\t\t\tif err != nil {\n\t\t\t\tif err != buntdb.ErrNotFound {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t\treturn nil\n\t})\n\tif err != nil {\n\t\tlog.Error(err)\n\t\treturn false\n\t}\n\n\t// send each val. on failure reinsert that one and all of the following\n\tfor i, key := range keys {\n\t\tval := vals[i]\n\t\tidx := stringToUint64(key[len(hookLogPrefix):])\n\t\tvar sent bool\n\t\tfor _, endpoint := range h.Endpoints {\n\t\t\terr := h.epm.Send(endpoint, val)\n\t\t\tif err != nil {\n\t\t\t\tlog.Debugf(\"Endpoint connect/send error: %v: %v: %v\",\n\t\t\t\t\tidx, endpoint, err)\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tlog.Debugf(\"Endpoint send ok: %v: %v: %v\", idx, endpoint, err)\n\t\t\tsent = true\n\t\t\th.counter.Add(1)\n\t\t\tbreak\n\t\t}\n\t\tif !sent {\n\t\t\t// failed to send. try to reinsert the remaining.\n\t\t\t// if this fails we lose log entries.\n\t\t\tkeys = keys[i:]\n\t\t\tvals = vals[i:]\n\t\t\tttls = ttls[i:]\n\t\t\th.db.Update(func(tx *buntdb.Tx) error {\n\t\t\t\tfor i, key := range keys {\n\t\t\t\t\tval := vals[i]\n\t\t\t\t\tttl := ttls[i] - time.Since(start)\n\t\t\t\t\tif ttl > 0 {\n\t\t\t\t\t\topts := &buntdb.SetOptions{\n\t\t\t\t\t\t\tExpires: true,\n\t\t\t\t\t\t\tTTL: ttl,\n\t\t\t\t\t\t}\n\t\t\t\t\t\t_, _, err := tx.Set(key, val, opts)\n\t\t\t\t\t\tif err != nil {\n\t\t\t\t\t\t\treturn err\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t\treturn nil\n\t\t\t})\n\t\t\treturn false\n\t\t}\n\t}\n\treturn true\n}\n" }, { "path": "internal/server/json.go", "content": "package server\n\nimport (\n\t\"bytes\"\n\t\"encoding/json\"\n\t\"strconv\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/tidwall/geojson\"\n\t\"github.com/tidwall/gjson\"\n\t\"github.com/tidwall/resp\"\n\t\"github.com/tidwall/sjson\"\n\t\"github.com/tidwall/tile38/internal/collection\"\n\t\"github.com/tidwall/tile38/internal/field\"\n\t\"github.com/tidwall/tile38/internal/object\"\n)\n\nfunc appendJSONString(b []byte, s string) []byte {\n\tfor i := 0; i < len(s); i++ {\n\t\tif s[i] < ' ' || s[i] == '\\\\' || s[i] == '\"' || s[i] > 126 {\n\t\t\td, _ := json.Marshal(s)\n\t\t\treturn append(b, string(d)...)\n\t\t}\n\t}\n\tb = append(b, '\"')\n\tb = append(b, s...)\n\tb = append(b, '\"')\n\treturn b\n}\n\nfunc jsonString(s string) string {\n\tfor i := 0; i < len(s); i++ {\n\t\tif s[i] < ' ' || s[i] == '\\\\' || s[i] == '\"' || s[i] > 126 {\n\t\t\td, _ := json.Marshal(s)\n\t\t\treturn string(d)\n\t\t}\n\t}\n\tb := make([]byte, len(s)+2)\n\tb[0] = '\"'\n\tcopy(b[1:], s)\n\tb[len(b)-1] = '\"'\n\treturn string(b)\n}\n\nfunc isJSONNumber(data string) bool {\n\t// Returns true if the given string can be encoded as a JSON number value.\n\t// See:\n\t// https://json.org\n\t// http://www.ecma-international.org/publications/files/ECMA-ST/ECMA-404.pdf\n\tif data == \"\" {\n\t\treturn false\n\t}\n\ti := 0\n\t// sign\n\tif data[i] == '-' {\n\t\ti++\n\t}\n\tif i == len(data) {\n\t\treturn false\n\t}\n\t// int\n\tif data[i] == '0' {\n\t\ti++\n\t} else {\n\t\tfor ; i < len(data); i++ {\n\t\t\tif data[i] >= '0' && data[i] <= '9' {\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tbreak\n\t\t}\n\t}\n\t// frac\n\tif i == len(data) {\n\t\treturn true\n\t}\n\tif data[i] == '.' {\n\t\ti++\n\t\tif i == len(data) {\n\t\t\treturn false\n\t\t}\n\t\tif data[i] < '0' || data[i] > '9' {\n\t\t\treturn false\n\t\t}\n\t\ti++\n\t\tfor ; i < len(data); i++ {\n\t\t\tif data[i] >= '0' && data[i] <= '9' {\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tbreak\n\t\t}\n\t}\n\t// exp\n\tif i == len(data) {\n\t\treturn true\n\t}\n\tif data[i] == 'e' || data[i] == 'E' {\n\t\ti++\n\t\tif i == len(data) {\n\t\t\treturn false\n\t\t}\n\t\tif data[i] == '+' || data[i] == '-' {\n\t\t\ti++\n\t\t}\n\t\tif i == len(data) {\n\t\t\treturn false\n\t\t}\n\t\tif data[i] < '0' || data[i] > '9' {\n\t\t\treturn false\n\t\t}\n\t\ti++\n\t\tfor ; i < len(data); i++ {\n\t\t\tif data[i] >= '0' && data[i] <= '9' {\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tbreak\n\t\t}\n\t}\n\treturn i == len(data)\n}\n\nfunc appendJSONSimpleBounds(dst []byte, o geojson.Object) []byte {\n\tbbox := o.Rect()\n\tdst = append(dst, `{\"sw\":{\"lat\":`...)\n\tdst = strconv.AppendFloat(dst, bbox.Min.Y, 'f', -1, 64)\n\tdst = append(dst, `,\"lon\":`...)\n\tdst = strconv.AppendFloat(dst, bbox.Min.X, 'f', -1, 64)\n\tdst = append(dst, `},\"ne\":{\"lat\":`...)\n\tdst = strconv.AppendFloat(dst, bbox.Max.Y, 'f', -1, 64)\n\tdst = append(dst, `,\"lon\":`...)\n\tdst = strconv.AppendFloat(dst, bbox.Max.X, 'f', -1, 64)\n\tdst = append(dst, `}}`...)\n\treturn dst\n}\n\nfunc appendJSONSimplePoint(dst []byte, o geojson.Object) []byte {\n\tpoint := o.Center()\n\tz := extractZCoordinate(o)\n\tdst = append(dst, `{\"lat\":`...)\n\tdst = strconv.AppendFloat(dst, point.Y, 'f', -1, 64)\n\tdst = append(dst, `,\"lon\":`...)\n\tdst = strconv.AppendFloat(dst, point.X, 'f', -1, 64)\n\tif z != 0 {\n\t\tdst = append(dst, `,\"z\":`...)\n\t\tdst = strconv.AppendFloat(dst, z, 'f', -1, 64)\n\t}\n\tdst = append(dst, '}')\n\treturn dst\n}\n\nfunc appendJSONTimeFormat(b []byte, t time.Time) []byte {\n\tb = append(b, '\"')\n\tb = t.AppendFormat(b, \"2006-01-02T15:04:05.999999999Z07:00\")\n\tb = append(b, '\"')\n\treturn b\n}\n\nfunc jsonTimeFormat(t time.Time) string {\n\tvar b []byte\n\tb = appendJSONTimeFormat(b, t)\n\treturn string(b)\n}\n\nfunc (s *Server) cmdJget(msg *Message) (resp.Value, error) {\n\tstart := time.Now()\n\n\tif len(msg.Args) < 3 {\n\t\treturn NOMessage, errInvalidNumberOfArguments\n\t}\n\tif len(msg.Args) > 5 {\n\t\treturn NOMessage, errInvalidNumberOfArguments\n\t}\n\tkey := msg.Args[1]\n\tid := msg.Args[2]\n\tvar doget bool\n\tvar path string\n\tvar raw bool\n\tif len(msg.Args) > 3 {\n\t\tdoget = true\n\t\tpath = msg.Args[3]\n\t\tif len(msg.Args) == 5 {\n\t\t\tif strings.ToLower(msg.Args[4]) == \"raw\" {\n\t\t\t\traw = true\n\t\t\t} else {\n\t\t\t\treturn NOMessage, errInvalidArgument(msg.Args[4])\n\t\t\t}\n\t\t}\n\t}\n\tcol, _ := s.cols.Get(key)\n\tif col == nil {\n\t\tif msg.OutputType == RESP {\n\t\t\treturn resp.NullValue(), nil\n\t\t}\n\t\treturn NOMessage, errKeyNotFound\n\t}\n\to := col.Get(id)\n\tif o == nil {\n\t\tif msg.OutputType == RESP {\n\t\t\treturn resp.NullValue(), nil\n\t\t}\n\t\treturn NOMessage, errIDNotFound\n\t}\n\tvar res gjson.Result\n\tif doget {\n\t\tres = gjson.Get(o.Geo().String(), path)\n\t} else {\n\t\tres = gjson.Parse(o.Geo().String())\n\t}\n\tvar val string\n\tif raw {\n\t\tval = res.Raw\n\t} else {\n\t\tval = res.String()\n\t}\n\tvar buf bytes.Buffer\n\tif msg.OutputType == JSON {\n\t\tbuf.WriteString(`{\"ok\":true`)\n\t}\n\tswitch msg.OutputType {\n\tcase JSON:\n\t\tif res.Exists() {\n\t\t\tbuf.WriteString(`,\"value\":` + jsonString(val))\n\t\t}\n\t\tbuf.WriteString(`,\"elapsed\":\"` + time.Since(start).String() + \"\\\"}\")\n\t\treturn resp.StringValue(buf.String()), nil\n\tcase RESP:\n\t\tif !res.Exists() {\n\t\t\treturn resp.NullValue(), nil\n\t\t}\n\t\treturn resp.StringValue(val), nil\n\t}\n\treturn NOMessage, nil\n}\n\nfunc (s *Server) cmdJset(msg *Message) (res resp.Value, d commandDetails, err error) {\n\t// JSET key path value [RAW]\n\tstart := time.Now()\n\n\tvar raw, str bool\n\tswitch len(msg.Args) {\n\tdefault:\n\t\treturn NOMessage, d, errInvalidNumberOfArguments\n\tcase 5:\n\tcase 6:\n\t\tswitch strings.ToLower(msg.Args[5]) {\n\t\tdefault:\n\t\t\treturn NOMessage, d, errInvalidArgument(msg.Args[5])\n\t\tcase \"raw\":\n\t\t\traw = true\n\t\tcase \"str\":\n\t\t\tstr = true\n\t\t}\n\t}\n\n\tkey := msg.Args[1]\n\tid := msg.Args[2]\n\tpath := msg.Args[3]\n\tval := msg.Args[4]\n\tif !str && !raw {\n\t\tswitch val {\n\t\tdefault:\n\t\t\traw = isJSONNumber(val)\n\t\tcase \"true\", \"false\", \"null\":\n\t\t\traw = true\n\t\t}\n\t}\n\tcol, _ := s.cols.Get(key)\n\tvar createcol bool\n\tif col == nil {\n\t\tcol = collection.New()\n\t\tcreatecol = true\n\t}\n\tvar json string\n\tvar geoobj bool\n\tvar fields field.List\n\to := col.Get(id)\n\tif o != nil {\n\t\tgeoobj = objIsSpatial(o.Geo())\n\t\tjson = o.Geo().String()\n\t\tfields = o.Fields()\n\t}\n\tif raw {\n\t\t// set as raw block\n\t\tjson, err = sjson.SetRaw(json, path, val)\n\t} else {\n\t\t// set as a string\n\t\tjson, err = sjson.Set(json, path, val)\n\t}\n\tif err != nil {\n\t\treturn NOMessage, d, err\n\t}\n\n\tif geoobj {\n\t\tnmsg := *msg\n\t\tnmsg.Args = []string{\"SET\", key, id, \"OBJECT\", json}\n\t\t// SET key id OBJECT json\n\t\treturn s.cmdSET(&nmsg)\n\t}\n\tif createcol {\n\t\ts.cols.Set(key, col)\n\t}\n\tvar oobj geojson.Object = collection.String(json)\n\tobj := object.New(id, oobj, 0, fields)\n\tcol.Set(obj)\n\n\td.key = key\n\td.obj = obj\n\td.timestamp = time.Now()\n\td.updated = true\n\n\tswitch msg.OutputType {\n\tcase JSON:\n\t\tvar buf bytes.Buffer\n\t\tbuf.WriteString(`{\"ok\":true`)\n\t\tbuf.WriteString(`,\"elapsed\":\"` + time.Since(start).String() + \"\\\"}\")\n\t\treturn resp.StringValue(buf.String()), d, nil\n\tcase RESP:\n\t\treturn resp.SimpleStringValue(\"OK\"), d, nil\n\t}\n\treturn NOMessage, d, nil\n}\n\nfunc (s *Server) cmdJdel(msg *Message) (res resp.Value, d commandDetails, err error) {\n\tstart := time.Now()\n\n\tif len(msg.Args) != 4 {\n\t\treturn NOMessage, d, errInvalidNumberOfArguments\n\t}\n\tkey := msg.Args[1]\n\tid := msg.Args[2]\n\tpath := msg.Args[3]\n\n\tcol, _ := s.cols.Get(key)\n\tif col == nil {\n\t\tif msg.OutputType == RESP {\n\t\t\treturn resp.IntegerValue(0), d, nil\n\t\t}\n\t\treturn NOMessage, d, errKeyNotFound\n\t}\n\n\tvar json string\n\tvar geoobj bool\n\tvar fields field.List\n\to := col.Get(id)\n\tif o != nil {\n\t\tgeoobj = objIsSpatial(o.Geo())\n\t\tjson = o.Geo().String()\n\t\tfields = o.Fields()\n\t}\n\tnjson, err := sjson.Delete(json, path)\n\tif err != nil {\n\t\treturn NOMessage, d, err\n\t}\n\tif njson == json {\n\t\tswitch msg.OutputType {\n\t\tcase JSON:\n\t\t\treturn NOMessage, d, errPathNotFound\n\t\tcase RESP:\n\t\t\treturn resp.IntegerValue(0), d, nil\n\t\t}\n\t\treturn NOMessage, d, nil\n\t}\n\tjson = njson\n\tif geoobj {\n\t\tnmsg := *msg\n\t\tnmsg.Args = []string{\"SET\", key, id, \"OBJECT\", json}\n\t\t// SET key id OBJECT json\n\t\treturn s.cmdSET(&nmsg)\n\t}\n\n\tvar oobj geojson.Object = collection.String(json)\n\tobj := object.New(id, oobj, 0, fields)\n\tcol.Set(obj)\n\n\td.key = key\n\td.obj = obj\n\td.timestamp = time.Now()\n\td.updated = true\n\tswitch msg.OutputType {\n\tcase JSON:\n\t\tvar buf bytes.Buffer\n\t\tbuf.WriteString(`{\"ok\":true`)\n\t\tbuf.WriteString(`,\"elapsed\":\"` + time.Since(start).String() + \"\\\"}\")\n\t\treturn resp.StringValue(buf.String()), d, nil\n\tcase RESP:\n\t\treturn resp.IntegerValue(1), d, nil\n\t}\n\treturn NOMessage, d, nil\n}\n" }, { "path": "internal/server/json_test.go", "content": "package server\n\nimport (\n\t\"encoding/json\"\n\t\"testing\"\n)\n\nfunc BenchmarkJSONString(t *testing.B) {\n\tvar s = \"the need for mead\"\n\tfor i := 0; i < t.N; i++ {\n\t\tjsonString(s)\n\t}\n}\n\nfunc BenchmarkJSONMarshal(t *testing.B) {\n\tvar s = \"the need for mead\"\n\tfor i := 0; i < t.N; i++ {\n\t\tjson.Marshal(s)\n\t}\n}\n\nfunc TestIsJsonNumber(t *testing.T) {\n\ttest := func(expected bool, val string) {\n\t\tactual := isJSONNumber(val)\n\t\tif expected != actual {\n\t\t\tt.Fatalf(\"Expected %t == isJsonNumber(\\\"%s\\\") but was %t\", expected, val, actual)\n\t\t}\n\t}\n\ttest(false, \"\")\n\ttest(false, \"-\")\n\ttest(false, \"foo\")\n\ttest(false, \"0123\")\n\ttest(false, \"1.\")\n\ttest(false, \"1.0e\")\n\ttest(false, \"1.0e-\")\n\ttest(false, \"1.0E10NaN\")\n\ttest(false, \"1.0ENaN\")\n\ttest(true, \"-1\")\n\ttest(true, \"0\")\n\ttest(true, \"0.0\")\n\ttest(true, \"42\")\n\ttest(true, \"1.0E10\")\n\ttest(true, \"1.0e10\")\n\ttest(true, \"1E+5\")\n\ttest(true, \"1E-10\")\n}\n" }, { "path": "internal/server/keys.go", "content": "package server\n\nimport (\n\t\"encoding/json\"\n\t\"time\"\n\n\t\"github.com/tidwall/resp\"\n\t\"github.com/tidwall/tile38/internal/collection\"\n\t\"github.com/tidwall/tile38/internal/glob\"\n)\n\n// KEYS pattern\nfunc (s *Server) cmdKEYS(msg *Message) (resp.Value, error) {\n\tvar start = time.Now()\n\n\t// >> Args\n\n\targs := msg.Args\n\tif len(args) != 2 {\n\t\treturn retrerr(errInvalidNumberOfArguments)\n\t}\n\tpattern := args[1]\n\n\t// >> Operation\n\n\tkeys := []string{}\n\tg := glob.Parse(pattern, false)\n\teverything := g.Limits[0] == \"\" && g.Limits[1] == \"\"\n\tif everything {\n\t\ts.cols.Scan(\n\t\t\tfunc(key string, _ *collection.Collection) bool {\n\t\t\t\tmatch, _ := glob.Match(pattern, key)\n\t\t\t\tif match {\n\t\t\t\t\tkeys = append(keys, key)\n\t\t\t\t}\n\t\t\t\treturn true\n\t\t\t},\n\t\t)\n\t} else {\n\t\ts.cols.Ascend(g.Limits[0],\n\t\t\tfunc(key string, _ *collection.Collection) bool {\n\t\t\t\tif key > g.Limits[1] {\n\t\t\t\t\treturn false\n\t\t\t\t}\n\t\t\t\tmatch, _ := glob.Match(pattern, key)\n\t\t\t\tif match {\n\t\t\t\t\tkeys = append(keys, key)\n\t\t\t\t}\n\t\t\t\treturn true\n\t\t\t},\n\t\t)\n\t}\n\n\t// >> Response\n\n\tif msg.OutputType == JSON {\n\t\tdata, _ := json.Marshal(keys)\n\t\treturn resp.StringValue(`{\"ok\":true,\"keys\":` + string(data) +\n\t\t\t`,\"elapsed\":\"` + time.Since(start).String() + `\"}`), nil\n\t}\n\n\tvar vals []resp.Value\n\tfor _, key := range keys {\n\t\tvals = append(vals, resp.StringValue(key))\n\t}\n\treturn resp.ArrayValue(vals), nil\n}\n" }, { "path": "internal/server/live.go", "content": "package server\n\nimport (\n\t\"bytes\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"net\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/tidwall/redcon\"\n\t\"github.com/tidwall/tile38/internal/log\"\n\t\"go.uber.org/atomic\"\n)\n\ntype liveBuffer struct {\n\tkey string\n\tglobs []string\n\tfence *liveFenceSwitches\n\tdetails []*commandDetails\n\tcond *sync.Cond\n}\n\nfunc (s *Server) processLives(wg *sync.WaitGroup) {\n\tdefer wg.Done()\n\tvar done atomic.Bool\n\twg.Add(1)\n\tgo func() {\n\t\tdefer wg.Done()\n\t\tfor {\n\t\t\tif done.Load() {\n\t\t\t\tbreak\n\t\t\t}\n\t\t\ts.lcond.Broadcast()\n\t\t\ttime.Sleep(time.Second / 4)\n\t\t}\n\t}()\n\ts.lcond.L.Lock()\n\tdefer s.lcond.L.Unlock()\n\tfor {\n\t\tif s.stopServer.Load() {\n\t\t\tdone.Store(true)\n\t\t\treturn\n\t\t}\n\t\tfor len(s.lstack) > 0 {\n\t\t\titem := s.lstack[0]\n\t\t\ts.lstack = s.lstack[1:]\n\t\t\tif len(s.lstack) == 0 {\n\t\t\t\ts.lstack = nil\n\t\t\t}\n\t\t\tfor lb := range s.lives {\n\t\t\t\tlb.cond.L.Lock()\n\t\t\t\tif lb.key != \"\" && lb.key == item.key {\n\t\t\t\t\tlb.details = append(lb.details, item)\n\t\t\t\t\tlb.cond.Broadcast()\n\t\t\t\t}\n\t\t\t\tlb.cond.L.Unlock()\n\t\t\t}\n\t\t}\n\t\ts.lcond.Wait()\n\t}\n}\n\nfunc writeLiveMessage(\n\tconn net.Conn,\n\tmessage []byte,\n\twrapRESP bool,\n\tconnType Type, websocket bool,\n) error {\n\tif len(message) == 0 {\n\t\treturn nil\n\t}\n\tif websocket {\n\t\treturn WriteWebSocketMessage(conn, message)\n\t}\n\tvar err error\n\tswitch connType {\n\tcase RESP:\n\t\tif wrapRESP {\n\t\t\t_, err = fmt.Fprintf(conn, \"$%d\\r\\n%s\\r\\n\", len(message), string(message))\n\t\t} else {\n\t\t\t_, err = conn.Write(message)\n\t\t}\n\tcase Native:\n\t\t_, err = fmt.Fprintf(conn, \"$%d %s\\r\\n\", len(message), string(message))\n\t}\n\treturn err\n}\n\nfunc (s *Server) goLive(\n\tinerr error, conn net.Conn, rd *PipelineReader, msg *Message, websocket bool,\n) error {\n\taddr := conn.RemoteAddr().String()\n\tlog.Info(\"live \" + addr)\n\tdefer func() {\n\t\tlog.Info(\"not live \" + addr)\n\t}()\n\tswitch lfs := inerr.(type) {\n\tdefault:\n\t\treturn errors.New(\"invalid live type switches\")\n\tcase liveAOFSwitches:\n\t\treturn s.liveAOF(lfs.pos, conn, rd, msg)\n\tcase liveSubscriptionSwitches:\n\t\treturn s.liveSubscription(conn, rd, msg, websocket)\n\tcase liveMonitorSwitches:\n\t\treturn s.liveMonitor(conn, rd, msg)\n\tcase liveFenceSwitches:\n\t\t// fallthrough\n\t}\n\n\t// everything below is for live geofences\n\tlb := &liveBuffer{\n\t\tcond: sync.NewCond(&sync.Mutex{}),\n\t}\n\tvar err error\n\tvar sw *scanWriter\n\tvar wr bytes.Buffer\n\tlfs := inerr.(liveFenceSwitches)\n\tlb.globs = lfs.globs\n\tlb.key = lfs.key\n\tlb.fence = &lfs\n\ts.mu.RLock()\n\tsw, err = s.newScanWriter(\n\t\t&wr, msg, lfs.key, lfs.output, lfs.precision, lfs.globs, false,\n\t\tlfs.cursor, lfs.limit, lfs.wheres, lfs.whereins, lfs.whereevals,\n\t\tlfs.nofields, lfs.mvt, lfs.tileX, lfs.tileY, lfs.tileZ)\n\ts.mu.RUnlock()\n\n\t// everything below if for live SCAN, NEARBY, WITHIN, INTERSECTS\n\tif err != nil {\n\t\treturn err\n\t}\n\ts.lcond.L.Lock()\n\ts.lives[lb] = true\n\ts.lcond.L.Unlock()\n\tdefer func() {\n\t\ts.lcond.L.Lock()\n\t\tdelete(s.lives, lb)\n\t\ts.lcond.L.Unlock()\n\t\tconn.Close()\n\t}()\n\n\tvar mustQuit bool\n\tgo func() {\n\t\tdefer func() {\n\t\t\tlb.cond.L.Lock()\n\t\t\tmustQuit = true\n\t\t\tlb.cond.Broadcast()\n\t\t\tlb.cond.L.Unlock()\n\t\t\tconn.Close()\n\t\t}()\n\t\tfor {\n\t\t\tvs, err := rd.ReadMessages()\n\t\t\tif err != nil {\n\t\t\t\tif err != io.EOF && !(websocket && err == io.ErrUnexpectedEOF) {\n\t\t\t\t\tlog.Error(err)\n\t\t\t\t}\n\t\t\t\treturn\n\t\t\t}\n\t\t\tfor _, v := range vs {\n\t\t\t\tif v == nil {\n\t\t\t\t\tcontinue\n\t\t\t\t}\n\t\t\t\tswitch v.Command() {\n\t\t\t\tdefault:\n\t\t\t\t\tlog.Error(\"received a live command that was not QUIT\")\n\t\t\t\t\treturn\n\t\t\t\tcase \"quit\", \"\":\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}()\n\toutputType := msg.OutputType\n\tconnType := msg.ConnType\n\tif websocket {\n\t\toutputType = JSON\n\t}\n\tvar livemsg []byte\n\tswitch outputType {\n\tcase JSON:\n\t\tlivemsg = redcon.AppendBulkString(nil, `{\"ok\":true,\"live\":true}`)\n\tcase RESP:\n\t\tlivemsg = redcon.AppendOK(nil)\n\t}\n\tif err := writeLiveMessage(conn, livemsg, false, connType, websocket); err != nil {\n\t\treturn nil // nil return is fine here\n\t}\n\tfor {\n\t\tlb.cond.L.Lock()\n\t\tif mustQuit {\n\t\t\tlb.cond.L.Unlock()\n\t\t\treturn nil\n\t\t}\n\t\tfor len(lb.details) > 0 {\n\t\t\tdetails := lb.details[0]\n\t\t\tlb.details = lb.details[1:]\n\t\t\tif len(lb.details) == 0 {\n\t\t\t\tlb.details = nil\n\t\t\t}\n\t\t\tfence := lb.fence\n\t\t\tlb.cond.L.Unlock()\n\t\t\tvar msgs []string\n\t\t\tfunc() {\n\t\t\t\t// safely lock the fence because we are outside the main loop\n\t\t\t\ts.mu.RLock()\n\t\t\t\tdefer s.mu.RUnlock()\n\t\t\t\tmsgs = FenceMatch(\"\", sw, fence, nil, details)\n\t\t\t}()\n\t\t\tfor _, msg := range msgs {\n\t\t\t\tif err := writeLiveMessage(conn, []byte(msg), true, connType, websocket); err != nil {\n\t\t\t\t\treturn nil // nil return is fine here\n\t\t\t\t}\n\t\t\t}\n\t\t\ts.statsTotalMsgsSent.Add(int64(len(msgs)))\n\t\t\tlb.cond.L.Lock()\n\n\t\t}\n\t\tlb.cond.Wait()\n\t\tlb.cond.L.Unlock()\n\t}\n}\n" }, { "path": "internal/server/metrics.go", "content": "package server\n\nimport (\n\t\"fmt\"\n\t\"net/http\"\n\n\t\"github.com/tidwall/tile38/core\"\n\t\"github.com/tidwall/tile38/internal/collection\"\n\n\t\"github.com/prometheus/client_golang/prometheus\"\n\t\"github.com/prometheus/client_golang/prometheus/collectors\"\n\t\"github.com/prometheus/client_golang/prometheus/promhttp\"\n)\n\nvar (\n\tmetricDescriptions = map[string]*prometheus.Desc{\n\t\t/*\n\t\t\tthese metrics are taken from basicStats() / extStats()\n\t\t\tby accessing the map and directly exporting the value found\n\t\t*/\n\t\t\"num_collections\": prometheus.NewDesc(\"tile38_collections\", \"Total number of collections\", nil, nil),\n\t\t\"pid\": prometheus.NewDesc(\"tile38_pid\", \"\", nil, nil),\n\t\t\"aof_size\": prometheus.NewDesc(\"tile38_aof_size_bytes\", \"\", nil, nil),\n\t\t\"num_hooks\": prometheus.NewDesc(\"tile38_hooks\", \"\", nil, nil),\n\t\t\"in_memory_size\": prometheus.NewDesc(\"tile38_in_memory_size_bytes\", \"\", nil, nil),\n\t\t\"heap_size\": prometheus.NewDesc(\"tile38_heap_size_bytes\", \"\", nil, nil),\n\t\t\"heap_released\": prometheus.NewDesc(\"tile38_memory_reap_released_bytes\", \"\", nil, nil),\n\t\t\"max_heap_size\": prometheus.NewDesc(\"tile38_memory_max_heap_size_bytes\", \"\", nil, nil),\n\t\t\"avg_item_size\": prometheus.NewDesc(\"tile38_avg_item_size_bytes\", \"\", nil, nil),\n\t\t\"pointer_size\": prometheus.NewDesc(\"tile38_pointer_size_bytes\", \"\", nil, nil),\n\t\t\"cpus\": prometheus.NewDesc(\"tile38_num_cpus\", \"\", nil, nil),\n\t\t\"tile38_connected_clients\": prometheus.NewDesc(\"tile38_connected_clients\", \"\", nil, nil),\n\n\t\t\"tile38_total_connections_received\": prometheus.NewDesc(\"tile38_connections_received_total\", \"\", nil, nil),\n\t\t\"tile38_total_messages_sent\": prometheus.NewDesc(\"tile38_messages_sent_total\", \"\", nil, nil),\n\t\t\"tile38_expired_keys\": prometheus.NewDesc(\"tile38_expired_keys_total\", \"\", nil, nil),\n\n\t\t/*\n\t\t\tthese metrics are NOT taken from basicStats() / extStats()\n\t\t\tbut are calculated independently\n\t\t*/\n\t\t\"collection_objects\": prometheus.NewDesc(\"tile38_collection_objects\", \"Total number of objects per collection\", []string{\"col\"}, nil),\n\t\t\"collection_points\": prometheus.NewDesc(\"tile38_collection_points\", \"Total number of points per collection\", []string{\"col\"}, nil),\n\t\t\"collection_strings\": prometheus.NewDesc(\"tile38_collection_strings\", \"Total number of strings per collection\", []string{\"col\"}, nil),\n\t\t\"collection_weight\": prometheus.NewDesc(\"tile38_collection_weight_bytes\", \"Total weight of collection in bytes\", []string{\"col\"}, nil),\n\t\t\"server_info\": prometheus.NewDesc(\"tile38_server_info\", \"Server info\", []string{\"id\", \"version\"}, nil),\n\t\t\"replication\": prometheus.NewDesc(\"tile38_replication_info\", \"Replication info\", []string{\"role\", \"following\", \"caught_up\", \"caught_up_once\"}, nil),\n\t\t\"start_time\": prometheus.NewDesc(\"tile38_start_time_seconds\", \"\", nil, nil),\n\t}\n\n\tcmdDurations = prometheus.NewSummaryVec(prometheus.SummaryOpts{\n\t\tName: \"tile38_cmd_duration_seconds\",\n\t\tObjectives: map[float64]float64{0.5: 0.05, 0.9: 0.01, 0.95: 0.005, 0.99: 0.001},\n\t}, []string{\"cmd\"},\n\t)\n)\n\nfunc (s *Server) MetricsIndexHandler(w http.ResponseWriter, r *http.Request) {\n\tw.Write([]byte(`\nTile38 ` + core.Version + `\n

Tile38 ` + core.Version + `

\n

Metrics

\n`))\n}\n\nfunc (s *Server) MetricsHandler(w http.ResponseWriter, r *http.Request) {\n\treg := prometheus.NewRegistry()\n\n\treg.MustRegister(\n\t\tcollectors.NewProcessCollector(collectors.ProcessCollectorOpts{}),\n\t\tcollectors.NewGoCollector(),\n\t\tcollectors.NewBuildInfoCollector(),\n\t\tcmdDurations,\n\t\ts,\n\t)\n\n\tpromhttp.HandlerFor(reg, promhttp.HandlerOpts{}).ServeHTTP(w, r)\n}\n\nfunc (s *Server) Describe(ch chan<- *prometheus.Desc) {\n\tfor _, desc := range metricDescriptions {\n\t\tch <- desc\n\t}\n}\n\nfunc (s *Server) Collect(ch chan<- prometheus.Metric) {\n\ts.mu.RLock()\n\tdefer s.mu.RUnlock()\n\n\tm := make(map[string]interface{})\n\ts.basicStats(m)\n\ts.extStats(m)\n\n\tfor metric, descr := range metricDescriptions {\n\t\tval, ok := toFloat(m[metric])\n\t\tif ok {\n\t\t\tch <- prometheus.MustNewConstMetric(descr, prometheus.GaugeValue, val)\n\t\t}\n\t}\n\n\tch <- prometheus.MustNewConstMetric(\n\t\tmetricDescriptions[\"server_info\"],\n\t\tprometheus.GaugeValue, 1.0,\n\t\ts.config.serverID(), core.Version)\n\n\tch <- prometheus.MustNewConstMetric(\n\t\tmetricDescriptions[\"start_time\"],\n\t\tprometheus.GaugeValue, float64(s.started.Unix()))\n\n\treplLbls := []string{\"leader\", \"\", \"\", \"\"}\n\tif s.config.followHost() != \"\" {\n\t\treplLbls = []string{\"follower\",\n\t\t\tfmt.Sprintf(\"%s:%d\", s.config.followHost(), s.config.followPort()),\n\t\t\tfmt.Sprintf(\"%t\", s.caughtUp()), fmt.Sprintf(\"%t\", s.caughtUpOnce())}\n\t}\n\tch <- prometheus.MustNewConstMetric(\n\t\tmetricDescriptions[\"replication\"],\n\t\tprometheus.GaugeValue, 1.0,\n\t\treplLbls...)\n\n\t/*\n\t\tadd objects/points/strings stats for each collection\n\t*/\n\ts.cols.Scan(func(key string, col *collection.Collection) bool {\n\t\tch <- prometheus.MustNewConstMetric(\n\t\t\tmetricDescriptions[\"collection_objects\"],\n\t\t\tprometheus.GaugeValue,\n\t\t\tfloat64(col.Count()),\n\t\t\tkey,\n\t\t)\n\t\tch <- prometheus.MustNewConstMetric(\n\t\t\tmetricDescriptions[\"collection_points\"],\n\t\t\tprometheus.GaugeValue,\n\t\t\tfloat64(col.PointCount()),\n\t\t\tkey,\n\t\t)\n\t\tch <- prometheus.MustNewConstMetric(\n\t\t\tmetricDescriptions[\"collection_strings\"],\n\t\t\tprometheus.GaugeValue,\n\t\t\tfloat64(col.StringCount()),\n\t\t\tkey,\n\t\t)\n\t\tch <- prometheus.MustNewConstMetric(\n\t\t\tmetricDescriptions[\"collection_weight\"],\n\t\t\tprometheus.GaugeValue,\n\t\t\tfloat64(col.TotalWeight()),\n\t\t\tkey,\n\t\t)\n\t\treturn true\n\t})\n}\n\nfunc toFloat(val interface{}) (float64, bool) {\n\tswitch v := val.(type) {\n\tcase float64:\n\t\treturn v, true\n\tcase int64:\n\t\treturn float64(v), true\n\tcase uint64:\n\t\treturn float64(v), true\n\tcase float32:\n\t\treturn float64(v), true\n\tcase int:\n\t\treturn float64(v), true\n\tcase int32:\n\t\treturn float64(v), true\n\tcase uint32:\n\t\treturn float64(v), true\n\tcase int16:\n\t\treturn float64(v), true\n\tcase uint16:\n\t\treturn float64(v), true\n\tcase int8:\n\t\treturn float64(v), true\n\tcase uint8:\n\t\treturn float64(v), true\n\t}\n\treturn 0, false\n}\n" }, { "path": "internal/server/monitor.go", "content": "package server\n\nimport (\n\t\"fmt\"\n\t\"io\"\n\t\"net\"\n\t\"strconv\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/tidwall/resp\"\n)\n\ntype liveMonitorSwitches struct {\n\t// no fields. everything is managed through the Message\n}\n\nfunc (sub liveMonitorSwitches) Error() string {\n\treturn goingLive\n}\n\nfunc (s *Server) cmdMonitor(msg *Message) (resp.Value, error) {\n\tif len(msg.Args) != 1 {\n\t\treturn resp.Value{}, errInvalidNumberOfArguments\n\t}\n\treturn NOMessage, liveMonitorSwitches{}\n}\n\nfunc (s *Server) liveMonitor(conn net.Conn, rd *PipelineReader, msg *Message) error {\n\ts.monconnsMu.Lock()\n\ts.monconns[conn] = true\n\ts.monconnsMu.Unlock()\n\tdefer func() {\n\t\ts.monconnsMu.Lock()\n\t\tdelete(s.monconns, conn)\n\t\ts.monconnsMu.Unlock()\n\t\tconn.Close()\n\t}()\n\ts.monconnsMu.Lock()\n\tconn.Write([]byte(\"+OK\\r\\n\"))\n\ts.monconnsMu.Unlock()\n\tmsgs, err := rd.ReadMessages()\n\tif err != nil {\n\t\tif err == io.EOF {\n\t\t\treturn nil\n\t\t}\n\t\treturn err\n\t}\n\tfor _, msg := range msgs {\n\t\tif len(msg.Args) == 1 && strings.ToLower(msg.Args[0]) == \"quit\" {\n\t\t\ts.monconnsMu.Lock()\n\t\t\tconn.Write([]byte(\"+OK\\r\\n\"))\n\t\t\ts.monconnsMu.Unlock()\n\t\t\treturn nil\n\t\t}\n\t}\n\treturn nil\n}\n\n// send messages to live MONITOR clients\nfunc (s *Server) sendMonitor(err error, msg *Message, c *Client, lua bool) {\n\ts.monconnsMu.RLock()\n\tn := len(s.monconns)\n\ts.monconnsMu.RUnlock()\n\tif n == 0 {\n\t\treturn\n\t}\n\tif (c == nil && !lua) ||\n\t\t(err != nil && (err == errInvalidNumberOfArguments ||\n\t\t\tstrings.HasPrefix(err.Error(), \"unknown command \"))) {\n\t\treturn\n\t}\n\n\t// accept all commands except for these:\n\tswitch strings.ToLower(msg.Command()) {\n\tcase \"config\", \"config set\", \"config get\", \"config rewrite\",\n\t\t\"auth\", \"follow\", \"slaveof\", \"replconf\",\n\t\t\"aof\", \"aofmd5\", \"client\",\n\t\t\"monitor\":\n\t\treturn\n\t}\n\n\tvar line []byte\n\tfor i, arg := range msg.Args {\n\t\tif i > 0 {\n\t\t\tline = append(line, ' ')\n\t\t}\n\t\tline = append(line, strconv.Quote(arg)...)\n\t}\n\ttstr := fmt.Sprintf(\"%.6f\", float64(time.Now().UnixNano())/1e9)\n\tvar addr string\n\tif lua {\n\t\taddr = \"lua\"\n\t} else {\n\t\taddr = c.remoteAddr\n\t}\n\ts.monconnsMu.Lock()\n\tfor conn := range s.monconns {\n\t\tfmt.Fprintf(conn, \"+%s [0 %s] %s\\r\\n\", tstr, addr, line)\n\t}\n\ts.monconnsMu.Unlock()\n}\n" }, { "path": "internal/server/must.go", "content": "package server\n\nfunc Must[T any](a T, err error) T {\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\treturn a\n}\n\nfunc Default[T comparable](a, b T) T {\n\tvar c T\n\tif a == c {\n\t\treturn b\n\t}\n\treturn a\n}\n" }, { "path": "internal/server/must_test.go", "content": "package server\n\nimport (\n\t\"errors\"\n\t\"testing\"\n)\n\nfunc TestMust(t *testing.T) {\n\tif Must(1, nil) != 1 {\n\t\tt.Fail()\n\t}\n\tfunc() {\n\t\tvar ended bool\n\t\tdefer func() {\n\t\t\tif ended {\n\t\t\t\tt.Fail()\n\t\t\t}\n\t\t\terr, ok := recover().(error)\n\t\t\tif !ok {\n\t\t\t\tt.Fail()\n\t\t\t}\n\t\t\tif err.Error() != \"ok\" {\n\t\t\t\tt.Fail()\n\t\t\t}\n\t\t}()\n\t\tMust(1, errors.New(\"ok\"))\n\t\tended = true\n\t}()\n}\n\nfunc TestDefault(t *testing.T) {\n\tif Default(\"\", \"2\") != \"2\" {\n\t\tt.Fail()\n\t}\n\tif Default(\"1\", \"2\") != \"1\" {\n\t\tt.Fail()\n\t}\n}\n" }, { "path": "internal/server/mvt.go", "content": "package server\n\nimport (\n\t\"net/url\"\n\t\"strings\"\n\n\t\"github.com/tidwall/geojson\"\n\t\"github.com/tidwall/geojson/geometry\"\n\t\"github.com/tidwall/mvt\"\n)\n\ntype mvtObj struct {\n\tid string\n\tobj geojson.Object\n}\n\nfunc mvtDrawRing(f *mvt.Feature, tileX, tileY, tileZ int, ring geometry.Series,\n\thole bool,\n) {\n\tnpoints := ring.NumPoints()\n\tif npoints < 3 {\n\t\treturn\n\t}\n\tcw := ring.Clockwise()\n\treverse := (cw && hole) || (!cw && !hole)\n\tif reverse {\n\t\tp := ring.PointAt(npoints - 1)\n\t\tf.MoveTo(mvt.LatLonXY(p.Y, p.X, tileX, tileY, tileZ))\n\t\tfor i := npoints - 2; i >= 0; i-- {\n\t\t\tp := ring.PointAt(i)\n\t\t\tf.LineTo(mvt.LatLonXY(p.Y, p.X, tileX, tileY, tileZ))\n\t\t}\n\t} else {\n\t\tp := ring.PointAt(0)\n\t\tf.MoveTo(mvt.LatLonXY(p.Y, p.X, tileX, tileY, tileZ))\n\t\tfor i := 1; i < npoints; i++ {\n\t\t\tp := ring.PointAt(i)\n\t\t\tf.LineTo(mvt.LatLonXY(p.Y, p.X, tileX, tileY, tileZ))\n\t\t}\n\t}\n\tf.ClosePath()\n}\n\nfunc mvtAddFeature(l *mvt.Layer, tileX, tileY, tileZ int, o mvtObj) {\n\tvar f *mvt.Feature\n\tswitch g := o.obj.(type) {\n\tcase *geojson.Point:\n\t\tf = l.AddFeature(mvt.Point)\n\t\tp := g.Base()\n\t\tf.MoveTo(mvt.LatLonXY(p.Y, p.X, tileX, tileY, tileZ))\n\t\tf.AddTag(\"type\", \"point\")\n\tcase *geojson.SimplePoint:\n\t\tf = l.AddFeature(mvt.Point)\n\t\tp := g\n\t\tf.MoveTo(mvt.LatLonXY(p.Y, p.X, tileX, tileY, tileZ))\n\t\tf.AddTag(\"type\", \"point\")\n\tcase *geojson.LineString:\n\t\tf = l.AddFeature(mvt.LineString)\n\t\tline := g.Base()\n\t\tnpoints := line.NumPoints()\n\t\tif npoints < 2 {\n\t\t\treturn\n\t\t}\n\t\tp := line.PointAt(0)\n\t\tf.MoveTo(mvt.LatLonXY(p.Y, p.X, tileX, tileY, tileZ))\n\t\tfor i := 1; i < npoints; i++ {\n\t\t\tp := line.PointAt(i)\n\t\t\tf.LineTo(mvt.LatLonXY(p.Y, p.X, tileX, tileY, tileZ))\n\t\t}\n\t\tf.AddTag(\"type\", \"linestring\")\n\tcase *geojson.Rect:\n\t\tf = l.AddFeature(mvt.Polygon)\n\t\tmvtDrawRing(f, tileX, tileY, tileZ, g.Base(), false)\n\t\tf.AddTag(\"type\", \"polygon\")\n\tcase *geojson.Polygon:\n\t\tf = l.AddFeature(mvt.Polygon)\n\t\tpoly := g.Base()\n\t\tmvtDrawRing(f, tileX, tileY, tileZ, poly.Exterior, false)\n\t\tfor _, hole := range poly.Holes {\n\t\t\tmvtDrawRing(f, tileX, tileY, tileZ, hole, true)\n\t\t}\n\t\tf.AddTag(\"type\", \"polygon\")\n\tcase *geojson.Feature:\n\t\tmvtAddFeature(l, tileX, tileY, tileZ, mvtObj{o.id, g.Base()})\n\t\treturn\n\tdefault:\n\t\tif g, ok := g.(geojson.Collection); ok {\n\t\t\tfor _, g := range g.Children() {\n\t\t\t\tmvtAddFeature(l, tileX, tileY, tileZ, mvtObj{o.id, g})\n\t\t\t}\n\t\t}\n\t\treturn\n\t}\n\tf.AddTag(\"id\", o.id)\n}\n\nfunc mvtRender(tileX, tileY, tileZ int, objs []mvtObj) []byte {\n\tvar tile mvt.Tile\n\tl := tile.AddLayer(\"tile38\")\n\tl.SetExtent(4096)\n\tfor _, obj := range objs {\n\t\tmvtAddFeature(l, tileX, tileY, tileZ, obj)\n\t}\n\treturn tile.Render()\n}\n\nfunc mvtFilterHTTPArgs(msg *Message, query string) (modified bool) {\n\tpath := msg.Args[0]\n\tparts := strings.Split(path, \"/\")\n\tif len(parts) != 4 {\n\t\treturn false\n\t}\n\tparts[3] = parts[3][:len(parts[3])-4]\n\tfor i := 0; i < len(parts); i++ {\n\t\tvar err error\n\t\tparts[i], err = url.PathUnescape(parts[i])\n\t\tif err != nil {\n\t\t\treturn false\n\t\t}\n\t}\n\tvar limit string\n\tvar sparse string\n\tif query != \"\" {\n\t\tq, _ := url.ParseQuery(query)\n\t\tsparse = q.Get(\"sparse\")\n\t\tlimit = q.Get(\"limit\")\n\t}\n\tmsg._command = \"\"\n\tmsg.Args = []string{\"INTERSECTS\", parts[0]}\n\tif sparse != \"\" {\n\t\tmsg.Args = append(msg.Args, \"SPARSE\", sparse)\n\t} else if limit != \"\" {\n\t\tmsg.Args = append(msg.Args, \"LIMIT\", limit)\n\t} else {\n\t\tmsg.Args = append(msg.Args, \"LIMIT\", \"100000000\")\n\t}\n\tmsg.Args = append(msg.Args, \"MVT\", parts[2], parts[3], parts[1])\n\treturn true\n}\n" }, { "path": "internal/server/mvt_test.go", "content": "package server\n\nimport (\n\t\"bytes\"\n\t\"testing\"\n\n\t\"github.com/tidwall/geojson\"\n\t\"github.com/tidwall/geojson/geometry\"\n\t\"github.com/tidwall/mvt\"\n)\n\nfunc ls(points []geometry.Point) *geojson.LineString {\n\treturn geojson.NewLineString(geometry.NewLine(points, nil))\n}\n\n// Ensure that LineString features are encoded using\n// all points in the geometry, not just the first one.\nfunc TestMVTAddFeatureLineStringUsesAllPoints(t *testing.T) {\n\ttileX, tileY, tileZ := 0, 0, 0\n\n\tline := ls([]geometry.Point{\n\t\t{X: 1, Y: 1},\n\t\t{X: 2, Y: 2},\n\t\t{X: 3, Y: 3},\n\t})\n\tid := \"line\"\n\n\tactual := mvtRender(tileX, tileY, tileZ, []mvtObj{{id: id, obj: line}})\n\n\tvar tile mvt.Tile\n\tlayer := tile.AddLayer(\"tile38\")\n\tlayer.SetExtent(4096)\n\n\tf := layer.AddFeature(mvt.LineString)\n\tseries := line.Base()\n\tnpoints := series.NumPoints()\n\tif npoints < 2 {\n\t\tt.Fatalf(\"expected at least two points, got %d\", npoints)\n\t}\n\tp := series.PointAt(0)\n\tx, y := mvt.LatLonXY(p.Y, p.X, tileX, tileY, tileZ)\n\tf.MoveTo(x, y)\n\tfor i := 1; i < npoints; i++ {\n\t\tp = series.PointAt(i)\n\t\tx, y = mvt.LatLonXY(p.Y, p.X, tileX, tileY, tileZ)\n\t\tf.LineTo(x, y)\n\t}\n\tf.AddTag(\"type\", \"linestring\")\n\tf.AddTag(\"id\", id)\n\n\texpected := tile.Render()\n\n\tif !bytes.Equal(actual, expected) {\n\t\tt.Fatalf(\"mvtAddFeature LineString encoding mismatch\")\n\t}\n}\n\n// LineStrings with fewer than two points should not\n// produce any geometry commands.\nfunc TestMVTAddFeatureLineStringTooShort(t *testing.T) {\n\ttileX, tileY, tileZ := 0, 0, 0\n\n\tline := ls([]geometry.Point{\n\t\t{X: 1, Y: 1},\n\t})\n\n\tactual := mvtRender(tileX, tileY, tileZ, []mvtObj{{id: \"short\", obj: line}})\n\n\tvar tile mvt.Tile\n\tlayer := tile.AddLayer(\"tile38\")\n\tlayer.SetExtent(4096)\n\t_ = layer.AddFeature(mvt.LineString)\n\n\texpected := tile.Render()\n\n\tif !bytes.Equal(actual, expected) {\n\t\tt.Fatalf(\"mvtAddFeature LineString with <2 points should not encode geometry\")\n\t}\n}\n\n" }, { "path": "internal/server/output.go", "content": "package server\n\nimport (\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/tidwall/resp\"\n)\n\n// OUTPUT [resp|json]\nfunc (s *Server) cmdOUTPUT(msg *Message) (resp.Value, error) {\n\tstart := time.Now()\n\n\targs := msg.Args\n\tswitch len(args) {\n\tcase 1:\n\t\tif msg.OutputType == JSON {\n\t\t\treturn resp.StringValue(`{\"ok\":true,\"output\":\"json\",\"elapsed\":` +\n\t\t\t\ttime.Since(start).String() + `}`), nil\n\t\t}\n\t\treturn resp.StringValue(\"resp\"), nil\n\tcase 2:\n\t\t// Setting the original message output type will be picked up by the\n\t\t// server prior to the next command being executed.\n\t\tswitch strings.ToLower(args[1]) {\n\t\tdefault:\n\t\t\treturn retrerr(errInvalidArgument(args[1]))\n\t\tcase \"json\":\n\t\t\tmsg.OutputType = JSON\n\t\tcase \"resp\":\n\t\t\tmsg.OutputType = RESP\n\t\t}\n\t\treturn OKMessage(msg, start), nil\n\tdefault:\n\t\treturn retrerr(errInvalidNumberOfArguments)\n\t}\n}\n" }, { "path": "internal/server/pubqueue.go", "content": "package server\n\nimport (\n\t\"net\"\n\t\"sync\"\n\n\t\"github.com/tidwall/redcon\"\n)\n\ntype pubQueue struct {\n\tcond *sync.Cond\n\tentries []pubQueueEntry // follower publish queue\n\tclosed bool\n}\n\ntype pubQueueEntry struct {\n\tchannel string\n\tmessages []string\n}\n\nfunc (s *Server) startPublishQueue(wg *sync.WaitGroup) {\n\tdefer wg.Done()\n\tvar buf []byte\n\tvar conns []net.Conn\n\ts.pubq.cond.L.Lock()\n\tfor {\n\t\tfor len(s.pubq.entries) > 0 {\n\t\t\tentries := s.pubq.entries\n\t\t\ts.pubq.entries = nil\n\t\t\ts.pubq.cond.L.Unlock()\n\t\t\t// Get follower connections\n\t\t\ts.mu.RLock()\n\t\t\tfor conn := range s.aofconnM {\n\t\t\t\tconns = append(conns, conn)\n\t\t\t}\n\t\t\ts.mu.RUnlock()\n\t\t\t// Buffer the PUBLISH command pipeline\n\t\t\tbuf = buf[:0]\n\t\t\tfor _, entry := range entries {\n\t\t\t\tfor _, message := range entry.messages {\n\t\t\t\t\tbuf = redcon.AppendArray(buf, 3)\n\t\t\t\t\tbuf = redcon.AppendBulkString(buf, \"PUBLISH\")\n\t\t\t\t\tbuf = redcon.AppendBulkString(buf, entry.channel)\n\t\t\t\t\tbuf = redcon.AppendBulkString(buf, message)\n\t\t\t\t}\n\t\t\t}\n\t\t\t// Publish to followers\n\t\t\tfor i, conn := range conns {\n\t\t\t\tconn.Write(buf)\n\t\t\t\tconns[i] = nil\n\t\t\t}\n\t\t\tconns = conns[:0]\n\t\t\ts.pubq.cond.L.Lock()\n\t\t}\n\t\tif s.pubq.closed {\n\t\t\tbreak\n\t\t}\n\t\ts.pubq.cond.Wait()\n\t}\n\ts.pubq.cond.L.Unlock()\n}\n\nfunc (s *Server) stopPublishQueue() {\n\ts.pubq.cond.L.Lock()\n\ts.pubq.closed = true\n\ts.pubq.cond.Broadcast()\n\ts.pubq.cond.L.Unlock()\n}\n\nfunc (s *Server) sendPublishQueue(channel string, message ...string) {\n\ts.pubq.cond.L.Lock()\n\tif !s.pubq.closed {\n\t\ts.pubq.entries = append(s.pubq.entries, pubQueueEntry{\n\t\t\tchannel: channel,\n\t\t\tmessages: message,\n\t\t})\n\t}\n\ts.pubq.cond.Broadcast()\n\ts.pubq.cond.L.Unlock()\n}\n" }, { "path": "internal/server/pubsub.go", "content": "package server\n\nimport (\n\t\"io\"\n\t\"net\"\n\t\"strconv\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/tidwall/gjson\"\n\t\"github.com/tidwall/match\"\n\t\"github.com/tidwall/redcon\"\n\t\"github.com/tidwall/resp\"\n\t\"github.com/tidwall/tile38/internal/log\"\n)\n\nconst (\n\tpubsubChannel = iota\n\tpubsubPattern\n)\n\ntype pubsub struct {\n\tmu sync.RWMutex\n\thubs [2]map[string]*subhub\n}\n\nfunc newPubsub() *pubsub {\n\treturn &pubsub{\n\t\thubs: [2]map[string]*subhub{\n\t\t\tmake(map[string]*subhub),\n\t\t\tmake(map[string]*subhub),\n\t\t},\n\t}\n}\n\n// Publish a message to subscribers\nfunc (s *Server) Publish(channel string, message ...string) int {\n\tvar msgs []submsg\n\ts.pubsub.mu.RLock()\n\tif hub := s.pubsub.hubs[pubsubChannel][channel]; hub != nil {\n\t\tfor target := range hub.targets {\n\t\t\tfor _, message := range message {\n\t\t\t\tmsgs = append(msgs, submsg{\n\t\t\t\t\tkind: pubsubChannel,\n\t\t\t\t\ttarget: target,\n\t\t\t\t\tchannel: channel,\n\t\t\t\t\tmessage: message,\n\t\t\t\t})\n\t\t\t}\n\t\t}\n\t}\n\tfor pattern, hub := range s.pubsub.hubs[pubsubPattern] {\n\t\tif match.Match(channel, pattern) {\n\t\t\tfor target := range hub.targets {\n\t\t\t\tfor _, message := range message {\n\t\t\t\t\tmsgs = append(msgs, submsg{\n\t\t\t\t\t\tkind: pubsubPattern,\n\t\t\t\t\t\ttarget: target,\n\t\t\t\t\t\tchannel: channel,\n\t\t\t\t\t\tpattern: pattern,\n\t\t\t\t\t\tmessage: message,\n\t\t\t\t\t})\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\ts.pubsub.mu.RUnlock()\n\n\t// broadcast to clients\n\tfor _, msg := range msgs {\n\t\tmsg.target.cond.L.Lock()\n\t\tmsg.target.msgs = append(msg.target.msgs, msg)\n\t\tmsg.target.cond.Broadcast()\n\t\tmsg.target.cond.L.Unlock()\n\t}\n\n\t// Broadcast to followers\n\ts.sendPublishQueue(channel, message...)\n\n\treturn len(msgs)\n}\n\nfunc (ps *pubsub) register(kind int, channel string, target *subtarget) {\n\tps.mu.Lock()\n\thub, ok := ps.hubs[kind][channel]\n\tif !ok {\n\t\thub = newSubhub()\n\t\tps.hubs[kind][channel] = hub\n\t}\n\thub.targets[target] = true\n\tps.mu.Unlock()\n}\n\nfunc (ps *pubsub) unregister(kind int, channel string, target *subtarget) {\n\tps.mu.Lock()\n\thub, ok := ps.hubs[kind][channel]\n\tif ok {\n\t\tdelete(hub.targets, target)\n\t\tif len(hub.targets) == 0 {\n\t\t\tdelete(ps.hubs[kind], channel)\n\t\t}\n\t}\n\tps.mu.Unlock()\n}\n\ntype submsg struct {\n\tkind byte\n\ttarget *subtarget\n\tpattern string\n\tchannel string\n\tmessage string\n}\n\ntype subtarget struct {\n\tcond *sync.Cond\n\tmsgs []submsg\n\tclosed bool\n}\n\nfunc newSubtarget() *subtarget {\n\ttarget := new(subtarget)\n\ttarget.cond = sync.NewCond(&sync.Mutex{})\n\treturn target\n}\n\ntype subhub struct {\n\ttargets map[*subtarget]bool\n}\n\nfunc newSubhub() *subhub {\n\thub := new(subhub)\n\thub.targets = make(map[*subtarget]bool)\n\treturn hub\n}\n\ntype liveSubscriptionSwitches struct {\n\t// no fields. everything is managed through the Message\n}\n\nfunc (sub liveSubscriptionSwitches) Error() string {\n\treturn goingLive\n}\n\nfunc (s *Server) cmdSubscribe(msg *Message) (resp.Value, error) {\n\tif len(msg.Args) < 2 {\n\t\treturn resp.Value{}, errInvalidNumberOfArguments\n\t}\n\treturn NOMessage, liveSubscriptionSwitches{}\n}\n\nfunc (s *Server) cmdPsubscribe(msg *Message) (resp.Value, error) {\n\tif len(msg.Args) < 2 {\n\t\treturn resp.Value{}, errInvalidNumberOfArguments\n\t}\n\treturn NOMessage, liveSubscriptionSwitches{}\n}\n\nfunc (s *Server) cmdPublish(msg *Message) (resp.Value, error) {\n\tstart := time.Now()\n\tif len(msg.Args) != 3 {\n\t\treturn resp.Value{}, errInvalidNumberOfArguments\n\t}\n\n\tchannel := msg.Args[1]\n\tmessage := msg.Args[2]\n\t//geofence := gjson.Valid(message) && gjson.Get(message, \"fence\").Bool()\n\tn := s.Publish(channel, message) //, geofence)\n\tvar res resp.Value\n\tswitch msg.OutputType {\n\tcase JSON:\n\t\tres = resp.StringValue(`{\"ok\":true` +\n\t\t\t`,\"published\":` + strconv.FormatInt(int64(n), 10) +\n\t\t\t`,\"elapsed\":\"` + time.Since(start).String() + `\"}`)\n\tcase RESP:\n\t\tres = resp.IntegerValue(n)\n\t}\n\treturn res, nil\n}\n\nfunc (s *Server) liveSubscription(\n\tconn net.Conn,\n\trd *PipelineReader,\n\tmsg *Message,\n\twebsocket bool,\n) error {\n\tdefer conn.Close() // close connection when we are done\n\n\toutputType := msg.OutputType\n\tconnType := msg.ConnType\n\tif websocket {\n\t\toutputType = JSON\n\t} else if msg.StrictRESP {\n\t\toutputType = RESP\n\t}\n\n\tvar start time.Time\n\n\t// write helpers\n\tvar writeLock sync.Mutex\n\twrite := func(data []byte) {\n\t\twriteLock.Lock()\n\t\tdefer writeLock.Unlock()\n\t\twriteLiveMessage(conn, data, outputType == JSON, connType, websocket)\n\t}\n\twriteOK := func() {\n\t\tswitch outputType {\n\t\tcase JSON:\n\t\t\twrite([]byte(`{\"ok\":true` +\n\t\t\t\t`,\"elapsed\":\"` + time.Since(start).String() + `\"}`))\n\t\tcase RESP:\n\t\t\twrite([]byte(\"+OK\\r\\n\"))\n\t\t}\n\t}\n\twritePing := func(m *Message) {\n\t\tswitch outputType {\n\t\tcase JSON:\n\t\t\tif len(m.Args) > 1 {\n\t\t\t\twrite([]byte(`{\"ok\":true,\"ping\":` + jsonString(m.Args[1]) + `,\"elapsed\":\"` + time.Since(start).String() + `\"}`))\n\t\t\t} else {\n\t\t\t\twrite([]byte(`{\"ok\":true,\"ping\":\"pong\",\"elapsed\":\"` + time.Since(start).String() + `\"}`))\n\t\t\t}\n\t\tcase RESP:\n\t\t\tdata := redcon.AppendArray(nil, 2)\n\t\t\tdata = redcon.AppendBulkString(data, \"PONG\")\n\t\t\tif len(m.Args) > 1 {\n\t\t\t\tdata = redcon.AppendBulkString(data, m.Args[1])\n\t\t\t} else {\n\t\t\t\tdata = redcon.AppendBulkString(data, \"\")\n\t\t\t}\n\t\t\twrite(data)\n\t\t}\n\t}\n\twriteWrongNumberOfArgsErr := func(command string) {\n\t\tswitch outputType {\n\t\tcase JSON:\n\t\t\twrite([]byte(`{\"ok\":false,\"err\":\"invalid number of arguments\"` +\n\t\t\t\t`,\"elapsed\":\"` + time.Since(start).String() + `\"}`))\n\t\tcase RESP:\n\t\t\twrite([]byte(\"-ERR wrong number of arguments \" +\n\t\t\t\t\"for '\" + command + \"' command\\r\\n\"))\n\t\t}\n\t}\n\twriteOnlyPubsubErr := func() {\n\t\tswitch outputType {\n\t\tcase JSON:\n\t\t\twrite([]byte(`{\"ok\":false` +\n\t\t\t\t`,\"err\":\"only (P)SUBSCRIBE / (P)UNSUBSCRIBE / ` +\n\t\t\t\t`PING / QUIT allowed in this context\"` +\n\t\t\t\t`,\"elapsed\":\"` + time.Since(start).String() + `\"}`))\n\t\tcase RESP:\n\t\t\twrite([]byte(\"-ERR only (P)SUBSCRIBE / (P)UNSUBSCRIBE / \" +\n\t\t\t\t\"PING / QUIT allowed in this context\\r\\n\"))\n\t\t}\n\t}\n\twriteSubscribe := func(command, channel string, num int) {\n\t\tswitch outputType {\n\t\tcase JSON:\n\t\t\twrite([]byte(`{\"ok\":true` +\n\t\t\t\t`,\"command\":` + jsonString(command) +\n\t\t\t\t`,\"channel\":` + jsonString(channel) +\n\t\t\t\t`,\"num\":` + strconv.FormatInt(int64(num), 10) +\n\t\t\t\t`,\"elapsed\":\"` + time.Since(start).String() + `\"}`))\n\t\tcase RESP:\n\t\t\tb := redcon.AppendArray(nil, 3)\n\t\t\tb = redcon.AppendBulkString(b, command)\n\t\t\tb = redcon.AppendBulkString(b, channel)\n\t\t\tb = redcon.AppendInt(b, int64(num))\n\t\t\twrite(b)\n\t\t}\n\t}\n\twriteMessage := func(msg submsg) {\n\t\tif msg.kind == pubsubChannel {\n\t\t\tswitch outputType {\n\t\t\tcase JSON:\n\t\t\t\tvar data []byte\n\t\t\t\tif !gjson.Valid(msg.message) {\n\t\t\t\t\tdata = appendJSONString(nil, msg.message)\n\t\t\t\t} else {\n\t\t\t\t\tdata = []byte(msg.message)\n\t\t\t\t}\n\t\t\t\twrite(data)\n\t\t\tcase RESP:\n\t\t\t\tb := redcon.AppendArray(nil, 3)\n\t\t\t\tb = redcon.AppendBulkString(b, \"message\")\n\t\t\t\tb = redcon.AppendBulkString(b, msg.channel)\n\t\t\t\tb = redcon.AppendBulkString(b, msg.message)\n\t\t\t\twrite(b)\n\t\t\t}\n\t\t} else {\n\t\t\tswitch outputType {\n\t\t\tcase JSON:\n\t\t\t\tvar data []byte\n\t\t\t\tif !gjson.Valid(msg.message) {\n\t\t\t\t\tdata = appendJSONString(nil, msg.message)\n\t\t\t\t} else {\n\t\t\t\t\tdata = []byte(msg.message)\n\t\t\t\t}\n\t\t\t\twrite(data)\n\t\t\tcase RESP:\n\t\t\t\tb := redcon.AppendArray(nil, 4)\n\t\t\t\tb = redcon.AppendBulkString(b, \"pmessage\")\n\t\t\t\tb = redcon.AppendBulkString(b, msg.pattern)\n\t\t\t\tb = redcon.AppendBulkString(b, msg.channel)\n\t\t\t\tb = redcon.AppendBulkString(b, msg.message)\n\t\t\t\twrite(b)\n\t\t\t}\n\t\t}\n\t\ts.statsTotalMsgsSent.Add(1)\n\t}\n\n\tm := [2]map[string]bool{\n\t\tmake(map[string]bool), // pubsubChannel\n\t\tmake(map[string]bool), // pubsubPattern\n\t}\n\n\ttarget := newSubtarget()\n\n\tdefer func() {\n\t\tfor i := 0; i < 2; i++ {\n\t\t\tfor channel := range m[i] {\n\t\t\t\ts.pubsub.unregister(i, channel, target)\n\t\t\t}\n\t\t}\n\t\ttarget.cond.L.Lock()\n\t\ttarget.closed = true\n\t\ttarget.cond.Broadcast()\n\t\ttarget.cond.L.Unlock()\n\t}()\n\tgo func() {\n\t\tlog.Debugf(\"pubsub open\")\n\t\tdefer log.Debugf(\"pubsub closed\")\n\t\ttarget.cond.L.Lock()\n\t\tdefer target.cond.L.Unlock()\n\t\tfor {\n\t\t\tfor len(target.msgs) > 0 {\n\t\t\t\tmsgs := target.msgs\n\t\t\t\ttarget.msgs = nil\n\t\t\t\ttarget.cond.L.Unlock()\n\t\t\t\tfor _, msg := range msgs {\n\t\t\t\t\twriteMessage(msg)\n\t\t\t\t}\n\t\t\t\ttarget.cond.L.Lock()\n\t\t\t}\n\t\t\tif target.closed {\n\t\t\t\tbreak\n\t\t\t}\n\t\t\ttarget.cond.Wait()\n\t\t}\n\t}()\n\n\tmsgs := []*Message{msg}\n\tfor {\n\t\tfor _, msg := range msgs {\n\t\t\tstart = time.Now()\n\t\t\tvar kind int\n\t\t\tvar un bool\n\t\t\tswitch msg.Command() {\n\t\t\tcase \"quit\":\n\t\t\t\twriteOK()\n\t\t\t\treturn nil\n\t\t\tcase \"ping\":\n\t\t\t\twritePing(msg)\n\t\t\t\tcontinue\n\t\t\tcase \"psubscribe\":\n\t\t\t\tkind, un = pubsubPattern, false\n\t\t\tcase \"punsubscribe\":\n\t\t\t\tkind, un = pubsubPattern, true\n\t\t\tcase \"subscribe\":\n\t\t\t\tkind, un = pubsubChannel, false\n\t\t\tcase \"unsubscribe\":\n\t\t\t\tkind, un = pubsubChannel, true\n\t\t\tdefault:\n\t\t\t\twriteOnlyPubsubErr()\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tif len(msg.Args) < 2 {\n\t\t\t\twriteWrongNumberOfArgsErr(msg.Command())\n\t\t\t}\n\t\t\tfor i := 1; i < len(msg.Args); i++ {\n\t\t\t\tchannel := msg.Args[i]\n\t\t\t\tif un {\n\t\t\t\t\tdelete(m[kind], channel)\n\t\t\t\t\ts.pubsub.unregister(kind, channel, target)\n\t\t\t\t} else {\n\t\t\t\t\tm[kind][channel] = true\n\t\t\t\t\ts.pubsub.register(kind, channel, target)\n\t\t\t\t}\n\t\t\t\twriteSubscribe(msg.Command(), channel, len(m[0])+len(m[1]))\n\t\t\t}\n\t\t}\n\t\tvar err error\n\t\tmsgs, err = rd.ReadMessages()\n\t\tif err != nil {\n\t\t\tif err == io.EOF {\n\t\t\t\treturn nil\n\t\t\t}\n\t\t\treturn err\n\t\t}\n\t}\n}\n" }, { "path": "internal/server/readonly.go", "content": "package server\n\nimport (\n\t\"time\"\n\n\t\"github.com/tidwall/resp\"\n\t\"github.com/tidwall/tile38/internal/log\"\n)\n\n// READONLY yes|no\nfunc (s *Server) cmdREADONLY(msg *Message) (resp.Value, error) {\n\tstart := time.Now()\n\n\t// >> Args\n\n\targs := msg.Args\n\tif len(args) != 2 {\n\t\treturn retrerr(errInvalidNumberOfArguments)\n\t}\n\n\tswitch args[1] {\n\tcase \"yes\", \"no\":\n\tdefault:\n\t\treturn retrerr(errInvalidArgument(args[1]))\n\t}\n\n\t// >> Operation\n\n\tvar updated bool\n\tif args[1] == \"yes\" {\n\t\tif !s.config.readOnly() {\n\t\t\tupdated = true\n\t\t\ts.config.setReadOnly(true)\n\t\t\tlog.Info(\"read only\")\n\t\t}\n\t} else {\n\t\tif s.config.readOnly() {\n\t\t\tupdated = true\n\t\t\ts.config.setReadOnly(false)\n\t\t\tlog.Info(\"read write\")\n\t\t}\n\t}\n\tif updated {\n\t\ts.config.write(false)\n\t}\n\n\t// >> Response\n\n\treturn OKMessage(msg, start), nil\n}\n" }, { "path": "internal/server/respconn.go", "content": "package server\n\nimport (\n\t\"net\"\n\t\"time\"\n\n\t\"github.com/tidwall/resp\"\n)\n\n// RESPConn represents a simple resp connection.\ntype RESPConn struct {\n\tconn net.Conn\n\trd *resp.Reader\n\twr *resp.Writer\n}\n\n// DialTimeout dials a resp\nfunc DialTimeout(address string, timeout time.Duration) (*RESPConn, error) {\n\ttcpconn, err := net.DialTimeout(\"tcp\", address, timeout)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tconn := &RESPConn{\n\t\tconn: tcpconn,\n\t\trd: resp.NewReader(tcpconn),\n\t\twr: resp.NewWriter(tcpconn),\n\t}\n\treturn conn, nil\n}\n\n// Close closes the connection.\nfunc (conn *RESPConn) Close() error {\n\tconn.wr.WriteMultiBulk(\"quit\")\n\treturn conn.conn.Close()\n}\n\n// Do performs a command and returns a resp value.\nfunc (conn *RESPConn) Do(commandName string, args ...interface{}) (\n\tval resp.Value, err error,\n) {\n\tif err := conn.wr.WriteMultiBulk(commandName, args...); err != nil {\n\t\treturn val, err\n\t}\n\tval, _, err = conn.rd.ReadValue()\n\treturn val, err\n}\n" }, { "path": "internal/server/scan.go", "content": "package server\n\nimport (\n\t\"bytes\"\n\t\"time\"\n\n\t\"github.com/tidwall/resp\"\n\t\"github.com/tidwall/tile38/internal/object\"\n)\n\nfunc (s *Server) cmdScanArgs(vs []string) (\n\tls liveFenceSwitches, err error,\n) {\n\tvar t searchScanBaseTokens\n\tvs, t, err = s.parseSearchScanBaseTokens(\"scan\", t, vs)\n\tif err != nil {\n\t\treturn\n\t}\n\tls.searchScanBaseTokens = t\n\tif len(vs) != 0 {\n\t\terr = errInvalidNumberOfArguments\n\t\treturn\n\t}\n\treturn\n}\n\nfunc (s *Server) cmdScan(msg *Message) (res resp.Value, err error) {\n\tstart := time.Now()\n\tvs := msg.Args[1:]\n\n\targs, err := s.cmdScanArgs(vs)\n\tif args.usingLua() {\n\t\tdefer args.Close()\n\t\tdefer func() {\n\t\t\t// if r := recover(); r != nil {\n\t\t\t// \tres = NOMessage\n\t\t\t// \terr = fmt.Errorf(\"%v\", r)\n\t\t\t// \treturn\n\t\t\t// }\n\t\t}()\n\t}\n\tif err != nil {\n\t\treturn NOMessage, err\n\t}\n\twr := &bytes.Buffer{}\n\tsw, err := s.newScanWriter(\n\t\twr, msg, args.key, args.output, args.precision, args.globs, false,\n\t\targs.cursor, args.limit, args.wheres, args.whereins, args.whereevals,\n\t\targs.nofields, args.mvt, args.tileX, args.tileY, args.tileZ)\n\tif err != nil {\n\t\treturn NOMessage, err\n\t}\n\tif msg.OutputType == JSON {\n\t\twr.WriteString(`{\"ok\":true`)\n\t}\n\tvar ierr error\n\tif sw.col != nil {\n\t\tif sw.output == outputCount && len(sw.wheres) == 0 &&\n\t\t\tlen(sw.whereins) == 0 && len(sw.whereevals) == 0 &&\n\t\t\tsw.globEverything {\n\t\t\tcount := sw.col.Count() - int(args.cursor)\n\t\t\tif count < 0 {\n\t\t\t\tcount = 0\n\t\t\t}\n\t\t\tsw.count = uint64(count)\n\t\t} else {\n\t\t\tlimits := multiGlobParse(sw.globs, args.desc)\n\t\t\tif limits[0] == \"\" && limits[1] == \"\" {\n\t\t\t\tsw.col.Scan(args.desc, sw,\n\t\t\t\t\tmsg.Deadline,\n\t\t\t\t\tfunc(o *object.Object) bool {\n\t\t\t\t\t\tkeepGoing, err := sw.pushObject(ScanWriterParams{\n\t\t\t\t\t\t\tobj: o,\n\t\t\t\t\t\t})\n\t\t\t\t\t\tif err != nil {\n\t\t\t\t\t\t\tierr = err\n\t\t\t\t\t\t\treturn false\n\t\t\t\t\t\t}\n\t\t\t\t\t\treturn keepGoing\n\t\t\t\t\t},\n\t\t\t\t)\n\t\t\t} else {\n\t\t\t\tsw.col.ScanRange(limits[0], limits[1], args.desc, sw,\n\t\t\t\t\tmsg.Deadline,\n\t\t\t\t\tfunc(o *object.Object) bool {\n\t\t\t\t\t\tkeepGoing, err := sw.pushObject(ScanWriterParams{\n\t\t\t\t\t\t\tobj: o,\n\t\t\t\t\t\t})\n\t\t\t\t\t\tif err != nil {\n\t\t\t\t\t\t\tierr = err\n\t\t\t\t\t\t\treturn false\n\t\t\t\t\t\t}\n\t\t\t\t\t\treturn keepGoing\n\t\t\t\t\t},\n\t\t\t\t)\n\t\t\t}\n\t\t}\n\t}\n\tif ierr != nil {\n\t\treturn retrerr(ierr)\n\t}\n\tsw.writeFoot()\n\tif msg.OutputType == JSON {\n\t\twr.WriteString(`,\"elapsed\":\"` + time.Since(start).String() + \"\\\"}\")\n\t\treturn resp.BytesValue(wr.Bytes()), nil\n\t}\n\treturn sw.respOut, nil\n}\n" }, { "path": "internal/server/scanner.go", "content": "package server\n\nimport (\n\t\"bytes\"\n\t\"encoding/base64\"\n\t\"errors\"\n\t\"math\"\n\t\"strconv\"\n\t\"strings\"\n\n\t\"github.com/mmcloughlin/geohash\"\n\t\"github.com/tidwall/btree\"\n\t\"github.com/tidwall/geojson\"\n\t\"github.com/tidwall/gjson\"\n\t\"github.com/tidwall/resp\"\n\t\"github.com/tidwall/tile38/internal/clip\"\n\t\"github.com/tidwall/tile38/internal/collection\"\n\t\"github.com/tidwall/tile38/internal/field\"\n\t\"github.com/tidwall/tile38/internal/glob\"\n\t\"github.com/tidwall/tile38/internal/object\"\n)\n\nconst limitItems = 100\n\ntype outputT int\n\nconst (\n\toutputUnknown outputT = iota\n\toutputIDs\n\toutputObjects\n\toutputCount\n\toutputPoints\n\toutputHashes\n\toutputBounds\n)\n\ntype scanWriter struct {\n\ts *Server\n\twr *bytes.Buffer\n\tname string\n\tmsg *Message\n\tcol *collection.Collection\n\tfkeys btree.Set[string]\n\toutput outputT\n\twheres []whereT\n\twhereins []whereinT\n\twhereevals []whereevalT\n\tnumberIters uint64\n\tnumberItems uint64\n\tnofields bool\n\tcursor uint64\n\tlimit uint64\n\thitLimit bool\n\tonce bool\n\tcount uint64\n\tprecision uint64\n\tglobs []string\n\tglobEverything bool\n\tfullFields bool\n\tvalues []resp.Value\n\tmatchValues bool\n\trespOut resp.Value\n\tfilled []ScanWriterParams\n\tmvtObjs []mvtObj\n\tmvt bool\n\ttileX int\n\ttileY int\n\ttileZ int\n}\n\ntype ScanWriterParams struct {\n\tobj *object.Object\n\tdist float64\n\tdistOutput bool // query or fence requested distance output\n\tnoTest bool\n\tignoreGlobMatch bool\n\tclip geojson.Object\n\tskipTesting bool\n}\n\nfunc (s *Server) newScanWriter(\n\twr *bytes.Buffer, msg *Message, name string, output outputT,\n\tprecision uint64, globs []string, matchValues bool,\n\tcursor, limit uint64, wheres []whereT, whereins []whereinT,\n\twhereevals []whereevalT, nofields, mvt bool, tileX, tileY, tileZ int,\n) (\n\t*scanWriter, error,\n) {\n\tswitch output {\n\tdefault:\n\t\treturn nil, errors.New(\"invalid output type\")\n\tcase outputIDs, outputObjects, outputCount, outputBounds, outputPoints,\n\t\toutputHashes:\n\t}\n\tif limit == 0 {\n\t\tif output == outputCount {\n\t\t\tlimit = math.MaxUint64\n\t\t} else {\n\t\t\tlimit = limitItems\n\t\t}\n\t}\n\tsw := &scanWriter{\n\t\ts: s,\n\t\twr: wr,\n\t\tname: name,\n\t\tmsg: msg,\n\t\tglobs: globs,\n\t\tlimit: limit,\n\t\tcursor: cursor,\n\t\toutput: output,\n\t\tnofields: nofields,\n\t\tprecision: precision,\n\t\twhereevals: whereevals,\n\t\tmatchValues: matchValues,\n\t}\n\n\tsw.mvt = mvt\n\tsw.tileX = tileX\n\tsw.tileY = tileY\n\tsw.tileZ = tileZ\n\n\tif len(globs) == 0 || (len(globs) == 1 && globs[0] == \"*\") {\n\t\tsw.globEverything = true\n\t}\n\tsw.wheres = wheres\n\tsw.whereins = whereins\n\tsw.col, _ = sw.s.cols.Get(sw.name)\n\treturn sw, nil\n}\n\nfunc (sw *scanWriter) hasFieldsOutput() bool {\n\tswitch sw.output {\n\tdefault:\n\t\treturn false\n\tcase outputObjects, outputPoints, outputHashes, outputBounds:\n\t\treturn !sw.nofields\n\t}\n}\n\nfunc (sw *scanWriter) writeFoot() {\n\tif sw.mvt {\n\t\tsw.wr.WriteString(`,\"mvt\":\"`)\n\t} else {\n\t\tswitch sw.msg.OutputType {\n\t\tcase JSON:\n\t\t\tif sw.fkeys.Len() > 0 && sw.hasFieldsOutput() {\n\t\t\t\tsw.wr.WriteString(`,\"fields\":[`)\n\t\t\t\tvar i int\n\t\t\t\tsw.fkeys.Scan(func(name string) bool {\n\t\t\t\t\tif i > 0 {\n\t\t\t\t\t\tsw.wr.WriteByte(',')\n\t\t\t\t\t}\n\t\t\t\t\tsw.wr.WriteString(jsonString(name))\n\t\t\t\t\ti++\n\t\t\t\t\treturn true\n\t\t\t\t})\n\t\t\t\tsw.wr.WriteByte(']')\n\t\t\t}\n\t\t\tswitch sw.output {\n\t\t\tcase outputIDs:\n\t\t\t\tsw.wr.WriteString(`,\"ids\":[`)\n\t\t\tcase outputObjects:\n\t\t\t\tsw.wr.WriteString(`,\"objects\":[`)\n\t\t\tcase outputPoints:\n\t\t\t\tsw.wr.WriteString(`,\"points\":[`)\n\t\t\tcase outputBounds:\n\t\t\t\tsw.wr.WriteString(`,\"bounds\":[`)\n\t\t\tcase outputHashes:\n\t\t\t\tsw.wr.WriteString(`,\"hashes\":[`)\n\t\t\tcase outputCount:\n\n\t\t\t}\n\t\tcase RESP:\n\t\t}\n\t}\n\tvar mvtTile []byte\n\tif sw.mvt {\n\t\tmvtTile = mvtRender(sw.tileX, sw.tileY, sw.tileZ, sw.mvtObjs)\n\t} else {\n\t\tfor _, opts := range sw.filled {\n\t\t\tsw.writeFilled(opts)\n\t\t}\n\t}\n\tcursor := sw.numberIters\n\tif !sw.hitLimit {\n\t\tcursor = 0\n\t}\n\tswitch sw.msg.OutputType {\n\tcase JSON:\n\t\tif sw.mvt {\n\t\t\tsw.wr.WriteString(base64.RawStdEncoding.EncodeToString(mvtTile))\n\t\t\tsw.wr.WriteByte('\"')\n\t\t} else {\n\t\t\tswitch sw.output {\n\t\t\tdefault:\n\t\t\t\tsw.wr.WriteByte(']')\n\t\t\tcase outputCount:\n\t\t\t}\n\t\t}\n\t\tsw.wr.WriteString(`,\"count\":` + strconv.FormatUint(sw.count, 10))\n\t\tsw.wr.WriteString(`,\"cursor\":` + strconv.FormatUint(cursor, 10))\n\tcase RESP:\n\t\tif sw.output == outputCount {\n\t\t\tsw.respOut = resp.IntegerValue(int(sw.count))\n\t\t} else {\n\t\t\tvalues := []resp.Value{resp.IntegerValue(int(cursor))}\n\t\t\tif sw.mvt {\n\t\t\t\tvalues = append(values, resp.BytesValue(mvtTile))\n\t\t\t} else {\n\t\t\t\tvalues = append(values, resp.ArrayValue(sw.values))\n\t\t\t}\n\t\t\tsw.respOut = resp.ArrayValue(values)\n\t\t}\n\t}\n}\n\nfunc extractZCoordinate(o geojson.Object) float64 {\n\tfor {\n\t\tswitch g := o.(type) {\n\t\tcase *geojson.Point:\n\t\t\treturn g.Z()\n\t\tcase *geojson.Feature:\n\t\t\to = g.Base()\n\t\tdefault:\n\t\t\treturn 0\n\t\t}\n\t}\n}\n\nfunc isPathKey(s string, key string) bool {\n\treturn strings.HasPrefix(s, key) && (s == key || s[len(key)] == '.')\n}\n\nfunc getFieldValue(o *object.Object, name string) field.Value {\n\tif isPathKey(name, \"properties\") {\n\t\tif g := o.Geo(); g != nil {\n\t\t\tif res := gjson.Get(g.Members(), \"properties\"); res.Exists() {\n\t\t\t\tif name != \"properties\" {\n\t\t\t\t\t// We have a dot path suffix.\n\t\t\t\t\tres = res.Get(name[len(\"properties\")+1:])\n\t\t\t\t}\n\t\t\t\tif res.Exists() {\n\t\t\t\t\treturn field.ValueOf(res.Raw)\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\tif name == \"z\" {\n\t\tz := extractZCoordinate(o.Geo())\n\t\treturn field.ValueOf(strconv.FormatFloat(z, 'f', -1, 64))\n\t}\n\treturn o.Fields().Get(name).Value()\n}\n\nfunc (sw *scanWriter) fieldMatch(o *object.Object) (bool, error) {\n\tfor _, where := range sw.wheres {\n\t\tif where.expr {\n\t\t\tif !where.matchExpr(sw.s, o) {\n\t\t\t\treturn false, nil\n\t\t\t}\n\t\t} else {\n\t\t\tif !where.matchField(getFieldValue(o, where.name)) {\n\t\t\t\treturn false, nil\n\t\t\t}\n\t\t}\n\t}\n\tfor _, wherein := range sw.whereins {\n\t\tif !wherein.match(getFieldValue(o, wherein.name)) {\n\t\t\treturn false, nil\n\t\t}\n\t}\n\tif len(sw.whereevals) > 0 {\n\t\tfieldNames := make(map[string]field.Value)\n\t\tvar props string\n\t\tif objIsSpatial(o.Geo()) {\n\t\t\tz := extractZCoordinate(o.Geo())\n\t\t\tfieldNames[\"z\"] = field.ValueOf(strconv.FormatFloat(z, 'f', -1, 64))\n\t\t\tprops = gjson.Get(o.Geo().Members(), \"properties\").Raw\n\t\t}\n\t\to.Fields().Scan(func(f field.Field) bool {\n\t\t\tfieldNames[f.Name()] = f.Value()\n\t\t\treturn true\n\t\t})\n\t\tfor _, whereval := range sw.whereevals {\n\t\t\tmatch, err := whereval.match(fieldNames, o.ID(), props)\n\t\t\tif err != nil {\n\t\t\t\treturn false, err\n\t\t\t}\n\t\t\tif !match {\n\t\t\t\treturn false, nil\n\t\t\t}\n\t\t}\n\t}\n\treturn true, nil\n}\n\nfunc (sw *scanWriter) globMatch(o *object.Object) (ok, keepGoing bool) {\n\tif sw.globEverything {\n\t\treturn true, true\n\t}\n\tvar val string\n\tif sw.matchValues {\n\t\tval = o.String()\n\t} else {\n\t\tval = o.ID()\n\t}\n\tfor _, pattern := range sw.globs {\n\t\tok, _ := glob.Match(pattern, val)\n\t\tif ok {\n\t\t\treturn true, true\n\t\t}\n\t}\n\treturn false, true\n}\n\n// Increment cursor\nfunc (sw *scanWriter) Offset() uint64 {\n\treturn sw.cursor\n}\n\nfunc (sw *scanWriter) Step(n uint64) {\n\tsw.numberIters += n\n}\n\n// ok is whether the object passes the test and should be written\n// keepGoing is whether there could be more objects to test\nfunc (sw *scanWriter) testObject(o *object.Object,\n) (ok, keepGoing bool, err error) {\n\tmatch, kg := sw.globMatch(o)\n\tif !match {\n\t\treturn false, kg, nil\n\t}\n\tok, err = sw.fieldMatch(o)\n\tif err != nil {\n\t\treturn false, false, err\n\t}\n\treturn ok, true, nil\n}\n\nfunc (sw *scanWriter) pushObject(opts ScanWriterParams) (keepGoing bool,\n\terr error,\n) {\n\tkeepGoing = true\n\tif !opts.noTest {\n\t\tvar ok bool\n\t\tvar err error\n\t\tok, keepGoing, err = sw.testObject(opts.obj)\n\t\tif err != nil {\n\t\t\treturn false, err\n\t\t}\n\t\tif !ok {\n\t\t\treturn keepGoing, nil\n\t\t}\n\t}\n\tsw.count++\n\tif sw.output == outputCount {\n\t\treturn sw.count < sw.limit, nil\n\t}\n\tif opts.clip != nil {\n\t\t// create a newly clipped object\n\t\topts.obj = object.New(\n\t\t\topts.obj.ID(),\n\t\t\tclip.Clip(opts.obj.Geo(), opts.clip, &sw.s.geomIndexOpts),\n\t\t\topts.obj.Expires(),\n\t\t\topts.obj.Fields(),\n\t\t)\n\t}\n\tif sw.mvt {\n\t\tsw.mvtObjs = append(sw.mvtObjs, mvtObj{opts.obj.ID(), opts.obj.Geo()})\n\t}\n\tif !sw.fullFields {\n\t\topts.obj.Fields().Scan(func(f field.Field) bool {\n\t\t\tsw.fkeys.Insert(f.Name())\n\t\t\treturn true\n\t\t})\n\t}\n\tsw.filled = append(sw.filled, opts)\n\tsw.numberItems++\n\tif sw.numberItems == sw.limit {\n\t\tsw.hitLimit = true\n\t\treturn false, nil\n\t}\n\treturn keepGoing, nil\n}\n\nfunc (sw *scanWriter) writeObject(opts ScanWriterParams) {\n\tn := len(sw.filled)\n\tsw.pushObject(opts)\n\tif len(sw.filled) > n {\n\t\tsw.writeFilled(sw.filled[len(sw.filled)-1])\n\t\tsw.filled = sw.filled[:n]\n\t}\n}\n\nfunc (sw *scanWriter) writeFilled(opts ScanWriterParams) {\n\tswitch sw.msg.OutputType {\n\tcase JSON:\n\t\tvar wr bytes.Buffer\n\t\tvar jsfields string\n\t\tif sw.once {\n\t\t\twr.WriteByte(',')\n\t\t} else {\n\t\t\tsw.once = true\n\t\t}\n\t\tfieldsOutput := sw.hasFieldsOutput()\n\t\tif fieldsOutput && sw.fullFields {\n\t\t\tif opts.obj.Fields().Len() > 0 {\n\t\t\t\tjsfields = `,\"fields\":{`\n\t\t\t\tvar i int\n\t\t\t\topts.obj.Fields().Scan(func(f field.Field) bool {\n\t\t\t\t\tif !f.Value().IsZero() {\n\t\t\t\t\t\tif i > 0 {\n\t\t\t\t\t\t\tjsfields += `,`\n\t\t\t\t\t\t}\n\t\t\t\t\t\tjsfields += jsonString(f.Name()) + \":\" + f.Value().JSON()\n\t\t\t\t\t\ti++\n\t\t\t\t\t}\n\t\t\t\t\treturn true\n\t\t\t\t})\n\t\t\t\tjsfields += `}`\n\t\t\t}\n\t\t} else if fieldsOutput && sw.fkeys.Len() > 0 && !sw.fullFields {\n\t\t\tjsfields = `,\"fields\":[`\n\t\t\tvar i int\n\t\t\tsw.fkeys.Scan(func(name string) bool {\n\t\t\t\tif i > 0 {\n\t\t\t\t\tjsfields += `,`\n\t\t\t\t}\n\t\t\t\tf := opts.obj.Fields().Get(name)\n\t\t\t\tjsfields += f.Value().JSON()\n\t\t\t\ti++\n\t\t\t\treturn true\n\t\t\t})\n\t\t\tjsfields += `]`\n\t\t}\n\t\tif sw.output == outputIDs {\n\t\t\tif opts.distOutput || opts.dist > 0 {\n\t\t\t\twr.WriteString(`{\"id\":` + jsonString(opts.obj.ID()) +\n\t\t\t\t\t`,\"distance\":` + strconv.FormatFloat(opts.dist, 'f', -1, 64) + \"}\")\n\t\t\t} else {\n\t\t\t\twr.WriteString(jsonString(opts.obj.ID()))\n\t\t\t}\n\t\t} else {\n\t\t\twr.WriteString(`{\"id\":` + jsonString(opts.obj.ID()))\n\t\t\tswitch sw.output {\n\t\t\tcase outputObjects:\n\t\t\t\twr.WriteString(`,\"object\":` + string(opts.obj.Geo().AppendJSON(nil)))\n\t\t\tcase outputPoints:\n\t\t\t\twr.WriteString(`,\"point\":` + string(appendJSONSimplePoint(nil, opts.obj.Geo())))\n\t\t\tcase outputHashes:\n\t\t\t\tcenter := opts.obj.Geo().Center()\n\t\t\t\tp := geohash.EncodeWithPrecision(center.Y, center.X, uint(sw.precision))\n\t\t\t\twr.WriteString(`,\"hash\":\"` + p + `\"`)\n\t\t\tcase outputBounds:\n\t\t\t\twr.WriteString(`,\"bounds\":` + string(appendJSONSimpleBounds(nil, opts.obj.Geo())))\n\t\t\t}\n\t\t\twr.WriteString(jsfields)\n\t\t\tif opts.distOutput || opts.dist > 0 {\n\t\t\t\twr.WriteString(`,\"distance\":` + strconv.FormatFloat(opts.dist, 'f', -1, 64))\n\t\t\t}\n\n\t\t\twr.WriteString(`}`)\n\t\t}\n\t\tsw.wr.Write(wr.Bytes())\n\tcase RESP:\n\t\tvals := make([]resp.Value, 1, 3)\n\t\tvals[0] = resp.StringValue(opts.obj.ID())\n\t\tif sw.output == outputIDs {\n\t\t\tif opts.distOutput || opts.dist > 0 {\n\t\t\t\tvals = append(vals, resp.FloatValue(opts.dist))\n\t\t\t\tsw.values = append(sw.values, resp.ArrayValue(vals))\n\t\t\t} else {\n\t\t\t\tsw.values = append(sw.values, vals[0])\n\t\t\t}\n\t\t} else {\n\t\t\tswitch sw.output {\n\t\t\tcase outputObjects:\n\t\t\t\tvals = append(vals, resp.StringValue(opts.obj.String()))\n\t\t\tcase outputPoints:\n\t\t\t\tpoint := opts.obj.Geo().Center()\n\t\t\t\tz := extractZCoordinate(opts.obj.Geo())\n\t\t\t\tif z != 0 {\n\t\t\t\t\tvals = append(vals, resp.ArrayValue([]resp.Value{\n\t\t\t\t\t\tresp.FloatValue(point.Y),\n\t\t\t\t\t\tresp.FloatValue(point.X),\n\t\t\t\t\t\tresp.FloatValue(z),\n\t\t\t\t\t}))\n\t\t\t\t} else {\n\t\t\t\t\tvals = append(vals, resp.ArrayValue([]resp.Value{\n\t\t\t\t\t\tresp.FloatValue(point.Y),\n\t\t\t\t\t\tresp.FloatValue(point.X),\n\t\t\t\t\t}))\n\t\t\t\t}\n\t\t\tcase outputHashes:\n\t\t\t\tcenter := opts.obj.Geo().Center()\n\t\t\t\tp := geohash.EncodeWithPrecision(center.Y, center.X, uint(sw.precision))\n\t\t\t\tvals = append(vals, resp.StringValue(p))\n\t\t\tcase outputBounds:\n\t\t\t\tbbox := opts.obj.Rect()\n\t\t\t\tvals = append(vals, resp.ArrayValue([]resp.Value{\n\t\t\t\t\tresp.ArrayValue([]resp.Value{\n\t\t\t\t\t\tresp.FloatValue(bbox.Min.Y),\n\t\t\t\t\t\tresp.FloatValue(bbox.Min.X),\n\t\t\t\t\t}),\n\t\t\t\t\tresp.ArrayValue([]resp.Value{\n\t\t\t\t\t\tresp.FloatValue(bbox.Max.Y),\n\t\t\t\t\t\tresp.FloatValue(bbox.Max.X),\n\t\t\t\t\t}),\n\t\t\t\t}))\n\t\t\t}\n\t\t\tif sw.hasFieldsOutput() {\n\t\t\t\tvar fvals []resp.Value\n\t\t\t\tvar i int\n\t\t\t\topts.obj.Fields().Scan(func(f field.Field) bool {\n\t\t\t\t\tif !f.Value().IsZero() {\n\t\t\t\t\t\tfvals = append(fvals, resp.StringValue(f.Name()), resp.StringValue(f.Value().Data()))\n\t\t\t\t\t\ti++\n\t\t\t\t\t}\n\t\t\t\t\treturn true\n\t\t\t\t})\n\t\t\t\tif len(fvals) > 0 {\n\t\t\t\t\tvals = append(vals, resp.ArrayValue(fvals))\n\t\t\t\t}\n\t\t\t}\n\t\t\tif opts.distOutput || opts.dist > 0 {\n\t\t\t\tvals = append(vals, resp.FloatValue(opts.dist))\n\t\t\t}\n\t\t\tsw.values = append(sw.values, resp.ArrayValue(vals))\n\t\t}\n\t}\n}\n" }, { "path": "internal/server/scanner_test.go", "content": "package server\n\nimport (\n\t\"fmt\"\n\t\"math\"\n\t\"math/rand\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/tidwall/geojson\"\n\t\"github.com/tidwall/geojson/geometry\"\n\t\"github.com/tidwall/tile38/internal/field\"\n\t\"github.com/tidwall/tile38/internal/object\"\n)\n\ntype testPointItem struct {\n\tobject geojson.Object\n\tfields field.List\n}\n\nfunc PO(x, y float64) *geojson.Point {\n\treturn geojson.NewPoint(geometry.Point{X: x, Y: y})\n}\n\nfunc BenchmarkFieldMatch(t *testing.B) {\n\trand.Seed(time.Now().UnixNano())\n\titems := make([]testPointItem, t.N)\n\tfor i := 0; i < t.N; i++ {\n\t\tvar fields field.List\n\t\tfields = fields.Set(field.Make(\"foo\", fmt.Sprintf(\"%f\", rand.Float64()*9+1)))\n\t\tfields = fields.Set(field.Make(\"bar\", fmt.Sprintf(\"%f\", math.Round(rand.Float64()*30)+1)))\n\t\titems[i] = testPointItem{\n\t\t\tPO(rand.Float64()*360-180, rand.Float64()*180-90),\n\t\t\tfields,\n\t\t}\n\t}\n\tsw := &scanWriter{\n\t\twheres: []whereT{\n\t\t\t{false, \"foo\", false, field.ValueOf(\"1\"), false, field.ValueOf(\"3\")},\n\t\t\t{false, \"bar\", false, field.ValueOf(\"10\"), false, field.ValueOf(\"30\")},\n\t\t},\n\t\twhereins: []whereinT{\n\t\t\t{\"foo\", []field.Value{field.ValueOf(\"1\"), field.ValueOf(\"2\")}},\n\t\t\t{\"bar\", []field.Value{field.ValueOf(\"11\"), field.ValueOf(\"25\")}},\n\t\t},\n\t}\n\tt.ResetTimer()\n\tfor i := 0; i < t.N; i++ {\n\t\t// one call is super fast, measurements are not reliable, let's do 100\n\t\tfor ix := 0; ix < 100; ix++ {\n\t\t\tsw.fieldMatch(object.New(\"\", items[i].object, 0, items[i].fields))\n\t\t}\n\t}\n}\n" }, { "path": "internal/server/scripts.go", "content": "package server\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"crypto/sha1\"\n\t\"encoding/hex\"\n\t\"encoding/json\"\n\t\"errors\"\n\t\"fmt\"\n\t\"math\"\n\t\"strconv\"\n\t\"strings\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/tidwall/geojson/geo\"\n\t\"github.com/tidwall/resp\"\n\t\"github.com/tidwall/tile38/internal/log\"\n\t\"github.com/tidwall/tinylru\"\n\tlua \"github.com/yuin/gopher-lua\"\n\tluajson \"layeh.com/gopher-json\"\n)\n\nconst (\n\tiniLuaPoolSize = 5\n\tmaxLuaPoolSize = 1000\n)\n\n// For Lua os.clock() impl\nvar startedAt = time.Now()\n\nvar errShaNotFound = errors.New(\"sha not found\")\nvar errCmdNotSupported = errors.New(\"command not supported in scripts\")\nvar errNotLeader = errors.New(\"not the leader\")\nvar errReadOnly = errors.New(\"read only\")\nvar errCatchingUp = errors.New(\"catching up to leader\")\nvar errNoLuasAvailable = errors.New(\"no interpreters available\")\nvar errTimeout = errors.New(\"timeout\")\n\n// Go-routine-safe pool of read-to-go lua states\ntype lStatePool struct {\n\tm sync.Mutex\n\ts *Server\n\tsaved []*lua.LState\n\ttotal int\n}\n\n// newPool returns a new pool of lua states\nfunc (s *Server) newPool() *lStatePool {\n\tpl := &lStatePool{\n\t\tsaved: make([]*lua.LState, iniLuaPoolSize),\n\t\ts: s,\n\t}\n\t// Fill the pool with some ready handlers\n\tfor i := 0; i < iniLuaPoolSize; i++ {\n\t\tpl.saved[i] = pl.New()\n\t\tpl.total++\n\t}\n\treturn pl\n}\n\nfunc (pl *lStatePool) Get() (*lua.LState, error) {\n\tpl.m.Lock()\n\tdefer pl.m.Unlock()\n\tn := len(pl.saved)\n\tif n == 0 {\n\t\tif pl.total >= maxLuaPoolSize {\n\t\t\treturn nil, errNoLuasAvailable\n\t\t}\n\t\tpl.total++\n\t\treturn pl.New(), nil\n\t}\n\tx := pl.saved[n-1]\n\tpl.saved = pl.saved[0 : n-1]\n\treturn x, nil\n}\n\n// Prune removes some of the idle lua states from the pool\nfunc (pl *lStatePool) Prune() {\n\tpl.m.Lock()\n\tn := len(pl.saved)\n\tif n > iniLuaPoolSize {\n\t\t// drop half of the idle states that is above the minimum\n\t\tdropNum := (n - iniLuaPoolSize) / 2\n\t\tif dropNum < 1 {\n\t\t\tdropNum = 1\n\t\t}\n\t\tnewSaved := make([]*lua.LState, n-dropNum)\n\t\tcopy(newSaved, pl.saved[dropNum:])\n\t\tpl.saved = newSaved\n\t\tpl.total -= dropNum\n\t}\n\tpl.m.Unlock()\n}\n\nfunc (pl *lStatePool) New() *lua.LState {\n\t// Prevent opening all Lua modules\n\tL := lua.NewState(lua.Options{SkipOpenLibs: true})\n\n\tallowedModules := []struct {\n\t\tmoduleName string\n\t\tmoduleFn lua.LGFunction\n\t}{\n\t\t{lua.BaseLibName, openBaseSubset},\n\t\t{lua.TabLibName, lua.OpenTable},\n\t\t{lua.MathLibName, lua.OpenMath},\n\t\t{lua.StringLibName, lua.OpenString},\n\t\t{lua.OsLibName, openOsSubset}, // See below for impl, only opens clock/difftime\n\t}\n\n\t// Open non-vulnerable modules (i.e. NOT io/os)\n\tfor _, pair := range allowedModules {\n\t\tif err := L.CallByParam(lua.P{\n\t\t\tFn: L.NewFunction(pair.moduleFn),\n\t\t\tNRet: 0,\n\t\t\tProtect: true,\n\t\t}, lua.LString(pair.moduleName)); err != nil {\n\t\t\tpanic(err)\n\t\t}\n\t}\n\n\tgetArgs := func(ls *lua.LState) (evalCmd string, args []string) {\n\t\tevalCmd = ls.GetGlobal(\"EVAL_CMD\").String()\n\n\t\t// Trying to work with unknown number of args.\n\t\t// When we see empty arg we call it enough.\n\t\tfor i := 1; ; i++ {\n\t\t\tif arg := ls.ToString(i); arg == \"\" {\n\t\t\t\tbreak\n\t\t\t} else {\n\t\t\t\targs = append(args, arg)\n\t\t\t}\n\t\t}\n\t\treturn\n\t}\n\tcall := func(ls *lua.LState) int {\n\t\tevalCmd, args := getArgs(ls)\n\t\tvar numRet int\n\t\tif res, err := pl.s.luaTile38Call(evalCmd, args[0], args[1:]...); err != nil {\n\t\t\tls.RaiseError(\"ERR %s\", err.Error())\n\t\t\tnumRet = 0\n\t\t} else {\n\t\t\tls.Push(ConvertToLua(ls, res))\n\t\t\tnumRet = 1\n\t\t}\n\t\treturn numRet\n\t}\n\tpcall := func(ls *lua.LState) int {\n\t\tevalCmd, args := getArgs(ls)\n\t\tif res, err := pl.s.luaTile38Call(evalCmd, args[0], args[1:]...); err != nil {\n\t\t\tls.Push(ConvertToLua(ls, resp.ErrorValue(err)))\n\t\t} else {\n\t\t\tls.Push(ConvertToLua(ls, res))\n\t\t}\n\t\treturn 1\n\n\t}\n\terrorReply := func(ls *lua.LState) int {\n\t\ttbl := L.CreateTable(0, 1)\n\t\ttbl.RawSetString(\"err\", lua.LString(ls.ToString(1)))\n\t\tls.Push(tbl)\n\t\treturn 1\n\t}\n\tstatusReply := func(ls *lua.LState) int {\n\t\ttbl := L.CreateTable(0, 1)\n\t\ttbl.RawSetString(\"ok\", lua.LString(ls.ToString(1)))\n\t\tls.Push(tbl)\n\t\treturn 1\n\t}\n\tsha1hex := func(ls *lua.LState) int {\n\t\tshaSum := Sha1Sum(ls.ToString(1))\n\t\tls.Push(lua.LString(shaSum))\n\t\treturn 1\n\t}\n\tdistanceTo := func(ls *lua.LState) int {\n\t\tdt := geo.DistanceTo(\n\t\t\tfloat64(ls.ToNumber(1)),\n\t\t\tfloat64(ls.ToNumber(2)),\n\t\t\tfloat64(ls.ToNumber(3)),\n\t\t\tfloat64(ls.ToNumber(4)))\n\t\tls.Push(lua.LNumber(dt))\n\t\treturn 1\n\t}\n\tvar exports = map[string]lua.LGFunction{\n\t\t\"call\": call,\n\t\t\"pcall\": pcall,\n\t\t\"error_reply\": errorReply,\n\t\t\"status_reply\": statusReply,\n\t\t\"sha1hex\": sha1hex,\n\t\t\"distance_to\": distanceTo,\n\t}\n\tL.SetGlobal(\"tile38\", L.SetFuncs(L.NewTable(), exports))\n\n\t// Load json\n\tL.SetGlobal(\"json\", L.Get(luajson.Loader(L)))\n\n\t// Prohibit creating new globals in this state\n\tlockNewGlobals := func(ls *lua.LState) int {\n\t\tls.RaiseError(\"attempt to create global variable '%s'\", ls.ToString(2))\n\t\treturn 0\n\t}\n\tmt := L.CreateTable(0, 1)\n\tmt.RawSetString(\"__newindex\", L.NewFunction(lockNewGlobals))\n\tL.SetMetatable(L.Get(lua.GlobalsIndex), mt)\n\n\treturn L\n}\n\nfunc (pl *lStatePool) Put(L *lua.LState) {\n\tpl.m.Lock()\n\tpl.saved = append(pl.saved, L)\n\tpl.m.Unlock()\n}\n\nfunc (pl *lStatePool) Shutdown() {\n\tpl.m.Lock()\n\tfor _, L := range pl.saved {\n\t\tL.Close()\n\t}\n\tpl.m.Unlock()\n}\n\n// Go-routine-safe map of compiled scripts\ntype lScriptMap struct {\n\tm sync.Mutex\n\tscripts map[string]*lua.FunctionProto\n\tlru tinylru.LRUG[string, *lua.FunctionProto]\n}\n\nfunc (sm *lScriptMap) Get(key string) (script *lua.FunctionProto, ok bool) {\n\tsm.m.Lock()\n\tscript, ok = sm.scripts[key]\n\tif !ok {\n\t\tscript, ok = sm.lru.Get(key)\n\t}\n\tsm.m.Unlock()\n\treturn script, ok\n}\n\nfunc (sm *lScriptMap) Put(key string, script *lua.FunctionProto) {\n\tsm.m.Lock()\n\tsm.scripts[key] = script\n\tsm.m.Unlock()\n}\n\nfunc (sm *lScriptMap) PutLRU(key string, script *lua.FunctionProto) {\n\tsm.m.Lock()\n\tsm.lru.Set(key, script)\n\tsm.m.Unlock()\n}\n\nfunc (sm *lScriptMap) Flush() {\n\tsm.m.Lock()\n\tsm.scripts = make(map[string]*lua.FunctionProto)\n\tsm.lru.Clear()\n\tsm.m.Unlock()\n}\n\n// NewScriptMap returns a new map with lua scripts\nfunc (s *Server) newScriptMap() *lScriptMap {\n\treturn &lScriptMap{\n\t\tscripts: make(map[string]*lua.FunctionProto),\n\t}\n}\n\n// ConvertToLua converts RESP value to lua LValue\nfunc ConvertToLua(L *lua.LState, val resp.Value) lua.LValue {\n\tif val.IsNull() {\n\t\treturn lua.LFalse\n\t}\n\tswitch val.Type() {\n\tcase resp.Integer:\n\t\treturn lua.LNumber(val.Integer())\n\tcase resp.BulkString:\n\t\treturn lua.LString(val.String())\n\tcase resp.Error:\n\t\ttbl := L.CreateTable(0, 1)\n\t\ttbl.RawSetString(\"err\", lua.LString(val.String()))\n\t\treturn tbl\n\tcase resp.SimpleString:\n\t\ttbl := L.CreateTable(0, 1)\n\t\ttbl.RawSetString(\"ok\", lua.LString(val.String()))\n\t\treturn tbl\n\tcase resp.Array:\n\t\ttbl := L.CreateTable(len(val.Array()), 0)\n\t\tfor _, item := range val.Array() {\n\t\t\ttbl.Append(ConvertToLua(L, item))\n\t\t}\n\t\treturn tbl\n\t}\n\treturn lua.LString(\"ERR: unknown RESP type: \" + val.Type().String())\n}\n\n// ConvertToRESP convert lua LValue to RESP value\nfunc ConvertToRESP(val lua.LValue) resp.Value {\n\tswitch val.Type() {\n\tcase lua.LTNil:\n\t\treturn resp.NullValue()\n\tcase lua.LTBool:\n\t\tif val == lua.LTrue {\n\t\t\treturn resp.IntegerValue(1)\n\t\t}\n\t\treturn resp.NullValue()\n\tcase lua.LTNumber:\n\t\tfloat := float64(val.(lua.LNumber))\n\t\tif math.IsNaN(float) || math.IsInf(float, 0) {\n\t\t\treturn resp.FloatValue(float)\n\t\t}\n\t\treturn resp.IntegerValue(int(math.Floor(float)))\n\tcase lua.LTString:\n\t\treturn resp.StringValue(val.String())\n\tcase lua.LTTable:\n\t\tvar values []resp.Value\n\t\tvar specialValues []resp.Value\n\t\tvar cb func(lk lua.LValue, lv lua.LValue)\n\t\ttbl := val.(*lua.LTable)\n\n\t\tif tbl.Len() != 0 { // list\n\t\t\tcb = func(lk lua.LValue, lv lua.LValue) {\n\t\t\t\tvalues = append(values, ConvertToRESP(lv))\n\t\t\t}\n\t\t} else { // map\n\t\t\tcb = func(lk lua.LValue, lv lua.LValue) {\n\t\t\t\tif lk.Type() == lua.LTString {\n\t\t\t\t\tlks := lk.String()\n\t\t\t\t\tswitch lks {\n\t\t\t\t\tcase \"ok\":\n\t\t\t\t\t\tspecialValues = append(specialValues, resp.SimpleStringValue(lv.String()))\n\t\t\t\t\tcase \"err\":\n\t\t\t\t\t\tspecialValues = append(specialValues, resp.ErrorValue(errors.New(lv.String())))\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t\tvalues = append(values, resp.ArrayValue(\n\t\t\t\t\t[]resp.Value{ConvertToRESP(lk), ConvertToRESP(lv)}))\n\t\t\t}\n\t\t}\n\t\ttbl.ForEach(cb)\n\t\tif len(values) == 1 && len(specialValues) == 1 {\n\t\t\treturn specialValues[0]\n\t\t}\n\t\treturn resp.ArrayValue(values)\n\t}\n\treturn resp.ErrorValue(errors.New(\"Unsupported lua type: \" + val.Type().String()))\n}\n\n// ConvertToJSON converts lua LValue to JSON string\nfunc ConvertToJSON(val lua.LValue) string {\n\tswitch val.Type() {\n\tcase lua.LTNil:\n\t\treturn \"null\"\n\tcase lua.LTBool:\n\t\tif val == lua.LTrue {\n\t\t\treturn \"true\"\n\t\t}\n\t\treturn \"false\"\n\tcase lua.LTNumber:\n\t\treturn val.String()\n\tcase lua.LTString:\n\t\tif b, err := json.Marshal(val.String()); err != nil {\n\t\t\tpanic(err)\n\t\t} else {\n\t\t\treturn string(b)\n\t\t}\n\tcase lua.LTTable:\n\t\tvar values []string\n\t\tvar cb func(lk lua.LValue, lv lua.LValue)\n\t\tvar start, end string\n\n\t\ttbl := val.(*lua.LTable)\n\t\tif tbl.Len() != 0 { // list\n\t\t\tstart = `[`\n\t\t\tend = `]`\n\t\t\tcb = func(lk lua.LValue, lv lua.LValue) {\n\t\t\t\tvalues = append(values, ConvertToJSON(lv))\n\t\t\t}\n\t\t} else { // map\n\t\t\tstart = `{`\n\t\t\tend = `}`\n\t\t\tcb = func(lk lua.LValue, lv lua.LValue) {\n\t\t\t\tvalues = append(\n\t\t\t\t\tvalues, ConvertToJSON(lk)+`:`+ConvertToJSON(lv))\n\t\t\t}\n\t\t}\n\t\ttbl.ForEach(cb)\n\t\treturn start + strings.Join(values, `,`) + end\n\t}\n\treturn \"Unsupported lua type: \" + val.Type().String()\n}\n\nfunc luaSetRawGlobals(ls *lua.LState, tbl map[string]lua.LValue) {\n\tgt := ls.Get(lua.GlobalsIndex).(*lua.LTable)\n\tfor key, val := range tbl {\n\t\tgt.RawSetString(key, val)\n\t}\n}\n\n// Sha1Sum returns a string with hex representation of sha1 sum of a given string\nfunc Sha1Sum(s string) string {\n\th := sha1.New()\n\th.Write([]byte(s))\n\treturn hex.EncodeToString(h.Sum(nil))\n}\n\n// Replace newlines with literal \\n since RESP errors cannot have newlines\nfunc makeSafeErr(err error) error {\n\treturn errors.New(strings.Replace(err.Error(), \"\\n\", `\\n`, -1))\n}\n\n// Run eval/evalro/evalna command or it's -sha variant\nfunc (s *Server) cmdEvalUnified(scriptIsSha bool, msg *Message) (res resp.Value, err error) {\n\tstart := time.Now()\n\tvs := msg.Args[1:]\n\n\tvar ok bool\n\tvar script, numkeysStr, key, arg string\n\tif vs, script, ok = tokenval(vs); !ok || script == \"\" {\n\t\treturn NOMessage, errInvalidNumberOfArguments\n\t}\n\n\tif vs, numkeysStr, ok = tokenval(vs); !ok || numkeysStr == \"\" {\n\t\treturn NOMessage, errInvalidNumberOfArguments\n\t}\n\n\tvar i, numkeys uint64\n\tif numkeys, err = strconv.ParseUint(numkeysStr, 10, 64); err != nil {\n\t\terr = errInvalidArgument(numkeysStr)\n\t\treturn\n\t}\n\n\tluaState, err := s.luapool.Get()\n\tif err != nil {\n\t\treturn\n\t}\n\tluaDeadline := lua.LNil\n\tif msg.Deadline != nil {\n\t\tdlTime := msg.Deadline.GetDeadlineTime()\n\t\tctx, cancel := context.WithDeadline(context.Background(), dlTime)\n\t\tdefer cancel()\n\t\tluaState.SetContext(ctx)\n\t\tdefer luaState.RemoveContext()\n\t\tluaDeadline = lua.LNumber(float64(dlTime.UnixNano()) / 1e9)\n\t}\n\tdefer s.luapool.Put(luaState)\n\n\tkeysTbl := luaState.CreateTable(int(numkeys), 0)\n\tfor i = 0; i < numkeys; i++ {\n\t\tif vs, key, ok = tokenval(vs); !ok || key == \"\" {\n\t\t\terr = errInvalidNumberOfArguments\n\t\t\treturn\n\t\t}\n\t\tkeysTbl.Append(lua.LString(key))\n\t}\n\n\targsTbl := luaState.CreateTable(len(vs), 0)\n\tfor len(vs) > 0 {\n\t\tif vs, arg, ok = tokenval(vs); !ok || arg == \"\" {\n\t\t\terr = errInvalidNumberOfArguments\n\t\t\treturn\n\t\t}\n\t\targsTbl.Append(lua.LString(arg))\n\t}\n\n\tvar shaSum string\n\tif scriptIsSha {\n\t\tshaSum = script\n\t} else {\n\t\tshaSum = Sha1Sum(script)\n\t}\n\n\tluaSetRawGlobals(\n\t\tluaState, map[string]lua.LValue{\n\t\t\t\"KEYS\": keysTbl,\n\t\t\t\"ARGV\": argsTbl,\n\t\t\t\"DEADLINE\": luaDeadline,\n\t\t\t\"EVAL_CMD\": lua.LString(msg.Command()),\n\t\t})\n\n\tcompiled, ok := s.luascripts.Get(shaSum)\n\tvar fn *lua.LFunction\n\tif ok {\n\t\tfn = &lua.LFunction{\n\t\t\tIsG: false,\n\t\t\tEnv: luaState.Env,\n\n\t\t\tProto: compiled,\n\t\t\tGFunction: nil,\n\t\t\tUpvalues: make([]*lua.Upvalue, 0),\n\t\t}\n\t} else if scriptIsSha {\n\t\terr = errShaNotFound\n\t\treturn\n\t} else {\n\t\tfn, err = luaState.Load(strings.NewReader(script), \"f_\"+shaSum)\n\t\tif err != nil {\n\t\t\treturn NOMessage, makeSafeErr(err)\n\t\t}\n\t\ts.luascripts.Put(shaSum, fn.Proto)\n\t}\n\tluaState.Push(fn)\n\tdefer luaSetRawGlobals(\n\t\tluaState, map[string]lua.LValue{\n\t\t\t\"KEYS\": lua.LNil,\n\t\t\t\"ARGV\": lua.LNil,\n\t\t\t\"DEADLINE\": lua.LNil,\n\t\t\t\"EVAL_CMD\": lua.LNil,\n\t\t})\n\tif err := luaState.PCall(0, 1, nil); err != nil {\n\t\tif strings.Contains(err.Error(), \"context deadline exceeded\") {\n\t\t\tmsg.Deadline.Check()\n\t\t}\n\t\tlog.Debugf(\"%v\", err.Error())\n\t\treturn NOMessage, makeSafeErr(err)\n\t}\n\tret := luaState.Get(-1) // returned value\n\tluaState.Pop(1)\n\n\tswitch msg.OutputType {\n\tcase JSON:\n\t\tvar buf bytes.Buffer\n\t\tbuf.WriteString(`{\"ok\":true`)\n\t\tbuf.WriteString(`,\"result\":` + ConvertToJSON(ret))\n\t\tbuf.WriteString(`,\"elapsed\":\"` + time.Since(start).String() + \"\\\"}\")\n\t\treturn resp.StringValue(buf.String()), nil\n\tcase RESP:\n\t\treturn ConvertToRESP(ret), nil\n\t}\n\treturn NOMessage, nil\n}\n\nfunc (s *Server) cmdScriptLoad(msg *Message) (resp.Value, error) {\n\tstart := time.Now()\n\tvs := msg.Args[1:]\n\n\tvar ok bool\n\tvar script string\n\tif _, script, ok = tokenval(vs); !ok || script == \"\" {\n\t\treturn NOMessage, errInvalidNumberOfArguments\n\t}\n\n\tshaSum := Sha1Sum(script)\n\n\tluaState, err := s.luapool.Get()\n\tif err != nil {\n\t\treturn NOMessage, err\n\t}\n\tdefer s.luapool.Put(luaState)\n\n\tfn, err := luaState.Load(strings.NewReader(script), \"f_\"+shaSum)\n\tif err != nil {\n\t\treturn NOMessage, makeSafeErr(err)\n\t}\n\ts.luascripts.Put(shaSum, fn.Proto)\n\n\tswitch msg.OutputType {\n\tcase JSON:\n\t\tvar buf bytes.Buffer\n\t\tbuf.WriteString(`{\"ok\":true`)\n\t\tbuf.WriteString(`,\"result\":\"` + shaSum + `\"`)\n\t\tbuf.WriteString(`,\"elapsed\":\"` + time.Since(start).String() + \"\\\"}\")\n\t\treturn resp.StringValue(buf.String()), nil\n\tcase RESP:\n\t\treturn resp.StringValue(shaSum), nil\n\t}\n\treturn NOMessage, nil\n}\n\nfunc (s *Server) cmdScriptExists(msg *Message) (resp.Value, error) {\n\tstart := time.Now()\n\tvs := msg.Args[1:]\n\n\tvar ok bool\n\tvar shaSum string\n\tvar results []int\n\tvar ires int\n\tfor len(vs) > 0 {\n\t\tif vs, shaSum, ok = tokenval(vs); !ok || shaSum == \"\" {\n\t\t\treturn NOMessage, errInvalidNumberOfArguments\n\t\t}\n\t\t_, ok = s.luascripts.Get(shaSum)\n\t\tif ok {\n\t\t\tires = 1\n\t\t} else {\n\t\t\tires = 0\n\t\t}\n\t\tresults = append(results, ires)\n\t}\n\n\tswitch msg.OutputType {\n\tcase JSON:\n\t\tvar buf bytes.Buffer\n\t\tbuf.WriteString(`{\"ok\":true`)\n\t\tvar resArray []string\n\t\tfor _, ires := range results {\n\t\t\tresArray = append(resArray, fmt.Sprintf(\"%d\", ires))\n\t\t}\n\t\tbuf.WriteString(`,\"result\":[` + strings.Join(resArray, \",\") + `]`)\n\t\tbuf.WriteString(`,\"elapsed\":\"` + time.Since(start).String() + \"\\\"}\")\n\t\treturn resp.StringValue(buf.String()), nil\n\tcase RESP:\n\t\tvar resArray []resp.Value\n\t\tfor _, ires := range results {\n\t\t\tresArray = append(resArray, resp.IntegerValue(ires))\n\t\t}\n\t\treturn resp.ArrayValue(resArray), nil\n\t}\n\treturn resp.SimpleStringValue(\"\"), nil\n}\n\nfunc (s *Server) cmdScriptFlush(msg *Message) (resp.Value, error) {\n\tstart := time.Now()\n\ts.luascripts.Flush()\n\n\tswitch msg.OutputType {\n\tcase JSON:\n\t\tvar buf bytes.Buffer\n\t\tbuf.WriteString(`{\"ok\":true`)\n\t\tbuf.WriteString(`,\"elapsed\":\"` + time.Since(start).String() + \"\\\"}\")\n\t\treturn resp.StringValue(buf.String()), nil\n\tcase RESP:\n\t\treturn resp.StringValue(\"OK\"), nil\n\t}\n\treturn resp.SimpleStringValue(\"\"), nil\n}\n\nfunc (s *Server) commandInScript(msg *Message) (\n\tres resp.Value, d commandDetails, err error,\n) {\n\tswitch msg.Command() {\n\tdefault:\n\t\terr = fmt.Errorf(\"unknown command '%s'\", msg.Args[0])\n\tcase \"set\":\n\t\tres, d, err = s.cmdSET(msg)\n\tcase \"fset\":\n\t\tres, d, err = s.cmdFSET(msg)\n\tcase \"del\":\n\t\tres, d, err = s.cmdDEL(msg)\n\tcase \"pdel\":\n\t\tres, d, err = s.cmdPDEL(msg)\n\tcase \"drop\":\n\t\tres, d, err = s.cmdDROP(msg)\n\tcase \"expire\":\n\t\tres, d, err = s.cmdEXPIRE(msg)\n\tcase \"rename\":\n\t\tres, d, err = s.cmdRENAME(msg)\n\tcase \"renamenx\":\n\t\tres, d, err = s.cmdRENAME(msg)\n\tcase \"persist\":\n\t\tres, d, err = s.cmdPERSIST(msg)\n\tcase \"ttl\":\n\t\tres, err = s.cmdTTL(msg)\n\tcase \"stats\":\n\t\tres, err = s.cmdSTATS(msg)\n\tcase \"scan\":\n\t\tres, err = s.cmdScan(msg)\n\tcase \"nearby\":\n\t\tres, err = s.cmdNearby(msg)\n\tcase \"within\":\n\t\tres, err = s.cmdWITHIN(msg)\n\tcase \"intersects\":\n\t\tres, err = s.cmdINTERSECTS(msg)\n\tcase \"search\":\n\t\tres, err = s.cmdSearch(msg)\n\tcase \"bounds\":\n\t\tres, err = s.cmdBOUNDS(msg)\n\tcase \"get\":\n\t\tres, err = s.cmdGET(msg)\n\tcase \"fget\":\n\t\tres, err = s.cmdFGET(msg)\n\tcase \"jget\":\n\t\tres, err = s.cmdJget(msg)\n\tcase \"jset\":\n\t\tres, d, err = s.cmdJset(msg)\n\tcase \"jdel\":\n\t\tres, d, err = s.cmdJdel(msg)\n\tcase \"type\":\n\t\tres, err = s.cmdTYPE(msg)\n\tcase \"keys\":\n\t\tres, err = s.cmdKEYS(msg)\n\tcase \"exists\":\n\t\tres, err = s.cmdEXISTS(msg)\n\tcase \"fexists\":\n\t\tres, err = s.cmdFEXISTS(msg)\n\tcase \"test\":\n\t\tres, err = s.cmdTEST(msg)\n\tcase \"server\":\n\t\tres, err = s.cmdSERVER(msg)\n\t}\n\ts.sendMonitor(err, msg, nil, true)\n\treturn\n}\n\nfunc (s *Server) luaTile38Call(evalcmd string, cmd string, args ...string) (resp.Value, error) {\n\tmsg := &Message{}\n\tmsg.OutputType = RESP\n\tmsg.Args = append([]string{cmd}, args...)\n\n\tif msg.Command() == \"timeout\" {\n\t\tif err := rewriteTimeoutMsg(msg); err != nil {\n\t\t\treturn resp.NullValue(), err\n\t\t}\n\t}\n\n\tswitch msg.Command() {\n\tcase \"ping\", \"echo\", \"auth\", \"massinsert\", \"shutdown\", \"gc\",\n\t\t\"sethook\", \"pdelhook\", \"delhook\",\n\t\t\"follow\", \"readonly\", \"config\", \"output\", \"client\",\n\t\t\"aofshrink\",\n\t\t\"script load\", \"script exists\", \"script flush\",\n\t\t\"eval\", \"evalsha\", \"evalro\", \"evalrosha\", \"evalna\", \"evalnasha\":\n\t\treturn resp.NullValue(), errCmdNotSupported\n\t}\n\n\tswitch evalcmd {\n\tcase \"eval\", \"evalsha\":\n\t\treturn s.luaTile38AtomicRW(msg)\n\tcase \"evalro\", \"evalrosha\":\n\t\treturn s.luaTile38AtomicRO(msg)\n\tcase \"evalna\", \"evalnasha\":\n\t\treturn s.luaTile38NonAtomic(msg)\n\t}\n\n\treturn resp.NullValue(), errCmdNotSupported\n}\n\n// The eval command has already got the lock. No locking on the call from within the script.\nfunc (s *Server) luaTile38AtomicRW(msg *Message) (resp.Value, error) {\n\tvar write bool\n\n\tswitch msg.Command() {\n\tdefault:\n\t\treturn resp.NullValue(), errCmdNotSupported\n\tcase \"set\", \"del\", \"drop\", \"fset\", \"flushdb\", \"expire\", \"persist\", \"jset\", \"pdel\",\n\t\t\"rename\", \"renamenx\":\n\t\t// write operations\n\t\twrite = true\n\t\tif s.config.followHost() != \"\" {\n\t\t\treturn resp.NullValue(), errNotLeader\n\t\t}\n\t\tif s.config.readOnly() {\n\t\t\treturn resp.NullValue(), errReadOnly\n\t\t}\n\tcase \"get\", \"keys\", \"scan\", \"nearby\", \"within\", \"intersects\", \"hooks\", \"search\",\n\t\t\"ttl\", \"bounds\", \"server\", \"info\", \"type\", \"jget\", \"fget\", \"exists\", \"fexists\", \"test\":\n\t\t// read operations\n\t\tif s.config.followHost() != \"\" && !s.caughtUpOnce() {\n\t\t\treturn resp.NullValue(), errCatchingUp\n\t\t}\n\t}\n\n\tres, d, err := func() (res resp.Value, d commandDetails, err error) {\n\t\tif msg.Deadline != nil {\n\t\t\tif write {\n\t\t\t\tres = NOMessage\n\t\t\t\terr = errTimeoutOnCmd(msg.Command())\n\t\t\t\treturn\n\t\t\t}\n\t\t\tdefer func() {\n\t\t\t\tif msg.Deadline.Hit() {\n\t\t\t\t\tv := recover()\n\t\t\t\t\tif v != nil {\n\t\t\t\t\t\tif s, ok := v.(string); !ok || s != \"deadline\" {\n\t\t\t\t\t\t\tpanic(v)\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t\tres = NOMessage\n\t\t\t\t\terr = errTimeout\n\t\t\t\t}\n\t\t\t}()\n\t\t}\n\t\treturn s.commandInScript(msg)\n\t}()\n\tif err != nil {\n\t\treturn resp.NullValue(), err\n\t}\n\n\tif write {\n\t\tif err := s.writeAOF(msg.Args, &d); err != nil {\n\t\t\treturn resp.NullValue(), err\n\t\t}\n\t}\n\n\treturn res, nil\n}\n\nfunc (s *Server) luaTile38AtomicRO(msg *Message) (resp.Value, error) {\n\tswitch msg.Command() {\n\tdefault:\n\t\treturn resp.NullValue(), errCmdNotSupported\n\n\tcase \"set\", \"del\", \"drop\", \"fset\", \"flushdb\", \"expire\", \"persist\", \"jset\", \"pdel\",\n\t\t\"rename\", \"renamenx\":\n\t\t// write operations\n\t\treturn resp.NullValue(), errReadOnly\n\n\tcase \"get\", \"keys\", \"scan\", \"nearby\", \"within\", \"intersects\", \"hooks\", \"search\",\n\t\t\"ttl\", \"bounds\", \"server\", \"info\", \"type\", \"jget\", \"fget\", \"exists\", \"fexists\", \"test\":\n\t\t// read operations\n\t\tif s.config.followHost() != \"\" && !s.caughtUpOnce() {\n\t\t\treturn resp.NullValue(), errCatchingUp\n\t\t}\n\t}\n\n\tres, _, err := func() (res resp.Value, d commandDetails, err error) {\n\t\tif msg.Deadline != nil {\n\t\t\tdefer func() {\n\t\t\t\tif msg.Deadline.Hit() {\n\t\t\t\t\tv := recover()\n\t\t\t\t\tif v != nil {\n\t\t\t\t\t\tif s, ok := v.(string); !ok || s != \"deadline\" {\n\t\t\t\t\t\t\tpanic(v)\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t\tres = NOMessage\n\t\t\t\t\terr = errTimeout\n\t\t\t\t}\n\t\t\t}()\n\t\t}\n\t\treturn s.commandInScript(msg)\n\t}()\n\tif err != nil {\n\t\treturn resp.NullValue(), err\n\t}\n\n\treturn res, nil\n}\n\nfunc (s *Server) luaTile38NonAtomic(msg *Message) (resp.Value, error) {\n\tvar write bool\n\n\t// choose the locking strategy\n\tswitch msg.Command() {\n\tdefault:\n\t\treturn resp.NullValue(), errCmdNotSupported\n\tcase \"set\", \"del\", \"drop\", \"fset\", \"flushdb\", \"expire\", \"persist\", \"jset\", \"pdel\",\n\t\t\"rename\", \"renamenx\":\n\t\t// write operations\n\t\twrite = true\n\t\ts.mu.Lock()\n\t\tdefer s.mu.Unlock()\n\t\tif s.config.followHost() != \"\" {\n\t\t\treturn resp.NullValue(), errNotLeader\n\t\t}\n\t\tif s.config.readOnly() {\n\t\t\treturn resp.NullValue(), errReadOnly\n\t\t}\n\tcase \"get\", \"keys\", \"scan\", \"nearby\", \"within\", \"intersects\", \"hooks\", \"search\",\n\t\t\"ttl\", \"bounds\", \"server\", \"info\", \"type\", \"jget\", \"fget\", \"exists\", \"fexists\", \"test\":\n\t\t// read operations\n\t\ts.mu.RLock()\n\t\tdefer s.mu.RUnlock()\n\t\tif s.config.followHost() != \"\" && !s.caughtUpOnce() {\n\t\t\treturn resp.NullValue(), errCatchingUp\n\t\t}\n\t}\n\n\tres, d, err := func() (res resp.Value, d commandDetails, err error) {\n\t\tif msg.Deadline != nil {\n\t\t\tif write {\n\t\t\t\tres = NOMessage\n\t\t\t\terr = errTimeoutOnCmd(msg.Command())\n\t\t\t\treturn\n\t\t\t}\n\t\t\tdefer func() {\n\t\t\t\tif msg.Deadline.Hit() {\n\t\t\t\t\tv := recover()\n\t\t\t\t\tif v != nil {\n\t\t\t\t\t\tif s, ok := v.(string); !ok || s != \"deadline\" {\n\t\t\t\t\t\t\tpanic(v)\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t\tres = NOMessage\n\t\t\t\t\terr = errTimeout\n\t\t\t\t}\n\t\t\t}()\n\t\t}\n\t\treturn s.commandInScript(msg)\n\t}()\n\tif err != nil {\n\t\treturn resp.NullValue(), err\n\t}\n\n\tif write {\n\t\tif err := s.writeAOF(msg.Args, &d); err != nil {\n\t\t\treturn resp.NullValue(), err\n\t\t}\n\t}\n\n\treturn res, nil\n}\n\n// Opens a subset of the Lua 5.1 base module (tonumber, tostring)\nfunc openBaseSubset(L *lua.LState) int {\n\tbasefns := map[string]lua.LGFunction{\n\t\t\"tonumber\": baseToNumber,\n\t\t\"tostring\": baseToString,\n\t}\n\n\tglobal := L.Get(lua.GlobalsIndex).(*lua.LTable)\n\tL.SetGlobal(\"_G\", global)\n\tL.SetGlobal(\"_VERSION\", lua.LString(lua.LuaVersion))\n\tL.SetGlobal(\"_GOPHER_LUA_VERSION\", lua.LString(lua.PackageName+\" \"+lua.PackageVersion))\n\tbasemod := L.RegisterModule(\"_G\", basefns)\n\tL.Push(basemod)\n\treturn 1\n\n}\n\n// Opens a subset of the Lua 5.1 os module (clock, difftime)\nfunc openOsSubset(L *lua.LState) int {\n\tosfns := map[string]lua.LGFunction{\n\t\t\"clock\": osClock,\n\t\t\"difftime\": osDiffTime,\n\t}\n\tosmod := L.RegisterModule(lua.OsLibName, osfns)\n\tL.Push(osmod)\n\treturn 1\n}\n\n// Lua tonumber()\nfunc baseToNumber(L *lua.LState) int {\n\tbase := L.OptInt(2, 10)\n\tnoBase := L.Get(2) == lua.LNil\n\n\tswitch lv := L.CheckAny(1).(type) {\n\tcase lua.LNumber:\n\t\tL.Push(lv)\n\tcase lua.LString:\n\t\tstr := strings.Trim(string(lv), \" \\n\\t\")\n\t\tif strings.Contains(str, \".\") {\n\t\t\tif v, err := strconv.ParseFloat(str, lua.LNumberBit); err != nil {\n\t\t\t\tL.Push(lua.LNil)\n\t\t\t} else {\n\t\t\t\tL.Push(lua.LNumber(v))\n\t\t\t}\n\t\t} else {\n\t\t\tif noBase && strings.HasPrefix(strings.ToLower(str), \"0x\") {\n\t\t\t\tbase, str = 16, str[2:] // Hex number\n\t\t\t}\n\t\t\tif v, err := strconv.ParseInt(str, base, lua.LNumberBit); err != nil {\n\t\t\t\tL.Push(lua.LNil)\n\t\t\t} else {\n\t\t\t\tL.Push(lua.LNumber(v))\n\t\t\t}\n\t\t}\n\tdefault:\n\t\tL.Push(lua.LNil)\n\t}\n\treturn 1\n}\n\n// Lua tostring()\nfunc baseToString(L *lua.LState) int {\n\tv1 := L.CheckAny(1)\n\tL.Push(L.ToStringMeta(v1))\n\treturn 1\n}\n\n// Lua os.clock()\nfunc osClock(L *lua.LState) int {\n\tL.Push(lua.LNumber(time.Since(startedAt).Seconds()))\n\treturn 1\n}\n\n// Lua os.difftime()\nfunc osDiffTime(L *lua.LState) int {\n\tL.Push(lua.LNumber(L.CheckInt64(1) - L.CheckInt64(2)))\n\treturn 1\n}\n" }, { "path": "internal/server/search.go", "content": "package server\n\nimport (\n\t\"bytes\"\n\t\"errors\"\n\t\"fmt\"\n\t\"strconv\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/iwpnd/sectr\"\n\t\"github.com/mmcloughlin/geohash\"\n\t\"github.com/tidwall/geojson\"\n\t\"github.com/tidwall/geojson/geometry\"\n\t\"github.com/tidwall/resp\"\n\t\"github.com/tidwall/tile38/internal/bing\"\n\t\"github.com/tidwall/tile38/internal/buffer\"\n\t\"github.com/tidwall/tile38/internal/clip\"\n\t\"github.com/tidwall/tile38/internal/glob\"\n\t\"github.com/tidwall/tile38/internal/object\"\n)\n\nconst defaultCircleSteps = 64\n\ntype liveFenceSwitches struct {\n\tsearchScanBaseTokens\n\tobj geojson.Object\n\tcmd string\n\troam roamSwitches\n}\n\ntype roamSwitches struct {\n\ton bool\n\tkey string\n\tid string\n\tpattern bool\n\tmeters float64\n\tscan string\n}\n\ntype roamMatch struct {\n\tid string\n\tobj geojson.Object\n\tmeters float64\n}\n\nfunc (lfs liveFenceSwitches) Error() string {\n\treturn goingLive\n}\n\nfunc (lfs liveFenceSwitches) Close() {\n\tfor _, whereeval := range lfs.whereevals {\n\t\twhereeval.Close()\n\t}\n}\n\nfunc (lfs liveFenceSwitches) usingLua() bool {\n\treturn len(lfs.whereevals) > 0\n}\n\nfunc parseRectArea(ltyp string, vs []string) (nvs []string,\n\tgrect geojson.Object, tileX, tileY, tileZ int, err error,\n) {\n\tvar rect geometry.Rect\n\tvar ok bool\n\tswitch ltyp {\n\tdefault:\n\t\terr = errNotRectangle\n\t\treturn\n\tcase \"bounds\":\n\t\tvar sminLat, sminLon, smaxlat, smaxlon string\n\t\tif vs, sminLat, ok = tokenval(vs); !ok || sminLat == \"\" {\n\t\t\terr = errInvalidNumberOfArguments\n\t\t\treturn\n\t\t}\n\t\tif vs, sminLon, ok = tokenval(vs); !ok || sminLon == \"\" {\n\t\t\terr = errInvalidNumberOfArguments\n\t\t\treturn\n\t\t}\n\t\tif vs, smaxlat, ok = tokenval(vs); !ok || smaxlat == \"\" {\n\t\t\terr = errInvalidNumberOfArguments\n\t\t\treturn\n\t\t}\n\t\tif vs, smaxlon, ok = tokenval(vs); !ok || smaxlon == \"\" {\n\t\t\terr = errInvalidNumberOfArguments\n\t\t\treturn\n\t\t}\n\t\tvar minLat, minLon, maxLat, maxLon float64\n\t\tif minLat, err = strconv.ParseFloat(sminLat, 64); err != nil {\n\t\t\terr = errInvalidArgument(sminLat)\n\t\t\treturn\n\t\t}\n\t\tif minLon, err = strconv.ParseFloat(sminLon, 64); err != nil {\n\t\t\terr = errInvalidArgument(sminLon)\n\t\t\treturn\n\t\t}\n\t\tif maxLat, err = strconv.ParseFloat(smaxlat, 64); err != nil {\n\t\t\terr = errInvalidArgument(smaxlat)\n\t\t\treturn\n\t\t}\n\t\tif maxLon, err = strconv.ParseFloat(smaxlon, 64); err != nil {\n\t\t\terr = errInvalidArgument(smaxlon)\n\t\t\treturn\n\t\t}\n\t\trect = geometry.Rect{\n\t\t\tMin: geometry.Point{X: minLon, Y: minLat},\n\t\t\tMax: geometry.Point{X: maxLon, Y: maxLat},\n\t\t}\n\tcase \"hash\":\n\t\tvar hash string\n\t\tif vs, hash, ok = tokenval(vs); !ok || hash == \"\" {\n\t\t\terr = errInvalidNumberOfArguments\n\t\t\treturn\n\t\t}\n\t\tbox := geohash.BoundingBox(hash)\n\t\trect = geometry.Rect{\n\t\t\tMin: geometry.Point{X: box.MinLng, Y: box.MinLat},\n\t\t\tMax: geometry.Point{X: box.MaxLng, Y: box.MaxLat},\n\t\t}\n\tcase \"quadkey\":\n\t\tvar key string\n\t\tif vs, key, ok = tokenval(vs); !ok || key == \"\" {\n\t\t\terr = errInvalidNumberOfArguments\n\t\t\treturn\n\t\t}\n\t\tvar minLat, minLon, maxLat, maxLon float64\n\t\tminLat, minLon, maxLat, maxLon, err = bing.QuadKeyToBounds(key)\n\t\tif err != nil {\n\t\t\terr = errInvalidArgument(key)\n\t\t\treturn\n\t\t}\n\t\trect = geometry.Rect{\n\t\t\tMin: geometry.Point{X: minLon, Y: minLat},\n\t\t\tMax: geometry.Point{X: maxLon, Y: maxLat},\n\t\t}\n\tcase \"tile\", \"mvt\":\n\t\tvar sx, sy, sz string\n\t\tif vs, sx, ok = tokenval(vs); !ok || sx == \"\" {\n\t\t\terr = errInvalidNumberOfArguments\n\t\t\treturn\n\t\t}\n\t\tif vs, sy, ok = tokenval(vs); !ok || sy == \"\" {\n\t\t\terr = errInvalidNumberOfArguments\n\t\t\treturn\n\t\t}\n\t\tif vs, sz, ok = tokenval(vs); !ok || sz == \"\" {\n\t\t\terr = errInvalidNumberOfArguments\n\t\t\treturn\n\t\t}\n\t\tvar x, y, z int\n\t\tif x, err = strconv.Atoi(sx); err != nil || x < 0 {\n\t\t\terr = errInvalidArgument(sx)\n\t\t\treturn\n\t\t}\n\t\tif y, err = strconv.Atoi(sy); err != nil || y < 0 {\n\t\t\terr = errInvalidArgument(sy)\n\t\t\treturn\n\t\t}\n\t\tif z, err = strconv.Atoi(sz); err != nil || z < 0 || z > 23 {\n\t\t\terr = errInvalidArgument(sz)\n\t\t\treturn\n\t\t}\n\t\tvar minLat, minLon, maxLat, maxLon float64\n\t\tminLat, minLon, maxLat, maxLon =\n\t\t\tbing.TileXYToBounds(int64(x), int64(y), uint64(z))\n\t\trect = geometry.Rect{\n\t\t\tMin: geometry.Point{X: minLon, Y: minLat},\n\t\t\tMax: geometry.Point{X: maxLon, Y: maxLat},\n\t\t}\n\t\ttileX = x\n\t\ttileY = y\n\t\ttileZ = z\n\t}\n\tnvs = vs\n\tif ltyp == \"mvt\" {\n\t\t// Expand rectangle by 0.0625%\n\t\trect.Min.Y -= (rect.Max.Y - rect.Min.Y) * 0.0625\n\t\trect.Max.Y += (rect.Max.Y - rect.Min.Y) * 0.0625\n\t\trect.Min.X -= (rect.Max.X - rect.Min.X) * 0.0625\n\t\trect.Max.X += (rect.Max.X - rect.Min.X) * 0.0625\n\t\tif rect.Min.Y < bing.MinLatitude {\n\t\t\trect.Min.Y = bing.MinLatitude\n\t\t}\n\t\tif rect.Max.Y > bing.MaxLatitude {\n\t\t\trect.Max.Y = bing.MaxLatitude\n\t\t}\n\t\tif rect.Min.X < bing.MinLongitude {\n\t\t\trect.Min.X = bing.MinLongitude\n\t\t}\n\t\tif rect.Max.X > bing.MaxLongitude {\n\t\t\trect.Max.X = bing.MaxLongitude\n\t\t}\n\t}\n\tgrect = geojson.NewRect(rect)\n\treturn\n}\n\nfunc (s *Server) cmdSearchArgs(\n\tfromFenceCmd bool, cmd string, vs []string, types map[string]bool,\n) (lfs liveFenceSwitches, err error) {\n\tvar t searchScanBaseTokens\n\tif fromFenceCmd {\n\t\tt.fence = true\n\t}\n\tvs, t, err = s.parseSearchScanBaseTokens(cmd, t, vs)\n\tif err != nil {\n\t\treturn\n\t}\n\tlfs.searchScanBaseTokens = t\n\tvar typ string\n\tvar ok bool\n\tif vs, typ, ok = tokenval(vs); !ok || typ == \"\" {\n\t\terr = errInvalidNumberOfArguments\n\t\treturn\n\t}\n\tif lfs.searchScanBaseTokens.output == outputBounds {\n\t\tif cmd == \"within\" || cmd == \"intersects\" {\n\t\t\tif _, err := strconv.ParseFloat(typ, 64); err == nil {\n\t\t\t\t// It's likely that the output was not specified, but rather the search bounds.\n\t\t\t\tlfs.searchScanBaseTokens.output = defaultSearchOutput\n\t\t\t\tvs = append([]string{typ}, vs...)\n\t\t\t\ttyp = \"BOUNDS\"\n\t\t\t}\n\t\t}\n\t}\n\tltyp := strings.ToLower(typ)\n\tfound := types[ltyp]\n\tif !found && lfs.searchScanBaseTokens.fence && ltyp == \"roam\" && cmd == \"nearby\" {\n\t\t// allow roaming for nearby fence searches.\n\t\tfound = true\n\t}\n\tif !found {\n\t\terr = errInvalidArgument(typ)\n\t\treturn\n\t}\n\tswitch ltyp {\n\tcase \"point\":\n\t\tvar slat, slon, smeters string\n\t\tif vs, slat, ok = tokenval(vs); !ok || slat == \"\" {\n\t\t\terr = errInvalidNumberOfArguments\n\t\t\treturn\n\t\t}\n\t\tif vs, slon, ok = tokenval(vs); !ok || slon == \"\" {\n\t\t\terr = errInvalidNumberOfArguments\n\t\t\treturn\n\t\t}\n\t\tvar lat, lon, meters float64\n\t\tif lat, err = strconv.ParseFloat(slat, 64); err != nil {\n\t\t\terr = errInvalidArgument(slat)\n\t\t\treturn\n\t\t}\n\t\tif lon, err = strconv.ParseFloat(slon, 64); err != nil {\n\t\t\terr = errInvalidArgument(slon)\n\t\t\treturn\n\t\t}\n\t\t// radius is optional for nearby, but mandatory for others\n\t\tif cmd == \"nearby\" {\n\t\t\tif vs, smeters, ok = tokenval(vs); ok && smeters != \"\" {\n\t\t\t\tmeters, err = strconv.ParseFloat(smeters, 64)\n\t\t\t\tif err != nil || meters < 0 {\n\t\t\t\t\terr = errInvalidArgument(smeters)\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\tmeters = -1\n\t\t\t}\n\t\t\t// Nearby used the Circle type\n\t\t\tlfs.obj = geojson.NewCircle(geometry.Point{X: lon, Y: lat}, meters, defaultCircleSteps)\n\t\t} else {\n\t\t\t// Intersects and Within use the Point type\n\t\t\tlfs.obj = geojson.NewPoint(geometry.Point{X: lon, Y: lat})\n\t\t}\n\tcase \"circle\":\n\t\tif lfs.clip {\n\t\t\terr = errInvalidArgument(\"cannot clip with \" + ltyp)\n\t\t\treturn\n\t\t}\n\t\tvar slat, slon, smeters string\n\t\tif vs, slat, ok = tokenval(vs); !ok || slat == \"\" {\n\t\t\terr = errInvalidNumberOfArguments\n\t\t\treturn\n\t\t}\n\t\tif vs, slon, ok = tokenval(vs); !ok || slon == \"\" {\n\t\t\terr = errInvalidNumberOfArguments\n\t\t\treturn\n\t\t}\n\t\tvar lat, lon, meters float64\n\t\tif lat, err = strconv.ParseFloat(slat, 64); err != nil {\n\t\t\terr = errInvalidArgument(slat)\n\t\t\treturn\n\t\t}\n\t\tif lon, err = strconv.ParseFloat(slon, 64); err != nil {\n\t\t\terr = errInvalidArgument(slon)\n\t\t\treturn\n\t\t}\n\t\tif vs, smeters, ok = tokenval(vs); !ok || smeters == \"\" {\n\t\t\terr = errInvalidNumberOfArguments\n\t\t\treturn\n\t\t}\n\t\tmeters, err = strconv.ParseFloat(smeters, 64)\n\t\tif err != nil || meters < 0 {\n\t\t\terr = errInvalidArgument(smeters)\n\t\t\treturn\n\t\t}\n\t\tlfs.obj = geojson.NewCircle(geometry.Point{X: lon, Y: lat}, meters, defaultCircleSteps)\n\tcase \"object\":\n\t\tif lfs.clip {\n\t\t\terr = errInvalidArgument(\"cannot clip with object\")\n\t\t\treturn\n\t\t}\n\t\tvar obj string\n\t\tif vs, obj, ok = tokenval(vs); !ok || obj == \"\" {\n\t\t\terr = errInvalidNumberOfArguments\n\t\t\treturn\n\t\t}\n\t\tlfs.obj, err = geojson.Parse(obj, &s.geomParseOpts)\n\t\tif err != nil {\n\t\t\treturn\n\t\t}\n\tcase \"sector\":\n\t\tif lfs.clip {\n\t\t\terr = errInvalidArgument(\"cannot clip with \" + ltyp)\n\t\t\treturn\n\t\t}\n\t\tvar slat, slon, smeters, sb1, sb2 string\n\t\tif vs, slat, ok = tokenval(vs); !ok || slat == \"\" {\n\t\t\terr = errInvalidNumberOfArguments\n\t\t\treturn\n\t\t}\n\t\tif vs, slon, ok = tokenval(vs); !ok || slon == \"\" {\n\t\t\terr = errInvalidNumberOfArguments\n\t\t\treturn\n\t\t}\n\t\tif vs, smeters, ok = tokenval(vs); !ok || smeters == \"\" {\n\t\t\terr = errInvalidNumberOfArguments\n\t\t\treturn\n\t\t}\n\t\tif vs, sb1, ok = tokenval(vs); !ok || sb1 == \"\" {\n\t\t\terr = errInvalidNumberOfArguments\n\t\t\treturn\n\t\t}\n\t\tif vs, sb2, ok = tokenval(vs); !ok || sb2 == \"\" {\n\t\t\terr = errInvalidNumberOfArguments\n\t\t\treturn\n\t\t}\n\t\tvar lat, lon, meters, b1, b2 float64\n\t\tif lat, err = strconv.ParseFloat(slat, 64); err != nil {\n\t\t\terr = errInvalidArgument(slat)\n\t\t\treturn\n\t\t}\n\t\tif lon, err = strconv.ParseFloat(slon, 64); err != nil {\n\t\t\terr = errInvalidArgument(slon)\n\t\t\treturn\n\t\t}\n\t\tif meters, err = strconv.ParseFloat(smeters, 64); err != nil {\n\t\t\terr = errInvalidArgument(smeters)\n\t\t\treturn\n\t\t}\n\t\tif b1, err = strconv.ParseFloat(sb1, 64); err != nil {\n\t\t\terr = errInvalidArgument(sb1)\n\t\t\treturn\n\t\t}\n\t\tif b2, err = strconv.ParseFloat(sb2, 64); err != nil {\n\t\t\terr = errInvalidArgument(sb2)\n\t\t\treturn\n\t\t}\n\n\t\tif b1 == b2 {\n\t\t\terr = fmt.Errorf(\"equal bearings (%s == %s), use CIRCLE instead\", sb1, sb2)\n\t\t\treturn\n\t\t}\n\n\t\torigin := sectr.Point{Lng: lon, Lat: lat}\n\t\tsector := sectr.NewSector(origin, meters, b1, b2)\n\n\t\tlfs.obj, err = geojson.Parse(string(sector.JSON()), &s.geomParseOpts)\n\t\tif err != nil {\n\t\t\treturn\n\t\t}\n\tcase \"bounds\", \"hash\", \"tile\", \"mvt\", \"quadkey\":\n\t\tvs, lfs.obj, lfs.tileX, lfs.tileY, lfs.tileZ, err =\n\t\t\tparseRectArea(ltyp, vs)\n\t\tif err != nil {\n\t\t\treturn\n\t\t}\n\t\tif ltyp == \"mvt\" {\n\t\t\tlfs.mvt = true\n\t\t\tlfs.clip = true\n\t\t}\n\tcase \"get\":\n\t\tif lfs.clip {\n\t\t\terr = errInvalidArgument(\"cannot clip with get\")\n\t\t}\n\t\tvar key, id string\n\t\tif vs, key, ok = tokenval(vs); !ok || key == \"\" {\n\t\t\terr = errInvalidNumberOfArguments\n\t\t\treturn\n\t\t}\n\t\tif vs, id, ok = tokenval(vs); !ok || id == \"\" {\n\t\t\terr = errInvalidNumberOfArguments\n\t\t\treturn\n\t\t}\n\t\tcol, _ := s.cols.Get(key)\n\t\tif col == nil {\n\t\t\terr = errKeyNotFound\n\t\t\treturn\n\t\t}\n\t\to := col.Get(id)\n\t\tif o == nil {\n\t\t\terr = errIDNotFound\n\t\t\treturn\n\t\t}\n\t\tlfs.obj = o.Geo()\n\tcase \"roam\":\n\t\tlfs.roam.on = true\n\t\tif vs, lfs.roam.key, ok = tokenval(vs); !ok || lfs.roam.key == \"\" {\n\t\t\terr = errInvalidNumberOfArguments\n\t\t\treturn\n\t\t}\n\t\tif vs, lfs.roam.id, ok = tokenval(vs); !ok || lfs.roam.id == \"\" {\n\t\t\terr = errInvalidNumberOfArguments\n\t\t\treturn\n\t\t}\n\t\tlfs.roam.pattern = glob.IsGlob(lfs.roam.id)\n\t\tvar smeters string\n\t\tif vs, smeters, ok = tokenval(vs); !ok || smeters == \"\" {\n\t\t\terr = errInvalidNumberOfArguments\n\t\t\treturn\n\t\t}\n\t\tif lfs.roam.meters, err = strconv.ParseFloat(smeters, 64); err != nil {\n\t\t\terr = errInvalidArgument(smeters)\n\t\t\treturn\n\t\t}\n\t\tvar scan string\n\t\tif vs, scan, ok = tokenval(vs); ok {\n\t\t\tif strings.ToLower(scan) != \"scan\" {\n\t\t\t\terr = errInvalidArgument(scan)\n\t\t\t\treturn\n\t\t\t}\n\t\t\tif vs, scan, ok = tokenval(vs); !ok || scan == \"\" {\n\t\t\t\terr = errInvalidNumberOfArguments\n\t\t\t\treturn\n\t\t\t}\n\t\t\tlfs.roam.scan = scan\n\t\t}\n\t}\n\n\tvar clipRect geojson.Object\n\tvar tok, ltok string\n\tfor len(vs) > 0 {\n\t\tif vs, tok, ok = tokenval(vs); !ok || tok == \"\" {\n\t\t\terr = errInvalidNumberOfArguments\n\t\t\treturn\n\t\t}\n\t\tif strings.ToLower(tok) != \"clipby\" {\n\t\t\terr = errInvalidNumberOfArguments\n\t\t\treturn\n\t\t}\n\t\tif vs, tok, ok = tokenval(vs); !ok || tok == \"\" {\n\t\t\terr = errInvalidNumberOfArguments\n\t\t\treturn\n\t\t}\n\t\tltok = strings.ToLower(tok)\n\t\tswitch ltok {\n\t\tcase \"bounds\", \"hash\", \"tile\", \"quadkey\":\n\t\t\tvs, clipRect, lfs.tileX, lfs.tileY, lfs.tileZ, err =\n\t\t\t\tparseRectArea(ltok, vs)\n\t\t\tif err == errNotRectangle {\n\t\t\t\terr = errInvalidArgument(\"cannot clipby \" + ltok)\n\t\t\t\treturn\n\t\t\t}\n\t\t\tif err != nil {\n\t\t\t\treturn\n\t\t\t}\n\t\t\tlfs.obj = clip.Clip(lfs.obj, clipRect, &s.geomIndexOpts)\n\t\tdefault:\n\t\t\terr = errInvalidArgument(\"cannot clipby \" + ltok)\n\t\t\treturn\n\t\t}\n\t}\n\n\tif lfs.hasbuffer {\n\t\tlfs.obj, err = buffer.Simple(lfs.obj, lfs.buffer)\n\t\tif err != nil {\n\t\t\treturn\n\t\t}\n\n\t}\n\treturn\n}\n\nvar nearbyTypes = map[string]bool{\n\t\"point\": true,\n}\nvar withinOrIntersectsTypes = map[string]bool{\n\t\"geo\": true, \"bounds\": true, \"hash\": true, \"tile\": true, \"quadkey\": true,\n\t\"get\": true, \"object\": true, \"circle\": true, \"point\": true, \"sector\": true,\n\t\"mvt\": true,\n}\n\nfunc (s *Server) cmdNearby(msg *Message) (res resp.Value, err error) {\n\tstart := time.Now()\n\tvs := msg.Args[1:]\n\twr := &bytes.Buffer{}\n\tsargs, err := s.cmdSearchArgs(false, \"nearby\", vs, nearbyTypes)\n\tif sargs.usingLua() {\n\t\tdefer sargs.Close()\n\t\tdefer func() {\n\t\t\tif r := recover(); r != nil {\n\t\t\t\tres = NOMessage\n\t\t\t\terr = errors.New(r.(string))\n\t\t\t\treturn\n\t\t\t}\n\t\t}()\n\t}\n\tif err != nil {\n\t\treturn NOMessage, err\n\t}\n\tsargs.cmd = \"nearby\"\n\tif sargs.fence {\n\t\treturn NOMessage, sargs\n\t}\n\tsw, err := s.newScanWriter(\n\t\twr, msg, sargs.key, sargs.output, sargs.precision, sargs.globs, false,\n\t\tsargs.cursor, sargs.limit, sargs.wheres, sargs.whereins,\n\t\tsargs.whereevals, sargs.nofields,\n\t\tsargs.mvt, sargs.tileX, sargs.tileY, sargs.tileZ)\n\tif err != nil {\n\t\treturn NOMessage, err\n\t}\n\tif msg.OutputType == JSON {\n\t\twr.WriteString(`{\"ok\":true`)\n\t}\n\tvar ierr error\n\tif sw.col != nil {\n\t\titerStep := func(o *object.Object, dist float64) bool {\n\t\t\tkeepGoing, err := sw.pushObject(ScanWriterParams{\n\t\t\t\tobj: o,\n\t\t\t\tdist: dist,\n\t\t\t\tdistOutput: sargs.distance,\n\t\t\t\tignoreGlobMatch: true,\n\t\t\t\tskipTesting: true,\n\t\t\t})\n\t\t\tif err != nil {\n\t\t\t\tierr = err\n\t\t\t\treturn false\n\t\t\t}\n\t\t\treturn keepGoing\n\t\t}\n\t\tmaxDist := sargs.obj.(*geojson.Circle).Meters()\n\t\tif sargs.sparse > 0 {\n\t\t\tif maxDist < 0 {\n\t\t\t\t// error cannot use SPARSE and KNN together\n\t\t\t\treturn NOMessage,\n\t\t\t\t\terrors.New(\"cannot use SPARSE without a point distance\")\n\t\t\t}\n\t\t\t// An intersects operation is required for SPARSE\n\t\t\titer := func(o *object.Object) bool {\n\t\t\t\tvar dist float64\n\t\t\t\tif sargs.distance {\n\t\t\t\t\tdist = o.Geo().Distance(sargs.obj)\n\t\t\t\t}\n\t\t\t\treturn iterStep(o, dist)\n\t\t\t}\n\t\t\tsw.col.Intersects(sargs.obj, sargs.sparse, sw, msg.Deadline, iter)\n\t\t} else {\n\t\t\titer := func(o *object.Object, dist float64) bool {\n\t\t\t\tif maxDist > 0 && dist > maxDist {\n\t\t\t\t\treturn false\n\t\t\t\t}\n\t\t\t\tvar meters float64\n\t\t\t\tif sargs.distance {\n\t\t\t\t\tmeters = dist\n\t\t\t\t}\n\t\t\t\treturn iterStep(o, meters)\n\t\t\t}\n\t\t\tsw.col.Nearby(sargs.obj, sw, msg.Deadline, iter)\n\t\t}\n\t}\n\tif ierr != nil {\n\t\treturn retrerr(ierr)\n\t}\n\tsw.writeFoot()\n\tif msg.OutputType == JSON {\n\t\twr.WriteString(`,\"elapsed\":\"` + time.Since(start).String() + \"\\\"}\")\n\t\treturn resp.BytesValue(wr.Bytes()), nil\n\t}\n\treturn sw.respOut, nil\n}\n\nfunc (s *Server) cmdWITHIN(msg *Message) (res resp.Value, err error) {\n\treturn s.cmdWITHINorINTERSECTS(\"within\", msg)\n}\n\nfunc (s *Server) cmdINTERSECTS(msg *Message) (res resp.Value, err error) {\n\treturn s.cmdWITHINorINTERSECTS(\"intersects\", msg)\n}\n\nfunc (s *Server) cmdWITHINorINTERSECTS(cmd string, msg *Message) (res resp.Value, err error) {\n\tstart := time.Now()\n\tvs := msg.Args[1:]\n\n\twr := &bytes.Buffer{}\n\tsargs, err := s.cmdSearchArgs(false, cmd, vs, withinOrIntersectsTypes)\n\tif sargs.usingLua() {\n\t\tdefer sargs.Close()\n\t\tdefer func() {\n\t\t\tif r := recover(); r != nil {\n\t\t\t\tres = NOMessage\n\t\t\t\terr = errors.New(r.(string))\n\t\t\t\treturn\n\t\t\t}\n\t\t}()\n\t}\n\tif err != nil {\n\t\treturn NOMessage, err\n\t}\n\tsargs.cmd = cmd\n\tif sargs.fence {\n\t\treturn NOMessage, sargs\n\t}\n\tsw, err := s.newScanWriter(\n\t\twr, msg, sargs.key, sargs.output, sargs.precision, sargs.globs, false,\n\t\tsargs.cursor, sargs.limit, sargs.wheres, sargs.whereins,\n\t\tsargs.whereevals, sargs.nofields,\n\t\tsargs.mvt, sargs.tileX, sargs.tileY, sargs.tileZ)\n\tif err != nil {\n\t\treturn NOMessage, err\n\t}\n\tif msg.OutputType == JSON {\n\t\twr.WriteString(`{\"ok\":true`)\n\t}\n\tvar ierr error\n\tif sw.col != nil {\n\t\tswitch cmd {\n\t\tcase \"within\":\n\t\t\tsw.col.Within(sargs.obj, sargs.sparse, sw, msg.Deadline,\n\t\t\t\tfunc(o *object.Object) bool {\n\t\t\t\t\tkeepGoing, err := sw.pushObject(ScanWriterParams{obj: o})\n\t\t\t\t\tif err != nil {\n\t\t\t\t\t\tierr = err\n\t\t\t\t\t\treturn false\n\t\t\t\t\t}\n\t\t\t\t\treturn keepGoing\n\t\t\t\t},\n\t\t\t)\n\t\tcase \"intersects\":\n\t\t\tsw.col.Intersects(sargs.obj, sargs.sparse, sw, msg.Deadline,\n\t\t\t\tfunc(o *object.Object) bool {\n\t\t\t\t\tparams := ScanWriterParams{obj: o}\n\t\t\t\t\tif sargs.clip {\n\t\t\t\t\t\tparams.clip = sargs.obj\n\t\t\t\t\t}\n\t\t\t\t\tkeepGoing, err := sw.pushObject(params)\n\t\t\t\t\tif err != nil {\n\t\t\t\t\t\tierr = err\n\t\t\t\t\t\treturn false\n\t\t\t\t\t}\n\t\t\t\t\treturn keepGoing\n\t\t\t\t},\n\t\t\t)\n\t\t}\n\t}\n\tif ierr != nil {\n\t\treturn retrerr(ierr)\n\t}\n\tsw.writeFoot()\n\tif msg.OutputType == JSON {\n\t\twr.WriteString(`,\"elapsed\":\"` + time.Since(start).String() + \"\\\"}\")\n\t\treturn resp.BytesValue(wr.Bytes()), nil\n\t}\n\treturn sw.respOut, nil\n}\n\nfunc (s *Server) cmdSeachValuesArgs(vs []string) (\n\tlfs liveFenceSwitches, err error,\n) {\n\tvar t searchScanBaseTokens\n\tvs, t, err = s.parseSearchScanBaseTokens(\"search\", t, vs)\n\tif err != nil {\n\t\treturn\n\t}\n\tlfs.searchScanBaseTokens = t\n\tif len(vs) != 0 {\n\t\terr = errInvalidNumberOfArguments\n\t\treturn\n\t}\n\treturn\n}\n\nfunc multiGlobParse(globs []string, desc bool) [2]string {\n\tvar limits [2]string\n\tfor i, pattern := range globs {\n\t\tg := glob.Parse(pattern, desc)\n\t\tif g.Limits[0] == \"\" && g.Limits[1] == \"\" {\n\t\t\tlimits[0], limits[1] = \"\", \"\"\n\t\t\tbreak\n\t\t}\n\t\tif i == 0 {\n\t\t\tlimits[0], limits[1] = g.Limits[0], g.Limits[1]\n\t\t} else if desc {\n\t\t\tif g.Limits[0] > limits[0] {\n\t\t\t\tlimits[0] = g.Limits[0]\n\t\t\t}\n\t\t\tif g.Limits[1] < limits[1] {\n\t\t\t\tlimits[1] = g.Limits[1]\n\t\t\t}\n\t\t} else {\n\t\t\tif g.Limits[0] < limits[0] {\n\t\t\t\tlimits[0] = g.Limits[0]\n\t\t\t}\n\t\t\tif g.Limits[1] > limits[1] {\n\t\t\t\tlimits[1] = g.Limits[1]\n\t\t\t}\n\t\t}\n\t}\n\treturn limits\n}\n\nfunc (s *Server) cmdSearch(msg *Message) (res resp.Value, err error) {\n\tstart := time.Now()\n\tvs := msg.Args[1:]\n\twr := &bytes.Buffer{}\n\tsargs, err := s.cmdSeachValuesArgs(vs)\n\tif sargs.usingLua() {\n\t\tdefer sargs.Close()\n\t\tdefer func() {\n\t\t\tif r := recover(); r != nil {\n\t\t\t\tres = NOMessage\n\t\t\t\terr = errors.New(r.(string))\n\t\t\t\treturn\n\t\t\t}\n\t\t}()\n\t}\n\tif err != nil {\n\t\treturn NOMessage, err\n\t}\n\tsw, err := s.newScanWriter(\n\t\twr, msg, sargs.key, sargs.output, sargs.precision, sargs.globs, true,\n\t\tsargs.cursor, sargs.limit, sargs.wheres, sargs.whereins,\n\t\tsargs.whereevals, sargs.nofields,\n\t\tsargs.mvt, sargs.tileX, sargs.tileY, sargs.tileZ)\n\tif err != nil {\n\t\treturn NOMessage, err\n\t}\n\tif msg.OutputType == JSON {\n\t\twr.WriteString(`{\"ok\":true`)\n\t}\n\tvar ierr error\n\tif sw.col != nil {\n\t\tif sw.output == outputCount && len(sw.wheres) == 0 && sw.globEverything {\n\t\t\tcount := sw.col.Count() - int(sargs.cursor)\n\t\t\tif count < 0 {\n\t\t\t\tcount = 0\n\t\t\t}\n\t\t\tsw.count = uint64(count)\n\t\t} else {\n\t\t\tlimits := multiGlobParse(sw.globs, sargs.desc)\n\t\t\tif limits[0] == \"\" && limits[1] == \"\" {\n\t\t\t\tsw.col.SearchValues(sargs.desc, sw, msg.Deadline,\n\t\t\t\t\tfunc(o *object.Object) bool {\n\t\t\t\t\t\tkeepGoing, err := sw.pushObject(ScanWriterParams{\n\t\t\t\t\t\t\tobj: o,\n\t\t\t\t\t\t})\n\t\t\t\t\t\tif err != nil {\n\t\t\t\t\t\t\tierr = err\n\t\t\t\t\t\t\treturn false\n\t\t\t\t\t\t}\n\t\t\t\t\t\treturn keepGoing\n\t\t\t\t\t},\n\t\t\t\t)\n\t\t\t} else {\n\t\t\t\t// must disable globSingle for string value type matching because\n\t\t\t\t// globSingle is only for ID matches, not values.\n\t\t\t\tsw.col.SearchValuesRange(limits[0], limits[1], sargs.desc, sw,\n\t\t\t\t\tmsg.Deadline,\n\t\t\t\t\tfunc(o *object.Object) bool {\n\t\t\t\t\t\tkeepGoing, err := sw.pushObject(ScanWriterParams{\n\t\t\t\t\t\t\tobj: o,\n\t\t\t\t\t\t})\n\t\t\t\t\t\tif err != nil {\n\t\t\t\t\t\t\tierr = err\n\t\t\t\t\t\t\treturn false\n\t\t\t\t\t\t}\n\t\t\t\t\t\treturn keepGoing\n\t\t\t\t\t},\n\t\t\t\t)\n\t\t\t}\n\t\t}\n\t}\n\tif ierr != nil {\n\t\treturn retrerr(ierr)\n\t}\n\tsw.writeFoot()\n\tif msg.OutputType == JSON {\n\t\twr.WriteString(`,\"elapsed\":\"` + time.Since(start).String() + \"\\\"}\")\n\t\treturn resp.BytesValue(wr.Bytes()), nil\n\t}\n\treturn sw.respOut, nil\n}\n" }, { "path": "internal/server/server.go", "content": "package server\n\nimport (\n\t\"bytes\"\n\t\"crypto/rand\"\n\t\"crypto/sha1\"\n\t\"encoding/base64\"\n\t\"encoding/binary\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"net\"\n\t\"net/http\"\n\t\"net/url\"\n\t\"os\"\n\t\"path\"\n\t\"path/filepath\"\n\t\"runtime\"\n\t\"runtime/debug\"\n\t\"strconv\"\n\t\"strings\"\n\t\"sync\"\n\t\"sync/atomic\"\n\t\"time\"\n\n\t\"github.com/prometheus/client_golang/prometheus\"\n\t\"github.com/tidwall/btree\"\n\t\"github.com/tidwall/buntdb\"\n\t\"github.com/tidwall/geojson\"\n\t\"github.com/tidwall/geojson/geometry\"\n\t\"github.com/tidwall/gjson\"\n\t\"github.com/tidwall/redcon\"\n\t\"github.com/tidwall/resp\"\n\t\"github.com/tidwall/rtree\"\n\t\"github.com/tidwall/tile38/core\"\n\t\"github.com/tidwall/tile38/internal/collection\"\n\t\"github.com/tidwall/tile38/internal/deadline\"\n\t\"github.com/tidwall/tile38/internal/endpoint\"\n\t\"github.com/tidwall/tile38/internal/log\"\n\t\"github.com/tidwall/tile38/internal/object\"\n\t\"github.com/tidwall/tile38/internal/viewer\"\n)\n\nvar errOOM = errors.New(\"OOM command not allowed when used memory > 'maxmemory'\")\n\nfunc errTimeoutOnCmd(cmd string) error {\n\treturn fmt.Errorf(\"timeout not supported for '%s'\", cmd)\n}\n\nconst (\n\tgoingLive = \"going live\"\n\thookLogPrefix = \"hook:log:\"\n)\n\n// commandDetails is detailed information about a mutable command. It's used\n// for geofence formulas.\ntype commandDetails struct {\n\tcommand string // client command, like \"SET\" or \"DEL\"\n\tkey string // collection key\n\tnewKey string // new key, for RENAME command\n\n\tobj *object.Object // target object\n\told *object.Object // previous object, if any\n\n\tupdated bool // object was updated\n\ttimestamp time.Time // timestamp when the update occurred\n\tparent bool // when true, only children are forwarded\n\tpattern string // PDEL key pattern\n\tchildren []*commandDetails // for multi actions such as \"PDEL\"\n}\n\ntype rwlocker interface {\n\tLockLowPriority()\n\tLock()\n\tUnlock()\n\tRLock()\n\tRUnlock()\n}\n\n// rwspinlock is the same as a RWLock, but uses spinlocks instead of mutexes.\ntype rwspinlock struct {\n\tstate atomic.Int32\n}\n\nfunc (l *rwspinlock) Lock() {\n\tfor {\n\t\tstate := l.state.Load()\n\t\tif state == 0 && l.state.CompareAndSwap(state, -1) {\n\t\t\treturn\n\t\t}\n\t\truntime.Gosched()\n\t}\n}\nfunc (l *rwspinlock) LockLowPriority() {\n\t// All write locks are low priority and unfair. First come first serve.\n\tl.Lock()\n}\nfunc (l *rwspinlock) Unlock() {\n\tif l.state.Add(1) > 0 {\n\t\tpanic(\"Unlock of unlocked rwspinlock\")\n\t}\n}\nfunc (l *rwspinlock) RLock() {\n\tfor {\n\t\tstate := l.state.Load()\n\t\tif state >= 0 && l.state.CompareAndSwap(state, state+1) {\n\t\t\treturn\n\t\t}\n\t\truntime.Gosched()\n\t}\n}\nfunc (l *rwspinlock) RUnlock() {\n\tif l.state.Add(-1) < 0 {\n\t\tpanic(\"RUnlock of unlocked rwspinlock\")\n\t}\n}\n\n// rwmutex is the same as a RWLock but includes a LockLowPriority for\n// performing write locks that do not block the queue for readers.\ntype rwmutex struct {\n\tmu sync.RWMutex\n}\n\nfunc (l *rwmutex) Lock() {\n\tstart := time.Now()\n\tfor {\n\t\tfor range 100 {\n\t\t\tif l.mu.TryLock() {\n\t\t\t\treturn\n\t\t\t}\n\t\t\truntime.Gosched()\n\t\t}\n\t\tif time.Since(start) > time.Millisecond*50 {\n\t\t\tbreak\n\t\t}\n\t}\n\tl.mu.Lock()\n}\nfunc (l *rwmutex) LockLowPriority() {\n\tfor !l.mu.TryLock() {\n\t\truntime.Gosched()\n\t}\n}\nfunc (l *rwmutex) Unlock() {\n\tl.mu.Unlock()\n}\nfunc (l *rwmutex) RLock() {\n\tl.mu.RLock()\n}\nfunc (l *rwmutex) RUnlock() {\n\tl.mu.RUnlock()\n}\n\n// Server is a tile38 controller\ntype Server struct {\n\t// user defined options\n\topts Options\n\n\t// static values\n\tunix string\n\thost string\n\tport int\n\thttp bool\n\tdir string\n\tstarted time.Time\n\tconfig *Config\n\tepc *endpoint.Manager\n\tepool *exprPool\n\n\tlnmu sync.Mutex\n\tln net.Listener // server listener\n\n\t// env opts\n\tgeomParseOpts geojson.ParseOptions\n\tgeomIndexOpts geometry.IndexOptions\n\thttp500Errors bool\n\n\t// atomics\n\tfollowc atomic.Int64 // counter when follow property changes\n\tstatsTotalConns atomic.Int64 // counter for total connections\n\tstatsTotalCommands atomic.Int64 // counter for total commands\n\tstatsTotalMsgsSent atomic.Int64 // counter for total sent webhook messages\n\tstatsExpired atomic.Int64 // item expiration counter\n\tlastShrinkDuration atomic.Int64\n\tstopServer atomic.Bool\n\toutOfMemory atomic.Bool\n\tloadedAndReady atomic.Bool // server is loaded and ready for commands\n\n\tconnsmu sync.RWMutex\n\tconns map[int]*Client\n\n\tmu rwlocker // sync.RWMutex\n\n\t// aof\n\taof *os.File // active aof file\n\taofdirty atomic.Bool // mark the aofbuf as having data\n\taofbuf []byte // prewrite buffer\n\taofsz int // active size of the aof file\n\tshrinking bool // aof shrinking flag\n\tshrinklog [][]string // aof shrinking log\n\n\t// database\n\tqdb *buntdb.DB // hook queue log\n\tqidx uint64 // hook queue log last idx\n\n\tcols *btree.Map[string, *collection.Collection] // data collections\n\n\thooks *btree.BTree // hook name -- [string]*Hook\n\thookCross *rtree.RTree // hook spatial tree for \"cross\" geofences\n\thookTree *rtree.RTree // hook spatial tree for all\n\thooksOut *btree.BTree // hooks with \"outside\" detection -- [string]*Hook\n\tgroupHooks *btree.BTree // hooks that are connected to objects\n\tgroupObjects *btree.BTree // objects that are connected to hooks\n\thookExpires *btree.BTree // queue of all hooks marked for expiration\n\n\t// followers (external aof readers)\n\tfollows map[*bytes.Buffer]bool\n\tfcond *sync.Cond\n\tlstack []*commandDetails\n\tlives map[*liveBuffer]bool\n\tlcond *sync.Cond // live geofence signal\n\tfaofsz int // last reported aofsize\n\tfcupflags atomic.Int32 // follow caught up (caughtUp and caughtUpOnce)\n\taofconnM map[net.Conn]io.Closer\n\tpubq pubQueue\n\n\t// lua scripts\n\tluascripts *lScriptMap\n\tluapool *lStatePool\n\n\t// pubsub system (SUBSCRIBE, PUBLISH, and SETCHAN)\n\tpubsub *pubsub\n\n\t// monitor connections (using the MONITOR command)\n\tmonconnsMu sync.RWMutex\n\tmonconns map[net.Conn]bool\n}\n\n// Options for Serve()\ntype Options struct {\n\tHost string\n\tPort int\n\tDir string\n\tUseHTTP bool\n\tMetricsAddr string\n\tUnixSocketPath string // path for unix socket\n\tClientOutput string // \"\" or \"resp\" or \"json\"\n\n\t// DevMode puts application in to dev mode\n\tDevMode bool\n\n\t// ShowDebugMessages allows for log.Debug to print to console.\n\tShowDebugMessages bool\n\n\t// ProtectedMode forces Tile38 to default in protected mode.\n\tProtectedMode string\n\n\t// AppendOnly allows for disabling the appendonly file.\n\tAppendOnly bool\n\n\t// AppendFileName allows for custom appendonly file path\n\tAppendFileName string\n\n\t// QueueFileName allows for custom queue.db file path\n\tQueueFileName string\n\n\t// Shutdown allows for shutting down the server.\n\tShutdown <-chan bool\n\n\t// Spinlock uses a spinlock instead of a mutex\n\tSpinlock bool\n}\n\n// Serve starts a new tile38 server\nfunc Serve(opts Options) error {\n\tif opts.AppendFileName == \"\" {\n\t\topts.AppendFileName = path.Join(opts.Dir, \"appendonly.aof\")\n\t}\n\tif opts.QueueFileName == \"\" {\n\t\topts.QueueFileName = path.Join(opts.Dir, \"queue.db\")\n\t}\n\tif opts.ProtectedMode == \"\" {\n\t\topts.ProtectedMode = \"no\"\n\t}\n\n\tlog.Infof(\"Server started, Tile38 version %s, git %s\", core.Version, core.GitSHA)\n\tdefer func() {\n\t\tlog.Warn(\"Server has shutdown, bye now\")\n\t\tif false {\n\t\t\t// prints the stack, looking for running goroutines.\n\t\t\tbuf := make([]byte, 10000)\n\t\t\tn := runtime.Stack(buf, true)\n\t\t\tprintln(string(buf[:n]))\n\t\t}\n\t}()\n\n\tvar lock rwlocker\n\n\tif opts.Spinlock {\n\t\tlock = new(rwspinlock)\n\t} else {\n\t\tlock = new(rwmutex)\n\t}\n\n\t// Initialize the s\n\ts := &Server{\n\t\tmu: lock,\n\t\tunix: opts.UnixSocketPath,\n\t\thost: opts.Host,\n\t\tport: opts.Port,\n\t\tdir: opts.Dir,\n\t\tfollows: make(map[*bytes.Buffer]bool),\n\t\tfcond: sync.NewCond(&sync.Mutex{}),\n\t\tlives: make(map[*liveBuffer]bool),\n\t\tlcond: sync.NewCond(&sync.Mutex{}),\n\t\thooks: btree.NewNonConcurrent(byHookName),\n\t\thooksOut: btree.NewNonConcurrent(byHookName),\n\t\thookCross: &rtree.RTree{},\n\t\thookTree: &rtree.RTree{},\n\t\taofconnM: make(map[net.Conn]io.Closer),\n\t\tstarted: time.Now(),\n\t\tconns: make(map[int]*Client),\n\t\thttp: opts.UseHTTP,\n\t\tpubsub: newPubsub(),\n\t\tpubq: pubQueue{cond: sync.NewCond(&sync.Mutex{})},\n\t\tmonconns: make(map[net.Conn]bool),\n\t\tcols: &btree.Map[string, *collection.Collection]{},\n\n\t\tgroupHooks: btree.NewNonConcurrent(byGroupHook),\n\t\tgroupObjects: btree.NewNonConcurrent(byGroupObject),\n\t\thookExpires: btree.NewNonConcurrent(byHookExpires),\n\t\topts: opts,\n\t}\n\ts.epool = newExprPool(s)\n\ts.epc = endpoint.NewManager(s)\n\tdefer s.epc.Shutdown()\n\ts.luascripts = s.newScriptMap()\n\ts.luapool = s.newPool()\n\tdefer s.luapool.Shutdown()\n\n\tif err := os.MkdirAll(opts.Dir, 0700); err != nil {\n\t\treturn err\n\t}\n\tvar err error\n\ts.config, err = loadConfig(filepath.Join(opts.Dir, \"config\"))\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// Send \"500 Internal Server\" error instead of \"200 OK\" for json responses\n\t// with `\"ok\":false`. T38HTTP500ERRORS=1\n\ts.http500Errors, _ = strconv.ParseBool(os.Getenv(\"T38HTTP500ERRORS\"))\n\n\t// Allow for geometry indexing options through environment variables:\n\t// T38IDXGEOMKIND -- None, RTree, QuadTree\n\t// T38IDXGEOM -- Min number of points in a geometry for indexing.\n\t// T38IDXMULTI -- Min number of object in a Multi/Collection for indexing.\n\ts.geomParseOpts = *geojson.DefaultParseOptions\n\ts.geomIndexOpts = *geometry.DefaultIndexOptions\n\tn, err := strconv.ParseUint(os.Getenv(\"T38IDXGEOM\"), 10, 32)\n\tif err == nil {\n\t\ts.geomParseOpts.IndexGeometry = int(n)\n\t\ts.geomIndexOpts.MinPoints = int(n)\n\t}\n\tn, err = strconv.ParseUint(os.Getenv(\"T38IDXMULTI\"), 10, 32)\n\tif err == nil {\n\t\ts.geomParseOpts.IndexChildren = int(n)\n\t}\n\trequireValid := os.Getenv(\"REQUIREVALID\")\n\tif requireValid != \"\" {\n\t\ts.geomParseOpts.RequireValid = true\n\t}\n\tindexKind := os.Getenv(\"T38IDXGEOMKIND\")\n\tswitch indexKind {\n\tdefault:\n\t\tlog.Errorf(\"Unknown index kind: %s\", indexKind)\n\tcase \"\":\n\tcase \"None\":\n\t\ts.geomParseOpts.IndexGeometryKind = geometry.None\n\t\ts.geomIndexOpts.Kind = geometry.None\n\tcase \"RTree\":\n\t\ts.geomParseOpts.IndexGeometryKind = geometry.RTree\n\t\ts.geomIndexOpts.Kind = geometry.RTree\n\tcase \"QuadTree\":\n\t\ts.geomParseOpts.IndexGeometryKind = geometry.QuadTree\n\t\ts.geomIndexOpts.Kind = geometry.QuadTree\n\t}\n\tif s.geomParseOpts.IndexGeometryKind == geometry.None {\n\t\tlog.Debugf(\"Geom indexing: %s\",\n\t\t\ts.geomParseOpts.IndexGeometryKind,\n\t\t)\n\t} else {\n\t\tlog.Debugf(\"Geom indexing: %s (%d points)\",\n\t\t\ts.geomParseOpts.IndexGeometryKind,\n\t\t\ts.geomParseOpts.IndexGeometry,\n\t\t)\n\t}\n\tlog.Debugf(\"Multi indexing: RTree (%d points)\", s.geomParseOpts.IndexChildren)\n\n\tnerr := make(chan error)\n\tgo func() {\n\t\t// Start the server in the background\n\t\tnerr <- s.netServe()\n\t}()\n\n\tvar fstop atomic.Bool\n\tgo func() {\n\t\tfor !fstop.Load() {\n\t\t\ts.fcond.Broadcast()\n\t\t\ttime.Sleep(time.Second / 4)\n\t\t}\n\t}()\n\n\tgo func() {\n\t\t<-opts.Shutdown\n\t\ts.stopServer.Store(true)\n\t\tlog.Warnf(\"Shutting down...\")\n\t\tfstop.Store(true)\n\t\ts.lnmu.Lock()\n\t\tln := s.ln\n\t\ts.ln = nil\n\t\ts.lnmu.Unlock()\n\t\tif ln != nil {\n\t\t\tln.Close()\n\t\t}\n\t\tfor conn, f := range s.aofconnM {\n\t\t\tconn.Close()\n\t\t\tf.Close()\n\t\t}\n\t}()\n\n\t// Load the queue before the aof\n\tqdb, err := buntdb.Open(opts.QueueFileName)\n\tif err != nil {\n\t\treturn err\n\t}\n\tvar qidx uint64\n\tif err := qdb.View(func(tx *buntdb.Tx) error {\n\t\tval, err := tx.Get(\"hook:idx\")\n\t\tif err != nil {\n\t\t\tif err == buntdb.ErrNotFound {\n\t\t\t\treturn nil\n\t\t\t}\n\t\t\treturn err\n\t\t}\n\t\tqidx = stringToUint64(val)\n\t\treturn nil\n\t}); err != nil {\n\t\treturn err\n\t}\n\terr = qdb.CreateIndex(\"hooks\", hookLogPrefix+\"*\", buntdb.IndexJSONCaseSensitive(\"hook\"))\n\tif err != nil {\n\t\treturn err\n\t}\n\n\ts.qdb = qdb\n\ts.qidx = qidx\n\tif err := s.migrateAOF(); err != nil {\n\t\treturn err\n\t}\n\tif opts.AppendOnly {\n\t\tf, err := os.OpenFile(opts.AppendFileName, os.O_CREATE|os.O_RDWR, 0600)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\ts.aof = f\n\t\tif err := s.loadAOF(); err != nil {\n\t\t\treturn err\n\t\t}\n\t\tdefer func() {\n\t\t\ts.flushAOF(false)\n\t\t\ts.aof.Sync()\n\t\t}()\n\t}\n\n\t// Start background routines\n\tvar bgwg sync.WaitGroup\n\n\tif s.config.followHost() != \"\" {\n\t\tbgwg.Add(1)\n\t\tgo func() {\n\t\t\tdefer bgwg.Done()\n\t\t\ts.follow(s.config.followHost(), s.config.followPort(),\n\t\t\t\tint(s.followc.Load()))\n\t\t}()\n\t}\n\n\tvar mln net.Listener\n\tif opts.MetricsAddr != \"\" {\n\t\tlog.Infof(\"Listening for metrics at: %s\", opts.MetricsAddr)\n\t\tmln, err = net.Listen(\"tcp\", opts.MetricsAddr)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tbgwg.Add(1)\n\t\tgo func() {\n\t\t\tdefer bgwg.Done()\n\t\t\tsmux := http.NewServeMux()\n\t\t\tsmux.HandleFunc(\"/\", s.MetricsIndexHandler)\n\t\t\tsmux.HandleFunc(\"/metrics\", s.MetricsHandler)\n\t\t\terr := http.Serve(mln, smux)\n\t\t\tif err != nil {\n\t\t\t\tif !s.stopServer.Load() {\n\t\t\t\t\tlog.Fatalf(\"metrics server: %s\", err)\n\t\t\t\t}\n\t\t\t}\n\t\t}()\n\t}\n\n\tbgwg.Add(1)\n\tgo s.processLives(&bgwg)\n\tbgwg.Add(1)\n\tgo s.watchOutOfMemory(&bgwg)\n\tbgwg.Add(1)\n\tgo s.watchLuaStatePool(&bgwg)\n\tbgwg.Add(1)\n\tgo s.watchAutoGC(&bgwg)\n\tbgwg.Add(1)\n\tgo s.backgroundExpiring(&bgwg)\n\tbgwg.Add(1)\n\tgo s.backgroundSyncAOF(&bgwg)\n\tbgwg.Add(1)\n\tgo s.startPublishQueue(&bgwg)\n\tdefer func() {\n\t\tlog.Debug(\"Stopping background routines\")\n\t\t// Stop background routines\n\t\ts.stopPublishQueue()\n\t\ts.followc.Add(1) // this will force any follow communication to die\n\t\ts.stopServer.Store(true)\n\t\tif mln != nil {\n\t\t\tmln.Close() // Stop the metrics server\n\t\t}\n\t\tbgwg.Wait()\n\t}()\n\n\t// Server is now loaded and ready. Wait for network error messages.\n\ts.loadedAndReady.Store(true)\n\treturn <-nerr\n}\n\nfunc (s *Server) isProtected() bool {\n\tif s.opts.ProtectedMode == \"no\" {\n\t\t// --protected-mode no\n\t\treturn false\n\t}\n\tif s.host != \"\" && s.host != \"127.0.0.1\" &&\n\t\ts.host != \"::1\" && s.host != \"localhost\" {\n\t\t// -h address\n\t\treturn false\n\t}\n\tis := s.config.protectedMode() != \"no\" && s.config.requirePass() == \"\"\n\treturn is\n}\n\nfunc (s *Server) netServe() error {\n\tvar ln net.Listener\n\tvar err error\n\tif s.unix != \"\" {\n\t\tos.RemoveAll(s.unix)\n\t\tln, err = net.Listen(\"unix\", s.unix)\n\t} else {\n\t\ttcpAddr := fmt.Sprintf(\"%s:%d\", s.host, s.port)\n\t\tln, err = net.Listen(\"tcp\", tcpAddr)\n\t}\n\tif err != nil {\n\t\treturn err\n\t}\n\ts.lnmu.Lock()\n\ts.ln = ln\n\ts.lnmu.Unlock()\n\n\tvar wg sync.WaitGroup\n\tdefer func() {\n\t\tlog.Debug(\"Closing client connections...\")\n\t\ts.connsmu.RLock()\n\t\tfor _, c := range s.conns {\n\t\t\tif c.closer != nil {\n\t\t\t\tc.closer.Close()\n\t\t\t}\n\t\t}\n\t\ts.connsmu.RUnlock()\n\t\twg.Wait()\n\t\tln.Close()\n\t\tlog.Debug(\"Client connection closed\")\n\t}()\n\n\tvar defaultOutputType Type\n\tswitch s.opts.ClientOutput {\n\tcase \"resp\":\n\t\tdefaultOutputType = RESP\n\tcase \"json\":\n\t\tdefaultOutputType = JSON\n\t}\n\n\tlog.Infof(\"Ready to accept connections at %s\", ln.Addr())\n\tvar clientID int64\n\tfor {\n\t\tconn, err := ln.Accept()\n\t\tif err != nil {\n\t\t\tif s.stopServer.Load() {\n\t\t\t\treturn nil\n\t\t\t}\n\t\t\tlog.Warn(err)\n\t\t\ttime.Sleep(time.Second / 5)\n\t\t\tcontinue\n\t\t}\n\t\twg.Add(1)\n\t\tgo func(conn net.Conn) {\n\t\t\tdetached := false\n\t\t\tdefer func() {\n\t\t\t\tif !detached {\n\t\t\t\t\twg.Done()\n\t\t\t\t}\n\t\t\t}()\n\n\t\t\t// open connection\n\t\t\t// create the client\n\t\t\tclient := new(Client)\n\t\t\tclient.id = int(atomic.AddInt64(&clientID, 1))\n\t\t\tclient.opened = time.Now()\n\t\t\tclient.remoteAddr = conn.RemoteAddr().String()\n\t\t\tclient.closer = conn\n\n\t\t\t// add client to server map\n\t\t\ts.connsmu.Lock()\n\t\t\ts.conns[client.id] = client\n\t\t\ts.connsmu.Unlock()\n\t\t\ts.statsTotalConns.Add(1)\n\n\t\t\t// set the client keep-alive, if needed\n\t\t\tif s.config.keepAlive() > 0 {\n\t\t\t\tif conn, ok := conn.(*net.TCPConn); ok {\n\t\t\t\t\tconn.SetKeepAlive(true)\n\t\t\t\t\tconn.SetKeepAlivePeriod(\n\t\t\t\t\t\ttime.Duration(s.config.keepAlive()) * time.Second,\n\t\t\t\t\t)\n\t\t\t\t}\n\t\t\t}\n\t\t\tlog.Debugf(\"Opened connection: %s\", client.remoteAddr)\n\n\t\t\tdefer func() {\n\t\t\t\t// close connection\n\t\t\t\t// delete from server map\n\t\t\t\ts.connsmu.Lock()\n\t\t\t\tdelete(s.conns, client.id)\n\t\t\t\ts.connsmu.Unlock()\n\t\t\t\tlog.Debugf(\"Closed connection: %s\", client.remoteAddr)\n\t\t\t\tconn.Close()\n\t\t\t}()\n\n\t\t\tvar lastConnType Type\n\t\t\tvar lastOutputType Type\n\n\t\t\t// check if the connection is protected\n\t\t\tif !strings.HasPrefix(client.remoteAddr, \"127.0.0.1:\") &&\n\t\t\t\t!strings.HasPrefix(client.remoteAddr, \"[::1]:\") {\n\t\t\t\tif s.isProtected() {\n\t\t\t\t\t// This is a protected server. Only loopback is allowed.\n\t\t\t\t\tconn.Write(deniedMessage)\n\t\t\t\t\treturn // close connection\n\t\t\t\t}\n\t\t\t}\n\t\t\tpacket := make([]byte, 0xFFFF)\n\t\t\tfor {\n\t\t\t\tvar close bool\n\t\t\t\tn, err := conn.Read(packet)\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t\tin := packet[:n]\n\n\t\t\t\t// read the payload packet from the client input stream.\n\t\t\t\tpacket := client.in.Begin(in)\n\n\t\t\t\t// load the pipeline reader\n\t\t\t\tpr := &client.pr\n\t\t\t\trdbuf := bytes.NewBuffer(packet)\n\t\t\t\tpr.rd = rdbuf\n\t\t\t\tpr.wr = client\n\t\t\t\tmsgs, err := pr.ReadMessages()\n\t\t\t\tfor _, msg := range msgs {\n\t\t\t\t\t// Just closing connection if we have deprecated HTTP or WS connection,\n\t\t\t\t\t// And --http-transport = false\n\t\t\t\t\tif !s.http && (msg.ConnType == WebSocket ||\n\t\t\t\t\t\tmsg.ConnType == HTTP) {\n\t\t\t\t\t\tclose = true // close connection\n\t\t\t\t\t\tbreak\n\t\t\t\t\t}\n\t\t\t\t\tif msg != nil && msg.Command() != \"\" {\n\t\t\t\t\t\tif client.outputType != Null {\n\t\t\t\t\t\t\tmsg.OutputType = client.outputType\n\t\t\t\t\t\t} else if defaultOutputType != Null {\n\t\t\t\t\t\t\tmsg.OutputType = defaultOutputType\n\t\t\t\t\t\t}\n\t\t\t\t\t\tmsg.StrictRESP = client.strictRESP\n\t\t\t\t\t\tif msg.Command() == \"quit\" {\n\t\t\t\t\t\t\tif msg.OutputType == RESP {\n\t\t\t\t\t\t\t\tio.WriteString(client, \"+OK\\r\\n\")\n\t\t\t\t\t\t\t}\n\t\t\t\t\t\t\tclose = true // close connection\n\t\t\t\t\t\t\tbreak\n\t\t\t\t\t\t}\n\n\t\t\t\t\t\t// increment last used\n\t\t\t\t\t\tclient.mu.Lock()\n\t\t\t\t\t\tclient.last = time.Now()\n\t\t\t\t\t\tclient.mu.Unlock()\n\n\t\t\t\t\t\t// update total command count\n\t\t\t\t\t\ts.statsTotalCommands.Add(1)\n\n\t\t\t\t\t\t// handle the command\n\t\t\t\t\t\terr := s.handleInputCommand(client, msg)\n\t\t\t\t\t\tif err != nil {\n\t\t\t\t\t\t\tif err.Error() == goingLive {\n\t\t\t\t\t\t\t\tclient.goLiveErr = err\n\t\t\t\t\t\t\t\tclient.goLiveMsg = msg\n\t\t\t\t\t\t\t\t// detach\n\t\t\t\t\t\t\t\tvar rwc io.ReadWriteCloser = conn\n\t\t\t\t\t\t\t\tclient.conn = rwc\n\t\t\t\t\t\t\t\tif len(client.out) > 0 {\n\t\t\t\t\t\t\t\t\tclient.conn.Write(client.out)\n\t\t\t\t\t\t\t\t\tclient.out = nil\n\t\t\t\t\t\t\t\t}\n\t\t\t\t\t\t\t\tclient.in = InputStream{}\n\t\t\t\t\t\t\t\tclient.pr.rd = rwc\n\t\t\t\t\t\t\t\tclient.pr.wr = rwc\n\t\t\t\t\t\t\t\tclient.closer = nil\n\t\t\t\t\t\t\t\twg.Done()\n\t\t\t\t\t\t\t\tdetached = true\n\t\t\t\t\t\t\t\tlog.Debugf(\"Detached connection: %s\", client.remoteAddr)\n\n\t\t\t\t\t\t\t\tvar wg2 sync.WaitGroup\n\t\t\t\t\t\t\t\twg2.Add(1)\n\t\t\t\t\t\t\t\tgo func() {\n\t\t\t\t\t\t\t\t\tdefer wg2.Done()\n\t\t\t\t\t\t\t\t\terr := s.goLive(\n\t\t\t\t\t\t\t\t\t\tclient.goLiveErr,\n\t\t\t\t\t\t\t\t\t\t&liveConn{conn.RemoteAddr(), rwc},\n\t\t\t\t\t\t\t\t\t\t&client.pr,\n\t\t\t\t\t\t\t\t\t\tclient.goLiveMsg,\n\t\t\t\t\t\t\t\t\t\tclient.goLiveMsg.ConnType == WebSocket,\n\t\t\t\t\t\t\t\t\t)\n\t\t\t\t\t\t\t\t\tif err != nil {\n\t\t\t\t\t\t\t\t\t\tlog.Error(err)\n\t\t\t\t\t\t\t\t\t}\n\t\t\t\t\t\t\t\t}()\n\t\t\t\t\t\t\t\twg2.Wait()\n\t\t\t\t\t\t\t\treturn // close connection\n\t\t\t\t\t\t\t}\n\t\t\t\t\t\t\tlog.Error(err)\n\t\t\t\t\t\t\treturn // close connection, NOW\n\t\t\t\t\t\t}\n\n\t\t\t\t\t\tclient.outputType = msg.OutputType\n\t\t\t\t\t\tclient.strictRESP = msg.StrictRESP\n\t\t\t\t\t} else {\n\t\t\t\t\t\tclient.Write([]byte(\"HTTP/1.1 500 Bad Request\\r\\nConnection: close\\r\\n\\r\\n\"))\n\t\t\t\t\t\tbreak\n\t\t\t\t\t}\n\t\t\t\t\tif msg.ConnType == HTTP || msg.ConnType == WebSocket {\n\t\t\t\t\t\tclose = true // close connection\n\t\t\t\t\t\tbreak\n\t\t\t\t\t}\n\t\t\t\t\tlastOutputType = msg.OutputType\n\t\t\t\t\tlastConnType = msg.ConnType\n\t\t\t\t}\n\n\t\t\t\tpacket = packet[len(packet)-rdbuf.Len():]\n\t\t\t\tclient.in.End(packet)\n\n\t\t\t\t// write to client\n\t\t\t\tif len(client.out) > 0 {\n\t\t\t\t\tif s.aofdirty.Load() {\n\t\t\t\t\t\tfunc() {\n\t\t\t\t\t\t\t// prewrite\n\t\t\t\t\t\t\ts.mu.Lock()\n\t\t\t\t\t\t\tdefer s.mu.Unlock()\n\t\t\t\t\t\t\ts.flushAOF(false)\n\t\t\t\t\t\t}()\n\t\t\t\t\t\ts.aofdirty.Store(false)\n\t\t\t\t\t}\n\t\t\t\t\tconn.Write(client.out)\n\t\t\t\t\tclient.out = nil\n\t\t\t\t}\n\t\t\t\tif close {\n\t\t\t\t\tbreak\n\t\t\t\t}\n\t\t\t\tif err != nil {\n\t\t\t\t\tlog.Error(err)\n\t\t\t\t\tif lastConnType == RESP {\n\t\t\t\t\t\tvar value resp.Value\n\t\t\t\t\t\tswitch lastOutputType {\n\t\t\t\t\t\tcase JSON:\n\t\t\t\t\t\t\tvalue = resp.StringValue(`{\"ok\":false,\"err\":` +\n\t\t\t\t\t\t\t\tjsonString(err.Error()) + \"}\")\n\t\t\t\t\t\tcase RESP:\n\t\t\t\t\t\t\tvalue = resp.ErrorValue(err)\n\t\t\t\t\t\t}\n\t\t\t\t\t\tbytes, _ := value.MarshalRESP()\n\t\t\t\t\t\tconn.Write(bytes)\n\t\t\t\t\t}\n\t\t\t\t\tbreak // close connection\n\t\t\t\t}\n\t\t\t}\n\t\t}(conn)\n\t}\n}\n\ntype liveConn struct {\n\tremoteAddr net.Addr\n\trwc io.ReadWriteCloser\n}\n\nfunc (conn *liveConn) Close() error {\n\treturn conn.rwc.Close()\n}\n\nfunc (conn *liveConn) LocalAddr() net.Addr {\n\tpanic(\"not supported\")\n}\n\nfunc (conn *liveConn) RemoteAddr() net.Addr {\n\treturn conn.remoteAddr\n}\nfunc (conn *liveConn) Read(b []byte) (n int, err error) {\n\treturn conn.rwc.Read(b)\n}\n\nfunc (conn *liveConn) Write(b []byte) (n int, err error) {\n\treturn conn.rwc.Write(b)\n}\n\nfunc (conn *liveConn) SetDeadline(deadline time.Time) error {\n\tpanic(\"not supported\")\n}\n\nfunc (conn *liveConn) SetReadDeadline(deadline time.Time) error {\n\tpanic(\"not supported\")\n}\n\nfunc (conn *liveConn) SetWriteDeadline(deadline time.Time) error {\n\tpanic(\"not supported\")\n}\n\nfunc (s *Server) watchAutoGC(wg *sync.WaitGroup) {\n\tdefer wg.Done()\n\tstart := time.Now()\n\ts.loopUntilServerStops(time.Second, func() {\n\t\tautoGC := s.config.autoGC()\n\t\tif autoGC == 0 {\n\t\t\treturn\n\t\t}\n\t\tif time.Since(start) < time.Second*time.Duration(autoGC) {\n\t\t\treturn\n\t\t}\n\t\tvar mem1, mem2 runtime.MemStats\n\t\truntime.ReadMemStats(&mem1)\n\t\tlog.Debugf(\"autogc(before): \"+\n\t\t\t\"alloc: %v, heap_alloc: %v, heap_released: %v\",\n\t\t\tmem1.Alloc, mem1.HeapAlloc, mem1.HeapReleased)\n\n\t\truntime.GC()\n\t\tdebug.FreeOSMemory()\n\t\truntime.ReadMemStats(&mem2)\n\t\tlog.Debugf(\"autogc(after): \"+\n\t\t\t\"alloc: %v, heap_alloc: %v, heap_released: %v\",\n\t\t\tmem2.Alloc, mem2.HeapAlloc, mem2.HeapReleased)\n\t\tstart = time.Now()\n\t})\n}\n\nfunc (s *Server) checkOutOfMemory() {\n\tif s.stopServer.Load() {\n\t\treturn\n\t}\n\toom := s.outOfMemory.Load()\n\tvar mem runtime.MemStats\n\tif s.config.maxMemory() == 0 {\n\t\tif oom {\n\t\t\ts.outOfMemory.Store(false)\n\t\t}\n\t\treturn\n\t}\n\tif oom {\n\t\truntime.GC()\n\t}\n\truntime.ReadMemStats(&mem)\n\ts.outOfMemory.Store(int(mem.HeapAlloc) > s.config.maxMemory())\n}\n\nfunc (s *Server) loopUntilServerStops(dur time.Duration, op func()) {\n\tvar last time.Time\n\tfor {\n\t\tif s.stopServer.Load() {\n\t\t\treturn\n\t\t}\n\t\tnow := time.Now()\n\t\tif now.Sub(last) > dur {\n\t\t\top()\n\t\t\tlast = now\n\t\t}\n\t\ttime.Sleep(time.Second / 5)\n\t}\n}\n\nfunc (s *Server) watchOutOfMemory(wg *sync.WaitGroup) {\n\tdefer wg.Done()\n\ts.loopUntilServerStops(time.Second*4, func() {\n\t\ts.checkOutOfMemory()\n\t})\n}\n\nfunc (s *Server) watchLuaStatePool(wg *sync.WaitGroup) {\n\tdefer wg.Done()\n\ts.loopUntilServerStops(time.Second*10, func() {\n\t\ts.luapool.Prune()\n\t})\n}\n\n// backgroundSyncAOF ensures that the aof buffer is does not grow too big.\nfunc (s *Server) backgroundSyncAOF(wg *sync.WaitGroup) {\n\tdefer wg.Done()\n\ts.loopUntilServerStops(time.Second, func() {\n\t\ts.mu.LockLowPriority()\n\t\tdefer s.mu.Unlock()\n\t\ts.flushAOF(true)\n\t})\n}\n\nfunc isReservedFieldName(field string) bool {\n\tswitch field {\n\tcase \"z\", \"lat\", \"lon\":\n\t\treturn true\n\t}\n\treturn false\n}\n\nfunc rewriteTimeoutMsg(msg *Message) (err error) {\n\tvs := msg.Args[1:]\n\tvar valStr string\n\tvar ok bool\n\tif vs, valStr, ok = tokenval(vs); !ok || valStr == \"\" || len(vs) == 0 {\n\t\terr = errInvalidNumberOfArguments\n\t\treturn\n\t}\n\ttimeoutSec, _err := strconv.ParseFloat(valStr, 64)\n\tif _err != nil || timeoutSec < 0 {\n\t\terr = errInvalidArgument(valStr)\n\t\treturn\n\t}\n\tmsg.Args = vs[:]\n\tmsg._command = \"\"\n\tmsg.Deadline = deadline.New(\n\t\ttime.Now().Add(time.Duration(timeoutSec * float64(time.Second))))\n\treturn\n}\n\nfunc (s *Server) handleInputCommand(client *Client, msg *Message) error {\n\tstart := time.Now()\n\tvar mvt bool\n\tif msg.ConnType == HTTP && len(msg.Args) == 1 {\n\t\tvar query string\n\t\tif i := strings.IndexByte(msg.Args[0], '?'); i != -1 {\n\t\t\tquery = msg.Args[0][i+1:]\n\t\t\tmsg.Args[0] = msg.Args[0][:i]\n\t\t}\n\t\tif strings.HasSuffix(msg.Args[0], \".mvt\") ||\n\t\t\tstrings.HasSuffix(msg.Args[0], \".pbf\") {\n\t\t\tmvt = mvtFilterHTTPArgs(msg, query)\n\t\t} else if strings.HasPrefix(msg.Args[0], \"viewer/\") ||\n\t\t\tmsg.Args[0] == \"viewer\" {\n\t\t\treturn viewer.HandleHTTP(client, \"/\"+strings.Join(msg.Args, \"/\"),\n\t\t\t\ts.opts.DevMode)\n\t\t}\n\t}\n\tserializeOutput := func(res resp.Value) (string, error) {\n\t\tvar resStr string\n\t\tvar err error\n\t\tswitch msg.OutputType {\n\t\tcase JSON:\n\t\t\tresStr = res.String()\n\t\tcase RESP:\n\t\t\tvar resBytes []byte\n\t\t\tresBytes, err = res.MarshalRESP()\n\t\t\tresStr = string(resBytes)\n\t\t}\n\t\treturn resStr, err\n\t}\n\twriteOutput := func(res string) error {\n\t\tswitch msg.ConnType {\n\t\tdefault:\n\t\t\terr := fmt.Errorf(\"unsupported conn type: %v\", msg.ConnType)\n\t\t\tlog.Error(err)\n\t\t\treturn err\n\t\tcase WebSocket:\n\t\t\treturn WriteWebSocketMessage(client, []byte(res))\n\t\tcase HTTP:\n\t\t\textraNL := 2\n\t\t\tcontentType := \"application/json; charset=utf-8\"\n\t\t\tstatus := \"200 OK\"\n\t\t\tif (s.http500Errors || msg._command == \"healthz\") &&\n\t\t\t\t!gjson.Get(res, \"ok\").Bool() {\n\t\t\t\tstatus = \"500 Internal Server Error\"\n\t\t\t} else if mvt {\n\t\t\t\tv := gjson.Get(res, \"mvt\")\n\t\t\t\tif !v.Exists() {\n\t\t\t\t\tstatus = \"500 Internal Server Error\"\n\t\t\t\t} else {\n\t\t\t\t\tres = v.String()\n\t\t\t\t\tout, err := base64.RawStdEncoding.DecodeString(res)\n\t\t\t\t\tif err != nil {\n\t\t\t\t\t\tstatus = \"500 Internal Server Error\"\n\t\t\t\t\t} else {\n\t\t\t\t\t\tres = string(out)\n\t\t\t\t\t\tcontentType = \"application/vnd.mapbox-vector-tile\"\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t\t_, err := fmt.Fprintf(client, \"\"+\n\t\t\t\t\"HTTP/1.1 %s\\r\\n\"+\n\t\t\t\t\"Connection: close\\r\\n\"+\n\t\t\t\t\"Content-Type: %s\\r\\n\"+\n\t\t\t\t\"Content-Length: %d\\r\\n\"+\n\t\t\t\t\"Access-Control-Allow-Origin: *\\r\\n\"+\n\t\t\t\t\"\\r\\n\", status, contentType, len(res)+extraNL)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\t_, err = io.WriteString(client, res)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tif extraNL == 2 {\n\t\t\t\t_, err = io.WriteString(client, \"\\r\\n\")\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\t\t\t}\n\t\t\treturn nil\n\t\tcase RESP:\n\t\t\tvar err error\n\t\t\tif msg.OutputType == JSON {\n\t\t\t\t_, err = fmt.Fprintf(client, \"$%d\\r\\n%s\\r\\n\", len(res), res)\n\t\t\t} else {\n\t\t\t\t_, err = io.WriteString(client, res)\n\t\t\t}\n\t\t\treturn err\n\t\tcase Native:\n\t\t\t_, err := fmt.Fprintf(client, \"$%d %s\\r\\n\", len(res), res)\n\t\t\treturn err\n\t\t}\n\t}\n\n\tcmd := msg.Command()\n\tdefer func() {\n\t\ttook := time.Since(start).Seconds()\n\t\tcmdDurations.With(prometheus.Labels{\"cmd\": cmd}).Observe(took)\n\t}()\n\n\t// Ping. Just send back the response. No need to put through the pipeline.\n\tif cmd == \"ping\" || cmd == \"echo\" {\n\t\tswitch msg.OutputType {\n\t\tcase JSON:\n\t\t\tif len(msg.Args) > 1 {\n\t\t\t\treturn writeOutput(`{\"ok\":true,\"` + cmd + `\":` + jsonString(msg.Args[1]) + `,\"elapsed\":\"` + time.Since(start).String() + `\"}`)\n\t\t\t}\n\t\t\treturn writeOutput(`{\"ok\":true,\"` + cmd + `\":\"pong\",\"elapsed\":\"` + time.Since(start).String() + `\"}`)\n\t\tcase RESP:\n\t\t\tif len(msg.Args) > 1 {\n\t\t\t\tdata := redcon.AppendBulkString(nil, msg.Args[1])\n\t\t\t\treturn writeOutput(string(data))\n\t\t\t}\n\t\t\treturn writeOutput(\"+PONG\\r\\n\")\n\t\t}\n\t\ts.sendMonitor(nil, msg, client, false)\n\t\treturn nil\n\t}\n\n\twriteErr := func(errMsg string) error {\n\t\tswitch msg.OutputType {\n\t\tcase JSON:\n\t\t\treturn writeOutput(`{\"ok\":false,\"err\":` + jsonString(errMsg) + `,\"elapsed\":\"` + time.Since(start).String() + \"\\\"}\")\n\t\tcase RESP:\n\t\t\tif errMsg == errInvalidNumberOfArguments.Error() {\n\t\t\t\treturn writeOutput(\"-ERR wrong number of arguments for '\" + cmd + \"' command\\r\\n\")\n\t\t\t}\n\t\t\tvar ucprefix bool\n\t\t\tword := strings.Split(errMsg, \" \")[0]\n\t\t\tif len(word) > 0 {\n\t\t\t\tucprefix = true\n\t\t\t\tfor i := 0; i < len(word); i++ {\n\t\t\t\t\tif word[i] < 'A' || word[i] > 'Z' {\n\t\t\t\t\t\tucprefix = false\n\t\t\t\t\t\tbreak\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t\tif !ucprefix {\n\t\t\t\terrMsg = \"ERR \" + errMsg\n\t\t\t}\n\t\t\tv, _ := resp.ErrorValue(errors.New(errMsg)).MarshalRESP()\n\t\t\treturn writeOutput(string(v))\n\t\t}\n\t\treturn nil\n\t}\n\n\tif !s.loadedAndReady.Load() {\n\t\tswitch msg.Command() {\n\t\tcase \"output\", \"ping\", \"echo\", \"auth\":\n\t\tdefault:\n\t\t\treturn writeErr(\"LOADING Tile38 is loading the dataset in memory\")\n\t\t}\n\t}\n\n\tif cmd == \"hello\" {\n\t\t// Not Supporting RESP3+, returns an ERR instead.\n\t\tot := msg.OutputType\n\t\tif len(msg.Args) > 1 && (msg.Args[1] >= \"0\" &&\n\t\t\tmsg.Args[1] <= \"9\") && s.opts.ClientOutput == \"json\" &&\n\t\t\tot == JSON && msg.ConnType == RESP {\n\t\t\t// Here we are making sure that we ignoring the '-o json' flag, if\n\t\t\t// used, otherwise the connection will fail for some redis clients\n\t\t\t// like \"github.com/redis/go-redis/v9\" are overly strict and expect\n\t\t\t// a map type or an error as the result of the HELLO command.\n\t\t\tmsg.OutputType = RESP\n\t\t\tmsg.StrictRESP = true\n\t\t}\n\t\terr := writeErr(\"unknown command '\" + msg.Args[0] + \"'\")\n\t\tmsg.OutputType = ot\n\t\treturn err\n\t}\n\n\tif cmd == \"command\" && len(msg.Args) > 1 && msg.Args[1] == \"DOCS\" &&\n\t\ts.opts.ClientOutput == \"json\" {\n\t\t// The standard Redis client typically sends a \"COMMAND DOCS\" to start\n\t\t// client connection.\n\t\tmsg.StrictRESP = true\n\t}\n\n\tif cmd == \"timeout\" {\n\t\tif err := rewriteTimeoutMsg(msg); err != nil {\n\t\t\treturn writeErr(err.Error())\n\t\t}\n\t}\n\n\tvar write bool\n\n\tif (!client.authd || cmd == \"auth\") && cmd != \"output\" && cmd != \"healthz\" {\n\t\tif s.config.requirePass() != \"\" {\n\t\t\tpassword := \"\"\n\t\t\t// This better be an AUTH command or the Message should contain an Auth\n\t\t\tif cmd != \"auth\" && msg.Auth == \"\" {\n\t\t\t\t// Just shut down the pipeline now. The less the client connection knows the better.\n\t\t\t\treturn writeErr(\"authentication required\")\n\t\t\t}\n\t\t\tif msg.Auth != \"\" {\n\t\t\t\tpassword = msg.Auth\n\t\t\t} else {\n\t\t\t\tif len(msg.Args) > 1 {\n\t\t\t\t\tpassword = msg.Args[1]\n\t\t\t\t}\n\t\t\t}\n\t\t\tif s.config.requirePass() != strings.TrimSpace(password) {\n\t\t\t\treturn writeErr(\"invalid password\")\n\t\t\t}\n\t\t\tclient.authd = true\n\t\t\tif msg.ConnType != HTTP {\n\t\t\t\tresStr, _ := serializeOutput(OKMessage(msg, start))\n\t\t\t\treturn writeOutput(resStr)\n\t\t\t}\n\t\t} else if msg.Command() == \"auth\" {\n\t\t\treturn writeErr(\"invalid password\")\n\t\t}\n\t}\n\n\t// choose the locking strategy\n\tswitch msg.Command() {\n\tdefault:\n\t\ts.mu.RLock()\n\t\tdefer s.mu.RUnlock()\n\tcase \"set\", \"del\", \"drop\", \"fset\", \"flushdb\",\n\t\t\"setchan\", \"pdelchan\", \"delchan\",\n\t\t\"sethook\", \"pdelhook\", \"delhook\",\n\t\t\"expire\", \"persist\", \"jset\", \"pdel\", \"rename\", \"renamenx\":\n\t\t// write operations\n\t\twrite = true\n\t\ts.mu.Lock()\n\t\tdefer s.mu.Unlock()\n\t\tif s.config.followHost() != \"\" {\n\t\t\treturn writeErr(\"not the leader\")\n\t\t}\n\t\tif s.config.readOnly() {\n\t\t\treturn writeErr(\"read only\")\n\t\t}\n\tcase \"eval\", \"evalsha\":\n\t\t// write operations (potentially) but no AOF for the script command itself\n\t\ts.mu.Lock()\n\t\tdefer s.mu.Unlock()\n\t\tif s.config.followHost() != \"\" {\n\t\t\treturn writeErr(\"not the leader\")\n\t\t}\n\t\tif s.config.readOnly() {\n\t\t\treturn writeErr(\"read only\")\n\t\t}\n\tcase \"get\", \"keys\", \"scan\", \"nearby\", \"within\", \"intersects\", \"hooks\",\n\t\t\"chans\", \"search\", \"ttl\", \"bounds\", \"server\", \"info\", \"type\", \"jget\",\n\t\t\"evalro\", \"evalrosha\", \"role\", \"fget\", \"exists\", \"fexists\":\n\t\t// read operations\n\t\ts.mu.RLock()\n\t\tdefer s.mu.RUnlock()\n\t\t// fallthrough to perform a \"catching up to leader\" check\n\t\tfallthrough\n\tcase \"healthz\":\n\t\t// healthz operation does not require a read lock. It only needs\n\t\t// a caughtup once check to leader, which is atomic.\n\t\tif s.config.followHost() != \"\" && !s.caughtUpOnce() {\n\t\t\treturn writeErr(\"catching up to leader\")\n\t\t}\n\tcase \"follow\", \"slaveof\", \"replconf\", \"readonly\", \"config\":\n\t\t// system operations\n\t\t// does not write to aof, but requires a write lock.\n\t\ts.mu.Lock()\n\t\tdefer s.mu.Unlock()\n\tcase \"output\":\n\t\t// this is local connection operation. Locks not needed.\n\tcase \"echo\":\n\tcase \"massinsert\":\n\t\t// dev operation\n\tcase \"sleep\":\n\t\t// dev operation\n\t\ts.mu.RLock()\n\t\tdefer s.mu.RUnlock()\n\tcase \"shutdown\":\n\t\t// dev operation\n\t\ts.mu.Lock()\n\t\tdefer s.mu.Unlock()\n\tcase \"aofshrink\":\n\t\ts.mu.RLock()\n\t\tdefer s.mu.RUnlock()\n\tcase \"client\":\n\t\ts.mu.Lock()\n\t\tdefer s.mu.Unlock()\n\tcase \"evalna\", \"evalnasha\":\n\t\t// No locking for scripts, otherwise writes cannot happen within scripts\n\tcase \"subscribe\", \"psubscribe\", \"publish\":\n\t\t// No locking for pubsub\n\tcase \"monitor\":\n\t\t// No locking for monitor\n\t}\n\tres, d, err := func() (res resp.Value, d commandDetails, err error) {\n\t\tif msg.Deadline != nil {\n\t\t\tif write {\n\t\t\t\tres = NOMessage\n\t\t\t\terr = errTimeoutOnCmd(msg.Command())\n\t\t\t\treturn\n\t\t\t}\n\t\t\tdefer func() {\n\t\t\t\tif msg.Deadline.Hit() {\n\t\t\t\t\tv := recover()\n\t\t\t\t\tif v != nil {\n\t\t\t\t\t\tif s, ok := v.(string); !ok || s != \"deadline\" {\n\t\t\t\t\t\t\tpanic(v)\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t\tres = NOMessage\n\t\t\t\t\terr = errTimeout\n\t\t\t\t}\n\t\t\t}()\n\t\t}\n\t\tres, d, err = s.command(msg, client)\n\t\tif msg.Deadline != nil {\n\t\t\tmsg.Deadline.Check()\n\t\t}\n\t\treturn res, d, err\n\t}()\n\tif res.Type() == resp.Error {\n\t\treturn writeErr(res.String())\n\t}\n\tif err != nil {\n\t\tif err.Error() == goingLive {\n\t\t\treturn err\n\t\t}\n\t\treturn writeErr(err.Error())\n\t}\n\tif write {\n\t\tif err := s.writeAOF(msg.Args, &d); err != nil {\n\t\t\tif _, ok := err.(errAOFHook); ok {\n\t\t\t\treturn writeErr(err.Error())\n\t\t\t}\n\t\t\tlog.Fatal(err)\n\t\t\treturn err\n\t\t}\n\t}\n\tvar resStr string\n\tresStr, err = serializeOutput(res)\n\tif err != nil {\n\t\treturn err\n\t}\n\tif err := writeOutput(resStr); err != nil {\n\t\treturn err\n\t}\n\treturn nil\n}\n\nfunc randomKey(n int) string {\n\tb := make([]byte, n)\n\tnn, err := rand.Read(b)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\tif nn != n {\n\t\tpanic(\"random failed\")\n\t}\n\treturn fmt.Sprintf(\"%x\", b)\n}\n\nfunc (s *Server) reset() {\n\ts.aofsz = 0\n\ts.cols.Clear()\n}\n\nfunc (s *Server) command(msg *Message, client *Client) (\n\tres resp.Value, d commandDetails, err error,\n) {\n\tswitch msg.Command() {\n\tdefault:\n\t\terr = fmt.Errorf(\"unknown command '%s'\", msg.Args[0])\n\tcase \"set\":\n\t\tres, d, err = s.cmdSET(msg)\n\tcase \"fset\":\n\t\tres, d, err = s.cmdFSET(msg)\n\tcase \"del\":\n\t\tres, d, err = s.cmdDEL(msg)\n\tcase \"pdel\":\n\t\tres, d, err = s.cmdPDEL(msg)\n\tcase \"drop\":\n\t\tres, d, err = s.cmdDROP(msg)\n\tcase \"flushdb\":\n\t\tres, d, err = s.cmdFLUSHDB(msg)\n\tcase \"rename\":\n\t\tres, d, err = s.cmdRENAME(msg)\n\tcase \"renamenx\":\n\t\tres, d, err = s.cmdRENAME(msg)\n\tcase \"sethook\":\n\t\tres, d, err = s.cmdSetHook(msg)\n\tcase \"delhook\":\n\t\tres, d, err = s.cmdDelHook(msg)\n\tcase \"pdelhook\":\n\t\tres, d, err = s.cmdPDelHook(msg)\n\tcase \"hooks\":\n\t\tres, err = s.cmdHooks(msg)\n\tcase \"setchan\":\n\t\tres, d, err = s.cmdSetHook(msg)\n\tcase \"delchan\":\n\t\tres, d, err = s.cmdDelHook(msg)\n\tcase \"pdelchan\":\n\t\tres, d, err = s.cmdPDelHook(msg)\n\tcase \"chans\":\n\t\tres, err = s.cmdHooks(msg)\n\tcase \"expire\":\n\t\tres, d, err = s.cmdEXPIRE(msg)\n\tcase \"persist\":\n\t\tres, d, err = s.cmdPERSIST(msg)\n\tcase \"ttl\":\n\t\tres, err = s.cmdTTL(msg)\n\tcase \"shutdown\":\n\t\tif !s.opts.DevMode {\n\t\t\terr = fmt.Errorf(\"unknown command '%s'\", msg.Args[0])\n\t\t\treturn\n\t\t}\n\t\tlog.Fatal(\"shutdown requested by developer\")\n\tcase \"massinsert\":\n\t\tif !s.opts.DevMode {\n\t\t\terr = fmt.Errorf(\"unknown command '%s'\", msg.Args[0])\n\t\t\treturn\n\t\t}\n\t\tres, err = s.cmdMassInsert(msg)\n\tcase \"sleep\":\n\t\tif !s.opts.DevMode {\n\t\t\terr = fmt.Errorf(\"unknown command '%s'\", msg.Args[0])\n\t\t\treturn\n\t\t}\n\t\tres, err = s.cmdSleep(msg)\n\tcase \"follow\", \"slaveof\":\n\t\tres, err = s.cmdFollow(msg)\n\tcase \"replconf\":\n\t\tres, err = s.cmdReplConf(msg, client)\n\tcase \"readonly\":\n\t\tres, err = s.cmdREADONLY(msg)\n\tcase \"stats\":\n\t\tres, err = s.cmdSTATS(msg)\n\tcase \"server\":\n\t\tres, err = s.cmdSERVER(msg)\n\tcase \"healthz\":\n\t\tres, err = s.cmdHEALTHZ(msg)\n\tcase \"info\":\n\t\tres, err = s.cmdINFO(msg)\n\tcase \"role\":\n\t\tres, err = s.cmdROLE(msg)\n\tcase \"scan\":\n\t\tres, err = s.cmdScan(msg)\n\tcase \"nearby\":\n\t\tres, err = s.cmdNearby(msg)\n\tcase \"within\":\n\t\tres, err = s.cmdWITHIN(msg)\n\tcase \"intersects\":\n\t\tres, err = s.cmdINTERSECTS(msg)\n\tcase \"search\":\n\t\tres, err = s.cmdSearch(msg)\n\tcase \"bounds\":\n\t\tres, err = s.cmdBOUNDS(msg)\n\tcase \"get\":\n\t\tres, err = s.cmdGET(msg)\n\tcase \"fget\":\n\t\tres, err = s.cmdFGET(msg)\n\tcase \"jget\":\n\t\tres, err = s.cmdJget(msg)\n\tcase \"jset\":\n\t\tres, d, err = s.cmdJset(msg)\n\tcase \"jdel\":\n\t\tres, d, err = s.cmdJdel(msg)\n\tcase \"type\":\n\t\tres, err = s.cmdTYPE(msg)\n\tcase \"keys\":\n\t\tres, err = s.cmdKEYS(msg)\n\tcase \"exists\":\n\t\tres, err = s.cmdEXISTS(msg)\n\tcase \"fexists\":\n\t\tres, err = s.cmdFEXISTS(msg)\n\tcase \"output\":\n\t\tres, err = s.cmdOUTPUT(msg)\n\tcase \"aof\":\n\t\tres, err = s.cmdAOF(msg)\n\tcase \"aofmd5\":\n\t\tres, err = s.cmdAOFMD5(msg)\n\tcase \"gc\":\n\t\truntime.GC()\n\t\tdebug.FreeOSMemory()\n\t\tres = OKMessage(msg, time.Now())\n\tcase \"aofshrink\":\n\t\tgo s.aofshrink()\n\t\tres = OKMessage(msg, time.Now())\n\tcase \"config get\":\n\t\tres, err = s.cmdConfigGet(msg)\n\tcase \"config set\":\n\t\tres, err = s.cmdConfigSet(msg)\n\tcase \"config rewrite\":\n\t\tres, err = s.cmdConfigRewrite(msg)\n\tcase \"config\", \"script\":\n\t\t// These get rewritten into \"config foo\" and \"script bar\"\n\t\terr = fmt.Errorf(\"unknown command '%s'\", msg.Args[0])\n\t\tif len(msg.Args) > 1 {\n\t\t\tmsg.Args[1] = msg.Args[0] + \" \" + msg.Args[1]\n\t\t\tmsg.Args = msg.Args[1:]\n\t\t\tmsg._command = \"\"\n\t\t\treturn s.command(msg, client)\n\t\t}\n\tcase \"client\":\n\t\tres, err = s.cmdCLIENT(msg, client)\n\tcase \"eval\", \"evalro\", \"evalna\":\n\t\tres, err = s.cmdEvalUnified(false, msg)\n\tcase \"evalsha\", \"evalrosha\", \"evalnasha\":\n\t\tres, err = s.cmdEvalUnified(true, msg)\n\tcase \"script load\":\n\t\tres, err = s.cmdScriptLoad(msg)\n\tcase \"script exists\":\n\t\tres, err = s.cmdScriptExists(msg)\n\tcase \"script flush\":\n\t\tres, err = s.cmdScriptFlush(msg)\n\tcase \"subscribe\":\n\t\tres, err = s.cmdSubscribe(msg)\n\tcase \"psubscribe\":\n\t\tres, err = s.cmdPsubscribe(msg)\n\tcase \"publish\":\n\t\tres, err = s.cmdPublish(msg)\n\tcase \"test\":\n\t\tres, err = s.cmdTEST(msg)\n\tcase \"monitor\":\n\t\tres, err = s.cmdMonitor(msg)\n\t}\n\n\ts.sendMonitor(err, msg, client, false)\n\treturn\n}\n\n// This phrase is copied nearly verbatim from Redis.\nvar deniedMessage = []byte(strings.Replace(strings.TrimSpace(`\n-DENIED Tile38 is running in protected mode because protected mode is enabled,\nno bind address was specified, no authentication password is requested to\nclients. In this mode connections are only accepted from the loopback\ninterface. If you want to connect from external computers to Tile38 you may\nadopt one of the following solutions: 1) Just disable protected mode sending\nthe command 'CONFIG SET protected-mode no' from the loopback interface by\nconnecting to Tile38 from the same host the server is running, however MAKE\nSURE Tile38 is not publicly accessible from internet if you do so. Use CONFIG\nREWRITE to make this change permanent. 2) Alternatively you can just disable\nthe protected mode by editing the Tile38 configuration file, and setting the\nprotected mode option to 'no', and then restarting the server. 3) If you\nstarted the server manually just for testing, restart it with the\n'--protected-mode no' option. 4) Setup a bind address or an authentication\npassword. NOTE: You only need to do one of the above things in order for the\nserver to start accepting connections from the outside.\n`), \"\\n\", \" \", -1) + \"\\r\\n\")\n\n// WriteWebSocketMessage write a websocket message to an io.Writer.\nfunc WriteWebSocketMessage(w io.Writer, data []byte) error {\n\tvar msg []byte\n\tbuf := make([]byte, 10+len(data))\n\tbuf[0] = 129 // FIN + TEXT\n\tif len(data) <= 125 {\n\t\tbuf[1] = byte(len(data))\n\t\tcopy(buf[2:], data)\n\t\tmsg = buf[:2+len(data)]\n\t} else if len(data) <= 0xFFFF {\n\t\tbuf[1] = 126\n\t\tbinary.BigEndian.PutUint16(buf[2:], uint16(len(data)))\n\t\tcopy(buf[4:], data)\n\t\tmsg = buf[:4+len(data)]\n\t} else {\n\t\tbuf[1] = 127\n\t\tbinary.BigEndian.PutUint64(buf[2:], uint64(len(data)))\n\t\tcopy(buf[10:], data)\n\t\tmsg = buf[:10+len(data)]\n\t}\n\t_, err := w.Write(msg)\n\treturn err\n}\n\n// OKMessage returns a default OK message in JSON or RESP.\nfunc OKMessage(msg *Message, start time.Time) resp.Value {\n\tswitch msg.OutputType {\n\tcase JSON:\n\t\treturn resp.StringValue(`{\"ok\":true,\"elapsed\":\"` + time.Since(start).String() + \"\\\"}\")\n\tcase RESP:\n\t\treturn resp.SimpleStringValue(\"OK\")\n\t}\n\treturn resp.SimpleStringValue(\"\")\n}\n\n// NOMessage is no message\nvar NOMessage = resp.SimpleStringValue(\"\")\n\nvar errInvalidHTTP = errors.New(\"invalid HTTP request\")\n\n// Type is resp type\ntype Type byte\n\n// Protocol Types\nconst (\n\tNull Type = iota\n\tRESP\n\tTelnet\n\tNative\n\tHTTP\n\tWebSocket\n\tJSON\n)\n\n// Message is a resp message\ntype Message struct {\n\t_command string\n\tArgs []string\n\tStrictRESP bool\n\tConnType Type\n\tOutputType Type\n\tAuth string\n\tAcceptEncoding string\n\tDeadline *deadline.Deadline\n}\n\n// Command returns the first argument as a lowercase string\nfunc (msg *Message) Command() string {\n\tif msg._command == \"\" {\n\t\tmsg._command = strings.ToLower(msg.Args[0])\n\t}\n\treturn msg._command\n}\n\n// PipelineReader ...\ntype PipelineReader struct {\n\trd io.Reader\n\twr io.Writer\n\tpacket [0xFFFF]byte\n\tbuf []byte\n}\n\nconst kindHTTP redcon.Kind = 9999\n\n// NewPipelineReader ...\nfunc NewPipelineReader(rd io.ReadWriter) *PipelineReader {\n\treturn &PipelineReader{rd: rd, wr: rd}\n}\n\nfunc readcrlfline(packet []byte) (line string, leftover []byte, ok bool) {\n\tfor i := 1; i < len(packet); i++ {\n\t\tif packet[i] == '\\n' && packet[i-1] == '\\r' {\n\t\t\treturn string(packet[:i-1]), packet[i+1:], true\n\t\t}\n\t}\n\treturn \"\", packet, false\n}\n\nfunc headerValue(header, name string) int {\n\ti := 0\n\tfor ; i < len(name) && i < len(header); i++ {\n\t\ta := header[i]\n\t\tif a >= 'A' && a <= 'Z' {\n\t\t\ta += 32\n\t\t}\n\t\tb := name[i]\n\t\tif b >= 'A' && b <= 'Z' {\n\t\t\tb += 32\n\t\t}\n\t\tif a != b {\n\t\t\treturn -1\n\t\t}\n\t}\n\tif i != len(name) || i == len(header) || header[i] != ':' {\n\t\treturn -1\n\t}\n\ti++\n\tfor ; i < len(header); i++ {\n\t\tif header[i] != ' ' && header[i] != '\\t' {\n\t\t\tbreak\n\t\t}\n\t}\n\treturn i\n}\n\nfunc readNextHTTPCommand(packet []byte, argsIn [][]byte, msg *Message, wr io.Writer) (\n\tcomplete bool, args [][]byte, kind redcon.Kind, leftover []byte, err error,\n) {\n\targs = argsIn[:0]\n\tmsg.ConnType = HTTP\n\tmsg.OutputType = JSON\n\tmsg.StrictRESP = false\n\topacket := packet\n\n\tready, err := func() (bool, error) {\n\t\tvar line string\n\t\tvar ok bool\n\n\t\t// read header\n\t\tvar headers []string\n\t\tfor {\n\t\t\tline, packet, ok = readcrlfline(packet)\n\t\t\tif !ok {\n\t\t\t\treturn false, nil\n\t\t\t}\n\t\t\tif line == \"\" {\n\t\t\t\tbreak\n\t\t\t}\n\t\t\theaders = append(headers, line)\n\t\t}\n\t\tparts := strings.Split(headers[0], \" \")\n\t\tif len(parts) != 3 {\n\t\t\treturn false, errInvalidHTTP\n\t\t}\n\t\tmethod := parts[0]\n\t\tpath := parts[1]\n\t\t// Handle CORS request for allowed origins\n\t\tif method == \"OPTIONS\" {\n\t\t\tif wr == nil {\n\t\t\t\treturn false, errors.New(\"connection is nil\")\n\t\t\t}\n\t\t\tcorshead := \"HTTP/1.1 204 No Content\\r\\n\" +\n\t\t\t\t\"Connection: close\\r\\n\" +\n\t\t\t\t\"Access-Control-Allow-Origin: *\\r\\n\" +\n\t\t\t\t\"Access-Control-Allow-Headers: *, Authorization\\r\\n\" +\n\t\t\t\t\"Access-Control-Allow-Methods: POST, GET, OPTIONS\\r\\n\\r\\n\"\n\n\t\t\tif _, err = wr.Write([]byte(corshead)); err != nil {\n\t\t\t\treturn false, err\n\t\t\t}\n\t\t\treturn false, nil\n\t\t}\n\t\tif len(path) == 0 || path[0] != '/' {\n\t\t\treturn false, errInvalidHTTP\n\t\t}\n\t\tpath, err = url.QueryUnescape(path[1:])\n\t\tif err != nil {\n\t\t\treturn false, errInvalidHTTP\n\t\t}\n\t\tif method != \"GET\" && method != \"POST\" {\n\t\t\treturn false, errInvalidHTTP\n\t\t}\n\t\tcontentLength := 0\n\t\twebsocket := false\n\t\twebsocketVersion := 0\n\t\twebsocketKey := \"\"\n\t\tacceptEncoding := \"\"\n\t\tfor _, hdr := range headers[1:] {\n\t\t\tvar i int\n\t\t\tif i = headerValue(hdr, \"Accept-Encoding\"); i != -1 {\n\t\t\t\tacceptEncoding = strings.TrimSpace(hdr[i:])\n\t\t\t} else if i = headerValue(hdr, \"Authorization\"); i != -1 {\n\t\t\t\tmsg.Auth = strings.TrimSpace(hdr[i:])\n\t\t\t} else if i = headerValue(hdr, \"Upgrade\"); i != -1 {\n\t\t\t\tval := strings.TrimSpace(hdr[i:])\n\t\t\t\tif strings.ToLower(val) == \"websocket\" {\n\t\t\t\t\twebsocket = true\n\t\t\t\t}\n\t\t\t} else if i = headerValue(hdr, \"Sec-Websocket-Version\"); i != -1 {\n\t\t\t\tval := strings.TrimSpace(hdr[i:])\n\t\t\t\tn, err := strconv.ParseUint(strings.TrimSpace(val), 10, 64)\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn false, err\n\t\t\t\t}\n\t\t\t\twebsocketVersion = int(n)\n\t\t\t} else if i = headerValue(hdr, \"Sec-Websocket-Key\"); i != -1 {\n\t\t\t\twebsocketKey = strings.TrimSpace(hdr[i:])\n\t\t\t} else if i = headerValue(hdr, \"Content-Length\"); i != -1 {\n\t\t\t\tval := strings.TrimSpace(hdr[i:])\n\t\t\t\tn, err := strconv.ParseUint(strings.TrimSpace(val), 10, 64)\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn false, err\n\t\t\t\t}\n\t\t\t\tcontentLength = int(n)\n\t\t\t}\n\t\t}\n\t\tif websocket && websocketVersion >= 13 && websocketKey != \"\" {\n\t\t\tmsg.ConnType = WebSocket\n\t\t\tif wr == nil {\n\t\t\t\treturn false, errors.New(\"connection is nil\")\n\t\t\t}\n\t\t\tsum := sha1.Sum([]byte(websocketKey + \"258EAFA5-E914-47DA-95CA-C5AB0DC85B11\"))\n\t\t\taccept := base64.StdEncoding.EncodeToString(sum[:])\n\t\t\twshead := \"HTTP/1.1 101 Switching Protocols\\r\\nUpgrade: websocket\\r\\nConnection: Upgrade\\r\\nSec-WebSocket-Accept: \" + accept + \"\\r\\n\\r\\n\"\n\t\t\tif _, err = wr.Write([]byte(wshead)); err != nil {\n\t\t\t\treturn false, err\n\t\t\t}\n\t\t} else if contentLength > 0 {\n\t\t\tmsg.ConnType = HTTP\n\t\t\tif len(packet) < contentLength {\n\t\t\t\treturn false, nil\n\t\t\t}\n\t\t\tpath += string(packet[:contentLength])\n\t\t\tpacket = packet[contentLength:]\n\t\t}\n\t\tif path == \"\" {\n\t\t\treturn true, nil\n\t\t}\n\t\tnmsg, err := readNativeMessageLine([]byte(path))\n\t\tif err != nil {\n\t\t\treturn false, err\n\t\t}\n\t\tmsg.AcceptEncoding = acceptEncoding\n\t\tmsg.OutputType = JSON\n\t\tmsg.StrictRESP = false\n\t\tmsg.Args = nmsg.Args\n\t\treturn true, nil\n\t}()\n\tif err != nil || !ready {\n\t\treturn false, args[:0], kindHTTP, opacket, err\n\t}\n\treturn true, args[:0], kindHTTP, packet, nil\n}\nfunc readNextCommand(packet []byte, argsIn [][]byte, msg *Message, wr io.Writer) (\n\tcomplete bool, args [][]byte, kind redcon.Kind, leftover []byte, err error,\n) {\n\tif packet[0] == 'G' || packet[0] == 'P' || packet[0] == 'O' {\n\t\t// could be an HTTP request\n\t\tvar line []byte\n\t\tfor i := 1; i < len(packet); i++ {\n\t\t\tif packet[i] == '\\n' {\n\t\t\t\tif packet[i-1] == '\\r' {\n\t\t\t\t\tline = packet[:i+1]\n\t\t\t\t\tbreak\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t\tif len(line) == 0 {\n\t\t\treturn false, argsIn[:0], redcon.Redis, packet, nil\n\t\t}\n\t\tif len(line) > 11 && string(line[len(line)-11:len(line)-5]) == \" HTTP/\" {\n\t\t\treturn readNextHTTPCommand(packet, argsIn, msg, wr)\n\t\t}\n\t}\n\treturn redcon.ReadNextCommand(packet, args)\n}\n\n// ReadMessages ...\nfunc (rd *PipelineReader) ReadMessages() ([]*Message, error) {\n\tvar msgs []*Message\nmoreData:\n\tn, err := rd.rd.Read(rd.packet[:])\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tif n == 0 {\n\t\t// need more data\n\t\tgoto moreData\n\t}\n\tdata := rd.packet[:n]\n\tif len(rd.buf) > 0 {\n\t\tdata = append(rd.buf, data...)\n\t}\n\tfor len(data) > 0 {\n\t\tmsg := &Message{}\n\t\tcomplete, args, kind, leftover, err2 :=\n\t\t\treadNextCommand(data, nil, msg, rd.wr)\n\t\tif err2 != nil {\n\t\t\terr = err2\n\t\t\tbreak\n\t\t}\n\t\tif !complete {\n\t\t\tbreak\n\t\t}\n\t\tif kind == kindHTTP {\n\t\t\tif len(msg.Args) == 0 {\n\t\t\t\treturn nil, errInvalidHTTP\n\t\t\t}\n\t\t\tmsgs = append(msgs, msg)\n\t\t} else if len(args) > 0 {\n\t\t\tfor i := 0; i < len(args); i++ {\n\t\t\t\tmsg.Args = append(msg.Args, string(args[i]))\n\t\t\t}\n\t\t\tswitch kind {\n\t\t\tcase redcon.Redis:\n\t\t\t\tmsg.ConnType = RESP\n\t\t\t\tmsg.OutputType = RESP\n\t\t\t\tmsg.StrictRESP = false\n\t\t\tcase redcon.Tile38:\n\t\t\t\tmsg.ConnType = Native\n\t\t\t\tmsg.OutputType = JSON\n\t\t\t\tmsg.StrictRESP = false\n\t\t\tcase redcon.Telnet:\n\t\t\t\tmsg.ConnType = RESP\n\t\t\t\tmsg.OutputType = RESP\n\t\t\t\tmsg.StrictRESP = false\n\t\t\t}\n\t\t\tmsgs = append(msgs, msg)\n\t\t}\n\t\tdata = leftover\n\t}\n\tif len(data) > 0 {\n\t\trd.buf = append(rd.buf[:0], data...)\n\t} else if len(rd.buf) > 0 {\n\t\trd.buf = rd.buf[:0]\n\t}\n\treturn msgs, err\n}\n\nfunc readNativeMessageLine(line []byte) (*Message, error) {\n\tvar args []string\nreading:\n\tfor len(line) != 0 {\n\t\tif line[0] == '{' {\n\t\t\t// The native protocol cannot understand json boundaries so it assumes that\n\t\t\t// a json element must be at the end of the line.\n\t\t\targs = append(args, string(line))\n\t\t\tbreak\n\t\t}\n\t\tif line[0] == '\"' && line[len(line)-1] == '\"' {\n\t\t\tif len(args) > 0 &&\n\t\t\t\tstrings.ToLower(args[0]) == \"set\" &&\n\t\t\t\tstrings.ToLower(args[len(args)-1]) == \"string\" {\n\t\t\t\t// Setting a string value that is contained inside double quotes.\n\t\t\t\t// This is only because of the boundary issues of the native protocol.\n\t\t\t\targs = append(args, string(line[1:len(line)-1]))\n\t\t\t\tbreak\n\t\t\t}\n\t\t}\n\t\ti := 0\n\t\tfor ; i < len(line); i++ {\n\t\t\tif line[i] == ' ' {\n\t\t\t\targ := string(line[:i])\n\t\t\t\tif arg != \"\" {\n\t\t\t\t\targs = append(args, arg)\n\t\t\t\t}\n\t\t\t\tline = line[i+1:]\n\t\t\t\tcontinue reading\n\t\t\t}\n\t\t}\n\t\targs = append(args, string(line))\n\t\tbreak\n\t}\n\treturn &Message{Args: args, ConnType: Native, OutputType: JSON}, nil\n}\n\n// InputStream is a helper type for managing input streams from inside\n// the Data event.\ntype InputStream struct{ b []byte }\n\n// Begin accepts a new packet and returns a working sequence of\n// unprocessed bytes.\nfunc (is *InputStream) Begin(packet []byte) (data []byte) {\n\tdata = packet\n\tif len(is.b) > 0 {\n\t\tis.b = append(is.b, data...)\n\t\tdata = is.b\n\t}\n\treturn data\n}\n\n// End shifts the stream to match the unprocessed data.\nfunc (is *InputStream) End(data []byte) {\n\tif len(data) > 0 {\n\t\tif len(data) != len(is.b) {\n\t\t\tis.b = append(is.b[:0], data...)\n\t\t}\n\t} else if len(is.b) > 0 {\n\t\tis.b = is.b[:0]\n\t}\n}\n\n// clientErrorf is the same as fmt.Errorf, but is intented for errors that are\n// sent back to the client. This allows for the Go static checker to ignore\n// throwing warning for certain error strings.\n// https://staticcheck.io/docs/checks#ST1005\nfunc clientErrorf(format string, args ...interface{}) error {\n\treturn fmt.Errorf(format, args...)\n}\n" }, { "path": "internal/server/stats.go", "content": "package server\n\nimport (\n\t\"bytes\"\n\t\"encoding/json\"\n\t\"errors\"\n\t\"fmt\"\n\t\"os\"\n\t\"runtime\"\n\t\"sort\"\n\t\"strconv\"\n\t\"strings\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/tidwall/buntdb\"\n\t\"github.com/tidwall/resp\"\n\t\"github.com/tidwall/tile38/core\"\n\t\"github.com/tidwall/tile38/internal/collection\"\n)\n\nvar memStats runtime.MemStats\nvar memStatsMu sync.Mutex\nvar memStatsBG bool\n\n// ReadMemStats returns the latest memstats. It provides an instant response.\nfunc readMemStats() runtime.MemStats {\n\tmemStatsMu.Lock()\n\tif !memStatsBG {\n\t\truntime.ReadMemStats(&memStats)\n\t\tgo func() {\n\t\t\tvar ms runtime.MemStats\n\t\t\tfor {\n\t\t\t\truntime.ReadMemStats(&ms)\n\t\t\t\tmemStatsMu.Lock()\n\t\t\t\tmemStats = ms\n\t\t\t\tmemStatsMu.Unlock()\n\t\t\t\ttime.Sleep(time.Second / 5)\n\t\t\t}\n\t\t}()\n\t\tmemStatsBG = true\n\t}\n\tms := memStats\n\tmemStatsMu.Unlock()\n\treturn ms\n}\n\n// STATS key [key...]\nfunc (s *Server) cmdSTATS(msg *Message) (resp.Value, error) {\n\tstart := time.Now()\n\n\t// >> Args\n\n\targs := msg.Args\n\tif len(args) < 2 {\n\t\treturn retrerr(errInvalidNumberOfArguments)\n\t}\n\n\t// >> Operation\n\n\tvar vals []resp.Value\n\tvar ms = []map[string]interface{}{}\n\tfor i := 1; i < len(args); i++ {\n\t\tkey := args[i]\n\t\tcol, _ := s.cols.Get(key)\n\t\tif col != nil {\n\t\t\tm := make(map[string]interface{})\n\t\t\tm[\"num_points\"] = col.PointCount()\n\t\t\tm[\"in_memory_size\"] = col.TotalWeight()\n\t\t\tm[\"num_objects\"] = col.Count()\n\t\t\tm[\"num_strings\"] = col.StringCount()\n\t\t\tswitch msg.OutputType {\n\t\t\tcase JSON:\n\t\t\t\tms = append(ms, m)\n\t\t\tcase RESP:\n\t\t\t\tvals = append(vals, resp.ArrayValue(respValuesSimpleMap(m)))\n\t\t\t}\n\t\t} else {\n\t\t\tswitch msg.OutputType {\n\t\t\tcase JSON:\n\t\t\t\tms = append(ms, nil)\n\t\t\tcase RESP:\n\t\t\t\tvals = append(vals, resp.NullValue())\n\t\t\t}\n\t\t}\n\t}\n\n\t// >> Response\n\n\tif msg.OutputType == JSON {\n\t\tdata, _ := json.Marshal(ms)\n\t\treturn resp.StringValue(`{\"ok\":true,\"stats\":` + string(data) +\n\t\t\t`,\"elapsed\":\"` + time.Since(start).String() + \"\\\"}\"), nil\n\t}\n\treturn resp.ArrayValue(vals), nil\n}\n\n// HEALTHZ\nfunc (s *Server) cmdHEALTHZ(msg *Message) (resp.Value, error) {\n\tstart := time.Now()\n\n\t// >> Args\n\n\targs := msg.Args\n\tif len(args) != 1 {\n\t\treturn retrerr(errInvalidNumberOfArguments)\n\t}\n\n\t// >> Operation\n\n\tif s.config.followHost() != \"\" {\n\t\tif !s.caughtUp() {\n\t\t\treturn retrerr(errors.New(\"not caught up\"))\n\t\t}\n\t}\n\n\t// >> Response\n\n\tif msg.OutputType == JSON {\n\t\treturn resp.StringValue(`{\"ok\":true,\"elapsed\":\"` +\n\t\t\ttime.Since(start).String() + \"\\\"}\"), nil\n\t}\n\treturn resp.SimpleStringValue(\"OK\"), nil\n}\n\n// SERVER [ext]\nfunc (s *Server) cmdSERVER(msg *Message) (resp.Value, error) {\n\tstart := time.Now()\n\n\t// >> Args\n\n\targs := msg.Args\n\tvar ext bool\n\tfor i := 1; i < len(args); i++ {\n\t\tswitch strings.ToLower(args[i]) {\n\t\tcase \"ext\":\n\t\t\text = true\n\t\tdefault:\n\t\t\treturn retrerr(errInvalidArgument(args[i]))\n\t\t}\n\t}\n\n\t// >> Operation\n\n\tm := make(map[string]interface{})\n\n\tif ext {\n\t\ts.extStats(m)\n\t} else {\n\t\ts.basicStats(m)\n\t}\n\n\t// >> Response\n\n\tif msg.OutputType == JSON {\n\t\tdata, _ := json.Marshal(m)\n\t\treturn resp.StringValue(`{\"ok\":true,\"stats\":` + string(data) +\n\t\t\t`,\"elapsed\":\"` + time.Since(start).String() + \"\\\"}\"), nil\n\t}\n\treturn resp.ArrayValue(respValuesSimpleMap(m)), nil\n}\n\n// basicStats populates the passed map with basic system/go/tile38 statistics\nfunc (s *Server) basicStats(m map[string]interface{}) {\n\tm[\"id\"] = s.config.serverID()\n\tif s.config.followHost() != \"\" {\n\t\tm[\"following\"] = fmt.Sprintf(\"%s:%d\", s.config.followHost(),\n\t\t\ts.config.followPort())\n\n\t\tm[\"caught_up\"] = s.caughtUp()\n\t\tm[\"caught_up_once\"] = s.caughtUpOnce()\n\t}\n\tm[\"http_transport\"] = s.http\n\tm[\"pid\"] = os.Getpid()\n\tm[\"aof_size\"] = s.aofsz\n\tm[\"num_collections\"] = s.cols.Len()\n\tm[\"num_hooks\"] = s.hooks.Len()\n\tsz := 0\n\ts.cols.Scan(func(key string, col *collection.Collection) bool {\n\t\tsz += col.TotalWeight()\n\t\treturn true\n\t})\n\tm[\"in_memory_size\"] = sz\n\tpoints := 0\n\tobjects := 0\n\tnstrings := 0\n\ts.cols.Scan(func(key string, col *collection.Collection) bool {\n\t\tpoints += col.PointCount()\n\t\tobjects += col.Count()\n\t\tnstrings += col.StringCount()\n\t\treturn true\n\t})\n\tm[\"num_points\"] = points\n\tm[\"num_objects\"] = objects\n\tm[\"num_strings\"] = nstrings\n\tmem := readMemStats()\n\tavgsz := 0\n\tif points != 0 {\n\t\tavgsz = int(mem.HeapAlloc) / points\n\t}\n\tm[\"mem_alloc\"] = mem.Alloc\n\tm[\"heap_size\"] = mem.HeapAlloc\n\tm[\"heap_released\"] = mem.HeapReleased\n\tm[\"max_heap_size\"] = s.config.maxMemory()\n\tm[\"avg_item_size\"] = avgsz\n\tm[\"version\"] = core.Version\n\tm[\"pointer_size\"] = (32 << uintptr(uint64(^uintptr(0))>>63)) / 8\n\tm[\"read_only\"] = s.config.readOnly()\n\tm[\"cpus\"] = runtime.NumCPU()\n\tn, _ := runtime.ThreadCreateProfile(nil)\n\tm[\"threads\"] = float64(n)\n\tvar nevents int\n\ts.qdb.View(func(tx *buntdb.Tx) error {\n\t\t// All entries in the buntdb log are events, except for one, which\n\t\t// is \"hook:idx\".\n\t\tnevents, _ = tx.Len()\n\t\tnevents -= 1 // Ignore the \"hook:idx\"\n\t\tif nevents < 0 {\n\t\t\tnevents = 0\n\t\t}\n\t\treturn nil\n\t})\n\tm[\"pending_events\"] = nevents\n}\n\n// extStats populates the passed map with extended system/go/tile38 statistics\nfunc (s *Server) extStats(m map[string]interface{}) {\n\tn, _ := runtime.ThreadCreateProfile(nil)\n\tmem := readMemStats()\n\n\t// Go/Memory Stats\n\n\t// Number of goroutines that currently exist\n\tm[\"go_goroutines\"] = runtime.NumGoroutine()\n\t// Number of OS threads created\n\tm[\"go_threads\"] = float64(n)\n\t// A summary of the GC invocation durations\n\tm[\"go_version\"] = runtime.Version()\n\t// Number of bytes allocated and still in use\n\tm[\"alloc_bytes\"] = mem.Alloc\n\t// Total number of bytes allocated, even if freed\n\tm[\"alloc_bytes_total\"] = mem.TotalAlloc\n\t// Number of CPUS available on the system\n\tm[\"sys_cpus\"] = runtime.NumCPU()\n\t// Number of bytes obtained from system\n\tm[\"sys_bytes\"] = mem.Sys\n\t// Total number of pointer lookups\n\tm[\"lookups_total\"] = mem.Lookups\n\t// Total number of mallocs\n\tm[\"mallocs_total\"] = mem.Mallocs\n\t// Total number of frees\n\tm[\"frees_total\"] = mem.Frees\n\t// Number of heap bytes allocated and still in use\n\tm[\"heap_alloc_bytes\"] = mem.HeapAlloc\n\t// Number of heap bytes obtained from system\n\tm[\"heap_sys_bytes\"] = mem.HeapSys\n\t// Number of heap bytes waiting to be used\n\tm[\"heap_idle_bytes\"] = mem.HeapIdle\n\t// Number of heap bytes that are in use\n\tm[\"heap_inuse_bytes\"] = mem.HeapInuse\n\t// Number of heap bytes released to OS\n\tm[\"heap_released_bytes\"] = mem.HeapReleased\n\t// Number of allocated objects\n\tm[\"heap_objects\"] = mem.HeapObjects\n\t// Number of bytes in use by the stack allocator\n\tm[\"stack_inuse_bytes\"] = mem.StackInuse\n\t// Number of bytes obtained from system for stack allocator\n\tm[\"stack_sys_bytes\"] = mem.StackSys\n\t// Number of bytes in use by mspan structures\n\tm[\"mspan_inuse_bytes\"] = mem.MSpanInuse\n\t// Number of bytes used for mspan structures obtained from system\n\tm[\"mspan_sys_bytes\"] = mem.MSpanSys\n\t// Number of bytes in use by mcache structures\n\tm[\"mcache_inuse_bytes\"] = mem.MCacheInuse\n\t// Number of bytes used for mcache structures obtained from system\n\tm[\"mcache_sys_bytes\"] = mem.MCacheSys\n\t// Number of bytes used by the profiling bucket hash table\n\tm[\"buck_hash_sys_bytes\"] = mem.BuckHashSys\n\t// Number of bytes used for garbage collection system metadata\n\tm[\"gc_sys_bytes\"] = mem.GCSys\n\t// Number of bytes used for other system allocations\n\tm[\"other_sys_bytes\"] = mem.OtherSys\n\t// Number of heap bytes when next garbage collection will take place\n\tm[\"next_gc_bytes\"] = mem.NextGC\n\t// Number of seconds since 1970 of last garbage collection\n\tm[\"last_gc_time_seconds\"] = float64(mem.LastGC) / 1e9\n\t// The fraction of this program's available CPU time used by the GC since\n\t// the program started\n\tm[\"gc_cpu_fraction\"] = mem.GCCPUFraction\n\n\t// Tile38 Stats\n\n\t// ID of the server\n\tm[\"tile38_id\"] = s.config.serverID()\n\t// The process ID of the server\n\tm[\"tile38_pid\"] = os.Getpid()\n\t// Version of Tile38 running\n\tm[\"tile38_version\"] = core.Version\n\t// Maximum heap size allowed\n\tm[\"tile38_max_heap_size\"] = s.config.maxMemory()\n\t// Type of instance running\n\tif s.config.followHost() == \"\" {\n\t\tm[\"tile38_type\"] = \"leader\"\n\t} else {\n\t\tm[\"tile38_type\"] = \"follower\"\n\t}\n\t// Whether or not the server is read-only\n\tm[\"tile38_read_only\"] = s.config.readOnly()\n\t// Size of pointer\n\tm[\"tile38_pointer_size\"] = (32 << uintptr(uint64(^uintptr(0))>>63)) / 8\n\t// Uptime of the Tile38 server in seconds\n\tm[\"tile38_uptime_in_seconds\"] = time.Since(s.started).Seconds()\n\t// Number of currently connected Tile38 clients\n\ts.connsmu.RLock()\n\tm[\"tile38_connected_clients\"] = len(s.conns)\n\ts.connsmu.RUnlock()\n\t// Whether or not a cluster is enabled\n\tm[\"tile38_cluster_enabled\"] = false\n\t// Whether or not the Tile38 AOF is enabled\n\tm[\"tile38_aof_enabled\"] = s.opts.AppendOnly\n\t// Whether or not an AOF shrink is currently in progress\n\tm[\"tile38_aof_rewrite_in_progress\"] = s.shrinking\n\t// Length of time the last AOF shrink took\n\tm[\"tile38_aof_last_rewrite_time_sec\"] = s.lastShrinkDuration.Load() / int64(time.Second)\n\t// Duration of the on-going AOF rewrite operation if any\n\tvar currentShrinkStart time.Time\n\tif currentShrinkStart.IsZero() {\n\t\tm[\"tile38_aof_current_rewrite_time_sec\"] = 0\n\t} else {\n\t\tm[\"tile38_aof_current_rewrite_time_sec\"] = time.Since(currentShrinkStart).Seconds()\n\t}\n\t// Total size of the AOF in bytes\n\tm[\"tile38_aof_size\"] = s.aofsz\n\t// Whether or no the HTTP transport is being served\n\tm[\"tile38_http_transport\"] = s.http\n\t// Number of connections accepted by the server\n\tm[\"tile38_total_connections_received\"] = s.statsTotalConns.Load()\n\t// Number of commands processed by the server\n\tm[\"tile38_total_commands_processed\"] = s.statsTotalCommands.Load()\n\t// Number of webhook messages sent by server\n\tm[\"tile38_total_messages_sent\"] = s.statsTotalMsgsSent.Load()\n\t// Number of key expiration events\n\tm[\"tile38_expired_keys\"] = s.statsExpired.Load()\n\t// Number of connected slaves\n\tm[\"tile38_connected_slaves\"] = len(s.aofconnM)\n\n\tpoints := 0\n\tobjects := 0\n\tstrings := 0\n\ts.cols.Scan(func(key string, col *collection.Collection) bool {\n\t\tpoints += col.PointCount()\n\t\tobjects += col.Count()\n\t\tstrings += col.StringCount()\n\t\treturn true\n\t})\n\n\t// Number of points in the database\n\tm[\"tile38_num_points\"] = points\n\t// Number of objects in the database\n\tm[\"tile38_num_objects\"] = objects\n\t// Number of string in the database\n\tm[\"tile38_num_strings\"] = strings\n\t// Number of collections in the database\n\tm[\"tile38_num_collections\"] = s.cols.Len()\n\t// Number of hooks in the database\n\tm[\"tile38_num_hooks\"] = s.hooks.Len()\n\t// Number of hook groups in the database\n\tm[\"tile38_num_hook_groups\"] = s.groupHooks.Len()\n\t// Number of object groups in the database\n\tm[\"tile38_num_object_groups\"] = s.groupObjects.Len()\n\n\tavgsz := 0\n\tif points != 0 {\n\t\tavgsz = int(mem.HeapAlloc) / points\n\t}\n\n\t// Average point size in bytes\n\tm[\"tile38_avg_point_size\"] = avgsz\n\n\tsz := 0\n\ts.cols.Scan(func(key string, col *collection.Collection) bool {\n\t\tsz += col.TotalWeight()\n\t\treturn true\n\t})\n\n\t// Total in memory size of all collections\n\tm[\"tile38_in_memory_size\"] = sz\n}\n\nfunc (s *Server) writeInfoServer(w *bytes.Buffer) {\n\tfmt.Fprintf(w, \"tile38_version:%s\\r\\n\", core.Version)\n\tfmt.Fprintf(w, \"redis_version:%s\\r\\n\", core.Version) // Version of the Redis server\n\tfmt.Fprintf(w, \"uptime_in_seconds:%d\\r\\n\", int(time.Since(s.started).Seconds())) // Number of seconds since Redis server start\n}\nfunc (s *Server) writeInfoClients(w *bytes.Buffer) {\n\ts.connsmu.RLock()\n\tfmt.Fprintf(w, \"connected_clients:%d\\r\\n\", len(s.conns)) // Number of client connections (excluding connections from slaves)\n\ts.connsmu.RUnlock()\n}\nfunc (s *Server) writeInfoMemory(w *bytes.Buffer) {\n\tmem := readMemStats()\n\tfmt.Fprintf(w, \"used_memory:%d\\r\\n\", mem.Alloc) // total number of bytes allocated by Redis using its allocator (either standard libc, jemalloc, or an alternative allocator such as tcmalloc\n}\nfunc boolInt(t bool) int {\n\tif t {\n\t\treturn 1\n\t}\n\treturn 0\n}\nfunc (s *Server) writeInfoPersistence(w *bytes.Buffer) {\n\tfmt.Fprintf(w, \"aof_enabled:%d\\r\\n\", boolInt(s.opts.AppendOnly))\n\tfmt.Fprintf(w, \"aof_rewrite_in_progress:%d\\r\\n\", boolInt(s.shrinking)) // Flag indicating a AOF rewrite operation is on-going\n\tfmt.Fprintf(w, \"aof_last_rewrite_time_sec:%d\\r\\n\", s.lastShrinkDuration.Load()/int64(time.Second)) // Duration of the last AOF rewrite operation in seconds\n\n\tvar currentShrinkStart time.Time // c.currentShrinkStart.get()\n\tif currentShrinkStart.IsZero() {\n\t\tfmt.Fprintf(w, \"aof_current_rewrite_time_sec:0\\r\\n\") // Duration of the on-going AOF rewrite operation if any\n\t} else {\n\t\tfmt.Fprintf(w, \"aof_current_rewrite_time_sec:%d\\r\\n\", time.Since(currentShrinkStart)/time.Second) // Duration of the on-going AOF rewrite operation if any\n\t}\n}\n\nfunc (s *Server) writeInfoStats(w *bytes.Buffer) {\n\tfmt.Fprintf(w, \"total_connections_received:%d\\r\\n\", s.statsTotalConns.Load()) // Total number of connections accepted by the server\n\tfmt.Fprintf(w, \"total_commands_processed:%d\\r\\n\", s.statsTotalCommands.Load()) // Total number of commands processed by the server\n\tfmt.Fprintf(w, \"total_messages_sent:%d\\r\\n\", s.statsTotalMsgsSent.Load()) // Total number of commands processed by the server\n\tfmt.Fprintf(w, \"expired_keys:%d\\r\\n\", s.statsExpired.Load()) // Total number of key expiration events\n}\n\nfunc replicaIPAndPort(cc *Client) (ip string, port int) {\n\tip = cc.remoteAddr\n\tif cc.replAddr != \"\" {\n\t\tip = cc.replAddr\n\t}\n\ti := strings.LastIndex(ip, \":\")\n\tif i != -1 {\n\t\tip = ip[:i]\n\t\tif ip == \"[::1]\" {\n\t\t\tip = \"localhost\"\n\t\t}\n\t}\n\tport = cc.replPort\n\treturn ip, port\n}\n\n// writeInfoReplication writes all replication data to the 'info' response\nfunc (s *Server) writeInfoReplication(w *bytes.Buffer) {\n\tif s.config.followHost() != \"\" {\n\t\tfmt.Fprintf(w, \"role:slave\\r\\n\")\n\t\tfmt.Fprintf(w, \"master_host:%s\\r\\n\", s.config.followHost())\n\t\tfmt.Fprintf(w, \"master_port:%v\\r\\n\", s.config.followPort())\n\t\tfmt.Fprintf(w, \"slave_repl_offset:%v\\r\\n\", int(s.faofsz))\n\t\tif s.config.replicaPriority() >= 0 {\n\t\t\tfmt.Fprintf(w, \"slave_priority:%v\\r\\n\", s.config.replicaPriority())\n\t\t}\n\t} else {\n\t\tfmt.Fprintf(w, \"role:master\\r\\n\")\n\t\tvar i int\n\t\ts.connsmu.RLock()\n\t\tfor _, cc := range s.conns {\n\t\t\tif cc.replPort != 0 {\n\t\t\t\tip, port := replicaIPAndPort(cc)\n\t\t\t\tfmt.Fprintf(w, \"slave%v:ip=%s,port=%v,state=online\\r\\n\", i,\n\t\t\t\t\tip, port)\n\t\t\t\ti++\n\t\t\t}\n\t\t}\n\t\ts.connsmu.RUnlock()\n\t}\n\tfmt.Fprintf(w, \"connected_slaves:%d\\r\\n\", len(s.aofconnM)) // Number of connected slaves\n}\n\nfunc (s *Server) writeInfoCluster(w *bytes.Buffer) {\n\tfmt.Fprintf(w, \"cluster_enabled:0\\r\\n\")\n}\n\n// INFO [section ...]\nfunc (s *Server) cmdINFO(msg *Message) (res resp.Value, err error) {\n\tstart := time.Now()\n\n\t// >> Args\n\n\targs := msg.Args\n\n\tmsects := make(map[string]bool)\n\tallsects := []string{\n\t\t\"server\", \"clients\", \"memory\", \"persistence\", \"stats\",\n\t\t\"replication\", \"cpu\", \"cluster\", \"keyspace\",\n\t}\n\n\tif len(args) == 1 {\n\t\tfor _, s := range allsects {\n\t\t\tmsects[s] = true\n\t\t}\n\t}\n\tfor i := 1; i < len(args); i++ {\n\t\tsection := strings.ToLower(args[i])\n\t\tswitch section {\n\t\tcase \"all\", \"default\":\n\t\t\tfor _, s := range allsects {\n\t\t\t\tmsects[s] = true\n\t\t\t}\n\t\tdefault:\n\t\t\tfor _, s := range allsects {\n\t\t\t\tif s == section {\n\t\t\t\t\tmsects[section] = true\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\n\t// >> Operation\n\n\tvar sects []string\n\tfor _, s := range allsects {\n\t\tif msects[s] {\n\t\t\tsects = append(sects, s)\n\t\t}\n\t}\n\n\tw := &bytes.Buffer{}\n\tfor i, section := range sects {\n\t\tif i > 0 {\n\t\t\tw.WriteString(\"\\r\\n\")\n\t\t}\n\t\tswitch strings.ToLower(section) {\n\t\tdefault:\n\t\t\tcontinue\n\t\tcase \"server\":\n\t\t\tw.WriteString(\"# Server\\r\\n\")\n\t\t\ts.writeInfoServer(w)\n\t\tcase \"clients\":\n\t\t\tw.WriteString(\"# Clients\\r\\n\")\n\t\t\ts.writeInfoClients(w)\n\t\tcase \"memory\":\n\t\t\tw.WriteString(\"# Memory\\r\\n\")\n\t\t\ts.writeInfoMemory(w)\n\t\tcase \"persistence\":\n\t\t\tw.WriteString(\"# Persistence\\r\\n\")\n\t\t\ts.writeInfoPersistence(w)\n\t\tcase \"stats\":\n\t\t\tw.WriteString(\"# Stats\\r\\n\")\n\t\t\ts.writeInfoStats(w)\n\t\tcase \"replication\":\n\t\t\tw.WriteString(\"# Replication\\r\\n\")\n\t\t\ts.writeInfoReplication(w)\n\t\tcase \"cpu\":\n\t\t\tw.WriteString(\"# CPU\\r\\n\")\n\t\t\ts.writeInfoCPU(w)\n\t\tcase \"cluster\":\n\t\t\tw.WriteString(\"# Cluster\\r\\n\")\n\t\t\ts.writeInfoCluster(w)\n\t\t}\n\t}\n\n\t// >> Response\n\n\tif msg.OutputType == JSON {\n\t\t// Create a map of all key/value info fields\n\t\tm := make(map[string]interface{})\n\t\tfor _, kv := range strings.Split(w.String(), \"\\r\\n\") {\n\t\t\tkv = strings.TrimSpace(kv)\n\t\t\tif !strings.HasPrefix(kv, \"#\") {\n\t\t\t\tif split := strings.SplitN(kv, \":\", 2); len(split) == 2 {\n\t\t\t\t\tm[split[0]] = tryParseType(split[1])\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\n\t\t// Marshal the map and use the output in the JSON response\n\t\tdata, _ := json.Marshal(m)\n\t\treturn resp.StringValue(`{\"ok\":true,\"info\":` + string(data) +\n\t\t\t`,\"elapsed\":\"` + time.Since(start).String() + \"\\\"}\"), nil\n\t}\n\treturn resp.BytesValue(w.Bytes()), nil\n}\n\n// tryParseType attempts to parse the passed string as an integer, float64 and\n// a bool returning any successful parsed values. It returns the passed string\n// if all tries fail\nfunc tryParseType(str string) interface{} {\n\tif v, err := strconv.ParseInt(str, 10, 64); err == nil {\n\t\treturn v\n\t}\n\tif v, err := strconv.ParseFloat(str, 64); err == nil {\n\t\treturn v\n\t}\n\tif v, err := strconv.ParseBool(str); err == nil {\n\t\treturn v\n\t}\n\treturn str\n}\n\nfunc respValuesSimpleMap(m map[string]interface{}) []resp.Value {\n\tvar keys []string\n\tfor key := range m {\n\t\tkeys = append(keys, key)\n\t}\n\tsort.Strings(keys)\n\tvar vals []resp.Value\n\tfor _, key := range keys {\n\t\tval := m[key]\n\t\tvals = append(vals, resp.StringValue(key))\n\t\tvals = append(vals, resp.StringValue(fmt.Sprintf(\"%v\", val)))\n\t}\n\treturn vals\n}\n\n// ROLE\nfunc (s *Server) cmdROLE(msg *Message) (res resp.Value, err error) {\n\tstart := time.Now()\n\tvar role string\n\tvar offset int\n\tvar ips []string\n\tvar ports []int\n\tvar offsets []int\n\tvar host string\n\tvar port int\n\tvar state string\n\tif s.config.followHost() == \"\" {\n\t\trole = \"master\"\n\t\toffset = s.aofsz\n\t\ts.connsmu.RLock()\n\t\tfor _, cc := range s.conns {\n\t\t\tif cc.replPort != 0 {\n\t\t\t\tip, port := replicaIPAndPort(cc)\n\t\t\t\tips = append(ips, ip)\n\t\t\t\tports = append(ports, port)\n\t\t\t\toffsets = append(offsets, s.aofsz)\n\t\t\t}\n\t\t}\n\t\ts.connsmu.RUnlock()\n\t} else {\n\t\trole = \"slave\"\n\t\thost = s.config.followHost()\n\t\tport = s.config.followPort()\n\t\toffset = int(s.faofsz)\n\t\tstate = \"connected\"\n\t}\n\tif msg.OutputType == JSON {\n\t\tvar json []byte\n\t\tjson = append(json, `{\"ok\":true,\"role\":{`...)\n\t\tjson = append(json, `\"role\":`...)\n\t\tjson = appendJSONString(json, role)\n\t\tif role == \"master\" {\n\t\t\tjson = append(json, `,\"offset\":`...)\n\t\t\tjson = strconv.AppendInt(json, int64(offset), 10)\n\t\t\tjson = append(json, `,\"slaves\":[`...)\n\t\t\tfor i := range ips {\n\t\t\t\tif i > 0 {\n\t\t\t\t\tjson = append(json, ',')\n\t\t\t\t}\n\t\t\t\tjson = append(json, '{')\n\t\t\t\tjson = append(json, `\"ip\":`...)\n\t\t\t\tjson = appendJSONString(json, ips[i])\n\t\t\t\tjson = append(json, `,\"port\":`...)\n\t\t\t\tjson = appendJSONString(json, fmt.Sprint(ports[i]))\n\t\t\t\tjson = append(json, `,\"offset\":`...)\n\t\t\t\tjson = appendJSONString(json, fmt.Sprint(offsets[i]))\n\t\t\t\tjson = append(json, '}')\n\t\t\t}\n\t\t\tjson = append(json, `]`...)\n\t\t} else if role == \"slave\" {\n\t\t\tjson = append(json, `,\"host\":`...)\n\t\t\tjson = appendJSONString(json, host)\n\t\t\tjson = append(json, `,\"port\":`...)\n\t\t\tjson = strconv.AppendInt(json, int64(port), 10)\n\t\t\tjson = append(json, `,\"state\":`...)\n\t\t\tjson = appendJSONString(json, state)\n\t\t\tjson = append(json, `,\"offset\":`...)\n\t\t\tjson = strconv.AppendInt(json, int64(offset), 10)\n\t\t}\n\t\tjson = append(json, `},\"elapsed\":`...)\n\t\tjson = appendJSONString(json, time.Since(start).String())\n\t\tjson = append(json, '}')\n\t\treturn resp.StringValue(string(json)), nil\n\t} else {\n\t\tvar vals []resp.Value\n\t\tvals = append(vals, resp.StringValue(role))\n\t\tif role == \"master\" {\n\t\t\tvals = append(vals, resp.IntegerValue(offset))\n\t\t\tvar replicaVals []resp.Value\n\t\t\tfor i := range ips {\n\t\t\t\tvar vals []resp.Value\n\t\t\t\tvals = append(vals, resp.StringValue(ips[i]))\n\t\t\t\tvals = append(vals, resp.StringValue(fmt.Sprint(ports[i])))\n\t\t\t\tvals = append(vals, resp.StringValue(fmt.Sprint(offsets[i])))\n\t\t\t\treplicaVals = append(replicaVals, resp.ArrayValue(vals))\n\t\t\t}\n\t\t\tvals = append(vals, resp.ArrayValue(replicaVals))\n\t\t} else if role == \"slave\" {\n\t\t\tvals = append(vals, resp.StringValue(host))\n\t\t\tvals = append(vals, resp.IntegerValue(port))\n\t\t\tvals = append(vals, resp.StringValue(state))\n\t\t\tvals = append(vals, resp.IntegerValue(offset))\n\t\t}\n\t\treturn resp.ArrayValue(vals), nil\n\t}\n}\n" }, { "path": "internal/server/stats_cpu.go", "content": "//go:build !linux && !darwin\n\npackage server\n\nimport (\n\t\"bytes\"\n\t\"fmt\"\n)\n\nfunc (s *Server) writeInfoCPU(w *bytes.Buffer) {\n\tfmt.Fprintf(w,\n\t\t\"used_cpu_sys:%.2f\\r\\n\"+\n\t\t\t\"used_cpu_user:%.2f\\r\\n\"+\n\t\t\t\"used_cpu_sys_children:%.2f\\r\\n\"+\n\t\t\t\"used_cpu_user_children:%.2f\\r\\n\",\n\t\t0.0, 0.0, 0.0, 0.0,\n\t)\n}\n" }, { "path": "internal/server/stats_cpu_darlin.go", "content": "//go:build linux || darwin\n\npackage server\n\nimport (\n\t\"bytes\"\n\t\"fmt\"\n\t\"syscall\"\n)\n\nfunc (s *Server) writeInfoCPU(w *bytes.Buffer) {\n\tvar selfRu syscall.Rusage\n\tvar cRu syscall.Rusage\n\tsyscall.Getrusage(syscall.RUSAGE_SELF, &selfRu)\n\tsyscall.Getrusage(syscall.RUSAGE_CHILDREN, &cRu)\n\tfmt.Fprintf(w,\n\t\t\"used_cpu_sys:%.2f\\r\\n\"+\n\t\t\t\"used_cpu_user:%.2f\\r\\n\"+\n\t\t\t\"used_cpu_sys_children:%.2f\\r\\n\"+\n\t\t\t\"used_cpu_user_children:%.2f\\r\\n\",\n\t\tfloat64(selfRu.Stime.Sec)+float64(selfRu.Stime.Usec/1000000),\n\t\tfloat64(selfRu.Utime.Sec)+float64(selfRu.Utime.Usec/1000000),\n\t\tfloat64(cRu.Stime.Sec)+float64(cRu.Stime.Usec/1000000),\n\t\tfloat64(cRu.Utime.Sec)+float64(cRu.Utime.Usec/1000000),\n\t)\n}\n" }, { "path": "internal/server/test.go", "content": "package server\n\n// TEST command: spatial tests without walking the tree.\n\nimport (\n\t\"bytes\"\n\t\"fmt\"\n\t\"strconv\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/iwpnd/sectr\"\n\t\"github.com/mmcloughlin/geohash\"\n\t\"github.com/tidwall/geojson\"\n\t\"github.com/tidwall/geojson/geometry\"\n\t\"github.com/tidwall/resp\"\n\t\"github.com/tidwall/tile38/internal/bing\"\n\t\"github.com/tidwall/tile38/internal/clip\"\n)\n\nfunc (s *Server) parseArea(ovs []string, doClip bool) (vs []string, o geojson.Object, err error) {\n\tvar ok bool\n\tvar typ string\n\tvs = ovs[:]\n\tif vs, typ, ok = tokenval(vs); !ok || typ == \"\" {\n\t\terr = errInvalidNumberOfArguments\n\t\treturn\n\t}\n\tltyp := strings.ToLower(typ)\n\tswitch ltyp {\n\tcase \"point\":\n\t\tvar slat, slon string\n\t\tif vs, slat, ok = tokenval(vs); !ok || slat == \"\" {\n\t\t\terr = errInvalidNumberOfArguments\n\t\t\treturn\n\t\t}\n\t\tif vs, slon, ok = tokenval(vs); !ok || slon == \"\" {\n\t\t\terr = errInvalidNumberOfArguments\n\t\t\treturn\n\t\t}\n\t\tvar lat, lon float64\n\t\tif lat, err = strconv.ParseFloat(slat, 64); err != nil {\n\t\t\terr = errInvalidArgument(slat)\n\t\t\treturn\n\t\t}\n\t\tif lon, err = strconv.ParseFloat(slon, 64); err != nil {\n\t\t\terr = errInvalidArgument(slon)\n\t\t\treturn\n\t\t}\n\t\to = geojson.NewPoint(geometry.Point{X: lon, Y: lat})\n\tcase \"sector\":\n\t\tif doClip {\n\t\t\terr = fmt.Errorf(\"invalid clip type '%s'\", typ)\n\t\t\treturn\n\t\t}\n\t\tvar slat, slon, smeters, sb1, sb2 string\n\t\tif vs, slat, ok = tokenval(vs); !ok || slat == \"\" {\n\t\t\terr = errInvalidNumberOfArguments\n\t\t\treturn\n\t\t}\n\t\tif vs, slon, ok = tokenval(vs); !ok || slon == \"\" {\n\t\t\terr = errInvalidNumberOfArguments\n\t\t\treturn\n\t\t}\n\t\tif vs, smeters, ok = tokenval(vs); !ok || smeters == \"\" {\n\t\t\terr = errInvalidNumberOfArguments\n\t\t\treturn\n\t\t}\n\t\tif vs, sb1, ok = tokenval(vs); !ok || sb1 == \"\" {\n\t\t\terr = errInvalidNumberOfArguments\n\t\t\treturn\n\t\t}\n\t\tif vs, sb2, ok = tokenval(vs); !ok || sb2 == \"\" {\n\t\t\terr = errInvalidNumberOfArguments\n\t\t\treturn\n\t\t}\n\t\tvar lat, lon, meters, b1, b2 float64\n\t\tif lat, err = strconv.ParseFloat(slat, 64); err != nil {\n\t\t\terr = errInvalidArgument(slat)\n\t\t\treturn\n\t\t}\n\t\tif lon, err = strconv.ParseFloat(slon, 64); err != nil {\n\t\t\terr = errInvalidArgument(slon)\n\t\t\treturn\n\t\t}\n\t\tif meters, err = strconv.ParseFloat(smeters, 64); err != nil {\n\t\t\terr = errInvalidArgument(smeters)\n\t\t\treturn\n\t\t}\n\t\tif b1, err = strconv.ParseFloat(sb1, 64); err != nil {\n\t\t\terr = errInvalidArgument(sb1)\n\t\t\treturn\n\t\t}\n\t\tif b2, err = strconv.ParseFloat(sb2, 64); err != nil {\n\t\t\terr = errInvalidArgument(sb2)\n\t\t\treturn\n\t\t}\n\n\t\tif b1 == b2 {\n\t\t\terr = fmt.Errorf(\"equal bearings (%s == %s), use CIRCLE instead\", sb1, sb2)\n\t\t\treturn\n\t\t}\n\n\t\torigin := sectr.Point{Lng: lon, Lat: lat}\n\t\tsector := sectr.NewSector(origin, meters, b1, b2)\n\n\t\to, err = geojson.Parse(string(sector.JSON()), &s.geomParseOpts)\n\t\tif err != nil {\n\t\t\treturn\n\t\t}\n\n\tcase \"circle\":\n\t\tif doClip {\n\t\t\terr = fmt.Errorf(\"invalid clip type '%s'\", typ)\n\t\t\treturn\n\t\t}\n\t\tvar slat, slon, smeters string\n\t\tif vs, slat, ok = tokenval(vs); !ok || slat == \"\" {\n\t\t\terr = errInvalidNumberOfArguments\n\t\t\treturn\n\t\t}\n\t\tif vs, slon, ok = tokenval(vs); !ok || slon == \"\" {\n\t\t\terr = errInvalidNumberOfArguments\n\t\t\treturn\n\t\t}\n\t\tvar lat, lon, meters float64\n\t\tif lat, err = strconv.ParseFloat(slat, 64); err != nil {\n\t\t\terr = errInvalidArgument(slat)\n\t\t\treturn\n\t\t}\n\t\tif lon, err = strconv.ParseFloat(slon, 64); err != nil {\n\t\t\terr = errInvalidArgument(slon)\n\t\t\treturn\n\t\t}\n\t\tif vs, smeters, ok = tokenval(vs); !ok || smeters == \"\" {\n\t\t\terr = errInvalidNumberOfArguments\n\t\t\treturn\n\t\t}\n\t\tif meters, err = strconv.ParseFloat(smeters, 64); err != nil {\n\t\t\terr = errInvalidArgument(smeters)\n\t\t\treturn\n\t\t}\n\t\tif meters < 0 {\n\t\t\terr = errInvalidArgument(smeters)\n\t\t\treturn\n\t\t}\n\t\to = geojson.NewCircle(geometry.Point{X: lon, Y: lat}, meters, defaultCircleSteps)\n\tcase \"object\":\n\t\tif doClip {\n\t\t\terr = fmt.Errorf(\"invalid clip type '%s'\", typ)\n\t\t\treturn\n\t\t}\n\t\tvar obj string\n\t\tif vs, obj, ok = tokenval(vs); !ok || obj == \"\" {\n\t\t\terr = errInvalidNumberOfArguments\n\t\t\treturn\n\t\t}\n\t\to, err = geojson.Parse(obj, &s.geomParseOpts)\n\t\tif err != nil {\n\t\t\treturn\n\t\t}\n\tcase \"bounds\":\n\t\tvar sminLat, sminLon, smaxlat, smaxlon string\n\t\tif vs, sminLat, ok = tokenval(vs); !ok || sminLat == \"\" {\n\t\t\terr = errInvalidNumberOfArguments\n\t\t\treturn\n\t\t}\n\t\tif vs, sminLon, ok = tokenval(vs); !ok || sminLon == \"\" {\n\t\t\terr = errInvalidNumberOfArguments\n\t\t\treturn\n\t\t}\n\t\tif vs, smaxlat, ok = tokenval(vs); !ok || smaxlat == \"\" {\n\t\t\terr = errInvalidNumberOfArguments\n\t\t\treturn\n\t\t}\n\t\tif vs, smaxlon, ok = tokenval(vs); !ok || smaxlon == \"\" {\n\t\t\terr = errInvalidNumberOfArguments\n\t\t\treturn\n\t\t}\n\t\tvar minLat, minLon, maxLat, maxLon float64\n\t\tif minLat, err = strconv.ParseFloat(sminLat, 64); err != nil {\n\t\t\terr = errInvalidArgument(sminLat)\n\t\t\treturn\n\t\t}\n\t\tif minLon, err = strconv.ParseFloat(sminLon, 64); err != nil {\n\t\t\terr = errInvalidArgument(sminLon)\n\t\t\treturn\n\t\t}\n\t\tif maxLat, err = strconv.ParseFloat(smaxlat, 64); err != nil {\n\t\t\terr = errInvalidArgument(smaxlat)\n\t\t\treturn\n\t\t}\n\t\tif maxLon, err = strconv.ParseFloat(smaxlon, 64); err != nil {\n\t\t\terr = errInvalidArgument(smaxlon)\n\t\t\treturn\n\t\t}\n\t\to = geojson.NewRect(geometry.Rect{\n\t\t\tMin: geometry.Point{X: minLon, Y: minLat},\n\t\t\tMax: geometry.Point{X: maxLon, Y: maxLat},\n\t\t})\n\tcase \"hash\":\n\t\tvar hash string\n\t\tif vs, hash, ok = tokenval(vs); !ok || hash == \"\" {\n\t\t\terr = errInvalidNumberOfArguments\n\t\t\treturn\n\t\t}\n\t\tbox := geohash.BoundingBox(hash)\n\t\to = geojson.NewRect(geometry.Rect{\n\t\t\tMin: geometry.Point{X: box.MinLng, Y: box.MinLat},\n\t\t\tMax: geometry.Point{X: box.MaxLng, Y: box.MaxLat},\n\t\t})\n\tcase \"quadkey\":\n\t\tvar key string\n\t\tif vs, key, ok = tokenval(vs); !ok || key == \"\" {\n\t\t\terr = errInvalidNumberOfArguments\n\t\t\treturn\n\t\t}\n\t\tvar minLat, minLon, maxLat, maxLon float64\n\t\tminLat, minLon, maxLat, maxLon, err = bing.QuadKeyToBounds(key)\n\t\tif err != nil {\n\t\t\terr = errInvalidArgument(key)\n\t\t\treturn\n\t\t}\n\t\to = geojson.NewRect(geometry.Rect{\n\t\t\tMin: geometry.Point{X: minLon, Y: minLat},\n\t\t\tMax: geometry.Point{X: maxLon, Y: maxLat},\n\t\t})\n\tcase \"tile\":\n\t\tvar sx, sy, sz string\n\t\tif vs, sx, ok = tokenval(vs); !ok || sx == \"\" {\n\t\t\terr = errInvalidNumberOfArguments\n\t\t\treturn\n\t\t}\n\t\tif vs, sy, ok = tokenval(vs); !ok || sy == \"\" {\n\t\t\terr = errInvalidNumberOfArguments\n\t\t\treturn\n\t\t}\n\t\tif vs, sz, ok = tokenval(vs); !ok || sz == \"\" {\n\t\t\terr = errInvalidNumberOfArguments\n\t\t\treturn\n\t\t}\n\t\tvar x, y int64\n\t\tvar z uint64\n\t\tif x, err = strconv.ParseInt(sx, 10, 64); err != nil {\n\t\t\terr = errInvalidArgument(sx)\n\t\t\treturn\n\t\t}\n\t\tif y, err = strconv.ParseInt(sy, 10, 64); err != nil {\n\t\t\terr = errInvalidArgument(sy)\n\t\t\treturn\n\t\t}\n\t\tif z, err = strconv.ParseUint(sz, 10, 64); err != nil {\n\t\t\terr = errInvalidArgument(sz)\n\t\t\treturn\n\t\t}\n\t\tvar minLat, minLon, maxLat, maxLon float64\n\t\tminLat, minLon, maxLat, maxLon = bing.TileXYToBounds(x, y, z)\n\t\to = geojson.NewRect(geometry.Rect{\n\t\t\tMin: geometry.Point{X: minLon, Y: minLat},\n\t\t\tMax: geometry.Point{X: maxLon, Y: maxLat},\n\t\t})\n\tcase \"get\":\n\t\tif doClip {\n\t\t\terr = fmt.Errorf(\"invalid clip type '%s'\", typ)\n\t\t\treturn\n\t\t}\n\t\tvar key, id string\n\t\tif vs, key, ok = tokenval(vs); !ok || key == \"\" {\n\t\t\terr = errInvalidNumberOfArguments\n\t\t\treturn\n\t\t}\n\t\tif vs, id, ok = tokenval(vs); !ok || id == \"\" {\n\t\t\terr = errInvalidNumberOfArguments\n\t\t\treturn\n\t\t}\n\t\tcol, _ := s.cols.Get(key)\n\t\tif col == nil {\n\t\t\terr = errKeyNotFound\n\t\t\treturn\n\t\t}\n\t\tobj := col.Get(id)\n\t\tif obj == nil {\n\t\t\terr = errIDNotFound\n\t\t\treturn\n\t\t}\n\t\to = obj.Geo()\n\t}\n\treturn\n}\n\n// TEST (POINT lat lon)|(GET key id)|(BOUNDS minlat minlon maxlat maxlon)|\n// (OBJECT geojson)|(CIRCLE lat lon meters)|(TILE x y z)|(QUADKEY quadkey)|\n// (HASH geohash) INTERSECTS|WITHIN [CLIP] (POINT lat lon)|(GET key id)|\n// (BOUNDS minlat minlon maxlat maxlon)|(OBJECT geojson)|\n// (CIRCLE lat lon meters)|(TILE x y z)|(QUADKEY quadkey)|(HASH geohash)|\n// (SECTOR lat lon meters bearing1 bearing2)\nfunc (s *Server) cmdTEST(msg *Message) (res resp.Value, err error) {\n\tstart := time.Now()\n\n\tvs := msg.Args[1:]\n\n\tvar ok bool\n\tvar test string\n\tvar clipped geojson.Object\n\tvar area1, area2 *areaExpression\n\tif vs, area1, err = s.parseAreaExpression(vs, false); err != nil {\n\t\treturn\n\t}\n\tif vs, test, ok = tokenval(vs); !ok || test == \"\" {\n\t\terr = errInvalidNumberOfArguments\n\t\treturn\n\t}\n\tlTest := strings.ToLower(test)\n\tif lTest != \"within\" && lTest != \"intersects\" {\n\t\terr = errInvalidArgument(test)\n\t\treturn\n\t}\n\tvar wtok string\n\tvar nvs []string\n\tvar doClip bool\n\tnvs, wtok, ok = tokenval(vs)\n\tif ok && len(wtok) > 0 {\n\t\tswitch strings.ToLower(wtok) {\n\t\tcase \"clip\":\n\t\t\tvs = nvs\n\t\t\tif lTest != \"intersects\" {\n\t\t\t\terr = errInvalidArgument(wtok)\n\t\t\t\treturn\n\t\t\t}\n\t\t\tdoClip = true\n\t\t}\n\t}\n\tif vs, area2, err = s.parseAreaExpression(vs, doClip); err != nil {\n\t\treturn\n\t}\n\tif doClip && (area1.obj == nil || area2.obj == nil) {\n\t\terr = errInvalidArgument(\"clip\")\n\t\treturn\n\t}\n\tif len(vs) != 0 {\n\t\terr = errInvalidNumberOfArguments\n\t\treturn\n\t}\n\n\tvar result int\n\tif lTest == \"within\" {\n\t\tif area1.WithinExpr(area2) {\n\t\t\tresult = 1\n\t\t}\n\t} else if lTest == \"intersects\" {\n\t\tif area1.IntersectsExpr(area2) {\n\t\t\tresult = 1\n\t\t\tif doClip {\n\t\t\t\tclipped = clip.Clip(area1.obj, area2.obj, nil)\n\t\t\t}\n\t\t}\n\t}\n\tif msg.OutputType == JSON {\n\t\tvar buf bytes.Buffer\n\t\tbuf.WriteString(`{\"ok\":true`)\n\t\tif result != 0 {\n\t\t\tbuf.WriteString(`,\"result\":true`)\n\t\t} else {\n\t\t\tbuf.WriteString(`,\"result\":false`)\n\t\t}\n\t\tif clipped != nil {\n\t\t\tbuf.WriteString(`,\"object\":` + clipped.JSON())\n\t\t}\n\t\tbuf.WriteString(`,\"elapsed\":\"` + time.Since(start).String() + \"\\\"}\")\n\t\treturn resp.StringValue(buf.String()), nil\n\t}\n\tif clipped != nil {\n\t\treturn resp.ArrayValue([]resp.Value{\n\t\t\tresp.IntegerValue(result),\n\t\t\tresp.StringValue(clipped.JSON())}), nil\n\t}\n\treturn resp.IntegerValue(result), nil\n}\n" }, { "path": "internal/server/token.go", "content": "package server\n\nimport (\n\t\"errors\"\n\t\"fmt\"\n\t\"math\"\n\t\"strconv\"\n\t\"strings\"\n\n\t\"github.com/tidwall/tile38/internal/field\"\n\t\"github.com/tidwall/tile38/internal/log\"\n\tlua \"github.com/yuin/gopher-lua\"\n\tluajson \"layeh.com/gopher-json\"\n)\n\nconst defaultSearchOutput = outputObjects\n\nvar errInvalidNumberOfArguments = errors.New(\"invalid number of arguments\")\nvar errKeyNotFound = errors.New(\"key not found\")\nvar errIDNotFound = errors.New(\"id not found\")\nvar errIDAlreadyExists = errors.New(\"id already exists\")\nvar errPathNotFound = errors.New(\"path not found\")\nvar errKeyHasHooksSet = errors.New(\"key has hooks set\")\nvar errKeyHasChannelsSet = errors.New(\"key has channels set\")\nvar errNotRectangle = errors.New(\"not a rectangle\")\n\nfunc errInvalidArgument(arg string) error {\n\treturn fmt.Errorf(\"invalid argument '%s'\", arg)\n}\nfunc errDuplicateArgument(arg string) error {\n\treturn fmt.Errorf(\"duplicate argument '%s'\", arg)\n}\nfunc token(line string) (newLine, token string) {\n\tfor i := 0; i < len(line); i++ {\n\t\tif line[i] == ' ' {\n\t\t\treturn line[i+1:], line[:i]\n\t\t}\n\t}\n\treturn \"\", line\n}\n\nfunc tokenval(vs []string) (nvs []string, token string, ok bool) {\n\tif len(vs) > 0 {\n\t\ttoken = vs[0]\n\t\tnvs = vs[1:]\n\t\tok = true\n\t}\n\treturn\n}\n\nfunc lc(s1, s2 string) bool {\n\tif len(s1) != len(s2) {\n\t\treturn false\n\t}\n\tfor i := 0; i < len(s1); i++ {\n\t\tch := s1[i]\n\t\tif ch >= 'A' && ch <= 'Z' {\n\t\t\tif ch+32 != s2[i] {\n\t\t\t\treturn false\n\t\t\t}\n\t\t} else if ch != s2[i] {\n\t\t\treturn false\n\t\t}\n\t}\n\treturn true\n}\n\ntype whereT struct {\n\texpr bool\n\tname string\n\tminx bool\n\tmin field.Value\n\tmaxx bool\n\tmax field.Value\n}\n\nfunc mLT(a, b field.Value) bool { return a.Less(b) }\nfunc mLTE(a, b field.Value) bool { return !mLT(b, a) }\nfunc mGT(a, b field.Value) bool { return mLT(b, a) }\nfunc mGTE(a, b field.Value) bool { return !mLT(a, b) }\nfunc mEQ(a, b field.Value) bool { return a.Equals(b) }\n\nfunc (where whereT) matchField(value field.Value) bool {\n\tswitch where.min.Data() {\n\tcase \"<\":\n\t\treturn mLT(value, where.max)\n\tcase \"<=\":\n\t\treturn mLTE(value, where.max)\n\tcase \">\":\n\t\treturn mGT(value, where.max)\n\tcase \">=\":\n\t\treturn mGTE(value, where.max)\n\tcase \"==\":\n\t\treturn mEQ(value, where.max)\n\tcase \"!=\":\n\t\treturn !mEQ(value, where.max)\n\t}\n\tif !where.minx {\n\t\tif mLT(value, where.min) { // if value < where.min {\n\t\t\treturn false\n\t\t}\n\t} else {\n\t\tif mLTE(value, where.min) { // if value <= where.min {\n\t\t\treturn false\n\t\t}\n\t}\n\tif !where.maxx {\n\t\tif mGT(value, where.max) { // if value > where.max {\n\t\t\treturn false\n\t\t}\n\t} else {\n\t\tif mGTE(value, where.max) { // if value >= where.max {\n\t\t\treturn false\n\t\t}\n\t}\n\treturn true\n}\n\ntype whereinT struct {\n\tname string\n\tvalArr []field.Value\n}\n\nfunc (wherein whereinT) match(value field.Value) bool {\n\tfor _, val := range wherein.valArr {\n\t\tif mEQ(val, value) {\n\t\t\treturn true\n\t\t}\n\t}\n\treturn false\n}\n\ntype whereevalT struct {\n\tc *Server\n\tluaState *lua.LState\n\tfn *lua.LFunction\n}\n\nfunc (whereeval whereevalT) Close() {\n\tluaSetRawGlobals(\n\t\twhereeval.luaState, map[string]lua.LValue{\n\t\t\t\"ARGV\": lua.LNil,\n\t\t})\n\twhereeval.c.luapool.Put(whereeval.luaState)\n}\n\nfunc luaSetField(tbl *lua.LTable, name string, val field.Value) {\n\tvar lval lua.LValue\n\tswitch val.Kind() {\n\tcase field.Null:\n\t\tlval = lua.LNil\n\tcase field.False:\n\t\tlval = lua.LFalse\n\tcase field.True:\n\t\tlval = lua.LTrue\n\tcase field.Number:\n\t\tlval = lua.LNumber(val.Num())\n\tdefault:\n\t\tlval = lua.LString(val.Data())\n\t}\n\ttbl.RawSetString(name, lval)\n}\n\nfunc (whereeval whereevalT) match(fieldsWithNames map[string]field.Value,\n\tid string, props string) (ok bool, err error,\n) {\n\tfieldsTbl := whereeval.luaState.CreateTable(0, len(fieldsWithNames))\n\tfor name, val := range fieldsWithNames {\n\t\tluaSetField(fieldsTbl, name, val)\n\t}\n\tpropsTbl := lua.LValue(whereeval.luaState.CreateTable(0, 0))\n\tif props != \"\" {\n\t\tvar err error\n\t\ttbl, err := luajson.Decode(whereeval.luaState, []byte(props))\n\t\tif err == nil {\n\t\t\tpropsTbl = tbl\n\t\t}\n\t}\n\tluaSetRawGlobals(\n\t\twhereeval.luaState, map[string]lua.LValue{\n\t\t\t\"ID\": lua.LString(id),\n\t\t\t\"FIELDS\": fieldsTbl,\n\t\t\t\"PROPERTIES\": propsTbl,\n\t\t})\n\tdefer func() {\n\t\tluaSetRawGlobals(\n\t\t\twhereeval.luaState, map[string]lua.LValue{\n\t\t\t\t\"ID\": lua.LNil,\n\t\t\t\t\"FIELDS\": lua.LNil,\n\t\t\t\t\"PROPERTIES\": lua.LNil,\n\t\t\t})\n\t}()\n\n\twhereeval.luaState.Push(whereeval.fn)\n\tif err := whereeval.luaState.PCall(0, 1, nil); err != nil {\n\t\temsg := err.Error()\n\t\tif strings.Contains(emsg, \"attempt to index a non-table\") {\n\t\t\tlog.Debugf(\"Lua error: %v\", emsg)\n\t\t\treturn false, nil\n\t\t}\n\t\treturn false, err\n\t}\n\tret := whereeval.luaState.Get(-1)\n\twhereeval.luaState.Pop(1)\n\n\tif ret == nil {\n\t\treturn false, nil\n\t}\n\t// Make bool out of returned lua value\n\tswitch ret.Type() {\n\tcase lua.LTNil:\n\t\treturn false, nil\n\tcase lua.LTBool:\n\t\treturn ret == lua.LTrue, nil\n\tcase lua.LTNumber:\n\t\treturn float64(ret.(lua.LNumber)) != 0, nil\n\tcase lua.LTString:\n\t\treturn ret.String() != \"\", nil\n\tcase lua.LTTable:\n\t\ttbl := ret.(*lua.LTable)\n\t\tif tbl.Len() != 0 {\n\t\t\treturn true, nil\n\t\t}\n\t\tvar match bool\n\t\ttbl.ForEach(func(lk lua.LValue, lv lua.LValue) { match = true })\n\t\treturn match, nil\n\t}\n\treturn false, fmt.Errorf(\"script returned value of type %s\", ret.Type())\n}\n\ntype searchScanBaseTokens struct {\n\tkey string\n\tcursor uint64\n\toutput outputT\n\tprecision uint64\n\tfence bool\n\tdistance bool\n\tnodwell bool\n\tdetect map[string]bool\n\taccept map[string]bool\n\tglobs []string\n\twheres []whereT\n\twhereins []whereinT\n\twhereevals []whereevalT\n\tnofields bool\n\tulimit bool\n\tlimit uint64\n\tusparse bool\n\tsparse uint8\n\tdesc bool\n\tclip bool\n\tbuffer float64\n\thasbuffer bool\n\tmvt bool\n\ttileX int\n\ttileY int\n\ttileZ int\n}\n\nfunc (s *Server) parseSearchScanBaseTokens(\n\tcmd string, t searchScanBaseTokens, vs []string,\n) (\n\tvsout []string, tout searchScanBaseTokens, err error,\n) {\n\tvar ok bool\n\tif vs, t.key, ok = tokenval(vs); !ok || t.key == \"\" {\n\t\terr = errInvalidNumberOfArguments\n\t\treturn\n\t}\n\n\tfromFence := t.fence\n\n\tvar slimit string\n\tvar ssparse string\n\tvar scursor string\n\tvar asc bool\n\tfor {\n\t\tnvs, wtok, ok := tokenval(vs)\n\t\tif ok && len(wtok) > 0 {\n\t\t\tswitch strings.ToLower(wtok) {\n\t\t\tcase \"buffer\":\n\t\t\t\tvs = nvs\n\t\t\t\tvar sbuf string\n\t\t\t\tif vs, sbuf, ok = tokenval(vs); !ok || sbuf == \"\" {\n\t\t\t\t\terr = errInvalidNumberOfArguments\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t\tvar buf float64\n\t\t\t\tbuf, err = strconv.ParseFloat(sbuf, 64)\n\t\t\t\tif err != nil || buf < 0 || math.IsInf(buf, 0) || math.IsNaN(buf) {\n\t\t\t\t\terr = errInvalidArgument(sbuf)\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t\tt.buffer = buf\n\t\t\t\tt.hasbuffer = true\n\t\t\t\tcontinue\n\t\t\tcase \"cursor\":\n\t\t\t\tvs = nvs\n\t\t\t\tif scursor != \"\" {\n\t\t\t\t\terr = errDuplicateArgument(strings.ToUpper(wtok))\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t\tif vs, scursor, ok = tokenval(vs); !ok || scursor == \"\" {\n\t\t\t\t\terr = errInvalidNumberOfArguments\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t\tcontinue\n\t\t\tcase \"where\":\n\t\t\t\tvs = nvs\n\t\t\t\tif detectExprToken(vs) {\n\t\t\t\t\t// using expressions\n\t\t\t\t\t// WHERE expr\n\t\t\t\t\tvar expr string\n\t\t\t\t\tif vs, expr, ok = tokenval(vs); !ok {\n\t\t\t\t\t\terr = errInvalidNumberOfArguments\n\t\t\t\t\t\treturn\n\t\t\t\t\t}\n\t\t\t\t\tt.wheres = append(t.wheres, whereT{name: expr, expr: true})\n\t\t\t\t\tcontinue\n\t\t\t\t} else {\n\t\t\t\t\t// using field filter\n\t\t\t\t\t// WHERE min max\n\t\t\t\t\tvar name, smin, smax string\n\t\t\t\t\tif vs, name, ok = tokenval(vs); !ok {\n\t\t\t\t\t\terr = errInvalidNumberOfArguments\n\t\t\t\t\t\treturn\n\t\t\t\t\t}\n\t\t\t\t\tif vs, smin, ok = tokenval(vs); !ok {\n\t\t\t\t\t\terr = errInvalidNumberOfArguments\n\t\t\t\t\t\treturn\n\t\t\t\t\t}\n\t\t\t\t\tif vs, smax, ok = tokenval(vs); !ok {\n\t\t\t\t\t\terr = errInvalidNumberOfArguments\n\t\t\t\t\t\treturn\n\t\t\t\t\t}\n\t\t\t\t\tvar minx, maxx bool\n\t\t\t\t\tsmin = strings.ToLower(smin)\n\t\t\t\t\tsmax = strings.ToLower(smax)\n\t\t\t\t\tif smax == \"+inf\" || smax == \"inf\" {\n\t\t\t\t\t\tsmax = \"inf\"\n\t\t\t\t\t}\n\t\t\t\t\tswitch smin {\n\t\t\t\t\tcase \"<\", \"<=\", \">\", \">=\", \"==\", \"!=\":\n\t\t\t\t\tdefault:\n\t\t\t\t\t\tif strings.HasPrefix(smin, \"(\") {\n\t\t\t\t\t\t\tminx = true\n\t\t\t\t\t\t\tsmin = smin[1:]\n\t\t\t\t\t\t}\n\t\t\t\t\t\tif strings.HasPrefix(smax, \"(\") {\n\t\t\t\t\t\t\tmaxx = true\n\t\t\t\t\t\t\tsmax = smax[1:]\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t\tt.wheres = append(t.wheres, whereT{\n\t\t\t\t\t\tname: name,\n\t\t\t\t\t\tminx: minx,\n\t\t\t\t\t\tmin: field.ValueOf(smin),\n\t\t\t\t\t\tmaxx: maxx,\n\t\t\t\t\t\tmax: field.ValueOf(smax),\n\t\t\t\t\t})\n\t\t\t\t\tcontinue\n\t\t\t\t}\n\t\t\tcase \"wherein\":\n\t\t\t\tvs = nvs\n\t\t\t\tvar name, nvalsStr, valStr string\n\t\t\t\tif vs, name, ok = tokenval(vs); !ok {\n\t\t\t\t\terr = errInvalidNumberOfArguments\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t\tif vs, nvalsStr, ok = tokenval(vs); !ok {\n\t\t\t\t\terr = errInvalidNumberOfArguments\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t\tvar i, nvals uint64\n\t\t\t\tif nvals, err = strconv.ParseUint(nvalsStr, 10, 64); err != nil {\n\t\t\t\t\terr = errInvalidArgument(nvalsStr)\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t\tvalArr := make([]field.Value, nvals)\n\t\t\t\tfor i = 0; i < nvals; i++ {\n\t\t\t\t\tif vs, valStr, ok = tokenval(vs); !ok {\n\t\t\t\t\t\terr = errInvalidNumberOfArguments\n\t\t\t\t\t\treturn\n\t\t\t\t\t}\n\t\t\t\t\tvalArr[i] = field.ValueOf(valStr)\n\t\t\t\t}\n\t\t\t\tt.whereins = append(t.whereins, whereinT{\n\t\t\t\t\tname: name,\n\t\t\t\t\tvalArr: valArr,\n\t\t\t\t})\n\t\t\t\tcontinue\n\t\t\tcase \"whereevalsha\":\n\t\t\t\tfallthrough\n\t\t\tcase \"whereeval\":\n\t\t\t\tscriptIsSha := strings.ToLower(wtok) == \"whereevalsha\"\n\t\t\t\tvs = nvs\n\t\t\t\tvar script, nargsStr, arg string\n\t\t\t\tif vs, script, ok = tokenval(vs); !ok || script == \"\" {\n\t\t\t\t\terr = errInvalidNumberOfArguments\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t\tif vs, nargsStr, ok = tokenval(vs); !ok || nargsStr == \"\" {\n\t\t\t\t\terr = errInvalidNumberOfArguments\n\t\t\t\t\treturn\n\t\t\t\t}\n\n\t\t\t\tvar i, nargs uint64\n\t\t\t\tif nargs, err = strconv.ParseUint(nargsStr, 10, 64); err != nil {\n\t\t\t\t\terr = errInvalidArgument(nargsStr)\n\t\t\t\t\treturn\n\t\t\t\t}\n\n\t\t\t\tvar luaState *lua.LState\n\t\t\t\tluaState, err = s.luapool.Get()\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn\n\t\t\t\t}\n\n\t\t\t\targsTbl := luaState.CreateTable(len(vs), 0)\n\t\t\t\tfor i = 0; i < nargs; i++ {\n\t\t\t\t\tif vs, arg, ok = tokenval(vs); !ok || arg == \"\" {\n\t\t\t\t\t\terr = errInvalidNumberOfArguments\n\t\t\t\t\t\treturn\n\t\t\t\t\t}\n\t\t\t\t\targsTbl.Append(lua.LString(arg))\n\t\t\t\t}\n\n\t\t\t\tvar shaSum string\n\t\t\t\tif scriptIsSha {\n\t\t\t\t\tshaSum = script\n\t\t\t\t} else {\n\t\t\t\t\tshaSum = Sha1Sum(script)\n\t\t\t\t}\n\n\t\t\t\tluaSetRawGlobals(\n\t\t\t\t\tluaState, map[string]lua.LValue{\n\t\t\t\t\t\t\"ARGV\": argsTbl,\n\t\t\t\t\t})\n\n\t\t\t\tcompiled, ok := s.luascripts.Get(shaSum)\n\t\t\t\tvar fn *lua.LFunction\n\t\t\t\tif ok {\n\t\t\t\t\tfn = &lua.LFunction{\n\t\t\t\t\t\tIsG: false,\n\t\t\t\t\t\tEnv: luaState.Env,\n\n\t\t\t\t\t\tProto: compiled,\n\t\t\t\t\t\tGFunction: nil,\n\t\t\t\t\t\tUpvalues: make([]*lua.Upvalue, 0),\n\t\t\t\t\t}\n\t\t\t\t} else if scriptIsSha {\n\t\t\t\t\terr = errShaNotFound\n\t\t\t\t\treturn\n\t\t\t\t} else {\n\t\t\t\t\tfn, err = luaState.Load(strings.NewReader(script), \"f_\"+shaSum)\n\t\t\t\t\tif err != nil {\n\t\t\t\t\t\terr = makeSafeErr(err)\n\t\t\t\t\t\treturn\n\t\t\t\t\t}\n\t\t\t\t\ts.luascripts.PutLRU(shaSum, fn.Proto)\n\t\t\t\t}\n\t\t\t\tt.whereevals = append(t.whereevals, whereevalT{\n\t\t\t\t\tc: s, luaState: luaState, fn: fn,\n\t\t\t\t})\n\t\t\t\tcontinue\n\t\t\tcase \"nofields\":\n\t\t\t\tvs = nvs\n\t\t\t\tif t.nofields {\n\t\t\t\t\terr = errDuplicateArgument(strings.ToUpper(wtok))\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t\tt.nofields = true\n\t\t\t\tcontinue\n\t\t\tcase \"limit\":\n\t\t\t\tvs = nvs\n\t\t\t\tif slimit != \"\" {\n\t\t\t\t\terr = errDuplicateArgument(strings.ToUpper(wtok))\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t\tif vs, slimit, ok = tokenval(vs); !ok || slimit == \"\" {\n\t\t\t\t\terr = errInvalidNumberOfArguments\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t\tcontinue\n\t\t\tcase \"sparse\":\n\t\t\t\tvs = nvs\n\t\t\t\tif ssparse != \"\" {\n\t\t\t\t\terr = errDuplicateArgument(strings.ToUpper(wtok))\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t\tif vs, ssparse, ok = tokenval(vs); !ok || ssparse == \"\" {\n\t\t\t\t\terr = errInvalidNumberOfArguments\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t\tcontinue\n\t\t\tcase \"fence\":\n\t\t\t\tvs = nvs\n\t\t\t\tif t.fence && !fromFence {\n\t\t\t\t\terr = errDuplicateArgument(strings.ToUpper(wtok))\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t\tt.fence = true\n\t\t\t\tcontinue\n\t\t\tcase \"commands\":\n\t\t\t\tvs = nvs\n\t\t\t\tif t.accept != nil {\n\t\t\t\t\terr = errDuplicateArgument(strings.ToUpper(wtok))\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t\tt.accept = make(map[string]bool)\n\t\t\t\tvar peek string\n\t\t\t\tif vs, peek, ok = tokenval(vs); !ok || peek == \"\" {\n\t\t\t\t\terr = errInvalidNumberOfArguments\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t\tfor _, s := range strings.Split(peek, \",\") {\n\t\t\t\t\tpart := strings.TrimSpace(strings.ToLower(s))\n\t\t\t\t\tif t.accept[part] {\n\t\t\t\t\t\terr = errDuplicateArgument(s)\n\t\t\t\t\t\treturn\n\t\t\t\t\t}\n\t\t\t\t\tt.accept[part] = true\n\t\t\t\t}\n\t\t\t\tif len(t.accept) == 0 {\n\t\t\t\t\tt.accept = nil\n\t\t\t\t}\n\t\t\t\tcontinue\n\t\t\tcase \"distance\":\n\t\t\t\tvs = nvs\n\t\t\t\tif t.distance {\n\t\t\t\t\terr = errDuplicateArgument(strings.ToUpper(wtok))\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t\tt.distance = true\n\t\t\t\tcontinue\n\t\t\tcase \"detect\":\n\t\t\t\tvs = nvs\n\t\t\t\tif t.detect != nil {\n\t\t\t\t\terr = errDuplicateArgument(strings.ToUpper(wtok))\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t\tt.detect = make(map[string]bool)\n\t\t\t\tvar peek string\n\t\t\t\tif vs, peek, ok = tokenval(vs); !ok || peek == \"\" {\n\t\t\t\t\terr = errInvalidNumberOfArguments\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t\tfor _, s := range strings.Split(peek, \",\") {\n\t\t\t\t\tpart := strings.TrimSpace(strings.ToLower(s))\n\t\t\t\t\tswitch part {\n\t\t\t\t\tdefault:\n\t\t\t\t\t\terr = errInvalidArgument(peek)\n\t\t\t\t\t\treturn\n\t\t\t\t\tcase \"inside\", \"outside\", \"enter\", \"exit\", \"cross\":\n\t\t\t\t\t}\n\t\t\t\t\tif t.detect[part] {\n\t\t\t\t\t\terr = errDuplicateArgument(s)\n\t\t\t\t\t\treturn\n\t\t\t\t\t}\n\t\t\t\t\tt.detect[part] = true\n\t\t\t\t}\n\t\t\t\tif len(t.detect) == 0 {\n\t\t\t\t\tt.detect = map[string]bool{\n\t\t\t\t\t\t\"inside\": true,\n\t\t\t\t\t\t\"outside\": true,\n\t\t\t\t\t\t\"enter\": true,\n\t\t\t\t\t\t\"exit\": true,\n\t\t\t\t\t\t\"cross\": true,\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t\tcontinue\n\t\t\tcase \"nodwell\":\n\t\t\t\tvs = nvs\n\t\t\t\tif t.desc || asc {\n\t\t\t\t\terr = errDuplicateArgument(strings.ToUpper(wtok))\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t\tt.nodwell = true\n\t\t\t\tcontinue\n\t\t\tcase \"desc\":\n\t\t\t\tvs = nvs\n\t\t\t\tif t.desc || asc {\n\t\t\t\t\terr = errDuplicateArgument(strings.ToUpper(wtok))\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t\tt.desc = true\n\t\t\t\tcontinue\n\t\t\tcase \"asc\":\n\t\t\t\tvs = nvs\n\t\t\t\tif t.desc || asc {\n\t\t\t\t\terr = errDuplicateArgument(strings.ToUpper(wtok))\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t\tasc = true\n\t\t\t\tcontinue\n\t\t\tcase \"match\":\n\t\t\t\tvs = nvs\n\t\t\t\tvar glob string\n\t\t\t\tif vs, glob, ok = tokenval(vs); !ok || glob == \"\" {\n\t\t\t\t\terr = errInvalidNumberOfArguments\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t\tt.globs = append(t.globs, glob)\n\t\t\t\tcontinue\n\t\t\tcase \"clip\":\n\t\t\t\tvs = nvs\n\t\t\t\tif t.clip {\n\t\t\t\t\terr = errDuplicateArgument(strings.ToUpper(wtok))\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t\tt.clip = true\n\t\t\t\tcontinue\n\t\t\t}\n\t\t}\n\t\tbreak\n\t}\n\n\t// check to make sure that there aren't any conflicts\n\tif cmd == \"scan\" || cmd == \"search\" {\n\t\tif ssparse != \"\" {\n\t\t\terr = errors.New(\"SPARSE is not allowed for \" + strings.ToUpper(cmd))\n\t\t\treturn\n\t\t}\n\t\tif t.fence {\n\t\t\terr = errors.New(\"FENCE is not allowed for \" + strings.ToUpper(cmd))\n\t\t\treturn\n\t\t}\n\t} else {\n\t\tif t.desc {\n\t\t\terr = errors.New(\"DESC is not allowed for \" + strings.ToUpper(cmd))\n\t\t\treturn\n\t\t}\n\t\tif asc {\n\t\t\terr = errors.New(\"ASC is not allowed for \" + strings.ToUpper(cmd))\n\t\t\treturn\n\t\t}\n\t}\n\tif ssparse != \"\" && slimit != \"\" {\n\t\terr = errors.New(\"LIMIT is not allowed when SPARSE is specified\")\n\t\treturn\n\t}\n\tif scursor != \"\" && ssparse != \"\" {\n\t\terr = errors.New(\"CURSOR is not allowed when SPARSE is specified\")\n\t\treturn\n\t}\n\tif scursor != \"\" && t.fence {\n\t\terr = errors.New(\"CURSOR is not allowed when FENCE is specified\")\n\t\treturn\n\t}\n\tif t.detect != nil && !t.fence {\n\t\terr = errors.New(\"DETECT is not allowed when FENCE is not specified\")\n\t\treturn\n\t}\n\n\tt.output = defaultSearchOutput\n\tvar nvs []string\n\tvar sprecision string\n\tvar which string\n\tif nvs, which, ok = tokenval(vs); ok && which != \"\" {\n\t\tupdline := true\n\t\tswitch strings.ToLower(which) {\n\t\tdefault:\n\t\t\tif cmd == \"scan\" {\n\t\t\t\terr = errInvalidArgument(which)\n\t\t\t\treturn\n\t\t\t}\n\t\t\tupdline = false\n\t\tcase \"count\":\n\t\t\tt.output = outputCount\n\t\tcase \"objects\":\n\t\t\tt.output = outputObjects\n\t\tcase \"points\":\n\t\t\tt.output = outputPoints\n\t\tcase \"hashes\":\n\t\t\tt.output = outputHashes\n\t\t\tif nvs, sprecision, ok = tokenval(nvs); !ok || sprecision == \"\" {\n\t\t\t\terr = errInvalidNumberOfArguments\n\t\t\t\treturn\n\t\t\t}\n\t\tcase \"bounds\":\n\t\t\tt.output = outputBounds\n\t\tcase \"ids\":\n\t\t\tt.output = outputIDs\n\t\t}\n\t\tif updline {\n\t\t\tvs = nvs\n\t\t}\n\t}\n\tif scursor != \"\" {\n\t\tif t.cursor, err = strconv.ParseUint(scursor, 10, 64); err != nil {\n\t\t\terr = errInvalidArgument(scursor)\n\t\t\treturn\n\t\t}\n\t}\n\tif sprecision != \"\" {\n\t\tt.precision, err = strconv.ParseUint(sprecision, 10, 64)\n\t\tif err != nil || t.precision == 0 || t.precision > 12 {\n\t\t\terr = errInvalidArgument(sprecision)\n\t\t\treturn\n\t\t}\n\t}\n\tif slimit != \"\" {\n\t\tt.ulimit = true\n\t\tif t.limit, err = strconv.ParseUint(slimit, 10, 64); err != nil || t.limit == 0 {\n\t\t\terr = errInvalidArgument(slimit)\n\t\t\treturn\n\t\t}\n\t}\n\tif ssparse != \"\" {\n\t\tt.usparse = true\n\t\tvar sparse uint64\n\t\tif sparse, err = strconv.ParseUint(ssparse, 10, 8); err != nil ||\n\t\t\tsparse == 0 || sparse > 16 {\n\t\t\terr = errInvalidArgument(ssparse)\n\t\t\treturn\n\t\t}\n\t\tt.sparse = uint8(sparse)\n\t\tt.limit = math.MaxUint64\n\t}\n\tvsout = vs\n\ttout = t\n\treturn\n}\n\nfunc detectExprToken(vs []string) bool {\n\t// Detect the kind of where, either:\n\t// - expr\n\t// - name min max\n\tif len(vs) == 0 {\n\t\treturn false\n\t} else if len(vs) == 1 || (len(vs) == 2 && len(vs[1]) == 0) {\n\t\treturn true\n\t}\n\tv := vs[1]\n\tif (v[0] >= 'a' && v[0] <= 'z') || (v[0] >= 'A' && v[0] <= 'Z') {\n\t\tif (v[0] == 'i' || v[0] == 'I') && strings.ToLower(v) == \"inf\" {\n\t\t\treturn false\n\t\t}\n\t\treturn true\n\t}\n\treturn false\n}\n\ntype parentStack []*areaExpression\n\nfunc (ps *parentStack) isEmpty() bool {\n\treturn len(*ps) == 0\n}\n\nfunc (ps *parentStack) push(e *areaExpression) {\n\t*ps = append(*ps, e)\n}\n\nfunc (ps *parentStack) pop() (e *areaExpression, empty bool) {\n\tn := len(*ps)\n\tif n == 0 {\n\t\treturn nil, true\n\t}\n\tx := (*ps)[n-1]\n\t*ps = (*ps)[:n-1]\n\treturn x, false\n}\n\nfunc (s *Server) parseAreaExpression(vsin []string, doClip bool) (vsout []string, ae *areaExpression, err error) {\n\tps := &parentStack{}\n\tvsout = vsin[:]\n\tvar negate, needObj bool\nloop:\n\tfor {\n\t\tnvs, wtok, ok := tokenval(vsout)\n\t\tif !ok || len(wtok) == 0 {\n\t\t\tbreak\n\t\t}\n\t\tswitch strings.ToLower(wtok) {\n\t\tcase tokenLParen:\n\t\t\tnewExpr := &areaExpression{negate: negate, op: NOOP}\n\t\t\tnegate = false\n\t\t\tneedObj = false\n\t\t\tif ae != nil {\n\t\t\t\tae.children = append(ae.children, newExpr)\n\t\t\t}\n\t\t\tae = newExpr\n\t\t\tps.push(ae)\n\t\t\tvsout = nvs\n\t\tcase tokenRParen:\n\t\t\tif needObj {\n\t\t\t\terr = errInvalidArgument(tokenRParen)\n\t\t\t\treturn\n\t\t\t}\n\t\t\tparent, empty := ps.pop()\n\t\t\tif empty {\n\t\t\t\terr = errInvalidArgument(tokenRParen)\n\t\t\t\treturn\n\t\t\t}\n\t\t\tae = parent\n\t\t\tvsout = nvs\n\t\tcase tokenNOT:\n\t\t\tnegate = !negate\n\t\t\tneedObj = true\n\t\t\tvsout = nvs\n\t\tcase tokenAND:\n\t\t\tif needObj {\n\t\t\t\terr = errInvalidArgument(tokenAND)\n\t\t\t\treturn\n\t\t\t}\n\t\t\tneedObj = true\n\t\t\tif ae == nil {\n\t\t\t\terr = errInvalidArgument(tokenAND)\n\t\t\t\treturn\n\t\t\t} else if ae.obj == nil {\n\t\t\t\tswitch ae.op {\n\t\t\t\tcase OR:\n\t\t\t\t\tnumChildren := len(ae.children)\n\t\t\t\t\tif numChildren < 2 {\n\t\t\t\t\t\terr = errInvalidNumberOfArguments\n\t\t\t\t\t\treturn\n\t\t\t\t\t}\n\t\t\t\t\tae.children = append(\n\t\t\t\t\t\tae.children[:numChildren-1],\n\t\t\t\t\t\t&areaExpression{\n\t\t\t\t\t\t\top: AND,\n\t\t\t\t\t\t\tchildren: []*areaExpression{ae.children[numChildren-1]}})\n\t\t\t\tcase NOOP:\n\t\t\t\t\tae.op = AND\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\tae = &areaExpression{op: AND, children: []*areaExpression{ae}}\n\t\t\t}\n\t\t\tvsout = nvs\n\t\tcase tokenOR:\n\t\t\tif needObj {\n\t\t\t\terr = errInvalidArgument(tokenOR)\n\t\t\t\treturn\n\t\t\t}\n\t\t\tneedObj = true\n\t\t\tif ae == nil {\n\t\t\t\terr = errInvalidArgument(tokenOR)\n\t\t\t\treturn\n\t\t\t} else if ae.obj == nil {\n\t\t\t\tswitch ae.op {\n\t\t\t\tcase AND:\n\t\t\t\t\tif len(ae.children) < 2 {\n\t\t\t\t\t\terr = errInvalidNumberOfArguments\n\t\t\t\t\t\treturn\n\t\t\t\t\t}\n\t\t\t\t\tae = &areaExpression{op: OR, children: []*areaExpression{ae}}\n\t\t\t\tcase NOOP:\n\t\t\t\t\tae.op = OR\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\tae = &areaExpression{op: OR, children: []*areaExpression{ae}}\n\t\t\t}\n\t\t\tvsout = nvs\n\t\tcase \"point\", \"circle\", \"object\", \"bounds\", \"hash\", \"quadkey\", \"tile\", \"get\", \"sector\":\n\t\t\tparsedVs, parsedObj, areaErr := s.parseArea(vsout, doClip)\n\t\t\tif areaErr != nil {\n\t\t\t\terr = areaErr\n\t\t\t\treturn\n\t\t\t}\n\t\t\tnewExpr := &areaExpression{negate: negate, obj: parsedObj, op: NOOP}\n\t\t\tnegate = false\n\t\t\tneedObj = false\n\t\t\tif ae == nil {\n\t\t\t\tae = newExpr\n\t\t\t} else {\n\t\t\t\tae.children = append(ae.children, newExpr)\n\t\t\t}\n\t\t\tvsout = parsedVs\n\t\tdefault:\n\t\t\tbreak loop\n\t\t}\n\t}\n\tif !ps.isEmpty() || needObj || ae == nil || (ae.obj == nil && len(ae.children) == 0) {\n\t\terr = errInvalidNumberOfArguments\n\t}\n\treturn\n}\n" }, { "path": "internal/server/token_test.go", "content": "package server\n\nimport (\n\t\"strings\"\n\t\"testing\"\n\n\t\"github.com/tidwall/tile38/internal/field\"\n)\n\nfunc TestLowerCompare(t *testing.T) {\n\tif !lc(\"hello\", \"hello\") {\n\t\tt.Fatal(\"failed\")\n\t}\n\tif !lc(\"Hello\", \"hello\") {\n\t\tt.Fatal(\"failed\")\n\t}\n\tif !lc(\"HeLLo World\", \"hello world\") {\n\t\tt.Fatal(\"failed\")\n\t}\n\tif !lc(\"\", \"\") {\n\t\tt.Fatal(\"failed\")\n\t}\n\tif lc(\"hello\", \"\") {\n\t\tt.Fatal(\"failed\")\n\t}\n\tif lc(\"\", \"hello\") {\n\t\tt.Fatal(\"failed\")\n\t}\n\tif lc(\"HeLLo World\", \"Hello world\") {\n\t\tt.Fatal(\"failed\")\n\t}\n}\n\nfunc TestParseWhereins(t *testing.T) {\n\ts := &Server{}\n\n\ttype tcase struct {\n\t\tinputWhereins []whereinT\n\t\texpWhereins []whereinT\n\t}\n\n\tfn := func(tc tcase) func(t *testing.T) {\n\t\treturn func(t *testing.T) {\n\n\t\t\t_, tout, err := s.parseSearchScanBaseTokens(\n\t\t\t\t\"scan\",\n\t\t\t\tsearchScanBaseTokens{\n\t\t\t\t\twhereins: tc.inputWhereins,\n\t\t\t\t},\n\t\t\t\t[]string{\"key\"},\n\t\t\t)\n\t\t\tgot := tout.whereins\n\t\t\texp := tc.expWhereins\n\n\t\t\tif err != nil {\n\t\t\t\tt.Fatalf(\"unexpected error while parsing search scan base tokens\")\n\t\t\t}\n\n\t\t\tif len(got) != len(exp) {\n\t\t\t\tt.Fatalf(\"expected equal length whereins\")\n\t\t\t}\n\n\t\t\tfor i := range got {\n\t\t\t\tif got[i].name != exp[i].name {\n\t\t\t\t\tt.Fatalf(\"expected equal field names\")\n\t\t\t\t}\n\n\t\t\t\tfor j := range exp[i].valArr {\n\t\t\t\t\tif !got[i].match(exp[i].valArr[j]) {\n\t\t\t\t\t\tt.Fatalf(\"expected matching value arrays\")\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\n\ttests := map[string]tcase{\n\t\t\"upper case\": {\n\t\t\tinputWhereins: []whereinT{\n\t\t\t\t{\n\t\t\t\t\tname: \"TEST\",\n\t\t\t\t\tvalArr: []field.Value{\n\t\t\t\t\t\tfield.ValueOf(\"1\"),\n\t\t\t\t\t\tfield.ValueOf(\"1\"),\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpWhereins: []whereinT{\n\t\t\t\t{\n\t\t\t\t\tname: \"TEST\",\n\t\t\t\t\tvalArr: []field.Value{\n\t\t\t\t\t\tfield.ValueOf(\"1\"),\n\t\t\t\t\t\tfield.ValueOf(\"1\"),\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t\"lower case\": {\n\t\t\tinputWhereins: []whereinT{\n\t\t\t\t{\n\t\t\t\t\tname: \"test\",\n\t\t\t\t\tvalArr: []field.Value{\n\t\t\t\t\t\tfield.ValueOf(\"1\"),\n\t\t\t\t\t\tfield.ValueOf(\"1\"),\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpWhereins: []whereinT{\n\t\t\t\t{\n\t\t\t\t\tname: \"test\",\n\t\t\t\t\tvalArr: []field.Value{\n\t\t\t\t\t\tfield.ValueOf(\"1\"),\n\t\t\t\t\t\tfield.ValueOf(\"1\"),\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t\t\"mixed case\": {\n\t\t\tinputWhereins: []whereinT{\n\t\t\t\t{\n\t\t\t\t\tname: \"teSt\",\n\t\t\t\t\tvalArr: []field.Value{\n\t\t\t\t\t\tfield.ValueOf(\"1\"),\n\t\t\t\t\t\tfield.ValueOf(\"1\"),\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\texpWhereins: []whereinT{\n\t\t\t\t{\n\t\t\t\t\tname: \"teSt\",\n\t\t\t\t\tvalArr: []field.Value{\n\t\t\t\t\t\tfield.ValueOf(\"1\"),\n\t\t\t\t\t\tfield.ValueOf(\"1\"),\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t},\n\t}\n\n\tfor name, tc := range tests {\n\t\tt.Run(name, fn(tc))\n\t}\n\n}\n\n// func testParseFloat(t testing.TB, s string, f float64, invalid bool) {\n// \tn, err := parseFloat(s)\n// \tif err != nil {\n// \t\tif invalid {\n// \t\t\treturn\n// \t\t}\n// \t\tt.Fatal(err)\n// \t}\n// \tif invalid {\n// \t\tt.Fatalf(\"expecting an error for %s\", s)\n// \t}\n// \tif n != f {\n// \t\tt.Fatalf(\"for '%s', expect %f, got %f\", s, f, n)\n// \t}\n// }\n\n// func TestParseFloat(t *testing.T) {\n// \ttestParseFloat(t, \"100\", 100, false)\n// \ttestParseFloat(t, \"0\", 0, false)\n// \ttestParseFloat(t, \"-1\", -1, false)\n// \ttestParseFloat(t, \"-0\", -0, false)\n\n// \ttestParseFloat(t, \"-100\", -100, false)\n// \ttestParseFloat(t, \"-0\", -0, false)\n// \ttestParseFloat(t, \"+1\", 1, false)\n// \ttestParseFloat(t, \"+0\", 0, false)\n\n// \ttestParseFloat(t, \"33.102938\", 33.102938, false)\n// \ttestParseFloat(t, \"-115.123123\", -115.123123, false)\n\n// \ttestParseFloat(t, \".1\", 0.1, false)\n// \ttestParseFloat(t, \"0.1\", 0.1, false)\n\n// \ttestParseFloat(t, \"00.1\", 0.1, false)\n// \ttestParseFloat(t, \"01.1\", 1.1, false)\n// \ttestParseFloat(t, \"01\", 1, false)\n// \ttestParseFloat(t, \"-00.1\", -0.1, false)\n// \ttestParseFloat(t, \"+00.1\", 0.1, false)\n// \ttestParseFloat(t, \"\", 0.1, true)\n// \ttestParseFloat(t, \" 0\", 0.1, true)\n// \ttestParseFloat(t, \"0 \", 0.1, true)\n\n// }\n\nfunc BenchmarkLowerCompare(t *testing.B) {\n\tfor i := 0; i < t.N; i++ {\n\t\tif !lc(\"HeLLo World\", \"hello world\") {\n\t\t\tt.Fatal(\"failed\")\n\t\t}\n\t}\n}\n\nfunc BenchmarkStringsLowerCompare(t *testing.B) {\n\tfor i := 0; i < t.N; i++ {\n\t\tif strings.ToLower(\"HeLLo World\") != \"hello world\" {\n\t\t\tt.Fatal(\"failed\")\n\t\t}\n\n\t}\n}\n\n// func BenchmarkParseFloat(t *testing.B) {\n// \ts := []string{\"33.10293\", \"-115.1203102\"}\n// \tfor i := 0; i < t.N; i++ {\n// \t\t_, err := parseFloat(s[i%2])\n// \t\tif err != nil {\n// \t\t\tt.Fatal(\"failed\")\n// \t\t}\n// \t}\n// }\n\n// func BenchmarkStrconvParseFloat(t *testing.B) {\n// \ts := []string{\"33.10293\", \"-115.1203102\"}\n// \tfor i := 0; i < t.N; i++ {\n// \t\t_, err := strconv.ParseFloat(s[i%2], 64)\n// \t\tif err != nil {\n// \t\t\tt.Fatal(\"failed\")\n// \t\t}\n// \t}\n// }\n" }, { "path": "internal/sstring/sstring.go", "content": "// Package shared allows for\npackage sstring\n\nimport (\n\t\"sync\"\n\t\"unsafe\"\n\n\t\"github.com/tidwall/hashmap\"\n)\n\nvar mu sync.Mutex\nvar nums hashmap.Map[string, int]\nvar strs []string\n\n// Load a shared string from its number.\n// Panics when there is no string assigned with that number.\nfunc Load(num int) (str string) {\n\tmu.Lock()\n\tif num >= 0 && num < len(strs) {\n\t\tstr = strs[num]\n\t\tmu.Unlock()\n\t\treturn str\n\t}\n\tmu.Unlock()\n\tpanic(\"string not found\")\n}\n\n// Store a shared string.\n// Returns a unique number that can be used to load the string later.\n// The number is al\nfunc Store(str string) (num int) {\n\tmu.Lock()\n\tvar ok bool\n\tnum, ok = nums.Get(str)\n\tif !ok {\n\t\t// Make a copy of the string to ensure we don't take in slices.\n\t\tb := make([]byte, len(str))\n\t\tcopy(b, str)\n\t\tstr = *(*string)(unsafe.Pointer(&b))\n\t\tnum = len(strs)\n\t\tstrs = append(strs, str)\n\t\tnums.Set(str, num)\n\t}\n\tmu.Unlock()\n\treturn num\n}\n\n// Len returns the number of shared strings\nfunc Len() int {\n\tmu.Lock()\n\tn := len(strs)\n\tmu.Unlock()\n\treturn n\n}\n" }, { "path": "internal/sstring/sstring_test.go", "content": "package sstring\n\nimport (\n\t\"math/rand\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/tidwall/assert\"\n)\n\nfunc TestShared(t *testing.T) {\n\tfor i := -1; i < 10; i++ {\n\t\tvar str string\n\t\tfunc() {\n\t\t\tdefer func() {\n\t\t\t\tassert.Assert(recover().(string) == \"string not found\")\n\t\t\t}()\n\t\t\tstr = Load(i)\n\t\t}()\n\t\tassert.Assert(str == \"\")\n\t}\n\tassert.Assert(Store(\"hello\") == 0)\n\tassert.Assert(Store(\"\") == 1)\n\tassert.Assert(Store(\"jello\") == 2)\n\tassert.Assert(Store(\"hello\") == 0)\n\tassert.Assert(Store(\"\") == 1)\n\tassert.Assert(Store(\"jello\") == 2)\n\tstr := Load(0)\n\tassert.Assert(str == \"hello\")\n\tstr = Load(1)\n\tassert.Assert(str == \"\")\n\tstr = Load(2)\n\tassert.Assert(str == \"jello\")\n\n\tassert.Assert(Len() == 3)\n\n}\n\nfunc randStr(n int) string {\n\tb := make([]byte, n)\n\trand.Read(b)\n\tfor i := 0; i < n; i++ {\n\t\tb[i] = 'a' + b[i]%26\n\t}\n\treturn string(b)\n}\n\nfunc BenchmarkStore(b *testing.B) {\n\trand.Seed(time.Now().UnixNano())\n\twmap := make(map[string]bool, b.N)\n\tfor len(wmap) < b.N {\n\t\twmap[randStr(10)] = true\n\t}\n\twords := make([]string, 0, b.N)\n\tfor word := range wmap {\n\t\twords = append(words, word)\n\t}\n\tb.ResetTimer()\n\tfor i := 0; i < b.N; i++ {\n\t\tStore(words[i])\n\t}\n}\n\nfunc BenchmarkLoad(b *testing.B) {\n\trand.Seed(time.Now().UnixNano())\n\twmap := make(map[string]bool, b.N)\n\tfor len(wmap) < b.N {\n\t\twmap[randStr(10)] = true\n\t}\n\twords := make([]string, 0, b.N)\n\tfor word := range wmap {\n\t\twords = append(words, word)\n\t}\n\tvar nums []int\n\tfor i := 0; i < b.N; i++ {\n\t\tnums = append(nums, Store(words[i]))\n\t}\n\trand.Shuffle(len(nums), func(i, j int) {\n\t\tnums[i], nums[j] = nums[j], nums[i]\n\t})\n\tb.ResetTimer()\n\tfor i := 0; i < b.N; i++ {\n\t\tLoad(nums[i])\n\t}\n}\n" }, { "path": "internal/viewer/index.html", "content": "\n\n\n\nTile38 Map Viewer\n\n\n\n\n\n\n\n\n\n
\n\nChoose Collection:\n\n
\n
\n
\n\n\n\n\n\n" }, { "path": "internal/viewer/viewer.go", "content": "package viewer\n\nimport (\n\t\"bytes\"\n\t\"embed\"\n\t\"fmt\"\n\t\"io\"\n\t\"mime\"\n\t\"net/http\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"strings\"\n)\n\n//go:embed *\nvar files embed.FS\n\nfunc HandleHTTP(wr io.Writer, url string, devMode bool) error {\n\tif (!strings.HasPrefix(url, \"/viewer/\") && url != \"/viewer\") ||\n\t\tstrings.Contains(url, \"..\") {\n\t\treturn writeHTTPResponse(wr, \"404 Not Found\", \"text/html\",\n\t\t\tnil, []byte(\"

404 Not Found

\\n\"))\n\t}\n\tif strings.HasSuffix(url, \"/\") {\n\t\treturn writeHTTPResponse(wr, \"307 Redirect\", \"text/html\",\n\t\t\t[]string{\"Location\", url[:len(url)-1]},\n\t\t\t[]byte(\"

307 Redirect

\\n\"))\n\t}\n\n\tif url == \"/viewer\" {\n\t\treturn writeHTTPFile(wr, \"/viewer/index.html\", devMode)\n\t}\n\treturn writeHTTPFile(wr, url, devMode)\n}\n\nfunc writeHTTPFile(wr io.Writer, path string, devMode bool) error {\n\tvar data []byte\n\terr := os.ErrNotExist\n\tif devMode {\n\t\tdata, err = os.ReadFile(\"internal\" + path)\n\t}\n\tpath = path[8:]\n\tif os.IsNotExist(err) {\n\t\tdata, err = files.ReadFile(path)\n\t}\n\tif os.IsNotExist(err) {\n\t\treturn writeHTTPResponse(wr, \"404 Not Found\", \"text/html\",\n\t\t\tnil, []byte(\"

404 Not Found

\\n\"))\n\t}\n\treturn writeHTTPResponse(wr, \"200 OK\",\n\t\tmime.TypeByExtension(filepath.Ext(path)), nil, data)\n}\n\nfunc writeHTTPResponse(wr io.Writer, status, contentType string,\n\theaders []string, body []byte,\n) error {\n\tvar sheaders string\n\tif len(headers) > 0 {\n\t\thdrs := http.Header{}\n\t\tfor i := 0; i < len(headers)-1; i += 2 {\n\t\t\thdrs.Set(headers[i], headers[i+1])\n\t\t}\n\t\tvar buf bytes.Buffer\n\t\thdrs.Write(&buf)\n\t\tsheaders = buf.String()\n\t}\n\tpayload := append([]byte(nil), fmt.Sprintf(\"\"+\n\t\t\"HTTP/1.1 %s\\r\\n\"+\n\t\t\"Connection: close\\r\\n\"+\n\t\t\"Content-Type: %s\\r\\n\"+\n\t\t\"Content-Length: %d\\r\\n\"+\n\t\t\"Access-Control-Allow-Origin: *\\r\\n\"+\n\t\tsheaders+\n\t\t\"\\r\\n\", status, contentType, len(body))...)\n\tpayload = append(payload, body...)\n\t_, err := wr.Write(payload)\n\treturn err\n}\n" }, { "path": "scripts/RELEASE.md", "content": "**To bump a new release of Tile38**\n\n- Update CHANGELOG.md to include the newest changes.\n- `git commit -m $vers` changes (where `$vers` is a semver)\n- `git tag $vers` (where `$vers` is a semver)\n- `git push --tags`\n- `git push` \n- `make package`\n- Add a new Github Release and add the zips from packages directory.\n " }, { "path": "scripts/build.sh", "content": "#!/bin/bash\n\nset -e\ncd $(dirname \"${BASH_SOURCE[0]}\")/..\n\nif [ \"$1\" == \"\" ]; then\n\techo \"error: missing argument (binary name)\"\n\texit 1\nfi\n\n# Check the Go installation\nif [ \"$(which go)\" == \"\" ]; then\n\techo \"error: Go is not installed. Please download and follow installation\"\\\n\t\t \"instructions at https://golang.org/dl to continue.\"\n\texit 1\nfi\n\n# Hardcode some values to the core package.\nif [ -d \".git\" ]; then\n\tVERSION=$(git describe --tags --abbrev=0)\n\tGITSHA=$(git rev-parse --short HEAD)\n\tLDFLAGS=\"$LDFLAGS -X github.com/tidwall/tile38/core.Version=${VERSION}\"\n\tLDFLAGS=\"$LDFLAGS -X github.com/tidwall/tile38/core.GitSHA=${GITSHA}\"\nfi\nLDFLAGS=\"$LDFLAGS -X github.com/tidwall/tile38/core.BuildTime=$(date +%FT%T%z)\"\n\n# Generate the core package\ncore/gen.sh\n\n# Set final Go environment options\nLDFLAGS=\"$LDFLAGS -extldflags '-static'\"\nexport CGO_ENABLED=0\n\nif [[ \"$GORACE\" == \"1\" ]]; then\n\texport CGO_ENABLED=1\n\tgoflags=\"$goflags -race\"\nfi\n\n# Build and store objects into original directory.\ngo build -ldflags \"$LDFLAGS\" $goflags -o $1 cmd/$1/*.go\n" }, { "path": "scripts/docker-push.sh", "content": "#!/bin/bash\n\nset -e\ncd $(dirname \"${BASH_SOURCE[0]}\")/..\n\n# GIT_BRANCH is the current branch name\nexport GIT_BRANCH=$(git branch --show-current)\n# GIT_VERSION - always the last verison number, like 1.12.1.\nexport GIT_VERSION=$(git describe --tags --abbrev=0)\n# GIT_COMMIT_SHORT - the short git commit number, like a718ef0.\nexport GIT_COMMIT_SHORT=$(git rev-parse --short HEAD)\n# DOCKER_REPO - the base repository name to push the docker build to.\nexport DOCKER_REPO=$DOCKER_USER/tile38\n\nif [ \"$GIT_BRANCH\" != \"master\" ]; then\n\techo \"Not pushing, not on master\"\nelif [ \"$DOCKER_USER\" == \"\" ]; then\n\techo \"Not pushing, DOCKER_USER not set\"\n\texit 1\nelif [ \"$DOCKER_LOGIN\" == \"\" ]; then\n\techo \"Not pushing, DOCKER_LOGIN not set\"\n\texit 1\nelif [ \"$DOCKER_PASSWORD\" == \"\" ]; then\n\techo \"Not pushing, DOCKER_PASSWORD not set\"\n\texit 1\nelse\n\t# setup cross platform builder\n\t# https://github.com/tonistiigi/binfmt\n\tdocker run --privileged --rm tonistiigi/binfmt --install all\n\tdocker buildx create --name multiarch --platform linux/amd64,linux/amd64/v2,linux/amd64/v3,linux/arm64,linux/386,linux/arm/v7 --use default\n\n\t# docker login\n\techo $DOCKER_PASSWORD | docker login -u $DOCKER_LOGIN --password-stdin\n\tif [ \"$(curl -s https://hub.docker.com/v2/repositories/$DOCKER_REPO/tags/$GIT_VERSION/ | grep \"digest\")\" == \"\" ]; then\n\t\t# build the docker image\n\t\tdocker buildx build \\\n\t\t\t-f Dockerfile \\\n\t\t\t--platform linux/arm64,linux/amd64 \\\n\t\t\t--build-arg VERSION=$GIT_VERSION \\\n\t\t\t--tag $DOCKER_REPO:$GIT_VERSION \\\n\t\t\t--tag $DOCKER_REPO:latest \\\n\t\t\t--tag $DOCKER_REPO:edge \\\n\t\t\t--push \\\n\t\t\t.\n\telse\n\t\t# build the docker image\n\t\tdocker buildx build \\\n\t\t\t-f Dockerfile \\\n\t\t\t--platform linux/arm64,linux/amd64 \\\n\t\t\t--build-arg VERSION=$GIT_VERSION \\\n\t\t\t--tag $DOCKER_REPO:edge \\\n\t\t\t--push \\\n\t\t\t.\n\tfi\nfi\n" }, { "path": "scripts/package.sh", "content": "#!/bin/bash\n\nset -e\ncd $(dirname \"${BASH_SOURCE[0]}\")/..\n\nPLATFORM=\"$1\"\nGOOS=\"$2\"\nGOARCH=\"$3\"\nVERSION=$(git describe --tags --abbrev=0)\n\necho Packaging $PLATFORM Binary\n\n# Remove previous build directory, if needed.\nbdir=tile38-$VERSION-$GOOS-$GOARCH\nrm -rf packages/$bdir && mkdir -p packages/$bdir\n\n# Make the binaries.\nGOOS=$GOOS GOARCH=$GOARCH make all\nrm -f tile38-luamemtest # not needed\n\n# Copy the executable binaries.\nif [ \"$GOOS\" == \"windows\" ]; then\n\tmv tile38-server packages/$bdir/tile38-server.exe\n\tmv tile38-cli packages/$bdir/tile38-cli.exe\n\tmv tile38-benchmark packages/$bdir/tile38-benchmark.exe\nelse\n\tmv tile38-server packages/$bdir\n\tmv tile38-cli packages/$bdir\n\tmv tile38-benchmark packages/$bdir\nfi\n\n# Copy documention and license.\ncp README.md packages/$bdir\ncp CHANGELOG.md packages/$bdir\ncp LICENSE packages/$bdir\n\n# Compress the package.\ncd packages\nif [ \"$GOOS\" == \"linux\" ]; then\n\ttar -zcf $bdir.tar.gz $bdir\nelse\n\tzip -r -q $bdir.zip $bdir\nfi\n\n" }, { "path": "scripts/test.sh", "content": "#!/bin/bash\n\nset -e\ncd $(dirname \"${BASH_SOURCE[0]}\")/..\n\nexport CGO_ENABLED=0\n\ncd tests\ngo test -coverpkg=../internal/server -coverprofile=/tmp/coverage.out $GOTEST\n\n\n# go test -coverpkg=../internal/... -coverprofile=/tmp/coverage.out \\\n# -v ./... $GOTEST\n\ngo tool cover -html=/tmp/coverage.out -o /tmp/coverage.html\necho \"details: file:///tmp/coverage.html\"\ncd ..\n\nif [[ \"$GOTEST\" == \"\" ]]; then\n go test $(go list ./... | grep -v /vendor/ | grep -v /tests)\nfi\n" }, { "path": "tests/107/.gitignore", "content": "appendonly.aof\nlog\ndata/\n" }, { "path": "tests/107/LINK", "content": "https://github.com/tidwall/tile38/issues/107\n" }, { "path": "tests/107/main.go", "content": "package main\n\nimport (\n\t\"archive/zip\"\n\t\"bytes\"\n\t\"flag\"\n\t\"fmt\"\n\t\"io\"\n\t\"io/ioutil\"\n\t\"log\"\n\t\"math/rand\"\n\t\"net/http\"\n\t\"os\"\n\t\"os/exec\"\n\t\"path\"\n\t\"strconv\"\n\t\"strings\"\n\t\"sync/atomic\"\n\t\"time\"\n\n\t\"github.com/gomodule/redigo/redis\"\n\t\"github.com/tidwall/gjson\"\n\t\"github.com/tidwall/tile38/internal/server\"\n)\n\nconst tile38Port = 9191\nconst httpPort = 9292\nconst dir = \"data\"\n\nvar tile38Addr string\nvar httpAddr string\n\nvar wd string\n\nvar minX float64\nvar minY float64\nvar maxX float64\nvar maxY float64\nvar pool = &redis.Pool{\n\tMaxIdle: 3,\n\tIdleTimeout: 240 * time.Second,\n\tDial: func() (redis.Conn, error) {\n\t\treturn redis.Dial(\"tcp\", tile38Addr)\n\t},\n}\nvar providedTile38 bool\nvar providedHTTP bool\n\nconst blank = false\nconst hookServer = true\n\nvar logf *os.File\n\nfunc main() {\n\tflag.StringVar(&tile38Addr, \"tile38\", \"\",\n\t\t\"Tile38 address, leave blank to start a new server\")\n\tflag.StringVar(&httpAddr, \"hook\", \"\",\n\t\t\"Hook HTTP url, leave blank to start a new server\")\n\tflag.Parse()\n\tlog.Println(\"mockfill-107 (Github #107: Memory leak)\")\n\n\tif tile38Addr == \"\" {\n\t\ttile38Addr = \"127.0.0.1:\" + strconv.FormatInt(int64(tile38Port), 10)\n\t} else {\n\t\tprovidedTile38 = true\n\t}\n\tif httpAddr == \"\" {\n\t\thttpAddr = \"http://127.0.0.1:\" + strconv.FormatInt(int64(httpPort), 10) + \"/hook\"\n\t} else {\n\t\tprovidedHTTP = true\n\t}\n\tvar err error\n\twd, err = os.Getwd()\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tlogf, err = os.Create(\"log\")\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tdefer logf.Close()\n\tif !providedTile38 {\n\t\tcopyAOF()\n\t\tgo startTile38Server()\n\t}\n\tif !providedHTTP {\n\t\tif hookServer {\n\t\t\tgo startHookServer()\n\t\t}\n\t}\n\tgo waitForServers(func() {\n\t\tlog.Printf(\"servers ready\")\n\t\tlogServer(\"START\")\n\t\tsetPoints()\n\t\tlogServer(\"DONE\")\n\t})\n\tselect {}\n\treturn\n}\n\nfunc startTile38Server() {\n\tlog.Println(\"start tile38 server\")\n\topts := server.Options{\n\t\tHost: \"localhost\",\n\t\tPort: tile38Port,\n\t\tDir: \"data\",\n\t\tUseHTTP: false,\n\t\tMetricsAddr: \"\",\n\t}\n\terr := server.Serve(opts)\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n}\n\nfunc startHookServer() {\n\tlog.Println(\"start hook server\")\n\thttp.HandleFunc(\"/ping\", func(w http.ResponseWriter, _ *http.Request) {\n\t\tio.WriteString(w, \"pong\")\n\t})\n\thttp.HandleFunc(\"/hook\", func(w http.ResponseWriter, req *http.Request) {\n\t\tdata, err := ioutil.ReadAll(req.Body)\n\t\tif err != nil {\n\t\t\tlog.Fatal(err)\n\t\t}\n\t\tlog.Println(string(data))\n\t})\n\terr := http.ListenAndServe(fmt.Sprintf(\"127.0.0.1:%d\", httpPort), nil)\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n}\n\nfunc waitForServers(cb func()) {\n\tlog.Println(\"wait for servers\")\n\tvar err error\n\tstart := time.Now()\n\tfor {\n\t\tif time.Since(start) > time.Second*5 {\n\t\t\tlog.Fatal(\"connection failed:\", err)\n\t\t}\n\t\tfunc() {\n\t\t\tconn := pool.Get()\n\t\t\tdefer conn.Close()\n\t\t\tvar s string\n\t\t\ts, err = redis.String(conn.Do(\"PING\"))\n\t\t\tif err != nil {\n\t\t\t\treturn\n\t\t\t}\n\t\t\tif s != \"PONG\" {\n\t\t\t\tlog.Fatalf(\"expected '%v', got '%v'\", \"PONG\", s)\n\t\t\t}\n\t\t}()\n\t\tif err == nil {\n\t\t\tbreak\n\t\t}\n\t\ttime.Sleep(time.Second / 5)\n\t}\n\tif hookServer {\n\t\tstart = time.Now()\n\t\tfor {\n\t\t\tif time.Since(start) > time.Second*5 {\n\t\t\t\tlog.Fatal(\"connection failed:\", err)\n\t\t\t}\n\t\t\tfunc() {\n\t\t\t\tvar resp *http.Response\n\t\t\t\tresp, err = http.Get(httpAddr + \"/notreal\")\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t\tdefer resp.Body.Close()\n\t\t\t\tif resp.StatusCode != 200 && resp.StatusCode != 404 {\n\t\t\t\t\tlog.Fatalf(\"expected '%v', got '%v'\", \"200 or 404\",\n\t\t\t\t\t\tresp.StatusCode)\n\t\t\t\t}\n\t\t\t}()\n\t\t\tif err == nil {\n\t\t\t\tbreak\n\t\t\t}\n\t\t\ttime.Sleep(time.Second / 5)\n\t\t}\n\t}\n\tcb()\n}\n\nfunc downloadAOF() {\n\tlog.Println(\"downloading aof\")\n\tresp, err := http.Get(\"https://github.com/tidwall/tile38/files/675225/appendonly.aof.zip\")\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tdefer resp.Body.Close()\n\n\tbody, err := ioutil.ReadAll(resp.Body)\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\trd, err := zip.NewReader(bytes.NewReader(body), int64(len(body)))\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tfor _, f := range rd.File {\n\t\tif path.Ext(f.Name) == \".aof\" {\n\t\t\trc, err := f.Open()\n\t\t\tif err != nil {\n\t\t\t\tlog.Fatal(err)\n\t\t\t}\n\t\t\tdefer rc.Close()\n\n\t\t\tdata, err := ioutil.ReadAll(rc)\n\t\t\tif err != nil {\n\t\t\t\tlog.Fatal(err)\n\t\t\t}\n\t\t\terr = ioutil.WriteFile(path.Join(wd, \"appendonly.aof\"), data, 0666)\n\t\t\tif err != nil {\n\t\t\t\tlog.Fatal(err)\n\t\t\t}\n\t\t\treturn\n\t\t}\n\t}\n\tlog.Fatal(\"invalid appendonly.aof.zip\")\n}\n\nfunc copyAOF() {\n\tif err := os.RemoveAll(path.Join(wd, \"data\")); err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tif err := os.MkdirAll(path.Join(wd, \"data\"), 0777); err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tfin, err := os.Open(path.Join(wd, \"appendonly.aof\"))\n\tif err != nil {\n\t\tif os.IsNotExist(err) {\n\t\t\tdownloadAOF()\n\t\t\tfin, err = os.Open(path.Join(wd, \"appendonly.aof\"))\n\t\t\tif err != nil {\n\t\t\t\tlog.Fatal(err)\n\t\t\t}\n\t\t} else {\n\t\t\tlog.Fatal(err)\n\t\t}\n\t}\n\tdefer fin.Close()\n\n\tlog.Println(\"load aof\")\n\tfout, err := os.Create(path.Join(wd, \"data\", \"appendonly.aof\"))\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tdefer fout.Close()\n\tdata, err := ioutil.ReadAll(fin)\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\trep := httpAddr\n\trep = \"$\" + strconv.FormatInt(int64(len(rep)), 10) + \"\\r\\n\" + rep + \"\\r\\n\"\n\tdata = bytes.Replace(data,\n\t\t[]byte(\"$23\\r\\nhttp://172.17.0.1:9999/\\r\\n\"), []byte(rep), -1)\n\tif blank {\n\t\tdata = nil\n\t}\n\tif _, err := fout.Write(data); err != nil {\n\t\tlog.Fatal(err)\n\t}\n}\n\nfunc respGet(resp interface{}, idx ...int) interface{} {\n\tfor i := 0; i < len(idx); i++ {\n\t\tarr, _ := redis.Values(resp, nil)\n\t\tresp = arr[idx[i]]\n\t}\n\treturn resp\n}\n\ntype PSAUX struct {\n\tUser string\n\tPID int\n\tCPU float64\n\tMem float64\n\tVSZ int\n\tRSS int\n\tTTY string\n\tStat string\n\tStart string\n\tTime string\n\tCommand string\n}\n\nfunc atoi(s string) int {\n\tn, _ := strconv.ParseInt(s, 10, 64)\n\treturn int(n)\n}\nfunc atof(s string) float64 {\n\tn, _ := strconv.ParseFloat(s, 64)\n\treturn float64(n)\n}\nfunc psaux(pid int) PSAUX {\n\tvar res []byte\n\tres, err := exec.Command(\"ps\", \"ux\", \"-p\", strconv.FormatInt(int64(pid), 10)).CombinedOutput()\n\tif err != nil {\n\t\treturn PSAUX{}\n\t}\n\tpids := strconv.FormatInt(int64(pid), 10)\n\tfor _, line := range strings.Split(string(res), \"\\n\") {\n\t\tvar words []string\n\t\tfor _, word := range strings.Split(line, \" \") {\n\t\t\tif word != \"\" {\n\t\t\t\twords = append(words, word)\n\t\t\t}\n\t\t}\n\t\tif len(words) >= 11 {\n\t\t\tif words[1] == pids {\n\t\t\t\treturn PSAUX{\n\t\t\t\t\tUser: words[0],\n\t\t\t\t\tPID: atoi(words[1]),\n\t\t\t\t\tCPU: atof(words[2]),\n\t\t\t\t\tMem: atof(words[3]),\n\t\t\t\t\tVSZ: atoi(words[4]),\n\t\t\t\t\tRSS: atoi(words[5]),\n\t\t\t\t\tTTY: words[6],\n\t\t\t\t\tStat: words[7],\n\t\t\t\t\tStart: words[8],\n\t\t\t\t\tTime: words[9],\n\t\t\t\t\tCommand: words[10],\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\treturn PSAUX{}\n}\nfunc respGetFloat(resp interface{}, idx ...int) float64 {\n\tresp = respGet(resp, idx...)\n\tf, _ := redis.Float64(resp, nil)\n\treturn f\n}\nfunc logServer(tag string) {\n\tconn := pool.Get()\n\tdefer conn.Close()\n\t_, err := conn.Do(\"OUTPUT\", \"json\")\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\t_, err = redis.String(conn.Do(\"GC\"))\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tjson, err := redis.String(conn.Do(\"SERVER\"))\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\t_, err = conn.Do(\"OUTPUT\", \"resp\")\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\trss := float64(psaux(int(gjson.Get(json, \"stats.pid\").Int())).RSS) / 1024\n\theapSize := gjson.Get(json, \"stats.heap_size\").Float() / 1024 / 1024\n\theapReleased := gjson.Get(json, \"stats.heap_released\").Float() / 1024 / 1024\n\tfmt.Fprintf(logf, \"%s %10.2f MB (heap) %10.2f MB (released) %10.2f MB (system)\\n\",\n\t\ttime.Now().Format(\"2006-01-02T15:04:05Z07:00\"),\n\t\theapSize, heapReleased, rss)\n}\nfunc setPoints() {\n\tgo func() {\n\t\tvar i int\n\t\tfor range time.NewTicker(time.Second * 1).C {\n\t\t\tlogServer(fmt.Sprintf(\"SECOND-%d\", i*1))\n\t\t\ti++\n\t\t}\n\t}()\n\n\trand.Seed(time.Now().UnixNano())\n\tn := 1000000\n\tex := time.Second * 10\n\tlog.Printf(\"time to pump data (%d points, expires %s)\", n, ex)\n\tconn := pool.Get()\n\tdefer conn.Close()\n\tif blank {\n\t\tminX = -124.40959167480469\n\t\tminY = 32.53415298461914\n\t\tmaxX = -114.13121032714844\n\t\tmaxY = 42.009521484375\n\t} else {\n\t\tresp, err := conn.Do(\"bounds\", \"boundies\")\n\t\tif err != nil {\n\t\t\tlog.Fatal(err)\n\t\t}\n\t\tminX = respGetFloat(resp, 0, 0)\n\t\tminY = respGetFloat(resp, 0, 1)\n\t\tmaxX = respGetFloat(resp, 1, 0)\n\t\tmaxY = respGetFloat(resp, 1, 1)\n\t}\n\tlog.Printf(\"bbox: [[%.4f,%.4f],[%.4f,%.4f]]\\n\", minX, minY, maxX, maxY)\n\tvar idx uint64\n\tfor i := 0; i < 4; i++ {\n\t\tgo func() {\n\t\t\tconn := pool.Get()\n\t\t\tdefer conn.Close()\n\t\t\tfor i := 0; i < n; i++ {\n\t\t\t\tatomic.AddUint64(&idx, 1)\n\t\t\t\tid := fmt.Sprintf(\"person:%d\", idx)\n\t\t\t\tx := rand.Float64()*(maxX-minX) + minX\n\t\t\t\ty := rand.Float64()*(maxY-minY) + minY\n\t\t\t\tok, err := redis.String(conn.Do(\"SET\", \"people\", id,\n\t\t\t\t\t\"EX\", float64(ex/time.Second),\n\t\t\t\t\t\"POINT\", y, x))\n\t\t\t\tif err != nil {\n\t\t\t\t\tlog.Fatal(err)\n\t\t\t\t}\n\t\t\t\tif ok != \"OK\" {\n\t\t\t\t\tlog.Fatalf(\"expected 'OK', got '%v\", ok)\n\t\t\t\t}\n\t\t\t\tlog.Printf(\"SET people %v EX %v POINT %v %v\",\n\t\t\t\t\tid, float64(ex/time.Second), y, x)\n\t\t\t}\n\t\t}()\n\t}\n\tselect {}\n}\n" }, { "path": "tests/616/main.go", "content": "// Test Tile38 for Expiration Drift\n// Issue #616\n\npackage main\n\nimport (\n\t\"fmt\"\n\t\"math/rand\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/gomodule/redigo/redis\"\n\t\"github.com/tidwall/btree\"\n\t\"github.com/tidwall/gjson\"\n\t\"github.com/tidwall/sjson\"\n)\n\nconst exsecs = 10\nconst key = \"__issue_616__\"\n\nfunc makeID() string {\n\tconst chars = \"0123456789abcdefghijklmnopqrstuvwxyz-\"\n\tvar buf [10]byte\n\trand.Read(buf[:])\n\tfor i := 0; i < len(buf); i++ {\n\t\tbuf[i] = chars[int(buf[i])%len(chars)]\n\t}\n\treturn string(buf[:])\n}\n\nfunc main() {\n\tfmt.Printf(\n\t\t\"The SCAN and ACTUAL values should reach about 1850 and stay\\n\" +\n\t\t\t\"roughly the same from there on.\\n\")\n\tvar mu sync.Mutex\n\tobjs := btree.NewNonConcurrent(func(a, b interface{}) bool {\n\t\tajson := a.(string)\n\t\tbjson := b.(string)\n\t\treturn gjson.Get(ajson, \"id\").String() < gjson.Get(bjson, \"id\").String()\n\t})\n\texpires := btree.NewNonConcurrent(func(a, b interface{}) bool {\n\t\tajson := a.(string)\n\t\tbjson := b.(string)\n\t\tif gjson.Get(ajson, \"properties.ex\").Int() < gjson.Get(bjson, \"properties.ex\").Int() {\n\t\t\treturn true\n\t\t}\n\t\tif gjson.Get(ajson, \"properties.ex\").Int() > gjson.Get(bjson, \"properties.ex\").Int() {\n\t\t\treturn false\n\t\t}\n\t\treturn gjson.Get(ajson, \"id\").String() < gjson.Get(bjson, \"id\").String()\n\t})\n\n\tconn := must(redis.Dial(\"tcp\", \":9851\")).(redis.Conn)\n\tmust(conn.Do(\"DROP\", key))\n\tmust(nil, conn.Close())\n\n\tgo func() {\n\t\tconn := must(redis.Dial(\"tcp\", \":9851\")).(redis.Conn)\n\t\tdefer conn.Close()\n\t\tfor {\n\t\t\tex := time.Now().UnixNano() + int64(exsecs*time.Second)\n\t\t\tfor i := 0; i < 10; i++ {\n\t\t\t\tid := makeID()\n\t\t\t\tx := rand.Float64()*360 - 180\n\t\t\t\ty := rand.Float64()*180 - 90\n\t\t\t\tobj := fmt.Sprintf(`{\"type\":\"Feature\",\"geometry\":{\"type\":\"Point\",\"coordinates\":[%f,%f]},\"properties\":{}}`, x, y)\n\t\t\t\tobj, _ = sjson.Set(obj, \"properties.ex\", ex)\n\t\t\t\tobj, _ = sjson.Set(obj, \"id\", id)\n\t\t\t\tres := must(redis.String(conn.Do(\"SET\", key, id, \"ex\", exsecs, \"OBJECT\", obj))).(string)\n\t\t\t\tif res != \"OK\" {\n\t\t\t\t\tpanic(fmt.Sprintf(\"expected 'OK', got '%s'\", res))\n\t\t\t\t}\n\t\t\t\tmu.Lock()\n\t\t\t\tprev := objs.Set(obj)\n\t\t\t\tif prev != nil {\n\t\t\t\t\texpires.Delete(obj)\n\t\t\t\t}\n\t\t\t\texpires.Set(obj)\n\t\t\t\tmu.Unlock()\n\t\t\t}\n\t\t\ttime.Sleep(time.Second / 20)\n\t\t}\n\t}()\n\n\tgo func() {\n\t\tconn := must(redis.Dial(\"tcp\", \":9851\")).(redis.Conn)\n\t\tdefer conn.Close()\n\t\tfor {\n\t\t\ttime.Sleep(time.Second * 5)\n\t\t\tmust(conn.Do(\"AOFSHRINK\"))\n\t\t}\n\t}()\n\n\tgo func() {\n\t\tconn := must(redis.Dial(\"tcp\", \":9851\")).(redis.Conn)\n\t\tdefer conn.Close()\n\t\tmust(conn.Do(\"OUTPUT\", \"JSON\"))\n\t\tfor {\n\t\t\ttime.Sleep(time.Second / 10)\n\t\t\tvar ids []string\n\t\t\tres := must(redis.String(conn.Do(\"SCAN\", key, \"LIMIT\", 100000000))).(string)\n\t\t\tgjson.Get(res, \"objects\").ForEach(func(_, res gjson.Result) bool {\n\t\t\t\tids = append(ids, res.Get(\"id\").String())\n\t\t\t\treturn true\n\t\t\t})\n\t\t\tnow := time.Now().UnixNano()\n\t\t\tmu.Lock()\n\t\t\tvar exobjs []string\n\t\t\texpires.Ascend(nil, func(v interface{}) bool {\n\t\t\t\tex := gjson.Get(v.(string), \"properties.ex\").Int()\n\t\t\t\tif ex > now {\n\t\t\t\t\treturn false\n\t\t\t\t}\n\t\t\t\texobjs = append(exobjs, v.(string))\n\t\t\t\treturn true\n\t\t\t})\n\t\t\tfor _, obj := range exobjs {\n\t\t\t\tobjs.Delete(obj)\n\t\t\t\texpires.Delete(obj)\n\t\t\t}\n\t\t\tfmt.Printf(\"\\rSCAN: %d, ACTUAL: %d \", len(ids), objs.Len())\n\t\t\tmu.Unlock()\n\t\t}\n\t}()\n\tselect {}\n}\n\nfunc must(v interface{}, err error) interface{} {\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\treturn v\n}\n" }, { "path": "tests/README.md", "content": "## Tile38 Integration Testing\n\n- Uses Redis protocol\n- The Tile38 data is flushed before every `DoBatch`\n\nA basic test operation looks something like:\n\n```go\nfunc keys_SET_test(mc *mockServer) error {\n\treturn mc.DoBatch([][]interface{}{\n {\"SET\", \"fleet\", \"truck1\", \"POINT\", 33.0001, -112.0001}, {\"OK\"},\n {\"GET\", \"fleet\", \"truck1\", \"POINT\"}, {\"[33.0001 -112.0001]\"},\n }\n}\n```\n\nUsing a custom function:\n\n```go\nfunc keys_MATCH_test(mc *mockServer) error {\n\treturn mc.DoBatch([][]interface{}{\n {\"SET\", \"fleet\", \"truck1\", \"POINT\", 33.0001, -112.0001}, {\n func(v interface{}) (resp, expect interface{}) {\n // v is the value as strings or slices of strings\n // test will pass as long as `resp` and `expect` are the same.\n return v, \"OK\"\n },\n\t\t},\n }\n}\n```\n\n\n" }, { "path": "tests/aof_test.go", "content": "package tests\n\nimport (\n\t\"bytes\"\n\t\"crypto/md5\"\n\t\"encoding/hex\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"math/rand\"\n\t\"net\"\n\t\"net/http\"\n\t\"sync/atomic\"\n\t\"time\"\n\n\t\"github.com/gomodule/redigo/redis\"\n\n\t_ \"embed\"\n)\n\nfunc subTestAOF(g *testGroup) {\n\tg.regSubTest(\"loading\", aof_loading_test)\n\tg.regSubTest(\"migrate\", aof_migrate_test)\n\tg.regSubTest(\"AOF\", aof_AOF_test)\n\tg.regSubTest(\"AOFMD5\", aof_AOFMD5_test)\n\tg.regSubTest(\"AOFSHRINK\", aof_AOFSHRINK_test)\n\tg.regSubTest(\"READONLY\", aof_READONLY_test)\n}\n\nfunc loadAOFAndClose(aof any) error {\n\tmc, err := loadAOF(aof)\n\tif mc != nil {\n\t\tmc.Close()\n\t}\n\treturn err\n}\n\nfunc loadAOF(aof any) (*mockServer, error) {\n\tvar aofb []byte\n\tswitch aof := aof.(type) {\n\tcase []byte:\n\t\taofb = []byte(aof)\n\tcase string:\n\t\taofb = []byte(aof)\n\tdefault:\n\t\treturn nil, errors.New(\"aof is not string or bytes\")\n\t}\n\treturn mockOpenServer(MockServerOptions{\n\t\tSilent: true,\n\t\tMetrics: false,\n\t\tAOFData: aofb,\n\t})\n}\n\nfunc aof_loading_test(mc *mockServer) error {\n\n\tvar err error\n\t// invalid command\n\terr = loadAOFAndClose(\"asdfasdf\\r\\n\")\n\tif err == nil || err.Error() != \"unknown command 'asdfasdf'\" {\n\t\treturn fmt.Errorf(\"expected '%v', got '%v'\",\n\t\t\t\"unknown command 'asdfasdf'\", err)\n\t}\n\n\t// incomplete command\n\terr = loadAOFAndClose(\"set fleet truck point 10 10\\r\\nasdfasdf\")\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// big aof file\n\tvar aof string\n\tfor i := 0; i < 10000; i++ {\n\t\taof += fmt.Sprintf(\"SET fleet truck%d POINT 10 10\\r\\n\", i)\n\t}\n\terr = loadAOFAndClose(aof)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// extra zeros at various places\n\taof = \"\"\n\tfor i := 0; i < 1000; i++ {\n\t\tif i%10 == 0 {\n\t\t\taof += string(bytes.Repeat([]byte{0}, 100))\n\t\t}\n\t\taof += fmt.Sprintf(\"SET fleet truck%d POINT 10 10\\r\\n\", i)\n\t}\n\taof += string(bytes.Repeat([]byte{0}, 5000))\n\terr = loadAOFAndClose(aof)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// bad protocol\n\taof = \"*2\\r\\n$1\\r\\nh\\r\\n+OK\\r\\n\"\n\terr = loadAOFAndClose(aof)\n\tif fmt.Sprintf(\"%v\", err) != \"Protocol error: expected '$', got '+'\" {\n\t\treturn fmt.Errorf(\"expected '%v', got '%v'\",\n\t\t\t\"Protocol error: expected '$', got '+'\", err)\n\t}\n\treturn nil\n}\n\nfunc aof_AOFMD5_test(mc *mockServer) error {\n\tfor i := 0; i < 10000; i++ {\n\t\t_, err := mc.Do(\"SET\", \"fleet\", rand.Int(),\n\t\t\t\"POINT\", rand.Float64()*180-90, rand.Float64()*360-180)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\taof, err := mc.readAOF()\n\tif err != nil {\n\t\treturn err\n\t}\n\tcheck := func(start, size int) func(s string) error {\n\t\treturn func(s string) error {\n\t\t\tsum := md5.Sum(aof[start : start+size])\n\t\t\tval := hex.EncodeToString(sum[:])\n\t\t\tif s != val {\n\t\t\t\treturn fmt.Errorf(\"expected '%s', got '%s'\", val, s)\n\t\t\t}\n\t\t\treturn nil\n\t\t}\n\t}\n\treturn mc.DoBatch(\n\t\tDo(\"AOFMD5\").Err(\"wrong number of arguments for 'aofmd5' command\"),\n\t\tDo(\"AOFMD5\", 0).Err(\"wrong number of arguments for 'aofmd5' command\"),\n\t\tDo(\"AOFMD5\", 0, 0, 1).Err(\"wrong number of arguments for 'aofmd5' command\"),\n\t\tDo(\"AOFMD5\", -1, 0).Err(\"invalid argument '-1'\"),\n\t\tDo(\"AOFMD5\", 1, -1).Err(\"invalid argument '-1'\"),\n\t\tDo(\"AOFMD5\", 0, 100000000000).Err(\"EOF\"),\n\t\tDo(\"AOFMD5\", 0, 0).Str(\"d41d8cd98f00b204e9800998ecf8427e\"),\n\t\tDo(\"AOFMD5\", 0, 0).JSON().Str(`{\"ok\":true,\"md5\":\"d41d8cd98f00b204e9800998ecf8427e\"}`),\n\t\tDo(\"AOFMD5\", 0, 0).Func(check(0, 0)),\n\t\tDo(\"AOFMD5\", 0, 1).Func(check(0, 1)),\n\t\tDo(\"AOFMD5\", 0, 100).Func(check(0, 100)),\n\t\tDo(\"AOFMD5\", 1002, 4321).Func(check(1002, 4321)),\n\t)\n}\n\nfunc openFollower(mc *mockServer) (conn redis.Conn, err error) {\n\tconn, err = redis.Dial(\"tcp\", fmt.Sprintf(\":%d\", mc.port),\n\t\tredis.DialReadTimeout(time.Second))\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tdefer func() {\n\t\tif err != nil {\n\t\t\tconn.Close()\n\t\t\tconn = nil\n\t\t}\n\t}()\n\tif err := conn.Send(\"AOF\", 0); err != nil {\n\t\treturn nil, err\n\t}\n\tif err := conn.Flush(); err != nil {\n\t\treturn nil, err\n\t}\n\tstr, err := redis.String(conn.Receive())\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tif str != \"OK\" {\n\t\treturn nil, fmt.Errorf(\"expected '%s', got '%s'\", \"OK\", str)\n\t}\n\treturn conn, nil\n}\n\nfunc aof_AOF_test(mc *mockServer) error {\n\tvar argss [][]interface{}\n\tfor i := 0; i < 10000; i++ {\n\t\targs := []interface{}{\"SET\", \"fleet\", fmt.Sprint(rand.Int()),\n\t\t\t\"POINT\", fmt.Sprint(rand.Float64()*180 - 90),\n\t\t\tfmt.Sprint(rand.Float64()*360 - 180)}\n\t\targss = append(argss, args)\n\t\t_, err := mc.Do(fmt.Sprint(args[0]), args[1:]...)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\treadAll := func() (conn redis.Conn, err error) {\n\t\tconn, err = openFollower(mc)\n\t\tif err != nil {\n\t\t\treturn\n\t\t}\n\t\tdefer func() {\n\t\t\tif err != nil {\n\t\t\t\tconn.Close()\n\t\t\t\tconn = nil\n\t\t\t}\n\t\t}()\n\t\tvar t bool\n\t\tfor i := 0; i < len(argss); i++ {\n\t\t\targs, err := redis.Values(conn.Receive())\n\t\t\tif err != nil {\n\t\t\t\treturn nil, err\n\t\t\t}\n\t\t\tif t || (len(args) == len(argss[0]) &&\n\t\t\t\tfmt.Sprintf(\"%s\", args[2]) == fmt.Sprintf(\"%s\", argss[0][2])) {\n\t\t\t\tt = true\n\t\t\t\tif fmt.Sprintf(\"%s\", args[2]) !=\n\t\t\t\t\tfmt.Sprintf(\"%s\", argss[i][2]) {\n\t\t\t\t\treturn nil, fmt.Errorf(\"expected '%s', got '%s'\",\n\t\t\t\t\t\targss[i][2], args[2])\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\ti--\n\t\t\t}\n\t\t}\n\t\treturn conn, nil\n\t}\n\n\tconn, err := readAll()\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer conn.Close()\n\t_, err = conn.Do(\"fancy\") // non-existent error\n\tif err == nil || err.Error() != \"EOF\" {\n\t\treturn fmt.Errorf(\"expected '%v', got '%v'\", \"EOF\", err)\n\t}\n\n\tconn, err = readAll()\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer conn.Close()\n\t_, err = conn.Do(\"quit\")\n\tif err == nil || err.Error() != \"EOF\" {\n\t\treturn fmt.Errorf(\"expected '%v', got '%v'\", \"EOF\", err)\n\t}\n\n\treturn mc.DoBatch(\n\t\tDo(\"AOF\").Err(\"wrong number of arguments for 'aof' command\"),\n\t\tDo(\"AOF\", 0, 0).Err(\"wrong number of arguments for 'aof' command\"),\n\t\tDo(\"AOF\", -1).Err(\"invalid argument '-1'\"),\n\t\tDo(\"AOF\", 1000000000000).Err(\"pos is too big, must be less that the aof_size of leader\"),\n\t)\n}\n\nfunc aof_AOFSHRINK_test(mc *mockServer) error {\n\tvar err error\n\thaddr := fmt.Sprintf(\"localhost:%d\", getNextPort())\n\tln, err := net.Listen(\"tcp\", haddr)\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer ln.Close()\n\tvar msgs atomic.Int32\n\tgo func() {\n\t\tmux := http.NewServeMux()\n\t\tmux.HandleFunc(\"/\", func(w http.ResponseWriter, r *http.Request) {\n\t\t\tmsgs.Add(1)\n\t\t\t// println(r.URL.Path)\n\t\t})\n\t\thttp.Serve(ln, mux)\n\t}()\n\terr = mc.DoBatch(\n\t\tDo(\"SETCHAN\", \"mychan\", \"INTERSECTS\", \"mi:0\", \"BOUNDS\", -10, -10, 10, 10).Str(\"1\"),\n\t\tDo(\"SETHOOK\", \"myhook\", \"http://\"+haddr, \"INTERSECTS\", \"mi:0\", \"BOUNDS\", -10, -10, 10, 10).Str(\"1\"),\n\t\tDo(\"MASSINSERT\", 5, 10000).OK(),\n\t)\n\tif err != nil {\n\t\treturn err\n\t}\n\terr = mc.DoBatch(\n\t\tDo(\"AOFSHRINK\").OK(),\n\t\tDo(\"MASSINSERT\", 5, 10000).OK(),\n\t)\n\tif err != nil {\n\t\treturn err\n\t}\n\tnmsgs := msgs.Load()\n\tif nmsgs == 0 {\n\t\treturn fmt.Errorf(\"expected > 0, got %d\", nmsgs)\n\t}\n\treturn err\n}\n\nfunc aof_READONLY_test(mc *mockServer) error {\n\treturn mc.DoBatch(\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"POINT\", \"10\", \"10\").OK(),\n\t\tDo(\"READONLY\", \"yes\").OK(),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"POINT\", \"10\", \"10\").Err(\"read only\"),\n\t\tDo(\"READONLY\", \"no\").OK(),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"POINT\", \"10\", \"10\").OK(),\n\t\tDo(\"READONLY\").Err(\"wrong number of arguments for 'readonly' command\"),\n\t\tDo(\"READONLY\", \"maybe\").Err(\"invalid argument 'maybe'\"),\n\t)\n}\n\n//go:embed aof_legacy\nvar aofLegacy []byte\n\nfunc aof_migrate_test(mc *mockServer) error {\n\tvar aof []byte\n\tfor i := 0; i < 10000; i++ {\n\t\taof = append(aof, aofLegacy...)\n\t}\n\tvar mc2 *mockServer\n\tvar err error\n\tdefer func() {\n\t\tmc2.Close()\n\t}()\n\tmc2, err = mockOpenServer(MockServerOptions{\n\t\tAOFFileName: \"aof\",\n\t\tAOFData: aof,\n\t\tSilent: true,\n\t\tMetrics: true,\n\t})\n\tif err != nil {\n\t\treturn err\n\t}\n\terr = mc2.DoBatch(\n\t\tDo(\"GET\", \"1\", \"2\").Str(`{\"type\":\"Point\",\"coordinates\":[20,10]}`),\n\t)\n\tif err != nil {\n\t\treturn err\n\t}\n\tmc2.Close()\n\n\tmc2, err = mockOpenServer(MockServerOptions{\n\t\tAOFFileName: \"aof\",\n\t\tAOFData: aofLegacy[:len(aofLegacy)-1],\n\t\tSilent: true,\n\t\tMetrics: true,\n\t})\n\tif err != io.ErrUnexpectedEOF {\n\t\treturn fmt.Errorf(\"expected '%v', got '%v'\", io.ErrUnexpectedEOF, err)\n\t}\n\tmc2.Close()\n\n\tmc2, err = mockOpenServer(MockServerOptions{\n\t\tAOFFileName: \"aof\",\n\t\tAOFData: aofLegacy[1:],\n\t\tSilent: true,\n\t\tMetrics: true,\n\t})\n\tif err != io.ErrUnexpectedEOF {\n\t\treturn fmt.Errorf(\"expected '%v', got '%v'\", io.ErrUnexpectedEOF, err)\n\t}\n\tmc2.Close()\n\n\treturn nil\n}\n" }, { "path": "tests/client_test.go", "content": "package tests\n\nimport (\n\t\"errors\"\n\t\"fmt\"\n\t\"strings\"\n\n\t\"github.com/gomodule/redigo/redis\"\n\t\"github.com/tidwall/gjson\"\n\t\"github.com/tidwall/pretty\"\n)\n\nfunc subTestClient(g *testGroup) {\n\tg.regSubTest(\"OUTPUT\", client_OUTPUT_test)\n\tg.regSubTest(\"CLIENT\", client_CLIENT_test)\n}\n\nfunc client_OUTPUT_test(mc *mockServer) error {\n\tif err := mc.DoBatch(\n\t\t// tests removal of \"elapsed\" member.\n\t\tDo(\"OUTPUT\", \"json\", \"yaml\").Err(`wrong number of arguments for 'output' command`),\n\t\tDo(\"OUTPUT\", \"json\").Str(`{\"ok\":true}`),\n\t\tDo(\"OUTPUT\").JSON().Str(`{\"ok\":true,\"output\":\"json\"}`),\n\t\tDo(\"OUTPUT\").Str(`resp`), // this is due to the internal Do test\n\t\tDo(\"OUTPUT\", \"resp\").OK(),\n\t\tDo(\"OUTPUT\", \"yaml\").Err(`invalid argument 'yaml'`),\n\t\tDo(\"OUTPUT\").Str(`resp`),\n\t\tDo(\"OUTPUT\").JSON().Str(`{\"ok\":true,\"output\":\"json\"}`),\n\t); err != nil {\n\t\treturn err\n\t}\n\n\t// run direct commands\n\tif _, err := mc.Do(\"OUTPUT\", \"json\"); err != nil {\n\t\treturn err\n\t}\n\tres, err := mc.Do(\"CLIENT\", \"list\")\n\tif err != nil {\n\t\treturn err\n\t}\n\tbres, ok := res.([]byte)\n\tif !ok {\n\t\treturn errors.New(\"Failed to type assert CLIENT response\")\n\t}\n\tsres := string(bres)\n\tif !gjson.Valid(sres) {\n\t\treturn errors.New(\"CLIENT response was invalid\")\n\t}\n\tinfo := gjson.Get(sres, \"list\").String()\n\tif !gjson.Valid(info) {\n\t\treturn errors.New(\"CLIENT.list response was invalid\")\n\t}\n\treturn nil\n}\n\nfunc client_CLIENT_test(mc *mockServer) error {\n\tnumConns := 20\n\tvar conns []redis.Conn\n\tdefer func() {\n\t\tfor i := range conns {\n\t\t\tconns[i].Close()\n\t\t}\n\t}()\n\tfor i := 0; i <= numConns; i++ {\n\t\tconn, err := redis.Dial(\"tcp\", fmt.Sprintf(\":%d\", mc.port))\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tconn.Do(\"PING\")\n\t\tconns = append(conns, conn)\n\t}\n\n\t_, err := conns[1].Do(\"CLIENT\", \"setname\", \"cl1\")\n\tif err != nil {\n\t\treturn err\n\t}\n\t_, err = conns[2].Do(\"CLIENT\", \"setname\", \"cl2\")\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tif _, err := mc.Do(\"OUTPUT\", \"JSON\"); err != nil {\n\t\treturn err\n\t}\n\tres, err := mc.Do(\"CLIENT\", \"list\")\n\tif err != nil {\n\t\treturn err\n\t}\n\tbres, ok := res.([]byte)\n\tif !ok {\n\t\treturn errors.New(\"Failed to type assert CLIENT response\")\n\t}\n\tsres := string(pretty.Pretty(bres))\n\tif int(gjson.Get(sres, \"list.#\").Int()) < numConns {\n\t\treturn errors.New(\"Invalid number of connections\")\n\t}\n\n\tclient13ID := gjson.Get(sres, \"list.13.id\").String()\n\tclient14Addr := gjson.Get(sres, \"list.14.addr\").String()\n\tclient15Addr := gjson.Get(sres, \"list.15.addr\").String()\n\n\treturn mc.DoBatch(\n\t\tDo(\"CLIENT\", \"list\").JSON().Func(func(s string) error {\n\t\t\tif int(gjson.Get(s, \"list.#\").Int()) < numConns {\n\t\t\t\treturn errors.New(\"Invalid number of connections\")\n\t\t\t}\n\t\t\treturn nil\n\t\t}),\n\t\tDo(\"CLIENT\", \"list\").Func(func(s string) error {\n\t\t\tif len(strings.Split(strings.TrimSpace(s), \"\\n\")) < numConns {\n\t\t\t\treturn errors.New(\"Invalid number of connections\")\n\t\t\t}\n\t\t\treturn nil\n\t\t}),\n\t\tDo(\"CLIENT\").Err(`wrong number of arguments for 'client' command`),\n\t\tDo(\"CLIENT\", \"hello\").Err(`Syntax error, try CLIENT (LIST | KILL | GETNAME | SETNAME)`),\n\t\tDo(\"CLIENT\", \"list\", \"arg3\").Err(`wrong number of arguments for 'client' command`),\n\t\tDo(\"CLIENT\", \"getname\", \"arg3\").Err(`wrong number of arguments for 'client' command`),\n\t\tDo(\"CLIENT\", \"getname\").JSON().Str(`{\"ok\":true,\"name\":\"\"}`),\n\t\tDo(\"CLIENT\", \"getname\").Str(``),\n\t\tDo(\"CLIENT\", \"setname\", \"abc\").OK(),\n\t\tDo(\"CLIENT\", \"getname\").Str(`abc`),\n\t\tDo(\"CLIENT\", \"getname\").JSON().Str(`{\"ok\":true,\"name\":\"abc\"}`),\n\t\tDo(\"CLIENT\", \"setname\", \"abc\", \"efg\").Err(`wrong number of arguments for 'client' command`),\n\t\tDo(\"CLIENT\", \"setname\", \" abc \").Err(`Client names cannot contain spaces, newlines or special characters.`),\n\t\tDo(\"CLIENT\", \"setname\", \"abcd\").JSON().OK(),\n\t\tDo(\"CLIENT\", \"kill\", \"name\", \"abcd\").Err(\"No such client\"),\n\t\tDo(\"CLIENT\", \"getname\").Str(`abcd`),\n\t\tDo(\"CLIENT\", \"kill\").Err(`wrong number of arguments for 'client' command`),\n\t\tDo(\"CLIENT\", \"kill\", \"\").Err(`No such client`),\n\t\tDo(\"CLIENT\", \"kill\", \"abcd\").Err(`No such client`),\n\t\tDo(\"CLIENT\", \"kill\", \"id\", client13ID).OK(),\n\t\tDo(\"CLIENT\", \"kill\", \"id\").Err(\"wrong number of arguments for 'client' command\"),\n\t\tDo(\"CLIENT\", \"kill\", client14Addr).OK(),\n\t\tDo(\"CLIENT\", \"kill\", client14Addr, \"yikes\").Err(\"wrong number of arguments for 'client' command\"),\n\t\tDo(\"CLIENT\", \"kill\", \"addr\").Err(\"wrong number of arguments for 'client' command\"),\n\t\tDo(\"CLIENT\", \"kill\", \"addr\", client15Addr).JSON().OK(),\n\t\tDo(\"CLIENT\", \"kill\", \"addr\", client14Addr, \"yikes\").Err(\"wrong number of arguments for 'client' command\"),\n\t\tDo(\"CLIENT\", \"kill\", \"id\", \"1000\").Err(\"No such client\"),\n\t)\n\n}\n" }, { "path": "tests/fence_roaming_test.go", "content": "package tests\n\nimport (\n\t\"fmt\"\n\t\"io\"\n\t\"net/http\"\n\t\"net/http/httptest\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/gomodule/redigo/redis\"\n\t\"github.com/tidwall/pretty\"\n\t\"github.com/tidwall/sjson\"\n)\n\nfunc fence_roaming_webhook_test(mc *mockServer) error {\n\tcar1, car2, expected := roamingTestData()\n\tfinalErr := make(chan error)\n\n\t// Create a connection for subscribing to geofence notifications\n\tsc, err := redis.Dial(\"tcp\", fmt.Sprintf(\":%d\", mc.port))\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer sc.Close()\n\n\tactual := []string{}\n\t// Create the test http server that will capture all messages\n\tts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tif err := func() error {\n\t\t\t// Read the request body\n\t\t\tbody, err := io.ReadAll(r.Body)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\n\t\t\t// If the new message doesn't match whats expected an error\n\t\t\t// should be returned\n\t\t\tactual = append(actual, cleanMessage(body))\n\t\t\tpos := len(actual) - 1\n\t\t\tif len(expected) < pos+1 {\n\t\t\t\treturn fmt.Errorf(\"More messages than expected were received : '%s'\", actual[pos])\n\t\t\t}\n\t\t\tif actual[pos] != expected[pos] {\n\t\t\t\treturn fmt.Errorf(\"Expected '%s' but got '%s'\", expected[pos],\n\t\t\t\t\tactual[pos])\n\t\t\t}\n\t\t\tif len(actual) == len(expected) {\n\t\t\t\tfinalErr <- nil\n\t\t\t}\n\t\t\treturn nil\n\t\t}(); err != nil {\n\t\t\tfinalErr <- err\n\t\t}\n\t\tfmt.Fprintln(w, \"OK!\")\n\t}))\n\tdefer ts.Close()\n\n\t_, err = sc.Do(\"SETHOOK\", \"carshook\", ts.URL, \"NEARBY\", \"cars\", \"FENCE\", \"ROAM\", \"cars\", \"*\", 1000)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// Create the base connection for setting up points and geofences\n\tbc, err := redis.Dial(\"tcp\", fmt.Sprintf(\":%d\", mc.port))\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer bc.Close()\n\n\t// Fire all car movement commands on the base client\n\tfor i := range car1 {\n\t\tif _, err := bc.Do(\"SET\", \"cars\", \"car1\", \"POINT\", car1[i][1],\n\t\t\tcar1[i][0]); err != nil {\n\t\t\treturn err\n\t\t}\n\t\tif _, err := bc.Do(\"SET\", \"cars\", \"car2\", \"POINT\", car2[i][1],\n\t\t\tcar2[i][0]); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\treturn <-finalErr\n}\n\nfunc goMultiFunc(mc *mockServer, fns ...func() error) error {\n\terrs := make([]error, len(fns))\n\tvar wg sync.WaitGroup\n\twg.Add(len(fns))\n\tfor i := 0; i < len(fns); i++ {\n\t\tgo func(i int) {\n\t\t\tdefer wg.Done()\n\t\t\terrs[i] = fns[i]()\n\t\t}(i)\n\t}\n\twg.Wait()\n\tvar ferrs []error\n\tfor i := 0; i < len(errs); i++ {\n\t\tif errs[i] != nil {\n\t\t\tferrs = append(ferrs, errs[i])\n\t\t}\n\t}\n\tif len(ferrs) == 0 {\n\t\treturn nil\n\t}\n\tif len(ferrs) == 1 {\n\t\treturn ferrs[0]\n\t}\n\treturn fmt.Errorf(\"%v\", ferrs)\n}\n\nfunc fence_roaming_live_test(mc *mockServer) error {\n\tcar1, car2, expected := roamingTestData()\n\tvar liveReady sync.WaitGroup\n\tliveReady.Add(1)\n\treturn goMultiFunc(mc,\n\t\tfunc() error {\n\t\t\tsc, err := redis.Dial(\"tcp\", fmt.Sprintf(\":%d\", mc.port),\n\t\t\t\tredis.DialConnectTimeout(0),\n\t\t\t\tredis.DialReadTimeout(time.Second*5),\n\t\t\t\tredis.DialWriteTimeout(time.Second*5))\n\t\t\tif err != nil {\n\t\t\t\tliveReady.Done()\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tdefer sc.Close()\n\t\t\t// Set up a live geofence stream\n\t\t\treply, err := redis.String(\n\t\t\t\tsc.Do(\"NEARBY\", \"cars\", \"FENCE\", \"ROAM\", \"cars\", \"*\", 1000),\n\t\t\t)\n\t\t\tif err != nil {\n\t\t\t\tliveReady.Done()\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tif reply != \"OK\" {\n\t\t\t\tliveReady.Done()\n\t\t\t\treturn fmt.Errorf(\"expected 'OK', got '%v'\", reply)\n\t\t\t}\n\t\t\tliveReady.Done()\n\t\t\tfor i := 0; i < len(expected); i++ {\n\t\t\t\treply, err := redis.String(sc.Receive())\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\t\t\t\treply = cleanMessage([]byte(reply))\n\t\t\t\tif reply != expected[i] {\n\t\t\t\t\treturn fmt.Errorf(\"Expected '%s' but got '%s'\",\n\t\t\t\t\t\texpected[i], reply)\n\t\t\t\t}\n\t\t\t}\n\t\t\treturn nil\n\t\t},\n\t\tfunc() error {\n\t\t\tliveReady.Wait()\n\t\t\tbc, err := redis.Dial(\"tcp\", fmt.Sprintf(\":%d\", mc.port))\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tdefer bc.Close()\n\n\t\t\t// Fire all car movement commands on the base client\n\t\t\tfor i := range car1 {\n\n\t\t\t\tif _, err := bc.Do(\"SET\", \"cars\", \"car1\", \"POINT\", car1[i][1],\n\t\t\t\t\tcar1[i][0]); err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\t\t\t\tif _, err := bc.Do(\"SET\", \"cars\", \"car2\", \"POINT\", car2[i][1],\n\t\t\t\t\tcar2[i][0]); err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\t\t\t}\n\n\t\t\treturn nil\n\t\t},\n\t)\n}\n\nfunc fence_roaming_channel_test(mc *mockServer) error {\n\tcar1, car2, expected := roamingTestData()\n\tfinalErr := make(chan error)\n\n\tgo func() {\n\t\t// Create a connection for subscribing to geofence notifications\n\t\tsc, err := redis.Dial(\"tcp\", fmt.Sprintf(\":%d\", mc.port))\n\t\tif err != nil {\n\t\t\tfinalErr <- err\n\t\t\treturn\n\t\t}\n\t\tdefer sc.Close()\n\n\t\tif _, err := sc.Do(\"SETCHAN\", \"carschan\", \"NEARBY\", \"cars\", \"FENCE\", \"ROAM\", \"cars\", \"*\", 1000); err != nil {\n\t\t\tfinalErr <- err\n\t\t\treturn\n\t\t}\n\n\t\t// Subscribe the subscription client to the * pattern\n\t\tpsc := redis.PubSubConn{Conn: sc}\n\t\tif err := psc.PSubscribe(\"carschan\"); err != nil {\n\t\t\tfinalErr <- err\n\t\t\treturn\n\t\t}\n\n\t\tactual := []string{}\n\t\tfor sc.Err() == nil {\n\t\t\tif err := func() error {\n\t\t\t\tvar body []byte\n\t\t\t\tswitch v := psc.Receive().(type) {\n\t\t\t\tcase redis.Message:\n\t\t\t\t\tbody = v.Data\n\t\t\t\tcase error:\n\t\t\t\t\treturn err\n\t\t\t\t}\n\t\t\t\tif len(body) == 0 {\n\t\t\t\t\treturn nil\n\t\t\t\t}\n\n\t\t\t\t// If the new message doesn't match whats expected an error\n\t\t\t\t// should be returned\n\t\t\t\tactual = append(actual, cleanMessage(body))\n\t\t\t\tpos := len(actual) - 1\n\t\t\t\tif len(expected) < pos+1 {\n\t\t\t\t\treturn fmt.Errorf(\"More messages than expected were received : '%s'\", actual[pos])\n\t\t\t\t}\n\t\t\t\tif actual[pos] != expected[pos] {\n\t\t\t\t\treturn fmt.Errorf(\"Expected '%s' but got '%s'\", expected[pos],\n\t\t\t\t\t\tactual[pos])\n\t\t\t\t}\n\t\t\t\tif len(actual) == len(expected) {\n\t\t\t\t\tfinalErr <- nil\n\t\t\t\t}\n\t\t\t\treturn nil\n\t\t\t}(); err != nil {\n\t\t\t\tfinalErr <- err\n\t\t\t}\n\t\t}\n\t}()\n\n\t// Create the base connection for setting up points and geofences\n\tbc, err := redis.Dial(\"tcp\", fmt.Sprintf(\":%d\", mc.port))\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer bc.Close()\n\n\t// Fire all car movement commands on the base client\n\tfor i := range car1 {\n\t\tif _, err := bc.Do(\"SET\", \"cars\", \"car1\", \"POINT\", car1[i][1],\n\t\t\tcar1[i][0]); err != nil {\n\t\t\treturn err\n\t\t}\n\t\tif _, err := bc.Do(\"SET\", \"cars\", \"car2\", \"POINT\", car2[i][1],\n\t\t\tcar2[i][0]); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\treturn <-finalErr\n}\n\nfunc cleanMessage(body []byte) string {\n\t// Remove fields that are non-deterministic or use case specific\n\tmsg, _ := sjson.Delete(string(body), \"group\")\n\tmsg, _ = sjson.Delete(msg, \"time\")\n\tmsg, _ = sjson.Delete(msg, \"hook\")\n\tmsg = string(pretty.Ugly([]byte(msg)))\n\treturn msg\n}\n\nfunc roamingTestData() (car1 [][]float64, car2 [][]float64, output []string) {\n\tcar1 = [][]float64{\n\t\t{-111.93669319152832, 33.414750027566235},\n\t\t{-111.93051338195801, 33.414750027566235},\n\t\t{-111.92416191101074, 33.414750027566235},\n\t\t{-111.91789627075195, 33.414750027566235},\n\t\t{-111.9111156463623, 33.414750027566235},\n\t\t{-111.90510749816895, 33.414750027566235},\n\t\t{-111.89746856689453, 33.414750027566235},\n\t}\n\tcar2 = [][]float64{\n\t\t{-111.89746856689453, 33.414750027566235},\n\t\t{-111.90519332885742, 33.414750027566235},\n\t\t{-111.91154479980467, 33.414750027566235},\n\t\t{-111.91781044006346, 33.414750027566235},\n\t\t{-111.92416191101074, 33.414750027566235},\n\t\t{-111.93059921264648, 33.414750027566235},\n\t\t{-111.93660736083984, 33.414750027566235},\n\t}\n\toutput = []string{\n\t\t`{\"command\":\"set\",\"detect\":\"roam\",\"key\":\"cars\",\"id\":\"car1\",\"object\":{\"type\":\"Point\",\"coordinates\":[-111.91789627075195,33.414750027566235]},\"nearby\":{\"key\":\"cars\",\"id\":\"car2\",\"object\":{\"type\":\"Point\",\"coordinates\":[-111.91154479980467,33.414750027566235]},\"meters\":589.512}}`,\n\t\t`{\"command\":\"set\",\"detect\":\"roam\",\"key\":\"cars\",\"id\":\"car2\",\"object\":{\"type\":\"Point\",\"coordinates\":[-111.91781044006346,33.414750027566235]},\"nearby\":{\"key\":\"cars\",\"id\":\"car1\",\"object\":{\"type\":\"Point\",\"coordinates\":[-111.91789627075195,33.414750027566235]},\"meters\":7.966}}`,\n\t\t`{\"command\":\"set\",\"detect\":\"roam\",\"key\":\"cars\",\"id\":\"car1\",\"object\":{\"type\":\"Point\",\"coordinates\":[-111.9111156463623,33.414750027566235]},\"nearby\":{\"key\":\"cars\",\"id\":\"car2\",\"object\":{\"type\":\"Point\",\"coordinates\":[-111.91781044006346,33.414750027566235]},\"meters\":621.377}}`,\n\t\t`{\"command\":\"set\",\"detect\":\"roam\",\"key\":\"cars\",\"id\":\"car2\",\"object\":{\"type\":\"Point\",\"coordinates\":[-111.92416191101074,33.414750027566235]},\"faraway\":{\"key\":\"cars\",\"id\":\"car1\",\"object\":{\"type\":\"Point\",\"coordinates\":[-111.9111156463623,33.414750027566235]},\"meters\":1210.89}}`,\n\t}\n\treturn\n}\n" }, { "path": "tests/fence_test.go", "content": "package tests\n\nimport (\n\t\"bufio\"\n\t\"encoding/json\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"log\"\n\t\"net\"\n\t\"strconv\"\n\t\"strings\"\n\t\"sync\"\n\t\"sync/atomic\"\n\t\"time\"\n\n\t\"github.com/gomodule/redigo/redis\"\n\t\"github.com/tidwall/gjson\"\n)\n\nfunc subTestFence(g *testGroup) {\n\n\t// Standard\n\tg.regSubTest(\"basic\", fence_basic_test)\n\tg.regSubTest(\"channel message order\", fence_channel_message_order_test)\n\tg.regSubTest(\"detect inside,outside\", fence_detect_inside_test)\n\n\t// Roaming\n\tg.regSubTest(\"roaming live\", fence_roaming_live_test)\n\tg.regSubTest(\"roaming channel\", fence_roaming_channel_test)\n\tg.regSubTest(\"roaming webhook\", fence_roaming_webhook_test)\n\n\t// channel meta\n\tg.regSubTest(\"channel meta\", fence_channel_meta_test)\n\n\t// various\n\tg.regSubTest(\"detect eecio\", fence_eecio_test)\n}\n\ntype fenceReader struct {\n\tconn net.Conn\n\trd *bufio.Reader\n}\n\nfunc (fr *fenceReader) receive() (string, error) {\n\tif err := fr.conn.SetReadDeadline(time.Now().Add(time.Second)); err != nil {\n\t\treturn \"\", err\n\t}\n\tline, err := fr.rd.ReadBytes('\\n')\n\tif err != nil {\n\t\treturn \"\", err\n\t}\n\tif len(line) < 4 || line[0] != '$' || line[len(line)-2] != '\\r' || line[len(line)-1] != '\\n' {\n\t\treturn \"\", errors.New(\"invalid message\")\n\t}\n\tn, err := strconv.ParseUint(string(line[1:len(line)-2]), 10, 64)\n\tif err != nil {\n\t\treturn \"\", err\n\t}\n\tbuf := make([]byte, int(n)+2)\n\t_, err = io.ReadFull(fr.rd, buf)\n\tif err != nil {\n\t\treturn \"\", err\n\t}\n\tif buf[len(buf)-2] != '\\r' || buf[len(buf)-1] != '\\n' {\n\t\treturn \"\", errors.New(\"invalid message\")\n\t}\n\tjs := buf[:len(buf)-2]\n\tvar m interface{}\n\tif err := json.Unmarshal(js, &m); err != nil {\n\t\treturn \"\", err\n\t}\n\treturn string(js), nil\n}\n\nfunc (fr *fenceReader) receiveExpect(valex ...string) error {\n\ts, err := fr.receive()\n\tif err != nil {\n\t\treturn err\n\t}\n\tfor i := 0; i < len(valex); i += 2 {\n\t\tif gjson.Get(s, valex[i]).String() != valex[i+1] {\n\t\t\treturn fmt.Errorf(\"expected '%s'='%s', got '%s'\", valex[i], valex[i+1], gjson.Get(s, valex[i]).String())\n\t\t}\n\t}\n\treturn nil\n}\n\nfunc fence_basic_test(mc *mockServer) error {\n\tconn, err := net.Dial(\"tcp\", fmt.Sprintf(\":%d\", mc.port))\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer conn.Close()\n\t_, err = fmt.Fprintf(conn, \"NEARBY mykey FENCE POINT 33 -115 5000\\r\\n\")\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tbuf := make([]byte, 4096)\n\tn, err := conn.Read(buf)\n\tif err != nil {\n\t\treturn err\n\t}\n\tres := string(buf[:n])\n\tif res != \"+OK\\r\\n\" {\n\t\treturn fmt.Errorf(\"expected OK, got '%v'\", res)\n\t}\n\trd := &fenceReader{conn, bufio.NewReader(conn)}\n\n\t// send a point\n\tc, err := redis.Dial(\"tcp\", fmt.Sprintf(\":%d\", mc.port))\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer c.Close()\n\n\tres, err = redis.String(c.Do(\"SET\", \"mykey\", \"myid1\", \"POINT\", 33, -115))\n\tif err != nil {\n\t\treturn err\n\t}\n\tif res != \"OK\" {\n\t\treturn fmt.Errorf(\"expected OK, got '%v'\", res)\n\t}\n\n\t// receive the message\n\tif err := rd.receiveExpect(\"command\", \"set\",\n\t\t\"detect\", \"enter\",\n\t\t\"key\", \"mykey\",\n\t\t\"id\", \"myid1\",\n\t\t\"object.type\", \"Point\",\n\t\t\"object.coordinates\", \"[-115,33]\"); err != nil {\n\t\treturn err\n\t}\n\n\tif err := rd.receiveExpect(\"command\", \"set\",\n\t\t\"detect\", \"inside\",\n\t\t\"key\", \"mykey\",\n\t\t\"id\", \"myid1\",\n\t\t\"object.type\", \"Point\",\n\t\t\"object.coordinates\", \"[-115,33]\"); err != nil {\n\t\treturn err\n\t}\n\n\tres, err = redis.String(c.Do(\"SET\", \"mykey\", \"myid1\", \"POINT\", 34, -115))\n\tif err != nil {\n\t\treturn err\n\t}\n\tif res != \"OK\" {\n\t\treturn fmt.Errorf(\"expected OK, got '%v'\", res)\n\t}\n\n\t// receive the message\n\tif err := rd.receiveExpect(\"command\", \"set\",\n\t\t\"detect\", \"exit\",\n\t\t\"key\", \"mykey\",\n\t\t\"id\", \"myid1\",\n\t\t\"object.type\", \"Point\",\n\t\t\"object.coordinates\", \"[-115,34]\"); err != nil {\n\t\treturn err\n\t}\n\n\tif err := rd.receiveExpect(\"command\", \"set\",\n\t\t\"detect\", \"outside\",\n\t\t\"key\", \"mykey\",\n\t\t\"id\", \"myid1\",\n\t\t\"object.type\", \"Point\",\n\t\t\"object.coordinates\", \"[-115,34]\"); err != nil {\n\t\treturn err\n\t}\n\treturn nil\n}\n\nfunc fence_channel_message_order_test(mc *mockServer) error {\n\t// Create a channel to store the goroutines error\n\tfinalErr := make(chan error)\n\n\tvar ready atomic.Bool\n\t// Concurrently subscribe for notifications\n\tgo func() {\n\t\t// Create the subscription connection to Tile38 to subscribe for updates\n\t\tsc, err := redis.Dial(\"tcp\", fmt.Sprintf(\":%d\", mc.port))\n\t\tif err != nil {\n\t\t\tlog.Println(err)\n\t\t\treturn\n\t\t}\n\t\tdefer sc.Close()\n\n\t\t// Subscribe the subscription client to the * pattern\n\t\tpsc := redis.PubSubConn{Conn: sc}\n\t\tif err := psc.PSubscribe(\"*\"); err != nil {\n\t\t\tlog.Println(err)\n\t\t\treturn\n\t\t}\n\t\tvar msgs []string\n\t\t// While not a permanent error on the connection.\n\tloop:\n\t\tfor sc.Err() == nil {\n\t\t\tswitch v := psc.Receive().(type) {\n\t\t\tcase redis.Message:\n\t\t\t\tif v.Channel == \"status\" && string(v.Data) == \"ready\" {\n\t\t\t\t\tready.Store(true)\n\t\t\t\t} else {\n\t\t\t\t\tmsgs = append(msgs, string(v.Data))\n\t\t\t\t\tif len(msgs) == 8 {\n\t\t\t\t\t\tbreak loop\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\tcase error:\n\t\t\t\tfmt.Printf(\"%s\\n\", err.Error())\n\t\t\t}\n\t\t}\n\t\t// Verify all messages\n\t\tcorrectOrder := []string{\"exit:A\", \"exit:B\", \"outside:A\", \"outside:B\", \"enter:C\", \"enter:D\", \"inside:C\", \"inside:D\"}\n\t\tfor i := range msgs {\n\t\t\tif gjson.Get(msgs[i], \"detect\").String()+\":\"+\n\t\t\t\tgjson.Get(msgs[i], \"hook\").String() != correctOrder[i] {\n\t\t\t\tfinalErr <- errors.New(\"INVALID MESSAGE ORDER\")\n\t\t\t}\n\t\t}\n\t\tfinalErr <- nil\n\t}()\n\t// Create the base connection for setting up points and geofences\n\tbc, err := redis.Dial(\"tcp\", fmt.Sprintf(\":%d\", mc.port))\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer bc.Close()\n\n\tfor !ready.Load() {\n\t\tif _, err := do(bc, \"PUBLISH status ready\"); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\n\t// Fire all setup commands on the base client\n\tfor _, cmd := range []string{\n\t\t\"SET points point POINT 33.412529053733444 -111.93368911743164\",\n\t\t`SETCHAN A WITHIN points FENCE OBJECT {\"type\":\"Polygon\",\"coordinates\":[[[-111.95205688476562,33.400491820565236],[-111.92630767822266,33.400491820565236],[-111.92630767822266,33.422272258866045],[-111.95205688476562,33.422272258866045],[-111.95205688476562,33.400491820565236]]]}`,\n\t\t`SETCHAN B WITHIN points FENCE OBJECT {\"type\":\"Polygon\",\"coordinates\":[[[-111.93952560424803,33.403501285221594],[-111.92630767822266,33.403501285221594],[-111.92630767822266,33.41997983836345],[-111.93952560424803,33.41997983836345],[-111.93952560424803,33.403501285221594]]]}`,\n\t\t`SETCHAN C WITHIN points FENCE OBJECT {\"type\":\"Polygon\",\"coordinates\":[[[-111.9255781173706,33.40342963251261],[-111.91201686859131,33.40342963251261],[-111.91201686859131,33.41994401881284],[-111.9255781173706,33.41994401881284],[-111.9255781173706,33.40342963251261]]]}`,\n\t\t`SETCHAN D WITHIN points FENCE OBJECT {\"type\":\"Polygon\",\"coordinates\":[[[-111.92562103271484,33.40063513076968],[-111.90021514892578,33.40063513076968],[-111.90021514892578,33.42212898435788],[-111.92562103271484,33.42212898435788],[-111.92562103271484,33.40063513076968]]]}`,\n\t\t\"SET points point POINT 33.412529053733444 -111.91909790039062\",\n\t} {\n\t\tif _, err := do(bc, cmd); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\treturn <-finalErr\n}\n\nfunc fence_detect_inside_test(mc *mockServer) error {\n\tconn, err := net.Dial(\"tcp\", fmt.Sprintf(\":%d\", mc.port))\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer conn.Close()\n\t_, err = fmt.Fprintf(conn, \"WITHIN users FENCE DETECT inside,outside POINTS BOUNDS 33.618824 -84.457973 33.654359 -84.399859\\r\\n\")\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tbuf := make([]byte, 4096)\n\tn, err := conn.Read(buf)\n\tif err != nil {\n\t\treturn err\n\t}\n\tres := string(buf[:n])\n\tif res != \"+OK\\r\\n\" {\n\t\treturn fmt.Errorf(\"expected OK, got '%v'\", res)\n\t}\n\trd := &fenceReader{conn, bufio.NewReader(conn)}\n\n\t// send a point\n\tc, err := redis.Dial(\"tcp\", fmt.Sprintf(\":%d\", mc.port))\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer c.Close()\n\n\tres, err = redis.String(c.Do(\"SET\", \"users\", \"200\", \"POINT\", \"33.642301\", \"-84.43118\"))\n\tif err != nil {\n\t\treturn err\n\t}\n\tif res != \"OK\" {\n\t\treturn fmt.Errorf(\"expected OK, got '%v'\", res)\n\t}\n\n\tif err := rd.receiveExpect(\"command\", \"set\",\n\t\t\"detect\", \"inside\",\n\t\t\"key\", \"users\",\n\t\t\"id\", \"200\",\n\t\t\"point\", `{\"lat\":33.642301,\"lon\":-84.43118}`); err != nil {\n\t\treturn err\n\t}\n\n\tres, err = redis.String(c.Do(\"SET\", \"users\", \"200\", \"POINT\", \"34.642301\", \"-84.43118\"))\n\tif err != nil {\n\t\treturn err\n\t}\n\tif res != \"OK\" {\n\t\treturn fmt.Errorf(\"expected OK, got '%v'\", res)\n\t}\n\n\t// receive the message\n\tif err := rd.receiveExpect(\"command\", \"set\",\n\t\t\"detect\", \"outside\",\n\t\t\"key\", \"users\",\n\t\t\"id\", \"200\",\n\t\t\"point\", `{\"lat\":34.642301,\"lon\":-84.43118}`); err != nil {\n\t\treturn err\n\t}\n\treturn nil\n}\n\n// do performs the passed command on the passed redis client\nfunc do(c redis.Conn, cmd string) (interface{}, error) {\n\t// Split out all parameters\n\tparams := strings.Split(cmd, \" \")\n\n\t// Produce a slice of interfaces for use in the arguments\n\tvar args []interface{}\n\tfor _, p := range params[1:] {\n\t\targs = append(args, p)\n\t}\n\n\t// Perform the request and return the response\n\treturn c.Do(params[0], args...)\n}\n\nfunc fence_channel_meta_test(mc *mockServer) error {\n\treturn mc.DoBatch([][]interface{}{\n\t\t{\"SETCHAN\", \"carbon\", \"NEARBY\", \"x\", \"MATCH\", \"carbon*\", \"FENCE\", \"NODWELL\", \"points\", \"ROAM\", \"x\", \"*\", \"200000\"}, {\"1\"},\n\t\t{\"OUTPUT\", \"json\"}, {`{\"ok\":true}`},\n\t\t// check for valid json on the chans command\n\t\t{\"CHANS\", \"*\"}, {\n\t\t\tfunc(v interface{}) (resp, expect interface{}) {\n\t\t\t\t// v is the value as strings or slices of strings\n\t\t\t\t// test will pass as long as `resp` and `expect` are the same.\n\t\t\t\tif !json.Valid([]byte(v.(string))) {\n\t\t\t\t\treturn v, \"Valid JSON\"\n\t\t\t\t}\n\t\t\t\treturn true, true\n\t\t\t},\n\t\t},\n\t})\n}\n\nfunc dialTile38(port int) (redis.Conn, error) {\n\tconn, err := redis.Dial(\"tcp\", fmt.Sprintf(\":%d\", port))\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tif _, err := conn.Do(\"OUTPUT\", \"json\"); err != nil {\n\t\tconn.Close()\n\t\treturn nil, err\n\t}\n\treturn conn, nil\n}\n\nfunc doTile38(c redis.Conn, cmd string, args ...interface{}) (string, error) {\n\tjs, err := redis.String(c.Do(cmd, args...))\n\tif !gjson.Get(js, \"ok\").Bool() {\n\t\treturn \"\", errors.New(gjson.Get(js, \"err\").String())\n\t}\n\treturn js, err\n}\n\nfunc fence_eecio_test(mc *mockServer) error {\n\t// simulates issue #578\n\tvar wg sync.WaitGroup\n\twg.Add(3)\n\tch := make(chan bool)\n\tvar err1, err2, err3 error\n\tvar msgs1, msgs2 []string\n\t// terminal 1\n\tgo func() {\n\t\tdefer wg.Done()\n\t\terr1 = func() error {\n\t\t\tconn, err := dialTile38(mc.port)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tdefer conn.Close()\n\t\t\t_, err = doTile38(conn,\n\t\t\t\t\"SETCHAN\", \"test-eec\", \"NEARBY\", \"fleet\",\n\t\t\t\t\"FENCE\", \"DETECT\", \"enter,exit,cross\",\n\t\t\t\t\"POINT\", \"10.000\", \"10.000\", \"10000\")\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\t_, err = doTile38(conn, \"SUBSCRIBE\", \"test-eec\")\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tch <- true\n\t\t\tfor {\n\t\t\t\tjs, err := redis.String(conn.Receive())\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\t\t\t\tif js == `\"DONE\"` {\n\t\t\t\t\tbreak\n\t\t\t\t}\n\t\t\t\tmsgs1 = append(msgs1, js)\n\t\t\t}\n\t\t\treturn nil\n\t\t}()\n\t}()\n\t// terminal 2\n\tgo func() {\n\t\tdefer wg.Done()\n\t\terr2 = func() error {\n\t\t\tconn, err := dialTile38(mc.port)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tdefer conn.Close()\n\t\t\t_, err = doTile38(conn,\n\t\t\t\t\"SETCHAN\", \"test-eecio\", \"NEARBY\", \"fleet\",\n\t\t\t\t\"FENCE\", \"DETECT\", \"enter,exit,cross,inside,outside\",\n\t\t\t\t\"POINT\", \"10.000\", \"10.000\", \"10000\")\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\t_, err = doTile38(conn, \"SUBSCRIBE\", \"test-eecio\")\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tch <- true\n\t\t\tfor {\n\t\t\t\tjs, err := redis.String(conn.Receive())\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\t\t\t\tif js == `\"DONE\"` {\n\t\t\t\t\tbreak\n\t\t\t\t}\n\t\t\t\tmsgs2 = append(msgs2, js)\n\t\t\t}\n\t\t\treturn nil\n\t\t}()\n\t}()\n\t// terminal 3\n\tvar ok bool\n\tgo func() {\n\t\tdefer wg.Done()\n\t\terr3 = func() error {\n\t\t\t<-ch // terminal 1\n\t\t\t<-ch // terminal 2\n\t\t\tconn, err := dialTile38(mc.port)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tdefer conn.Close()\n\t\t\tif _, err = doTile38(conn,\n\t\t\t\t\"SET\", \"fleet\", \"vehicle_1\",\n\t\t\t\t\"POINT\", \"10.0\", \"10.0\"); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tif _, err = doTile38(conn,\n\t\t\t\t\"SET\", \"fleet\", \"vehicle_1\",\n\t\t\t\t\"POINT\", \"0.0\", \"0.0\"); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tif _, err = doTile38(conn,\n\t\t\t\t\"SET\", \"fleet\", \"vehicle_1\",\n\t\t\t\t\"POINT\", \"20.0\", \"20.0\"); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tif _, err = doTile38(conn, \"PUBLISH\", \"test-eecio\",\n\t\t\t\t\"DONE\"); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tif _, err = doTile38(conn, \"PUBLISH\", \"test-eec\",\n\t\t\t\t\"DONE\"); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tok = true\n\t\t\treturn nil\n\t\t}()\n\t}()\n\tvar timeok atomic.Bool\n\tgo func() {\n\t\ttime.Sleep(time.Second * 30)\n\t\tif !timeok.Load() {\n\t\t\tpanic(\"timeout\")\n\t\t}\n\t}()\n\twg.Wait()\n\ttimeok.Store(true)\n\tif err3 != nil {\n\t\treturn err3\n\t}\n\tif !ok {\n\t\tif err2 != nil {\n\t\t\treturn err2\n\t\t}\n\t\tif err1 != nil {\n\t\t\treturn err1\n\t\t}\n\t}\n\tvar detects []string\n\tfor i := 0; i < len(msgs1); i++ {\n\t\tdetects = append(detects, gjson.Get(msgs1[i], \"detect\").String())\n\t}\n\tif strings.Join(detects, \",\") != \"enter,exit,cross\" {\n\t\terrmsg := fmt.Sprintf(\"expected 'enter,exit,cross', got '%s'\\n\",\n\t\t\tstrings.Join(detects, \",\"))\n\t\treturn errors.New(errmsg)\n\t}\n\tdetects = nil\n\tfor i := 0; i < len(msgs2); i++ {\n\t\tdetects = append(detects, gjson.Get(msgs2[i], \"detect\").String())\n\t}\n\n\tif strings.Join(detects, \",\") != \"enter,inside,exit,outside,cross,outside\" {\n\t\terrmsg := fmt.Sprintf(\n\t\t\t\"expected 'enter,inside,exit,outside,cross,outside', got '%s'\\n\",\n\t\t\tstrings.Join(detects, \",\"))\n\t\treturn errors.New(errmsg)\n\t}\n\n\treturn nil\n}\n" }, { "path": "tests/follower_test.go", "content": "package tests\n\nimport \"time\"\n\nfunc subTestFollower(g *testGroup) {\n\tg.regSubTest(\"follow\", follower_follow_test)\n}\n\nfunc follower_follow_test(mc *mockServer) error {\n\tmc2, err := mockOpenServer(MockServerOptions{\n\t\tSilent: true, Metrics: false,\n\t})\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer mc2.Close()\n\terr = mc.DoBatch(\n\t\tDo(\"SET\", \"mykey\", \"truck1\", \"POINT\", 10, 10).OK(),\n\t\tDo(\"SET\", \"mykey\", \"truck2\", \"POINT\", 10, 10).OK(),\n\t\tDo(\"SET\", \"mykey\", \"truck3\", \"POINT\", 10, 10).OK(),\n\t\tDo(\"SET\", \"mykey\", \"truck4\", \"POINT\", 10, 10).OK(),\n\t\tDo(\"SET\", \"mykey\", \"truck5\", \"POINT\", 10, 10).OK(),\n\t\tDo(\"SET\", \"mykey\", \"truck6\", \"POINT\", 10, 10).OK(),\n\t\tDo(\"CONFIG\", \"SET\", \"requirepass\", \"1234\").OK(),\n\t\tDo(\"AUTH\", \"1234\").OK(),\n\t)\n\tif err != nil {\n\t\treturn err\n\t}\n\terr = mc2.DoBatch(\n\t\tDo(\"SET\", \"mykey2\", \"truck1\", \"POINT\", 10, 10).OK(),\n\t\tDo(\"SET\", \"mykey2\", \"truck2\", \"POINT\", 10, 10).OK(),\n\t\tDo(\"GET\", \"mykey2\", \"truck1\").Str(`{\"type\":\"Point\",\"coordinates\":[10,10]}`),\n\t\tDo(\"GET\", \"mykey2\", \"truck2\").Str(`{\"type\":\"Point\",\"coordinates\":[10,10]}`),\n\n\t\tDo(\"CONFIG\", \"SET\", \"leaderauth\", \"1234\").OK(),\n\t\tDo(\"FOLLOW\", \"localhost\", mc.port).OK(),\n\t\tDo(\"GET\", \"mykey\", \"truck1\").Err(\"catching up to leader\"),\n\t\tSleep(time.Second/2),\n\n\t\tDo(\"GET\", \"mykey\", \"truck1\").Err(`{\"type\":\"Point\",\"coordinates\":[10,10]}`),\n\t\tDo(\"GET\", \"mykey\", \"truck2\").Err(`{\"type\":\"Point\",\"coordinates\":[10,10]}`),\n\t)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\terr = mc.DoBatch(\n\t\tDo(\"SET\", \"mykey\", \"truck7\", \"POINT\", 10, 10).OK(),\n\t\tDo(\"SET\", \"mykey\", \"truck8\", \"POINT\", 10, 10).OK(),\n\t\tDo(\"SET\", \"mykey\", \"truck9\", \"POINT\", 10, 10).OK(),\n\t)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\terr = mc2.DoBatch(\n\t\tSleep(time.Second/2),\n\t\tDo(\"GET\", \"mykey\", \"truck7\").Str(`{\"type\":\"Point\",\"coordinates\":[10,10]}`),\n\t\tDo(\"GET\", \"mykey\", \"truck8\").Str(`{\"type\":\"Point\",\"coordinates\":[10,10]}`),\n\t\tDo(\"GET\", \"mykey\", \"truck9\").Str(`{\"type\":\"Point\",\"coordinates\":[10,10]}`),\n\t)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\treturn nil\n}\n" }, { "path": "tests/json_test.go", "content": "package tests\n\nfunc subTestJSON(g *testGroup) {\n\tg.regSubTest(\"basic\", json_JSET_basic_test)\n\tg.regSubTest(\"geojson\", json_JSET_geojson_test)\n\tg.regSubTest(\"number\", json_JSET_number_test)\n\n}\nfunc json_JSET_basic_test(mc *mockServer) error {\n\treturn mc.DoBatch([][]interface{}{\n\t\t{\"JSET\", \"mykey\", \"myid1\", \"hello\", \"world\"}, {\"OK\"},\n\t\t{\"JGET\", \"mykey\", \"myid1\"}, {`{\"hello\":\"world\"}`},\n\t\t{\"JSET\", \"mykey\", \"myid1\", \"hello\", \"planet\"}, {\"OK\"},\n\t\t{\"JGET\", \"mykey\", \"myid1\"}, {`{\"hello\":\"planet\"}`},\n\t\t{\"JSET\", \"mykey\", \"myid1\", \"user.name.last\", \"tom\"}, {\"OK\"},\n\t\t{\"JSET\", \"mykey\", \"myid1\", \"user.name.first\", \"andrew\"}, {\"OK\"},\n\t\t{\"JGET\", \"mykey\", \"myid1\"}, {`{\"hello\":\"planet\",\"user\":{\"name\":{\"last\":\"tom\",\"first\":\"andrew\"}}}`},\n\t\t{\"JDEL\", \"mykey\", \"myid1\", \"user.name.last\"}, {1},\n\t\t{\"JGET\", \"mykey\", \"myid1\"}, {`{\"hello\":\"planet\",\"user\":{\"name\":{\"first\":\"andrew\"}}}`},\n\t\t{\"JDEL\", \"mykey\", \"myid1\", \"user.name.last\"}, {0},\n\t\t{\"JGET\", \"mykey\", \"myid1\"}, {`{\"hello\":\"planet\",\"user\":{\"name\":{\"first\":\"andrew\"}}}`},\n\t\t{\"JDEL\", \"mykey2\", \"myid1\", \"user.name.last\"}, {0},\n\t})\n}\n\nfunc json_JSET_geojson_test(mc *mockServer) error {\n\treturn mc.DoBatch([][]interface{}{\n\t\t{\"SET\", \"mykey\", \"myid1\", \"POINT\", 33, -115}, {\"OK\"},\n\t\t{\"JGET\", \"mykey\", \"myid1\"}, {`{\"type\":\"Point\",\"coordinates\":[-115,33]}`},\n\t\t{\"JSET\", \"mykey\", \"myid1\", \"coordinates.1\", 44}, {\"OK\"},\n\t\t{\"JGET\", \"mykey\", \"myid1\"}, {`{\"type\":\"Point\",\"coordinates\":[-115,44]}`},\n\t\t{\"SET\", \"mykey\", \"myid1\", \"OBJECT\", `{\"type\":\"Feature\",\"geometry\":{\"type\":\"Point\",\"coordinates\":[-115,44]}}`}, {\"OK\"},\n\t\t{\"JGET\", \"mykey\", \"myid1\"}, {`{\"type\":\"Feature\",\"geometry\":{\"type\":\"Point\",\"coordinates\":[-115,44]},\"properties\":{}}`},\n\t\t{\"JGET\", \"mykey\", \"myid1\", \"geometry.type\"}, {\"Point\"},\n\t\t{\"JSET\", \"mykey\", \"myid1\", \"properties.tags.-1\", \"southwest\"}, {\"OK\"},\n\t\t{\"JSET\", \"mykey\", \"myid1\", \"properties.tags.-1\", \"united states\"}, {\"OK\"},\n\t\t{\"JSET\", \"mykey\", \"myid1\", \"properties.tags.-1\", \"hot\"}, {\"OK\"},\n\t\t{\"JGET\", \"mykey\", \"myid1\"}, {`{\"type\":\"Feature\",\"geometry\":{\"type\":\"Point\",\"coordinates\":[-115,44]},\"properties\":{\"tags\":[\"southwest\",\"united states\",\"hot\"]}}`},\n\t\t{\"JDEL\", \"mykey\", \"myid1\", \"type\"}, {\"ERR missing type\"},\n\t})\n}\n\nfunc json_JSET_number_test(mc *mockServer) error {\n\treturn mc.DoBatch([][]interface{}{\n\t\t{\"JSET\", \"mykey\", \"myid1\", \"hello\", \"0\"}, {\"OK\"},\n\t\t{\"JGET\", \"mykey\", \"myid1\"}, {`{\"hello\":0}`},\n\t\t{\"JSET\", \"mykey\", \"myid1\", \"hello\", \"0123\"}, {\"OK\"},\n\t\t{\"JGET\", \"mykey\", \"myid1\"}, {`{\"hello\":\"0123\"}`},\n\t\t{\"JSET\", \"mykey\", \"myid1\", \"hello\", \"3.14\"}, {\"OK\"},\n\t\t{\"JGET\", \"mykey\", \"myid1\"}, {`{\"hello\":3.14}`},\n\t\t{\"JSET\", \"mykey\", \"myid1\", \"hello\", \"1.0e10\"}, {\"OK\"},\n\t\t{\"JGET\", \"mykey\", \"myid1\"}, {`{\"hello\":1.0e10}`},\n\t})\n}\n" }, { "path": "tests/keys_search_test.go", "content": "package tests\n\nimport (\n\t\"errors\"\n\t\"fmt\"\n\t\"math\"\n\t\"math/rand\"\n\t\"sort\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/gomodule/redigo/redis\"\n\t\"github.com/tidwall/gjson\"\n)\n\nfunc subTestSearch(g *testGroup) {\n\tg.regSubTest(\"KNN_BASIC\", keys_KNN_basic_test)\n\tg.regSubTest(\"KNN_RANDOM\", keys_KNN_random_test)\n\tg.regSubTest(\"KNN_CURSOR\", keys_KNN_cursor_test)\n\tg.regSubTest(\"NEARBY_SPARSE\", keys_NEARBY_SPARSE_test)\n\tg.regSubTest(\"WITHIN_CIRCLE\", keys_WITHIN_CIRCLE_test)\n\tg.regSubTest(\"WITHIN_SECTOR\", keys_WITHIN_SECTOR_test)\n\tg.regSubTest(\"INTERSECTS_CIRCLE\", keys_INTERSECTS_CIRCLE_test)\n\tg.regSubTest(\"INTERSECTS_SECTOR\", keys_INTERSECTS_SECTOR_test)\n\tg.regSubTest(\"WITHIN\", keys_WITHIN_test)\n\tg.regSubTest(\"WITHIN_CURSOR\", keys_WITHIN_CURSOR_test)\n\tg.regSubTest(\"WITHIN_CLIPBY\", keys_WITHIN_CLIPBY_test)\n\tg.regSubTest(\"INTERSECTS\", keys_INTERSECTS_test)\n\tg.regSubTest(\"INTERSECTS_CURSOR\", keys_INTERSECTS_CURSOR_test)\n\tg.regSubTest(\"INTERSECTS_CLIPBY\", keys_INTERSECTS_CLIPBY_test)\n\tg.regSubTest(\"SCAN_CURSOR\", keys_SCAN_CURSOR_test)\n\tg.regSubTest(\"SEARCH_CURSOR\", keys_SEARCH_CURSOR_test)\n\tg.regSubTest(\"MATCH\", keys_MATCH_test)\n\tg.regSubTest(\"FIELDS\", keys_FIELDS_search_test)\n\tg.regSubTest(\"BUFFER\", keys_BUFFER_search_test)\n}\n\nfunc keys_KNN_basic_test(mc *mockServer) error {\n\treturn mc.DoBatch([][]interface{}{\n\t\t{\"SET\", \"mykey\", \"1\", \"POINT\", 5, 5}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"2\", \"POINT\", 19, 19}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"3\", \"POINT\", 12, 19}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"4\", \"POINT\", -5, 5}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"5\", \"POINT\", 33, 21}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"6\", \"POINT\", 52, 13}, {\"OK\"},\n\t\t{\"NEARBY\", \"mykey\", \"LIMIT\", 10, \"POINTS\", \"POINT\", 20, 20}, {\n\t\t\t\"[0 [[2 [19 19]] [3 [12 19]] [5 [33 21]] [1 [5 5]] [4 [-5 5]] [6 [52 13]]]]\"},\n\t\t{\"NEARBY\", \"mykey\", \"LIMIT\", 10, \"IDS\", \"POINT\", 20, 20, 4000000}, {\"[0 [2 3 5 1 4 6]]\"},\n\t\t{\"NEARBY\", \"mykey\", \"LIMIT\", 10, \"DISTANCE\", \"IDS\", \"POINT\", 20, 20, 1500000}, {\"[0 [[2 152808.67164037024] [3 895945.1409106688] [5 1448929.5916252395]]]\"},\n\t\t{\"NEARBY\", \"mykey\", \"LIMIT\", 10, \"DISTANCE\", \"POINT\", 52, 13, 100}, {`[0 [[6 {\"type\":\"Point\",\"coordinates\":[13,52]} 0]]]`},\n\t\t{\"NEARBY\", \"mykey\", \"LIMIT\", 10, \"POINT\", 52.1, 13.1, 100000}, {`[0 [[6 {\"type\":\"Point\",\"coordinates\":[13,52]}]]]`},\n\t\t{\"OUTPUT\", \"json\"}, {func(res string) bool { return gjson.Get(res, \"ok\").Bool() }},\n\t\t{\"NEARBY\", \"mykey\", \"LIMIT\", 10, \"DISTANCE\", \"IDS\", \"POINT\", 20, 20, 1500000}, {\n\t\t\tfunc(res string) error {\n\t\t\t\tif !gjson.Get(res, \"ok\").Bool() {\n\t\t\t\t\treturn errors.New(\"not ok\")\n\t\t\t\t}\n\t\t\t\tif gjson.Get(res, \"ids.#\").Int() != 3 {\n\t\t\t\t\treturn fmt.Errorf(\"expected '%d' objects, got '%d'\", 3, gjson.Get(res, \"ids.#\").Int())\n\t\t\t\t}\n\t\t\t\tif gjson.Get(res, \"ids.#.distance|#\").Int() != 3 {\n\t\t\t\t\treturn fmt.Errorf(\"expected '%d' distances, got '%d'\", 3, gjson.Get(res, \"ids.#.distance|#\").Int())\n\t\t\t\t}\n\n\t\t\t\tfor i, d := range gjson.Get(res, \"ids.#.distance\").Array() {\n\t\t\t\t\tif d.Float() <= 0 {\n\t\t\t\t\t\treturn fmt.Errorf(\"expected all distances to be greater than 0: (%d, %f)\", i, d.Float())\n\t\t\t\t\t}\n\t\t\t\t}\n\n\t\t\t\treturn nil\n\t\t\t},\n\t\t},\n\t\t{\"NEARBY\", \"mykey\", \"LIMIT\", 10, \"DISTANCE\", \"IDS\", \"POINT\", 52, 13, 100}, {\n\t\t\tfunc(res string) error {\n\t\t\t\texpected := 0.0\n\n\t\t\t\tif !gjson.Get(res, \"ok\").Bool() {\n\t\t\t\t\treturn errors.New(\"not ok\")\n\t\t\t\t}\n\n\t\t\t\tif gjson.Get(res, \"ids.0.distance\").Float() != expected {\n\t\t\t\t\treturn fmt.Errorf(\"expected '%f' distances, got '%f'\", expected, gjson.Get(res, \"ids.0.distance\").Float())\n\t\t\t\t}\n\n\t\t\t\treturn nil\n\t\t\t},\n\t\t},\n\t})\n}\n\nfunc keys_KNN_random_test(mc *mockServer) error {\n\n\t// do random points\n\tmc.Do(\"OUTPUT\", \"resp\")\n\tmc.Do(\"DROP\", \"points\")\n\tdefer mc.Do(\"DROP\", \"points\")\n\n\tseed := time.Now().UnixNano()\n\t// seed = 98123098\n\trng := rand.New(rand.NewSource(seed))\n\trpoint := func() [2]float64 {\n\t\treturn [2]float64{\n\t\t\trng.Float64()*360 - 180,\n\t\t\trng.Float64()*180 - 90,\n\t\t}\n\t}\n\tN := 5000\n\tpoints := make([][2]float64, N)\n\tfor i := 0; i < len(points); i++ {\n\t\tpoints[i] = rpoint()\n\t\tres, err := redis.String(mc.Do(\"SET\", \"points\", i, \"POINT\", points[i][1], points[i][0]))\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tif res != \"OK\" {\n\t\t\treturn fmt.Errorf(\"expected 'OK', got '%s'\", res)\n\t\t}\n\t}\n\ttarget := rpoint()\n\n\tmc.Do(\"OUTPUT\", \"json\")\n\tdefer mc.Do(\"OUTPUT\", \"resp\")\n\n\tres, err := redis.String(mc.Do(\"NEARBY\", \"points\", \"LIMIT\", N, \"POINT\", target[1], target[0]))\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tldist := math.Inf(-1)\n\tfor _, dist := range gjson.Get(res, \"objects.#.distance\").Array() {\n\t\tif ldist > dist.Float() {\n\t\t\treturn fmt.Errorf(\"out of order\")\n\t\t}\n\t\tldist = dist.Float()\n\t}\n\treturn nil\n}\n\nfunc keys_KNN_cursor_test(mc *mockServer) error {\n\treturn mc.DoBatch([][]interface{}{\n\t\t{\"SET\", \"mykey\", \"1\", \"FIELD\", \"foo\", 5.5, \"POINT\", 5, 5}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"2\", \"FIELD\", \"foo\", 19.19, \"POINT\", 19, 19}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"3\", \"FIELD\", \"foo\", 12.19, \"POINT\", 12, 19}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"4\", \"FIELD\", \"foo\", -5.5, \"POINT\", -5, 5}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"5\", \"FIELD\", \"foo\", 13.21, \"POINT\", 33, 21}, {\"OK\"},\n\t\t{\"NEARBY\", \"mykey\", \"LIMIT\", 2, \"POINTS\", \"POINT\", 20, 20}, {\n\t\t\t\"[2 [[2 [19 19] [foo 19.19]] [3 [12 19] [foo 12.19]]]]\"},\n\t\t{\"NEARBY\", \"mykey\", \"CURSOR\", 2, \"LIMIT\", 1, \"POINTS\", \"POINT\", 20, 20}, {\n\t\t\t\"[3 [[5 [33 21] [foo 13.21]]]]\"},\n\t\t{\"NEARBY\", \"mykey\", \"LIMIT\", 2, \"WHERE\", \"foo\", -10, 15, \"POINTS\", \"POINT\", 20, 20}, {\n\t\t\t\"[3 [[3 [12 19] [foo 12.19]] [5 [33 21] [foo 13.21]]]]\"},\n\t\t{\"NEARBY\", \"mykey\", \"CURSOR\", 3, \"LIMIT\", 1, \"WHERE\", \"foo\", -10, 15, \"POINTS\", \"POINT\", 20, 20}, {\n\t\t\t\"[4 [[1 [5 5] [foo 5.5]]]]\"},\n\t\t{\"NEARBY\", \"mykey\", \"CURSOR\", 4, \"LIMIT\", 1, \"WHERE\", \"foo\", -10, 15, \"POINTS\", \"POINT\", 20, 20}, {\n\t\t\t\"[5 [[4 [-5 5] [foo -5.5]]]]\"},\n\t\t{\"NEARBY\", \"mykey\", \"CURSOR\", 4, \"LIMIT\", 10, \"WHERE\", \"foo\", -10, 15, \"POINTS\", \"POINT\", 20, 20}, {\n\t\t\t\"[0 [[4 [-5 5] [foo -5.5]]]]\"},\n\t})\n}\n\nfunc keys_WITHIN_test(mc *mockServer) error {\n\treturn mc.DoBatch([][]interface{}{\n\t\t{\"SET\", \"mykey\", \"point1\", \"POINT\", 37.7335, -122.4412}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"point2\", \"POINT\", 37.7335, -122.44121}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"line3\", \"OBJECT\", `{\"type\":\"LineString\",\"coordinates\":[[-122.4408378,37.7341129],[-122.4408378,37.733]]}`}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"poly4\", \"OBJECT\", `{\"type\":\"Polygon\",\"coordinates\":[[[-122.4408378,37.7341129],[-122.4408378,37.733],[-122.44,37.733],[-122.44,37.7341129],[-122.4408378,37.7341129]]]}`}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"multipoly5\", \"OBJECT\", `{\"type\":\"MultiPolygon\",\"coordinates\":[[[[-122.4408378,37.7341129],[-122.4408378,37.733],[-122.44,37.733],[-122.44,37.7341129],[-122.4408378,37.7341129]]],[[[-122.44091033935547,37.731981251280985],[-122.43994474411011,37.731981251280985],[-122.43994474411011,37.73254976045042],[-122.44091033935547,37.73254976045042],[-122.44091033935547,37.731981251280985]]]]}`}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"point6\", \"POINT\", -5, 5}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"point7\", \"POINT\", 33, 21}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"poly8\", \"OBJECT\", `{\"type\":\"Polygon\",\"coordinates\":[[[-122.4408378,37.7341129],[-122.4408378,37.733],[-122.44,37.733],[-122.44,37.7341129],[-122.4408378,37.7341129]],[[-122.44060993194579,37.73345766902749],[-122.44044363498686,37.73345766902749],[-122.44044363498686,37.73355524732416],[-122.44060993194579,37.73355524732416],[-122.44060993194579,37.73345766902749]],[[-122.44060724973677,37.7336888869566],[-122.4402102828026,37.7336888869566],[-122.4402102828026,37.7339752567853],[-122.44060724973677,37.7339752567853],[-122.44060724973677,37.7336888869566]]]}`}, {\"OK\"},\n\t\t{\"WITHIN\", \"mykey\", \"IDS\", \"OBJECT\",\n\t\t\t`{\n\t\t\t\t\"type\": \"Polygon\",\n\t\t\t\t\"coordinates\": [\n\t\t\t\t\t[\n\t\t\t\t\t\t[-122.44126439094543,37.72906137107],\n\t\t\t\t\t\t[-122.43980526924135,37.72906137107],\n\t\t\t\t\t\t[-122.43980526924135,37.73421283683962],\n\t\t\t\t\t\t[-122.44126439094543,37.73421283683962],\n\t\t\t\t\t\t[-122.44126439094543,37.72906137107]\n\t\t\t\t\t]\n\t\t\t\t]\n\t\t\t}`}, {\"[0 [point2 point1 multipoly5 poly8 poly4 line3]]\"},\n\t\t{\"WITHIN\", \"mykey\", \"IDS\", \"SECTOR\", \"37.72999\", \"-122.44760\", \"1000\", \"0\", \"90\"}, {\n\t\t\t\"[0 [point2 point1 multipoly5 poly8 poly4 line3]]\"},\n\t\t{\"WITHIN\", \"mykey\", \"IDS\", \"SECTOR\", \"37.72999\", \"-122.44760\", \"1000\", \"0\", \"0\"}, {\"ERR equal bearings (0 == 0), use CIRCLE instead\"},\n\n\t\t{\"SET\", \"key2\", \"poly9\", \"OBJECT\", `{\"type\":\"Polygon\",\"coordinates\":[[[-122.44037926197052,37.73313523548048],[-122.44017541408539,37.73313523548048],[-122.44017541408539,37.73336857568778],[-122.44037926197052,37.73336857568778],[-122.44037926197052,37.73313523548048]]]}`}, {\"OK\"},\n\t\t{\"SET\", \"key2\", \"poly10\", \"OBJECT\", `{\"type\":\"Polygon\",\"coordinates\":[[[-122.44040071964262,37.73359343010089],[-122.4402666091919,37.73359343010089],[-122.4402666091919,37.73373767596864],[-122.44040071964262,37.73373767596864],[-122.44040071964262,37.73359343010089]]]}`}, {\"OK\"},\n\t\t{\"WITHIN\", \"key2\", \"IDS\", \"GET\", \"mykey\", \"poly8\"}, {\"[0 [poly9]]\"},\n\n\t\t{\"SET\", \"key3\", \"poly11\", \"OBJECT\", `{\"type\":\"Polygon\",\"coordinates\":[[[-122.44059920310973,37.733279482240874],[-122.4402344226837,37.733279482240874],[-122.4402344226837,37.73375464605226],[-122.44059920310973,37.73375464605226],[-122.44059920310973,37.733279482240874]]]}`}, {\"OK\"},\n\t\t{\"SET\", \"key3\", \"poly12\", \"OBJECT\", `{\"type\":\"Polygon\",\"coordinates\":[[[-122.44060993194579,37.73238005667773],[-122.44013786315918,37.73238005667773],[-122.44013786315918,37.73316917591997],[-122.44060993194579,37.73316917591997],[-122.44060993194579,37.73238005667773]]]}`}, {\"OK\"},\n\t\t{\"WITHIN\", \"key3\", \"IDS\", \"GET\", \"mykey\", \"multipoly5\"}, {\"[0 [poly11]]\"},\n\n\t\t{\"SET\", \"key5\", \"poly13\", \"OBJECT\", `{\"type\":\"Polygon\",\"coordinates\":[[[-122.44073867797852,37.733211601447465],[-122.44011640548705,37.733211601447465],[-122.44011640548705,37.7340516218859],[-122.44073867797852,37.7340516218859],[-122.44073867797852,37.733211601447465]],[[-122.44060993194579,37.73345766902749],[-122.44060993194579,37.73355524732416],[-122.44044363498686,37.73355524732416],[-122.44044363498686,37.73345766902749],[-122.44060993194579,37.73345766902749]],[[-122.44060724973677,37.7336888869566],[-122.44060724973677,37.7339752567853],[-122.4402102828026,37.7339752567853],[-122.4402102828026,37.7336888869566],[-122.44060724973677,37.7336888869566]]]}`}, {\"OK\"},\n\t\t{\"SET\", \"key5\", \"poly14\", \"OBJECT\", `{\"type\":\"Polygon\",\"coordinates\":[[[-122.44154334068298,37.73179457567642],[-122.43935465812682,37.73179457567642],[-122.43935465812682,37.7343740514423],[-122.44154334068298,37.7343740514423],[-122.44154334068298,37.73179457567642]],[[-122.44104981422423,37.73286371140448],[-122.44104981422423,37.73424677678513],[-122.43990182876587,37.73424677678513],[-122.43990182876587,37.73286371140448],[-122.44104981422423,37.73286371140448]],[[-122.44109272956847,37.731870943026074],[-122.43976235389708,37.731870943026074],[-122.43976235389708,37.7326855231885],[-122.44109272956847,37.7326855231885],[-122.44109272956847,37.731870943026074]]]}`}, {\"OK\"},\n\t\t{\"WITHIN\", \"key5\", \"IDS\", \"GET\", \"mykey\", \"multipoly5\"}, {\"[0 [poly13]]\"},\n\n\t\t{\"SET\", \"key6\", \"multipoly5\", \"OBJECT\", `{\"type\":\"MultiPolygon\",\"coordinates\":[[[[-122.4408378,37.7341129],[-122.4408378,37.733],[-122.44,37.733],[-122.44,37.7341129],[-122.4408378,37.7341129]]],[[[-122.44091033935547,37.731981251280985],[-122.43994474411011,37.731981251280985],[-122.43994474411011,37.73254976045042],[-122.44091033935547,37.73254976045042],[-122.44091033935547,37.731981251280985]]]]}`}, {\"OK\"},\n\t\t{\"SET\", \"key6\", \"poly13\", \"OBJECT\", `{\"type\":\"Polygon\",\"coordinates\":[[[-122.44073867797852,37.733211601447465],[-122.44011640548705,37.733211601447465],[-122.44011640548705,37.7340516218859],[-122.44073867797852,37.7340516218859],[-122.44073867797852,37.733211601447465]],[[-122.44060993194579,37.73345766902749],[-122.44060993194579,37.73355524732416],[-122.44044363498686,37.73355524732416],[-122.44044363498686,37.73345766902749],[-122.44060993194579,37.73345766902749]],[[-122.44060724973677,37.7336888869566],[-122.44060724973677,37.7339752567853],[-122.4402102828026,37.7339752567853],[-122.4402102828026,37.7336888869566],[-122.44060724973677,37.7336888869566]]]}`}, {\"OK\"},\n\t\t{\"WITHIN\", \"key6\", \"IDS\", \"GET\", \"key5\", \"poly14\"}, {\"[0 []]\"},\n\n\t\t{\"SET\", \"key7\", \"multipoly15\", \"OBJECT\", `{\"type\":\"MultiPolygon\",\"coordinates\":[[[[-122.44056701660155,37.7332964524295],[-122.44029879570007,37.7332964524295],[-122.44029879570007,37.73375464605226],[-122.44056701660155,37.73375464605226],[-122.44056701660155,37.7332964524295]]],[[[-122.44067430496216,37.73217641163713],[-122.44034171104431,37.73217641163713],[-122.44034171104431,37.732430967850384],[-122.44067430496216,37.732430967850384],[-122.44067430496216,37.73217641163713]]]]}`}, {\"OK\"},\n\t\t{\"SET\", \"key7\", \"multipoly16\", \"OBJECT\", `{\"type\":\"MultiPolygon\",\"coordinates\":[[[[-122.44056701660155,37.7332964524295],[-122.44029879570007,37.7332964524295],[-122.44029879570007,37.73375464605226],[-122.44056701660155,37.73375464605226],[-122.44056701660155,37.7332964524295]]],[[[-122.4402666091919,37.733109780140644],[-122.4401271343231,37.733109780140644],[-122.4401271343231,37.73323705675229],[-122.4402666091919,37.73323705675229],[-122.4402666091919,37.733109780140644]]]]}`}, {\"OK\"},\n\t\t{\"SET\", \"key7\", \"multipoly17\", \"OBJECT\", `{\"type\":\"MultiPolygon\",\"coordinates\":[[[[-122.44056701660155,37.7332964524295],[-122.44029879570007,37.7332964524295],[-122.44029879570007,37.73375464605226],[-122.44056701660155,37.73375464605226],[-122.44056701660155,37.7332964524295]]],[[[-122.44032025337218,37.73267703802467],[-122.44013786315918,37.73267703802467],[-122.44013786315918,37.732838255971316],[-122.44032025337218,37.732838255971316],[-122.44032025337218,37.73267703802467]]]]}`}, {\"OK\"},\n\t\t{\"WITHIN\", \"key7\", \"IDS\", \"GET\", \"mykey\", \"multipoly5\"}, {\"[0 [multipoly15 multipoly16]]\"},\n\t})\n}\n\nfunc keys_WITHIN_CURSOR_test(mc *mockServer) error {\n\ttestArea := `{\n\t\t\"type\": \"Polygon\",\n\t\t\t\"coordinates\": [\n\t\t\t\t[\n\t\t\t\t\t[-122.44126439094543,37.72906137107],\n\t\t\t\t\t[-122.43980526924135,37.72906137107],\n\t\t\t\t\t[-122.43980526924135,37.73421283683962],\n\t\t\t\t\t[-122.44126439094543,37.73421283683962],\n\t\t\t\t\t[-122.44126439094543,37.72906137107]\n\t\t\t\t]\n\t\t\t]\n\t\t}`\n\treturn mc.DoBatch([][]interface{}{\n\t\t{\"SET\", \"mykey\", \"point1\", \"FIELD\", \"foo\", 1, \"POINT\", 37.7335, -122.4412}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"point2\", \"FIELD\", \"foo\", 2, \"POINT\", 37.7335, -122.44121}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"line3\", \"FIELD\", \"foo\", 3, \"OBJECT\", `{\"type\":\"LineString\",\"coordinates\":[[-122.4408378,37.7341129],[-122.4408378,37.733]]}`}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"poly4\", \"FIELD\", \"foo\", 4, \"OBJECT\", `{\"type\":\"Polygon\",\"coordinates\":[[[-122.4408378,37.7341129],[-122.4408378,37.733],[-122.44,37.733],[-122.44,37.7341129],[-122.4408378,37.7341129]]]}`}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"multipoly5\", \"FIELD\", \"foo\", 5, \"OBJECT\", `{\"type\":\"MultiPolygon\",\"coordinates\":[[[[-122.4408378,37.7341129],[-122.4408378,37.733],[-122.44,37.733],[-122.44,37.7341129],[-122.4408378,37.7341129]]],[[[-122.44091033935547,37.731981251280985],[-122.43994474411011,37.731981251280985],[-122.43994474411011,37.73254976045042],[-122.44091033935547,37.73254976045042],[-122.44091033935547,37.731981251280985]]]]}`}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"point6\", \"FIELD\", \"foo\", 6, \"POINT\", -5, 5}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"point7\", \"FIELD\", \"foo\", 7, \"POINT\", 33, 21}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"poly8\", \"FIELD\", \"foo\", 8, \"OBJECT\", `{\"type\":\"Polygon\",\"coordinates\":[[[-122.4408378,37.7341129],[-122.4408378,37.733],[-122.44,37.733],[-122.44,37.7341129],[-122.4408378,37.7341129]],[[-122.44060993194579,37.73345766902749],[-122.44044363498686,37.73345766902749],[-122.44044363498686,37.73355524732416],[-122.44060993194579,37.73355524732416],[-122.44060993194579,37.73345766902749]],[[-122.44060724973677,37.7336888869566],[-122.4402102828026,37.7336888869566],[-122.4402102828026,37.7339752567853],[-122.44060724973677,37.7339752567853],[-122.44060724973677,37.7336888869566]]]}`}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"point9\", \"FIELD\", \"foo\", 9, \"POINT\", 37.7335, -122.4412}, {\"OK\"},\n\t\t{\"WITHIN\", \"mykey\", \"LIMIT\", 3, \"IDS\", \"OBJECT\", testArea}, {\n\t\t\t\"[3 [point2 point9 point1]]\"},\n\t\t{\"WITHIN\", \"mykey\", \"CURSOR\", 3, \"LIMIT\", 3, \"IDS\", \"OBJECT\", testArea}, {\n\t\t\t\"[6 [multipoly5 poly8 poly4]]\"},\n\t\t{\"WITHIN\", \"mykey\", \"WHERE\", \"foo\", 3, 5, \"IDS\", \"OBJECT\", testArea}, {\n\t\t\t\"[0 [multipoly5 poly4 line3]]\"},\n\t\t{\"WITHIN\", \"mykey\", \"LIMIT\", 1, \"WHERE\", \"foo\", 3, 5, \"IDS\", \"OBJECT\", testArea}, {\n\t\t\t\"[4 [multipoly5]]\"},\n\t\t{\"WITHIN\", \"mykey\", \"CURSOR\", 0, \"LIMIT\", 1, \"WHERE\", \"foo\", 8, 9, \"IDS\", \"OBJECT\", testArea}, {\n\t\t\t\"[2 [point9]]\"},\n\t\t{\"WITHIN\", \"mykey\", \"CURSOR\", 1, \"LIMIT\", 1, \"WHERE\", \"foo\", 8, 9, \"IDS\", \"OBJECT\", testArea}, {\n\t\t\t\"[2 [point9]]\"},\n\t\t{\"WITHIN\", \"mykey\", \"CURSOR\", 2, \"LIMIT\", 1, \"WHERE\", \"foo\", 8, 9, \"IDS\", \"OBJECT\", testArea}, {\n\t\t\t\"[5 [poly8]]\"},\n\t\t{\"WITHIN\", \"mykey\", \"CURSOR\", 3, \"LIMIT\", 1, \"WHERE\", \"foo\", 8, 9, \"IDS\", \"OBJECT\", testArea}, {\n\t\t\t\"[5 [poly8]]\"},\n\t\t{\"WITHIN\", \"mykey\", \"CURSOR\", 4, \"LIMIT\", 1, \"WHERE\", \"foo\", 8, 9, \"IDS\", \"OBJECT\", testArea}, {\n\t\t\t\"[5 [poly8]]\"},\n\t\t{\"WITHIN\", \"mykey\", \"CURSOR\", 5, \"LIMIT\", 1, \"WHERE\", \"foo\", 8, 9, \"IDS\", \"OBJECT\", testArea}, {\n\t\t\t\"[0 []]\"},\n\t})\n}\n\nfunc keys_WITHIN_CLIPBY_test(mc *mockServer) error {\n\tjagged := `{\n\t\t\"type\":\"Polygon\",\n\t\t\"coordinates\":[[\n\t\t\t[-122.47781753540039,37.74655746554895],\n\t\t\t[-122.48777389526366,37.7355619376922],\n\t\t\t[-122.4707794189453,37.73271097867418],\n\t\t\t[-122.46528625488281,37.735969208590504],\n\t\t\t[-122.45189666748047,37.73922729512254],\n\t\t\t[-122.4565315246582,37.75008654795525],\n\t\t\t[-122.46683120727538,37.75307256315459],\n\t\t\t[-122.47781753540039,37.74655746554895]\n\t\t]]\n\t}`\n\n\treturn mc.DoBatch([][]interface{}{\n\t\t{\"SET\", \"mykey\", \"point1\", \"FIELD\", \"foo\", 1, \"POINT\", 37.73963454585715, -122.4810791015625}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"point2\", \"FIELD\", \"foo\", 2, \"POINT\", 37.75130811419222, -122.47438430786133}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"point3\", \"FIELD\", \"foo\", 1, \"POINT\", 37.74816932695052, -122.47713088989258}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"point4\", \"FIELD\", \"foo\", 2, \"POINT\", 37.74503040657439, -122.47571468353271}, {\"OK\"},\n\t\t{\"SET\", \"other\", \"jagged\", \"OBJECT\", jagged}, {\"OK\"},\n\n\t\t{\"WITHIN\", \"mykey\", \"IDS\", \"GET\", \"other\", \"jagged\"}, {\"[0 [point1 point4]]\"},\n\t\t{\"WITHIN\", \"mykey\", \"IDS\", \"BOUNDS\",\n\t\t\t37.737734023260884, -122.47816085815431, 37.74886496155229, -122.45464324951172,\n\t\t}, {\"[0 [point3 point4]]\"},\n\t\t{\"WITHIN\", \"mykey\", \"IDS\", \"GET\", \"other\", \"jagged\", \"CLIPBY\", \"BOUNDS\",\n\t\t\t37.737734023260884, -122.47816085815431, 37.74886496155229, -122.45464324951172,\n\t\t}, {\"[0 [point4]]\"},\n\t\t{\"WITHIN\", \"mykey\", \"IDS\", \"BOUNDS\",\n\t\t\t37.74411415606583, -122.48034954071045, 37.7536833241461, -122.47163772583008,\n\t\t}, {\"[0 [point3 point4 point2]]\"},\n\t\t{\"WITHIN\", \"mykey\", \"IDS\", \"GET\", \"other\", \"jagged\", \"CLIPBY\", \"BOUNDS\",\n\t\t\t37.74411415606583, -122.48034954071045, 37.7536833241461, -122.47163772583008,\n\t\t}, {\"[0 [point4]]\"},\n\t\t{\"WITHIN\", \"mykey\", \"IDS\", \"GET\", \"other\", \"jagged\",\n\t\t\t\"CLIPBY\", \"BOUNDS\",\n\t\t\t37.74411415606583, -122.48034954071045, 37.7536833241461, -122.47163772583008,\n\t\t\t\"CLIPBY\", \"BOUNDS\",\n\t\t\t37.737734023260884, -122.47816085815431, 37.74886496155229, -122.45464324951172,\n\t\t}, {\"[0 [point4]]\"},\n\t})\n}\n\nfunc keys_INTERSECTS_test(mc *mockServer) error {\n\treturn mc.DoBatch([][]interface{}{\n\t\t{\"SET\", \"mykey\", \"point1\", \"POINT\", 37.7335, -122.4412}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"point2\", \"POINT\", 37.7335, -122.44121}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"line3\", \"OBJECT\", `{\"type\":\"LineString\",\"coordinates\":[[-122.4408378,37.7341129],[-122.4408378,37.733]]}`}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"poly4\", \"OBJECT\", `{\"type\":\"Polygon\",\"coordinates\":[[[-122.4408378,37.7341129],[-122.4408378,37.733],[-122.44,37.733],[-122.44,37.7341129],[-122.4408378,37.7341129]]]}`}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"multipoly5\", \"OBJECT\", `{\"type\":\"MultiPolygon\",\"coordinates\":[[[[-122.4408378,37.7341129],[-122.4408378,37.733],[-122.44,37.733],[-122.44,37.7341129],[-122.4408378,37.7341129]]],[[[-122.44091033935547,37.731981251280985],[-122.43994474411011,37.731981251280985],[-122.43994474411011,37.73254976045042],[-122.44091033935547,37.73254976045042],[-122.44091033935547,37.731981251280985]]]]}`}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"point6\", \"POINT\", -5, 5}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"point7\", \"POINT\", 33, 21}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"poly8\", \"OBJECT\", `{\"type\":\"Polygon\",\"coordinates\":[[[-122.4408378,37.7341129],[-122.4408378,37.733],[-122.44,37.733],[-122.44,37.7341129],[-122.4408378,37.7341129]],[[-122.44060993194579,37.73345766902749],[-122.44044363498686,37.73345766902749],[-122.44044363498686,37.73355524732416],[-122.44060993194579,37.73355524732416],[-122.44060993194579,37.73345766902749]],[[-122.44060724973677,37.7336888869566],[-122.4402102828026,37.7336888869566],[-122.4402102828026,37.7339752567853],[-122.44060724973677,37.7339752567853],[-122.44060724973677,37.7336888869566]]]}`}, {\"OK\"},\n\t\t{\"INTERSECTS\", \"mykey\", \"IDS\", \"OBJECT\",\n\t\t\t`{\n\t\t\t\t\"type\": \"Polygon\",\n\t\t\t\t\"coordinates\": [\n\t\t\t\t\t[\n\t\t\t\t\t\t[-122.44126439094543,37.732906137107],\n\t\t\t\t\t\t[-122.43980526924135,37.732906137107],\n\t\t\t\t\t\t[-122.43980526924135,37.73421283683962],\n\t\t\t\t\t\t[-122.44126439094543,37.73421283683962],\n\t\t\t\t\t\t[-122.44126439094543,37.732906137107]\n\t\t\t\t\t]\n\t\t\t\t]\n\t\t\t}`}, {\"[0 [point2 point1 multipoly5 poly8 poly4 line3]]\"},\n\t\t{\"INTERSECTS\", \"mykey\", \"IDS\", \"SECTOR\", \"37.72999\", \"-122.44760\", \"1000\", \"0\", \"90\"}, {\n\t\t\t\"[0 [point2 point1 multipoly5 poly8 poly4 line3]]\"},\n\t\t{\"INTERSECTS\", \"mykey\", \"IDS\", \"SECTOR\", \"37.72999\", \"-122.44760\", \"1000\", \"0\", \"0\"}, {\"ERR equal bearings (0 == 0), use CIRCLE instead\"},\n\n\t\t{\"SET\", \"key2\", \"poly9\", \"OBJECT\", `{\"type\": \"Polygon\",\"coordinates\": [[[-122.44037926197052,37.73313523548048],[-122.44017541408539,37.73313523548048],[-122.44017541408539,37.73336857568778],[-122.44037926197052,37.73336857568778],[-122.44037926197052,37.73313523548048]]]}`}, {\"OK\"},\n\t\t{\"SET\", \"key2\", \"poly10\", \"OBJECT\", `{\"type\": \"Polygon\",\"coordinates\": [[[-122.44040071964262,37.73359343010089],[-122.4402666091919,37.73359343010089],[-122.4402666091919,37.73373767596864],[-122.44040071964262,37.73373767596864],[-122.44040071964262,37.73359343010089]]]}`}, {\"OK\"},\n\t\t{\"SET\", \"key2\", \"poly10.1\", \"OBJECT\", `{\"type\":\"Polygon\",\"coordinates\":[[[-122.44051605463028,37.73375464605226],[-122.44028002023695,37.73375464605226],[-122.44028002023695,37.733903134117966],[-122.44051605463028,37.733903134117966],[-122.44051605463028,37.73375464605226]]]}`}, {\"OK\"},\n\t\t{\"INTERSECTS\", \"key2\", \"IDS\", \"GET\", \"mykey\", \"poly8\"}, {\n\t\t\t\"[0 [poly10 poly9]]\"},\n\n\t\t{\"SET\", \"key3\", \"poly11\", \"OBJECT\", `{\"type\":\"Polygon\",\"coordinates\":[[[-122.44059920310973,37.733279482240874],[-122.4402344226837,37.733279482240874],[-122.4402344226837,37.73375464605226],[-122.44059920310973,37.73375464605226],[-122.44059920310973,37.733279482240874]]]}`}, {\"OK\"},\n\t\t{\"SET\", \"key3\", \"poly12\", \"OBJECT\", `{\"type\":\"Polygon\",\"coordinates\":[[[-122.44060993194579,37.73238005667773],[-122.44013786315918,37.73238005667773],[-122.44013786315918,37.73316917591997],[-122.44060993194579,37.73316917591997],[-122.44060993194579,37.73238005667773]]]}`}, {\"OK\"},\n\t\t{\"SET\", \"key3\", \"poly12.1\", \"OBJECT\", `{\"type\":\"Polygon\",\"coordinates\":[[[-122.44074940681458,37.73270249351328],[-122.44008421897887,37.73270249351328],[-122.44008421897887,37.732872196546936],[-122.44074940681458,37.732872196546936],[-122.44074940681458,37.73270249351328]]]}`}, {\"OK\"},\n\t\t{\"INTERSECTS\", \"key3\", \"IDS\", \"GET\", \"mykey\", \"multipoly5\"}, {\n\t\t\t\"[0 [poly12 poly11]]\"},\n\n\t\t{\"SET\", \"key5\", \"poly13\", \"OBJECT\", `{\"type\":\"Polygon\",\"coordinates\":[[[-122.44073867797852,37.733211601447465],[-122.44011640548705,37.733211601447465],[-122.44011640548705,37.7340516218859],[-122.44073867797852,37.7340516218859],[-122.44073867797852,37.733211601447465]],[[-122.44060993194579,37.73345766902749],[-122.44060993194579,37.73355524732416],[-122.44044363498686,37.73355524732416],[-122.44044363498686,37.73345766902749],[-122.44060993194579,37.73345766902749]],[[-122.44060724973677,37.7336888869566],[-122.44060724973677,37.7339752567853],[-122.4402102828026,37.7339752567853],[-122.4402102828026,37.7336888869566],[-122.44060724973677,37.7336888869566]]]}`}, {\"OK\"},\n\t\t{\"SET\", \"key5\", \"poly14\", \"OBJECT\", `{\"type\":\"Polygon\",\"coordinates\":[[[-122.44154334068298,37.73179457567642],[-122.43935465812682,37.73179457567642],[-122.43935465812682,37.7343740514423],[-122.44154334068298,37.7343740514423],[-122.44154334068298,37.73179457567642]],[[-122.44104981422423,37.73286371140448],[-122.44104981422423,37.73424677678513],[-122.43990182876587,37.73424677678513],[-122.43990182876587,37.73286371140448],[-122.44104981422423,37.73286371140448]],[[-122.44109272956847,37.731870943026074],[-122.43976235389708,37.731870943026074],[-122.43976235389708,37.7326855231885],[-122.44109272956847,37.7326855231885],[-122.44109272956847,37.731870943026074]]]}`}, {\"OK\"},\n\t\t{\"INTERSECTS\", \"key5\", \"IDS\", \"GET\", \"mykey\", \"multipoly5\"}, {\n\t\t\t\"[0 [poly13]]\"},\n\n\t\t{\"SET\", \"key6\", \"multipoly5\", \"OBJECT\", `{\"type\":\"MultiPolygon\",\"coordinates\":[[[[-122.4408378,37.7341129],[-122.4408378,37.733],[-122.44,37.733],[-122.44,37.7341129],[-122.4408378,37.7341129]]],[[[-122.44091033935547,37.731981251280985],[-122.43994474411011,37.731981251280985],[-122.43994474411011,37.73254976045042],[-122.44091033935547,37.73254976045042],[-122.44091033935547,37.731981251280985]]]]}`}, {\"OK\"},\n\t\t{\"SET\", \"key6\", \"poly13\", \"OBJECT\", `{\"type\":\"Polygon\",\"coordinates\":[[[-122.44073867797852,37.733211601447465],[-122.44011640548705,37.733211601447465],[-122.44011640548705,37.7340516218859],[-122.44073867797852,37.7340516218859],[-122.44073867797852,37.733211601447465]],[[-122.44060993194579,37.73345766902749],[-122.44060993194579,37.73355524732416],[-122.44044363498686,37.73355524732416],[-122.44044363498686,37.73345766902749],[-122.44060993194579,37.73345766902749]],[[-122.44060724973677,37.7336888869566],[-122.44060724973677,37.7339752567853],[-122.4402102828026,37.7339752567853],[-122.4402102828026,37.7336888869566],[-122.44060724973677,37.7336888869566]]]}`}, {\"OK\"},\n\t\t{\"INTERSECTS\", \"key6\", \"IDS\", \"GET\", \"key5\", \"poly14\"}, {\n\t\t\t\"[0 []]\"},\n\n\t\t{\"SET\", \"key7\", \"multipoly15\", \"OBJECT\", `{\"type\":\"MultiPolygon\",\"coordinates\":[[[[-122.44056701660155,37.7332964524295],[-122.44029879570007,37.7332964524295],[-122.44029879570007,37.73375464605226],[-122.44056701660155,37.73375464605226],[-122.44056701660155,37.7332964524295]]],[[[-122.44067430496216,37.73217641163713],[-122.44034171104431,37.73217641163713],[-122.44034171104431,37.732430967850384],[-122.44067430496216,37.732430967850384],[-122.44067430496216,37.73217641163713]]]]}`}, {\"OK\"},\n\t\t{\"SET\", \"key7\", \"multipoly16\", \"OBJECT\", `{\"type\":\"MultiPolygon\",\"coordinates\":[[[[-122.44056701660155,37.7332964524295],[-122.44029879570007,37.7332964524295],[-122.44029879570007,37.73375464605226],[-122.44056701660155,37.73375464605226],[-122.44056701660155,37.7332964524295]]],[[[-122.4402666091919,37.733109780140644],[-122.4401271343231,37.733109780140644],[-122.4401271343231,37.73323705675229],[-122.4402666091919,37.73323705675229],[-122.4402666091919,37.733109780140644]]]]}`}, {\"OK\"},\n\t\t{\"SET\", \"key7\", \"multipoly17\", \"OBJECT\", `{\"type\":\"MultiPolygon\",\"coordinates\":[[[[-122.44056701660155,37.7332964524295],[-122.44029879570007,37.7332964524295],[-122.44029879570007,37.73375464605226],[-122.44056701660155,37.73375464605226],[-122.44056701660155,37.7332964524295]]],[[[-122.44032025337218,37.73267703802467],[-122.44013786315918,37.73267703802467],[-122.44013786315918,37.732838255971316],[-122.44032025337218,37.732838255971316],[-122.44032025337218,37.73267703802467]]]]}`}, {\"OK\"},\n\t\t{\"SET\", \"key7\", \"multipoly17.1\", \"OBJECT\", `{\"type\":\"MultiPolygon\",\"coordinates\":[[[[-122.4407172203064,37.73270249351328],[-122.44049191474916,37.73270249351328],[-122.44049191474916,37.73286371140448],[-122.4407172203064,37.73286371140448],[-122.4407172203064,37.73270249351328]]],[[[-122.44032025337218,37.73267703802467],[-122.44013786315918,37.73267703802467],[-122.44013786315918,37.732838255971316],[-122.44032025337218,37.732838255971316],[-122.44032025337218,37.73267703802467]]]]}`}, {\"OK\"},\n\t\t{\"INTERSECTS\", \"key7\", \"IDS\", \"GET\", \"mykey\", \"multipoly5\"}, {\n\t\t\t\"[0 [multipoly15 multipoly17 multipoly16]]\"},\n\t})\n}\n\nfunc keys_INTERSECTS_CLIPBY_test(mc *mockServer) error {\n\tjagged := `{\n\t\t\"type\":\"Polygon\",\n\t\t\"coordinates\":[[\n\t\t\t[-122.47781753540039,37.74655746554895],\n\t\t\t[-122.48777389526366,37.7355619376922],\n\t\t\t[-122.4707794189453,37.73271097867418],\n\t\t\t[-122.46528625488281,37.735969208590504],\n\t\t\t[-122.45189666748047,37.73922729512254],\n\t\t\t[-122.4565315246582,37.75008654795525],\n\t\t\t[-122.46683120727538,37.75307256315459],\n\t\t\t[-122.47781753540039,37.74655746554895]\n\t\t]]\n\t}`\n\n\treturn mc.DoBatch([][]interface{}{\n\t\t{\"SET\", \"mykey\", \"point1\", \"FIELD\", \"foo\", 1, \"POINT\", 37.73963454585715, -122.4810791015625}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"point2\", \"FIELD\", \"foo\", 2, \"POINT\", 37.75130811419222, -122.47438430786133}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"point3\", \"FIELD\", \"foo\", 1, \"POINT\", 37.74816932695052, -122.47713088989258}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"point4\", \"FIELD\", \"foo\", 2, \"POINT\", 37.74503040657439, -122.47571468353271}, {\"OK\"},\n\t\t{\"SET\", \"other\", \"jagged\", \"OBJECT\", jagged}, {\"OK\"},\n\n\t\t{\"INTERSECTS\", \"mykey\", \"IDS\", \"GET\", \"other\", \"jagged\"}, {\"[0 [point1 point4]]\"},\n\t\t{\"INTERSECTS\", \"mykey\", \"IDS\", \"BOUNDS\",\n\t\t\t37.737734023260884, -122.47816085815431, 37.74886496155229, -122.45464324951172,\n\t\t}, {\"[0 [point3 point4]]\"},\n\t\t{\"INTERSECTS\", \"mykey\", \"IDS\", \"GET\", \"other\", \"jagged\", \"CLIPBY\", \"BOUNDS\",\n\t\t\t37.737734023260884, -122.47816085815431, 37.74886496155229, -122.45464324951172,\n\t\t}, {\"[0 [point4]]\"},\n\t\t{\"INTERSECTS\", \"mykey\", \"IDS\", \"BOUNDS\",\n\t\t\t37.74411415606583, -122.48034954071045, 37.7536833241461, -122.47163772583008,\n\t\t}, {\"[0 [point3 point4 point2]]\"},\n\t\t{\"INTERSECTS\", \"mykey\", \"IDS\", \"GET\", \"other\", \"jagged\", \"CLIPBY\", \"BOUNDS\",\n\t\t\t37.74411415606583, -122.48034954071045, 37.7536833241461, -122.47163772583008,\n\t\t}, {\"[0 [point4]]\"},\n\t\t{\"INTERSECTS\", \"mykey\", \"IDS\", \"GET\", \"other\", \"jagged\",\n\t\t\t\"CLIPBY\", \"BOUNDS\",\n\t\t\t37.74411415606583, -122.48034954071045, 37.7536833241461, -122.47163772583008,\n\t\t\t\"CLIPBY\", \"BOUNDS\",\n\t\t\t37.737734023260884, -122.47816085815431, 37.74886496155229, -122.45464324951172,\n\t\t}, {\"[0 [point4]]\"},\n\t})\n}\n\nfunc keys_INTERSECTS_CURSOR_test(mc *mockServer) error {\n\ttestArea := `{\n\t\t\"type\": \"Polygon\",\n\t\t\t\"coordinates\": [\n\t\t\t\t[\n\t\t\t\t\t[-122.44126439094543,37.732906137107],\n\t\t\t\t\t[-122.43980526924135,37.732906137107],\n\t\t\t\t\t[-122.43980526924135,37.73421283683962],\n\t\t\t\t\t[-122.44126439094543,37.73421283683962],\n\t\t\t\t\t[-122.44126439094543,37.732906137107]\n\t\t\t\t]\n\t\t\t]\n\t\t}`\n\treturn mc.DoBatch([][]interface{}{\n\t\t{\"SET\", \"mykey\", \"point1\", \"FIELD\", \"foo\", 1, \"POINT\", 37.7335, -122.4412}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"point2\", \"FIELD\", \"foo\", 2, \"POINT\", 37.7335, -122.44121}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"line3\", \"FIELD\", \"foo\", 3, \"OBJECT\", `{\"type\":\"LineString\",\"coordinates\":[[-122.4408378,37.7341129],[-122.4408378,37.733]]}`}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"poly4\", \"FIELD\", \"foo\", 4, \"OBJECT\", `{\"type\":\"Polygon\",\"coordinates\":[[[-122.4408378,37.7341129],[-122.4408378,37.733],[-122.44,37.733],[-122.44,37.7341129],[-122.4408378,37.7341129]]]}`}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"multipoly5\", \"FIELD\", \"foo\", 5, \"OBJECT\", `{\"type\":\"MultiPolygon\",\"coordinates\":[[[[-122.4408378,37.7341129],[-122.4408378,37.733],[-122.44,37.733],[-122.44,37.7341129],[-122.4408378,37.7341129]]],[[[-122.44091033935547,37.731981251280985],[-122.43994474411011,37.731981251280985],[-122.43994474411011,37.73254976045042],[-122.44091033935547,37.73254976045042],[-122.44091033935547,37.731981251280985]]]]}`}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"point6\", \"FIELD\", \"foo\", 6, \"POINT\", -5, 5}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"point7\", \"FIELD\", \"foo\", 7, \"POINT\", 33, 21}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"poly8\", \"FIELD\", \"foo\", 8, \"OBJECT\", `{\"type\":\"Polygon\",\"coordinates\":[[[-122.4408378,37.7341129],[-122.4408378,37.733],[-122.44,37.733],[-122.44,37.7341129],[-122.4408378,37.7341129]],[[-122.44060993194579,37.73345766902749],[-122.44044363498686,37.73345766902749],[-122.44044363498686,37.73355524732416],[-122.44060993194579,37.73355524732416],[-122.44060993194579,37.73345766902749]],[[-122.44060724973677,37.7336888869566],[-122.4402102828026,37.7336888869566],[-122.4402102828026,37.7339752567853],[-122.44060724973677,37.7339752567853],[-122.44060724973677,37.7336888869566]]]}`}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"point9\", \"FIELD\", \"foo\", 9, \"POINT\", 37.7335, -122.4412}, {\"OK\"},\n\t\t{\"INTERSECTS\", \"mykey\", \"LIMIT\", 3, \"IDS\", \"OBJECT\", testArea}, {\n\t\t\t\"[3 [point2 point9 point1]]\"},\n\t\t{\"INTERSECTS\", \"mykey\", \"CURSOR\", 3, \"LIMIT\", 3, \"IDS\", \"OBJECT\", testArea}, {\n\t\t\t\"[6 [multipoly5 poly8 poly4]]\"},\n\t\t{\"INTERSECTS\", \"mykey\", \"WHERE\", \"foo\", 3, 5, \"IDS\", \"OBJECT\", testArea}, {\n\t\t\t\"[0 [multipoly5 poly4 line3]]\"},\n\t\t{\"INTERSECTS\", \"mykey\", \"LIMIT\", 1, \"WHERE\", \"foo\", 3, 5, \"IDS\", \"OBJECT\", testArea}, {\n\t\t\t\"[4 [multipoly5]]\"},\n\t\t{\"INTERSECTS\", \"mykey\", \"CURSOR\", 1, \"LIMIT\", 1, \"WHERE\", \"foo\", 8, 9, \"IDS\", \"OBJECT\", testArea}, {\n\t\t\t\"[2 [point9]]\"},\n\t\t{\"INTERSECTS\", \"mykey\", \"CURSOR\", 2, \"LIMIT\", 1, \"WHERE\", \"foo\", 8, 9, \"IDS\", \"OBJECT\", testArea}, {\n\t\t\t\"[5 [poly8]]\"},\n\t\t{\"INTERSECTS\", \"mykey\", \"CURSOR\", 3, \"LIMIT\", 1, \"WHERE\", \"foo\", 8, 9, \"IDS\", \"OBJECT\", testArea}, {\n\t\t\t\"[5 [poly8]]\"},\n\t\t{\"INTERSECTS\", \"mykey\", \"CURSOR\", 4, \"LIMIT\", 1, \"WHERE\", \"foo\", 8, 9, \"IDS\", \"OBJECT\", testArea}, {\n\t\t\t\"[5 [poly8]]\"},\n\t\t{\"INTERSECTS\", \"mykey\", \"CURSOR\", 5, \"LIMIT\", 1, \"WHERE\", \"foo\", 8, 9, \"IDS\", \"OBJECT\", testArea}, {\n\t\t\t\"[0 []]\"},\n\t\t{\"INTERSECTS\", \"mykey\", \"CURSOR\", 6, \"LIMIT\", 1, \"WHERE\", \"foo\", 8, 9, \"IDS\", \"OBJECT\", testArea}, {\n\t\t\t\"[0 []]\"},\n\t})\n}\n\nfunc keys_WITHIN_CIRCLE_test(mc *mockServer) error {\n\treturn mc.DoBatch([][]interface{}{\n\t\t{\"SET\", \"mykey\", \"1\", \"POINT\", 37.7335, -122.4412}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"2\", \"POINT\", 37.7335, -122.44121}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"3\", \"OBJECT\", `{\"type\":\"LineString\",\"coordinates\":[[-122.4408378,37.7341129],[-122.4408378,37.733]]}`}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"4\", \"OBJECT\", `{\"type\":\"Polygon\",\"coordinates\":[[[-122.4408378,37.7341129],[-122.4408378,37.733],[-122.44,37.733],[-122.44,37.7341129],[-122.4408378,37.7341129]]]}`}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"5\", \"OBJECT\", `{\"type\":\"MultiPolygon\",\"coordinates\":[[[[-122.4408378,37.7341129],[-122.4408378,37.733],[-122.44,37.733],[-122.44,37.7341129],[-122.4408378,37.7341129]]]]}`}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"6\", \"POINT\", -5, 5}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"7\", \"POINT\", 33, 21}, {\"OK\"},\n\t\t{\"WITHIN\", \"mykey\", \"IDS\", \"CIRCLE\", 37.7335, -122.4412, 1000}, {\n\t\t\t\"[0 [2 1 5 4 3]]\"},\n\t\t{\"WITHIN\", \"mykey\", \"IDS\", \"CIRCLE\", 37.7335, -122.4412, 10}, {\n\t\t\t\"[0 [2 1]]\"},\n\t})\n}\n\nfunc keys_WITHIN_SECTOR_test(mc *mockServer) error {\n\treturn mc.DoBatch([][]interface{}{\n\t\t{\"SET\", \"mykey\", \"1\", \"POINT\", 37.7324, -122.4424}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"2\", \"POINT\", 37.73241, -122.44241}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"3\", \"OBJECT\", `{\"type\":\"LineString\",\"coordinates\":[[-122.4408378,37.7341129],[-122.4408378,37.733]]}`}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"4\", \"OBJECT\", `{\"type\":\"Polygon\",\"coordinates\":[[[-122.4408378,37.7341129],[-122.4408378,37.733],[-122.44,37.733],[-122.44,37.7341129],[-122.4408378,37.7341129]]]}`}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"5\", \"OBJECT\", `{\"type\":\"MultiPolygon\",\"coordinates\":[[[[-122.4408378,37.7341129],[-122.4408378,37.733],[-122.44,37.733],[-122.44,37.7341129],[-122.4408378,37.7341129]]]]}`}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"6\", \"POINT\", -5, 5}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"7\", \"POINT\", 33, 21}, {\"OK\"},\n\t\t{\"WITHIN\", \"mykey\", \"IDS\", \"SECTOR\", 37.731930, -122.443270, 1000, 0, 90}, {\n\t\t\t\"[0 [2 1 5 4 3]]\"},\n\t\t{\"WITHIN\", \"mykey\", \"IDS\", \"SECTOR\", 37.731930, -122.443270, 100, 0, 90}, {\n\t\t\t\"[0 [2 1]]\"},\n\t})\n}\n\nfunc keys_NEARBY_SPARSE_test(mc *mockServer) error {\n\t// https://github.com/tidwall/tile38/issues/618\n\treturn mc.DoBatch([][]interface{}{\n\t\t{\"SET\", \"location\", \"379\", \"FIELD\", \"story\", \"214\", \"POINT\", \"38.343763352486\", \"-0.48118065817742\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"380\", \"FIELD\", \"story\", \"216\", \"POINT\", \"38.343210451684\", \"-0.48164476701469\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"381\", \"FIELD\", \"story\", \"217\", \"POINT\", \"38.343548609904\", \"-0.4815616494057\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"382\", \"FIELD\", \"story\", \"219\", \"POINT\", \"38.342949723291\", \"-0.48180543529947\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"383\", \"FIELD\", \"story\", \"222\", \"POINT\", \"38.343527453662\", \"-0.48118542576204\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"384\", \"FIELD\", \"story\", \"223\", \"POINT\", \"38.342894310235\", \"-0.48121117150688\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"385\", \"FIELD\", \"story\", \"224\", \"POINT\", \"38.343665011791\", \"-0.48128042649575\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"386\", \"FIELD\", \"story\", \"226\", \"POINT\", \"38.34300663218\", \"-0.48136455958069\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"387\", \"FIELD\", \"story\", \"227\", \"POINT\", \"38.343561105586\", \"-0.48133979329476\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"388\", \"FIELD\", \"story\", \"228\", \"POINT\", \"38.343021516797\", \"-0.48111203609768\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"389\", \"FIELD\", \"story\", \"229\", \"POINT\", \"38.34377906915\", \"-0.48100754592639\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"390\", \"FIELD\", \"story\", \"230\", \"POINT\", \"38.343028862949\", \"-0.48107204744577\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"391\", \"FIELD\", \"story\", \"231\", \"POINT\", \"38.342956973955\", \"-0.48123798785545\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"392\", \"FIELD\", \"story\", \"233\", \"POINT\", \"38.343342938888\", \"-0.48181034196501\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"393\", \"FIELD\", \"story\", \"234\", \"POINT\", \"38.343323273543\", \"-0.48119160635951\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"394\", \"FIELD\", \"story\", \"235\", \"POINT\", \"38.343475947604\", \"-0.48128444286906\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"395\", \"FIELD\", \"story\", \"236\", \"POINT\", \"38.343553742872\", \"-0.48161695699988\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"396\", \"FIELD\", \"story\", \"237\", \"POINT\", \"38.343657786414\", \"-0.48109919689955\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"397\", \"FIELD\", \"story\", \"238\", \"POINT\", \"38.342934456291\", \"-0.48126912781599\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"398\", \"FIELD\", \"story\", \"239\", \"POINT\", \"38.343254792078\", \"-0.48115765613124\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"399\", \"FIELD\", \"story\", \"240\", \"POINT\", \"38.342851143141\", \"-0.48151031587298\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"400\", \"FIELD\", \"story\", \"244\", \"POINT\", \"38.343298791244\", \"-0.48121409612892\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"401\", \"FIELD\", \"story\", \"246\", \"POINT\", \"38.343436945653\", \"-0.48141198331599\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"402\", \"FIELD\", \"story\", \"248\", \"POINT\", \"38.343033046491\", \"-0.48183781756703\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"403\", \"FIELD\", \"story\", \"249\", \"POINT\", \"38.343115572723\", \"-0.48114768365296\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"404\", \"FIELD\", \"story\", \"250\", \"POINT\", \"38.343318663597\", \"-0.48120263102647\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"405\", \"FIELD\", \"story\", \"251\", \"POINT\", \"38.343434654108\", \"-0.4814578363497\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"406\", \"FIELD\", \"story\", \"252\", \"POINT\", \"38.343810655958\", \"-0.48181221112942\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"407\", \"FIELD\", \"story\", \"253\", \"POINT\", \"38.342910776509\", \"-0.48124848403503\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"408\", \"FIELD\", \"story\", \"176\", \"POINT\", \"38.343429050328\", \"-0.48134829622424\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"409\", \"FIELD\", \"story\", \"177\", \"POINT\", \"38.343375167926\", \"-0.4813182716687\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"410\", \"FIELD\", \"story\", \"178\", \"POINT\", \"38.343686937911\", \"-0.48184949541056\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"411\", \"FIELD\", \"story\", \"179\", \"POINT\", \"38.343095509246\", \"-0.48121296750565\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"412\", \"FIELD\", \"story\", \"243\", \"POINT\", \"38.343052434763\", \"-0.48133792363582\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"413\", \"FIELD\", \"story\", \"174\", \"POINT\", \"38.343556877562\", \"-0.4814408412531\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"414\", \"FIELD\", \"story\", \"182\", \"POINT\", \"38.34352896108\", \"-0.48127167080998\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"415\", \"FIELD\", \"story\", \"183\", \"POINT\", \"38.343458562741\", \"-0.48113117383504\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"416\", \"FIELD\", \"story\", \"187\", \"POINT\", \"38.343372242633\", \"-0.48198426529928\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"417\", \"FIELD\", \"story\", \"200\", \"POINT\", \"38.343365745635\", \"-0.48145747589433\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"418\", \"FIELD\", \"story\", \"206\", \"POINT\", \"38.343019183653\", \"-0.48177065402226\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"419\", \"FIELD\", \"story\", \"180\", \"POINT\", \"38.343492978961\", \"-0.48146214309728\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"420\", \"FIELD\", \"story\", \"181\", \"POINT\", \"38.343614147661\", \"-0.48178183237141\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"421\", \"FIELD\", \"story\", \"172\", \"POINT\", \"38.34365219519\", \"-0.48163252690471\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"422\", \"FIELD\", \"story\", \"193\", \"POINT\", \"38.343284579937\", \"-0.48191851957019\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"423\", \"FIELD\", \"story\", \"194\", \"POINT\", \"38.342957462369\", \"-0.48169612941468\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"424\", \"FIELD\", \"story\", \"195\", \"POINT\", \"38.343050765851\", \"-0.48189678247055\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"425\", \"FIELD\", \"story\", \"196\", \"POINT\", \"38.343767590125\", \"-0.48171070193171\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"426\", \"FIELD\", \"story\", \"197\", \"POINT\", \"38.343547519997\", \"-0.4813692941909\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"427\", \"FIELD\", \"story\", \"198\", \"POINT\", \"38.342914769086\", \"-0.48155727196514\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"428\", \"FIELD\", \"story\", \"211\", \"POINT\", \"38.342873132946\", \"-0.48120151934304\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"429\", \"FIELD\", \"story\", \"212\", \"POINT\", \"38.343776804477\", \"-0.48175041955478\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"430\", \"FIELD\", \"story\", \"218\", \"POINT\", \"38.343321288826\", \"-0.48138129717684\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"431\", \"FIELD\", \"story\", \"241\", \"POINT\", \"38.34353344767\", \"-0.4814278700903\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"432\", \"FIELD\", \"story\", \"247\", \"POINT\", \"38.34366410657\", \"-0.48163485684748\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"433\", \"FIELD\", \"story\", \"203\", \"POINT\", \"38.343237196083\", \"-0.48114844901293\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"434\", \"FIELD\", \"story\", \"204\", \"POINT\", \"38.342949966718\", \"-0.48104381934163\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"435\", \"FIELD\", \"story\", \"205\", \"POINT\", \"38.343334803169\", \"-0.48143352609798\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"436\", \"FIELD\", \"story\", \"215\", \"POINT\", \"38.343231760033\", \"-0.48177962151034\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"437\", \"FIELD\", \"story\", \"220\", \"POINT\", \"38.34381041238\", \"-0.48184807353803\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"438\", \"FIELD\", \"story\", \"232\", \"POINT\", \"38.3437321952\", \"-0.4810338033529\"}, {\"OK\"},\n\t\t{\"SET\", \"location\", \"439\", \"FIELD\", \"story\", \"221\", \"POINT\", \"38.343038197665\", \"-0.48194660158614\"}, {\"OK\"},\n\t\t{\"OUTPUT\", \"json\"}, {func(res string) bool { return gjson.Get(res, \"ok\").Bool() }},\n\t\t{\"NEARBY\", \"location\", \"SPARSE\", \"1\", \"DISTANCE\", \"POINT\", \"38.342940855731506\", \"-0.48126081948077476\", \"25\"}, {\n\t\t\t// should return 4 objects that include a \"distance\" field\n\t\t\tfunc(res string) error {\n\t\t\t\tif !gjson.Get(res, \"ok\").Bool() {\n\t\t\t\t\treturn errors.New(\"not ok\")\n\t\t\t\t}\n\t\t\t\tif gjson.Get(res, \"objects.#\").Int() != 4 {\n\t\t\t\t\treturn fmt.Errorf(\"expected '%d' objects, got '%d'\", 4, gjson.Get(res, \"objects.#\").Int())\n\t\t\t\t}\n\t\t\t\tif gjson.Get(res, \"objects.#.distance|#\").Int() != 4 {\n\t\t\t\t\treturn fmt.Errorf(\"expected '%d' distances, got '%d'\", 4, gjson.Get(res, \"objects.#.distance|#\").Int())\n\t\t\t\t}\n\t\t\t\treturn nil\n\t\t\t},\n\t\t},\n\t})\n}\n\nfunc keys_INTERSECTS_CIRCLE_test(mc *mockServer) error {\n\treturn mc.DoBatch([][]interface{}{\n\t\t{\"SET\", \"mykey\", \"1\", \"POINT\", 37.7335, -122.4412}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"2\", \"POINT\", 37.7335, -122.44121}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"3\", \"OBJECT\", `{\"type\":\"LineString\",\"coordinates\":[[-122.4408378,37.7341129],[-122.4408378,37.733]]}`}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"4\", \"OBJECT\", `{\"type\":\"Polygon\",\"coordinates\":[[[-122.4408378,37.7341129],[-122.4408378,37.733],[-122.44,37.733],[-122.44,37.7341129],[-122.4408378,37.7341129]]]}`}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"5\", \"OBJECT\", `{\"type\":\"MultiPolygon\",\"coordinates\":[[[[-122.4408378,37.7341129],[-122.4408378,37.733],[-122.44,37.733],[-122.44,37.7341129],[-122.4408378,37.7341129]]]]}`}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"6\", \"POINT\", -5, 5}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"7\", \"POINT\", 33, 21}, {\"OK\"},\n\t\t{\"INTERSECTS\", \"mykey\", \"IDS\", \"CIRCLE\", 37.7335, -122.4412, 70}, {\n\t\t\t\"[0 [2 1 5 4 3]]\"},\n\t\t{\"INTERSECTS\", \"mykey\", \"IDS\", \"CIRCLE\", 37.7335, -122.4412, 10}, {\n\t\t\t\"[0 [2 1]]\"},\n\t})\n}\n\nfunc keys_INTERSECTS_SECTOR_test(mc *mockServer) error {\n\treturn mc.DoBatch([][]interface{}{\n\t\t{\"SET\", \"mykey\", \"1\", \"POINT\", 37.7324, -122.4424}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"2\", \"POINT\", 37.73241, -122.44241}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"3\", \"OBJECT\", `{\"type\":\"LineString\",\"coordinates\":[[-122.4408378,37.7341129],[-122.4408378,37.733]]}`}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"4\", \"OBJECT\", `{\"type\":\"Polygon\",\"coordinates\":[[[-122.4408378,37.7341129],[-122.4408378,37.733],[-122.44,37.733],[-122.44,37.7341129],[-122.4408378,37.7341129]]]}`}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"5\", \"OBJECT\", `{\"type\":\"MultiPolygon\",\"coordinates\":[[[[-122.4408378,37.7341129],[-122.4408378,37.733],[-122.44,37.733],[-122.44,37.7341129],[-122.4408378,37.7341129]]]]}`}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"6\", \"POINT\", -5, 5}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"7\", \"POINT\", 33, 21}, {\"OK\"},\n\t\t{\"INTERSECTS\", \"mykey\", \"IDS\", \"SECTOR\", 37.731930, -122.443270, 1000, 0, 90}, {\n\t\t\t\"[0 [2 1 5 4 3]]\"},\n\t\t{\"INTERSECTS\", \"mykey\", \"IDS\", \"SECTOR\", 37.731930, -122.443270, 100, 0, 90}, {\n\t\t\t\"[0 [2 1]]\"},\n\t})\n}\n\nfunc keys_SCAN_CURSOR_test(mc *mockServer) error {\n\treturn mc.DoBatch([][]interface{}{\n\t\t{\"SET\", \"mykey\", \"id1\", \"FIELD\", \"foo\", 1, \"STRING\", \"bar1\"}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"id2\", \"FIELD\", \"foo\", 2, \"STRING\", \"bar2\"}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"id3\", \"FIELD\", \"foo\", 3, \"STRING\", \"bar3\"}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"id4\", \"FIELD\", \"foo\", 4, \"STRING\", \"bar4\"}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"id5\", \"FIELD\", \"foo\", 5, \"STRING\", \"bar5\"}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"id6\", \"FIELD\", \"foo\", 6, \"STRING\", \"bar6\"}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"id7\", \"FIELD\", \"foo\", 7, \"STRING\", \"bar7\"}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"id8\", \"FIELD\", \"foo\", 8, \"STRING\", \"bar8\"}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"id9\", \"FIELD\", \"foo\", 9, \"STRING\", \"bar9\"}, {\"OK\"},\n\t\t{\"SCAN\", \"mykey\", \"LIMIT\", 3, \"IDS\"}, {\"[3 [id1 id2 id3]]\"},\n\t\t{\"SCAN\", \"mykey\", \"CURSOR\", 3, \"LIMIT\", 3, \"IDS\"}, {\"[6 [id4 id5 id6]]\"},\n\t\t{\"SCAN\", \"mykey\", \"WHERE\", \"foo\", 3, 5, \"IDS\"}, {\"[0 [id3 id4 id5]]\"},\n\t\t{\"SCAN\", \"mykey\", \"LIMIT\", 1, \"WHERE\", \"foo\", 3, 5, \"IDS\"}, {\"[3 [id3]]\"},\n\t\t{\"SCAN\", \"mykey\", \"CURSOR\", 3, \"LIMIT\", 1, \"WHERE\", \"foo\", 8, 9, \"IDS\"}, {\n\t\t\t\"[8 [id8]]\"},\n\t\t{\"SCAN\", \"mykey\", \"CURSOR\", 6, \"LIMIT\", 1, \"WHERE\", \"foo\", 8, 9, \"IDS\"}, {\n\t\t\t\"[8 [id8]]\"},\n\t})\n}\n\nfunc keys_SEARCH_CURSOR_test(mc *mockServer) error {\n\treturn mc.DoBatch([][]interface{}{\n\t\t{\"SET\", \"mykey\", \"id1\", \"FIELD\", \"foo\", 1, \"STRING\", \"bar1\"}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"id2\", \"FIELD\", \"foo\", 2, \"STRING\", \"bar2\"}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"id3\", \"FIELD\", \"foo\", 3, \"STRING\", \"bar3\"}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"id4\", \"FIELD\", \"foo\", 4, \"STRING\", \"bar4\"}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"id5\", \"FIELD\", \"foo\", 5, \"STRING\", \"bar5\"}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"id6\", \"FIELD\", \"foo\", 6, \"STRING\", \"bar6\"}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"id7\", \"FIELD\", \"foo\", 7, \"STRING\", \"bar7\"}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"id8\", \"FIELD\", \"foo\", 8, \"STRING\", \"bar8\"}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"id9\", \"FIELD\", \"foo\", 9, \"STRING\", \"bar9\"}, {\"OK\"},\n\t\t{\"SEARCH\", \"mykey\", \"LIMIT\", 3, \"IDS\"}, {\"[3 [id1 id2 id3]]\"},\n\t\t{\"SEARCH\", \"mykey\", \"CURSOR\", 3, \"LIMIT\", 3, \"IDS\"}, {\"[6 [id4 id5 id6]]\"},\n\t\t{\"SEARCH\", \"mykey\", \"WHERE\", \"foo\", 3, 5, \"IDS\"}, {\"[0 [id3 id4 id5]]\"},\n\t\t{\"SEARCH\", \"mykey\", \"LIMIT\", 1, \"WHERE\", \"foo\", 3, 5, \"IDS\"}, {\"[3 [id3]]\"},\n\t\t{\"SEARCH\", \"mykey\", \"CURSOR\", 3, \"LIMIT\", 1, \"WHERE\", \"foo\", 8, 9, \"IDS\"}, {\n\t\t\t\"[8 [id8]]\"},\n\t\t{\"SEARCH\", \"mykey\", \"CURSOR\", 6, \"LIMIT\", 1, \"WHERE\", \"foo\", 8, 9, \"IDS\"}, {\n\t\t\t\"[8 [id8]]\"},\n\t\t{\"SEARCH\", \"mykey\", \"LIMIT\", 3, \"DESC\", \"IDS\"}, {\"[3 [id9 id8 id7]]\"},\n\t})\n}\n\nfunc keys_MATCH_test(mc *mockServer) error {\n\treturn mc.DoBatch([][]interface{}{\n\t\t{\"SET\", \"fleet\", \"truck1\", \"POINT\", \"33.0001\", \"-112.0001\"}, {\"OK\"},\n\t\t{\"SET\", \"fleet\", \"truck2\", \"POINT\", \"33.0002\", \"-112.0002\"}, {\"OK\"},\n\t\t{\"SET\", \"fleet\", \"luck1\", \"POINT\", \"33.0003\", \"-112.0003\"}, {\"OK\"},\n\t\t{\"SET\", \"fleet\", \"luck2\", \"POINT\", \"33.0004\", \"-112.0004\"}, {\"OK\"},\n\t\t{\"SET\", \"fleet\", \"train1\", \"POINT\", \"33.0005\", \"-112.0005\"}, {\"OK\"},\n\n\t\t{\"SCAN\", \"fleet\", \"IDS\"}, {\"[0 [luck1 luck2 train1 truck1 truck2]]\"},\n\t\t{\"SCAN\", \"fleet\", \"MATCH\", \"*\", \"IDS\"}, {\"[0 [luck1 luck2 train1 truck1 truck2]]\"},\n\t\t{\"SCAN\", \"fleet\", \"MATCH\", \"truck*\", \"IDS\"}, {\"[0 [truck1 truck2]]\"},\n\t\t{\"SCAN\", \"fleet\", \"MATCH\", \"luck*\", \"IDS\"}, {\"[0 [luck1 luck2]]\"},\n\t\t{\"SCAN\", \"fleet\", \"MATCH\", \"*2\", \"IDS\"}, {\"[0 [luck2 truck2]]\"},\n\t\t{\"SCAN\", \"fleet\", \"MATCH\", \"*2*\", \"IDS\"}, {\"[0 [luck2 truck2]]\"},\n\t\t{\"SCAN\", \"fleet\", \"MATCH\", \"*u*\", \"IDS\"}, {\"[0 [luck1 luck2 truck1 truck2]]\"},\n\t\t{\"SCAN\", \"fleet\", \"MATCH\", \"*u*\", \"MATCH\", \"*u*\", \"IDS\"}, {\"[0 [luck1 luck2 truck1 truck2]]\"},\n\t\t{\"SCAN\", \"fleet\", \"MATCH\", \"*u*\", \"MATCH\", \"*a*\", \"IDS\"}, {\"[0 [luck1 luck2 train1 truck1 truck2]]\"},\n\t\t{\"SCAN\", \"fleet\", \"MATCH\", \"train*\", \"MATCH\", \"truck*\", \"IDS\"}, {\"[0 [train1 truck1 truck2]]\"},\n\t\t{\"SCAN\", \"fleet\", \"MATCH\", \"train*\", \"MATCH\", \"truck*\", \"MATCH\", \"luck1\", \"IDS\"}, {\"[0 [luck1 train1 truck1 truck2]]\"},\n\n\t\t{\"NEARBY\", \"fleet\", \"IDS\", \"POINT\", 33.00005, -112.00005, 100000}, {\n\t\t\tmatch(\"[0 [luck1 luck2 train1 truck1 truck2]]\"),\n\t\t},\n\t\t{\"NEARBY\", \"fleet\", \"MATCH\", \"*\", \"IDS\", \"POINT\", 33.00005, -112.00005, 100000}, {\n\t\t\tmatch(\"[0 [luck1 luck2 train1 truck1 truck2]]\"),\n\t\t},\n\t\t{\"NEARBY\", \"fleet\", \"MATCH\", \"t*\", \"IDS\", \"POINT\", 33.00005, -112.00005, 100000}, {\n\t\t\tmatch(\"[0 [train1 truck1 truck2]]\"),\n\t\t},\n\t\t{\"NEARBY\", \"fleet\", \"MATCH\", \"t*2\", \"IDS\", \"POINT\", 33.00005, -112.00005, 100000}, {\n\t\t\tmatch(\"[0 [truck2]]\"),\n\t\t},\n\t\t{\"NEARBY\", \"fleet\", \"MATCH\", \"*2\", \"IDS\", \"POINT\", 33.00005, -112.00005, 100000}, {\n\t\t\tmatch(\"[0 [luck2 truck2]]\"),\n\t\t},\n\n\t\t{\"INTERSECTS\", \"fleet\", \"IDS\", \"BOUNDS\", 33, -113, 34, -112}, {\n\t\t\tmatch(\"[0 [luck1 luck2 train1 truck1 truck2]]\"),\n\t\t},\n\t\t{\"INTERSECTS\", \"fleet\", \"MATCH\", \"*\", \"IDS\", \"BOUNDS\", 33, -113, 34, -112}, {\n\t\t\tmatch(\"[0 [luck1 luck2 train1 truck1 truck2]]\"),\n\t\t},\n\t\t{\"INTERSECTS\", \"fleet\", \"MATCH\", \"t*\", \"IDS\", \"BOUNDS\", 33, -113, 34, -112}, {\n\t\t\tmatch(\"[0 [train1 truck1 truck2]]\"),\n\t\t},\n\t\t{\"INTERSECTS\", \"fleet\", \"MATCH\", \"t*2\", \"IDS\", \"BOUNDS\", 33, -113, 34, -112}, {\n\t\t\tmatch(\"[0 [truck2]]\"),\n\t\t},\n\t\t{\"INTERSECTS\", \"fleet\", \"MATCH\", \"*2\", \"IDS\", \"BOUNDS\", 33, -113, 34, -112}, {\n\t\t\tmatch(\"[0 [luck2 truck2]]\"),\n\t\t},\n\t})\n}\n\nfunc keys_FIELDS_search_test(mc *mockServer) error {\n\treturn mc.DoBatch([][]interface{}{\n\t\t{\"SET\", \"mykey\", \"1\", \"FIELD\", \"field1\", 10, \"FIELD\", \"field2\", 11 /* field3 undefined */, \"OBJECT\", `{\"type\":\"Point\",\"coordinates\":[-112.2791,33.5220]}`}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"2\", \"FIELD\", \"field1\", 20, \"FIELD\", \"field2\", 10 /* field3 undefined */, \"OBJECT\", `{\"type\":\"Point\",\"coordinates\":[-112.2793,33.5222]}`}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"3\", \"FIELD\", \"field1\", 30, \"FIELD\", \"field2\", 13 /* field3 undefined */, \"OBJECT\", `{\"type\":\"Point\",\"coordinates\":[-112.2795,33.5224]}`}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"4\", \"FIELD\", \"field1\", 40, \"FIELD\", \"field2\", 14 /* field3 undefined */, \"OBJECT\", `{\"type\":\"Point\",\"coordinates\":[-112.2797,33.5226]}`}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"5\" /* field1 undefined */, \"FIELD\", \"field2\", 15, \"FIELD\", \"field3\", 28, \"OBJECT\", `{\"type\":\"Point\",\"coordinates\":[-112.2799,33.5228]}`}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"6\" /* field1 & field2 undefined */, \"FIELD\", \"field3\", 29, \"OBJECT\", `{\"type\":\"Point\",\"coordinates\":[-112.2801,33.5230]}`}, {\"OK\"},\n\t\t{\"SET\", \"mykey\", \"7\" /* field1, field2, & field3 undefined */, \"OBJECT\", `{\"type\":\"Point\",\"coordinates\":[-112.2803,33.5232]}`}, {\"OK\"},\n\n\t\t// test RESP output\n\t\t{\"NEARBY\", \"mykey\", \"WHERE\", \"field2\", 11, \"+inf\", \"POINT\", 33.462, -112.268, 60000}, {\n\t\t\t`[0 [` +\n\t\t\t\t`[1 {\"type\":\"Point\",\"coordinates\":[-112.2791,33.522]} [field1 10 field2 11]] ` +\n\t\t\t\t`[3 {\"type\":\"Point\",\"coordinates\":[-112.2795,33.5224]} [field1 30 field2 13]] ` +\n\t\t\t\t`[4 {\"type\":\"Point\",\"coordinates\":[-112.2797,33.5226]} [field1 40 field2 14]] ` +\n\t\t\t\t`[5 {\"type\":\"Point\",\"coordinates\":[-112.2799,33.5228]} [field2 15 field3 28]]]]`,\n\t\t},\n\t\t{\"NEARBY\", \"mykey\", \"WHERE\", \"field2\", 0, 2, \"POINT\", 33.462, -112.268, 60000}, {\n\t\t\t`[0 [` +\n\t\t\t\t`[6 {\"type\":\"Point\",\"coordinates\":[-112.2801,33.523]} [field3 29]] ` +\n\t\t\t\t`[7 {\"type\":\"Point\",\"coordinates\":[-112.2803,33.5232]}]]]`},\n\n\t\t{\"WITHIN\", \"mykey\", \"WHERE\", \"field2\", 11, \"+inf\", \"CIRCLE\", 33.462, -112.268, 60000}, {\n\t\t\t`[0 [` +\n\t\t\t\t`[5 {\"type\":\"Point\",\"coordinates\":[-112.2799,33.5228]} [field2 15 field3 28]] ` +\n\t\t\t\t`[4 {\"type\":\"Point\",\"coordinates\":[-112.2797,33.5226]} [field1 40 field2 14]] ` +\n\t\t\t\t`[3 {\"type\":\"Point\",\"coordinates\":[-112.2795,33.5224]} [field1 30 field2 13]] ` +\n\t\t\t\t`[1 {\"type\":\"Point\",\"coordinates\":[-112.2791,33.522]} [field1 10 field2 11]]]]`,\n\t\t},\n\t\t{\"WITHIN\", \"mykey\", \"WHERE\", \"field2\", 0, 2, \"CIRCLE\", 33.462, -112.268, 60000}, {\n\t\t\t`[0 [` +\n\t\t\t\t`[7 {\"type\":\"Point\",\"coordinates\":[-112.2803,33.5232]}] ` +\n\t\t\t\t`[6 {\"type\":\"Point\",\"coordinates\":[-112.2801,33.523]} [field3 29]]]]`},\n\n\t\t// test JSON output\n\t\t{\"OUTPUT\", \"json\"}, {`{\"ok\":true}`},\n\t\t{\"NEARBY\", \"mykey\", \"WHERE\", \"field2\", 11, \"+inf\", \"POINT\", 33.462, -112.268, 60000}, {\n\t\t\t`{\"ok\":true,\"fields\":[\"field1\",\"field2\",\"field3\"],\"objects\":[` +\n\t\t\t\t`{\"id\":\"1\",\"object\":{\"type\":\"Point\",\"coordinates\":[-112.2791,33.522]},\"fields\":[10,11,0]},` +\n\t\t\t\t`{\"id\":\"3\",\"object\":{\"type\":\"Point\",\"coordinates\":[-112.2795,33.5224]},\"fields\":[30,13,0]},` +\n\t\t\t\t`{\"id\":\"4\",\"object\":{\"type\":\"Point\",\"coordinates\":[-112.2797,33.5226]},\"fields\":[40,14,0]},` +\n\t\t\t\t`{\"id\":\"5\",\"object\":{\"type\":\"Point\",\"coordinates\":[-112.2799,33.5228]},\"fields\":[0,15,28]}` +\n\t\t\t\t`],\"count\":4,\"cursor\":0}`},\n\t\t{\"NEARBY\", \"mykey\", \"WHERE\", \"field2\", 0, 2, \"POINT\", 33.462, -112.268, 60000}, {\n\t\t\t`{\"ok\":true,\"fields\":[\"field3\"],\"objects\":[` +\n\t\t\t\t`{\"id\":\"6\",\"object\":{\"type\":\"Point\",\"coordinates\":[-112.2801,33.523]},\"fields\":[29]},` +\n\t\t\t\t`{\"id\":\"7\",\"object\":{\"type\":\"Point\",\"coordinates\":[-112.2803,33.5232]},\"fields\":[0]}` +\n\t\t\t\t`],\"count\":2,\"cursor\":0}`},\n\n\t\t{\"WITHIN\", \"mykey\", \"WHERE\", \"field2\", 11, \"+inf\", \"CIRCLE\", 33.462, -112.268, 60000}, {\n\t\t\t`{\"ok\":true,\"fields\":[\"field1\",\"field2\",\"field3\"],\"objects\":[` +\n\t\t\t\t`{\"id\":\"5\",\"object\":{\"type\":\"Point\",\"coordinates\":[-112.2799,33.5228]},\"fields\":[0,15,28]},` +\n\t\t\t\t`{\"id\":\"4\",\"object\":{\"type\":\"Point\",\"coordinates\":[-112.2797,33.5226]},\"fields\":[40,14,0]},` +\n\t\t\t\t`{\"id\":\"3\",\"object\":{\"type\":\"Point\",\"coordinates\":[-112.2795,33.5224]},\"fields\":[30,13,0]},` +\n\t\t\t\t`{\"id\":\"1\",\"object\":{\"type\":\"Point\",\"coordinates\":[-112.2791,33.522]},\"fields\":[10,11,0]}` +\n\t\t\t\t`],\"count\":4,\"cursor\":0}`},\n\t\t{\"WITHIN\", \"mykey\", \"WHERE\", \"field2\", 0, 2, \"CIRCLE\", 33.462, -112.268, 60000}, {\n\t\t\t`{\"ok\":true,\"fields\":[\"field3\"],\"objects\":[` +\n\t\t\t\t`{\"id\":\"7\",\"object\":{\"type\":\"Point\",\"coordinates\":[-112.2803,33.5232]},\"fields\":[0]},` +\n\t\t\t\t`{\"id\":\"6\",\"object\":{\"type\":\"Point\",\"coordinates\":[-112.2801,33.523]},\"fields\":[29]}` +\n\t\t\t\t`],\"count\":2,\"cursor\":0}`},\n\t})\n}\n\nfunc keys_BUFFER_search_test(mc *mockServer) error {\n\tconst lineString = `{\"type\":\"LineString\",\"coordinates\":[\n\t\t[-116.40289306640624,34.125447565116126],\n\t\t[-116.36444091796875,34.14818102254435],\n\t\t[-116.0980224609375,34.15045403191448],\n\t\t[-115.74920654296874,34.127721186043985],\n\t\t[-115.54870605468749,34.075412438417395],\n\t\t[-115.5267333984375,34.11407854333859],\n\t\t[-115.21911621093749,34.048108084909835],\n\t\t[-115.25207519531249,33.8339199536547],\n\t\t[-115.40588378906249,33.71748624018193]\n\t]}`\n\n\treturn mc.DoBatch([][]interface{}{\n\t\t// points in\n\t\t{\"SET\", \"fleet\", \"truck01\", \"POINT\", \"34.10825132729329\", \"-115.6436347961428\"}, {\"OK\"},\n\t\t{\"SET\", \"fleet\", \"truck02\", \"POINT\", \"34.07199987534163\", \"-115.5435562133782\"}, {\"OK\"},\n\t\t{\"SET\", \"fleet\", \"truck03\", \"POINT\", \"34.05123715497616\", \"-115.2148246765137\"}, {\"OK\"},\n\t\t{\"SET\", \"fleet\", \"truck04\", \"POINT\", \"33.71520164474084\", \"-115.4110336303711\"}, {\"OK\"},\n\t\t{\"SET\", \"fleet\", \"truck05\", \"POINT\", \"34.12345809664606\", \"-116.4070129394531\"}, {\"OK\"},\n\t\t// points out\n\t\t{\"SET\", \"fleet\", \"truck06\", \"POINT\", \"35.10825132729329\", \"-115.6436347961428\"}, {\"OK\"},\n\t\t{\"SET\", \"fleet\", \"truck07\", \"POINT\", \"35.07199987534163\", \"-115.5435562133782\"}, {\"OK\"},\n\t\t{\"SET\", \"fleet\", \"truck08\", \"POINT\", \"35.05123715497616\", \"-115.2148246765137\"}, {\"OK\"},\n\t\t{\"SET\", \"fleet\", \"truck09\", \"POINT\", \"35.71520164474084\", \"-115.4110336303711\"}, {\"OK\"},\n\t\t{\"SET\", \"fleet\", \"truck10\", \"POINT\", \"35.12345809664606\", \"-116.4070129394531\"}, {\"OK\"},\n\t\t// buffered intersects\n\t\t{\"INTERSECTS\", \"fleet\", \"BUFFER\", \"1000\", \"COUNT\", \"OBJECT\", lineString}, {\"5\"},\n\t})\n}\n\n// match sorts the response and compares to the expected input\nfunc match(expectIn string) func(org, v interface{}) (resp, expect interface{}) {\n\treturn func(v, org interface{}) (resp, expect interface{}) {\n\t\tsort.Slice(org.([]interface{})[1], func(i, j int) bool {\n\t\t\treturn org.([]interface{})[1].([]interface{})[i].(string) <\n\t\t\t\torg.([]interface{})[1].([]interface{})[j].(string)\n\t\t})\n\t\treturn fmt.Sprintf(\"%v\", org), expectIn\n\t}\n}\n\nfunc subBenchSearch(b *testing.B, mc *mockServer) {\n\trunBenchStep(b, mc, \"KNN\", keys_KNN_bench)\n}\n\nfunc keys_KNN_bench(mc *mockServer) error {\n\tlat := rand.Float64()*180 - 90\n\tlon := rand.Float64()*360 - 180\n\t_, err := mc.conn.Do(\"NEARBY\",\n\t\t\"mykey\",\n\t\t\"LIMIT\", 50,\n\t\t\"DISTANCE\",\n\t\t\"POINTS\",\n\t\t\"POINT\", lat, lon)\n\treturn err\n}\n" }, { "path": "tests/keys_test.go", "content": "package tests\n\nimport (\n\t\"errors\"\n\t\"fmt\"\n\t\"math/rand\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/gomodule/redigo/redis\"\n\t\"github.com/tidwall/gjson\"\n)\n\nfunc subTestKeys(g *testGroup) {\n\tg.regSubTest(\"BOUNDS\", keys_BOUNDS_test)\n\tg.regSubTest(\"DEL\", keys_DEL_test)\n\tg.regSubTest(\"DROP\", keys_DROP_test)\n\tg.regSubTest(\"RENAME\", keys_RENAME_test)\n\tg.regSubTest(\"RENAMENX\", keys_RENAMENX_test)\n\tg.regSubTest(\"EXPIRE\", keys_EXPIRE_test)\n\tg.regSubTest(\"FSET\", keys_FSET_test)\n\tg.regSubTest(\"FGET\", keys_FGET_test)\n\tg.regSubTest(\"GET\", keys_GET_test)\n\tg.regSubTest(\"KEYS\", keys_KEYS_test)\n\tg.regSubTest(\"PERSIST\", keys_PERSIST_test)\n\tg.regSubTest(\"SET\", keys_SET_test)\n\tg.regSubTest(\"STATS\", keys_STATS_test)\n\tg.regSubTest(\"TTL\", keys_TTL_test)\n\tg.regSubTest(\"EXIST\", keys_EXISTS_test)\n\tg.regSubTest(\"FEXIST\", keys_FEXISTS_test)\n\tg.regSubTest(\"SET EX\", keys_SET_EX_test)\n\tg.regSubTest(\"PDEL\", keys_PDEL_test)\n\tg.regSubTest(\"FIELDS\", keys_FIELDS_test)\n\tg.regSubTest(\"WHEREIN\", keys_WHEREIN_test)\n\tg.regSubTest(\"WHEREEVAL\", keys_WHEREEVAL_test)\n\tg.regSubTest(\"TYPE\", keys_TYPE_test)\n\tg.regSubTest(\"FLUSHDB\", keys_FLUSHDB_test)\n\tg.regSubTest(\"HEALTHZ\", keys_HEALTHZ_test)\n\tg.regSubTest(\"SERVER\", keys_SERVER_test)\n\tg.regSubTest(\"INFO\", keys_INFO_test)\n}\n\nfunc keys_BOUNDS_test(mc *mockServer) error {\n\treturn mc.DoBatch(\n\t\tDo(\"BOUNDS\", \"mykey\").Str(\"\"),\n\t\tDo(\"BOUNDS\", \"mykey\").JSON().Err(\"key not found\"),\n\t\tDo(\"SET\", \"mykey\", \"myid1\", \"POINT\", 33, -115).OK(),\n\t\tDo(\"BOUNDS\", \"mykey\").Str(\"[[-115 33] [-115 33]]\"),\n\t\tDo(\"BOUNDS\", \"mykey\").JSON().Str(`{\"ok\":true,\"bounds\":{\"type\":\"Polygon\",\"coordinates\":[[[-115,33],[-115,33],[-115,33],[-115,33],[-115,33]]]}}`),\n\t\tDo(\"SET\", \"mykey\", \"myid2\", \"POINT\", 34, -112).OK(),\n\t\tDo(\"BOUNDS\", \"mykey\").Str(\"[[-115 33] [-112 34]]\"),\n\t\tDo(\"DEL\", \"mykey\", \"myid2\").Str(\"1\"),\n\t\tDo(\"BOUNDS\", \"mykey\").Str(\"[[-115 33] [-115 33]]\"),\n\t\tDo(\"SET\", \"mykey\", \"myid3\", \"OBJECT\", `{\"type\":\"Point\",\"coordinates\":[-130,38,10]}`).OK(),\n\t\tDo(\"SET\", \"mykey\", \"myid4\", \"OBJECT\", `{\"type\":\"Point\",\"coordinates\":[-110,25,-8]}`).OK(),\n\t\tDo(\"BOUNDS\", \"mykey\").Str(\"[[-130 25] [-110 38]]\"),\n\t\tDo(\"BOUNDS\", \"mykey\", \"hello\").Err(\"wrong number of arguments for 'bounds' command\"),\n\t\tDo(\"BOUNDS\", \"nada\").Str(\"\"),\n\t\tDo(\"BOUNDS\", \"nada\").JSON().Err(\"key not found\"),\n\t\tDo(\"BOUNDS\", \"\").Str(\"\"),\n\t\tDo(\"BOUNDS\", \"mykey\").JSON().Str(`{\"ok\":true,\"bounds\":{\"type\":\"Polygon\",\"coordinates\":[[[-130,25],[-110,25],[-110,38],[-130,38],[-130,25]]]}}`),\n\t)\n}\n\nfunc keys_DEL_test(mc *mockServer) error {\n\treturn mc.DoBatch(\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"POINT\", 33, -115).OK(),\n\t\tDo(\"GET\", \"mykey\", \"myid\", \"POINT\").Str(\"[33 -115]\"),\n\t\tDo(\"DEL\", \"mykey\", \"myid2\", \"ERRON404\").Err(\"id not found\"),\n\t\tDo(\"DEL\", \"mykey\", \"myid\").Str(\"1\"),\n\t\tDo(\"DEL\", \"mykey\", \"myid\").Str(\"0\"),\n\t\tDo(\"DEL\", \"mykey\").Err(\"wrong number of arguments for 'del' command\"),\n\t\tDo(\"GET\", \"mykey\", \"myid\").Str(\"\"),\n\t\tDo(\"DEL\", \"mykey\", \"myid\", \"ERRON404\").Err(\"key not found\"),\n\t\tDo(\"DEL\", \"mykey\", \"myid\", \"invalid-arg\").Err(\"invalid argument 'invalid-arg'\"),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"POINT\", 33, -115).OK(),\n\t\tDo(\"DEL\", \"mykey\", \"myid2\", \"ERRON404\").JSON().Err(\"id not found\"),\n\t\tDo(\"DEL\", \"mykey\", \"myid\").JSON().OK(),\n\t\tDo(\"DEL\", \"mykey\", \"myid\").JSON().OK(),\n\t\tDo(\"DEL\", \"mykey\", \"myid\", \"ERRON404\").JSON().Err(\"key not found\"),\n\t)\n}\n\nfunc keys_DROP_test(mc *mockServer) error {\n\treturn mc.DoBatch(\n\t\tDo(\"SET\", \"mykey\", \"myid1\", \"HASH\", \"9my5xp7\").OK(),\n\t\tDo(\"SET\", \"mykey\", \"myid2\", \"HASH\", \"9my5xp8\").OK(),\n\t\tDo(\"SCAN\", \"mykey\", \"COUNT\").Str(\"2\"),\n\t\tDo(\"DROP\").Err(\"wrong number of arguments for 'drop' command\"),\n\t\tDo(\"DROP\", \"mykey\", \"arg3\").Err(\"wrong number of arguments for 'drop' command\"),\n\t\tDo(\"DROP\", \"mykey\").Str(\"1\"),\n\t\tDo(\"SCAN\", \"mykey\", \"COUNT\").Str(\"0\"),\n\t\tDo(\"DROP\", \"mykey\").Str(\"0\"),\n\t\tDo(\"SCAN\", \"mykey\", \"COUNT\").Str(\"0\"),\n\t\tDo(\"SET\", \"mykey\", \"myid1\", \"HASH\", \"9my5xp7\").OK(),\n\t\tDo(\"DROP\", \"mykey\").JSON().OK(),\n\t\tDo(\"DROP\", \"mykey\").JSON().OK(),\n\t)\n}\nfunc keys_RENAME_test(mc *mockServer) error {\n\treturn mc.DoBatch(\n\t\tDo(\"SET\", \"mykey\", \"myid1\", \"HASH\", \"9my5xp7\").OK(),\n\t\tDo(\"SET\", \"mykey\", \"myid2\", \"HASH\", \"9my5xp8\").OK(),\n\t\tDo(\"SCAN\", \"mykey\", \"COUNT\").Str(\"2\"),\n\t\tDo(\"RENAME\", \"foo\", \"mynewkey\", \"arg3\").Err(\"wrong number of arguments for 'rename' command\"),\n\t\tDo(\"RENAME\", \"mykey\", \"mynewkey\").OK(),\n\t\tDo(\"SCAN\", \"mykey\", \"COUNT\").Str(\"0\"),\n\t\tDo(\"SCAN\", \"mynewkey\", \"COUNT\").Str(\"2\"),\n\t\tDo(\"SET\", \"mykey\", \"myid3\", \"HASH\", \"9my5xp7\").OK(),\n\t\tDo(\"RENAME\", \"mykey\", \"mynewkey\").OK(),\n\t\tDo(\"SCAN\", \"mykey\", \"COUNT\").Str(\"0\"),\n\t\tDo(\"SCAN\", \"mynewkey\", \"COUNT\").Str(\"1\"),\n\t\tDo(\"RENAME\", \"foo\", \"mynewkey\").Err(\"key not found\"),\n\t\tDo(\"SCAN\", \"mynewkey\", \"COUNT\").Str(\"1\"),\n\t\tDo(\"SETCHAN\", \"mychan\", \"INTERSECTS\", \"mynewkey\", \"BOUNDS\", 10, 10, 20, 20).Str(\"1\"),\n\t\tDo(\"RENAME\", \"mynewkey\", \"foo2\").Err(\"key has channels set\"),\n\t\tDo(\"RENAMENX\", \"mynewkey\", \"foo2\").Err(\"key has channels set\"),\n\t\tDo(\"SET\", \"mykey\", \"myid1\", \"HASH\", \"9my5xp7\").OK(),\n\t\tDo(\"RENAME\", \"mykey\", \"foo2\").OK(),\n\t\tDo(\"RENAMENX\", \"foo2\", \"foo3\").Str(\"1\"),\n\t\tDo(\"RENAMENX\", \"foo2\", \"foo3\").Err(\"key not found\"),\n\t\tDo(\"RENAME\", \"foo2\", \"foo3\").JSON().Err(\"key not found\"),\n\t\tDo(\"SET\", \"mykey\", \"myid1\", \"HASH\", \"9my5xp7\").OK(),\n\t\tDo(\"RENAMENX\", \"mykey\", \"foo3\").Str(\"0\"),\n\t\tDo(\"RENAME\", \"foo3\", \"foo4\").JSON().OK(),\n\t)\n}\nfunc keys_RENAMENX_test(mc *mockServer) error {\n\treturn mc.DoBatch(\n\t\tDo(\"SET\", \"mykey\", \"myid1\", \"HASH\", \"9my5xp7\").OK(),\n\t\tDo(\"SET\", \"mykey\", \"myid2\", \"HASH\", \"9my5xp8\").OK(),\n\t\tDo(\"SCAN\", \"mykey\", \"COUNT\").Str(\"2\"),\n\t\tDo(\"RENAMENX\", \"mykey\", \"mynewkey\").Str(\"1\"),\n\t\tDo(\"SCAN\", \"mykey\", \"COUNT\").Str(\"0\"),\n\t\tDo(\"DROP\", \"mykey\").Str(\"0\"),\n\t\tDo(\"SCAN\", \"mykey\", \"COUNT\").Str(\"0\"),\n\t\tDo(\"SCAN\", \"mynewkey\", \"COUNT\").Str(\"2\"),\n\t\tDo(\"SET\", \"mykey\", \"myid3\", \"HASH\", \"9my5xp7\").OK(),\n\t\tDo(\"RENAMENX\", \"mykey\", \"mynewkey\").Str(\"0\"),\n\t\tDo(\"SCAN\", \"mykey\", \"COUNT\").Str(\"1\"),\n\t\tDo(\"SCAN\", \"mynewkey\", \"COUNT\").Str(\"2\"),\n\t\tDo(\"RENAMENX\", \"foo\", \"mynewkey\").Str(\"ERR key not found\"),\n\t\tDo(\"SCAN\", \"mynewkey\", \"COUNT\").Str(\"2\"),\n\t)\n}\nfunc keys_EXPIRE_test(mc *mockServer) error {\n\treturn mc.DoBatch(\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"STRING\", \"value\").OK(),\n\t\tDo(\"EXPIRE\", \"mykey\", \"myid\").Err(\"wrong number of arguments for 'expire' command\"),\n\t\tDo(\"EXPIRE\", \"mykey\", \"myid\", \"y\").Err(\"invalid argument 'y'\"),\n\t\tDo(\"EXPIRE\", \"mykey\", \"myid\", 1).Str(\"1\"),\n\t\tDo(\"EXPIRE\", \"mykey\", \"myid\", 1).JSON().OK(),\n\t\tSleep(time.Second/4),\n\t\tDo(\"GET\", \"mykey\", \"myid\").Str(\"value\"),\n\t\tSleep(time.Second),\n\t\tDo(\"GET\", \"mykey\", \"myid\").Str(\"\"),\n\t\tDo(\"EXPIRE\", \"mykey\", \"myid\", 1).JSON().Err(\"key not found\"),\n\t\tDo(\"SET\", \"mykey\", \"myid1\", \"STRING\", \"value1\").OK(),\n\t\tDo(\"SET\", \"mykey\", \"myid2\", \"STRING\", \"value2\").OK(),\n\t\tDo(\"EXPIRE\", \"mykey\", \"myid1\", 1).Str(\"1\"),\n\t\tSleep(time.Second/4),\n\t\tDo(\"GET\", \"mykey\", \"myid1\").Str(\"value1\"),\n\t\tSleep(time.Second),\n\t\tDo(\"EXPIRE\", \"mykey\", \"myid1\", 1).Str(\"0\"),\n\t\tDo(\"EXPIRE\", \"mykey\", \"myid1\", 1).JSON().Err(\"id not found\"),\n\t)\n}\nfunc keys_FSET_test(mc *mockServer) error {\n\treturn mc.DoBatch(\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"HASH\", \"9my5xp7\").OK(),\n\t\tDo(\"GET\", \"mykey\", \"myid\", \"WITHFIELDS\", \"HASH\", 7).Str(\"[9my5xp7]\"),\n\t\tDo(\"FSET\", \"mykey\", \"myid\", \"f1\", 105.6).Str(\"1\"),\n\t\tDo(\"GET\", \"mykey\", \"myid\", \"WITHFIELDS\", \"HASH\", 7).Str(\"[9my5xp7 [f1 105.6]]\"),\n\t\tDo(\"FSET\", \"mykey\", \"myid\", \"f1\", 1.1, \"f2\", 2.2).Str(\"2\"),\n\t\tDo(\"GET\", \"mykey\", \"myid\", \"WITHFIELDS\", \"HASH\", 7).Str(\"[9my5xp7 [f1 1.1 f2 2.2]]\"),\n\t\tDo(\"FSET\", \"mykey\", \"myid\", \"f1\", 1.1, \"f2\", 22.22).Str(\"1\"),\n\t\tDo(\"GET\", \"mykey\", \"myid\", \"WITHFIELDS\", \"HASH\", 7).Str(\"[9my5xp7 [f1 1.1 f2 22.22]]\"),\n\t\tDo(\"FSET\", \"mykey\", \"myid\", \"f1\", 0).Str(\"1\"),\n\t\tDo(\"GET\", \"mykey\", \"myid\", \"WITHFIELDS\", \"HASH\", 7).Str(\"[9my5xp7 [f2 22.22]]\"),\n\t\tDo(\"FSET\", \"mykey\", \"myid\", \"f2\", 0).Str(\"1\"),\n\t\tDo(\"GET\", \"mykey\", \"myid\", \"WITHFIELDS\", \"HASH\", 7).Str(\"[9my5xp7]\"),\n\t\tDo(\"FSET\", \"mykey\", \"myid2\", \"xx\", \"f1\", 1.1, \"f2\", 2.2).Str(\"0\"),\n\n\t\tDo(\"FSET\", \"mykey\", \"myid\", \"f1\", 1, \"RETURN\", \"HASH\", 7, \"WITHFIELDS\").Str(\"[9my5xp7 [f1 1]]\"),\n\t\tDo(\"FSET\", \"mykey\", \"myid\", \"f2\", 2, \"RETURN\", \"HASH\", 7, \"WITHFIELDS\").Str(\"[9my5xp7 [f1 1 f2 2]]\"),\n\t\tDo(\"FSET\", \"mykey\", \"myid\", \"f1\", 0, \"RETURN\", \"HASH\", 7, \"WITHFIELDS\").Str(\"[9my5xp7 [f2 2]]\"),\n\t\tDo(\"FSET\", \"mykey\", \"myid\", \"f2\", 0, \"RETURN\", \"HASH\", 7, \"WITHFIELDS\").Str(\"[9my5xp7]\"),\n\n\t\tDo(\"GET\", \"mykey\", \"myid2\").Str(\"\"),\n\t\tDo(\"DEL\", \"mykey\", \"myid\").Str(\"1\"),\n\t\tDo(\"GET\", \"mykey\", \"myid\").Str(\"\"),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"POINT\", 1, 2).OK(),\n\t\tDo(\"GET\", \"mykey\", \"myid\", \"WITHFIELDS\").JSON().Str(`{\"ok\":true,\"object\":{\"type\":\"Point\",\"coordinates\":[2,1]}}`),\n\t\tDo(\"FSET\", \"mykey\", \"myid\", \"f2\", 1).JSON().OK(),\n\t\tDo(\"GET\", \"mykey\", \"myid\", \"WITHFIELDS\").JSON().Str(`{\"ok\":true,\"object\":{\"type\":\"Point\",\"coordinates\":[2,1]},\"fields\":{\"f2\":1}}`),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"POINT\", 3, 4).OK(),\n\t\tDo(\"GET\", \"mykey\", \"myid\", \"WITHFIELDS\").JSON().Str(`{\"ok\":true,\"object\":{\"type\":\"Point\",\"coordinates\":[4,3]},\"fields\":{\"f2\":1}}`),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"HASH\", \"9my5xp7\").OK(),\n\t\tDo(\"DEL\", \"mykey\", \"myid\").Str(\"1\"),\n\t\tDo(\"GET\", \"mykey\", \"myid\").Str(\"\"),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"HASH\", \"9my5xp7\").OK(),\n\t\tDo(\"CONFIG\", \"SET\", \"maxmemory\", \"1\").OK(),\n\t\tDo(\"FSET\", \"mykey\", \"myid\", \"xx\", \"f1\", 1.1, \"f2\", 2.2).Err(`OOM command not allowed when used memory > 'maxmemory'`),\n\t\tDo(\"CONFIG\", \"SET\", \"maxmemory\", \"0\").OK(),\n\t\tDo(\"FSET\", \"mykey\", \"myid\", \"xx\").Err(\"wrong number of arguments for 'fset' command\"),\n\t\tDo(\"FSET\", \"mykey\", \"myid\", \"f1\", \"a\", \"f2\").Err(\"wrong number of arguments for 'fset' command\"),\n\t\tDo(\"FSET\", \"mykey\", \"myid\", \"z\", \"a\").Err(\"invalid argument 'z'\"),\n\t\tDo(\"FSET\", \"mykey2\", \"myid\", \"a\", \"b\").Err(\"key not found\"),\n\t\tDo(\"FSET\", \"mykey\", \"myid2\", \"a\", \"b\").Err(\"id not found\"),\n\t\tDo(\"FSET\", \"mykey\", \"myid\", \"f2\", 0).JSON().OK(),\n\t\tDo(\"SET\", \"cases\", \"lower\", \"POINT\", 1, 2).OK(),\n\t\tDo(\"FSET\", \"cases\", \"lower\", \"lowercase\", 1).JSON().OK(),\n\t\tDo(\"GET\", \"cases\", \"lower\", \"WITHFIELDS\").JSON().Str(\n\t\t\t`{\"ok\":true,\"object\":{\"type\":\"Point\",\"coordinates\":[2,1]},\"fields\":{\"lowercase\":1}}`,\n\t\t),\n\t\tDo(\"SET\", \"cases\", \"upper\", \"POINT\", 1, 2).OK(),\n\t\tDo(\"FSET\", \"cases\", \"upper\", \"UPPERCASE\", 1).JSON().OK(),\n\t\tDo(\"GET\", \"cases\", \"upper\", \"WITHFIELDS\").JSON().Str(\n\t\t\t`{\"ok\":true,\"object\":{\"type\":\"Point\",\"coordinates\":[2,1]},\"fields\":{\"UPPERCASE\":1}}`,\n\t\t),\n\t\tDo(\"SET\", \"cases\", \"camel\", \"POINT\", 1, 2).OK(),\n\t\tDo(\"FSET\", \"cases\", \"camel\", \"camelCase\", 1).JSON().OK(),\n\t\tDo(\"GET\", \"cases\", \"camel\", \"WITHFIELDS\").JSON().Str(\n\t\t\t`{\"ok\":true,\"object\":{\"type\":\"Point\",\"coordinates\":[2,1]},\"fields\":{\"camelCase\":1}}`,\n\t\t),\n\t\tDo(\"SET\", \"cases\", \"allcases\", \"POINT\", 1, 2).OK(),\n\t\tDo(\"FSET\", \"cases\", \"allcases\", \"UPPERCASE\", 1).JSON().OK(),\n\t\tDo(\"FSET\", \"cases\", \"allcases\", \"lowercase\", 1).JSON().OK(),\n\t\tDo(\"FSET\", \"cases\", \"allcases\", \"camelCase\", 1).JSON().OK(),\n\t\tDo(\"GET\", \"cases\", \"allcases\", \"WITHFIELDS\").JSON().Str(\n\t\t\t`{\"ok\":true,\"object\":{\"type\":\"Point\",\"coordinates\":[2,1]},\"fields\":{\"UPPERCASE\":1,\"camelCase\":1,\"lowercase\":1}}`,\n\t\t),\n\t)\n}\nfunc keys_FGET_test(mc *mockServer) error {\n\treturn mc.DoBatch(\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"HASH\", \"9my5xp7\").OK(),\n\t\tDo(\"GET\", \"mykey\", \"myid\", \"WITHFIELDS\", \"HASH\", 7).Str(\"[9my5xp7]\"),\n\t\tDo(\"FSET\", \"mykey\", \"myid\", \"f1\", 105.6).Str(\"1\"),\n\t\tDo(\"FGET\", \"mykey\", \"myid\", \"f1\").Str(\"105.6\"),\n\t\tDo(\"FSET\", \"mykey\", \"myid\", \"f1\", 1.1, \"f2\", 2.2).Str(\"2\"),\n\t\tDo(\"FGET\", \"mykey\", \"myid\", \"f2\").Str(\"2.2\"),\n\t\tDo(\"FGET\", \"mykey\", \"myid\", \"f1\").Str(\"1.1\"),\n\t\tDo(\"FGET\", \"mykey\", \"myid\", \"f1\").JSON().Str(`{\"ok\":true,\"value\":1.1}`),\n\t\tDo(\"FSET\", \"mykey\", \"myid\", \"f3\", \"a\").Str(\"1\"),\n\t\tDo(\"FGET\", \"mykey\", \"myid\", \"f3\").Str(\"a\"),\n\t\tDo(\"FGET\", \"mykey\", \"myid\", \"f4\").Str(\"0\"),\n\t\tDo(\"FGET\", \"mykey\", \"myid\", \"f4\").JSON().Str(`{\"ok\":true,\"value\":0}`),\n\t\tDo(\"FGET\", \"mykey\", \"myid\").Err(\"wrong number of arguments for 'fget' command\"),\n\t\tDo(\"FGET\", \"mykey2\", \"myid\", \"a\", \"b\").Err(\"key not found\"),\n\t\tDo(\"FGET\", \"mykey\", \"myid2\", \"a\", \"b\").Err(\"id not found\"),\n\t)\n}\nfunc keys_GET_test(mc *mockServer) error {\n\treturn mc.DoBatch(\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"STRING\", \"value\").OK(),\n\t\tDo(\"GET\", \"mykey\", \"myid\").Str(\"value\"),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"STRING\", \"value2\").OK(),\n\t\tDo(\"GET\", \"mykey\", \"myid\").Str(\"value2\"),\n\t\tDo(\"DEL\", \"mykey\", \"myid\").Str(\"1\"),\n\t\tDo(\"GET\", \"mykey\", \"myid\").Str(\"\"),\n\t\tDo(\"GET\", \"mykey\").Err(\"wrong number of arguments for 'get' command\"),\n\t\tDo(\"GET\", \"mykey\", \"myid\", \"hash\").Err(\"wrong number of arguments for 'get' command\"),\n\t\tDo(\"GET\", \"mykey\", \"myid\", \"hash\", \"0\").Err(\"invalid argument '0'\"),\n\t\tDo(\"GET\", \"mykey\", \"myid\", \"hash\", \"-1\").Err(\"invalid argument '-1'\"),\n\t\tDo(\"GET\", \"mykey\", \"myid\", \"hash\", \"13\").Err(\"invalid argument '13'\"),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"field\", \"hello\", \"world\", \"field\", \"hiya\", 55, \"point\", 33, -112).OK(),\n\t\tDo(\"GET\", \"mykey\", \"myid\", \"hash\", \"1\").Str(\"9\"),\n\t\tDo(\"GET\", \"mykey\", \"myid\", \"point\").Str(\"[33 -112]\"),\n\t\tDo(\"GET\", \"mykey\", \"myid\", \"bounds\").Str(\"[[33 -112] [33 -112]]\"),\n\t\tDo(\"GET\", \"mykey\", \"myid\", \"object\").Str(`{\"type\":\"Point\",\"coordinates\":[-112,33]}`),\n\t\tDo(\"GET\", \"mykey\", \"myid\", \"object\").Str(`{\"type\":\"Point\",\"coordinates\":[-112,33]}`),\n\t\tDo(\"GET\", \"mykey\", \"myid\", \"withfields\", \"point\").Str(`[[33 -112] [hello world hiya 55]]`),\n\t\tDo(\"GET\", \"mykey\", \"myid\", \"joint\").Err(\"wrong number of arguments for 'get' command\"),\n\t\tDo(\"GET\", \"mykey2\", \"myid\").Str(\"\"),\n\t\tDo(\"GET\", \"mykey2\", \"myid\").JSON().Err(\"key not found\"),\n\t\tDo(\"GET\", \"mykey\", \"myid2\").Str(\"\"),\n\t\tDo(\"GET\", \"mykey\", \"myid2\").JSON().Err(\"id not found\"),\n\t\tDo(\"GET\", \"mykey\", \"myid\", \"point\").JSON().Str(`{\"ok\":true,\"point\":{\"lat\":33,\"lon\":-112}}`),\n\t\tDo(\"GET\", \"mykey\", \"myid\", \"object\").JSON().Str(`{\"ok\":true,\"object\":{\"type\":\"Point\",\"coordinates\":[-112,33]}}`),\n\t\tDo(\"GET\", \"mykey\", \"myid\", \"hash\", \"1\").JSON().Str(`{\"ok\":true,\"hash\":\"9\"}`),\n\t\tDo(\"GET\", \"mykey\", \"myid\", \"bounds\").JSON().Str(`{\"ok\":true,\"bounds\":{\"sw\":{\"lat\":33,\"lon\":-112},\"ne\":{\"lat\":33,\"lon\":-112}}}`),\n\t\tDo(\"SET\", \"mykey\", \"myid2\", \"point\", 33, -112, 55).OK(),\n\t\tDo(\"GET\", \"mykey\", \"myid2\", \"point\").Str(\"[33 -112 55]\"),\n\t\tDo(\"GET\", \"mykey\", \"myid2\", \"point\").JSON().Str(`{\"ok\":true,\"point\":{\"lat\":33,\"lon\":-112,\"z\":55}}`),\n\t\tDo(\"GET\", \"mykey\", \"myid\", \"withfields\").JSON().Str(`{\"ok\":true,\"object\":{\"type\":\"Point\",\"coordinates\":[-112,33]},\"fields\":{\"hello\":\"world\",\"hiya\":55}}`),\n\t)\n}\nfunc keys_KEYS_test(mc *mockServer) error {\n\treturn mc.DoBatch(\n\t\tDo(\"SET\", \"mykey11\", \"myid4\", \"STRING\", \"value\").OK(),\n\t\tDo(\"SET\", \"mykey22\", \"myid2\", \"HASH\", \"9my5xp7\").OK(),\n\t\tDo(\"SET\", \"mykey22\", \"myid1\", \"OBJECT\", `{\"type\":\"Point\",\"coordinates\":[-130,38,10]}`).OK(),\n\t\tDo(\"SET\", \"mykey11\", \"myid3\", \"OBJECT\", `{\"type\":\"Point\",\"coordinates\":[-110,25,-8]}`).OK(),\n\t\tDo(\"SET\", \"mykey42\", \"myid2\", \"HASH\", \"9my5xp7\").OK(),\n\t\tDo(\"SET\", \"mykey31\", \"myid4\", \"STRING\", \"value\").OK(),\n\t\tDo(\"SET\", \"mykey310\", \"myid5\", \"STRING\", \"value\").OK(),\n\t\tDo(\"KEYS\", \"*\").Str(\"[mykey11 mykey22 mykey31 mykey310 mykey42]\"),\n\t\tDo(\"KEYS\", \"*key*\").Str(\"[mykey11 mykey22 mykey31 mykey310 mykey42]\"),\n\t\tDo(\"KEYS\", \"mykey*\").Str(\"[mykey11 mykey22 mykey31 mykey310 mykey42]\"),\n\t\tDo(\"KEYS\", \"mykey4*\").Str(\"[mykey42]\"),\n\t\tDo(\"KEYS\", \"mykey*1\").Str(\"[mykey11 mykey31]\"),\n\t\tDo(\"KEYS\", \"mykey*1*\").Str(\"[mykey11 mykey31 mykey310]\"),\n\t\tDo(\"KEYS\", \"mykey*10\").Str(\"[mykey310]\"),\n\t\tDo(\"KEYS\", \"mykey*2\").Str(\"[mykey22 mykey42]\"),\n\t\tDo(\"KEYS\", \"*2\").Str(\"[mykey22 mykey42]\"),\n\t\tDo(\"KEYS\", \"*1*\").Str(\"[mykey11 mykey31 mykey310]\"),\n\t\tDo(\"KEYS\", \"mykey\").Str(\"[]\"),\n\t\tDo(\"KEYS\", \"mykey31\").Str(\"[mykey31]\"),\n\t\tDo(\"KEYS\", \"mykey[^3]*\").Str(\"[mykey11 mykey22 mykey42]\"),\n\t\tDo(\"KEYS\").Err(\"wrong number of arguments for 'keys' command\"),\n\t\tDo(\"KEYS\", \"*\").JSON().Str(`{\"ok\":true,\"keys\":[\"mykey11\",\"mykey22\",\"mykey31\",\"mykey310\",\"mykey42\"]}`),\n\t)\n}\nfunc keys_PERSIST_test(mc *mockServer) error {\n\treturn mc.DoBatch(\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"STRING\", \"value\").OK(),\n\t\tDo(\"EXPIRE\", \"mykey\", \"myid\", 2).Str(\"1\"),\n\t\tDo(\"PERSIST\", \"mykey\", \"myid\").Str(\"1\"),\n\t\tDo(\"PERSIST\", \"mykey\", \"myid\").Str(\"0\"),\n\t\tDo(\"PERSIST\", \"mykey\").Err(\"wrong number of arguments for 'persist' command\"),\n\t\tDo(\"PERSIST\", \"mykey2\", \"myid\").Str(\"0\"),\n\t\tDo(\"PERSIST\", \"mykey2\", \"myid\").JSON().Err(\"key not found\"),\n\t\tDo(\"PERSIST\", \"mykey\", \"myid2\").Str(\"0\"),\n\t\tDo(\"PERSIST\", \"mykey\", \"myid2\").JSON().Err(\"id not found\"),\n\t\tDo(\"EXPIRE\", \"mykey\", \"myid\", 2).Str(\"1\"),\n\t\tDo(\"PERSIST\", \"mykey\", \"myid\").JSON().OK(),\n\t)\n}\nfunc keys_SET_test(mc *mockServer) error {\n\treturn mc.DoBatch(\n\t\t// Section: point\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"POINT\", 33, -115).OK(),\n\t\tDo(\"GET\", \"mykey\", \"myid\", \"POINT\").Str(\"[33 -115]\"),\n\t\tDo(\"GET\", \"mykey\", \"myid\", \"BOUNDS\").Str(\"[[33 -115] [33 -115]]\"),\n\t\tDo(\"GET\", \"mykey\", \"myid\", \"OBJECT\").Str(`{\"type\":\"Point\",\"coordinates\":[-115,33]}`),\n\t\tDo(\"GET\", \"mykey\", \"myid\", \"HASH\", 7).Str(\"9my5xp7\"),\n\t\tDo(\"DEL\", \"mykey\", \"myid\").Str(\"1\"),\n\t\tDo(\"GET\", \"mykey\", \"myid\").Str(\"\"),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"point\", \"33\", \"-112\", \"99\").OK(),\n\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"POINT\", 33, -115, \"RETURN\").Str(`{\"type\":\"Point\",\"coordinates\":[-115,33]}`),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"POINT\", 33, -115, \"RETURN\", \"POINT\").Str(\"[33 -115]\"),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"POINT\", 33, -115, \"RETURN\", \"OBJECT\").Str(`{\"type\":\"Point\",\"coordinates\":[-115,33]}`),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"POINT\", 33, -115, \"RETURN\", \"BOUNDS\").Str(\"[[33 -115] [33 -115]]\"),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"POINT\", 33, -115, \"RETURN\", \"OBJECT\").Str(`{\"type\":\"Point\",\"coordinates\":[-115,33]}`),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"POINT\", 33, -115, \"RETURN\", \"HASH\", 7).Str(\"9my5xp7\"),\n\t\tDo(\"DEL\", \"mykey\", \"myid\").Str(\"1\"),\n\t\tDo(\"GET\", \"mykey\", \"myid\").Str(\"\"),\n\n\t\t// Section: object\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"OBJECT\", `{\"type\":\"Point\",\"coordinates\":[-115,33]}`).OK(),\n\t\tDo(\"GET\", \"mykey\", \"myid\", \"POINT\").Str(\"[33 -115]\"),\n\t\tDo(\"GET\", \"mykey\", \"myid\", \"BOUNDS\").Str(\"[[33 -115] [33 -115]]\"),\n\t\tDo(\"GET\", \"mykey\", \"myid\", \"OBJECT\").Str(`{\"type\":\"Point\",\"coordinates\":[-115,33]}`),\n\t\tDo(\"GET\", \"mykey\", \"myid\", \"HASH\", 7).Str(\"9my5xp7\"),\n\t\tDo(\"DEL\", \"mykey\", \"myid\").Str(\"1\"),\n\t\tDo(\"GET\", \"mykey\", \"myid\").Str(\"\"),\n\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"OBJECT\", `{\"type\":\"Point\",\"coordinates\":[-115,33]}`, \"RETURN\").Str(`{\"type\":\"Point\",\"coordinates\":[-115,33]}`),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"OBJECT\", `{\"type\":\"Point\",\"coordinates\":[-115,33]}`, \"RETURN\", \"POINT\").Str(\"[33 -115]\"),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"OBJECT\", `{\"type\":\"Point\",\"coordinates\":[-115,33]}`, \"RETURN\", \"BOUNDS\").Str(\"[[33 -115] [33 -115]]\"),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"OBJECT\", `{\"type\":\"Point\",\"coordinates\":[-115,33]}`, \"RETURN\", \"OBJECT\").Str(`{\"type\":\"Point\",\"coordinates\":[-115,33]}`),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"OBJECT\", `{\"type\":\"Point\",\"coordinates\":[-115,33]}`, \"RETURN\", \"HASH\", 7).Str(\"9my5xp7\"),\n\t\tDo(\"DEL\", \"mykey\", \"myid\").Str(\"1\"),\n\t\tDo(\"GET\", \"mykey\", \"myid\").Str(\"\"),\n\n\t\t// Section: bounds\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"BOUNDS\", 33, -115, 33, -115).OK(),\n\t\tDo(\"GET\", \"mykey\", \"myid\", \"POINT\").Str(\"[33 -115]\"),\n\t\tDo(\"GET\", \"mykey\", \"myid\", \"BOUNDS\").Str(\"[[33 -115] [33 -115]]\"),\n\t\tDo(\"GET\", \"mykey\", \"myid\", \"OBJECT\").Str(`{\"type\":\"Polygon\",\"coordinates\":[[[-115,33],[-115,33],[-115,33],[-115,33],[-115,33]]]}`),\n\t\tDo(\"GET\", \"mykey\", \"myid\", \"HASH\", 7).Str(\"9my5xp7\"),\n\t\tDo(\"DEL\", \"mykey\", \"myid\").Str(\"1\"),\n\t\tDo(\"GET\", \"mykey\", \"myid\").Str(\"\"),\n\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"BOUNDS\", 33, -115, 33, -115, \"RETURN\").Str(`{\"type\":\"Polygon\",\"coordinates\":[[[-115,33],[-115,33],[-115,33],[-115,33],[-115,33]]]}`),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"BOUNDS\", 33, -115, 33, -115, \"RETURN\", \"POINT\").Str(\"[33 -115]\"),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"BOUNDS\", 33, -115, 33, -115, \"RETURN\", \"BOUNDS\").Str(\"[[33 -115] [33 -115]]\"),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"BOUNDS\", 33, -115, 33, -115, \"RETURN\", \"OBJECT\").Str(`{\"type\":\"Polygon\",\"coordinates\":[[[-115,33],[-115,33],[-115,33],[-115,33],[-115,33]]]}`),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"BOUNDS\", 33, -115, 33, -115, \"RETURN\", \"HASH\", 7).Str(\"9my5xp7\"),\n\t\tDo(\"DEL\", \"mykey\", \"myid\").Str(\"1\"),\n\t\tDo(\"GET\", \"mykey\", \"myid\").Str(\"\"),\n\n\t\t// Section: hash\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"HASH\", \"9my5xp7\").OK(),\n\t\tDo(\"GET\", \"mykey\", \"myid\", \"HASH\", 7).Str(\"9my5xp7\"),\n\t\tDo(\"DEL\", \"mykey\", \"myid\").Str(\"1\"),\n\t\tDo(\"GET\", \"mykey\", \"myid\").Str(\"\"),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"HASH\", \"9my5xp7\").JSON().OK(),\n\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"HASH\", \"9my5xp7\", \"RETURN\").JSON().OK(),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"HASH\", \"9my5xp7\", \"RETURN\", \"HASH\", 7).JSON().Str(`{\"ok\":true,\"hash\":\"9my5xp7\"}`),\n\t\tDo(\"DEL\", \"mykey\", \"myid\").Str(\"1\"),\n\t\tDo(\"GET\", \"mykey\", \"myid\").Str(\"\"),\n\n\t\t// Section: field\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"FIELD\", \"f1\", 33, \"FIELD\", \"a2\", 44.5, \"HASH\", \"9my5xp7\").OK(),\n\t\tDo(\"GET\", \"mykey\", \"myid\", \"WITHFIELDS\", \"HASH\", 7).Str(\"[9my5xp7 [a2 44.5 f1 33]]\"),\n\t\tDo(\"DEL\", \"mykey\", \"myid\").Str(\"1\"),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"FIELD\", \"f1\", 33, \"FIELD\", \"a2\", 44.5, \"HASH\", \"9my5xp7\", \"RETURN\", \"WITHFIELDS\", \"HASH\", 7).Str(\"[9my5xp7 [a2 44.5 f1 33]]\"),\n\t\tDo(\"FSET\", \"mykey\", \"myid\", \"f1\", 0).Str(\"1\"),\n\t\tDo(\"FSET\", \"mykey\", \"myid\", \"f1\", 0).Str(\"0\"),\n\t\tDo(\"GET\", \"mykey\", \"myid\", \"WITHFIELDS\", \"HASH\", 7).Str(\"[9my5xp7 [a2 44.5]]\"),\n\t\tDo(\"DEL\", \"mykey\", \"myid\").Str(\"1\"),\n\t\tDo(\"GET\", \"mykey\", \"myid\").Str(\"\"),\n\n\t\t// Section: string\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"STRING\", \"value\").OK(),\n\t\tDo(\"GET\", \"mykey\", \"myid\").Str(\"value\"),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"STRING\", \"value2\").OK(),\n\t\tDo(\"GET\", \"mykey\", \"myid\").Str(\"value2\"),\n\t\tDo(\"DEL\", \"mykey\", \"myid\").Str(\"1\"),\n\t\tDo(\"GET\", \"mykey\", \"myid\").Str(\"\"),\n\n\t\t// Test error conditions\n\t\tDo(\"CONFIG\", \"SET\", \"maxmemory\", \"1\").OK(),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"STRING\", \"value2\").Err(\"OOM command not allowed when used memory > 'maxmemory'\"),\n\t\tDo(\"CONFIG\", \"SET\", \"maxmemory\", \"0\").OK(),\n\t\tDo(\"SET\").Err(\"wrong number of arguments for 'set' command\"),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"FIELD\", \"f1\").Err(\"wrong number of arguments for 'set' command\"),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"FIELD\", \"z\", \"1\").Err(\"invalid argument 'z'\"),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"EX\").Err(\"wrong number of arguments for 'set' command\"),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"EX\", \"yyy\").Err(\"invalid argument 'yyy'\"),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"EX\", \"123\").Err(\"wrong number of arguments for 'set' command\"),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"nx\").Err(\"wrong number of arguments for 'set' command\"),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"nx\", \"xx\").Err(\"invalid argument 'xx'\"),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"xx\", \"nx\").Err(\"invalid argument 'nx'\"),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"string\").Err(\"wrong number of arguments for 'set' command\"),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"point\").Err(\"wrong number of arguments for 'set' command\"),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"point\", \"33\").Err(\"wrong number of arguments for 'set' command\"),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"point\", \"33f\", \"-112\").Err(\"invalid argument '33f'\"),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"point\", \"33\", \"-112f\").Err(\"invalid argument '-112f'\"),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"point\", \"33\", \"-112f\", \"99\").Err(\"invalid argument '-112f'\"),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"bounds\").Err(\"wrong number of arguments for 'set' command\"),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"bounds\", \"fff\", \"1\", \"2\", \"3\").Err(\"invalid argument 'fff'\"),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"hash\").Err(\"wrong number of arguments for 'set' command\"),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"object\").Err(\"wrong number of arguments for 'set' command\"),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"object\", \"asd\").Err(\"invalid data\"),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"joint\").Err(\"invalid argument 'joint'\"),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"XX\", \"HASH\", \"9my5xp7\").Err(\"\"),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"XX\", \"HASH\", \"9my5xp7\").JSON().Err(\"id not found\"),\n\t\tDo(\"SET\", \"mykey\", \"myid1\", \"HASH\", \"9my5xp7\").OK(),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"XX\", \"HASH\", \"9my5xp7\").Err(\"\"),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"NX\", \"HASH\", \"9my5xp7\").OK(),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"XX\", \"HASH\", \"9my5xp7\").OK(),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"NX\", \"HASH\", \"9my5xp7\").Err(\"\"),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"NX\", \"HASH\", \"9my5xp7\").JSON().Err(\"id already exists\"),\n\t)\n}\n\nfunc keys_STATS_test(mc *mockServer) error {\n\treturn mc.DoBatch(\n\t\tDo(\"STATS\", \"mykey\").Str(\"[nil]\"),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"STRING\", \"value\").OK(),\n\t\tDo(\"STATS\", \"mykey\").Str(\"[[in_memory_size 9 num_objects 1 num_points 0 num_strings 1]]\"),\n\t\tDo(\"STATS\", \"mykey\", \"hello\").JSON().Str(`{\"ok\":true,\"stats\":[{\"in_memory_size\":9,\"num_objects\":1,\"num_points\":0,\"num_strings\":1},null]}`),\n\t\tDo(\"SET\", \"mykey\", \"myid2\", \"STRING\", \"value\").OK(),\n\t\tDo(\"STATS\", \"mykey\").Str(\"[[in_memory_size 19 num_objects 2 num_points 0 num_strings 2]]\"),\n\t\tDo(\"SET\", \"mykey\", \"myid3\", \"OBJECT\", `{\"type\":\"Point\",\"coordinates\":[-115,33]}`).OK(),\n\t\tDo(\"STATS\", \"mykey\").Str(\"[[in_memory_size 40 num_objects 3 num_points 1 num_strings 2]]\"),\n\t\tDo(\"DEL\", \"mykey\", \"myid\").Str(\"1\"),\n\t\tDo(\"STATS\", \"mykey\").Str(\"[[in_memory_size 31 num_objects 2 num_points 1 num_strings 1]]\"),\n\t\tDo(\"DEL\", \"mykey\", \"myid3\").Str(\"1\"),\n\t\tDo(\"STATS\", \"mykey\").Str(\"[[in_memory_size 10 num_objects 1 num_points 0 num_strings 1]]\"),\n\t\tDo(\"STATS\", \"mykey\", \"mykey2\").Str(\"[[in_memory_size 10 num_objects 1 num_points 0 num_strings 1] nil]\"),\n\t\tDo(\"DEL\", \"mykey\", \"myid2\").Str(\"1\"),\n\t\tDo(\"STATS\", \"mykey\").Str(\"[nil]\"),\n\t\tDo(\"STATS\", \"mykey\", \"mykey2\").Str(\"[nil nil]\"),\n\t\tDo(\"STATS\", \"mykey\").Str(\"[nil]\"),\n\t\tDo(\"STATS\").Err(`wrong number of arguments for 'stats' command`),\n\t)\n}\nfunc keys_TTL_test(mc *mockServer) error {\n\treturn mc.DoBatch(\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"STRING\", \"value\").OK(),\n\t\tDo(\"EXPIRE\", \"mykey\", \"myid\", 2).Str(\"1\"),\n\t\tDo(\"EXPIRE\", \"mykey\", \"myid\", 2).JSON().OK(),\n\t\tSleep(time.Millisecond*10),\n\t\tDo(\"TTL\", \"mykey\", \"myid\").Str(\"1\"),\n\t\tDo(\"EXPIRE\", \"mykey\", \"myid\", 1).Str(\"1\"),\n\t\tSleep(time.Millisecond*10),\n\t\tDo(\"TTL\", \"mykey\", \"myid\").Str(\"0\"),\n\t\tDo(\"TTL\", \"mykey\", \"myid\").JSON().Str(`{\"ok\":true,\"ttl\":0}`),\n\t\tDo(\"TTL\", \"mykey2\", \"myid\").Str(\"-2\"),\n\t\tDo(\"TTL\", \"mykey\", \"myid2\").Str(\"-2\"),\n\t\tDo(\"TTL\", \"mykey\").Err(\"wrong number of arguments for 'ttl' command\"),\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"STRING\", \"value\").OK(),\n\t\tDo(\"TTL\", \"mykey\", \"myid\").Str(\"-1\"),\n\t\tDo(\"TTL\", \"mykey2\", \"myid\").JSON().Err(\"key not found\"),\n\t\tDo(\"TTL\", \"mykey\", \"myid2\").JSON().Err(\"id not found\"),\n\t)\n}\nfunc keys_EXISTS_test(mc *mockServer) error {\n\treturn mc.DoBatch(\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"STRING\", \"value\").OK(),\n\t\tDo(\"EXISTS\", \"mykey\", \"myid\").Str(\"1\"),\n\t\tDo(\"EXISTS\", \"mykey\", \"myid\").JSON().Str(`{\"ok\":true,\"exists\":true}`),\n\t\tDo(\"EXISTS\", \"mykey\", \"myid2\").Str(\"0\"),\n\t\tDo(\"EXISTS\", \"mykey\", \"myid2\").JSON().Str(`{\"ok\":true,\"exists\":false}`),\n\t\tDo(\"EXISTS\", \"mykey\").Err(\"wrong number of arguments for 'exists' command\"),\n\t\tDo(\"EXISTS\", \"mykey2\", \"myid\").JSON().Err(\"key not found\"),\n\t)\n}\nfunc keys_FEXISTS_test(mc *mockServer) error {\n\treturn mc.DoBatch(\n\t\tDo(\"SET\", \"mykey\", \"myid\", \"FIELD\", \"f1\", \"123\", \"STRING\", \"value\").OK(),\n\t\tDo(\"FEXISTS\", \"mykey\", \"myid\", \"f1\").Str(\"1\"),\n\t\tDo(\"FEXISTS\", \"mykey\", \"myid\", \"f1\").JSON().Str(`{\"ok\":true,\"exists\":true}`),\n\t\tDo(\"FEXISTS\", \"mykey\", \"myid\", \"f2\").Str(\"0\"),\n\t\tDo(\"FEXISTS\", \"mykey\", \"myid\", \"f2\").JSON().Str(`{\"ok\":true,\"exists\":false}`),\n\t\tDo(\"FEXISTS\", \"mykey\", \"myid\").Err(\"wrong number of arguments for 'fexists' command\"),\n\t\tDo(\"FEXISTS\", \"mykey2\", \"myid\", \"f2\").JSON().Err(\"key not found\"),\n\t\tDo(\"FEXISTS\", \"mykey\", \"myid2\", \"f2\").JSON().Err(\"id not found\"),\n\t)\n}\n\nfunc keys_SET_EX_test(mc *mockServer) (err error) {\n\trand.Seed(time.Now().UnixNano())\n\n\t// add a bunch of points\n\tfor i := 0; i < 20000; i++ {\n\t\tval := fmt.Sprintf(\"val:%d\", i)\n\t\tvar resp string\n\t\tvar lat, lon float64\n\t\tlat = rand.Float64()*180 - 90\n\t\tlon = rand.Float64()*360 - 180\n\t\tresp, err = redis.String(mc.conn.Do(\"SET\",\n\t\t\tfmt.Sprintf(\"mykey%d\", i%3), val,\n\t\t\t\"EX\", 1+rand.Float64(),\n\t\t\t\"POINT\", lat, lon))\n\t\tif err != nil {\n\t\t\treturn\n\t\t}\n\t\tif resp != \"OK\" {\n\t\t\terr = fmt.Errorf(\"expected 'OK', got '%s'\", resp)\n\t\t\treturn\n\t\t}\n\t\ttime.Sleep(time.Nanosecond)\n\t}\n\ttime.Sleep(time.Second * 3)\n\tmc.conn.Do(\"OUTPUT\", \"json\")\n\tjson, _ := redis.String(mc.conn.Do(\"SERVER\"))\n\tif !gjson.Get(json, \"ok\").Bool() {\n\t\treturn errors.New(\"not ok\")\n\t}\n\tif gjson.Get(json, \"stats.num_objects\").Int() > 0 {\n\t\treturn errors.New(\"items left in database\")\n\t}\n\tmc.conn.Do(\"FLUSHDB\")\n\treturn nil\n}\n\nfunc keys_FIELDS_test(mc *mockServer) error {\n\treturn mc.DoBatch(\n\t\tDo(\"SET\", \"mykey\", \"myid1a\", \"FIELD\", \"a\", 1, \"POINT\", 33, -115).OK(),\n\t\tDo(\"GET\", \"mykey\", \"myid1a\", \"WITHFIELDS\").Str(`[{\"type\":\"Point\",\"coordinates\":[-115,33]} [a 1]]`),\n\t\tDo(\"SET\", \"mykey\", \"myid1a\", \"FIELD\", \"a\", \"a\", \"POINT\", 33, -115).OK(),\n\t\tDo(\"GET\", \"mykey\", \"myid1a\", \"WITHFIELDS\").Str(`[{\"type\":\"Point\",\"coordinates\":[-115,33]} [a a]]`),\n\t\tDo(\"SET\", \"mykey\", \"myid1a\", \"FIELD\", \"a\", 1, \"FIELD\", \"b\", 2, \"POINT\", 33, -115).OK(),\n\t\tDo(\"GET\", \"mykey\", \"myid1a\", \"WITHFIELDS\").Str(`[{\"type\":\"Point\",\"coordinates\":[-115,33]} [a 1 b 2]]`),\n\t\tDo(\"SET\", \"mykey\", \"myid1a\", \"FIELD\", \"b\", 2, \"POINT\", 33, -115).OK(),\n\t\tDo(\"GET\", \"mykey\", \"myid1a\", \"WITHFIELDS\").Str(`[{\"type\":\"Point\",\"coordinates\":[-115,33]} [a 1 b 2]]`),\n\t\tDo(\"SET\", \"mykey\", \"myid1a\", \"FIELD\", \"b\", 2, \"FIELD\", \"a\", \"1\", \"FIELD\", \"c\", 3, \"POINT\", 33, -115).OK(),\n\t\tDo(\"GET\", \"mykey\", \"myid1a\", \"WITHFIELDS\").Str(`[{\"type\":\"Point\",\"coordinates\":[-115,33]} [a 1 b 2 c 3]]`),\n\n\t\tDo(\"GET\", \"fleet\", \"truck1\", \"WITHFIELDS\").JSON().Err(\"key not found\"),\n\t\tDo(\"SET\", \"fleet\", \"truck1\", \"FIELD\", \"speed\", \"0\", \"POINT\", \"-112\", \"33\").JSON().OK(),\n\t\tDo(\"GET\", \"fleet\", \"truck1\", \"WITHFIELDS\").JSON().Str(`{\"ok\":true,\"object\":{\"type\":\"Point\",\"coordinates\":[33,-112]}}`),\n\t\tDo(\"SET\", \"fleet\", \"truck1\", \"FIELD\", \"speed\", \"1\", \"POINT\", \"-112\", \"33\").JSON().OK(),\n\t\tDo(\"GET\", \"fleet\", \"truck1\", \"WITHFIELDS\").JSON().Str(`{\"ok\":true,\"object\":{\"type\":\"Point\",\"coordinates\":[33,-112]},\"fields\":{\"speed\":1}}`),\n\t\tDo(\"SET\", \"fleet\", \"truck1\", \"FIELD\", \"speed\", \"0\", \"POINT\", \"-112\", \"33\").JSON().OK(),\n\t\tDo(\"GET\", \"fleet\", \"truck1\", \"WITHFIELDS\").JSON().Str(`{\"ok\":true,\"object\":{\"type\":\"Point\",\"coordinates\":[33,-112]}}`),\n\t\tDo(\"SET\", \"fleet\", \"truck1\", \"FIELD\", \"speed\", \"2\", \"POINT\", \"-112\", \"33\").JSON().OK(),\n\t\tDo(\"GET\", \"fleet\", \"truck1\", \"WITHFIELDS\").JSON().Str(`{\"ok\":true,\"object\":{\"type\":\"Point\",\"coordinates\":[33,-112]},\"fields\":{\"speed\":2}}`),\n\n\t\t// Do some whereins queries\n\t\tDo(\"SET\", \"whereins\", \"id1\", \"FIELD\", \"test\", \"2\", \"POINT\", \"-112\", \"33\").JSON().OK(),\n\t\tDo(\"SET\", \"whereins\", \"id2\", \"FIELD\", \"TEST\", \"3\", \"POINT\", \"-111\", \"32\").JSON().OK(),\n\t\tDo(\"SET\", \"whereins\", \"id3\", \"FIELD\", \"teSt\", \"4\", \"POINT\", \"-110\", \"31\").JSON().OK(),\n\t\tDo(\"SCAN\", \"whereins\", \"WHEREIN\", \"test\", \"1\", \"2\", \"OBJECTS\").JSON().Str(`{\"ok\":true,\"fields\":[\"test\"],\"objects\":[{\"id\":\"id1\",\"object\":{\"type\":\"Point\",\"coordinates\":[33,-112]},\"fields\":[2]}],\"count\":1,\"cursor\":0}`),\n\t\tDo(\"SCAN\", \"whereins\", \"WHEREIN\", \"TEST\", \"1\", \"3\", \"OBJECTS\").JSON().Str(`{\"ok\":true,\"fields\":[\"TEST\"],\"objects\":[{\"id\":\"id2\",\"object\":{\"type\":\"Point\",\"coordinates\":[32,-111]},\"fields\":[3]}],\"count\":1,\"cursor\":0}`),\n\t\tDo(\"SCAN\", \"whereins\", \"WHEREIN\", \"teSt\", \"1\", \"4\", \"OBJECTS\").JSON().Str(`{\"ok\":true,\"fields\":[\"teSt\"],\"objects\":[{\"id\":\"id3\",\"object\":{\"type\":\"Point\",\"coordinates\":[31,-110]},\"fields\":[4]}],\"count\":1,\"cursor\":0}`),\n\t\tDo(\"SCAN\", \"whereins\", \"OBJECTS\").JSON().Str(`{\"ok\":true,\"fields\":[\"TEST\",\"teSt\",\"test\"],\"objects\":[{\"id\":\"id1\",\"object\":{\"type\":\"Point\",\"coordinates\":[33,-112]},\"fields\":[0,0,2]},{\"id\":\"id2\",\"object\":{\"type\":\"Point\",\"coordinates\":[32,-111]},\"fields\":[3,0,0]},{\"id\":\"id3\",\"object\":{\"type\":\"Point\",\"coordinates\":[31,-110]},\"fields\":[0,4,0]}],\"count\":3,\"cursor\":0}`),\n\n\t\t// Do some GJSON queries.\n\t\tDo(\"SET\", \"fleet\", \"truck2\", \"FIELD\", \"hello\", `{\"world\":\"tom\"}`, \"POINT\", \"-112\", \"33\").JSON().OK(),\n\t\tDo(\"SCAN\", \"fleet\", \"WHERE\", \"hello\", `{\"world\":\"tom\"}`, `{\"world\":\"tom\"}`, \"COUNT\").JSON().Str(`{\"ok\":true,\"count\":1,\"cursor\":0}`),\n\t\tDo(\"SCAN\", \"fleet\", \"WHERE\", \"hello.world == 'tom'\", \"COUNT\").JSON().Str(`{\"ok\":true,\"count\":1,\"cursor\":0}`),\n\t\t// The next scan does not match on anything, but since we're matching\n\t\t// on zeros, which is the default, then all (two) objects are returned.\n\t\tDo(\"SCAN\", \"fleet\", \"WHERE\", \"hello.world.1\", `0`, `0`, \"IDS\").JSON().Str(`{\"ok\":true,\"ids\":[\"truck1\",\"truck2\"],\"count\":2,\"cursor\":0}`),\n\t\tDo(\"SCAN\", \"fleet\", \"WHERE\", \"hello.world\", \">\", `tom`, \"IDS\").JSON().Str(`{\"ok\":true,\"ids\":[],\"count\":0,\"cursor\":0}`),\n\t\tDo(\"SCAN\", \"fleet\", \"WHERE\", \"hello.world\", \">=\", `Tom`, \"IDS\").JSON().Str(`{\"ok\":true,\"ids\":[\"truck2\"],\"count\":1,\"cursor\":0}`),\n\t\tDo(\"SCAN\", \"fleet\", \"WHERE\", \"hello.world\", \">=\", `tom`, \"IDS\").JSON().Str(`{\"ok\":true,\"ids\":[\"truck2\"],\"count\":1,\"cursor\":0}`),\n\t\tDo(\"SCAN\", \"fleet\", \"WHERE\", \"hello.world\", \"==\", `tom`, \"IDS\").JSON().Str(`{\"ok\":true,\"ids\":[\"truck2\"],\"count\":1,\"cursor\":0}`),\n\t\tDo(\"SCAN\", \"fleet\", \"WHERE\", \"hello.world\", \"<\", `tom`, \"IDS\").JSON().Str(`{\"ok\":true,\"ids\":[\"truck1\"],\"count\":1,\"cursor\":0}`),\n\t\tDo(\"SCAN\", \"fleet\", \"WHERE\", \"hello.world\", \"<=\", `tom`, \"IDS\").JSON().Str(`{\"ok\":true,\"ids\":[\"truck1\",\"truck2\"],\"count\":2,\"cursor\":0}`),\n\t\tDo(\"SCAN\", \"fleet\", \"WHERE\", \"hello.world\", \"<\", `uom`, \"IDS\").JSON().Str(`{\"ok\":true,\"ids\":[\"truck1\",\"truck2\"],\"count\":2,\"cursor\":0}`),\n\t\tDo(\"SCAN\", \"fleet\", \"WHERE\", \"hello.world\", \"!=\", `tom`, \"IDS\").JSON().Str(`{\"ok\":true,\"ids\":[\"truck1\"],\"count\":1,\"cursor\":0}`),\n\t\t// Test REGEX on FIELD\n\t\tDo(\"SCAN\", \"fleet\", \"WHERE\", \"hello.world =~ 'tom.*'\", \"IDS\").JSON().Str(`{\"ok\":true,\"ids\":[\"truck2\"],\"count\":1,\"cursor\":0}`),\n\t\tDo(\"SCAN\", \"fleet\", \"WHERE\", \"hello.world =~ 'foo.*'\", \"IDS\").JSON().Str(`{\"ok\":true,\"ids\":[],\"count\":0,\"cursor\":0}`),\n\t\tDo(\"SCAN\", \"fleet\", \"WHERE\", \"hello.world =~ '(*'\", \"IDS\").JSON().Str(`{\"ok\":true,\"ids\":[],\"count\":0,\"cursor\":0}`),\n\n\t\tDo(\"SET\", \"fleet\", \"truck1\", \"OBJECT\", `{\"type\":\"Feature\",\"geometry\":{\"type\":\"Point\",\"coordinates\":[-112,33]},\"properties\":{\"speed\":50},\"asdf\":\"Adsf\"}`).JSON().OK(),\n\t\tDo(\"SCAN\", \"fleet\", \"WHERE\", \"properties.speed\", \">\", 49, \"IDS\").JSON().Str(`{\"ok\":true,\"ids\":[\"truck1\"],\"count\":1,\"cursor\":0}`),\n\t\tDo(\"SCAN\", \"fleet\", \"WHERE\", \"properties.speed\", \">\", 50, \"IDS\").JSON().Str(`{\"ok\":true,\"ids\":[],\"count\":0,\"cursor\":0}`),\n\t\tDo(\"SCAN\", \"fleet\", \"WHERE\", \"properties.speed\", \"<\", 51, \"IDS\").JSON().Str(`{\"ok\":true,\"ids\":[\"truck1\",\"truck2\"],\"count\":2,\"cursor\":0}`),\n\t\t// Test REGEX on OBJECT properties\n\t\tDo(\"SET\", \"fleet\", \"truck3\", \"OBJECT\", `{\"type\":\"Feature\",\"geometry\":{\"type\":\"Point\",\"coordinates\":[-112,33]},\"properties\":{\"name\":\"truck01\"}}`).JSON().OK(),\n\t\tDo(\"SCAN\", \"fleet\", \"WHERE\", \"properties.name =~ 'truck.*'\", \"IDS\").JSON().Str(`{\"ok\":true,\"ids\":[\"truck3\"],\"count\":1,\"cursor\":0}`),\n\t\tDo(\"SCAN\", \"fleet\", \"WHERE\", \"properties.name =~ 'foo.*'\", \"IDS\").JSON().Str(`{\"ok\":true,\"ids\":[],\"count\":0,\"cursor\":0}`),\n\t\tDo(\"SCAN\", \"fleet\", \"WHERE\", \"properties.name =~ '(*'\", \"IDS\").JSON().Str(`{\"ok\":true,\"ids\":[],\"count\":0,\"cursor\":0}`),\n\n\t\tDo(\"DROP\", \"fleet\").JSON().OK(),\n\t\tDo(\"SET\", \"fleet\", \"truck1\", \"FIELD\", \"speed\", \"50\", \"POINT\", \"-112\", \"33\").JSON().OK(),\n\t\tDo(\"SET\", \"fleet\", \"truck1\", \"FIELD\", \"Speed\", \"51\", \"POINT\", \"-112\", \"33\").JSON().OK(),\n\t\tDo(\"SET\", \"fleet\", \"truck1\", \"FIELD\", \"speeD\", \"52\", \"POINT\", \"-112\", \"33\").JSON().OK(),\n\t\tDo(\"SET\", \"fleet\", \"truck1\", \"FIELD\", \"SpeeD\", \"53\", \"POINT\", \"-112\", \"33\").JSON().OK(),\n\t\tDo(\"GET\", \"fleet\", \"truck1\", \"WITHFIELDS\").JSON().Str(`{\"ok\":true,\"object\":{\"type\":\"Point\",\"coordinates\":[33,-112]},\"fields\":{\"SpeeD\":53,\"Speed\":51,\"speeD\":52,\"speed\":50}}`),\n\t\tDo(\"SCAN\", \"fleet\", \"WHERE\", \"speed == 50\", \"IDS\").JSON().Str(`{\"ok\":true,\"ids\":[\"truck1\"],\"count\":1,\"cursor\":0}`),\n\t\tDo(\"SCAN\", \"fleet\", \"WHERE\", \"Speed == 50\", \"IDS\").JSON().Str(`{\"ok\":true,\"ids\":[],\"count\":0,\"cursor\":0}`),\n\t\tDo(\"SCAN\", \"fleet\", \"WHERE\", \"Speed == 51\", \"IDS\").JSON().Str(`{\"ok\":true,\"ids\":[\"truck1\"],\"count\":1,\"cursor\":0}`),\n\t\tDo(\"SCAN\", \"fleet\", \"WHERE\", \"speed\", 50, 50, \"IDS\").JSON().Str(`{\"ok\":true,\"ids\":[\"truck1\"],\"count\":1,\"cursor\":0}`),\n\t\tDo(\"SCAN\", \"fleet\", \"WHERE\", \"Speed\", 51, 51, \"IDS\").JSON().Str(`{\"ok\":true,\"ids\":[\"truck1\"],\"count\":1,\"cursor\":0}`),\n\t\tDo(\"SCAN\", \"fleet\", \"WHERE\", \"Speed\", 50, 50, \"IDS\").JSON().Str(`{\"ok\":true,\"ids\":[],\"count\":0,\"cursor\":0}`),\n\t\tDo(\"SCAN\", \"fleet\", \"WHERE\", \"speed\", 51, 51, \"IDS\").JSON().Str(`{\"ok\":true,\"ids\":[],\"count\":0,\"cursor\":0}`),\n\n\t\tDo(\"DROP\", \"fleet\").JSON().OK(),\n\t\tDo(\"SET\", \"fleet\", \"truck1\", \"field\", \"props\", `{\"speed\":50,\"Speed\":51}`, \"point\", \"33\", \"-112\").JSON().OK(),\n\t\tDo(\"SET\", \"fleet\", \"truck1\", \"field\", \"Props\", `{\"speed\":52,\"Speed\":53}`, \"point\", \"33\", \"-112\").JSON().OK(),\n\t\tDo(\"GET\", \"fleet\", \"truck1\", \"WITHFIELDS\").JSON().Str(`{\"ok\":true,\"object\":{\"type\":\"Point\",\"coordinates\":[-112,33]},\"fields\":{\"Props\":{\"speed\":52,\"Speed\":53},\"props\":{\"speed\":50,\"Speed\":51}}}`),\n\t\tDo(\"SCAN\", \"fleet\", \"WHERE\", \"props.speed == 50\", \"IDS\").JSON().Str(`{\"ok\":true,\"ids\":[\"truck1\"],\"count\":1,\"cursor\":0}`),\n\t\tDo(\"SCAN\", \"fleet\", \"WHERE\", \"props.Speed == 51\", \"IDS\").JSON().Str(`{\"ok\":true,\"ids\":[\"truck1\"],\"count\":1,\"cursor\":0}`),\n\t\tDo(\"SCAN\", \"fleet\", \"WHERE\", \"Props.speed == 52\", \"IDS\").JSON().Str(`{\"ok\":true,\"ids\":[\"truck1\"],\"count\":1,\"cursor\":0}`),\n\t\tDo(\"SCAN\", \"fleet\", \"WHERE\", \"Props.Speed == 53\", \"IDS\").JSON().Str(`{\"ok\":true,\"ids\":[\"truck1\"],\"count\":1,\"cursor\":0}`),\n\t\tDo(\"SCAN\", \"fleet\", \"WHERE\", \"props.Speed == 52\", \"IDS\").JSON().Str(`{\"ok\":true,\"ids\":[],\"count\":0,\"cursor\":0}`),\n\t\tDo(\"SCAN\", \"fleet\", \"WHERE\", \"props.speed == 51\", \"IDS\").JSON().Str(`{\"ok\":true,\"ids\":[],\"count\":0,\"cursor\":0}`),\n\t\tDo(\"SCAN\", \"fleet\", \"WHERE\", \"Props.speed == 53\", \"IDS\").JSON().Str(`{\"ok\":true,\"ids\":[],\"count\":0,\"cursor\":0}`),\n\t\tDo(\"SCAN\", \"fleet\", \"WHERE\", \"Props.Speed == 50\", \"IDS\").JSON().Str(`{\"ok\":true,\"ids\":[],\"count\":0,\"cursor\":0}`),\n\t\tDo(\"SCAN\", \"fleet\", \"WHERE\", \"props.speed > 49\", \"IDS\").JSON().Str(`{\"ok\":true,\"ids\":[\"truck1\"],\"count\":1,\"cursor\":0}`),\n\t\tDo(\"SCAN\", \"fleet\", \"WHERE\", \"props.Speed > 49\", \"IDS\").JSON().Str(`{\"ok\":true,\"ids\":[\"truck1\"],\"count\":1,\"cursor\":0}`),\n\t\tDo(\"SCAN\", \"fleet\", \"WHERE\", \"Props.speed > 49\", \"IDS\").JSON().Str(`{\"ok\":true,\"ids\":[\"truck1\"],\"count\":1,\"cursor\":0}`),\n\t\tDo(\"SCAN\", \"fleet\", \"WHERE\", \"Props.Speed > 49\", \"IDS\").JSON().Str(`{\"ok\":true,\"ids\":[\"truck1\"],\"count\":1,\"cursor\":0}`),\n\t\tDo(\"SCAN\", \"fleet\", \"WHERE\", \"props.Speed > 53\", \"IDS\").JSON().Str(`{\"ok\":true,\"ids\":[],\"count\":0,\"cursor\":0}`),\n\t\tDo(\"SCAN\", \"fleet\", \"WHERE\", \"props.speed > 53\", \"IDS\").JSON().Str(`{\"ok\":true,\"ids\":[],\"count\":0,\"cursor\":0}`),\n\t\tDo(\"SCAN\", \"fleet\", \"WHERE\", \"Props.speed > 53\", \"IDS\").JSON().Str(`{\"ok\":true,\"ids\":[],\"count\":0,\"cursor\":0}`),\n\t\tDo(\"SCAN\", \"fleet\", \"WHERE\", \"Props.Speed > 53\", \"IDS\").JSON().Str(`{\"ok\":true,\"ids\":[],\"count\":0,\"cursor\":0}`),\n\n\t\tDo(\"DROP\", \"fleet\").JSON().OK(),\n\t\tDo(\"SET\", \"fleet\", \"1\", \"field\", \"teamId\", \"1\", \"field\", \"optionalId\", \"999\", \"point\", \"0\", \"0\").JSON().OK(),\n\t\tDo(\"SET\", \"fleet\", \"2\", \"field\", \"teamId\", \"1\", \"point\", \"0\", \"0\").JSON().OK(),\n\t\tDo(\"SCAN\", \"fleet\", \"COUNT\").JSON().Str(`{\"ok\":true,\"count\":2,\"cursor\":0}`),\n\t\tDo(\"SCAN\", \"fleet\", \"WHEREIN\", \"teamId\", \"1\", \"1\", \"COUNT\").JSON().Str(`{\"ok\":true,\"count\":2,\"cursor\":0}`),\n\n\t\tDo(\"SCAN\", \"fleet\", \"WHEREIN\", \"teamId\", \"1\", \"1\", \"WHERE\", \"!optionalId || optionalId == 999\", \"count\").JSON().Str(`{\"ok\":true,\"count\":2,\"cursor\":0}`),\n\t\tDo(\"SCAN\", \"fleet\", \"WHEREIN\", \"teamId\", \"1\", \"1\", \"WHERE\", \"!!!optionalId || optionalId == 999\", \"count\").JSON().Str(`{\"ok\":true,\"count\":2,\"cursor\":0}`),\n\t\tDo(\"SCAN\", \"fleet\", \"WHEREIN\", \"teamId\", \"1\", \"1\", \"WHERE\", \"optionalId == 0 || optionalId == 999\", \"count\").JSON().Str(`{\"ok\":true,\"count\":2,\"cursor\":0}`),\n\t\tDo(\"SCAN\", \"fleet\", \"WHEREIN\", \"teamId\", \"1\", \"1\", \"WHERE\", \"1 == 1 || optionalId == 999\", \"count\").JSON().Str(`{\"ok\":true,\"count\":2,\"cursor\":0}`),\n\t)\n}\n\nfunc keys_PDEL_test(mc *mockServer) error {\n\treturn mc.DoBatch(\n\t\tDo(\"SET\", \"mykey\", \"myid1a\", \"POINT\", 33, -115).OK(),\n\t\tDo(\"SET\", \"mykey\", \"myid1b\", \"POINT\", 33, -115).OK(),\n\t\tDo(\"SET\", \"mykey\", \"myid2a\", \"POINT\", 33, -115).OK(),\n\t\tDo(\"SET\", \"mykey\", \"myid2b\", \"POINT\", 33, -115).OK(),\n\t\tDo(\"SET\", \"mykey\", \"myid3a\", \"POINT\", 33, -115).OK(),\n\t\tDo(\"SET\", \"mykey\", \"myid3b\", \"POINT\", 33, -115).OK(),\n\t\tDo(\"SET\", \"mykey\", \"myid4a\", \"POINT\", 33, -115).OK(),\n\t\tDo(\"SET\", \"mykey\", \"myid4b\", \"POINT\", 33, -115).OK(),\n\t\tDo(\"PDEL\", \"mykey\").Err(\"wrong number of arguments for 'pdel' command\"),\n\t\tDo(\"PDEL\", \"mykeyNA\", \"*\").Str(\"0\"),\n\t\tDo(\"PDEL\", \"mykey\", \"myid1a\").Str(\"1\"),\n\t\tDo(\"PDEL\", \"mykey\", \"myid1a\").Str(\"0\"),\n\t\tDo(\"PDEL\", \"mykey\", \"myid1*\").Str(\"1\"),\n\t\tDo(\"PDEL\", \"mykey\", \"myid2*\").Str(\"2\"),\n\t\tDo(\"PDEL\", \"mykey\", \"*b\").Str(\"2\"),\n\t\tDo(\"PDEL\", \"mykey\", \"*\").Str(\"2\"),\n\t\tDo(\"PDEL\", \"mykey\", \"*\").Str(\"0\"),\n\t\tDo(\"SET\", \"mykey\", \"myid1a\", \"POINT\", 33, -115).OK(),\n\t\tDo(\"SET\", \"mykey\", \"myid1b\", \"POINT\", 33, -115).OK(),\n\t\tDo(\"SET\", \"mykey\", \"myid2a\", \"POINT\", 33, -115).OK(),\n\t\tDo(\"SET\", \"mykey\", \"myid2b\", \"POINT\", 33, -115).OK(),\n\t\tDo(\"SET\", \"mykey\", \"myid3a\", \"POINT\", 33, -115).OK(),\n\t\tDo(\"PDEL\", \"mykey\", \"*\").JSON().OK(),\n\t)\n}\n\nfunc keys_WHEREIN_test(mc *mockServer) error {\n\treturn mc.DoBatch(\n\t\tDo(\"SET\", \"mykey\", \"myid_a1\", \"FIELD\", \"a\", 1, \"POINT\", 33, -115).OK(),\n\t\tDo(\"WITHIN\", \"mykey\", \"WHEREIN\", \"a\", 3, 0, 1, 2, \"BOUNDS\", 32.8, -115.2, 33.2, -114.8).Str(`[0 [[myid_a1 {\"type\":\"Point\",\"coordinates\":[-115,33]} [a 1]]]]`),\n\t\tDo(\"WITHIN\", \"mykey\", \"WHEREIN\", \"a\", \"a\", 0, 1, 2, \"BOUNDS\", 32.8, -115.2, 33.2, -114.8).Err(\"invalid argument 'a'\"),\n\t\tDo(\"WITHIN\", \"mykey\", \"WHEREIN\", \"a\", 1, 0, 1, 2, \"BOUNDS\", 32.8, -115.2, 33.2, -114.8).Err(\"invalid argument '1'\"),\n\t\tDo(\"WITHIN\", \"mykey\", \"WHEREIN\", \"a\", 3, 0, \"a\", 2, \"BOUNDS\", 32.8, -115.2, 33.2, -114.8).Str(\"[0 []]\"),\n\t\tDo(\"WITHIN\", \"mykey\", \"WHEREIN\", \"a\", 4, 0, \"a\", 1, 2, \"BOUNDS\", 32.8, -115.2, 33.2, -114.8).Str(`[0 [[myid_a1 {\"type\":\"Point\",\"coordinates\":[-115,33]} [a 1]]]]`),\n\t\tDo(\"SET\", \"mykey\", \"myid_a2\", \"FIELD\", \"a\", 2, \"POINT\", 32.99, -115).OK(),\n\t\tDo(\"SET\", \"mykey\", \"myid_a3\", \"FIELD\", \"a\", 3, \"POINT\", 33, -115.02).OK(),\n\t\tDo(\"WITHIN\", \"mykey\", \"WHEREIN\", \"a\", 3, 0, 1, 2, \"BOUNDS\", 32.8, -115.2, 33.2, -114.8).Str(`[0 [[myid_a2 {\"type\":\"Point\",\"coordinates\":[-115,32.99]} [a 2]] [myid_a1 {\"type\":\"Point\",\"coordinates\":[-115,33]} [a 1]]]]`),\n\t\t// zero value should not match 1 and 2\n\t\tDo(\"SET\", \"mykey\", \"myid_a0\", \"FIELD\", \"a\", 0, \"POINT\", 33, -115.02).OK(),\n\t\tDo(\"WITHIN\", \"mykey\", \"WHEREIN\", \"a\", 2, 1, 2, \"BOUNDS\", 32.8, -115.2, 33.2, -114.8).Str(`[0 [[myid_a2 {\"type\":\"Point\",\"coordinates\":[-115,32.99]} [a 2]] [myid_a1 {\"type\":\"Point\",\"coordinates\":[-115,33]} [a 1]]]]`),\n\t)\n}\n\nfunc keys_WHEREEVAL_test(mc *mockServer) error {\n\treturn mc.DoBatch(\n\t\tDo(\"SET\", \"mykey\", \"myid_a1\", \"FIELD\", \"a\", 1, \"POINT\", 33, -115).OK(),\n\t\tDo(\"WITHIN\", \"mykey\", \"WHEREEVAL\", \"return FIELDS.a > tonumber(ARGV[1])\", 1, 0.5, \"BOUNDS\", 32.8, -115.2, 33.2, -114.8).Str(`[0 [[myid_a1 {\"type\":\"Point\",\"coordinates\":[-115,33]} [a 1]]]]`),\n\t\tDo(\"WITHIN\", \"mykey\", \"WHEREEVAL\", \"return FIELDS.a > tonumber(ARGV[1])\", \"a\", 0.5, \"BOUNDS\", 32.8, -115.2, 33.2, -114.8).Err(\"invalid argument 'a'\"),\n\t\tDo(\"WITHIN\", \"mykey\", \"WHEREEVAL\", \"return FIELDS.a > tonumber(ARGV[1])\", 1, 0.5, 4, \"BOUNDS\", 32.8, -115.2, 33.2, -114.8).Err(\"invalid argument '4'\"),\n\t\tDo(\"SET\", \"mykey\", \"myid_a2\", \"FIELD\", \"a\", 2, \"POINT\", 32.99, -115).OK(),\n\t\tDo(\"SET\", \"mykey\", \"myid_a3\", \"FIELD\", \"a\", 3, \"POINT\", 33, -115.02).OK(),\n\t\tDo(\"WITHIN\", \"mykey\", \"WHEREEVAL\", \"return FIELDS.a > tonumber(ARGV[1]) and FIELDS.a ~= tonumber(ARGV[2])\", 2, 0.5, 3, \"BOUNDS\", 32.8, -115.2, 33.2, -114.8).Str(`[0 [[myid_a2 {\"type\":\"Point\",\"coordinates\":[-115,32.99]} [a 2]] [myid_a1 {\"type\":\"Point\",\"coordinates\":[-115,33]} [a 1]]]]`),\n\t)\n}\n\nfunc keys_TYPE_test(mc *mockServer) error {\n\treturn mc.DoBatch(\n\t\tDo(\"SET\", \"mykey\", \"myid1\", \"POINT\", 33, -115).OK(),\n\t\tDo(\"TYPE\", \"mykey\").Str(\"hash\"),\n\t\tDo(\"TYPE\", \"mykey\", \"hello\").Err(\"wrong number of arguments for 'type' command\"),\n\t\tDo(\"TYPE\", \"mykey2\").Str(\"none\"),\n\t\tDo(\"TYPE\", \"mykey2\").JSON().Err(\"key not found\"),\n\t\tDo(\"TYPE\", \"mykey\").JSON().Str(`{\"ok\":true,\"type\":\"hash\"}`),\n\t)\n}\n\nfunc keys_FLUSHDB_test(mc *mockServer) error {\n\treturn mc.DoBatch(\n\t\tDo(\"SET\", \"mykey1\", \"myid1\", \"POINT\", 33, -115).OK(),\n\t\tDo(\"SET\", \"mykey2\", \"myid1\", \"POINT\", 33, -115).OK(),\n\t\tDo(\"SETCHAN\", \"mychan\", \"INTERSECTS\", \"mykey1\", \"BOUNDS\", 10, 10, 10, 10).Str(\"1\"),\n\t\tDo(\"KEYS\", \"*\").Str(\"[mykey1 mykey2]\"),\n\t\tDo(\"CHANS\", \"*\").JSON().Func(func(s string) error {\n\t\t\tif gjson.Get(s, \"chans.#\").Int() != 1 {\n\t\t\t\treturn fmt.Errorf(\"expected '%d', got '%d'\", 1, gjson.Get(s, \"chans.#\").Int())\n\t\t\t}\n\t\t\treturn nil\n\t\t}),\n\t\tDo(\"FLUSHDB\", \"arg2\").Err(\"wrong number of arguments for 'flushdb' command\"),\n\t\tDo(\"FLUSHDB\").OK(),\n\t\tDo(\"KEYS\", \"*\").Str(\"[]\"),\n\t\tDo(\"CHANS\", \"*\").Str(\"[]\"),\n\t\tDo(\"SET\", \"mykey1\", \"myid1\", \"POINT\", 33, -115).OK(),\n\t\tDo(\"SET\", \"mykey2\", \"myid1\", \"POINT\", 33, -115).OK(),\n\t\tDo(\"SETCHAN\", \"mychan\", \"INTERSECTS\", \"mykey1\", \"BOUNDS\", 10, 10, 10, 10).Str(\"1\"),\n\t\tDo(\"FLUSHDB\").JSON().OK(),\n\t)\n}\n\nfunc keys_HEALTHZ_test(mc *mockServer) error {\n\treturn mc.DoBatch(\n\t\tDo(\"HEALTHZ\").OK(),\n\t\tDo(\"HEALTHZ\").JSON().OK(),\n\t\tDo(\"HEALTHZ\", \"arg\").Err(`wrong number of arguments for 'healthz' command`),\n\t)\n}\n\nfunc keys_SERVER_test(mc *mockServer) error {\n\treturn mc.DoBatch(\n\t\tDo(\"SERVER\").Func(func(s string) error {\n\t\t\tvalid := strings.HasPrefix(s, \"[\") && strings.HasSuffix(s, \"]\") &&\n\t\t\t\tstrings.Contains(s, \"cpus\") && strings.Contains(s, \"mem_alloc\")\n\t\t\tif !valid {\n\t\t\t\treturn errors.New(\"looks invalid\")\n\t\t\t}\n\t\t\treturn nil\n\t\t}),\n\t\tDo(\"SERVER\").JSON().Func(func(s string) error {\n\t\t\tif !gjson.Get(s, \"ok\").Bool() {\n\t\t\t\treturn errors.New(\"not ok\")\n\t\t\t}\n\t\t\tvalid := gjson.Get(s, \"stats.cpus\").Exists() &&\n\t\t\t\tgjson.Get(s, \"stats.mem_alloc\").Exists()\n\t\t\tif !valid {\n\t\t\t\treturn errors.New(\"looks invalid\")\n\t\t\t}\n\t\t\treturn nil\n\t\t}),\n\t\tDo(\"SERVER\", \"ext\").Func(func(s string) error {\n\t\t\tvalid := strings.HasPrefix(s, \"[\") && strings.HasSuffix(s, \"]\") &&\n\t\t\t\tstrings.Contains(s, \"sys_cpus\") &&\n\t\t\t\tstrings.Contains(s, \"tile38_connected_clients\")\n\n\t\t\tif !valid {\n\t\t\t\treturn errors.New(\"looks invalid\")\n\t\t\t}\n\t\t\treturn nil\n\t\t}),\n\t\tDo(\"SERVER\", \"ext\").JSON().Func(func(s string) error {\n\t\t\tif !gjson.Get(s, \"ok\").Bool() {\n\t\t\t\treturn errors.New(\"not ok\")\n\t\t\t}\n\t\t\tvalid := gjson.Get(s, \"stats.sys_cpus\").Exists() &&\n\t\t\t\tgjson.Get(s, \"stats.tile38_connected_clients\").Exists()\n\t\t\tif !valid {\n\t\t\t\treturn errors.New(\"looks invalid\")\n\t\t\t}\n\t\t\treturn nil\n\t\t}),\n\t\tDo(\"SERVER\", \"ett\").Err(`invalid argument 'ett'`),\n\t\tDo(\"SERVER\", \"ett\").JSON().Err(`invalid argument 'ett'`),\n\t)\n}\n\nfunc keys_INFO_test(mc *mockServer) error {\n\treturn mc.DoBatch(\n\t\tDo(\"INFO\").Func(func(s string) error {\n\t\t\tif !strings.Contains(s, \"# Clients\") ||\n\t\t\t\t!strings.Contains(s, \"# Stats\") {\n\t\t\t\treturn errors.New(\"looks invalid\")\n\t\t\t}\n\t\t\treturn nil\n\t\t}),\n\t\tDo(\"INFO\", \"all\").Func(func(s string) error {\n\t\t\tif !strings.Contains(s, \"# Clients\") ||\n\t\t\t\t!strings.Contains(s, \"# Stats\") {\n\t\t\t\treturn errors.New(\"looks invalid\")\n\t\t\t}\n\t\t\treturn nil\n\t\t}),\n\t\tDo(\"INFO\", \"default\").Func(func(s string) error {\n\t\t\tif !strings.Contains(s, \"# Clients\") ||\n\t\t\t\t!strings.Contains(s, \"# Stats\") {\n\t\t\t\treturn errors.New(\"looks invalid\")\n\t\t\t}\n\t\t\treturn nil\n\t\t}),\n\t\tDo(\"INFO\", \"cpu\").Func(func(s string) error {\n\t\t\tif !strings.Contains(s, \"# CPU\") ||\n\t\t\t\tstrings.Contains(s, \"# Clients\") ||\n\t\t\t\tstrings.Contains(s, \"# Stats\") {\n\t\t\t\treturn errors.New(\"looks invalid\")\n\t\t\t}\n\t\t\treturn nil\n\t\t}),\n\t\tDo(\"INFO\", \"cpu\", \"clients\").Func(func(s string) error {\n\t\t\tif !strings.Contains(s, \"# CPU\") ||\n\t\t\t\t!strings.Contains(s, \"# Clients\") ||\n\t\t\t\tstrings.Contains(s, \"# Stats\") {\n\t\t\t\treturn errors.New(\"looks invalid\")\n\t\t\t}\n\t\t\treturn nil\n\t\t}),\n\t\tDo(\"INFO\").JSON().Func(func(s string) error {\n\t\t\tif gjson.Get(s, \"info.tile38_version\").String() == \"\" {\n\t\t\t\treturn errors.New(\"looks invalid\")\n\t\t\t}\n\t\t\treturn nil\n\t\t}),\n\t)\n}\n" }, { "path": "tests/metrics_test.go", "content": "package tests\n\nimport (\n\t\"fmt\"\n\t\"io\"\n\t\"net/http\"\n\t\"strings\"\n)\n\nfunc subTestMetrics(g *testGroup) {\n\tg.regSubTest(\"basic\", metrics_basic_test)\n}\n\nfunc downloadURLWithStatusCode(u string) (int, string, error) {\n\tresp, err := http.Get(u)\n\tif err != nil {\n\t\treturn 0, \"\", err\n\t}\n\tdefer resp.Body.Close()\n\tbody, err := io.ReadAll(resp.Body)\n\tif err != nil {\n\t\treturn 0, \"\", err\n\t}\n\treturn resp.StatusCode, string(body), nil\n}\n\nfunc metrics_basic_test(mc *mockServer) error {\n\n\tmaddr := fmt.Sprintf(\"http://127.0.0.1:%d/\", mc.metricsPort())\n\n\tmc.Do(\"SET\", \"metrics_test_1\", \"1\", \"FIELD\", \"foo\", 5.5, \"POINT\", 5, 5)\n\tmc.Do(\"SET\", \"metrics_test_2\", \"2\", \"FIELD\", \"foo\", 19.19, \"POINT\", 19, 19)\n\tmc.Do(\"SET\", \"metrics_test_2\", \"3\", \"FIELD\", \"foo\", 19.19, \"POINT\", 19, 19)\n\tmc.Do(\"SET\", \"metrics_test_2\", \"truck1:driver\", \"STRING\", \"John Denton\")\n\n\tstatus, index, err := downloadURLWithStatusCode(maddr)\n\tif err != nil {\n\t\treturn err\n\t}\n\tif status != 200 {\n\t\treturn fmt.Errorf(\"Expected status code 200, got: %d\", status)\n\t}\n\tif !strings.Contains(index, \" 0 {\n\t\tif opts.AOFFileName == \"\" {\n\t\t\topts.AOFFileName = \"appendonly.aof\"\n\t\t}\n\t\tif err := os.MkdirAll(dir, 0777); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\terr := os.WriteFile(filepath.Join(dir, opts.AOFFileName),\n\t\t\topts.AOFData, 0666)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\n\tshutdown := make(chan bool)\n\ts := &mockServer{port: port, dir: dir, shutdown: shutdown}\n\tif opts.Metrics {\n\t\ts.mport = getNextPort()\n\t}\n\tvar ferr atomic.Pointer[error] // ferr for when the server fails to start\n\tgo func() {\n\t\tsopts := server.Options{\n\t\t\tHost: \"localhost\",\n\t\t\tPort: port,\n\t\t\tDir: dir,\n\t\t\tUseHTTP: true,\n\t\t\tDevMode: true,\n\t\t\tAppendOnly: true,\n\t\t\tShutdown: shutdown,\n\t\t\tShowDebugMessages: true,\n\t\t}\n\t\tif opts.Metrics {\n\t\t\tsopts.MetricsAddr = fmt.Sprintf(\":%d\", s.mport)\n\t\t}\n\t\terr := server.Serve(sopts)\n\t\tif err != nil {\n\t\t\tferr.CompareAndSwap(nil, &err)\n\t\t}\n\t}()\n\tif err := s.waitForStartup(&ferr); err != nil {\n\t\ts.Close()\n\t\treturn nil, err\n\t}\n\treturn s, nil\n}\n\nfunc (s *mockServer) waitForStartup(ferr *atomic.Pointer[error]) error {\n\tvar lerr error\n\tstart := time.Now()\n\tfor {\n\t\tif perr := ferr.Load(); perr != nil {\n\t\t\treturn *perr\n\t\t}\n\t\tif time.Since(start) > time.Second*5 {\n\t\t\tif lerr != nil {\n\t\t\t\treturn lerr\n\t\t\t}\n\t\t\treturn errTimeout\n\t\t}\n\t\tresp, err := redis.String(s.Do(\"SET\", \"please\", \"allow\", \"POINT\", \"33\", \"-115\"))\n\t\tif err != nil {\n\t\t\tlerr = err\n\t\t} else if resp != \"OK\" {\n\t\t\tlerr = errors.New(\"not OK\")\n\t\t} else {\n\t\t\tresp, err := redis.Int(s.Do(\"DEL\", \"please\", \"allow\"))\n\t\t\tif err != nil {\n\t\t\t\tlerr = err\n\t\t\t} else if resp != 1 {\n\t\t\t\tlerr = errors.New(\"not 1\")\n\t\t\t} else {\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\ttime.Sleep(time.Millisecond * 100)\n\t}\n}\n\nfunc (mc *mockServer) Close() {\n\tif mc == nil || mc.closed {\n\t\treturn\n\t}\n\tmc.closed = true\n\tmc.shutdown <- true\n\tif mc.conn != nil {\n\t\tmc.conn.Close()\n\t}\n\tif mc.dir != \"\" {\n\t\tos.RemoveAll(mc.dir)\n\t}\n}\n\nfunc (mc *mockServer) ResetConn() {\n\tif mc.conn != nil {\n\t\tmc.conn.Close()\n\t\tmc.conn = nil\n\t}\n}\n\nfunc (s *mockServer) DoPipeline(cmds [][]interface{}) ([]interface{}, error) {\n\tif s.conn == nil {\n\t\tvar err error\n\t\ts.conn, err = redis.Dial(\"tcp\", fmt.Sprintf(\":%d\", s.port))\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\t//defer conn.Close()\n\tfor _, cmd := range cmds {\n\t\tif err := s.conn.Send(cmd[0].(string), cmd[1:]...); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\tif err := s.conn.Flush(); err != nil {\n\t\treturn nil, err\n\t}\n\tvar resps []interface{}\n\tfor i := 0; i < len(cmds); i++ {\n\t\tresp, err := s.conn.Receive()\n\t\tif err != nil {\n\t\t\tresps = append(resps, err)\n\t\t} else {\n\t\t\tresps = append(resps, resp)\n\t\t}\n\t}\n\treturn resps, nil\n}\nfunc (s *mockServer) Do(commandName string, args ...interface{}) (interface{}, error) {\n\tresps, err := s.DoPipeline([][]interface{}{\n\t\tappend([]interface{}{commandName}, args...),\n\t})\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tif len(resps) != 1 {\n\t\treturn nil, errors.New(\"invalid number or responses\")\n\t}\n\treturn resps[0], nil\n}\n\nfunc (mc *mockServer) DoBatch(commands ...interface{}) error {\n\t// Probe for I/O tests\n\tif len(commands) > 0 {\n\t\tif _, ok := commands[0].(*IO); ok {\n\t\t\tvar cmds []*IO\n\t\t\t// If the first is an I/O test then all must be\n\t\t\tfor _, cmd := range commands {\n\t\t\t\tif cmd, ok := cmd.(*IO); ok {\n\t\t\t\t\tcmds = append(cmds, cmd)\n\t\t\t\t} else {\n\t\t\t\t\treturn errors.New(\"DoBatch cannot mix I/O tests with other kinds\")\n\t\t\t\t}\n\t\t\t}\n\t\t\tfor i, cmd := range cmds {\n\t\t\t\tif err := mc.doIOTest(i, cmd); err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\t\t\t}\n\t\t\treturn nil\n\t\t}\n\t}\n\n\tvar tag string\n\tfor _, commands := range commands {\n\t\tswitch commands := commands.(type) {\n\t\tcase string:\n\t\t\ttag = commands\n\t\tcase [][]interface{}:\n\t\t\tfor i := 0; i < len(commands); i += 2 {\n\t\t\t\tcmds := commands[i]\n\t\t\t\tif dur, ok := cmds[0].(time.Duration); ok {\n\t\t\t\t\ttime.Sleep(dur)\n\t\t\t\t} else {\n\t\t\t\t\tif err := mc.DoExpect(commands[i+1], cmds[0].(string), cmds[1:]...); err != nil {\n\t\t\t\t\t\tif tag == \"\" {\n\t\t\t\t\t\t\treturn fmt.Errorf(\"batch[%d]: %v\", i/2, err)\n\t\t\t\t\t\t} else {\n\t\t\t\t\t\t\treturn fmt.Errorf(\"batch[%d][%v]: %v\", i/2, tag, err)\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t\ttag = \"\"\n\t\tcase *IO:\n\t\t\treturn errors.New(\"DoBatch cannot mix I/O tests with other kinds\")\n\t\tdefault:\n\t\t\treturn fmt.Errorf(\"Unknown command input\")\n\t\t}\n\t}\n\treturn nil\n}\n\nfunc normalize(v interface{}) interface{} {\n\tswitch v := v.(type) {\n\tdefault:\n\t\treturn v\n\tcase []interface{}:\n\t\tfor i := 0; i < len(v); i++ {\n\t\t\tv[i] = normalize(v[i])\n\t\t}\n\tcase []uint8:\n\t\treturn string(v)\n\t}\n\treturn v\n}\nfunc (mc *mockServer) DoExpect(expect interface{}, commandName string, args ...interface{}) error {\n\tif v, ok := expect.([]interface{}); ok {\n\t\texpect = v[0]\n\t}\n\tresp, err := mc.Do(commandName, args...)\n\tif err != nil {\n\t\tif exs, ok := expect.(string); ok {\n\t\t\tif err.Error() == exs {\n\t\t\t\treturn nil\n\t\t\t}\n\t\t}\n\t\treturn err\n\t}\n\tif b, ok := resp.([]byte); ok && len(b) > 1 && b[0] == '{' {\n\t\tb, err = sjson.DeleteBytes(b, \"elapsed\")\n\t\tif err == nil {\n\t\t\tresp = b\n\t\t}\n\t}\n\toresp := resp\n\tresp = normalize(resp)\n\tif expect == nil && resp != nil {\n\t\treturn fmt.Errorf(\"expected '%v', got '%v'\", expect, resp)\n\t}\n\tif vv, ok := resp.([]interface{}); ok {\n\t\tvar ss []string\n\t\tfor _, v := range vv {\n\t\t\tif v == nil {\n\t\t\t\tss = append(ss, \"nil\")\n\t\t\t} else if s, ok := v.(string); ok {\n\t\t\t\tss = append(ss, s)\n\t\t\t} else if b, ok := v.([]uint8); ok {\n\t\t\t\tif b == nil {\n\t\t\t\t\tss = append(ss, \"nil\")\n\t\t\t\t} else {\n\t\t\t\t\tss = append(ss, string(b))\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\tss = append(ss, fmt.Sprintf(\"%v\", v))\n\t\t\t}\n\t\t}\n\t\tresp = ss\n\t}\n\tif b, ok := resp.([]uint8); ok {\n\t\tif b == nil {\n\t\t\tresp = nil\n\t\t} else {\n\t\t\tresp = string([]byte(b))\n\t\t}\n\t}\n\terr = func() (err error) {\n\t\tdefer func() {\n\t\t\tv := recover()\n\t\t\tif v != nil {\n\t\t\t\terr = fmt.Errorf(\"panic '%v'\", v)\n\t\t\t}\n\t\t}()\n\t\tif fn, ok := expect.(func(v, org interface{}) (resp, expect interface{})); ok {\n\t\t\tresp, expect = fn(resp, oresp)\n\t\t}\n\t\tif fn, ok := expect.(func(v interface{}) (resp, expect interface{})); ok {\n\t\t\tresp, expect = fn(resp)\n\t\t}\n\t\treturn nil\n\t}()\n\tif err != nil {\n\t\treturn err\n\t}\n\tif fn, ok := expect.(func(string) bool); ok {\n\t\tif !fn(fmt.Sprintf(\"%v\", resp)) {\n\t\t\treturn fmt.Errorf(\"unexpected for response '%v'\", resp)\n\t\t}\n\t} else if fn, ok := expect.(func(string) error); ok {\n\t\terr := fn(fmt.Sprintf(\"%v\", resp))\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"%s, for response '%v'\", err.Error(), resp)\n\t\t}\n\t} else if fmt.Sprintf(\"%v\", resp) != fmt.Sprintf(\"%v\", expect) {\n\t\treturn fmt.Errorf(\"expected '%v', got '%v'\", expect, resp)\n\t}\n\treturn nil\n}\n" }, { "path": "tests/monitor_test.go", "content": "package tests\n\nimport (\n\t\"fmt\"\n\t\"strings\"\n\t\"sync\"\n\n\t\"github.com/gomodule/redigo/redis\"\n)\n\nfunc subTestMonitor(g *testGroup) {\n\tg.regSubTest(\"monitor\", follower_monitor_test)\n}\n\nfunc follower_monitor_test(mc *mockServer) error {\n\tN := 1000\n\tch := make(chan error)\n\tvar wg sync.WaitGroup\n\twg.Add(1)\n\tgo func() {\n\t\tch <- func() error {\n\t\t\tconn, err := redis.Dial(\"tcp\", fmt.Sprintf(\"localhost:%d\", mc.port))\n\t\t\tif err != nil {\n\t\t\t\twg.Done()\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tdefer conn.Close()\n\t\t\ts, err := redis.String(conn.Do(\"MONITOR\"))\n\t\t\tif err != nil {\n\t\t\t\twg.Done()\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tif s != \"OK\" {\n\t\t\t\twg.Done()\n\t\t\t\treturn fmt.Errorf(\"expected '%s', got '%s'\", \"OK\", s)\n\t\t\t}\n\t\t\twg.Done()\n\n\t\t\tfor i := 0; i < N; i++ {\n\t\t\t\ts, err := redis.String(conn.Receive())\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\t\t\t\tex := fmt.Sprintf(`\"mykey\" \"%d\"`, i)\n\t\t\t\tif !strings.Contains(s, ex) {\n\t\t\t\t\treturn fmt.Errorf(\"expected '%s', got '%s'\", ex, s)\n\t\t\t\t}\n\t\t\t}\n\t\t\treturn nil\n\t\t}()\n\t}()\n\n\twg.Wait()\n\n\tconn, err := redis.Dial(\"tcp\", fmt.Sprintf(\"localhost:%d\", mc.port))\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer conn.Close()\n\n\tfor i := 0; i < N; i++ {\n\t\ts, err := redis.String(conn.Do(\"SET\", \"mykey\", i, \"POINT\", 10, 10))\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tif s != \"OK\" {\n\t\t\treturn fmt.Errorf(\"expected '%s', got '%s'\", \"OK\", s)\n\t\t}\n\t}\n\n\terr = <-ch\n\tif err != nil {\n\t\terr = fmt.Errorf(\"monitor client: %w\", err)\n\t}\n\n\treturn err\n}\n" }, { "path": "tests/proto_test.go", "content": "package tests\n\nimport (\n\t\"fmt\"\n\t\"net/http\"\n)\n\nfunc subTestProto(g *testGroup) {\n\tg.regSubTest(\"HTTP CORS\", proto_HTTP_CORS_test)\n}\n\nfunc proto_HTTP_CORS_test(mc *mockServer) error {\n\t// Make CORS request for GET /SERVER\n\tmorigin := \"http://my-test-origin\"\n\turl := fmt.Sprintf(\"http://127.0.0.1:%d/SERVER\", mc.port)\n\treq, err := http.NewRequest(http.MethodOptions, url, nil)\n\tif err != nil {\n\t\treturn err\n\t}\n\treq.Header.Add(\"Origin\", morigin)\n\treq.Header.Add(\"Access-Control-Request-Method\", \"GET\")\n\treq.Header.Add(\"Access-Control-Request-Headers\", \"Authorization\")\n\tresp, err := http.DefaultClient.Do(req)\n\n\t// Validate CORS response\n\tif err != nil {\n\t\treturn err\n\t}\n\tif resp.StatusCode != 204 {\n\t\treturn fmt.Errorf(\"expected http stuats '204', got '%d'\", resp.StatusCode)\n\t}\n\torigin := resp.Header.Get(\"Access-Control-Allow-Origin\")\n\tmethods := resp.Header.Get(\"Access-Control-Allow-Methods\")\n\theaders := resp.Header.Get(\"Access-Control-Allow-Headers\")\n\tif !(origin == \"*\" || origin == morigin) {\n\t\treturn fmt.Errorf(\"expected http access-control-allow-origin value '*', got '%s'\", origin)\n\t}\n\tif methods != \"POST, GET, OPTIONS\" {\n\t\treturn fmt.Errorf(\"expected http access-control-allow-Methods value 'POST, GET, OPTIONS', got '%s'\", methods)\n\t}\n\tif headers != \"*, Authorization\" {\n\t\treturn fmt.Errorf(\"expected http access-control-allow-headers value '*, Authorization', got '%s'\", headers)\n\t}\n\n\t// Make the actual request now\n\tresp, err = http.Get(url)\n\tif err != nil {\n\t\treturn err\n\t}\n\torigin = resp.Header.Get(\"Access-Control-Allow-Origin\")\n\tif !(origin == \"*\" || origin == morigin) {\n\t\treturn fmt.Errorf(\"expected http access-control-allow-origin value '*', got '%s'\", origin)\n\t}\n\n\treturn nil\n}\n" }, { "path": "tests/scripts_test.go", "content": "package tests\n\nimport (\n\t\"fmt\"\n\t\"strings\"\n)\n\nfunc subTestScripts(g *testGroup) {\n\tg.regSubTest(\"BASIC\", scripts_BASIC_test)\n\tg.regSubTest(\"ATOMIC\", scripts_ATOMIC_test)\n\tg.regSubTest(\"READONLY\", scripts_READONLY_test)\n\tg.regSubTest(\"NONATOMIC\", scripts_NONATOMIC_test)\n\tg.regSubTest(\"VULN\", scripts_VULN_test)\n}\n\nfunc scripts_BASIC_test(mc *mockServer) error {\n\treturn mc.DoBatch([][]interface{}{\n\t\t{\"EVAL\", \"return 2 + 2\", 0}, {\"4\"},\n\t\t{\"SCRIPT LOAD\", \"return 2 + 2\"}, {\"2dd1b44209ecb49617af05caf0491390a03c1cc4\"},\n\t\t{\"SCRIPT EXISTS\", \"2dd1b44209ecb49617af05caf0491390a03c1cc4\", \"no_script\"}, {\"[1 0]\"},\n\t\t{\"EVALSHA\", \"2dd1b44209ecb49617af05caf0491390a03c1cc4\", \"0\"}, {\"4\"},\n\t\t{\"SCRIPT FLUSH\"}, {\"OK\"},\n\t\t{\"SCRIPT EXISTS\", \"2dd1b44209ecb49617af05caf0491390a03c1cc4\", \"no_script\"}, {\"[0 0]\"},\n\t\t{\"EVAL\", \"return KEYS[1] .. ' only'\", 1, \"key1\"}, {\"key1 only\"},\n\t\t{\"EVAL\", \"return KEYS[1] .. ' and ' .. ARGV[1]\", 1, \"key1\", \"arg1\"}, {\"key1 and arg1\"},\n\t\t{\"EVAL\", \"return ARGV[1] .. ' and ' .. ARGV[2]\", 0, \"arg1\", \"arg2\"}, {\"arg1 and arg2\"},\n\t\t{\"EVAL\", \"return tile38.sha1hex('asdf')\", 0}, {\"3da541559918a808c2402bba5012f6c60b27661c\"},\n\t\t{\"EVAL\", \"return tile38.distance_to(37.7341129, -122.4408378, 37.733, -122.43)\", 0}, {\"961\"},\n\t})\n}\n\nfunc scripts_ATOMIC_test(mc *mockServer) error {\n\treturn mc.DoBatch([][]interface{}{\n\t\t{\"EVAL\", \"return tile38.call('get', KEYS[1], ARGV[1])\", \"1\", \"mykey\", \"myid\"}, {nil},\n\t\t{\"EVAL\", \"return tile38.call('set', KEYS[1], ARGV[1], 'point', 33, -115)\", \"1\", \"mykey\", \"myid1\"}, {\"OK\"},\n\t\t{\"EVAL\", \"return tile38.call('get', KEYS[1], ARGV[1], ARGV[2])\", \"1\", \"mykey\", \"myid1\", \"point\"}, {\"[33 -115]\"},\n\t})\n}\n\nfunc scripts_READONLY_test(mc *mockServer) error {\n\treturn mc.DoBatch([][]interface{}{\n\t\t{\"EVALRO\", \"return tile38.call('get', KEYS[1], ARGV[1])\", \"1\", \"mykey\", \"myid\"}, {nil},\n\t\t{\"EVALRO\", \"return tile38.call('set', KEYS[1], ARGV[1], 'point', 33, -115)\", \"1\", \"mykey\", \"myid1\"}, {\n\t\t\tfunc(v interface{}) (resp, expect interface{}) {\n\t\t\t\ts := fmt.Sprintf(\"%v\", v)\n\t\t\t\tif strings.Contains(s, \"ERR read only\") {\n\t\t\t\t\treturn v, v\n\t\t\t\t}\n\t\t\t\treturn v, \"A lua stack containing 'ERR read only'\"\n\t\t\t},\n\t\t},\n\t\t{\"EVALRO\", \"return tile38.pcall('set', KEYS[1], ARGV[1], 'point', 33, -115)\", \"1\", \"mykey\", \"myid1\"}, {\"ERR read only\"},\n\t\t{\"SET\", \"mykey\", \"myid1\", \"POINT\", 33, -115}, {\"OK\"},\n\t\t{\"EVALRO\", \"return tile38.call('get', KEYS[1], ARGV[1], ARGV[2])\", \"1\", \"mykey\", \"myid1\", \"point\"}, {\"[33 -115]\"},\n\t})\n}\n\nfunc scripts_NONATOMIC_test(mc *mockServer) error {\n\treturn mc.DoBatch([][]interface{}{\n\t\t{\"EVALNA\", \"return tile38.call('get', KEYS[1], ARGV[1])\", \"1\", \"mykey\", \"myid\"}, {nil},\n\t\t{\"EVALNA\", \"return tile38.call('set', KEYS[1], ARGV[1], 'point', 33, -115)\", \"1\", \"mykey\", \"myid1\"}, {\"OK\"},\n\t\t{\"EVALNA\", \"return tile38.call('get', KEYS[1], ARGV[1], ARGV[2])\", \"1\", \"mykey\", \"myid1\", \"point\"}, {\"[33 -115]\"},\n\t})\n}\n\nfunc scripts_VULN_test(mc *mockServer) error {\n\treturn mc.DoBatch([][]interface{}{\n\t\t{\"EVAL\", \"return io\", \"0\"}, {nil},\n\t\t{\"EVAL\", \"return file\", \"0\"}, {nil},\n\t\t{\"EVAL\", \"return os.execute\", \"0\"}, {nil},\n\t\t{\"EVAL\", \"return os.getenv\", \"0\"}, {nil},\n\t\t{\"EVAL\", \"return os.clock\", \"0\"}, {\"ERR Unsupported lua type: function\"},\n\t\t{\"EVAL\", \"return loadfile\", \"0\"}, {nil},\n\t\t{\"EVAL\", \"return tonumber(ARGV[1])\", \"0\", \"38\"}, {\"38\"},\n\t\t{\"EVAL\", \"return package\", \"0\"}, {nil},\n\t})\n}\n" }, { "path": "tests/stats_test.go", "content": "package tests\n\nimport (\n\t\"errors\"\n\n\t\"github.com/tidwall/gjson\"\n)\n\nfunc subTestInfo(g *testGroup) {\n\tg.regSubTest(\"valid json\", info_valid_json_test)\n}\n\nfunc info_valid_json_test(mc *mockServer) error {\n\tif _, err := mc.Do(\"OUTPUT\", \"JSON\"); err != nil {\n\t\treturn err\n\t}\n\tres, err := mc.Do(\"INFO\")\n\tif err != nil {\n\t\treturn err\n\t}\n\tbres, ok := res.([]byte)\n\tif !ok {\n\t\treturn errors.New(\"Failed to type assert INFO response\")\n\t}\n\tsres := string(bres)\n\tif !gjson.Valid(sres) {\n\t\treturn errors.New(\"INFO response was invalid\")\n\t}\n\tinfo := gjson.Get(sres, \"info\").String()\n\tif !gjson.Valid(info) {\n\t\treturn errors.New(\"INFO.info response was invalid\")\n\t}\n\treturn nil\n}\n" }, { "path": "tests/testcmd_test.go", "content": "package tests\n\nfunc subTestTestCmd(g *testGroup) {\n\tg.regSubTest(\"WITHIN\", testcmd_WITHIN_test)\n\tg.regSubTest(\"INTERSECTS\", testcmd_INTERSECTS_test)\n\tg.regSubTest(\"INTERSECTS_CLIP\", testcmd_INTERSECTS_CLIP_test)\n\tg.regSubTest(\"ExpressionErrors\", testcmd_expressionErrors_test)\n\tg.regSubTest(\"Expressions\", testcmd_expression_test)\n}\n\nfunc testcmd_WITHIN_test(mc *mockServer) error {\n\tpoly := `{\n\t\t\t\t\"type\": \"Polygon\",\n\t\t\t\t\"coordinates\": [\n\t\t\t\t\t[\n\t\t\t\t\t\t[-122.44126439094543,37.72906137107],\n\t\t\t\t\t\t[-122.43980526924135,37.72906137107],\n\t\t\t\t\t\t[-122.43980526924135,37.73421283683962],\n\t\t\t\t\t\t[-122.44126439094543,37.73421283683962],\n\t\t\t\t\t\t[-122.44126439094543,37.72906137107]\n\t\t\t\t\t]\n\t\t\t\t]\n\t\t\t}`\n\tpoly8 := `{\"type\":\"Polygon\",\"coordinates\":[[[-122.4408378,37.7341129],[-122.4408378,37.733],[-122.44,37.733],[-122.44,37.7341129],[-122.4408378,37.7341129]],[[-122.44060993194579,37.73345766902749],[-122.44044363498686,37.73345766902749],[-122.44044363498686,37.73355524732416],[-122.44060993194579,37.73355524732416],[-122.44060993194579,37.73345766902749]],[[-122.44060724973677,37.7336888869566],[-122.4402102828026,37.7336888869566],[-122.4402102828026,37.7339752567853],[-122.44060724973677,37.7339752567853],[-122.44060724973677,37.7336888869566]]]}`\n\tpoly9 := `{\"type\":\"Polygon\",\"coordinates\":[[[-122.44037926197052,37.73313523548048],[-122.44017541408539,37.73313523548048],[-122.44017541408539,37.73336857568778],[-122.44037926197052,37.73336857568778],[-122.44037926197052,37.73313523548048]]]}`\n\tpoly10 := `{\"type\":\"Polygon\",\"coordinates\":[[[-122.44040071964262,37.73359343010089],[-122.4402666091919,37.73359343010089],[-122.4402666091919,37.73373767596864],[-122.44040071964262,37.73373767596864],[-122.44040071964262,37.73359343010089]]]}`\n\n\treturn mc.DoBatch(\n\t\tDo(\"SET\", \"mykey\", \"point1\", \"POINT\", 37.7335, -122.4412).OK(),\n\t\tDo(\"SET\", \"mykey\", \"point2\", \"POINT\", 37.7335, -122.44121).OK(),\n\t\tDo(\"SET\", \"mykey\", \"line3\", \"OBJECT\", `{\"type\":\"LineString\",\"coordinates\":[[-122.4408378,37.7341129],[-122.4408378,37.733]]}`).OK(),\n\t\tDo(\"SET\", \"mykey\", \"poly4\", \"OBJECT\", `{\"type\":\"Polygon\",\"coordinates\":[[[-122.4408378,37.7341129],[-122.4408378,37.733],[-122.44,37.733],[-122.44,37.7341129],[-122.4408378,37.7341129]]]}`).OK(),\n\t\tDo(\"SET\", \"mykey\", \"multipoly5\", \"OBJECT\", `{\"type\":\"MultiPolygon\",\"coordinates\":[[[[-122.4408378,37.7341129],[-122.4408378,37.733],[-122.44,37.733],[-122.44,37.7341129],[-122.4408378,37.7341129]]],[[[-122.44091033935547,37.731981251280985],[-122.43994474411011,37.731981251280985],[-122.43994474411011,37.73254976045042],[-122.44091033935547,37.73254976045042],[-122.44091033935547,37.731981251280985]]]]}`).OK(),\n\t\tDo(\"SET\", \"mykey\", \"point6\", \"POINT\", -5, 5).OK(),\n\t\tDo(\"SET\", \"mykey\", \"point7\", \"POINT\", 33, 21).OK(),\n\t\tDo(\"SET\", \"mykey\", \"poly8\", \"OBJECT\", poly8).OK(),\n\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point1\", \"WITHIN\", \"OBJECT\", poly).Str(\"1\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point1\", \"WITHIN\", \"SECTOR\", \"37.72999\", \"-122.44760\", \"1000\", \"0\", \"90\").Str(\"1\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"line3\", \"WITHIN\", \"OBJECT\", poly).Str(\"1\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"poly4\", \"WITHIN\", \"OBJECT\", poly).Str(\"1\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"multipoly5\", \"WITHIN\", \"OBJECT\", poly).Str(\"1\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"poly8\", \"WITHIN\", \"OBJECT\", poly).Str(\"1\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"poly8\", \"WITHIN\", \"SECTOR\", \"37.72999\", \"-122.44760\", \"1000\", \"0\", \"90\").Str(\"1\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"poly8\", \"WITHIN\", \"SECTOR\", \"37.72999\", \"-122.44760\", \"1000\", \"0\", \"90\").JSON().Str(`{\"ok\":true,\"result\":true}`),\n\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point6\", \"WITHIN\", \"OBJECT\", poly).Str(\"0\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point6\", \"WITHIN\", \"OBJECT\", poly).JSON().Str(`{\"ok\":true,\"result\":false}`),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point7\", \"WITHIN\", \"OBJECT\", poly).Str(\"0\"),\n\n\t\tDo(\"TEST\", \"OBJECT\", poly9, \"WITHIN\", \"OBJECT\", poly8).Str(\"1\"),\n\t\tDo(\"TEST\", \"OBJECT\", poly10, \"WITHIN\", \"OBJECT\", poly8).Str(\"0\"),\n\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point1\", \"WITHIN\", \"SECTOR\", \"37.72999\", \"-122.44760\", \"1000\", \"0\", \"ff\").Err(\"invalid argument 'ff'\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point1\", \"WITHIN\", \"SECTOR\", \"37.72999\", \"-122.44760\", \"1000\", \"ee\", \"ff\").Err(\"invalid argument 'ee'\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point1\", \"WITHIN\", \"SECTOR\", \"37.72999\", \"-122.44760\", \"dd\", \"ee\", \"ff\").Err(\"invalid argument 'dd'\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point1\", \"WITHIN\", \"SECTOR\", \"37.72999\", \"cc\", \"dd\", \"ee\", \"ff\").Err(\"invalid argument 'cc'\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point1\", \"WITHIN\", \"SECTOR\", \"bb\", \"cc\", \"dd\", \"ee\", \"ff\").Err(\"invalid argument 'bb'\"),\n\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point1\", \"WITHIN\", \"SECTOR\", \"37.72999\", \"-122.44760\", \"1000\", \"0\").Err(\"wrong number of arguments for 'test' command\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point1\", \"WITHIN\", \"SECTOR\", \"37.72999\", \"-122.44760\", \"1000\").Err(\"wrong number of arguments for 'test' command\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point1\", \"WITHIN\", \"SECTOR\", \"37.72999\", \"-122.44760\").Err(\"wrong number of arguments for 'test' command\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point1\", \"WITHIN\", \"SECTOR\", \"37.72999\").Err(\"wrong number of arguments for 'test' command\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point1\", \"WITHIN\", \"SECTOR\").Err(\"wrong number of arguments for 'test' command\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point1\", \"WITHIN\", \"SECTOR\", \"37.72999\", \"-122.44760\", \"1000\", \"1\", \"1\").Err(\"equal bearings (1 == 1), use CIRCLE instead\"),\n\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point1\", \"WITHIN\", \"CIRCLE\", \"37.72999\", \"-122.44760\", \"10000\").Str(\"1\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point1\", \"WITHIN\", \"CIRCLE\", \"37.72999\", \"-122.44760\", \"10000\", \"10000\").Err(\"wrong number of arguments for 'test' command\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point1\", \"WITHIN\", \"CIRCLE\", \"37.72999\", \"-122.44760\").Err(\"wrong number of arguments for 'test' command\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point1\", \"WITHIN\", \"CIRCLE\", \"37.72999\").Err(\"wrong number of arguments for 'test' command\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point1\", \"WITHIN\", \"CIRCLE\").Err(\"wrong number of arguments for 'test' command\"),\n\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point1\", \"WITHIN\", \"CIRCLE\", \"37.72999\", \"-122.44760\", \"cc\").Err(\"invalid argument 'cc'\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point1\", \"WITHIN\", \"CIRCLE\", \"37.72999\", \"bb\", \"cc\").Err(\"invalid argument 'bb'\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point1\", \"WITHIN\", \"CIRCLE\", \"aa\", \"bb\", \"cc\").Err(\"invalid argument 'aa'\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point1\", \"WITHIN\", \"CIRCLE\", \"37.72999\", \"-122.44760\", \"-10000\").Err(\"invalid argument '-10000'\"),\n\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point1\", \"WITHIN\", \"hash\").Err(\"wrong number of arguments for 'test' command\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point1\", \"WITHIN\", \"hash\", \"123\").Str(\"0\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point1\", \"WITHIN\", \"hash\", \"123\", \"asdf\").Err(\"wrong number of arguments for 'test' command\"),\n\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point1\", \"WITHIN\", \"quadkey\").Err(\"wrong number of arguments for 'test' command\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point1\", \"WITHIN\", \"quadkey\", \"123\").Str(\"0\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point1\", \"WITHIN\", \"quadkey\", \"pqowie\").Err(\"invalid argument 'pqowie'\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point1\", \"WITHIN\", \"quadkey\", \"123\", \"asdf\").Err(\"wrong number of arguments for 'test' command\"),\n\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point1\", \"WITHIN\", \"tile\").Err(\"wrong number of arguments for 'test' command\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point1\", \"WITHIN\", \"tile\", \"1\").Err(\"wrong number of arguments for 'test' command\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point1\", \"WITHIN\", \"tile\", \"1\", \"2\").Err(\"wrong number of arguments for 'test' command\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point1\", \"WITHIN\", \"tile\", \"1\", \"2\", \"3\").Str(\"0\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point1\", \"WITHIN\", \"tile\", \"1\", \"2\", \"cc\").Err(\"invalid argument 'cc'\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point1\", \"WITHIN\", \"tile\", \"1\", \"bb\", \"cc\").Err(\"invalid argument 'bb'\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point1\", \"WITHIN\", \"tile\", \"aa\", \"bb\", \"cc\").Err(\"invalid argument 'aa'\"),\n\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point1\", \"WITHIN\", \"point\", \"1\", \"2\").Str(\"0\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point1\", \"WITHIN\", \"point\").Err(\"wrong number of arguments for 'test' command\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point1\", \"WITHIN\", \"point\", \"1\").Err(\"wrong number of arguments for 'test' command\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point1\", \"WITHIN\", \"point\", \"1\", \"2\", \"3\").Err(\"wrong number of arguments for 'test' command\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point1\", \"WITHIN\", \"point\", \"1\", \"bb\").Err(\"invalid argument 'bb'\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point1\", \"WITHIN\", \"point\", \"aa\", \"bb\").Err(\"invalid argument 'aa'\"),\n\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point1\", \"WITHIN\").Err(\"wrong number of arguments for 'test' command\"),\n\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point1\", \"WITHIN\", \"bounds\", \"1\", \"2\", \"3\", \"4\").Str(\"0\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point1\", \"WITHIN\", \"bounds\", \"1\", \"2\", \"3\", \"4\", \"5\").Err(\"wrong number of arguments for 'test' command\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point1\", \"WITHIN\", \"bounds\", \"1\", \"2\", \"3\").Err(\"wrong number of arguments for 'test' command\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point1\", \"WITHIN\", \"bounds\", \"1\", \"2\").Err(\"wrong number of arguments for 'test' command\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point1\", \"WITHIN\", \"bounds\", \"1\").Err(\"wrong number of arguments for 'test' command\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point1\", \"WITHIN\", \"bounds\").Err(\"wrong number of arguments for 'test' command\"),\n\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point1\", \"WITHIN\", \"bounds\", \"1\", \"2\", \"3\", \"dd\").Err(\"invalid argument 'dd'\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point1\", \"WITHIN\", \"bounds\", \"1\", \"2\", \"cc\", \"dd\").Err(\"invalid argument 'cc'\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point1\", \"WITHIN\", \"bounds\", \"1\", \"bb\", \"cc\", \"dd\").Err(\"invalid argument 'bb'\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point1\", \"WITHIN\", \"bounds\", \"aa\", \"bb\", \"cc\", \"dd\").Err(\"invalid argument 'aa'\"),\n\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point6\", \"WITHIN\", \"GET\").Err(\"wrong number of arguments for 'test' command\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point6\", \"WITHIN\", \"GET\", \"mykey\").Err(\"wrong number of arguments for 'test' command\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point6\", \"WITHIN\", \"GET\", \"mykey\", \"point6\").Str(\"1\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point6\", \"WITHIN\", \"GET\", \"mykey__\", \"point6\").Err(\"key not found\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point6\", \"WITHIN\", \"GET\", \"mykey\", \"point6__\").Err(\"id not found\"),\n\t)\n}\n\nfunc testcmd_INTERSECTS_test(mc *mockServer) error {\n\tpoly := `{\n\t\t\t\t\"type\": \"Polygon\",\n\t\t\t\t\"coordinates\": [\n\t\t\t\t\t[\n\t\t\t\t\t\t[-122.44126439094543,37.732906137107],\n\t\t\t\t\t\t[-122.43980526924135,37.732906137107],\n\t\t\t\t\t\t[-122.43980526924135,37.73421283683962],\n\t\t\t\t\t\t[-122.44126439094543,37.73421283683962],\n\t\t\t\t\t\t[-122.44126439094543,37.732906137107]\n\t\t\t\t\t]\n\t\t\t\t]\n\t\t\t}`\n\tpoly8 := `{\"type\":\"Polygon\",\"coordinates\":[[[-122.4408378,37.7341129],[-122.4408378,37.733],[-122.44,37.733],[-122.44,37.7341129],[-122.4408378,37.7341129]],[[-122.44060993194579,37.73345766902749],[-122.44044363498686,37.73345766902749],[-122.44044363498686,37.73355524732416],[-122.44060993194579,37.73355524732416],[-122.44060993194579,37.73345766902749]],[[-122.44060724973677,37.7336888869566],[-122.4402102828026,37.7336888869566],[-122.4402102828026,37.7339752567853],[-122.44060724973677,37.7339752567853],[-122.44060724973677,37.7336888869566]]]}`\n\tpoly9 := `{\"type\": \"Polygon\",\"coordinates\": [[[-122.44037926197052,37.73313523548048],[-122.44017541408539,37.73313523548048],[-122.44017541408539,37.73336857568778],[-122.44037926197052,37.73336857568778],[-122.44037926197052,37.73313523548048]]]}`\n\tpoly10 := `{\"type\": \"Polygon\",\"coordinates\": [[[-122.44040071964262,37.73359343010089],[-122.4402666091919,37.73359343010089],[-122.4402666091919,37.73373767596864],[-122.44040071964262,37.73373767596864],[-122.44040071964262,37.73359343010089]]]}`\n\tpoly101 := `{\"type\":\"Polygon\",\"coordinates\":[[[-122.44051605463028,37.73375464605226],[-122.44028002023695,37.73375464605226],[-122.44028002023695,37.733903134117966],[-122.44051605463028,37.733903134117966],[-122.44051605463028,37.73375464605226]]]}`\n\n\treturn mc.DoBatch(\n\t\tDo(\"SET\", \"mykey\", \"point1\", \"POINT\", 37.7335, -122.4412).OK(),\n\t\tDo(\"SET\", \"mykey\", \"point2\", \"POINT\", 37.7335, -122.44121).OK(),\n\t\tDo(\"SET\", \"mykey\", \"line3\", \"OBJECT\", `{\"type\":\"LineString\",\"coordinates\":[[-122.4408378,37.7341129],[-122.4408378,37.733]]}`).OK(),\n\t\tDo(\"SET\", \"mykey\", \"poly4\", \"OBJECT\", `{\"type\":\"Polygon\",\"coordinates\":[[[-122.4408378,37.7341129],[-122.4408378,37.733],[-122.44,37.733],[-122.44,37.7341129],[-122.4408378,37.7341129]]]}`).OK(),\n\t\tDo(\"SET\", \"mykey\", \"multipoly5\", \"OBJECT\", `{\"type\":\"MultiPolygon\",\"coordinates\":[[[[-122.4408378,37.7341129],[-122.4408378,37.733],[-122.44,37.733],[-122.44,37.7341129],[-122.4408378,37.7341129]]],[[[-122.44091033935547,37.731981251280985],[-122.43994474411011,37.731981251280985],[-122.43994474411011,37.73254976045042],[-122.44091033935547,37.73254976045042],[-122.44091033935547,37.731981251280985]]]]}`).OK(),\n\t\tDo(\"SET\", \"mykey\", \"point6\", \"POINT\", -5, 5).OK(),\n\t\tDo(\"SET\", \"mykey\", \"point7\", \"POINT\", 33, 21).OK(),\n\t\tDo(\"SET\", \"mykey\", \"poly8\", \"OBJECT\", poly8).OK(),\n\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point1\", \"INTERSECTS\", \"OBJECT\", poly).Str(\"1\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point2\", \"INTERSECTS\", \"OBJECT\", poly).Str(\"1\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"line3\", \"INTERSECTS\", \"OBJECT\", poly).Str(\"1\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"poly4\", \"INTERSECTS\", \"OBJECT\", poly).Str(\"1\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"multipoly5\", \"INTERSECTS\", \"OBJECT\", poly).Str(\"1\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"poly8\", \"INTERSECTS\", \"OBJECT\", poly).Str(\"1\"),\n\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point6\", \"INTERSECTS\", \"OBJECT\", poly).Str(\"0\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"point7\", \"INTERSECTS\", \"OBJECT\", poly).Str(\"0\"),\n\n\t\tDo(\"TEST\", \"OBJECT\", poly9, \"INTERSECTS\", \"OBJECT\", poly8).Str(\"1\"),\n\t\tDo(\"TEST\", \"OBJECT\", poly10, \"INTERSECTS\", \"OBJECT\", poly8).Str(\"1\"),\n\t\tDo(\"TEST\", \"OBJECT\", poly101, \"INTERSECTS\", \"OBJECT\", poly8).Str(\"0\"),\n\t)\n}\n\nfunc testcmd_INTERSECTS_CLIP_test(mc *mockServer) error {\n\tpoly8 := `{\"type\":\"Polygon\",\"coordinates\":[[[-122.4408378,37.7341129],[-122.4408378,37.733],[-122.44,37.733],[-122.44,37.7341129],[-122.4408378,37.7341129]],[[-122.44060993194579,37.73345766902749],[-122.44044363498686,37.73345766902749],[-122.44044363498686,37.73355524732416],[-122.44060993194579,37.73355524732416],[-122.44060993194579,37.73345766902749]],[[-122.44060724973677,37.7336888869566],[-122.4402102828026,37.7336888869566],[-122.4402102828026,37.7339752567853],[-122.44060724973677,37.7339752567853],[-122.44060724973677,37.7336888869566]]]}`\n\tpoly9 := `{\"type\":\"Polygon\",\"coordinates\":[[[-122.44037926197052,37.73313523548048],[-122.44017541408539,37.73313523548048],[-122.44017541408539,37.73336857568778],[-122.44037926197052,37.73336857568778],[-122.44037926197052,37.73313523548048]]]}`\n\tmultipoly5 := `{\"type\":\"MultiPolygon\",\"coordinates\":[[[[-122.4408378,37.7341129],[-122.4408378,37.733],[-122.44,37.733],[-122.44,37.7341129],[-122.4408378,37.7341129]]],[[[-122.44091033935547,37.731981251280985],[-122.43994474411011,37.731981251280985],[-122.43994474411011,37.73254976045042],[-122.44091033935547,37.73254976045042],[-122.44091033935547,37.731981251280985]]]]}`\n\tpoly101 := `{\"type\":\"Polygon\",\"coordinates\":[[[-122.44051605463028,37.73375464605226],[-122.44028002023695,37.73375464605226],[-122.44028002023695,37.733903134117966],[-122.44051605463028,37.733903134117966],[-122.44051605463028,37.73375464605226]]]}`\n\n\treturn mc.DoBatch(\n\t\tDo(\"SET\", \"mykey\", \"point1\", \"POINT\", 37.7335, -122.4412).OK(),\n\n\t\tDo(\"TEST\", \"OBJECT\", poly9, \"INTERSECTS\", \"CLIP\", \"OBJECT\", \"{}\").Err(\"invalid clip type 'OBJECT'\"),\n\t\tDo(\"TEST\", \"OBJECT\", poly9, \"INTERSECTS\", \"CLIP\", \"CIRCLE\", \"1\", \"2\", \"3\").Err(\"invalid clip type 'CIRCLE'\"),\n\t\tDo(\"TEST\", \"OBJECT\", poly9, \"INTERSECTS\", \"CLIP\", \"GET\", \"mykey\", \"point1\").Err(\"invalid clip type 'GET'\"),\n\t\tDo(\"TEST\", \"OBJECT\", poly9, \"WITHIN\", \"CLIP\", \"BOUNDS\", 10, 10, 20, 20).Err(\"invalid argument 'CLIP'\"),\n\n\t\tDo(\"TEST\", \"OBJECT\", poly9, \"INTERSECTS\", \"CLIP\", \"SECTOR\").Err(\"invalid clip type 'SECTOR'\"),\n\n\t\tDo(\"TEST\", \"OBJECT\", poly9, \"INTERSECTS\", \"CLIP\", \"BOUNDS\", 37.732906137107, -122.44126439094543, 37.73421283683962, -122.43980526924135).Str(\"[1 \"+poly9+\"]\"),\n\t\tDo(\"TEST\", \"OBJECT\", poly9, \"INTERSECTS\", \"CLIP\", \"BOUNDS\", 37.732906137107, -122.44126439094543, 37.73421283683962, -122.43980526924135).JSON().Str(`{\"ok\":true,\"result\":true,\"object\":`+poly9+`}`),\n\t\tDo(\"TEST\", \"OBJECT\", poly8, \"INTERSECTS\", \"CLIP\", \"BOUNDS\", 37.733, -122.4408378, 37.7341129, -122.44).Str(\"[1 \"+poly8+\"]\"),\n\t\tDo(\"TEST\", \"OBJECT\", multipoly5, \"INTERSECTS\", \"CLIP\", \"BOUNDS\", 37.73227823422744, -122.44120001792908, 37.73319038868677, -122.43955314159392).Str(\"[1 \"+`{\"type\":\"FeatureCollection\",\"features\":[{\"type\":\"Feature\",\"geometry\":{\"type\":\"Polygon\",\"coordinates\":[[[-122.4408378,37.73319038868677],[-122.4408378,37.733],[-122.44,37.733],[-122.44,37.73319038868677],[-122.4408378,37.73319038868677]]]},\"properties\":{}},{\"type\":\"Feature\",\"geometry\":{\"type\":\"Polygon\",\"coordinates\":[[[-122.44091033935547,37.73227823422744],[-122.43994474411011,37.73227823422744],[-122.43994474411011,37.73254976045042],[-122.44091033935547,37.73254976045042],[-122.44091033935547,37.73227823422744]]]},\"properties\":{}}]}`+\"]\"),\n\t\tDo(\"TEST\", \"OBJECT\", poly101, \"INTERSECTS\", \"CLIP\", \"BOUNDS\", 37.73315644825698, -122.44054287672043, 37.73349585185455, -122.44008690118788).Str(\"0\"),\n\t)\n}\n\nfunc testcmd_expressionErrors_test(mc *mockServer) error {\n\treturn mc.DoBatch(\n\t\tDo(\"SET\", \"mykey\", \"foo\", \"OBJECT\", `{\"type\":\"LineString\",\"coordinates\":[[-122.4408378,37.7341129],[-122.4408378,37.733]]}`).OK(),\n\t\tDo(\"SET\", \"mykey\", \"bar\", \"OBJECT\", `{\"type\":\"LineString\",\"coordinates\":[[-122.4408378,37.7341129],[-122.4408378,37.733]]}`).OK(),\n\t\tDo(\"SET\", \"mykey\", \"baz\", \"OBJECT\", `{\"type\":\"LineString\",\"coordinates\":[[-122.4408378,37.7341129],[-122.4408378,37.733]]}`).OK(),\n\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"foo\", \"INTERSECTS\", \"(\", \"GET\", \"mykey\", \"bar\").Err(\"wrong number of arguments for 'test' command\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"foo\", \"INTERSECTS\", \"GET\", \"mykey\", \"bar\", \")\").Err(\"invalid argument ')'\"),\n\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"foo\", \"INTERSECTS\", \"OR\", \"GET\", \"mykey\", \"bar\").Err(\"invalid argument 'or'\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"foo\", \"INTERSECTS\", \"AND\", \"GET\", \"mykey\", \"bar\").Err(\"invalid argument 'and'\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"foo\", \"INTERSECTS\", \"GET\", \"mykey\", \"bar\", \"OR\", \"AND\", \"GET\", \"mykey\", \"baz\").Err(\"invalid argument 'and'\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"foo\", \"INTERSECTS\", \"GET\", \"mykey\", \"bar\", \"AND\", \"OR\", \"GET\", \"mykey\", \"baz\").Err(\"invalid argument 'or'\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"foo\", \"INTERSECTS\", \"GET\", \"mykey\", \"bar\", \"OR\", \"OR\", \"GET\", \"mykey\", \"baz\").Err(\"invalid argument 'or'\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"foo\", \"INTERSECTS\", \"GET\", \"mykey\", \"bar\", \"AND\", \"AND\", \"GET\", \"mykey\", \"baz\").Err(\"invalid argument 'and'\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"foo\", \"INTERSECTS\", \"GET\", \"mykey\", \"bar\", \"OR\").Err(\"wrong number of arguments for 'test' command\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"foo\", \"INTERSECTS\", \"GET\", \"mykey\", \"bar\", \"AND\").Err(\"wrong number of arguments for 'test' command\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"foo\", \"INTERSECTS\", \"GET\", \"mykey\", \"bar\", \"NOT\").Err(\"wrong number of arguments for 'test' command\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"foo\", \"INTERSECTS\", \"GET\", \"mykey\", \"bar\", \"NOT\", \"AND\", \"GET\", \"mykey\", \"baz\").Err(\"invalid argument 'and'\"),\n\n\t\tDo(\"TEST\").Err(\"wrong number of arguments for 'test' command\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"foo\").Err(\"wrong number of arguments for 'test' command\"),\n\t\tDo(\"TEST\", \"GET\", \"mykey\", \"foo\", \"jello\").Err(\"invalid argument 'jello'\"),\n\t)\n}\n\nfunc testcmd_expression_test(mc *mockServer) error {\n\tpoly := `{\n\t\t\t\t\"type\": \"Polygon\",\n\t\t\t\t\"coordinates\": [\n\t\t\t\t\t[\n\t\t\t\t\t\t[-122.44126439094543,37.732906137107],\n\t\t\t\t\t\t[-122.43980526924135,37.732906137107],\n\t\t\t\t\t\t[-122.43980526924135,37.73421283683962],\n\t\t\t\t\t\t[-122.44126439094543,37.73421283683962],\n\t\t\t\t\t\t[-122.44126439094543,37.732906137107]\n\t\t\t\t\t]\n\t\t\t\t]\n\t\t\t}`\n\tpoly8 := `{\"type\":\"Polygon\",\"coordinates\":[[[-122.4408378,37.7341129],[-122.4408378,37.733],[-122.44,37.733],[-122.44,37.7341129],[-122.4408378,37.7341129]],[[-122.44060993194579,37.73345766902749],[-122.44044363498686,37.73345766902749],[-122.44044363498686,37.73355524732416],[-122.44060993194579,37.73355524732416],[-122.44060993194579,37.73345766902749]],[[-122.44060724973677,37.7336888869566],[-122.4402102828026,37.7336888869566],[-122.4402102828026,37.7339752567853],[-122.44060724973677,37.7339752567853],[-122.44060724973677,37.7336888869566]]]}`\n\tpoly9 := `{\"type\": \"Polygon\",\"coordinates\": [[[-122.44037926197052,37.73313523548048],[-122.44017541408539,37.73313523548048],[-122.44017541408539,37.73336857568778],[-122.44037926197052,37.73336857568778],[-122.44037926197052,37.73313523548048]]]}`\n\n\treturn mc.DoBatch(\n\t\tDo(\"SET\", \"mykey\", \"line3\", \"OBJECT\", `{\"type\":\"LineString\",\"coordinates\":[[-122.4408378,37.7341129],[-122.4408378,37.733]]}`).OK(),\n\t\tDo(\"SET\", \"mykey\", \"poly8\", \"OBJECT\", poly8).OK(),\n\n\t\tDo(\"TEST\", \"OBJECT\", poly9, \"INTERSECTS\", \"NOT\", \"OBJECT\", poly).Str(\"0\"),\n\t\tDo(\"TEST\", \"OBJECT\", poly9, \"INTERSECTS\", \"NOT\", \"NOT\", \"OBJECT\", poly).Str(\"1\"),\n\t\tDo(\"TEST\", \"OBJECT\", poly9, \"INTERSECTS\", \"NOT\", \"NOT\", \"NOT\", \"OBJECT\", poly).Str(\"0\"),\n\n\t\tDo(\"TEST\", \"OBJECT\", poly9, \"INTERSECTS\", \"OBJECT\", poly8, \"OR\", \"OBJECT\", poly).Str(\"1\"),\n\t\tDo(\"TEST\", \"OBJECT\", poly9, \"INTERSECTS\", \"OBJECT\", poly8, \"AND\", \"OBJECT\", poly).Str(\"1\"),\n\t\tDo(\"TEST\", \"OBJECT\", poly9, \"INTERSECTS\", \"GET\", \"mykey\", \"poly8\", \"OR\", \"OBJECT\", poly).Str(\"1\"),\n\n\t\tDo(\"TEST\", \"OBJECT\", poly9, \"INTERSECTS\", \"GET\", \"mykey\", \"line3\").Str(\"0\"),\n\t\tDo(\"TEST\", \"OBJECT\", poly9, \"INTERSECTS\", \"GET\", \"mykey\", \"poly8\", \"AND\", \"(\", \"OBJECT\", poly, \"AND\", \"GET\", \"mykey\", \"line3\", \")\").Str(\"0\"),\n\t\tDo(\"TEST\", \"OBJECT\", poly9, \"INTERSECTS\", \"GET\", \"mykey\", \"poly8\", \"AND\", \"(\", \"OBJECT\", poly, \"OR\", \"GET\", \"mykey\", \"line3\", \")\").Str(\"1\"),\n\t\tDo(\"TEST\", \"OBJECT\", poly9, \"INTERSECTS\", \"GET\", \"mykey\", \"poly8\", \"AND\", \"(\", \"OBJECT\", poly, \"AND\", \"NOT\", \"GET\", \"mykey\", \"line3\", \")\").Str(\"1\"),\n\t\tDo(\"TEST\", \"OBJECT\", poly9, \"INTERSECTS\", \"NOT\", \"GET\", \"mykey\", \"line3\").Str(\"1\"),\n\t\tDo(\"TEST\", \"NOT\", \"OBJECT\", poly9, \"INTERSECTS\", \"GET\", \"mykey\", \"line3\").Str(\"1\"),\n\t\tDo(\"TEST\", \"OBJECT\", poly9, \"INTERSECTS\", \"GET\", \"mykey\", \"line3\", \"OR\", \"OBJECT\", poly8, \"AND\", \"OBJECT\", poly).Str(\"1\"),\n\t\tDo(\"TEST\", \"OBJECT\", poly9, \"INTERSECTS\", \"OBJECT\", poly8, \"AND\", \"OBJECT\", poly, \"OR\", \"GET\", \"mykey\", \"line3\").Str(\"1\"),\n\t\tDo(\"TEST\", \"OBJECT\", poly9, \"INTERSECTS\", \"GET\", \"mykey\", \"line3\", \"OR\", \"(\", \"OBJECT\", poly8, \"AND\", \"OBJECT\", poly, \")\").Str(\"1\"),\n\t\tDo(\"TEST\", \"OBJECT\", poly9, \"INTERSECTS\", \"(\", \"GET\", \"mykey\", \"line3\", \"OR\", \"OBJECT\", poly8, \")\", \"AND\", \"OBJECT\", poly).Str(\"1\"),\n\n\t\tDo(\"TEST\", \"OBJECT\", poly9, \"WITHIN\", \"OBJECT\", poly8, \"OR\", \"OBJECT\", poly).Str(\"1\"),\n\t\tDo(\"TEST\", \"OBJECT\", poly9, \"WITHIN\", \"OBJECT\", poly8, \"AND\", \"OBJECT\", poly).Str(\"1\"),\n\n\t\tDo(\"TEST\", \"OBJECT\", poly9, \"WITHIN\", \"GET\", \"mykey\", \"line3\").Str(\"0\"),\n\t\tDo(\"TEST\", \"OBJECT\", poly9, \"WITHIN\", \"GET\", \"mykey\", \"poly8\", \"AND\", \"(\", \"OBJECT\", poly, \"AND\", \"GET\", \"mykey\", \"line3\", \")\").Str(\"0\"),\n\t\tDo(\"TEST\", \"OBJECT\", poly9, \"WITHIN\", \"GET\", \"mykey\", \"poly8\", \"AND\", \"(\", \"OBJECT\", poly, \"OR\", \"GET\", \"mykey\", \"line3\", \")\").Str(\"1\"),\n\t\tDo(\"TEST\", \"OBJECT\", poly9, \"WITHIN\", \"GET\", \"mykey\", \"poly8\", \"AND\", \"(\", \"OBJECT\", poly, \"AND\", \"NOT\", \"GET\", \"mykey\", \"line3\", \")\").Str(\"1\"),\n\t\tDo(\"TEST\", \"OBJECT\", poly9, \"WITHIN\", \"NOT\", \"GET\", \"mykey\", \"line3\").Str(\"1\"),\n\t)\n}\n" }, { "path": "tests/tests_test.go", "content": "package tests\n\nimport (\n\t\"fmt\"\n\t\"math/rand\"\n\t\"os\"\n\t\"os/signal\"\n\t\"runtime\"\n\t\"strings\"\n\t\"sync\"\n\t\"syscall\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/gomodule/redigo/redis\"\n\t\"github.com/tidwall/limiter\"\n\t\"go.uber.org/atomic\"\n)\n\nconst (\n\tclear = \"\\x1b[0m\"\n\tbright = \"\\x1b[1m\"\n\tdim = \"\\x1b[2m\"\n\tblack = \"\\x1b[30m\"\n\tred = \"\\x1b[31m\"\n\tgreen = \"\\x1b[32m\"\n\tyellow = \"\\x1b[33m\"\n\tblue = \"\\x1b[34m\"\n\tmagenta = \"\\x1b[35m\"\n\tcyan = \"\\x1b[36m\"\n\twhite = \"\\x1b[37m\"\n)\n\nfunc TestIntegration(t *testing.T) {\n\n\tmockCleanup(true)\n\tdefer mockCleanup(true)\n\n\tch := make(chan os.Signal, 1)\n\tsignal.Notify(ch, os.Interrupt, syscall.SIGTERM)\n\tgo func() {\n\t\t<-ch\n\t\tmockCleanup(false)\n\t\tos.Exit(1)\n\t}()\n\n\tregTestGroup(\"keys\", subTestKeys)\n\tregTestGroup(\"json\", subTestJSON)\n\tregTestGroup(\"search\", subTestSearch)\n\tregTestGroup(\"testcmd\", subTestTestCmd)\n\tregTestGroup(\"client\", subTestClient)\n\tregTestGroup(\"scripts\", subTestScripts)\n\tregTestGroup(\"fence\", subTestFence)\n\tregTestGroup(\"info\", subTestInfo)\n\tregTestGroup(\"timeouts\", subTestTimeout)\n\tregTestGroup(\"metrics\", subTestMetrics)\n\tregTestGroup(\"follower\", subTestFollower)\n\tregTestGroup(\"aof\", subTestAOF)\n\tregTestGroup(\"monitor\", subTestMonitor)\n\tregTestGroup(\"proto\", subTestProto)\n\trunTestGroups(t)\n}\n\nvar allGroups []*testGroup\n\nfunc runTestGroups(t *testing.T) {\n\tlimit := runtime.NumCPU()\n\tif limit > 16 {\n\t\tlimit = 16\n\t}\n\tl := limiter.New(limit)\n\n\t// Initialize all stores as \"skipped\", but they'll be unset if the test is\n\t// not actually skipped.\n\tfor _, g := range allGroups {\n\t\tfor _, s := range g.subs {\n\t\t\ts.skipped.Store(true)\n\t\t}\n\t}\n\tfor _, g := range allGroups {\n\t\tfunc(g *testGroup) {\n\t\t\tt.Run(g.name, func(t *testing.T) {\n\t\t\t\tfor _, s := range g.subs {\n\t\t\t\t\tfunc(s *testGroupSub) {\n\t\t\t\t\t\tt.Run(s.name, func(t *testing.T) {\n\t\t\t\t\t\t\ts.skipped.Store(false)\n\t\t\t\t\t\t\tvar wg sync.WaitGroup\n\t\t\t\t\t\t\twg.Add(1)\n\t\t\t\t\t\t\tvar err error\n\t\t\t\t\t\t\tgo func() {\n\t\t\t\t\t\t\t\tl.Begin()\n\t\t\t\t\t\t\t\tdefer func() {\n\t\t\t\t\t\t\t\t\tl.End()\n\t\t\t\t\t\t\t\t\twg.Done()\n\t\t\t\t\t\t\t\t}()\n\t\t\t\t\t\t\t\terr = s.run()\n\t\t\t\t\t\t\t}()\n\t\t\t\t\t\t\tif false {\n\t\t\t\t\t\t\t\tt.Parallel()\n\t\t\t\t\t\t\t\tt.Run(\"bg\", func(t *testing.T) {\n\t\t\t\t\t\t\t\t\twg.Wait()\n\t\t\t\t\t\t\t\t\tif err != nil {\n\t\t\t\t\t\t\t\t\t\tt.Fatal(err)\n\t\t\t\t\t\t\t\t\t}\n\t\t\t\t\t\t\t\t})\n\t\t\t\t\t\t\t}\n\t\t\t\t\t\t})\n\t\t\t\t\t}(s)\n\t\t\t\t}\n\t\t\t})\n\t\t}(g)\n\t}\n\n\tdone := make(chan bool)\n\tgo func() {\n\t\tdefer func() { done <- true }()\n\t\t// count the largest sub test name\n\t\tvar largest int\n\t\tfor _, g := range allGroups {\n\t\t\tfor _, s := range g.subs {\n\t\t\t\tif !s.skipped.Load() {\n\t\t\t\t\tif len(s.name) > largest {\n\t\t\t\t\t\tlargest = len(s.name)\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\n\t\tfor {\n\t\t\tfinished := true\n\t\t\tfor _, g := range allGroups {\n\t\t\t\tskipped := true\n\t\t\t\tfor _, s := range g.subs {\n\t\t\t\t\tif !s.skipped.Load() {\n\t\t\t\t\t\tskipped = false\n\t\t\t\t\t\tbreak\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t\tif !skipped && !g.printed.Load() {\n\t\t\t\t\tfmt.Printf(\"\\n\"+bright+\"Testing %s\"+clear+\"\\n\", g.name)\n\t\t\t\t\tg.printed.Store(true)\n\t\t\t\t}\n\t\t\t\tconst frtmp = \"%s ... \"\n\t\t\t\tfor _, s := range g.subs {\n\t\t\t\t\tif !s.skipped.Load() && !s.printedName.Load() {\n\t\t\t\t\t\tpref := fmt.Sprintf(frtmp, s.name)\n\t\t\t\t\t\tnspaces := largest - len(pref) + 5\n\t\t\t\t\t\tif nspaces < 0 {\n\t\t\t\t\t\t\tnspaces = 0\n\t\t\t\t\t\t}\n\t\t\t\t\t\tspaces := strings.Repeat(\" \", nspaces)\n\t\t\t\t\t\tfmt.Printf(\"%s%s\", pref, spaces)\n\t\t\t\t\t\ts.printedName.Store(true)\n\t\t\t\t\t}\n\t\t\t\t\tif s.done.Load() && !s.printedResult.Load() {\n\t\t\t\t\t\tif s.err != nil {\n\t\t\t\t\t\t\tfmt.Printf(\"[\" + red + \"fail\" + clear + \"]\\n\")\n\t\t\t\t\t\t} else {\n\t\t\t\t\t\t\tfmt.Printf(\"[\" + green + \"ok\" + clear + \"]\\n\")\n\t\t\t\t\t\t}\n\t\t\t\t\t\ts.printedResult.Store(true)\n\t\t\t\t\t}\n\t\t\t\t\tif !s.skipped.Load() && !s.done.Load() {\n\t\t\t\t\t\tfinished = false\n\t\t\t\t\t\tbreak\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t\tif !finished {\n\t\t\t\t\tbreak\n\t\t\t\t}\n\t\t\t}\n\t\t\tif finished {\n\t\t\t\tbreak\n\t\t\t}\n\t\t\ttime.Sleep(time.Second / 4)\n\t\t}\n\t}()\n\t<-done\n\tvar fail bool\n\tfor _, g := range allGroups {\n\t\tfor _, s := range g.subs {\n\t\t\tif s.err != nil {\n\t\t\t\tt.Errorf(\"%s/%s/%s\\n%s\", t.Name(), g.name, s.name, s.err)\n\t\t\t\tfail = true\n\t\t\t}\n\t\t}\n\t}\n\tif fail {\n\t\tt.Fail()\n\t}\n\n}\n\ntype testGroup struct {\n\tname string\n\tsubs []*testGroupSub\n\tprinted atomic.Bool\n}\n\ntype testGroupSub struct {\n\tg *testGroup\n\tname string\n\tfn func(mc *mockServer) error\n\terr error\n\tskipped atomic.Bool\n\tdone atomic.Bool\n\tprintedName atomic.Bool\n\tprintedResult atomic.Bool\n}\n\nfunc regTestGroup(name string, fn func(g *testGroup)) {\n\tg := &testGroup{name: name}\n\tallGroups = append(allGroups, g)\n\tfn(g)\n}\n\nfunc (g *testGroup) regSubTest(name string, fn func(mc *mockServer) error) {\n\ts := &testGroupSub{g: g, name: name, fn: fn}\n\tg.subs = append(g.subs, s)\n}\n\nfunc (s *testGroupSub) run() (err error) {\n\t// This all happens in a background routine.\n\tdefer func() {\n\t\ts.err = err\n\t\ts.done.Store(true)\n\t}()\n\treturn func() error {\n\t\tmc, err := mockOpenServer(MockServerOptions{\n\t\t\tSilent: true,\n\t\t\tMetrics: true,\n\t\t})\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tdefer mc.Close()\n\t\treturn s.fn(mc)\n\t}()\n}\n\nfunc BenchmarkAll(b *testing.B) {\n\tmockCleanup(true)\n\tdefer mockCleanup(true)\n\n\tch := make(chan os.Signal, 1)\n\tsignal.Notify(ch, os.Interrupt, syscall.SIGTERM)\n\tgo func() {\n\t\t<-ch\n\t\tmockCleanup(true)\n\t\tos.Exit(1)\n\t}()\n\n\tmc, err := mockOpenServer(MockServerOptions{\n\t\tSilent: true, Metrics: true,\n\t})\n\tif err != nil {\n\t\tb.Fatal(err)\n\t}\n\tdefer mc.Close()\n\trunSubBenchmark(b, \"search\", mc, subBenchSearch)\n}\n\nfunc loadBenchmarkPoints(b *testing.B, mc *mockServer) (err error) {\n\tconst nPoints = 200000\n\trand.Seed(time.Now().UnixNano())\n\n\t// add a bunch of points\n\tfor i := 0; i < nPoints; i++ {\n\t\tval := fmt.Sprintf(\"val:%d\", i)\n\t\tvar resp string\n\t\tvar lat, lon, fval float64\n\t\tfval = rand.Float64()\n\t\tlat = rand.Float64()*180 - 90\n\t\tlon = rand.Float64()*360 - 180\n\t\tresp, err = redis.String(mc.conn.Do(\"SET\",\n\t\t\t\"mykey\", val,\n\t\t\t\"FIELD\", \"foo\", fval,\n\t\t\t\"POINT\", lat, lon))\n\t\tif err != nil {\n\t\t\treturn\n\t\t}\n\t\tif resp != \"OK\" {\n\t\t\terr = fmt.Errorf(\"expected 'OK', got '%s'\", resp)\n\t\t\treturn\n\t\t}\n\t}\n\treturn\n}\n\nfunc runSubBenchmark(b *testing.B, name string, mc *mockServer, bench func(t *testing.B, mc *mockServer)) {\n\tb.Run(name, func(b *testing.B) {\n\t\tbench(b, mc)\n\t})\n}\n\nfunc runBenchStep(b *testing.B, mc *mockServer, name string, step func(mc *mockServer) error) {\n\tb.Helper()\n\tb.Run(name, func(b *testing.B) {\n\t\tb.Helper()\n\t\tif err := func() error {\n\t\t\t// reset the current server\n\t\t\tmc.ResetConn()\n\t\t\tdefer mc.ResetConn()\n\t\t\t// clear the database so the test is consistent\n\t\t\tif err := mc.DoBatch([][]interface{}{\n\t\t\t\t{\"OUTPUT\", \"resp\"}, {\"OK\"},\n\t\t\t\t{\"FLUSHDB\"}, {\"OK\"},\n\t\t\t}); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\terr := loadBenchmarkPoints(b, mc)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tb.ResetTimer()\n\t\t\tfor i := 0; i < b.N; i++ {\n\t\t\t\tif err := step(mc); err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\t\t\t}\n\t\t\treturn nil\n\t\t}(); err != nil {\n\t\t\tb.Fatal(err)\n\t\t}\n\t})\n}\n" }, { "path": "tests/timeout_test.go", "content": "package tests\n\nimport (\n\t\"fmt\"\n\t\"math/rand\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/gomodule/redigo/redis\"\n)\n\nfunc subTestTimeout(g *testGroup) {\n\tg.regSubTest(\"spatial\", timeout_spatial_test)\n\tg.regSubTest(\"search\", timeout_search_test)\n\tg.regSubTest(\"scripts\", timeout_scripts_test)\n\tg.regSubTest(\"no writes\", timeout_no_writes_test)\n\tg.regSubTest(\"within scripts\", timeout_within_scripts_test)\n\tg.regSubTest(\"no writes within scripts\", timeout_no_writes_within_scripts_test)\n}\n\nfunc setup(mc *mockServer, count int, points bool) (err error) {\n\trand.Seed(time.Now().UnixNano())\n\n\t// add a bunch of points\n\tfor i := 0; i < count; i++ {\n\t\tval := fmt.Sprintf(\"val:%d\", i)\n\t\tvar resp string\n\t\tvar lat, lon, fval float64\n\t\tfval = rand.Float64()\n\t\tif points {\n\t\t\tlat = rand.Float64()*180 - 90\n\t\t\tlon = rand.Float64()*360 - 180\n\t\t\tresp, err = redis.String(mc.conn.Do(\"SET\",\n\t\t\t\t\"mykey\", val,\n\t\t\t\t\"FIELD\", \"foo\", fval,\n\t\t\t\t\"POINT\", lat, lon))\n\t\t} else {\n\t\t\tresp, err = redis.String(mc.conn.Do(\"SET\",\n\t\t\t\t\"mykey\", val,\n\t\t\t\t\"FIELD\", \"foo\", fval,\n\t\t\t\t\"STRING\", val))\n\t\t}\n\t\tif err != nil {\n\t\t\treturn\n\t\t}\n\t\tif resp != \"OK\" {\n\t\t\terr = fmt.Errorf(\"expected 'OK', got '%s'\", resp)\n\t\t\treturn\n\t\t}\n\t\ttime.Sleep(time.Nanosecond)\n\t}\n\ttime.Sleep(time.Second * 3)\n\treturn\n}\n\nfunc timeout_spatial_test(mc *mockServer) error {\n\terr := setup(mc, 10000, true)\n\tif err != nil {\n\t\treturn err\n\t}\n\treturn mc.DoBatch(\n\t\tDo(\"SCAN\", \"mykey\", \"WHERE\", \"foo\", -1, 2, \"COUNT\").Str(\"10000\"),\n\t\tDo(\"INTERSECTS\", \"mykey\", \"WHERE\", \"foo\", -1, 2, \"COUNT\", \"BOUNDS\", -90, -180, 90, 180).Str(\"10000\"),\n\t\tDo(\"WITHIN\", \"mykey\", \"WHERE\", \"foo\", -1, 2, \"COUNT\", \"BOUNDS\", -90, -180, 90, 180).Str(\"10000\"),\n\n\t\tDo(\"TIMEOUT\", \"0.000001\", \"SCAN\", \"mykey\", \"WHERE\", \"foo\", -1, 2, \"COUNT\").Err(\"timeout\"),\n\t\tDo(\"TIMEOUT\", \"0.000001\", \"INTERSECTS\", \"mykey\", \"WHERE\", \"foo\", -1, 2, \"COUNT\", \"BOUNDS\", -90, -180, 90, 180).Err(\"timeout\"),\n\t\tDo(\"TIMEOUT\", \"0.000001\", \"WITHIN\", \"mykey\", \"WHERE\", \"foo\", -1, 2, \"COUNT\", \"BOUNDS\", -90, -180, 90, 180).Err(\"timeout\"),\n\t)\n}\n\nfunc timeout_search_test(mc *mockServer) (err error) {\n\terr = setup(mc, 10000, false)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\treturn mc.DoBatch([][]interface{}{\n\t\t{\"SEARCH\", \"mykey\", \"MATCH\", \"val:*\", \"COUNT\"}, {\"10000\"},\n\t\t{\"TIMEOUT\", \"0.000001\", \"SEARCH\", \"mykey\", \"MATCH\", \"val:*\", \"COUNT\"}, {\"ERR timeout\"},\n\t})\n}\n\nfunc timeout_scripts_test(mc *mockServer) (err error) {\n\tscript := `\n\t\tlocal clock = os.clock\n\t\tlocal function sleep(n)\n\t\t\tlocal t0 = clock()\n\t\t\twhile clock() - t0 <= n do end\n\t\tend\n\t\tsleep(0.5)\n\t`\n\tsha := \"e3ce9449853a622327f30c727a6e086ccd91d9d4\"\n\n\treturn mc.DoBatch([][]interface{}{\n\t\t{\"SCRIPT LOAD\", script}, {sha},\n\n\t\t{\"EVALSHA\", sha, 0}, {nil},\n\t\t{\"EVALROSHA\", sha, 0}, {nil},\n\t\t{\"EVALNASHA\", sha, 0}, {nil},\n\n\t\t{\"TIMEOUT\", \"0.1\", \"EVALSHA\", sha, 0}, {\"ERR timeout\"},\n\t\t{\"TIMEOUT\", \"0.1\", \"EVALROSHA\", sha, 0}, {\"ERR timeout\"},\n\t\t{\"TIMEOUT\", \"0.1\", \"EVALNASHA\", sha, 0}, {\"ERR timeout\"},\n\n\t\t{\"TIMEOUT\", \"0.9\", \"EVALSHA\", sha, 0}, {nil},\n\t\t{\"TIMEOUT\", \"0.9\", \"EVALROSHA\", sha, 0}, {nil},\n\t\t{\"TIMEOUT\", \"0.9\", \"EVALNASHA\", sha, 0}, {nil},\n\t})\n}\n\nfunc timeout_no_writes_test(mc *mockServer) (err error) {\n\treturn mc.DoBatch([][]interface{}{\n\t\t{\"SET\", \"mykey\", \"myid\", \"STRING\", \"foo\"}, {\"OK\"},\n\t\t{\"TIMEOUT\", 1, \"SET\", \"mykey\", \"myid\", \"STRING\", \"foo\"}, {\"ERR timeout not supported for 'set'\"},\n\t})\n}\n\nfunc scriptTimeoutErr(v interface{}) (resp, expect interface{}) {\n\ts := fmt.Sprintf(\"%v\", v)\n\tif strings.Contains(s, \"ERR timeout\") {\n\t\treturn v, v\n\t}\n\treturn v, \"A lua stack containing 'ERR timeout'\"\n}\n\nfunc timeout_within_scripts_test(mc *mockServer) (err error) {\n\terr = setup(mc, 10000, true)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tscript1 := \"return tile38.call('timeout', 10, 'SCAN', 'mykey', 'WHERE', 'foo', -1, 2, 'COUNT')\"\n\tscript2 := \"return tile38.call('timeout', 0.000001, 'SCAN', 'mykey', 'WHERE', 'foo', -1, 2, 'COUNT')\"\n\tsha1 := \"27a364b4e46ef493f6b70371086c286e2d5b5f49\"\n\tsha2 := \"2da9c05b54abfe870bdc8383a143f9d3aa656192\"\n\n\treturn mc.DoBatch([][]interface{}{\n\t\t{\"SCRIPT LOAD\", script1}, {sha1},\n\t\t{\"SCRIPT LOAD\", script2}, {sha2},\n\n\t\t{\"EVALSHA\", sha1, 0}, {\"10000\"},\n\t\t{\"EVALROSHA\", sha1, 0}, {\"10000\"},\n\t\t{\"EVALNASHA\", sha1, 0}, {\"10000\"},\n\t\t{\"EVALSHA\", sha2, 0}, {scriptTimeoutErr},\n\t\t{\"EVALROSHA\", sha2, 0}, {scriptTimeoutErr},\n\t\t{\"EVALNASHA\", sha2, 0}, {scriptTimeoutErr},\n\t})\n}\n\nfunc scriptTimeoutNotSupportedErr(v interface{}) (resp, expect interface{}) {\n\ts := fmt.Sprintf(\"%v\", v)\n\tif strings.Contains(s, \"ERR timeout not supported for\") {\n\t\treturn v, v\n\t}\n\treturn v, \"A lua stack containing 'ERR timeout not supported for'\"\n}\n\nfunc timeout_no_writes_within_scripts_test(mc *mockServer) (err error) {\n\tscript1 := \"return tile38.call('SET', 'mykey', 'myval', 'STRING', 'foo')\"\n\tscript2 := \"return tile38.call('timeout', 10, 'SET', 'mykey', 'myval', 'STRING', 'foo')\"\n\tsha1 := \"393d0adff113fdda45e3b5aff93c188c30099f48\"\n\tsha2 := \"5287c158d15eb53d800b7389d82df0d73b004bf1\"\n\n\treturn mc.DoBatch([][]interface{}{\n\t\t{\"SCRIPT LOAD\", script1}, {sha1},\n\t\t{\"SCRIPT LOAD\", script2}, {sha2},\n\t\t{\"EVALSHA\", sha1, 0, \"foo\"}, {\"OK\"},\n\t\t{\"EVALSHA\", sha2, 0, \"foo\"}, {scriptTimeoutNotSupportedErr},\n\t})\n}\n" } ]