[
  {
    "path": ".gitignore",
    "content": "src/test_cleanup.sh\nsrc/test_reindex.sh\nsrc/node_modules\npackage-lock.json\nnode_modules\n.idea\n"
  },
  {
    "path": "CHANGELOG.md",
    "content": "# Changelog\nAll notable changes to this project will be documented in this file.\n\nThe format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),\nand this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).\n## [1.11.2] - 2020-11-12\n### Fixed\n- Enable indexing of review ratings - @soyamore (#107)\n\n## [1.11.1] - 2020.09.15\n### Added\n - Updates magento2-rest-client dependecy from version 0.0.2 to 0.0.12 (#106) \n - Bump lodash from 4.17.13 to 4.17.19 (#103)\n\n## [1.11] - 2020.04.15\n### Added\n - Elastic7 support - @pkarw (#96) \n - Add product attributes_metadata - @andrzejewsky (#99)\n\n## [1.10] - 2019.07.10\n### Added\n - Added optional Redis Auth functionality. - @rain2o (#42)\n - MSI support - @dimasch (#86)\n \n### Fixed\n - Import throwing an error when product's first category name was empty - @Loac-fr (#92)\n - Typos in documentation - @kkdg, @adityasharma7 (#90, #91)\n \n## [1.9] - 2019.03.14\n### Added\n- New ENV variable `SEO_USE_URL_DISPATCHER` (default = true) added. When set, then the `product.url_path` and `category.url_path` are automatically populated for the UrlDispatche featu$\n\n## [1.8.1] - 2019.02.13\n### Changed / improved\n - `elasticsearch.apiVesion` with default = 5.6 added to the config\n\n## [1.8.0] - 2019.02.08\n### Added\n- Video data mapper @rain2go [#75](https://github.com/DivanteLtd/mage2vuestorefront/pull/75)\n\n## [1.8.0]\n### Added\n - Setting `configurable_options.label` from the attribute meta descriptor. **Note:** When You modify any configurable attribute label in Magento You should reindex all products now\n - Configurable parent refresh sync - enabled in the `productsdelta` and `productsworker` modes and in `products --sku=<singleSku>`. This mode is refreshing the configurable parent product for the simple child which requires update. Its' required to start the `clis.js productsworker` (example call: `test_product_worker.sh`) for processing these parent updates,\n - Example calls added: `test_product_delta.sh` - for delta indexer, `test_product_worker.sh` for products worker, `test_fullreindex_multiprocess.sh` for multi process/parallel updates, `test_by_sky.sh` - for single SKU updates.\n\n ### Removed\n -  Mongodb support has been removed\n - old `package.json` and `yarn.lock` from `src` directory\n\n## [1.7.1] - 2019-01-29\n### Added\n- This changelog file\n\n### Fixed\n- Slugify funtion properly parse regional characters changing them to latin\n- yarn.lock file rebuild after some issues with merging\n\n### Changed\n- Slugify funtion moved outside adapter and exposed as helper\n"
  },
  {
    "path": "LICENSE",
    "content": "MIT License\n\nCopyright (c) 2017 Divante Ltd.\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n"
  },
  {
    "path": "README.md",
    "content": "# mage2alokai\n\n### Stay connected\n\n[![GitHub Repo stars](https://img.shields.io/github/stars/vuestorefront/vue-storefront?style=social)](https://github.com/vuestorefront/vue-storefront)\n[![Twitter Follow](https://img.shields.io/twitter/follow/vuestorefront?style=social)](https://twitter.com/vuestorefront)\n[![YouTube Channel Subscribers](https://img.shields.io/youtube/channel/subscribers/UCkm1F3Cglty3CE1QwKQUhhg?style=social)](https://www.youtube.com/c/VueStorefront)\n[![Discord](https://img.shields.io/discord/770285988244750366?label=join%20discord&logo=Discord&logoColor=white)](https://discord.vuestorefront.io)\n\n\nFor those who would love to work with Magento on backend but use NoSQL power on the frontend. Two way / real time data synchronizer.\n\nIt's part of [alokai project - first Progressive Web App for eCommerce](https://github.com/DivanteLtd/vue-storefront) with Magento2 support.\nSome details about the rationale and our goals [here](https://www.linkedin.com/pulse/magento2-nosql-database-pwa-support-piotr-karwatka)\n\nIt synchronizes all the products, attributes, taxrules, categories and links between products and categories.\n\nThis is multi-process data synchronizer between Magento to Alokai ElasticSearch database.\n\nAt this point synchronization works with following entities:\n- Products\n- Categories\n- Taxrules\n- Attributes\n- Product-to-categories\n- Reviews (require custom module Divante/ReviewApi to work)\n- Cms Blocks & Pages (require custom module [SnowdogApps/magento2-cms-api](https://github.com/SnowdogApps/magento2-cms-api))\n\nCategories and Product-to-categories links are additionaly stored in Redis cache for rapid-requests (for example from your WebAPI). Our other project [alokai-api](https://github.com/DivanteLtd/vue-storefront-api) exposes this database to be used in PWA/JS webapps.\n\nDatasync uses oauth + magento2 rest API to get the data.\nKUE is used for job queueing and multi-process/multi-tenant processing is enabled by default\nElasticSearch is used for NoSQL database\nRedis is used for KUE queue backend\n\nBy default all services are used without authorization and on default ports (check out config.js or ENV variables for change of this behavior). \n\n\n**Tutorial on installation / integration [manual for Alokai connectivity](https://medium.com/@piotrkarwatka/vue-storefront-how-to-install-and-integrate-with-magento2-227767dd65b2)**\n\n\n## How to perform full / initial import for Alokai\n\nTo get started with VS we must start with some very basics about the architecture; the project is backed by three separate Node.js applications\n\n### Alokai Architecture\n[alokai (Github)](https://github.com/DivanteLtd/vue-storefront) — is the main project where you can also find most of the documentation, issues mapped to further releases and other resources to start with — Vue.js on webpack.\n\n[alokai-api (Github)](https://github.com/DivanteLtd/vue-storefront-api) — is the API layer which provides the data to vue-storefront app — Node.js, Express; This project consist of docker instances for Redis and ElasticSearch required by mage2vuestorefront and pimcore2vuestorefront\n\nmage2alokai — THIS project -data bridges which are in charge of moving data back from Magento2 to Alokai data store.\n\nYou must install `alokai-api` locally. You may install it using the Alokai installer - see details. Or manually by executing the sequence of commands:\n\n```bash\ngit clone https://github.com/DivanteLtd/vue-storefront-api\ncd vue-storefront-api\nnpm install\nnpm run migrate\ndocker-compose up -d\nnpm run dev\n```\n\nThe key command is `docker-compose up -d` which runs the ElasticSearch and Redis instances - both required by `mage2vuestorefront`\n\n### Elastic 7 Support\n\nBy default, Alokai API docker files and config are based on Elastic 5.6. We plan to change the default Elastic version to 7 with the 1.11 stable release. As for now, the [Elastic 7 support](https://github.com/DivanteLtd/vue-storefront-api/pull/342) is marked as **experimental**. \n\nIn order to index data to Elastic 7 please make sure you set the proper `apiVersion` in the `config.js`:\n\n```js\n  elasticsearch: {\n    apiVersion: process.env.ELASTICSEARCH_API_VERSION || '7.1'\n  },\n```\n\nor just use the env variable:\n\n```bash\nexport ELASTICSEARCH_API_VERSION=7.1\n```\n\nStarting from [Elasitc 6 and 7](https://www.elastic.co/guide/en/elasticsearch/reference/current/breaking-changes-7.0.html) we can have **just single** document type per single index. Alokai used to have `product`, `category` ... types defined in the `vue_storefront_catalog`.\n\nFrom now on, we're using the separate indexes per each entity type. The convention is: `${indexName}_${entityType}`. If your' **logical index name** is `vue_storefront_catalog` then it will be mapped to the **physical indexes** of: `vue_storefront_catalog_product`, `vue_storefront_catalog_category` ...\n\n### Initial Alokai import\n\nNow, You're ready to run the importer. Please check the [config file](https://github.com/DivanteLtd/mage2vuestorefront/blob/master/src/config.js). You may setup the Magento access data and URLs by config values or ENV variables.\n\nWe'll use in the following example - the ENV variables. The simplest command sequence to perform full reindex is:\n\n```bash\nexport TIME_TO_EXIT=2000\nexport MAGENTO_CONSUMER_KEY=byv3730rhoulpopcq64don8ukb8lf2gq\nexport MAGENTO_CONSUMER_SECRET=u9q4fcobv7vfx9td80oupa6uhexc27rb\nexport MAGENTO_ACCESS_TOKEN=040xx3qy7s0j28o3q0exrfop579cy20m\nexport MAGENTO_ACCESS_TOKEN_SECRET=7qunl3p505rubmr7u1ijt7odyialnih9\n\necho 'Default store - in our case United States / en'\nexport MAGENTO_URL=http://demo-magento2.vuestorefront.io/rest\nexport INDEX_NAME=vue_storefront_catalog\n\nnode --harmony cli.js categories --removeNonExistent=true\nnode --harmony cli.js productcategories --partitions=1\nnode --harmony cli.js attributes --removeNonExistent=true\nnode --harmony cli.js taxrule --removeNonExistent=true\nnode --harmony cli.js products --removeNonExistent=true --partitions=1\nnode --harmony cli.js reviews\n```\n\nAfter installing the 3rd party Magneto module ([SnowdogApps/magento2-cms-api](https://github.com/SnowdogApps/magento2-cms-api)) there are two additional imports available:\n```\nnode --harmony cli.js blocks\nnode --harmony cli.js pages\n```\n\n**Please note:**\n- `--removeNonExistent` option means - all records that were found in the index but currently don't exist in the API feed - will be removed. Please use this option ONLY for the full reindex!\n- `INDEX_NAME` by default is set to the `vue_storefront_catalog` but You may set it to any other elastic search index name.\n- The `categories` importer option `--generateUniqueUrlKeys` is by default set to true. This is due the fact that in Magento2, the `category.url_key` field is not mandatory unique and from v. 1.7 Alokai uses the `category.url_key` to display the category details without any client's side modification.\n- `PRODUCTS_EXCLUDE_DISABLED` by default is set to `false`. To only import enabled products set this to `true`.\n\n**Cache invalidation:** Recent version of Alokai do support output caching. Output cache is being tagged with the product and categories id (products and categories used on specific page). Mage2vuestorefront can invalidate cache of product and category pages if You set the following ENV variables:\n\n```bash\nexport VS_INVALIDATE_CACHE_URL=http://localhost:3000/invalidate?key=aeSu7aip&tag=\nexport VS_INVALIDATE_CACHE=1\n```\n\n- `VS_INVALIDATE_CACHE_URL` is a cache to the Alokai instance - used as a webhook to clear the output cache.\n\nPlease note:\nAfter data import - especially when You're not sure about the product attributes data types - please **reindex** ElasticSearch to establish the correct / current database schema. You may do this using [Database tool](https://github.com/DivanteLtd/vue-storefront/blob/master/doc/Database%20tool.md) in the `vue-storefront-api` folder:\n\n```bash\ncd vue-storefront-api\nnpm run db rebuild -- --indexName=vue_storefront_catalog\n```\n\nIf You like to create a new, empty index please run:\n```bash\ncd vue-storefront-api\nnpm run db new -- --indexName=vue_storefront_catalog\n```\n\n### Checking indexed data\n\nIf you want to see how many products were stored into Elastic data store, you can use Kibana to do so. Kibana is part of alokai-api. Once you start docker containers of alokai-api you can access it on http://localhost:5601/.\n\nTo see count of indexed products go to DEV tools and run following query:\n\n```\nGET vue_storefront_catalog/product/_count\n```\n\nSee https://www.elastic.co/guide/en/kibana/current/console-kibana.html to find out more.\n\n### Delta indexer\n\nAfter initial setup and full-reindex You may want to add indexer to the `crontab` to index only modified product records.\nThis is fairly easy - You just need to add the following command to crontab:\n\n```bash\nnode --harmony cli.js productsdelta --partitions=1\n```\n\nThis command will execute full reindex at first call - and then will be storing the last index date in the `.lastIndex.json` and downloading only these products which have `updated_at` > last index date.\n\nIf you have a multistore setup and would like to use the delta indexer for each storeview you can not use the delta timestamp from `.lastIndex.json` for all stores; instead\nyou will need to set the `INDEX_META_PATH` to a unique value for each store you are indexing. For instance:\n\n```\nexport INDEX_META_PATH=.lastIndex-UK.json && node --harmony cli.js productsdelta --partitions=1\n```\n\nPlease note: Magento2 has a bug with altering `updated_at` field. Please install [a fix for that](https://github.com/codepeak/magento2-productfix) before using this method: \n\n\n```bash\ncomposer require codepeak/magento2-productfix\nphp bin/magento cache:flush\n```\n\n### Parent products updates \n\nPlease note if there is a `simple` product update request coming from Delta Indexer or On Demand indexer `mage2vuestorefront` will - by default - check and update the `configurable`/parent product as well. The parent product update is scheduled in the `productsworker` mode - using KUE queue. Please make sure You're runinng at least one worker instance to process these on-demand request:\n\n```bash\ncd mage2vuestorefront/src\nexport TIME_TO_EXIT=2000\nexport MAGENTO_CONSUMER_KEY=byv3730rhoulpopcq64don8ukb8lf2gq\nexport MAGENTO_CONSUMER_SECRET=u9q4fcobv7vfx9td80oupa6uhexc27rb\nexport MAGENTO_ACCESS_TOKEN=040xx3qy7s0j28o3q0exrfop579cy20m\nexport MAGENTO_ACCESS_TOKEN_SECRET=7qunl3p505rubmr7u1ijt7odyialnih9\nexport PORT=6060\nexport MAGENTO_URL=http://demo-magento2.vuestorefront.io/rest\n\nnode --harmony cli.js productsworker\n```\n\nThis second process will take care of indexing the parent products updates whetever any of it's  `configurable_children` simple products has been changed.\n\n### On-demand indexer (experimental!)\n\nMage2nosql supports an on-demand indexer - where Magento calls a special webhook to update modified products.\nIn the current version the webhook notifies mage2vuestorefront about changed product SKUs and mage2vuestorefront pulls the modified products data via Magento2 APIs.\n\nFirst. You should install [Magento2 module called VsBridge](https://github.com/DivanteLtd/magento2-vsbridge)\nSecond. Deploy mage2vuestorefront on the server.\n\nThen You may want to start a webapi process:\n\n```bash\ncd mage2vuestorefront/src\nexport TIME_TO_EXIT=2000\nexport MAGENTO_CONSUMER_KEY=byv3730rhoulpopcq64don8ukb8lf2gq\nexport MAGENTO_CONSUMER_SECRET=u9q4fcobv7vfx9td80oupa6uhexc27rb\nexport MAGENTO_ACCESS_TOKEN=040xx3qy7s0j28o3q0exrfop579cy20m\nexport MAGENTO_ACCESS_TOKEN_SECRET=7qunl3p505rubmr7u1ijt7odyialnih9\nexport PORT=6060\nexport MAGENTO_URL=http://demo-magento2.vuestorefront.io/rest\n\nnode --harmony webapi.js\n```\n\nThe API will be listening on port 6060. Typically non-standard ports like this one are not exposed on the firewall. Please consider setting up [simple nginx proxy for this service](https://www.digitalocean.com/community/tutorials/how-to-set-up-a-node-js-application-for-production-on-ubuntu-16-04).\n\nAnyway - this API must be publicly available via Internet OR You must have the mage2vuestorefront installed on the same machine like Magento2.\n\nGo to Your Magento2 admin panel, then to Stores -> Configuration -> VsBridge and set-up \"Edit product\" url to: `http://localhost:6060/magento/products/update`. **Please note:** Product delete endpoint hasn't been implemented yet and it's good chance for Your PR.\n\nAfter having the webapi up and runing and this endpoint set, any Product save action will call `POST http://localhost:6060/magento/products/update` with the body set to `{\"sku\": [\"modified-sku-list\"]}`.\n\nWebapi will [add the products to the queue](https://github.com/DivanteLtd/mage2vuestorefront/blob/b44e7ede9aeb27f308e2a87033251a2491640da8/src/api/routes/magento.js#L19).\n\nPlease run the queue worker to process all the queued updates (You may run multiple queue workers even distributed across many machines):\n\n```bash\ncd mage2vuestorefront/src\nexport TIME_TO_EXIT=2000\nexport MAGENTO_CONSUMER_KEY=byv3730rhoulpopcq64don8ukb8lf2gq\nexport MAGENTO_CONSUMER_SECRET=u9q4fcobv7vfx9td80oupa6uhexc27rb\nexport MAGENTO_ACCESS_TOKEN=040xx3qy7s0j28o3q0exrfop579cy20m\nexport MAGENTO_ACCESS_TOKEN_SECRET=7qunl3p505rubmr7u1ijt7odyialnih9\nexport PORT=6060\nexport MAGENTO_URL=http://demo-magento2.vuestorefront.io/rest\n\nnode --harmony cli.js productsworker\n```\n\n**Please note:** We're using [kue based on Redis queue](https://github.com/Automattic/kue) which may be configured via `src/config.js` - `kue` + `redis` section.\n\n **Please note:** Redis now supports auth. In order to use Redis with auth simply pass the password to the `REDIS_AUTH` env variable.\n\n\n### Multistore setup\n\nMultiwebsite support starts with the ElasticSearch indexing. Basically - each store has it's own ElasticSearch index and should be populated separately using [mage2vuestorefront](https://github.com/DivanteLtd/mage2vuestorefront) tool.\n\nThe simplest script to index multi site:\n\n```bash\nexport TIME_TO_EXIT=2000\nexport MAGENTO_CONSUMER_KEY=byv3730rhoulpopcq64don8ukb8lf2gq\nexport MAGENTO_CONSUMER_SECRET=u9q4fcobv7vfx9td80oupa6uhexc27rb\nexport MAGENTO_ACCESS_TOKEN=040xx3qy7s0j28o3q0exrfop579cy20m\nexport MAGENTO_ACCESS_TOKEN_SECRET=7qunl3p505rubmr7u1ijt7odyialnih9\n\necho 'German store - de'\nexport MAGENTO_URL=http://demo-magento2.vuestorefront.io/rest/de\nexport INDEX_NAME=vue_storefront_catalog_de\n\nnode --harmony cli.js categories --removeNonExistent=true\nnode --harmony cli.js productcategories --partitions=1\nnode --harmony cli.js attributes --removeNonExistent=true\nnode --harmony cli.js taxrule --removeNonExistent=true\nnode --harmony cli.js products --removeNonExistent=true --partitions=1\n\necho 'Italian store - it'\nexport MAGENTO_URL=http://demo-magento2.vuestorefront.io/rest/it  \nexport INDEX_NAME=vue_storefront_catalog_it\n\nnode --harmony cli.js categories --removeNonExistent=true\nnode --harmony cli.js productcategories --partitions=1\nnode --harmony cli.js attributes --removeNonExistent=true\nnode --harmony cli.js taxrule --removeNonExistent=true\nnode --harmony cli.js products --removeNonExistent=true --partitions=1\n\necho 'Default store - in our case United States / en'\nexport MAGENTO_URL=http://demo-magento2.vuestorefront.io/rest\nexport INDEX_NAME=vue_storefront_catalog\n\nnode --harmony cli.js categories --removeNonExistent=true\nnode --harmony cli.js productcategories --partitions=1\nnode --harmony cli.js attributes --removeNonExistent=true\nnode --harmony cli.js taxrule --removeNonExistent=true\nnode --harmony cli.js products --removeNonExistent=true --partitions=1\n```\n\nAs You may see it's just a **it** or **de** store code which is added to the base Magento2 REST API urls that makes the difference and then the **INDEX_NAME** set to the dedicated index name.\n\nIn the result You should get:\n- *vue_storefront_catalog_it* - populated with the \"it\" store data\n- *vue_storefront_catalog_de* - populated with the \"it\" store data\n- *vue_storefront_catalog* - populated with the \"default\" store data\n\nThen, to use these indexes in the Alokai You should index the database schema using the `alokai-api` db tool:\n\n```bash\nnpm run db rebuild -- --indexName=vue_storefront_catalog_it\nnpm run db rebuild -- --indexName=vue_storefront_catalog_de\nnpm run db rebuild -- --indexName=vue_storefront_catalog\n```\n\nMore on <a href=\"https://github.com/DivanteLtd/vue-storefront/blob/master/doc/Multistore%20setup.md\">how to setup Alokai in the Multistore mode</a>.\n\n### Indexing configurable products attributes for filters\n\nIf You like to have Category filter working with configurable products - You need to expand the `product.configurable_children.attrName` to `product.attrName_options` array. This is automatically done by [mage2vuestorefront](https://github.com/DivanteLtd/mage2vuestorefront) for all attributes set as `product.configurable_options` (by default: color, size). If You like to add additional fields like `manufacturer` to the filters You need to expand `product.manufacturer_options` field. The easiest way to do so is to set `config.product.expandConfigurableFilters` to `['manufacturer']` and re-run the `mage2vuestorefront` indexer.\n\n## FAQ\n\nHere You can find some frequently asked questions answered:\n\n### I've been playing with Alokai for quite a while now and now I see that my catalog rule (-20% on all products) is not applied in one shop.\n\nPlease make sure that You've got the `config.synchronizeCatalogSpecialPrices` (env: `PRODUCTS_SPECIAL_PRICES`) and `config.renderCatalogRegularPrices` (env: `PRODUCTS_RENDER_PRICES`) set to `true` (default is `false`). Otherwise only the catalog prices will be synced (without dynamic pricing rules applied). You can also use the Alokai [dynamic-pricing option](https://divanteltd.github.io/vue-storefront/guide/integrations/direct-prices-sync.html) for the same purpose.\n\n\n## Advanced usage\n\nStart Elasticsearch and Redis:\n- `docker-compose up`\n\nInstall:\n- `npm install`\n- `cd src`\n\nConfig - see: config.js or use following ENV variables: \n- `MAGENTO_URL`\n- `MAGENTO_CONSUMER_KEY`\n- `MAGENTO_CONSUMER_SECRET`\n- `MAGENTO_ACCESS_TOKEN`\n- `MAGENTO_ACCESS_TOKEN_SECRET`\n- `DATABASE_URL` (default: 'elasticsearch://localhost:9200/vue_storefront_catalog')\n\n\nRun:\n- `cd src/` and then:\n- `node --harmony cli.js fullreindex` - synchronizes all the categories, products and links between products and categories\n\nOther commands supported:\n- `node --harmony cli.js products --partitions=10`\n- `node --harmony cli.js products --partitions=10 --initQueue=false` - run the products sync worker (product sync jobs should be populated eslewhere - it's used to run multi-tenant environment of workers)\n- `node --harmony cli.js products --partitions=10 --delta=true` - check products changed since last run; compared by updated_at field\n- `node --harmony cli.js productcategories` - to synchronize the links between products and categories it *should be run before* products synchronization because it populates Redis cache assigments for product-to-category link\n- `node --harmony cli.js categories`\n- `node --harmony cli.js products --adapter=magento --partitions=1 --skus=24-WG082-blue,24-WG082-pink`  - to pull out only selected SKUs\n- `node --harmony cli.js productsworker --adapter=magento --partitions=10`  - run queue worker for pulling out individual products (jobs can be assigned by webapi.js microservice triggers; it can be called by webhook for example from within Magento2 plugin)\n- `node --harmony webapi.js` - run localhost:3000 service endpoint for adding queue tasks\n\nWebAPI:\n- `node --harmony webapi.js`\n- `curl localhost:8080/api/magento/products/pull/WT09-XS-Purple` - to schedule data refresh for SKU=WT09-XS-Purple\n- `node --harmony cli.js productsworker` - to run pull request processor \n\nAvailable options:\n- `partitions=10` - number of concurent processes, by default number of CPUs core given\n- `adapter=magento` - for now only Magento is supported\n- `delta` - sync products changed from last run\n- command names: `products` / `attributes` / `taxrule` / `categories` / `productsworker` / `productcategories` / `productsdelta`\n\n\n"
  },
  {
    "path": "doc/destination_es_product.json",
    "content": "\"entity_id\": \"1351\",\n\"attribute_set_id\": \"10\",\n\"type_id\": \"simple\",\n\"sku\": \"6343908\",\n\"has_options\": \"0\",\n\"required_options\": \"0\",\n\"created_at\": \"2017-04-06 16:25:59\",\n\"updated_at\": \"2017-04-06 16:25:59\",\n\"visibility\": \"4\",\n\"price\": [\n    {\n        \"price\": \"32.5000\",\n        \"original_price\": \"34.9000\",\n        \"is_discount\": true,\n        \"customer_group_id\": \"0\"\n    },\n    {\n        \"price\": \"32.5000\",\n        \"original_price\": \"34.9000\",\n        \"is_discount\": true,\n        \"customer_group_id\": \"1\"\n    },\n    {\n        \"price\": \"32.5000\",\n        \"original_price\": \"34.9000\",\n        \"is_discount\": true,\n        \"customer_group_id\": \"4\"\n    },\n    {\n        \"price\": \"32.5000\",\n        \"original_price\": \"34.9000\",\n        \"is_discount\": true,\n        \"customer_group_id\": \"5\"\n    }\n],\n\"category\": [\n    {\n        \"category_id\": 2\n    },\n    {\n        \"category_id\": 4,\n        \"is_parent\": true,\n        \"name\": \"Książki\"\n    },\n    {\n        \"category_id\": 26,\n        \"is_parent\": true,\n        \"name\": \"biznes\"\n    },\n    {\n        \"category_id\": 232,\n        \"is_parent\": true,\n        \"name\": \" praca\"\n    },\n    {\n        \"category_id\": 3101,\n        \"name\": \"GRUPY\"\n    },\n    {\n        \"category_id\": 3220,\n        \"is_parent\": true,\n        \"name\": \" Donald Trump. 45. Prezydent Stanów Zjednoczonych \"\n    }\n],\n\"name\": [\n    \"Sukces mimo wszystko. Donald Trump\"\n],\n\"image\": [\n    \"/9/9/99906343908.jpg\"\n],\n\"ean\": [\n    \"9788328333772\"\n],\n\"pkwiu\": [\n    \"58.11\"\n],\n\"availability\": [\n    \"7\"\n],\n\"isbn\": [\n    \"978-83-283-3377-2\"\n],\n\"authors_es\": [\n    \"_68450_,_68449_,_68450_,_68449_\"\n],\n\"publishers_es\": [\n    \"_62556_,_62556_,_62556_,_62556_\"\n],\n\"producers_es\": [\n    \"_62556_,_62556_,_62556_,_62556_\"\n],\n\"status\": [\n    1\n],\n\"option_text_status\": [\n    \"Enabled\"\n],\n\"tax_class_id\": [\n    2\n],\n\"option_text_tax_class_id\": [\n    \"Taxable Goods\"\n],\n\"description\": [\n\"<h3>Ucz się od mistrza, a będziesz mieć świat u st&oacute;p!</h3> Kto powiedział, że cokolwiek na świecie jest poza Twoim zasięgiem? Pewnie nie zostaniesz już ani pierwszym kosmonautą, ani odkrywcą Ameryki, ale w gruncie rzeczy możesz osiągnąć niemal wszystko! Kluczem do sukcesu &mdash; Twojego i wszystkich innych ludzi &mdash; jest edukacja! Badania i wiedza, eksperymentowanie i doświadczanie: to wszystko stoi za sukcesem <strong>Donalda Trumpa</strong>. To dzięki nim ten ekscentryczny miliarder odnosi sukcesy we wszystkim, czego się dotknie. To nie magia &mdash; to twarde dane, chłodne analizy, podstawowe zasady relacji biznesowych i społecznych.<br /> <br /> W tej książce <strong>Donald Trump</strong> pokazuje nam, skąd sam czerpie wiedzę i jak wykorzystuje ją na co dzień. Dzieli się swoimi przemyśleniami, odpowiada na pytania zadawane mu na blogu i daje wskaz&oacute;wki swoim ewentualnym naśladowcom. Jeśli chcesz wspiąć się na sam szczyt ekonomicznej (lub społecznej) piramidy, jeśli nie boisz się wyzwań i ciężkiej pracy, jesteś głodny sukcesu i got&oacute;w na wiele wyrzeczeń, sprawdź, jak może Ci w tym pom&oacute;c <strong>Donald Trump</strong>. Zobacz, co naprawdę napędza świat biznesu, jak nakreślić skuteczny plan działania i nieustępliwie walczyć o swoje. Odkryj w sobie pasję i pozytywną motywację, stale poszerzaj horyzonty i nikomu nie daj się zastraszyć ani zepchnąć z obranej drogi. A przede wszystkim r&oacute;b to, co Ci sprawia przyjemność, co stanowi Twoją prawdziwą pasję, co budzi w Tobie dreszcz podniecenia. Podążaj za wskaz&oacute;wkami nowego prezydenta Stan&oacute;w Zjednoczonych, a świat stanie przed Tobą otworem! <ul> <li>Zawsze mierz wysoko.</li> <li>Stale dąż do perfekcji.</li> <li>Nie lekceważ przeczuć.</li> <li>Stawiaj na aktywność.</li> <li>Wykonuj pracę, kt&oacute;rą kochasz.</li> </ul> <h3>Odważni ludzie wyrastają ponad szklany sufit!</h3> <hr /><strong>Donald J. Trump</strong> &mdash; człowiek, kt&oacute;rego właściwie nie trzeba nikomu przedstawiać. Krezus i ekscentryk, prezes Organizacji Trumpa, nowy prezydent Stan&oacute;w Zjednoczonych. Z zawodu inwestor nieruchomości i autor bestseller&oacute;w, z zamiłowania filantrop.\t\"\n],\n\"short_description\": [\n    \"Ucz się od mistrza, a będziesz mieć świat u st&oacute;p! Kto powiedział, że cokolwiek na świecie jest poza Twoim zasięgiem? Pewnie nie zostaniesz już ani pierwszym kosmonautą, ani odkrywcą Ameryki, ale w gruncie rzeczy możesz osiągnąć niemal wszystko!...\"\n],\n\"special_price\": [\n    32.5\n],\n\"stock\": {\n    \"is_in_stock\": true,\n    \"qty\": 0\n}\n},\n\"sort\": [\n\"59\",\n\"59\"\n]\n}"
  },
  {
    "path": "doc/product_mapping.json",
    "content": "{\n    \"vue_storefront_catalog\": {\n        \"mappings\": {\n            \"product\": {\n                \"properties\": {\n                    \"actors_es\": {\n                        \"type\": \"string\"\n                    },\n                    \"attribute_set_id\": {\n                        \"type\": \"string\"\n                    },\n                    \"authors_es\": {\n                        \"type\": \"string\"\n                    },\n                    \"availability\": {\n                        \"type\": \"string\"\n                    },\n                    \"category\": {\n                        \"properties\": {\n                            \"category_id\": {\n                                \"type\": \"long\"\n                            },\n                            \"is_parent\": {\n                                \"type\": \"boolean\"\n                            },\n                            \"is_virtual\": {\n                                \"type\": \"string\"\n                            },\n                            \"name\": {\n                                \"type\": \"string\"\n                            },\n                            \"position\": {\n                                \"type\": \"long\"\n                            }\n                        }\n                    },\n                    \"composers_es\": {\n                        \"type\": \"string\"\n                    },\n                    \"created_at\": {\n                        \"type\": \"string\"\n                    },\n                    \"description\": {\n                        \"type\": \"string\"\n                    },\n                    \"directors_es\": {\n                        \"type\": \"string\"\n                    },\n                    \"ean\": {\n                        \"type\": \"string\"\n                    },\n                    \"entity_id\": {\n                        \"type\": \"string\"\n                    },\n                    \"has_options\": {\n                        \"type\": \"string\"\n                    },\n                    \"image\": {\n                        \"type\": \"string\"\n                    },\n                    \"is_new\": {\n                        \"type\": \"long\"\n                    },\n                    \"is_preorder\": {\n                        \"type\": \"long\"\n                    },\n                    \"isbn\": {\n                        \"type\": \"string\"\n                    },\n                    \"name\": {\n                        \"type\": \"string\"\n                    },\n                    \"option_text_is_new\": {\n                        \"type\": \"string\"\n                    },\n                    \"option_text_is_preorder\": {\n                        \"type\": \"string\"\n                    },\n                    \"option_text_status\": {\n                        \"type\": \"string\"\n                    },\n                    \"option_text_tax_class_id\": {\n                        \"type\": \"string\"\n                    },\n                    \"performers_es\": {\n                        \"type\": \"string\"\n                    },\n                    \"pkwiu\": {\n                        \"type\": \"string\"\n                    },\n                    \"price\": {\n                        \"properties\": {\n                            \"customer_group_id\": {\n                                \"type\": \"string\"\n                            },\n                            \"is_discount\": {\n                                \"type\": \"boolean\"\n                            },\n                            \"original_price\": {\n                                \"type\": \"string\"\n                            },\n                            \"price\": {\n                                \"type\": \"string\"\n                            }\n                        }\n                    },\n                    \"producers_es\": {\n                        \"type\": \"string\"\n                    },\n                    \"publishers_es\": {\n                        \"type\": \"string\"\n                    },\n                    \"required_options\": {\n                        \"type\": \"string\"\n                    },\n                    \"short_description\": {\n                        \"type\": \"string\"\n                    },\n                    \"sku\": {\n                        \"type\": \"string\"\n                    },\n                    \"special_price\": {\n                        \"type\": \"double\"\n                    },\n                    \"status\": {\n                        \"type\": \"long\"\n                    },\n                    \"stock\": {\n                        \"properties\": {\n                            \"is_in_stock\": {\n                                \"type\": \"boolean\"\n                            },\n                            \"qty\": {\n                                \"type\": \"long\"\n                            }\n                        }\n                    },\n                    \"tax_class_id\": {\n                        \"type\": \"long\"\n                    },\n                    \"type_id\": {\n                        \"type\": \"string\"\n                    },\n                    \"updated_at\": {\n                        \"type\": \"string\"\n                    },\n                    \"visibility\": {\n                        \"type\": \"string\"\n                    }\n                }\n            }\n        }\n    }\n}"
  },
  {
    "path": "doc/source_magento_category.json",
    "content": "{\n    \"id\": 0,\n    \"parent_id\": 0,\n    \"name\": \"string\",\n    \"is_active\": true,\n    \"position\": 0,\n    \"level\": 0,\n    \"product_count\": 0,\n    \"children_data\": [\n      {}\n    ]\n  }"
  },
  {
    "path": "doc/source_magento_product.json",
    "content": "\nModelModel Schema\n{\n  \"items\": [\n    {\n      \"id\": 0,\n      \"sku\": \"string\",\n      \"name\": \"string\",\n      \"attribute_set_id\": 0,\n      \"price\": 0,\n      \"status\": 0,\n      \"visibility\": 0,\n      \"type_id\": \"string\",\n      \"created_at\": \"string\",\n      \"updated_at\": \"string\",\n      \"weight\": 0,\n      \"extension_attributes\": {\n        \"stock_item\": {\n          \"item_id\": 0,\n          \"product_id\": 0,\n          \"stock_id\": 0,\n          \"qty\": 0,\n          \"is_in_stock\": true,\n          \"is_qty_decimal\": true,\n          \"show_default_notification_message\": true,\n          \"use_config_min_qty\": true,\n          \"min_qty\": 0,\n          \"use_config_min_sale_qty\": 0,\n          \"min_sale_qty\": 0,\n          \"use_config_max_sale_qty\": true,\n          \"max_sale_qty\": 0,\n          \"use_config_backorders\": true,\n          \"backorders\": 0,\n          \"use_config_notify_stock_qty\": true,\n          \"notify_stock_qty\": 0,\n          \"use_config_qty_increments\": true,\n          \"qty_increments\": 0,\n          \"use_config_enable_qty_inc\": true,\n          \"enable_qty_increments\": true,\n          \"use_config_manage_stock\": true,\n          \"manage_stock\": true,\n          \"low_stock_date\": \"string\",\n          \"is_decimal_divided\": true,\n          \"stock_status_changed_auto\": 0,\n          \"extension_attributes\": {}\n        },\n        \"bundle_product_options\": [\n          {\n            \"option_id\": 0,\n            \"title\": \"string\",\n            \"required\": true,\n            \"type\": \"string\",\n            \"position\": 0,\n            \"sku\": \"string\",\n            \"product_links\": [\n              {\n                \"id\": \"string\",\n                \"sku\": \"string\",\n                \"option_id\": 0,\n                \"qty\": 0,\n                \"position\": 0,\n                \"is_default\": true,\n                \"price\": 0,\n                \"price_type\": 0,\n                \"can_change_quantity\": 0,\n                \"extension_attributes\": {}\n              }\n            ],\n            \"extension_attributes\": {}\n          }\n        ],\n        \"downloadable_product_links\": [\n          {\n            \"id\": 0,\n            \"title\": \"string\",\n            \"sort_order\": 0,\n            \"is_shareable\": 0,\n            \"price\": 0,\n            \"number_of_downloads\": 0,\n            \"link_type\": \"string\",\n            \"link_file\": \"string\",\n            \"link_file_content\": {\n              \"file_data\": \"string\",\n              \"name\": \"string\",\n              \"extension_attributes\": {}\n            },\n            \"link_url\": \"string\",\n            \"sample_type\": \"string\",\n            \"sample_file\": \"string\",\n            \"sample_file_content\": {\n              \"file_data\": \"string\",\n              \"name\": \"string\",\n              \"extension_attributes\": {}\n            },\n            \"sample_url\": \"string\",\n            \"extension_attributes\": {}\n          }\n        ],\n        \"downloadable_product_samples\": [\n          {\n            \"id\": 0,\n            \"title\": \"string\",\n            \"sort_order\": 0,\n            \"sample_type\": \"string\",\n            \"sample_file\": \"string\",\n            \"sample_file_content\": {\n              \"file_data\": \"string\",\n              \"name\": \"string\",\n              \"extension_attributes\": {}\n            },\n            \"sample_url\": \"string\",\n            \"extension_attributes\": {}\n          }\n        ],\n        \"giftcard_amounts\": [\n          {\n            \"attribute_id\": 0,\n            \"website_id\": 0,\n            \"value\": 0,\n            \"website_value\": 0,\n            \"extension_attributes\": {}\n          }\n        ],\n        \"configurable_product_options\": [\n          {\n            \"id\": 0,\n            \"attribute_id\": \"string\",\n            \"label\": \"string\",\n            \"position\": 0,\n            \"is_use_default\": true,\n            \"values\": [\n              {\n                \"value_index\": 0,\n                \"extension_attributes\": {}\n              }\n            ],\n            \"extension_attributes\": {},\n            \"product_id\": 0\n          }\n        ],\n        \"configurable_product_links\": [\n          0\n        ]\n      },\n      \"product_links\": [\n        {\n          \"sku\": \"string\",\n          \"link_type\": \"string\",\n          \"linked_product_sku\": \"string\",\n          \"linked_product_type\": \"string\",\n          \"position\": 0,\n          \"extension_attributes\": {\n            \"qty\": 0\n          }\n        }\n      ],\n      \"options\": [\n        {\n          \"product_sku\": \"string\",\n          \"option_id\": 0,\n          \"title\": \"string\",\n          \"type\": \"string\",\n          \"sort_order\": 0,\n          \"is_require\": true,\n          \"price\": 0,\n          \"price_type\": \"string\",\n          \"sku\": \"string\",\n          \"file_extension\": \"string\",\n          \"max_characters\": 0,\n          \"image_size_x\": 0,\n          \"image_size_y\": 0,\n          \"values\": [\n            {\n              \"title\": \"string\",\n              \"sort_order\": 0,\n              \"price\": 0,\n              \"price_type\": \"string\",\n              \"sku\": \"string\",\n              \"option_type_id\": 0\n            }\n          ],\n          \"extension_attributes\": {}\n        }\n      ],\n      \"media_gallery_entries\": [\n        {\n          \"id\": 0,\n          \"media_type\": \"string\",\n          \"label\": \"string\",\n          \"position\": 0,\n          \"disabled\": true,\n          \"types\": [\n            \"string\"\n          ],\n          \"file\": \"string\",\n          \"content\": {\n            \"base64_encoded_data\": \"string\",\n            \"type\": \"string\",\n            \"name\": \"string\"\n          },\n          \"extension_attributes\": {\n            \"video_content\": {\n              \"media_type\": \"string\",\n              \"video_provider\": \"string\",\n              \"video_url\": \"string\",\n              \"video_title\": \"string\",\n              \"video_description\": \"string\",\n              \"video_metadata\": \"string\"\n            }\n          }\n        }\n      ],\n      \"tier_prices\": [\n        {\n          \"customer_group_id\": 0,\n          \"qty\": 0,\n          \"value\": 0,\n          \"extension_attributes\": {}\n        }\n      ],\n      \"custom_attributes\": [\n        {\n          \"attribute_code\": \"string\",\n          \"value\": \"string\"\n        }\n      ]\n    }\n  ],\n  \"search_criteria\": {\n    \"filter_groups\": [\n      {\n        \"filters\": [\n          {\n            \"field\": \"string\",\n            \"value\": \"string\",\n            \"condition_type\": \"string\"\n          }\n        ]\n      }\n    ],\n    \"sort_orders\": [\n      {\n        \"field\": \"string\",\n        \"direction\": \"string\"\n      }\n    ],\n    \"page_size\": 0,\n    \"current_page\": 0\n  },\n  \"total_count\": 0\n}"
  },
  {
    "path": "doc/source_magento_review.json",
    "content": "{\n  \"id\": 0,\n  \"title\": \"string\",\n  \"detail\": \"string\",\n  \"nickname\": \"string\",\n  \"ratings\": [{\n    \"id\": 0,\n    \"review_id\": 0,\n    \"attribute_code\": \"string\",\n    \"value\": 0\n  }],\n  \"customer_id\": null,\n  \"review_entity\": \"product\",\n  \"review_type\": 0,\n  \"review_status\": 0,\n  \"created_at\": \"string\",\n  \"entity_pk_value\": 0,\n  \"stores\": []\n}"
  },
  {
    "path": "docker-compose.yml",
    "content": "version: '2'\nservices:\n  esm1:\n    image: elasticsearch:7.3.2\n    container_name: esm1\n    environment:\n      - cluster.name=docker-cluster\n      - bootstrap.memory_lock=true\n      - \"ES_JAVA_OPTS=-Xms512m -Xmx512m\"\n    ulimits:\n      memlock:\n        soft: -1\n        hard: -1\n    mem_limit: 1g\n    volumes:\n      - esdat1:/usr/share/elasticsearch/data\n    ports:\n      - 9200:9200\n    networks:\n      - esnet\n  esm2:\n    image: elasticsearch:7.3.2\n    container_name: esm2\n    environment:\n      - cluster.name=docker-cluster\n      - bootstrap.memory_lock=true\n      - \"ES_JAVA_OPTS=-Xms512m -Xmx512m\"\n      - \"discovery.zen.ping.unicast.hosts=esm1\"\n    ulimits:\n      memlock:\n        soft: -1\n        hard: -1\n    mem_limit: 1g\n    volumes:\n      - esdat2:/usr/share/elasticsearch/data\n    networks:\n      - esnet\n  \n  redis:\n    image: redis\n\n    ports:\n      - \"6379:6379\"\n\nvolumes:\n  esdat1:\n    driver: local\n  esdat2:\n    driver: local\n\nnetworks:\n  esnet:\n"
  },
  {
    "path": "package.json",
    "content": "{\n  \"name\": \"mage2vuestorefront\",\n  \"private\": true,\n  \"version\": \"1.11.1\",\n  \"description\": \"Magento sync for products, categories, users and orders\",\n  \"author\": \"Piotr Karwatka\",\n  \"license\": \"MIT\",\n  \"main\": \"src/cli.js\",\n  \"scripts\": {\n    \"test\": \"echo \\\"Error: no test specified\\\" && exit 1\"\n  },\n  \"dependencies\": {\n    \"@elastic/elasticsearch\": \"^7.3.0\",\n    \"agentkeepalive\": \"^3.3.0\",\n    \"body-parser\": \"^1.17.1\",\n    \"commander\": \"^2.18.0\",\n    \"elasticsearch-deletebyquery\": \"^1.6.0\",\n    \"express\": \"^4.15.2\",\n    \"jsonfile\": \"^4.0.0\",\n    \"jwt-simple\": \"^0.5.1\",\n    \"kue\": \"^0.11.5\",\n    \"lodash\": \"^4.17.19\",\n    \"magento2-rest-client\": \"https://github.com/DivanteLtd/magento2-rest-client\",\n    \"moment\": \"^2.22.2\",\n    \"node-sync\": \"^0.2.1\",\n    \"passport\": \"^0.3.2\",\n    \"passport-jwt\": \"^4.0.0\",\n    \"promise-queue\": \"^2.2.3\",\n    \"queue\": \"^4.4.0\",\n    \"redis-jsonify\": \"^1.0.1\",\n    \"remove-accents\": \"^0.4.2\",\n    \"request\": \"^2.88.0\",\n    \"sha1\": \"^1.1.1\"\n  },\n  \"devDependencies\": {}\n}\n"
  },
  {
    "path": "src/adapters/abstract.js",
    "content": "'use strict';\n\nconst AdapterFactory = require('./factory');\nconst Redis = require('redis');\n\nclass AbstractAdapter {\n\n  validateConfig(config) {\n    if (!config['db']['url'])\n      throw Error('db.url must be set up in config');\n  }\n\n  constructor(app_config) {\n    this.config = app_config;\n\n    let factory = new AdapterFactory(app_config);\n    this.db = factory.getAdapter('nosql', app_config.db.driver);\n\n    if (global.cache == null) {\n      this.cache = Redis.createClient(this.config.redis); // redis client\n      this.cache.on('error', (err) => { // workaround for https://github.com/NodeRedis/node_redis/issues/713\n        this.cache = Redis.createClient(this.config.redis); // redis client\n      });\n      // redis auth if provided\n      if (this.config.redis.auth) {\n        this.cache.auth(this.config.redis.auth);\n      }\n      global.cache = this.cache;\n    } else this.cache = global.cache;\n\n    this.update_document = true; // should we update database with new data from API? @see productcategory where this is disabled\n\n    this.total_count = 0;\n    this.page_count = 0;\n    this.page_size = 50;\n    this.page = 1;\n    this.current_context = {};\n\n    this.use_paging = false;\n    this.is_federated = false;\n\n    this.validateConfig(this.config);\n\n    this.tasks_count = 0;\n  }\n\n  isValidFor(entity_type) {\n    throw Error('isValidFor must be implemented in specific class');\n  }\n\n  getCurrentContext() {\n    return this.current_context;\n  }\n\n  /**\n   * Default done callback called after all main items are processed by processItems\n   */\n  defaultDoneCallback() {\n    return;\n  }\n\n  /**\n   * Run products/categories/ ... import\n   * @param {Object} context import context with parameter such \"page\", \"size\" and other search parameters\n   */\n  run(context) {\n    this.current_context = context;\n\n    if (!(this.current_context.transaction_key))\n      this.current_context.transaction_key = new Date().getTime(); // the key used to filter out records NOT ADDED by this import\n\n    this.db.connect(() => {\n      logger.info('Connected correctly to server');\n      logger.info(`TRANSACTION KEY = ${this.current_context.transaction_key}`);\n\n      this.onDone = this.current_context.done_callback ? (\n        () => {\n          this.defaultDoneCallback();\n          this.current_context.done_callback();\n        }\n      ): this.defaultDoneCallback;\n\n      let exitCallback = this.onDone;\n      this.getSourceData(this.current_context)\n        .then(this.processItems.bind(this))\n        .catch((err) => {\n          logger.error(err);\n          exitCallback();\n        });\n    });\n  }\n\n  /**\n   * Implement some item related operations - executed BEFORE saving to the database\n   * @param {Object} item\n   */\n  preProcessItem(item) {\n    return new Promise((done, reject) => { done(); });\n  }\n\n  /**\n   * Remove records from database other than specific transaction_key\n   * @param {int} transaction_key\n   */\n  cleanUp(transaction_key) {\n    this.db.connect(() => {\n      logger.info(`Cleaning up with tsk = ${transaction_key}`);\n      this.db.cleanupByTransactionkey(this.getCollectionName(), transaction_key);\n    });\n  }\n\n  prepareItems(items) {\n    if(!items)\n      return items;\n\n    if (items.total_count)\n      this.total_count = items.total_count;\n\n    if (!Array.isArray(items))\n      items = new Array(items);\n\n    return items;\n  }\n\n  isFederated() {\n    return this.is_federated;\n  }\n\n  processItems(items, level) {\n\n    if (isNaN(level))\n      level = 0;\n    items = this.prepareItems(items);\n\n    if (!items) {\n      logger.error('No items given to processItems call!');\n      return;\n    }\n\n    let count = items.length;\n    let index = 0;\n\n    if (count == 0) {\n      logger.warn('No records to process!');\n      return this.onDone(this);\n    } else\n      this.tasks_count += count;\n\n    let db = this.db;\n    if (!db)\n      throw new Error('No db adapter connection established!');\n\n    if (this.total_count)\n      logger.info(`Total count is: ${this.total_count}`)\n\n    items.map((item) => {\n\n      this.preProcessItem(item).then((item) => {\n\n        this.tasks_count--;\n\n        item.tsk = this.getCurrentContext().transaction_key; // transaction key for items that can be then cleaned up\n\n        logger.info(`Importing ${index} of ${count} - ${this.getLabel(item)} with tsk = ${item.tsk}`);\n        logger.info(`Tasks count = ${this.tasks_count}`);\n\n        if (this.update_document)\n          this.db.updateDocument(this.getCollectionName(), this.normalizeDocumentFormat(item))\n        else\n          logger.debug('Skipping database update');\n\n        if (item.children_data && item.children_data.length > 0) {\n          logger.info(`--L:${level} Processing child items ...`);\n          this.processItems(item.children_data, level + 1);\n        }\n\n        if (this.tasks_count == 0 && !this.use_paging) { // this is the last item!\n          logger.info('No tasks to process. All records processed!');\n          this.db.close();\n\n          return this.onDone(this);\n        } else {\n\n          if (index == (count - 1)) { // page done!\n            logger.debug(`--L:${level} Level done! Current page: ${this.page} of ${this.page_count}`);\n            if (parseInt(level) == 0) {\n\n              if (this.use_paging && !this.isFederated()) { //TODO: paging should be refactored using queueing\n\n                if (this.page >= (this.page_count)) {\n                  logger.info('All pages processed!');\n                  this.db.close();\n\n                  this.onDone(this);\n                } else {\n                  const context = this.getCurrentContext()\n                  if (context.page) {\n                    context.page++\n                    this.page++;\n                  } else {\n                    context.page = ++this.page;\n                  }\n                  logger.debug(`Switching page to ${this.page}`);\n                  let exitCallback = this.onDone;\n                  this.getSourceData(context)\n                    .then(this.processItems.bind(this))\n                    .catch((err) => {\n                      logger.error(err);\n                      exitCallback()\n                    });\n                }\n              }\n            }\n          }\n        }\n\n        index++;\n      }).catch((reason) => {\n        logger.error(reason);\n        return this.onDone(this);\n      });\n    })\n  }\n}\n\nmodule.exports = AbstractAdapter;\n"
  },
  {
    "path": "src/adapters/factory.js",
    "content": "'use strict';\n\nclass AdapterFactory {\n\n  constructor (app_config) {\n    this.config = app_config;\n  }\n\n   getAdapter (adapter_type, driver) {\n\n    let adapter_class = require('./' + adapter_type + '/' + driver);\n\n    if (!adapter_class) {\n      throw new Error(`Invalid adapter ${adapter_type} / ${driver}`);\n    } else {\n      let adapter_instance = new adapter_class(this.config);\n\n      if((typeof adapter_instance.isValidFor == 'function') && !adapter_instance.isValidFor(driver))\n        throw new Error(`Not valid adapter class or adapter is not valid for ${driver}`);\n\n      return adapter_instance;\n    }\n  }\n}\n\nmodule.exports = AdapterFactory;\n"
  },
  {
    "path": "src/adapters/magento/abstract.js",
    "content": "'use strict';\n\nlet AbstractAdapter = require('../abstract');\n\nclass AbstractMagentoAdapter extends AbstractAdapter{\n\n  constructor(config){\n    super(config);\n\n    let Magento2Client = require('./magento2-rest-client').Magento2Client;\n    this.api = Magento2Client(this.config.magento);\n  }\n\n  getEntityType(){\n    throw new Error('getEntityType must be implemented');\n  }\n\n  getCollectionName(){\n    return this.getEntityType();\n  }\n\n  validateConfig(config){\n\n      super.validateConfig(config);\n\n      if(!config['magento']['url'] ||\n         !config['magento']['consumerKey'] ||\n         !config['magento']['consumerSecret'] ||\n         !config['magento']['accessToken'] ||\n         !config['magento']['accessTokenSecret'])\n             throw Error('magento.{url,consumerKey,consumerSecret,accessToken,accessTokenSecret} must be set in config');\n  }\n\n  isValidFor(entity_type){\n    return (entity_type == this.getEntityType());\n  }\n\n  getSourceData(){\n    throw new Error('getSourceData must be implemented');\n  }\n\n  getLabel(source_item){\n    return source_item.id;\n  }\n\n  /**\n   * We're transorming the data structure of item to be compliant with Smile.fr Elastic Search Suite\n   * @param {object} item  document to be updated in elastic search\n   */\n  normalizeDocumentFormat(item) {\n    return item;\n  }\n\n}\n\nmodule.exports = AbstractMagentoAdapter;\n"
  },
  {
    "path": "src/adapters/magento/attribute.js",
    "content": "'use strict';\n\nlet AbstractMagentoAdapter = require('./abstract');\nconst CacheKeys = require('./cache_keys');\nconst util = require('util');\n\nclass AttributeAdapter extends AbstractMagentoAdapter {\n\n  getEntityType() {\n    return 'attribute';\n  }\n\n  getName() {\n    return 'adapters/magento/AttributeAdapter';\n  }\n\n  getSourceData(context) {\n    return this.api.attributes.list();\n  }\n\n  /**  Regarding Magento2 api docs and reality we do have an exception here that items aren't listed straight in the response but under \"items\" key */\n  prepareItems(items) {\n    if(!items)\n      return items;\n \n    if (items.total_count)\n      this.total_count = items.total_count;\n    \n    if(items.items)\n      items = items.items; // this is an exceptional behavior for Magento2 api  for attributes\n\n    return items;\n  }\n\n  getLabel(source_item) {\n    return `[(${source_item.attribute_code}) ${source_item.default_frontend_label}]`;\n  }\n\n  isFederated() {\n    return false;\n  }\n\n  preProcessItem(item) {\n    return new Promise((done, reject) => {\n      if (item) {\n        item.id = item.attribute_id;\n        // store the item into local redis cache\n        let key = util.format(CacheKeys.CACHE_KEY_ATTRIBUTE, item.attribute_code);\n        logger.debug(`Storing attribute data to cache under: ${key}`);\n        this.cache.set(key, JSON.stringify(item));\n\n        key = util.format(CacheKeys.CACHE_KEY_ATTRIBUTE, item.attribute_id); // store under attribute id as an second option\n        logger.debug(`Storing attribute data to cache under: ${key}`);\n        this.cache.set(key, JSON.stringify(item));\n      }\n\n      return done(item);\n    });\n  }\n\n  /**\n   * We're transorming the data structure of item to be compliant with Smile.fr Elastic Search Suite\n   * @param {object} item  document to be updated in elastic search\n   */\n  normalizeDocumentFormat(item) {\n    return item;\n  }\n}\n\nmodule.exports = AttributeAdapter;\n"
  },
  {
    "path": "src/adapters/magento/cache_keys.js",
    "content": "var config = require('../../config')\n\nmodule.exports = {\n  CACHE_KEY_CATEGORY: config.db.indexName + '_cat_%s',\n  CACHE_KEY_PRODUCT: config.db.indexName + '_prd_%s',\n  CACHE_KEY_PRODUCT_CATEGORIES: config.db.indexName + '_prd_cat_%s',\n  CACHE_KEY_PRODUCT_CATEGORIES_TEMPORARY: config.db.indexName + '_prd_cat_ts_%s',\n  CACHE_KEY_ATTRIBUTE: config.db.indexName + '_attr_%s',\n  CACHE_KEY_STOCKITEM: config.db.indexName + '_stock_%s'\n}\n"
  },
  {
    "path": "src/adapters/magento/category.js",
    "content": "'use strict';\n\nlet AbstractMagentoAdapter = require('./abstract');\nconst CacheKeys = require('./cache_keys');\nconst util = require('util');\nconst request = require('request');\nconst _slugify = require('../../helpers/slugify')\n\nconst _normalizeExtendedData = function (result, generateUrlKey = true, config = null) {\n  if (result.custom_attributes) {\n    for (let customAttribute of result.custom_attributes) { // map custom attributes directly to document root scope\n      result[customAttribute.attribute_code] = customAttribute.value;\n    }\n    delete result.custom_attributes;\n  }\n  if (generateUrlKey) {\n    result.url_key = _slugify(result.name) + '-' + result.id;\n  }\n  result.slug = result.url_key\n  if (config.seo.useUrlDispatcher) {\n    result.url_path = config.seo.categoryUrlPathMapper(result)\n  } else {\n    result.url_path = result.url_key;\n  }\n  return result\n}\n\nclass CategoryAdapter extends AbstractMagentoAdapter {\n\n  constructor (config) {\n    super(config);\n    this.extendedCategories = false;\n    this.generateUniqueUrlKeys = true;\n  }\n\n  getEntityType() {\n    return 'category';\n  }\n\n  getName() {\n    return 'adapters/magento/CategoryAdapter';\n  }\n\n  getSourceData(context) {\n    this.generateUniqueUrlKeys = context.generateUniqueUrlKeys;\n    this.extendedCategories = context.extendedCategories;\n    return this.api.categories.list();\n  }\n\n  getLabel(source_item) {\n    return `[(${source_item.id}) ${source_item.name}]`;\n  }\n\n  isFederated() {\n    return false;\n  }\n\n  _addSingleCategoryData(item, result) {\n    item = Object.assign(item, _normalizeExtendedData(result, this.generateUniqueUrlKeys, this.config));\n  }\n\n  _extendSingleCategory(rootId, catToExtend) {\n    const generateUniqueUrlKeys = this.generateUniqueUrlKeys\n    const config = this.config\n    return this.api.categories.getSingle(catToExtend.id).then(function(result) {\n      Object.assign(catToExtend, _normalizeExtendedData(result, generateUniqueUrlKeys, config))\n      logger.info(`Subcategory data extended for ${rootId}, children object ${catToExtend.id}`)\n    }).catch(function(err) {\n      logger.error(err)\n    });\n  }\n\n  _extendChildrenCategories(rootId, children, result, subpromises) {\n    for (const child of children) {\n      if (Array.isArray(child.children_data)) {\n        this._extendChildrenCategories(rootId, child.children_data, result, subpromises);\n        subpromises.push(this._extendSingleCategory(rootId, child));\n      } else {\n        subpromises.push(this._extendSingleCategory(rootId, child));\n      }\n    }\n    return result;\n  };\n\n  preProcessItem(item) {\n    return new Promise((done, reject) => {\n\n      if (!item) {\n        return done(item);\n      }\n\n      if (!item.url_key || this.generateUniqueUrlKeys) {\n        item.url_key = _slugify(item.name) + '-' + item.id\n      }\n      item.slug = item.url_key;\n      if (this.config.seo.useUrlDispatcher) {\n        item.url_path = this.config.seo.categoryUrlPathMapper(item)\n      } else {               \n        item.url_path = item.url_key;\n      }\n\n      if (this.extendedCategories) {\n\n        this.api.categories.getSingle(item.id).then((result) => {\n          this._addSingleCategoryData(item, result);\n\n          const key = util.format(CacheKeys.CACHE_KEY_CATEGORY, item.id);\n          logger.debug(`Storing extended category data to cache under: ${key}`);\n          this.cache.set(key, JSON.stringify(item));\n\n          const subpromises = []\n          if (item.children_data && item.children_data.length) {\n            this._extendChildrenCategories(item.id, item.children_data, result, subpromises)\n\n            Promise.all(subpromises).then(function (results) {\n              done(item)\n            }).catch(function (err) {\n              logger.error(err)\n              done(item)\n            })\n          } else {\n            done(item);\n          }\n        }).catch(function (err) {\n          logger.error(err);\n          done(item);\n        });\n\n      } else {\n        const key = util.format(CacheKeys.CACHE_KEY_CATEGORY, item.id);\n        logger.debug(`Storing category data to cache under: ${key}`);\n        this.cache.set(key, JSON.stringify(item));\n        return done(item);\n      }\n\n    });\n  }\n\n  /**\n   * We're transforming the data structure of item to be compliant with Smile.fr Elastic Search Suite\n   * @param {object} item  document to be updated in elastic search\n   */\n  normalizeDocumentFormat(item) {\n    if (this.config.vuestorefront && this.config.vuestorefront.invalidateCache) {\n      request(this.config.vuestorefront.invalidateCacheUrl + 'C' + item.id, {}, (err, res, body) => {\n        if (err) { return console.error(err); }\n        console.log(body);\n      });\n    }\n    return item;\n  }\n}\n\nmodule.exports = CategoryAdapter;\n"
  },
  {
    "path": "src/adapters/magento/cms_block.js",
    "content": "'use strict';\n\nlet AbstractMagentoAdapter = require('./abstract');\n\nclass BlockAdapter extends AbstractMagentoAdapter {\n    constructor(config) {\n        super(config);\n        this.use_paging = false;\n    }\n\n    getEntityType() {\n        return 'cms_block';\n    }\n\n    getName() {\n        return 'adapters/magento/BlockAdapter';\n    }\n\n    getSourceData(context) {\n        if (this.use_paging) {\n            return this.api.blocks.list('&searchCriteria[currentPage]=' + this.page + '&searchCriteria[pageSize]=' + this.page_size + (query ? '&' + query : '')).catch((err) => {\n                throw new Error(err);\n            });\n        }\n\n        return this.api.blocks.list().catch((err) => {\n            throw new Error(err);\n        });\n    }\n\n    prepareItems(items) {\n        if(!items) {\n            return items;\n        }\n\n        if (items.total_count) {\n            this.total_count = items.total_count;\n        }\n\n        if (items.items) {\n            items = items.items; // this is an exceptional behavior for Magento2 api for lists\n        }\n        \n        return items;\n    }\n\n    isFederated() {\n        return false;\n    }\n    \n    preProcessItem(item) {\n        //\n        return new Promise((done, reject) => {\n            if (item) {\n                item.type = 'cms_block'\n            }\n          \n          return done(item);\n        });\n    }\n    /**\n     * We're transorming the data structure of item to be compliant with Smile.fr Elastic Search Suite\n     * @param {object} item  document to be updated in elastic search\n     */\n    normalizeDocumentFormat(item) {\n        return item;\n    }\n}\n\nmodule.exports = BlockAdapter;\n"
  },
  {
    "path": "src/adapters/magento/cms_page.js",
    "content": "'use strict';\n\nlet AbstractMagentoAdapter = require('./abstract');\n\nclass PageAdapter extends AbstractMagentoAdapter {\n    constructor(config) {\n        super(config);\n        this.use_paging = false;\n    }\n\n    getEntityType() {\n        return 'cms_page';\n    }\n\n    getName() {\n        return 'adapters/magento/PageAdapter';\n    }\n\n    getSourceData(context) {\n        if (this.use_paging) {\n            return this.api.pages.list('&searchCriteria[currentPage]=' + this.page + '&searchCriteria[pageSize]=' + this.page_size + (query ? '&' + query : '')).catch((err) => {\n                throw new Error(err);\n            });\n        }\n\n        return this.api.pages.list().catch((err) => {\n            throw new Error(err);\n        });\n    }\n\n    prepareItems(items) {\n        if(!items)\n          return items;\n\n        if (items.total_count)\n          this.total_count = items.total_count;\n\n        if (items.items) {\n          items = items.items; // this is an exceptional behavior for Magento2 api for lists\n        }\n\n        return items;\n    }\n\n    preProcessItem(item) {\n\n        return new Promise((done, reject) => {\n            if (item) {\n                item.type = 'cms_page'\n            }\n          \n          return done(item);\n        });\n\n    }\n\n    normalizeDocumentFormat(item) {\n        return item;\n    }\n}\n\nmodule.exports = PageAdapter;\n"
  },
  {
    "path": "src/adapters/magento/magento2-rest-client/.npmignore",
    "content": "# Logs\nlogs\n*.log\nnpm-debug.log*\n\n# Runtime data\npids\n*.pid\n*.seed\n\n# Directory for instrumented libs generated by jscoverage/JSCover\nlib-cov\n\n# Coverage directory used by tools like istanbul\ncoverage\n\n# Grunt intermediate storage (http://gruntjs.com/creating-plugins#storing-task-files)\n.grunt\n\n# node-waf configuration\n.lock-wscript\n\n# Compiled binary addons (http://nodejs.org/api/addons.html)\nbuild/Release\n\n# Dependency directory\nnode_modules\n\n# Optional npm cache directory\n.npm\n\n# Optional REPL history\n.node_repl_history\n"
  },
  {
    "path": "src/adapters/magento/magento2-rest-client/LICENSE",
    "content": "The MIT License (MIT)\n\nCopyright (c) 2016 Marko Novak\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n"
  },
  {
    "path": "src/adapters/magento/magento2-rest-client/README.md",
    "content": "# Magento2 REST client\n\nThis Node.js library enables JavaScript applications to communicate with Magento2 sites using their REST API.\n\n**NOTE: the library is not finished yet! Only a subset of Magento2 API is currently implemented.**\n\n## Installation\n\nThe library can be installed using the Npm package manager:\n\n```\n    npm install magento2-rest-client\n```\n\n## Usage\n\nThe code sample below shows the usage of the library:\n\n```javascript\n    var Magento2Client = require('magento2-rest-client').Magento2Client;\n\n    var options = {\n          'url': 'http://www.test.com/index.php/rest',\n          'consumerKey': '<OAuth 1.0a consumer key>',\n          'consumerSecret': '<OAuth 1.0a consumer secret>',\n          'accessToken': '<OAuth 1.0a access token>',\n          'accessTokenSecret': '<OAuth 1.0a access token secret>'\n    };\n    var client = Magento2Client(options);\n    client.categories.list()\n        .then(function (categories) {\n            assert.equal(categories.parentId, 1);\n        })\n```"
  },
  {
    "path": "src/adapters/magento/magento2-rest-client/index.js",
    "content": "'use strict';\n\nvar RestClient = require('./lib/rest_client').RestClient;\nvar categories = require('./lib/categories');\nvar attributes = require('./lib/attributes');\nvar products = require('./lib/products');\nvar productMedia = require('./lib/product_media');\nvar categoryProducts = require('./lib/category_products');\nvar configurableChildren = require('./lib/configurable_children');\nvar configurableOptions = require('./lib/configurable_options');\nvar customOptions = require('./lib/custom_options');\nvar bundleOptions = require('./lib/bundle_options');\nvar taxRates = require('./lib/tax_rates');\nvar taxRules = require('./lib/tax_rules');\nvar stockItems = require('./lib/stock_items');\nvar productLinks = require('./lib/product_links');\nvar reviews = require('./lib/reviews');\nvar blocks = require('./lib/blocks');\nvar pages = require('./lib/pages');\n\nconst MAGENTO_API_VERSION = 'V1';\n\nmodule.exports.Magento2Client = function (options) {\n    var instance = {};\n\n    options.version = MAGENTO_API_VERSION;\n\n    var client = RestClient(options);\n\n    instance.attributes = attributes(client);\n    instance.categories = categories(client);\n    instance.products = products(client);\n    instance.productMedia = productMedia(client);\n    instance.categoryProducts = categoryProducts(client);\n    instance.configurableChildren = configurableChildren(client);\n    instance.configurableOptions = configurableOptions(client);\n    instance.stockItems = stockItems(client);\n    instance.taxRates = taxRates(client);\n    instance.taxRules = taxRules(client);\n    instance.customOptions = customOptions(client);\n    instance.bundleOptions = bundleOptions(client);\n    instance.productLinks = productLinks(client);\n    instance.reviews = reviews(client);\n    instance.blocks = blocks(client);\n    instance.pages = pages(client);\n\n    return instance;\n}\n"
  },
  {
    "path": "src/adapters/magento/magento2-rest-client/lib/attributes.js",
    "content": "var util = require('util');\n\nmodule.exports = function (restClient) {\n    var module = {};\n\n    module.list = function (searchCriteria) {\n        var query = 'searchCriteria=' + searchCriteria;\n        var endpointUrl = util.format('/products/attributes?%s', query);\n        return restClient.get(endpointUrl);\n    }\n    \n    module.create = function (categoryAttributes) {\n        return restClient.post('/products/attributes', categoryAttributes);\n    }\n\n    module.update = function (attributeId, categoryAttributes) {\n        var endpointUrl = util.format('/products/attributes/%d', attributeId);\n        return restClient.put(endpointUrl, categoryAttributes);\n    }\n\n    module.delete = function (attributeId) {\n        var endpointUrl = util.format('/products/attributes/%d', attributeId);\n        return restClient.delete(endpointUrl);\n    }\n\n    return module;\n}\n"
  },
  {
    "path": "src/adapters/magento/magento2-rest-client/lib/blocks.js",
    "content": "var util = require('util');\n\nmodule.exports = function (restClient) {\n    var module = {};\n\n    module.list = function(searchCriteria) {\n        var query = 'searchCriteria=' + searchCriteria;\n        var endpointUrl = util.format('/snowdog/cmsBlock/search?%s', query);\n        return restClient.get(endpointUrl);\n    }\n\n    return module;\n}\n"
  },
  {
    "path": "src/adapters/magento/magento2-rest-client/lib/bundle_options.js",
    "content": "var util = require('util');\n\nmodule.exports = function (restClient) {\n    var module = {};\n\n    module.list = function (sku) {\n        var endpointUrl = util.format('/bundle-products/%s/options/all', encodeURIComponent(sku));\n        return restClient.get(endpointUrl);\n    }\n\n\n    return module;\n}\n"
  },
  {
    "path": "src/adapters/magento/magento2-rest-client/lib/categories.js",
    "content": "var util = require('util');\n\nmodule.exports = function (restClient) {\n    var module = {};\n\n    module.list = function () {\n        return restClient.get('/categories');\n    }\n    \n    module.getSingle = function (categoryId) {\n        var endpointUrl = util.format('/categories/%d', categoryId);\n        return restClient.get(endpointUrl);\n    }\n    \n    module.create = function (categoryAttributes) {\n        return restClient.post('/categories', categoryAttributes);\n    }\n\n    module.update = function (categoryId, categoryAttributes) {\n        var endpointUrl = util.format('/categories/%d', categoryId);\n        return restClient.put(endpointUrl, categoryAttributes);\n    }\n\n    module.delete = function (categoryId) {\n        var endpointUrl = util.format('/categories/%d', categoryId);\n        return restClient.delete(endpointUrl);\n    }\n\n    return module;\n}\n"
  },
  {
    "path": "src/adapters/magento/magento2-rest-client/lib/category_products.js",
    "content": "var util = require('util');\n\nmodule.exports = function (restClient) {\n    var module = {};\n\n    module.list = function (categoryId) {\n        var endpointUrl = util.format('/categories/%d/products', categoryId);\n        return restClient.get(endpointUrl);\n    }\n\n\n    return module;\n}\n"
  },
  {
    "path": "src/adapters/magento/magento2-rest-client/lib/configurable_children.js",
    "content": "var util = require('util');\n\nmodule.exports = function (restClient) {\n    var module = {};\n\n    module.list = function (sku) {\n        var endpointUrl = util.format('/configurable-products/%s/children', encodeURIComponent(sku));\n        return restClient.get(endpointUrl);\n    }\n\n\n    return module;\n}\n"
  },
  {
    "path": "src/adapters/magento/magento2-rest-client/lib/configurable_options.js",
    "content": "var util = require('util');\n\nmodule.exports = function (restClient) {\n    var module = {};\n\n    module.list = function (sku) {\n        var endpointUrl = util.format('/configurable-products/%s/options/all', encodeURIComponent(sku));\n        return restClient.get(endpointUrl);\n    }\n\n\n    return module;\n}\n"
  },
  {
    "path": "src/adapters/magento/magento2-rest-client/lib/custom_options.js",
    "content": "var util = require('util');\n\nmodule.exports = function (restClient) {\n    var module = {};\n\n    module.list = function (sku) {\n        var endpointUrl = util.format('/products/%s/options', encodeURIComponent(sku));\n        return restClient.get(endpointUrl);\n    }\n\n\n    return module;\n}\n"
  },
  {
    "path": "src/adapters/magento/magento2-rest-client/lib/log.js",
    "content": "var winston = require('winston');\n\nwinston.emitErrs = true;\n\nvar logger = new winston.Logger({\n    transports: [\n        new winston.transports.Console({\n            level: 'debug',\n            handleExceptions: true,\n            json: false,\n            colorize: true\n        })\n    ],\n    exitOnError: false\n});\n\nlogger.info('Winston logging library initialized.');\n\nmodule.exports = logger;\n"
  },
  {
    "path": "src/adapters/magento/magento2-rest-client/lib/pages.js",
    "content": "var util = require('util');\n\nmodule.exports = function (restClient) {\n    var module = {};\n\n    module.list = function (searchCriteria) {\n        var query = 'searchCriteria=' + searchCriteria;\n        var endpointUrl = util.format('/snowdog/cmsPage/search?%s', query);\n        return restClient.get(endpointUrl);\n    }\n\n    return module;\n}\n"
  },
  {
    "path": "src/adapters/magento/magento2-rest-client/lib/product_links.js",
    "content": "var util = require('util');\n\nmodule.exports = function (restClient) {\n    var module = {};\n    var typesCache = null;\n\n    module.list = function (sku, type) {\n        var endpointUrl = util.format('/products/%s/links/%s', encodeURIComponent(sku), type);\n        return restClient.get(endpointUrl);\n    }\n\n    module.types = function () {\n        var endpointUrl = util.format('/products/links/types');\n        if (typesCache !== null) {\n            return new Promise((resolve, reject) => {\n                resolve (typesCache)\n            })\n        } else {\n            return restClient.get(endpointUrl).then((result) => {\n                typesCache = result\n            })\n        }\n    }\n    return module;\n}\n"
  },
  {
    "path": "src/adapters/magento/magento2-rest-client/lib/product_media.js",
    "content": "var util = require('util');\n\nmodule.exports = function (restClient) {\n    var module = {};\n\n    module.list = function (productSku) {\n        var endpointUrl = util.format('/products/%s/media', encodeURIComponent(productSku));\n        return restClient.get(endpointUrl);\n    }\n\n    module.get = function (productSku, mediaId) {\n        var endpointUrl = util.format('/products/%s/media/%d', encodeURIComponent(productSku), mediaId);\n        return restClient.get(endpointUrl);\n    }\n\n    module.create = function (productSku, productMediaAttributes) {\n        var endpointUrl = util.format('/products/%s/media', encodeURIComponent(productSku));\n        return restClient.post(endpointUrl, productMediaAttributes);\n    }\n\n    module.update = function (productSku, mediaId, productMediaAttributes) {\n        var endpointUrl = util.format('/products/%s/media/%d', encodeURIComponent(productSku), mediaId);\n        return restClient.put(endpointUrl, productMediaAttributes);\n    }\n\n    module.delete = function (productSku, mediaId) {\n        var endpointUrl = util.format('/products/%s/media/%d', encodeURIComponent(productSku), mediaId);\n        return restClient.delete(endpointUrl);\n    }\n\n    return module;\n}\n"
  },
  {
    "path": "src/adapters/magento/magento2-rest-client/lib/products.js",
    "content": "var util = require('util');\n\nmodule.exports = function (restClient) {\n    var module = {};\n\n    module.list = function (searchCriteria) {\n        var query = 'searchCriteria=' + searchCriteria;\n        var endpointUrl = util.format('/products?%s', query);\n        return restClient.get(endpointUrl);\n    }\n    \n    module.renderList = function (searchCriteria, storeId = 1, currencyCode = 'USD') {\n        var query = 'searchCriteria=' + searchCriteria;\n        var endpointUrl = util.format('/products-render-info?%s&storeId=%d&currencyCode=' + encodeURIComponent(currencyCode), query, storeId);\n        return restClient.get(endpointUrl);\n    }\n\n    module.create = function (productAttributes) {\n        return restClient.post('/products', productAttributes);\n    }\n\n    module.update = function (productSku, productAttributes) {\n        var endpointUrl = util.format('/products/%s', encodeURIComponent(productSku));\n        return restClient.put(endpointUrl, productAttributes);\n    }\n\n    module.delete = function (productSku) {\n        var endpointUrl = util.format('/products/%s', encodeURIComponent(productSku));\n        return restClient.delete(endpointUrl);\n    }\n\n    return module;\n}\n\n"
  },
  {
    "path": "src/adapters/magento/magento2-rest-client/lib/rest_client.js",
    "content": "'use strict';\n\nvar OAuth = require('oauth-1.0a');\nvar request = require('request');\nvar humps = require('humps');\nvar sprintf = require('util').format;\n\nvar logger = require('./log');\n\nmodule.exports.RestClient = function (options) {\n    var instance = {};\n\n    var servelrUrl = options.url;\n    var apiVersion = options.version;\n    var oauth = OAuth({\n        consumer: {\n            public: options.consumerKey,\n            secret: options.consumerSecret\n        },\n        signature_method: 'HMAC-SHA1'\n    });\n    var token = {\n        public: options.accessToken,\n        secret: options.accessTokenSecret\n    };\n\n    function apiCall(request_data) {\n        logger.debug('Calling API endpoint: ' + request_data.method + ' ' + request_data.url);\n        return new Promise(function (resolve, reject) {\n            request({\n                url: request_data.url,\n                method: request_data.method,\n                headers: oauth.toHeader(oauth.authorize(request_data, token)),\n                json: true,\n                body: request_data.body,\n            }, function (error, response, body) {\n                logger.debug('Response received.');\n                if (error) {\n                    logger.error('Error occured: ' + error);\n                    reject(error);\n                    return;\n                } else if (!httpCallSucceeded(response)) {\n                    var errorMessage = 'HTTP ERROR ' + response.code;\n                    if(body && body.hasOwnProperty('message') )\n                        errorMessage = errorString(body.message, body.hasOwnProperty('parameters') ? body.parameters : {});\n                    \n                    logger.error('API call failed: ' + errorMessage);\n                    reject(errorMessage);\n                }\n//                var bodyCamelized = humps.camelizeKeys(body);\n//                resolve(bodyCamelized);\n                resolve(body);\n            });\n        });\n    }\n\n    function httpCallSucceeded(response) {\n        return response.statusCode >= 200 && response.statusCode < 300;\n    }\n\n    function errorString(message, parameters) {\n        if (parameters === null) {\n            return message;\n        }\n        if (parameters instanceof Array) {\n            for (var i = 0; i < parameters.length; i++) {\n                var parameterPlaceholder = '%' + (i + 1).toString();\n                message = message.replace(parameterPlaceholder, parameters[i]);\n            }\n        } else if (parameters instanceof Object) {\n            for (var key in parameters) {\n                var parameterPlaceholder = '%' + key;\n                message = message.replace(parameterPlaceholder, parameters[key]);\n            }\n        }\n\n        return message;\n    }\n\n    instance.get = function (resourceUrl) {\n        var request_data = {\n            url: createUrl(resourceUrl),\n            method: 'GET'\n        };\n        return apiCall(request_data);\n    }\n\n    function createUrl(resourceUrl) {\n        return servelrUrl + '/' + apiVersion + resourceUrl;\n    }\n\n    instance.post = function (resourceUrl, data) {\n        var request_data = {\n            url: createUrl(resourceUrl),\n            method: 'POST',\n            body: data\n        };\n        return apiCall(request_data);\n    }\n\n    instance.put = function (resourceUrl, data) {\n        var request_data = {\n            url: createUrl(resourceUrl),\n            method: 'PUT',\n            body: data\n        };\n        return apiCall(request_data);\n    }\n\n    instance.delete = function (resourceUrl) {\n        var request_data = {\n            url: createUrl(resourceUrl),\n            method: 'DELETE'\n        };\n        return apiCall(request_data);\n    }\n\n    return instance;\n}\n"
  },
  {
    "path": "src/adapters/magento/magento2-rest-client/lib/reviews.js",
    "content": "const util = require('util');\n\nmodule.exports = function (restClient) {\n    var module = {};\n\n    module.getByProductSku = function (sku) {\n        const endpointUrl = util.format('/products/%s/review', encodeURIComponent(sku));\n        return restClient.get(endpointUrl);\n    };\n\n    module.list = function(searchCriteria) {\n        const query = 'searchCriteria=' + searchCriteria;\n        const endpointUrl = util.format('/reviews/?%s', query);\n        return restClient.get(endpointUrl);\n    };\n\n    module.create = function (reviewData) {\n        return restClient.post('/reviews', {review: reviewData})\n    }\n\n    module.delete = function (reviewId) {\n        var endpointUrl = util.format('/reviews/%d', reviewId);\n        return restClient.delete(endpointUrl);\n    }\n\n    return module;\n};\n"
  },
  {
    "path": "src/adapters/magento/magento2-rest-client/lib/stock_items.js",
    "content": "var util = require('util');\n\nmodule.exports = function (restClient) {\n    var module = {};\n\n    module.list = function (sku) {\n        var endpointUrl = util.format('/stockItems/%s', encodeURIComponent(sku));\n        return restClient.get(endpointUrl);\n    }\n\n    // MSI\n    module.getSalableQty = function (sku, stockId) {\n        var endpointUrl = util.format(\n            '/inventory/get-product-salable-quantity/%s/%d',\n            encodeURIComponent(sku),\n            encodeURIComponent(stockId)\n        );\n        return restClient.get(endpointUrl);\n    }\n\n    // MSI\n    module.isSalable = function (sku, stockId) {\n        var endpointUrl = util.format(\n            '/inventory/is-product-salable/%s/%d',\n            encodeURIComponent(sku),\n            encodeURIComponent(stockId)\n        );\n        return restClient.get(endpointUrl);\n    }\n\n    return module;\n}\n"
  },
  {
    "path": "src/adapters/magento/magento2-rest-client/lib/tax_rates.js",
    "content": "var util = require('util');\n\nmodule.exports = function (restClient) {\n    var module = {};\n\n    module.list = function (rateId) {\n        var endpointUrl = util.format('/taxRates/%d', rateId);\n        return restClient.get(endpointUrl);\n    }\n    \n    module.create = function (rateAttributes) {\n        return restClient.post('/taxRates', rateAttributes);\n    }\n\n    module.update = function (rateId, rateAttributes) {\n        var endpointUrl = util.format('/taxRates/%d', rateId);\n        return restClient.put(endpointUrl, rateAttributes);\n    }\n\n    module.delete = function (rateId) {\n        var endpointUrl = util.format('/taxRates/%d', rateId);\n        return restClient.delete(endpointUrl);\n    }\n\n    return module;\n}\n"
  },
  {
    "path": "src/adapters/magento/magento2-rest-client/lib/tax_rules.js",
    "content": "var util = require('util');\n\nmodule.exports = function (restClient) {\n    var module = {};\n\n    module.list = function (searchCriteria) {\n        var query = 'searchCriteria=' + searchCriteria;\n        var endpointUrl = util.format('/taxRules/search?%s', query);\n        return restClient.get(endpointUrl);\n    }\n    \n    module.create = function (ruleAttributes) {\n        return restClient.post('/taxRules', ruleAttributes);\n    }\n\n    module.update = function (ruleId, ruleAttributes) {\n        var endpointUrl = util.format('/taxRules/%d', ruleId);\n        return restClient.put(endpointUrl, ruleAttributes);\n    }\n\n    module.delete = function (ruleId) {\n        var endpointUrl = util.format('/taxRules/%d', ruleId);\n        return restClient.delete(endpointUrl);\n    }\n\n    return module;\n}\n"
  },
  {
    "path": "src/adapters/magento/magento2-rest-client/magento2-rest-client.iml",
    "content": "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<module type=\"WEB_MODULE\" version=\"4\">\n  <component name=\"NewModuleRootManager\" inherit-compiler-output=\"true\">\n    <exclude-output />\n    <content url=\"file://$MODULE_DIR$\" />\n    <orderEntry type=\"inheritedJdk\" />\n    <orderEntry type=\"sourceFolder\" forTests=\"false\" />\n    <orderEntry type=\"library\" name=\"magento2-rest-client node_modules\" level=\"project\" />\n  </component>\n</module>"
  },
  {
    "path": "src/adapters/magento/magento2-rest-client/package.json",
    "content": "{\n  \"_from\": \"magento2-rest-client@0.0.2\",\n  \"_id\": \"magento2-rest-client@0.0.2\",\n  \"_inBundle\": false,\n  \"_integrity\": \"sha1-Km7yMMiBKahoC2Pq03k+p+mb9qQ=\",\n  \"_location\": \"/magento2-rest-client\",\n  \"_phantomChildren\": {},\n  \"_requested\": {\n    \"type\": \"version\",\n    \"registry\": true,\n    \"raw\": \"magento2-rest-client@0.0.2\",\n    \"name\": \"magento2-rest-client\",\n    \"escapedName\": \"magento2-rest-client\",\n    \"rawSpec\": \"0.0.2\",\n    \"saveSpec\": null,\n    \"fetchSpec\": \"0.0.2\"\n  },\n  \"_requiredBy\": [\n    \"/\"\n  ],\n  \"_resolved\": \"https://registry.npmjs.org/magento2-rest-client/-/magento2-rest-client-0.0.2.tgz\",\n  \"_shasum\": \"2a6ef230c88129a8680b63ead3793ea7e99bf6a4\",\n  \"_spec\": \"magento2-rest-client@0.0.2\",\n  \"_where\": \"/Users/pkarwatka/Documents/_PROJEKTY/mage2nosql/src\",\n  \"author\": {\n    \"name\": \"Marko Novak\",\n    \"email\": \"nouvak@gmail.com\"\n  },\n  \"bugs\": {\n    \"url\": \"https://github.com/nouvak/magento2-rest-client/issues\"\n  },\n  \"bundleDependencies\": false,\n  \"dependencies\": {\n    \"humps\": \"^1.1.0\",\n    \"oauth-1.0a\": \"^1.0.1\",\n    \"request\": \"^2.72.0\",\n    \"winston\": \"^2.2.0\"\n  },\n  \"deprecated\": false,\n  \"description\": \"REST client for accessing Magento 2 functionality.\",\n  \"devDependencies\": {\n    \"chai\": \"^3.5.0\",\n    \"mocha\": \"^2.4.5\"\n  },\n  \"directories\": {\n    \"lib\": \"./lib\"\n  },\n  \"homepage\": \"https://github.com/nouvak/magento2-rest-client#readme\",\n  \"keywords\": [\n    \"magento2\",\n    \"REST\",\n    \"API\"\n  ],\n  \"license\": \"MIT\",\n  \"main\": \"index.js\",\n  \"name\": \"magento2-rest-client\",\n  \"repository\": {\n    \"type\": \"git\",\n    \"url\": \"git+https://github.com/nouvak/magento2-rest-client.git\"\n  },\n  \"scripts\": {\n    \"test\": \"test\"\n  },\n  \"version\": \"0.0.2\"\n}\n"
  },
  {
    "path": "src/adapters/magento/magento2-rest-client/test/config.json",
    "content": "{\n  \"url\": \"http://www.igracke.si/index.php/rest\",\n  \"consumerKey\": \"h0vun4y13ov0pg1drcndfhnim761omcu\",\n  \"consumerSecret\": \"prdemsvf7usypg1f6q24cpis2k1pdtkw\",\n  \"accessToken\": \"kdrmfbudawyquoelylm6syjtb94hmalw\",\n  \"accessTokenSecret\": \"gqjls6qef730m3mniid443m799cu1i8d\"\n}"
  },
  {
    "path": "src/adapters/magento/magento2-rest-client/test/integration/categories.integration.test.js",
    "content": "var chai = require('chai');\nvar credentials = require('../config');\nvar assert = chai.assert;\n\nvar Magento2Client = require('../../index').Magento2Client;\n\nsuite('categories tests', function () {\n    var client;\n\n    before(function() {\n        client = Magento2Client(credentials);\n    });\n\n    test('list categories test', function (done) {\n        client.categories.list()\n            .then(function (categories) {\n                assert.equal(categories.parentId, 1);\n            })\n            .then(done, done);\n    });\n\n    test('create category test', function (done) {\n        var newCategory = {\n            category: {\n                parentId: 3,\n                name: 'Category from integration test',\n                isActive: true,\n                includeInMenu: true,\n            }\n        };\n        client.categories.create(newCategory)\n            .then(function (result) {\n                assert.equal(result.parentId, 3);\n            })\n            .then(done, done);\n    });\n\n    test('update category test', function (done) {\n        var categoryUpdate = {\n            category: {\n                parentId: 3,\n                name: 'Podkategorija 1 updated',\n                isActive: true,\n                includeInMenu: true,\n            }\n        };\n        client.categories.update(4, categoryUpdate)\n            .then(function (result) {\n                assert.equal(result.parentId, 3);\n            })\n            .then(done, done);\n    });\n    \n    test('delete category test', function (done) {\n        client.categories.delete(23)\n            .then(function (result) {\n                assert.isTrue(result);\n            })\n            .then(done, done);\n    })\n});\n"
  },
  {
    "path": "src/adapters/magento/magento2-rest-client/test/integration/product_media.integration.test.js",
    "content": "var chai = require('chai');\nvar credentials = require('../config');\nvar assert = chai.assert;\n\nvar Magento2Client = require('../../index').Magento2Client;\n\nsuite('products media tests', function () {\n    var client;\n\n    before(function () {\n        client = Magento2Client(credentials);\n    });\n\n    test('list product media test', function (done) {\n        client.productMedia.list('test123')\n            .then(function (productMedia) {\n                assert.isTrue(productMedia.length > 0);\n            })\n            .then(done, done);\n    });\n\n    test('get product media test', function (done) {\n        client.productMedia.get('test123', 15)\n            .then(function (productMedia) {\n                assert.isNotNull(productMedia);\n            })\n            .then(done, done);\n    });\n\n    test('create product media test', function (done) {\n        var newProductMedia = {\n            'entry': {\n                'media_type': 'image',\n                'label': 'Image',\n                'position': 1,\n                'disabled': false,\n                'types': [\n                    'image',\n                    'small_image',\n                    'thumbnail'\n                ],\n                'file': '/m/b/mb01-blue-0.png',\n                'content': {\n                    'base64EncodedData': 'iVBORw0KGgoAAAANSUhEUgAAACgAAAAoCAYAAACM/rhtAAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAAWtJREFUeNpi/P//P8NgBkwMgxyMOnDUgTDAyMhIDNYF4vNA/B+IDwCxHLoakgEoFxODiQRXQUYi4e3k2gfDjMRajsP3zED8F8pmA+JvUDEYeArEMugOpFcanA/Ef6A0CPwC4uNoag5SnAjJjGI2tKhkg4rLAfFGIH4IxEuBWIjSKKYkDfZCHddLiwChVhokK8YGohwEZYy3aBmEKmDEhOCgreomo+VmZHxsMEQxIc2MAx3FO/DI3RxMmQTZkI9ALDCaSUYdOOrAIeRAPzQ+PxCHUM2FFDb5paGNBPRa5C20bUhxc4sSB4JaLnvxVHWHsbVu6OnACjyOg+HqgXKgGRD/JMKBoD6LDb0dyAPE94hwHAw/hGYcujlwEQmOg+EV9HJgLBmOg+FMWjsQVKR8psCBoDSrQqoDSSmoG6Hpj1wA6ju30LI9+BBX4UsC+Ai0T4BWVd1EIL5PgeO+APECmoXgaGtm1IE0AgABBgAJAICuV8dAUAAAAABJRU5ErkJggg==',\n                    'type': 'image/png',\n                    'name': 'new_image.png'\n                }\n            }\n        };\n        client.productMedia.create('test123', newProductMedia)\n            .then(function (result) {\n                assert.isNotNull(result);\n            })\n            .then(done, done);\n    });\n\n    test('update product test', function (done) {\n        var productMediaUpdate = {\n                'entry': {\n                    'id': 15,\n                    'label': 'Image updated',\n                }\n            };\n        client.productMedia.update('test123', 15, productMediaUpdate)\n            .then(function (result) {\n                assert.isNotNull(result);\n            })\n            .then(done, done);\n    });\n\n    test('delete product test', function (done) {\n        client.productMedia.delete('test123', 10)\n            .then(function (result) {\n                assert.isTrue(result);\n            })\n            .then(done, done);\n    })\n});\n\n"
  },
  {
    "path": "src/adapters/magento/magento2-rest-client/test/integration/products.integration.test.js",
    "content": "var chai = require('chai');\nvar credentials = require('../config');\nvar assert = chai.assert;\n\nvar Magento2Client = require('../../index').Magento2Client;\n\nsuite('products tests', function () {\n    var client;\n\n    before(function() {\n        client = Magento2Client(credentials);\n    });\n\n    test('list products test', function (done) {\n        client.products.list('Test')\n            .then(function (products) {\n                assert.isTrue(products.totalCount > 0);\n            })\n            .then(done, done);\n    });\n\n    test('create product test', function (done) {\n        var newProduct = {\n            product: {\n                'sku': 'test123',\n                'name': 'Integration test product',\n                'typeId': 'simple',\n                'price': 12.3,\n                'attributeSetId': 4,\n                'status': 1,\n                'visibility': 4,\n            }\n        };\n        client.products.create(newProduct)\n            .then(function (result) {\n                assert.equal(result.name, 'Integration test product');\n            })\n            .then(done, done);\n    });\n\n    test('update product test', function (done) {\n        var productUpdate = {\n            product: {\n                'sku': 'test123',\n                'name': 'Integration test product updated',\n                'typeId': 'simple',\n                'price': 12.3,\n                'attributeSetId': 4,\n                'status': 1,\n                'visibility': 4,\n            }\n        };\n        client.products.update('test123', productUpdate)\n            .then(function (result) {\n                assert.equal(result.name, 'Integration test product updated');\n            })\n            .then(done, done);\n    });\n\n    test('delete product test', function (done) {\n        client.products.delete(23)\n            .then(function (result) {\n                assert.isTrue(result);\n            })\n            .then(done, done);\n    })\n});\n\n"
  },
  {
    "path": "src/adapters/magento/product.js",
    "content": "'use strict';\n\nlet AbstractMagentoAdapter = require('./abstract');\nconst util = require('util');\nconst CacheKeys = require('./cache_keys');\nconst moment = require('moment')\nconst _ = require('lodash')\nconst request = require('request');\nconst HTTP_RETRIES = 3\nlet kue = require('kue');\nconst _slugify = require('../../helpers/slugify')\n\n/*\n * serial executes Promises sequentially.\n * @param {funcs} An array of funcs that return promises.\n * @example\n * const urls = ['/url1', '/url2', '/url3']\n * serial(urls.map(url => () => $.ajax(url)))\n *     .then(console.log(console))\n */\nconst serial = funcs =>\nfuncs.reduce((promise, func) =>\n    promise.then(result => func().then(Array.prototype.concat.bind(result))), Promise.resolve([]))\n\n const optionLabel = (attr, optionId) => {\n  if (attr) {\n    let opt = attr.options.find((op) => { // TODO: cache it in memory\n      if (_.toString(op.value) === _.toString(optionId)) {\n        return op\n      }\n    }) // TODO: i18n support with multi website attribute names\n    return opt ? opt.label : optionId\n  } else {\n    return optionId\n  }\n}\n\nclass ProductAdapter extends AbstractMagentoAdapter {\n\n  constructor(config) {\n    super(config);\n    this.use_paging = true;\n    this.stock_sync = true;\n    this.custom_sync = true;\n    this.parent_sync = true;\n    this.media_sync = true;\n    this.category_sync = true;\n    this.links_sync = true;\n    this.configurable_sync = true;\n    this.is_federated = true; // by default use federated behaviour\n\n  }\n\n  getEntityType() {\n    return 'product';\n  }\n\n  getName() {\n    return 'adapters/magento/ProductAdapter';\n  }\n\n  prepareItems(items) {\n    if(!items)\n      return null;\n\n    this.total_count = items.total_count;\n\n    if (this.use_paging) {\n      this.page_count = Math.ceil(this.total_count / this.page_size);\n      logger.info('Page count', this.page_count)\n    }\n\n    return items.items;\n  }\n\n  getFilterQuery(context) {\n    let query = '';\n\n    if (context.skus) { // pul individual products\n      if (!Array.isArray(context.skus))\n        context.skus = new Array(context.skus);\n\n      query += 'searchCriteria[filter_groups][0][filters][0][field]=sku&' +\n        'searchCriteria[filter_groups][0][filters][0][value]=' + encodeURIComponent(context.skus.join(',')) + '&' +\n        'searchCriteria[filter_groups][0][filters][0][condition_type]=in';\n\n    } else if (context.updated_after && typeof context.updated_after == 'object') {\n      query += 'searchCriteria[filter_groups][0][filters][0][field]=updated_at&' +\n        'searchCriteria[filter_groups][0][filters][0][value]=' + encodeURIComponent(moment(context.updated_after).utc().format()) + '&' +\n        'searchCriteria[filter_groups][0][filters][0][condition_type]=gt';\n    }\n    return query;\n  }\n\n  getSourceData(context) {\n    const that = this\n    const retryHandler = (context, err, reject) => {\n      context.retry_count = context.retry_count ? context.retry_count + 1 : 1;\n      if (err == null || context.retry_count < HTTP_RETRIES ) {\n        if (err) {\n          logger.error(err);\n          logger.info('Retrying getSourceData() request ' + context.retry_count);\n        }\n        if (this.config.product && this.config.product.synchronizeCatalogSpecialPrices) {\n          return new Promise((resolve, reject) => {\n            this.getProductSourceData(context).then((result) => {\n              // download rendered list items\n              const products = result.items\n              let skus = products.map((p) => { return p.sku })\n\n              if (products.length === 1) { // single product - download child data\n                const childSkus = _.flattenDeep(products.map((p) => { return (p.configurable_children) ? p.configurable_children.map((cc) => { return cc.sku }) : null }))\n                skus = _.union(skus, childSkus)\n              }\n              const query = '&searchCriteria[filter_groups][0][filters][0][field]=sku&' +\n              'searchCriteria[filter_groups][0][filters][0][value]=' + encodeURIComponent(skus.join(',')) + '&' +\n              'searchCriteria[filter_groups][0][filters][0][condition_type]=in';\n\n              this.api.products.renderList(query, this.config.magento.storeId, this.config.magento.currencyCode).then(renderedProducts => {\n                context.renderedProducts = renderedProducts\n                for (let product of result.items) {\n                  const productAdditionalInfo = renderedProducts.items.find(p => p.id === product.id)\n\n                  if (productAdditionalInfo && productAdditionalInfo.price_info) {\n                    delete productAdditionalInfo.price_info.formatted_prices\n                    delete productAdditionalInfo.price_info.extension_attributes\n                    // delete productAdditionalInfo.price_info.special_price\n                    product = Object.assign(product, productAdditionalInfo.price_info)\n\n                    if (product.final_price < product.price) {\n                      product.special_price = product.final_price\n                    }\n\n                    if (this.config.product.renderCatalogRegularPrices) {\n                      product.price = product.regular_price\n                    }\n                  }\n                }\n                resolve(result)\n              })\n            }).catch(err => {\n              retryHandler(context, err, reject)\n            })\n          })\n        } else {\n          return this.getProductSourceData(context).catch(err => {\n              retryHandler(context, err, null)\n            })\n        }\n      } else {\n        if (reject) {\n          reject(err)\n        } else {\n          throw err\n        }\n      }\n    }\n\n    // run the import logick\n    return retryHandler(context, null, null)\n  }\n\n  getProductSourceData(context) {\n    let query = this.getFilterQuery(context);\n    let searchCriteria = '&searchCriteria[currentPage]=%d&searchCriteria[pageSize]=%d';\n\n    if(this.config.product && JSON.parse(this.config.product.excludeDisabledProducts)) {\n      searchCriteria += '&searchCriteria[filterGroups][0][filters][0][field]=status'+\n                        '&searchCriteria[filterGroups][0][filters][0][value]=1';\n    }\n\n    if(typeof context.stock_sync !== 'undefined')\n      this.stock_sync = context.stock_sync;\n\n    if(typeof context.parent_sync !== 'undefined')\n    {\n      logger.info('Configurable parent sync is ', context.parent_sync)\n      this.parent_sync = context.parent_sync;\n    }\n\n    if(typeof context.category_sync !== 'undefined')\n      this.category_sync = context.category_sync;\n\n    if(typeof context.configurable_sync !== 'undefined')\n      this.configurable_sync = context.configurable_sync;\n\n    if (context.for_total_count) { // get total counts\n      return this.api.products.list(util.format(searchCriteria, 1, 1)).catch((err) => {\n        throw new Error(err);\n      });\n    } else if (context.page && context.page_size) {\n\n      this.use_paging = context.use_paging || false\n      this.is_federated = context.use_paging ? false : true;\n      this.page = context.page;\n      this.page_size = context.page_size\n      if (!context.use_paging) this.page_count = 1; // process only one page - used for partitioning purposes\n\n      logger.debug(`Using specific paging options from adapter context: ${context.page} / ${context.page_size}`);\n\n      return this.api.products.list(util.format(searchCriteria, context.page, context.page_size) + (query ? '&' + query : '')).catch((err) => {\n        throw new Error(err);\n      });\n\n    } else if (this.use_paging) {\n      this.is_federated = false; // federated execution is not compliant with paging\n      logger.debug(util.format(searchCriteria, this.page, this.page_size) + (query ? '&' + query : ''));\n      return this.api.products.list(util.format(searchCriteria, this.page, this.page_size) + (query ? '&' + query : '')).catch((err) => {\n        throw new Error(err);\n      });\n    } else {\n      return this.api.products.list().catch((err) => {\n        throw new Error(err);\n      });\n    }\n  }\n\n  getTotalCount(context) {\n    context = context ? Object.assign(context, { for_total_count: 1 }) : { for_total_count: 1 };\n    return this.getSourceData(context); //api.products.list('&searchCriteria[currentPage]=1&searchCriteria[pageSize]=1');\n  }\n\n  getLabel(source_item) {\n    return `[(${source_item.id} - ${source_item.sku}) ${source_item.name}]`;\n  }\n\n  isNumeric(value) {\n    return /^\\d+$/.test(value);\n  }\n\n  processAttributes(customAttributes, configurableOptions) {\n    const loadFromCache = (key) => new Promise((resolve) =>\n      this.cache.get(key, (err, serializedAtr) => resolve(JSON.parse(serializedAtr)))\n    )\n    const findConfigurableOptionsValues = attributeId => {\n      const attribute = configurableOptions.find(\n        opt => parseInt(opt.attribute_id) === parseInt(attributeId)\n      )\n\n      if (attribute) {\n        return attribute.values.map(val => parseInt(val.value_index))\n      }\n\n      return []\n    }\n\n    const findCustomAttributesValues = (attributeCode) => {\n      const attribute = customAttributes.find(\n        opt => opt.attribute_code === attributeCode\n      )\n\n      return attribute ? [parseInt(attribute.value)] : []\n    }\n\n    const findOptionValues = option => ([\n      ...findConfigurableOptionsValues(option.attribute_id),\n      ...findCustomAttributesValues(option.attribute_code)\n    ])\n\n    const selectFields = (res) => res.map(o => {\n      const attributeOptionValues = findOptionValues(o)\n      const options = o.options.filter(opt => attributeOptionValues.includes(parseInt(opt.value)))\n\n      return {\n        is_visible_on_front: o.is_visible_on_front,\n        is_visible: o.is_visible,\n        default_frontend_label: o.default_frontend_label,\n        attribute_id: o.attribute_id,\n        entity_type_id: o.entity_type_id,\n        id: o.id,\n        frontend_input: o.frontend_input,\n        is_user_defined: o.is_user_defined,\n        is_comparable: o.is_comparable,\n        attribute_code: o.attribute_code,\n        slug: o.slug,\n        options\n      }\n    })\n\n    const attributeCodes = customAttributes.map(obj => new Promise((resolve) => {\n      const key = util.format(CacheKeys.CACHE_KEY_ATTRIBUTE, obj.attribute_code);\n      loadFromCache(key).then(resolve)\n    }))\n\n    const attributeIds = configurableOptions.map(obj => new Promise((resolve) => {\n      const key = util.format(CacheKeys.CACHE_KEY_ATTRIBUTE, obj.attribute_id);\n      loadFromCache(key).then(resolve)\n    }))\n\n    return Promise.all([\n      ...attributeCodes,\n      ...attributeIds\n    ])\n      .then(selectFields)\n  }\n\n  /**\n   *\n   * @param {Object} item\n   */\n  preProcessItem(item) {\n    for (let customAttribute of item.custom_attributes) { // map custom attributes directly to document root scope\n      let valueArray = String(customAttribute['value']).split(',');\n      let attrValue = valueArray.map(Number);\n      if (valueArray.length > 1){\n        for (let element of valueArray){\n          if (!this.isNumeric(element)) {\n            attrValue = customAttribute.value;\n            break;\n          }\n        }\n      } else {\n        attrValue = customAttribute.value;\n      }\n      item[customAttribute.attribute_code] = attrValue;\n    }\n    item.slug = _slugify(item.name + '-' + item.id);\n\n    return new Promise((done, reject) => {\n      // TODO: add denormalization of productcategories into product categories\n      // DO NOT use \"productcategories\" type but rather do search categories with assigned products\n\n      let subSyncPromises = [];\n      const config = this.config;\n\n      // TODO: Refactor the following to \"Chain of responsibility\"\n      // STOCK SYNC\n      if (this.stock_sync) {\n        logger.info(`Product sub-stage 1: Getting stock items for ${item.sku}`);\n        subSyncPromises.push(() => {\n          return this.api.stockItems.list(item.sku).then((result) => {\n            item.stock = result;\n\n            if (this.config.magento.msi.enabled) {\n              return this.api.stockItems.getSalableQty(item.sku, this.config.magento.msi.stockId).then((salableQty) => {\n                item.stock.qty = salableQty;\n                return item;\n              }).then((item) => {\n                return this.api.stockItems.isSalable(item.sku, this.config.magento.msi.stockId).then((isSalable) => {\n                  item.stock.is_in_stock = isSalable;\n\n                  const key = util.format(CacheKeys.CACHE_KEY_STOCKITEM, item.id);\n                  logger.debug(`Storing stock data to cache under: ${key}`);\n                  this.cache.set(key, JSON.stringify(item.stock));\n\n                  return item;\n                })\n              })\n            } else {\n              const key = util.format(CacheKeys.CACHE_KEY_STOCKITEM, item.id);\n              logger.debug(`Storing stock data to cache under: ${key}`);\n              this.cache.set(key, JSON.stringify(result));\n\n              return item;\n            }\n          })\n        })\n      }\n\n      // MEDIA SYNC\n      if (this.media_sync) {\n        logger.info(`Product sub-stage 2: Getting media gallery ${item.sku}`);\n        subSyncPromises.push(() => {\n          return this.api.productMedia.list(item.sku).then((result) => {\n            let media_gallery = []\n            for (let mediaItem of result){\n              if (!mediaItem.disabled) {\n                media_gallery.push({\n                  image: mediaItem.file,\n                  pos: mediaItem.position,\n                  typ: mediaItem.media_type,\n                  lab: mediaItem.label,\n                  vid: this.computeVideoData(mediaItem)\n                })\n              }\n            }\n            item.media_gallery = media_gallery\n            return item\n          })\n        })\n      }\n\n      // CUSTOM OPTIONS SYNC\n      if (this.custom_sync) {\n        logger.info(`Product sub-stage 3: Getting product custom options ${item.sku}`);\n        subSyncPromises.push(() => {\n          return this.api.customOptions.list(item.sku).then((result) => {\n            if (result && result.length > 0) {\n              item.custom_options = result\n              logger.info(`Found custom options for ${item.sku}: ${result.length}`)\n            }\n            return item\n          })\n        })\n      }\n\n      // BUNDLE OPTIONS SYNC\n      if (this.custom_sync && item.type_id == 'bundle') {\n        logger.info(`Product sub-stage 4: Getting bundle custom options ${item.sku}`);\n        subSyncPromises.push(() => {\n          return this.api.bundleOptions.list(item.sku).then((result) => {\n            if(result && result.length > 0) {\n              item.bundle_options = result\n              logger.info(`Found bundle options for ${item.sku}: ${result.length}`)\n            }\n            return item\n          })\n       })\n      }\n\n      // PRODUCT LINKS - as it seems magento returns these links anyway in the \"product_links\"\n      if (this.links_sync) {\n        logger.info(`Product sub-stage 5: Getting product links ${item.sku}`);\n        item.links = {}\n\n        subSyncPromises.push(() => {\n          return new Promise ((opResolve, opReject) => {\n            return this.api.productLinks.types().then((result) => {\n            if(result && result.length > 0) {\n              let subPromises = []\n              for (const linkType of result) {\n                logger.info(`Getting the product links ${item.sku}: ${linkType.name}`)\n                subPromises.push(this.api.productLinks.list(item.sku, linkType.name).then((links) => {\n                  if(links && links.length > 0) {\n                    item.links[linkType.name] = links.map((r) => { return { sku: r.linked_product_sku, pos: r.position } })\n                    logger.info(`Found related products for ${item.sku}: ${item.links[linkType.name]}`)\n                  }\n                  return item\n                }))\n              }\n              Promise.all(subPromises).then((res) => {\n                logger.info('Product links expanded!')\n                opResolve(item)\n              }).catch((err) => {\n                logger.error(err)\n                opResolve(item)\n              })\n            } else {\n              opResolve (item)\n            }\n            return item\n          })\n        })})\n      }\n\n      if (this.parent_sync && (item.type_id == 'simple')) {\n        subSyncPromises.push(() => {\n          return new Promise ((opResolve, opReject) => {\n            // Find the parent product and schedule a sync after subsequent configurable_children got modified\n            this.db.getDocuments('product', { query: { match: {'configurable_children.sku': item.sku } }}).then((docs) => {\n              if (docs && docs.length > 0) {\n                let queue = kue.createQueue(Object.assign(config.kue, { redis: config.redis }));\n                docs.map(parentProduct => { // schedule for update\n                  queue.createJob('product', { skus: [parentProduct.sku], adapter: 'magento' }).save();\n                  logger.info('Parent product update scheduled (make sure `cli.js productsworker` queue is running)', parentProduct.sku)\n                })\n                opResolve(item)\n              } else {\n                opResolve(item)\n              }\n            }).catch(err => {\n              logger.error(err)\n              opResolve(item)\n            })\n          })\n        })\n      }\n\n      // CONFIGURABLE AND BUNDLE SYNC\n      if (this.configurable_sync && (item.type_id == 'configurable')) {\n        logger.info(`Product sub-stage 6: Getting product options for ${item.sku}`);\n\n        // q.push(() => {\n        subSyncPromises.push(() => {\n          return new Promise ((opResolve, opReject) => {\n            this.api.configurableChildren.list(item.sku).then((result) => {\n\n              item.configurable_children = new Array()\n              for (let prOption of result) {\n                let confChild = {\n                  sku: prOption.sku,\n                  id: prOption.id,\n                  status: prOption.status,\n                  visibility: prOption.visibility,\n                  name: prOption.name,\n                  price: prOption.price,\n                  tier_prices: prOption.tier_prices,\n                };\n\n                if (prOption.custom_attributes) {\n                  for (let opt of prOption.custom_attributes) {\n                    confChild[opt.attribute_code] = opt.value\n                  }\n                }\n                const context = this.current_context\n                if (context.renderedProducts && context.renderedProducts.items.length) {\n                  const renderedProducts = context.renderedProducts\n                  const subProductAdditionalInfo = renderedProducts.items.find(p => p.id === confChild.id)\n\n                  if (subProductAdditionalInfo && subProductAdditionalInfo.price_info) {\n                    delete subProductAdditionalInfo.price_info.formatted_prices\n                    delete subProductAdditionalInfo.price_info.extension_attributes\n                    // delete subProductAdditionalInfo.price_info.special_price // always empty :-(\n                    confChild = Object.assign(confChild, subProductAdditionalInfo.price_info)\n                    if (confChild.final_price < confChild.price) {\n                      confChild.special_price = confChild.final_price\n                    }\n                    if(config.product.renderCatalogRegularPrices) {\n                      confChild.price = confChild.regular_price\n                    }\n\n                  }\n                }\n\n                item.configurable_children.push(confChild);\n                if(item.price  == 0) // if price is zero fix it with first children\n                  item.price = prOption.price;\n              }\n\n              // EXPAND CONFIGURABLE CHILDREN ATTRS\n              if (config.product && config.product.expandConfigurableFilters) {\n                for (const attrToExpand of config.product.expandConfigurableFilters){\n                  const expandedSet = new Set()\n                  if (item[attrToExpand]) {\n                    expandedSet.add(item[attrToExpand])\n                  }\n                  for (const confChild of item.configurable_children) {\n                    if (confChild[attrToExpand]) {\n                      expandedSet.add(confChild[attrToExpand])\n                    }\n                  }\n                  if (expandedSet.size > 0) {\n                    item[attrToExpand + '_options'] = Array.from(expandedSet)\n                  }\n                }\n              }\n\n              this.api.configurableOptions.list(item.sku).then((result) => {\n                item.configurable_options = result;\n\n                let subPromises = []\n                for (let option of item.configurable_options) {\n                  let atrKey = util.format(CacheKeys.CACHE_KEY_ATTRIBUTE, option.attribute_id);\n\n                  subPromises.push(new Promise ((resolve, reject) => {\n                    logger.info(`Configurable options for ${atrKey}`)\n                    this.cache.get(atrKey, (err, serializedAtr) => {\n                      let atr = JSON.parse(serializedAtr); // category object\n                      if (atr != null) {\n                        option.attribute_code = atr.attribute_code;\n                        option.values.map((el) => {\n                          el.label = optionLabel(atr, el.value_index)\n                        } )\n                        logger.info(`Product options for ${atr.attribute_code} for ${item.sku} set`);\n                        item[atr.attribute_code + '_options'] = option.values.map((el) => { return el.value_index } )\n                      }\n                      resolve(item)\n                    })\n                  }))\n                }\n\n                Promise.all(subPromises).then((res) => {\n                  logger.info('Configurable options expanded!')\n                  opResolve(item)\n                })\n\n              }).catch((err) => {\n                logger.error(err);\n                opResolve(item)\n              })\n\n            }).catch((err) => {\n              logger.error(err);\n              opResolve(item)\n            })\n          })\n        })\n      }\n\n      subSyncPromises.push(() => {\n        return new Promise((resolve) => {\n          this.processAttributes(item.custom_attributes, item.configurable_options || []).then(res => {\n            item.attributes_metadata = res\n            item.custom_attributes = null\n            resolve(item)\n          })\n        })\n      });\n\n      // CATEGORIES SYNC\n      subSyncPromises.push(() => {\n        return new Promise((resolve, reject) => {\n        logger.info(`Product sub-stage 6: Getting product categories for ${item.sku}`);\n\n        const key = util.format(CacheKeys.CACHE_KEY_PRODUCT_CATEGORIES, item.sku); // store under SKU of the product the categories assigned\n\n        if(this.category_sync) {\n          item.category = new Array();\n\n          const catBinder = (categories) => {\n\n            let catPromises = new Array();\n            for (let catId of categories) {\n              catPromises.push(\n                new Promise((resolve, reject) => {\n                  let cat = this.cache.get(util.format(CacheKeys.CACHE_KEY_CATEGORY, catId), (err, serializedCat) => {\n                    let cat = JSON.parse(serializedCat); // category object\n                    if (cat != null) {\n                      resolve({\n                        category_id: cat.id,\n                        name: cat.name,\n                        slug: cat.slug,\n                        path: cat.url_path\n                      })\n                    } else {\n                      resolve({\n                        category_id: catId\n                      });\n                    }\n                  });\n                })\n              );\n            }\n\n            Promise.all(catPromises).then((values) => {\n              if(this.category_sync) // TODO: refactor the code above to not get cache categorylinks when no category_sync required\n                item.category = values; // here we get configurable options\n                if (this.config.seo.useUrlDispatcher) {\n                  item.url_path = this.config.seo.productUrlPathMapper(item)\n                }\n                resolve(item)\n            });\n          }\n          if (item.category_ids && Array.isArray(item.category_ids) && item.category_ids.length > 0) {\n            const catIdsArray = item.category_ids.map(item => { return parseInt(item)})\n            logger.info(`Using category_ids binding for ${item.sku}: ${catIdsArray}`)\n            catBinder(catIdsArray)\n          } else {\n            this.cache.smembers(key, (err, categories) => {\n              if (categories == null) {\n                resolve(item);\n              }\n              else {\n                catBinder(categories)\n              }\n            })\n          }\n        }\n      })})\n\n      serial(subSyncPromises).then((res) => {\n        logger.info(`Product sub-stages done for ${item.sku}`)\n        return done(item) // all subpromises return refernce to the product\n      }).catch(err => {\n        logger.error(err);\n        return done(item)\n      });\n    });\n  }\n\n  /**\n   * Process video data to provide the proper\n   * provider and attributes.\n   * Currently supports YouTube and Vimeo\n   *\n   * @param {Object} mediaItem\n   */\n  computeVideoData(mediaItem) {\n    let videoData = null;\n\n    if (mediaItem.extension_attributes && mediaItem.extension_attributes.video_content) {\n      let videoId = null,\n          type = null,\n          youtubeRegex = /^(?:https?:\\/\\/)?(?:www\\.)?(?:youtu\\.be\\/|youtube\\.com\\/(?:embed\\/|v\\/|watch\\?v=|watch\\?.+&v=))((\\w|-){11})(?:\\S+)?$/,\n          vimeoRegex = new RegExp(['https?:\\\\/\\\\/(?:www\\\\.|player\\\\.)?vimeo.com\\\\/(?:channels\\\\/(?:\\\\w+\\\\/)',\n            '?|groups\\\\/([^\\\\/]*)\\\\/videos\\\\/|album\\\\/(\\\\d+)\\\\/video\\\\/|video\\\\/|)(\\\\d+)(?:$|\\\\/|\\\\?)'\n          ].join(''));\n\n      if (mediaItem.extension_attributes.video_content.video_url.match(youtubeRegex)) {\n        videoId = RegExp.$1\n        type = 'youtube'\n      } else if (mediaItem.extension_attributes.video_content.video_url.match(vimeoRegex)) {\n        videoId = RegExp.$3\n        type = 'vimeo'\n      }\n\n      videoData = {\n        url: mediaItem.extension_attributes.video_content.video_url,\n        title: mediaItem.extension_attributes.video_content.video_title,\n        desc: mediaItem.extension_attributes.video_content.video_description,\n        meta: mediaItem.extension_attributes.video_content.video_metadata,\n        video_id: videoId,\n        type: type\n      }\n    }\n\n    return videoData;\n  }\n\n  /**\n   * We're transorming the data structure of item to be compliant with Smile.fr Elastic Search Suite\n   * @param {object} item  document to be updated in elastic search\n   */\n  normalizeDocumentFormat(item) {\n    let prices = new Array();\n\n    /*for (let priceTag of item.tier_prices) {\n      prices.push({\n        \"price\": priceTag.value,\n        \"original_price\": priceTag.original_price,\n        \"customer_group_id\": priceTag.customerGroupId,\n        \"qty\": priceTag.qty\n      });\n    }*/\n\n    if (this.config.vuestorefront && this.config.vuestorefront.invalidateCache) {\n      request(this.config.vuestorefront.invalidateCacheUrl + 'P' + item.id, {}, (err, res, body) => {\n        if (err) { return console.error(err); }\n        try {\n          if (body && JSON.parse(body).code !== 200) console.log(body);\n        } catch (e) {\n          return console.error('Invalid Cache Invalidation response format', e)\n        }\n      });\n    }\n\n    let resultItem = Object.assign(item, {\n    // \"price\": prices, // ES stores prices differently\n    // TODO: HOW TO GET product stock from Magento API call for product?\n    });\n    return resultItem;\n  }\n}\n\nmodule.exports = ProductAdapter;\n"
  },
  {
    "path": "src/adapters/magento/productcategories.js",
    "content": "'use strict';\n\nlet AbstractMagentoAdapter = require('./abstract');\n\nlet queue = require('queue');\nlet util = require('util');\nlet q = queue({ concurrency: 4 });\n\nconst ProductCategoriesModes = {\n  SEPARATE_ELASTICSEARCH: 1,\n  UPDATE_CATEGORY: 2,\n  SEPARATE_REDIS: 3\n}\n\nconst CacheKeys = require('./cache_keys');\n\n/**\n * This adapter retrieves and stores all product / category links from Magento2\n */\nclass ProductcategoriesAdapter extends AbstractMagentoAdapter {\n\n  constructor(config) {\n    super(config);\n    this.update_document = false; // this adapter works on categories but doesn't update them itself but is focused on category links instead!\n    this._productHisto = new Set();\n\n    this.mode = ProductCategoriesModes.SEPARATE_REDIS;\n\n    this.catLinkQueue = new Array();\n  }\n\n  getEntityType() {\n    return 'productcategories';\n  }\n\n  getName() {\n    return 'adapters/magento/ProductcategoryAdapter';\n  }\n\n  getSourceData(context) {\n    return this.api.categories.list();\n  }\n\n  getLabel(source_item) {\n    return `[(${source_item.id}) ${source_item.name}]`;\n  }\n\n  /**\n   * Process category link product/category\n   * @param {Object} result \n   */\n  _storeCatLinks(item, result) {\n    let index = 0;\n    let length = result.length;\n\n    switch (this.mode) {\n      case ProductCategoriesModes.SEPARATE_ELASTICSEARCH: {\n        for (let catLink of result) {\n          // logger.debug('(' +index +'/' + length +  ') Storing categoryproduct link for ' + catLink.sku +' - ' + catLink.category_id);\n          catLink.id = catLink.category_id * MAX_PRODUCTS_IN_CAT + index;\n\n          index++;\n\n          catLink = this.normalizeDocumentFormat(catLink);\n        }\n\n        logger.debug('Performing bulk update...');\n        this.db.updateDocumentBulk('productcategories', result); // TODO: add support for BULK operations and DELETE \n      }\n\n      case ProductCategoriesModes.UPDATE_CATEGORY: {\n        // TODO: update products to set valid category\n        item.products = result;\n        this.db.updateDocument('category', item);\n        logger.debug(`Updating category object for ${item.id}`);\n      }\n\n      default: { // update in redis = ProductCategoriesModes.SEPARATE_REDIS\n        logger.debug(`Storing category assigments in REDIS cache for ${item.id}`);\n\n        for (let catLink of result) {\n          const key = util.format(CacheKeys.CACHE_KEY_PRODUCT_CATEGORIES_TEMPORARY, catLink.sku); // store under SKU of the product the categories assigned\n\n          this.cache.sadd(key, catLink.category_id); // add categories to sets\n          this._productHisto.add(catLink.sku);\n        }\n      }\n    }\n\n    logger.info(`The task count still is = ${this.tasks_count}`);\n  }\n\n  /**\n   * Get the product category links for this specific category and update the products\n   * @param {Object} item \n   */\n  preProcessItem(item) {\n    return new Promise((done, reject) => {\n// q.push(() => {\n      this.api.categoryProducts.list(item.id).then((result) => {\n        this._storeCatLinks(item, result);\n        done(item);\n      }).catch((err) => {\n        logger.error(err);\n        done(item);\n      });\n  // });\n    });\n  }\n\n  /**\n   * Default done callback called after all main items are processed by processItems\n   */\n  defaultDoneCallback() {\n    logger.info('Renaming the cache key to production! ');\n\n    for(let sku of this._productHisto){\n      const origKey = util.format(CacheKeys.CACHE_KEY_PRODUCT_CATEGORIES_TEMPORARY, sku); \n      const destKey = util.format(CacheKeys.CACHE_KEY_PRODUCT_CATEGORIES, sku); \n      // logger.debug(util.format('Moving %s to %s', origKey, destKey));\n\n      this.cache.rename(origKey, destKey);\n    }\n\n    logger.info('Publishing cache keys to production finished!');\n\n  /*  q.start((err) => {\n      if (err) throw err\n      logger.info('all done:', results)\n    });*/\n  }\n\n  isFederated() {\n    return false;\n  }\n\n  /**\n   * We're transorming the data structure of item to be compliant with Smile.fr Elastic Search Suite\n   * @param {object} item  document to be updated in elastic search\n   */\n  normalizeDocumentFormat(item) {\n    return item;\n  }\n\n}\n\nmodule.exports = ProductcategoriesAdapter;\n"
  },
  {
    "path": "src/adapters/magento/review.js",
    "content": "'use strict';\n\nlet AbstractMagentoAdapter = require('./abstract');\n\nclass ReviewAdapter extends AbstractMagentoAdapter {\n  constructor(config) {\n    super(config);\n    this.use_paging = false;\n  }\n\n  getEntityType() {\n    return 'review';\n  }\n\n  getName() {\n    return 'adapters/magento/ReviewAdapter';\n  }\n\n  getSourceData(context) {\n    if (this.use_paging) {\n      return this.api.reviews.list('&searchCriteria[currentPage]=' + this.page + '&searchCriteria[pageSize]=' + this.page_size + (query ? '&' + query : '')).catch((err) => {\n        throw new Error(err);\n      });\n    }\n\n    return this.api.reviews.list().catch((err) => {\n      throw new Error(err);\n    });\n  }\n\n  /**\n   * Regarding Magento2 api docs and reality\n   * we do have an exception here that items aren't listed straight\n   * in the response but under \"items\" key */\n  prepareItems(items) {\n    if (!items) {\n      return items;\n    }\n\n    if (items.total_count) {\n      this.total_count = items.total_count;\n    }\n\n    if (this.use_paging) {\n      this.page_count = Math.ceil(this.total_count / this.page_size);\n    }\n\n    if (items.items) {\n      items = items.items; // this is an exceptional behavior for Magento2 api for lists\n    }\n\n    return items;\n  }\n\n  isFederated() {\n    return false;\n  }\n\n  preProcessItem(item) {\n    logger.debug(item);\n    //\n    return new Promise((done, reject) => {\n      if (item) {\n        item.product_id = item.entity_pk_value;\n\n        delete item.entity_pk_value;\n\n        logger.debug(`Review ${item.id}`);\n      }\n\n      return done(item);\n    });\n  }\n\n  /**\n   * We're transorming the data structure of item to be compliant with Smile.fr Elastic Search Suite\n   * @param {object} item  document to be updated in elastic search\n   */\n  normalizeDocumentFormat(item) {\n    return item;\n  }\n}\n\nmodule.exports = ReviewAdapter;\n"
  },
  {
    "path": "src/adapters/magento/taxrule.js",
    "content": "'use strict';\n\nlet AbstractMagentoAdapter = require('./abstract');\n\nclass TaxruleAdapter extends AbstractMagentoAdapter {\n\n  getEntityType() {\n    return 'taxrule';\n  }\n\n  getName() {\n    return 'adapters/magento/TaxrulesAdapter';\n  }\n\n  getSourceData(context) {\n    return this.api.taxRules.list();\n  }\n\n  getLabel(source_item) {\n    return `[(${source_item.id}) ${source_item.code}]`;\n  }\n\n  isFederated() {\n    return false;\n  }\n\n  preProcessItem(item) {\n    return new Promise((done, reject) => {\n\n      // TODO get tax rates for this tax rule\n      let subPromises = []\n      item.rates = []\n      logger.info(item)\n\n      for (let ruleId of item.tax_rate_ids) {\n        subPromises.push(new Promise((resolve, reject) => { \n          this.api.taxRates.list(ruleId).then(function(result) { \n            result.rate = parseFloat(result.rate)\n            item.rates.push(result)\n            resolve (result)\n          })\n        }))\n      }\n\n      Promise.all(subPromises).then(function(results) {\n        logger.info(`Rates downloaded for ${item.code}`)\n        logger.info(item)\n        return done(item);\n      })\n    });\n  }\n\n  prepareItems(items) {\n    if(!items)\n      return null;\n\n    this.total_count = items.total_count;\n    return items.items;\n  }\n  \n  /**\n   * We're transorming the data structure of item to be compliant with Smile.fr Elastic Search Suite\n   * @param {object} item  document to be updated in elastic search\n   */\n  normalizeDocumentFormat(item) {\n    return item;\n  }\n}\n\nmodule.exports = TaxruleAdapter;\n"
  },
  {
    "path": "src/adapters/nosql/abstract.js",
    "content": "'use strict';\n\n\nclass AbstractNosqlAdapter{\n\n  validateConfig(config){\n\n    if(!config['db']['url'])\n      throw Error('db.url must be set up in config');\n\n  }\n\n  constructor(app_config){\n    this.config = app_config;\n    this.db = null;\n    this.validateConfig(this.config);\n    this.updateDocument.bind(this);\n  }\n\n  /**\n   * Get where query to identify the base record\n   * @param {Object} source_item \n   */\n  getWhereQuery(source_item){\n    return { id: source_item.id }; \n  }\n\n  /**\n   * Close the nosql database connection - abstract to the driver\n   */\n  close(){\n    throw new Error('close needs implementation!');\n  }\n\n/**\n * Update single document in database\n * @param {object} item document to be updated in database\n */\n  updateDocument(collectionName, item) {\n    throw new Error('updateDocument needs implementation!');\n  }\n\n  /**\n   * Get documents\n   * @param collectionName collection name\n   * @param query query object\n   */  \n  getDocuments(collectionName, query) {\n    throw new Error('getDocum ent needs implementation!');\n  }\n  /**\n   * Update multiple documents in database\n   * @param {array} items to be updated\n   */\n  updateDocumentBulk(collectionName, items){\n    throw new Error('updateDocumentBulk needs implementation!');\n  }\n\n   /**\n   * Remove records other than <record>.tsk = \"transactionKey\"\n   * @param {String} collectionName\n   * @param {int} transactionKey transaction key - which is usualy a timestamp\n   */\n  cleanupByTransactionkey(collectionName, transactionKey){\n    throw new Error('cleanupByTransactionkey needs implementation!');\n  }\n  \n/**\n * Connect / prepare driver\n */\n  connect (done){\n    throw new Error('connect needs implementation!');\n  }\n\n}\n\nmodule.exports = AbstractNosqlAdapter;\n"
  },
  {
    "path": "src/adapters/nosql/elasticsearch.js",
    "content": "'use strict';\nconst AbstractNosqlAdapter = require('./abstract');\nconst elasticsearch = require('@elastic/elasticsearch');\nconst AgentKeepAlive = require('agentkeepalive');\nconst AgentKeepAliveHttps = require('agentkeepalive').HttpsAgent;\n\n\nclass ElasticsearchAdapter extends AbstractNosqlAdapter {\n\n  validateConfig(config) {\n\n    if (!config['db']['url'])\n      throw Error('db.url must be set up in config');\n\n    if (!config['db']['indexName'])\n      throw Error('db.indexName must be set up in config');\n\n  }\n\n  constructor(app_config) {\n    super(app_config);\n\n    this.config = app_config;\n    this.db = null;\n    this.validateConfig(this.config);\n\n    logger.debug('Elasticsearch module initialized!');\n    this.updateDocument.bind(this);\n  }\n\n  /**\n   * Get physical Elastic index name; since 7.x we're adding an entity name to get real index: vue_storefront_catalog_product, vue_storefront_catalog_category and so on\n   * @param {*} baseIndexName \n   * @param {*} config \n   */\n  getPhysicalIndexName(collectionName, config) {\n    if (parseInt(config.elasticsearch.apiVersion) >= 6) {\n      return `${config.db.indexName}_${collectionName}`\n    } else {\n      return config.db.indexName\n    }\n  }\n\n  /**\n   * Get physical Elastic type name; since 7.x index can have one type _doc\n   * @param {*} baseIndexName \n   * @param {*} config \n   */\n  getPhysicalTypeName(collectionName, config) {\n    if (parseInt(config.elasticsearch.apiVersion) >= 6) {\n      return `_doc`\n    } else {\n      return collectionName\n    }\n  }  \n\n  /**\n   * Close the nosql database connection - abstract to the driver\n   */\n  close() { // switched to global singleton\n    //this.db.close();\n  }\n\n  /**\n   * Get documents\n   * @param collectionName collection name\n   * @param query query object\n  */  \n  getDocuments(collectionName, queryBody) {\n    return new Promise((resolve, reject) => {\n      const searchQueryBody = {\n        index: this.getPhysicalIndexName(collectionName, this.config),\n        body: queryBody\n      }\n      if (parseInt(this.config.elasticsearch.apiVersion) < 6)\n       searchQueryBody.type  = this.getPhysicalTypeName(collectionName, this.config)\n\n      this.db.search(searchQueryBody, function (error, { body: response }) {\n        if (error) reject(error);\n        if (response.hits && response.hits.hits) {\n          resolve(response.hits.hits.map(h => h._source))\n        } else {\n          reject(new Error('Invalid Elastic response'))\n        }\n      });\n    })\n  }\n\n  /**\n   * Update single document in database\n   * @param {object} item document to be updated in database\n   */\n  updateDocument(collectionName, item) {\n    const itemtbu = item;\n    const updateRequestBody = {\n      index: this.getPhysicalIndexName(collectionName, this.config),\n      id: item.id,\n      body: {\n        // put the partial document under the `doc` key\n        upsert: itemtbu,\n        doc: itemtbu\n\n      }\n    }\n    if (parseInt(this.config.elasticsearch.apiVersion) < 6)\n      updateRequestBody.type = this.getPhysicalTypeName(collectionName, this.config)\n\n    this.db.update(updateRequestBody, function (error, response) {\n      if (error)\n        throw new Error(error);\n    });\n  }\n\n  /**\n  * Remove records other than <record>.tsk = \"transactionKey\"\n  * @param {String} collectionName\n  * @param {int} transactionKey transaction key - which is usually a timestamp\n  */\n  cleanupByTransactionkey(collectionName, transactionKey) {\n\n    if (transactionKey) {\n      const query = {\n        index: this.getPhysicalIndexName(collectionName, this.config),\n        conflicts: 'proceed',\n        body: {\n          query: {\n            bool: {\n              must_not: {\n                term: { tsk: transactionKey }\n              }\n            }\n          }\n        }\n      };\n      if (parseInt(this.config.elasticsearch.apiVersion) < 6)\n        query.type = this.getPhysicalTypeName(collectionName, this.config)\n\n      this.db.deleteByQuery(query, function (error, response) {\n        if (error) throw new Error(error);\n      });\n    }\n  }\n\n  /**\n   * Update multiple documents in database\n   * @param {array} items to be updated\n   */\n  updateDocumentBulk(collectionName, items) {\n\n    let requests = new Array();\n    let index = 0;\n    let bulkSize = 500;\n\n    for (let doc of items) {\n      const query = {\n        _index: this.getPhysicalIndexName(collectionName, this.config),\n        _id: doc.id,\n      };\n      if (parseInt(this.config.elasticsearch.apiVersion) < 6)\n        query.type = this.getPhysicalTypeName(collectionName, this.config)\n\n      requests.push({\n        update: query\n      });\n\n      requests.push({\n\n        // put the partial document under the `doc` key\n        doc: doc,\n        \"doc_as_upsert\": true\n\n      });\n\n      if ((index % bulkSize) == 0) {\n        logger.debug('Splitting bulk query ' + index);\n        this.db.bulk({\n          body: requests\n        }, function (error, response) {\n          if (error)\n            throw new Error(error);\n        });\n\n        requests = new Array();\n      }\n\n      index++;\n    }\n\n  }\n\n  /**\n   * Connect / prepare driver\n   * @param {Function} done callback to be called after connection is established\n   */\n  connect(done) {\n\n    if (!global.es) {\n      this.db = new elasticsearch.Client({\n        node: this.config.db.url,\n        log: 'debug',\n        apiVersion: this.config.elasticsearch.apiVersion,\n\n        maxRetries: 10,\n        keepAlive: true,\n        maxSockets: 10,\n        minSockets: 10,\n        requestTimeout: 1800000,        \n\n        createNodeAgent: function (connection, config) {\n          if (connection.useSsl) {\n            return new AgentKeepAliveHttps(connection.makeAgentConfig(config));\n          }\n          return new AgentKeepAlive(connection.makeAgentConfig(config));\n        }\n\n      });\n      global.es = this.db;\n    } else\n      this.db = global.es;\n\n    done(this.db);\n  }\n\n}\n\nmodule.exports = ElasticsearchAdapter;\n"
  },
  {
    "path": "src/api/routes/magento.js",
    "content": "'use strict';\nlet express = require('express');\nlet kue = require('kue');\n\nlet config = require('../../config');\nlet AdapterFactory = require('../../adapters/factory');\nlet factory = new AdapterFactory(config);\n\nlet router = express.Router();\n\nrouter.post('/products/update', function(req, res) { // TODO: add api key middleware\n\n  let skus_array = req.body.sku;\n\n  console.log('Incoming pull request for', skus_array)\n  if(skus_array.length > 0){\n    let queue = kue.createQueue(Object.assign(config.kue, { redis: config.redis }));\n\n    queue.createJob('product', { skus: skus_array, adapter: 'magento' }).save();\n    res.json({ status: 'done', message: 'Products ' + skus_array + ' scheduled to be refreshed'});\n  } else {\n    res.json({ status: 'error', message: 'Please provide product SKU separated by comma'});\n  }\n});\n\nmodule.exports = router;\n"
  },
  {
    "path": "src/cli.js",
    "content": "'use strict';\n\nconst program = require('commander');\nconst fs = require('fs');\nconst path = require('path');\nlet AdapterFactory = require('./adapters/factory');\n\nconst TIME_TO_EXIT = process.env.TIME_TO_EXIT ? process.env.TIME_TO_EXIT : 30000; // wait 30s before quiting after task is done\n\nlet config = require('./config');\nlet logger = require('./log');\nlet factory = new AdapterFactory(config);\nconst jsonFile = require('jsonfile')\nconst INDEX_META_PATH = process.env.INDEX_META_PATH ? path.join('tmp', process.env.INDEX_META_PATH) : path.join('tmp', '.lastIndex.json')\n\nlet kue = require('kue');\nlet queue = kue.createQueue(Object.assign(config.kue, { redis: config.redis }));\n\nconst _handleBoolParam = (value) => {\n  return JSON.parse(value) // simple way to handle all the '0', '1', 'true', true, false ...\n}\n\nconst reindexAttributes = (adapterName, removeNonExistent) => {\n  removeNonExistent = _handleBoolParam(removeNonExistent)\n\n  return new Promise((resolve, reject) => {\n    let adapter = factory.getAdapter(adapterName, 'attribute');\n    let tsk = new Date().getTime();\n  \n    adapter.run({\n      transaction_key: tsk,\n      done_callback: () => {\n        if (removeNonExistent) {\n          adapter.cleanUp(tsk);\n        }\n  \n        logger.info('Task done! Exiting in 30s...');\n        setTimeout(process.exit, TIME_TO_EXIT); // let ES commit all changes made\n        resolve();\n      }\n    });\n  })\n}\n\nconst reindexReviews = (adapterName, removeNonExistent) => {\n  removeNonExistent = _handleBoolParam(removeNonExistent)\n\n  return new Promise((resolve, reject) => {\n    let adapter = factory.getAdapter(adapterName, 'review');\n    let tsk = new Date().getTime();\n  \n    adapter.cleanUp(tsk);\n  \n    adapter.run({\n      transaction_key: tsk,\n      done_callback: () => {\n        if (removeNonExistent) {\n          adapter.cleanUp(tsk);\n        }\n  \n        logger.info('Task done! Exiting in 30s...');\n        setTimeout(process.exit, TIME_TO_EXIT); // let ES commit all changes made\n        resolve();\n      }\n    });\n  });\n}\n\n/**\n * Re-index cms blocks\n */\nconst reindexBlocks = (adapterName, removeNonExistent) => {\n  removeNonExistent = _handleBoolParam(removeNonExistent)\n\n  return new Promise((resolve, reject) => {\n    let adapter = factory.getAdapter(adapterName, 'cms_block');\n    let tsk = new Date().getTime();\n\n    adapter.run({\n      transaction_key: tsk,\n      done_callback: () => {\n        if (removeNonExistent) {\n          adapter.cleanUp(tsk);\n        }\n\n        logger.info('Task done! Exiting in 30s...');\n        setTimeout(process.exit, TIME_TO_EXIT); // let ES commit all changes made\n        resolve();\n      }\n    })\n  })\n}\n\n/**\n * Re-index cms pages\n */\nconst reindexPages = (adapterName, removeNonExistent) => {\n  removeNonExistent = _handleBoolParam(removeNonExistent)\n\n  return new Promise((resolve, reject) => {\n    let adapter = factory.getAdapter(adapterName, 'cms_page');\n    let tsk = new Date().getTime();\n    adapter.run({\n      transaction_key: tsk,\n      done_callback: () => {\n        if (removeNonExistent) {\n          adapter.cleanUp(tsk);\n        }\n\n        logger.info('Task done! Exiting in 30s...');\n        setTimeout(process.exit, TIME_TO_EXIT); // let ES commit all changes made\n        resolve();\n      }\n    })\n  })\n}\n\nconst reindexCategories = (adapterName, removeNonExistent, extendedCategories, generateUniqueUrlKeys) => {\n  removeNonExistent = _handleBoolParam(removeNonExistent)\n  extendedCategories = _handleBoolParam(extendedCategories)\n  generateUniqueUrlKeys = (_handleBoolParam(generateUniqueUrlKeys))\n\n  return new Promise((resolve, reject) => {\n    let adapter = factory.getAdapter(adapterName, 'category');\n    let tsk = new Date().getTime();\n\n    adapter.run({\n      transaction_key: tsk,\n      extendedCategories: extendedCategories,\n      generateUniqueUrlKeys: generateUniqueUrlKeys,\n      done_callback: () => {\n        if (removeNonExistent) {\n          adapter.cleanUp(tsk);\n        }\n\n        logger.info('Task done! Exiting in 30s...');\n        setTimeout(process.exit, TIME_TO_EXIT); // let ES commit all changes made\n        resolve();\n      }\n    });\n  });\n}\n\nconst reindexTaxRules = (adapterName, removeNonExistent) => {\n  removeNonExistent = _handleBoolParam(removeNonExistent)\n\n  return new Promise((resolve, reject) => {\n    let adapter = factory.getAdapter(adapterName, 'taxrule');\n    let tsk = new Date().getTime();\n  \n    adapter.run({\n      transaction_key: tsk,\n      done_callback: () => {\n        if (removeNonExistent) {\n          adapter.cleanUp(tsk);\n        }\n  \n        logger.info('Task done! Exiting in 30s...');\n        setTimeout(process.exit, TIME_TO_EXIT); // let ES commit all changes made\n        resolve();\n      }\n    });\n  });\n}\n\nconst reindexProductCategories = (adapterName) => {\n  return new Promise((resolve, reject) => {\n    let adapter = factory.getAdapter(adapterName, 'productcategories');\n    adapter.run({\n      done_callback: () => {\n        logger.info('Task done! Exiting in 30s...');\n        setTimeout(process.exit, TIME_TO_EXIT); // let ES commit all changes made\n        resolve();\n      }\n    });\n  });\n}\n\nfunction cleanup(adapterName, cleanupType, transactionKey) {\n  let adapter = factory.getAdapter(adapterName, cleanupType);\n  let tsk = transactionKey;\n\n  if (tsk) {\n    logger.info('Cleaning up for TRANSACTION KEY = ' + tsk);\n    adapter.connect\n    adapter.cleanUp(tsk);\n  } else {\n    logger.error('No \"transactionKey\" given as a parameter');\n  }\n}\n\nfunction reindexProducts(adapterName, removeNonExistent, partitions, partitionSize, initQueue, skus, updatedAfter = null, page = null) {\n  removeNonExistent = _handleBoolParam(removeNonExistent)\n  initQueue = _handleBoolParam(initQueue)\n\n  let adapter = factory.getAdapter(adapterName, 'product');\n\n  if (updatedAfter) {\n    logger.info('Delta indexer started for', updatedAfter)\n  }\n  let tsk = new Date().getTime();\n\n  if (partitions > 1 && adapter.isFederated()) { // standard case\n    let partition_count = partitions;\n\n    logger.info(`Running in MPM (Multi Process Mode) with partitions count = ${partition_count}`);\n\n    adapter.getTotalCount({ updated_after: updatedAfter }).then((result) => {\n\n      let total_count = result.total_count;\n      let page_size = partitionSize;\n      let page_count = Math.ceil(total_count / page_size);\n\n      let transaction_key = new Date().getTime();\n\n      if (initQueue) {\n        logger.info('Propagating job queue... ');\n\n        for (let i = 1; i <= page_count; i++) {\n          logger.debug(`Adding job for: ${i} / ${page_count}, page_size = ${page_size}`);\n          queue.createJob('products', { page_size: page_size, page: i, updated_after: updatedAfter }).save();\n        }\n      } else {\n        logger.info('Not propagating queue - only worker mode!');\n      }\n\n      // TODO: separate the execution part to run in multi-tenant env\n      queue.process('products', partition_count, (job, done) => {\n        let adapter = factory.getAdapter(adapterName, 'product');\n        if (job && job.data.page && job.data.page_size) {\n          logger.info(`Processing job: ${job.data.page}`);\n\n          adapter.run({\n            transaction_key: transaction_key,\n            page_size: job.data.page_size,\n            page: job.data.page,\n            parent_sync: job.data.updatedAfter !== null,\n            updated_after: job.data.updatedAfter,\n            done_callback: () => {\n              logger.info('Task done!');\n              return done();\n            }\n          });\n        } else return done();\n      });\n\n      if (initQueue) { // if this is not true it meant that process is runing to process the queue in the loop and shouldnt be \"killed\"\n        setInterval(() => {\n          queue.inactiveCount((err, total) => { // others are activeCount, completeCount, failedCount, delayedCount\n            if (total == 0) {\n\n              if (removeNonExistent) {\n                logger.info('CleaningUp products!');\n                let adapter = factory.getAdapter(adapterName, 'product');\n                adapter.cleanUp(transaction_key);\n              }\n\n              logger.info('Queue processed. Exiting!');\n              setTimeout(process.exit, TIME_TO_EXIT); // let ES commit all changes made\n            }\n          });\n        }, 2000);\n      }\n    });\n\n  } else {\n    logger.info('Running in SPM (Single Process Mode)');\n    let context = {\n      page: page !== null ? parseInt(page) : null,\n      page_size: partitionSize,\n      use_paging: true,\n      updated_after: updatedAfter,\n      transaction_key: tsk,\n      parent_sync: updatedAfter !== null,\n      done_callback: () => {\n        if (removeNonExistent) {\n          adapter.cleanUp(tsk);\n        }\n        logger.info('Task done! Exiting in 30s...');\n        setTimeout(process.exit, TIME_TO_EXIT); // let ES commit all changes made\n      }\n    };\n    if (page!== null) logger.info('Current page is: ', page, partitionSize)\n    if (skus) {\n      context.skus = skus.split(','); // update individual producs\n      context.parent_sync = true\n    }\n\n    adapter.run(context);\n  }\n}\n\nfunction fullReindex(adapterName, removeNonExistent, partitions, partitionSize, initQueue, skus, extendedCategories, generateUniqueUrlKeys) {\n  // The sequence is important because commands operate on some cache resources - especially for product/category assignments\n  Promise.all([\n    reindexAttributes(adapterName, removeNonExistent), // 0. It stores attributes in redis cache\n    reindexTaxRules(adapterName, removeNonExistent), // 1. It indexes the taxRules\n    reindexCategories(adapterName, removeNonExistent, extendedCategories, generateUniqueUrlKeys), //2. It stores categories in redis cache\n    reindexProductCategories(adapterName) // 3. It stores product/cateogry links in redis cache\n  ]).then((results) => {\n    logger.info('Starting full products reindex!');\n    reindexProducts(adapterName, removeNonExistent, partitions, partitionSize, initQueue, skus); //4. It indexes all the products\n  }).catch((err) => {\n    logger.error(err);\n    process.exit(1)\n  });\n}\n\n/**\n * Run worker listening to \"product\" command on KUE queue\n */\nfunction runProductsworker(adapterName, partitions) {\n\n  logger.info('Starting `productsworker` worker. Waiting for jobs ...');\n  let partition_count = partitions;\n\n  // TODO: separte the execution part to run in multi-tenant env\n  queue.process('product', partition_count, (job, done) => {\n\n    if (job && job.data.skus && Array.isArray(job.data.skus)) {\n\n      logger.info('Starting product pull job for ' + job.data.skus.join(','));\n\n      let adapter = factory.getAdapter(job.data.adapter ? job.data.adapter : adapterName, 'product');\n\n      adapter.run({\n        skus: job.data.skus,\n        parent_sync: true,\n        done_callback: () => {\n          logger.info('Task done!');\n          return done();\n        }\n      });\n    } else return done();\n\n  });\n}\n\nprogram\n  .command('attributes')\n  .option('--adapter <adapter>', 'name of the adapter', 'magento')\n  .option('--removeNonExistent <removeNonExistent>', 'remove non existent products', false)\n  .action(async (cmd) => {\n    await reindexAttributes(cmd.adapter, cmd.removeNonExistent);\n  });\n\nprogram\n  .command('categories')\n  .option('--adapter <adapter>', 'name of the adapter', 'magento')\n  .option('--removeNonExistent <removeNonExistent>', 'remove non existent products', false)\n  .option('--extendedCategories <extendedCategories>', 'extended categories import', true)\n  .option('--generateUniqueUrlKeys <generateUniqueUrlKeys>', 'make sure that category url keys are uniqe', true)\n  .action(async (cmd) => {\n    await reindexCategories(cmd.adapter, cmd.removeNonExistent, cmd.extendedCategories, cmd.generateUniqueUrlKeys);\n  });\n\nprogram\n  .command('cleanup')\n  .option('--adapter <adapter>', 'name of the adapter', 'magento')\n  .option('--cleanupType <cleanupType>', 'type of the entity to clean up: product|category', 'product')\n  .option('--transactionKey <transactionKey>', 'transaction key', 0)\n  .action((cmd) => {\n    cleanup(cmd.adapter, cmd.cleanupType, cmd.transactionKey);\n  });\n\nprogram\n  .command('fullreindex')\n  .option('--adapter <adapter>', 'name of the adapter', 'magento')\n  .option('--partitions <partitions>', 'number of partitions', 1)\n  .option('--partitionSize <partitionSize>', 'size of the partitions', 50)\n  .option('--initQueue <initQueue>', 'use the queue', true)\n  .option('--skus <skus>', 'comma delimited list of SKUs to fetch fresh informations from', '')\n  .option('--extendedCategories <extendedCategories>', 'extended categories import', true)\n  .option('--generateUniqueUrlKeys <generateUniqueUrlKeys>', 'generate unique url_keys', true)\n  .action((cmd) => {\n    fullReindex(cmd.adapter, true, cmd.partitions, cmd.partitionSize, cmd.initQueue, cmd.skus, cmd.extendedCategories, cmd.generateUniqueUrlKeys);\n  });\n\nprogram\n  .command('productcategories')\n  .option('--adapter <adapter>', 'name of the adapter', 'magento')\n  .action((cmd) => {\n    reindexProductCategories(cmd.adapter);\n  });\n\nprogram\n  .command('products')\n  .option('--adapter <adapter>', 'name of the adapter', 'magento')\n  .option('--partitions <partitions>', 'number of partitions', 1)\n  .option('--partitionSize <partitionSize>', 'size of the partitions', 50)\n  .option('--initQueue <initQueue>', 'use the queue', true)\n  .option('--skus <skus>', 'comma delimited list of SKUs to fetch fresh informations from', '')\n  .option('--removeNonExistent <removeNonExistent>', 'remove non existent products', false)\n  .option('--updatedAfter <updatedAfter>', 'timestamp to start the synchronization from', '')\n  .option('--page <page>', 'start from specific page', null)\n  .action((cmd) => {\n    if (cmd.updatedAfter) {\n      reindexProducts(cmd.adapter, cmd.removeNonExistent, cmd.partitions, cmd.partitionSize, cmd.initQueue, cmd.skus, new Date(cmd.updatedAfter), cmd.page);\n    } else {\n      reindexProducts(cmd.adapter, cmd.removeNonExistent, cmd.partitions, cmd.partitionSize, cmd.initQueue, cmd.skus, null, cmd.page);\n    }\n  });\n\nprogram\n  .command('productsdelta')\n  .option('--adapter <adapter>', 'name of the adapter', 'magento')\n  .option('--partitions <partitions>', 'number of partitions', 1)\n  .option('--partitionSize <partitionSize>', 'size of the partitions', 50)\n  .option('--initQueue <initQueue>', 'use the queue', true)\n  .option('--skus <skus>', 'comma delimited list of SKUs to fetch fresh informations from', '')\n  .option('--removeNonExistent <removeNonExistent>', 'remove non existent products', false)\n  .action((cmd) => {\n    let indexMeta = { lastIndexDate: new Date() }\n    let updatedAfter = null\n    try {\n      // Make sure so the temporary folder exists.\n      if (!fs.existsSync('tmp')) {\n        fs.mkdirSync('tmp')\n      }\n      indexMeta = jsonFile.readFileSync(INDEX_META_PATH)\n      updatedAfter = new Date(indexMeta.lastIndexDate)\n    } catch (err) {\n      console.log('Seems like first time run!')\n      updatedAfter = null // full reindex\n    }\n\n    reindexProducts(cmd.adapter, cmd.removeNonExistent, cmd.partitions, cmd.partitionSize, cmd.initQueue, cmd.skus, updatedAfter);\n\n    try {\n      indexMeta.lastIndexDate = new Date()\n      jsonFile.writeFile(INDEX_META_PATH, indexMeta)\n    } catch (err) {\n      console.log('Error writing index meta!', err)\n    }\n  })\n\nprogram\n  .command('productsworker')\n  .option('--adapter <adapter>', 'name of the adapter', 'magento')\n  .option('--partitions <partitions>', 'number of partitions', 1)\n  .action((cmd) => {\n    runProductsworker(cmd.adapter, cmd.partitions);\n  })\n      \nprogram\n  .command('reviews')\n  .option('--adapter <adapter>', 'name of the adapter', 'magento')\n  .option('--removeNonExistent <removeNonExistent>', 'remove non existent products', false)\n  .action(async (cmd) => {\n    await reindexReviews(cmd.adapter, cmd.removeNonExistent);\n  })\n  \nprogram\n  .command('taxrule')\n  .option('--adapter <adapter>', 'name of the adapter', 'magento')\n  .option('--removeNonExistent <removeNonExistent>', 'remove non existent products', false)\n  .action(async (cmd) => {\n    await reindexTaxRules(cmd.adapter, cmd.removeNonExistent);\n  })\n      \n/**\n* Sync cms blocks\n*/\nprogram\n  .command('blocks')\n  .option('--adapter <adapter>', 'name of the adapter', 'magento')\n  .option('--removeNonExistent <removeNonExistent>', 'remove non existent products', false)\n  .action(async (cmd) => {\n    await reindexBlocks(cmd.adapter, cmd.removeNonExistent);\n  })\n\nprogram\n  .command('pages')\n  .option('--adapter <adapter>', 'name of the adapter', 'magento')\n  .option('--removeNonExistent <removeNonExistent>', 'remove non existent products', false)\n  .action(async (cmd) => {\n    await reindexPages(cmd.adapter, cmd.removeNonExistent);\n  })\n\nprogram\n  .on('command:*', () => {\n    console.error('Invalid command: %s\\nSee --help for a list of available commands.', program.args.join(' '));\n    process.exit(1);\n  });\n\nprogram\n  .parse(process.argv);\n\nprocess\n  .on('unhandledRejection', (reason, p) => {\n    logger.error('Unhandled Rejection at: Promise', p, 'reason:', reason);\n    // application specific logging, throwing an error, or other logic here\n  });\n"
  },
  {
    "path": "src/config.js",
    "content": "const _slugify = require('./helpers/slugify')\n\nmodule.exports = {\n\n  seo: {\n    useUrlDispatcher: JSON.parse(process.env.SEO_USE_URL_DISPATCHER || true),\n    productUrlPathMapper: (product) => {\n      let destPath = ''\n      if (product.category && product.category.length > 0) {\n        for (let i = 0; i < product.category.length; i++) {\n          if (typeof product.category[i].name !== 'undefined') {\n            const firstValidCat = product.category[i]\n            destPath = (firstValidCat.path ? (firstValidCat.path) : _slugify(firstValidCat.name)) + '/' + (product.slug ? product.slug : _slugify(product.name + '-' + product.id))\n            break\n          }\n        }\n      }\n      if (destPath === '') {\n        destPath = (product.slug ? product.slug : _slugify(product.name + '-' + product.id))\n      }\n      destPath += '.html'\n      console.log('Dest. product path = ', destPath)\n      return destPath\n    },\n    categoryUrlPathMapper: (category) => {\n      const destSlug = (category.url_path ? category.url_path + '/': '') + category.url_key\n      console.log('Dest. cat path = ', destSlug)\n      return destSlug\n    },\n  },\n\n  magento: {\n    url: process.env.MAGENTO_URL || 'http://magento2.demo-1.divante.pl/rest/',\n    consumerKey: process.env.MAGENTO_CONSUMER_KEY || 'alva6h6hku9qxrpfe02c2jalopx7od1q',\n    consumerSecret: process.env.MAGENTO_CONSUMER_SECRET || '9tgfpgoojlx9tfy21b8kw7ssfu2aynpm',\n    accessToken: process.env.MAGENTO_ACCESS_TOKEN || 'rw5w0si9imbu45h3m9hkyrfr4gjina8q',\n    accessTokenSecret: process.env.MAGENTO_ACCESS_TOKEN_SECRET || '00y9dl4vpxgcef3gn5mntbxtylowjcc9',\n    storeId: process.env.MAGENTO_STORE_ID || 1,\n    currencyCode: process.env.MAGENTO_CURRENCY_CODE || 'USD',\n    msi: { enabled: process.env.MAGENTO_MSI_ENABLED || false, stockId: process.env.MAGENTO_MSI_STOCK_ID || 1 }\n  },\n\n  vuestorefront: {\n    invalidateCache: JSON.parse(typeof process.env.VS_INVALIDATE_CACHE === 'undefined' ? false : process.env.VS_INVALIDATE_CACHE),\n    invalidateCacheUrl: process.env.VS_INVALIDATE_CACHE_URL || 'http://localhost:3000/invalidate?key=aeSu7aip&tag='\n  },\n\n  product: {\n    expandConfigurableFilters: ['manufacturer'],\n    synchronizeCatalogSpecialPrices: process.env.PRODUCTS_SPECIAL_PRICES || true,\n    renderCatalogRegularPrices: process.env.PRODUCTS_RENDER_PRICES || false,\n    excludeDisabledProducts: process.env.PRODUCTS_EXCLUDE_DISABLED || false\n  },\n\n  kue: {}, // default KUE config works on local redis instance. See KUE docs for non standard redis connections\n\n  db: {\n    driver: 'elasticsearch',\n    url: process.env.DATABASE_URL || 'http://localhost:9200',\n    indexName: process.env.INDEX_NAME || 'vue_storefront_catalog'\n  },\n\n  elasticsearch: {\n    apiVersion: process.env.ELASTICSEARCH_API_VERSION || '5.6'\n  },\n\n  redis: {\n    host: process.env.REDIS_HOST || '127.0.0.1',\n    port: process.env.REDIS_PORT || 6379,\n    auth: process.env.REDIS_AUTH || false,\n    db: process.env.REDIS_DB || 0\n  },\n\n  passport: {\n    jwtSecret: \"MyS3cr3tK3Y\",\n    jwtSession: {\n      session: false\n    }\n  }\n\n}\n"
  },
  {
    "path": "src/helpers/slugify.js",
    "content": "const removeAccents = require('remove-accents').remove\n\n/**\n * Create slugify -> \"create-slugify\" permalink  of text\n * @param {String} text\n * @returns {String}\n */\nfunction slugify (text) {\n  // to be sure that text is String\n  text = text.toString()\n\n  // remove regional characters\n  text = removeAccents(text)\n\n  return text\n    .toLowerCase()\n    .replace(/\\s+/g, '-') // Replace spaces with -\n    .replace(/&/g, '-and-') // Replace & with 'and'\n    .replace(/[^\\w-]+/g, '') // Remove all non-word chars\n    .replace(/--+/g, '-') // Replace multiple - with single -\n}\n\nmodule.exports = slugify\n"
  },
  {
    "path": "src/log.js",
    "content": "var winston = require('winston');\n\nwinston.emitErrs = true;\n\nif(!global.logger) {\n  global.logger = new winston.Logger({\n    transports: [\n      new winston.transports.Console({\n        level: 'debug',\n        handleExceptions: false,\n        json: false,\n        prettyPrint: true,\n        colorize: true,\n        timestamp: true\n      })\n    ],\n    exitOnError: false\n  });\n}\n\nmodule.exports = global.logger;\n"
  },
  {
    "path": "src/test_by_sku.sh",
    "content": "\nexport TIME_TO_EXIT=2000\nexport VS_INVALIDATE_CACHE_URL=http://localhost:3000/invalidate?key=aeSu7aip&tag=\nexport VS_INVALIDATE_CACHE=1\nexport PRODUCTS_SPECIAL_PRICES=true\nexport MAGENTO_CONSUMER_KEY=byv3730rhoulpopcq64don8ukb8lf2gq\nexport MAGENTO_CONSUMER_SECRET=u9q4fcobv7vfx9td80oupa6uhexc27rb\nexport MAGENTO_ACCESS_TOKEN=040xx3qy7s0j28o3q0exrfop579cy20m\nexport MAGENTO_ACCESS_TOKEN_SECRET=7qunl3p505rubmr7u1ijt7odyialnih9\n\necho 'Default store - in our case United States / en'\nexport MAGENTO_URL=http://demo-magento2.vuestorefront.io/rest\n\nnode --harmony cli.js products --partitions=1 --skus=WS01-XS-BLACK\n\n"
  },
  {
    "path": "src/test_categoryextended.sh",
    "content": "\nexport TIME_TO_EXIT=2000\nexport VS_INVALIDATE_CACHE_URL=http://localhost:3000/invalidate?key=aeSu7aip&tag=\nexport VS_INVALIDATE_CACHE=1\nexport MAGENTO_CONSUMER_KEY=byv3730rhoulpopcq64don8ukb8lf2gq\nexport MAGENTO_CONSUMER_SECRET=u9q4fcobv7vfx9td80oupa6uhexc27rb\nexport MAGENTO_ACCESS_TOKEN=040xx3qy7s0j28o3q0exrfop579cy20m\nexport MAGENTO_ACCESS_TOKEN_SECRET=7qunl3p505rubmr7u1ijt7odyialnih9\n\necho 'Default store - in our case United States / en'\nexport MAGENTO_URL=http://demo-magento2.vuestorefront.io/rest\n\nnode --harmony cli.js categories --removeNonExistent=true --extendedCategories=true\n"
  },
  {
    "path": "src/test_fullreindex.sh",
    "content": "\nexport TIME_TO_EXIT=2000\nexport VS_INVALIDATE_CACHE_URL=http://localhost:3000/invalidate?key=aeSu7aip&tag=\nexport VS_INVALIDATE_CACHE=1\nexport PRODUCTS_SPECIAL_PRICES=true\nexport MAGENTO_CONSUMER_KEY=byv3730rhoulpopcq64don8ukb8lf2gq\nexport MAGENTO_CONSUMER_SECRET=u9q4fcobv7vfx9td80oupa6uhexc27rb\nexport MAGENTO_ACCESS_TOKEN=040xx3qy7s0j28o3q0exrfop579cy20m\nexport MAGENTO_ACCESS_TOKEN_SECRET=7qunl3p505rubmr7u1ijt7odyialnih9\nexport SEO_USE_URL_DISPATCHER=1\n\necho 'Default store - in our case United States / en'\nexport MAGENTO_URL=http://demo-magento2.vuestorefront.io/rest\n\nnode --harmony cli.js blocks\nnode --harmony cli.js pages \nnode --harmony cli.js reviews\nnode --harmony cli.js categories --removeNonExistent=true --extendedCategories=true\nnode --harmony cli.js productcategories\nnode --harmony cli.js attributes --removeNonExistent=true\nnode --harmony cli.js taxrule --removeNonExistent=true\nnode --harmony cli.js products --removeNonExistent=true --partitions=1\n"
  },
  {
    "path": "src/test_fullreindex_de.sh",
    "content": "\nexport TIME_TO_EXIT=2000\nexport VS_INVALIDATE_CACHE_URL=http://localhost:3000/invalidate?key=aeSu7aip&tag=\nexport VS_INVALIDATE_CACHE=1\nexport PRODUCTS_SPECIAL_PRICES=true\nexport MAGENTO_CONSUMER_KEY=byv3730rhoulpopcq64don8ukb8lf2gq\nexport MAGENTO_CONSUMER_SECRET=u9q4fcobv7vfx9td80oupa6uhexc27rb\nexport MAGENTO_ACCESS_TOKEN=040xx3qy7s0j28o3q0exrfop579cy20m\nexport MAGENTO_ACCESS_TOKEN_SECRET=7qunl3p505rubmr7u1ijt7odyialnih9\nexport SEO_USE_URL_DISPATCHER=1\n\necho 'Italian store - de'\nexport MAGENTO_URL=http://demo-magento2.vuestorefront.io/rest/de\nexport INDEX_NAME=vue_storefront_catalog_de\n\nnode --harmony cli.js blocks\nnode --harmony cli.js pages\nnode --harmony cli.js reviews\nnode --harmony cli.js categories --removeNonExistent=true --extendedCategories=true\nnode --harmony cli.js productcategories\nnode --harmony cli.js attributes --removeNonExistent=true\nnode --harmony cli.js taxrule --removeNonExistent=true\nnode --harmony cli.js products --removeNonExistent=true --partitions=1\n\n\n\n"
  },
  {
    "path": "src/test_fullreindex_it.sh",
    "content": "\nexport TIME_TO_EXIT=2000\nexport VS_INVALIDATE_CACHE_URL=http://localhost:3000/invalidate?key=aeSu7aip&tag=\nexport VS_INVALIDATE_CACHE=1\nexport PRODUCTS_SPECIAL_PRICES=true\nexport MAGENTO_CONSUMER_KEY=byv3730rhoulpopcq64don8ukb8lf2gq\nexport MAGENTO_CONSUMER_SECRET=u9q4fcobv7vfx9td80oupa6uhexc27rb\nexport MAGENTO_ACCESS_TOKEN=040xx3qy7s0j28o3q0exrfop579cy20m\nexport MAGENTO_ACCESS_TOKEN_SECRET=7qunl3p505rubmr7u1ijt7odyialnih9\nexport SEO_USE_URL_DISPATCHER=1\n\necho 'Italian store - it'\nexport MAGENTO_URL=http://demo-magento2.vuestorefront.io/rest/it\nexport INDEX_NAME=vue_storefront_catalog_it\n\nnode --harmony cli.js blocks\nnode --harmony cli.js pages\nnode --harmony cli.js reviews\nnode --harmony cli.js categories --removeNonExistent=true --extendedCategories=true\nnode --harmony cli.js productcategories\nnode --harmony cli.js attributes --removeNonExistent=true\nnode --harmony cli.js taxrule --removeNonExistent=true\nnode --harmony cli.js products --removeNonExistent=true --partitions=1\n\n\n\n"
  },
  {
    "path": "src/test_fullreindex_multiprocess.sh",
    "content": "\nexport TIME_TO_EXIT=2000\nexport VS_INVALIDATE_CACHE_URL=http://localhost:3000/invalidate?key=aeSu7aip&tag=\nexport VS_INVALIDATE_CACHE=1\nexport PRODUCTS_SPECIAL_PRICES=true\nexport MAGENTO_CONSUMER_KEY=byv3730rhoulpopcq64don8ukb8lf2gq\nexport MAGENTO_CONSUMER_SECRET=u9q4fcobv7vfx9td80oupa6uhexc27rb\nexport MAGENTO_ACCESS_TOKEN=040xx3qy7s0j28o3q0exrfop579cy20m\nexport MAGENTO_ACCESS_TOKEN_SECRET=7qunl3p505rubmr7u1ijt7odyialnih9\n\necho 'Default store - in our case United States / en'\nexport MAGENTO_URL=http://demo-magento2.vuestorefront.io/rest\n\nnode --harmony cli.js blocks\nnode --harmony cli.js pages \nnode --harmony cli.js reviews\nnode --harmony cli.js categories --removeNonExistent=true --extendedCategories=true\nnode --harmony cli.js productcategories\nnode --harmony cli.js attributes --removeNonExistent=true\nnode --harmony cli.js taxrule --removeNonExistent=true\nnode --harmony cli.js products --partitions=4\n"
  },
  {
    "path": "src/test_multistore.sh",
    "content": "\nexport TIME_TO_EXIT=2000\nexport VS_INVALIDATE_CACHE_URL=http://localhost:3000/invalidate?key=aeSu7aip&tag=\nexport VS_INVALIDATE_CACHE=1\nexport PRODUCTS_SPECIAL_PRICES=true\nexport MAGENTO_CONSUMER_KEY=byv3730rhoulpopcq64don8ukb8lf2gq\nexport MAGENTO_CONSUMER_SECRET=u9q4fcobv7vfx9td80oupa6uhexc27rb\nexport MAGENTO_ACCESS_TOKEN=040xx3qy7s0j28o3q0exrfop579cy20m\nexport MAGENTO_ACCESS_TOKEN_SECRET=7qunl3p505rubmr7u1ijt7odyialnih9\nexport SEO_USE_URL_DISPATCHER=1\n\necho 'German store - de'\nexport MAGENTO_URL=http://demo-magento2.vuestorefront.io/rest/de\nexport INDEX_NAME=vue_storefront_catalog_de\n\nnode --harmony cli.js blocks\nnode --harmony cli.js pages \nnode --harmony cli.js reviews\nnode --harmony cli.js categories --removeNonExistent=true --extendedCategories=true\nnode --harmony cli.js productcategories\nnode --harmony cli.js attributes --removeNonExistent=true\nnode --harmony cli.js taxrule --removeNonExistent=true\nnode --harmony cli.js products --removeNonExistent=true --partitions=1\n\n\necho 'Italian store - it'\nexport MAGENTO_URL=http://demo-magento2.vuestorefront.io/rest/it\nexport INDEX_NAME=vue_storefront_catalog_it\n\nnode --harmony cli.js blocks\nnode --harmony cli.js pages \nnode --harmony cli.js reviews\nnode --harmony cli.js categories --removeNonExistent=true --extendedCategories=true\nnode --harmony cli.js productcategories\nnode --harmony cli.js attributes --removeNonExistent=true\nnode --harmony cli.js taxrule --removeNonExistent=true\nnode --harmony cli.js products --removeNonExistent=true --partitions=1\n\n\n\necho 'Default store - in our case United States / en'\nexport MAGENTO_URL=http://demo-magento2.vuestorefront.io/rest\nexport INDEX_NAME=vue_storefront_catalog\n\nnode --harmony cli.js blocks\nnode --harmony cli.js pages \nnode --harmony cli.js reviews\nnode --harmony cli.js categories --removeNonExistent=true --extendedCategories=true\nnode --harmony cli.js productcategories\nnode --harmony cli.js attributes --removeNonExistent=true\nnode --harmony cli.js taxrule --removeNonExistent=true\nnode --harmony cli.js products --removeNonExistent=true --partitions=1\n"
  },
  {
    "path": "src/test_product.sh",
    "content": "\nexport TIME_TO_EXIT=2000\nexport VS_INVALIDATE_CACHE_URL=http://localhost:3000/invalidate?key=aeSu7aip&tag=\nexport VS_INVALIDATE_CACHE=1\nexport PRODUCTS_SPECIAL_PRICES=true\nexport MAGENTO_CONSUMER_KEY=byv3730rhoulpopcq64don8ukb8lf2gq\nexport MAGENTO_CONSUMER_SECRET=u9q4fcobv7vfx9td80oupa6uhexc27rb\nexport MAGENTO_ACCESS_TOKEN=040xx3qy7s0j28o3q0exrfop579cy20m\nexport MAGENTO_ACCESS_TOKEN_SECRET=7qunl3p505rubmr7u1ijt7odyialnih9\n\necho 'Default store - in our case United States / en'\nexport MAGENTO_URL=http://demo-magento2.vuestorefront.io/rest\n\nnode --harmony cli.js products --removeNonExistent=true --partitions=1\n"
  },
  {
    "path": "src/test_product_delta.sh",
    "content": "\nexport TIME_TO_EXIT=2000\nexport VS_INVALIDATE_CACHE_URL=http://localhost:3000/invalidate?key=aeSu7aip&tag=\nexport VS_INVALIDATE_CACHE=1\nexport PRODUCTS_SPECIAL_PRICES=true\nexport MAGENTO_CONSUMER_KEY=byv3730rhoulpopcq64don8ukb8lf2gq\nexport MAGENTO_CONSUMER_SECRET=u9q4fcobv7vfx9td80oupa6uhexc27rb\nexport MAGENTO_ACCESS_TOKEN=040xx3qy7s0j28o3q0exrfop579cy20m\nexport MAGENTO_ACCESS_TOKEN_SECRET=7qunl3p505rubmr7u1ijt7odyialnih9\n\necho 'Default store - in our case United States / en'\nexport MAGENTO_URL=http://demo-magento2.vuestorefront.io/rest\n\nnode --harmony cli.js productsdelta\n\n"
  },
  {
    "path": "src/test_product_msi.sh",
    "content": "\nexport TIME_TO_EXIT=2000\nexport VS_INVALIDATE_CACHE_URL=http://localhost:3000/invalidate?key=aeSu7aip&tag=\nexport VS_INVALIDATE_CACHE=1\nexport PRODUCTS_SPECIAL_PRICES=true\nexport MAGENTO_MSI_ENABLED=true\nexport MAGENTO_MSI_STOCK_ID=1\nexport MAGENTO_CONSUMER_KEY=byv3730rhoulpopcq64don8ukb8lf2gq\nexport MAGENTO_CONSUMER_SECRET=u9q4fcobv7vfx9td80oupa6uhexc27rb\nexport MAGENTO_ACCESS_TOKEN=040xx3qy7s0j28o3q0exrfop579cy20m\nexport MAGENTO_ACCESS_TOKEN_SECRET=7qunl3p505rubmr7u1ijt7odyialnih9\n\necho 'Default store - in our case United States / en'\nexport MAGENTO_URL=http://demo-magento2.vuestorefront.io/rest\n\nnode --harmony cli.js products --removeNonExistent=true --partitions=1\n"
  },
  {
    "path": "src/test_product_worker.sh",
    "content": "\nexport TIME_TO_EXIT=2000\nexport VS_INVALIDATE_CACHE_URL=http://localhost:3000/invalidate?key=aeSu7aip&tag=\nexport VS_INVALIDATE_CACHE=1\nexport PRODUCTS_SPECIAL_PRICES=true\nexport MAGENTO_CONSUMER_KEY=byv3730rhoulpopcq64don8ukb8lf2gq\nexport MAGENTO_CONSUMER_SECRET=u9q4fcobv7vfx9td80oupa6uhexc27rb\nexport MAGENTO_ACCESS_TOKEN=040xx3qy7s0j28o3q0exrfop579cy20m\nexport MAGENTO_ACCESS_TOKEN_SECRET=7qunl3p505rubmr7u1ijt7odyialnih9\n\necho 'Default store - in our case United States / en'\nexport MAGENTO_URL=http://demo-magento2.vuestorefront.io/rest\n\nnode --harmony cli.js productsworker\n\n"
  },
  {
    "path": "src/tmp/.gitignore",
    "content": "*.json\n\n"
  },
  {
    "path": "src/webapi.js",
    "content": "'use strict';\n\n/**\n * Webhook API to add specific products or categories to be synchronized by the service\n */\n\nlet logger = require('./log');\nvar express = require('express');\nvar app = express();\nvar body_parser = require('body-parser');\n\n// configure app to use bodyParser()\n// this will let us get the data from a POST\napp.use(body_parser.urlencoded({ extended: true }));\napp.use(body_parser.json());\n\nvar port = process.env.PORT || 8080; // set our port\napp.use('/magento', require('./api/routes/magento'));\n\napp.listen(port);\nlogger.info('Magic happens on port ' + port);\n"
  }
]