Repository: postmanlabs/newman Branch: develop Commit: 942d1390f81e Files: 223 Total size: 804.4 KB Directory structure: gitextract_6f7i_n3s/ ├── .dockerignore ├── .editorconfig ├── .eslintrc ├── .gitattributes ├── .github/ │ ├── CONTRIBUTING.md │ ├── ISSUE_TEMPLATE.md │ ├── dependabot.yml │ └── workflows/ │ ├── ci.yml │ └── codeql-analysis.yml ├── .gitignore ├── .npmignore ├── .nycrc.js ├── CHANGELOG.yaml ├── LICENSE.md ├── MIGRATION.md ├── README.md ├── SECURITY.md ├── bin/ │ ├── newman.js │ └── util.js ├── codecov.yml ├── docker/ │ ├── README.md │ └── images/ │ ├── alpine/ │ │ ├── Dockerfile │ │ └── README.md │ └── ubuntu/ │ ├── Dockerfile │ └── README.md ├── examples/ │ ├── find-unique-urls-in-run.js │ ├── parallel-collection-runs.js │ ├── read-collection-from-file.js │ ├── reports/ │ │ ├── sample-collection-report.json │ │ └── sample-collection-report.xml │ ├── run-collections-in-directory.js │ ├── sample-collection.json │ ├── ssl-client-cert-list.json │ └── v1.postman_collection.json ├── index.js ├── lib/ │ ├── config/ │ │ ├── index.js │ │ ├── process-env.js │ │ └── rc-file.js │ ├── index.js │ ├── node-version-check/ │ │ └── index.js │ ├── print/ │ │ └── index.js │ ├── reporters/ │ │ ├── cli/ │ │ │ ├── cli-utils-symbols.js │ │ │ ├── cli-utils.js │ │ │ └── index.js │ │ ├── emojitrain.js │ │ ├── json/ │ │ │ └── index.js │ │ ├── junit/ │ │ │ └── index.js │ │ └── progress.js │ ├── run/ │ │ ├── export-file.js │ │ ├── index.js │ │ ├── options.js │ │ ├── secure-fs.js │ │ └── summary.js │ └── util.js ├── npm/ │ ├── create-release.js │ ├── docker-publish.sh │ ├── server.js │ ├── test-cli.js │ ├── test-integration.js │ ├── test-library.js │ ├── test-lint.js │ ├── test-system.js │ └── test-unit.js ├── package.json └── test/ ├── .eslintrc ├── cli/ │ ├── cli-reporter-failure-details.test.js │ ├── cli-reporter-no-success-assertsions.test.js │ ├── cli-reporter-show-timestamp.test.js │ ├── color-tty.test.js │ ├── export-environment.test.js │ ├── export-globals.test.js │ ├── iteration-count.test.js │ ├── run-options.test.js │ ├── shallow-junit-reporter.test.js │ ├── ssl-client-cert.test.js │ ├── suppress-exit-code.test.js │ ├── verbose.test.js │ └── working-directory.test.js ├── fixtures/ │ ├── files/ │ │ ├── outside.json │ │ ├── ssl-client-cert-config.json │ │ └── work-dir/ │ │ └── test.json │ ├── overrides/ │ │ ├── env.json │ │ ├── failing-collection.json │ │ └── pmcollection.json │ ├── run/ │ │ ├── failed-request.json │ │ ├── nested-requests-fail-report-test.json │ │ ├── nested-requests-report-test.json │ │ ├── newman-report-test.json │ │ ├── response-bodies.json │ │ ├── simple-variables.json │ │ ├── single-file-inside.json │ │ ├── single-file-outside.json │ │ ├── single-get-request.json │ │ ├── single-request-failing.json │ │ ├── spaces/ │ │ │ ├── data.json │ │ │ ├── simple-cookie-jar.json │ │ │ ├── simple-variables.json │ │ │ └── single-get-request.json │ │ ├── ssl-client-cert-list.json │ │ ├── ssl-client-cert.json │ │ ├── test-data.postman_data.json │ │ └── undefined-test-checks.json │ └── ssl/ │ ├── SSL.md │ ├── ca.crt │ ├── ca.key │ ├── ca2.crt │ ├── ca2.key │ ├── ca3.crt │ ├── ca3.key │ ├── client.crt │ ├── client.csr │ ├── client.key │ ├── client.p12 │ ├── client2.crt │ ├── client2.csr │ ├── client2.key │ ├── client3.crt │ ├── client3.csr │ ├── client3.key │ ├── config.cnf │ ├── generate-certificates.sh │ ├── server.crt │ ├── server.csr │ ├── server.key │ ├── server2.crt │ ├── server2.csr │ ├── server2.key │ ├── server3.crt │ ├── server3.csr │ ├── server3.key │ ├── sslClientCertList.json │ └── v3.ext ├── integration/ │ ├── bom-and-encoding/ │ │ ├── bom-in-collection-file.postman_collection.json │ │ ├── bom-in-environment-file.postman_collection.json │ │ ├── bom-in-environment-file.postman_environment.json │ │ ├── utf16-le-encoding-in-env.postman_collection.json │ │ └── utf16-le-encoding-in-env.postman_environment.json │ ├── case-insen-header-sandbox.postman_collection.json │ ├── clear-vars-sandbox.postman_collection.json │ ├── comma-test-csv/ │ │ ├── comma-test-csv.postman_collection.json │ │ └── comma-test-csv.postman_data.csv │ ├── cookie-jar.postman_collection.json │ ├── crypto-md5.postman_collection.json │ ├── csv-with-bom/ │ │ ├── csv-with-bom.postman_collection.json │ │ └── csv-with-bom.postman_data.csv │ ├── custom-http-method/ │ │ ├── custom-http-method.postman_collection.json │ │ └── upload-file.csv │ ├── data-file-var-replacement/ │ │ ├── data-file-var-replacement.postman_collection.json │ │ ├── data-file-var-replacement.postman_data.json │ │ └── data-file-var-replacement.postman_environment.json │ ├── disabled-and-duplicate-variables/ │ │ ├── disabled-and-duplicates.postman_collection.json │ │ └── disabled-and-duplicates.postman_environment.json │ ├── distinct-random-int.postman_collection.json │ ├── dynamic-var-replacement.postman_collection.json │ ├── echo-v2.postman_collection.json │ ├── esc-formdata/ │ │ ├── esc-formdata.postman_collection.json │ │ └── esc-formdata.postman_environment.json │ ├── globals-env-data-files/ │ │ ├── globals-env-data-files.postman_collection.json │ │ ├── globals-env-data-files.postman_data.json │ │ ├── globals-env-data-files.postman_environment.json │ │ └── globals-env-data-files.postman_globals.json │ ├── hawk-auth-test.postman_collection.json │ ├── head-requests.postman_collection.json │ ├── helper.postman_collection.json │ ├── inherited-entities/ │ │ ├── inherited-entities.postman_collection.json │ │ └── inherited-entities.postman_environment.json │ ├── multi-level-folders-v1.postman_collection.json │ ├── multi-level-folders-v2.postman_collection.json │ ├── multi-value-data.postman_collection.json │ ├── multiple-form-values.postman_collection.json │ ├── newman-gzip-test.postman_collection.json │ ├── oauth1-var-in-url-params.postman_collection.json │ ├── protocol-profile-behavior.postman_collection.json │ ├── prototype-check.postman_collection.json │ ├── redirect-test/ │ │ ├── redirect-test.postman_collection.json │ │ ├── redirect-test.postman_config.json │ │ └── redirect-with-body.postman_collection.json │ ├── request-body-with-get.postman_collection.json │ ├── request-chaining-test.postman_collection.json │ ├── request-name-in-script.postman_collection.json │ ├── sandbox-libraries.postman_collection.json │ ├── semicolon-tests.postman_collection.json │ ├── set-next-request.postman_collection.json │ ├── slashed-variable-names/ │ │ ├── slashed-variable-names.postman_collection.json │ │ └── slashed-variable-names.postman_environment.json │ ├── steph/ │ │ ├── steph.postman_collection.json │ │ └── steph.postman_data.json │ ├── super-sandbox-test.postman_collection.json │ ├── tc4/ │ │ ├── tc4.postman_collection.json │ │ └── upload-file.json │ ├── timeout/ │ │ ├── timeout.postman_collection.json │ │ └── timeout.postman_config.json │ ├── v2-regression-tests/ │ │ ├── v2-regression-tests.postman_collection.json │ │ ├── v2-regression-tests.postman_config.json │ │ └── v2-regression-tests.postman_environment.json │ ├── var-replacement.postman_collection.json │ └── whatwg-url.postman_collection.json ├── library/ │ ├── cookie-jar.test.js │ ├── export-environment.test.js │ ├── export-globals.test.js │ ├── folder-variants.test.js │ ├── iteration-count.test.js │ ├── postman-api-key.test.js │ ├── requestAgents.test.js │ ├── run-options.test.js │ ├── shallow-junit-reporter.test.js │ ├── ssl-client-cert.test.js │ ├── suppress-exit-code.test.js │ └── working-directory.test.js ├── system/ │ ├── dockerfile_rules.yml │ ├── dockerfiles.test.js │ ├── editorconfig.test.js │ ├── npm-publish.test.js │ └── repository.test.js └── unit/ ├── cli-reporter-symbols.test.js ├── cli.test.js ├── defaultReporter.test.js ├── externalReporter.test.js ├── options.test.js ├── run-summary.test.js ├── run.test.js ├── secure-fs.test.js └── util.test.js ================================================ FILE CONTENTS ================================================ ================================================ FILE: .dockerignore ================================================ # PLATFORM # ======== # All exclusions that are specific to the NPM, GIT, IDE and Operating Systems. # - Do not allow installed node modules to be committed. Doing `npm install -d` will bring them in root or other places. node_modules # - Do not commit any log file from anywhere *.log *.log.* # - Prevent addition of OS specific file explorer files Thumbs.db .DS_Store # Prevent IDE stuff .idea .vscode # PROJECT # ======= # Configuration pertaining to project specific repository structure. # - Prevent Sublime text IDE files from being committed to repository *.sublime-* # - Allow sublime text project file to be committed in the development directory. !/develop/*.sublime-project # - Prevent CI output files from being Added /out/ /newman/ # - Prevent diff backups from SourceTree from showing as commit. *.BACKUP.* *.BASE.* *.LOCAL.* *.REMOTE.* *.orig # Prevent unit test coverage reports from being added .coverage ================================================ FILE: .editorconfig ================================================ # editorconfig.org root = true [*] indent_style = space indent_size = 4 end_of_line = lf charset = utf-8 max_length = 120 trim_trailing_whitespace = true insert_final_newline = true [*.{json, yml, html, hbs}] indent_size = 2 ================================================ FILE: .eslintrc ================================================ { "plugins": ["jsdoc", "lodash", "security"], "env": { "browser": false, "node": true, "es6": true }, "globals": {}, "settings": { "lodash": { "pragma": "_" } }, "parserOptions": { "ecmaVersion": 2018 }, "rules": { // Possible Errors "for-direction": "error", "default-param-last": "error", "function-call-argument-newline": "off", "getter-return": "error", "no-async-promise-executor": "error", "no-await-in-loop": "error", "no-compare-neg-zero": "error", "no-cond-assign": "error", "no-console": [ "error", { "allow": ["info", "warn", "error"] } ], "no-constant-condition": "error", "no-control-regex": "error", "no-constructor-return": "error", "no-debugger": "error", "no-dupe-args": "error", "no-dupe-else-if": "error", "no-dupe-keys": "error", "no-duplicate-case": "error", "no-empty": "error", "no-empty-character-class": "error", "no-ex-assign": "error", "no-extra-boolean-cast": "error", "no-extra-parens": "off", "no-extra-semi": "error", "no-func-assign": "error", "no-import-assign": "error", "no-inner-declarations": "error", "no-invalid-regexp": "error", "no-irregular-whitespace": "error", "no-loss-of-precision": "error", "no-misleading-character-class": "error", "no-nonoctal-decimal-escape": "error", "no-obj-calls": "error", "no-prototype-builtins": "error", "no-regex-spaces": "error", "no-sparse-arrays": "error", "no-template-curly-in-string": "error", "no-unexpected-multiline": "error", "no-unreachable": "error", "no-unsafe-finally": "error", "no-unsafe-negation": "error", "no-unsafe-optional-chaining": "error", "prefer-regex-literals": "error", "require-atomic-updates": "error", "require-unicode-regexp": "off", "use-isnan": "error", "valid-typeof": "error", // Best Practices "accessor-pairs": "error", "array-callback-return": "off", "block-scoped-var": "error", "class-methods-use-this": "error", "complexity": "off", "consistent-return": "off", "curly": "error", "default-case": "error", "default-case-last": "error", "dot-location": ["error", "property"], "dot-notation": "error", "eqeqeq": "error", "grouped-accessor-pairs": "warn", "guard-for-in": "warn", "id-denylist": "warn", "max-classes-per-file": ["error", 1], "max-lines-per-function": "off", "no-alert": "error", "no-caller": "error", "no-case-declarations": "error", "no-div-regex": "error", "no-else-return": "error", "no-empty-function": "error", "no-empty-pattern": "error", "no-eq-null": "error", "no-eval": "error", "no-extend-native": "error", "no-extra-bind": "error", "no-extra-label": "error", "no-fallthrough": "error", "no-floating-decimal": "error", "no-global-assign": "error", "no-implicit-coercion": "error", "no-implicit-globals": "error", "no-implied-eval": "error", "no-invalid-this": "error", "no-iterator": "error", "no-labels": "error", "no-lone-blocks": "error", "no-loop-func": "error", "no-magic-numbers": "off", "no-multi-spaces": "error", "no-multi-str": "error", "no-new": "error", "no-new-func": "error", "no-new-wrappers": "error", "no-octal": "error", "no-octal-escape": "error", "no-param-reassign": "off", "no-promise-executor-return": "error", "no-proto": "error", "no-redeclare": "error", "no-restricted-exports": "error", "no-restricted-properties": "error", "no-return-assign": "error", "no-return-await": "error", "no-script-url": "error", "no-self-assign": "error", "no-self-compare": "error", "no-sequences": "error", "no-setter-return": "error", "no-throw-literal": "error", "no-unmodified-loop-condition": "error", "no-unreachable-loop": "error", "no-unused-expressions": "off", "no-unused-labels": "error", "no-useless-call": "error", "no-useless-concat": "error", "no-useless-backreference": "error", "no-useless-escape": "error", "no-useless-return": "error", "no-void": "error", "no-warning-comments": "off", "no-with": "error", "prefer-named-capture-group": "off", "prefer-exponentiation-operator": "warn", "prefer-object-spread": "error", "prefer-promise-reject-errors": "error", "radix": "error", "require-await": "error", "vars-on-top": "off", "wrap-iife": "error", "yoda": "error", // Strict Mode "strict": "off", // Variables "init-declarations": "off", "no-delete-var": "error", "no-label-var": "error", "no-restricted-globals": "error", "no-shadow": "off", "no-shadow-restricted-names": "error", "no-undef": "error", "no-undef-init": "error", "no-undefined": "off", "no-unused-vars": "error", "no-use-before-define": "error", // Node.js and CommonJS "callback-return": "error", "global-require": "off", "handle-callback-err": "error", "no-buffer-constructor": "error", "no-mixed-requires": "off", "no-new-require": "off", "no-path-concat": "error", "no-process-env": "error", "no-process-exit": "off", "no-restricted-modules": "error", "no-sync": "off", // Stylistic Issues "array-bracket-newline": "off", "array-bracket-spacing": "error", "array-element-newline": "off", "block-spacing": "error", "brace-style": [ "error", "stroustrup", { "allowSingleLine": true } ], "camelcase": "off", "capitalized-comments": "off", "comma-dangle": ["error", "never"], "comma-spacing": [ "error", { "before": false, "after": true } ], "comma-style": ["error", "last"], "computed-property-spacing": "error", "consistent-this": "off", "eol-last": "error", "func-call-spacing": "error", "func-name-matching": "off", "func-names": "off", "func-style": "off", "function-paren-newline": ["error", "never"], "id-blacklist": "error", "id-length": "off", "id-match": "error", "implicit-arrow-linebreak": ["error", "beside"], "indent": [ "error", 4, { "VariableDeclarator": { "var": 1, "let": 1, "const": 1 }, "SwitchCase": 1 } ], "jsx-quotes": ["error", "prefer-single"], "key-spacing": "error", "keyword-spacing": "error", "line-comment-position": "off", "linebreak-style": ["error", "unix"], "lines-around-comment": [ "error", { "beforeBlockComment": true, "afterBlockComment": false, "beforeLineComment": false, "afterLineComment": false, "allowBlockStart": true, "allowBlockEnd": false, "allowObjectStart": true, "allowObjectEnd": false, "allowArrayStart": true, "allowArrayEnd": false } ], "lines-between-class-members": [ "error", "always", { "exceptAfterSingleLine": true } ], "max-depth": "error", "max-len": [ "error", { "code": 120 } ], "max-lines": "off", "max-nested-callbacks": "error", "max-params": "off", "max-statements": "off", "max-statements-per-line": [ "error", { "max": 2 } ], "multiline-comment-style": "off", "multiline-ternary": "off", "new-cap": "off", "new-parens": "error", "newline-per-chained-call": [ "error", { "ignoreChainWithDepth": 4 } ], "no-array-constructor": "error", "no-bitwise": "off", "no-continue": "off", "no-inline-comments": "off", "no-lonely-if": "error", "no-mixed-operators": "off", "no-mixed-spaces-and-tabs": "error", "no-multi-assign": "off", "no-multiple-empty-lines": "error", "no-negated-condition": "off", "no-nested-ternary": "off", "no-new-object": "error", "no-plusplus": "off", "no-restricted-syntax": "error", "no-tabs": "error", "no-ternary": "off", "no-trailing-spaces": "error", "no-underscore-dangle": "off", "no-unneeded-ternary": "error", "no-whitespace-before-property": "error", "nonblock-statement-body-position": "error", "object-curly-newline": "off", "object-curly-spacing": ["error", "always"], "object-property-newline": "off", "one-var": ["error", "always"], "one-var-declaration-per-line": "error", "operator-assignment": "error", "operator-linebreak": ["error", "after"], "padded-blocks": ["error", "never"], "padding-line-between-statements": [ "error", { "blankLine": "always", "prev": "*", "next": "return" }, { "blankLine": "always", "prev": ["const", "let", "var"], "next": "*" }, { "blankLine": "any", "prev": ["const", "let", "var"], "next": ["const", "let", "var"] } ], "quote-props": ["error", "as-needed"], "quotes": ["error", "single"], "semi": "error", "semi-spacing": "error", "semi-style": ["error", "last"], "sort-keys": "off", "sort-vars": "off", "space-before-blocks": "error", "space-before-function-paren": "error", "space-in-parens": "error", "space-infix-ops": "error", "space-unary-ops": "error", "spaced-comment": [ "error", "always", { "block": { "exceptions": ["!"] } } ], "switch-colon-spacing": "error", "template-tag-spacing": "error", "unicode-bom": "error", "wrap-regex": "error", // ECMAScript 6 "arrow-body-style": ["error", "always"], "arrow-parens": ["error", "always"], "arrow-spacing": "error", "constructor-super": "error", "generator-star-spacing": "error", "no-class-assign": "error", "no-confusing-arrow": "error", "no-const-assign": "error", "no-dupe-class-members": "error", "no-duplicate-imports": "error", "no-new-symbol": "error", "no-restricted-imports": "error", "no-this-before-super": "error", "no-useless-catch": "error", "no-useless-computed-key": "error", "no-useless-constructor": "error", "no-useless-rename": "error", "no-var": "off", "object-shorthand": ["error", "consistent-as-needed"], "prefer-arrow-callback": "off", "prefer-const": "off", "prefer-destructuring": "off", "prefer-numeric-literals": "off", "prefer-rest-params": "off", "prefer-spread": "error", "prefer-template": "off", "require-yield": "error", "rest-spread-spacing": "error", "sort-imports": "off", "symbol-description": "off", "template-curly-spacing": "error", "yield-star-spacing": "error", // Lodash "lodash/callback-binding": "error", "lodash/collection-method-value": "off", "lodash/collection-return": "error", "lodash/no-double-unwrap": "error", "lodash/no-extra-args": "error", "lodash/no-unbound-this": "error", "lodash/unwrap": "error", "lodash/chain-style": ["error", "as-needed"], "lodash/chaining": ["error", "always", 3], "lodash/consistent-compose": ["error", "flow"], "lodash/identity-shorthand": ["error", "always"], "lodash/import-scope": "off", "lodash/matches-prop-shorthand": ["error", "always"], "lodash/matches-shorthand": ["error", "always", 3], "lodash/no-commit": "error", "lodash/path-style": ["error", "as-needed"], "lodash/prefer-compact": "error", "lodash/prefer-filter": ["off", 3], "lodash/prefer-flat-map": "error", "lodash/prefer-invoke-map": "error", "lodash/prefer-map": "error", "lodash/prefer-reject": ["error", 3], "lodash/prefer-thru": "error", "lodash/prefer-wrapper-method": "error", "lodash/preferred-alias": "error", "lodash/prop-shorthand": ["error", "always"], "lodash/prefer-constant": "off", "lodash/prefer-get": ["warn", 4], "lodash/prefer-includes": [ "error", { "includeNative": true } ], "lodash/prefer-is-nil": "error", "lodash/prefer-lodash-chain": "error", "lodash/prefer-lodash-method": "off", "lodash/prefer-lodash-typecheck": "off", "lodash/prefer-matches": ["off", 3], "lodash/prefer-noop": "off", "lodash/prefer-over-quantifier": "warn", "lodash/prefer-some": "off", "lodash/prefer-startswith": "off", "lodash/prefer-times": "off", // JsDoc "jsdoc/check-param-names": "error", "jsdoc/check-tag-names": "off", "jsdoc/check-types": "off", "jsdoc/require-description-complete-sentence": "off", "jsdoc/require-example": "off", "jsdoc/require-hyphen-before-param-description": "off", "jsdoc/require-param": "error", "jsdoc/require-param-description": "error", "jsdoc/require-param-type": "error", "jsdoc/require-returns-description": "off", "jsdoc/require-returns-type": "error", // Security "security/detect-unsafe-regex": "error", "security/detect-buffer-noassert": "error", "security/detect-child-process": "error", "security/detect-disable-mustache-escape": "error", "security/detect-eval-with-expression": "error", "security/detect-no-csrf-before-method-override": "error", "security/detect-non-literal-fs-filename": "off", "security/detect-non-literal-regexp": "error", "security/detect-non-literal-require": "off", "security/detect-object-injection": "off", "security/detect-possible-timing-attacks": "error", "security/detect-pseudoRandomBytes": "error" } } ================================================ FILE: .gitattributes ================================================ # Set the default behavior * text=auto # JavaScript files will always have LF line endings *.js text eol=lf # JSON files always have LF line endings *.json text eol=lf ================================================ FILE: .github/CONTRIBUTING.md ================================================ # Contributing to Postman Newman - [Getting Started Quick](#getting-started-quick) - [NPM Command Reference](#npm-command-reference) - [`npm install`](#npm-install) - [`npm test`](#npm-test) - [`npm run test-system`](#npm-run-test-system) - [`npm run test-lint`](#npm-run-test-lint) - [`npm run test-unit`](#npm-run-test-unit) - [`npm run test-integration`](#npm-run-test-integration) - [Repository](#repository) - [Structure](#structure) - [Branching and Tagging Policy](#branching-and-tagging-policy) - [Preferred IDE](#preferred-ide) - [Commit Guidelines](#commit-guidelines) - [Check for errors before committing](#check-for-errors-before-committing) - [Atomic commits](#atomic-commits) - [Clean commit message](#clean-commit-message) - [Writing tests](#writing-tests) - [Guidelines for sending a Pull Request](#guidelines-for-sending-a-pull-request) - [Documentation guidelines](#documentation-guidelines) - [The CI Platform](#the-ci-platform) - [Ensuring your commits will not fail build](#ensuring-your-commits-will-not-fail-build) - [Accessing build log on CI server](#accessing-build-log-on-ci-server) - [Security](#security) ## Getting Started Quick In order to contribute to this project, you should: 1. Clone this repository / your fork of this repository using `git` 2. If not on `develop`, switch to the `develop` branch with `git checkout develop` 3. Run `npm install` in the project directory 4. Make sure everything is working by running `npm test`. 5. Create a new feature branch from `develop` with `feature/your-feature-name`. 6. Make the necessary changes in line with the objective(s) of the pull request 7. You can debug newman locally by running `node ./bin/newman `. 8. Ensure that you have added unit and integration tests for any new features added / bugs fixed 9. Run `npm test`. If any tests fail, resolve the issue with the code, and re-try 10. Once the tests pass, commit and push. **Do not** alter the `version` in `package.json` 11. Create a pull request to `develop` ## NPM Command Reference ### `npm install` Installs all `dependencies` listed in `package.json` ### `npm test` The script associated with `npm test` will run all tests that ensures that your commit does not break anything in the repository. As such run `npm test` before you push. In addition to performing a few logging and pre-test configuration actions, this test bootstrap script will also trigger the various sub test tasks, which include code lint checks, unit and integration tests. At present, the following sub tests can be run on a standalone basis: * `npm run test-system`: Runs system tests * `npm run test-lint`: Performs code style checks, flagging inconsistencies and other miscellaneous anomalies * `npm run test-unit`: Runs unit tests to verify the correctness of various methods used within the project * `npm run test-integration`: Checks Newman sanity with a sample requests categorized by collection * `npm run test-cli`: Runs CLI integration tests * `npm run test-library`: Runs library integration tests ## Repository ### Structure Directory | Summary ------------------------|----------------------------------------------------------------------------------------------- `bin` | Contains scripts that wrap around all the other methods provided within the project `examples` | A sample script to demonstrate collection file parsing within Newman `lib` | Houses the core logic and configuration sets for Newman, including reporters, and the runtime `npm` | All CI scripts (triggered by NPM run-script) `test` | Contains all the test scripts for Newman `test/unit` | Method wise tests for Newman `test/system` | Erstwhile `infra` tests, checks for proper code structuring and division across the project `test/integration` | Contains the integration test runner, validates Newman as a library ### Branching and Tagging Policy This repository uses standard `git-flow` branch management policy/strategy. If you want to learn more on `git-flow`, refer to [tutorial from Atlassian](https://www.atlassian.com/git/workflows#!workflow-gitflow) and more details at [http://nvie.com/posts/a-successful-git-branching-model/](http://nvie.com/posts/a-successful-git-branching-model). > Deletion of `main` and `develop`. > Rebasing on `main` is blocked. ## Preferred IDE It is advised to use an IDE that provides [EditorConfig](http://editorconfig.org) support via `.editorconfig` files, either natively, or through plugins. In addition, the `.gitignore` file has been populated with entries to support ignoring metadata / manifest files for various IDEs. ## Commit Guidelines The following best practices, coupled with a pinch of common-sense will keep the repository clean and usable in future. The idea is that everything that goes into the repository is not for an individual, but someone else who will be directly or indirectly affected by it. ### Check for errors before committing Checking for errors should be done for each commit whether it is being pushed to remote or not. First, you don't want to submit any whitespace errors. Git provides an easy way to check for this before you commit, run `git diff --check`, which identifies possible whitespace errors and lists them for you. If you run that command before committing, you can tell if you're about to commit whitespace issues that may annoy other developers. Secondly, you should ensure that your commit does not break builds. Run `npm test` on the repository to execute all sanity and smoke tests. If any test fail, do not change the test to pass your commit. The tests were there with a purpose. Discuss within your team to ensure that the changes that you do to test specs are valid. If you are adding a new feature, accompanying them with new tests are a good practice. ### Atomic commits Try to make each commit a logically separate change set. If you can, try to make your changes digestible don't code for a whole weekend on five different issues and then submit them all as one massive commit on Monday. Even if you don't commit during the weekend, use the staging area on Monday to split your work into at least one commit per issue, with a useful message per commit. If some of the changes modify the same file, try to use `git add --patch` to partially stage files. The project snapshot at the tip of the branch is identical whether you do one commit or five, as long as all the changes are added at some point, so try to make things easier on your fellow developers when they have to review your changes. This approach also makes it easier to pull out or revert one of the change sets if you need to later. There are a number of useful Git tricks for rewriting history and interactively staging files use these tools to help craft a clean and understandable history. ### Clean commit message *More detailed explanation include your motivation for the change and contrast its implementation with previous behavior this is a good guideline to follow.* Getting in the habit of creating quality commit messages makes using and collaborating with Git a lot easier. As a general rule, your messages should start with a single line that is no more than 50 characters and that describes the change set concisely, followed by a blank line, followed by a more detailed explanation. It's also a good idea to use the imperative present tense in these messages. In other words, use commands. Instead of "I added tests for" or "Adding tests for," use "Add tests for." You should see if your commit message answers the following questions: Answer the following questions: 1. **Why is this change necessary?** 2. **How does it address the issue?** 3. **What side effects does this change have?** The first question tells reviewers of your pull request what to expect in the commit, allowing them to more easily identify and point out unrelated changes. The second question describes, at a high level, what was done to affect change. If your change is obvious, you may be able to omit addressing this question. The third is the most important question to answer, as it can point out problems where you are making too many changes in one commit or branch. One or two bullet points for related changes may be okay, but five or six are likely indicators of a commit that is doing too many things. A good commit message template ``` Short (50 chars or less) summary of changes with relevant project management issue ID. More detailed explanatory text, if necessary. Wrap it to about 72 characters or so. In some contexts, the first line is treated as the subject of an email and the rest of the text as the body. The blank line separating the summary from the body is critical (unless you omit the body entirely); tools like rebase can get confused if you run the two together. Further paragraphs come after blank lines. - Bullet points are okay, too - Typically a hyphen or asterisk is used for the bullet, preceded by a single space, with blank lines in between, but conventions vary here ``` Run `git log --no-merges` to see what a nicely formatted project-commit history looks like. ### Writing tests Over the course of contributing to Newman, several new features will be added and discovered bugs / glitches will be fixed. It is important to ensure that all of these changes are cross-checked via their respective tests. Two important points in this context are `unit` and `integration` tests. #### Unit tests Unit tests are focused toward validating one method, and one method only. These tests are to be added to `test/unit`. Each unit test file contains a `describe` block, which may or may not contain describe blocks within itself. Any given describe block is meant to group together tests for one particular purpose or section of code. All unit test files must conform to the following standard: * The file name must end with `.test.js` * The file must be named using lower case characters exclusively * Should the first part of the file name need multiple words, they must be separated with dashes `(-)`. Taking up the [existing test files](https://github.com/postmanlabs/newman/tree/feature/v3/test/unit) as an example, we can see that all tests for cli options are grouped together in one file, while tests for the collection runner reside in their own file. Within each `describe` block, individual tests are denoted by distinct `it` blocks. Each `it` block will test exactly one method, with a given set of parameters. Note that multiple `it` blocks can effectively map to one method, as a method may need to be assessed for a variety of runtime conditions. Within each `it` block, correctness checks for expected method behaviour are done using `expect` calls. For more on `expect`, check out [Chai Expect](http://www.chaijs.com/api/bdd) Lastly, the information provided within `describe` and `it` blocks should form a cogent sentence when combined. For instance: ```javascript describe('Logic tests', function () { it('should throw an error if parameters are missing', function () {}); it('should pass when valid values are provided', function () {}); }); ``` The result sentence derived by combining the descriptions in the `describe` and `it` blocks are: **Logic tests** should throw an error if parameters are missing **Logic tests** should pass when valid values are provided #### Integration tests Integration tests test Newman from the outside, and as a whole. As you might have already guessed, these tests don't test one method at a time. With regard to Newman, each integration test consists of a postman collection, and optional set of environment, data, and globals files. All these files are arranged without any directory segregation, as can be seen in [test/integration](https://github.com/postmanlabs/newman/tree/feature/v3/test/integration). Each of these file names should follow the convention outlined below: * File names with multiple words must have `-` as a separator, not `_`. * Under no circumstances do file names involve `camelCase`. * The type of the file must be designated with `postman_{{type}}` in the file name, where `type` is one of `collection`, `data`, `environment`, or `globals`. See examples below for clarity. * All postman_collection files must be in the *V2* format. * `.json` is a common file extension across all files, data files can have `.csv` as an extension as well. For instance: > file-name.postman_collection.json file-name.postman_environment.json file-name.postman_globals.json file-name.postman_data.json file-name.postman_data.csv ## Guidelines for sending a Pull Request Commit to `main` branch and `develop` branch is locked. As such, `git-flow` for feature completion and release will not work. Thus, the last steps of feature completion in `git-flow` will happen as a Pull Request from website. Avoid changing the `version` field in `package.json` for feature pull requests, as `version` bumps are handled separately. 1. Pull Request comment and commit comments should explicitly discuss what changes were made. The Pull Request reviewer should not need to communicate out of scope of issue tracker and the pull request description in order to understand what changes has been done. 2. Pull Requests with build failures will not be merged. Ensure that `npm test` passes on the `head` of your feature branch. The same goes for pull requests with untested new features / resolved bugs. 3. Ensure that your feature branch has been tested and if it is associated with issues from corresponding issue-tracker, the issue must be in a "resolved" state, implying that the issue has been fully tested, and accepted for inclusion. 4. Pull Requests with merge conflict are very difficult to review, and are at a higher likelihood of being rejected. Ensure that the `head` of your feature branch is either already merged with `develop` or has no conflict when it is merged with `develop`. 5. The turn around time to close a Pull Request is directly proportional to the delta of changes done - more the change in files, more time it would take. As such, if you anticipate a feature branch to have a large delta on feature completion, break it into sub-issues of the issue-tracker, test them, close them, and then send PR for that branch. 6. In addition, the turn around time for Pull Request would get affected if commit messages are unclear. 7. If you have deadlines to ensure feature completion, send Pull Request ahead of time. Better still, ensure that your feature development time window accounts for PR acceptance time as well. 8. If you have mentioned issue tracker references in Pull Request description, the severity and priority of those issues will be taken into account. Otherwise, no Pull Request will take priority over others already in queue - it is first-pull first-merge! ## Documentation guidelines Details on usage, sample use cases, licensing and current project status, and community support are to be specified within `README.md`. All information related to contribution is to be added to `CONTRIBUTING.md`. More in depth information on aspects like project architecture, design, and so on is to be added to the project [wiki](https://github.com/postmanlabs/newman/wiki) ## The CI Platform The CI system is built as a bunch of bash scripts to execute a set of tasks. These scripts are meant to execute tasks that can run on every local machine. In general, knowledge about these scripts are not necessary for development **The scripts are to be only accessed using `npm run