[
  {
    "path": ".editorconfig",
    "content": "root = true\n\n[*]\ncharset = utf-8\nend_of_line = lf\nindent_size = 4\nindent_style = space\ninsert_final_newline = true\ntrim_trailing_whitespace = true\n"
  },
  {
    "path": ".gitignore",
    "content": "bin/\ncache/\ndata/\nlogs/\nnode_modules/\noutput/\nvalidateOutput/\n\\#*\n.\\#*\n"
  },
  {
    "path": ".travis.yml",
    "content": "language: node_js\n\nnode_js:\n  - 'node'\n\nsudo: false\n\nbranches:\n  only:\n  - master\n\ninstall:\n  - npm ci\n\ngit:\n  depth: 1\n"
  },
  {
    "path": ".vscode/launch.json",
    "content": "{\n    // Use IntelliSense to learn about possible attributes.\n    // Hover to view descriptions of existing attributes.\n    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387\n    \"version\": \"0.2.0\",\n    \"configurations\": [\n        {\n            \"type\": \"node\",\n            \"request\": \"launch\",\n            \"name\": \"parse-definitions.js\",\n            \"cwd\": \"${workspaceFolder}\",\n            \"program\": \"${workspaceFolder}/bin/parse-definitions.js\",\n            \"args\": [],\n            \"sourceMaps\": true\n        },\n        {\n            \"type\": \"node\",\n            \"request\": \"launch\",\n            \"name\": \"calculate-versions.js\",\n            \"cwd\": \"${workspaceFolder}\",\n            \"program\": \"${workspaceFolder}/bin/calculate-versions.js\",\n            \"args\": [],\n            \"sourceMaps\": true\n        },\n        {\n            \"type\": \"node\",\n            \"request\": \"launch\",\n            \"name\": \"publish-packages.js --dry\",\n            \"cwd\": \"${workspaceFolder}\",\n            \"program\": \"${workspaceFolder}/bin/publish-packages.js\",\n            \"args\": [\"--dry\"],\n            \"sourceMaps\": true\n        }\n    ]\n}"
  },
  {
    "path": "CODEOWNERS",
    "content": "* @sandersn @RyanCavanaugh"
  },
  {
    "path": "DEBUGGING.md",
    "content": "# DEBUGGING\n\nDebugging an Azure App Service has been a miserable learning\nexperience for me, so here are some tips to help.\n\nFirst, repro and debug anything you can locally. Everything except the\nwebhook-server either includes a dry-run mode or runs identically\nlocally. But you may still need to test the non-dry-run code.\n\nYour basic workflow looks like this:\n\n1. `npm run push-production`\n2. Resend webhook or merge a new PR.\n3. Look at ftp logs.\n\n(2) and (3) have some complexity.\n\nStep (2) is required because an App Service doesn't seem to properly\nrestart until a web request comes in. If you're just testing startup,\nor something else that happens even if no PRs need to be merged,\nresending a webhook should be fine. Types-publisher only cares that\nthe push was to master and not to some other branch. This is true for\nall non-team-member PRs.\n\nOtherwise, you'll need to find a easy PR to merge.\n\nFor step (3), proceed to the FTP logs. If you can't remember the\naddress, you can find it in the Diagnostics logs section of the\nTypesPublisher App Service page on Azure. You can also set up new\nusername/password combinations somewhere, but I don't remember where\nexactly. I think the README may tell where.\n\nThen look in LogFiles/Application/ and page down to index.html, which\ncontains a *sorted* list of log files. You'll see a new log file\nevery time the server restarts, which is at least every time you push\nto production, but may be more often if the server is crashing."
  },
  {
    "path": "IISNode.yml",
    "content": "# For all options see https://github.com/tjanczuk/iisnode/blob/master/src/samples/configuration/iisnode.yml\nnodeProcessCommandLine: node.exe --stack-trace-limit=1000\nloggingEnabled: true\ndevErrorsEnabled: false\nmaxLogFileSizeInKB: 2048\nmaxTotalLogFileSizeInKB: 65536\nmaxLogFiles: 16384\n"
  },
  {
    "path": "LICENSE",
    "content": "    MIT License\n\n    Copyright (c) Microsoft Corporation. All rights reserved.\n\n    Permission is hereby granted, free of charge, to any person obtaining a copy\n    of this software and associated documentation files (the \"Software\"), to deal\n    in the Software without restriction, including without limitation the rights\n    to use, copy, modify, merge, publish, distribute, sublicense, and/or sell\n    copies of the Software, and to permit persons to whom the Software is\n    furnished to do so, subject to the following conditions:\n\n    The above copyright notice and this permission notice shall be included in all\n    copies or substantial portions of the Software.\n\n    THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\n    IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\n    FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\n    AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\n    LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\n    OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\n    SOFTWARE\n"
  },
  {
    "path": "README.md",
    "content": "# This repo has moved.\n\nThis service is now part of [microsoft/DefinitelyTyped-tools](https://github.com/microsoft/DefinitelyTyped-tools/tree/master/packages/publisher).\n\nIf you need to update allowedPackageJsonDependencies.txt (formerly named dependenciesWhitelist.txt), please do so at its [new location](https://github.com/microsoft/DefinitelyTyped-tools/blob/master/packages/definitions-parser/allowedPackageJsonDependencies.txt).\n"
  },
  {
    "path": "dependenciesWhitelist.txt",
    "content": "@babel/code-frame\n@babel/core\n@babel/generator\n@babel/parser\n@babel/template\n@babel/traverse\n@babel/types\n@elastic/elasticsearch\n@electron/get\n@emotion/core\n@emotion/serialize\n@emotion/styled\n@hapi/boom\n@hapi/iron\n@hapi/wreck\n@jest/environment\n@jest/fake-timers\n@jest/globals\n@jest/transform\n@jest/types\n@loadable/component\n@material-ui/core\n@material-ui/types\n@sentry/browser\n@storybook/addons\n@storybook/react\n@types/base-x\n@types/expect\n@types/firebase\n@types/highcharts\n@types/hoist-non-react-statics\n@types/mkdirp-promise\n@types/next-redux-wrapper\n@types/ink\n@types/js-data\n@types/react-native-tab-view\n@types/react-navigation\n@types/three\n@types/rsmq\n@types/vue\n@types/webdriverio\n@types/winston\n@types/wonder-commonlib\n@types/wonder-frp\n@uirouter/angularjs\n@vue/test-utils\n@wordpress/element\nabort-controller\nactions-on-google\nactivex-helpers\nadal-node\najv\nalgoliasearch\nalgoliasearch-helper\nanydb-sql\naphrodite\napollo-client\napollo-link\napollo-link-http-common\nast-types\naws-sdk\naxe-core\naxios\nbase-x\nbignumber.js\nbookshelf\nbroadcast-channel\ncac\ncash-dom\ncassandra-driver\nchalk\nchokidar\ncids\ncommander\nconnect-mongo\ncordova-plugin-camera\ncordova-plugin-file\ncordova-plugin-file-transfer\ncosmiconfig\ncropperjs\ncsstype\ncypress\ndate-fns\ndecimal.js\ndel\ndexie\ndnd-core\ndotenv\negg\nelectron\nelectron-notarize\nelectron-osx-sign\nelectron-store\nethers\neventemitter2\neventemitter3\nexpect\nexpress-graphql\nexpress-rate-limit\nfast-glob\nfastify\nfinal-form\nflatpickr\nform-data\ngl-matrix\nglaze\nglobby\ngoogle-auth-library\ngoogle-gax\ngot\ngraphql\ngraphql-tools\ngrpc\nhandlebars\nhoist-non-react-statics\nhtmlparser2\ni18next\nimmutable\nindefinite-observable\ninversify\njest-cucumber\njest-diff\njest-environment-node\njest-environment-jsdom\njest-mock\njest-snapshot\njimp\njointjs\nlevelup\nlit-element\nlocalforage\nlog4js\nlogform\nloglevel\nlogrocket\nknex\nknockout\nmagic-string\nmali\nmeteor-typings\nmoment\nmonaco-editor\nmoment-range\nmqtt\nnext\nnock\nnormalize-url\nobjection\nopentracing\nparchment\nparse5\npath-to-regexp\npkcs11js\npopper.js\npostcss\npolished\npreact\npretty-format\nprobot\nprom-client\nprotobufjs\nprotractor\nquery-string\nquill-delta\nraven-js\nre-resizable\nreact-autosize-textarea\nreact-dnd\nreact-dnd-touch-backend\nreact-intl\nreact-native-fs\nreact-native-maps\nreact-native-modal\nreact-native-svg\nreact-native-tab-view\nreact-navigation\nrecast\nredux\nredux-observable\nredux-persist\nredux-saga\nredux-thunk\nrollup\nrrule\nrxjs\nsafe-buffer\nshould\nsmooth-scrollbar\nscroll-into-view-if-needed\nsource-map\nstyled-components\nsw-toolbox\nswagger-parser\nterser\nthree\ntreat\ntslint\nts-toolbelt\ntweetnacl\ntypescript\ntypescript-tuple\nutility-types\nvega-typings\nvfile\nvfile-message\nvue\nvue-router\nvuex\nwebpack-chain\nwinston\nwinston-transport\nxmlbuilder\nxpath\nzipkin\nzipkin-transport-http\n"
  },
  {
    "path": "jest.config.js",
    "content": "module.exports = {\n    \"roots\": [\n        \"src\"\n    ],\n    \"preset\": \"ts-jest\",\n    \"testRegex\": \"(/__tests__/.*|\\\\.(test|spec))\\\\.tsx?$\",\n    \"moduleFileExtensions\": [\n        \"ts\",\n        \"tsx\",\n        \"js\",\n        \"jsx\",\n        \"json\",\n        \"node\"\n    ],\n}\n"
  },
  {
    "path": "package.json",
    "content": "{\n  \"private\": true,\n  \"name\": \"types-publisher\",\n  \"version\": \"0.0.0\",\n  \"main\": \"./bin/index.js\",\n  \"types\": \"./bin/index.d.ts\",\n  \"description\": \"Publish DefinitelyTyped definitions to NPM\",\n  \"dependencies\": {\n    \"@octokit/rest\": \"^16.1.0\",\n    \"@types/tar-stream\": \"^1.6.0\",\n    \"adal-node\": \"^0.1.22\",\n    \"applicationinsights\": \"^1.0.7\",\n    \"azure-keyvault\": \"^3.0.4\",\n    \"azure-storage\": \"^2.0.0\",\n    \"charm\": \"^1.0.2\",\n    \"definitelytyped-header-parser\": \"3.8.2\",\n    \"dtslint\": \"latest\",\n    \"fs-extra\": \"4.0.0\",\n    \"fstream\": \"^1.0.12\",\n    \"longjohn\": \"^0.2.11\",\n    \"moment\": \"^2.18.1\",\n    \"npm\": \"^6.13.4\",\n    \"npm-registry-client\": \"^8.1.0\",\n    \"oboe\": \"^2.1.3\",\n    \"source-map-support\": \"^0.4.0\",\n    \"tar\": \"^2.2.2\",\n    \"tar-stream\": \"^1.6.2\",\n    \"travis-fold\": \"^0.1.2\",\n    \"typescript\": \"next\",\n    \"yargs\": \"^8.0.2\"\n  },\n  \"devDependencies\": {\n    \"@types/charm\": \"^1.0.0\",\n    \"@types/fs-extra\": \"4.0.0\",\n    \"@types/jest\": \"^23.3.9\",\n    \"@types/mz\": \"^0.0.31\",\n    \"@types/node\": \"^10.11.3\",\n    \"@types/oboe\": \"^2.0.28\",\n    \"@types/source-map-support\": \"^0.4.0\",\n    \"@types/tar\": \"^1.0.27\",\n    \"@types/travis-fold\": \"^0.1.0\",\n    \"@types/yargs\": \"^8.0.1\",\n    \"jest\": \"^24.9.0\",\n    \"ts-jest\": \"^24.2.0\",\n    \"tslint\": \"^5.13.0\"\n  },\n  \"scripts\": {\n    \"test\": \"npm run build && jest && npm run lint\",\n    \"build\": \"node node_modules/typescript/lib/tsc.js\",\n    \"watch\": \"node node_modules/typescript/lib/tsc.js --watch\",\n    \"clean\": \"node -r source-map-support/register bin/clean.js\",\n    \"get-definitely-typed\": \"node -r source-map-support/register bin/get-definitely-typed.js\",\n    \"parse\": \"node -r source-map-support/register bin/parse-definitions.js\",\n    \"check\": \"node -r source-map-support/register bin/check-parse-results.js\",\n    \"calculate-versions\": \"node -r source-map-support/register bin/calculate-versions.js\",\n    \"generate\": \"node -r source-map-support/register bin/generate-packages.js\",\n    \"validate\": \"node -r source-map-support/register bin/validate.js\",\n    \"index\": \"node -r source-map-support/register bin/create-search-index.js\",\n    \"publish\": \"node -r source-map-support/register bin/publish-packages.js\",\n    \"publish-dry\": \"node -r source-map-support/register bin/publish-packages.js --dry\",\n    \"publish-registry\": \"node -r source-map-support/register bin/publish-registry.js\",\n    \"upload-blobs\": \"node -r source-map-support/register bin/upload-blobs.js\",\n    \"full\": \"node -r source-map-support/register bin/full.js\",\n    \"full-dry\": \"node -r source-map-support/register bin/full.js --dry\",\n    \"lint\": \"node node_modules/tslint/bin/tslint --format stylish --project tsconfig.json\",\n    \"webhook-dry\": \"node -r source-map-support/register bin/webhook.js --dry\",\n    \"make-server-run\": \"node -r source-map-support/register bin/make-server-run.js\",\n    \"make-production-server-run\": \"node -r source-map-support/register bin/make-server-run.js --remote\",\n    \"test-tsNext\": \"node -r source-map-support/register bin/tester/test.js --all --tsNext\",\n    \"code-owners\": \"node -r source-map-support/register bin/code-owners.js\",\n    \"push-production\": \"npm run build && git checkout production && git merge master && npm run build && git add bin && git commit -m \\\"Update bin\\\" && git push -u origin production\"\n  },\n  \"repository\": {\n    \"type\": \"git\",\n    \"url\": \"git+https://github.com/Microsoft/types-publisher.git\"\n  },\n  \"author\": \"Microsoft\",\n  \"license\": \"MIT\",\n  \"bugs\": {\n    \"url\": \"https://github.com/Microsoft/types-publisher/issues\"\n  },\n  \"engines\": {\n    \"node\": \">=6.10.0\"\n  },\n  \"homepage\": \"https://github.com/Microsoft/types-publisher#readme\"\n}\n"
  },
  {
    "path": "server.js",
    "content": "require(\"./bin/webhook.js\").default().catch(console.error);\n"
  },
  {
    "path": "src/.vscode/launch.json",
    "content": "{\n    // Use IntelliSense to learn about possible attributes.\n    // Hover to view descriptions of existing attributes.\n    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387\n    \"version\": \"0.2.0\",\n    \"configurations\": [\n       {\n            \"type\": \"node\",\n            \"request\": \"attach\",\n            \"name\": \"9229\",\n            \"port\": 9229,\n            \"sourceMaps\": true,\n            \"stopOnEntry\": false\n            \n        }\n    ]\n}"
  },
  {
    "path": "src/calculate-versions.ts",
    "content": "import assert = require(\"assert\");\n\nimport { FS, getDefinitelyTyped } from \"./get-definitely-typed\";\nimport { Options, writeDataFile } from \"./lib/common\";\nimport { CachedNpmInfoClient, NpmInfoVersion, UncachedNpmInfoClient, withNpmCache } from \"./lib/npm-client\";\nimport { AllPackages, NotNeededPackage, TypingsData } from \"./lib/packages\";\nimport { ChangedPackages, ChangedPackagesJson, ChangedTypingJson, Semver, versionsFilename } from \"./lib/versions\";\nimport { loggerWithErrors, LoggerWithErrors } from \"./util/logging\";\nimport { assertDefined, best, logUncaughtErrors, mapDefined, mapDefinedAsync } from \"./util/util\";\nif (!module.parent) {\n    const log = loggerWithErrors()[0];\n    logUncaughtErrors(async () => calculateVersions(await getDefinitelyTyped(Options.defaults, log), new UncachedNpmInfoClient(), log));\n}\n\nexport default async function calculateVersions(\n    dt: FS,\n    uncachedClient: UncachedNpmInfoClient,\n    log: LoggerWithErrors,\n): Promise<ChangedPackages> {\n    log.info(\"=== Calculating versions ===\");\n    return withNpmCache(uncachedClient, async client => {\n        log.info(\"* Reading packages...\");\n        const packages = await AllPackages.read(dt);\n        return computeAndSaveChangedPackages(packages, log, client);\n    });\n}\n\nasync function computeAndSaveChangedPackages(\n    allPackages: AllPackages,\n    log: LoggerWithErrors,\n    client: CachedNpmInfoClient,\n): Promise<ChangedPackages> {\n    const cp = await computeChangedPackages(allPackages, log, client);\n    const json: ChangedPackagesJson = {\n        changedTypings: cp.changedTypings.map(({ pkg: { id }, version, latestVersion }): ChangedTypingJson => ({ id, version, latestVersion })),\n        changedNotNeededPackages: cp.changedNotNeededPackages.map(p => p.name),\n    };\n    await writeDataFile(versionsFilename, json);\n    return cp;\n}\n\nasync function computeChangedPackages(\n    allPackages: AllPackages,\n    log: LoggerWithErrors,\n    client: CachedNpmInfoClient,\n): Promise<ChangedPackages> {\n    log.info(\"# Computing changed packages...\");\n    const changedTypings = await mapDefinedAsync(allPackages.allTypings(), async pkg => {\n        const { version, needsPublish } = await fetchTypesPackageVersionInfo(pkg, client, /*publish*/ true, log);\n        if (needsPublish) {\n            log.info(`Changed: ${pkg.desc}`);\n            for (const { name } of pkg.packageJsonDependencies) {\n                assertDefined(\n                    await client.fetchAndCacheNpmInfo(name),\n                    `'${pkg.name}' depends on '${name}' which does not exist on npm. All dependencies must exist.`);\n            }\n            const latestVersion = pkg.isLatest ?\n                undefined :\n                (await fetchTypesPackageVersionInfo(allPackages.getLatest(pkg), client, /*publish*/ true)).version;\n            return { pkg, version, latestVersion };\n        }\n        return undefined;\n    });\n    log.info(\"# Computing deprecated packages...\");\n    const changedNotNeededPackages = await mapDefinedAsync(allPackages.allNotNeeded(), async pkg => {\n        if (!await isAlreadyDeprecated(pkg, client, log)) {\n            assertDefined(\n                await client.fetchAndCacheNpmInfo(pkg.unescapedName),\n                `To deprecate '@types/${pkg.name}', '${pkg.unescapedName}' must exist on npm.`);\n            log.info(`To be deprecated: ${pkg.name}`);\n            return pkg;\n        }\n        return undefined;\n    });\n    return { changedTypings, changedNotNeededPackages };\n}\n\nasync function fetchTypesPackageVersionInfo(\n    pkg: TypingsData,\n    client: CachedNpmInfoClient,\n    canPublish: boolean,\n    log?: LoggerWithErrors,\n): Promise<{ version: string, needsPublish: boolean }> {\n    let info = client.getNpmInfoFromCache(pkg.fullEscapedNpmName);\n    let latestVersion = info && getHighestVersionForMajor(info.versions, pkg);\n    let latestVersionInfo = latestVersion && assertDefined(info!.versions.get(latestVersion.versionString));\n    if (!latestVersionInfo || latestVersionInfo.typesPublisherContentHash !== pkg.contentHash) {\n        if (log) { log.info(`Version info not cached for ${pkg.desc}`); }\n        info = await client.fetchAndCacheNpmInfo(pkg.fullEscapedNpmName);\n        latestVersion = info && getHighestVersionForMajor(info.versions, pkg);\n        latestVersionInfo = latestVersion && assertDefined(info!.versions.get(latestVersion.versionString));\n        if (!latestVersionInfo) { return { version: versionString(pkg, /*patch*/ 0), needsPublish: true }; }\n    }\n\n    if (latestVersionInfo.deprecated) {\n        // https://github.com/DefinitelyTyped/DefinitelyTyped/pull/22306\n        assert(\n            pkg.name === \"angular-ui-router\" || pkg.name === \"ui-router-extras\",\n            `Package ${pkg.name} has been deprecated, so we shouldn't have parsed it. Was it re-added?`);\n    }\n    const needsPublish = canPublish && pkg.contentHash !== latestVersionInfo.typesPublisherContentHash;\n    const patch = needsPublish ? (latestVersion!.minor === pkg.minor ? latestVersion!.patch + 1 : 0) : latestVersion!.patch;\n    return { version: versionString(pkg, patch), needsPublish };\n}\n\nfunction versionString(pkg: TypingsData, patch: number): string {\n    return new Semver(pkg.major, pkg.minor, patch).versionString;\n}\n\nasync function isAlreadyDeprecated(pkg: NotNeededPackage, client: CachedNpmInfoClient, log: LoggerWithErrors): Promise<boolean> {\n    const cachedInfo = client.getNpmInfoFromCache(pkg.fullEscapedNpmName);\n    let latestVersion = cachedInfo && assertDefined(cachedInfo.distTags.get(\"latest\"));\n    let latestVersionInfo = cachedInfo && latestVersion && assertDefined(cachedInfo.versions.get(latestVersion));\n    if (!latestVersionInfo || !latestVersionInfo.deprecated) {\n        log.info(`Version info not cached for deprecated package ${pkg.desc}`);\n        const info = assertDefined(await client.fetchAndCacheNpmInfo(pkg.fullEscapedNpmName));\n        latestVersion = assertDefined(info.distTags.get(\"latest\"));\n        latestVersionInfo = assertDefined(info.versions.get(latestVersion));\n    }\n    return !!latestVersionInfo.deprecated;\n}\n\nfunction getHighestVersionForMajor(versions: ReadonlyMap<string, NpmInfoVersion>, { major, minor }: TypingsData): Semver | undefined {\n    const patch = latestPatchMatchingMajorAndMinor(versions.keys(), major, minor);\n    return patch === undefined ? undefined : new Semver(major, minor, patch);\n}\n\n/** Finds the version with matching major/minor with the latest patch version. */\nfunction latestPatchMatchingMajorAndMinor(versions: Iterable<string>, newMajor: number, newMinor: number): number | undefined {\n    const versionsWithTypings = mapDefined(versions, v => {\n        const semver = Semver.tryParse(v);\n        if (!semver) {\n            return undefined;\n        }\n        const { major, minor, patch } = semver;\n        return major === newMajor && minor === newMinor ? patch : undefined;\n    });\n    return best(versionsWithTypings, (a, b) => a > b);\n}\n\nexport async function getLatestTypingVersion(pkg: TypingsData, client: CachedNpmInfoClient): Promise<string> {\n    return (await fetchTypesPackageVersionInfo(pkg, client, /*publish*/ false)).version;\n}\n"
  },
  {
    "path": "src/check-parse-results.ts",
    "content": "import { FS, getDefinitelyTyped } from \"./get-definitely-typed\";\nimport { Options } from \"./lib/common\";\nimport { NpmInfoRawVersions, NpmInfoVersion, UncachedNpmInfoClient } from \"./lib/npm-client\";\nimport { AllPackages, formatTypingVersion, TypingsData, TypingVersion } from \"./lib/packages\";\nimport { Semver } from \"./lib/versions\";\nimport { Logger, logger, loggerWithErrors, writeLog } from \"./util/logging\";\nimport { assertDefined, best, logUncaughtErrors, mapDefined, nAtATime } from \"./util/util\";\n\nif (!module.parent) {\n    const log = loggerWithErrors()[0];\n    logUncaughtErrors(\n        async () => checkParseResults(true, await getDefinitelyTyped(Options.defaults, log), Options.defaults, new UncachedNpmInfoClient()));\n}\n\nexport default async function checkParseResults(includeNpmChecks: boolean, dt: FS, options: Options, client: UncachedNpmInfoClient): Promise<void> {\n    const allPackages = await AllPackages.read(dt);\n    const [log, logResult] = logger();\n\n    checkTypeScriptVersions(allPackages);\n\n    checkPathMappings(allPackages);\n\n    const dependedOn = new Set<string>();\n    const packages = allPackages.allPackages();\n    for (const pkg of packages) {\n        if (pkg instanceof TypingsData) {\n            for (const dep of pkg.dependencies) {\n                dependedOn.add(dep.name);\n            }\n            for (const dep of pkg.testDependencies) {\n                dependedOn.add(dep);\n            }\n        }\n    }\n\n    if (includeNpmChecks) {\n        await nAtATime(10, allPackages.allTypings(), pkg => checkNpm(pkg, log, dependedOn, client), {\n            name: \"Checking for typed packages...\",\n            flavor: pkg => pkg.desc,\n            options,\n        });\n    }\n\n    await writeLog(\"conflicts.md\", logResult());\n}\n\nfunction checkTypeScriptVersions(allPackages: AllPackages): void {\n    for (const pkg of allPackages.allTypings()) {\n        for (const dep of allPackages.allDependencyTypings(pkg)) {\n            if (dep.minTypeScriptVersion > pkg.minTypeScriptVersion) {\n                throw new Error(`${pkg.desc} depends on ${dep.desc} but has a lower required TypeScript version.`);\n            }\n        }\n    }\n}\n\nfunction checkPathMappings(allPackages: AllPackages): void {\n    for (const pkg of allPackages.allTypings()) {\n        const pathMappings = new Map<string, TypingVersion>(pkg.pathMappings.map(p => [p.packageName, p.version]));\n        const unusedPathMappings = new Set(pathMappings.keys());\n\n        // If A depends on B, and B has path mappings, A must have the same mappings.\n        for (const dependency of allPackages.allDependencyTypings(pkg)) {\n            for (const { packageName: transitiveDependencyName, version: transitiveDependencyVersion } of dependency.pathMappings) {\n                const pathMappingVersion = pathMappings.get(transitiveDependencyName);\n                if (\n                    pathMappingVersion\n                    && (\n                        pathMappingVersion.major !== transitiveDependencyVersion.major\n                        || pathMappingVersion.minor !== transitiveDependencyVersion.minor\n                    )\n                ) {\n                    const expectedPathMapping = `${transitiveDependencyName}/v${formatTypingVersion(transitiveDependencyVersion)}`;\n                    throw new Error(\n                        `${pkg.desc} depends on ${dependency.desc}, which has a path mapping for ${expectedPathMapping}. ` +\n                        `${pkg.desc} must have the same path mappings as its dependencies.`);\n                }\n                unusedPathMappings.delete(transitiveDependencyName);\n            }\n\n            unusedPathMappings.delete(dependency.name);\n        }\n\n        for (const unusedPathMapping of unusedPathMappings) {\n            if (pkg.name !== unusedPathMapping) {\n                throw new Error(`${pkg.desc} has unused path mapping for ${unusedPathMapping}`);\n            }\n        }\n    }\n}\n\nasync function checkNpm(\n    { major, minor, name, libraryName, projectName, contributors }: TypingsData,\n    log: Logger,\n    dependedOn: ReadonlySet<string>,\n    client: UncachedNpmInfoClient,\n): Promise<void> {\n    if (notNeededExceptions.has(name)) {\n        return;\n    }\n\n    const info = await client.fetchRawNpmInfo(name); // Gets info for the real package, not the @types package\n    if (!info) { return; }\n\n    const versions = getRegularVersions(info.versions);\n    const firstTypedVersion = best(mapDefined(versions, ({ hasTypes, version }) => hasTypes ? version : undefined), (a, b) => b.greaterThan(a));\n    // A package might have added types but removed them later, so check the latest version too\n    if (firstTypedVersion === undefined || !best(versions, (a, b) => a.version.greaterThan(b.version))!.hasTypes) {\n        return;\n    }\n\n    const ourVersion = `${major}.${minor}`;\n\n    log(\"\");\n    log(`Typings already defined for ${name} (${libraryName}) as of ${firstTypedVersion.versionString} (our version: ${ourVersion})`);\n    const contributorUrls = contributors.map(c => {\n        const gh = \"https://github.com/\";\n        return c.url.startsWith(gh) ? `@${c.url.slice(gh.length)}` : `${c.name} (${c.url})`;\n    }).join(\", \");\n    log(\"  To fix this:\");\n    log(`  git checkout -b not-needed-${name}`);\n    const yarnargs = [name, firstTypedVersion.versionString, projectName];\n    if (libraryName !== name) {\n        yarnargs.push(JSON.stringify(libraryName));\n    }\n    log(\"  yarn not-needed \" + yarnargs.join(\" \"));\n    log(`  git add --all && git commit -m \"${name}: Provides its own types\" && git push -u origin not-needed-${name}`);\n    log(`  And comment PR: This will deprecate \\`@types/${name}\\` in favor of just \\`${name}\\`. CC ${contributorUrls}`);\n    if (new Semver(major, minor, 0).greaterThan(firstTypedVersion)) {\n        log(\"  WARNING: our version is greater!\");\n    }\n    if (dependedOn.has(name)) {\n        log(\"  WARNING: other packages depend on this!\");\n    }\n}\n\nexport async function packageHasTypes(packageName: string, client: UncachedNpmInfoClient): Promise<boolean> {\n    const info = assertDefined(await client.fetchRawNpmInfo(packageName));\n    return versionHasTypes(info.versions[info[\"dist-tags\"].latest]);\n}\n\nfunction getRegularVersions(versions: NpmInfoRawVersions): ReadonlyArray<{ readonly version: Semver; readonly hasTypes: boolean; }> {\n    return mapDefined(Object.entries(versions), ([versionString, info]) => {\n        const version = Semver.tryParse(versionString);\n        return version === undefined ? undefined : { version, hasTypes: versionHasTypes(info) };\n    });\n}\n\nfunction versionHasTypes(info: NpmInfoVersion): boolean {\n    return \"types\" in info || \"typings\" in info;\n}\n\nconst notNeededExceptions: ReadonlySet<string> = new Set([\n    // https://github.com/DefinitelyTyped/DefinitelyTyped/pull/22306\n    \"angular-ui-router\", \"ui-router-extras\",\n    // Declares to bundle types, but they're also in the `.npmignore` (https://github.com/nkovacic/angular-touchspin/issues/21)\n    \"angular-touchspin\",\n    // \"typings\" points to the wrong file (https://github.com/Microsoft/Bing-Maps-V8-TypeScript-Definitions/issues/31)\n    \"bingmaps\",\n    // Types are bundled, but not officially released (https://github.com/DefinitelyTyped/DefinitelyTyped/pull/22313#issuecomment-353225893)\n    \"dwt\",\n    // Waiting on some typing errors to be fixed (https://github.com/julien-c/epub/issues/30)\n    \"epub\",\n    // Typings file is not in package.json \"files\" list (https://github.com/silentmatt/expr-eval/issues/127)\n    \"expr-eval\",\n    // NPM package \"express-serve-static-core\" isn't a real package -- express-serve-static-core exists only for the purpose of types\n    \"express-serve-static-core\",\n    // Has \"typings\": \"index.d.ts\" but does not actually bundle typings. https://github.com/kolodny/immutability-helper/issues/79\n    \"immutability-helper\",\n    // Has `\"typings\": \"compiled/typings/node-mysql-wrapper/node-mysql-wrapper.d.ts\",`, but `compiled/typings` doesn't exist.\n    // Package hasn't updated in 2 years and author seems to have deleted their account, so no chance of being fixed.\n    \"node-mysql-wrapper\",\n    // raspi packages bundle types, but can only be installed on a Raspberry Pi, so they are duplicated to DefinitelyTyped.\n    // See https://github.com/DefinitelyTyped/DefinitelyTyped/pull/21618\n    \"raspi\", \"raspi-board\", \"raspi-gpio\", \"raspi-i2c\", \"raspi-led\", \"raspi-onewire\",\n    \"raspi-peripheral\", \"raspi-pwm\", \"raspi-serial\", \"raspi-soft-pwm\",\n    // Declare \"typings\" but don't actually have them yet (https://github.com/stampit-org/stampit/issues/245)\n    \"stampit\",\n]);\n"
  },
  {
    "path": "src/clean.ts",
    "content": "import { removeSync } from \"fs-extra\";\n\nif (!module.parent) {\n    clean();\n}\n\nexport function clean() {\n    for (const dir of [\"data\", \"logs\", \"output\"]) {\n        console.log(`Clean ${dir}`);\n        removeSync(dir);\n    }\n}\n"
  },
  {
    "path": "src/crawl-npm.ts",
    "content": "import assert = require(\"assert\");\nimport oboe = require(\"oboe\");\n\nimport { packageHasTypes } from \"./check-parse-results\";\nimport { Options, writeDataFile } from \"./lib/common\";\nimport { UncachedNpmInfoClient } from \"./lib/npm-client\";\nimport { npmRegistry } from \"./lib/settings\";\nimport ProgressBar, { strProgress } from \"./util/progress\";\nimport { filterNAtATimeOrdered, logUncaughtErrors } from \"./util/util\";\n\nif (!module.parent) {\n    logUncaughtErrors(main(Options.defaults));\n}\n\n/** Prints out every package on NPM with 'types'. */\nasync function main(options: Options): Promise<void> {\n    const all = await allNpmPackages();\n    await writeDataFile(\"all-npm-packages.json\", all);\n    const client = new UncachedNpmInfoClient();\n    const allTyped = await filterNAtATimeOrdered(10, all, pkg => packageHasTypes(pkg, client), {\n        name: \"Checking for types...\",\n        flavor: (name, isTyped) => isTyped ? name : undefined,\n        options,\n    });\n    await writeDataFile(\"all-typed-packages.json\", allTyped);\n    console.log(allTyped.join(\"\\n\"));\n    console.log(`Found ${allTyped.length} typed packages.`);\n}\n\nfunction allNpmPackages(): Promise<string[]> {\n    const progress = new ProgressBar({ name: \"Loading NPM packages...\" });\n\n    // https://github.com/npm/registry/blob/master/docs/REGISTRY-API.md\n    const url = `${npmRegistry}-/all`;\n    const all: string[] = [];\n    return new Promise<string[]>((resolve, reject) => {\n        oboe(url)\n        .node(\"!.*\", (x, path) => {\n            assert((path as string).length > 0);\n            if (typeof x !== \"number\") {\n                const { name } = x as { name: string };\n                assert(typeof name === \"string\" && name.length > 0); // tslint:disable-line strict-type-predicates\n                progress.update(strProgress(name), name);\n                all.push(name);\n            }\n            return oboe.drop;\n        })\n        .done(() => {\n            progress.done();\n            resolve(all);\n        })\n        .fail(err => { reject(err.thrown); });\n    });\n}\n"
  },
  {
    "path": "src/create-search-index.ts",
    "content": "import * as yargs from \"yargs\";\n\nimport { getDefinitelyTyped } from \"./get-definitely-typed\";\nimport { Options, writeDataFile } from \"./lib/common\";\nimport { UncachedNpmInfoClient } from \"./lib/npm-client\";\nimport { AllPackages, TypingsData } from \"./lib/packages\";\nimport { loggerWithErrors } from \"./util/logging\";\nimport { logUncaughtErrors } from \"./util/util\";\n\nif (!module.parent) {\n    const log = loggerWithErrors()[0];\n    const single = yargs.argv.single as string | undefined;\n    if (single) {\n        logUncaughtErrors(doSingle(single, new UncachedNpmInfoClient()));\n    } else {\n        logUncaughtErrors(\n            async () => createSearchIndex(await AllPackages.read(await getDefinitelyTyped(Options.defaults, log)), new UncachedNpmInfoClient()));\n    }\n}\n\nexport interface SearchRecord {\n    // types package name\n    readonly t: string;\n    // globals\n    readonly g: ReadonlyArray<string>;\n    // modules\n    readonly m: ReadonlyArray<string>;\n    // project name\n    readonly p: string;\n    // library name\n    readonly l: string;\n    // downloads in the last month from NPM\n    readonly d: number;\n}\n\nexport default async function createSearchIndex(packages: AllPackages, client: UncachedNpmInfoClient): Promise<void> {\n    console.log(\"Generating search index...\");\n    const records = await createSearchRecords(packages.allLatestTypings(), client);\n    console.log(\"Done generating search index. Writing out data files...\");\n    await writeDataFile(\"search-index-min.json\", records, false);\n}\n\nasync function doSingle(name: string, client: UncachedNpmInfoClient): Promise<void> {\n    const pkg = await AllPackages.readSingle(name);\n    const record = (await createSearchRecords([pkg], client))[0];\n    console.log(record);\n}\n\nasync function createSearchRecords(packages: ReadonlyArray<TypingsData>, client: UncachedNpmInfoClient): Promise<ReadonlyArray<SearchRecord>> {\n    // TODO: Would like to just use pkg.unescapedName unconditionally, but npm doesn't allow scoped packages.\n    const dl = await client.getDownloads(packages.map((pkg, i) => pkg.name === pkg.unescapedName ? pkg.name : `dummy${i}`));\n    return packages.map((pkg, i): SearchRecord => ({\n        p: pkg.projectName,\n        l: pkg.libraryName,\n        g: pkg.globals,\n        t: pkg.name,\n        m: pkg.declaredModules,\n        d: dl[i],\n    })).sort((a, b) => b.d - a.d);\n}\n"
  },
  {
    "path": "src/full.ts",
    "content": "import appInsights = require(\"applicationinsights\");\nimport * as yargs from \"yargs\";\n\nimport calculateVersions from \"./calculate-versions\";\nimport { clean } from \"./clean\";\nimport createSearchIndex from \"./create-search-index\";\nimport generatePackages from \"./generate-packages\";\nimport { getDefinitelyTyped } from \"./get-definitely-typed\";\nimport { Options } from \"./lib/common\";\nimport { UncachedNpmInfoClient } from \"./lib/npm-client\";\nimport parseDefinitions from \"./parse-definitions\";\nimport publishPackages from \"./publish-packages\";\nimport publishRegistry from \"./publish-registry\";\nimport uploadBlobsAndUpdateIssue from \"./upload-blobs\";\nimport { Fetcher } from \"./util/io\";\nimport { LoggerWithErrors, loggerWithErrors } from \"./util/logging\";\nimport { assertDefined, currentTimeStamp, logUncaughtErrors, numberOfOsProcesses } from \"./util/util\";\nimport validate from \"./validate\";\n\nif (!module.parent) {\n    if (process.env.APPINSIGHTS_INSTRUMENTATIONKEY) {\n        appInsights.setup();\n        appInsights.start();\n    }\n    const dry = !!yargs.argv.dry;\n    logUncaughtErrors(full(dry, currentTimeStamp(), process.env.GH_API_TOKEN || \"\", new Fetcher(), Options.defaults, loggerWithErrors()[0]));\n}\n\nexport default async function full(\n    dry: boolean,\n    timeStamp: string,\n    githubAccessToken: string,\n    fetcher: Fetcher,\n    options: Options,\n    log: LoggerWithErrors): Promise<void> {\n    const infoClient = new UncachedNpmInfoClient();\n    clean();\n    const dt = await getDefinitelyTyped(options, log);\n    const allPackages = await parseDefinitions(\n        dt,\n        options.parseInParallel\n            ? { nProcesses: numberOfOsProcesses, definitelyTypedPath: assertDefined(options.definitelyTypedPath) }\n            : undefined,\n        log);\n    const changedPackages = await calculateVersions(dt, infoClient, log);\n    await generatePackages(dt, allPackages, changedPackages);\n    await createSearchIndex(allPackages, infoClient);\n    await publishPackages(changedPackages, dry, githubAccessToken, fetcher);\n    await publishRegistry(dt, allPackages, dry, infoClient);\n    await validate(dt);\n    if (!dry) {\n        await uploadBlobsAndUpdateIssue(timeStamp);\n    }\n}\n"
  },
  {
    "path": "src/generate-packages.test.ts",
    "content": "import { createNotNeededPackageJSON, createPackageJSON, createReadme, getLicenseFileText } from \"./generate-packages\";\nimport { Registry } from \"./lib/common\";\nimport { AllPackages, License, NotNeededPackage, readNotNeededPackages, TypesDataFile, TypingsData, TypingsDataRaw } from \"./lib/packages\";\nimport { createMockDT } from \"./mocks\";\nimport { testo } from \"./util/test\";\n\nfunction createRawPackage(license: License): TypingsDataRaw {\n    return {\n        libraryName: \"jquery\",\n        typingsPackageName: \"jquery\",\n        dependencies: [{ name: \"madeira\", version: { major: 1 } }],\n        testDependencies: [],\n        pathMappings: [],\n        contributors: [{ name: \"A\", url: \"b@c.d\", githubUsername: \"e\" }],\n        libraryMajorVersion: 1,\n        libraryMinorVersion: 0,\n        minTsVersion: \"3.0\",\n        typesVersions: [],\n        files: [\"index.d.ts\", \"jquery.test.ts\"],\n        license,\n        packageJsonDependencies: [{ name: \"balzac\", version: \"~3\" }],\n        contentHash: \"11\",\n        projectName: \"jquery.org\",\n        globals: [],\n        declaredModules: [\"jquery\"],\n    };\n}\nfunction createTypesData(): TypesDataFile {\n    return {\n        jquery: {\n            1: createRawPackage(License.MIT),\n        },\n        madeira: {\n            1: createRawPackage(License.Apache20),\n        },\n    };\n}\nfunction createUnneededPackage() {\n    return new NotNeededPackage({\n        libraryName: \"absalom\",\n        typingsPackageName: \"absalom\",\n        asOfVersion: \"1.1.1\",\n        sourceRepoURL: \"https://github.com/aardwulf/absalom\",\n    });\n}\ntesto({\n    mitLicenseText() {\n        const typing = new TypingsData(createRawPackage(License.MIT), /*isLatest*/ true);\n        expect(getLicenseFileText(typing)).toEqual(expect.stringContaining(\"MIT License\"));\n    },\n    apacheLicenseText() {\n        const typing = new TypingsData(createRawPackage(License.Apache20), /*isLatest*/ true);\n        expect(getLicenseFileText(typing)).toEqual(expect.stringContaining(\"Apache License, Version 2.0\"));\n    },\n    basicReadme() {\n        const typing = new TypingsData(createRawPackage(License.Apache20), /*isLatest*/ true);\n        expect(createReadme(typing)).toEqual(expect.stringContaining(\"This package contains type definitions for\"));\n    },\n    readmeContainsProjectName() {\n        const typing = new TypingsData(createRawPackage(License.Apache20), /*isLatest*/ true);\n        expect(createReadme(typing)).toEqual(expect.stringContaining(\"jquery.org\"));\n    },\n    readmeOneDependency() {\n        const typing = new TypingsData(createRawPackage(License.Apache20), /*isLatest*/ true);\n        expect(createReadme(typing)).toEqual(expect.stringContaining(\"Dependencies: [@types/madeira](https://npmjs.com/package/@types/madeira)\"));\n    },\n    readmeNoGlobals() {\n        const typing = new TypingsData(createRawPackage(License.Apache20), /*isLatest*/ true);\n        expect(createReadme(typing)).toEqual(expect.stringContaining(\"Global values: none\"));\n    },\n    basicPackageJson() {\n        const packages = AllPackages.from(createTypesData(), readNotNeededPackages(createMockDT().fs));\n        const typing = new TypingsData(createRawPackage(License.MIT), /*isLatest*/ true);\n        expect(createPackageJSON(typing, \"1.0\", packages, Registry.NPM)).toEqual(`{\n    \"name\": \"@types/jquery\",\n    \"version\": \"1.0\",\n    \"description\": \"TypeScript definitions for jquery\",\n    \"license\": \"MIT\",\n    \"contributors\": [\n        {\n            \"name\": \"A\",\n            \"url\": \"b@c.d\",\n            \"githubUsername\": \"e\"\n        }\n    ],\n    \"main\": \"\",\n    \"types\": \"index.d.ts\",\n    \"repository\": {\n        \"type\": \"git\",\n        \"url\": \"https://github.com/DefinitelyTyped/DefinitelyTyped.git\",\n        \"directory\": \"types/jquery\"\n    },\n    \"scripts\": {},\n    \"dependencies\": {\n        \"@types/madeira\": \"^1\",\n        \"balzac\": \"~3\"\n    },\n    \"typesPublisherContentHash\": \"11\",\n    \"typeScriptVersion\": \"3.0\"\n}`);\n    },\n    githubPackageJsonName() {\n        const packages = AllPackages.from(createTypesData(), readNotNeededPackages(createMockDT().fs));\n        const typing = new TypingsData(createRawPackage(License.MIT), /*isLatest*/ true);\n        expect(createPackageJSON(typing, \"1.0\", packages, Registry.Github)).toEqual(\n            expect.stringContaining('\"name\": \"@types/jquery\"'));\n    },\n    githubPackageJsonRegistry() {\n        const packages = AllPackages.from(createTypesData(), readNotNeededPackages(createMockDT().fs));\n        const typing = new TypingsData(createRawPackage(License.MIT), /*isLatest*/ true);\n        const s = createPackageJSON(typing, \"1.0\", packages, Registry.Github);\n        expect(s).toEqual(expect.stringContaining(\"publishConfig\"));\n        expect(s).toEqual(expect.stringContaining('\"registry\": \"https://npm.pkg.github.com/\"'));\n    },\n    basicNotNeededPackageJson() {\n        const s = createNotNeededPackageJSON(createUnneededPackage(), Registry.NPM);\n        expect(s).toEqual(`{\n    \"name\": \"@types/absalom\",\n    \"version\": \"1.1.1\",\n    \"typings\": null,\n    \"description\": \"Stub TypeScript definitions entry for absalom, which provides its own types definitions\",\n    \"main\": \"\",\n    \"scripts\": {},\n    \"author\": \"\",\n    \"repository\": \"https://github.com/aardwulf/absalom\",\n    \"license\": \"MIT\",\n    \"dependencies\": {\n        \"absalom\": \"*\"\n    }\n}`);\n    },\n    scopedNotNeededPackageJson() {\n        const scopedUnneeded = new NotNeededPackage({\n        libraryName: \"@google-cloud/pubsub\",\n        typingsPackageName: \"google-cloud__pubsub\",\n        asOfVersion: \"0.26.0\",\n        sourceRepoURL: \"https://github.com/googleapis/nodejs-storage\",\n    });\n        const s = createNotNeededPackageJSON(scopedUnneeded, Registry.NPM);\n        expect(s).toEqual(`{\n    \"name\": \"@types/google-cloud__pubsub\",\n    \"version\": \"0.26.0\",\n    \"typings\": null,\n    \"description\": \"Stub TypeScript definitions entry for @google-cloud/pubsub, which provides its own types definitions\",\n    \"main\": \"\",\n    \"scripts\": {},\n    \"author\": \"\",\n    \"repository\": \"https://github.com/googleapis/nodejs-storage\",\n    \"license\": \"MIT\",\n    \"dependencies\": {\n        \"@google-cloud/pubsub\": \"*\"\n    }\n}`);\n    },\n    githubNotNeededPackageJson() {\n        const s = createNotNeededPackageJSON(createUnneededPackage(), Registry.Github);\n        expect(s).toEqual(expect.stringContaining(\"@types\"));\n        expect(s).toEqual(expect.stringContaining(\"npm.pkg.github.com\"));\n    },\n});\n"
  },
  {
    "path": "src/generate-packages.ts",
    "content": "import { makeTypesVersionsForPackageJson } from \"definitelytyped-header-parser\";\nimport { emptyDir, mkdir, mkdirp, readFileSync } from \"fs-extra\";\nimport * as path from \"path\";\nimport * as yargs from \"yargs\";\n\nimport { FS, getDefinitelyTyped } from \"./get-definitely-typed\";\nimport { Options, Registry } from \"./lib/common\";\nimport { CachedNpmInfoClient, UncachedNpmInfoClient, withNpmCache } from \"./lib/npm-client\";\nimport {\n    AllPackages, AnyPackage, DependencyVersion, formatTypingVersion, getFullNpmName, License, NotNeededPackage, PackageJsonDependency, TypingsData,\n} from \"./lib/packages\";\nimport { outputDirPath, sourceBranch } from \"./lib/settings\";\nimport { ChangedPackages, readChangedPackages, skipBadPublishes } from \"./lib/versions\";\nimport { writeFile } from \"./util/io\";\nimport { logger, Logger, loggerWithErrors, writeLog } from \"./util/logging\";\nimport { writeTgz } from \"./util/tgz\";\nimport { assertNever, joinPaths, logUncaughtErrors, sortObjectKeys } from \"./util/util\";\n\nconst mitLicense = readFileSync(joinPaths(__dirname, \"..\", \"LICENSE\"), \"utf-8\");\n\nif (!module.parent) {\n    const tgz = !!yargs.argv.tgz;\n    logUncaughtErrors(async () => {\n        const log = loggerWithErrors()[0];\n        const dt = await getDefinitelyTyped(Options.defaults, log);\n        const allPackages = await AllPackages.read(dt);\n        await generatePackages(dt, allPackages, await readChangedPackages(allPackages), tgz);\n    });\n}\n\nexport default async function generatePackages(dt: FS, allPackages: AllPackages, changedPackages: ChangedPackages, tgz = false): Promise<void> {\n    const [log, logResult] = logger();\n    log(\"\\n## Generating packages\");\n\n    await emptyDir(outputDirPath);\n\n    for (const { pkg, version } of changedPackages.changedTypings) {\n        await generateTypingPackage(pkg, allPackages, version, dt);\n        if (tgz) {\n            await writeTgz(pkg.outputDirectory, `${pkg.outputDirectory}.tgz`);\n        }\n        log(` * ${pkg.desc}`);\n    }\n    log(\"## Generating deprecated packages\");\n    await withNpmCache(new UncachedNpmInfoClient(), async client => {\n        for (const pkg of changedPackages.changedNotNeededPackages) {\n            log(` * ${pkg.libraryName}`);\n            await generateNotNeededPackage(pkg, client, log);\n        }\n    });\n    await writeLog(\"package-generator.md\", logResult());\n}\nasync function generateTypingPackage(typing: TypingsData, packages: AllPackages, version: string, dt: FS): Promise<void> {\n    const typesDirectory = dt.subDir(\"types\").subDir(typing.name);\n    const packageFS = typing.isLatest ? typesDirectory : typesDirectory.subDir(typing.versionDirectoryName!);\n\n    await writeCommonOutputs(typing, createPackageJSON(typing, version, packages, Registry.NPM), createReadme(typing), Registry.NPM);\n    await writeCommonOutputs(typing, createPackageJSON(typing, version, packages, Registry.Github), createReadme(typing), Registry.Github);\n    await Promise.all(\n        typing.files.map(async file => writeFile(await outputFilePath(typing, Registry.NPM, file), packageFS.readFile(file))));\n    await Promise.all(\n        typing.files.map(async file => writeFile(await outputFilePath(typing, Registry.Github, file), packageFS.readFile(file))));\n}\n\nasync function generateNotNeededPackage(pkg: NotNeededPackage, client: CachedNpmInfoClient, log: Logger): Promise<void> {\n    pkg = skipBadPublishes(pkg, client, log);\n    await writeCommonOutputs(pkg, createNotNeededPackageJSON(pkg, Registry.NPM), pkg.readme(), Registry.NPM);\n    await writeCommonOutputs(pkg, createNotNeededPackageJSON(pkg, Registry.Github), pkg.readme(), Registry.Github);\n}\n\nasync function writeCommonOutputs(pkg: AnyPackage, packageJson: string, readme: string, registry: Registry): Promise<void> {\n    await mkdir(pkg.outputDirectory + (registry === Registry.Github ? \"-github\" : \"\"));\n\n    await Promise.all([\n        writeOutputFile(\"package.json\", packageJson),\n        writeOutputFile(\"README.md\", readme),\n        writeOutputFile(\"LICENSE\", getLicenseFileText(pkg)),\n    ]);\n\n    async function writeOutputFile(filename: string, content: string): Promise<void> {\n        await writeFile(await outputFilePath(pkg, registry, filename), content);\n    }\n}\n\nasync function outputFilePath(pkg: AnyPackage, registry: Registry, filename: string): Promise<string> {\n    const full = joinPaths(pkg.outputDirectory + (registry === Registry.Github ? \"-github\" : \"\"), filename);\n    const dir = path.dirname(full);\n    if (dir !== pkg.outputDirectory) {\n        await mkdirp(dir);\n    }\n    return full;\n}\n\ninterface Dependencies { [name: string]: string; }\n\nexport function createPackageJSON(typing: TypingsData, version: string, packages: AllPackages, registry: Registry): string {\n    // Use the ordering of fields from https://docs.npmjs.com/files/package.json\n    const out: {} = {\n        name: typing.fullNpmName,\n        version,\n        description: `TypeScript definitions for ${typing.libraryName}`,\n        // keywords,\n        // homepage,\n        // bugs,\n        license: typing.license,\n        contributors: typing.contributors,\n        main: \"\",\n        types: \"index.d.ts\",\n        typesVersions:  makeTypesVersionsForPackageJson(typing.typesVersions),\n        repository: {\n            type: \"git\",\n            url: registry === Registry.Github\n                ? \"https://github.com/types/_definitelytypedmirror.git\"\n                : \"https://github.com/DefinitelyTyped/DefinitelyTyped.git\",\n            directory: `types/${typing.name}`,\n        },\n        scripts: {},\n        dependencies: getDependencies(typing.packageJsonDependencies, typing, packages),\n        typesPublisherContentHash: typing.contentHash,\n        typeScriptVersion: typing.minTypeScriptVersion,\n    };\n    if (registry === Registry.Github) {\n        (out as any).publishConfig = { registry: \"https://npm.pkg.github.com/\" };\n    }\n\n    return JSON.stringify(out, undefined, 4);\n}\n\nconst definitelyTypedURL = \"https://github.com/DefinitelyTyped/DefinitelyTyped\";\n\n/** Adds inferred dependencies to `dependencies`, if they are not already specified in either `dependencies` or `peerDependencies`. */\nfunction getDependencies(packageJsonDependencies: ReadonlyArray<PackageJsonDependency>, typing: TypingsData, allPackages: AllPackages): Dependencies {\n    const dependencies: Dependencies = {};\n    for (const { name, version } of packageJsonDependencies) {\n        dependencies[name] = version;\n    }\n\n    for (const dependency of typing.dependencies) {\n        const typesDependency = getFullNpmName(dependency.name);\n        // A dependency \"foo\" is already handled if we already have a dependency on the package \"foo\" or \"@types/foo\".\n        if (!packageJsonDependencies.some(d => d.name === dependency.name || d.name === typesDependency) && allPackages.hasTypingFor(dependency)) {\n            dependencies[typesDependency] = dependencySemver(dependency.version);\n        }\n    }\n    return sortObjectKeys(dependencies);\n}\n\nfunction dependencySemver(dependency: DependencyVersion): string {\n    return dependency === \"*\" ? dependency : \"^\" + formatTypingVersion(dependency);\n}\n\nexport function createNotNeededPackageJSON(\n    {\n        libraryName,\n        license,\n        unescapedName,\n        fullNpmName,\n        sourceRepoURL,\n        version,\n    }: NotNeededPackage,\n    registry: Registry,\n): string {\n    const out = {\n        name: fullNpmName,\n        version: version.versionString,\n        typings: null, // tslint:disable-line no-null-keyword\n        description: `Stub TypeScript definitions entry for ${libraryName}, which provides its own types definitions`,\n        main: \"\",\n        scripts: {},\n        author: \"\",\n        repository: registry === Registry.NPM ? sourceRepoURL : \"https://github.com/types/_definitelytypedmirror.git\",\n        license,\n        // No `typings`, that's provided by the dependency.\n        dependencies: {\n            [unescapedName]: \"*\",\n        },\n    };\n    if (registry === Registry.Github) {\n        (out as any).publishConfig = { registry: \"https://npm.pkg.github.com/\" };\n    }\n    return JSON.stringify(out, undefined, 4);\n}\n\nexport function createReadme(typing: TypingsData): string {\n    const lines: string[] = [];\n    lines.push(\"# Installation\");\n    lines.push(`> \\`npm install --save ${typing.fullNpmName}\\``);\n    lines.push(\"\");\n\n    lines.push(\"# Summary\");\n    if (typing.projectName) {\n        lines.push(`This package contains type definitions for ${typing.libraryName} (${typing.projectName}).`);\n    } else {\n        lines.push(`This package contains type definitions for ${typing.libraryName}.`);\n    }\n    lines.push(\"\");\n\n    lines.push(\"# Details\");\n    lines.push(`Files were exported from ${definitelyTypedURL}/tree/${sourceBranch}/types/${typing.subDirectoryPath}.`);\n\n    lines.push(\"\");\n    lines.push(\"### Additional Details\");\n    lines.push(` * Last updated: ${(new Date()).toUTCString()}`);\n    const dependencies = Array.from(typing.dependencies).map(d => getFullNpmName(d.name));\n    lines.push(` * Dependencies: ${dependencies.length ? dependencies.map(d => `[${d}](https://npmjs.com/package/${d})`).join(\", \") : \"none\"}`);\n    lines.push(` * Global values: ${typing.globals.length ? typing.globals.map(g => `\\`${g}\\``).join(\", \") : \"none\"}`);\n    lines.push(\"\");\n\n    lines.push(\"# Credits\");\n    const contributors = typing.contributors.map(({ name, url }) => `[${name}](${url})`).join(\", \").replace(/, ([^,]+)$/, \", and $1\");\n    lines.push(`These definitions were written by ${contributors}.`);\n    lines.push(\"\");\n\n    return lines.join(\"\\r\\n\");\n}\n\nexport function getLicenseFileText(typing: AnyPackage): string {\n    switch (typing.license) {\n        case License.MIT:\n            return mitLicense;\n        case License.Apache20:\n            return apacheLicense(typing);\n        default:\n            throw assertNever(typing);\n    }\n}\n\nfunction apacheLicense(typing: TypingsData): string {\n    const year = new Date().getFullYear();\n    const names = typing.contributors.map(c => c.name);\n    // tslint:disable max-line-length\n    return `Copyright ${year} ${names.join(\", \")}\nLicensed under the Apache License, Version 2.0 (the \"License\"); you may not use this file except in compliance with the License. You may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software distributed under the License is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.`;\n    // tslint:enable max-line-length\n}\n"
  },
  {
    "path": "src/get-definitely-typed.test.ts",
    "content": "import { Dir, FS, getDefinitelyTyped, InMemoryDT } from \"./get-definitely-typed\";\nimport { Options } from \"./lib/common\";\nimport { loggerWithErrors } from \"./util/logging\";\nimport { testo } from \"./util/test\";\n\ntesto({\n    async downloadDefinitelyTyped() {\n        const dt = await getDefinitelyTyped(Options.azure, loggerWithErrors()[0]);\n        expect(dt.exists(\"types\")).toBe(true);\n        expect(dt.exists(\"buncho\")).toBe(false);\n    },\n    createDirs() {\n        const root = new Dir(undefined);\n        root.set(\"file1.txt\", \"ok\");\n        expect(root.has(\"file1.txt\")).toBe(true);\n        expect(root.get(\"file1.txt\")).toBe(\"ok\");\n    },\n    simpleMemoryFS() {\n        const root = new Dir(undefined);\n        root.set(\"file1.txt\", \"ok\");\n        const dir = root.subdir(\"sub1\");\n        dir.set(\"file2.txt\", \"x\");\n        const fs: FS = new InMemoryDT(root, \"test/\");\n        expect(fs.exists(\"file1.txt\")).toBe(true);\n        expect(fs.readFile(\"file1.txt\")).toBe(\"ok\");\n        expect(fs.readFile(\"sub1/file2.txt\")).toBe(\"x\");\n    },\n});\n"
  },
  {
    "path": "src/get-definitely-typed.ts",
    "content": "import appInsights = require(\"applicationinsights\");\nimport assert = require(\"assert\");\nimport { ensureDir, pathExistsSync, readdirSync, statSync } from \"fs-extra\";\nimport https = require(\"https\");\nimport tarStream = require(\"tar-stream\");\nimport * as yargs from \"yargs\";\nimport * as zlib from \"zlib\";\n\nimport { Options } from \"./lib/common\";\nimport { dataDirPath, definitelyTypedZipUrl } from \"./lib/settings\";\nimport { readFileSync, readJsonSync, stringOfStream } from \"./util/io\";\nimport { LoggerWithErrors, loggerWithErrors } from \"./util/logging\";\nimport { assertDefined, exec, joinPaths, logUncaughtErrors, withoutStart } from \"./util/util\";\n\n/**\n * Readonly filesystem.\n * Paths provided to these methods should be relative to the FS object's root but not start with '/' or './'.\n */\nexport interface FS {\n    /**\n     * Alphabetically sorted list of files and subdirectories.\n     * If dirPath is missing, reads the root.\n     */\n    readdir(dirPath?: string): ReadonlyArray<string>;\n    readJson(path: string): unknown;\n    readFile(path: string): string;\n    isDirectory(dirPath: string): boolean;\n    exists(path: string): boolean;\n    /** FileSystem rooted at a child directory. */\n    subDir(path: string): FS;\n    /** Representation of current location, for debugging. */\n    debugPath(): string;\n}\n\nif (!module.parent) {\n    if (process.env.APPINSIGHTS_INSTRUMENTATIONKEY) {\n        appInsights.setup();\n        appInsights.start();\n    }\n    const dry = !!yargs.argv.dry;\n    console.log(\"gettingDefinitelyTyped: \" + (dry ? \"from github\" : \"locally\"));\n    logUncaughtErrors(async () => {\n        const dt = await getDefinitelyTyped(dry ? Options.azure : Options.defaults, loggerWithErrors()[0]);\n        assert(dt.exists(\"types\"));\n        assert(!(dt.exists(\"buncho\")));\n    });\n}\n\nexport async function getDefinitelyTyped(options: Options, log: LoggerWithErrors): Promise<FS> {\n    if (options.definitelyTypedPath === undefined) {\n        log.info(\"Downloading Definitely Typed ...\");\n        await ensureDir(dataDirPath);\n        return downloadAndExtractFile(definitelyTypedZipUrl);\n    }\n    const { error, stderr, stdout } = await exec(\"git diff --name-only\", options.definitelyTypedPath);\n    if (error) { throw error; }\n    if (stderr) { throw new Error(stderr); }\n    if (stdout) { throw new Error(`'git diff' should be empty. Following files changed:\\n${stdout}`); }\n    log.info(`Using local Definitely Typed at ${options.definitelyTypedPath}.`);\n    return new DiskFS(`${options.definitelyTypedPath}/`);\n}\n\nexport function getLocallyInstalledDefinitelyTyped(path: string): FS {\n    return new DiskFS(`${path}/`);\n}\n\nfunction downloadAndExtractFile(url: string): Promise<FS> {\n    return new Promise<FS>((resolve, reject) => {\n        const root = new Dir(undefined);\n        function insertFile(path: string, content: string): void {\n            const components = path.split(\"/\");\n            const baseName = assertDefined(components.pop());\n            let dir = root;\n            for (const component of components) {\n                dir = dir.subdir(component);\n            }\n            dir.set(baseName, content);\n        }\n\n        https.get(url, response => {\n            const extract = tarStream.extract();\n            response.pipe(zlib.createGunzip()).pipe(extract);\n            interface Header {\n                readonly name: string;\n                readonly type: \"file\" | \"directory\";\n            }\n            extract.on(\"entry\", (header: Header, stream: NodeJS.ReadableStream, next: () => void) => {\n                const name = assertDefined(withoutStart(header.name, \"DefinitelyTyped-master/\"));\n                switch (header.type) {\n                    case \"file\":\n                        stringOfStream(stream, name).then(s => {\n                            insertFile(name, s);\n                            next();\n                        }).catch(reject);\n                        break;\n                    case \"directory\":\n                        next();\n                        break;\n                    default:\n                        throw new Error(`Unexpected file system entry kind ${header.type}`);\n                }\n            });\n            extract.on(\"error\", reject);\n            extract.on(\"finish\", () => { resolve(new InMemoryDT(root.finish(), \"\")); });\n        });\n    });\n}\n\ninterface ReadonlyDir extends ReadonlyMap<string, ReadonlyDir | string> {\n    readonly parent: Dir | undefined;\n}\n\n// Map entries are Dir for directory and string for file.\nexport class Dir extends Map<string, Dir | string> implements ReadonlyDir {\n    constructor(readonly parent: Dir | undefined) { super(); }\n\n    subdir(name: string): Dir {\n        const x = this.get(name);\n        if (x !== undefined) {\n            if (typeof x === \"string\") {\n                throw new Error(`File ${name} has same name as a directory?`);\n            }\n            return x;\n        }\n        const res = new Dir(this);\n        this.set(name, res);\n        return res;\n    }\n\n    finish(): Dir {\n        const out = new Dir(this.parent);\n        for (const key of Array.from(this.keys()).sort()) {\n            const subDirOrFile = this.get(key)!;\n            out.set(key, typeof subDirOrFile === \"string\" ? subDirOrFile : subDirOrFile.finish());\n        }\n        return out;\n    }\n}\n\nexport class InMemoryDT implements FS {\n    /** pathToRoot is just for debugging */\n    constructor(readonly curDir: ReadonlyDir, readonly pathToRoot: string) {}\n\n    private tryGetEntry(path: string): ReadonlyDir | string | undefined {\n        validatePath(path);\n        if (path === \"\") {\n            return this.curDir;\n        }\n        const components = path.split(\"/\");\n        const baseName = assertDefined(components.pop());\n        let dir = this.curDir;\n        for (const component of components) {\n            const entry = component === \"..\" ? dir.parent : dir.get(component);\n            if (entry === undefined) {\n                return undefined;\n            }\n            if (!(entry instanceof Dir)) {\n                throw new Error(`No file system entry at ${this.pathToRoot}/${path}. Siblings are: ${Array.from(dir.keys()).toString()}`);\n            }\n            dir = entry;\n        }\n        return dir.get(baseName);\n    }\n\n    private getEntry(path: string): ReadonlyDir | string {\n        const entry = this.tryGetEntry(path);\n        if (entry === undefined) { throw new Error(`No file system entry at ${this.pathToRoot}/${path}`); }\n        return entry;\n    }\n\n    private getDir(dirPath: string): Dir {\n        const res = this.getEntry(dirPath);\n        if (!(res instanceof Dir)) {\n            throw new Error(`${this.pathToRoot}/${dirPath} is a file, not a directory.`);\n        }\n        return res;\n    }\n\n    readFile(filePath: string): string {\n        const res = this.getEntry(filePath);\n        if (typeof res !== \"string\") {\n            throw new Error(`${this.pathToRoot}/${filePath} is a directory, not a file.`);\n        }\n        return res;\n    }\n\n    readdir(dirPath?: string): ReadonlyArray<string> {\n        return Array.from((dirPath === undefined ? this.curDir : this.getDir(dirPath)).keys());\n    }\n\n    readJson(path: string): unknown {\n        return JSON.parse(this.readFile(path)) as unknown;\n    }\n\n    isDirectory(path: string): boolean {\n        return typeof this.getEntry(path) !== \"string\";\n    }\n\n    exists(path: string): boolean {\n        return this.tryGetEntry(path) !== undefined;\n    }\n\n    subDir(path: string): FS {\n        return new InMemoryDT(this.getDir(path), joinPaths(this.pathToRoot, path));\n    }\n\n    debugPath(): string {\n        return this.pathToRoot;\n    }\n}\n\nclass DiskFS implements FS {\n    constructor(private readonly rootPrefix: string) {\n        assert(rootPrefix.endsWith(\"/\"));\n    }\n\n    private getPath(path: string | undefined): string {\n        if (path === undefined) {\n            return this.rootPrefix;\n        }\n        validatePath(path);\n        return this.rootPrefix + path;\n    }\n\n    readdir(dirPath?: string): ReadonlyArray<string> {\n        return readdirSync(this.getPath(dirPath)).sort().filter(name => name !== \".DS_Store\");\n    }\n\n    isDirectory(dirPath: string): boolean {\n        return statSync(this.getPath(dirPath)).isDirectory();\n    }\n\n    readJson(path: string): unknown {\n        return readJsonSync(this.getPath(path));\n    }\n\n    readFile(path: string): string {\n        return readFileSync(this.getPath(path));\n    }\n\n    exists(path: string): boolean {\n        return pathExistsSync(this.getPath(path));\n    }\n\n    subDir(path: string): FS {\n        return new DiskFS(`${this.rootPrefix}${path}/`);\n    }\n\n    debugPath(): string {\n        return this.rootPrefix.slice(0, this.rootPrefix.length - 1); // remove trailing '/'\n    }\n}\n\n/** FS only handles simple paths like `foo/bar` or `../foo`. No `./foo` or `/foo`. */\nfunction validatePath(path: string): void {\n    if (path.startsWith(\".\") && path !== \".editorconfig\" && !path.startsWith(\"../\")) {\n        throw new Error(`${path}: filesystem doesn't support paths of the form './x'.`);\n    }\n    if (path.startsWith(\"/\")) {\n        throw new Error(`${path}: filesystem doesn't support paths of the form '/xxx'.`);\n    }\n    if (path.endsWith(\"/\")) {\n        throw new Error(`${path}: filesystem doesn't support paths of the form 'xxx/'.`);\n    }\n}\n"
  },
  {
    "path": "src/index.ts",
    "content": "export { getDefinitelyTyped } from \"./get-definitely-typed\";\nexport { withNpmCache, CachedNpmInfoClient, NpmPublishClient, UncachedNpmInfoClient } from \"./lib/npm-client\";\nexport { AllPackages } from \"./lib/packages\";\nexport { clean } from \"./clean\";\nexport { getLatestTypingVersion } from \"./calculate-versions\";\nexport { default as parseDefinitions } from \"./parse-definitions\";\n\nexport { parseNProcesses } from \"./tester/test-runner\";\nexport { consoleLogger, loggerWithErrors } from \"./util/logging\";\nexport { logUncaughtErrors, nAtATime } from \"./util/util\";\n\nexport { updateLatestTag, updateTypeScriptVersionTags } from \"./lib/package-publisher\";\n"
  },
  {
    "path": "src/lib/azure-container.ts",
    "content": "import { BlobService, common, createBlobService, ErrorOrResponse, ErrorOrResult } from \"azure-storage\";\nimport * as fs from \"fs\";\nimport * as https from \"https\";\n\nimport { streamDone, streamOfString, stringOfStream } from \"../util/io\";\nimport { gzip, unGzip } from \"../util/tgz\";\nimport { parseJson } from \"../util/util\";\n\nimport { getSecret, Secret } from \"./secrets\";\nimport { azureContainer, azureStorageAccount } from \"./settings\";\n\nexport default class BlobWriter {\n    static async create(): Promise<BlobWriter> {\n        return new BlobWriter(createBlobService(azureStorageAccount, await getSecret(Secret.AZURE_STORAGE_ACCESS_KEY)));\n    }\n\n    private constructor(private readonly service: BlobService) {}\n\n    setCorsProperties(): Promise<void> {\n        const properties: common.models.ServicePropertiesResult.ServiceProperties = {\n            Cors: {\n                CorsRule: [\n                    {\n                        AllowedOrigins: [\"*\"],\n                        AllowedMethods: [\"GET\"],\n                        AllowedHeaders: [],\n                        ExposedHeaders: [],\n                        MaxAgeInSeconds: 60 * 60 * 24, // 1 day\n                    },\n                ],\n            },\n        };\n        return promisifyErrorOrResponse(cb => { this.service.setServiceProperties(properties, cb); });\n    }\n\n    ensureCreated(options: BlobService.CreateContainerOptions): Promise<void> {\n        return promisifyErrorOrResult<BlobService.ContainerResult>(cb => {\n            this.service.createContainerIfNotExists(azureContainer, options, cb);\n        }) as unknown as Promise<void>;\n    }\n\n    createBlobFromFile(blobName: string, fileName: string): Promise<void> {\n        return this.createBlobFromStream(blobName, fs.createReadStream(fileName));\n    }\n\n    createBlobFromText(blobName: string, text: string): Promise<void> {\n        return this.createBlobFromStream(blobName, streamOfString(text));\n    }\n\n    async listBlobs(prefix: string): Promise<BlobService.BlobResult[]> {\n        const once = (tkn: common.ContinuationToken | undefined) =>\n            promisifyErrorOrResult<BlobService.ListBlobsResult>(cb => {\n                this.service.listBlobsSegmentedWithPrefix(azureContainer, prefix, tkn!, cb);\n            });\n\n        const out: BlobService.BlobResult[] = [];\n        let token: common.ContinuationToken | undefined;\n        do {\n            const {entries, continuationToken}: BlobService.ListBlobsResult = await once(token);\n            out.push(...entries);\n            token = continuationToken;\n        } while (token);\n\n        return out;\n    }\n\n    deleteBlob(blobName: string): Promise<void> {\n        return promisifyErrorOrResponse(cb => {\n            this.service.deleteBlob(azureContainer, blobName, cb);\n        });\n    }\n\n    private createBlobFromStream(blobName: string, stream: NodeJS.ReadableStream): Promise<void> {\n        const options: BlobService.CreateBlobRequestOptions =  {\n            contentSettings: {\n                contentEncoding: \"gzip\",\n                contentType: \"application/json; charset=utf-8\",\n            },\n        };\n        // Remove `undefined!` once https://github.com/Azure/azure-storage-node/pull/267 is in\n        return streamDone(gzip(stream).pipe(this.service.createWriteStreamToBlockBlob(azureContainer, blobName, options, undefined!)));\n    }\n}\n\nexport async function readBlob(blobName: string): Promise<string> {\n    return new Promise<string>((resolve, reject) => {\n        const url = urlOfBlob(blobName);\n        const req = https.get(url, res => {\n            switch (res.statusCode) {\n                case 200:\n                    if (res.headers[\"content-encoding\"] !== \"gzip\") {\n                        reject(new Error(`${url} is not gzipped`));\n                    } else {\n                        resolve(stringOfStream(unGzip(res), blobName));\n                    }\n                    break;\n                default:\n                    reject(new Error(`Can't get ${url}: ${res.statusCode} error`));\n            }\n        });\n        req.on(\"error\", reject);\n    });\n}\n\nexport async function readJsonBlob(blobName: string): Promise<object> {\n    return parseJson(await readBlob(blobName));\n}\n\nexport function urlOfBlob(blobName: string): string {\n    return `https://${azureContainer}.blob.core.windows.net/${azureContainer}/${blobName}`;\n}\n\nfunction promisifyErrorOrResult<A>(callsBack: (x: ErrorOrResult<A>) => void): Promise<A> {\n    return new Promise<A>((resolve, reject) => {\n        callsBack((err, result) => {\n            if (err) {\n                reject(err);\n            } else {\n                resolve(result);\n            }\n        });\n    });\n}\n\nfunction promisifyErrorOrResponse(callsBack: (x: ErrorOrResponse) => void): Promise<void> {\n    return new Promise<void>((resolve, reject) => {\n        callsBack(err => {\n            if (err) {\n                reject(err);\n            } else {\n                resolve();\n            }\n        });\n    });\n}\n"
  },
  {
    "path": "src/lib/blob-uploader.ts",
    "content": "import assert = require(\"assert\");\nimport { readdir } from \"fs-extra\";\nimport * as path from \"path\";\n\nimport { Logger, logger, logPath, writeLog } from \"../util/logging\";\nimport { joinPaths, unique } from \"../util/util\";\n\nimport BlobWriter, { urlOfBlob } from \"./azure-container\";\n\nconst maxNumberOfOldLogsDirectories = 5;\n\nexport default async function uploadBlobsAndUpdateIssue(timeStamp: string): Promise<void> {\n    const container = await BlobWriter.create();\n    await container.ensureCreated({ publicAccessLevel: \"blob\" });\n    await container.setCorsProperties();\n    const [dataUrls, logUrls] = await uploadBlobs(container, timeStamp);\n    await uploadIndex(container, timeStamp, dataUrls, logUrls);\n}\n\n// View uploaded files at: https://ms.portal.azure.com under \"typespublisher\"\nasync function uploadBlobs(container: BlobWriter, timeStamp: string): Promise<[string[], string[]]> {\n    const [log, logResult] = logger();\n    const [dataUrls, logUrls] = await Promise.all([\n        await uploadDirectory(container, \"data\", \"data\", log),\n        await uploadLogs(container, timeStamp, log),\n    ]);\n\n    // Finally, output blob logs and upload them.\n    const blobLogs = \"upload-blobs.md\";\n    await writeLog(blobLogs, logResult());\n    logUrls.push(await uploadFile(container, `${logsUploadedLocation(timeStamp)}/${blobLogs}`, logPath(blobLogs)));\n\n    return [dataUrls, logUrls];\n}\n\nconst logsDirectoryName = \"logs\";\nconst logsPrefix = `${logsDirectoryName}/`;\n\nfunction logsUploadedLocation(timeStamp: string): string {\n    return logsPrefix + timeStamp;\n}\n\nasync function uploadLogs(container: BlobWriter, timeStamp: string, log: Logger): Promise<string[]> {\n    await removeOldDirectories(container, logsPrefix, maxNumberOfOldLogsDirectories - 1, log);\n    return uploadDirectory(container, logsUploadedLocation(timeStamp), logsDirectoryName, log, f => f !== \"upload-blobs.md\");\n}\n\nasync function uploadDirectory(\n    container: BlobWriter, uploadedDirPath: string, dirPath: string, log: Logger,\n    filter?: (fileName: string) => boolean): Promise<string[]> {\n\n    let files = await readdir(dirPath);\n    if (filter) {\n        files = files.filter(filter);\n    }\n    return Promise.all(files.map(fileName => {\n        const fullPath = joinPaths(dirPath, fileName);\n        const blobName = joinPaths(uploadedDirPath, fileName);\n        return logAndUploadFile(container, blobName, fullPath, log);\n    }));\n}\n\nasync function logAndUploadFile(container: BlobWriter, blobName: string, filePath: string, log: Logger): Promise<string> {\n    const url = urlOfBlob(blobName);\n    log(`Uploading ${filePath} to ${url}`);\n    await container.createBlobFromFile(blobName, filePath);\n    return url;\n}\nasync function uploadFile(container: BlobWriter, blobName: string, filePath: string): Promise<string> {\n    const url = urlOfBlob(blobName);\n    await container.createBlobFromFile(blobName, filePath);\n    return url;\n}\n\nasync function deleteDirectory(container: BlobWriter, uploadedDirPath: string, log: Logger): Promise<void> {\n    const blobs = await container.listBlobs(uploadedDirPath);\n    const blobNames = blobs.map(b => b.name);\n    log(`Deleting directory ${uploadedDirPath}: delete files ${blobNames.toString()}`);\n    await Promise.all(blobNames.map(b => container.deleteBlob(b)));\n}\n\nasync function removeOldDirectories(container: BlobWriter, prefix: string, maxDirectories: number, log: Logger): Promise<void> {\n    const list = await container.listBlobs(prefix);\n\n    const dirNames = unique(list.map(({name}) => {\n        assert(name.startsWith(prefix));\n        return path.dirname(name.slice(prefix.length));\n    }));\n\n    if (dirNames.length <= maxDirectories) {\n        log(`No need to remove old directories: have ${dirNames.length}, can go up to ${maxDirectories}.`);\n        return;\n    }\n\n    // For ISO 8601 times, sorting lexicographically *is* sorting by time.\n    const sortedNames = dirNames.sort();\n    const toDelete = sortedNames.slice(0, sortedNames.length - maxDirectories);\n\n    log(`Too many old logs, so removing the following directories: [${toDelete.toString()}]`);\n    await Promise.all(toDelete.map(d => deleteDirectory(container, prefix + d, log)));\n}\n\n// Provides links to the latest blobs.\n// These are at: https://typespublisher.blob.core.windows.net/typespublisher/index.html\nfunction uploadIndex(container: BlobWriter, timeStamp: string, dataUrls: ReadonlyArray<string>, logUrls: ReadonlyArray<string>): Promise<void> {\n    return container.createBlobFromText(\"index.html\", createIndex());\n\n    function createIndex(): string {\n        const lines: string[] = [];\n        lines.push(\"<html><head></head><body>\");\n        lines.push(`<h3>Here is the latest data as of **${timeStamp}**:</h3>`);\n        lines.push(\"<h4>Data</h4>\");\n        lines.push(...dataUrls.map(link));\n        lines.push(\"<h4>Logs</h4>\");\n        lines.push(...logUrls.map(link));\n        lines.push(\"</body></html>\");\n        return lines.join(\"\\n\");\n\n        function link(url: string): string {\n            const short = url.slice(url.lastIndexOf(\"/\") + 1);\n            return `<li><a href='${url}'>${short}</a></li>`;\n        }\n    }\n}\n"
  },
  {
    "path": "src/lib/common.ts",
    "content": "import { ensureDir } from \"fs-extra\";\n\nimport { readJson, writeJson } from \"../util/io\";\nimport { joinPaths } from \"../util/util\";\n\nimport { dataDirPath } from \"./settings\";\n\nif (process.env.LONGJOHN) {\n    console.log(\"=== USING LONGJOHN ===\");\n    const longjohn = require(\"longjohn\") as { async_trace_limit: number }; // tslint:disable-line no-var-requires\n    longjohn.async_trace_limit = -1; // unlimited\n}\n\n/** Which registry to publish to */\nexport enum Registry {\n    /** types-registry and @types/* on NPM */\n    NPM,\n    /** @definitelytyped/types-registry and @types/* on Github */\n    Github,\n}\n\n/** Settings that may be determined dynamically. */\nexport interface Options {\n    /**\n     * e.g. '../DefinitelyTyped'\n     * This is overridden to `cwd` when running the tester, as that is run from within DefinitelyTyped.\n     * If undefined, downloads instead.\n     */\n    readonly definitelyTypedPath: string | undefined;\n    /** Whether to show progress bars. Good when running locally, bad when running on travis / azure. */\n    readonly progress: boolean;\n    /** Disabled on azure since it has problems logging errors from other processes. */\n    readonly parseInParallel: boolean;\n}\nexport namespace Options {\n    /** Options for running locally. */\n    export const defaults: TesterOptions = { definitelyTypedPath: \"../DefinitelyTyped\", progress: true, parseInParallel: true };\n    export const azure: Options = { definitelyTypedPath: undefined, progress: false, parseInParallel: false };\n}\nexport interface TesterOptions extends Options {\n    // Tester can only run on files stored on-disk.\n    readonly definitelyTypedPath: string;\n}\n\nexport function readDataFile(generatedBy: string, fileName: string): Promise<object> {\n    return readFileAndWarn(generatedBy, dataFilePath(fileName));\n}\n\n/** If a file doesn't exist, warn and tell the step it should have been generated by. */\nexport async function readFileAndWarn(generatedBy: string, filePath: string): Promise<object> {\n    try {\n        return await readJson(filePath);\n    } catch (e) {\n        console.error(`Run ${generatedBy} first!`);\n        throw e;\n    }\n}\n\nexport async function writeDataFile(filename: string, content: {}, formatted = true): Promise<void> {\n    await ensureDir(dataDirPath);\n    await writeJson(dataFilePath(filename), content, formatted);\n}\n\nexport function dataFilePath(filename: string): string {\n    return joinPaths(dataDirPath, filename);\n}\n"
  },
  {
    "path": "src/lib/definition-parser-worker.ts",
    "content": "import assert = require(\"assert\");\nimport process = require(\"process\");\n\nimport { getLocallyInstalledDefinitelyTyped } from \"../get-definitely-typed\";\nimport { logUncaughtErrors } from \"../util/util\";\n\nimport { getTypingInfo } from \"./definition-parser\";\n\n// This file is \"called\" by runWithChildProcesses from parse-definition.ts\nexport const definitionParserWorkerFilename = __filename;\n\nif (!module.parent) {\n    process.on(\"message\", message => {\n        assert(process.argv.length === 3);\n        const typesPath = process.argv[2];\n        // tslint:disable-next-line no-async-without-await\n        logUncaughtErrors(async () => {\n            for (const packageName of message as ReadonlyArray<string>) {\n                const data = getTypingInfo(packageName, getLocallyInstalledDefinitelyTyped(typesPath).subDir(packageName));\n                process.send!({ data, packageName });\n            }\n        });\n    });\n}\n"
  },
  {
    "path": "src/lib/definition-parser.test.ts",
    "content": "// tslint:disable:object-literal-key-quotes\n\nimport { createMockDT } from \"../mocks\";\n\nimport { getTypingInfo } from \"./definition-parser\";\n\ndescribe(getTypingInfo, () => {\n    it(\"keys data by major.minor version\", () => {\n        const dt = createMockDT();\n        dt.addOldVersionOfPackage(\"jquery\", \"1.42\");\n        dt.addOldVersionOfPackage(\"jquery\", \"2\");\n        const info = getTypingInfo(\"jquery\", dt.pkgFS(\"jquery\"));\n\n        expect(Object.keys(info).sort()).toEqual([\"1.42\", \"2.0\", \"3.3\"]);\n    });\n\n    describe(\"concerning multiple versions\", () => {\n        it(\"records what the version directory looks like on disk\", () => {\n            const dt = createMockDT();\n            dt.addOldVersionOfPackage(\"jquery\", \"2\");\n            dt.addOldVersionOfPackage(\"jquery\", \"1.5\");\n            const info = getTypingInfo(\"jquery\", dt.pkgFS(\"jquery\"));\n\n            expect(info).toEqual({\n                \"1.5\": expect.objectContaining({\n                    libraryVersionDirectoryName: \"1.5\",\n                }),\n                \"2.0\": expect.objectContaining({\n                    libraryVersionDirectoryName: \"2\",\n                }),\n                \"3.3\": expect.objectContaining({\n                    // The latest version does not have its own version directory\n                    libraryVersionDirectoryName: undefined,\n                }),\n            });\n        });\n\n        it(\"records a path mapping to the version directory\", () => {\n            const dt = createMockDT();\n            dt.addOldVersionOfPackage(\"jquery\", \"2\");\n            dt.addOldVersionOfPackage(\"jquery\", \"1.5\");\n            const info = getTypingInfo(\"jquery\", dt.pkgFS(\"jquery\"));\n\n            expect(info).toEqual({\n                \"1.5\": expect.objectContaining({\n                    pathMappings: [{\n                        packageName: \"jquery\",\n                        version: { major: 1, minor: 5 },\n                    }],\n                }),\n                \"2.0\": expect.objectContaining({\n                    pathMappings: [{\n                        packageName: \"jquery\",\n                        version: { major: 2, minor: undefined },\n                    }],\n                }),\n                \"3.3\": expect.objectContaining({\n                    // The latest version does not have path mappings of its own\n                    pathMappings: [],\n                }),\n            });\n        });\n\n        describe(\"validation thereof\", () => {\n            it(\"throws if a directory exists for the latest major version\", () => {\n                const dt = createMockDT();\n                dt.addOldVersionOfPackage(\"jquery\", \"3\");\n\n                expect(() => {\n                    getTypingInfo(\"jquery\", dt.pkgFS(\"jquery\"));\n                }).toThrow(\n                    \"The latest version is 3.3, so the subdirectory 'v3' is not allowed; \" +\n                        \"since it applies to any 3.* version, up to and including 3.3.\",\n                );\n            });\n\n            it(\"throws if a directory exists for the latest minor version\", () => {\n                const dt = createMockDT();\n                dt.addOldVersionOfPackage(\"jquery\", \"3.3\");\n\n                expect(() => {\n                    getTypingInfo(\"jquery\", dt.pkgFS(\"jquery\"));\n                }).toThrow(\n                    \"The latest version is 3.3, so the subdirectory 'v3.3' is not allowed.\",\n                );\n            });\n\n            it(\"does not throw when a minor version is older than the latest\", () => {\n                const dt = createMockDT();\n                dt.addOldVersionOfPackage(\"jquery\", \"3.0\");\n\n                expect(() => {\n                    getTypingInfo(\"jquery\", dt.pkgFS(\"jquery\"));\n                }).not.toThrow();\n            });\n        });\n    });\n});\n"
  },
  {
    "path": "src/lib/definition-parser.ts",
    "content": "import { parseHeaderOrFail, TypeScriptVersion } from \"definitelytyped-header-parser\";\nimport * as ts from \"typescript\";\n\nimport { FS } from \"../get-definitely-typed\";\nimport {\n    computeHash, filter, flatMap, hasWindowsSlashes, join, mapDefined, sort, split, unique, unmangleScopedPackage, withoutStart,\n} from \"../util/util\";\n\nimport { allReferencedFiles, createSourceFile, getModuleInfo, getTestDependencies } from \"./module-info\";\nimport {\n    formatTypingVersion,\n    getLicenseFromPackageJson,\n    PackageId,\n    PackageJsonDependency,\n    PathMapping,\n    TypingsDataRaw,\n    TypingsVersionsRaw,\n    TypingVersion,\n} from \"./packages\";\nimport { dependenciesWhitelist } from \"./settings\";\n\nfunction matchesVersion(typingsDataRaw: TypingsDataRaw, version: TypingVersion, considerLibraryMinorVersion: boolean) {\n    return typingsDataRaw.libraryMajorVersion === version.major\n        && (considerLibraryMinorVersion ?\n            (version.minor === undefined || typingsDataRaw.libraryMinorVersion === version.minor)\n            : true);\n}\n\nfunction formattedLibraryVersion(typingsDataRaw: TypingsDataRaw) {\n    return `${typingsDataRaw.libraryMajorVersion}.${typingsDataRaw.libraryMinorVersion}`;\n}\n\n/** @param fs Rooted at the package's directory, e.g. `DefinitelyTyped/types/abs` */\nexport function getTypingInfo(packageName: string, fs: FS): TypingsVersionsRaw {\n    if (packageName !== packageName.toLowerCase()) {\n        throw new Error(`Package name \\`${packageName}\\` should be strictly lowercase`);\n    }\n    interface OlderVersionDir { readonly directoryName: string; readonly version: TypingVersion; }\n    const [rootDirectoryLs, olderVersionDirectories] = split<string, OlderVersionDir>(fs.readdir(), fileOrDirectoryName => {\n        const version = parseVersionFromDirectoryName(fileOrDirectoryName);\n        return version === undefined ? undefined : { directoryName: fileOrDirectoryName, version };\n    });\n\n    const considerLibraryMinorVersion = olderVersionDirectories.some(({ version }) => version.minor !== undefined);\n\n    const latestData: TypingsDataRaw = {\n        libraryVersionDirectoryName: undefined,\n        ...combineDataForAllTypesVersions(packageName, rootDirectoryLs, fs, undefined),\n    };\n\n    const older = olderVersionDirectories.map(({ directoryName, version: directoryVersion }) => {\n        if (matchesVersion(latestData, directoryVersion, considerLibraryMinorVersion)) {\n            const latest = `${latestData.libraryMajorVersion}.${latestData.libraryMinorVersion}`;\n            throw new Error(\n                `The latest version is ${latest}, so the subdirectory '${directoryName}' is not allowed` +\n                    (`v${latest}` === directoryName ?\n                        \".\" : `; since it applies to any ${latestData.libraryMajorVersion}.* version, up to and including ${latest}.`),\n            );\n        }\n\n        const ls = fs.readdir(directoryName);\n        const data: TypingsDataRaw = {\n            libraryVersionDirectoryName: formatTypingVersion(directoryVersion),\n            ...combineDataForAllTypesVersions(packageName, ls, fs.subDir(directoryName), directoryVersion),\n        };\n\n        if (!matchesVersion(data, directoryVersion, considerLibraryMinorVersion)) {\n            if (considerLibraryMinorVersion) {\n                throw new Error(\n                    `Directory ${directoryName} indicates major.minor version ${directoryVersion.major}.${directoryVersion.minor}, ` +\n                    `but header indicates major.minor version ${data.libraryMajorVersion}.${data.libraryMinorVersion}`,\n                );\n            }\n            throw new Error(\n                `Directory ${directoryName} indicates major version ${directoryVersion.major}, but header indicates major version ` +\n                data.libraryMajorVersion.toString(),\n            );\n        }\n        return data;\n    });\n\n    const res: TypingsVersionsRaw = {};\n    res[formattedLibraryVersion(latestData)] = latestData;\n    for (const o of older) {\n        res[formattedLibraryVersion(o)] = o;\n    }\n    return res;\n}\n\nconst packageJsonName = \"package.json\";\n\ninterface LsMinusTypesVersionsAndPackageJson {\n    readonly remainingLs: ReadonlyArray<string>;\n    readonly typesVersions: ReadonlyArray<TypeScriptVersion>;\n    readonly hasPackageJson: boolean;\n}\n\nfunction getTypesVersionsAndPackageJson(ls: ReadonlyArray<string>): LsMinusTypesVersionsAndPackageJson {\n    const withoutPackageJson = ls.filter(name => name !== packageJsonName);\n    const [remainingLs, typesVersions] = split(withoutPackageJson, fileOrDirectoryName => {\n        const match = /^ts(\\d+\\.\\d+)$/.exec(fileOrDirectoryName);\n        if (match === null) { return undefined; }\n\n        const version = match[1];\n        if (parseInt(version, 10) < 3) {\n            throw new Error(`Directory name starting with 'ts' should be a TypeScript version newer than 3.0. Got: ${version}`);\n        }\n        return version as TypeScriptVersion;\n    });\n    return { remainingLs, typesVersions, hasPackageJson: withoutPackageJson.length !== ls.length };\n}\n\n/**\n * Parses a directory name into a version that either holds a single major version or a major and minor version.\n *\n * @example\n *\n * ```ts\n * parseVersionFromDirectoryName(\"v1\") // { major: 1 }\n * parseVersionFromDirectoryName(\"v0.61\") // { major: 0, minor: 61 }\n * ```\n */\nexport function parseVersionFromDirectoryName(directoryName: string): TypingVersion | undefined {\n    const match = /^v(\\d+)(\\.(\\d+))?$/.exec(directoryName);\n    if (match === null) {\n        return undefined;\n    }\n    return {\n        major: Number(match[1]),\n        minor: match[3] !== undefined ? Number(match[3]) : undefined, // tslint:disable-line strict-type-predicates (false positive)\n    };\n}\n\nfunction combineDataForAllTypesVersions(\n    typingsPackageName: string,\n    ls: ReadonlyArray<string>,\n    fs: FS,\n    directoryVersion: TypingVersion | undefined,\n): Omit<TypingsDataRaw, \"libraryVersionDirectoryName\"> {\n    const { remainingLs, typesVersions, hasPackageJson } = getTypesVersionsAndPackageJson(ls);\n\n    // Every typesVersion has an index.d.ts, but only the root index.d.ts should have a header.\n    const { contributors, libraryMajorVersion, libraryMinorVersion, typeScriptVersion: minTsVersion, libraryName, projects } =\n        parseHeaderOrFail(readFileAndThrowOnBOM(\"index.d.ts\", fs));\n\n    const dataForRoot = getTypingDataForSingleTypesVersion(undefined, typingsPackageName, fs.debugPath(), remainingLs, fs, directoryVersion);\n    const dataForOtherTypesVersions = typesVersions.map(tsVersion => {\n        const subFs = fs.subDir(`ts${tsVersion}`);\n        return getTypingDataForSingleTypesVersion(tsVersion, typingsPackageName, fs.debugPath(), subFs.readdir(), subFs, directoryVersion);\n    });\n    const allTypesVersions = [dataForRoot, ...dataForOtherTypesVersions];\n\n    const packageJson = hasPackageJson ? fs.readJson(packageJsonName) as { readonly license?: unknown, readonly dependencies?: unknown } : {};\n    const license = getLicenseFromPackageJson(packageJson.license);\n    const packageJsonDependencies = checkPackageJsonDependencies(packageJson.dependencies, packageJsonName);\n\n    const files = Array.from(flatMap(allTypesVersions, ({ typescriptVersion, declFiles }) =>\n        declFiles.map(file =>\n            typescriptVersion === undefined ? file : `ts${typescriptVersion}/${file}`)));\n\n    return {\n        libraryName,\n        typingsPackageName,\n        projectName: projects[0], // TODO: collect multiple project names\n        contributors,\n        libraryMajorVersion,\n        libraryMinorVersion,\n        minTsVersion,\n        typesVersions,\n        files,\n        license,\n        // TODO: Explicit type arguments shouldn't be necessary. https://github.com/Microsoft/TypeScript/issues/27507\n        dependencies: getAllUniqueValues<\"dependencies\", PackageId>(allTypesVersions, \"dependencies\"),\n        testDependencies: getAllUniqueValues<\"testDependencies\", string>(allTypesVersions, \"testDependencies\"),\n        pathMappings: getAllUniqueValues<\"pathMappings\", PathMapping>(allTypesVersions, \"pathMappings\"),\n        packageJsonDependencies,\n        contentHash: hash(hasPackageJson ? [...files, packageJsonName] : files, mapDefined(allTypesVersions, a => a.tsconfigPathsForHash), fs),\n        globals: getAllUniqueValues<\"globals\", string>(allTypesVersions, \"globals\"),\n        declaredModules: getAllUniqueValues<\"declaredModules\", string>(allTypesVersions, \"declaredModules\"),\n    };\n}\n\nfunction getAllUniqueValues<K extends string, T>(records: ReadonlyArray<Record<K, ReadonlyArray<T>>>, key: K): ReadonlyArray<T> {\n    return unique(flatMap(records, x => x[key]));\n}\n\ninterface TypingDataFromIndividualTypeScriptVersion {\n    /** Undefined for root (which uses `// TypeScript Version: ` comment instead) */\n    readonly typescriptVersion: TypeScriptVersion | undefined;\n    readonly dependencies: ReadonlyArray<PackageId>;\n    readonly testDependencies: ReadonlyArray<string>;\n    readonly pathMappings: ReadonlyArray<PathMapping>;\n    readonly declFiles: ReadonlyArray<string>;\n    readonly tsconfigPathsForHash: string | undefined;\n    readonly globals: ReadonlyArray<string>;\n    readonly declaredModules: ReadonlyArray<string>;\n}\n\n/**\n * @param typescriptVersion Set if this is in e.g. a `ts3.1` directory.\n * @param packageName Name of the outermost directory; e.g. for \"node/v4\" this is just \"node\".\n * @param ls All file/directory names in `directory`.\n * @param fs FS rooted at the directory for this particular TS version, e.g. `types/abs/ts3.1` or `types/abs` when typescriptVersion is undefined.\n */\nfunction getTypingDataForSingleTypesVersion(\n    typescriptVersion: TypeScriptVersion | undefined,\n    packageName: string,\n    packageDirectory: string,\n    ls: ReadonlyArray<string>,\n    fs: FS,\n    directoryVersion: TypingVersion | undefined,\n): TypingDataFromIndividualTypeScriptVersion {\n    const tsconfig = fs.readJson(\"tsconfig.json\") as TsConfig;\n    checkFilesFromTsConfig(packageName, tsconfig, fs.debugPath());\n    const { types, tests } = allReferencedFiles(tsconfig.files!, fs, packageName, packageDirectory);\n    const usedFiles = new Set([...types.keys(), ...tests.keys(), \"tsconfig.json\", \"tslint.json\"]);\n    const otherFiles = ls.indexOf(unusedFilesName) > -1 ? (fs.readFile(unusedFilesName)).split(/\\r?\\n/g).filter(Boolean) : [];\n    checkAllFilesUsed(ls, usedFiles, otherFiles, packageName, fs);\n    for (const untestedTypeFile of filter(otherFiles, name => name.endsWith(\".d.ts\"))) {\n        // add d.ts files from OTHER_FILES.txt in order get their dependencies\n        types.set(untestedTypeFile, createSourceFile(untestedTypeFile, fs.readFile(untestedTypeFile)));\n    }\n\n    const { dependencies: dependenciesWithDeclaredModules, globals, declaredModules } = getModuleInfo(packageName, types);\n    const declaredModulesSet = new Set(declaredModules);\n    // Don't count an import of \"x\" as a dependency if we saw `declare module \"x\"` somewhere.\n    const dependenciesSet = new Set(filter(dependenciesWithDeclaredModules, m => !declaredModulesSet.has(m)));\n    const testDependencies = Array.from(\n        filter(\n            getTestDependencies(packageName, types, tests.keys(), dependenciesSet, fs),\n            m => !declaredModulesSet.has(m),\n        ),\n    );\n\n    const { dependencies, pathMappings } = calculateDependencies(packageName, tsconfig, dependenciesSet, directoryVersion);\n    const tsconfigPathsForHash = JSON.stringify(tsconfig.compilerOptions.paths);\n    return {\n        typescriptVersion,\n        dependencies,\n        testDependencies,\n        pathMappings,\n        globals,\n        declaredModules,\n        declFiles: sort(types.keys()),\n        tsconfigPathsForHash,\n    };\n}\n\nfunction checkPackageJsonDependencies(dependencies: unknown, path: string): ReadonlyArray<PackageJsonDependency> {\n    if (dependencies === undefined) { // tslint:disable-line strict-type-predicates (false positive)\n        return [];\n    }\n    if (dependencies === null || typeof dependencies !== \"object\") { // tslint:disable-line strict-type-predicates\n        throw new Error(`${path} should contain \"dependencies\" or not exist.`);\n    }\n\n    const deps: PackageJsonDependency[] = [];\n\n    for (const dependencyName of Object.keys(dependencies!)) { // `dependencies` cannot be null because of check above.\n        if (!dependenciesWhitelist.has(dependencyName)) {\n            const msg = dependencyName.startsWith(\"@types/\")\n                ? `Dependency ${dependencyName} not in whitelist.\nDon't use a 'package.json' for @types dependencies unless this package relies on\nan old version of types that have since been moved to the source repo.\nFor example, if package *P* used to have types on Definitely Typed at @types/P,\nbut now has its own types, a dependent package *D* will need to use package.json\nto refer to @types/P if it relies on old versions of P's types.\nIn this case, please make a pull request to types-publisher adding @types/P to \\`dependenciesWhitelist.txt\\`.`\n                : `Dependency ${dependencyName} not in whitelist.\nIf you are depending on another \\`@types\\` package, do *not* add it to a \\`package.json\\`. Path mapping should make the import work.\nFor namespaced dependencies you then have to add a \\`paths\\` mapping from \\`@namespace/library\\` to \\`namespace__library\\` in \\`tsconfig.json\\`.\nIf this is an external library that provides typings,  please make a pull request to types-publisher adding it to \\`dependenciesWhitelist.txt\\`.`;\n            throw new Error(`In ${path}: ${msg}`);\n        }\n\n        const version = (dependencies as { [key: string]: unknown })[dependencyName];\n        if (typeof version !== \"string\") { // tslint:disable-line strict-type-predicates\n            throw new Error(`In ${path}: Dependency version for ${dependencyName} should be a string.`);\n        }\n        deps.push({ name: dependencyName, version });\n    }\n\n    return deps;\n}\n\nfunction checkFilesFromTsConfig(packageName: string, tsconfig: TsConfig, directoryPath: string): void {\n    const tsconfigPath = `${directoryPath}/tsconfig.json`;\n    if (tsconfig.include) {\n        throw new Error(`In tsconfig, don't use \"include\", must use \"files\"`);\n    }\n\n    const files = tsconfig.files;\n    if (!files) {\n        throw new Error(`${tsconfigPath} needs to specify  \"files\"`);\n    }\n    for (const file of files) {\n        if (file.startsWith(\"./\")) {\n            throw new Error(`In ${tsconfigPath}: Unnecessary \"./\" at the start of ${file}`);\n        }\n        if (file.endsWith(\".d.ts\") && file !== \"index.d.ts\") {\n            throw new Error(`${packageName}: Only index.d.ts may be listed explicitly in tsconfig's \"files\" entry.\nOther d.ts files must either be referenced through index.d.ts, tests, or added to OTHER_FILES.txt.`);\n        }\n\n        if (!file.endsWith(\".d.ts\") && !file.startsWith(\"test/\")) {\n            const expectedName = `${packageName}-tests.ts`;\n            if (file !== expectedName && file !== `${expectedName}x`) {\n                const message = file.endsWith(\".ts\") || file.endsWith(\".tsx\")\n                    ? `Expected file '${file}' to be named '${expectedName}' or to be inside a '${directoryPath}/test/' directory`\n                    : (`Unexpected file extension for '${file}' -- expected '.ts' or '.tsx' (maybe this should not be in \"files\", but ` +\n                        \"OTHER_FILES.txt)\");\n                throw new Error(message);\n            }\n        }\n    }\n}\n\ninterface TsConfig {\n    include?: ReadonlyArray<string>;\n    files?: ReadonlyArray<string>;\n    compilerOptions: ts.CompilerOptions;\n}\n\n/** In addition to dependencies found in source code, also get dependencies from tsconfig. */\ninterface DependenciesAndPathMappings { readonly dependencies: ReadonlyArray<PackageId>; readonly pathMappings: ReadonlyArray<PathMapping>; }\nfunction calculateDependencies(\n    packageName: string,\n    tsconfig: TsConfig,\n    dependencyNames: ReadonlySet<string>,\n    directoryVersion: TypingVersion | undefined,\n): DependenciesAndPathMappings {\n    const paths = tsconfig.compilerOptions && tsconfig.compilerOptions.paths || {};\n\n    const dependencies: PackageId[] = [];\n    const pathMappings: PathMapping[] = [];\n\n    for (const dependencyName of Object.keys(paths)) {\n        // Might have a path mapping for \"foo/*\" to support subdirectories\n        const rootDirectory = withoutEnd(dependencyName, \"/*\");\n        if (rootDirectory !== undefined) {\n            if (!(rootDirectory in paths)) {\n                throw new Error(`In ${packageName}: found path mapping for ${dependencyName} but not for ${rootDirectory}`);\n            }\n            continue;\n        }\n\n        const pathMappingList = paths[dependencyName];\n        if (pathMappingList.length !== 1) {\n            throw new Error(`In ${packageName}: Path mapping for ${dependencyName} may only have 1 entry.`);\n        }\n        const pathMapping = pathMappingList[0];\n\n        // Path mapping may be for \"@foo/bar\" -> \"foo__bar\".\n        const scopedPackageName = unmangleScopedPackage(pathMapping);\n        if (scopedPackageName !== undefined) {\n            if (dependencyName !== scopedPackageName) {\n                throw new Error(`Expected directory ${pathMapping} to be the path mapping for ${dependencyName}`);\n            }\n            continue;\n        }\n\n        const pathMappingVersion = parseDependencyVersionFromPath(dependencyName, dependencyName, pathMapping);\n        if (dependencyName === packageName) {\n            if (directoryVersion === undefined) {\n                throw new Error(`In ${packageName}: Latest version of a package should not have a path mapping for itself.`);\n            }\n            if (\n                directoryVersion.major !== pathMappingVersion.major\n                || directoryVersion.minor !== pathMappingVersion.minor\n             ) {\n                const correctPathMapping = [`${dependencyName}/v${formatTypingVersion(directoryVersion)}`];\n                throw new Error(`In ${packageName}: Must have a \"paths\" entry of \"${dependencyName}\": ${JSON.stringify(correctPathMapping)}`);\n            }\n        } else {\n            if (dependencyNames.has(dependencyName)) {\n                dependencies.push({ name: dependencyName, version: pathMappingVersion });\n            }\n        }\n        // Else, the path mapping may be necessary if it is for a transitive dependency. We will check this in check-parse-results.\n        pathMappings.push({ packageName: dependencyName, version: pathMappingVersion });\n    }\n\n    if (directoryVersion !== undefined && !(paths && packageName in paths)) {\n        const mapping = JSON.stringify([`${packageName}/v${formatTypingVersion(directoryVersion)}`]);\n        throw new Error(\n            `${packageName}: Older version ${formatTypingVersion(directoryVersion)} must have a \"paths\" entry of \"${packageName}\": ${mapping}`,\n        );\n    }\n\n    for (const dependency of dependencyNames) {\n        if (!dependencies.some(d => d.name === dependency) && !nodeBuiltins.has(dependency)) {\n            dependencies.push({ name: dependency, version: \"*\" });\n        }\n    }\n\n    return { dependencies, pathMappings };\n}\n\nconst nodeBuiltins: ReadonlySet<string> = new Set([\n    \"assert\", \"async_hooks\", \"buffer\", \"child_process\", \"cluster\", \"console\", \"constants\", \"crypto\",\n    \"dgram\", \"dns\", \"domain\", \"events\", \"fs\", \"http\", \"http2\", \"https\", \"module\", \"net\", \"os\",\n    \"path\", \"perf_hooks\", \"process\", \"punycode\", \"querystring\", \"readline\", \"repl\", \"stream\",\n    \"string_decoder\", \"timers\", \"tls\", \"tty\", \"url\", \"util\", \"v8\", \"vm\", \"zlib\",\n]);\n\nfunction parseDependencyVersionFromPath(packageName: string, dependencyName: string, dependencyPath: string): TypingVersion {\n    const versionString = withoutStart(dependencyPath, `${dependencyName}/`);\n    const version = versionString === undefined ? undefined : parseVersionFromDirectoryName(versionString);\n    if (version === undefined) {\n        throw new Error(`In ${packageName}, unexpected path mapping for ${dependencyName}: '${dependencyPath}'`);\n    }\n    return version;\n}\n\nfunction withoutEnd(s: string, end: string): string | undefined {\n    if (s.endsWith(end)) {\n        return s.slice(0, s.length - end.length);\n    }\n    return undefined;\n}\n\nfunction hash(files: ReadonlyArray<string>, tsconfigPathsForHash: ReadonlyArray<string>, fs: FS): string {\n    const fileContents = files.map(f => `${f}**${readFileAndThrowOnBOM(f, fs)}`);\n    let allContent = fileContents.join(\"||\");\n    for (const path of tsconfigPathsForHash) {\n        allContent += path;\n    }\n    return computeHash(allContent);\n}\n\nexport function readFileAndThrowOnBOM(fileName: string, fs: FS): string {\n    const text = fs.readFile(fileName);\n    if (text.charCodeAt(0) === 0xFEFF) {\n        const commands = [\n            \"npm install -g strip-bom-cli\",\n            `strip-bom ${fileName} > fix`,\n            `mv fix ${fileName}`,\n        ];\n        throw new Error(`File '${fileName}' has a BOM. Try using:\\n${commands.join(\"\\n\")}`);\n    }\n    return text;\n}\n\nconst unusedFilesName = \"OTHER_FILES.txt\";\n\nfunction checkAllFilesUsed(ls: ReadonlyArray<string>, usedFiles: Set<string>, otherFiles: string[], packageName: string, fs: FS): void {\n    // Double-check that no windows \"\\\\\" broke in.\n    for (const fileName of usedFiles) {\n        if (hasWindowsSlashes(fileName)) {\n            throw new Error(`In ${packageName}: windows slash detected in ${fileName}`);\n        }\n    }\n    checkAllUsedRecur(new Set(ls), usedFiles,  new Set(otherFiles), fs);\n}\n\nfunction checkAllUsedRecur(ls: Iterable<string>, usedFiles: Set<string>, unusedFiles: Set<string>, fs: FS): void {\n    for (const lsEntry of ls) {\n        if (usedFiles.has(lsEntry)) {\n            continue;\n        }\n        if (unusedFiles.has(lsEntry)) {\n            unusedFiles.delete(lsEntry);\n            continue;\n        }\n\n        if (fs.isDirectory(lsEntry)) {\n            const subdir = fs.subDir(lsEntry);\n            // We allow a \"scripts\" directory to be used for scripts.\n            if (lsEntry === \"node_modules\" || lsEntry === \"scripts\") {\n                continue;\n            }\n\n            const lssubdir = subdir.readdir();\n            if (lssubdir.length === 0) {\n                // tslint:disable-next-line strict-string-expressions\n                throw new Error(`Empty directory ${subdir} (${join(usedFiles)})`);\n            }\n\n            function takeSubdirectoryOutOfSet(originalSet: Set<string>): Set<string> {\n                const subdirSet = new Set<string>();\n                for (const file of originalSet) {\n                    const sub = withoutStart(file, `${lsEntry}/`);\n                    if (sub !== undefined) {\n                        originalSet.delete(file);\n                        subdirSet.add(sub);\n                    }\n                }\n                return subdirSet;\n            }\n            checkAllUsedRecur(lssubdir, takeSubdirectoryOutOfSet(usedFiles), takeSubdirectoryOutOfSet(unusedFiles), subdir);\n        } else {\n            if (lsEntry.toLowerCase() !== \"readme.md\" && lsEntry !== \"NOTICE\" && lsEntry !== \".editorconfig\" && lsEntry !== unusedFilesName) {\n                throw new Error(`Unused file ${fs.debugPath()}/${lsEntry} (used files: ${JSON.stringify(Array.from(usedFiles))})`);\n            }\n        }\n    }\n\n    for (const unusedFile of unusedFiles) {\n        if (usedFiles.has(unusedFile)) {\n            throw new Error(`File ${fs.debugPath()}/${unusedFile} listed in ${unusedFilesName} is already reachable from tsconfig.json.`);\n        }\n        throw new Error(`File ${fs.debugPath()}/${unusedFile} listed in ${unusedFilesName} does not exist.`);\n    }\n}\n"
  },
  {
    "path": "src/lib/module-info.test.ts",
    "content": "import * as ts from \"typescript\";\n\nimport { Dir, InMemoryDT } from \"../get-definitely-typed\";\nimport { createMockDT } from \"../mocks\";\nimport { testo } from \"../util/test\";\n\nimport { allReferencedFiles, getModuleInfo, getTestDependencies } from \"./module-info\";\nconst fs = createMockDT().fs;\nfunction getBoringReferences() {\n    return allReferencedFiles([\"index.d.ts\", \"boring-tests.ts\"], fs.subDir(\"types\").subDir(\"boring\"), \"boring\", \"types/boring\");\n}\ntesto({\n    allReferencedFilesFromTsconfigFiles() {\n        const { types, tests } = getBoringReferences();\n        expect(Array.from(types.keys())).toEqual([\"index.d.ts\", \"secondary.d.ts\", \"quaternary.d.ts\", \"tertiary.d.ts\", \"commonjs.d.ts\", \"v1.d.ts\"]);\n        expect(Array.from(tests.keys())).toEqual([\"boring-tests.ts\"]);\n    },\n    allReferencedFilesFromTestIncludesSecondaryInternalFiles() {\n        const { types, tests } = allReferencedFiles([\"boring-tests.ts\"], fs.subDir(\"types\").subDir(\"boring\"), \"boring\", \"types/boring\");\n        expect(Array.from(types.keys())).toEqual([\"secondary.d.ts\", \"quaternary.d.ts\", \"tertiary.d.ts\", \"commonjs.d.ts\", \"v1.d.ts\"]);\n        expect(Array.from(tests.keys())).toEqual([\"boring-tests.ts\"]);\n    },\n    allReferencedFilesFromTsconfigGlobal() {\n        const { types, tests } = allReferencedFiles([\"jquery-tests.ts\", \"index.d.ts\"], fs.subDir(\"types\").subDir(\"jquery\"), \"jquery\", \"types/jquery\");\n        expect(Array.from(types.keys())).toEqual([\"index.d.ts\", \"JQuery.d.ts\"]);\n        expect(Array.from(tests.keys())).toEqual([\"jquery-tests.ts\"]);\n    },\n    allReferencedFilesFromTestIncludesSecondaryTripleSlashTypes() {\n        const { types, tests } = allReferencedFiles(\n            [\"globby-tests.ts\", \"test/other-tests.ts\"],\n            fs.subDir(\"types\").subDir(\"globby\"),\n            \"globby\",\n            \"types/globby\",\n        );\n        expect(Array.from(types.keys())).toEqual([\"merges.d.ts\"]);\n        expect(Array.from(tests.keys())).toEqual([\"globby-tests.ts\", \"test/other-tests.ts\"]);\n    },\n    getModuleInfoWorksWithOtherFiles() {\n        const { types } = getBoringReferences();\n        // written as if it were from OTHER_FILES.txt\n        types.set(\n            \"untested.d.ts\",\n            ts.createSourceFile(\"untested.d.ts\", fs.subDir(\"types\").subDir(\"boring\").readFile(\"untested.d.ts\"), ts.ScriptTarget.Latest, false),\n        );\n        const i = getModuleInfo(\"boring\", types);\n        expect(i.dependencies).toEqual(new Set([\"manual\", \"react\", \"react-default\", \"things\", \"vorticon\"]));\n    },\n    getModuleInfoForNestedTypeReferences() {\n        const { types } = allReferencedFiles(\n            [\"index.d.ts\", \"globby-tests.ts\", \"test/other-tests.ts\"],\n            fs.subDir(\"types\").subDir(\"globby\"),\n            \"globby\",\n            \"types/globby\",\n        );\n        expect(Array.from(types.keys())).toEqual([\"index.d.ts\", \"sneaky.d.ts\", \"merges.d.ts\"]);\n        const i = getModuleInfo(\"globby\", types);\n        expect(i.dependencies).toEqual(new Set([\"andere\"]));\n    },\n    versionTypeRefThrows() {\n        const fail = new Dir(undefined);\n        const memFS = new InMemoryDT(fail, \"typeref-fails\");\n        fail.set(\"index.d.ts\", `// Type definitions for fail 1.0\n// Project: https://youtube.com/typeref-fails\n// Definitions by: Type Ref Fails <https://github.com/typeref-fails>\n// Definitions: https://github.com/DefinitelyTyped/DefinitelyTyped\n\n/// <reference types=\"elser/v3\" />\n`);\n        const { types } = allReferencedFiles([\"index.d.ts\"], memFS, \"typeref-fails\", \"types/typeref-fails\");\n        expect(Array.from(types.keys())).toEqual([\"index.d.ts\"]);\n        expect(() => getModuleInfo(\"typeref-fails\", types)).toThrow(\"do not directly import specific versions of another types package\");\n    },\n    selfVersionTypeRefAllowed() {\n        const fail = new Dir(undefined);\n        const memFS = new InMemoryDT(fail, \"typeref-fails\");\n        fail.set(\"index.d.ts\", `// Type definitions for fail 1.0\n// Project: https://youtube.com/typeref-fails\n// Definitions by: Type Ref Fails <https://github.com/typeref-fails>\n// Definitions: https://github.com/DefinitelyTyped/DefinitelyTyped\n\n/// <reference types=\"fail/v3\" />\n`);\n        const { types } = allReferencedFiles([\"index.d.ts\"], memFS, \"typeref-fails\", \"types/typeref-fails\");\n        expect(Array.from(types.keys())).toEqual([\"index.d.ts\"]);\n        const i = getModuleInfo(\"fail\", types);\n        expect(i.dependencies).toEqual(new Set([]));\n    },\n    getTestDependenciesWorks() {\n        const { types, tests } = getBoringReferences();\n        const i = getModuleInfo(\"boring\", types);\n        const d = getTestDependencies(\"boring\", types, tests.keys(), i.dependencies, fs.subDir(\"types\").subDir(\"boring\"));\n        expect(d).toEqual(new Set([\"super-big-fun-hus\"]));\n    },\n});\n"
  },
  {
    "path": "src/lib/module-info.ts",
    "content": "import assert = require(\"assert\");\nimport * as path from \"path\";\nimport * as ts from \"typescript\";\n\nimport { FS } from \"../get-definitely-typed\";\nimport { hasWindowsSlashes, joinPaths, normalizeSlashes, sort } from \"../util/util\";\n\nimport { readFileAndThrowOnBOM } from \"./definition-parser\";\n\nexport function getModuleInfo(packageName: string, all: Map<string, ts.SourceFile>): ModuleInfo {\n\n    const dependencies = new Set<string>();\n    const declaredModules: string[] = [];\n    const globals = new Set<string>();\n\n    function addDependency(ref: string): void {\n        if (ref.startsWith(\".\")) { return; }\n        const dependency = rootName(ref, all, packageName);\n        if (dependency !== packageName) {\n            dependencies.add(dependency);\n        }\n        // TODO: else throw new Error(`Package ${packageName} references itself. (via ${src.fileName})`);\n    }\n\n    for (const sourceFile of all.values()) {\n        for (const ref of imports(sourceFile)) {\n            addDependency(ref);\n        }\n        for (const ref of sourceFile.typeReferenceDirectives) {\n            addDependency(ref.fileName);\n        }\n        if (ts.isExternalModule(sourceFile)) {\n            if (sourceFileExportsSomething(sourceFile)) {\n                declaredModules.push(properModuleName(packageName, sourceFile.fileName));\n                const namespaceExport = sourceFile.statements.find(ts.isNamespaceExportDeclaration);\n                if (namespaceExport) {\n                    globals.add(namespaceExport.name.text);\n                }\n            }\n        } else {\n            for (const node of sourceFile.statements) {\n                switch (node.kind) {\n                    case ts.SyntaxKind.ModuleDeclaration: {\n                        const decl = node as ts.ModuleDeclaration;\n                        const name = decl.name.text;\n                        if (decl.name.kind === ts.SyntaxKind.StringLiteral) {\n                            declaredModules.push(assertNoWindowsSlashes(packageName, name));\n                        } else if (isValueNamespace(decl)) {\n                            globals.add(name);\n                        }\n                        break;\n                    }\n                    case ts.SyntaxKind.VariableStatement:\n                        for (const decl of (node as ts.VariableStatement).declarationList.declarations) {\n                            if (decl.name.kind === ts.SyntaxKind.Identifier) {\n                                globals.add(decl.name.text);\n                            }\n                        }\n                        break;\n                    case ts.SyntaxKind.EnumDeclaration:\n                    case ts.SyntaxKind.ClassDeclaration:\n                    case ts.SyntaxKind.FunctionDeclaration: {\n                        // Deliberately not doing this for types, because those won't show up in JS code and can't be used for ATA\n                        const nameNode = (node as ts.EnumDeclaration | ts.ClassDeclaration | ts.FunctionDeclaration).name;\n                        if (nameNode) {\n                            globals.add(nameNode.text);\n                        }\n                        break;\n                    }\n                    case ts.SyntaxKind.ImportEqualsDeclaration:\n                    case ts.SyntaxKind.InterfaceDeclaration:\n                    case ts.SyntaxKind.TypeAliasDeclaration:\n                        break;\n                    default:\n                        throw new Error(`Unexpected node kind ${ts.SyntaxKind[node.kind]}`);\n                }\n            }\n        }\n    }\n\n    return { dependencies, declaredModules, globals: sort(globals) };\n}\n\n/**\n * A file is a proper module if it is an external module *and* it has at least one export.\n * A module with only imports is not a proper module; it likely just augments some other module.\n */\nfunction sourceFileExportsSomething({ statements }: ts.SourceFile): boolean {\n    return statements.some(statement => {\n        switch (statement.kind) {\n            case ts.SyntaxKind.ImportEqualsDeclaration:\n            case ts.SyntaxKind.ImportDeclaration:\n                return false;\n            case ts.SyntaxKind.ModuleDeclaration:\n                return (statement as ts.ModuleDeclaration).name.kind === ts.SyntaxKind.Identifier;\n            default:\n                return true;\n        }\n    });\n}\n\ninterface ModuleInfo {\n    dependencies: Set<string>;\n    // Anything from a `declare module \"foo\"`\n    declaredModules: string[];\n    // Every global symbol\n    globals: string[];\n}\n\n/**\n * Given a file name, get the name of the module it declares.\n * `foo/index.d.ts` declares \"foo\", `foo/bar.d.ts` declares \"foo/bar\", \"foo/bar/index.d.ts\" declares \"foo/bar\"\n */\nfunction properModuleName(folderName: string, fileName: string): string {\n    const part = path.basename(fileName) === \"index.d.ts\" ? path.dirname(fileName) : withoutExtension(fileName, \".d.ts\");\n    return part === \".\" ? folderName : joinPaths(folderName, part);\n}\n\n/**\n * \"foo/bar/baz\" -> \"foo\"; \"@foo/bar/baz\" -> \"@foo/bar\"\n * Note: Throws an error for references like\n * \"bar/v3\" because referencing old versions of *other* packages is illegal;\n * those directories won't exist in the published @types package.\n */\nfunction rootName(importText: string, typeFiles: Map<string, unknown>, packageName: string): string {\n    let slash = importText.indexOf(\"/\");\n    // Root of `@foo/bar/baz` is `@foo/bar`\n    if (importText.startsWith(\"@\")) {\n        // Use second \"/\"\n        slash = importText.indexOf(\"/\", slash + 1);\n    }\n    const root = importText.slice(0, slash);\n    const postImport = importText.slice(slash + 1);\n    if (slash > -1 && postImport.match(/v\\d+$/) && !typeFiles.has(postImport + \".d.ts\") && root !== packageName) {\n        throw new Error(`${importText}: do not directly import specific versions of another types package.\nYou should work with the latest version of ${root} instead.`);\n    }\n    return slash === -1 ? importText : root;\n}\n\nfunction withoutExtension(str: string, ext: string): string {\n    assert(str.endsWith(ext));\n    return str.slice(0, str.length - ext.length);\n}\n\n/** Returns a map from filename (path relative to `directory`) to the SourceFile we parsed for it. */\nexport function allReferencedFiles(\n    entryFilenames: ReadonlyArray<string>, fs: FS, packageName: string, baseDirectory: string,\n): { types: Map<string, ts.SourceFile>, tests: Map<string, ts.SourceFile> } {\n    const seenReferences = new Set<string>();\n    const types = new Map<string, ts.SourceFile>();\n    const tests = new Map<string, ts.SourceFile>();\n    entryFilenames.forEach(text => recur({ text, exact: true }));\n    return { types, tests };\n\n    function recur({ text, exact }: Reference): void {\n        if (seenReferences.has(text)) {\n            return;\n        }\n        seenReferences.add(text);\n\n        const resolvedFilename = exact ? text : resolveModule(text, fs);\n        if (fs.exists(resolvedFilename)) {\n            const src = createSourceFile(resolvedFilename, readFileAndThrowOnBOM(resolvedFilename, fs));\n            if (resolvedFilename.endsWith(\".d.ts\")) {\n                types.set(resolvedFilename, src);\n            } else {\n                tests.set(resolvedFilename, src);\n            }\n\n            const refs = findReferencedFiles(\n                src,\n                packageName,\n                path.dirname(resolvedFilename),\n                normalizeSlashes(path.relative(baseDirectory, fs.debugPath())),\n            );\n            refs.forEach(recur);\n        }\n    }\n\n}\n\nfunction resolveModule(importSpecifier: string, fs: FS): string {\n    importSpecifier = importSpecifier.endsWith(\"/\") ? importSpecifier.slice(0, importSpecifier.length - 1) : importSpecifier;\n    if (importSpecifier !== \".\" && importSpecifier !== \"..\") {\n        if (fs.exists(importSpecifier + \".d.ts\")) {\n            return importSpecifier + \".d.ts\";\n        }\n        if (fs.exists(importSpecifier + \".ts\")) {\n            return importSpecifier + \".ts\";\n        }\n        if (fs.exists(importSpecifier + \".tsx\")) {\n            return importSpecifier + \".tsx\";\n        }\n    }\n    return importSpecifier === \".\" ? \"index.d.ts\" : joinPaths(importSpecifier, \"index.d.ts\");\n\n}\n\ninterface Reference {\n    /** <reference path> includes exact filename, so true. import \"foo\" may reference \"foo.d.ts\" or \"foo/index.d.ts\", so false. */\n    readonly exact: boolean;\n    text: string;\n}\n\n/**\n * @param subDirectory The specific directory within the DefinitelyTyped directory we are in.\n * For example, `baseDirectory` may be `react-router` and `subDirectory` may be `react-router/lib`.\n * versionsBaseDirectory may be \"\" when not in typesVersions or \"..\" when inside `react-router/ts3.1`\n */\nfunction findReferencedFiles(src: ts.SourceFile, packageName: string, subDirectory: string, baseDirectory: string) {\n    const refs: Reference[] = [];\n\n    for (const ref of src.referencedFiles) {\n        // Any <reference path=\"foo\"> is assumed to be local\n        addReference({ text: ref.fileName, exact: true });\n    }\n    for (const ref of src.typeReferenceDirectives) {\n        // only <reference types=\"../packagename/x\" /> references are local (or \"packagename/x\", though in 3.7 that doesn't work in DT).\n        if (ref.fileName.startsWith(\"../\" + packageName + \"/\")) {\n            addReference({ text: ref.fileName, exact: false });\n        } else if (ref.fileName.startsWith(packageName + \"/\")) {\n            addReference({ text: convertToRelativeReference(ref.fileName), exact: false });\n        }\n    }\n\n    for (const ref of imports(src)) {\n        if (ref.startsWith(\".\")) {\n            addReference({ text: ref, exact: false });\n        }\n        if (ref.startsWith(packageName + \"/\")) {\n            addReference({ text: convertToRelativeReference(ref), exact: false });\n        }\n    }\n    return refs;\n\n    function addReference(ref: Reference): void {\n        // `path.normalize` may add windows slashes\n        const full = normalizeSlashes(path.normalize(joinPaths(subDirectory, assertNoWindowsSlashes(src.fileName, ref.text))));\n        // allow files in typesVersions directories (i.e. 'ts3.1') to reference files in parent directory\n        if (full.startsWith(\"../\" + packageName + \"/\")) {\n            ref.text = full.slice(packageName.length + 4);\n            refs.push(ref);\n            return;\n        }\n        if (full.startsWith(\"..\")\n            && (baseDirectory === \"\" || path.normalize(joinPaths(baseDirectory, full)).startsWith(\"..\"))) {\n            throw new Error(\n                `${src.fileName}: ` +\n                'Definitions must use global references to other packages, not parent (\"../xxx\") references.' +\n                `(Based on reference '${ref.text}')`);\n        }\n        ref.text = full;\n        refs.push(ref);\n    }\n\n    /** boring/foo -> ./foo when subDirectory === '.'; ../foo when it's === 'x'; ../../foo when it's 'x/y' */\n    function convertToRelativeReference(name: string) {\n        const relative = \".\" + \"/..\".repeat(subDirectory === \".\" ? 0 : subDirectory.split(\"/\").length);\n        return relative + name.slice(packageName.length);\n    }\n}\n\n/**\n * All strings referenced in `import` statements.\n * Does *not* include <reference> directives.\n */\nfunction* imports({ statements }: ts.SourceFile | ts.ModuleBlock): Iterable<string> {\n    for (const node of statements) {\n        switch (node.kind) {\n            case ts.SyntaxKind.ImportDeclaration:\n            case ts.SyntaxKind.ExportDeclaration: {\n                const { moduleSpecifier } = node as ts.ImportDeclaration | ts.ExportDeclaration;\n                if (moduleSpecifier && moduleSpecifier.kind === ts.SyntaxKind.StringLiteral) {\n                    yield (moduleSpecifier as ts.StringLiteral).text;\n                }\n                break;\n            }\n\n            case ts.SyntaxKind.ImportEqualsDeclaration: {\n                const { moduleReference } = node as ts.ImportEqualsDeclaration;\n                if (moduleReference.kind === ts.SyntaxKind.ExternalModuleReference) {\n                    yield parseRequire(moduleReference);\n                }\n                break;\n            }\n\n            case ts.SyntaxKind.ModuleDeclaration: {\n                const { name, body } = node as ts.ModuleDeclaration;\n                if (name.kind === ts.SyntaxKind.StringLiteral && body) {\n                    yield* imports(body as ts.ModuleBlock);\n                }\n                break;\n            }\n\n            default:\n        }\n    }\n}\n\nfunction parseRequire(reference: ts.ExternalModuleReference): string {\n    const { expression } = reference;\n    if (!expression || !ts.isStringLiteral(expression)) {\n        throw new Error(`Bad 'import =' reference: ${reference.getText()}`);\n    }\n    return expression.text;\n}\n\nfunction isValueNamespace(ns: ts.ModuleDeclaration): boolean {\n    if (!ns.body) {\n        throw new Error(\"@types should not use shorthand ambient modules\");\n    }\n    return ns.body.kind === ts.SyntaxKind.ModuleDeclaration\n        ? isValueNamespace(ns.body as ts.ModuleDeclaration)\n        : (ns.body as ts.ModuleBlock).statements.some(statementDeclaresValue);\n}\n\nfunction statementDeclaresValue(statement: ts.Statement): boolean {\n    switch (statement.kind) {\n        case ts.SyntaxKind.VariableStatement:\n        case ts.SyntaxKind.ClassDeclaration:\n        case ts.SyntaxKind.FunctionDeclaration:\n        case ts.SyntaxKind.EnumDeclaration:\n            return true;\n\n        case ts.SyntaxKind.ModuleDeclaration:\n            return isValueNamespace(statement as ts.ModuleDeclaration);\n\n        case ts.SyntaxKind.InterfaceDeclaration:\n        case ts.SyntaxKind.TypeAliasDeclaration:\n        case ts.SyntaxKind.ImportEqualsDeclaration:\n            return false;\n\n        default:\n            throw new Error(`Forgot to implement ambient namespace statement ${ts.SyntaxKind[statement.kind]}`);\n    }\n}\n\nfunction assertNoWindowsSlashes(packageName: string, fileName: string): string {\n    if (hasWindowsSlashes(fileName)) {\n        throw new Error(`In ${packageName}: Use forward slash instead when referencing ${fileName}`);\n    }\n    return fileName;\n}\n\nexport function getTestDependencies(\n    packageName: string,\n    typeFiles: Map<string, unknown>,\n    testFiles: Iterable<string>,\n    dependencies: ReadonlySet<string>,\n    fs: FS,\n): Iterable<string> {\n    const testDependencies = new Set<string>();\n    for (const filename of testFiles) {\n        const content = readFileAndThrowOnBOM(filename, fs);\n        const sourceFile = createSourceFile(filename, content);\n        const { fileName, referencedFiles, typeReferenceDirectives } = sourceFile;\n        const filePath = () => path.join(packageName, fileName);\n        let hasImports = false;\n        let isModule = false;\n        let referencesSelf = false;\n\n        for (const { fileName: ref } of referencedFiles) {\n            throw new Error(`Test files should not use '<reference path=\"\" />'. '${filePath()}' references '${ref}'.`);\n        }\n        for (const { fileName: referencedPackage } of typeReferenceDirectives) {\n            if (dependencies.has(referencedPackage)) {\n                throw new Error(\n                    `'${filePath()}' unnecessarily references '${referencedPackage}', which is already referenced in the type definition.`);\n            }\n            if (referencedPackage === packageName) {\n                referencesSelf = true;\n            }\n            testDependencies.add(referencedPackage);\n        }\n        for (const imported of imports(sourceFile)) {\n            hasImports = true;\n            if (!imported.startsWith(\".\")) {\n                const dep = rootName(imported, typeFiles, packageName);\n                if (!dependencies.has(dep) && dep !== packageName) {\n                    testDependencies.add(dep);\n                }\n            }\n        }\n\n        isModule = hasImports || (() => {\n            // FIXME: This results in files without imports to be walked twice,\n            // once in the `imports(...)` function, and once more here:\n            for (const node of sourceFile.statements) {\n                if (\n                    node.kind === ts.SyntaxKind.ExportAssignment ||\n                    node.kind === ts.SyntaxKind.ExportDeclaration\n                ) {\n                    return true;\n                }\n            }\n            return false;\n        })();\n\n        if (isModule && referencesSelf) {\n            throw new Error(`'${filePath()}' unnecessarily references the package. This can be removed.`);\n        }\n    }\n    return testDependencies;\n}\n\nexport function createSourceFile(filename: string, content: string): ts.SourceFile {\n    return ts.createSourceFile(filename, content, ts.ScriptTarget.Latest, /*setParentNodes*/false);\n}\n"
  },
  {
    "path": "src/lib/npm-client.ts",
    "content": "import assert = require(\"assert\");\nimport { ensureFile, pathExists } from \"fs-extra\";\nimport RegClient = require(\"npm-registry-client\");\nimport { resolve as resolveUrl } from \"url\";\n\nimport { Fetcher, readFile, readJson, sleep, writeJson } from \"../util/io\";\nimport { Logger, loggerWithErrors } from \"../util/logging\";\nimport { createTgz } from \"../util/tgz\";\nimport { assertNever, identity, joinPaths, mapToRecord, recordToMap } from \"../util/util\";\n\nimport { Registry } from \"./common\";\nimport { getSecret, Secret } from \"./secrets\";\nimport { githubRegistry, npmApi, npmRegistry, npmRegistryHostName } from \"./settings\";\n\nconst cacheFile = joinPaths(__dirname, \"..\", \"..\", \"cache\", \"npmInfo.json\");\n\nexport type NpmInfoCache = ReadonlyMap<string, NpmInfo>;\n\nexport interface NpmInfoRaw {\n    readonly \"dist-tags\": {\n        readonly [tag: string]: string;\n    };\n    readonly versions: NpmInfoRawVersions;\n    readonly time: {\n        readonly [s: string]: string;\n    };\n}\nexport interface NpmInfoRawVersions {\n    readonly [version: string]: NpmInfoVersion;\n}\n\n// Processed npm info. Intentially kept small so it can be cached.\nexport interface NpmInfo {\n    readonly distTags: Map<string, string>;\n    readonly versions: Map<string, NpmInfoVersion>;\n    readonly time: Map<string, string>;\n}\nexport interface NpmInfoVersion {\n    readonly typesPublisherContentHash?: string;\n    readonly deprecated?: string;\n}\n\nexport interface CachedNpmInfoClient {\n    getNpmInfoFromCache(escapedPackageName: string): NpmInfo | undefined;\n    fetchAndCacheNpmInfo(escapedPackageName: string): Promise<NpmInfo | undefined>;\n}\n\nexport async function withNpmCache<T>(uncachedClient: UncachedNpmInfoClient, cb: (client: CachedNpmInfoClient) => Promise<T>): Promise<T> {\n    const log = loggerWithErrors()[0];\n    let unroll: Map<string, NpmInfo>;\n    log.info(`Checking for cache file at ${cacheFile}...`);\n    const cacheFileExists = await pathExists(cacheFile);\n    if (cacheFileExists) {\n        log.info(\"Reading cache file...\");\n        const cachedJson = await readJson(cacheFile) as Record<string, NpmInfoRaw>;\n        log.info(`Cache file ${cacheFile} exists, copying to map...`);\n        unroll = recordToMap(cachedJson, npmInfoFromJson);\n    } else {\n        log.info(\"Cache file doesn't exist, using empty map.\");\n        unroll = new Map();\n    }\n\n    const res = await cb({ getNpmInfoFromCache, fetchAndCacheNpmInfo });\n    log.info(\"Writing npm cache.\");\n    await ensureFile(cacheFile);\n    await writeJson(cacheFile, mapToRecord(unroll, jsonFromNpmInfo));\n    return res;\n\n    /** May return old info -- caller should check that this looks up-to-date. */\n    function getNpmInfoFromCache(escapedPackageName: string): NpmInfo | undefined {\n        return unroll.get(escapedPackageName);\n    }\n\n    /** Call this when the result of getNpmInfoFromCache looks potentially out-of-date. */\n    async function fetchAndCacheNpmInfo(escapedPackageName: string): Promise<NpmInfo | undefined> {\n        const info = await uncachedClient.fetchNpmInfo(escapedPackageName);\n        if (info) { unroll.set(escapedPackageName, info); }\n        return info;\n    }\n\n}\n\nexport class UncachedNpmInfoClient {\n    private readonly fetcher = new Fetcher();\n\n    async fetchNpmInfo(escapedPackageName: string): Promise<NpmInfo | undefined> {\n        const raw = await this.fetchRawNpmInfo(escapedPackageName);\n        await sleep(0.01); // If we don't do this, npm resets the connection?\n        return raw === undefined ? undefined : npmInfoFromJson(raw);\n    }\n\n    async fetchRawNpmInfo(escapedPackageName: string): Promise<NpmInfoRaw | undefined> {\n        const info = await this.fetcher.fetchJson({\n            hostname: npmRegistryHostName,\n            path: escapedPackageName,\n            retries: true,\n        }) as { readonly error: string } | NpmInfoRaw;\n        if (\"error\" in info) {\n            if (info.error === \"Not found\") { return undefined; }\n            throw new Error(`Error getting version at ${escapedPackageName}: ${info.error}`);\n        }\n        if (!info[\"dist-tags\"] && !info.versions) {\n            // Unpublished\n            return undefined;\n        }\n        return info;\n    }\n\n    // See https://github.com/npm/download-counts\n    async getDownloads(packageNames: ReadonlyArray<string>): Promise<ReadonlyArray<number>> {\n        // NPM uses a different API if there's only a single name, so ensure there's at least 2 for every batch of 128.\n        const names = (packageNames.length % 128) === 1 ? [...packageNames, \"dummy\"] : packageNames;\n        const nameGroups = Array.from(splitToFixedSizeGroups(names, 128)); // NPM has a limit of 128 packages at a time.\n\n        const out: number[] = [];\n        for (const nameGroup of nameGroups) {\n            const data = await this.fetcher.fetchJson({\n                hostname: npmApi,\n                path: `/downloads/point/last-month/${nameGroup.join(\",\")}`,\n                retries: true,\n            }) as { readonly error: string } | { readonly [key: string]: { readonly downloads: number } };\n            if (\"error\" in data) { throw new Error(data.error as string); }\n            for (const key of Object.keys(data)) {\n                assert(key === names[out.length], `at index ${out.length} of ${Object.keys(data).toString()} : ${key} !== ${names[out.length]}`);\n                out.push(data[key] ? data[key].downloads : 0);\n            }\n        }\n        return out;\n    }\n}\n\nfunction splitToFixedSizeGroups(names: ReadonlyArray<string>, chunkSize: number): ReadonlyArray<ReadonlyArray<string>> {\n    const out: string[][] = [];\n    for (let i = 0; i < names.length; i += chunkSize) {\n        out.push(names.slice(i, i + chunkSize));\n    }\n    return out;\n}\n\nexport class NpmPublishClient {\n    static async create(config?: RegClient.Config, registry: Registry = Registry.NPM): Promise<NpmPublishClient> {\n        switch (registry) {\n            case Registry.NPM:\n                return new NpmPublishClient(new RegClient(config), { token: await getSecret(Secret.NPM_TOKEN) }, npmRegistry);\n            case Registry.Github:\n                return new NpmPublishClient(new RegClient(config), { token: await getSecret(Secret.GITHUB_PUBLISH_ACCESS_TOKEN) }, githubRegistry);\n            default:\n                assertNever(registry);\n        }\n    }\n\n    private constructor(private readonly client: RegClient, private readonly auth: RegClient.Credentials, private readonly registry: string) {}\n\n    async publish(publishedDirectory: string, packageJson: {}, dry: boolean, log: Logger): Promise<void> {\n        const readme = await readFile(joinPaths(publishedDirectory, \"README.md\"));\n\n        return new Promise<void>((resolve, reject) => {\n            const body = createTgz(publishedDirectory, reject);\n            const metadata = { readme, ...packageJson };\n            if (dry) {\n                log(`(dry) Skip publish of ${publishedDirectory} to ${this.registry}`);\n            }\n            resolve(dry ? undefined : promisifyVoid(cb => {\n                this.client.publish(this.registry, { access: \"public\", auth: this.auth, metadata, body }, cb);\n            }));\n        });\n    }\n\n    tag(packageName: string, version: string, distTag: string, dry: boolean, log: Logger): Promise<void> {\n        if (dry) {\n            log(`(dry) Skip tag of ${packageName}@${distTag} as ${version}`);\n            return Promise.resolve();\n        }\n        return promisifyVoid(cb => {\n            this.client.distTags.add(this.registry, { package: packageName, version, distTag, auth: this.auth }, cb);\n        });\n    }\n\n    deprecate(packageName: string, version: string, message: string): Promise<void> {\n        const url = resolveUrl(npmRegistry, packageName.replace(\"/\", \"%2f\"));\n        const params = {\n            message,\n            version,\n            auth: this.auth,\n        };\n        return promisifyVoid(cb => { this.client.deprecate(url, params, cb); });\n    }\n}\n\nfunction npmInfoFromJson(n: NpmInfoRaw): NpmInfo {\n    return {\n        distTags: recordToMap(n[\"dist-tags\"], identity),\n        // Callback ensures we remove any other properties\n        versions: recordToMap(n.versions, ({ typesPublisherContentHash, deprecated }) => ({ typesPublisherContentHash, deprecated })),\n        time: recordToMap(n.time),\n    };\n}\n\nfunction jsonFromNpmInfo(n: NpmInfo): NpmInfoRaw {\n    return {\n        \"dist-tags\": mapToRecord(n.distTags),\n        versions: mapToRecord(n.versions),\n        time: mapToRecord(n.time),\n    };\n}\n\nfunction promisifyVoid(callsBack: (cb: (error: Error | undefined) => void) => void): Promise<void> {\n    return new Promise<void>((resolve, reject) => {\n        callsBack(error => {\n            if (error) {\n                reject(error);\n            } else {\n                resolve();\n            }\n        });\n    });\n}\n"
  },
  {
    "path": "src/lib/package-publisher.ts",
    "content": "import assert = require(\"assert\");\nimport { TypeScriptVersion } from \"definitelytyped-header-parser\";\n\nimport { Logger } from \"../util/logging\";\nimport { joinPaths } from \"../util/util\";\n\nimport { readFileAndWarn, Registry } from \"./common\";\nimport { NpmPublishClient } from \"./npm-client\";\nimport { AnyPackage, NotNeededPackage } from \"./packages\";\nimport { ChangedTyping } from \"./versions\";\n\nexport async function publishTypingsPackage(\n    client: NpmPublishClient,\n    changedTyping: ChangedTyping,\n    dry: boolean,\n    log: Logger,\n    registry: Registry,\n): Promise<void> {\n    const { pkg, version, latestVersion } = changedTyping;\n    await common(client, pkg, log, dry, registry);\n    if (pkg.isLatest) {\n        await updateTypeScriptVersionTags(pkg, version, client, log, dry);\n    }\n    assert((latestVersion === undefined) === pkg.isLatest);\n    if (latestVersion !== undefined) {\n        // If this is an older version of the package, we still update tags for the *latest*.\n        // NPM will update \"latest\" even if we are publishing an older version of a package (https://github.com/npm/npm/issues/6778),\n        // so we must undo that by re-tagging latest.\n        await updateLatestTag(pkg.fullNpmName, latestVersion, client, log, dry);\n    }\n}\n\nexport async function publishNotNeededPackage(\n    client: NpmPublishClient,\n    pkg: NotNeededPackage,\n    dry: boolean,\n    log: Logger,\n    registry: Registry,\n): Promise<void> {\n    log(`Deprecating ${pkg.name}`);\n    await common(client, pkg, log, dry, registry);\n    // Don't use a newline in the deprecation message because it will be displayed as \"\\n\" and not as a newline.\n    await deprecateNotNeededPackage(client, pkg, dry, log);\n}\n\nasync function common(client: NpmPublishClient, pkg: AnyPackage, log: Logger, dry: boolean, registry: Registry): Promise<void> {\n    const packageDir = pkg.outputDirectory;\n    const packageJson = await readFileAndWarn(\"generate\", joinPaths(packageDir + (registry === Registry.Github ? \"-github\" : \"\"), \"package.json\"));\n    await client.publish(packageDir, packageJson, dry, log);\n}\n\nexport async function deprecateNotNeededPackage(client: NpmPublishClient, pkg: NotNeededPackage, dry = false, log: Logger): Promise<void> {\n    const name = pkg.fullNpmName;\n    if (dry) {\n        log(\"(dry) Skip deprecate not needed package \" + name + \" at \" + pkg.version.versionString);\n    } else {\n        log(`Deprecating ${name} at ${pkg.version.versionString} with message: ${pkg.deprecatedMessage()}.`);\n        await client.deprecate(name, pkg.version.versionString, pkg.deprecatedMessage());\n    }\n}\n\nexport async function updateTypeScriptVersionTags(\n    pkg: AnyPackage, version: string, client: NpmPublishClient, log: Logger, dry: boolean,\n): Promise<void> {\n    const tags = TypeScriptVersion.tagsToUpdate(pkg.minTypeScriptVersion);\n    const name = pkg.fullNpmName;\n    log(`Tag ${name}@${version} as ${JSON.stringify(tags)}`);\n    if (dry) {\n        log(\"(dry) Skip tag\");\n    } else {\n        for (const tagName of tags) {\n            await client.tag(name, version, tagName, dry, log);\n        }\n    }\n}\n\nexport async function updateLatestTag(\n    fullName: string, version: string, client: NpmPublishClient, log: Logger, dry: boolean): Promise<void> {\n    log(`   but tag ${fullName}@${version} as \"latest\"`);\n    if (dry) {\n        log(\"   (dry) Skip move \\\"latest\\\" back to newest version\");\n    } else {\n        await client.tag(fullName, version, \"latest\", dry, log);\n    }\n}\n"
  },
  {
    "path": "src/lib/packages.test.ts",
    "content": "import { createMockDT } from \"../mocks\";\n\nimport { getTypingInfo } from \"./definition-parser\";\nimport { TypingsVersions } from \"./packages\";\n\ndescribe(TypingsVersions, () => {\n    let versions: TypingsVersions;\n\n    beforeAll(() => {\n        const dt = createMockDT();\n        dt.addOldVersionOfPackage(\"jquery\", \"1\");\n        dt.addOldVersionOfPackage(\"jquery\", \"2\");\n        dt.addOldVersionOfPackage(\"jquery\", \"2.5\");\n        versions = new TypingsVersions(getTypingInfo(\"jquery\", dt.pkgFS(\"jquery\")));\n    });\n\n    it(\"sorts the data from latest to oldest version\", () => {\n        expect(Array.from(versions.getAll()).map(v => v.major)).toEqual([3, 2, 2, 1]);\n    });\n\n    it(\"returns the latest version\", () => {\n        expect(versions.getLatest().major).toEqual(3);\n    });\n\n    it(\"finds the latest version when any version is wanted\", () => {\n        expect(versions.get(\"*\").major).toEqual(3);\n    });\n\n    it(\"finds the latest minor version for the given major version\", () => {\n        expect(versions.get({ major: 2 }).major).toEqual(2);\n        expect(versions.get({ major: 2 }).minor).toEqual(5);\n    });\n\n    it(\"finds a specific version\", () => {\n        expect(versions.get({ major: 2, minor: 0 }).major).toEqual(2);\n        expect(versions.get({ major: 2, minor: 0 }).minor).toEqual(0);\n    });\n\n    it(\"formats a version directory names\", () => {\n        expect(versions.get({ major: 2, minor: 0 }).versionDirectoryName).toEqual(\"v2\");\n        expect(versions.get({ major: 2, minor: 0 }).subDirectoryPath).toEqual(\"jquery/v2\");\n    });\n\n    it(\"formats missing version error nicely\", () => {\n        expect(() => versions.get({ major: 111, minor: 1001 })).toThrow(\"Could not find version 111.1001\");\n        expect(() => versions.get({ major: 111 })).toThrow(\"Could not find version 111.*\");\n    });\n});\n"
  },
  {
    "path": "src/lib/packages.ts",
    "content": "import assert = require(\"assert\");\nimport { AllTypeScriptVersion, Author, TypeScriptVersion } from \"definitelytyped-header-parser\";\n\nimport { FS } from \"../get-definitely-typed\";\nimport { assertSorted, joinPaths, mapValues, unmangleScopedPackage } from \"../util/util\";\n\nimport { readDataFile } from \"./common\";\nimport { outputDirPath, scopeName } from \"./settings\";\nimport { compare as compareSemver, Semver } from \"./versions\";\n\nexport class AllPackages {\n    static async read(dt: FS): Promise<AllPackages> {\n        return AllPackages.from(await readTypesDataFile(), readNotNeededPackages(dt));\n    }\n\n    static from(data: TypesDataFile, notNeeded: ReadonlyArray<NotNeededPackage>): AllPackages {\n        return new AllPackages(mapValues(new Map(Object.entries(data)), raw => new TypingsVersions(raw)), notNeeded);\n    }\n\n    static async readTypings(): Promise<ReadonlyArray<TypingsData>> {\n        return AllPackages.from(await readTypesDataFile(), []).allTypings();\n    }\n    static async readLatestTypings(): Promise<ReadonlyArray<TypingsData>> {\n        return AllPackages.from(await readTypesDataFile(), []).allLatestTypings();\n    }\n\n    /** Use for `--single` tasks only. Do *not* call this in a loop! */\n    static async readSingle(name: string): Promise<TypingsData> {\n        const data = await readTypesDataFile();\n        const raw = data[name];\n        if (!raw) {\n            throw new Error(`Can't find package ${name}`);\n        }\n        const versions = Object.keys(raw);\n        if (versions.length > 1) {\n            throw new Error(`Package ${name} has multiple versions.`);\n        }\n        return new TypingsData(raw[versions[0]], /*isLatest*/ true);\n    }\n\n    static readSingleNotNeeded(name: string, dt: FS): NotNeededPackage {\n        const notNeeded = readNotNeededPackages(dt);\n        const pkg = notNeeded.find(p => p.name === name);\n        if (pkg === undefined) {\n            throw new Error(`Cannot find not-needed package ${name}`);\n        }\n        return pkg;\n    }\n\n    private constructor(\n        private readonly data: ReadonlyMap<string, TypingsVersions>,\n        private readonly notNeeded: ReadonlyArray<NotNeededPackage>) {}\n\n    getNotNeededPackage(name: string): NotNeededPackage | undefined {\n        return this.notNeeded.find(p => p.name === name);\n    }\n\n    hasTypingFor(dep: PackageId): boolean {\n        return this.tryGetTypingsData(dep) !== undefined;\n    }\n\n    tryResolve(dep: PackageId): PackageId {\n        const versions = this.data.get(getMangledNameForScopedPackage(dep.name));\n        return versions ? versions.get(dep.version).id : dep;\n    }\n\n    /** Gets the latest version of a package. E.g. getLatest(node v6) was node v10 (before node v11 came out). */\n    getLatest(pkg: TypingsData): TypingsData {\n        return pkg.isLatest ? pkg : this.getLatestVersion(pkg.name);\n    }\n\n    private getLatestVersion(packageName: string): TypingsData {\n        const latest = this.tryGetLatestVersion(packageName);\n        if (!latest) {\n            throw new Error(`No such package ${packageName}.`);\n        }\n        return latest;\n    }\n\n    tryGetLatestVersion(packageName: string): TypingsData | undefined {\n        const versions = this.data.get(getMangledNameForScopedPackage(packageName));\n        return versions && versions.getLatest();\n    }\n\n    getTypingsData(id: PackageId): TypingsData {\n        const pkg = this.tryGetTypingsData(id);\n        if (!pkg) {\n            throw new Error(`No typings available for ${JSON.stringify(id)}`);\n        }\n        return pkg;\n    }\n\n    tryGetTypingsData({ name, version }: PackageId): TypingsData | undefined {\n        const versions = this.data.get(getMangledNameForScopedPackage(name));\n        return versions && versions.tryGet(version);\n    }\n\n    allPackages(): ReadonlyArray<AnyPackage> {\n        return [ ...this.allTypings(), ...this.allNotNeeded() ];\n    }\n\n    /** Note: this includes older version directories (`foo/v0`) */\n    allTypings(): ReadonlyArray<TypingsData> {\n        return assertSorted(Array.from(flattenData(this.data)), t => t.name);\n    }\n\n    allLatestTypings(): ReadonlyArray<TypingsData> {\n        return assertSorted(Array.from(this.data.values()).map(versions => versions.getLatest()), t => t.name);\n    }\n\n    allNotNeeded(): ReadonlyArray<NotNeededPackage> {\n        return this.notNeeded;\n    }\n\n    /** Returns all of the dependences *that have typings*, ignoring others, and including test dependencies. */\n    *allDependencyTypings(pkg: TypingsData): Iterable<TypingsData> {\n        for (const { name, version } of pkg.dependencies) {\n            const versions = this.data.get(getMangledNameForScopedPackage(name));\n            if (versions) {\n                yield versions.get(version);\n            }\n        }\n\n        for (const name of pkg.testDependencies) {\n            const versions = this.data.get(getMangledNameForScopedPackage(name));\n            if (versions) {\n                yield versions.getLatest();\n            }\n        }\n    }\n}\n\n// Same as the function in moduleNameResolver.ts in typescript\nexport function getMangledNameForScopedPackage(packageName: string): string {\n    if (packageName.startsWith(\"@\")) {\n        const replaceSlash = packageName.replace(\"/\", \"__\");\n        if (replaceSlash !== packageName) {\n            return replaceSlash.slice(1); // Take off the \"@\"\n        }\n    }\n    return packageName;\n}\n\nexport const typesDataFilename = \"definitions.json\";\n\nfunction* flattenData(data: ReadonlyMap<string, TypingsVersions>): Iterable<TypingsData> {\n    for (const versions of data.values()) {\n        yield* versions.getAll();\n    }\n}\n\nexport type AnyPackage = NotNeededPackage | TypingsData;\n\ninterface BaseRaw {\n    /**\n     * The name of the library.\n     *\n     * A human readable version, e.g. it might be \"Moment.js\" even though `packageName` is \"moment\".\n     */\n    readonly libraryName: string;\n\n    /**\n     * The NPM name to publish this under, e.g. \"jquery\".\n     *\n     * This does not include \"@types\".\n     */\n    readonly typingsPackageName: string;\n}\n\n/** Prefer to use `AnyPackage` instead of this. */\nexport abstract class PackageBase {\n    static compare(a: PackageBase, b: PackageBase): number { return a.name.localeCompare(b.name); }\n\n    /** Note: for \"foo__bar\" this is still \"foo__bar\", not \"@foo/bar\". */\n    readonly name: string;\n    readonly libraryName: string;\n\n    get unescapedName(): string {\n        return unmangleScopedPackage(this.name) || this.name;\n    }\n\n    /** Short description for debug output. */\n    get desc(): string {\n        return this.isLatest ? this.name : `${this.name} v${this.major}.${this.minor}`;\n    }\n\n    constructor(data: BaseRaw) {\n        this.name = data.typingsPackageName;\n        this.libraryName = data.libraryName;\n    }\n\n    isNotNeeded(): this is NotNeededPackage {\n        return this instanceof NotNeededPackage;\n    }\n\n    abstract readonly isLatest: boolean;\n    abstract readonly projectName: string;\n    abstract readonly declaredModules: ReadonlyArray<string>;\n    abstract readonly globals: ReadonlyArray<string>;\n    abstract readonly minTypeScriptVersion: TypeScriptVersion;\n\n    /** '@types/foo' for a package 'foo'. */\n    get fullNpmName(): string {\n        return getFullNpmName(this.name);\n    }\n\n    /** '@types%2ffoo' for a package 'foo'. */\n    get fullEscapedNpmName(): string {\n        return `@${scopeName}%2f${this.name}`;\n    }\n\n    abstract readonly major: number;\n    abstract readonly minor: number;\n\n    get id(): PackageId {\n        return { name: this.name, version: { major: this.major, minor: this.minor }};\n    }\n\n    get outputDirectory(): string {\n        return joinPaths(outputDirPath, this.desc);\n    }\n}\n\nexport function getFullNpmName(packageName: string): string {\n    return `@${scopeName}/${getMangledNameForScopedPackage(packageName)}`;\n}\n\ninterface NotNeededPackageRaw extends BaseRaw {\n    /**\n     * If this is available, @types typings are deprecated as of this version.\n     * This is useful for packages that previously had DefinitelyTyped definitions but which now provide their own.\n     */\n    // This must be \"major.minor.patch\"\n    readonly asOfVersion: string;\n    /** The package's own url, *not* DefinitelyTyped's. */\n    readonly sourceRepoURL: string;\n}\n\nexport class NotNeededPackage extends PackageBase {\n    readonly version: Semver;\n\n    get license(): License.MIT { return License.MIT; }\n\n    readonly sourceRepoURL: string;\n\n    constructor(raw: NotNeededPackageRaw) {\n        super(raw);\n        this.sourceRepoURL = raw.sourceRepoURL;\n\n        for (const key of Object.keys(raw)) {\n            if (![\"libraryName\", \"typingsPackageName\", \"sourceRepoURL\", \"asOfVersion\"].includes(key)) {\n                throw new Error(`Unexpected key in not-needed package: ${key}`);\n            }\n        }\n        assert(raw.libraryName && raw.typingsPackageName && raw.sourceRepoURL && raw.asOfVersion);\n\n        this.version = Semver.parse(raw.asOfVersion);\n    }\n\n    get major(): number { return this.version.major; }\n    get minor(): number { return this.version.minor; }\n\n    // A not-needed package has no other versions. (TODO: allow that?)\n    get isLatest(): boolean { return true; }\n    get projectName(): string { return this.sourceRepoURL; }\n    get declaredModules(): ReadonlyArray<string> { return []; }\n    get globals(): ReadonlyArray<string> { return this.globals; }\n    get minTypeScriptVersion(): TypeScriptVersion { return TypeScriptVersion.lowest; }\n\n    readme(): string {\n        return `This is a stub types definition for ${this.libraryName} (${this.sourceRepoURL}).\\n\n${this.libraryName} provides its own type definitions, so you don't need ${getFullNpmName(this.name)} installed!`;\n    }\n\n    deprecatedMessage(): string {\n        return `This is a stub types definition. ${this.name} provides its own type definitions, so you do not need this installed.`;\n    }\n}\n\nexport interface TypingsVersionsRaw {\n    [version: string]: TypingsDataRaw;\n}\n\nexport interface TypingVersion {\n    major: number;\n    minor?: number;\n}\n\nexport function formatTypingVersion(version: TypingVersion) {\n    return `${version.major}${version.minor === undefined ? \"\" : `.${version.minor}`}`;\n}\n\n/** If no version is specified, uses \"*\". */\nexport type DependencyVersion = TypingVersion | \"*\";\n\nexport function formatDependencyVersion(version: DependencyVersion) {\n    return version === \"*\" ? \"*\" : formatTypingVersion(version);\n}\n\nexport interface PackageJsonDependency {\n    readonly name: string;\n    readonly version: string;\n}\n\nexport interface TypingsDataRaw extends BaseRaw {\n    /**\n     * Other definitions, that exist in the same typings repo, that this package depends on.\n     *\n     * These will refer to *package names*, not *folder names*.\n     */\n    readonly dependencies: ReadonlyArray<PackageId>;\n\n    /**\n     * Other definitions, that exist in the same typings repo, that the tests, but not the types, of this package depend on.\n     *\n     * These are always the latest version and will not include anything already in `dependencies`.\n     */\n    readonly testDependencies: ReadonlyArray<string>;\n\n    /**\n     * External packages, from outside the typings repo, that provide definitions that this package depends on.\n     */\n    readonly packageJsonDependencies: ReadonlyArray<PackageJsonDependency>;\n\n    /**\n     * Represents that there was a path mapping to a package.\n     *\n     * Not all path mappings are direct dependencies, they may be necessary for transitive dependencies. However, where `dependencies` and\n     * `pathMappings` share a key, they *must* share the same value.\n     */\n    readonly pathMappings: ReadonlyArray<PathMapping>;\n\n    /**\n     * List of people that have contributed to the definitions in this package.\n     *\n     * These people will be requested for issue/PR review in the https://github.com/DefinitelyTyped/DefinitelyTyped repo.\n     */\n    readonly contributors: ReadonlyArray<Author>;\n\n    /**\n     * The [older] version of the library that this definition package refers to, as represented *on-disk*.\n     *\n     * @note The latest version always exists in the root of the package tree and thus does not have a value for this property.\n     */\n    readonly libraryVersionDirectoryName?: string;\n\n    /**\n     * Major version of the library.\n     *\n     * This data is parsed from a header comment in the entry point `.d.ts` and will be `0` if the file did not specify a version.\n     */\n    readonly libraryMajorVersion: number;\n\n    /**\n     * Minor version of the library.\n     *\n     * This data is parsed from a header comment in the entry point `.d.ts` and will be `0` if the file did not specify a version.\n     */\n    readonly libraryMinorVersion: number;\n\n    /**\n     * Minimum required TypeScript version to consume the definitions from this package.\n     */\n    readonly minTsVersion: AllTypeScriptVersion;\n\n    /**\n     * List of TS versions that have their own directories, and corresponding \"typesVersions\" in package.json.\n     * Usually empty.\n     */\n    readonly typesVersions: ReadonlyArray<TypeScriptVersion>;\n\n    /**\n     * Files that should be published with this definition, e.g. [\"jquery.d.ts\", \"jquery-extras.d.ts\"]\n     *\n     * Does *not* include a partial `package.json` because that will not be copied directly.\n     */\n    readonly files: ReadonlyArray<string>;\n\n    /**\n     * The license that this definition package is released under.\n     *\n     * Can be either MIT or Apache v2, defaults to MIT when not explicitly defined in this package’s \"package.json\".\n     */\n    readonly license: License;\n\n    /**\n     * A hash of the names and contents of the `files` list, used for versioning.\n     */\n    readonly contentHash: string;\n\n    /**\n     * Name or URL of the project, e.g. \"http://cordova.apache.org\".\n     */\n    readonly projectName: string;\n\n    /**\n     * A list of *values* declared in the global namespace.\n     *\n     * @note This does not include *types* declared in the global namespace.\n     */\n    readonly globals: ReadonlyArray<string>;\n\n    /**\n     * External modules declared by this package. Includes the containing folder name when applicable (e.g. proper module).\n     */\n    readonly declaredModules: ReadonlyArray<string>;\n}\n\n/**\n * @see {TypingsDataRaw.pathMappings}\n */\nexport interface PathMapping {\n    readonly packageName: string;\n    readonly version: TypingVersion;\n}\n\n// TODO: support BSD -- but must choose a *particular* BSD license from the list at https://spdx.org/licenses/\nexport const enum License { MIT = \"MIT\", Apache20 = \"Apache-2.0\" }\nconst allLicenses = [License.MIT, License.Apache20];\nexport function getLicenseFromPackageJson(packageJsonLicense: unknown): License {\n    if (packageJsonLicense === undefined) { // tslint:disable-line strict-type-predicates (false positive)\n        return License.MIT;\n    }\n    if (typeof packageJsonLicense === \"string\" && packageJsonLicense === \"MIT\") {\n        throw new Error(`Specifying '\"license\": \"MIT\"' is redundant, this is the default.`);\n    }\n    if (allLicenses.includes(packageJsonLicense as License)) {\n        return packageJsonLicense as License;\n    }\n    throw new Error(`'package.json' license is ${JSON.stringify(packageJsonLicense)}.\\nExpected one of: ${JSON.stringify(allLicenses)}}`);\n}\n\nexport class TypingsVersions {\n    private readonly map: ReadonlyMap<Semver, TypingsData>;\n\n    /**\n     * Sorted from latest to oldest.\n     */\n    private readonly versions: Semver[];\n\n    constructor(data: TypingsVersionsRaw) {\n        const versionMappings = new Map(Object.keys(data).map(key => {\n            const version = Semver.parse(key, true);\n            if (version) {\n                return [version, key];\n            }\n            throw new Error(`Unable to parse version ${key}`);\n        }));\n\n        /**\n         * Sorted from latest to oldest so that we publish the current version first.\n         * This is important because older versions repeatedly reset the \"latest\" tag to the current version.\n         */\n        this.versions = Array.from(versionMappings.keys()).sort(compareSemver).reverse();\n\n        this.map = new Map(this.versions.map(version => {\n            const dataKey = versionMappings.get(version)!;\n            return [version, new TypingsData(data[dataKey], version.equals(this.versions[0]))];\n        }));\n    }\n\n    getAll(): Iterable<TypingsData> {\n        return this.map.values();\n    }\n\n    get(version: DependencyVersion): TypingsData {\n        return version === \"*\" ? this.getLatest() : this.getLatestMatch(version);\n    }\n\n    tryGet(version: DependencyVersion): TypingsData | undefined {\n        return version === \"*\" ? this.getLatest() : this.tryGetLatestMatch(version);\n    }\n\n    getLatest(): TypingsData {\n        return this.map.get(this.versions[0])!;\n    }\n\n    private getLatestMatch(version: TypingVersion): TypingsData {\n        const data = this.tryGetLatestMatch(version);\n        if (!data) {\n            throw new Error(`Could not find version ${version.major}.${version.minor ?? \"*\"}`);\n        }\n        return data;\n    }\n\n    private tryGetLatestMatch(version: TypingVersion): TypingsData | undefined {\n        const found = this.versions.find(v => v.major === version.major && (version.minor === undefined || v.minor === version.minor));\n        return found && this.map.get(found);\n    }\n}\n\nexport class TypingsData extends PackageBase {\n    constructor(private readonly data: TypingsDataRaw, readonly isLatest: boolean) {\n        super(data);\n    }\n\n    get testDependencies(): ReadonlyArray<string> { return this.data.testDependencies; }\n    get contributors(): ReadonlyArray<Author> { return this.data.contributors; }\n    get major(): number { return this.data.libraryMajorVersion; }\n    get minor(): number { return this.data.libraryMinorVersion; }\n\n    get minTypeScriptVersion(): TypeScriptVersion {\n        return TypeScriptVersion.isSupported(this.data.minTsVersion) ? this.data.minTsVersion : TypeScriptVersion.lowest;\n    }\n    get typesVersions(): ReadonlyArray<TypeScriptVersion> { return this.data.typesVersions; }\n\n    get files(): ReadonlyArray<string> { return this.data.files; }\n    get license(): License { return this.data.license; }\n    get packageJsonDependencies(): ReadonlyArray<PackageJsonDependency> { return this.data.packageJsonDependencies; }\n    get contentHash(): string { return this.data.contentHash; }\n    get declaredModules(): ReadonlyArray<string> { return this.data.declaredModules; }\n    get projectName(): string { return this.data.projectName; }\n    get globals(): ReadonlyArray<string> { return this.data.globals; }\n    get pathMappings(): ReadonlyArray<PathMapping> { return this.data.pathMappings; }\n\n    get dependencies(): ReadonlyArray<PackageId> {\n        return this.data.dependencies;\n    }\n\n    get versionDirectoryName() {\n        return this.data.libraryVersionDirectoryName && `v${this.data.libraryVersionDirectoryName}`;\n    }\n\n    /** Path to this package, *relative* to the DefinitelyTyped directory. */\n    get subDirectoryPath(): string {\n        return this.isLatest ? this.name : `${this.name}/${this.versionDirectoryName}`;\n    }\n}\n\n/** Uniquely identifies a package. */\nexport interface PackageId {\n    readonly name: string;\n    readonly version: DependencyVersion;\n}\n\nexport interface TypesDataFile {\n    readonly [packageName: string]: TypingsVersionsRaw;\n}\nfunction readTypesDataFile(): Promise<TypesDataFile> {\n    return readDataFile(\"parse-definitions\", typesDataFilename) as Promise<TypesDataFile>;\n}\n\nexport function readNotNeededPackages(dt: FS): ReadonlyArray<NotNeededPackage> {\n    const rawJson = dt.readJson(\"notNeededPackages.json\"); // tslint:disable-line await-promise (tslint bug)\n    return (rawJson as { readonly packages: ReadonlyArray<NotNeededPackageRaw> }).packages.map(raw => new NotNeededPackage(raw));\n}\n"
  },
  {
    "path": "src/lib/rolling-logs.ts",
    "content": "import BlobWriter, { readBlob } from \"./azure-container\";\n\nexport default class RollingLogs {\n    static async create(name: string, maxLines: number): Promise<RollingLogs> {\n        return new RollingLogs(name, maxLines, await BlobWriter.create());\n    }\n\n    private allLogs: string[] | undefined;\n\n    constructor(readonly name: string, readonly maxLines: number, private readonly container: BlobWriter) {}\n\n    async write(lines: string[]): Promise<void> {\n        const logs = this.allLogs || (this.allLogs = await this.readAllLogs());\n        const totalLines = logs.length + lines.length;\n        logs.splice(0, totalLines - this.maxLines);\n        logs.push(...lines);\n        await this.container.createBlobFromText(this.name, logs.join(\"\\n\"));\n    }\n\n    private async readAllLogs(): Promise<string[]> {\n        try {\n            return (await readBlob(this.name)).split(\"\\n\");\n        } catch (err) {\n            // 404\n            return [];\n        }\n    }\n}\n"
  },
  {
    "path": "src/lib/secrets.ts",
    "content": "import { AuthenticationContext } from \"adal-node\";\nimport { KeyVaultClient, KeyVaultCredentials } from \"azure-keyvault\";\n\nimport { mapDefined } from \"../util/util\";\n\nimport { azureKeyvault } from \"./settings\";\n\nexport enum Secret {\n    /**\n     * Used to upload blobs.\n     * To find (or refresh) this value, go to https://ms.portal.azure.com -> All resources -> typespublisher -> General -> Access keys\n     */\n    AZURE_STORAGE_ACCESS_KEY,\n    /**\n     * Lets the server update an issue (https://github.com/Microsoft/types-publisher/issues/40) on GitHub in case of an error.\n     * Create a token at: https://github.com/settings/tokens\n     */\n    GITHUB_ACCESS_TOKEN,\n    /**\n     * This is used to ensure that only GitHub can send messages to our server.\n     * This should match the secret value set on GitHub: https://github.com/DefinitelyTyped/DefinitelyTyped/settings/hooks\n     * The Payload URL should be the URL of the Azure service.\n     * The webhook ignores the `sourceRepository` setting and can be triggered by *anything* with the secret,\n     * so make sure only DefinitelyTyped has the secret.\n     */\n    GITHUB_SECRET,\n    /**\n     * Token used to perform request to NPM's API.\n     * This was generated by doing:\n     * - `npm login`\n     * - copy the token value (comes after `authToken=`) in `~/.npmrc`\n     * - `rm ~/.npmrc` (don't want to accidentally log out this token.)\n     *\n     * We only need one token in existence, so delete old tokens at: https://www.npmjs.com/settings/tokens\n     */\n    NPM_TOKEN,\n    /**\n     * Token used to publish @definitelytyped/types-registry and @types/* to Github.\n     * This *could* be the same as GITHUB_ACCESS_TOKEN, but I think it's better if they remain separate.\n     */\n    GITHUB_PUBLISH_ACCESS_TOKEN,\n}\n\nexport const allSecrets: Secret[] = mapDefined(Object.keys(Secret), key => {\n    const value = (Secret as unknown as { [key: string]: unknown })[key];\n    return typeof value === \"number\" ? value : undefined; // tslint:disable-line strict-type-predicates (tslint bug)\n});\n\nexport async function getSecret(secret: Secret): Promise<string> {\n    const clientId = process.env.TYPES_PUBLISHER_CLIENT_ID;\n    const clientSecret = process.env.TYPES_PUBLISHER_CLIENT_SECRET;\n    if (!(clientId && clientSecret)) {\n        throw new Error(\"Must set the TYPES_PUBLISHER_CLIENT_ID and TYPES_PUBLISHER_CLIENT_SECRET environment variables.\");\n    }\n\n    // Copied from example usage at https://www.npmjs.com/package/azure-keyvault\n    const credentials = new KeyVaultCredentials((challenge, callback) => {\n        const context = new AuthenticationContext(challenge.authorization);\n        context.acquireTokenWithClientCredentials(challenge.resource, clientId, clientSecret, (error, tokenResponse) => {\n            if (error) {\n                throw error;\n            }\n            callback(undefined, `${tokenResponse!.tokenType} ${tokenResponse!.accessToken}`);\n        });\n    });\n\n    const client = new KeyVaultClient(credentials);\n\n    // Convert `AZURE_STORAGE_ACCESS_KEY` to `azure-storage-access-key` -- for some reason, Azure wouldn't allow secret names with underscores.\n    const azureSecretName = Secret[secret].toLowerCase().replace(/_/g, \"-\");\n    // console.log(\"Getting secret versions for: \" + azureSecretName);\n    const versions = await client.getSecretVersions(azureKeyvault, azureSecretName);\n    versions.sort((a, b) => a.attributes.created.getTime() < b.attributes.created.getTime() ? 1 : -1);\n    // console.log(versions);\n    const urlParts = versions[0].id.split(\"/\");\n    const latest = urlParts[urlParts.length - 1];\n    return (await client.getSecret(azureKeyvault, azureSecretName, latest)).value;\n}\n"
  },
  {
    "path": "src/lib/settings.ts",
    "content": "import { readFileSync } from \"fs\";\nimport { join as joinPaths } from \"path\";\n\n/** URL of the NPM registry to upload to. */\nexport const npmRegistryHostName = \"registry.npmjs.org\";\nexport const githubRegistryHostName = \"npm.pkg.github.com\";\nexport const npmRegistry = `https://${npmRegistryHostName}/`;\nexport const githubRegistry = `https://${githubRegistryHostName}/`;\nexport const npmApi = \"api.npmjs.org\";\n/** Note: this is 'types' and not '@types' */\nexport const scopeName = \"types\";\nconst root = joinPaths(__dirname, \"..\", \"..\");\nexport const dataDirPath = joinPaths(root, \"data\");\nexport const outputDirPath = joinPaths(root, \"output\");\nexport const validateOutputPath = joinPaths(root, \"validateOutput\");\nexport const logDir = joinPaths(root, \"logs\");\n\n/** URL to download the repository from. */\nexport const definitelyTypedZipUrl = \"https://codeload.github.com/DefinitelyTyped/DefinitelyTyped/tar.gz/master\";\n/** The branch that DefinitelyTyped is sourced from. */\nexport const sourceBranch = \"master\";\n\n/** Name of the azure storage account. Used for uploading data and logs. */\nexport const azureStorageAccount = \"typespublisher\";\n/** Name of the azure container. */\nexport const azureContainer = \"typespublisher\";\n/** URL of azure keyvault. */\nexport const azureKeyvault = \"https://types-publisher-keys.vault.azure.net\";\n/** Issue in types-publisher that we will use to report webhook errors. */\nexport const errorsIssue = \"Microsoft/types-publisher/issues/40\";\n\nexport const typesDirectoryName = \"types\";\n\nexport const dependenciesWhitelist: ReadonlySet<string> =\n    new Set(readFileSync(joinPaths(root, \"dependenciesWhitelist.txt\"), \"utf-8\").split(/\\r?\\n/));\n"
  },
  {
    "path": "src/lib/versions.test.ts",
    "content": "import { Semver } from \"./versions\";\n\ndescribe(Semver, () => {\n    it(\"returns a formatted description\", () => {\n        expect(new Semver(1, 2, 3).versionString).toEqual(\"1.2.3\");\n    });\n\n    it(\"parses semver versions\", () => {\n        expect(Semver.parse(\"0.42.1\").versionString).toEqual(\"0.42.1\");\n    });\n\n    it(\"parses versions that do not strictly adhere to semver\", () => {\n        expect(Semver.parse(\"1\", true).versionString).toEqual(\"1.0.0\");\n        expect(Semver.parse(\"0.42\", true).versionString).toEqual(\"0.42.0\");\n    });\n\n    it(\"throws when a version cannot be parsed\", () => {\n        expect(() => Semver.parse(\"1\")).toThrow();\n        expect(() => Semver.parse(\"1\", false)).toThrow();\n    });\n\n    it(\"returns whether or not it's equal to another Semver\", () => {\n        expect(Semver.parse(\"1.2.3\").equals(new Semver(1, 2, 3))).toBe(true);\n        expect(Semver.parse(\"1.2.3\").equals(new Semver(3, 2, 1))).toBe(false);\n    });\n\n    it(\"returns whether or not it's greater than another Semver\", () => {\n        expect(Semver.parse(\"1.2.3\").greaterThan(new Semver(1, 2, 2))).toBe(true);\n        expect(Semver.parse(\"1.2.3\").equals(new Semver(1, 2, 4))).toBe(false);\n    });\n});\n"
  },
  {
    "path": "src/lib/versions.ts",
    "content": "import { Logger } from \"../util/logging\";\nimport { assertDefined, best, intOfString } from \"../util/util\";\n\nimport { readDataFile } from \"./common\";\nimport { CachedNpmInfoClient } from \"./npm-client\";\nimport { AllPackages, NotNeededPackage, PackageId, TypingsData } from \"./packages\";\n\nexport const versionsFilename = \"versions.json\";\n\nexport interface ChangedTyping {\n    readonly pkg: TypingsData;\n    /** This is the version to be published, meaning it's the version that doesn't exist yet. */\n    readonly version: string;\n    /** For a non-latest version, this is the latest version; publishing an old version updates the 'latest' tag and we want to change it back. */\n    readonly latestVersion: string | undefined;\n}\n\nexport interface ChangedPackagesJson {\n    readonly changedTypings: ReadonlyArray<ChangedTypingJson>;\n    readonly changedNotNeededPackages: ReadonlyArray<string>;\n}\n\nexport interface ChangedTypingJson {\n    readonly id: PackageId;\n    readonly version: string;\n    readonly latestVersion?: string;\n}\n\nexport interface ChangedPackages {\n    readonly changedTypings: ReadonlyArray<ChangedTyping>;\n    readonly changedNotNeededPackages: ReadonlyArray<NotNeededPackage>;\n}\n\nexport async function readChangedPackages(allPackages: AllPackages): Promise<ChangedPackages> {\n    const json = await readDataFile(\"calculate-versions\", versionsFilename) as ChangedPackagesJson;\n    return {\n        changedTypings: json.changedTypings.map(({ id, version, latestVersion }): ChangedTyping =>\n            ({ pkg: allPackages.getTypingsData(id), version, latestVersion })),\n        changedNotNeededPackages: json.changedNotNeededPackages.map(id => assertDefined(allPackages.getNotNeededPackage(id))),\n    };\n}\n\n/**\n * When we fail to publish a deprecated package, it leaves behind an entry in the time property.\n * So the keys of 'time' give the actual 'latest'.\n * If that's not equal to the expected latest, try again by bumping the patch version of the last attempt by 1.\n */\nexport function skipBadPublishes(pkg: NotNeededPackage, client: CachedNpmInfoClient, log: Logger) {\n    // because this is called right after isAlreadyDeprecated, we can rely on the cache being up-to-date\n    const info = assertDefined(client.getNpmInfoFromCache(pkg.fullEscapedNpmName));\n    const notNeeded = pkg.version;\n    const latest = Semver.parse(findActualLatest(info.time));\n    if (latest.equals(notNeeded) || latest.greaterThan(notNeeded) ||\n        info.versions.has(notNeeded.versionString) && !assertDefined(info.versions.get(notNeeded.versionString)).deprecated) {\n        const plusOne = new Semver(latest.major, latest.minor, latest.patch + 1);\n        log(`Deprecation of ${notNeeded.versionString} failed, instead using ${plusOne.versionString}.`);\n        return new NotNeededPackage({\n            asOfVersion: plusOne.versionString,\n            libraryName: pkg.libraryName,\n            sourceRepoURL: pkg.sourceRepoURL,\n            typingsPackageName: pkg.name,\n        });\n    }\n    return pkg;\n}\n\nfunction findActualLatest(times: Map<string, string>) {\n    const actual = best(\n        times, ([k, v], [bestK, bestV]) =>\n            (bestK === \"modified\" || bestK === \"created\") ? true :\n            (k === \"modified\" || k === \"created\") ? false :\n            new Date(v).getTime() > new Date(bestV).getTime());\n    if (!actual) {\n        throw new Error(\"failed to find actual latest\");\n    }\n    return actual[0];\n}\n\n/** Version of a package published to NPM. */\nexport class Semver {\n    static parse(semver: string, coerce?: boolean): Semver {\n        const result = Semver.tryParse(semver, coerce);\n        if (!result) {\n            throw new Error(`Unexpected semver: ${semver}`);\n        }\n        return result;\n    }\n\n    static fromRaw({ major, minor, patch }: Semver): Semver {\n        return new Semver(major, minor, patch);\n    }\n\n    /**\n     * Per the semver spec <http://semver.org/#spec-item-2>:\n     *\n     *   A normal version number MUST take the form X.Y.Z where X, Y, and Z are non-negative integers, and MUST NOT contain leading zeroes.\n     *\n     * @note This must parse the output of `versionString`.\n     *\n     * @param semver The version string.\n     * @param coerce Without this optional parameter the version MUST follow the above semver spec. However, when set to `true` components after the\n     *               major version may be omitted. I.e. `1` equals `1.0` and `1.0.0`.\n     */\n    static tryParse(semver: string, coerce?: boolean): Semver | undefined {\n        const rgx = /^(\\d+)(\\.(\\d+))?(\\.(\\d+))?$/;\n        const match = rgx.exec(semver);\n        if (match) {\n            const { 1: major, 3: minor, 5: patch } = match;\n            if ((minor !== undefined && patch !== undefined) || coerce) { // tslint:disable-line:strict-type-predicates\n                return new Semver(intOfString(major), intOfString(minor || \"0\"), intOfString(patch || \"0\"));\n            }\n        }\n        return undefined;\n    }\n\n    constructor(readonly major: number, readonly minor: number, readonly patch: number) {}\n\n    get versionString(): string {\n        const { major, minor, patch } = this;\n        return `${major}.${minor}.${patch}`;\n    }\n\n    equals(other: Semver): boolean {\n        return compare(this, other) === 0;\n    }\n\n    greaterThan(other: Semver): boolean {\n        return compare(this, other) === 1;\n    }\n}\n\n/**\n * Returns 0 if equal, 1 if x > y, -1 if x < y\n */\nexport function compare(x: Semver, y: Semver) {\n    const versions: Array<[number, number]> = [[x.major, y.major], [x.minor, y.minor], [x.patch, y.patch]];\n    for (const [componentX, componentY] of versions) {\n        if (componentX > componentY) {\n            return 1;\n        }\n        if (componentX < componentY) {\n            return -1;\n        }\n    }\n    return 0;\n}\n"
  },
  {
    "path": "src/lib/webhook-server.ts",
    "content": "import { createHmac, timingSafeEqual } from \"crypto\";\nimport { createServer, IncomingMessage, Server, ServerResponse } from \"http\";\nimport { setInterval } from \"timers\";\n\nimport full from \"../full\";\nimport { Fetcher, stringOfStream } from \"../util/io\";\nimport { joinLogWithErrors, LoggerWithErrors, loggerWithErrors, LogWithErrors } from \"../util/logging\";\nimport { currentTimeStamp, parseJson } from \"../util/util\";\n\nimport { Options } from \"./common\";\nimport RollingLogs from \"./rolling-logs\";\nimport { sourceBranch } from \"./settings\";\n\nexport default async function webhookServer(\n    key: string,\n    githubAccessToken: string,\n    dry: boolean,\n    fetcher: Fetcher,\n    options: Options,\n): Promise<Server> {\n    const fullOne = updateOneAtATime(async (log, timestamp) => {\n        log.info(\"\"); log.info(\"\");\n        log.info(`# ${timestamp}`);\n        log.info(\"\");\n        log.info(\"Starting full...\");\n        await full(dry, timestamp, githubAccessToken, fetcher, options, log);\n    });\n    setInterval((log, timestamp) => {\n        const result = fullOne(log, timestamp);\n        if (!result) { return; } // already working, so do nothing.\n        result.catch(e => { log.info(e.toString()); console.error(e); });\n    },          2_000_000, loggerWithErrors()[0], currentTimeStamp());\n    return listenToGithub(key, fullOne);\n}\n\nfunction writeLog(rollingLogs: RollingLogs, logs: LogWithErrors): Promise<void> {\n    return rollingLogs.write(joinLogWithErrors(logs));\n}\n\n/** @param onUpdate: returns a promise in case it may error. Server will shut down on errors. */\nfunction listenToGithub(\n    key: string,\n    onUpdate: (log: LoggerWithErrors, timeStamp: string) => Promise<void> | undefined,\n): Server {\n\n    console.log(\"Before starting server\");\n    const rollingLogs = RollingLogs.create(\"webhook-logs.md\", 1000);\n    const server = createServer((req, resp) => {\n        switch (req.method) {\n            case \"POST\":\n                receiveUpdate(req, resp);\n                break;\n            default:\n                // Don't respond\n        }\n    });\n    return server;\n\n    function receiveUpdate(req: IncomingMessage, resp: ServerResponse): void {\n        const [log, logResult] = loggerWithErrors();\n        const timeStamp = currentTimeStamp();\n        try {\n            log.info(\"Before starting work\");\n            work().then(() => rollingLogs.then(logs => writeLog(logs, logResult()))).catch(onError);\n        } catch (error) {\n            rollingLogs\n                .then(logs => writeLog(logs, logResult()))\n                .then(onError)\n                .catch(onError);\n        }\n\n        function onError(): void {\n            server.close();\n        }\n\n        async function work(): Promise<void> {\n            const data = await stringOfStream(req, \"Request to webhook\");\n            if (!checkSignature(key, data, req.headers, log)) {\n                return;\n            }\n\n            log.info(`Message from github: ${data.slice(0, 200)}...`);\n            const expectedRef = `refs/heads/${sourceBranch}`;\n\n            const actualRef = (parseJson(data) as { readonly ref: string }).ref;\n            if (actualRef === expectedRef) {\n                respond(\"Thanks for the update! Running full.\");\n                await onUpdate(log, timeStamp);\n            } else {\n                const text = `Ignoring push to ${actualRef}, expected ${expectedRef}.`;\n                respond(text);\n                log.info(text);\n            }\n        }\n\n        // This is for the benefit of `npm run make-[production-]server-run`. GitHub ignores this.\n        function respond(text: string): void {\n            resp.write(text);\n            resp.end();\n        }\n    }\n}\n\n// Even if there are many changes to DefinitelyTyped in a row, we only perform one update at a time.\nfunction updateOneAtATime(\n    doOnce: (log: LoggerWithErrors, timeStamp: string) => Promise<void>,\n): (log: LoggerWithErrors, timeStamp: string) => Promise<void> | undefined {\n    let working = false;\n\n    return (log, timeStamp) => {\n        if (working) {\n            log.info(\"Not starting update, because already performing one.\");\n            return undefined;\n        }\n        return work();\n\n        async function work(): Promise<void> {\n            log.info(\"Starting update\");\n            working = true;\n            try {\n                await doOnce(log, timeStamp);\n            } catch (e) {\n                log.info(e.toString());\n            } finally {\n                working = false;\n            }\n        }\n    };\n}\n\nfunction checkSignature(key: string, data: string, headers: { readonly [key: string]: unknown }, log: LoggerWithErrors): boolean {\n    const signature = headers[\"x-hub-signature\"];\n    const expected = expectedSignature(key, data);\n    // tslint:disable-next-line strict-type-predicates (TODO: tslint bug)\n    if (typeof signature === \"string\" && stringEqualsConstantTime(signature, expected)) {\n        return true;\n    }\n    // tslint:disable-next-line strict-string-expressions\n    log.error(`Invalid request: expected ${expected}, got ${signature}`);\n    log.error(`Headers are: ${JSON.stringify(headers, undefined, 4)}`);\n    log.error(`Data is: ${data}`);\n    log.error(\"\");\n    return false;\n}\n\n// Use a constant-time compare to prevent timing attacks\nfunction stringEqualsConstantTime(actual: string, expected: string): boolean {\n    // `timingSafeEqual` throws if they don't have the same length.\n    const actualBuffer = Buffer.alloc(expected.length);\n    actualBuffer.write(actual);\n    return timingSafeEqual(actualBuffer, Buffer.from(expected));\n}\n\nexport function expectedSignature(key: string, data: string): string {\n    const hmac = createHmac(\"sha1\", key);\n    hmac.write(data);\n    const digest = hmac.digest(\"hex\");\n    return `sha1=${digest}`;\n}\n"
  },
  {
    "path": "src/make-server-run.ts",
    "content": "import * as yargs from \"yargs\";\n\nimport { getSecret, Secret } from \"./lib/secrets\";\nimport { sourceBranch } from \"./lib/settings\";\nimport { expectedSignature } from \"./lib/webhook-server\";\nimport { makeHttpRequest } from \"./util/io\";\nimport { logUncaughtErrors } from \"./util/util\";\n\nif (!module.parent) {\n    const remote = yargs.argv.remote;\n    logUncaughtErrors(main(remote ? { hostname: \"typespublisher.azurewebsites.net\" }  : { hostname: \"localhost\", port: getPort() }));\n}\n\nfunction getPort(): number {\n    const port = parseInt(process.env.PORT!, 10);\n    if (!port) {\n        throw new Error(\"Must provide PORT\");\n    }\n    return port;\n}\n\nasync function main(options: { hostname: string, port?: number }): Promise<void> {\n    const key = await getSecret(Secret.GITHUB_SECRET);\n    const body = JSON.stringify({ ref: `refs/heads/${sourceBranch}` });\n    console.log(await makeHttpRequest({\n        hostname: options.hostname,\n        port: options.port,\n        path: \"\",\n        method: \"POST\",\n        body,\n        headers: { \"x-hub-signature\": expectedSignature(key, body) },\n    }));\n}\n"
  },
  {
    "path": "src/mocks.ts",
    "content": "import { parseHeaderOrFail } from \"definitelytyped-header-parser\";\n\nimport { Dir, FS, InMemoryDT } from \"./get-definitely-typed\";\nimport { Semver } from \"./lib/versions\";\n\nclass DTMock {\n    public readonly fs: FS;\n    private readonly root: Dir;\n\n    constructor() {\n        this.root = new Dir(undefined);\n        this.root.set(\"notNeededPackages.json\", `{\n            \"packages\": [{\n            \"libraryName\": \"Angular 2\",\n            \"typingsPackageName\": \"angular\",\n            \"asOfVersion\": \"1.2.3\",\n            \"sourceRepoURL\": \"https://github.com/angular/angular2\"\n          }]\n        }`);\n        this.fs = new InMemoryDT(this.root, \"DefinitelyTyped\");\n    }\n\n    public pkgDir(packageName: string): Dir {\n        return this.root.subdir(\"types\").subdir(packageName);\n    }\n\n    public pkgFS(packageName: string): FS {\n        return this.fs.subDir(\"types\").subDir(packageName);\n    }\n\n    /**\n     * Creates a shallow copy of a package, meaning all entries in the old version directory that will be created refer to the copied entry from the\n     * latest version. The only exceptions are the `index.d.ts` and `tsconfig.json` files.\n     *\n     * The directory name will exactly follow the given `olderVersion`. I.e. `2` will become `v2`, whereas `2.2` will become `v2.2`.\n     *\n     * @param packageName The package of which an old version is to be added.\n     * @param olderVersion The older version that's to be added.\n     */\n    public addOldVersionOfPackage(packageName: string, olderVersion: string) {\n        const latestDir = this.pkgDir(packageName);\n        const index = latestDir.get(\"index.d.ts\") as string;\n        const latestHeader = parseHeaderOrFail(index);\n        const latestVersion = `${latestHeader.libraryMajorVersion}.${latestHeader.libraryMinorVersion}`;\n        const olderVersionParsed = Semver.parse(olderVersion, true)!;\n\n        const oldDir = latestDir.subdir(`v${olderVersion}`);\n        const tsconfig = JSON.parse(latestDir.get(\"tsconfig.json\") as string);\n\n        oldDir.set(\"index.d.ts\", index.replace(latestVersion, `${olderVersionParsed.major}.${olderVersionParsed.minor}`));\n        oldDir.set(\"tsconfig.json\", JSON.stringify({\n            ...tsconfig,\n            compilerOptions: {\n                ...tsconfig.compilerOptions,\n                paths: {\n                    [packageName]: [`${packageName}/v${olderVersion}`],\n                },\n            },\n        }));\n\n        latestDir.forEach((content, entry) => {\n            if (\n                content !== oldDir\n                && entry !== \"index.d.ts\"\n                && entry !== \"tsconfig.json\"\n                && !(content instanceof Dir && /^v\\d+(\\.\\d+)?$/.test(entry))\n            ) {\n                oldDir.set(entry, content);\n            }\n        });\n\n        return oldDir;\n    }\n}\n\nexport function createMockDT() {\n    const dt = new DTMock();\n\n    const boring = dt.pkgDir(\"boring\");\n    boring.set(\"index.d.ts\", `// Type definitions for boring 1.0\n// Project: https://boring.com\n// Definitions by: Some Guy From Space <https://github.com/goodspaceguy420>\n// Definitions: https://github.com/DefinitelyTyped/DefinitelyTyped\n\nimport * as React from 'react';\nexport const drills: number;\n`);\n    boring.set(\"secondary.d.ts\", `\nimport deffo from 'react-default';\nimport { mammoths } from 'boring/quaternary';\nexport const hovercars: unknown;\ndeclare module \"boring/fake\" {\n    import { stock } from 'boring/tertiary';\n}\ndeclare module \"other\" {\n    export const augmented: true;\n}\n`);\n    boring.set(\"tertiary.d.ts\", `\nimport { stuff } from 'things';\nexport var stock: number;\n`);\n    boring.set(\"quaternary.d.ts\", `\nexport const mammoths: object;\n`);\n    boring.set(\"commonjs.d.ts\", `\nimport vortex = require('vorticon');\ndeclare const australia: {};\nexport = australia;\n`);\n    boring.set(\"v1.d.ts\", `\nexport const inane: true | false;\n`);\n    boring.set(\"untested.d.ts\", `\nimport { help } from 'manual';\nexport const fungible: false;\n`);\n    boring.set(\"boring-tests.ts\", `\nimport { superstor } from \"super-big-fun-hus\";\nimport { drills } from \"boring\";\nimport { hovercars } from \"boring/secondary\";\nimport australia = require('boring/commonjs');\nimport { inane } from \"boring/v1\";\n`);\n    boring.set(\"OTHER_FILES.txt\", `\nuntested.d.ts\n`);\n    boring.set(\"tsconfig.json\", `{\n    \"compilerOptions\": {\n        \"module\": \"commonjs\",\n        \"lib\": [\n            \"es6\",\n            \"dom\"\n        ],\n        \"target\": \"es6\",\n        \"noImplicitAny\": true,\n        \"noImplicitThis\": true,\n        \"strictNullChecks\": true,\n        \"strictFunctionTypes\": true,\n        \"baseUrl\": \"../\",\n        \"typeRoots\": [\n            \"../\"\n        ],\n        \"types\": [],\n        \"noEmit\": true,\n        \"forceConsistentCasingInFileNames\": true\n    },\n    \"files\": [\n        \"index.d.ts\",\n        \"boring-tests.ts\"\n    ]\n}`);\n\n    const globby = dt.pkgDir(\"globby\");\n    globby.set(\"index.d.ts\", `// Type definitions for globby 0.2\n// Project: https://globby-gloopy.com\n// Definitions by: The Dragon Quest Slime <https://github.com/gloopyslime>\n// Definitions: https://github.com/DefinitelyTyped/DefinitelyTyped\n\n/// <reference path=\"./sneaky.d.ts\" />\n/// <reference types=\"andere/snee\" />\ndeclare var x: number\n`);\n    globby.set(\"merges.d.ts\", `\ndeclare var y: number\n`);\n    globby.set(\"sneaky.d.ts\", `\ndeclare var ka: number\n`);\n    globby.set(\"globby-tests.ts\", `\nvar z = x;\n`);\n    const tests = globby.subdir(\"test\");\n    tests.set(\"other-tests.ts\", `\n/// <reference types=\"globby/merges\" />\nvar z = y;\n`);\n    globby.set(\"tsconfig.json\", `{\n    \"compilerOptions\": {\n        \"module\": \"commonjs\",\n        \"lib\": [\n            \"es6\",\n            \"dom\"\n        ],\n        \"target\": \"es6\",\n        \"noImplicitAny\": true,\n        \"noImplicitThis\": true,\n        \"strictNullChecks\": true,\n        \"strictFunctionTypes\": true,\n        \"baseUrl\": \"../\",\n        \"typeRoots\": [\n            \"../\"\n        ],\n        \"types\": [],\n        \"noEmit\": true,\n        \"forceConsistentCasingInFileNames\": true\n    },\n    \"files\": [\n        \"index.d.ts\",\n        \"globby-tests.ts\",\n        \"test/other-tests.ts\"\n    ]\n}`);\n    const jquery = dt.pkgDir(\"jquery\");\n    jquery.set(\"JQuery.d.ts\", `\ndeclare var jQuery: 1;\n`);\n    jquery.set(\"index.d.ts\", `// Type definitions for jquery 3.3\n// Project: https://jquery.com\n// Definitions by: Leonard Thieu <https://github.com/leonard-thieu>\n// Definitions: https://github.com/DefinitelyTyped/DefinitelyTyped\n// TypeScript Version: 2.3\n\n/// <reference path=\"JQuery.d.ts\" />\n\nexport = jQuery;\n`);\n    jquery.set(\"jquery-tests.ts\", `\nconsole.log(jQuery);\n`);\n    jquery.set(\"tsconfig.json\", `{\n    \"compilerOptions\": {\n        \"module\": \"commonjs\",\n        \"lib\": [\n            \"es6\",\n            \"dom\"\n        ],\n        \"target\": \"es6\",\n        \"noImplicitAny\": true,\n        \"noImplicitThis\": true,\n        \"strictNullChecks\": true,\n        \"strictFunctionTypes\": true,\n        \"baseUrl\": \"../\",\n        \"typeRoots\": [\n            \"../\"\n        ],\n        \"types\": [],\n        \"noEmit\": true,\n        \"forceConsistentCasingInFileNames\": true\n    },\n    \"files\": [\n        \"index.d.ts\",\n        \"jquery-tests.ts\"\n    ]\n}\n\n`);\n\n    return dt;\n}\n"
  },
  {
    "path": "src/parse-definitions.test.ts",
    "content": "import { createMockDT } from \"./mocks\";\nimport parseDefinitions from \"./parse-definitions\";\nimport { loggerWithErrors } from \"./util/logging\";\nimport { testo } from \"./util/test\";\n\ntesto({\n    // async parseDefinitions() {\n    //     const log = loggerWithErrors()[0]\n    //     const dt = await getDefinitelyTyped(Options.defaults, log);\n    //     const defs = await parseDefinitions(dt, undefined, log)\n    //     expect(defs.allNotNeeded().length).toBeGreaterThan(0)\n    //     expect(defs.allTypings().length).toBeGreaterThan(5000)\n    //     const j = defs.tryGetLatestVersion(\"jquery\")\n    //     expect(j).toBeDefined()\n    //     expect(j!.fullNpmName).toContain(\"types\")\n    //     expect(j!.fullNpmName).toContain(\"jquery\")\n    //     expect(defs.allPackages().length).toEqual(defs.allTypings().length + defs.allNotNeeded().length)\n    // },\n    async mockParse() {\n        const log = loggerWithErrors()[0];\n        const defs = await parseDefinitions(createMockDT().fs, undefined, log);\n        expect(defs.allNotNeeded().length).toBe(1);\n        expect(defs.allTypings().length).toBe(3);\n        const j = defs.tryGetLatestVersion(\"jquery\");\n        expect(j).toBeDefined();\n        expect(j!.fullNpmName).toContain(\"types\");\n        expect(j!.fullNpmName).toContain(\"jquery\");\n        expect(defs.allPackages().length).toEqual(defs.allTypings().length + defs.allNotNeeded().length);\n    },\n});\n"
  },
  {
    "path": "src/parse-definitions.ts",
    "content": "import * as yargs from \"yargs\";\n\nimport { FS, getDefinitelyTyped } from \"./get-definitely-typed\";\nimport { Options, writeDataFile } from \"./lib/common\";\nimport { getTypingInfo } from \"./lib/definition-parser\";\nimport { definitionParserWorkerFilename } from \"./lib/definition-parser-worker\";\nimport { AllPackages, readNotNeededPackages, typesDataFilename, TypingsVersionsRaw } from \"./lib/packages\";\nimport { parseNProcesses } from \"./tester/test-runner\";\nimport { LoggerWithErrors, loggerWithErrors } from \"./util/logging\";\nimport { assertDefined, filterNAtATimeOrdered, logUncaughtErrors, runWithChildProcesses } from \"./util/util\";\n\nif (!module.parent) {\n    const singleName = yargs.argv.single as string | undefined;\n    const options = Options.defaults;\n    logUncaughtErrors(async () => {\n        const log = loggerWithErrors()[0];\n        const dt = await getDefinitelyTyped(options, log);\n        if (singleName)  {\n            await single(singleName, dt);\n        } else {\n            await parseDefinitions(\n                dt,\n                options.parseInParallel\n                    ? { nProcesses: parseNProcesses(), definitelyTypedPath: assertDefined(options.definitelyTypedPath) }\n                    : undefined,\n                log);\n        }\n    });\n}\n\nexport interface ParallelOptions { readonly nProcesses: number; readonly definitelyTypedPath: string; }\nexport default async function parseDefinitions(dt: FS, parallel: ParallelOptions | undefined, log: LoggerWithErrors): Promise<AllPackages> {\n    log.info(\"Parsing definitions...\");\n    const typesFS = dt.subDir(\"types\");\n    const packageNames = await filterNAtATimeOrdered(parallel ? parallel.nProcesses : 1, typesFS.readdir(), name => typesFS.isDirectory(name));\n    log.info(`Found ${packageNames.length} packages.`);\n\n    const typings: { [name: string]: TypingsVersionsRaw } = {};\n\n    const start = Date.now();\n    if (parallel) {\n        log.info(\"Parsing in parallel...\");\n        await runWithChildProcesses({\n            inputs: packageNames,\n            commandLineArgs: [`${parallel.definitelyTypedPath}/types`],\n            workerFile: definitionParserWorkerFilename,\n            nProcesses: parallel.nProcesses,\n            handleOutput({ data, packageName}: { data: TypingsVersionsRaw, packageName: string }) {\n                typings[packageName] = data;\n            },\n        });\n    } else {\n        log.info(\"Parsing non-parallel...\");\n        for (const packageName of packageNames) {\n            typings[packageName] = getTypingInfo(packageName, typesFS.subDir(packageName));\n        }\n    }\n    log.info(\"Parsing took \" + ((Date.now() - start) / 1000) + \" s\");\n    await writeDataFile(typesDataFilename, sorted(typings));\n    return AllPackages.from(typings, readNotNeededPackages(dt));\n}\n\nfunction sorted<T>(obj: { [name: string]: T }): { [name: string]: T } {\n    const out: { [name: string]: T } = {};\n    for (const key of Object.keys(obj).sort()) {\n        out[key] = obj[key];\n    }\n    return out;\n}\n\nasync function single(singleName: string, dt: FS): Promise<void> {\n    const data = getTypingInfo(singleName, dt.subDir(\"types\").subDir(singleName));\n    const typings = { [singleName]: data };\n    await writeDataFile(typesDataFilename, typings);\n    console.log(JSON.stringify(data, undefined, 4));\n}\n"
  },
  {
    "path": "src/publish-packages.ts",
    "content": "import appInsights = require(\"applicationinsights\");\nimport * as yargs from \"yargs\";\n\nimport { getDefinitelyTyped } from \"./get-definitely-typed\";\nimport { Options, Registry } from \"./lib/common\";\nimport { NpmPublishClient, UncachedNpmInfoClient, withNpmCache } from \"./lib/npm-client\";\nimport { deprecateNotNeededPackage, publishNotNeededPackage, publishTypingsPackage } from \"./lib/package-publisher\";\nimport { AllPackages } from \"./lib/packages\";\nimport { ChangedPackages, readChangedPackages, skipBadPublishes } from \"./lib/versions\";\nimport { Fetcher } from \"./util/io\";\nimport { logger, loggerWithErrors, writeLog } from \"./util/logging\";\nimport { logUncaughtErrors } from \"./util/util\";\n\nif (!module.parent) {\n    const dry = !!yargs.argv.dry;\n    const deprecateName = yargs.argv.deprecate as string | undefined;\n    logUncaughtErrors(async () => {\n        const dt = await getDefinitelyTyped(Options.defaults, loggerWithErrors()[0]);\n        if (deprecateName !== undefined) {\n            // A '--deprecate' command is available in case types-publisher got stuck *while* trying to deprecate a package.\n            // Normally this should not be needed.\n\n            const log = logger()[0];\n            try {\n                await deprecateNotNeededPackage(\n                    await NpmPublishClient.create(undefined, Registry.Github),\n                    AllPackages.readSingleNotNeeded(deprecateName, dt),\n                    false, /*dry*/\n                    log,\n                );\n            } catch (e) {\n                // log and continue\n                log(\"publishing to github failed: \" + e.toString());\n            }\n            await deprecateNotNeededPackage(\n                await NpmPublishClient.create(undefined, Registry.NPM), AllPackages.readSingleNotNeeded(deprecateName, dt), /*dry*/ false, log);\n        } else {\n            await publishPackages(await readChangedPackages(await AllPackages.read(dt)), dry, process.env.GH_API_TOKEN || \"\", new Fetcher());\n        }\n    });\n}\n\nexport default async function publishPackages(\n    changedPackages: ChangedPackages,\n    dry: boolean,\n    githubAccessToken: string,\n    fetcher: Fetcher): Promise<void> {\n    const [log, logResult] = logger();\n    if (dry) {\n        log(\"=== DRY RUN ===\");\n    } else {\n        log(\"=== Publishing packages ===\");\n    }\n\n    const client = await NpmPublishClient.create(undefined, Registry.NPM);\n    const ghClient = await NpmPublishClient.create(undefined, Registry.Github);\n\n    for (const cp of changedPackages.changedTypings) {\n        log(`Publishing ${cp.pkg.desc}...`);\n\n        try {\n            await publishTypingsPackage(ghClient, cp, dry, log, Registry.Github);\n        } catch (e) {\n            // log and continue\n            log(\"publishing to github failed: \" + e.toString());\n        }\n        await publishTypingsPackage(client, cp, dry, log, Registry.NPM);\n\n        const commits = await queryGithub(\n            `repos/DefinitelyTyped/DefinitelyTyped/commits?path=types%2f${cp.pkg.subDirectoryPath}`,\n            githubAccessToken,\n            fetcher) as Array<{\n            sha: string,\n            commit: {\n                message: string,\n                author: {\n                    date: string,\n                },\n            },\n        }>;\n\n        const firstCommit = commits[0];\n        if (firstCommit && !firstCommit.commit.message.includes(\"#no-publishing-comment\")) {\n            log(\"Found related commits; hash: \" + commits[0].sha);\n            const prs = await queryGithub(\n                `search/issues?q=is:pr%20is:merged%20${commits[0].sha}`,\n                githubAccessToken,\n                fetcher) as { items: Array<{ number: number }> };\n            let latestPr = 0;\n            for (const pr of prs.items) {\n                if (pr.number > latestPr) {\n                    latestPr = pr.number;\n                }\n            }\n            log(\"Latest PR: \" + latestPr);\n            if (latestPr === 0) {\n                continue;\n            }\n            const latest =\n                await queryGithub(`repos/DefinitelyTyped/DefinitelyTyped/pulls/${latestPr}`, githubAccessToken, fetcher) as { merged_at: string };\n            const latency = Date.now() - new Date(latest.merged_at).valueOf();\n            const commitlatency = Date.now() - new Date(commits[0].commit.author.date).valueOf();\n            log(\"Current date is \" + new Date(Date.now()).toString());\n            log(\"  Merge date is \" + new Date(latest.merged_at).toString());\n\n            const published = cp.pkg.fullNpmName + \"@\" + cp.version;\n            const publishNotification =\n                \"I just published [`\" + published + \"` to npm](https://www.npmjs.com/package/\" + cp.pkg.fullNpmName + \").\";\n            log(publishNotification);\n            if (dry) {\n                log(\"(dry) Skip publishing notification to github.\");\n            } else {\n                const commented = await postGithub(\n                    `repos/DefinitelyTyped/DefinitelyTyped/issues/${latestPr}/comments`,\n                    { body: publishNotification },\n                    githubAccessToken,\n                    fetcher);\n                log(\"From github: \" + JSON.stringify(commented).slice(0, 200));\n            }\n            if (dry) {\n                log(\"(dry) Not logging latency\");\n            } else {\n                appInsights.defaultClient.trackEvent({\n                    name: \"publish package\",\n                    properties: {\n                        name: cp.pkg.desc,\n                        latency: latency.toString(),\n                        commitLatency: commitlatency.toString(),\n                        authorCommit: commits[0].sha,\n                        pr: latestPr.toString(),\n                    },\n                });\n                appInsights.defaultClient.trackMetric({ name: \"publish latency\", value: latency });\n                appInsights.defaultClient.trackMetric({ name: \"author commit latency\", value: commitlatency });\n                log(\"Done logging latency\");\n            }\n        }\n    }\n\n    await withNpmCache(new UncachedNpmInfoClient(), async infoClient => {\n        for (const n of changedPackages.changedNotNeededPackages) {\n            const target = skipBadPublishes(n, infoClient, log);\n            try {\n                await publishNotNeededPackage(ghClient, target, dry, log, Registry.Github);\n            } catch (e) {\n                // log and continue\n                log(\"publishing to github failed: \" + e.toString());\n            }\n            await publishNotNeededPackage(client, target, dry, log, Registry.NPM);\n        }\n    });\n\n    await writeLog(\"publishing.md\", logResult());\n    console.log(\"Done!\");\n}\n\nasync function postGithub(path: string, data: any, githubToken: string, fetcher: Fetcher) {\n    const [log] = logger();\n    const body = JSON.stringify(data);\n    log(`Posting to github at ${path}: ${body}`);\n    return fetcher.fetchJson({\n        hostname: \"api.github.com\",\n        method: \"POST\",\n        path,\n        body,\n        headers: {\n            // arbitrary string, but something must be provided\n            \"User-Agent\": \"types-publisher\",\n            \"Content-Type\": \"application/json\",\n            Authorization: \"token \" + githubToken,\n            \"Content-Length\": Buffer.byteLength(body),\n        },\n    });\n}\n\nasync function queryGithub(path: string, githubToken: string, fetcher: Fetcher) {\n    const [log] = logger();\n    log(\"Requesting from github: \" + path);\n    return fetcher.fetchJson({\n        hostname: \"api.github.com\",\n        method: \"GET\",\n        path: path + \"&access_token=\" + githubToken,\n        headers: {\n            // arbitrary string, but something must be provided\n            \"User-Agent\": \"types-publisher\",\n        },\n    });\n}\n"
  },
  {
    "path": "src/publish-registry.ts",
    "content": "import assert = require(\"assert\");\nimport { emptyDir } from \"fs-extra\";\nimport * as yargs from \"yargs\";\n\nimport { FS, getDefinitelyTyped } from \"./get-definitely-typed\";\nimport { Options, Registry as RegistryName } from \"./lib/common\";\nimport { CachedNpmInfoClient, NpmPublishClient, UncachedNpmInfoClient, withNpmCache } from \"./lib/npm-client\";\nimport { AllPackages, NotNeededPackage, readNotNeededPackages, TypingsData } from \"./lib/packages\";\nimport { outputDirPath, validateOutputPath } from \"./lib/settings\";\nimport { Semver } from \"./lib/versions\";\nimport { npmInstallFlags, readJson, sleep, writeFile, writeJson } from \"./util/io\";\nimport { logger, Logger, loggerWithErrors, writeLog } from \"./util/logging\";\nimport { assertDefined, best, computeHash, execAndThrowErrors, joinPaths, logUncaughtErrors, mapDefined } from \"./util/util\";\n\nconst typesRegistry = \"types-registry\";\nconst registryOutputPath = joinPaths(outputDirPath, typesRegistry);\nconst readme =\n    `This package contains a listing of all packages published to the @types scope on NPM.\nGenerated by [types-publisher](https://github.com/Microsoft/types-publisher).`;\n\nif (!module.parent) {\n    const dry = !!yargs.argv.dry;\n    logUncaughtErrors(async () => {\n        const dt = await getDefinitelyTyped(Options.defaults, loggerWithErrors()[0]);\n        await publishRegistry(dt, await AllPackages.read(dt), dry, new UncachedNpmInfoClient());\n    });\n}\n\nexport default async function publishRegistry(dt: FS, allPackages: AllPackages, dry: boolean, client: UncachedNpmInfoClient): Promise<void> {\n    const [log, logResult] = logger();\n    log(\"=== Publishing types-registry ===\");\n\n    const { npmVersion, highestSemverVersion, npmContentHash, lastModified } =\n        await fetchAndProcessNpmInfo(typesRegistry, client);\n    assert.strictEqual(npmVersion.major, 0);\n    assert.strictEqual(npmVersion.minor, 1);\n\n    // Don't include not-needed packages in the registry.\n    const registryJsonData = await withNpmCache(client, cachedClient => generateRegistry(allPackages.allLatestTypings(), cachedClient));\n    const registry = JSON.stringify(registryJsonData);\n    const newContentHash = computeHash(registry);\n    const newVersion = `0.1.${npmVersion.patch + 1}`;\n    const isTimeForNewVersion = isSevenDaysAfter(lastModified);\n\n    try {\n        await publishToRegistry(RegistryName.Github);\n    } catch (e) {\n        // log and continue\n        log(\"publishing to github failed: \" + e.toString());\n    }\n    await publishToRegistry(RegistryName.NPM);\n    await writeLog(\"publish-registry.md\", logResult());\n\n    async function publishToRegistry(registryName: RegistryName) {\n        const packageName = registryName === RegistryName.Github ? \"@definitelytyped/\" + typesRegistry : typesRegistry;\n        const packageJson = generatePackageJson(packageName, registryName, newVersion, newContentHash);\n        await generate(registry, packageJson);\n\n        const publishClient = () => NpmPublishClient.create({ defaultTag: \"next\" }, registryName);\n        if (!highestSemverVersion.equals(npmVersion)) {\n            // There was an error in the last publish and types-registry wasn't validated.\n            // This may have just been due to a timeout, so test if types-registry@next is a subset of the one we're about to publish.\n            // If so, we should just update it to \"latest\" now.\n            log(\"Old version of types-registry was never tagged latest, so updating\");\n            await validateIsSubset(readNotNeededPackages(dt), log);\n            await (await publishClient()).tag(packageName, highestSemverVersion.versionString, \"latest\", dry, log);\n        } else if (npmContentHash !== newContentHash && isTimeForNewVersion) {\n            log(\"New packages have been added, so publishing a new registry.\");\n            await publish(await publishClient(), packageName, packageJson, newVersion, dry, log);\n        } else {\n            const reason = npmContentHash === newContentHash ? \"No new packages published\" : \"Was modified less than a week ago\";\n            log(`${reason}, so no need to publish new registry.`);\n            // Just making sure...\n            await validate(log);\n        }\n    }\n\n}\n\nconst millisecondsPerDay = 1000 * 60 * 60 * 24;\nfunction isSevenDaysAfter(time: Date): boolean {\n    const diff = Date.now() - time.getTime();\n    const days = diff / millisecondsPerDay;\n    return days > 7;\n}\n\nasync function generate(registry: string, packageJson: {}): Promise<void> {\n    await emptyDir(registryOutputPath);\n    await writeOutputJson(\"package.json\", packageJson);\n    await writeOutputFile(\"index.json\", registry);\n    await writeOutputFile(\"README.md\", readme);\n\n    function writeOutputJson(filename: string, content: object): Promise<void> {\n        return writeJson(outputPath(filename), content);\n    }\n\n    function writeOutputFile(filename: string, content: string): Promise<void> {\n        return writeFile(outputPath(filename), content);\n    }\n\n    function outputPath(filename: string): string {\n        return joinPaths(registryOutputPath, filename);\n    }\n}\n\nasync function publish(client: NpmPublishClient, packageName: string, packageJson: {}, version: string, dry: boolean, log: Logger): Promise<void> {\n    await client.publish(registryOutputPath, packageJson, dry, log);\n    // Sleep for 60 seconds to let NPM update.\n    if (dry) {\n        log(\"(dry) Skipping 60 second sleep...\");\n    } else {\n        log(\"Sleeping for 60 seconds ...\");\n        await sleep(60);\n    }\n    // Don't set it as \"latest\" until *after* it's been validated.\n    await validate(log);\n    await client.tag(packageName, version, \"latest\", dry, log);\n}\n\nasync function installForValidate(log: Logger): Promise<void> {\n    await emptyDir(validateOutputPath);\n    await writeJson(joinPaths(validateOutputPath, \"package.json\"), {\n        name: \"validate\",\n        version: \"0.0.0\",\n        description: \"description\",\n        readme: \"\",\n        license: \"\",\n        repository: {},\n    });\n\n    const npmPath = joinPaths(__dirname, \"..\", \"node_modules\", \"npm\", \"bin\", \"npm-cli.js\");\n    const cmd = `node ${npmPath} install types-registry@next ${npmInstallFlags}`;\n    log(cmd);\n    const err = (await execAndThrowErrors(cmd, validateOutputPath)).trim();\n    if (err) {\n        console.error(err);\n    }\n}\n\nconst validateTypesRegistryPath = joinPaths(validateOutputPath, \"node_modules\", \"types-registry\");\n\nasync function validate(log: Logger): Promise<void> {\n    await installForValidate(log);\n    const output = joinPaths(registryOutputPath, \"index.json\");\n    const nodeModules = joinPaths(validateTypesRegistryPath, \"index.json\");\n    log(`Checking that ${output} is newer than ${nodeModules}`);\n    assertJsonNewer(await readJson(output), await readJson(nodeModules), log);\n}\n\nasync function validateIsSubset(notNeeded: ReadonlyArray<NotNeededPackage>, log: Logger): Promise<void> {\n    await installForValidate(log);\n    const indexJson = \"index.json\";\n    const actual = await readJson(joinPaths(validateTypesRegistryPath, indexJson)) as Registry;\n    const expected = await readJson(joinPaths(registryOutputPath, indexJson)) as Registry;\n    for (const key of Object.keys(actual.entries)) {\n        if (!(key in expected.entries) && !notNeeded.some(p => p.name === key)) {\n            throw new Error(`Actual types-registry has unexpected key ${key}`);\n        }\n    }\n}\n\nfunction assertJsonNewer(newer: { [s: string]: any }, older: { [s: string]: any }, log: Logger, parent = \"\") {\n    for (const key of Object.keys(older)) {\n        if (!newer.hasOwnProperty(key)) {\n            log(`${key} in ${parent} was not found in newer -- assumed to be deprecated.`);\n            continue;\n        }\n        switch (typeof newer[key]) {\n            case \"string\":\n                const newerver = Semver.tryParse(newer[key]);\n                const olderver = Semver.tryParse(older[key]);\n                const condition = newerver && olderver ?\n                    newerver.greaterThan(olderver) || newerver.equals(olderver) :\n                    newer[key] >= older[key];\n                assert(condition, `${key} in ${parent} did not match: newer[key] (${newer[key]}) < older[key] (${older[key]})`);\n                break;\n            case \"number\":\n                assert(newer[key] >= older[key], `${key} in ${parent} did not match: newer[key] (${newer[key]}) < older[key] (${older[key]})`);\n                break;\n            case \"boolean\":\n                assert(newer[key] === older[key], `${key} in ${parent} did not match: newer[key] (${newer[key]}) !== older[key] (${older[key]})`);\n                break;\n            default:\n                assertJsonNewer(newer[key], older[key], log, key);\n        }\n    }\n}\n\nfunction generatePackageJson(name: string, registryName: RegistryName, version: string, typesPublisherContentHash: string): object {\n    const json = {\n        name,\n        version,\n        description: \"A registry of TypeScript declaration file packages published within the @types scope.\",\n        repository: {\n            type: \"git\",\n            url: registryName === RegistryName.Github\n                ? \"https://github.com/DefinitelyTyped/DefinitelyTyped.git\"\n                : \"https://github.com/Microsoft/types-publisher.git\",\n        },\n        keywords: [\n            \"TypeScript\",\n            \"declaration\",\n            \"files\",\n            \"types\",\n            \"packages\",\n        ],\n        author: \"Microsoft Corp.\",\n        license: \"MIT\",\n        typesPublisherContentHash,\n    };\n    if (registryName === RegistryName.Github) {\n        (json as any).publishConfig = { registry: \"https://npm.pkg.github.com/\" };\n    }\n    return json;\n}\n\ninterface Registry {\n    readonly entries: {\n        readonly [packageName: string]: {\n            readonly [distTags: string]: string,\n        },\n    };\n}\nasync function generateRegistry(typings: ReadonlyArray<TypingsData>, client: CachedNpmInfoClient): Promise<Registry> {\n    const entries: { [packageName: string]: { [distTags: string]: string } } = {};\n    for (const typing of typings) {\n        // Unconditionally use cached info, this should have been set in calculate-versions so should be recent enough.\n        const info = client.getNpmInfoFromCache(typing.fullEscapedNpmName);\n        if (!info) {\n            const missings = typings.filter(t => !client.getNpmInfoFromCache(t.fullEscapedNpmName)).map(t => t.fullEscapedNpmName);\n            throw new Error(`${missings.toString()} not found in cached npm info.`);\n        }\n        entries[typing.name] = filterTags(info.distTags);\n    }\n    return { entries };\n\n    function filterTags(tags: Map<string, string>): { readonly [tag: string]: string; } {\n        const latestTag = \"latest\";\n        const latestVersion = tags.get(latestTag);\n        const out: { [tag: string]: string } = {};\n        tags.forEach((value, tag) => {\n            if (tag === latestTag || value !== latestVersion) {\n                out[tag] = value;\n            }\n        });\n        return out;\n    }\n}\n\ninterface ProcessedNpmInfo {\n    readonly npmVersion: Semver;\n    readonly highestSemverVersion: Semver;\n    readonly npmContentHash: string;\n    readonly lastModified: Date;\n}\n\nasync function fetchAndProcessNpmInfo(escapedPackageName: string, client: UncachedNpmInfoClient): Promise<ProcessedNpmInfo> {\n    const info = assertDefined(await client.fetchNpmInfo(escapedPackageName));\n    const npmVersion = Semver.parse(assertDefined(info.distTags.get(\"latest\")));\n    const { distTags, versions, time } = info;\n    const highestSemverVersion = getLatestVersion(versions.keys());\n    assert.strictEqual(highestSemverVersion.versionString, distTags.get(\"next\"));\n    const npmContentHash = versions.get(npmVersion.versionString)!.typesPublisherContentHash || \"\";\n    return { npmVersion, highestSemverVersion, npmContentHash, lastModified: new Date(time.get(\"modified\")!) };\n}\nfunction getLatestVersion(versions: Iterable<string>): Semver {\n    return best(mapDefined(versions, v => Semver.tryParse(v)), (a, b) => a.greaterThan(b))!;\n}\n"
  },
  {
    "path": "src/test-get-secrets.ts",
    "content": "// Run `node ./bin/test-get-secrets.js` to test that we can fetch secrets from Azure Keyvault\n\nimport { allSecrets, getSecret, Secret } from \"./lib/secrets\";\nimport { logUncaughtErrors } from \"./util/util\";\n\nif (!module.parent) {\n    logUncaughtErrors(main());\n}\n\nasync function main(): Promise<void> {\n    for (const secret of allSecrets) {\n        console.log(`Fetching secret '${Secret[secret]}'...`);\n        console.log(await getSecret(secret));\n    }\n}\n"
  },
  {
    "path": "src/tester/get-affected-packages.test.ts",
    "content": "import { AllPackages, NotNeededPackage, TypesDataFile } from \"../lib/packages\";\nimport { createTypingsVersionRaw, testo } from \"../util/test\";\n\nimport { getAffectedPackages } from \"./get-affected-packages\";\nconst typesData: TypesDataFile = {\n    jquery: createTypingsVersionRaw(\"jquery\", [], []),\n    known: createTypingsVersionRaw(\"known\", [{ name: \"jquery\", version: { major: 1 }}], []),\n    \"known-test\": createTypingsVersionRaw(\"known-test\", [], [\"jquery\"]),\n    \"most-recent\": createTypingsVersionRaw(\"most-recent\", [{ name: \"jquery\", version: \"*\" }], []),\n    unknown: createTypingsVersionRaw(\"unknown\", [{ name: \"COMPLETELY-UNKNOWN\", version: { major: 1 }}], []),\n    \"unknown-test\": createTypingsVersionRaw(\"unknown-test\", [], [\"WAT\"]),\n};\n\nconst notNeeded = [\n    new NotNeededPackage({ typingsPackageName: \"jest\", libraryName: \"jest\", asOfVersion: \"100.0.0\", sourceRepoURL: \"jest.com\" }),\n];\nconst allPackages = AllPackages.from(typesData, notNeeded);\n\ntesto({\n    updatedPackage() {\n        const affected = getAffectedPackages(allPackages, [{ name: \"jquery\", version: { major: 1 }}]);\n        expect(affected.changedPackages.length).toEqual(1);\n        expect((affected.changedPackages[0] as any).data).toEqual(typesData.jquery[\"1.0.0\"]);\n        expect(affected.dependentPackages.length).toEqual(3);\n    },\n    deletedPackage() {\n        const affected = getAffectedPackages(allPackages, [{ name: \"WAT\", version: \"*\" }]);\n        expect(affected.changedPackages.length).toEqual(0);\n        expect(affected.dependentPackages.length).toEqual(1);\n    },\n});\n"
  },
  {
    "path": "src/tester/get-affected-packages.ts",
    "content": "import { AllPackages, formatDependencyVersion, getMangledNameForScopedPackage, PackageBase, PackageId, TypingsData } from \"../lib/packages\";\nimport { mapDefined, mapIter, sort } from \"../util/util\";\n\nexport interface Affected {\n    readonly changedPackages: ReadonlyArray<TypingsData>;\n    readonly dependentPackages: ReadonlyArray<TypingsData>;\n    allPackages: AllPackages;\n}\n\n/** Gets all packages that have changed on this branch, plus all packages affected by the change. */\nexport function getAffectedPackages(allPackages: AllPackages, changedPackageIds: PackageId[]): Affected {\n    const resolved = changedPackageIds.map(id => allPackages.tryResolve(id));\n    // If a package doesn't exist, that's because it was deleted.\n    const changed = mapDefined(resolved, id => allPackages.tryGetTypingsData(id));\n    const dependent = mapIter(collectDependers(resolved, getReverseDependencies(allPackages, resolved)), p => allPackages.getTypingsData(p));\n    return { changedPackages: changed, dependentPackages: sortPackages(dependent), allPackages };\n}\n\n/** Every package name in the original list, plus their dependencies (incl. dependencies' dependencies). */\nexport function allDependencies(allPackages: AllPackages, packages: Iterable<TypingsData>): TypingsData[] {\n    return sortPackages(transitiveClosure(packages, pkg => allPackages.allDependencyTypings(pkg)));\n}\n\n/** Collect all packages that depend on changed packages, and all that depend on those, etc. */\nfunction collectDependers(changedPackages: PackageId[], reverseDependencies: Map<PackageId, Set<PackageId>>): Set<PackageId> {\n    const dependers = transitiveClosure(changedPackages, pkg => reverseDependencies.get(pkg) || []);\n    // Don't include the original changed packages, just their dependers\n    for (const original of changedPackages) {\n        dependers.delete(original);\n    }\n    return dependers;\n}\n\nfunction sortPackages(packages: Iterable<TypingsData>): TypingsData[] {\n    return sort<TypingsData>(packages, PackageBase.compare); // tslint:disable-line no-unbound-method\n}\n\nfunction transitiveClosure<T>(initialItems: Iterable<T>, getRelatedItems: (item: T) => Iterable<T>): Set<T> {\n    const all = new Set<T>();\n    const workList: T[] = [];\n\n    function add(item: T): void {\n        if (!all.has(item)) {\n            all.add(item);\n            workList.push(item);\n        }\n    }\n\n    for (const item of initialItems) {\n        add(item);\n    }\n\n    while (workList.length) {\n        const item = workList.pop()!;\n        for (const newItem of getRelatedItems(item)) {\n            add(newItem);\n        }\n    }\n\n    return all;\n}\n\n/** Generate a map from a package to packages that depend on it. */\nfunction getReverseDependencies(allPackages: AllPackages, changedPackages: PackageId[]): Map<PackageId, Set<PackageId>> {\n   const map = new Map<string, [PackageId, Set<PackageId>]>();\n   for (const changed of changedPackages) {\n         map.set(packageIdToKey(changed), [changed, new Set()]);\n    }\n   for (const typing of allPackages.allTypings()) {\n        if (!map.has(packageIdToKey(typing.id))) {\n            map.set(packageIdToKey(typing.id), [typing.id, new Set()]);\n        }\n    }\n   for (const typing of allPackages.allTypings()) {\n        for (const dependency of typing.dependencies) {\n            const dependencies = map.get(packageIdToKey(allPackages.tryResolve(dependency)));\n            if (dependencies) {\n                dependencies[1].add(typing.id);\n            }\n        }\n        for (const dependencyName of typing.testDependencies) {\n            const latest: PackageId = { name: dependencyName, version: \"*\" };\n            const dependencies = map.get(packageIdToKey(allPackages.tryResolve(latest)));\n            if (dependencies) {\n                dependencies[1].add(typing.id);\n            }\n        }\n    }\n   return new Map(map.values());\n}\n\nfunction packageIdToKey(pkg: PackageId): string {\n    return getMangledNameForScopedPackage(pkg.name) + \"/v\" + formatDependencyVersion(pkg.version);\n}\n"
  },
  {
    "path": "src/tester/test-runner.test.ts",
    "content": "import { NpmInfo } from \"../lib/npm-client\";\nimport { AllPackages, NotNeededPackage, TypesDataFile } from \"../lib/packages\";\nimport { createTypingsVersionRaw, testo } from \"../util/test\";\n\nimport { checkNotNeededPackage, getNotNeededPackages, GitDiff } from \"./test-runner\";\n\nconst typesData: TypesDataFile = {\n    jquery: createTypingsVersionRaw(\"jquery\", [], []),\n    known: createTypingsVersionRaw(\"known\", [{ name: \"jquery\", version: { major: 1 }}], []),\n    \"known-test\": createTypingsVersionRaw(\"known-test\", [], [\"jquery\"]),\n    \"most-recent\": createTypingsVersionRaw(\"most-recent\", [{ name: \"jquery\", version: \"*\" }], []),\n    unknown: createTypingsVersionRaw(\"unknown\", [{ name: \"COMPLETELY-UNKNOWN\", version: { major: 1 }}], []),\n    \"unknown-test\": createTypingsVersionRaw(\"unknown-test\", [], [\"WAT\"]),\n};\n\nconst jestNotNeeded = [\n    new NotNeededPackage({ typingsPackageName: \"jest\", libraryName: \"jest\", asOfVersion: \"100.0.0\", sourceRepoURL: \"jest.com\" }),\n];\nconst allPackages = AllPackages.from(typesData, jestNotNeeded);\n\nconst deleteJestDiffs: GitDiff[] = [\n    { status: \"M\", file: \"notNeededPackages.json\" },\n    { status: \"D\", file: \"types/jest/index.d.ts\" },\n    { status: \"D\", file: \"types/jest/jest-tests.d.ts\" },\n];\n\ntesto({\n    ok() {\n        expect(Array.from(getNotNeededPackages(allPackages, deleteJestDiffs))).toEqual(jestNotNeeded);\n    },\n    forgotToDeleteFiles() {\n        expect(() =>\n            Array.from(getNotNeededPackages(\n                AllPackages.from({ jest: createTypingsVersionRaw(\"jest\", [], []) }, jestNotNeeded),\n                deleteJestDiffs))).toThrow(\"Please delete all files in jest\");\n\n    },\n    tooManyDeletes() {\n        expect(() => Array.from(getNotNeededPackages(allPackages, [{ status: \"D\", file: \"oops.txt\" }]))).toThrow(\n            \"Unexpected file deleted: oops.txt\");\n    },\n    extraneousFile() {\n        Array.from(getNotNeededPackages(allPackages, [\n            { status: \"A\", file: \"oooooooooooops.txt\" },\n            { status: \"M\", file: \"notNeededPackages.json\" },\n            { status: \"D\", file: \"types/jest/index.d.ts\" },\n            { status: \"D\", file: \"types/jest/jest-tests.d.ts\" },\n        ]));\n    },\n    forgotToUpdateNotNeededJson() {\n        expect(() => Array.from(getNotNeededPackages(AllPackages.from(typesData, []), [{status: \"D\", file: \"types/jest/index.d.ts\" }]))).toThrow(\n            \"Deleted package jest is not in notNeededPackages.json.\");\n    },\n    scoped() {\n        Array.from(getNotNeededPackages(\n            AllPackages.from(\n                typesData,\n                [new NotNeededPackage({\n                    typingsPackageName: \"ember__object\",\n                    libraryName: \"@ember/object\",\n                    asOfVersion: \"1.0.0\",\n                    sourceRepoURL: \"ember.js\",\n                })],\n            ),\n            [{ status: \"D\", file: \"types/ember__object/index.d.ts\" }]));\n    },\n    // TODO: Test npm info (and with scoped names)\n    // TODO: Test with dependents, etc etc\n});\n\nconst empty: NpmInfo = {\n    distTags: new Map(),\n    versions: new Map(),\n    time: new Map(),\n};\ntesto({\n    missingSource() {\n        expect(() => checkNotNeededPackage(jestNotNeeded[0], undefined, empty))\n            .toThrow(\"The entry for @types/jest in notNeededPackages.json\");\n    },\n    missingTypings() {\n        expect(() => checkNotNeededPackage(jestNotNeeded[0], empty, undefined))\n            .toThrow(\"@types package not found for @types/jest\");\n    },\n    missingTypingsLatest() {\n        expect(() => checkNotNeededPackage(jestNotNeeded[0], empty, empty))\n            .toThrow(\"@types/jest is missing the \\\"latest\\\" tag\");\n    },\n    deprecatedSameVersion() {\n        expect(() => {\n            checkNotNeededPackage(\n                jestNotNeeded[0],\n                empty,\n                { distTags: new Map([[\"latest\", \"100.0.0\"]]), versions: new Map(), time: new Map([[\"modified\", \"\"]]) },\n            );\n        }).toThrow(`The specified version 100.0.0 of jest must be newer than the version\nit is supposed to replace, 100.0.0 of @types/jest.`);\n    },\n    deprecatedOlderVersion() {\n        expect(() => {\n            checkNotNeededPackage(\n                jestNotNeeded[0],\n                empty,\n                { distTags: new Map([[\"latest\", \"999.0.0\"]]), versions: new Map(), time: new Map([[\"modified\", \"\"]]) },\n            );\n        }).toThrow(`The specified version 100.0.0 of jest must be newer than the version\nit is supposed to replace, 999.0.0 of @types/jest.`);\n    },\n    missingNpmVersion() {\n        expect(() => {\n            checkNotNeededPackage(\n                jestNotNeeded[0],\n                empty,\n                { distTags: new Map([[\"latest\", \"4.0.0\"]]), versions: new Map(), time: new Map([[\"modified\", \"\"]]) },\n            );\n        }).toThrow(\"The specified version 100.0.0 of jest is not on npm.\");\n    },\n    olderNpmVersion() {\n        expect(() => checkNotNeededPackage(\n            jestNotNeeded[0],\n            { distTags: new Map(), versions: new Map([[\"50.0.0\", {}]]), time: new Map([[\"modified\", \"\"]]) },\n            { distTags: new Map([[\"latest\", \"4.0.0\"]]), versions: new Map(), time: new Map([[\"modified\", \"\"]]) }))\n            .toThrow(\"The specified version 100.0.0 of jest is not on npm.\");\n    },\n    ok() {\n        checkNotNeededPackage(\n            jestNotNeeded[0],\n            { distTags: new Map(), versions: new Map([[\"100.0.0\", {}]]), time: new Map([[\"modified\", \"\"]]) },\n            { distTags: new Map([[\"latest\", \"4.0.0\"]]), versions: new Map(), time: new Map([[\"modified\", \"\"]]) });\n    },\n});\n"
  },
  {
    "path": "src/tester/test-runner.ts",
    "content": "import assert = require(\"assert\");\nimport { existsSync, readFileSync } from \"fs\";\nimport { pathExists, remove } from \"fs-extra\";\nimport os = require(\"os\");\nimport * as fold from \"travis-fold\";\nimport * as yargs from \"yargs\";\n\nimport { FS, getDefinitelyTyped } from \"../get-definitely-typed\";\nimport { Options, TesterOptions } from \"../lib/common\";\nimport { parseVersionFromDirectoryName } from \"../lib/definition-parser\";\nimport { NpmInfo, UncachedNpmInfoClient } from \"../lib/npm-client\";\nimport { AllPackages, DependencyVersion, formatDependencyVersion, NotNeededPackage, PackageId, TypingsData } from \"../lib/packages\";\nimport { sourceBranch, typesDirectoryName } from \"../lib/settings\";\nimport { Semver } from \"../lib/versions\";\nimport { npmInstallFlags } from \"../util/io\";\nimport { consoleLogger, Logger, LoggerWithErrors, loggerWithErrors } from \"../util/logging\";\nimport {\n    assertDefined,\n    CrashRecoveryState,\n    exec,\n    execAndThrowErrors,\n    flatMap,\n    joinPaths,\n    logUncaughtErrors,\n    mapIter,\n    numberOfOsProcesses,\n    runWithListeningChildProcesses,\n} from \"../util/util\";\n\nimport { allDependencies, getAffectedPackages } from \"./get-affected-packages\";\n\nconst perfDir = joinPaths(os.homedir(), \".dts\", \"perf\");\nconst suggestionsDir = joinPaths(os.homedir(), \".dts\", \"suggestions\");\n\nif (!module.parent) {\n    if (yargs.argv.affected) {\n        logUncaughtErrors(testAffectedOnly(Options.defaults));\n    } else {\n        const selection = yargs.argv.all ? \"all\" : yargs.argv._[0] ? new RegExp(yargs.argv._[0]) : \"affected\";\n        const options = testerOptions(!!yargs.argv.runFromDefinitelyTyped);\n        logUncaughtErrors(\n            getDefinitelyTyped(options, loggerWithErrors()[0]).then(dt => runTests(dt, options.definitelyTypedPath, parseNProcesses(), selection)));\n    }\n}\n\nexport interface GitDiff {\n    status: \"A\" | \"D\" | \"M\";\n    file: string;\n}\n\nasync function testAffectedOnly(options: TesterOptions): Promise<void> {\n    const changes = getAffectedPackages(\n        await AllPackages.read(await getDefinitelyTyped(options, loggerWithErrors()[0])),\n        gitChanges(await gitDiff(consoleLogger.info, options.definitelyTypedPath)));\n    console.log({ changedPackages: changes.changedPackages.map(t => t.desc), dependersLength: changes.dependentPackages.map(t => t.desc).length });\n}\n\nexport function parseNProcesses(): number {\n    const str = yargs.argv.nProcesses as string | undefined;\n    if (!str) {\n        return numberOfOsProcesses;\n    }\n    const nProcesses = Number.parseInt(str, 10);\n    if (Number.isNaN(nProcesses)) {\n        throw new Error(\"Expected nProcesses to be a number.\");\n    }\n    return nProcesses;\n}\n\nexport function testerOptions(runFromDefinitelyTyped: boolean): TesterOptions {\n    return runFromDefinitelyTyped\n        ? { definitelyTypedPath: process.cwd(), progress: false, parseInParallel: true }\n        : Options.defaults;\n}\n\nexport default async function runTests(\n    dt: FS,\n    definitelyTypedPath: string,\n    nProcesses: number,\n    selection: \"all\" | \"affected\" | RegExp,\n): Promise<void> {\n    const { changedPackages, dependentPackages, allPackages } = await getAffectedPackagesFromDiff(dt, definitelyTypedPath, selection);\n    console.log(`Running with ${nProcesses} processes.`);\n\n    const typesPath = `${definitelyTypedPath}/types`;\n    await doInstalls(allPackages, [...changedPackages, ...dependentPackages], typesPath);\n\n    console.log(\"Testing...\");\n    await doRunTests([...changedPackages, ...dependentPackages], new Set(changedPackages), typesPath, nProcesses);\n}\n\nexport async function getAffectedPackagesFromDiff(dt: FS, definitelyTypedPath: string, selection: \"all\" | \"affected\" | RegExp) {\n    const allPackages = await AllPackages.read(dt);\n    const diffs = await gitDiff(consoleLogger.info, definitelyTypedPath);\n    if (diffs.find(d => d.file === \"notNeededPackages.json\")) {\n        const uncached = new UncachedNpmInfoClient();\n        for (const deleted of getNotNeededPackages(allPackages, diffs)) {\n            const source = await uncached.fetchNpmInfo(deleted.libraryName); // eg @babel/parser\n            const typings = await uncached.fetchNpmInfo(deleted.fullNpmName); // eg @types/babel__parser\n            checkNotNeededPackage(deleted, source, typings);\n        }\n    }\n\n    const affected = selection === \"all\" ? { changedPackages: allPackages.allTypings(), dependentPackages: [], allPackages }\n        : selection === \"affected\" ? getAffectedPackages(allPackages, gitChanges(diffs))\n        : { changedPackages: allPackages.allTypings().filter(t => selection.test(t.name)), dependentPackages: [], allPackages };\n\n    console.log(`Testing ${affected.changedPackages.length} changed packages: ${affected.changedPackages.map(t => t.desc).toString()}`);\n    console.log(`Testing ${affected.dependentPackages.length} dependent packages: ${affected.dependentPackages.map(t => t.desc).toString()}`);\n    return affected;\n}\n\n/**\n * 1. find all the deleted files and group by toplevel\n * 2. Make sure that there are no packages left with deleted entries\n * 3. make sure that each toplevel deleted has a matching entry in notNeededPackages\n */\nexport function getNotNeededPackages(allPackages: AllPackages, diffs: GitDiff[]): Iterable<NotNeededPackage> {\n    const deletedPackages = new Set(diffs.filter(d => d.status === \"D\").map(d =>\n        assertDefined(getDependencyFromFile(d.file),\n                      `Unexpected file deleted: ${d.file}\nWhen removing packages, you should only delete files that are a part of removed packages.`)\n        .name));\n    return mapIter(deletedPackages, p => {\n        if (allPackages.hasTypingFor({ name: p, version: \"*\" })) {\n            throw new Error(`Please delete all files in ${p} when adding it to notNeededPackages.json.`);\n        }\n        return assertDefined(allPackages.getNotNeededPackage(p), `Deleted package ${p} is not in notNeededPackages.json.`);\n    });\n}\n\n/**\n * 1. libraryName must exist on npm (SKIPPED and preferably/optionally have been the libraryName in just-deleted header)\n * (SKIPPED 2.) sourceRepoURL must exist and be the npm homepage\n * 3. asOfVersion must be newer than `@types/name@latest` on npm\n * 4. `name@asOfVersion` must exist on npm\n *\n * I skipped (2) because the cached npm info doesn't include it. I might add it later.\n */\nexport function checkNotNeededPackage(unneeded: NotNeededPackage, source: NpmInfo | undefined, typings: NpmInfo | undefined) {\n    source = assertDefined(source, `The entry for ${unneeded.fullNpmName} in notNeededPackages.json has\n\"libraryName\": \"${unneeded.libraryName}\", but there is no npm package with this name.\nUnneeded packages have to be replaced with a package on npm.`);\n    typings = assertDefined(typings, `Unexpected error: @types package not found for ${unneeded.fullNpmName}`);\n    const latestTypings = Semver.parse(\n        assertDefined(typings.distTags.get(\"latest\"), `Unexpected error: ${unneeded.fullNpmName} is missing the \"latest\" tag.`),\n    );\n    assert(\n        unneeded.version.greaterThan(latestTypings),\n        `The specified version ${unneeded.version.versionString} of ${unneeded.libraryName} must be newer than the version\nit is supposed to replace, ${latestTypings.versionString} of ${unneeded.fullNpmName}.`,\n    );\n    assert(\n        source.versions.has(unneeded.version.versionString),\n        `The specified version ${unneeded.version.versionString} of ${unneeded.libraryName} is not on npm.`,\n    );\n}\n\nasync function doInstalls(allPackages: AllPackages, packages: Iterable<TypingsData>, typesPath: string): Promise<void> {\n    console.log(\"Installing NPM dependencies...\");\n\n    // We need to run `npm install` for all dependencies, too, so that we have dependencies' dependencies installed.\n    for (const pkg of allDependencies(allPackages, packages)) {\n        const cwd = directoryPath(typesPath, pkg);\n        if (!await pathExists(joinPaths(cwd, \"package.json\"))) {\n            continue;\n        }\n\n        // Scripts may try to compile native code.\n        // This doesn't work reliably on travis, and we're just installing for the types, so ignore.\n        const cmd = `npm install ${npmInstallFlags}`;\n        console.log(`  ${cwd}: ${cmd}`);\n        const stdout = await execAndThrowErrors(cmd, cwd);\n        if (stdout) {\n            // Must specify what this is for since these run in parallel.\n            console.log(` from ${cwd}: ${stdout}`);\n        }\n    }\n\n    await runCommand(console, undefined, require.resolve(\"dtslint\"), [\"--installAll\"]);\n}\n\nfunction directoryPath(typesPath: string, pkg: TypingsData): string {\n    return joinPaths(typesPath, pkg.subDirectoryPath);\n}\n\nasync function doRunTests(\n    packages: ReadonlyArray<TypingsData>,\n    changed: ReadonlySet<TypingsData>,\n    typesPath: string,\n    nProcesses: number,\n): Promise<void> {\n    await remove(suggestionsDir);\n    const allFailures: Array<[string, string]> = [];\n    if (fold.isTravis()) { console.log(fold.start(\"tests\")); }\n    await runWithListeningChildProcesses({\n        inputs: packages.map(p => ({ path: p.subDirectoryPath, onlyTestTsNext: !changed.has(p), expectOnly: !changed.has(p) })),\n        commandLineArgs: [\"--listen\"],\n        workerFile: require.resolve(\"dtslint\"),\n        nProcesses,\n        crashRecovery: true,\n        crashRecoveryMaxOldSpaceSize: 0, // disable retry with more memory\n        cwd: typesPath,\n        handleStart(input, processIndex): void {\n            const prefix = processIndex === undefined ? \"\" : `${processIndex}> `;\n            console.log(`${prefix}${input.path} START`);\n        },\n        handleOutput(output, processIndex): void {\n            const prefix = processIndex === undefined ? \"\" : `${processIndex}> `;\n            const { path, status } = output as { path: string, status: string };\n            if (status === \"OK\") {\n                console.log(`${prefix}${path} OK`);\n            } else {\n                console.error(`${prefix}${path} failing:`);\n                console.error(prefix ? status.split(/\\r?\\n/).map(line => `${prefix}${line}`).join(\"\\n\") : status);\n                allFailures.push([path, status]);\n            }\n        },\n        handleCrash(input, state, processIndex) {\n            const prefix = processIndex === undefined ? \"\" : `${processIndex}> `;\n            switch (state) {\n                case CrashRecoveryState.Retry:\n                    console.warn(`${prefix}${input.path} Out of memory: retrying`);\n                    break;\n                case CrashRecoveryState.RetryWithMoreMemory:\n                    console.warn(`${prefix}${input.path} Out of memory: retrying with increased memory (4096M)`);\n                    break;\n                case CrashRecoveryState.Crashed:\n                    console.error(`${prefix}${input.path} Out of memory: failed`);\n                    allFailures.push([input.path, \"Out of memory\"]);\n                    break;\n                default:\n            }\n        },\n    });\n    if (fold.isTravis()) { console.log(fold.end(\"tests\")); }\n\n    console.log(\"\\n\\n=== SUGGESTIONS ===\\n\");\n    const suggestionLines: string[] = [];\n    for (const change of changed) {\n        const pkgPath = change.versionDirectoryName ? change.name + change.versionDirectoryName : change.name;\n        const path = joinPaths(suggestionsDir, pkgPath + \".txt\");\n        if (existsSync(path)) {\n            const suggestions = readFileSync(path, \"utf8\").split(\"\\n\");\n            suggestionLines.push(`\"${change.subDirectoryPath}\": [${suggestions.join(\",\")}]`);\n        }\n    }\n    console.log(`{${suggestionLines.join(\",\")}}`);\n\n    console.log(\"\\n\\n=== PERFORMANCE ===\\n\");\n    console.log(\"{\");\n    for (const change of changed) {\n        const path = joinPaths(perfDir, change.name + \".json\");\n        if (existsSync(path)) {\n            const perf = JSON.parse(readFileSync(path, \"utf8\")) as { [name: string]: { typeCount: number } };\n            console.log(`  \"${change.name}\": ${perf[change.name].typeCount},`);\n        }\n    }\n    console.log(\"}\");\n\n    if (allFailures.length === 0) {\n        return;\n    }\n\n    console.error(\"\\n\\n=== ERRORS ===\\n\");\n\n    for (const [path, error] of allFailures) {\n        console.error(`\\n\\nError in ${path}`);\n        console.error(error);\n    }\n\n    throw new Error(`The following packages had errors: ${allFailures.map(e => e[0]).join(\", \")}`);\n}\n\ninterface TesterError {\n    message: string;\n}\n\nasync function runCommand(log: LoggerWithErrors, cwd: string | undefined, cmd: string, args: string[]): Promise<TesterError | undefined> {\n    const nodeCmd = `node ${cmd} ${args.join(\" \")}`;\n    log.info(`Running: ${nodeCmd}`);\n    try {\n        const { error, stdout, stderr } = await exec(nodeCmd, cwd);\n        if (stdout) {\n            log.info(stdout);\n        }\n        if (stderr) {\n            log.error(stderr);\n        }\n\n        return error && { message: `${error.message}\\n${stdout}\\n${stderr}` };\n    } catch (e) {\n        return e as TesterError;\n    }\n}\n\n/** Returns all immediate subdirectories of the root directory that have changed. */\nexport function gitChanges(diffs: GitDiff[]): PackageId[] {\n    const changedPackages = new Map<string, Map<string, DependencyVersion>>();\n\n    for (const diff of diffs) {\n        const dep = getDependencyFromFile(diff.file);\n        if (dep) {\n            const versions = changedPackages.get(dep.name);\n            if (!versions) {\n                changedPackages.set(dep.name, new Map([[formatDependencyVersion(dep.version), dep.version]]));\n            } else {\n                versions.set(formatDependencyVersion(dep.version), dep.version);\n            }\n        }\n    }\n\n    return Array.from(flatMap(changedPackages, ([name, versions]) =>\n        mapIter(versions, ([_, version]) => ({ name, version }))));\n}\n\n/*\nWe have to be careful about how we get the diff because travis uses a shallow clone.\n\nTravis runs:\n    git clone --depth=50 https://github.com/DefinitelyTyped/DefinitelyTyped.git DefinitelyTyped\n    cd DefinitelyTyped\n    git fetch origin +refs/pull/123/merge\n    git checkout -qf FETCH_HEAD\n\nIf editing this code, be sure to test on both full and shallow clones.\n*/\nexport async function gitDiff(log: Logger, definitelyTypedPath: string): Promise<GitDiff[]> {\n    try {\n        await run(`git rev-parse --verify ${sourceBranch}`);\n        // If this succeeds, we got the full clone.\n    } catch (_) {\n        // This is a shallow clone.\n        await run(`git fetch origin ${sourceBranch}`);\n        await run(`git branch ${sourceBranch} FETCH_HEAD`);\n    }\n\n    let diff = (await run(`git diff ${sourceBranch} --name-status`)).trim();\n    if (diff === \"\") {\n        // We are probably already on master, so compare to the last commit.\n        diff = (await run(`git diff ${sourceBranch}~1 --name-status`)).trim();\n    }\n    return diff.split(\"\\n\").map(line => {\n        const [status, file] = line.split(/\\s+/, 2);\n        return { status: status.trim(), file: file.trim() } as GitDiff;\n    });\n\n    async function run(cmd: string): Promise<string> {\n        log(`Running: ${cmd}`);\n        const stdout = await execAndThrowErrors(cmd, definitelyTypedPath);\n        log(stdout);\n        return stdout;\n    }\n}\n\n/**\n * For \"types/a/b/c\", returns { name: \"a\", version: \"*\" }.\n * For \"types/a/v3/c\", returns { name: \"a\", version: 3 }.\n * For \"x\", returns undefined.\n */\nfunction getDependencyFromFile(file: string): PackageId | undefined {\n    const parts = file.split(\"/\");\n    if (parts.length <= 2) {\n        // It's not in a typings directory at all.\n        return undefined;\n    }\n\n    const [typesDirName, name, subDirName] = parts; // Ignore any other parts\n\n    if (typesDirName !== typesDirectoryName) {\n        return undefined;\n    }\n\n    if (subDirName) {\n        const version = parseVersionFromDirectoryName(subDirName);\n        if (version !== undefined) {\n            return { name, version };\n        }\n    }\n\n    return { name, version: \"*\" };\n}\n"
  },
  {
    "path": "src/tester/test.ts",
    "content": "import yargs = require(\"yargs\");\n\nimport checkParseResults from \"../check-parse-results\";\nimport { clean } from \"../clean\";\nimport { getDefinitelyTyped } from \"../get-definitely-typed\";\nimport { TesterOptions } from \"../lib/common\";\nimport { UncachedNpmInfoClient } from \"../lib/npm-client\";\nimport parseDefinitions from \"../parse-definitions\";\nimport { loggerWithErrors } from \"../util/logging\";\nimport { logUncaughtErrors } from \"../util/util\";\n\nimport runTests, { getAffectedPackagesFromDiff, parseNProcesses, testerOptions } from \"./test-runner\";\n\nif (!module.parent) {\n    const options = testerOptions(!!yargs.argv.runFromDefinitelyTyped);\n    const all = !!yargs.argv.all;\n    logUncaughtErrors(main(options, parseNProcesses(), all));\n}\n\nasync function main(options: TesterOptions, nProcesses: number, all: boolean): Promise<void> {\n    clean();\n    const log = loggerWithErrors()[0];\n    const dt = await getDefinitelyTyped(options, log);\n    await parseDefinitions(dt, { nProcesses, definitelyTypedPath: options.definitelyTypedPath }, log);\n    try {\n        await checkParseResults(/*includeNpmChecks*/false, dt, options, new UncachedNpmInfoClient());\n    } catch (e) {\n        if (!all) {\n            await getAffectedPackagesFromDiff(dt, options.definitelyTypedPath, \"affected\");\n        }\n\n        throw e;\n    }\n    await runTests(dt, options.definitelyTypedPath, nProcesses, all ? \"all\" : \"affected\");\n}\n"
  },
  {
    "path": "src/types/adal-node.d.ts",
    "content": "export class AuthenticationContext {\n    constructor(authorization: string);\n\n    acquireTokenWithClientCredentials(\n        resource: string, clientId: string, clientSecret: string,\n        callback: (error: Error | null | undefined, tokenResponse: TokenResponse | null | undefined) => void): void;\n}\n\ninterface TokenResponse {\n    tokenType: string;\n    accessToken: string;\n}\n"
  },
  {
    "path": "src/types/azure-keyvault.d.ts",
    "content": "type AuthenticatorCallback = (error: Error | null | undefined, authorization?: string) => void;\nexport class KeyVaultCredentials {\n    constructor(authenticator: (challenge: Challenge, callback: AuthenticatorCallback) => void);\n}\ninterface Challenge {\n    authorization: string;\n    resource: string;\n}\n\nexport class KeyVaultClient {\n    constructor(credentials: KeyVaultCredentials);\n    getSecret(baseUrl: string, name: string, version: string): Promise<SecretBundle>;\n    getSecretVersions(url: string, name: string): Promise<SecretVersion[]>;\n}\n\ninterface SecretVersion {\n    id: string,\n    attributes: {\n        enabled: Date,\n        created: Date,\n        updated: Date,\n    }\n}\n\ninterface SecretBundle {\n    id: string;\n    value: string;\n}\n"
  },
  {
    "path": "src/types/fstream.d.ts",
    "content": "export function Reader(options: ReaderOptions): NodeJS.ReadableStream;\ninterface ReaderOptions {\n    path: string;\n    type: \"Directory\";\n    filter(entry: FStreamEntry): boolean;\n}\ninterface FStreamEntry {\n    props: { type: string, mode: number };\n}\n"
  },
  {
    "path": "src/types/npm-registry-client.d.ts",
    "content": "// Definitions transcribed from https://github.com/npm/npm-registry-client\ndeclare class RegClient {\n    constructor(config?: RegClient.Config);\n    request(uri: string, params: RegClient.RequestParams, cb: (error: Error, data: unknown, json: unknown, response: unknown) => void): void;\n    publish(uri: string, params: RegClient.PublishParams, cb: (error: Error) => void): void;\n    deprecate(uri: string, params: RegClient.DeprecateParams, cb: (error: Error, data: unknown, raw: string, response: unknown) => void): void;\n    distTags: {\n        add(uri: string, params: RegClient.AddTagParams, cb: (error: Error) => void): void;\n    }\n}\n\ndeclare namespace RegClient {\n    interface Config {\n        defaultTag?: string;\n    }\n    interface RequestParams {\n        method?: string;\n        body?: {};\n    }\n    interface PublishParams {\n        metadata: {};\n        access: \"public\" | \"restricted\";\n        body: NodeJS.ReadableStream;\n        auth: Credentials;\n    }\n    interface AddTagParams {\n        package: string;\n        version: string;\n        distTag: string;\n        auth: Credentials;\n    }\n    interface DeprecateParams {\n        version: string;\n        message: string;\n        auth: Credentials;\n    }\n    interface Credentials {\n        token: string;\n    }\n}\n\nexport = RegClient;\n"
  },
  {
    "path": "src/upload-blobs.ts",
    "content": "import uploadBlobsAndUpdateIssue from \"./lib/blob-uploader\";\nimport { currentTimeStamp, logUncaughtErrors } from \"./util/util\";\n\nif (!module.parent) {\n    logUncaughtErrors(uploadBlobsAndUpdateIssue(currentTimeStamp()));\n}\n\nexport default uploadBlobsAndUpdateIssue;\n"
  },
  {
    "path": "src/util/io.ts",
    "content": "import {\n    readFile as readFileWithEncoding,\n    readFileSync as readFileWithEncodingSync,\n    stat,\n    writeFile as writeFileWithEncoding,\n    writeJson as writeJsonRaw,\n} from \"fs-extra\";\nimport { request as httpRequest } from \"http\";\nimport { Agent, request } from \"https\";\nimport { Readable as ReadableStream } from \"stream\";\nimport { StringDecoder } from \"string_decoder\";\n\nimport { parseJson } from \"./util\";\n\nexport async function readFile(path: string): Promise<string> {\n    const res = await readFileWithEncoding(path, { encoding: \"utf8\" });\n    if (res.includes(\"�\")) {\n        throw new Error(`Bad character in ${path}`);\n    }\n    return res;\n}\n\nexport function readFileSync(path: string): string {\n    const res = readFileWithEncodingSync(path, { encoding: \"utf8\" });\n    if (res.includes(\"�\")) {\n        throw new Error(`Bad character in ${path}`);\n    }\n    return res;\n}\n\nexport function readJsonSync(path: string): object {\n    return parseJson(readFileSync(path));\n}\n\nexport async function readJson(path: string): Promise<object> {\n    return parseJson(await readFile(path));\n}\n\nexport function writeFile(path: string, content: string): Promise<void> {\n    return writeFileWithEncoding(path, content, { encoding: \"utf8\" });\n}\n\nexport function writeJson(path: string, content: unknown, formatted = true): Promise<void> {\n    return writeJsonRaw(path, content, { spaces: formatted ? 4 : 0 });\n}\n\nexport function streamOfString(text: string): NodeJS.ReadableStream {\n    const s = new ReadableStream();\n    s.push(text);\n    s.push(null); // tslint:disable-line no-null-keyword\n    return s;\n}\n\nexport function stringOfStream(stream: NodeJS.ReadableStream, description: string): Promise<string> {\n    const decoder = new StringDecoder(\"utf8\");\n    let body = \"\";\n    stream.on(\"data\", (data: Buffer) => {\n        body += decoder.write(data);\n    });\n    return new Promise<string>((resolve, reject) => {\n        stream.on(\"error\", reject);\n        stream.on(\"end\", () => {\n            body += decoder.end();\n            if (body.includes(\"�\")) {\n                reject(`Bad character decode in ${description}`);\n            } else {\n                resolve(body);\n            }\n        });\n    });\n}\n\nexport function streamDone(stream: NodeJS.WritableStream): Promise<void> {\n    return new Promise<void>((resolve, reject) => {\n        stream.on(\"error\", reject).on(\"finish\", resolve);\n    });\n}\n\nexport interface FetchOptions {\n    readonly hostname: string;\n    readonly port?: number;\n    readonly path: string;\n    readonly retries?: boolean | number;\n    readonly body?: string;\n    readonly method?: \"GET\" | \"PATCH\" | \"POST\";\n    readonly headers?: {};\n}\nexport class Fetcher {\n    private readonly agent = new Agent({ keepAlive: true });\n\n    async fetchJson(options: FetchOptions): Promise<unknown> {\n        const text = await this.fetch(options);\n        try {\n            return JSON.parse(text) as unknown;\n        } catch (e) {\n            throw new Error(`Bad response from server:\\noptions: ${JSON.stringify(options)}\\n\\n${text}`);\n        }\n    }\n\n    async fetch(options: FetchOptions): Promise<string> {\n        const maxRetries = options.retries === false || options.retries === undefined ? 0 : options.retries === true ? 10 : options.retries;\n        for (let retries = maxRetries; retries > 1; retries--) {\n            try {\n                return await doRequest(options, request, this.agent);\n            } catch (err) {\n                if (!/EAI_AGAIN|ETIMEDOUT|ECONNRESET/.test((err as Error).message)) {\n                    throw err;\n                }\n            }\n            await sleep(1);\n        }\n        return doRequest(options, request, this.agent);\n    }\n}\n\n/** Only used for testing. */\nexport function makeHttpRequest(options: FetchOptions): Promise<string> {\n    return doRequest(options, httpRequest);\n}\n\nfunction doRequest(options: FetchOptions, makeRequest: typeof request, agent?: Agent): Promise<string> {\n    return new Promise((resolve, reject) => {\n        const req = makeRequest(\n            {\n                hostname: options.hostname,\n                port: options.port,\n                path: `/${options.path}`,\n                agent,\n                method: options.method || \"GET\",\n                headers: options.headers,\n            },\n            res => {\n                let text = \"\";\n                res.on(\"data\", (d: string) => { text += d; });\n                res.on(\"error\", reject);\n                res.on(\"end\", () => { resolve(text); });\n            });\n        if (options.body !== undefined) {\n            req.write(options.body);\n        }\n        req.end();\n    });\n}\n\nexport async function sleep(seconds: number): Promise<void> {\n    return new Promise<void>(resolve => setTimeout(resolve, seconds * 1000));\n}\n\nexport async function isDirectory(path: string): Promise<boolean> {\n    return (await stat(path)).isDirectory();\n}\n\nexport const npmInstallFlags = \"--ignore-scripts --no-shrinkwrap --no-package-lock --no-bin-links --no-save\";\n"
  },
  {
    "path": "src/util/logging.ts",
    "content": "import { ensureDir } from \"fs-extra\";\n\nimport { logDir } from \"../lib/settings\";\nimport { joinPaths } from \"../util/util\";\n\nimport { writeFile } from \"./io\";\n\n/** Anything capable of receiving messages is a logger. */\nexport type Logger = (message: string) => void;\n\n/** Recording of every message sent to a Logger. */\nexport type Log = string[];\n\n/** Stores two separate loggers. */\nexport interface LoggerWithErrors {\n    info: Logger;\n    error: Logger;\n}\n\n/** Recording of every message sent to a LoggerWithErrors. */\nexport interface LogWithErrors {\n    infos: Log;\n    errors: Log;\n}\n\n/** Logger that *just* outputs to the console and does not save anything. */\nexport const consoleLogger: LoggerWithErrors = {\n    info: console.log, // tslint:disable-line no-unbound-method\n    error: console.error, // tslint:disable-line no-unbound-method\n};\n\n/** Logger that *just* records writes and does not output to console. */\nexport function quietLogger(): [Logger, () => Log] {\n    const logged: Log = [];\n    return [ (message: string) => logged.push(message), () => logged ];\n}\n\n/** Performs a side-effect and also records all logs. */\nfunction alsoConsoleLogger(consoleLog: Logger): [Logger, () => Log] {\n    const [log, logResult] = quietLogger();\n    return [\n        (message: string) => {\n            consoleLog(message);\n            log(message);\n        },\n        logResult,\n    ];\n}\n\n/** Logger that writes to console in addition to recording a result. */\nexport function logger(): [Logger, () => Log]  {\n    return alsoConsoleLogger(consoleLogger.info);\n}\n\n/** Helper for creating `info` and `error` loggers together. */\nfunction loggerWithErrorsHelper(loggerOrQuietLogger: () => [Logger, () => Log]): [LoggerWithErrors, () => LogWithErrors] {\n    const [info, infoResult] = loggerOrQuietLogger();\n    const [error, errorResult] = loggerOrQuietLogger();\n    return [\n        { info, error },\n        () => ({ infos: infoResult(), errors: errorResult() }),\n    ];\n}\n\n/** Records `info` and `error` messages without writing to console. */\nexport function quietLoggerWithErrors(): [LoggerWithErrors, () => LogWithErrors] {\n    return loggerWithErrorsHelper(quietLogger);\n}\n\n/** Records `info` and `error` messages, calling appropriate console methods as well. */\nexport function loggerWithErrors(): [LoggerWithErrors, () => LogWithErrors] {\n    return loggerWithErrorsHelper(logger);\n}\n\n/**\n * Move everything from one Log to another logger.\n * This is useful for performing several tasks in parallel, but outputting their logs in sequence.\n */\nexport function moveLogs(dest: Logger, src: Log, mapper?: (message: string) => string): void {\n    for (const line of src) {\n        dest(mapper ? mapper(line) : line);\n    }\n}\n\n/** Perform `moveLogs` for both parts of a LogWithErrors. */\nexport function moveLogsWithErrors(dest: LoggerWithErrors, {infos, errors}: LogWithErrors, mapper?: (message: string) => string): void {\n    moveLogs(dest.info, infos, mapper);\n    moveLogs(dest.error, errors, mapper);\n}\n\nexport function logPath(logName: string): string {\n    return joinPaths(logDir, logName);\n}\n\nexport async function writeLog(logName: string, contents: ReadonlyArray<string>): Promise<void> {\n    await ensureDir(logDir);\n    await writeFile(logPath(logName), contents.join(\"\\r\\n\"));\n}\n\nexport function joinLogWithErrors({infos, errors}: LogWithErrors): Log {\n    return errors.length ? infos.concat([\"\", \"=== ERRORS ===\", \"\"], errors) : infos;\n}\n"
  },
  {
    "path": "src/util/progress.ts",
    "content": "import charm = require(\"charm\");\n\nexport interface Options {\n    /** Text to display in front of the progress bar. */\n    name: string;\n    /** Length of the progress bar. */\n    width?: number;\n    /** Only render an update if this many milliseconds have passed. */\n    updateMinTime?: number;\n}\n\nexport default class ProgressBar {\n    private readonly console = new UpdatableConsole();\n\n    private readonly name: string;\n    private readonly width: number;\n    private readonly updateMinTime: number;\n\n    /** Most recent flavor text. */\n    private flavor = \"\";\n    private lastUpdateMillis = 0;\n\n    constructor(options: Options) {\n        this.name = options.name;\n        this.width = options.width === undefined ? 20 : options.width;\n        this.updateMinTime = options.updateMinTime === undefined ? 250 : options.updateMinTime;\n    }\n\n    update(current: number, flavor?: string): void {\n        if (flavor !== undefined) {\n            this.flavor = flavor;\n        }\n        const now = +(new Date());\n        const diff = now - this.lastUpdateMillis;\n        if (diff > this.updateMinTime) {\n            this.lastUpdateMillis = now;\n            this.doUpdate(current);\n        }\n    }\n\n    private doUpdate(current: number): void {\n        const nCellsFilled = Math.ceil(this.width * Math.min(1, Math.max(0, current)));\n        this.console.update(c => {\n            c.write(this.name);\n            c.write(\" [\");\n            c.write(\"█\".repeat(nCellsFilled));\n            if (nCellsFilled < this.width) {\n                c.right(this.width - nCellsFilled);\n            }\n            c.write(\"]\");\n            if (this.flavor.length) {\n                c.write(` ${this.flavor}`);\n            }\n        });\n    }\n\n    done(): void {\n        this.flavor = \"Done!\";\n        this.doUpdate(1);\n        this.console.end();\n    }\n}\n\n/** A mutable line of text on the console. */\nclass UpdatableConsole {\n    private readonly charm = charm(process.stdout);\n\n    update(action: (charm: charm.CharmInstance) => void): void {\n        this.charm.push();\n        this.charm.erase(\"line\");\n        action(this.charm);\n        this.charm.pop();\n    }\n\n    end(): void {\n        this.charm.write(\"\\n\");\n        this.charm.end();\n    }\n}\n\nconst firstLetter = \"a\".charCodeAt(0);\nconst lastLetter = \"z\".charCodeAt(0);\nconst charWidth = lastLetter - firstLetter;\nconst strProgressTotal = charWidth * charWidth; // 2 characters\n\n/** Tracks a string's progress through the alphabet. */\nexport function strProgress(str: string): number {\n    const x = charProgress(str.charCodeAt(0)) * charWidth + charProgress(str.charCodeAt(1));\n    return x / strProgressTotal;\n\n    function charProgress(ch: number): number {\n        if (Number.isNaN(ch) || ch <= firstLetter) {\n            return 0;\n        }\n        if (ch >= lastLetter) {\n            return charWidth;\n        }\n        return ch - firstLetter;\n    }\n}\n"
  },
  {
    "path": "src/util/test.ts",
    "content": "import { License, PackageId, TypingsVersionsRaw } from \"../lib/packages\";\n\nexport function testo(o: { [s: string]: () => void }) {\n    for (const k of Object.keys(o)) {\n        test(k, o[k], 100_000);\n    }\n}\n\nexport function createTypingsVersionRaw(\n    name: string, dependencies: PackageId[], testDependencies: string[],\n): TypingsVersionsRaw {\n    return {\n        \"1.0.0\": {\n            libraryName: name,\n            typingsPackageName: name,\n            dependencies,\n            testDependencies,\n            files: [\"index.d.ts\"],\n            libraryMajorVersion: 1,\n            libraryMinorVersion: 0,\n            pathMappings: [],\n            contributors: [{ name: \"Bender\", url: \"futurama.com\", githubUsername: \"bender\" }],\n            minTsVersion: \"2.3\",\n            typesVersions: [],\n            license: License.MIT,\n            packageJsonDependencies: [],\n            contentHash: \"11111111111111\",\n            projectName: \"zombo.com\",\n            globals: [],\n            declaredModules: [],\n        },\n    };\n}\n"
  },
  {
    "path": "src/util/tgz.ts",
    "content": "import { createWriteStream } from \"fs\";\nimport { FStreamEntry, Reader } from \"fstream\";\nimport { Pack } from \"tar\";\nimport * as zlib from \"zlib\";\n\nimport { streamDone } from \"./io\";\n\nexport function gzip(input: NodeJS.ReadableStream): NodeJS.ReadableStream {\n    return input.pipe(zlib.createGzip());\n}\n\nexport function unGzip(input: NodeJS.ReadableStream): NodeJS.ReadableStream {\n    const output = zlib.createGunzip();\n    input.pipe(output);\n    return output;\n}\n\nexport function writeTgz(inputDirectory: string, outFileName: string): Promise<void> {\n    return new Promise<void>((resolve, reject) => {\n        resolve(streamDone(createTgz(inputDirectory, reject).pipe(createWriteStream(outFileName))));\n    });\n}\n\n// To output this for testing: Export it and:\n// `require(\"./bin/lib/npm-client\").createTgz(\"./output/foo\", err => { throw err }).pipe(fs.createWriteStream(\"foo.tgz\"))`\nexport function createTgz(dir: string, onError: (error: Error) => void): NodeJS.ReadableStream {\n    return gzip(createTar(dir, onError));\n}\n\nfunction createTar(dir: string, onError: (error: Error) => void): NodeJS.ReadableStream {\n    const packer = Pack({ noProprietary: true })\n        .on(\"error\", onError);\n\n    return Reader({ path: dir, type: \"Directory\", filter: addDirectoryExecutablePermission })\n        .on(\"error\", onError)\n        .pipe(packer);\n}\n\n/**\n * Work around a bug where directories bundled on Windows do not have executable permission when extracted on Linux.\n * https://github.com/npm/node-tar/issues/7#issuecomment-17572926\n */\nfunction addDirectoryExecutablePermission(entry: FStreamEntry): boolean {\n    if (entry.props.type === \"Directory\") {\n        entry.props.mode = addExecutePermissionsFromReadPermissions(entry.props.mode);\n    }\n    return true;\n}\n\nfunction addExecutePermissionsFromReadPermissions(mode: number): number {\n    // Constant that gives execute permissions to owner, group, and others. \"+x\"\n    const allExecutePermissions = 0o111;\n    // Moves the bits for read permissions into the place for execute permissions.\n    // In other words, a component will have execute permissions if it has read permissions.\n    const readPermissionsAsExecutePermissions = (mode >>> 2) & allExecutePermissions; // tslint:disable-line no-bitwise\n    // Add these additional execute permissions to the mode.\n    return mode | readPermissionsAsExecutePermissions; // tslint:disable-line no-bitwise\n}\n"
  },
  {
    "path": "src/util/util.ts",
    "content": "import assert = require(\"assert\");\nimport { ChildProcess, exec as node_exec, fork } from \"child_process\";\nimport * as crypto from \"crypto\";\nimport moment = require(\"moment\");\nimport * as os from \"os\";\n\nimport { Options } from \"../lib/common\";\n\nimport ProgressBar from \"./progress\";\n\nexport function assertDefined<T>(x: T | undefined, message?: string | Error | undefined): T {\n    assert(x !== undefined, message);\n    return x!;\n}\n\nconst DEFAULT_CRASH_RECOVERY_MAX_OLD_SPACE_SIZE = 4096;\n\nexport function parseJson(text: string): object {\n    try {\n        return JSON.parse(text) as object;\n    } catch (err) {\n        throw new Error(`${(err as Error).message} due to JSON: ${text}`);\n    }\n}\n\nexport function currentTimeStamp(): string {\n    return moment().format(\"YYYY-MM-DDTHH:mm:ss.SSSZZ\");\n}\n\nexport const numberOfOsProcesses = process.env.TRAVIS === \"true\" ? 2 : os.cpus().length;\n\n/** Progress options needed for `nAtATime`. Other options will be inferred. */\ninterface ProgressOptions<T, U> {\n    readonly name: string;\n    flavor(input: T, output: U): string | undefined;\n    readonly options: Options;\n}\n\nexport async function nAtATime<T, U>(\n    n: number,\n    inputs: ReadonlyArray<T>,\n    use: (t: T) => Awaitable<U>,\n    progressOptions?: ProgressOptions<T, U>): Promise<U[]> {\n    const progress = progressOptions && progressOptions.options.progress ? new ProgressBar({ name: progressOptions.name }) : undefined;\n\n    const results = new Array(inputs.length);\n    // We have n \"threads\" which each run `continuouslyWork`.\n    // They all share `nextIndex`, so each work item is done only once.\n    let nextIndex = 0;\n    await Promise.all(initArray(n, async () => {\n        while (nextIndex !== inputs.length) {\n            const index = nextIndex;\n            nextIndex++;\n            const input = inputs[index];\n            const output = await use(input);\n            results[index] = output;\n            if (progress) {\n                progress!.update(index / inputs.length, progressOptions!.flavor(input, output));\n            }\n        }\n    }));\n    if (progress) {\n        progress.done();\n    }\n    return results;\n}\n\nexport function filter<T>(iterable: Iterable<T>, predicate: (value: T) => boolean): IterableIterator<T> {\n    const iter = iterable[Symbol.iterator]();\n    return {\n        [Symbol.iterator](): IterableIterator<T> { return this; },\n        next(): IteratorResult<T> {\n            while (true) {\n                const res = iter.next();\n                if (res.done || predicate(res.value)) {\n                    return res;\n                }\n            }\n        },\n    };\n}\n\nexport type Awaitable<T> = T | Promise<T>;\n\nexport async function filterNAtATimeOrdered<T>(\n    n: number, inputs: ReadonlyArray<T>, shouldKeep: (input: T) => Awaitable<boolean>, progress?: ProgressOptions<T, boolean>): Promise<T[]> {\n    const shouldKeeps: boolean[] = await nAtATime(n, inputs, shouldKeep, progress);\n    return inputs.filter((_, idx) => shouldKeeps[idx]);\n}\n\nexport function unique<T>(arr: Iterable<T>): T[] {\n    return [...new Set(arr)];\n}\n\nexport function logUncaughtErrors(promise: Promise<unknown> | (() => Promise<unknown>)): void {\n    (typeof promise === \"function\" ? promise() : promise).catch(error => {\n        console.error(error);\n        process.exit(1);\n    });\n}\n\nfunction initArray<T>(length: number, makeElement: (i: number) => T): T[] {\n    const arr = new Array(length);\n    for (let i = 0; i < length; i++) {\n        arr[i] = makeElement(i);\n    }\n    return arr;\n}\n\n/** Always use \"/\" for consistency. (This affects package content hash.) */\nexport function joinPaths(...paths: string[]): string {\n    return paths.join(\"/\");\n}\n\n/** Convert a path to use \"/\" instead of \"\\\\\" for consistency. (This affects content hash.) */\nexport function normalizeSlashes(path: string): string {\n    return path.replace(/\\\\/g, \"/\");\n}\n\nexport function hasWindowsSlashes(path: string): boolean {\n    return path.includes(\"\\\\\");\n}\n\nexport function intOfString(str: string): number {\n    const n = Number.parseInt(str, 10);\n    if (Number.isNaN(n)) {\n        throw new Error(`Error in parseInt(${JSON.stringify(str)})`);\n    }\n    return n;\n}\n\nexport function sortObjectKeys<T extends { [key: string]: unknown }>(data: T): T {\n    const out = {} as T; // tslint:disable-line no-object-literal-type-assertion\n    for (const key of Object.keys(data).sort()) {\n        out[key as keyof T] = data[key as keyof T];\n    }\n    return out;\n}\n\n/** Run a command and return the error, stdout, and stderr. (Never throws.) */\nexport function exec(cmd: string, cwd?: string): Promise<{ error: Error | undefined, stdout: string, stderr: string }> {\n    return new Promise<{ error: Error | undefined, stdout: string, stderr: string }>(resolve => {\n        // Fix \"stdout maxBuffer exceeded\" error\n        // See https://github.com/DefinitelyTyped/DefinitelyTyped/pull/26545#issuecomment-402274021\n        const maxBuffer = 1024 * 1024 * 1; // Max = 1 MiB, default is 200 KiB\n\n        node_exec(cmd, { encoding: \"utf8\", cwd, maxBuffer }, (error, stdout, stderr) => {\n            resolve({ error: error === null ? undefined : error, stdout: stdout.trim(), stderr: stderr.trim() });\n        });\n    });\n}\n\n/** Run a command and return the stdout, or if there was an error, throw. */\nexport async function execAndThrowErrors(cmd: string, cwd?: string): Promise<string> {\n    const { error, stdout, stderr } = await exec(cmd, cwd);\n    if (error) {\n        throw new Error(`${error.stack}\\n${stderr}`);\n    }\n    return stdout + stderr;\n}\n\n/**\n * Returns the input that is better than all others, or `undefined` if there are no inputs.\n * @param isBetter Returns true if `a` should be preferred over `b`.\n */\nexport function best<T>(inputs: Iterable<T>, isBetter: (a: T, b: T) => boolean): T | undefined {\n    const iter = inputs[Symbol.iterator]();\n\n    const first = iter.next();\n    if (first.done) {\n        return undefined;\n    }\n\n    let res = first.value;\n    while (true) {\n        const { value, done } = iter.next();\n        if (done) {\n            break;\n        }\n        if (isBetter(value, res)) {\n            res = value;\n        }\n    }\n    return res;\n}\n\nexport function computeHash(content: string): string {\n    // Normalize line endings\n    const normalContent = content.replace(/\\r\\n?/g, \"\\n\");\n\n    const h = crypto.createHash(\"sha256\");\n    h.update(normalContent, \"utf8\");\n    return h.digest(\"hex\");\n}\n\nexport function mapValues<K, V1, V2>(map: Map<K, V1>, valueMapper: (value: V1) => V2): Map<K, V2> {\n    const out = new Map<K, V2>();\n    map.forEach((value, key) => {\n        out.set(key, valueMapper(value));\n    });\n    return out;\n}\n\nexport function mapDefined<T, U>(arr: Iterable<T>, mapper: (t: T) => U | undefined): U[] {\n    const out = [];\n    for (const a of arr) {\n        const res = mapper(a);\n        if (res !== undefined) {\n            out.push(res);\n        }\n    }\n    return out;\n}\n\nexport async function mapDefinedAsync<T, U>(arr: Iterable<T>, mapper: (t: T) => Promise<U | undefined>): Promise<U[]> {\n    const out = [];\n    for (const a of arr) {\n        const res = await mapper(a);\n        if (res !== undefined) {\n            out.push(res);\n        }\n    }\n    return out;\n}\n\nexport function* mapIter<T, U>(inputs: Iterable<T>, mapper: (t: T) => U): Iterable<U> {\n    for (const input of inputs) {\n        yield mapper(input);\n    }\n}\n\nexport function* flatMap<T, U>(inputs: Iterable<T>, mapper: (t: T) => Iterable<U>): Iterable<U> {\n    for (const input of inputs) {\n        yield* mapper(input);\n    }\n}\n\nexport function sort<T>(values: Iterable<T>, comparer?: (a: T, b: T) => number): T[] {\n    return Array.from(values).sort(comparer);\n}\n\nexport function join<T>(values: Iterable<T>, joiner = \", \"): string {\n    let s = \"\";\n    for (const v of values) {\n        // tslint:disable-next-line strict-string-expressions\n        s += `${v}${joiner}`;\n    }\n    return s.slice(0, s.length - joiner.length);\n}\n\nexport interface RunWithChildProcessesOptions<In> {\n    readonly inputs: ReadonlyArray<In>;\n    readonly commandLineArgs: string[];\n    readonly workerFile: string;\n    readonly nProcesses: number;\n    handleOutput(output: unknown): void;\n}\nexport function runWithChildProcesses<In>(\n    { inputs, commandLineArgs, workerFile, nProcesses, handleOutput }: RunWithChildProcessesOptions<In>,\n): Promise<void> {\n    return new Promise((resolve, reject) => {\n        const nPerProcess = Math.floor(inputs.length / nProcesses);\n        let processesLeft = nProcesses;\n        let rejected = false;\n        const allChildren: ChildProcess[] = [];\n        for (let i = 0; i < nProcesses; i++) {\n            const lo = nPerProcess * i;\n            const hi = i === nProcesses - 1 ? inputs.length : lo + nPerProcess;\n            let outputsLeft = hi - lo; // Expect one output per input\n            if (outputsLeft === 0) {\n                // No work for this process to do, so don't launch it\n                processesLeft--;\n                continue;\n            }\n            const child = fork(workerFile, commandLineArgs);\n            allChildren.push(child);\n            child.send(inputs.slice(lo, hi));\n            child.on(\"message\", outputMessage => {\n                handleOutput(outputMessage as unknown);\n                assert(outputsLeft > 0);\n                outputsLeft--;\n                if (outputsLeft === 0) {\n                    assert(processesLeft > 0);\n                    processesLeft--;\n                    if (processesLeft === 0) {\n                        resolve();\n                    }\n                    child.kill();\n                }\n            });\n            child.on(\"disconnect\", () => {\n                if (outputsLeft !== 0) {\n                    fail();\n                }\n            });\n            child.on(\"close\", () => { assert(rejected || outputsLeft === 0); });\n            child.on(\"error\", fail);\n        }\n\n        function fail(): void {\n            rejected = true;\n            for (const child of allChildren) {\n                child.kill();\n            }\n            reject(new Error(\"Parsing failed.\"));\n        }\n    });\n}\n\nexport const enum CrashRecoveryState {\n    Normal,\n    Retry,\n    RetryWithMoreMemory,\n    Crashed,\n}\n\ninterface RunWithListeningChildProcessesOptions<In> {\n    readonly inputs: ReadonlyArray<In>;\n    readonly commandLineArgs: string[];\n    readonly workerFile: string;\n    readonly nProcesses: number;\n    readonly cwd: string;\n    readonly crashRecovery?: boolean;\n    readonly crashRecoveryMaxOldSpaceSize?: number;\n    readonly softTimeoutMs?: number;\n    handleOutput(output: unknown, processIndex: number | undefined): void;\n    handleStart?(input: In, processIndex: number | undefined): void;\n    handleCrash?(input: In, state: CrashRecoveryState, processIndex: number | undefined): void;\n}\nexport function runWithListeningChildProcesses<In>(\n    { inputs, commandLineArgs, workerFile, nProcesses, cwd, handleOutput, crashRecovery,\n      crashRecoveryMaxOldSpaceSize = DEFAULT_CRASH_RECOVERY_MAX_OLD_SPACE_SIZE,\n      handleStart, handleCrash, softTimeoutMs = Infinity }: RunWithListeningChildProcessesOptions<In>,\n): Promise<void> {\n    return new Promise((resolve, reject) => {\n        let inputIndex = 0;\n        let processesLeft = nProcesses;\n        let rejected = false;\n        const runningChildren = new Set<ChildProcess>();\n        const maxOldSpaceSize = getMaxOldSpaceSize(process.execArgv) || 0;\n        const startTime = Date.now();\n        for (let i = 0; i < nProcesses; i++) {\n            if (inputIndex === inputs.length) {\n                processesLeft--;\n                continue;\n            }\n\n            const processIndex = nProcesses > 1 ? i + 1 : undefined;\n            let child: ChildProcess;\n            let crashRecoveryState = CrashRecoveryState.Normal;\n            let currentInput: In;\n\n            const onMessage = (outputMessage: unknown) => {\n                try {\n                    const oldCrashRecoveryState = crashRecoveryState;\n                    crashRecoveryState = CrashRecoveryState.Normal;\n                    handleOutput(outputMessage as {}, processIndex);\n                    if (inputIndex === inputs.length || Date.now() - startTime > softTimeoutMs) {\n                        stopChild(/*done*/ true);\n                    } else {\n                        if (oldCrashRecoveryState !== CrashRecoveryState.Normal) {\n                            // retry attempt succeeded, restart the child for further tests.\n                            console.log(`${processIndex}> Restarting...`);\n                            restartChild(nextTask, process.execArgv);\n                        } else {\n                            nextTask();\n                        }\n                    }\n                } catch (e) {\n                    onError(e);\n                }\n            };\n\n            const onClose = () => {\n                if (rejected || !runningChildren.has(child)) {\n                    return;\n                }\n\n                try {\n                    // treat any unhandled closures of the child as a crash\n                    if (crashRecovery) {\n                        switch (crashRecoveryState) {\n                            case CrashRecoveryState.Normal:\n                                crashRecoveryState = CrashRecoveryState.Retry;\n                                break;\n                            case CrashRecoveryState.Retry:\n                                // skip crash recovery if we're already passing a value for --max_old_space_size that\n                                // is >= crashRecoveryMaxOldSpaceSize\n                                crashRecoveryState = maxOldSpaceSize < crashRecoveryMaxOldSpaceSize\n                                    ? CrashRecoveryState.RetryWithMoreMemory\n                                    : crashRecoveryState = CrashRecoveryState.Crashed;\n                                break;\n                            default:\n                                crashRecoveryState = CrashRecoveryState.Crashed;\n                        }\n                    } else {\n                        crashRecoveryState = CrashRecoveryState.Crashed;\n                    }\n\n                    if (handleCrash) {\n                        handleCrash(currentInput, crashRecoveryState, processIndex);\n                    }\n\n                    switch (crashRecoveryState) {\n                        case CrashRecoveryState.Retry:\n                            restartChild(resumeTask, process.execArgv);\n                            break;\n                        case CrashRecoveryState.RetryWithMoreMemory:\n                            restartChild(resumeTask, [\n                                ...getExecArgvWithoutMaxOldSpaceSize(),\n                                `--max_old_space_size=${crashRecoveryMaxOldSpaceSize}`,\n                            ]);\n                            break;\n                        case CrashRecoveryState.Crashed:\n                            crashRecoveryState = CrashRecoveryState.Normal;\n                            if (inputIndex === inputs.length || Date.now() - startTime > softTimeoutMs) {\n                                stopChild(/*done*/ true);\n                            } else {\n                                restartChild(nextTask, process.execArgv);\n                            }\n                            break;\n                        default:\n                            assert.fail(`${processIndex}> Unexpected crashRecoveryState: ${crashRecoveryState}`);\n                    }\n                } catch (e) {\n                    onError(e);\n                }\n            };\n\n            const onError = (err?: Error) => {\n                child.removeAllListeners();\n                runningChildren.delete(child);\n                fail(err);\n            };\n\n            const startChild = (taskAction: () => void, execArgv: string[]) => {\n                try {\n                    child = fork(workerFile, commandLineArgs, { cwd, execArgv });\n                    runningChildren.add(child);\n                } catch (e) {\n                    fail(e);\n                    return;\n                }\n\n                try {\n                    let closed = false;\n                    const thisChild = child;\n                    const onChildClosed = () => {\n                        // Don't invoke `onClose` more than once for a single child.\n                        if (!closed && child === thisChild) {\n                            closed = true;\n                            onClose();\n                        }\n                    };\n                    const onChildDisconnectedOrExited = () => {\n                        if (!closed && thisChild === child) {\n                            // Invoke `onClose` after enough time has elapsed to allow `close` to be triggered.\n                            // This is to ensure our `onClose` logic gets called in some conditions\n                            const timeout = 1000;\n                            setTimeout(onChildClosed, timeout);\n                        }\n                    };\n                    child.on(\"message\", onMessage);\n                    child.on(\"close\", onChildClosed);\n                    child.on(\"disconnect\", onChildDisconnectedOrExited);\n                    child.on(\"exit\", onChildDisconnectedOrExited);\n                    child.on(\"error\", onError);\n                    taskAction();\n                } catch (e) {\n                    onError(e);\n                }\n            };\n\n            const stopChild = (done: boolean) => {\n                try {\n                    assert(runningChildren.has(child), `${processIndex}> Child not running`);\n                    if (done) {\n                        processesLeft--;\n                        if (processesLeft === 0) {\n                            resolve();\n                        }\n                    }\n                    runningChildren.delete(child);\n                    child.removeAllListeners();\n                    child.kill();\n                } catch (e) {\n                    onError(e);\n                }\n            };\n\n            const restartChild = (taskAction: () => void, execArgv: string[]) => {\n                try {\n                    assert(runningChildren.has(child), `${processIndex}> Child not running`);\n                    console.log(`${processIndex}> Restarting...`);\n                    stopChild(/*done*/ false);\n                    startChild(taskAction, execArgv);\n                } catch (e) {\n                    onError(e);\n                }\n            };\n\n            const resumeTask = () => {\n                try {\n                    assert(runningChildren.has(child), `${processIndex}> Child not running`);\n                    child.send(currentInput);\n                } catch (e) {\n                    onError(e);\n                }\n            };\n\n            const nextTask = () => {\n                try {\n                    assert(runningChildren.has(child), `${processIndex}> Child not running`);\n                    currentInput = inputs[inputIndex];\n                    inputIndex++;\n                    if (handleStart) {\n                        handleStart(currentInput, processIndex);\n                    }\n                    child.send(currentInput);\n                } catch (e) {\n                    onError(e);\n                }\n            };\n\n            startChild(nextTask, process.execArgv);\n        }\n\n        function fail(err?: Error): void {\n            if (!rejected) {\n                rejected = true;\n                for (const child of runningChildren) {\n                    try {\n                        child.removeAllListeners();\n                        child.kill();\n                    } catch {\n                        // do nothing\n                    }\n                }\n                const message = err ? `: ${err.message}` : \"\";\n                reject(new Error(`Something went wrong in ${runWithListeningChildProcesses.name}${message}`));\n            }\n        }\n    });\n}\n\nconst maxOldSpaceSizeRegExp = /^--max[-_]old[-_]space[-_]size(?:$|=(\\d+))/;\n\ninterface MaxOldSpaceSizeArgument {\n    index: number;\n    size: number;\n    value: number | undefined;\n}\n\nfunction getMaxOldSpaceSizeArg(argv: ReadonlyArray<string>): MaxOldSpaceSizeArgument | undefined {\n    for (let index = 0; index < argv.length; index++) {\n        const match = maxOldSpaceSizeRegExp.exec(argv[index]);\n        if (match) {\n            const value = match[1] ? parseInt(match[1], 10) :\n                argv[index + 1] ? parseInt(argv[index + 1], 10) :\n                undefined;\n            const size = match[1] ? 1 : 2; // tslint:disable-line:no-magic-numbers\n            return { index, size, value };\n        }\n    }\n    return undefined;\n}\n\nfunction getMaxOldSpaceSize(argv: ReadonlyArray<string>): number | undefined {\n    const arg = getMaxOldSpaceSizeArg(argv);\n    return arg && arg.value;\n}\n\nlet execArgvWithoutMaxOldSpaceSize: ReadonlyArray<string> | undefined;\n\nfunction getExecArgvWithoutMaxOldSpaceSize(): ReadonlyArray<string> {\n    if (!execArgvWithoutMaxOldSpaceSize) {\n        // remove --max_old_space_size from execArgv\n        const execArgv = process.execArgv.slice();\n        let maxOldSpaceSizeArg = getMaxOldSpaceSizeArg(execArgv);\n        while (maxOldSpaceSizeArg) {\n            execArgv.splice(maxOldSpaceSizeArg.index, maxOldSpaceSizeArg.size);\n            maxOldSpaceSizeArg = getMaxOldSpaceSizeArg(execArgv);\n        }\n        execArgvWithoutMaxOldSpaceSize = execArgv;\n    }\n    return execArgvWithoutMaxOldSpaceSize;\n}\n\nexport function assertNever(_: never): never {\n    throw new Error();\n}\n\nexport function recordToMap<T>(record: Record<string, T>): Map<string, T>;\nexport function recordToMap<T, U>(record: Record<string, T>, cb: (t: T) => U): Map<string, U>;\nexport function recordToMap<T, U>(record: Record<string, T>, cb?: (t: T) => U): Map<string, T | U> {\n    const m = new Map<string, T | U>();\n    for (const key of Object.keys(record)) {\n        m.set(key, cb ? cb(record[key]) : record[key]);\n    }\n    return m;\n}\n\nexport function mapToRecord<T>(map: Map<string, T>): Record<string, T>;\nexport function mapToRecord<T, U>(map: Map<string, T>, cb: (t: T) => U): Record<string, U>;\nexport function mapToRecord<T, U>(map: Map<string, T>, cb?: (t: T) => U): Record<string, T | U> {\n    const o: Record<string, T | U> = {};\n    map.forEach((value, key) => { o[key] = cb ? cb(value) : value; });\n    return o;\n}\n\nexport function identity<T>(t: T): T { return t; }\n\nexport function withoutStart(s: string, start: string): string | undefined {\n    return s.startsWith(start) ? s.slice(start.length) : undefined;\n}\n\n// Based on `getPackageNameFromAtTypesDirectory` in TypeScript.\nexport function unmangleScopedPackage(packageName: string): string | undefined {\n    const separator = \"__\";\n    return packageName.includes(separator) ? `@${packageName.replace(separator, \"/\")}` : undefined;\n}\n\n/** Returns [values that cb returned undefined for, defined results of cb]. */\nexport function split<T, U>(inputs: ReadonlyArray<T>, cb: (t: T) => U | undefined): [ReadonlyArray<T>, ReadonlyArray<U>] {\n    const keep: T[] = [];\n    const splitOut: U[] = [];\n    for (const input of inputs) {\n        const res = cb(input);\n        if (res === undefined) { keep.push(input); } else { splitOut.push(res); }\n    }\n    return [keep, splitOut];\n}\n\nexport function assertSorted(a: ReadonlyArray<string>): ReadonlyArray<string>;\nexport function assertSorted<T>(a: ReadonlyArray<T>, cb: (t: T) => string): ReadonlyArray<T>;\nexport function assertSorted<T>(a: ReadonlyArray<T>, cb: (t: T) => string = (t: T) => t as unknown as string): ReadonlyArray<T> {\n    let prev = a[0];\n    for (let i = 1; i < a.length; i++) {\n        const x = a[i];\n        assert(cb(x) >= cb(prev), `${JSON.stringify(x)} >= ${JSON.stringify(prev)}`);\n        prev = x;\n    }\n    return a;\n}\n"
  },
  {
    "path": "src/validate.ts",
    "content": "import { mkdirp, remove } from \"fs-extra\";\nimport * as yargs from \"yargs\";\n\nimport { FS, getDefinitelyTyped } from \"./get-definitely-typed\";\nimport { Options } from \"./lib/common\";\nimport { AllPackages, getFullNpmName } from \"./lib/packages\";\nimport { validateOutputPath } from \"./lib/settings\";\nimport { readChangedPackages } from \"./lib/versions\";\nimport { writeFile, writeJson } from \"./util/io\";\nimport { LoggerWithErrors, loggerWithErrors, moveLogsWithErrors, quietLoggerWithErrors, writeLog } from \"./util/logging\";\nimport { exec, joinPaths, logUncaughtErrors, nAtATime } from \"./util/util\";\n\nif (!module.parent) {\n    const all = !!yargs.argv.all;\n    const packageNames = yargs.argv._;\n    if (all && packageNames.length) {\n        throw new Error(\"Can't combine --all with listed package names.\");\n    }\n\n    if (all) {\n        console.log(\"Validating all packages\");\n        logUncaughtErrors(doAll());\n    } else if (packageNames.length) {\n        console.log(`Validating: ${JSON.stringify(packageNames)}`);\n        logUncaughtErrors(doValidate(packageNames));\n    } else {\n        const log = loggerWithErrors()[0];\n        logUncaughtErrors(getDefinitelyTyped(Options.defaults, log).then(validate));\n    }\n}\n\nexport default async function validate(dt: FS): Promise<void> {\n    await doValidate((await readChangedPackages(await AllPackages.read(dt))).changedTypings.map(c => c.pkg.name));\n}\n\nasync function doAll(): Promise<void> {\n    // todo: validate older versions too\n    const packageNames = (await AllPackages.readTypings()).map(t => t.name).sort();\n    await doValidate(packageNames);\n}\n\nasync function doValidate(packageNames: ReadonlyArray<string>): Promise<void> {\n    const [log, logResult] = loggerWithErrors();\n    await validatePackages(packageNames, validateOutputPath, log);\n    const {infos, errors} = logResult();\n    await Promise.all([\n        writeLog(\"validate.md\", infos),\n        writeLog(\"validate-errors.md\", errors),\n    ]);\n}\n\nasync function validatePackages(packageNames: ReadonlyArray<string>, outPath: string, log: LoggerWithErrors): Promise<void> {\n    log.info(\"\");\n    log.info(`Using output path: ${outPath}`);\n    log.info(\"Running tests....\");\n    log.info(\"\");\n    const failed: string[] = [];\n    const passed: string[] = [];\n    try {\n        await remove(outPath);\n        await mkdirp(outPath);\n    } catch (e) {\n        log.error(`Could not recreate output directory. ${e}`);\n        return;\n    }\n\n    // Run the tests\n    await nAtATime(25, packageNames, async packageName => {\n        if (await validatePackage(packageName, outPath, log)) {\n            passed.push(packageName);\n        } else {\n            failed.push(packageName);\n        }\n    });\n\n    // Write results\n    log.info(\"\");\n    log.info(\"\");\n    log.info(`Total  ${packageNames.length}`);\n    log.info(`Passed ${passed.length}`);\n    log.info(`Failed ${failed.length}`);\n    log.info(\"\");\n\n    if (failed.length) {\n        log.info(`These packages failed: ${failed.toString()}`);\n    }\n}\n\nasync function validatePackage(packageName: string, outputDirecory: string, mainLog: LoggerWithErrors): Promise<boolean> {\n    const [log, logResult] = quietLoggerWithErrors();\n    let passed = false;\n    try {\n        const packageDirectory = joinPaths(outputDirecory, packageName);\n        log.info(\"\");\n        log.info(`Processing \\`${packageName}\\`...`);\n        await mkdirp(packageDirectory);\n        await writePackage(packageDirectory, packageName);\n        if (await runCommand(\"npm\", log, packageDirectory, \"../../node_modules/npm/bin/npm-cli.js\", \"install\") &&\n            await runCommand(\"tsc\", log, packageDirectory, \"../../node_modules/typescript/lib/tsc.js\")) {\n            await remove(packageDirectory);\n            log.info(\"Passed.\");\n            passed = true;\n        }\n    } catch (e) {\n        log.info(`Error: ${e}`);\n        log.info(\"Failed!\");\n    }\n\n    // Write the log as one entry to the main log\n    moveLogsWithErrors(mainLog, logResult());\n\n    console.info(`${packageName} -- ${passed ? \"Passed\" : \"Failed\"}.`);\n    return passed;\n}\n\nasync function writePackage(packageDirectory: string, packageName: string): Promise<void> {\n    // Write package.json\n    await writeJson(joinPaths(packageDirectory, \"package.json\"), {\n        name: `${packageName}_test`,\n        version: \"1.0.0\",\n        description: \"test\",\n        author: \"\",\n        license: \"ISC\",\n        repository: \"https://github.com/Microsoft/types-publisher\",\n        dependencies: { [getFullNpmName(packageName)]: \"latest\" },\n    });\n\n    // Write tsconfig.json\n    await writeJson(joinPaths(packageDirectory, \"tsconfig.json\"), {\n        compilerOptions: {\n            module: \"commonjs\",\n            target: \"es5\",\n            noImplicitAny: false,\n            strictNullChecks: false,\n            noEmit: true,\n            lib: [\"es5\", \"es2015.promise\", \"dom\"],\n        },\n    });\n\n    // Write index.ts\n    await writeFile(joinPaths(packageDirectory, \"index.ts\"), `/// <reference types=\"${packageName}\" />\\r\\n`);\n}\n\n// Returns whether the command succeeded.\nasync function runCommand(commandDescription: string, log: LoggerWithErrors, directory: string, cmd: string, ...args: string[]): Promise<boolean> {\n    const nodeCmd = `node ${cmd} ${args.join(\" \")}`;\n    log.info(`Run ${nodeCmd}`);\n    const { error, stdout, stderr } = await exec(nodeCmd, directory);\n    if (error) {\n        log.error(stderr);\n        log.info(stdout);\n        log.error(`${commandDescription} failed: ${JSON.stringify(error)}`);\n        log.info(`${commandDescription} failed, refer to error log`);\n        return false;\n    }\n    log.info(stdout);\n    return true;\n}\n"
  },
  {
    "path": "src/webhook.ts",
    "content": "import appInsights = require(\"applicationinsights\");\nimport * as yargs from \"yargs\";\n\nimport { Options } from \"./lib/common\";\nimport { getSecret, Secret } from \"./lib/secrets\";\nimport webhookServer from \"./lib/webhook-server\";\nimport { Fetcher } from \"./util/io\";\nimport { logUncaughtErrors } from \"./util/util\";\n\nif (!module.parent) {\n    logUncaughtErrors(main());\n}\n\nexport default async function main(): Promise<void> {\n    const key = await getSecret(Secret.GITHUB_SECRET);\n    const githubAccessToken = await getSecret(Secret.GITHUB_ACCESS_TOKEN);\n    const dry = !!(yargs.argv.dry || process.env.WEBHOOK_FORCE_DRY);\n    const port = process.env.PORT;\n\n    if (!(key && githubAccessToken && port)) {\n        console.log(\"The environment variables GITHUB_SECRET and GITHUB_ACCESS_TOKEN and PORT must be set.\");\n    } else {\n        console.log(`=== ${dry ? \"DRY\" : \"PRODUCTION\"} RUN ===`);\n        if (process.env.APPINSIGHTS_INSTRUMENTATIONKEY) {\n            appInsights.setup().start();\n            console.log(\"Done initialising App Insights\");\n        }\n        const fetcher = new Fetcher();\n        try {\n            const s = await webhookServer(key, githubAccessToken, dry, fetcher, Options.azure);\n            console.log(`Listening on port ${port}`);\n            s.listen(port);\n        } catch (e) {\n            appInsights.defaultClient.trackEvent({\n                name: \"crash\",\n                properties: {\n                    error: e.toString(),\n                },\n            });\n            throw e;\n        }\n    }\n}\n"
  },
  {
    "path": "tsconfig.json",
    "content": "{\n    \"include\": [\"src\"],\n    \"compilerOptions\": {\n        \"module\": \"commonjs\",\n        \"strict\": true,\n        \"declaration\": true,\n        \"noImplicitReturns\": true,\n        \"noFallthroughCasesInSwitch\": true,\n        \"noUnusedLocals\": true,\n        \"noUnusedParameters\": true,\n        \"allowUnreachableCode\": false,\n        \"allowUnusedLabels\": false,\n        \"outDir\": \"bin\",\n        \"sourceMap\": true,\n        \"target\": \"es2017\",\n        \"newLine\": \"crlf\",\n\n        \"baseUrl\": \".\",\n        \"paths\": {\n            \"*\": [\"src/types/*\"]\n        }\n    }\n}\n"
  },
  {
    "path": "tslint.json",
    "content": "{\n    \"extends\": \"tslint:all\",\n    \"rules\": {\n        \"arrow-parens\": [true, \"ban-single-arg-parens\"],\n        \"comment-format\": [true, \"check-space\"],\n        \"indent\": [true, \"spaces\"],\n        \"interface-name\": [true, \"never-prefix\"],\n        \"max-line-length\": [true, 150],\n        \"object-literal-key-quotes\": [true, \"as-needed\"],\n        \"only-arrow-functions\": [true, \"allow-declarations\"],\n        \"variable-name\": [true, \"check-format\", \"allow-leading-underscore\"],\n        \"strict-comparisons\": [\n            true,\n            {\n                \"allow-object-equal-comparison\": true,\n                \"allow-string-order-comparison\": true\n            }\n        ],\n\n        // TODO\n        \"strict-boolean-expressions\": false,\n        \"no-unnecessary-type-assertion\": false, // false positive\n\n        \"no-this-assignment\": [true, { \"allow-destructuring\": true }],\n        \"typedef\": false,\n        \"no-any\": false,\n        \"no-inferred-empty-object-type\": false,\n        \"no-unsafe-any\": false,\n        \"prefer-template\": false,\n        \"restrict-plus-operands\": false,\n\n        \"completed-docs\": false,\n        \"file-name-casing\": false,\n        \"forin\": false,\n        \"increment-decrement\": false,\n        \"linebreak-style\": false,\n        \"max-classes-per-file\": false,\n        \"member-access\": false,\n        \"member-ordering\": false,\n        \"newline-before-return\": false,\n        \"newline-per-chained-call\": false,\n        \"no-console\": false,\n        \"no-default-export\": false,\n        \"no-default-import\": false,\n        \"no-unused-variable\": false,\n        \"no-magic-numbers\": false,\n        \"no-namespace\": false,\n        \"no-non-null-assertion\": false,\n        \"no-object-literal-type-assertion\": false,\n        \"no-use-before-declare\": false,\n        \"no-parameter-properties\": false,\n        \"no-parameter-reassignment\": false,\n        \"no-require-imports\": false,\n        \"no-void-expression\": false,\n        \"object-literal-sort-keys\": false,\n        \"promise-function-async\": false,\n        \"type-literal-delimiter\": false\n    }\n}\n"
  }
]