- Code in a `browser/` folder always lives on the browser process, and it can use `window` and other browser items.
- Code in an `electron-main/` folder always lives on the main process, and it can import `node_modules`.
- Code in `common/` can be used by either process, but doesn't get any special imports.
- The browser environment is not allowed to import `node_modules`. We came up with two workarounds:
1. Bundle the raw node_module code to the browser - we're doing this for React.
2. Implement the code on `electron-main/` and set up a channel between main/browser - we're doing this for sendLLMMessage.
### Terminology
Here's some terminology you might want to know about when working inside VSCode:
- An **Editor** is the thing that you type your code in. If you have 10 tabs open, that's just one editor! Editors contain tabs (or "models").
- A **Model** is an internal representation of a file's contents. It's shared between editors (for example, if you press `Cmd+\` to make a new editor, then the model of a file like `A.ts` is shared between them. Two editors, one model. That's how changes sync.).
- Each model has a **URI** it represents, like `/Users/.../my_file.txt`. (A URI or "resource" is generally just a path).
- The **Workbench** is the wrapper that contains all the editors, the terminal, the file system tree, etc.
- Usually you use the `ITextModel` type for models and the `ICodeEditor` type for editors. There aren't that many other types.
- VSCode is organized into "**Services**". A service is just a class that mounts a single time (in computer science theory this is called a "singleton"). You can register services with `registerSingleton` so that you can easily use them in any constructor with `@`. See _dummyContrib for an example we put together on how to register them. The registration is the same every time.
- "**Actions**" are functions you register on VSCode so that either you or the user can call them later. They're also called "**Commands**".
- You can run actions as a user by pressing Cmd+Shift+P (opens the command pallete), or you can run them internally by using the commandService to call them by ID. We use actions to register keybinding listeners like Cmd+L, Cmd+K, etc. The nice thing about actions is the user can change the keybindings.
### Internal LLM Message Pipeline
Here's a picture of all the dependencies that are relevent between the time you first send a message through Void's sidebar, and the time a request is sent to your provider.
Sending LLM messages from the main process avoids CSP issues with local providers and lets us use node_modules more easily.
**Notes:** `modelCapabilities` is an important file that must be updated when new models come out!
### Apply
Void has two types of Apply: **Fast Apply** (uses Search/Replace, see below), and **Slow Apply** (rewrites whole file).
When you click Apply and Fast Apply is enabled, we prompt the LLM to output Search/Replace block(s) like this:
```
<<<<<<< ORIGINAL
// original code goes here
=======
// replaced code goes here
>>>>>>> UPDATED
```
This is what allows Void to quickly apply code even on 1000-line files. It's the same as asking the LLM to press Ctrl+F and enter in a search/replace query.
### Apply Inner Workings
The `editCodeService` file runs Apply. The same exact code is also used when the LLM calls the Edit tool, and when you submit Cmd+K. Just different versions of Fast/Slow Apply mode.
Here is some important terminology:
- A **DiffZone** is a {startLine, endLine} region of text where we compute and show red/green areas, or **Diffs**. When any changes are made to a file, we loop through all the DiffAreas on that file and refresh its Diffs.
- A **DiffArea** is a generalization that just tracks line numbers like a DiffZone.
- The only type of DiffArea that can "stream" is a DiffZone. Each DiffZone has an llmCancelToken if it's streaming.
How Apply works:
- When you click Apply, we create a **DiffZone** over that the full file so that any changes that the LLM makes will show up in red/green. We then stream the change.
- When an LLM calls Edit, it's really calling Apply.
- When you submit Cmd+K, it's the same as Apply except we create a smaller DiffZone (not on the whole file).
### Writing Files Inner Workings
When Void wants to change your code, it just writes to a text model. This means all you need to know to write to a file is its URI - you don't have to load it, save it, etc. There are some annoying background URI/model things to think about to get this to work, but we handled them all in `voidModelService`.
### Void Settings Inner Workings
We have a service `voidSettingsService` that stores all your Void settings (providers, models, global Void settings, etc). Imagine this as an implicit dependency for any of the core Void services:
Here's a guide to some of the terminology we're using:
- **FeatureName**: Autocomplete | Chat | CtrlK | Apply
- **ModelSelection**: a {providerName, modelName} pair.
- **ProviderName**: The name of a provider: `'ollama'`, `'openAI'`, etc.
- **ModelName**: The name of a model (string type, eg `'gpt-4o'`).
- **RefreshProvider**: a provider that we ping repeatedly to update the models list.
- **ChatMode** = normal | gather | agent
### Approval State
`editCodeService`'s data structures contain all the information about changes that the user needs to review. However, they don't store that information in a useful format. We wrote the following service to get a more useful derived state:
### Build process
If you want to know how our build pipeline works, see our build repo [here](https://github.com/voideditor/void-builder).
## VSCode Codebase Guide
For additional references, the Void team put together this list of links to get up and running with VSCode.
#### Links for Beginners
- [VSCode UI guide](https://code.visualstudio.com/docs/getstarted/userinterface) - covers auxbar, panels, etc.
- [UX guide](https://code.visualstudio.com/api/ux-guidelines/overview) - covers Containers, Views, Items, etc.
#### Links for Contributors
- [How VSCode's sourcecode is organized](https://github.com/microsoft/vscode/wiki/Source-Code-Organization) - this explains where the entry point files are, what `browser/` and `common/` mean, etc. This is the most important read on this whole list! We recommend reading the whole thing.
- [Built-in VSCode styles](https://code.visualstudio.com/api/references/theme-color) - CSS variables that are built into VSCode. Use `var(--vscode-{theme but replacing . with -})`. You can also see their [Webview theming guide](https://code.visualstudio.com/api/extension-guides/webview#theming-webview-content).
#### Misc
- [Every command](https://code.visualstudio.com/api/references/commands) built-in to VSCode - not used often, but here for reference.
- Note: VSCode's repo is the source code for the Monaco editor! An "editor" is a Monaco editor, and it shares the code for ITextModel, etc.
#### VSCode's Extension API
Void is no longer an extension, so these links are no longer required, but they might be useful if we ever build an extension again.
- [Files you need in an extension](https://code.visualstudio.com/api/get-started/extension-anatomy).
- [An extension's `package.json` schema](https://code.visualstudio.com/api/references/extension-manifest).
- ["Contributes" Guide](https://code.visualstudio.com/api/references/contribution-points) - the `"contributes"` part of `package.json` is how an extension mounts.
- [The Full VSCode Extension API](https://code.visualstudio.com/api/references/vscode-api) - look on the right side for organization. The [bottom](https://code.visualstudio.com/api/references/vscode-api#api-patterns) of the page is easy to miss but is useful - cancellation tokens, events, disposables.
- [Activation events](https://code.visualstudio.com/api/references/activation-events) you can define in `package.json` (not the most useful).
================================================
FILE: build/.cachesalt
================================================
2024-12-11T00:28:56.838Z
================================================
FILE: build/.gitattributes
================================================
* text eol=lf
*.exe binary
*.dll binary
================================================
FILE: build/.gitignore
================================================
*.js.map
================================================
FILE: build/.moduleignore
================================================
# cleanup rules for node modules, .gitignore style
# native node modules
nan/**
*/node_modules/nan/**
fsevents/binding.gyp
fsevents/fsevents.cc
fsevents/build/**
fsevents/src/**
fsevents/test/**
!fsevents/**/*.node
@vscode/spdlog/binding.gyp
@vscode/spdlog/build/**
@vscode/spdlog/deps/**
@vscode/spdlog/src/**
@vscode/spdlog/test/**
@vscode/spdlog/*.yml
!@vscode/spdlog/build/Release/*.node
@vscode/deviceid/binding.gyp
@vscode/deviceid/build/**
@vscode/deviceid/deps/**
@vscode/deviceid/src/**
@vscode/deviceid/test/**
@vscode/deviceid/*.yml
!@vscode/deviceid/build/Release/*.node
@vscode/sqlite3/binding.gyp
@vscode/sqlite3/benchmark/**
@vscode/sqlite3/cloudformation/**
@vscode/sqlite3/deps/**
@vscode/sqlite3/test/**
@vscode/sqlite3/build/**
@vscode/sqlite3/src/**
!@vscode/sqlite3/build/Release/*.node
@vscode/windows-mutex/binding.gyp
@vscode/windows-mutex/build/**
@vscode/windows-mutex/src/**
!@vscode/windows-mutex/**/*.node
@vscode/windows-process-tree/binding.gyp
@vscode/windows-process-tree/build/**
@vscode/windows-process-tree/src/**
@vscode/windows-process-tree/tsconfig.json
@vscode/windows-process-tree/tslint.json
!@vscode/windows-process-tree/**/*.node
@vscode/windows-registry/binding.gyp
@vscode/windows-registry/src/**
@vscode/windows-registry/build/**
!@vscode/windows-registry/build/Release/*.node
@vscode/tree-sitter-wasm/wasm/tree-sitter-*.wasm
!@vscode/tree-sitter-wasm/wasm/tree-sitter-typescript.wasm
!@vscode/tree-sitter-wasm/wasm/tree-sitter-regex.wasm
!@vscode/tree-sitter-wasm/wasm/tree-sitter-ini.wasm
!@vscode/tree-sitter-wasm/wasm/tree-sitter-css.wasm
native-keymap/binding.gyp
native-keymap/build/**
native-keymap/src/**
native-keymap/deps/**
!native-keymap/build/Release/*.node
native-is-elevated/binding.gyp
native-is-elevated/build/**
native-is-elevated/src/**
native-is-elevated/deps/**
!native-is-elevated/build/Release/*.node
native-watchdog/binding.gyp
native-watchdog/build/**
native-watchdog/src/**
!native-watchdog/build/Release/*.node
@vscode/vsce-sign/**
!@vscode/vsce-sign/src/main.d.ts
!@vscode/vsce-sign/src/main.js
!@vscode/vsce-sign/package.json
!@vscode/vsce-sign/bin/**
windows-foreground-love/binding.gyp
windows-foreground-love/build/**
windows-foreground-love/src/**
!windows-foreground-love/**/*.node
kerberos/binding.gyp
kerberos/build/**
kerberos/src/**
kerberos/node_modules/**
!kerberos/**/*.node
node-pty/binding.gyp
node-pty/build/**
node-pty/src/**
node-pty/lib/*.test.js
node-pty/tools/**
node-pty/deps/**
node-pty/scripts/**
node-pty/third_party/**
!node-pty/build/Release/spawn-helper
!node-pty/build/Release/*.exe
!node-pty/build/Release/*.dll
!node-pty/build/Release/*.node
!node-pty/build/Release/conpty/conpty.dll
!node-pty/build/Release/conpty/OpenConsole.exe
@parcel/watcher/binding.gyp
@parcel/watcher/build/**
@parcel/watcher/prebuilds/**
@parcel/watcher/src/**
!@parcel/watcher/build/Release/*.node
vsda/**
!vsda/index.js
!vsda/index.d.ts
!vsda/package.json
!vsda/build/Release/vsda.node
!vsda/rust/web/**
!vsda/rust/bundler/**
@vscode/policy-watcher/build/**
@vscode/policy-watcher/.husky/**
@vscode/policy-watcher/src/**
@vscode/policy-watcher/binding.gyp
@vscode/policy-watcher/README.md
@vscode/policy-watcher/index.d.ts
!@vscode/policy-watcher/build/Release/vscode-policy-watcher.node
@vscode/windows-ca-certs/**/*
!@vscode/windows-ca-certs/package.json
!@vscode/windows-ca-certs/**/*.node
@vscode/node-addon-api/**/*
node-addon-api/**/*
prebuild-install/**/*
# other node modules
**/docs/**
**/example/**
**/examples/**
**/test/**
**/tests/**
**/History.md
**/CHANGELOG.md
**/README.md
**/readme.md
**/readme.markdown
**/CODE_OF_CONDUCT.md
**/SUPPORT.md
**/CONTRIBUTING.md
**/*.ts
# Exclude TS files that aren't needed by TS extension
typescript/lib/tsc.js
typescript/lib/typescriptServices.js
typescript/lib/tsserverlibrary.js
# We still need to include stdlib d.ts
!typescript/lib/lib*.d.ts
jschardet/index.js
jschardet/src/**
jschardet/dist/jschardet.js
es6-promise/lib/**
vscode-textmate/webpack.config.js
zone.js/dist/**
!zone.js/dist/zone-node.js
# https://github.com/xtermjs/xterm.js/issues/3137
@xterm/xterm/src/**
@xterm/xterm/tsconfig.all.json
# https://github.com/xtermjs/xterm.js/issues/3138
@xterm/xterm-addon-*/src/**
@xterm/xterm-addon-*/fixtures/**
@xterm/xterm-addon-*/out/**
@xterm/xterm-addon-*/out-test/**
================================================
FILE: build/.moduleignore.darwin
================================================
@vscode/windows-mutex/index.js
@vscode/windows-mutex/**/*.node
@vscode/windows-mutex/*.md
@vscode/windows-mutex/package.json
@vscode/windows-process-tree/lib/**
@vscode/windows-process-tree/**/*.node
@vscode/windows-process-tree/LICENSE
@vscode/windows-process-tree/package.json
@vscode/windows-process-tree/*.md
@vscode/windows-registry/dist/**
@vscode/windows-registry/**/*.node
@vscode/windows-registry/*.md
@vscode/windows-registry/*.txt
@vscode/windows-registry/package.json
!@vscode/windows-registry/dist/index.d.ts
================================================
FILE: build/.moduleignore.linux
================================================
@vscode/windows-mutex/index.js
@vscode/windows-mutex/**/*.node
@vscode/windows-mutex/*.md
@vscode/windows-mutex/package.json
@vscode/windows-process-tree/lib/**
@vscode/windows-process-tree/**/*.node
@vscode/windows-process-tree/LICENSE
@vscode/windows-process-tree/package.json
@vscode/windows-process-tree/*.md
@vscode/windows-registry/dist/**
@vscode/windows-registry/**/*.node
@vscode/windows-registry/*.md
@vscode/windows-registry/*.txt
@vscode/windows-registry/package.json
!@vscode/windows-registry/dist/index.d.ts
================================================
FILE: build/.moduleignore.win32
================================================
================================================
FILE: build/.npmrc
================================================
disturl="https://nodejs.org/dist"
runtime="node"
build_from_source="true"
legacy-peer-deps="true"
force_process_config="true"
timeout=180000
================================================
FILE: build/.webignore
================================================
# cleanup rules for web node modules, .gitignore style
**/*.txt
**/*.json
**/*.md
**/*.d.ts
**/*.js.map
**/LICENSE
**/CONTRIBUTORS
**/docs/**
**/example/**
**/examples/**
jschardet/index.js
jschardet/src/**
jschardet/dist/jschardet.js
vscode-textmate/webpack.config.js
@xterm/xterm/src/**
@xterm/addon-clipboard/src/**
@xterm/addon-clipboard/out/**
@xterm/addon-image/src/**
@xterm/addon-image/out/**
@xterm/addon-ligatures/src/**
@xterm/addon-ligatures/out/**
@xterm/addon-search/src/**
@xterm/addon-search/out/**
@xterm/addon-search/fixtures/**
@xterm/addon-unicode11/src/**
@xterm/addon-unicode11/out/**
@xterm/addon-webgl/src/**
@xterm/addon-webgl/out/**
# This makes sure the model is included in the package
!@vscode/vscode-languagedetection/model/**
!@vscode/tree-sitter-wasm/wasm/**
# Ensure only the required telemetry pieces are loaded in web to reduce bundle size
@microsoft/1ds-core-js/**
@microsoft/1ds-post-js/**
@microsoft/applicationinsights-core-js/**
@microsoft/applicationinsights-shims/**
!@microsoft/1ds-core-js/dist/ms.core.min.js
!@microsoft/1ds-core-js/bundle/ms.core.min.js
!@microsoft/1ds-post-js/dist/ms.post.min.js
!@microsoft/1ds-post-js/bundle/ms.post.min.js
!@microsoft/applicationinsights-core-js/browser/applicationinsights-core-js.min.js
!@microsoft/applicationinsights-shims/dist/umd/applicationinsights-shims.min.js
vsda/**
!vsda/rust/web/**
================================================
FILE: build/azure-pipelines/alpine/cli-build-alpine.yml
================================================
parameters:
- name: VSCODE_BUILD_ALPINE
type: boolean
default: false
- name: VSCODE_BUILD_ALPINE_ARM64
type: boolean
default: false
- name: VSCODE_QUALITY
type: string
- name: VSCODE_CHECK_ONLY
type: boolean
default: false
steps:
- task: NodeTool@0
inputs:
versionSource: fromFile
versionFilePath: .nvmrc
nodejsMirror: https://github.com/joaomoreno/node-mirror/releases/download
- ${{ if ne(parameters.VSCODE_QUALITY, 'oss') }}:
- template: ../cli/cli-apply-patches.yml@self
- script: |
set -e
npm ci
workingDirectory: build
env:
GITHUB_TOKEN: "$(github-distro-mixin-password)"
displayName: Install build dependencies
- task: Npm@1
displayName: Download openssl prebuilt
inputs:
command: custom
customCommand: pack @vscode-internal/openssl-prebuilt@0.0.11
customRegistry: useFeed
customFeed: "Monaco/openssl-prebuilt"
workingDir: $(Build.ArtifactStagingDirectory)
- script: |
set -e
mkdir $(Build.ArtifactStagingDirectory)/openssl
tar -xvzf $(Build.ArtifactStagingDirectory)/vscode-internal-openssl-prebuilt-0.0.11.tgz --strip-components=1 --directory=$(Build.ArtifactStagingDirectory)/openssl
displayName: Extract openssl prebuilt
# inspired by: https://github.com/emk/rust-musl-builder/blob/main/Dockerfile
- bash: |
set -e
sudo apt-get update
sudo apt-get install -yq build-essential musl-dev musl-tools linux-libc-dev pkgconf xutils-dev lld
sudo ln -s "/usr/bin/g++" "/usr/bin/musl-g++" || echo "link exists"
displayName: Install musl build dependencies
- template: ../cli/install-rust-posix.yml@self
parameters:
targets:
- ${{ if eq(parameters.VSCODE_BUILD_ALPINE_ARM64, true) }}:
- aarch64-unknown-linux-musl
- ${{ if eq(parameters.VSCODE_BUILD_ALPINE, true) }}:
- x86_64-unknown-linux-musl
- ${{ if eq(parameters.VSCODE_BUILD_ALPINE_ARM64, true) }}:
- template: ../cli/cli-compile.yml@self
parameters:
VSCODE_CLI_TARGET: aarch64-unknown-linux-musl
VSCODE_CLI_ARTIFACT: vscode_cli_alpine_arm64_cli
VSCODE_QUALITY: ${{ parameters.VSCODE_QUALITY }}
VSCODE_CLI_ENV:
CXX_aarch64-unknown-linux-musl: musl-g++
CC_aarch64-unknown-linux-musl: musl-gcc
OPENSSL_LIB_DIR: $(Build.ArtifactStagingDirectory)/openssl/arm64-linux-musl/lib
OPENSSL_INCLUDE_DIR: $(Build.ArtifactStagingDirectory)/openssl/arm64-linux-musl/include
OPENSSL_STATIC: "1"
- ${{ if eq(parameters.VSCODE_BUILD_ALPINE, true) }}:
- template: ../cli/cli-compile.yml@self
parameters:
VSCODE_CLI_TARGET: x86_64-unknown-linux-musl
VSCODE_CLI_ARTIFACT: vscode_cli_alpine_x64_cli
VSCODE_QUALITY: ${{ parameters.VSCODE_QUALITY }}
VSCODE_CLI_ENV:
CXX_aarch64-unknown-linux-musl: musl-g++
CC_aarch64-unknown-linux-musl: musl-gcc
OPENSSL_LIB_DIR: $(Build.ArtifactStagingDirectory)/openssl/x64-linux-musl/lib
OPENSSL_INCLUDE_DIR: $(Build.ArtifactStagingDirectory)/openssl/x64-linux-musl/include
OPENSSL_STATIC: "1"
- ${{ if not(parameters.VSCODE_CHECK_ONLY) }}:
- ${{ if eq(parameters.VSCODE_BUILD_ALPINE_ARM64, true) }}:
- task: 1ES.PublishPipelineArtifact@1
inputs:
targetPath: $(Build.ArtifactStagingDirectory)/vscode_cli_alpine_arm64_cli.tar.gz
artifactName: vscode_cli_alpine_arm64_cli
sbomBuildDropPath: $(Build.ArtifactStagingDirectory)/cli
sbomPackageName: "VS Code Alpine arm64 CLI"
sbomPackageVersion: $(Build.SourceVersion)
displayName: Publish vscode_cli_alpine_arm64_cli artifact
- ${{ if eq(parameters.VSCODE_BUILD_ALPINE, true) }}:
- task: 1ES.PublishPipelineArtifact@1
inputs:
targetPath: $(Build.ArtifactStagingDirectory)/vscode_cli_alpine_x64_cli.tar.gz
artifactName: vscode_cli_alpine_x64_cli
sbomBuildDropPath: $(Build.ArtifactStagingDirectory)/cli
sbomPackageName: "VS Code Alpine x64 CLI"
sbomPackageVersion: $(Build.SourceVersion)
displayName: Publish vscode_cli_alpine_x64_cli artifact
================================================
FILE: build/azure-pipelines/alpine/product-build-alpine.yml
================================================
steps:
- task: NodeTool@0
inputs:
versionSource: fromFile
versionFilePath: .nvmrc
nodejsMirror: https://github.com/joaomoreno/node-mirror/releases/download
- template: ../distro/download-distro.yml@self
- task: AzureKeyVault@2
displayName: "Azure Key Vault: Get Secrets"
inputs:
azureSubscription: vscode
KeyVaultName: vscode-build-secrets
SecretsFilter: "github-distro-mixin-password"
- task: DownloadPipelineArtifact@2
inputs:
artifact: Compilation
path: $(Build.ArtifactStagingDirectory)
displayName: Download compilation output
- script: tar -xzf $(Build.ArtifactStagingDirectory)/compilation.tar.gz
displayName: Extract compilation output
- script: node build/setup-npm-registry.js $NPM_REGISTRY
condition: and(succeeded(), ne(variables['NPM_REGISTRY'], 'none'))
displayName: Setup NPM Registry
- script: mkdir -p .build && node build/azure-pipelines/common/computeNodeModulesCacheKey.js alpine $VSCODE_ARCH > .build/packagelockhash
displayName: Prepare node_modules cache key
- task: Cache@2
inputs:
key: '"node_modules" | .build/packagelockhash'
path: .build/node_modules_cache
cacheHitVar: NODE_MODULES_RESTORED
displayName: Restore node_modules cache
- script: tar -xzf .build/node_modules_cache/cache.tgz
condition: and(succeeded(), eq(variables.NODE_MODULES_RESTORED, 'true'))
displayName: Extract node_modules cache
- script: |
set -e
# Set the private NPM registry to the global npmrc file
# so that authentication works for subfolders like build/, remote/, extensions/ etc
# which does not have their own .npmrc file
npm config set registry "$NPM_REGISTRY"
echo "##vso[task.setvariable variable=NPMRC_PATH]$(npm config get userconfig)"
condition: and(succeeded(), ne(variables.NODE_MODULES_RESTORED, 'true'), ne(variables['NPM_REGISTRY'], 'none'))
displayName: Setup NPM
- task: npmAuthenticate@0
inputs:
workingFile: $(NPMRC_PATH)
condition: and(succeeded(), ne(variables.NODE_MODULES_RESTORED, 'true'), ne(variables['NPM_REGISTRY'], 'none'))
displayName: Setup NPM Authentication
- task: Docker@1
inputs:
azureSubscriptionEndpoint: vscode
azureContainerRegistry: vscodehub.azurecr.io
command: "Run an image"
imageName: "vscode-linux-build-agent:alpine-$(VSCODE_ARCH)"
containerCommand: uname
displayName: "Pull image"
condition: and(succeeded(), ne(variables.NODE_MODULES_RESTORED, 'true'))
- script: sudo apt-get update && sudo apt-get install -y libkrb5-dev
displayName: Install build dependencies
condition: and(succeeded(), ne(variables.NODE_MODULES_RESTORED, 'true'))
- script: |
set -e
for i in {1..5}; do # try 5 times
npm ci && break
if [ $i -eq 5 ]; then
echo "Npm install failed too many times" >&2
exit 1
fi
echo "Npm install failed $i, trying again..."
done
env:
npm_config_arch: $(NPM_ARCH)
ELECTRON_SKIP_BINARY_DOWNLOAD: 1
PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD: 1
GITHUB_TOKEN: "$(github-distro-mixin-password)"
VSCODE_REMOTE_DEPENDENCIES_CONTAINER_NAME: vscodehub.azurecr.io/vscode-linux-build-agent:alpine-$(VSCODE_ARCH)
VSCODE_HOST_MOUNT: "/mnt/vss/_work/1/s"
displayName: Install dependencies
condition: and(succeeded(), ne(variables.NODE_MODULES_RESTORED, 'true'))
- script: node build/azure-pipelines/distro/mixin-npm
displayName: Mixin distro node modules
condition: and(succeeded(), ne(variables.NODE_MODULES_RESTORED, 'true'))
- script: |
set -e
node build/azure-pipelines/common/listNodeModules.js .build/node_modules_list.txt
mkdir -p .build/node_modules_cache
tar -czf .build/node_modules_cache/cache.tgz --files-from .build/node_modules_list.txt
condition: and(succeeded(), ne(variables.NODE_MODULES_RESTORED, 'true'))
displayName: Create node_modules archive
- script: node build/azure-pipelines/distro/mixin-quality
displayName: Mixin distro quality
- template: ../common/install-builtin-extensions.yml@self
- script: |
set -e
TARGET=$([ "$VSCODE_ARCH" == "x64" ] && echo "linux-alpine" || echo "alpine-arm64") # TODO@joaomoreno
npm run gulp vscode-reh-$TARGET-min-ci
(cd .. && mv vscode-reh-$TARGET vscode-server-$TARGET) # TODO@joaomoreno
ARCHIVE_PATH=".build/linux/server/vscode-server-$TARGET.tar.gz"
DIR_PATH="$(realpath ../vscode-server-$TARGET)"
mkdir -p $(dirname $ARCHIVE_PATH)
tar --owner=0 --group=0 -czf $ARCHIVE_PATH -C .. vscode-server-$TARGET
echo "##vso[task.setvariable variable=SERVER_DIR_PATH]$DIR_PATH"
echo "##vso[task.setvariable variable=SERVER_PATH]$ARCHIVE_PATH"
env:
GITHUB_TOKEN: "$(github-distro-mixin-password)"
displayName: Build server
- script: |
set -e
TARGET=$([ "$VSCODE_ARCH" == "x64" ] && echo "linux-alpine" || echo "alpine-arm64")
npm run gulp vscode-reh-web-$TARGET-min-ci
(cd .. && mv vscode-reh-web-$TARGET vscode-server-$TARGET-web) # TODO@joaomoreno
ARCHIVE_PATH=".build/linux/web/vscode-server-$TARGET-web.tar.gz"
DIR_PATH="$(realpath ../vscode-server-$TARGET-web)"
mkdir -p $(dirname $ARCHIVE_PATH)
tar --owner=0 --group=0 -czf $ARCHIVE_PATH -C .. vscode-server-$TARGET-web
echo "##vso[task.setvariable variable=WEB_DIR_PATH]$DIR_PATH"
echo "##vso[task.setvariable variable=WEB_PATH]$ARCHIVE_PATH"
env:
GITHUB_TOKEN: "$(github-distro-mixin-password)"
displayName: Build server (web)
- script: echo "##vso[task.setvariable variable=ARTIFACT_PREFIX]attempt$(System.JobAttempt)_"
condition: and(succeededOrFailed(), notIn(variables['Agent.JobStatus'], 'Succeeded', 'SucceededWithIssues'))
displayName: Generate artifact prefix
- task: 1ES.PublishPipelineArtifact@1
inputs:
targetPath: $(SERVER_PATH)
artifactName: $(ARTIFACT_PREFIX)vscode_server_alpine_$(VSCODE_ARCH)_archive-unsigned
sbomBuildDropPath: $(SERVER_DIR_PATH)
sbomPackageName: "VS Code Alpine $(VSCODE_ARCH) Server"
sbomPackageVersion: $(Build.SourceVersion)
displayName: Publish server archive
condition: and(succeededOrFailed(), ne(variables['SERVER_PATH'], ''), ne(variables['VSCODE_ARCH'], 'x64'))
- task: 1ES.PublishPipelineArtifact@1
inputs:
targetPath: $(WEB_PATH)
artifactName: $(ARTIFACT_PREFIX)vscode_web_alpine_$(VSCODE_ARCH)_archive-unsigned
sbomBuildDropPath: $(WEB_DIR_PATH)
sbomPackageName: "VS Code Alpine $(VSCODE_ARCH) Web"
sbomPackageVersion: $(Build.SourceVersion)
displayName: Publish web server archive
condition: and(succeededOrFailed(), ne(variables['WEB_PATH'], ''), ne(variables['VSCODE_ARCH'], 'x64'))
# same as above, keep legacy name
- task: 1ES.PublishPipelineArtifact@1
inputs:
targetPath: $(SERVER_PATH)
artifactName: $(ARTIFACT_PREFIX)vscode_server_linux_alpine_archive-unsigned
sbomEnabled: false
displayName: Publish x64 server archive
condition: and(succeededOrFailed(), ne(variables['SERVER_PATH'], ''), eq(variables['VSCODE_ARCH'], 'x64'))
# same as above, keep legacy name
- task: 1ES.PublishPipelineArtifact@1
inputs:
targetPath: $(WEB_PATH)
artifactName: $(ARTIFACT_PREFIX)vscode_web_linux_alpine_archive-unsigned
sbomEnabled: false
displayName: Publish x64 web server archive
condition: and(succeededOrFailed(), ne(variables['WEB_PATH'], ''), eq(variables['VSCODE_ARCH'], 'x64'))
================================================
FILE: build/azure-pipelines/cli/cli-apply-patches.yml
================================================
steps:
- template: ../distro/download-distro.yml@self
- script: node build/azure-pipelines/distro/mixin-quality
displayName: Mixin distro quality
- script: node .build/distro/cli-patches/index.js
displayName: Apply distro patches
================================================
FILE: build/azure-pipelines/cli/cli-compile.yml
================================================
parameters:
- name: VSCODE_QUALITY
type: string
- name: VSCODE_CLI_TARGET
type: string
- name: VSCODE_CLI_ARTIFACT
type: string
- name: VSCODE_CLI_ENV
type: object
default: {}
- name: VSCODE_CHECK_ONLY
type: boolean
default: false
steps:
- ${{ if contains(parameters.VSCODE_CLI_TARGET, '-windows-') }}:
- ${{ if eq(parameters.VSCODE_QUALITY, 'oss') }}:
- pwsh: Write-Host "##vso[task.setvariable variable=VSCODE_CLI_PRODUCT_JSON]$(Build.SourcesDirectory)/product.json"
displayName: Set product.json path
- ${{ else }}:
- pwsh: Write-Host "##vso[task.setvariable variable=VSCODE_CLI_PRODUCT_JSON]$(Build.SourcesDirectory)/.build/distro/mixin/${{ parameters.VSCODE_QUALITY }}/product.json"
displayName: Set product.json path
- ${{ else }}:
- ${{ if eq(parameters.VSCODE_QUALITY, 'oss') }}:
- script: echo "##vso[task.setvariable variable=VSCODE_CLI_PRODUCT_JSON]$(Build.SourcesDirectory)/product.json"
displayName: Set product.json path
- ${{ else }}:
- script: echo "##vso[task.setvariable variable=VSCODE_CLI_PRODUCT_JSON]$(Build.SourcesDirectory)/.build/distro/mixin/${{ parameters.VSCODE_QUALITY }}/product.json"
displayName: Set product.json path
- ${{ if parameters.VSCODE_CHECK_ONLY }}:
- script: cargo clippy --target ${{ parameters.VSCODE_CLI_TARGET }} --bin=code
displayName: Lint ${{ parameters.VSCODE_CLI_TARGET }}
workingDirectory: $(Build.SourcesDirectory)/cli
env:
CARGO_NET_GIT_FETCH_WITH_CLI: true
${{ each pair in parameters.VSCODE_CLI_ENV }}:
${{ pair.key }}: ${{ pair.value }}
- ${{ else }}:
- ${{ if contains(parameters.VSCODE_CLI_TARGET, '-linux-') }}:
- script: |
set -e
if [ -n "$SYSROOT_ARCH" ]; then
export VSCODE_SYSROOT_DIR=$(Build.SourcesDirectory)/.build/sysroots
node -e '(async () => { const { getVSCodeSysroot } = require("../build/linux/debian/install-sysroot.js"); await getVSCodeSysroot(process.env["SYSROOT_ARCH"]); })()'
if [ "$SYSROOT_ARCH" == "arm64" ]; then
export CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER="$VSCODE_SYSROOT_DIR/aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc"
export CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_RUSTFLAGS="-C link-arg=--sysroot=$VSCODE_SYSROOT_DIR/aarch64-linux-gnu/aarch64-linux-gnu/sysroot"
export CC_aarch64_unknown_linux_gnu="$VSCODE_SYSROOT_DIR/aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc --sysroot=$VSCODE_SYSROOT_DIR/aarch64-linux-gnu/aarch64-linux-gnu/sysroot"
export PKG_CONFIG_LIBDIR_aarch64_unknown_linux_gnu="$VSCODE_SYSROOT_DIR/aarch64-linux-gnu/aarch64-linux-gnu/sysroot/usr/lib/aarch64-linux-gnu/pkgconfig:$VSCODE_SYSROOT_DIR/aarch64-linux-gnu/aarch64-linux-gnu/sysroot/usr/share/pkgconfig"
export PKG_CONFIG_SYSROOT_DIR_aarch64_unknown_linux_gnu="$VSCODE_SYSROOT_DIR/aarch64-linux-gnu/aarch64-linux-gnu/sysroot"
export OBJDUMP="$VSCODE_SYSROOT_DIR/aarch64-linux-gnu/aarch64-linux-gnu/bin/objdump"
elif [ "$SYSROOT_ARCH" == "amd64" ]; then
export CARGO_TARGET_X86_64_UNKNOWN_LINUX_GNU_LINKER="$VSCODE_SYSROOT_DIR/x86_64-linux-gnu/bin/x86_64-linux-gnu-gcc"
export CARGO_TARGET_X86_64_UNKNOWN_LINUX_GNU_RUSTFLAGS="-C link-arg=--sysroot=$VSCODE_SYSROOT_DIR/x86_64-linux-gnu/x86_64-linux-gnu/sysroot -C link-arg=-L$VSCODE_SYSROOT_DIR/x86_64-linux-gnu/x86_64-linux-gnu/sysroot/usr/lib/x86_64-linux-gnu"
export CC_x86_64_unknown_linux_gnu="$VSCODE_SYSROOT_DIR/x86_64-linux-gnu/bin/x86_64-linux-gnu-gcc --sysroot=$VSCODE_SYSROOT_DIR/x86_64-linux-gnu/x86_64-linux-gnu/sysroot"
export PKG_CONFIG_LIBDIR_x86_64_unknown_linux_gnu="$VSCODE_SYSROOT_DIR/x86_64-linux-gnu/x86_64-linux-gnu/sysroot/usr/lib/x86_64-linux-gnu/pkgconfig:$VSCODE_SYSROOT_DIR/x86_64-linux-gnu/x86_64-linux-gnu/sysroot/usr/share/pkgconfig"
export PKG_CONFIG_SYSROOT_DIR_x86_64_unknown_linux_gnu="$VSCODE_SYSROOT_DIR/x86_64-linux-gnu/x86_64-linux-gnu/sysroot"
export OBJDUMP="$VSCODE_SYSROOT_DIR/x86_64-linux-gnu/x86_64-linux-gnu/bin/objdump"
elif [ "$SYSROOT_ARCH" == "armhf" ]; then
export CARGO_TARGET_ARMV7_UNKNOWN_LINUX_GNUEABIHF_LINKER="$VSCODE_SYSROOT_DIR/arm-rpi-linux-gnueabihf/bin/arm-rpi-linux-gnueabihf-gcc"
export CARGO_TARGET_ARMV7_UNKNOWN_LINUX_GNUEABIHF_RUSTFLAGS="-C link-arg=--sysroot=$VSCODE_SYSROOT_DIR/arm-rpi-linux-gnueabihf/arm-rpi-linux-gnueabihf/sysroot"
export CC_armv7_unknown_linux_gnueabihf="$VSCODE_SYSROOT_DIR/arm-rpi-linux-gnueabihf/bin/arm-rpi-linux-gnueabihf-gcc --sysroot=$VSCODE_SYSROOT_DIR/arm-rpi-linux-gnueabihf/arm-rpi-linux-gnueabihf/sysroot"
export PKG_CONFIG_LIBDIR_armv7_unknown_linux_gnueabihf="$VSCODE_SYSROOT_DIR/arm-rpi-linux-gnueabihf/arm-rpi-linux-gnueabihf/sysroot/usr/lib/arm-rpi-linux-gnueabihf/pkgconfig:$VSCODE_SYSROOT_DIR/arm-rpi-linux-gnueabihf/arm-rpi-linux-gnueabihf/sysroot/usr/share/pkgconfig"
export PKG_CONFIG_SYSROOT_DIR_armv7_unknown_linux_gnueabihf="$VSCODE_SYSROOT_DIR/arm-rpi-linux-gnueabihf/arm-rpi-linux-gnueabihf/sysroot"
export OBJDUMP="$VSCODE_SYSROOT_DIR/arm-rpi-linux-gnueabihf/arm-rpi-linux-gnueabihf/bin/objdump"
fi
fi
cargo build --release --target ${{ parameters.VSCODE_CLI_TARGET }} --bin=code
# verify glibc requirement
if [ -n "$SYSROOT_ARCH" ]; then
glibc_version="2.28"
while IFS= read -r line; do
if [[ $line == *"GLIBC_"* ]]; then
version=$(echo "$line" | awk '{print $5}' | tr -d '()')
version=${version#*_}
if [[ $(printf "%s\n%s" "$version" "$glibc_version" | sort -V | tail -n1) == "$version" ]]; then
glibc_version=$version
fi
fi
done < <("$OBJDUMP" -T "$PWD/target/${{ parameters.VSCODE_CLI_TARGET }}/release/code")
if [[ "$glibc_version" != "2.28" ]]; then
echo "Error: binary has dependency on GLIBC > 2.28, found $glibc_version"
exit 1
fi
fi
displayName: Compile ${{ parameters.VSCODE_CLI_TARGET }}
workingDirectory: $(Build.SourcesDirectory)/cli
env:
CARGO_NET_GIT_FETCH_WITH_CLI: true
VSCODE_CLI_COMMIT: $(Build.SourceVersion)
GITHUB_TOKEN: "$(github-distro-mixin-password)"
${{ each pair in parameters.VSCODE_CLI_ENV }}:
${{ pair.key }}: ${{ pair.value }}
- ${{ else }}:
- script: cargo build --release --target ${{ parameters.VSCODE_CLI_TARGET }} --bin=code
displayName: Compile ${{ parameters.VSCODE_CLI_TARGET }}
workingDirectory: $(Build.SourcesDirectory)/cli
env:
CARGO_NET_GIT_FETCH_WITH_CLI: true
VSCODE_CLI_COMMIT: $(Build.SourceVersion)
${{ each pair in parameters.VSCODE_CLI_ENV }}:
${{ pair.key }}: ${{ pair.value }}
- ${{ if contains(parameters.VSCODE_CLI_TARGET, '-windows-') }}:
- task: PublishSymbols@2
inputs:
IndexSources: false
SymbolsFolder: $(Build.SourcesDirectory)/cli/target/${{ parameters.VSCODE_CLI_TARGET }}/release
SearchPattern: 'code.pdb'
SymbolServerType: TeamServices
SymbolsProduct: 'code'
ArtifactServices.Symbol.AccountName: microsoft
ArtifactServices.Symbol.PAT: $(System.AccessToken)
ArtifactServices.Symbol.UseAAD: false
displayName: Publish Symbols
- powershell: |
. build/azure-pipelines/win32/exec.ps1
$ErrorActionPreference = "Stop"
$AppProductJson = Get-Content -Raw -Path "$env:VSCODE_CLI_PRODUCT_JSON" | ConvertFrom-Json
$env:VSCODE_CLI_APPLICATION_NAME = $AppProductJson.applicationName
Write-Host "##vso[task.setvariable variable=VSCODE_CLI_APPLICATION_NAME]$env:VSCODE_CLI_APPLICATION_NAME"
New-Item -ItemType Directory -Force -Path "$(Build.ArtifactStagingDirectory)/cli"
Move-Item -Path $(Build.SourcesDirectory)/cli/target/${{ parameters.VSCODE_CLI_TARGET }}/release/code.exe -Destination "$(Build.ArtifactStagingDirectory)/cli/${env:VSCODE_CLI_APPLICATION_NAME}.exe"
displayName: Stage CLI
- task: ArchiveFiles@2
displayName: Archive CLI
inputs:
rootFolderOrFile: $(Build.ArtifactStagingDirectory)/cli/$(VSCODE_CLI_APPLICATION_NAME).exe
includeRootFolder: false
archiveType: zip
archiveFile: $(Build.ArtifactStagingDirectory)/${{ parameters.VSCODE_CLI_ARTIFACT }}.zip
- ${{ else }}:
- script: |
set -e
VSCODE_CLI_APPLICATION_NAME=$(node -p "require(\"$VSCODE_CLI_PRODUCT_JSON\").applicationName")
echo "##vso[task.setvariable variable=VSCODE_CLI_APPLICATION_NAME]$VSCODE_CLI_APPLICATION_NAME"
mkdir -p $(Build.ArtifactStagingDirectory)/cli
mv $(Build.SourcesDirectory)/cli/target/${{ parameters.VSCODE_CLI_TARGET }}/release/code $(Build.ArtifactStagingDirectory)/cli/$VSCODE_CLI_APPLICATION_NAME
displayName: Stage CLI
- task: ArchiveFiles@2
displayName: Archive CLI
inputs:
rootFolderOrFile: $(Build.ArtifactStagingDirectory)/cli/$(VSCODE_CLI_APPLICATION_NAME)
includeRootFolder: false
${{ if contains(parameters.VSCODE_CLI_TARGET, '-darwin') }}:
archiveType: zip
archiveFile: $(Build.ArtifactStagingDirectory)/${{ parameters.VSCODE_CLI_ARTIFACT }}.zip
${{ else }}:
archiveType: tar
tarCompression: gz
archiveFile: $(Build.ArtifactStagingDirectory)/${{ parameters.VSCODE_CLI_ARTIFACT }}.tar.gz
================================================
FILE: build/azure-pipelines/cli/cli-darwin-sign.yml
================================================
parameters:
- name: VSCODE_CLI_ARTIFACTS
type: object
default: []
steps:
- task: UseDotNet@2
inputs:
version: 6.x
- task: EsrpCodeSigning@5
inputs:
UseMSIAuthentication: true
ConnectedServiceName: vscode-esrp
AppRegistrationClientId: $(ESRP_CLIENT_ID)
AppRegistrationTenantId: $(ESRP_TENANT_ID)
AuthAKVName: vscode-esrp
AuthSignCertName: esrp-sign
FolderPath: .
Pattern: noop
displayName: 'Install ESRP Tooling'
- ${{ each target in parameters.VSCODE_CLI_ARTIFACTS }}:
- task: DownloadPipelineArtifact@2
displayName: Download ${{ target }}
inputs:
artifact: ${{ target }}
path: $(Build.ArtifactStagingDirectory)/pkg/${{ target }}
- task: ExtractFiles@1
displayName: Extract artifact
inputs:
archiveFilePatterns: $(Build.ArtifactStagingDirectory)/pkg/${{ target }}/*.zip
destinationFolder: $(Build.ArtifactStagingDirectory)/sign/${{ target }}
- script: node build/azure-pipelines/common/sign $(Agent.RootDirectory)/_tasks/EsrpCodeSigning_*/*/net6.0/esrpcli.dll sign-darwin $(Build.ArtifactStagingDirectory)/pkg "*.zip"
env:
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
displayName: Codesign
- script: node build/azure-pipelines/common/sign $(Agent.RootDirectory)/_tasks/EsrpCodeSigning_*/*/net6.0/esrpcli.dll notarize-darwin $(Build.ArtifactStagingDirectory)/pkg "*.zip"
env:
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
displayName: Notarize
- ${{ each target in parameters.VSCODE_CLI_ARTIFACTS }}:
- script: |
set -e
ASSET_ID=$(echo "${{ target }}" | sed "s/unsigned_//")
mv $(Build.ArtifactStagingDirectory)/pkg/${{ target }}/${{ target }}.zip $(Build.ArtifactStagingDirectory)/pkg/${{ target }}/$ASSET_ID.zip
echo "##vso[task.setvariable variable=ASSET_ID]$ASSET_ID"
displayName: Set asset id variable
- task: 1ES.PublishPipelineArtifact@1
inputs:
targetPath: $(Build.ArtifactStagingDirectory)/pkg/${{ target }}/$(ASSET_ID).zip
artifactName: $(ASSET_ID)
sbomBuildDropPath: $(Build.ArtifactStagingDirectory)/sign/${{ target }}
sbomPackageName: "VS Code macOS ${{ target }} CLI"
sbomPackageVersion: $(Build.SourceVersion)
displayName: Publish signed artifact with ID $(ASSET_ID)
================================================
FILE: build/azure-pipelines/cli/cli-win32-sign.yml
================================================
parameters:
- name: VSCODE_CLI_ARTIFACTS
type: object
default: []
steps:
- task: UseDotNet@2
inputs:
version: 6.x
- task: EsrpCodeSigning@5
inputs:
UseMSIAuthentication: true
ConnectedServiceName: vscode-esrp
AppRegistrationClientId: $(ESRP_CLIENT_ID)
AppRegistrationTenantId: $(ESRP_TENANT_ID)
AuthAKVName: vscode-esrp
AuthSignCertName: esrp-sign
FolderPath: .
Pattern: noop
displayName: 'Install ESRP Tooling'
- powershell: |
. build/azure-pipelines/win32/exec.ps1
$ErrorActionPreference = "Stop"
$EsrpCodeSigningTool = (gci -directory -filter EsrpCodeSigning_* $(Agent.RootDirectory)\_tasks | Select-Object -last 1).FullName
$Version = (gci -directory $EsrpCodeSigningTool | Select-Object -last 1).FullName
echo "##vso[task.setvariable variable=EsrpCliDllPath]$Version\net6.0\esrpcli.dll"
displayName: Find ESRP CLI
- ${{ each target in parameters.VSCODE_CLI_ARTIFACTS }}:
- task: DownloadPipelineArtifact@2
displayName: Download artifact
inputs:
artifact: ${{ target }}
path: $(Build.ArtifactStagingDirectory)/pkg/${{ target }}
- task: ExtractFiles@1
displayName: Extract artifact
inputs:
archiveFilePatterns: $(Build.ArtifactStagingDirectory)/pkg/${{ target }}/*.zip
destinationFolder: $(Build.ArtifactStagingDirectory)/sign/${{ target }}
- powershell: node build\azure-pipelines\common\sign $env:EsrpCliDllPath sign-windows $(Build.ArtifactStagingDirectory)/sign "*.exe"
env:
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
displayName: Codesign
- ${{ each target in parameters.VSCODE_CLI_ARTIFACTS }}:
- powershell: |
$ASSET_ID = "${{ target }}".replace("unsigned_", "");
echo "##vso[task.setvariable variable=ASSET_ID]$ASSET_ID"
displayName: Set asset id variable
- task: ArchiveFiles@2
displayName: Archive signed files
inputs:
rootFolderOrFile: $(Build.ArtifactStagingDirectory)/sign/${{ target }}
includeRootFolder: false
archiveType: zip
archiveFile: $(Build.ArtifactStagingDirectory)/$(ASSET_ID).zip
- task: 1ES.PublishPipelineArtifact@1
inputs:
targetPath: $(Build.ArtifactStagingDirectory)/$(ASSET_ID).zip
artifactName: $(ASSET_ID)
sbomBuildDropPath: $(Build.ArtifactStagingDirectory)/sign/${{ target }}
sbomPackageName: "VS Code Windows ${{ target }} CLI"
sbomPackageVersion: $(Build.SourceVersion)
displayName: Publish signed artifact with ID $(ASSET_ID)
================================================
FILE: build/azure-pipelines/cli/install-rust-posix.yml
================================================
parameters:
- name: channel
type: string
default: 1.85
- name: targets
default: []
type: object
# Todo: use 1ES pipeline once extension is installed in ADO
steps:
- task: RustInstaller@1
inputs:
rustVersion: ms-${{ parameters.channel }}
cratesIoFeedOverride: $(CARGO_REGISTRY)
additionalTargets: ${{ join(' ', parameters.targets) }}
toolchainFeed: https://pkgs.dev.azure.com/monacotools/Monaco/_packaging/vscode/nuget/v3/index.json
default: true
addToPath: true
displayName: Install MSFT Rust
condition: and(succeeded(), ne(variables['CARGO_REGISTRY'], 'none'))
- script: |
set -e
curl https://sh.rustup.rs -sSf | sh -s -- -y --profile minimal --default-toolchain $RUSTUP_TOOLCHAIN
echo "##vso[task.setvariable variable=PATH;]$PATH:$HOME/.cargo/bin"
env:
RUSTUP_TOOLCHAIN: ${{ parameters.channel }}
displayName: Install OSS Rust
condition: and(succeeded(), eq(variables['CARGO_REGISTRY'], 'none'))
- script: |
set -e
rustup default $RUSTUP_TOOLCHAIN
rustup update $RUSTUP_TOOLCHAIN
rustup component add clippy
env:
RUSTUP_TOOLCHAIN: ${{ parameters.channel }}
displayName: "Set Rust version"
condition: and(succeeded(), eq(variables['CARGO_REGISTRY'], 'none'))
- ${{ each target in parameters.targets }}:
- script: rustup target add ${{ target }}
displayName: "Adding Rust target '${{ target }}'"
condition: and(succeeded(), eq(variables['CARGO_REGISTRY'], 'none'))
- script: |
set -e
rustc --version
cargo --version
displayName: "Check Rust versions"
================================================
FILE: build/azure-pipelines/cli/install-rust-win32.yml
================================================
parameters:
- name: channel
type: string
default: 1.85
- name: targets
default: []
type: object
# Todo: use 1ES pipeline once extension is installed in ADO
steps:
- task: RustInstaller@1
inputs:
rustVersion: ms-${{ parameters.channel }}
cratesIoFeedOverride: $(CARGO_REGISTRY)
additionalTargets: ${{ join(' ', parameters.targets) }}
toolchainFeed: https://pkgs.dev.azure.com/monacotools/Monaco/_packaging/vscode/nuget/v3/index.json
default: true
addToPath: true
displayName: Install MSFT Rust
condition: and(succeeded(), ne(variables['CARGO_REGISTRY'], 'none'))
- powershell: |
. build/azure-pipelines/win32/exec.ps1
Invoke-WebRequest -Uri "https://win.rustup.rs" -Outfile $(Build.ArtifactStagingDirectory)/rustup-init.exe
exec { $(Build.ArtifactStagingDirectory)/rustup-init.exe -y --profile minimal --default-toolchain $env:RUSTUP_TOOLCHAIN --default-host x86_64-pc-windows-msvc }
echo "##vso[task.prependpath]$env:USERPROFILE\.cargo\bin"
env:
RUSTUP_TOOLCHAIN: ${{ parameters.channel }}
displayName: Install OSS Rust
condition: and(succeeded(), eq(variables['CARGO_REGISTRY'], 'none'))
- powershell: |
. build/azure-pipelines/win32/exec.ps1
exec { rustup default $RUSTUP_TOOLCHAIN }
exec { rustup update $RUSTUP_TOOLCHAIN }
env:
RUSTUP_TOOLCHAIN: ${{ parameters.channel }}
displayName: "Set Rust version"
condition: and(succeeded(), eq(variables['CARGO_REGISTRY'], 'none'))
- ${{ each target in parameters.targets }}:
- script: rustup target add ${{ target }}
displayName: "Adding Rust target '${{ target }}'"
condition: and(succeeded(), eq(variables['CARGO_REGISTRY'], 'none'))
- powershell: |
. build/azure-pipelines/win32/exec.ps1
exec { rustc --version }
exec { cargo --version }
displayName: "Check Rust versions"
================================================
FILE: build/azure-pipelines/cli/test.yml
================================================
steps:
- template: ./install-rust-posix.yml@self
- script: cargo clippy -- -D warnings
workingDirectory: cli
displayName: Clippy lint
- script: cargo test
workingDirectory: cli
displayName: Run unit tests
================================================
FILE: build/azure-pipelines/common/computeBuiltInDepsCacheKey.js
================================================
"use strict";
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
Object.defineProperty(exports, "__esModule", { value: true });
const fs_1 = __importDefault(require("fs"));
const path_1 = __importDefault(require("path"));
const crypto_1 = __importDefault(require("crypto"));
const productjson = JSON.parse(fs_1.default.readFileSync(path_1.default.join(__dirname, '../../../product.json'), 'utf8'));
const shasum = crypto_1.default.createHash('sha256');
for (const ext of productjson.builtInExtensions) {
shasum.update(`${ext.name}@${ext.version}`);
}
process.stdout.write(shasum.digest('hex'));
//# sourceMappingURL=computeBuiltInDepsCacheKey.js.map
================================================
FILE: build/azure-pipelines/common/computeBuiltInDepsCacheKey.ts
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
import fs from 'fs';
import path from 'path';
import crypto from 'crypto';
const productjson = JSON.parse(fs.readFileSync(path.join(__dirname, '../../../product.json'), 'utf8'));
const shasum = crypto.createHash('sha256');
for (const ext of productjson.builtInExtensions) {
shasum.update(`${ext.name}@${ext.version}`);
}
process.stdout.write(shasum.digest('hex'));
================================================
FILE: build/azure-pipelines/common/computeNodeModulesCacheKey.js
================================================
"use strict";
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
Object.defineProperty(exports, "__esModule", { value: true });
const fs_1 = __importDefault(require("fs"));
const path_1 = __importDefault(require("path"));
const crypto_1 = __importDefault(require("crypto"));
const { dirs } = require('../../npm/dirs');
const ROOT = path_1.default.join(__dirname, '../../../');
const shasum = crypto_1.default.createHash('sha256');
shasum.update(fs_1.default.readFileSync(path_1.default.join(ROOT, 'build/.cachesalt')));
shasum.update(fs_1.default.readFileSync(path_1.default.join(ROOT, '.npmrc')));
shasum.update(fs_1.default.readFileSync(path_1.default.join(ROOT, 'build', '.npmrc')));
shasum.update(fs_1.default.readFileSync(path_1.default.join(ROOT, 'remote', '.npmrc')));
// Add `package.json` and `package-lock.json` files
for (const dir of dirs) {
const packageJsonPath = path_1.default.join(ROOT, dir, 'package.json');
const packageJson = JSON.parse(fs_1.default.readFileSync(packageJsonPath).toString());
const relevantPackageJsonSections = {
dependencies: packageJson.dependencies,
devDependencies: packageJson.devDependencies,
optionalDependencies: packageJson.optionalDependencies,
resolutions: packageJson.resolutions,
distro: packageJson.distro
};
shasum.update(JSON.stringify(relevantPackageJsonSections));
const packageLockPath = path_1.default.join(ROOT, dir, 'package-lock.json');
shasum.update(fs_1.default.readFileSync(packageLockPath));
}
// Add any other command line arguments
for (let i = 2; i < process.argv.length; i++) {
shasum.update(process.argv[i]);
}
process.stdout.write(shasum.digest('hex'));
//# sourceMappingURL=computeNodeModulesCacheKey.js.map
================================================
FILE: build/azure-pipelines/common/computeNodeModulesCacheKey.ts
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
import fs from 'fs';
import path from 'path';
import crypto from 'crypto';
const { dirs } = require('../../npm/dirs');
const ROOT = path.join(__dirname, '../../../');
const shasum = crypto.createHash('sha256');
shasum.update(fs.readFileSync(path.join(ROOT, 'build/.cachesalt')));
shasum.update(fs.readFileSync(path.join(ROOT, '.npmrc')));
shasum.update(fs.readFileSync(path.join(ROOT, 'build', '.npmrc')));
shasum.update(fs.readFileSync(path.join(ROOT, 'remote', '.npmrc')));
// Add `package.json` and `package-lock.json` files
for (const dir of dirs) {
const packageJsonPath = path.join(ROOT, dir, 'package.json');
const packageJson = JSON.parse(fs.readFileSync(packageJsonPath).toString());
const relevantPackageJsonSections = {
dependencies: packageJson.dependencies,
devDependencies: packageJson.devDependencies,
optionalDependencies: packageJson.optionalDependencies,
resolutions: packageJson.resolutions,
distro: packageJson.distro
};
shasum.update(JSON.stringify(relevantPackageJsonSections));
const packageLockPath = path.join(ROOT, dir, 'package-lock.json');
shasum.update(fs.readFileSync(packageLockPath));
}
// Add any other command line arguments
for (let i = 2; i < process.argv.length; i++) {
shasum.update(process.argv[i]);
}
process.stdout.write(shasum.digest('hex'));
================================================
FILE: build/azure-pipelines/common/createBuild.js
================================================
"use strict";
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
Object.defineProperty(exports, "__esModule", { value: true });
const identity_1 = require("@azure/identity");
const cosmos_1 = require("@azure/cosmos");
const retry_1 = require("./retry");
if (process.argv.length !== 3) {
console.error('Usage: node createBuild.js VERSION');
process.exit(-1);
}
function getEnv(name) {
const result = process.env[name];
if (typeof result === 'undefined') {
throw new Error('Missing env: ' + name);
}
return result;
}
async function main() {
const [, , _version] = process.argv;
const quality = getEnv('VSCODE_QUALITY');
const commit = getEnv('BUILD_SOURCEVERSION');
const queuedBy = getEnv('BUILD_QUEUEDBY');
const sourceBranch = getEnv('BUILD_SOURCEBRANCH');
const version = _version + (quality === 'stable' ? '' : `-${quality}`);
console.log('Creating build...');
console.log('Quality:', quality);
console.log('Version:', version);
console.log('Commit:', commit);
const build = {
id: commit,
timestamp: (new Date()).getTime(),
version,
isReleased: false,
private: process.env['VSCODE_PRIVATE_BUILD']?.toLowerCase() === 'true',
sourceBranch,
queuedBy,
assets: [],
updates: {}
};
const aadCredentials = new identity_1.ClientAssertionCredential(process.env['AZURE_TENANT_ID'], process.env['AZURE_CLIENT_ID'], () => Promise.resolve(process.env['AZURE_ID_TOKEN']));
const client = new cosmos_1.CosmosClient({ endpoint: process.env['AZURE_DOCUMENTDB_ENDPOINT'], aadCredentials });
const scripts = client.database('builds').container(quality).scripts;
await (0, retry_1.retry)(() => scripts.storedProcedure('createBuild').execute('', [{ ...build, _partitionKey: '' }]));
}
main().then(() => {
console.log('Build successfully created');
process.exit(0);
}, err => {
console.error(err);
process.exit(1);
});
//# sourceMappingURL=createBuild.js.map
================================================
FILE: build/azure-pipelines/common/createBuild.ts
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
import { ClientAssertionCredential } from '@azure/identity';
import { CosmosClient } from '@azure/cosmos';
import { retry } from './retry';
if (process.argv.length !== 3) {
console.error('Usage: node createBuild.js VERSION');
process.exit(-1);
}
function getEnv(name: string): string {
const result = process.env[name];
if (typeof result === 'undefined') {
throw new Error('Missing env: ' + name);
}
return result;
}
async function main(): Promise {
const [, , _version] = process.argv;
const quality = getEnv('VSCODE_QUALITY');
const commit = getEnv('BUILD_SOURCEVERSION');
const queuedBy = getEnv('BUILD_QUEUEDBY');
const sourceBranch = getEnv('BUILD_SOURCEBRANCH');
const version = _version + (quality === 'stable' ? '' : `-${quality}`);
console.log('Creating build...');
console.log('Quality:', quality);
console.log('Version:', version);
console.log('Commit:', commit);
const build = {
id: commit,
timestamp: (new Date()).getTime(),
version,
isReleased: false,
private: process.env['VSCODE_PRIVATE_BUILD']?.toLowerCase() === 'true',
sourceBranch,
queuedBy,
assets: [],
updates: {}
};
const aadCredentials = new ClientAssertionCredential(process.env['AZURE_TENANT_ID']!, process.env['AZURE_CLIENT_ID']!, () => Promise.resolve(process.env['AZURE_ID_TOKEN']!));
const client = new CosmosClient({ endpoint: process.env['AZURE_DOCUMENTDB_ENDPOINT']!, aadCredentials });
const scripts = client.database('builds').container(quality).scripts;
await retry(() => scripts.storedProcedure('createBuild').execute('', [{ ...build, _partitionKey: '' }]));
}
main().then(() => {
console.log('Build successfully created');
process.exit(0);
}, err => {
console.error(err);
process.exit(1);
});
================================================
FILE: build/azure-pipelines/common/extract-telemetry.sh
================================================
#!/usr/bin/env bash
set -e
cd $BUILD_STAGINGDIRECTORY
mkdir extraction
cd extraction
git clone --depth 1 https://github.com/microsoft/vscode-extension-telemetry.git
git clone --depth 1 https://github.com/microsoft/vscode-chrome-debug-core.git
git clone --depth 1 https://github.com/microsoft/vscode-node-debug2.git
git clone --depth 1 https://github.com/microsoft/vscode-node-debug.git
git clone --depth 1 https://github.com/microsoft/vscode-html-languageservice.git
git clone --depth 1 https://github.com/microsoft/vscode-json-languageservice.git
node $BUILD_SOURCESDIRECTORY/node_modules/.bin/vscode-telemetry-extractor --sourceDir $BUILD_SOURCESDIRECTORY --excludedDir $BUILD_SOURCESDIRECTORY/extensions --outputDir . --applyEndpoints
node $BUILD_SOURCESDIRECTORY/node_modules/.bin/vscode-telemetry-extractor --config $BUILD_SOURCESDIRECTORY/build/azure-pipelines/common/telemetry-config.json -o .
mkdir -p $BUILD_SOURCESDIRECTORY/.build/telemetry
mv declarations-resolved.json $BUILD_SOURCESDIRECTORY/.build/telemetry/telemetry-core.json
mv config-resolved.json $BUILD_SOURCESDIRECTORY/.build/telemetry/telemetry-extensions.json
cd ..
rm -rf extraction
================================================
FILE: build/azure-pipelines/common/getPublishAuthTokens.js
================================================
"use strict";
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
Object.defineProperty(exports, "__esModule", { value: true });
exports.getAccessToken = getAccessToken;
const msal_node_1 = require("@azure/msal-node");
function e(name) {
const result = process.env[name];
if (typeof result !== 'string') {
throw new Error(`Missing env: ${name}`);
}
return result;
}
async function getAccessToken(endpoint, tenantId, clientId, idToken) {
const app = new msal_node_1.ConfidentialClientApplication({
auth: {
clientId,
authority: `https://login.microsoftonline.com/${tenantId}`,
clientAssertion: idToken
}
});
const result = await app.acquireTokenByClientCredential({ scopes: [`${endpoint}.default`] });
if (!result) {
throw new Error('Failed to get access token');
}
return {
token: result.accessToken,
expiresOnTimestamp: result.expiresOn.getTime(),
refreshAfterTimestamp: result.refreshOn?.getTime()
};
}
async function main() {
const cosmosDBAccessToken = await getAccessToken(e('AZURE_DOCUMENTDB_ENDPOINT'), e('AZURE_TENANT_ID'), e('AZURE_CLIENT_ID'), e('AZURE_ID_TOKEN'));
const blobServiceAccessToken = await getAccessToken(`https://${e('VSCODE_STAGING_BLOB_STORAGE_ACCOUNT_NAME')}.blob.core.windows.net/`, process.env['AZURE_TENANT_ID'], process.env['AZURE_CLIENT_ID'], process.env['AZURE_ID_TOKEN']);
console.log(JSON.stringify({ cosmosDBAccessToken, blobServiceAccessToken }));
}
if (require.main === module) {
main().then(() => {
process.exit(0);
}, err => {
console.error(err);
process.exit(1);
});
}
//# sourceMappingURL=getPublishAuthTokens.js.map
================================================
FILE: build/azure-pipelines/common/getPublishAuthTokens.ts
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
import { AccessToken } from '@azure/core-auth';
import { ConfidentialClientApplication } from '@azure/msal-node';
function e(name: string): string {
const result = process.env[name];
if (typeof result !== 'string') {
throw new Error(`Missing env: ${name}`);
}
return result;
}
export async function getAccessToken(endpoint: string, tenantId: string, clientId: string, idToken: string): Promise {
const app = new ConfidentialClientApplication({
auth: {
clientId,
authority: `https://login.microsoftonline.com/${tenantId}`,
clientAssertion: idToken
}
});
const result = await app.acquireTokenByClientCredential({ scopes: [`${endpoint}.default`] });
if (!result) {
throw new Error('Failed to get access token');
}
return {
token: result.accessToken,
expiresOnTimestamp: result.expiresOn!.getTime(),
refreshAfterTimestamp: result.refreshOn?.getTime()
};
}
async function main() {
const cosmosDBAccessToken = await getAccessToken(e('AZURE_DOCUMENTDB_ENDPOINT')!, e('AZURE_TENANT_ID')!, e('AZURE_CLIENT_ID')!, e('AZURE_ID_TOKEN')!);
const blobServiceAccessToken = await getAccessToken(`https://${e('VSCODE_STAGING_BLOB_STORAGE_ACCOUNT_NAME')}.blob.core.windows.net/`, process.env['AZURE_TENANT_ID']!, process.env['AZURE_CLIENT_ID']!, process.env['AZURE_ID_TOKEN']!);
console.log(JSON.stringify({ cosmosDBAccessToken, blobServiceAccessToken }));
}
if (require.main === module) {
main().then(() => {
process.exit(0);
}, err => {
console.error(err);
process.exit(1);
});
}
================================================
FILE: build/azure-pipelines/common/install-builtin-extensions.yml
================================================
steps:
- pwsh: mkdir .build -ea 0
condition: and(succeeded(), contains(variables['Agent.OS'], 'windows'))
displayName: Create .build folder
- script: mkdir -p .build
condition: and(succeeded(), not(contains(variables['Agent.OS'], 'windows')))
displayName: Create .build folder
- script: node build/azure-pipelines/common/computeBuiltInDepsCacheKey.js > .build/builtindepshash
displayName: Prepare built-in extensions cache key
- task: Cache@2
inputs:
key: '"builtin-extensions" | .build/builtindepshash'
path: .build/builtInExtensions
cacheHitVar: BUILTIN_EXTENSIONS_RESTORED
displayName: Restore built-in extensions cache
- script: node build/lib/builtInExtensions.js
env:
GITHUB_TOKEN: "$(github-distro-mixin-password)"
condition: and(succeeded(), ne(variables.BUILTIN_EXTENSIONS_RESTORED, 'true'))
displayName: Download built-in extensions
================================================
FILE: build/azure-pipelines/common/installPlaywright.js
================================================
"use strict";
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
process.env.DEBUG = 'pw:install'; // enable logging for this (https://github.com/microsoft/playwright/issues/17394)
const { installDefaultBrowsersForNpmInstall } = require('playwright-core/lib/server');
async function install() {
await installDefaultBrowsersForNpmInstall();
}
install();
//# sourceMappingURL=installPlaywright.js.map
================================================
FILE: build/azure-pipelines/common/listNodeModules.js
================================================
"use strict";
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
Object.defineProperty(exports, "__esModule", { value: true });
const fs_1 = __importDefault(require("fs"));
const path_1 = __importDefault(require("path"));
if (process.argv.length !== 3) {
console.error('Usage: node listNodeModules.js OUTPUT_FILE');
process.exit(-1);
}
const ROOT = path_1.default.join(__dirname, '../../../');
function findNodeModulesFiles(location, inNodeModules, result) {
const entries = fs_1.default.readdirSync(path_1.default.join(ROOT, location));
for (const entry of entries) {
const entryPath = `${location}/${entry}`;
if (/(^\/out)|(^\/src$)|(^\/.git$)|(^\/.build$)/.test(entryPath)) {
continue;
}
let stat;
try {
stat = fs_1.default.statSync(path_1.default.join(ROOT, entryPath));
}
catch (err) {
continue;
}
if (stat.isDirectory()) {
findNodeModulesFiles(entryPath, inNodeModules || (entry === 'node_modules'), result);
}
else {
if (inNodeModules) {
result.push(entryPath.substr(1));
}
}
}
}
const result = [];
findNodeModulesFiles('', false, result);
fs_1.default.writeFileSync(process.argv[2], result.join('\n') + '\n');
//# sourceMappingURL=listNodeModules.js.map
================================================
FILE: build/azure-pipelines/common/listNodeModules.ts
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
import fs from 'fs';
import path from 'path';
if (process.argv.length !== 3) {
console.error('Usage: node listNodeModules.js OUTPUT_FILE');
process.exit(-1);
}
const ROOT = path.join(__dirname, '../../../');
function findNodeModulesFiles(location: string, inNodeModules: boolean, result: string[]) {
const entries = fs.readdirSync(path.join(ROOT, location));
for (const entry of entries) {
const entryPath = `${location}/${entry}`;
if (/(^\/out)|(^\/src$)|(^\/.git$)|(^\/.build$)/.test(entryPath)) {
continue;
}
let stat: fs.Stats;
try {
stat = fs.statSync(path.join(ROOT, entryPath));
} catch (err) {
continue;
}
if (stat.isDirectory()) {
findNodeModulesFiles(entryPath, inNodeModules || (entry === 'node_modules'), result);
} else {
if (inNodeModules) {
result.push(entryPath.substr(1));
}
}
}
}
const result: string[] = [];
findNodeModulesFiles('', false, result);
fs.writeFileSync(process.argv[2], result.join('\n') + '\n');
================================================
FILE: build/azure-pipelines/common/publish.js
================================================
"use strict";
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
Object.defineProperty(exports, "__esModule", { value: true });
const fs_1 = __importDefault(require("fs"));
const path_1 = __importDefault(require("path"));
const stream_1 = require("stream");
const promises_1 = require("node:stream/promises");
const yauzl_1 = __importDefault(require("yauzl"));
const crypto_1 = __importDefault(require("crypto"));
const retry_1 = require("./retry");
const cosmos_1 = require("@azure/cosmos");
const child_process_1 = __importDefault(require("child_process"));
const os_1 = __importDefault(require("os"));
const node_worker_threads_1 = require("node:worker_threads");
const msal_node_1 = require("@azure/msal-node");
const storage_blob_1 = require("@azure/storage-blob");
const jws_1 = __importDefault(require("jws"));
const node_timers_1 = require("node:timers");
function e(name) {
const result = process.env[name];
if (typeof result !== 'string') {
throw new Error(`Missing env: ${name}`);
}
return result;
}
function hashStream(hashName, stream) {
return new Promise((c, e) => {
const shasum = crypto_1.default.createHash(hashName);
stream
.on('data', shasum.update.bind(shasum))
.on('error', e)
.on('close', () => c(shasum.digest()));
});
}
var StatusCode;
(function (StatusCode) {
StatusCode["Pass"] = "pass";
StatusCode["Aborted"] = "aborted";
StatusCode["Inprogress"] = "inprogress";
StatusCode["FailCanRetry"] = "failCanRetry";
StatusCode["FailDoNotRetry"] = "failDoNotRetry";
StatusCode["PendingAnalysis"] = "pendingAnalysis";
StatusCode["Cancelled"] = "cancelled";
})(StatusCode || (StatusCode = {}));
function getCertificateBuffer(input) {
return Buffer.from(input.replace(/-----BEGIN CERTIFICATE-----|-----END CERTIFICATE-----|\n/g, ''), 'base64');
}
function getThumbprint(input, algorithm) {
const buffer = getCertificateBuffer(input);
return crypto_1.default.createHash(algorithm).update(buffer).digest();
}
function getKeyFromPFX(pfx) {
const pfxCertificatePath = path_1.default.join(os_1.default.tmpdir(), 'cert.pfx');
const pemKeyPath = path_1.default.join(os_1.default.tmpdir(), 'key.pem');
try {
const pfxCertificate = Buffer.from(pfx, 'base64');
fs_1.default.writeFileSync(pfxCertificatePath, pfxCertificate);
child_process_1.default.execSync(`openssl pkcs12 -in "${pfxCertificatePath}" -nocerts -nodes -out "${pemKeyPath}" -passin pass:`);
const raw = fs_1.default.readFileSync(pemKeyPath, 'utf-8');
const result = raw.match(/-----BEGIN PRIVATE KEY-----[\s\S]+?-----END PRIVATE KEY-----/g)[0];
return result;
}
finally {
fs_1.default.rmSync(pfxCertificatePath, { force: true });
fs_1.default.rmSync(pemKeyPath, { force: true });
}
}
function getCertificatesFromPFX(pfx) {
const pfxCertificatePath = path_1.default.join(os_1.default.tmpdir(), 'cert.pfx');
const pemCertificatePath = path_1.default.join(os_1.default.tmpdir(), 'cert.pem');
try {
const pfxCertificate = Buffer.from(pfx, 'base64');
fs_1.default.writeFileSync(pfxCertificatePath, pfxCertificate);
child_process_1.default.execSync(`openssl pkcs12 -in "${pfxCertificatePath}" -nokeys -out "${pemCertificatePath}" -passin pass:`);
const raw = fs_1.default.readFileSync(pemCertificatePath, 'utf-8');
const matches = raw.match(/-----BEGIN CERTIFICATE-----[\s\S]+?-----END CERTIFICATE-----/g);
return matches ? matches.reverse() : [];
}
finally {
fs_1.default.rmSync(pfxCertificatePath, { force: true });
fs_1.default.rmSync(pemCertificatePath, { force: true });
}
}
class ESRPReleaseService {
log;
clientId;
accessToken;
requestSigningCertificates;
requestSigningKey;
containerClient;
stagingSasToken;
static async create(log, tenantId, clientId, authCertificatePfx, requestSigningCertificatePfx, containerClient, stagingSasToken) {
const authKey = getKeyFromPFX(authCertificatePfx);
const authCertificate = getCertificatesFromPFX(authCertificatePfx)[0];
const requestSigningKey = getKeyFromPFX(requestSigningCertificatePfx);
const requestSigningCertificates = getCertificatesFromPFX(requestSigningCertificatePfx);
const app = new msal_node_1.ConfidentialClientApplication({
auth: {
clientId,
authority: `https://login.microsoftonline.com/${tenantId}`,
clientCertificate: {
thumbprintSha256: getThumbprint(authCertificate, 'sha256').toString('hex'),
privateKey: authKey,
x5c: authCertificate
}
}
});
const response = await app.acquireTokenByClientCredential({
scopes: ['https://api.esrp.microsoft.com/.default']
});
return new ESRPReleaseService(log, clientId, response.accessToken, requestSigningCertificates, requestSigningKey, containerClient, stagingSasToken);
}
static API_URL = 'https://api.esrp.microsoft.com/api/v3/releaseservices/clients/';
constructor(log, clientId, accessToken, requestSigningCertificates, requestSigningKey, containerClient, stagingSasToken) {
this.log = log;
this.clientId = clientId;
this.accessToken = accessToken;
this.requestSigningCertificates = requestSigningCertificates;
this.requestSigningKey = requestSigningKey;
this.containerClient = containerClient;
this.stagingSasToken = stagingSasToken;
}
async createRelease(version, filePath, friendlyFileName) {
const correlationId = crypto_1.default.randomUUID();
const blobClient = this.containerClient.getBlockBlobClient(correlationId);
this.log(`Uploading ${filePath} to ${blobClient.url}`);
await blobClient.uploadFile(filePath);
this.log('Uploaded blob successfully');
try {
this.log(`Submitting release for ${version}: ${filePath}`);
const submitReleaseResult = await this.submitRelease(version, filePath, friendlyFileName, correlationId, blobClient);
this.log(`Successfully submitted release ${submitReleaseResult.operationId}. Polling for completion...`);
// Poll every 5 seconds, wait 60 minutes max -> poll 60/5*60=720 times
for (let i = 0; i < 720; i++) {
await new Promise(c => setTimeout(c, 5000));
const releaseStatus = await this.getReleaseStatus(submitReleaseResult.operationId);
if (releaseStatus.status === 'pass') {
break;
}
else if (releaseStatus.status === 'aborted') {
this.log(JSON.stringify(releaseStatus));
throw new Error(`Release was aborted`);
}
else if (releaseStatus.status !== 'inprogress') {
this.log(JSON.stringify(releaseStatus));
throw new Error(`Unknown error when polling for release`);
}
}
const releaseDetails = await this.getReleaseDetails(submitReleaseResult.operationId);
if (releaseDetails.status !== 'pass') {
throw new Error(`Timed out waiting for release: ${JSON.stringify(releaseDetails)}`);
}
this.log('Successfully created release:', releaseDetails.files[0].fileDownloadDetails[0].downloadUrl);
return releaseDetails.files[0].fileDownloadDetails[0].downloadUrl;
}
finally {
this.log(`Deleting blob ${blobClient.url}`);
await blobClient.delete();
this.log('Deleted blob successfully');
}
}
async submitRelease(version, filePath, friendlyFileName, correlationId, blobClient) {
const size = fs_1.default.statSync(filePath).size;
const hash = await hashStream('sha256', fs_1.default.createReadStream(filePath));
const blobUrl = `${blobClient.url}?${this.stagingSasToken}`;
const message = {
customerCorrelationId: correlationId,
esrpCorrelationId: correlationId,
driEmail: ['joao.moreno@microsoft.com'],
createdBy: { userPrincipalName: 'jomo@microsoft.com' },
owners: [{ owner: { userPrincipalName: 'jomo@microsoft.com' } }],
approvers: [{ approver: { userPrincipalName: 'jomo@microsoft.com' }, isAutoApproved: true, isMandatory: false }],
releaseInfo: {
title: 'VS Code',
properties: {
'ReleaseContentType': 'InstallPackage'
},
minimumNumberOfApprovers: 1
},
productInfo: {
name: 'VS Code',
version,
description: 'VS Code'
},
accessPermissionsInfo: {
mainPublisher: 'VSCode',
channelDownloadEntityDetails: {
AllDownloadEntities: ['VSCode']
}
},
routingInfo: {
intent: 'filedownloadlinkgeneration'
},
files: [{
name: path_1.default.basename(filePath),
friendlyFileName,
tenantFileLocation: blobUrl,
tenantFileLocationType: 'AzureBlob',
sourceLocation: {
type: 'azureBlob',
blobUrl
},
hashType: 'sha256',
hash: Array.from(hash),
sizeInBytes: size
}]
};
message.jwsToken = await this.generateJwsToken(message);
const res = await fetch(`${ESRPReleaseService.API_URL}${this.clientId}/workflows/release/operations`, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Authorization': `Bearer ${this.accessToken}`
},
body: JSON.stringify(message)
});
if (!res.ok) {
const text = await res.text();
throw new Error(`Failed to submit release: ${res.statusText}\n${text}`);
}
return await res.json();
}
async getReleaseStatus(releaseId) {
const url = `${ESRPReleaseService.API_URL}${this.clientId}/workflows/release/operations/grs/${releaseId}`;
const res = await fetch(url, {
headers: {
'Authorization': `Bearer ${this.accessToken}`
}
});
if (!res.ok) {
const text = await res.text();
throw new Error(`Failed to get release status: ${res.statusText}\n${text}`);
}
return await res.json();
}
async getReleaseDetails(releaseId) {
const url = `${ESRPReleaseService.API_URL}${this.clientId}/workflows/release/operations/grd/${releaseId}`;
const res = await fetch(url, {
headers: {
'Authorization': `Bearer ${this.accessToken}`
}
});
if (!res.ok) {
const text = await res.text();
throw new Error(`Failed to get release status: ${res.statusText}\n${text}`);
}
return await res.json();
}
async generateJwsToken(message) {
return jws_1.default.sign({
header: {
alg: 'RS256',
crit: ['exp', 'x5t'],
// Release service uses ticks, not seconds :roll_eyes: (https://stackoverflow.com/a/7968483)
exp: ((Date.now() + (6 * 60 * 1000)) * 10000) + 621355968000000000,
// Release service uses hex format, not base64url :roll_eyes:
x5t: getThumbprint(this.requestSigningCertificates[0], 'sha1').toString('hex'),
// Release service uses a '.' separated string, not an array of strings :roll_eyes:
x5c: this.requestSigningCertificates.map(c => getCertificateBuffer(c).toString('base64url')).join('.'),
},
payload: message,
privateKey: this.requestSigningKey,
});
}
}
class State {
statePath;
set = new Set();
constructor() {
const pipelineWorkspacePath = e('PIPELINE_WORKSPACE');
const previousState = fs_1.default.readdirSync(pipelineWorkspacePath)
.map(name => /^artifacts_processed_(\d+)$/.exec(name))
.filter((match) => !!match)
.map(match => ({ name: match[0], attempt: Number(match[1]) }))
.sort((a, b) => b.attempt - a.attempt)[0];
if (previousState) {
const previousStatePath = path_1.default.join(pipelineWorkspacePath, previousState.name, previousState.name + '.txt');
fs_1.default.readFileSync(previousStatePath, 'utf8').split(/\n/).filter(name => !!name).forEach(name => this.set.add(name));
}
const stageAttempt = e('SYSTEM_STAGEATTEMPT');
this.statePath = path_1.default.join(pipelineWorkspacePath, `artifacts_processed_${stageAttempt}`, `artifacts_processed_${stageAttempt}.txt`);
fs_1.default.mkdirSync(path_1.default.dirname(this.statePath), { recursive: true });
fs_1.default.writeFileSync(this.statePath, [...this.set.values()].map(name => `${name}\n`).join(''));
}
get size() {
return this.set.size;
}
has(name) {
return this.set.has(name);
}
add(name) {
this.set.add(name);
fs_1.default.appendFileSync(this.statePath, `${name}\n`);
}
[Symbol.iterator]() {
return this.set[Symbol.iterator]();
}
}
const azdoFetchOptions = {
headers: {
// Pretend we're a web browser to avoid download rate limits
'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36 Edg/119.0.0.0',
'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7',
'Accept-Encoding': 'gzip, deflate, br',
'Accept-Language': 'en-US,en;q=0.9',
'Referer': 'https://dev.azure.com',
Authorization: `Bearer ${e('SYSTEM_ACCESSTOKEN')}`
}
};
async function requestAZDOAPI(path) {
const abortController = new AbortController();
const timeout = setTimeout(() => abortController.abort(), 2 * 60 * 1000);
try {
const res = await fetch(`${e('BUILDS_API_URL')}${path}?api-version=6.0`, { ...azdoFetchOptions, signal: abortController.signal });
if (!res.ok) {
throw new Error(`Unexpected status code: ${res.status}`);
}
return await res.json();
}
finally {
clearTimeout(timeout);
}
}
async function getPipelineArtifacts() {
const result = await requestAZDOAPI('artifacts');
return result.value.filter(a => /^vscode_/.test(a.name) && !/sbom$/.test(a.name));
}
async function getPipelineTimeline() {
return await requestAZDOAPI('timeline');
}
async function downloadArtifact(artifact, downloadPath) {
const abortController = new AbortController();
const timeout = setTimeout(() => abortController.abort(), 4 * 60 * 1000);
try {
const res = await fetch(artifact.resource.downloadUrl, { ...azdoFetchOptions, signal: abortController.signal });
if (!res.ok) {
throw new Error(`Unexpected status code: ${res.status}`);
}
await (0, promises_1.pipeline)(stream_1.Readable.fromWeb(res.body), fs_1.default.createWriteStream(downloadPath));
}
finally {
clearTimeout(timeout);
}
}
async function unzip(packagePath, outputPath) {
return new Promise((resolve, reject) => {
yauzl_1.default.open(packagePath, { lazyEntries: true, autoClose: true }, (err, zipfile) => {
if (err) {
return reject(err);
}
const result = [];
zipfile.on('entry', entry => {
if (/\/$/.test(entry.fileName)) {
zipfile.readEntry();
}
else {
zipfile.openReadStream(entry, (err, istream) => {
if (err) {
return reject(err);
}
const filePath = path_1.default.join(outputPath, entry.fileName);
fs_1.default.mkdirSync(path_1.default.dirname(filePath), { recursive: true });
const ostream = fs_1.default.createWriteStream(filePath);
ostream.on('finish', () => {
result.push(filePath);
zipfile.readEntry();
});
istream?.on('error', err => reject(err));
istream.pipe(ostream);
});
}
});
zipfile.on('close', () => resolve(result));
zipfile.readEntry();
});
});
}
// Contains all of the logic for mapping details to our actual product names in CosmosDB
function getPlatform(product, os, arch, type) {
switch (os) {
case 'win32':
switch (product) {
case 'client': {
switch (type) {
case 'archive':
return `win32-${arch}-archive`;
case 'setup':
return `win32-${arch}`;
case 'user-setup':
return `win32-${arch}-user`;
default:
throw new Error(`Unrecognized: ${product} ${os} ${arch} ${type}`);
}
}
case 'server':
return `server-win32-${arch}`;
case 'web':
return `server-win32-${arch}-web`;
case 'cli':
return `cli-win32-${arch}`;
default:
throw new Error(`Unrecognized: ${product} ${os} ${arch} ${type}`);
}
case 'alpine':
switch (product) {
case 'server':
return `server-alpine-${arch}`;
case 'web':
return `server-alpine-${arch}-web`;
case 'cli':
return `cli-alpine-${arch}`;
default:
throw new Error(`Unrecognized: ${product} ${os} ${arch} ${type}`);
}
case 'linux':
switch (type) {
case 'snap':
return `linux-snap-${arch}`;
case 'archive-unsigned':
switch (product) {
case 'client':
return `linux-${arch}`;
case 'server':
return `server-linux-${arch}`;
case 'web':
if (arch === 'standalone') {
return 'web-standalone';
}
return `server-linux-${arch}-web`;
default:
throw new Error(`Unrecognized: ${product} ${os} ${arch} ${type}`);
}
case 'deb-package':
return `linux-deb-${arch}`;
case 'rpm-package':
return `linux-rpm-${arch}`;
case 'cli':
return `cli-linux-${arch}`;
default:
throw new Error(`Unrecognized: ${product} ${os} ${arch} ${type}`);
}
case 'darwin':
switch (product) {
case 'client':
if (arch === 'x64') {
return 'darwin';
}
return `darwin-${arch}`;
case 'server':
if (arch === 'x64') {
return 'server-darwin';
}
return `server-darwin-${arch}`;
case 'web':
if (arch === 'x64') {
return 'server-darwin-web';
}
return `server-darwin-${arch}-web`;
case 'cli':
return `cli-darwin-${arch}`;
default:
throw new Error(`Unrecognized: ${product} ${os} ${arch} ${type}`);
}
default:
throw new Error(`Unrecognized: ${product} ${os} ${arch} ${type}`);
}
}
// Contains all of the logic for mapping types to our actual types in CosmosDB
function getRealType(type) {
switch (type) {
case 'user-setup':
return 'setup';
case 'deb-package':
case 'rpm-package':
return 'package';
default:
return type;
}
}
async function withLease(client, fn) {
const lease = client.getBlobLeaseClient();
for (let i = 0; i < 360; i++) { // Try to get lease for 30 minutes
try {
await client.uploadData(new ArrayBuffer()); // blob needs to exist for lease to be acquired
await lease.acquireLease(60);
try {
const abortController = new AbortController();
const refresher = new Promise((c, e) => {
abortController.signal.onabort = () => {
(0, node_timers_1.clearInterval)(interval);
c();
};
const interval = (0, node_timers_1.setInterval)(() => {
lease.renewLease().catch(err => {
(0, node_timers_1.clearInterval)(interval);
e(new Error('Failed to renew lease ' + err));
});
}, 30_000);
});
const result = await Promise.race([fn(), refresher]);
abortController.abort();
return result;
}
finally {
await lease.releaseLease();
}
}
catch (err) {
if (err.statusCode !== 409 && err.statusCode !== 412) {
throw err;
}
await new Promise(c => setTimeout(c, 5000));
}
}
throw new Error('Failed to acquire lease on blob after 30 minutes');
}
async function processArtifact(artifact, filePath) {
const log = (...args) => console.log(`[${artifact.name}]`, ...args);
const match = /^vscode_(?[^_]+)_(?[^_]+)(?:_legacy)?_(?[^_]+)_(?[^_]+)$/.exec(artifact.name);
if (!match) {
throw new Error(`Invalid artifact name: ${artifact.name}`);
}
const { cosmosDBAccessToken, blobServiceAccessToken } = JSON.parse(e('PUBLISH_AUTH_TOKENS'));
const quality = e('VSCODE_QUALITY');
const version = e('BUILD_SOURCEVERSION');
const friendlyFileName = `${quality}/${version}/${path_1.default.basename(filePath)}`;
const blobServiceClient = new storage_blob_1.BlobServiceClient(`https://${e('VSCODE_STAGING_BLOB_STORAGE_ACCOUNT_NAME')}.blob.core.windows.net/`, { getToken: async () => blobServiceAccessToken });
const leasesContainerClient = blobServiceClient.getContainerClient('leases');
await leasesContainerClient.createIfNotExists();
const leaseBlobClient = leasesContainerClient.getBlockBlobClient(friendlyFileName);
log(`Acquiring lease for: ${friendlyFileName}`);
await withLease(leaseBlobClient, async () => {
log(`Successfully acquired lease for: ${friendlyFileName}`);
const url = `${e('PRSS_CDN_URL')}/${friendlyFileName}`;
const res = await (0, retry_1.retry)(() => fetch(url));
if (res.status === 200) {
log(`Already released and provisioned: ${url}`);
}
else {
const stagingContainerClient = blobServiceClient.getContainerClient('staging');
await stagingContainerClient.createIfNotExists();
const now = new Date().valueOf();
const oneHour = 60 * 60 * 1000;
const oneHourAgo = new Date(now - oneHour);
const oneHourFromNow = new Date(now + oneHour);
const userDelegationKey = await blobServiceClient.getUserDelegationKey(oneHourAgo, oneHourFromNow);
const sasOptions = { containerName: 'staging', permissions: storage_blob_1.ContainerSASPermissions.from({ read: true }), startsOn: oneHourAgo, expiresOn: oneHourFromNow };
const stagingSasToken = (0, storage_blob_1.generateBlobSASQueryParameters)(sasOptions, userDelegationKey, e('VSCODE_STAGING_BLOB_STORAGE_ACCOUNT_NAME')).toString();
const releaseService = await ESRPReleaseService.create(log, e('RELEASE_TENANT_ID'), e('RELEASE_CLIENT_ID'), e('RELEASE_AUTH_CERT'), e('RELEASE_REQUEST_SIGNING_CERT'), stagingContainerClient, stagingSasToken);
await releaseService.createRelease(version, filePath, friendlyFileName);
}
const { product, os, arch, unprocessedType } = match.groups;
const platform = getPlatform(product, os, arch, unprocessedType);
const type = getRealType(unprocessedType);
const size = fs_1.default.statSync(filePath).size;
const stream = fs_1.default.createReadStream(filePath);
const [hash, sha256hash] = await Promise.all([hashStream('sha1', stream), hashStream('sha256', stream)]); // CodeQL [SM04514] Using SHA1 only for legacy reasons, we are actually only respecting SHA256
const asset = { platform, type, url, hash: hash.toString('hex'), sha256hash: sha256hash.toString('hex'), size, supportsFastUpdate: true };
log('Creating asset...');
const result = await (0, retry_1.retry)(async (attempt) => {
log(`Creating asset in Cosmos DB (attempt ${attempt})...`);
const client = new cosmos_1.CosmosClient({ endpoint: e('AZURE_DOCUMENTDB_ENDPOINT'), tokenProvider: () => Promise.resolve(`type=aad&ver=1.0&sig=${cosmosDBAccessToken.token}`) });
const scripts = client.database('builds').container(quality).scripts;
const { resource: result } = await scripts.storedProcedure('createAsset').execute('', [version, asset, true]);
return result;
});
if (result === 'already exists') {
log('Asset already exists!');
}
else {
log('Asset successfully created: ', JSON.stringify(asset, undefined, 2));
}
});
log(`Successfully released lease for: ${friendlyFileName}`);
}
// It is VERY important that we don't download artifacts too much too fast from AZDO.
// AZDO throttles us SEVERELY if we do. Not just that, but they also close open
// sockets, so the whole things turns to a grinding halt. So, downloading and extracting
// happens serially in the main thread, making the downloads are spaced out
// properly. For each extracted artifact, we spawn a worker thread to upload it to
// the CDN and finally update the build in Cosmos DB.
async function main() {
if (!node_worker_threads_1.isMainThread) {
const { artifact, artifactFilePath } = node_worker_threads_1.workerData;
await processArtifact(artifact, artifactFilePath);
return;
}
const done = new State();
const processing = new Set();
for (const name of done) {
console.log(`\u2705 ${name}`);
}
const stages = new Set(['Compile']);
if (e('VSCODE_BUILD_STAGE_LINUX') === 'True' ||
e('VSCODE_BUILD_STAGE_ALPINE') === 'True' ||
e('VSCODE_BUILD_STAGE_MACOS') === 'True' ||
e('VSCODE_BUILD_STAGE_WINDOWS') === 'True') {
stages.add('CompileCLI');
}
if (e('VSCODE_BUILD_STAGE_WINDOWS') === 'True') {
stages.add('Windows');
}
if (e('VSCODE_BUILD_STAGE_LINUX') === 'True') {
stages.add('Linux');
}
if (e('VSCODE_BUILD_STAGE_ALPINE') === 'True') {
stages.add('Alpine');
}
if (e('VSCODE_BUILD_STAGE_MACOS') === 'True') {
stages.add('macOS');
}
if (e('VSCODE_BUILD_STAGE_WEB') === 'True') {
stages.add('Web');
}
let resultPromise = Promise.resolve([]);
const operations = [];
while (true) {
const [timeline, artifacts] = await Promise.all([(0, retry_1.retry)(() => getPipelineTimeline()), (0, retry_1.retry)(() => getPipelineArtifacts())]);
const stagesCompleted = new Set(timeline.records.filter(r => r.type === 'Stage' && r.state === 'completed' && stages.has(r.name)).map(r => r.name));
const stagesInProgress = [...stages].filter(s => !stagesCompleted.has(s));
const artifactsInProgress = artifacts.filter(a => processing.has(a.name));
if (stagesInProgress.length === 0 && artifacts.length === done.size + processing.size) {
break;
}
else if (stagesInProgress.length > 0) {
console.log('Stages in progress:', stagesInProgress.join(', '));
}
else if (artifactsInProgress.length > 0) {
console.log('Artifacts in progress:', artifactsInProgress.map(a => a.name).join(', '));
}
else {
console.log(`Waiting for a total of ${artifacts.length}, ${done.size} done, ${processing.size} in progress...`);
}
for (const artifact of artifacts) {
if (done.has(artifact.name) || processing.has(artifact.name)) {
continue;
}
console.log(`[${artifact.name}] Found new artifact`);
const artifactZipPath = path_1.default.join(e('AGENT_TEMPDIRECTORY'), `${artifact.name}.zip`);
await (0, retry_1.retry)(async (attempt) => {
const start = Date.now();
console.log(`[${artifact.name}] Downloading (attempt ${attempt})...`);
await downloadArtifact(artifact, artifactZipPath);
const archiveSize = fs_1.default.statSync(artifactZipPath).size;
const downloadDurationS = (Date.now() - start) / 1000;
const downloadSpeedKBS = Math.round((archiveSize / 1024) / downloadDurationS);
console.log(`[${artifact.name}] Successfully downloaded after ${Math.floor(downloadDurationS)} seconds(${downloadSpeedKBS} KB/s).`);
});
const artifactFilePaths = await unzip(artifactZipPath, e('AGENT_TEMPDIRECTORY'));
const artifactFilePath = artifactFilePaths.filter(p => !/_manifest/.test(p))[0];
processing.add(artifact.name);
const promise = new Promise((resolve, reject) => {
const worker = new node_worker_threads_1.Worker(__filename, { workerData: { artifact, artifactFilePath } });
worker.on('error', reject);
worker.on('exit', code => {
if (code === 0) {
resolve();
}
else {
reject(new Error(`[${artifact.name}] Worker stopped with exit code ${code}`));
}
});
});
const operation = promise.then(() => {
processing.delete(artifact.name);
done.add(artifact.name);
console.log(`\u2705 ${artifact.name} `);
});
operations.push({ name: artifact.name, operation });
resultPromise = Promise.allSettled(operations.map(o => o.operation));
}
await new Promise(c => setTimeout(c, 10_000));
}
console.log(`Found all ${done.size + processing.size} artifacts, waiting for ${processing.size} artifacts to finish publishing...`);
const artifactsInProgress = operations.filter(o => processing.has(o.name));
if (artifactsInProgress.length > 0) {
console.log('Artifacts in progress:', artifactsInProgress.map(a => a.name).join(', '));
}
const results = await resultPromise;
for (let i = 0; i < operations.length; i++) {
const result = results[i];
if (result.status === 'rejected') {
console.error(`[${operations[i].name}]`, result.reason);
}
}
if (results.some(r => r.status === 'rejected')) {
throw new Error('Some artifacts failed to publish');
}
console.log(`All ${done.size} artifacts published!`);
}
if (require.main === module) {
main().then(() => {
process.exit(0);
}, err => {
console.error(err);
process.exit(1);
});
}
//# sourceMappingURL=publish.js.map
================================================
FILE: build/azure-pipelines/common/publish.ts
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
import fs from 'fs';
import path from 'path';
import { Readable } from 'stream';
import type { ReadableStream } from 'stream/web';
import { pipeline } from 'node:stream/promises';
import yauzl from 'yauzl';
import crypto from 'crypto';
import { retry } from './retry';
import { CosmosClient } from '@azure/cosmos';
import cp from 'child_process';
import os from 'os';
import { Worker, isMainThread, workerData } from 'node:worker_threads';
import { ConfidentialClientApplication } from '@azure/msal-node';
import { BlobClient, BlobServiceClient, BlockBlobClient, ContainerClient, ContainerSASPermissions, generateBlobSASQueryParameters } from '@azure/storage-blob';
import jws from 'jws';
import { clearInterval, setInterval } from 'node:timers';
function e(name: string): string {
const result = process.env[name];
if (typeof result !== 'string') {
throw new Error(`Missing env: ${name}`);
}
return result;
}
function hashStream(hashName: string, stream: Readable): Promise {
return new Promise((c, e) => {
const shasum = crypto.createHash(hashName);
stream
.on('data', shasum.update.bind(shasum))
.on('error', e)
.on('close', () => c(shasum.digest()));
});
}
interface ReleaseSubmitResponse {
operationId: string;
esrpCorrelationId: string;
code?: string;
message?: string;
target?: string;
innerError?: any;
}
interface ReleaseActivityInfo {
activityId: string;
activityType: string;
name: string;
status: string;
errorCode: number;
errorMessages: string[];
beginTime?: Date;
endTime?: Date;
lastModifiedAt?: Date;
}
interface InnerServiceError {
code: string;
details: { [key: string]: string };
innerError?: InnerServiceError;
}
interface ReleaseError {
errorCode: number;
errorMessages: string[];
}
const enum StatusCode {
Pass = 'pass',
Aborted = 'aborted',
Inprogress = 'inprogress',
FailCanRetry = 'failCanRetry',
FailDoNotRetry = 'failDoNotRetry',
PendingAnalysis = 'pendingAnalysis',
Cancelled = 'cancelled'
}
interface ReleaseResultMessage {
activities: ReleaseActivityInfo[];
childWorkflowType: string;
clientId: string;
customerCorrelationId: string;
errorInfo: InnerServiceError;
groupId: string;
lastModifiedAt: Date;
operationId: string;
releaseError: ReleaseError;
requestSubmittedAt: Date;
routedRegion: string;
status: StatusCode;
totalFileCount: number;
totalReleaseSize: number;
version: string;
}
interface ReleaseFileInfo {
name?: string;
hash?: number[];
sourceLocation?: FileLocation;
sizeInBytes?: number;
hashType?: FileHashType;
fileId?: any;
distributionRelativePath?: string;
partNumber?: string;
friendlyFileName?: string;
tenantFileLocationType?: string;
tenantFileLocation?: string;
signedEngineeringCopyLocation?: string;
encryptedDistributionBlobLocation?: string;
preEncryptedDistributionBlobLocation?: string;
secondaryDistributionHashRequired?: boolean;
secondaryDistributionHashType?: FileHashType;
lastModifiedAt?: Date;
cultureCodes?: string[];
displayFileInDownloadCenter?: boolean;
isPrimaryFileInDownloadCenter?: boolean;
fileDownloadDetails?: FileDownloadDetails[];
}
interface ReleaseDetailsFileInfo extends ReleaseFileInfo { }
interface ReleaseDetailsMessage extends ReleaseResultMessage {
clusterRegion: string;
correlationVector: string;
releaseCompletedAt?: Date;
releaseInfo: ReleaseInfo;
productInfo: ProductInfo;
createdBy: UserInfo;
owners: OwnerInfo[];
accessPermissionsInfo: AccessPermissionsInfo;
files: ReleaseDetailsFileInfo[];
comments: string[];
cancellationReason: string;
downloadCenterInfo: DownloadCenterInfo;
}
interface ProductInfo {
name?: string;
version?: string;
description?: string;
}
interface ReleaseInfo {
title?: string;
minimumNumberOfApprovers: number;
properties?: { [key: string]: string };
isRevision?: boolean;
revisionNumber?: string;
}
type FileLocationType = 'azureBlob';
interface FileLocation {
type: FileLocationType;
blobUrl: string;
uncPath?: string;
url?: string;
}
type FileHashType = 'sha256' | 'sha1';
interface FileDownloadDetails {
portalName: string;
downloadUrl: string;
}
interface RoutingInfo {
intent?: string;
contentType?: string;
contentOrigin?: string;
productState?: string;
audience?: string;
}
interface ReleaseFileInfo {
name?: string;
hash?: number[];
sourceLocation?: FileLocation;
sizeInBytes?: number;
hashType?: FileHashType;
fileId?: any;
distributionRelativePath?: string;
partNumber?: string;
friendlyFileName?: string;
tenantFileLocationType?: string;
tenantFileLocation?: string;
signedEngineeringCopyLocation?: string;
encryptedDistributionBlobLocation?: string;
preEncryptedDistributionBlobLocation?: string;
secondaryDistributionHashRequired?: boolean;
secondaryDistributionHashType?: FileHashType;
lastModifiedAt?: Date;
cultureCodes?: string[];
displayFileInDownloadCenter?: boolean;
isPrimaryFileInDownloadCenter?: boolean;
fileDownloadDetails?: FileDownloadDetails[];
}
interface UserInfo {
userPrincipalName?: string;
}
interface OwnerInfo {
owner: UserInfo;
}
interface ApproverInfo {
approver: UserInfo;
isAutoApproved: boolean;
isMandatory: boolean;
}
interface AccessPermissionsInfo {
mainPublisher?: string;
releasePublishers?: string[];
channelDownloadEntityDetails?: { [key: string]: string[] };
}
interface DownloadCenterLocaleInfo {
cultureCode?: string;
downloadTitle?: string;
shortName?: string;
shortDescription?: string;
longDescription?: string;
instructions?: string;
additionalInfo?: string;
keywords?: string[];
version?: string;
relatedLinks?: { [key: string]: URL };
}
interface DownloadCenterInfo {
downloadCenterId: number;
publishToDownloadCenter?: boolean;
publishingGroup?: string;
operatingSystems?: string[];
relatedReleases?: string[];
kbNumbers?: string[];
sbNumbers?: string[];
locales?: DownloadCenterLocaleInfo[];
additionalProperties?: { [key: string]: string };
}
interface ReleaseRequestMessage {
driEmail: string[];
groupId?: string;
customerCorrelationId: string;
esrpCorrelationId: string;
contextData?: { [key: string]: string };
releaseInfo: ReleaseInfo;
productInfo: ProductInfo;
files: ReleaseFileInfo[];
routingInfo?: RoutingInfo;
createdBy: UserInfo;
owners: OwnerInfo[];
approvers: ApproverInfo[];
accessPermissionsInfo: AccessPermissionsInfo;
jwsToken?: string;
publisherId?: string;
downloadCenterInfo?: DownloadCenterInfo;
}
function getCertificateBuffer(input: string) {
return Buffer.from(input.replace(/-----BEGIN CERTIFICATE-----|-----END CERTIFICATE-----|\n/g, ''), 'base64');
}
function getThumbprint(input: string, algorithm: string): Buffer {
const buffer = getCertificateBuffer(input);
return crypto.createHash(algorithm).update(buffer).digest();
}
function getKeyFromPFX(pfx: string): string {
const pfxCertificatePath = path.join(os.tmpdir(), 'cert.pfx');
const pemKeyPath = path.join(os.tmpdir(), 'key.pem');
try {
const pfxCertificate = Buffer.from(pfx, 'base64');
fs.writeFileSync(pfxCertificatePath, pfxCertificate);
cp.execSync(`openssl pkcs12 -in "${pfxCertificatePath}" -nocerts -nodes -out "${pemKeyPath}" -passin pass:`);
const raw = fs.readFileSync(pemKeyPath, 'utf-8');
const result = raw.match(/-----BEGIN PRIVATE KEY-----[\s\S]+?-----END PRIVATE KEY-----/g)![0];
return result;
} finally {
fs.rmSync(pfxCertificatePath, { force: true });
fs.rmSync(pemKeyPath, { force: true });
}
}
function getCertificatesFromPFX(pfx: string): string[] {
const pfxCertificatePath = path.join(os.tmpdir(), 'cert.pfx');
const pemCertificatePath = path.join(os.tmpdir(), 'cert.pem');
try {
const pfxCertificate = Buffer.from(pfx, 'base64');
fs.writeFileSync(pfxCertificatePath, pfxCertificate);
cp.execSync(`openssl pkcs12 -in "${pfxCertificatePath}" -nokeys -out "${pemCertificatePath}" -passin pass:`);
const raw = fs.readFileSync(pemCertificatePath, 'utf-8');
const matches = raw.match(/-----BEGIN CERTIFICATE-----[\s\S]+?-----END CERTIFICATE-----/g);
return matches ? matches.reverse() : [];
} finally {
fs.rmSync(pfxCertificatePath, { force: true });
fs.rmSync(pemCertificatePath, { force: true });
}
}
class ESRPReleaseService {
static async create(
log: (...args: any[]) => void,
tenantId: string,
clientId: string,
authCertificatePfx: string,
requestSigningCertificatePfx: string,
containerClient: ContainerClient,
stagingSasToken: string
) {
const authKey = getKeyFromPFX(authCertificatePfx);
const authCertificate = getCertificatesFromPFX(authCertificatePfx)[0];
const requestSigningKey = getKeyFromPFX(requestSigningCertificatePfx);
const requestSigningCertificates = getCertificatesFromPFX(requestSigningCertificatePfx);
const app = new ConfidentialClientApplication({
auth: {
clientId,
authority: `https://login.microsoftonline.com/${tenantId}`,
clientCertificate: {
thumbprintSha256: getThumbprint(authCertificate, 'sha256').toString('hex'),
privateKey: authKey,
x5c: authCertificate
}
}
});
const response = await app.acquireTokenByClientCredential({
scopes: ['https://api.esrp.microsoft.com/.default']
});
return new ESRPReleaseService(log, clientId, response!.accessToken, requestSigningCertificates, requestSigningKey, containerClient, stagingSasToken);
}
private static API_URL = 'https://api.esrp.microsoft.com/api/v3/releaseservices/clients/';
private constructor(
private readonly log: (...args: any[]) => void,
private readonly clientId: string,
private readonly accessToken: string,
private readonly requestSigningCertificates: string[],
private readonly requestSigningKey: string,
private readonly containerClient: ContainerClient,
private readonly stagingSasToken: string
) { }
async createRelease(version: string, filePath: string, friendlyFileName: string) {
const correlationId = crypto.randomUUID();
const blobClient = this.containerClient.getBlockBlobClient(correlationId);
this.log(`Uploading ${filePath} to ${blobClient.url}`);
await blobClient.uploadFile(filePath);
this.log('Uploaded blob successfully');
try {
this.log(`Submitting release for ${version}: ${filePath}`);
const submitReleaseResult = await this.submitRelease(version, filePath, friendlyFileName, correlationId, blobClient);
this.log(`Successfully submitted release ${submitReleaseResult.operationId}. Polling for completion...`);
// Poll every 5 seconds, wait 60 minutes max -> poll 60/5*60=720 times
for (let i = 0; i < 720; i++) {
await new Promise(c => setTimeout(c, 5000));
const releaseStatus = await this.getReleaseStatus(submitReleaseResult.operationId);
if (releaseStatus.status === 'pass') {
break;
} else if (releaseStatus.status === 'aborted') {
this.log(JSON.stringify(releaseStatus));
throw new Error(`Release was aborted`);
} else if (releaseStatus.status !== 'inprogress') {
this.log(JSON.stringify(releaseStatus));
throw new Error(`Unknown error when polling for release`);
}
}
const releaseDetails = await this.getReleaseDetails(submitReleaseResult.operationId);
if (releaseDetails.status !== 'pass') {
throw new Error(`Timed out waiting for release: ${JSON.stringify(releaseDetails)}`);
}
this.log('Successfully created release:', releaseDetails.files[0].fileDownloadDetails![0].downloadUrl);
return releaseDetails.files[0].fileDownloadDetails![0].downloadUrl;
} finally {
this.log(`Deleting blob ${blobClient.url}`);
await blobClient.delete();
this.log('Deleted blob successfully');
}
}
private async submitRelease(
version: string,
filePath: string,
friendlyFileName: string,
correlationId: string,
blobClient: BlobClient
): Promise {
const size = fs.statSync(filePath).size;
const hash = await hashStream('sha256', fs.createReadStream(filePath));
const blobUrl = `${blobClient.url}?${this.stagingSasToken}`;
const message: ReleaseRequestMessage = {
customerCorrelationId: correlationId,
esrpCorrelationId: correlationId,
driEmail: ['joao.moreno@microsoft.com'],
createdBy: { userPrincipalName: 'jomo@microsoft.com' },
owners: [{ owner: { userPrincipalName: 'jomo@microsoft.com' } }],
approvers: [{ approver: { userPrincipalName: 'jomo@microsoft.com' }, isAutoApproved: true, isMandatory: false }],
releaseInfo: {
title: 'VS Code',
properties: {
'ReleaseContentType': 'InstallPackage'
},
minimumNumberOfApprovers: 1
},
productInfo: {
name: 'VS Code',
version,
description: 'VS Code'
},
accessPermissionsInfo: {
mainPublisher: 'VSCode',
channelDownloadEntityDetails: {
AllDownloadEntities: ['VSCode']
}
},
routingInfo: {
intent: 'filedownloadlinkgeneration'
},
files: [{
name: path.basename(filePath),
friendlyFileName,
tenantFileLocation: blobUrl,
tenantFileLocationType: 'AzureBlob',
sourceLocation: {
type: 'azureBlob',
blobUrl
},
hashType: 'sha256',
hash: Array.from(hash),
sizeInBytes: size
}]
};
message.jwsToken = await this.generateJwsToken(message);
const res = await fetch(`${ESRPReleaseService.API_URL}${this.clientId}/workflows/release/operations`, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Authorization': `Bearer ${this.accessToken}`
},
body: JSON.stringify(message)
});
if (!res.ok) {
const text = await res.text();
throw new Error(`Failed to submit release: ${res.statusText}\n${text}`);
}
return await res.json() as ReleaseSubmitResponse;
}
private async getReleaseStatus(releaseId: string): Promise {
const url = `${ESRPReleaseService.API_URL}${this.clientId}/workflows/release/operations/grs/${releaseId}`;
const res = await fetch(url, {
headers: {
'Authorization': `Bearer ${this.accessToken}`
}
});
if (!res.ok) {
const text = await res.text();
throw new Error(`Failed to get release status: ${res.statusText}\n${text}`);
}
return await res.json() as ReleaseResultMessage;
}
private async getReleaseDetails(releaseId: string): Promise {
const url = `${ESRPReleaseService.API_URL}${this.clientId}/workflows/release/operations/grd/${releaseId}`;
const res = await fetch(url, {
headers: {
'Authorization': `Bearer ${this.accessToken}`
}
});
if (!res.ok) {
const text = await res.text();
throw new Error(`Failed to get release status: ${res.statusText}\n${text}`);
}
return await res.json() as ReleaseDetailsMessage;
}
private async generateJwsToken(message: ReleaseRequestMessage): Promise {
return jws.sign({
header: {
alg: 'RS256',
crit: ['exp', 'x5t'],
// Release service uses ticks, not seconds :roll_eyes: (https://stackoverflow.com/a/7968483)
exp: ((Date.now() + (6 * 60 * 1000)) * 10000) + 621355968000000000,
// Release service uses hex format, not base64url :roll_eyes:
x5t: getThumbprint(this.requestSigningCertificates[0], 'sha1').toString('hex'),
// Release service uses a '.' separated string, not an array of strings :roll_eyes:
x5c: this.requestSigningCertificates.map(c => getCertificateBuffer(c).toString('base64url')).join('.') as any,
},
payload: message,
privateKey: this.requestSigningKey,
});
}
}
class State {
private statePath: string;
private set = new Set();
constructor() {
const pipelineWorkspacePath = e('PIPELINE_WORKSPACE');
const previousState = fs.readdirSync(pipelineWorkspacePath)
.map(name => /^artifacts_processed_(\d+)$/.exec(name))
.filter((match): match is RegExpExecArray => !!match)
.map(match => ({ name: match![0], attempt: Number(match![1]) }))
.sort((a, b) => b.attempt - a.attempt)[0];
if (previousState) {
const previousStatePath = path.join(pipelineWorkspacePath, previousState.name, previousState.name + '.txt');
fs.readFileSync(previousStatePath, 'utf8').split(/\n/).filter(name => !!name).forEach(name => this.set.add(name));
}
const stageAttempt = e('SYSTEM_STAGEATTEMPT');
this.statePath = path.join(pipelineWorkspacePath, `artifacts_processed_${stageAttempt}`, `artifacts_processed_${stageAttempt}.txt`);
fs.mkdirSync(path.dirname(this.statePath), { recursive: true });
fs.writeFileSync(this.statePath, [...this.set.values()].map(name => `${name}\n`).join(''));
}
get size(): number {
return this.set.size;
}
has(name: string): boolean {
return this.set.has(name);
}
add(name: string): void {
this.set.add(name);
fs.appendFileSync(this.statePath, `${name}\n`);
}
[Symbol.iterator](): IterableIterator {
return this.set[Symbol.iterator]();
}
}
const azdoFetchOptions = {
headers: {
// Pretend we're a web browser to avoid download rate limits
'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36 Edg/119.0.0.0',
'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7',
'Accept-Encoding': 'gzip, deflate, br',
'Accept-Language': 'en-US,en;q=0.9',
'Referer': 'https://dev.azure.com',
Authorization: `Bearer ${e('SYSTEM_ACCESSTOKEN')}`
}
};
async function requestAZDOAPI(path: string): Promise {
const abortController = new AbortController();
const timeout = setTimeout(() => abortController.abort(), 2 * 60 * 1000);
try {
const res = await fetch(`${e('BUILDS_API_URL')}${path}?api-version=6.0`, { ...azdoFetchOptions, signal: abortController.signal });
if (!res.ok) {
throw new Error(`Unexpected status code: ${res.status}`);
}
return await res.json();
} finally {
clearTimeout(timeout);
}
}
interface Artifact {
readonly name: string;
readonly resource: {
readonly downloadUrl: string;
readonly properties: {
readonly artifactsize: number;
};
};
}
async function getPipelineArtifacts(): Promise {
const result = await requestAZDOAPI<{ readonly value: Artifact[] }>('artifacts');
return result.value.filter(a => /^vscode_/.test(a.name) && !/sbom$/.test(a.name));
}
interface Timeline {
readonly records: {
readonly name: string;
readonly type: string;
readonly state: string;
}[];
}
async function getPipelineTimeline(): Promise {
return await requestAZDOAPI('timeline');
}
async function downloadArtifact(artifact: Artifact, downloadPath: string): Promise {
const abortController = new AbortController();
const timeout = setTimeout(() => abortController.abort(), 4 * 60 * 1000);
try {
const res = await fetch(artifact.resource.downloadUrl, { ...azdoFetchOptions, signal: abortController.signal });
if (!res.ok) {
throw new Error(`Unexpected status code: ${res.status}`);
}
await pipeline(Readable.fromWeb(res.body as ReadableStream), fs.createWriteStream(downloadPath));
} finally {
clearTimeout(timeout);
}
}
async function unzip(packagePath: string, outputPath: string): Promise {
return new Promise((resolve, reject) => {
yauzl.open(packagePath, { lazyEntries: true, autoClose: true }, (err, zipfile) => {
if (err) {
return reject(err);
}
const result: string[] = [];
zipfile!.on('entry', entry => {
if (/\/$/.test(entry.fileName)) {
zipfile!.readEntry();
} else {
zipfile!.openReadStream(entry, (err, istream) => {
if (err) {
return reject(err);
}
const filePath = path.join(outputPath, entry.fileName);
fs.mkdirSync(path.dirname(filePath), { recursive: true });
const ostream = fs.createWriteStream(filePath);
ostream.on('finish', () => {
result.push(filePath);
zipfile!.readEntry();
});
istream?.on('error', err => reject(err));
istream!.pipe(ostream);
});
}
});
zipfile!.on('close', () => resolve(result));
zipfile!.readEntry();
});
});
}
interface Asset {
platform: string;
type: string;
url: string;
mooncakeUrl?: string;
prssUrl?: string;
hash: string;
sha256hash: string;
size: number;
supportsFastUpdate?: boolean;
}
// Contains all of the logic for mapping details to our actual product names in CosmosDB
function getPlatform(product: string, os: string, arch: string, type: string): string {
switch (os) {
case 'win32':
switch (product) {
case 'client': {
switch (type) {
case 'archive':
return `win32-${arch}-archive`;
case 'setup':
return `win32-${arch}`;
case 'user-setup':
return `win32-${arch}-user`;
default:
throw new Error(`Unrecognized: ${product} ${os} ${arch} ${type}`);
}
}
case 'server':
return `server-win32-${arch}`;
case 'web':
return `server-win32-${arch}-web`;
case 'cli':
return `cli-win32-${arch}`;
default:
throw new Error(`Unrecognized: ${product} ${os} ${arch} ${type}`);
}
case 'alpine':
switch (product) {
case 'server':
return `server-alpine-${arch}`;
case 'web':
return `server-alpine-${arch}-web`;
case 'cli':
return `cli-alpine-${arch}`;
default:
throw new Error(`Unrecognized: ${product} ${os} ${arch} ${type}`);
}
case 'linux':
switch (type) {
case 'snap':
return `linux-snap-${arch}`;
case 'archive-unsigned':
switch (product) {
case 'client':
return `linux-${arch}`;
case 'server':
return `server-linux-${arch}`;
case 'web':
if (arch === 'standalone') {
return 'web-standalone';
}
return `server-linux-${arch}-web`;
default:
throw new Error(`Unrecognized: ${product} ${os} ${arch} ${type}`);
}
case 'deb-package':
return `linux-deb-${arch}`;
case 'rpm-package':
return `linux-rpm-${arch}`;
case 'cli':
return `cli-linux-${arch}`;
default:
throw new Error(`Unrecognized: ${product} ${os} ${arch} ${type}`);
}
case 'darwin':
switch (product) {
case 'client':
if (arch === 'x64') {
return 'darwin';
}
return `darwin-${arch}`;
case 'server':
if (arch === 'x64') {
return 'server-darwin';
}
return `server-darwin-${arch}`;
case 'web':
if (arch === 'x64') {
return 'server-darwin-web';
}
return `server-darwin-${arch}-web`;
case 'cli':
return `cli-darwin-${arch}`;
default:
throw new Error(`Unrecognized: ${product} ${os} ${arch} ${type}`);
}
default:
throw new Error(`Unrecognized: ${product} ${os} ${arch} ${type}`);
}
}
// Contains all of the logic for mapping types to our actual types in CosmosDB
function getRealType(type: string) {
switch (type) {
case 'user-setup':
return 'setup';
case 'deb-package':
case 'rpm-package':
return 'package';
default:
return type;
}
}
async function withLease(client: BlockBlobClient, fn: () => Promise) {
const lease = client.getBlobLeaseClient();
for (let i = 0; i < 360; i++) { // Try to get lease for 30 minutes
try {
await client.uploadData(new ArrayBuffer()); // blob needs to exist for lease to be acquired
await lease.acquireLease(60);
try {
const abortController = new AbortController();
const refresher = new Promise((c, e) => {
abortController.signal.onabort = () => {
clearInterval(interval);
c();
};
const interval = setInterval(() => {
lease.renewLease().catch(err => {
clearInterval(interval);
e(new Error('Failed to renew lease ' + err));
});
}, 30_000);
});
const result = await Promise.race([fn(), refresher]);
abortController.abort();
return result;
} finally {
await lease.releaseLease();
}
} catch (err) {
if (err.statusCode !== 409 && err.statusCode !== 412) {
throw err;
}
await new Promise(c => setTimeout(c, 5000));
}
}
throw new Error('Failed to acquire lease on blob after 30 minutes');
}
async function processArtifact(
artifact: Artifact,
filePath: string
) {
const log = (...args: any[]) => console.log(`[${artifact.name}]`, ...args);
const match = /^vscode_(?[^_]+)_(?[^_]+)(?:_legacy)?_(?[^_]+)_(?[^_]+)$/.exec(artifact.name);
if (!match) {
throw new Error(`Invalid artifact name: ${artifact.name}`);
}
const { cosmosDBAccessToken, blobServiceAccessToken } = JSON.parse(e('PUBLISH_AUTH_TOKENS'));
const quality = e('VSCODE_QUALITY');
const version = e('BUILD_SOURCEVERSION');
const friendlyFileName = `${quality}/${version}/${path.basename(filePath)}`;
const blobServiceClient = new BlobServiceClient(`https://${e('VSCODE_STAGING_BLOB_STORAGE_ACCOUNT_NAME')}.blob.core.windows.net/`, { getToken: async () => blobServiceAccessToken });
const leasesContainerClient = blobServiceClient.getContainerClient('leases');
await leasesContainerClient.createIfNotExists();
const leaseBlobClient = leasesContainerClient.getBlockBlobClient(friendlyFileName);
log(`Acquiring lease for: ${friendlyFileName}`);
await withLease(leaseBlobClient, async () => {
log(`Successfully acquired lease for: ${friendlyFileName}`);
const url = `${e('PRSS_CDN_URL')}/${friendlyFileName}`;
const res = await retry(() => fetch(url));
if (res.status === 200) {
log(`Already released and provisioned: ${url}`);
} else {
const stagingContainerClient = blobServiceClient.getContainerClient('staging');
await stagingContainerClient.createIfNotExists();
const now = new Date().valueOf();
const oneHour = 60 * 60 * 1000;
const oneHourAgo = new Date(now - oneHour);
const oneHourFromNow = new Date(now + oneHour);
const userDelegationKey = await blobServiceClient.getUserDelegationKey(oneHourAgo, oneHourFromNow);
const sasOptions = { containerName: 'staging', permissions: ContainerSASPermissions.from({ read: true }), startsOn: oneHourAgo, expiresOn: oneHourFromNow };
const stagingSasToken = generateBlobSASQueryParameters(sasOptions, userDelegationKey, e('VSCODE_STAGING_BLOB_STORAGE_ACCOUNT_NAME')).toString();
const releaseService = await ESRPReleaseService.create(
log,
e('RELEASE_TENANT_ID'),
e('RELEASE_CLIENT_ID'),
e('RELEASE_AUTH_CERT'),
e('RELEASE_REQUEST_SIGNING_CERT'),
stagingContainerClient,
stagingSasToken
);
await releaseService.createRelease(version, filePath, friendlyFileName);
}
const { product, os, arch, unprocessedType } = match.groups!;
const platform = getPlatform(product, os, arch, unprocessedType);
const type = getRealType(unprocessedType);
const size = fs.statSync(filePath).size;
const stream = fs.createReadStream(filePath);
const [hash, sha256hash] = await Promise.all([hashStream('sha1', stream), hashStream('sha256', stream)]); // CodeQL [SM04514] Using SHA1 only for legacy reasons, we are actually only respecting SHA256
const asset: Asset = { platform, type, url, hash: hash.toString('hex'), sha256hash: sha256hash.toString('hex'), size, supportsFastUpdate: true };
log('Creating asset...');
const result = await retry(async (attempt) => {
log(`Creating asset in Cosmos DB (attempt ${attempt})...`);
const client = new CosmosClient({ endpoint: e('AZURE_DOCUMENTDB_ENDPOINT')!, tokenProvider: () => Promise.resolve(`type=aad&ver=1.0&sig=${cosmosDBAccessToken.token}`) });
const scripts = client.database('builds').container(quality).scripts;
const { resource: result } = await scripts.storedProcedure('createAsset').execute<'ok' | 'already exists'>('', [version, asset, true]);
return result;
});
if (result === 'already exists') {
log('Asset already exists!');
} else {
log('Asset successfully created: ', JSON.stringify(asset, undefined, 2));
}
});
log(`Successfully released lease for: ${friendlyFileName}`);
}
// It is VERY important that we don't download artifacts too much too fast from AZDO.
// AZDO throttles us SEVERELY if we do. Not just that, but they also close open
// sockets, so the whole things turns to a grinding halt. So, downloading and extracting
// happens serially in the main thread, making the downloads are spaced out
// properly. For each extracted artifact, we spawn a worker thread to upload it to
// the CDN and finally update the build in Cosmos DB.
async function main() {
if (!isMainThread) {
const { artifact, artifactFilePath } = workerData;
await processArtifact(artifact, artifactFilePath);
return;
}
const done = new State();
const processing = new Set();
for (const name of done) {
console.log(`\u2705 ${name}`);
}
const stages = new Set(['Compile']);
if (
e('VSCODE_BUILD_STAGE_LINUX') === 'True' ||
e('VSCODE_BUILD_STAGE_ALPINE') === 'True' ||
e('VSCODE_BUILD_STAGE_MACOS') === 'True' ||
e('VSCODE_BUILD_STAGE_WINDOWS') === 'True'
) {
stages.add('CompileCLI');
}
if (e('VSCODE_BUILD_STAGE_WINDOWS') === 'True') { stages.add('Windows'); }
if (e('VSCODE_BUILD_STAGE_LINUX') === 'True') { stages.add('Linux'); }
if (e('VSCODE_BUILD_STAGE_ALPINE') === 'True') { stages.add('Alpine'); }
if (e('VSCODE_BUILD_STAGE_MACOS') === 'True') { stages.add('macOS'); }
if (e('VSCODE_BUILD_STAGE_WEB') === 'True') { stages.add('Web'); }
let resultPromise = Promise.resolve[]>([]);
const operations: { name: string; operation: Promise }[] = [];
while (true) {
const [timeline, artifacts] = await Promise.all([retry(() => getPipelineTimeline()), retry(() => getPipelineArtifacts())]);
const stagesCompleted = new Set(timeline.records.filter(r => r.type === 'Stage' && r.state === 'completed' && stages.has(r.name)).map(r => r.name));
const stagesInProgress = [...stages].filter(s => !stagesCompleted.has(s));
const artifactsInProgress = artifacts.filter(a => processing.has(a.name));
if (stagesInProgress.length === 0 && artifacts.length === done.size + processing.size) {
break;
} else if (stagesInProgress.length > 0) {
console.log('Stages in progress:', stagesInProgress.join(', '));
} else if (artifactsInProgress.length > 0) {
console.log('Artifacts in progress:', artifactsInProgress.map(a => a.name).join(', '));
} else {
console.log(`Waiting for a total of ${artifacts.length}, ${done.size} done, ${processing.size} in progress...`);
}
for (const artifact of artifacts) {
if (done.has(artifact.name) || processing.has(artifact.name)) {
continue;
}
console.log(`[${artifact.name}] Found new artifact`);
const artifactZipPath = path.join(e('AGENT_TEMPDIRECTORY'), `${artifact.name}.zip`);
await retry(async (attempt) => {
const start = Date.now();
console.log(`[${artifact.name}] Downloading (attempt ${attempt})...`);
await downloadArtifact(artifact, artifactZipPath);
const archiveSize = fs.statSync(artifactZipPath).size;
const downloadDurationS = (Date.now() - start) / 1000;
const downloadSpeedKBS = Math.round((archiveSize / 1024) / downloadDurationS);
console.log(`[${artifact.name}] Successfully downloaded after ${Math.floor(downloadDurationS)} seconds(${downloadSpeedKBS} KB/s).`);
});
const artifactFilePaths = await unzip(artifactZipPath, e('AGENT_TEMPDIRECTORY'));
const artifactFilePath = artifactFilePaths.filter(p => !/_manifest/.test(p))[0];
processing.add(artifact.name);
const promise = new Promise((resolve, reject) => {
const worker = new Worker(__filename, { workerData: { artifact, artifactFilePath } });
worker.on('error', reject);
worker.on('exit', code => {
if (code === 0) {
resolve();
} else {
reject(new Error(`[${artifact.name}] Worker stopped with exit code ${code}`));
}
});
});
const operation = promise.then(() => {
processing.delete(artifact.name);
done.add(artifact.name);
console.log(`\u2705 ${artifact.name} `);
});
operations.push({ name: artifact.name, operation });
resultPromise = Promise.allSettled(operations.map(o => o.operation));
}
await new Promise(c => setTimeout(c, 10_000));
}
console.log(`Found all ${done.size + processing.size} artifacts, waiting for ${processing.size} artifacts to finish publishing...`);
const artifactsInProgress = operations.filter(o => processing.has(o.name));
if (artifactsInProgress.length > 0) {
console.log('Artifacts in progress:', artifactsInProgress.map(a => a.name).join(', '));
}
const results = await resultPromise;
for (let i = 0; i < operations.length; i++) {
const result = results[i];
if (result.status === 'rejected') {
console.error(`[${operations[i].name}]`, result.reason);
}
}
if (results.some(r => r.status === 'rejected')) {
throw new Error('Some artifacts failed to publish');
}
console.log(`All ${done.size} artifacts published!`);
}
if (require.main === module) {
main().then(() => {
process.exit(0);
}, err => {
console.error(err);
process.exit(1);
});
}
================================================
FILE: build/azure-pipelines/common/releaseBuild.js
================================================
"use strict";
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
Object.defineProperty(exports, "__esModule", { value: true });
const identity_1 = require("@azure/identity");
const cosmos_1 = require("@azure/cosmos");
const retry_1 = require("./retry");
function getEnv(name) {
const result = process.env[name];
if (typeof result === 'undefined') {
throw new Error('Missing env: ' + name);
}
return result;
}
function createDefaultConfig(quality) {
return {
id: quality,
frozen: false
};
}
async function getConfig(client, quality) {
const query = `SELECT TOP 1 * FROM c WHERE c.id = "${quality}"`;
const res = await client.database('builds').container('config').items.query(query).fetchAll();
if (res.resources.length === 0) {
return createDefaultConfig(quality);
}
return res.resources[0];
}
async function main(force) {
const commit = getEnv('BUILD_SOURCEVERSION');
const quality = getEnv('VSCODE_QUALITY');
const aadCredentials = new identity_1.ClientAssertionCredential(process.env['AZURE_TENANT_ID'], process.env['AZURE_CLIENT_ID'], () => Promise.resolve(process.env['AZURE_ID_TOKEN']));
const client = new cosmos_1.CosmosClient({ endpoint: process.env['AZURE_DOCUMENTDB_ENDPOINT'], aadCredentials });
if (!force) {
const config = await getConfig(client, quality);
console.log('Quality config:', config);
if (config.frozen) {
console.log(`Skipping release because quality ${quality} is frozen.`);
return;
}
}
console.log(`Releasing build ${commit}...`);
const scripts = client.database('builds').container(quality).scripts;
await (0, retry_1.retry)(() => scripts.storedProcedure('releaseBuild').execute('', [commit]));
}
const [, , force] = process.argv;
console.log(process.argv);
main(/^true$/i.test(force)).then(() => {
console.log('Build successfully released');
process.exit(0);
}, err => {
console.error(err);
process.exit(1);
});
//# sourceMappingURL=releaseBuild.js.map
================================================
FILE: build/azure-pipelines/common/releaseBuild.ts
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
import { ClientAssertionCredential } from '@azure/identity';
import { CosmosClient } from '@azure/cosmos';
import { retry } from './retry';
function getEnv(name: string): string {
const result = process.env[name];
if (typeof result === 'undefined') {
throw new Error('Missing env: ' + name);
}
return result;
}
interface Config {
id: string;
frozen: boolean;
}
function createDefaultConfig(quality: string): Config {
return {
id: quality,
frozen: false
};
}
async function getConfig(client: CosmosClient, quality: string): Promise {
const query = `SELECT TOP 1 * FROM c WHERE c.id = "${quality}"`;
const res = await client.database('builds').container('config').items.query(query).fetchAll();
if (res.resources.length === 0) {
return createDefaultConfig(quality);
}
return res.resources[0] as Config;
}
async function main(force: boolean): Promise {
const commit = getEnv('BUILD_SOURCEVERSION');
const quality = getEnv('VSCODE_QUALITY');
const aadCredentials = new ClientAssertionCredential(process.env['AZURE_TENANT_ID']!, process.env['AZURE_CLIENT_ID']!, () => Promise.resolve(process.env['AZURE_ID_TOKEN']!));
const client = new CosmosClient({ endpoint: process.env['AZURE_DOCUMENTDB_ENDPOINT']!, aadCredentials });
if (!force) {
const config = await getConfig(client, quality);
console.log('Quality config:', config);
if (config.frozen) {
console.log(`Skipping release because quality ${quality} is frozen.`);
return;
}
}
console.log(`Releasing build ${commit}...`);
const scripts = client.database('builds').container(quality).scripts;
await retry(() => scripts.storedProcedure('releaseBuild').execute('', [commit]));
}
const [, , force] = process.argv;
console.log(process.argv);
main(/^true$/i.test(force)).then(() => {
console.log('Build successfully released');
process.exit(0);
}, err => {
console.error(err);
process.exit(1);
});
================================================
FILE: build/azure-pipelines/common/retry.js
================================================
"use strict";
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
Object.defineProperty(exports, "__esModule", { value: true });
exports.retry = retry;
async function retry(fn) {
let lastError;
for (let run = 1; run <= 10; run++) {
try {
return await fn(run);
}
catch (err) {
if (!/fetch failed|terminated|aborted|timeout|TimeoutError|Timeout Error|RestError|Client network socket disconnected|socket hang up|ECONNRESET|CredentialUnavailableError|endpoints_resolution_error|Audience validation failed|end of central directory record signature not found/i.test(err.message)) {
throw err;
}
lastError = err;
// maximum delay is 10th retry: ~3 seconds
const millis = Math.floor((Math.random() * 200) + (50 * Math.pow(1.5, run)));
await new Promise(c => setTimeout(c, millis));
}
}
console.error(`Too many retries, aborting.`);
throw lastError;
}
//# sourceMappingURL=retry.js.map
================================================
FILE: build/azure-pipelines/common/retry.ts
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
export async function retry(fn: (attempt: number) => Promise): Promise {
let lastError: Error | undefined;
for (let run = 1; run <= 10; run++) {
try {
return await fn(run);
} catch (err) {
if (!/fetch failed|terminated|aborted|timeout|TimeoutError|Timeout Error|RestError|Client network socket disconnected|socket hang up|ECONNRESET|CredentialUnavailableError|endpoints_resolution_error|Audience validation failed|end of central directory record signature not found/i.test(err.message)) {
throw err;
}
lastError = err;
// maximum delay is 10th retry: ~3 seconds
const millis = Math.floor((Math.random() * 200) + (50 * Math.pow(1.5, run)));
await new Promise(c => setTimeout(c, millis));
}
}
console.error(`Too many retries, aborting.`);
throw lastError;
}
================================================
FILE: build/azure-pipelines/common/sign-win32.js
================================================
"use strict";
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
Object.defineProperty(exports, "__esModule", { value: true });
const sign_1 = require("./sign");
const path_1 = __importDefault(require("path"));
(0, sign_1.main)([
process.env['EsrpCliDllPath'],
'sign-windows',
path_1.default.dirname(process.argv[2]),
path_1.default.basename(process.argv[2])
]);
//# sourceMappingURL=sign-win32.js.map
================================================
FILE: build/azure-pipelines/common/sign-win32.ts
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
import { main } from './sign';
import path from 'path';
main([
process.env['EsrpCliDllPath']!,
'sign-windows',
path.dirname(process.argv[2]),
path.basename(process.argv[2])
]);
================================================
FILE: build/azure-pipelines/common/sign.js
================================================
"use strict";
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
Object.defineProperty(exports, "__esModule", { value: true });
exports.Temp = void 0;
exports.main = main;
const child_process_1 = __importDefault(require("child_process"));
const fs_1 = __importDefault(require("fs"));
const crypto_1 = __importDefault(require("crypto"));
const path_1 = __importDefault(require("path"));
const os_1 = __importDefault(require("os"));
class Temp {
_files = [];
tmpNameSync() {
const file = path_1.default.join(os_1.default.tmpdir(), crypto_1.default.randomBytes(20).toString('hex'));
this._files.push(file);
return file;
}
dispose() {
for (const file of this._files) {
try {
fs_1.default.unlinkSync(file);
}
catch (err) {
// noop
}
}
}
}
exports.Temp = Temp;
function getParams(type) {
switch (type) {
case 'sign-windows':
return [
{
keyCode: 'CP-230012',
operationSetCode: 'SigntoolSign',
parameters: [
{ parameterName: 'OpusName', parameterValue: 'VS Code' },
{ parameterName: 'OpusInfo', parameterValue: 'https://code.visualstudio.com/' },
{ parameterName: 'Append', parameterValue: '/as' },
{ parameterName: 'FileDigest', parameterValue: '/fd "SHA256"' },
{ parameterName: 'PageHash', parameterValue: '/NPH' },
{ parameterName: 'TimeStamp', parameterValue: '/tr "http://rfc3161.gtm.corp.microsoft.com/TSS/HttpTspServer" /td sha256' }
],
toolName: 'sign',
toolVersion: '1.0'
},
{
keyCode: 'CP-230012',
operationSetCode: 'SigntoolVerify',
parameters: [
{ parameterName: 'VerifyAll', parameterValue: '/all' }
],
toolName: 'sign',
toolVersion: '1.0'
}
];
case 'sign-windows-appx':
return [
{
keyCode: 'CP-229979',
operationSetCode: 'SigntoolSign',
parameters: [
{ parameterName: 'OpusName', parameterValue: 'VS Code' },
{ parameterName: 'OpusInfo', parameterValue: 'https://code.visualstudio.com/' },
{ parameterName: 'FileDigest', parameterValue: '/fd "SHA256"' },
{ parameterName: 'PageHash', parameterValue: '/NPH' },
{ parameterName: 'TimeStamp', parameterValue: '/tr "http://rfc3161.gtm.corp.microsoft.com/TSS/HttpTspServer" /td sha256' }
],
toolName: 'sign',
toolVersion: '1.0'
},
{
keyCode: 'CP-229979',
operationSetCode: 'SigntoolVerify',
parameters: [],
toolName: 'sign',
toolVersion: '1.0'
}
];
case 'sign-pgp':
return [{
keyCode: 'CP-450779-Pgp',
operationSetCode: 'LinuxSign',
parameters: [],
toolName: 'sign',
toolVersion: '1.0'
}];
case 'sign-darwin':
return [{
keyCode: 'CP-401337-Apple',
operationSetCode: 'MacAppDeveloperSign',
parameters: [{ parameterName: 'Hardening', parameterValue: '--options=runtime' }],
toolName: 'sign',
toolVersion: '1.0'
}];
case 'notarize-darwin':
return [{
keyCode: 'CP-401337-Apple',
operationSetCode: 'MacAppNotarize',
parameters: [],
toolName: 'sign',
toolVersion: '1.0'
}];
case 'nuget':
return [{
keyCode: 'CP-401405',
operationSetCode: 'NuGetSign',
parameters: [],
toolName: 'sign',
toolVersion: '1.0'
}, {
keyCode: 'CP-401405',
operationSetCode: 'NuGetVerify',
parameters: [],
toolName: 'sign',
toolVersion: '1.0'
}];
default:
throw new Error(`Sign type ${type} not found`);
}
}
function main([esrpCliPath, type, folderPath, pattern]) {
const tmp = new Temp();
process.on('exit', () => tmp.dispose());
const key = crypto_1.default.randomBytes(32);
const iv = crypto_1.default.randomBytes(16);
const cipher = crypto_1.default.createCipheriv('aes-256-cbc', key, iv);
const encryptedToken = cipher.update(process.env['SYSTEM_ACCESSTOKEN'].trim(), 'utf8', 'hex') + cipher.final('hex');
const encryptionDetailsPath = tmp.tmpNameSync();
fs_1.default.writeFileSync(encryptionDetailsPath, JSON.stringify({ key: key.toString('hex'), iv: iv.toString('hex') }));
const encryptedTokenPath = tmp.tmpNameSync();
fs_1.default.writeFileSync(encryptedTokenPath, encryptedToken);
const patternPath = tmp.tmpNameSync();
fs_1.default.writeFileSync(patternPath, pattern);
const paramsPath = tmp.tmpNameSync();
fs_1.default.writeFileSync(paramsPath, JSON.stringify(getParams(type)));
const dotnetVersion = child_process_1.default.execSync('dotnet --version', { encoding: 'utf8' }).trim();
const adoTaskVersion = path_1.default.basename(path_1.default.dirname(path_1.default.dirname(esrpCliPath)));
const federatedTokenData = {
jobId: process.env['SYSTEM_JOBID'],
planId: process.env['SYSTEM_PLANID'],
projectId: process.env['SYSTEM_TEAMPROJECTID'],
hub: process.env['SYSTEM_HOSTTYPE'],
uri: process.env['SYSTEM_COLLECTIONURI'],
managedIdentityId: process.env['VSCODE_ESRP_CLIENT_ID'],
managedIdentityTenantId: process.env['VSCODE_ESRP_TENANT_ID'],
serviceConnectionId: process.env['VSCODE_ESRP_SERVICE_CONNECTION_ID'],
tempDirectory: os_1.default.tmpdir(),
systemAccessToken: encryptedTokenPath,
encryptionKey: encryptionDetailsPath
};
const args = [
esrpCliPath,
'vsts.sign',
'-a', process.env['ESRP_CLIENT_ID'],
'-d', process.env['ESRP_TENANT_ID'],
'-k', JSON.stringify({ akv: 'vscode-esrp' }),
'-z', JSON.stringify({ akv: 'vscode-esrp', cert: 'esrp-sign' }),
'-f', folderPath,
'-p', patternPath,
'-u', 'false',
'-x', 'regularSigning',
'-b', 'input.json',
'-l', 'AzSecPack_PublisherPolicyProd.xml',
'-y', 'inlineSignParams',
'-j', paramsPath,
'-c', '9997',
'-t', '120',
'-g', '10',
'-v', 'Tls12',
'-s', 'https://api.esrp.microsoft.com/api/v1',
'-m', '0',
'-o', 'Microsoft',
'-i', 'https://www.microsoft.com',
'-n', '5',
'-r', 'true',
'-w', dotnetVersion,
'-skipAdoReportAttachment', 'false',
'-pendingAnalysisWaitTimeoutMinutes', '5',
'-adoTaskVersion', adoTaskVersion,
'-resourceUri', 'https://msazurecloud.onmicrosoft.com/api.esrp.microsoft.com',
'-esrpClientId', process.env['ESRP_CLIENT_ID'],
'-useMSIAuthentication', 'true',
'-federatedTokenData', JSON.stringify(federatedTokenData)
];
try {
child_process_1.default.execFileSync('dotnet', args, { stdio: 'inherit' });
}
catch (err) {
console.error('ESRP failed');
console.error(err);
process.exit(1);
}
}
if (require.main === module) {
main(process.argv.slice(2));
process.exit(0);
}
//# sourceMappingURL=sign.js.map
================================================
FILE: build/azure-pipelines/common/sign.ts
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
import cp from 'child_process';
import fs from 'fs';
import crypto from 'crypto';
import path from 'path';
import os from 'os';
export class Temp {
private _files: string[] = [];
tmpNameSync(): string {
const file = path.join(os.tmpdir(), crypto.randomBytes(20).toString('hex'));
this._files.push(file);
return file;
}
dispose(): void {
for (const file of this._files) {
try {
fs.unlinkSync(file);
} catch (err) {
// noop
}
}
}
}
interface Params {
readonly keyCode: string;
readonly operationSetCode: string;
readonly parameters: {
readonly parameterName: string;
readonly parameterValue: string;
}[];
readonly toolName: string;
readonly toolVersion: string;
}
function getParams(type: string): Params[] {
switch (type) {
case 'sign-windows':
return [
{
keyCode: 'CP-230012',
operationSetCode: 'SigntoolSign',
parameters: [
{ parameterName: 'OpusName', parameterValue: 'VS Code' },
{ parameterName: 'OpusInfo', parameterValue: 'https://code.visualstudio.com/' },
{ parameterName: 'Append', parameterValue: '/as' },
{ parameterName: 'FileDigest', parameterValue: '/fd "SHA256"' },
{ parameterName: 'PageHash', parameterValue: '/NPH' },
{ parameterName: 'TimeStamp', parameterValue: '/tr "http://rfc3161.gtm.corp.microsoft.com/TSS/HttpTspServer" /td sha256' }
],
toolName: 'sign',
toolVersion: '1.0'
},
{
keyCode: 'CP-230012',
operationSetCode: 'SigntoolVerify',
parameters: [
{ parameterName: 'VerifyAll', parameterValue: '/all' }
],
toolName: 'sign',
toolVersion: '1.0'
}
];
case 'sign-windows-appx':
return [
{
keyCode: 'CP-229979',
operationSetCode: 'SigntoolSign',
parameters: [
{ parameterName: 'OpusName', parameterValue: 'VS Code' },
{ parameterName: 'OpusInfo', parameterValue: 'https://code.visualstudio.com/' },
{ parameterName: 'FileDigest', parameterValue: '/fd "SHA256"' },
{ parameterName: 'PageHash', parameterValue: '/NPH' },
{ parameterName: 'TimeStamp', parameterValue: '/tr "http://rfc3161.gtm.corp.microsoft.com/TSS/HttpTspServer" /td sha256' }
],
toolName: 'sign',
toolVersion: '1.0'
},
{
keyCode: 'CP-229979',
operationSetCode: 'SigntoolVerify',
parameters: [],
toolName: 'sign',
toolVersion: '1.0'
}
];
case 'sign-pgp':
return [{
keyCode: 'CP-450779-Pgp',
operationSetCode: 'LinuxSign',
parameters: [],
toolName: 'sign',
toolVersion: '1.0'
}];
case 'sign-darwin':
return [{
keyCode: 'CP-401337-Apple',
operationSetCode: 'MacAppDeveloperSign',
parameters: [{ parameterName: 'Hardening', parameterValue: '--options=runtime' }],
toolName: 'sign',
toolVersion: '1.0'
}];
case 'notarize-darwin':
return [{
keyCode: 'CP-401337-Apple',
operationSetCode: 'MacAppNotarize',
parameters: [],
toolName: 'sign',
toolVersion: '1.0'
}];
case 'nuget':
return [{
keyCode: 'CP-401405',
operationSetCode: 'NuGetSign',
parameters: [],
toolName: 'sign',
toolVersion: '1.0'
}, {
keyCode: 'CP-401405',
operationSetCode: 'NuGetVerify',
parameters: [],
toolName: 'sign',
toolVersion: '1.0'
}];
default:
throw new Error(`Sign type ${type} not found`);
}
}
export function main([esrpCliPath, type, folderPath, pattern]: string[]) {
const tmp = new Temp();
process.on('exit', () => tmp.dispose());
const key = crypto.randomBytes(32);
const iv = crypto.randomBytes(16);
const cipher = crypto.createCipheriv('aes-256-cbc', key, iv);
const encryptedToken = cipher.update(process.env['SYSTEM_ACCESSTOKEN']!.trim(), 'utf8', 'hex') + cipher.final('hex');
const encryptionDetailsPath = tmp.tmpNameSync();
fs.writeFileSync(encryptionDetailsPath, JSON.stringify({ key: key.toString('hex'), iv: iv.toString('hex') }));
const encryptedTokenPath = tmp.tmpNameSync();
fs.writeFileSync(encryptedTokenPath, encryptedToken);
const patternPath = tmp.tmpNameSync();
fs.writeFileSync(patternPath, pattern);
const paramsPath = tmp.tmpNameSync();
fs.writeFileSync(paramsPath, JSON.stringify(getParams(type)));
const dotnetVersion = cp.execSync('dotnet --version', { encoding: 'utf8' }).trim();
const adoTaskVersion = path.basename(path.dirname(path.dirname(esrpCliPath)));
const federatedTokenData = {
jobId: process.env['SYSTEM_JOBID'],
planId: process.env['SYSTEM_PLANID'],
projectId: process.env['SYSTEM_TEAMPROJECTID'],
hub: process.env['SYSTEM_HOSTTYPE'],
uri: process.env['SYSTEM_COLLECTIONURI'],
managedIdentityId: process.env['VSCODE_ESRP_CLIENT_ID'],
managedIdentityTenantId: process.env['VSCODE_ESRP_TENANT_ID'],
serviceConnectionId: process.env['VSCODE_ESRP_SERVICE_CONNECTION_ID'],
tempDirectory: os.tmpdir(),
systemAccessToken: encryptedTokenPath,
encryptionKey: encryptionDetailsPath
};
const args = [
esrpCliPath,
'vsts.sign',
'-a', process.env['ESRP_CLIENT_ID']!,
'-d', process.env['ESRP_TENANT_ID']!,
'-k', JSON.stringify({ akv: 'vscode-esrp' }),
'-z', JSON.stringify({ akv: 'vscode-esrp', cert: 'esrp-sign' }),
'-f', folderPath,
'-p', patternPath,
'-u', 'false',
'-x', 'regularSigning',
'-b', 'input.json',
'-l', 'AzSecPack_PublisherPolicyProd.xml',
'-y', 'inlineSignParams',
'-j', paramsPath,
'-c', '9997',
'-t', '120',
'-g', '10',
'-v', 'Tls12',
'-s', 'https://api.esrp.microsoft.com/api/v1',
'-m', '0',
'-o', 'Microsoft',
'-i', 'https://www.microsoft.com',
'-n', '5',
'-r', 'true',
'-w', dotnetVersion,
'-skipAdoReportAttachment', 'false',
'-pendingAnalysisWaitTimeoutMinutes', '5',
'-adoTaskVersion', adoTaskVersion,
'-resourceUri', 'https://msazurecloud.onmicrosoft.com/api.esrp.microsoft.com',
'-esrpClientId', process.env['ESRP_CLIENT_ID']!,
'-useMSIAuthentication', 'true',
'-federatedTokenData', JSON.stringify(federatedTokenData)
];
try {
cp.execFileSync('dotnet', args, { stdio: 'inherit' });
} catch (err) {
console.error('ESRP failed');
console.error(err);
process.exit(1);
}
}
if (require.main === module) {
main(process.argv.slice(2));
process.exit(0);
}
================================================
FILE: build/azure-pipelines/common/telemetry-config.json
================================================
[
{
"eventPrefix": "typescript-language-features/",
"sourceDirs": [
"../../s/extensions/typescript-language-features"
],
"excludedDirs": [],
"applyEndpoints": true
},
{
"eventPrefix": "git/",
"sourceDirs": [
"../../s/extensions/git"
],
"excludedDirs": [],
"applyEndpoints": true
},
{
"eventPrefix": "extension-telemetry/",
"sourceDirs": [
"vscode-extension-telemetry"
],
"excludedDirs": [],
"applyEndpoints": true
},
{
"eventPrefix": "vscode-markdown/",
"sourceDirs": [
"../../s/extensions/markdown-language-features"
],
"excludedDirs": [],
"applyEndpoints": true
},
{
"eventPrefix": "html-language-features/",
"sourceDirs": [
"../../s/extensions/html-language-features",
"vscode-html-languageservice"
],
"excludedDirs": [],
"applyEndpoints": true
},
{
"eventPrefix": "json-language-features/",
"sourceDirs": [
"../../s/extensions/json-language-features",
"vscode-json-languageservice"
],
"excludedDirs": [],
"applyEndpoints": true
},
{
"eventPrefix": "ms-vscode.node/",
"sourceDirs": [
"vscode-chrome-debug-core",
"vscode-node-debug"
],
"excludedDirs": [],
"applyEndpoints": true,
"patchDebugEvents": true
}
]
================================================
FILE: build/azure-pipelines/config/CredScanSuppressions.json
================================================
{
"tool": "Credential Scanner",
"suppressions": [
{
"file": [
"src/vs/base/test/common/uri.test.ts",
"src/vs/workbench/api/test/browser/extHostTelemetry.test.ts"
],
"_justification": "These are dummy credentials in tests."
},
{
"file": [
".build/linux/rpm/x86_64/rpmbuild/BUILD/usr/share/code/resources/app/extensions/github-authentication/dist/extension.js",
".build/linux/rpm/x86_64/rpmbuild/BUILD/usr/share/code/resources/app/extensions/emmet/dist/node/emmetNodeMain.js",
".build/linux/rpm/armv7hl/rpmbuild/BUILD/usr/share/code/resources/app/extensions/github-authentication/dist/extension.js",
".build/linux/rpm/armv7hl/rpmbuild/BUILD/usr/share/code/resources/app/extensions/emmet/dist/node/emmetNodeMain.js",
".build/linux/rpm/aarch64/rpmbuild/BUILD/usr/share/code/resources/app/extensions/github-authentication/dist/extension.js",
".build/linux/rpm/aarch64/rpmbuild/BUILD/usr/share/code/resources/app/extensions/emmet/dist/node/emmetNodeMain.js",
".build/linux/snap/x64/code-x64/usr/share/code/resources/app/extensions/github-authentication/dist/extension.js",
".build/linux/snap/x64/code-x64/usr/share/code/resources/app/extensions/emmet/dist/node/emmetNodeMain.js",
".build/linux/snap/x64/code-x64/stage/usr/share/code/resources/app/extensions/github-authentication/dist/extension.js",
".build/linux/snap/x64/code-x64/stage/usr/share/code/resources/app/extensions/emmet/dist/node/emmetNodeMain.js",
".build/linux/snap/x64/code-x64/prime/usr/share/code/resources/app/extensions/github-authentication/dist/extension.js",
".build/linux/snap/x64/code-x64/prime/usr/share/code/resources/app/extensions/emmet/dist/node/emmetNodeMain.js",
".build/linux/snap/x64/code-x64/parts/code/build/usr/share/code/resources/app/extensions/github-authentication/dist/extension.js",
".build/linux/snap/x64/code-x64/parts/code/install/usr/share/code/resources/app/extensions/github-authentication/dist/extension.js",
".build/linux/snap/x64/code-x64/parts/code/src/usr/share/code/resources/app/extensions/github-authentication/dist/extension.js",
".build/linux/snap/x64/code-x64/parts/code/build/usr/share/code/resources/app/extensions/emmet/dist/node/emmetNodeMain.js",
".build/linux/snap/x64/code-x64/parts/code/install/usr/share/code/resources/app/extensions/emmet/dist/node/emmetNodeMain.js",
".build/linux/snap/x64/code-x64/parts/code/src/usr/share/code/resources/app/extensions/emmet/dist/node/emmetNodeMain.js"
],
"_justification": "These are safe to ignore, since they are built artifacts (stable)."
},
{
"file": [
".build/linux/rpm/x86_64/rpmbuild/BUILD/usr/share/code-insiders/resources/app/extensions/github-authentication/dist/extension.js",
".build/linux/rpm/x86_64/rpmbuild/BUILD/usr/share/code-insiders/resources/app/extensions/emmet/dist/node/emmetNodeMain.js",
".build/linux/rpm/armv7hl/rpmbuild/BUILD/usr/share/code-insiders/resources/app/extensions/github-authentication/dist/extension.js",
".build/linux/rpm/armv7hl/rpmbuild/BUILD/usr/share/code-insiders/resources/app/extensions/emmet/dist/node/emmetNodeMain.js",
".build/linux/rpm/aarch64/rpmbuild/BUILD/usr/share/code-insiders/resources/app/extensions/github-authentication/dist/extension.js",
".build/linux/rpm/aarch64/rpmbuild/BUILD/usr/share/code-insiders/resources/app/extensions/emmet/dist/node/emmetNodeMain.js",
".build/linux/snap/x64/code-insiders-x64/usr/share/code-insiders/resources/app/extensions/github-authentication/dist/extension.js",
".build/linux/snap/x64/code-insiders-x64/usr/share/code-insiders/resources/app/extensions/emmet/dist/node/emmetNodeMain.js",
".build/linux/snap/x64/code-insiders-x64/stage/usr/share/code-insiders/resources/app/extensions/github-authentication/dist/extension.js",
".build/linux/snap/x64/code-insiders-x64/stage/usr/share/code-insiders/resources/app/extensions/emmet/dist/node/emmetNodeMain.js",
".build/linux/snap/x64/code-insiders-x64/prime/usr/share/code-insiders/resources/app/extensions/github-authentication/dist/extension.js",
".build/linux/snap/x64/code-insiders-x64/prime/usr/share/code-insiders/resources/app/extensions/emmet/dist/node/emmetNodeMain.js",
".build/linux/snap/x64/code-insiders-x64/parts/code/build/usr/share/code-insiders/resources/app/extensions/github-authentication/dist/extension.js",
".build/linux/snap/x64/code-insiders-x64/parts/code/install/usr/share/code-insiders/resources/app/extensions/github-authentication/dist/extension.js",
".build/linux/snap/x64/code-insiders-x64/parts/code/src/usr/share/code-insiders/resources/app/extensions/github-authentication/dist/extension.js",
".build/linux/snap/x64/code-insiders-x64/parts/code/build/usr/share/code-insiders/resources/app/extensions/emmet/dist/node/emmetNodeMain.js",
".build/linux/snap/x64/code-insiders-x64/parts/code/install/usr/share/code-insiders/resources/app/extensions/emmet/dist/node/emmetNodeMain.js",
".build/linux/snap/x64/code-insiders-x64/parts/code/src/usr/share/code-insiders/resources/app/extensions/emmet/dist/node/emmetNodeMain.js"
],
"_justification": "These are safe to ignore, since they are built artifacts (insiders)."
},
{
"file": [
".build/linux/rpm/x86_64/rpmbuild/BUILD/usr/share/code-exploration/resources/app/extensions/github-authentication/dist/extension.js",
".build/linux/rpm/x86_64/rpmbuild/BUILD/usr/share/code-exploration/resources/app/extensions/emmet/dist/node/emmetNodeMain.js",
".build/linux/rpm/armv7hl/rpmbuild/BUILD/usr/share/code-exploration/resources/app/extensions/github-authentication/dist/extension.js",
".build/linux/rpm/armv7hl/rpmbuild/BUILD/usr/share/code-exploration/resources/app/extensions/emmet/dist/node/emmetNodeMain.js",
".build/linux/rpm/aarch64/rpmbuild/BUILD/usr/share/code-exploration/resources/app/extensions/github-authentication/dist/extension.js",
".build/linux/rpm/aarch64/rpmbuild/BUILD/usr/share/code-exploration/resources/app/extensions/emmet/dist/node/emmetNodeMain.js",
".build/linux/snap/x64/code-exploration-x64/usr/share/code-exploration/resources/app/extensions/github-authentication/dist/extension.js",
".build/linux/snap/x64/code-exploration-x64/usr/share/code-exploration/resources/app/extensions/emmet/dist/node/emmetNodeMain.js",
".build/linux/snap/x64/code-exploration-x64/stage/usr/share/code-exploration/resources/app/extensions/github-authentication/dist/extension.js",
".build/linux/snap/x64/code-exploration-x64/stage/usr/share/code-exploration/resources/app/extensions/emmet/dist/node/emmetNodeMain.js",
".build/linux/snap/x64/code-exploration-x64/prime/usr/share/code-exploration/resources/app/extensions/github-authentication/dist/extension.js",
".build/linux/snap/x64/code-exploration-x64/prime/usr/share/code-exploration/resources/app/extensions/emmet/dist/node/emmetNodeMain.js",
".build/linux/snap/x64/code-exploration-x64/parts/code/build/usr/share/code-exploration/resources/app/extensions/github-authentication/dist/extension.js",
".build/linux/snap/x64/code-exploration-x64/parts/code/install/usr/share/code-exploration/resources/app/extensions/github-authentication/dist/extension.js",
".build/linux/snap/x64/code-exploration-x64/parts/code/src/usr/share/code-exploration/resources/app/extensions/github-authentication/dist/extension.js",
".build/linux/snap/x64/code-exploration-x64/parts/code/build/usr/share/code-exploration/resources/app/extensions/emmet/dist/node/emmetNodeMain.js",
".build/linux/snap/x64/code-exploration-x64/parts/code/install/usr/share/code-exploration/resources/app/extensions/emmet/dist/node/emmetNodeMain.js",
".build/linux/snap/x64/code-exploration-x64/parts/code/src/usr/share/code-exploration/resources/app/extensions/emmet/dist/node/emmetNodeMain.js"
],
"_justification": "These are safe to ignore, since they are built artifacts (exploration)."
},
{
"file": [
".build/web/extensions/github-authentication/dist/browser/extension.js",
".build/web/extensions/emmet/dist/browser/emmetBrowserMain.js.map",
".build/web/extensions/emmet/dist/browser/emmetBrowserMain.js"
],
"_justification": "These are safe to ignore, since they are built artifacts (web)."
}
]
}
================================================
FILE: build/azure-pipelines/config/tsaoptions.json
================================================
{
"codebaseName": "devdiv_microsoft_vscode",
"serviceTreeID": "79c048b2-322f-4ed5-a1ea-252a1250e4b3",
"instanceUrl": "https://devdiv.visualstudio.com/defaultcollection",
"projectName": "DevDiv",
"areaPath": "DevDiv\\VS Code (compliance tracking only)\\Visual Studio Code Client",
"notificationAliases": [
"monacotools@microsoft.com"
],
"validateToolOutput": "None",
"allTools": true
}
================================================
FILE: build/azure-pipelines/darwin/app-entitlements.plist
================================================
com.apple.security.cs.allow-jitcom.apple.security.device.audio-inputcom.apple.security.device.cameracom.apple.security.automation.apple-events
================================================
FILE: build/azure-pipelines/darwin/cli-build-darwin.yml
================================================
parameters:
- name: VSCODE_QUALITY
type: string
- name: VSCODE_BUILD_MACOS
type: boolean
default: false
- name: VSCODE_BUILD_MACOS_ARM64
type: boolean
default: false
- name: VSCODE_CHECK_ONLY
type: boolean
default: false
steps:
- task: NodeTool@0
inputs:
versionSource: fromFile
versionFilePath: .nvmrc
nodejsMirror: https://github.com/joaomoreno/node-mirror/releases/download
- ${{ if ne(parameters.VSCODE_QUALITY, 'oss') }}:
- template: ../cli/cli-apply-patches.yml@self
- task: Npm@1
displayName: Download openssl prebuilt
inputs:
command: custom
customCommand: pack @vscode-internal/openssl-prebuilt@0.0.11
customRegistry: useFeed
customFeed: "Monaco/openssl-prebuilt"
workingDir: $(Build.ArtifactStagingDirectory)
- script: |
set -e
mkdir $(Build.ArtifactStagingDirectory)/openssl
tar -xvzf $(Build.ArtifactStagingDirectory)/vscode-internal-openssl-prebuilt-0.0.11.tgz --strip-components=1 --directory=$(Build.ArtifactStagingDirectory)/openssl
displayName: Extract openssl prebuilt
- template: ../cli/install-rust-posix.yml@self
parameters:
targets:
- ${{ if eq(parameters.VSCODE_BUILD_MACOS, true) }}:
- x86_64-apple-darwin
- ${{ if eq(parameters.VSCODE_BUILD_MACOS_ARM64, true) }}:
- aarch64-apple-darwin
- ${{ if eq(parameters.VSCODE_BUILD_MACOS, true) }}:
- template: ../cli/cli-compile.yml@self
parameters:
VSCODE_QUALITY: ${{ parameters.VSCODE_QUALITY }}
VSCODE_CLI_TARGET: x86_64-apple-darwin
VSCODE_CLI_ARTIFACT: unsigned_vscode_cli_darwin_x64_cli
VSCODE_CHECK_ONLY: ${{ parameters.VSCODE_CHECK_ONLY }}
VSCODE_CLI_ENV:
OPENSSL_LIB_DIR: $(Build.ArtifactStagingDirectory)/openssl/x64-osx/lib
OPENSSL_INCLUDE_DIR: $(Build.ArtifactStagingDirectory)/openssl/x64-osx/include
- ${{ if eq(parameters.VSCODE_BUILD_MACOS_ARM64, true) }}:
- template: ../cli/cli-compile.yml@self
parameters:
VSCODE_QUALITY: ${{ parameters.VSCODE_QUALITY }}
VSCODE_CLI_TARGET: aarch64-apple-darwin
VSCODE_CLI_ARTIFACT: unsigned_vscode_cli_darwin_arm64_cli
VSCODE_CHECK_ONLY: ${{ parameters.VSCODE_CHECK_ONLY }}
VSCODE_CLI_ENV:
OPENSSL_LIB_DIR: $(Build.ArtifactStagingDirectory)/openssl/arm64-osx/lib
OPENSSL_INCLUDE_DIR: $(Build.ArtifactStagingDirectory)/openssl/arm64-osx/include
- ${{ if not(parameters.VSCODE_CHECK_ONLY) }}:
- ${{ if eq(parameters.VSCODE_BUILD_MACOS, true) }}:
- task: 1ES.PublishPipelineArtifact@1
inputs:
targetPath: $(Build.ArtifactStagingDirectory)/unsigned_vscode_cli_darwin_x64_cli.zip
artifactName: unsigned_vscode_cli_darwin_x64_cli
sbomBuildDropPath: $(Build.ArtifactStagingDirectory)/cli
sbomPackageName: "VS Code macOS x64 CLI (unsigned)"
sbomPackageVersion: $(Build.SourceVersion)
displayName: Publish unsigned_vscode_cli_darwin_x64_cli artifact
- ${{ if eq(parameters.VSCODE_BUILD_MACOS_ARM64, true) }}:
- task: 1ES.PublishPipelineArtifact@1
inputs:
targetPath: $(Build.ArtifactStagingDirectory)/unsigned_vscode_cli_darwin_arm64_cli.zip
artifactName: unsigned_vscode_cli_darwin_arm64_cli
sbomBuildDropPath: $(Build.ArtifactStagingDirectory)/cli
sbomPackageName: "VS Code macOS arm64 CLI (unsigned)"
sbomPackageVersion: $(Build.SourceVersion)
displayName: Publish unsigned_vscode_cli_darwin_arm64_cli artifact
================================================
FILE: build/azure-pipelines/darwin/helper-gpu-entitlements.plist
================================================
com.apple.security.cs.allow-jit
================================================
FILE: build/azure-pipelines/darwin/helper-plugin-entitlements.plist
================================================
com.apple.security.cs.allow-jitcom.apple.security.cs.allow-unsigned-executable-memorycom.apple.security.cs.disable-library-validation
================================================
FILE: build/azure-pipelines/darwin/helper-renderer-entitlements.plist
================================================
com.apple.security.cs.allow-jit
================================================
FILE: build/azure-pipelines/darwin/product-build-darwin-cli-sign.yml
================================================
parameters:
- name: VSCODE_BUILD_MACOS
type: boolean
- name: VSCODE_BUILD_MACOS_ARM64
type: boolean
- name: VSCODE_QUALITY
type: string
steps:
- task: NodeTool@0
inputs:
versionSource: fromFile
versionFilePath: .nvmrc
nodejsMirror: https://github.com/joaomoreno/node-mirror/releases/download
- ${{ if ne(parameters.VSCODE_QUALITY, 'oss') }}:
- task: AzureKeyVault@2
displayName: "Azure Key Vault: Get Secrets"
inputs:
azureSubscription: vscode
KeyVaultName: vscode-build-secrets
SecretsFilter: "github-distro-mixin-password"
- script: node build/setup-npm-registry.js $NPM_REGISTRY build
condition: and(succeeded(), ne(variables['NPM_REGISTRY'], 'none'))
displayName: Setup NPM Registry
- script: |
set -e
# Set the private NPM registry to the global npmrc file
# so that authentication works for subfolders like build/, remote/, extensions/ etc
# which does not have their own .npmrc file
npm config set registry "$NPM_REGISTRY"
echo "##vso[task.setvariable variable=NPMRC_PATH]$(npm config get userconfig)"
condition: and(succeeded(), ne(variables['NPM_REGISTRY'], 'none'))
displayName: Setup NPM
- task: npmAuthenticate@0
inputs:
workingFile: $(NPMRC_PATH)
condition: and(succeeded(), ne(variables['NPM_REGISTRY'], 'none'))
displayName: Setup NPM Authentication
- script: |
set -e
for i in {1..5}; do # try 5 times
npm ci && break
if [ $i -eq 5 ]; then
echo "Npm install failed too many times" >&2
exit 1
fi
echo "Npm install failed $i, trying again..."
done
workingDirectory: build
env:
GITHUB_TOKEN: "$(github-distro-mixin-password)"
displayName: Install build dependencies
- template: ../cli/cli-darwin-sign.yml@self
parameters:
VSCODE_CLI_ARTIFACTS:
- ${{ if eq(parameters.VSCODE_BUILD_MACOS, true) }}:
- unsigned_vscode_cli_darwin_x64_cli
- ${{ if eq(parameters.VSCODE_BUILD_MACOS_ARM64, true) }}:
- unsigned_vscode_cli_darwin_arm64_cli
================================================
FILE: build/azure-pipelines/darwin/product-build-darwin-sign.yml
================================================
steps:
- task: NodeTool@0
inputs:
versionSource: fromFile
versionFilePath: .nvmrc
nodejsMirror: https://github.com/joaomoreno/node-mirror/releases/download
- task: UseDotNet@2
inputs:
version: 6.x
- task: EsrpCodeSigning@5
inputs:
UseMSIAuthentication: true
ConnectedServiceName: vscode-esrp
AppRegistrationClientId: $(ESRP_CLIENT_ID)
AppRegistrationTenantId: $(ESRP_TENANT_ID)
AuthAKVName: vscode-esrp
AuthSignCertName: esrp-sign
FolderPath: .
Pattern: noop
displayName: 'Install ESRP Tooling'
- script: |
# For legacy purposes, arch for x64 is just 'darwin'
case $VSCODE_ARCH in
x64) ASSET_ID="darwin" ;;
arm64) ASSET_ID="darwin-arm64" ;;
universal) ASSET_ID="darwin-universal" ;;
esac
echo "##vso[task.setvariable variable=ASSET_ID]$ASSET_ID"
displayName: Set asset id variable
- script: |
if [ -z "$(ASSET_ID)" ]; then
echo "ASSET_ID is empty"
exit 1
else
echo "ASSET_ID is set to $(ASSET_ID)"
fi
displayName: Check ASSET_ID variable
- download: current
artifact: unsigned_vscode_client_darwin_$(VSCODE_ARCH)_archive
displayName: Download $(VSCODE_ARCH) artifact
- script: node build/azure-pipelines/common/sign $(Agent.RootDirectory)/_tasks/EsrpCodeSigning_*/*/net6.0/esrpcli.dll sign-darwin $(Pipeline.Workspace)/unsigned_vscode_client_darwin_$(VSCODE_ARCH)_archive VSCode-darwin-$(VSCODE_ARCH).zip
env:
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
displayName: Codesign
- script: node build/azure-pipelines/common/sign $(Agent.RootDirectory)/_tasks/EsrpCodeSigning_*/*/net6.0/esrpcli.dll notarize-darwin $(Pipeline.Workspace)/unsigned_vscode_client_darwin_$(VSCODE_ARCH)_archive VSCode-darwin-$(VSCODE_ARCH).zip
env:
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
displayName: Notarize
- script: unzip $(Pipeline.Workspace)/unsigned_vscode_client_darwin_$(VSCODE_ARCH)_archive/VSCode-darwin-$(VSCODE_ARCH).zip -d $(Agent.BuildDirectory)/VSCode-darwin-$(VSCODE_ARCH)
displayName: Extract signed app
- script: |
set -e
APP_ROOT="$(Agent.BuildDirectory)/VSCode-darwin-$(VSCODE_ARCH)"
APP_NAME="`ls $APP_ROOT | head -n 1`"
APP_PATH="$APP_ROOT/$APP_NAME"
codesign -dv --deep --verbose=4 "$APP_PATH"
"$APP_PATH/Contents/Resources/app/bin/code" --export-default-configuration=.build
displayName: Verify signature
condition: and(succeeded(), ne(variables['VSCODE_ARCH'], 'arm64'))
- script: mv $(Pipeline.Workspace)/unsigned_vscode_client_darwin_$(VSCODE_ARCH)_archive/VSCode-darwin-x64.zip $(Pipeline.Workspace)/unsigned_vscode_client_darwin_$(VSCODE_ARCH)_archive/VSCode-darwin.zip
displayName: Rename x64 build to its legacy name
condition: and(succeeded(), eq(variables['VSCODE_ARCH'], 'x64'))
- task: 1ES.PublishPipelineArtifact@1
inputs:
targetPath: $(Pipeline.Workspace)/unsigned_vscode_client_darwin_$(VSCODE_ARCH)_archive/VSCode-$(ASSET_ID).zip
artifactName: vscode_client_darwin_$(VSCODE_ARCH)_archive
sbomBuildDropPath: $(Agent.BuildDirectory)/VSCode-darwin-$(VSCODE_ARCH)
sbomPackageName: "VS Code macOS $(VSCODE_ARCH)"
sbomPackageVersion: $(Build.SourceVersion)
displayName: Publish client archive
================================================
FILE: build/azure-pipelines/darwin/product-build-darwin-test.yml
================================================
parameters:
- name: VSCODE_QUALITY
type: string
- name: VSCODE_RUN_UNIT_TESTS
type: boolean
- name: VSCODE_RUN_INTEGRATION_TESTS
type: boolean
- name: VSCODE_RUN_SMOKE_TESTS
type: boolean
steps:
- script: npm exec -- npm-run-all -lp "electron $(VSCODE_ARCH)" "playwright-install"
env:
GITHUB_TOKEN: "$(github-distro-mixin-password)"
displayName: Download Electron and Playwright
retryCountOnTaskFailure: 3
- ${{ if eq(parameters.VSCODE_RUN_UNIT_TESTS, true) }}:
- ${{ if eq(parameters.VSCODE_QUALITY, 'oss') }}:
- script: ./scripts/test.sh --tfs "Unit Tests"
displayName: Run unit tests (Electron)
timeoutInMinutes: 15
- script: npm run test-node
displayName: Run unit tests (node.js)
timeoutInMinutes: 15
- script: npm run test-browser-no-install -- --browser webkit --tfs "Browser Unit Tests"
env:
DEBUG: "*browser*"
displayName: Run unit tests (Browser, Webkit)
timeoutInMinutes: 30
- ${{ if ne(parameters.VSCODE_QUALITY, 'oss') }}:
- script: ./scripts/test.sh --build --tfs "Unit Tests"
displayName: Run unit tests (Electron)
timeoutInMinutes: 15
- script: npm run test-node -- --build
displayName: Run unit tests (node.js)
timeoutInMinutes: 15
- script: npm run test-browser-no-install -- --build --browser webkit --tfs "Browser Unit Tests"
env:
DEBUG: "*browser*"
displayName: Run unit tests (Browser, Webkit)
timeoutInMinutes: 30
- ${{ if eq(parameters.VSCODE_RUN_INTEGRATION_TESTS, true) }}:
- script: |
set -e
npm run gulp \
compile-extension:configuration-editing \
compile-extension:css-language-features-server \
compile-extension:emmet \
compile-extension:git \
compile-extension:github-authentication \
compile-extension:html-language-features-server \
compile-extension:ipynb \
compile-extension:notebook-renderers \
compile-extension:json-language-features-server \
compile-extension:markdown-language-features \
compile-extension-media \
compile-extension:microsoft-authentication \
compile-extension:typescript-language-features \
compile-extension:vscode-api-tests \
compile-extension:vscode-colorize-tests \
compile-extension:vscode-colorize-perf-tests \
compile-extension:vscode-test-resolver
displayName: Build integration tests
- ${{ if eq(parameters.VSCODE_QUALITY, 'oss') }}:
- script: ./scripts/test-integration --tfs "Integration Tests"
displayName: Run integration tests (Electron)
timeoutInMinutes: 20
- ${{ if ne(parameters.VSCODE_QUALITY, 'oss') }}:
- script: |
# Figure out the full absolute path of the product we just built
# including the remote server and configure the integration tests
# to run with these builds instead of running out of sources.
set -e
APP_ROOT="$(agent.builddirectory)/VSCode-darwin-$(VSCODE_ARCH)"
APP_NAME="`ls $APP_ROOT | head -n 1`"
INTEGRATION_TEST_ELECTRON_PATH="$APP_ROOT/$APP_NAME/Contents/MacOS/Electron" \
./scripts/test-integration.sh --build --tfs "Integration Tests"
env:
VSCODE_REMOTE_SERVER_PATH: $(agent.builddirectory)/vscode-server-darwin-$(VSCODE_ARCH)
displayName: Run integration tests (Electron)
timeoutInMinutes: 20
- script: ./scripts/test-web-integration.sh --browser webkit
env:
VSCODE_REMOTE_SERVER_PATH: $(agent.builddirectory)/vscode-server-darwin-$(VSCODE_ARCH)-web
displayName: Run integration tests (Browser, Webkit)
timeoutInMinutes: 20
- script: |
set -e
APP_ROOT=$(agent.builddirectory)/VSCode-darwin-$(VSCODE_ARCH)
APP_NAME="`ls $APP_ROOT | head -n 1`"
INTEGRATION_TEST_ELECTRON_PATH="$APP_ROOT/$APP_NAME/Contents/MacOS/Electron" \
./scripts/test-remote-integration.sh
env:
VSCODE_REMOTE_SERVER_PATH: $(agent.builddirectory)/vscode-server-darwin-$(VSCODE_ARCH)
displayName: Run integration tests (Remote)
timeoutInMinutes: 20
- ${{ if eq(parameters.VSCODE_RUN_SMOKE_TESTS, true) }}:
- script: ps -ef
displayName: Diagnostics before smoke test run
continueOnError: true
condition: succeededOrFailed()
- ${{ if eq(parameters.VSCODE_QUALITY, 'oss') }}:
- script: npm run compile
workingDirectory: test/smoke
displayName: Compile smoke tests
- script: npm run gulp compile-extension-media
displayName: Compile extensions for smoke tests
- script: npm run smoketest-no-compile -- --tracing
timeoutInMinutes: 20
displayName: Run smoke tests (Electron)
- ${{ if ne(parameters.VSCODE_QUALITY, 'oss') }}:
- script: |
set -e
APP_ROOT=$(agent.builddirectory)/VSCode-darwin-$(VSCODE_ARCH)
APP_NAME="`ls $APP_ROOT | head -n 1`"
npm run smoketest-no-compile -- --tracing --build "$APP_ROOT/$APP_NAME"
timeoutInMinutes: 20
displayName: Run smoke tests (Electron)
- script: npm run smoketest-no-compile -- --web --tracing --headless
env:
VSCODE_REMOTE_SERVER_PATH: $(agent.builddirectory)/vscode-server-darwin-$(VSCODE_ARCH)-web
timeoutInMinutes: 20
displayName: Run smoke tests (Browser, Chromium)
- script: |
set -e
npm run gulp compile-extension:vscode-test-resolver
APP_ROOT=$(agent.builddirectory)/VSCode-darwin-$(VSCODE_ARCH)
APP_NAME="`ls $APP_ROOT | head -n 1`"
npm run smoketest-no-compile -- --tracing --remote --build "$APP_ROOT/$APP_NAME"
env:
VSCODE_REMOTE_SERVER_PATH: $(agent.builddirectory)/vscode-server-darwin-$(VSCODE_ARCH)
timeoutInMinutes: 20
displayName: Run smoke tests (Remote)
- script: ps -ef
displayName: Diagnostics after smoke test run
continueOnError: true
condition: succeededOrFailed()
- ${{ if or(eq(parameters.VSCODE_RUN_INTEGRATION_TESTS, true), eq(parameters.VSCODE_RUN_SMOKE_TESTS, true)) }}:
- task: 1ES.PublishPipelineArtifact@1
inputs:
targetPath: .build/crashes
${{ if and(eq(parameters.VSCODE_RUN_INTEGRATION_TESTS, true), eq(parameters.VSCODE_RUN_SMOKE_TESTS, false)) }}:
artifactName: crash-dump-macos-$(VSCODE_ARCH)-integration-$(System.JobAttempt)
${{ elseif and(eq(parameters.VSCODE_RUN_INTEGRATION_TESTS, false), eq(parameters.VSCODE_RUN_SMOKE_TESTS, true)) }}:
artifactName: crash-dump-macos-$(VSCODE_ARCH)-smoke-$(System.JobAttempt)
${{ else }}:
artifactName: crash-dump-macos-$(VSCODE_ARCH)-$(System.JobAttempt)
sbomEnabled: false
displayName: "Publish Crash Reports"
continueOnError: true
condition: failed()
# In order to properly symbolify above crash reports
# (if any), we need the compiled native modules too
- task: 1ES.PublishPipelineArtifact@1
inputs:
targetPath: node_modules
${{ if and(eq(parameters.VSCODE_RUN_INTEGRATION_TESTS, true), eq(parameters.VSCODE_RUN_SMOKE_TESTS, false)) }}:
artifactName: node-modules-macos-$(VSCODE_ARCH)-integration-$(System.JobAttempt)
${{ elseif and(eq(parameters.VSCODE_RUN_INTEGRATION_TESTS, false), eq(parameters.VSCODE_RUN_SMOKE_TESTS, true)) }}:
artifactName: node-modules-macos-$(VSCODE_ARCH)-smoke-$(System.JobAttempt)
${{ else }}:
artifactName: node-modules-macos-$(VSCODE_ARCH)-$(System.JobAttempt)
sbomEnabled: false
displayName: "Publish Node Modules"
continueOnError: true
condition: failed()
- task: 1ES.PublishPipelineArtifact@1
inputs:
targetPath: .build/logs
${{ if and(eq(parameters.VSCODE_RUN_INTEGRATION_TESTS, true), eq(parameters.VSCODE_RUN_SMOKE_TESTS, false)) }}:
artifactName: logs-macos-$(VSCODE_ARCH)-integration-$(System.JobAttempt)
${{ elseif and(eq(parameters.VSCODE_RUN_INTEGRATION_TESTS, false), eq(parameters.VSCODE_RUN_SMOKE_TESTS, true)) }}:
artifactName: logs-macos-$(VSCODE_ARCH)-smoke-$(System.JobAttempt)
${{ else }}:
artifactName: logs-macos-$(VSCODE_ARCH)-$(System.JobAttempt)
sbomEnabled: false
displayName: "Publish Log Files"
continueOnError: true
condition: succeededOrFailed()
- task: PublishTestResults@2
displayName: Publish Tests Results
inputs:
testResultsFiles: "*-results.xml"
searchFolder: "$(Build.ArtifactStagingDirectory)/test-results"
condition: succeededOrFailed()
================================================
FILE: build/azure-pipelines/darwin/product-build-darwin-universal.yml
================================================
steps:
- task: NodeTool@0
inputs:
versionSource: fromFile
versionFilePath: .nvmrc
nodejsMirror: https://github.com/joaomoreno/node-mirror/releases/download
- template: ../distro/download-distro.yml@self
- task: AzureKeyVault@2
displayName: "Azure Key Vault: Get Secrets"
inputs:
azureSubscription: vscode
KeyVaultName: vscode-build-secrets
SecretsFilter: "github-distro-mixin-password,macos-developer-certificate,macos-developer-certificate-key"
- script: node build/setup-npm-registry.js $NPM_REGISTRY build
condition: and(succeeded(), ne(variables['NPM_REGISTRY'], 'none'))
displayName: Setup NPM Registry
- script: |
set -e
# Set the private NPM registry to the global npmrc file
# so that authentication works for subfolders like build/, remote/, extensions/ etc
# which does not have their own .npmrc file
npm config set registry "$NPM_REGISTRY"
echo "##vso[task.setvariable variable=NPMRC_PATH]$(npm config get userconfig)"
condition: and(succeeded(), ne(variables['NPM_REGISTRY'], 'none'))
displayName: Setup NPM
- task: npmAuthenticate@0
inputs:
workingFile: $(NPMRC_PATH)
condition: and(succeeded(), ne(variables['NPM_REGISTRY'], 'none'))
displayName: Setup NPM Authentication
- script: |
set -e
for i in {1..5}; do # try 5 times
npm ci && break
if [ $i -eq 5 ]; then
echo "Npm install failed too many times" >&2
exit 1
fi
echo "Npm install failed $i, trying again..."
done
workingDirectory: build
env:
GITHUB_TOKEN: "$(github-distro-mixin-password)"
displayName: Install build dependencies
- download: current
artifact: unsigned_vscode_client_darwin_x64_archive
displayName: Download x64 artifact
- download: current
artifact: unsigned_vscode_client_darwin_arm64_archive
displayName: Download arm64 artifact
- script: node build/azure-pipelines/distro/mixin-quality
displayName: Mixin distro quality
- script: |
set -e
unzip $(Pipeline.Workspace)/unsigned_vscode_client_darwin_x64_archive/VSCode-darwin-x64.zip -d $(agent.builddirectory)/VSCode-darwin-x64
unzip $(Pipeline.Workspace)/unsigned_vscode_client_darwin_arm64_archive/VSCode-darwin-arm64.zip -d $(agent.builddirectory)/VSCode-darwin-arm64
DEBUG=* node build/darwin/create-universal-app.js $(agent.builddirectory)
displayName: Create Universal App
- script: |
set -e
APP_ROOT="$(Agent.BuildDirectory)/VSCode-darwin-$(VSCODE_ARCH)"
APP_NAME="`ls $APP_ROOT | head -n 1`"
APP_PATH="$APP_ROOT/$APP_NAME" node build/darwin/verify-macho.js universal
displayName: Verify arch of Mach-O objects
- script: |
set -e
security create-keychain -p pwd $(agent.tempdirectory)/buildagent.keychain
security default-keychain -s $(agent.tempdirectory)/buildagent.keychain
security unlock-keychain -p pwd $(agent.tempdirectory)/buildagent.keychain
echo "$(macos-developer-certificate)" | base64 -D > $(agent.tempdirectory)/cert.p12
security import $(agent.tempdirectory)/cert.p12 -k $(agent.tempdirectory)/buildagent.keychain -P "$(macos-developer-certificate-key)" -T /usr/bin/codesign
export CODESIGN_IDENTITY=$(security find-identity -v -p codesigning $(agent.tempdirectory)/buildagent.keychain | grep -oEi "([0-9A-F]{40})" | head -n 1)
security set-key-partition-list -S apple-tool:,apple:,codesign: -s -k pwd $(agent.tempdirectory)/buildagent.keychain
DEBUG=electron-osx-sign* node build/darwin/sign.js $(agent.builddirectory)
displayName: Set Hardened Entitlements
- script: pushd $(agent.builddirectory)/VSCode-darwin-$(VSCODE_ARCH) && zip -r -X -y $(agent.builddirectory)/VSCode-darwin-$(VSCODE_ARCH).zip * && popd
displayName: Archive build
- task: 1ES.PublishPipelineArtifact@1
inputs:
targetPath: $(Agent.BuildDirectory)/VSCode-darwin-$(VSCODE_ARCH).zip
artifactName: unsigned_vscode_client_darwin_$(VSCODE_ARCH)_archive
sbomBuildDropPath: $(Agent.BuildDirectory)/VSCode-darwin-$(VSCODE_ARCH)
sbomPackageName: "VS Code macOS $(VSCODE_ARCH) (unsigned)"
sbomPackageVersion: $(Build.SourceVersion)
displayName: Publish client archive
================================================
FILE: build/azure-pipelines/darwin/product-build-darwin.yml
================================================
parameters:
- name: VSCODE_QUALITY
type: string
- name: VSCODE_CIBUILD
type: boolean
- name: VSCODE_RUN_UNIT_TESTS
type: boolean
- name: VSCODE_RUN_INTEGRATION_TESTS
type: boolean
- name: VSCODE_RUN_SMOKE_TESTS
type: boolean
steps:
- ${{ if eq(parameters.VSCODE_QUALITY, 'oss') }}:
- checkout: self
fetchDepth: 1
retryCountOnTaskFailure: 3
- task: NodeTool@0
inputs:
versionSource: fromFile
versionFilePath: .nvmrc
nodejsMirror: https://github.com/joaomoreno/node-mirror/releases/download
- ${{ if ne(parameters.VSCODE_QUALITY, 'oss') }}:
- template: ../distro/download-distro.yml@self
- task: AzureKeyVault@2
displayName: "Azure Key Vault: Get Secrets"
inputs:
azureSubscription: vscode
KeyVaultName: vscode-build-secrets
SecretsFilter: "github-distro-mixin-password,macos-developer-certificate,macos-developer-certificate-key"
- task: DownloadPipelineArtifact@2
inputs:
artifact: Compilation
path: $(Build.ArtifactStagingDirectory)
displayName: Download compilation output
- script: tar -xzf $(Build.ArtifactStagingDirectory)/compilation.tar.gz
displayName: Extract compilation output
- script: node build/setup-npm-registry.js $NPM_REGISTRY
condition: and(succeeded(), ne(variables['NPM_REGISTRY'], 'none'))
displayName: Setup NPM Registry
- script: mkdir -p .build && node build/azure-pipelines/common/computeNodeModulesCacheKey.js darwin $VSCODE_ARCH > .build/packagelockhash
displayName: Prepare node_modules cache key
- task: Cache@2
inputs:
key: '"node_modules" | .build/packagelockhash'
path: .build/node_modules_cache
cacheHitVar: NODE_MODULES_RESTORED
displayName: Restore node_modules cache
- script: tar -xzf .build/node_modules_cache/cache.tgz
condition: and(succeeded(), eq(variables.NODE_MODULES_RESTORED, 'true'))
displayName: Extract node_modules cache
- script: |
set -e
# Set the private NPM registry to the global npmrc file
# so that authentication works for subfolders like build/, remote/, extensions/ etc
# which does not have their own .npmrc file
npm config set registry "$NPM_REGISTRY"
echo "##vso[task.setvariable variable=NPMRC_PATH]$(npm config get userconfig)"
condition: and(succeeded(), ne(variables.NODE_MODULES_RESTORED, 'true'), ne(variables['NPM_REGISTRY'], 'none'))
displayName: Setup NPM
- task: npmAuthenticate@0
inputs:
workingFile: $(NPMRC_PATH)
condition: and(succeeded(), ne(variables.NODE_MODULES_RESTORED, 'true'), ne(variables['NPM_REGISTRY'], 'none'))
displayName: Setup NPM Authentication
- script: |
set -e
c++ --version
xcode-select -print-path
python3 -m pip install setuptools
for i in {1..5}; do # try 5 times
npm ci && break
if [ $i -eq 5 ]; then
echo "Npm install failed too many times" >&2
exit 1
fi
echo "Npm install failed $i, trying again..."
done
env:
npm_config_arch: $(VSCODE_ARCH)
ELECTRON_SKIP_BINARY_DOWNLOAD: 1
PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD: 1
GITHUB_TOKEN: "$(github-distro-mixin-password)"
# Avoid using dlopen to load Kerberos on macOS which can cause missing libraries
# https://github.com/mongodb-js/kerberos/commit/04044d2814ad1d01e77f1ce87f26b03d86692cf2
# flipped the default to support legacy linux distros which shouldn't happen
# on macOS.
GYP_DEFINES: "kerberos_use_rtld=false"
displayName: Install dependencies
condition: and(succeeded(), ne(variables.NODE_MODULES_RESTORED, 'true'))
- ${{ if ne(parameters.VSCODE_QUALITY, 'oss') }}:
- script: node build/azure-pipelines/distro/mixin-npm
condition: and(succeeded(), ne(variables.NODE_MODULES_RESTORED, 'true'))
displayName: Mixin distro node modules
- script: |
set -e
node build/azure-pipelines/common/listNodeModules.js .build/node_modules_list.txt
mkdir -p .build/node_modules_cache
tar -czf .build/node_modules_cache/cache.tgz --files-from .build/node_modules_list.txt
condition: and(succeeded(), ne(variables.NODE_MODULES_RESTORED, 'true'))
displayName: Create node_modules archive
- ${{ if ne(parameters.VSCODE_QUALITY, 'oss') }}:
- script: node build/azure-pipelines/distro/mixin-quality
displayName: Mixin distro quality
- template: ../common/install-builtin-extensions.yml@self
- ${{ if and(ne(parameters.VSCODE_CIBUILD, true), ne(parameters.VSCODE_QUALITY, 'oss')) }}:
- script: node build/lib/policies darwin
displayName: Generate policy definitions
retryCountOnTaskFailure: 3
- ${{ if ne(parameters.VSCODE_QUALITY, 'oss') }}:
- script: |
set -e
npm run gulp vscode-darwin-$(VSCODE_ARCH)-min-ci
echo "##vso[task.setvariable variable=BUILT_CLIENT]true"
env:
GITHUB_TOKEN: "$(github-distro-mixin-password)"
displayName: Build client
- script: |
set -e
npm run gulp vscode-reh-darwin-$(VSCODE_ARCH)-min-ci
mv ../vscode-reh-darwin-$(VSCODE_ARCH) ../vscode-server-darwin-$(VSCODE_ARCH) # TODO@joaomoreno
ARCHIVE_PATH=".build/darwin/server/vscode-server-darwin-$(VSCODE_ARCH).zip"
mkdir -p $(dirname $ARCHIVE_PATH)
(cd .. && zip -Xry $(Build.SourcesDirectory)/$ARCHIVE_PATH vscode-server-darwin-$(VSCODE_ARCH))
echo "##vso[task.setvariable variable=SERVER_PATH]$ARCHIVE_PATH"
env:
GITHUB_TOKEN: "$(github-distro-mixin-password)"
displayName: Build server
- script: |
set -e
npm run gulp vscode-reh-web-darwin-$(VSCODE_ARCH)-min-ci
mv ../vscode-reh-web-darwin-$(VSCODE_ARCH) ../vscode-server-darwin-$(VSCODE_ARCH)-web # TODO@joaomoreno
ARCHIVE_PATH=".build/darwin/server/vscode-server-darwin-$(VSCODE_ARCH)-web.zip"
mkdir -p $(dirname $ARCHIVE_PATH)
(cd .. && zip -Xry $(Build.SourcesDirectory)/$ARCHIVE_PATH vscode-server-darwin-$(VSCODE_ARCH)-web)
echo "##vso[task.setvariable variable=WEB_PATH]$ARCHIVE_PATH"
env:
GITHUB_TOKEN: "$(github-distro-mixin-password)"
displayName: Build server (web)
- ${{ else }}:
- script: npm run gulp transpile-client-esbuild transpile-extensions
env:
GITHUB_TOKEN: "$(github-distro-mixin-password)"
displayName: Transpile
- ${{ if or(eq(parameters.VSCODE_RUN_UNIT_TESTS, true), eq(parameters.VSCODE_RUN_INTEGRATION_TESTS, true), eq(parameters.VSCODE_RUN_SMOKE_TESTS, true)) }}:
- template: product-build-darwin-test.yml@self
parameters:
VSCODE_QUALITY: ${{ parameters.VSCODE_QUALITY }}
VSCODE_RUN_UNIT_TESTS: ${{ parameters.VSCODE_RUN_UNIT_TESTS }}
VSCODE_RUN_INTEGRATION_TESTS: ${{ parameters.VSCODE_RUN_INTEGRATION_TESTS }}
VSCODE_RUN_SMOKE_TESTS: ${{ parameters.VSCODE_RUN_SMOKE_TESTS }}
- ${{ elseif and(ne(parameters.VSCODE_CIBUILD, true), ne(parameters.VSCODE_QUALITY, 'oss')) }}:
- task: DownloadPipelineArtifact@2
inputs:
artifact: unsigned_vscode_cli_darwin_$(VSCODE_ARCH)_cli
patterns: "**"
path: $(Build.ArtifactStagingDirectory)/cli
displayName: Download VS Code CLI
- script: |
set -e
APP_ROOT="$(Agent.BuildDirectory)/VSCode-darwin-$(VSCODE_ARCH)"
APP_NAME="`ls $APP_ROOT | head -n 1`"
APP_PATH="$APP_ROOT/$APP_NAME"
unzip $(Build.ArtifactStagingDirectory)/cli/*.zip -d $(Build.ArtifactStagingDirectory)/cli
CLI_APP_NAME=$(node -p "require(\"$APP_PATH/Contents/Resources/app/product.json\").tunnelApplicationName")
APP_NAME=$(node -p "require(\"$APP_PATH/Contents/Resources/app/product.json\").applicationName")
mv "$(Build.ArtifactStagingDirectory)/cli/$APP_NAME" "$APP_PATH/Contents/Resources/app/bin/$CLI_APP_NAME"
chmod +x "$APP_PATH/Contents/Resources/app/bin/$CLI_APP_NAME"
displayName: Make CLI executable
- script: |
set -e
APP_ROOT="$(Agent.BuildDirectory)/VSCode-darwin-$(VSCODE_ARCH)"
APP_NAME="`ls $APP_ROOT | head -n 1`"
APP_PATH="$APP_ROOT/$APP_NAME" node build/darwin/verify-macho.js $(VSCODE_ARCH)
APP_PATH="$(Agent.BuildDirectory)/vscode-server-darwin-$(VSCODE_ARCH)" node build/darwin/verify-macho.js $(VSCODE_ARCH)
displayName: Verify arch of Mach-O objects
# Setting hardened entitlements is a requirement for:
# * Apple notarization
# * Running tests on Big Sur (because Big Sur has additional security precautions)
- script: |
set -e
security create-keychain -p pwd $(agent.tempdirectory)/buildagent.keychain
security default-keychain -s $(agent.tempdirectory)/buildagent.keychain
security unlock-keychain -p pwd $(agent.tempdirectory)/buildagent.keychain
echo "$(macos-developer-certificate)" | base64 -D > $(agent.tempdirectory)/cert.p12
security import $(agent.tempdirectory)/cert.p12 -k $(agent.tempdirectory)/buildagent.keychain -P "$(macos-developer-certificate-key)" -T /usr/bin/codesign
export CODESIGN_IDENTITY=$(security find-identity -v -p codesigning $(agent.tempdirectory)/buildagent.keychain | grep -oEi "([0-9A-F]{40})" | head -n 1)
security set-key-partition-list -S apple-tool:,apple:,codesign: -s -k pwd $(agent.tempdirectory)/buildagent.keychain
DEBUG=electron-osx-sign* node build/darwin/sign.js $(agent.builddirectory)
displayName: Set Hardened Entitlements
- script: |
set -e
ARCHIVE_PATH=".build/darwin/client/VSCode-darwin-$(VSCODE_ARCH).zip"
mkdir -p $(dirname $ARCHIVE_PATH)
(cd ../VSCode-darwin-$(VSCODE_ARCH) && zip -Xry $(Build.SourcesDirectory)/$ARCHIVE_PATH *)
echo "##vso[task.setvariable variable=CLIENT_PATH]$ARCHIVE_PATH"
condition: and(succeededOrFailed(), eq(variables['BUILT_CLIENT'], 'true'))
displayName: Package client
- script: echo "##vso[task.setvariable variable=ARTIFACT_PREFIX]attempt$(System.JobAttempt)_"
condition: and(succeededOrFailed(), notIn(variables['Agent.JobStatus'], 'Succeeded', 'SucceededWithIssues'))
displayName: Generate artifact prefix
- task: 1ES.PublishPipelineArtifact@1
inputs:
targetPath: $(CLIENT_PATH)
artifactName: $(ARTIFACT_PREFIX)unsigned_vscode_client_darwin_$(VSCODE_ARCH)_archive
sbomBuildDropPath: $(Agent.BuildDirectory)/VSCode-darwin-$(VSCODE_ARCH)
sbomPackageName: "VS Code macOS $(VSCODE_ARCH) (unsigned)"
sbomPackageVersion: $(Build.SourceVersion)
displayName: Publish client archive
- task: 1ES.PublishPipelineArtifact@1
inputs:
targetPath: $(SERVER_PATH)
artifactName: $(ARTIFACT_PREFIX)vscode_server_darwin_$(VSCODE_ARCH)_archive-unsigned
sbomBuildDropPath: $(Agent.BuildDirectory)/vscode-server-darwin-$(VSCODE_ARCH)
sbomPackageName: "VS Code macOS $(VSCODE_ARCH) Server"
sbomPackageVersion: $(Build.SourceVersion)
condition: and(succeededOrFailed(), ne(variables['SERVER_PATH'], ''))
displayName: Publish server archive
- task: 1ES.PublishPipelineArtifact@1
inputs:
targetPath: $(WEB_PATH)
artifactName: $(ARTIFACT_PREFIX)vscode_web_darwin_$(VSCODE_ARCH)_archive-unsigned
sbomBuildDropPath: $(Agent.BuildDirectory)/vscode-server-darwin-$(VSCODE_ARCH)-web
sbomPackageName: "VS Code macOS $(VSCODE_ARCH) Web"
sbomPackageVersion: $(Build.SourceVersion)
condition: and(succeededOrFailed(), ne(variables['WEB_PATH'], ''))
displayName: Publish web server archive
================================================
FILE: build/azure-pipelines/distro/download-distro.yml
================================================
steps:
- task: AzureKeyVault@2
displayName: "Azure Key Vault: Get Secrets"
inputs:
azureSubscription: vscode
KeyVaultName: vscode-build-secrets
SecretsFilter: "github-distro-mixin-password"
# TODO@joaomoreno: Keep pwsh once we move out of running entire jobs in containers
- pwsh: |
"machine github.com`nlogin vscode`npassword $(github-distro-mixin-password)" | Out-File "$Home/_netrc" -Encoding ASCII
condition: and(succeeded(), contains(variables['Agent.OS'], 'windows'))
displayName: Setup distro auth (Windows)
- pwsh: |
$ErrorActionPreference = "Stop"
$ArchivePath = "$(Agent.TempDirectory)/distro.zip"
$PackageJson = Get-Content -Path package.json -Raw | ConvertFrom-Json
$DistroVersion = $PackageJson.distro
Invoke-WebRequest -Uri "https://api.github.com/repos/microsoft/vscode-distro/zipball/$DistroVersion" `
-OutFile $ArchivePath `
-Headers @{ "Accept" = "application/vnd.github+json"; "Authorization" = "Bearer $(github-distro-mixin-password)"; "X-GitHub-Api-Version" = "2022-11-28" }
New-Item -ItemType Directory -Path .build -Force
Expand-Archive -Path $ArchivePath -DestinationPath .build
Rename-Item -Path ".build/microsoft-vscode-distro-$DistroVersion" -NewName distro
condition: and(succeeded(), contains(variables['Agent.OS'], 'windows'))
displayName: Download distro (Windows)
- script: |
mkdir -p .build
cat << EOF | tee ~/.netrc .build/.netrc > /dev/null
machine github.com
login vscode
password $(github-distro-mixin-password)
EOF
condition: and(succeeded(), not(contains(variables['Agent.OS'], 'windows')))
displayName: Setup distro auth (non-Windows)
- script: |
set -e
ArchivePath="$(Agent.TempDirectory)/distro.zip"
DistroVersion=$(node -p "require('./package.json').distro")
curl -H "Accept: application/vnd.github+json" \
-H "Authorization: Bearer $(github-distro-mixin-password)" \
-H "X-GitHub-Api-Version: 2022-11-28" \
-o $ArchivePath \
-L "https://api.github.com/repos/microsoft/vscode-distro/zipball/$DistroVersion"
unzip $ArchivePath -d .build
mv .build/microsoft-vscode-distro-$DistroVersion .build/distro
condition: and(succeeded(), not(contains(variables['Agent.OS'], 'windows')))
displayName: Download distro (non-Windows)
================================================
FILE: build/azure-pipelines/distro/mixin-npm.js
================================================
"use strict";
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
Object.defineProperty(exports, "__esModule", { value: true });
const fs_1 = __importDefault(require("fs"));
const path_1 = __importDefault(require("path"));
const { dirs } = require('../../npm/dirs');
function log(...args) {
console.log(`[${new Date().toLocaleTimeString('en', { hour12: false })}]`, '[distro]', ...args);
}
function mixin(mixinPath) {
if (!fs_1.default.existsSync(`${mixinPath}/node_modules`)) {
log(`Skipping distro npm dependencies: ${mixinPath} (no node_modules)`);
return;
}
log(`Mixing in distro npm dependencies: ${mixinPath}`);
const distroPackageJson = JSON.parse(fs_1.default.readFileSync(`${mixinPath}/package.json`, 'utf8'));
const targetPath = path_1.default.relative('.build/distro/npm', mixinPath);
for (const dependency of Object.keys(distroPackageJson.dependencies)) {
fs_1.default.rmSync(`./${targetPath}/node_modules/${dependency}`, { recursive: true, force: true });
fs_1.default.cpSync(`${mixinPath}/node_modules/${dependency}`, `./${targetPath}/node_modules/${dependency}`, { recursive: true, force: true, dereference: true });
}
log(`Mixed in distro npm dependencies: ${mixinPath} ✔︎`);
}
function main() {
log(`Mixing in distro npm dependencies...`);
const mixinPaths = dirs.filter(d => /^.build\/distro\/npm/.test(d));
for (const mixinPath of mixinPaths) {
mixin(mixinPath);
}
}
main();
//# sourceMappingURL=mixin-npm.js.map
================================================
FILE: build/azure-pipelines/distro/mixin-npm.ts
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
import fs from 'fs';
import path from 'path';
const { dirs } = require('../../npm/dirs') as { dirs: string[] };
function log(...args: any[]): void {
console.log(`[${new Date().toLocaleTimeString('en', { hour12: false })}]`, '[distro]', ...args);
}
function mixin(mixinPath: string) {
if (!fs.existsSync(`${mixinPath}/node_modules`)) {
log(`Skipping distro npm dependencies: ${mixinPath} (no node_modules)`);
return;
}
log(`Mixing in distro npm dependencies: ${mixinPath}`);
const distroPackageJson = JSON.parse(fs.readFileSync(`${mixinPath}/package.json`, 'utf8'));
const targetPath = path.relative('.build/distro/npm', mixinPath);
for (const dependency of Object.keys(distroPackageJson.dependencies)) {
fs.rmSync(`./${targetPath}/node_modules/${dependency}`, { recursive: true, force: true });
fs.cpSync(`${mixinPath}/node_modules/${dependency}`, `./${targetPath}/node_modules/${dependency}`, { recursive: true, force: true, dereference: true });
}
log(`Mixed in distro npm dependencies: ${mixinPath} ✔︎`);
}
function main() {
log(`Mixing in distro npm dependencies...`);
const mixinPaths = dirs.filter(d => /^.build\/distro\/npm/.test(d));
for (const mixinPath of mixinPaths) {
mixin(mixinPath);
}
}
main();
================================================
FILE: build/azure-pipelines/distro/mixin-quality.js
================================================
"use strict";
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
Object.defineProperty(exports, "__esModule", { value: true });
const fs_1 = __importDefault(require("fs"));
const path_1 = __importDefault(require("path"));
function log(...args) {
console.log(`[${new Date().toLocaleTimeString('en', { hour12: false })}]`, '[distro]', ...args);
}
function main() {
const quality = process.env['VSCODE_QUALITY'];
if (!quality) {
throw new Error('Missing VSCODE_QUALITY, skipping mixin');
}
log(`Mixing in distro quality...`);
const basePath = `.build/distro/mixin/${quality}`;
for (const name of fs_1.default.readdirSync(basePath)) {
const distroPath = path_1.default.join(basePath, name);
const ossPath = path_1.default.relative(basePath, distroPath);
if (ossPath === 'product.json') {
const distro = JSON.parse(fs_1.default.readFileSync(distroPath, 'utf8'));
const oss = JSON.parse(fs_1.default.readFileSync(ossPath, 'utf8'));
let builtInExtensions = oss.builtInExtensions;
if (Array.isArray(distro.builtInExtensions)) {
log('Overwriting built-in extensions:', distro.builtInExtensions.map(e => e.name));
builtInExtensions = distro.builtInExtensions;
}
else if (distro.builtInExtensions) {
const include = distro.builtInExtensions['include'] ?? [];
const exclude = distro.builtInExtensions['exclude'] ?? [];
log('OSS built-in extensions:', builtInExtensions.map(e => e.name));
log('Including built-in extensions:', include.map(e => e.name));
log('Excluding built-in extensions:', exclude);
builtInExtensions = builtInExtensions.filter(ext => !include.find(e => e.name === ext.name) && !exclude.find(name => name === ext.name));
builtInExtensions = [...builtInExtensions, ...include];
log('Final built-in extensions:', builtInExtensions.map(e => e.name));
}
else {
log('Inheriting OSS built-in extensions', builtInExtensions.map(e => e.name));
}
const result = { webBuiltInExtensions: oss.webBuiltInExtensions, ...distro, builtInExtensions };
fs_1.default.writeFileSync(ossPath, JSON.stringify(result, null, '\t'), 'utf8');
}
else {
fs_1.default.cpSync(distroPath, ossPath, { force: true, recursive: true });
}
log(distroPath, '✔︎');
}
}
main();
//# sourceMappingURL=mixin-quality.js.map
================================================
FILE: build/azure-pipelines/distro/mixin-quality.ts
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
import fs from 'fs';
import path from 'path';
interface IBuiltInExtension {
readonly name: string;
readonly version: string;
readonly repo: string;
readonly metadata: any;
}
interface OSSProduct {
readonly builtInExtensions: IBuiltInExtension[];
readonly webBuiltInExtensions?: IBuiltInExtension[];
}
interface Product {
readonly builtInExtensions?: IBuiltInExtension[] | { 'include'?: IBuiltInExtension[]; 'exclude'?: string[] };
readonly webBuiltInExtensions?: IBuiltInExtension[];
}
function log(...args: any[]): void {
console.log(`[${new Date().toLocaleTimeString('en', { hour12: false })}]`, '[distro]', ...args);
}
function main() {
const quality = process.env['VSCODE_QUALITY'];
if (!quality) {
throw new Error('Missing VSCODE_QUALITY, skipping mixin');
}
log(`Mixing in distro quality...`);
const basePath = `.build/distro/mixin/${quality}`;
for (const name of fs.readdirSync(basePath)) {
const distroPath = path.join(basePath, name);
const ossPath = path.relative(basePath, distroPath);
if (ossPath === 'product.json') {
const distro = JSON.parse(fs.readFileSync(distroPath, 'utf8')) as Product;
const oss = JSON.parse(fs.readFileSync(ossPath, 'utf8')) as OSSProduct;
let builtInExtensions = oss.builtInExtensions;
if (Array.isArray(distro.builtInExtensions)) {
log('Overwriting built-in extensions:', distro.builtInExtensions.map(e => e.name));
builtInExtensions = distro.builtInExtensions;
} else if (distro.builtInExtensions) {
const include = distro.builtInExtensions['include'] ?? [];
const exclude = distro.builtInExtensions['exclude'] ?? [];
log('OSS built-in extensions:', builtInExtensions.map(e => e.name));
log('Including built-in extensions:', include.map(e => e.name));
log('Excluding built-in extensions:', exclude);
builtInExtensions = builtInExtensions.filter(ext => !include.find(e => e.name === ext.name) && !exclude.find(name => name === ext.name));
builtInExtensions = [...builtInExtensions, ...include];
log('Final built-in extensions:', builtInExtensions.map(e => e.name));
} else {
log('Inheriting OSS built-in extensions', builtInExtensions.map(e => e.name));
}
const result = { webBuiltInExtensions: oss.webBuiltInExtensions, ...distro, builtInExtensions };
fs.writeFileSync(ossPath, JSON.stringify(result, null, '\t'), 'utf8');
} else {
fs.cpSync(distroPath, ossPath, { force: true, recursive: true });
}
log(distroPath, '✔︎');
}
}
main();
================================================
FILE: build/azure-pipelines/distro-build.yml
================================================
pool:
name: 1es-ubuntu-22.04-x64
os: linux
trigger:
branches:
include: ["main", "release/*"]
pr: none
steps:
- task: NodeTool@0
inputs:
versionSource: fromFile
versionFilePath: .nvmrc
nodejsMirror: https://github.com/joaomoreno/node-mirror/releases/download
- template: ./distro/download-distro.yml@self
================================================
FILE: build/azure-pipelines/linux/.gitignore
================================================
pat
================================================
FILE: build/azure-pipelines/linux/apt-retry.sh
================================================
#!/bin/sh
################################################################################
## Copied from https://github.com/actions/runner-images/blob/ubuntu22/20240825.1/images/ubuntu/scripts/build/configure-apt-mock.sh
################################################################################
i=1
while [ $i -le 30 ];do
err=$(mktemp)
"$@" 2>$err
# no errors, break the loop and continue normal flow
test -f $err || break
cat $err >&2
retry=false
if grep -q 'Could not get lock' $err;then
# apt db locked needs retry
retry=true
elif grep -q 'Could not open file /var/lib/apt/lists' $err;then
# apt update is not completed, needs retry
retry=true
elif grep -q 'IPC connect call failed' $err;then
# the delay should help with gpg-agent not ready
retry=true
elif grep -q 'Temporary failure in name resolution' $err;then
# It looks like DNS is not updated with random generated hostname yet
retry=true
elif grep -q 'dpkg frontend is locked by another process' $err;then
# dpkg process is busy by another process
retry=true
fi
rm $err
if [ $retry = false ]; then
break
fi
sleep 5
echo "...retry $i"
i=$((i + 1))
done
================================================
FILE: build/azure-pipelines/linux/cli-build-linux.yml
================================================
parameters:
- name: VSCODE_BUILD_LINUX
type: boolean
default: false
- name: VSCODE_BUILD_LINUX_ARM64
type: boolean
default: false
- name: VSCODE_BUILD_LINUX_ARMHF
type: boolean
default: false
- name: VSCODE_CHECK_ONLY
type: boolean
default: false
- name: VSCODE_QUALITY
type: string
steps:
- task: NodeTool@0
inputs:
versionSource: fromFile
versionFilePath: .nvmrc
nodejsMirror: https://github.com/joaomoreno/node-mirror/releases/download
- ${{ if ne(parameters.VSCODE_QUALITY, 'oss') }}:
- template: ../cli/cli-apply-patches.yml@self
- task: Npm@1
displayName: Download openssl prebuilt
inputs:
command: custom
customCommand: pack @vscode-internal/openssl-prebuilt@0.0.11
customRegistry: useFeed
customFeed: "Monaco/openssl-prebuilt"
workingDir: $(Build.ArtifactStagingDirectory)
- script: |
set -e
mkdir $(Build.ArtifactStagingDirectory)/openssl
tar -xvzf $(Build.ArtifactStagingDirectory)/vscode-internal-openssl-prebuilt-0.0.11.tgz --strip-components=1 --directory=$(Build.ArtifactStagingDirectory)/openssl
displayName: Extract openssl prebuilt
- ${{ if ne(parameters.VSCODE_QUALITY, 'oss') }}:
- script: node build/setup-npm-registry.js $NPM_REGISTRY build
condition: and(succeeded(), ne(variables['NPM_REGISTRY'], 'none'))
displayName: Setup NPM Registry
- script: |
set -e
# Set the private NPM registry to the global npmrc file
# so that authentication works for subfolders like build/, remote/, extensions/ etc
# which does not have their own .npmrc file
npm config set registry "$NPM_REGISTRY"
echo "##vso[task.setvariable variable=NPMRC_PATH]$(npm config get userconfig)"
condition: and(succeeded(), ne(variables['NPM_REGISTRY'], 'none'))
displayName: Setup NPM
- task: npmAuthenticate@0
inputs:
workingFile: $(NPMRC_PATH)
condition: and(succeeded(), ne(variables['NPM_REGISTRY'], 'none'))
displayName: Setup NPM Authentication
- script: |
set -e
for i in {1..5}; do # try 5 times
npm ci && break
if [ $i -eq 5 ]; then
echo "Npm install failed too many times" >&2
exit 1
fi
echo "Npm install failed $i, trying again..."
done
workingDirectory: build
env:
GITHUB_TOKEN: "$(github-distro-mixin-password)"
displayName: Install build dependencies
- script: |
set -e
mkdir -p $(Build.SourcesDirectory)/.build
displayName: Create .build folder for misc dependencies
- template: ../cli/install-rust-posix.yml@self
parameters:
targets:
- ${{ if eq(parameters.VSCODE_BUILD_LINUX_ARM64, true) }}:
- aarch64-unknown-linux-gnu
- ${{ if eq(parameters.VSCODE_BUILD_LINUX, true) }}:
- x86_64-unknown-linux-gnu
- ${{ if eq(parameters.VSCODE_BUILD_LINUX_ARMHF, true) }}:
- armv7-unknown-linux-gnueabihf
- ${{ if eq(parameters.VSCODE_BUILD_LINUX_ARM64, true) }}:
- template: ../cli/cli-compile.yml@self
parameters:
VSCODE_QUALITY: ${{ parameters.VSCODE_QUALITY }}
VSCODE_CLI_TARGET: aarch64-unknown-linux-gnu
VSCODE_CLI_ARTIFACT: vscode_cli_linux_arm64_cli
VSCODE_CHECK_ONLY: ${{ parameters.VSCODE_CHECK_ONLY }}
VSCODE_CLI_ENV:
OPENSSL_LIB_DIR: $(Build.ArtifactStagingDirectory)/openssl/arm64-linux/lib
OPENSSL_INCLUDE_DIR: $(Build.ArtifactStagingDirectory)/openssl/arm64-linux/include
SYSROOT_ARCH: arm64
- ${{ if eq(parameters.VSCODE_BUILD_LINUX, true) }}:
- template: ../cli/cli-compile.yml@self
parameters:
VSCODE_QUALITY: ${{ parameters.VSCODE_QUALITY }}
VSCODE_CLI_TARGET: x86_64-unknown-linux-gnu
VSCODE_CLI_ARTIFACT: vscode_cli_linux_x64_cli
VSCODE_CHECK_ONLY: ${{ parameters.VSCODE_CHECK_ONLY }}
VSCODE_CLI_ENV:
OPENSSL_LIB_DIR: $(Build.ArtifactStagingDirectory)/openssl/x64-linux/lib
OPENSSL_INCLUDE_DIR: $(Build.ArtifactStagingDirectory)/openssl/x64-linux/include
SYSROOT_ARCH: amd64
- ${{ if eq(parameters.VSCODE_BUILD_LINUX_ARMHF, true) }}:
- template: ../cli/cli-compile.yml@self
parameters:
VSCODE_QUALITY: ${{ parameters.VSCODE_QUALITY }}
VSCODE_CLI_TARGET: armv7-unknown-linux-gnueabihf
VSCODE_CLI_ARTIFACT: vscode_cli_linux_armhf_cli
VSCODE_CHECK_ONLY: ${{ parameters.VSCODE_CHECK_ONLY }}
VSCODE_CLI_ENV:
OPENSSL_LIB_DIR: $(Build.ArtifactStagingDirectory)/openssl/arm-linux/lib
OPENSSL_INCLUDE_DIR: $(Build.ArtifactStagingDirectory)/openssl/arm-linux/include
SYSROOT_ARCH: armhf
- ${{ if not(parameters.VSCODE_CHECK_ONLY) }}:
- ${{ if eq(parameters.VSCODE_BUILD_LINUX_ARMHF, true) }}:
- task: 1ES.PublishPipelineArtifact@1
inputs:
targetPath: $(Build.ArtifactStagingDirectory)/vscode_cli_linux_armhf_cli.tar.gz
artifactName: vscode_cli_linux_armhf_cli
sbomBuildDropPath: $(Build.ArtifactStagingDirectory)/cli
sbomPackageName: "VS Code Linux armhf CLI"
sbomPackageVersion: $(Build.SourceVersion)
displayName: Publish vscode_cli_linux_armhf_cli artifact
- ${{ if eq(parameters.VSCODE_BUILD_LINUX, true) }}:
- task: 1ES.PublishPipelineArtifact@1
inputs:
targetPath: $(Build.ArtifactStagingDirectory)/vscode_cli_linux_x64_cli.tar.gz
artifactName: vscode_cli_linux_x64_cli
sbomBuildDropPath: $(Build.ArtifactStagingDirectory)/cli
sbomPackageName: "VS Code Linux x64 CLI"
sbomPackageVersion: $(Build.SourceVersion)
displayName: Publish vscode_cli_linux_x64_cli artifact
- ${{ if eq(parameters.VSCODE_BUILD_LINUX_ARM64, true) }}:
- task: 1ES.PublishPipelineArtifact@1
inputs:
targetPath: $(Build.ArtifactStagingDirectory)/vscode_cli_linux_arm64_cli.tar.gz
artifactName: vscode_cli_linux_arm64_cli
sbomBuildDropPath: $(Build.ArtifactStagingDirectory)/cli
sbomPackageName: "VS Code Linux arm64 CLI"
sbomPackageVersion: $(Build.SourceVersion)
displayName: Publish vscode_cli_linux_arm64_cli artifact
================================================
FILE: build/azure-pipelines/linux/product-build-linux-test.yml
================================================
parameters:
- name: VSCODE_QUALITY
type: string
- name: VSCODE_RUN_UNIT_TESTS
type: boolean
- name: VSCODE_RUN_INTEGRATION_TESTS
type: boolean
- name: VSCODE_RUN_SMOKE_TESTS
type: boolean
- name: PUBLISH_TASK_NAME
type: string
default: PublishPipelineArtifact@0
steps:
- script: npm exec -- npm-run-all -lp "electron $(VSCODE_ARCH)" "playwright-install"
env:
GITHUB_TOKEN: "$(github-distro-mixin-password)"
displayName: Download Electron and Playwright
retryCountOnTaskFailure: 3
- ${{ if ne(parameters.VSCODE_QUALITY, 'oss') }}:
- script: |
set -e
APP_ROOT=$(agent.builddirectory)/VSCode-linux-$(VSCODE_ARCH)
ELECTRON_ROOT=.build/electron
sudo chown root $APP_ROOT/chrome-sandbox
sudo chown root $ELECTRON_ROOT/chrome-sandbox
sudo chmod 4755 $APP_ROOT/chrome-sandbox
sudo chmod 4755 $ELECTRON_ROOT/chrome-sandbox
stat $APP_ROOT/chrome-sandbox
stat $ELECTRON_ROOT/chrome-sandbox
displayName: Change setuid helper binary permission
- ${{ if eq(parameters.VSCODE_RUN_UNIT_TESTS, true) }}:
- ${{ if eq(parameters.VSCODE_QUALITY, 'oss') }}:
- script: ./scripts/test.sh --tfs "Unit Tests"
env:
DISPLAY: ":10"
displayName: Run unit tests (Electron)
timeoutInMinutes: 15
- script: npm run test-node
displayName: Run unit tests (node.js)
timeoutInMinutes: 15
- script: npm run test-browser-no-install -- --browser chromium --tfs "Browser Unit Tests"
env:
DEBUG: "*browser*"
displayName: Run unit tests (Browser, Chromium)
timeoutInMinutes: 15
- ${{ if ne(parameters.VSCODE_QUALITY, 'oss') }}:
- script: ./scripts/test.sh --build --tfs "Unit Tests"
displayName: Run unit tests (Electron)
timeoutInMinutes: 15
- script: npm run test-node -- --build
displayName: Run unit tests (node.js)
timeoutInMinutes: 15
- script: npm run test-browser-no-install -- --build --browser chromium --tfs "Browser Unit Tests"
env:
DEBUG: "*browser*"
displayName: Run unit tests (Browser, Chromium)
timeoutInMinutes: 15
- ${{ if eq(parameters.VSCODE_RUN_INTEGRATION_TESTS, true) }}:
- script: |
set -e
npm run gulp \
compile-extension:configuration-editing \
compile-extension:css-language-features-server \
compile-extension:emmet \
compile-extension:git \
compile-extension:github-authentication \
compile-extension:html-language-features-server \
compile-extension:ipynb \
compile-extension:notebook-renderers \
compile-extension:json-language-features-server \
compile-extension:markdown-language-features \
compile-extension-media \
compile-extension:microsoft-authentication \
compile-extension:typescript-language-features \
compile-extension:vscode-api-tests \
compile-extension:vscode-colorize-tests \
compile-extension:vscode-colorize-perf-tests \
compile-extension:vscode-test-resolver
displayName: Build integration tests
- ${{ if eq(parameters.VSCODE_RUN_INTEGRATION_TESTS, true) }}:
- ${{ if eq(parameters.VSCODE_QUALITY, 'oss') }}:
- script: ./scripts/test-integration.sh --tfs "Integration Tests"
env:
DISPLAY: ":10"
displayName: Run integration tests (Electron)
timeoutInMinutes: 20
- script: ./scripts/test-web-integration.sh --browser chromium
displayName: Run integration tests (Browser, Chromium)
timeoutInMinutes: 20
- script: ./scripts/test-remote-integration.sh
displayName: Run integration tests (Remote)
timeoutInMinutes: 20
- ${{ if ne(parameters.VSCODE_QUALITY, 'oss') }}:
- script: |
# Figure out the full absolute path of the product we just built
# including the remote server and configure the integration tests
# to run with these builds instead of running out of sources.
set -e
APP_ROOT=$(agent.builddirectory)/VSCode-linux-$(VSCODE_ARCH)
APP_NAME=$(node -p "require(\"$APP_ROOT/resources/app/product.json\").applicationName")
INTEGRATION_TEST_APP_NAME="$APP_NAME" \
INTEGRATION_TEST_ELECTRON_PATH="$APP_ROOT/$APP_NAME" \
./scripts/test-integration.sh --build --tfs "Integration Tests"
env:
VSCODE_REMOTE_SERVER_PATH: $(agent.builddirectory)/vscode-server-linux-$(VSCODE_ARCH)
displayName: Run integration tests (Electron)
timeoutInMinutes: 20
- script: ./scripts/test-web-integration.sh --browser chromium
env:
VSCODE_REMOTE_SERVER_PATH: $(agent.builddirectory)/vscode-server-linux-$(VSCODE_ARCH)-web
displayName: Run integration tests (Browser, Chromium)
timeoutInMinutes: 20
- script: |
set -e
APP_ROOT=$(agent.builddirectory)/VSCode-linux-$(VSCODE_ARCH)
APP_NAME=$(node -p "require(\"$APP_ROOT/resources/app/product.json\").applicationName")
INTEGRATION_TEST_APP_NAME="$APP_NAME" \
INTEGRATION_TEST_ELECTRON_PATH="$APP_ROOT/$APP_NAME" \
./scripts/test-remote-integration.sh
env:
VSCODE_REMOTE_SERVER_PATH: $(agent.builddirectory)/vscode-server-linux-$(VSCODE_ARCH)
displayName: Run integration tests (Remote)
timeoutInMinutes: 20
- ${{ if eq(parameters.VSCODE_RUN_SMOKE_TESTS, true) }}:
- script: |
set -e
ps -ef
cat /proc/sys/fs/inotify/max_user_watches
lsof | wc -l
displayName: Diagnostics before smoke test run (processes, max_user_watches, number of opened file handles)
continueOnError: true
condition: succeededOrFailed()
- ${{ if eq(parameters.VSCODE_QUALITY, 'oss') }}:
- script: npm run compile
workingDirectory: test/smoke
displayName: Compile smoke tests
- script: npm run gulp compile-extension:markdown-language-features compile-extension:ipynb compile-extension-media compile-extension:vscode-test-resolver
displayName: Build extensions for smoke tests
- script: npm run gulp node
displayName: Download node.js for remote smoke tests
retryCountOnTaskFailure: 3
- script: npm run smoketest-no-compile -- --tracing
timeoutInMinutes: 20
displayName: Run smoke tests (Electron)
- script: npm run smoketest-no-compile -- --web --tracing --headless --electronArgs="--disable-dev-shm-usage"
timeoutInMinutes: 20
displayName: Run smoke tests (Browser, Chromium)
- script: npm run smoketest-no-compile -- --remote --tracing
timeoutInMinutes: 20
displayName: Run smoke tests (Remote)
- ${{ if ne(parameters.VSCODE_QUALITY, 'oss') }}:
- script: npm run smoketest-no-compile -- --tracing --build "$(agent.builddirectory)/VSCode-linux-$(VSCODE_ARCH)"
timeoutInMinutes: 20
displayName: Run smoke tests (Electron)
- script: npm run smoketest-no-compile -- --web --tracing --headless --electronArgs="--disable-dev-shm-usage"
env:
VSCODE_REMOTE_SERVER_PATH: $(agent.builddirectory)/vscode-server-linux-$(VSCODE_ARCH)-web
timeoutInMinutes: 20
displayName: Run smoke tests (Browser, Chromium)
- script: |
set -e
npm run gulp compile-extension:vscode-test-resolver
APP_PATH=$(agent.builddirectory)/VSCode-linux-$(VSCODE_ARCH)
VSCODE_REMOTE_SERVER_PATH="$(agent.builddirectory)/vscode-server-linux-$(VSCODE_ARCH)" \
npm run smoketest-no-compile -- --tracing --remote --build "$APP_PATH"
timeoutInMinutes: 20
displayName: Run smoke tests (Remote)
- script: |
set -e
ps -ef
cat /proc/sys/fs/inotify/max_user_watches
lsof | wc -l
displayName: Diagnostics after smoke test run (processes, max_user_watches, number of opened file handles)
continueOnError: true
condition: succeededOrFailed()
- ${{ if or(eq(parameters.VSCODE_RUN_INTEGRATION_TESTS, true), eq(parameters.VSCODE_RUN_SMOKE_TESTS, true)) }}:
- task: ${{ parameters.PUBLISH_TASK_NAME }}
inputs:
targetPath: .build/crashes
${{ if and(eq(parameters.VSCODE_RUN_INTEGRATION_TESTS, true), eq(parameters.VSCODE_RUN_SMOKE_TESTS, false)) }}:
artifactName: crash-dump-linux-$(VSCODE_ARCH)-integration-$(System.JobAttempt)
${{ elseif and(eq(parameters.VSCODE_RUN_INTEGRATION_TESTS, false), eq(parameters.VSCODE_RUN_SMOKE_TESTS, true)) }}:
artifactName: crash-dump-linux-$(VSCODE_ARCH)-smoke-$(System.JobAttempt)
${{ else }}:
artifactName: crash-dump-linux-$(VSCODE_ARCH)-$(System.JobAttempt)
sbomEnabled: false
displayName: "Publish Crash Reports"
continueOnError: true
condition: failed()
# In order to properly symbolify above crash reports
# (if any), we need the compiled native modules too
- task: ${{ parameters.PUBLISH_TASK_NAME }}
inputs:
targetPath: node_modules
${{ if and(eq(parameters.VSCODE_RUN_INTEGRATION_TESTS, true), eq(parameters.VSCODE_RUN_SMOKE_TESTS, false)) }}:
artifactName: node-modules-linux-$(VSCODE_ARCH)-integration-$(System.JobAttempt)
${{ elseif and(eq(parameters.VSCODE_RUN_INTEGRATION_TESTS, false), eq(parameters.VSCODE_RUN_SMOKE_TESTS, true)) }}:
artifactName: node-modules-linux-$(VSCODE_ARCH)-smoke-$(System.JobAttempt)
${{ else }}:
artifactName: node-modules-linux-$(VSCODE_ARCH)-$(System.JobAttempt)
sbomEnabled: false
displayName: "Publish Node Modules"
continueOnError: true
condition: failed()
- task: ${{ parameters.PUBLISH_TASK_NAME }}
inputs:
targetPath: .build/logs
${{ if and(eq(parameters.VSCODE_RUN_INTEGRATION_TESTS, true), eq(parameters.VSCODE_RUN_SMOKE_TESTS, false)) }}:
artifactName: logs-linux-$(VSCODE_ARCH)-integration-$(System.JobAttempt)
${{ elseif and(eq(parameters.VSCODE_RUN_INTEGRATION_TESTS, false), eq(parameters.VSCODE_RUN_SMOKE_TESTS, true)) }}:
artifactName: logs-linux-$(VSCODE_ARCH)-smoke-$(System.JobAttempt)
${{ else }}:
artifactName: logs-linux-$(VSCODE_ARCH)-$(System.JobAttempt)
sbomEnabled: false
displayName: "Publish Log Files"
continueOnError: true
condition: succeededOrFailed()
- task: PublishTestResults@2
displayName: Publish Tests Results
inputs:
testResultsFiles: "*-results.xml"
searchFolder: "$(Build.ArtifactStagingDirectory)/test-results"
condition: succeededOrFailed()
================================================
FILE: build/azure-pipelines/linux/product-build-linux.yml
================================================
parameters:
- name: VSCODE_QUALITY
type: string
- name: VSCODE_CIBUILD
type: boolean
- name: VSCODE_RUN_UNIT_TESTS
type: boolean
- name: VSCODE_RUN_INTEGRATION_TESTS
type: boolean
- name: VSCODE_RUN_SMOKE_TESTS
type: boolean
- name: VSCODE_ARCH
type: string
steps:
- ${{ if eq(parameters.VSCODE_QUALITY, 'oss') }}:
- checkout: self
fetchDepth: 1
retryCountOnTaskFailure: 3
- task: NodeTool@0
inputs:
versionSource: fromFile
versionFilePath: .nvmrc
nodejsMirror: https://github.com/joaomoreno/node-mirror/releases/download
- ${{ if ne(parameters.VSCODE_QUALITY, 'oss') }}:
- template: ../distro/download-distro.yml@self
- task: AzureKeyVault@2
displayName: "Azure Key Vault: Get Secrets"
inputs:
azureSubscription: vscode
KeyVaultName: vscode-build-secrets
SecretsFilter: "github-distro-mixin-password"
- task: DownloadPipelineArtifact@2
inputs:
artifact: Compilation
path: $(Build.ArtifactStagingDirectory)
displayName: Download compilation output
- script: tar -xzf $(Build.ArtifactStagingDirectory)/compilation.tar.gz
displayName: Extract compilation output
- script: |
set -e
# Start X server
./build/azure-pipelines/linux/apt-retry.sh sudo apt-get update
./build/azure-pipelines/linux/apt-retry.sh sudo apt-get install -y pkg-config \
dbus \
xvfb \
libgtk-3-0 \
libxkbfile-dev \
libkrb5-dev \
libgbm1 \
rpm
sudo cp build/azure-pipelines/linux/xvfb.init /etc/init.d/xvfb
sudo chmod +x /etc/init.d/xvfb
sudo update-rc.d xvfb defaults
sudo service xvfb start
# Start dbus session
sudo mkdir -p /var/run/dbus
DBUS_LAUNCH_RESULT=$(sudo dbus-daemon --config-file=/usr/share/dbus-1/system.conf --print-address)
echo "##vso[task.setvariable variable=DBUS_SESSION_BUS_ADDRESS]$DBUS_LAUNCH_RESULT"
displayName: Setup system services
- script: node build/setup-npm-registry.js $NPM_REGISTRY
condition: and(succeeded(), ne(variables['NPM_REGISTRY'], 'none'))
displayName: Setup NPM Registry
- script: mkdir -p .build && node build/azure-pipelines/common/computeNodeModulesCacheKey.js linux $VSCODE_ARCH > .build/packagelockhash
displayName: Prepare node_modules cache key
- task: Cache@2
inputs:
key: '"node_modules" | .build/packagelockhash'
path: .build/node_modules_cache
cacheHitVar: NODE_MODULES_RESTORED
displayName: Restore node_modules cache
- script: tar -xzf .build/node_modules_cache/cache.tgz
condition: and(succeeded(), eq(variables.NODE_MODULES_RESTORED, 'true'))
displayName: Extract node_modules cache
- script: |
set -e
# Set the private NPM registry to the global npmrc file
# so that authentication works for subfolders like build/, remote/, extensions/ etc
# which does not have their own .npmrc file
npm config set registry "$NPM_REGISTRY"
echo "##vso[task.setvariable variable=NPMRC_PATH]$(npm config get userconfig)"
condition: and(succeeded(), ne(variables.NODE_MODULES_RESTORED, 'true'), ne(variables['NPM_REGISTRY'], 'none'))
displayName: Setup NPM
- task: npmAuthenticate@0
inputs:
workingFile: $(NPMRC_PATH)
condition: and(succeeded(), ne(variables.NODE_MODULES_RESTORED, 'true'), ne(variables['NPM_REGISTRY'], 'none'))
displayName: Setup NPM Authentication
- script: |
set -e
for i in {1..5}; do # try 5 times
npm ci && break
if [ $i -eq 5 ]; then
echo "Npm install failed too many times" >&2
exit 1
fi
echo "Npm install failed $i, trying again..."
done
workingDirectory: build
env:
GITHUB_TOKEN: "$(github-distro-mixin-password)"
displayName: Install build dependencies
condition: and(succeeded(), ne(variables.NODE_MODULES_RESTORED, 'true'))
# Step will be used by both Install dependencies and building rpm package,
# hence avoid adding it behind NODE_MODULES_RESTORED condition.
- script: |
set -e
SYSROOT_ARCH=$VSCODE_ARCH
if [ "$SYSROOT_ARCH" == "x64" ]; then
SYSROOT_ARCH="amd64"
fi
export VSCODE_SYSROOT_DIR=$(Build.SourcesDirectory)/.build/sysroots
SYSROOT_ARCH="$SYSROOT_ARCH" node -e '(async () => { const { getVSCodeSysroot } = require("./build/linux/debian/install-sysroot.js"); await getVSCodeSysroot(process.env["SYSROOT_ARCH"]); })()'
env:
VSCODE_ARCH: $(VSCODE_ARCH)
GITHUB_TOKEN: "$(github-distro-mixin-password)"
displayName: Download vscode sysroots
- ${{ if or(eq(parameters.VSCODE_ARCH, 'arm64'), eq(parameters.VSCODE_ARCH, 'armhf')) }}:
- script: |
set -e
includes=$(cat << 'EOF'
{
"target_defaults": {
"conditions": [
["OS=='linux'", {
'cflags_cc!': [ '-std=gnu++20' ],
'cflags_cc': [ '-std=gnu++2a' ],
}]
]
}
}
EOF
)
if [ ! -d "$HOME/.gyp" ]; then
mkdir -p "$HOME/.gyp"
fi
echo "$includes" > "$HOME/.gyp/include.gypi"
displayName: Override gnu target for arm64 and arm
- script: |
set -e
source ./build/azure-pipelines/linux/setup-env.sh
for i in {1..5}; do # try 5 times
npm ci && break
if [ $i -eq 5 ]; then
echo "Npm install failed too many times" >&2
exit 1
fi
echo "Npm install failed $i, trying again..."
done
env:
npm_config_arch: $(NPM_ARCH)
VSCODE_ARCH: $(VSCODE_ARCH)
ELECTRON_SKIP_BINARY_DOWNLOAD: 1
PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD: 1
GITHUB_TOKEN: "$(github-distro-mixin-password)"
displayName: Install dependencies
condition: and(succeeded(), ne(variables.NODE_MODULES_RESTORED, 'true'))
- ${{ if ne(parameters.VSCODE_QUALITY, 'oss') }}:
- script: node build/azure-pipelines/distro/mixin-npm
condition: and(succeeded(), ne(variables.NODE_MODULES_RESTORED, 'true'))
displayName: Mixin distro node modules
- script: |
set -e
node build/azure-pipelines/common/listNodeModules.js .build/node_modules_list.txt
mkdir -p .build/node_modules_cache
tar -czf .build/node_modules_cache/cache.tgz --files-from .build/node_modules_list.txt
condition: and(succeeded(), ne(variables.NODE_MODULES_RESTORED, 'true'))
displayName: Create node_modules archive
- ${{ if ne(parameters.VSCODE_QUALITY, 'oss') }}:
- script: node build/azure-pipelines/distro/mixin-quality
displayName: Mixin distro quality
- template: ../common/install-builtin-extensions.yml@self
- ${{ if ne(parameters.VSCODE_QUALITY, 'oss') }}:
- script: |
set -e
npm run gulp vscode-linux-$(VSCODE_ARCH)-min-ci
ARCHIVE_PATH=".build/linux/client/code-${{ parameters.VSCODE_QUALITY }}-$(VSCODE_ARCH)-$(date +%s).tar.gz"
mkdir -p $(dirname $ARCHIVE_PATH)
echo "##vso[task.setvariable variable=CLIENT_PATH]$ARCHIVE_PATH"
env:
GITHUB_TOKEN: "$(github-distro-mixin-password)"
displayName: Build client
- ${{ if ne(parameters.VSCODE_CIBUILD, true) }}:
- task: DownloadPipelineArtifact@2
inputs:
artifact: $(ARTIFACT_PREFIX)vscode_cli_linux_$(VSCODE_ARCH)_cli
patterns: "**"
path: $(Build.ArtifactStagingDirectory)/cli
displayName: Download VS Code CLI
- script: |
set -e
tar -xzvf $(Build.ArtifactStagingDirectory)/cli/*.tar.gz -C $(Build.ArtifactStagingDirectory)/cli
CLI_APP_NAME=$(node -p "require(\"$(agent.builddirectory)/VSCode-linux-$(VSCODE_ARCH)/resources/app/product.json\").tunnelApplicationName")
APP_NAME=$(node -p "require(\"$(agent.builddirectory)/VSCode-linux-$(VSCODE_ARCH)/resources/app/product.json\").applicationName")
mv $(Build.ArtifactStagingDirectory)/cli/$APP_NAME $(agent.builddirectory)/VSCode-linux-$(VSCODE_ARCH)/bin/$CLI_APP_NAME
displayName: Mix in CLI
- script: |
set -e
tar -czf $CLIENT_PATH -C .. VSCode-linux-$(VSCODE_ARCH)
env:
GITHUB_TOKEN: "$(github-distro-mixin-password)"
displayName: Archive client
- script: |
set -e
npm run gulp vscode-reh-linux-$(VSCODE_ARCH)-min-ci
mv ../vscode-reh-linux-$(VSCODE_ARCH) ../vscode-server-linux-$(VSCODE_ARCH) # TODO@joaomoreno
ARCHIVE_PATH=".build/linux/server/vscode-server-linux-$(VSCODE_ARCH).tar.gz"
UNARCHIVE_PATH="`pwd`/../vscode-server-linux-$(VSCODE_ARCH)"
mkdir -p $(dirname $ARCHIVE_PATH)
tar --owner=0 --group=0 -czf $ARCHIVE_PATH -C .. vscode-server-linux-$(VSCODE_ARCH)
echo "##vso[task.setvariable variable=SERVER_PATH]$ARCHIVE_PATH"
echo "##vso[task.setvariable variable=SERVER_UNARCHIVE_PATH]$UNARCHIVE_PATH"
env:
GITHUB_TOKEN: "$(github-distro-mixin-password)"
displayName: Build server
- script: |
set -e
npm run gulp vscode-reh-web-linux-$(VSCODE_ARCH)-min-ci
mv ../vscode-reh-web-linux-$(VSCODE_ARCH) ../vscode-server-linux-$(VSCODE_ARCH)-web # TODO@joaomoreno
ARCHIVE_PATH=".build/linux/web/vscode-server-linux-$(VSCODE_ARCH)-web.tar.gz"
mkdir -p $(dirname $ARCHIVE_PATH)
tar --owner=0 --group=0 -czf $ARCHIVE_PATH -C .. vscode-server-linux-$(VSCODE_ARCH)-web
echo "##vso[task.setvariable variable=WEB_PATH]$ARCHIVE_PATH"
env:
GITHUB_TOKEN: "$(github-distro-mixin-password)"
displayName: Build server (web)
- ${{ if or(eq(parameters.VSCODE_ARCH, 'x64'), eq(parameters.VSCODE_ARCH, 'arm64')) }}:
- script: |
set -e
EXPECTED_GLIBC_VERSION="2.28" \
EXPECTED_GLIBCXX_VERSION="3.4.25" \
./build/azure-pipelines/linux/verify-glibc-requirements.sh
env:
SEARCH_PATH: $(SERVER_UNARCHIVE_PATH)
npm_config_arch: $(NPM_ARCH)
VSCODE_ARCH: $(VSCODE_ARCH)
displayName: Check GLIBC and GLIBCXX dependencies in server archive
- ${{ else }}:
- script: |
set -e
EXPECTED_GLIBC_VERSION="2.28" \
EXPECTED_GLIBCXX_VERSION="3.4.26" \
./build/azure-pipelines/linux/verify-glibc-requirements.sh
env:
SEARCH_PATH: $(SERVER_UNARCHIVE_PATH)
npm_config_arch: $(NPM_ARCH)
VSCODE_ARCH: $(VSCODE_ARCH)
displayName: Check GLIBC and GLIBCXX dependencies in server archive
- ${{ else }}:
- script: npm run gulp "transpile-client-esbuild" "transpile-extensions"
env:
GITHUB_TOKEN: "$(github-distro-mixin-password)"
displayName: Transpile client and extensions
- ${{ if or(eq(parameters.VSCODE_RUN_UNIT_TESTS, true), eq(parameters.VSCODE_RUN_INTEGRATION_TESTS, true), eq(parameters.VSCODE_RUN_SMOKE_TESTS, true)) }}:
- template: product-build-linux-test.yml@self
parameters:
VSCODE_QUALITY: ${{ parameters.VSCODE_QUALITY }}
VSCODE_RUN_UNIT_TESTS: ${{ parameters.VSCODE_RUN_UNIT_TESTS }}
VSCODE_RUN_INTEGRATION_TESTS: ${{ parameters.VSCODE_RUN_INTEGRATION_TESTS }}
VSCODE_RUN_SMOKE_TESTS: ${{ parameters.VSCODE_RUN_SMOKE_TESTS }}
${{ if ne(parameters.VSCODE_QUALITY, 'oss') }}:
PUBLISH_TASK_NAME: 1ES.PublishPipelineArtifact@1
- ${{ if and(ne(parameters.VSCODE_CIBUILD, true), ne(parameters.VSCODE_QUALITY, 'oss')) }}:
- script: |
set -e
npm run gulp "vscode-linux-$(VSCODE_ARCH)-prepare-deb"
env:
GITHUB_TOKEN: "$(github-distro-mixin-password)"
displayName: Prepare deb package
- script: |
set -e
npm run gulp "vscode-linux-$(VSCODE_ARCH)-build-deb"
file_output=$(file $(ls .build/linux/deb/*/deb/*.deb))
if [[ "$file_output" != *"data compression xz"* ]]; then
echo "Error: unknown compression. $file_output"
exit 1
fi
echo "##vso[task.setvariable variable=DEB_PATH]$(ls .build/linux/deb/*/deb/*.deb)"
displayName: Build deb package
- script: |
set -e
TRIPLE=""
if [ "$VSCODE_ARCH" == "x64" ]; then
TRIPLE="x86_64-linux-gnu"
elif [ "$VSCODE_ARCH" == "arm64" ]; then
TRIPLE="aarch64-linux-gnu"
elif [ "$VSCODE_ARCH" == "armhf" ]; then
TRIPLE="arm-rpi-linux-gnueabihf"
fi
export VSCODE_SYSROOT_DIR=$(Build.SourcesDirectory)/.build/sysroots
export STRIP="$VSCODE_SYSROOT_DIR/$TRIPLE/$TRIPLE/bin/strip"
npm run gulp "vscode-linux-$(VSCODE_ARCH)-prepare-rpm"
env:
VSCODE_ARCH: $(VSCODE_ARCH)
displayName: Prepare rpm package
- script: |
set -e
npm run gulp "vscode-linux-$(VSCODE_ARCH)-build-rpm"
echo "##vso[task.setvariable variable=RPM_PATH]$(ls .build/linux/rpm/*/*.rpm)"
displayName: Build rpm package
- script: |
set -e
npm run gulp "vscode-linux-$(VSCODE_ARCH)-prepare-snap"
ARCHIVE_PATH=".build/linux/snap-tarball/snap-$(VSCODE_ARCH).tar.gz"
mkdir -p $(dirname $ARCHIVE_PATH)
tar -czf $ARCHIVE_PATH -C .build/linux snap
echo "##vso[task.setvariable variable=SNAP_PATH]$ARCHIVE_PATH"
displayName: Prepare snap package
- task: UseDotNet@2
inputs:
version: 6.x
- task: EsrpCodeSigning@5
inputs:
UseMSIAuthentication: true
ConnectedServiceName: vscode-esrp
AppRegistrationClientId: $(ESRP_CLIENT_ID)
AppRegistrationTenantId: $(ESRP_TENANT_ID)
AuthAKVName: vscode-esrp
AuthSignCertName: esrp-sign
FolderPath: .
Pattern: noop
displayName: 'Install ESRP Tooling'
- script: node build/azure-pipelines/common/sign $(Agent.RootDirectory)/_tasks/EsrpCodeSigning_*/*/net6.0/esrpcli.dll sign-pgp .build/linux/deb '*.deb'
env:
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
displayName: Codesign deb
- script: node build/azure-pipelines/common/sign $(Agent.RootDirectory)/_tasks/EsrpCodeSigning_*/*/net6.0/esrpcli.dll sign-pgp .build/linux/rpm '*.rpm'
env:
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
displayName: Codesign rpm
- script: echo "##vso[task.setvariable variable=ARTIFACT_PREFIX]attempt$(System.JobAttempt)_"
condition: and(succeededOrFailed(), notIn(variables['Agent.JobStatus'], 'Succeeded', 'SucceededWithIssues'))
displayName: Generate artifact prefix
- task: 1ES.PublishPipelineArtifact@1
inputs:
targetPath: $(CLIENT_PATH)
artifactName: $(ARTIFACT_PREFIX)vscode_client_linux_$(VSCODE_ARCH)_archive-unsigned
sbomBuildDropPath: $(Agent.BuildDirectory)/VSCode-linux-$(VSCODE_ARCH)
sbomPackageName: "VS Code Linux $(VSCODE_ARCH) (unsigned)"
sbomPackageVersion: $(Build.SourceVersion)
condition: and(succeededOrFailed(), ne(variables['CLIENT_PATH'], ''))
displayName: Publish client archive
- task: 1ES.PublishPipelineArtifact@1
inputs:
targetPath: $(SERVER_PATH)
artifactName: $(ARTIFACT_PREFIX)vscode_server_linux_$(VSCODE_ARCH)_archive-unsigned
sbomBuildDropPath: $(Agent.BuildDirectory)/vscode-server-linux-$(VSCODE_ARCH)
sbomPackageName: "VS Code Linux $(VSCODE_ARCH) Server"
sbomPackageVersion: $(Build.SourceVersion)
condition: and(succeededOrFailed(), ne(variables['SERVER_PATH'], ''))
displayName: Publish server archive
- task: 1ES.PublishPipelineArtifact@1
inputs:
targetPath: $(WEB_PATH)
artifactName: $(ARTIFACT_PREFIX)vscode_web_linux_$(VSCODE_ARCH)_archive-unsigned
sbomBuildDropPath: $(Agent.BuildDirectory)/vscode-server-linux-$(VSCODE_ARCH)-web
sbomPackageName: "VS Code Linux $(VSCODE_ARCH) Web"
sbomPackageVersion: $(Build.SourceVersion)
condition: and(succeededOrFailed(), ne(variables['WEB_PATH'], ''))
displayName: Publish web server archive
- task: 1ES.PublishPipelineArtifact@1
inputs:
targetPath: $(DEB_PATH)
artifactName: $(ARTIFACT_PREFIX)vscode_client_linux_$(VSCODE_ARCH)_deb-package
sbomBuildDropPath: .build/linux/deb
sbomPackageName: "VS Code Linux $(VSCODE_ARCH) DEB"
sbomPackageVersion: $(Build.SourceVersion)
condition: and(succeededOrFailed(), ne(variables['DEB_PATH'], ''))
displayName: Publish deb package
- task: 1ES.PublishPipelineArtifact@1
inputs:
targetPath: $(RPM_PATH)
artifactName: $(ARTIFACT_PREFIX)vscode_client_linux_$(VSCODE_ARCH)_rpm-package
sbomBuildDropPath: .build/linux/rpm
sbomPackageName: "VS Code Linux $(VSCODE_ARCH) RPM"
sbomPackageVersion: $(Build.SourceVersion)
condition: and(succeededOrFailed(), ne(variables['RPM_PATH'], ''))
displayName: Publish rpm package
- task: 1ES.PublishPipelineArtifact@1
inputs:
targetPath: $(SNAP_PATH)
artifactName: $(ARTIFACT_PREFIX)snap-$(VSCODE_ARCH)
sbomEnabled: false
condition: and(succeededOrFailed(), ne(variables['SNAP_PATH'], ''))
displayName: Publish snap pre-package
================================================
FILE: build/azure-pipelines/linux/setup-env.sh
================================================
#!/usr/bin/env bash
set -e
SYSROOT_ARCH=$VSCODE_ARCH
if [ "$SYSROOT_ARCH" == "x64" ]; then
SYSROOT_ARCH="amd64"
fi
export VSCODE_SYSROOT_DIR=$PWD/.build/sysroots
if [ -d "$VSCODE_SYSROOT_DIR" ]; then
echo "Using cached sysroot"
else
echo "Downloading sysroot"
SYSROOT_ARCH="$SYSROOT_ARCH" node -e '(async () => { const { getVSCodeSysroot } = require("./build/linux/debian/install-sysroot.js"); await getVSCodeSysroot(process.env["SYSROOT_ARCH"]); })()'
fi
if [ "$npm_config_arch" == "x64" ]; then
# Download clang based on chromium revision used by vscode
curl -s https://raw.githubusercontent.com/chromium/chromium/132.0.6834.210/tools/clang/scripts/update.py | python - --output-dir=$PWD/.build/CR_Clang --host-os=linux
# Download libcxx headers and objects from upstream electron releases
DEBUG=libcxx-fetcher \
VSCODE_LIBCXX_OBJECTS_DIR=$PWD/.build/libcxx-objects \
VSCODE_LIBCXX_HEADERS_DIR=$PWD/.build/libcxx_headers \
VSCODE_LIBCXXABI_HEADERS_DIR=$PWD/.build/libcxxabi_headers \
VSCODE_ARCH="$npm_config_arch" \
node build/linux/libcxx-fetcher.js
# Set compiler toolchain
# Flags for the client build are based on
# https://source.chromium.org/chromium/chromium/src/+/refs/tags/132.0.6834.210:build/config/arm.gni
# https://source.chromium.org/chromium/chromium/src/+/refs/tags/132.0.6834.210:build/config/compiler/BUILD.gn
# https://source.chromium.org/chromium/chromium/src/+/refs/tags/132.0.6834.210:build/config/c++/BUILD.gn
export CC="$PWD/.build/CR_Clang/bin/clang --gcc-toolchain=$VSCODE_SYSROOT_DIR/x86_64-linux-gnu"
export CXX="$PWD/.build/CR_Clang/bin/clang++ --gcc-toolchain=$VSCODE_SYSROOT_DIR/x86_64-linux-gnu"
export CXXFLAGS="-nostdinc++ -D__NO_INLINE__ -DSPDLOG_USE_STD_FORMAT -I$PWD/.build/libcxx_headers -isystem$PWD/.build/libcxx_headers/include -isystem$PWD/.build/libcxxabi_headers/include -fPIC -flto=thin -fsplit-lto-unit -D_LIBCPP_ABI_NAMESPACE=Cr -D_LIBCPP_HARDENING_MODE=_LIBCPP_HARDENING_MODE_EXTENSIVE --sysroot=$VSCODE_SYSROOT_DIR/x86_64-linux-gnu/x86_64-linux-gnu/sysroot"
export LDFLAGS="-stdlib=libc++ --sysroot=$VSCODE_SYSROOT_DIR/x86_64-linux-gnu/x86_64-linux-gnu/sysroot -fuse-ld=lld -flto=thin -L$PWD/.build/libcxx-objects -lc++abi -L$VSCODE_SYSROOT_DIR/x86_64-linux-gnu/x86_64-linux-gnu/sysroot/usr/lib/x86_64-linux-gnu -L$VSCODE_SYSROOT_DIR/x86_64-linux-gnu/x86_64-linux-gnu/sysroot/lib/x86_64-linux-gnu -Wl,--lto-O0"
# Set compiler toolchain for remote server
export VSCODE_REMOTE_CC=$VSCODE_SYSROOT_DIR/x86_64-linux-gnu/bin/x86_64-linux-gnu-gcc
export VSCODE_REMOTE_CXX=$VSCODE_SYSROOT_DIR/x86_64-linux-gnu/bin/x86_64-linux-gnu-g++
export VSCODE_REMOTE_CXXFLAGS="--sysroot=$VSCODE_SYSROOT_DIR/x86_64-linux-gnu/x86_64-linux-gnu/sysroot"
export VSCODE_REMOTE_LDFLAGS="--sysroot=$VSCODE_SYSROOT_DIR/x86_64-linux-gnu/x86_64-linux-gnu/sysroot -L$VSCODE_SYSROOT_DIR/x86_64-linux-gnu/x86_64-linux-gnu/sysroot/usr/lib/x86_64-linux-gnu -L$VSCODE_SYSROOT_DIR/x86_64-linux-gnu/x86_64-linux-gnu/sysroot/lib/x86_64-linux-gnu"
elif [ "$npm_config_arch" == "arm64" ]; then
# Set compiler toolchain for client native modules
export CC=$VSCODE_SYSROOT_DIR/aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc
export CXX=$VSCODE_SYSROOT_DIR/aarch64-linux-gnu/bin/aarch64-linux-gnu-g++
export CXXFLAGS="--sysroot=$VSCODE_SYSROOT_DIR/aarch64-linux-gnu/aarch64-linux-gnu/sysroot"
export LDFLAGS="--sysroot=$VSCODE_SYSROOT_DIR/aarch64-linux-gnu/aarch64-linux-gnu/sysroot -L$VSCODE_SYSROOT_DIR/aarch64-linux-gnu/aarch64-linux-gnu/sysroot/usr/lib/aarch64-linux-gnu -L$VSCODE_SYSROOT_DIR/aarch64-linux-gnu/aarch64-linux-gnu/sysroot/lib/aarch64-linux-gnu"
# Set compiler toolchain for remote server
export VSCODE_REMOTE_CC=$VSCODE_SYSROOT_DIR/aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc
export VSCODE_REMOTE_CXX=$VSCODE_SYSROOT_DIR/aarch64-linux-gnu/bin/aarch64-linux-gnu-g++
export VSCODE_REMOTE_CXXFLAGS="--sysroot=$VSCODE_SYSROOT_DIR/aarch64-linux-gnu/aarch64-linux-gnu/sysroot"
export VSCODE_REMOTE_LDFLAGS="--sysroot=$VSCODE_SYSROOT_DIR/aarch64-linux-gnu/aarch64-linux-gnu/sysroot -L$VSCODE_SYSROOT_DIR/aarch64-linux-gnu/aarch64-linux-gnu/sysroot/usr/lib/aarch64-linux-gnu -L$VSCODE_SYSROOT_DIR/aarch64-linux-gnu/aarch64-linux-gnu/sysroot/lib/aarch64-linux-gnu"
elif [ "$npm_config_arch" == "arm" ]; then
# Set compiler toolchain for client native modules
export CC=$VSCODE_SYSROOT_DIR/arm-rpi-linux-gnueabihf/bin/arm-rpi-linux-gnueabihf-gcc
export CXX=$VSCODE_SYSROOT_DIR/arm-rpi-linux-gnueabihf/bin/arm-rpi-linux-gnueabihf-g++
export CXXFLAGS="--sysroot=$VSCODE_SYSROOT_DIR/arm-rpi-linux-gnueabihf/arm-rpi-linux-gnueabihf/sysroot"
export LDFLAGS="--sysroot=$VSCODE_SYSROOT_DIR/arm-rpi-linux-gnueabihf/arm-rpi-linux-gnueabihf/sysroot -L$VSCODE_SYSROOT_DIR/arm-rpi-linux-gnueabihf/arm-rpi-linux-gnueabihf/sysroot/usr/lib/arm-linux-gnueabihf -L$VSCODE_SYSROOT_DIR/arm-rpi-linux-gnueabihf/arm-rpi-linux-gnueabihf/sysroot/lib/arm-linux-gnueabihf"
# Set compiler toolchain for remote server
export VSCODE_REMOTE_CC=$VSCODE_SYSROOT_DIR/arm-rpi-linux-gnueabihf/bin/arm-rpi-linux-gnueabihf-gcc
export VSCODE_REMOTE_CXX=$VSCODE_SYSROOT_DIR/arm-rpi-linux-gnueabihf/bin/arm-rpi-linux-gnueabihf-g++
export VSCODE_REMOTE_CXXFLAGS="--sysroot=$VSCODE_SYSROOT_DIR/arm-rpi-linux-gnueabihf/arm-rpi-linux-gnueabihf/sysroot"
export VSCODE_REMOTE_LDFLAGS="--sysroot=$VSCODE_SYSROOT_DIR/arm-rpi-linux-gnueabihf/arm-rpi-linux-gnueabihf/sysroot -L$VSCODE_SYSROOT_DIR/arm-rpi-linux-gnueabihf/arm-rpi-linux-gnueabihf/sysroot/usr/lib/arm-linux-gnueabihf -L$VSCODE_SYSROOT_DIR/arm-rpi-linux-gnueabihf/arm-rpi-linux-gnueabihf/sysroot/lib/arm-linux-gnueabihf"
fi
================================================
FILE: build/azure-pipelines/linux/snap-build-linux.yml
================================================
steps:
- task: NodeTool@0
inputs:
versionSource: fromFile
versionFilePath: .nvmrc
nodejsMirror: https://github.com/joaomoreno/node-mirror/releases/download
- task: DownloadPipelineArtifact@2
displayName: "Download Pipeline Artifact"
inputs:
artifact: snap-$(VSCODE_ARCH)
path: .build/linux/snap-tarball
- script: |
set -e
# Get snapcraft version
snapcraft --version
# Make sure we get latest packages
sudo apt-get update
sudo apt-get upgrade -y
sudo apt-get install -y curl apt-transport-https ca-certificates
# Define variables
SNAP_ROOT="$(pwd)/.build/linux/snap/$(VSCODE_ARCH)"
# Unpack snap tarball artifact, in order to preserve file perms
(cd .build/linux && tar -xzf snap-tarball/snap-$(VSCODE_ARCH).tar.gz)
# Create snap package
BUILD_VERSION="$(date +%s)"
SNAP_FILENAME="code-$VSCODE_QUALITY-$(VSCODE_ARCH)-$BUILD_VERSION.snap"
SNAP_PATH="$SNAP_ROOT/$SNAP_FILENAME"
case $(VSCODE_ARCH) in
x64) SNAPCRAFT_TARGET_ARGS="" ;;
*) SNAPCRAFT_TARGET_ARGS="--target-arch $(VSCODE_ARCH)" ;;
esac
(cd $SNAP_ROOT/code-* && sudo --preserve-env snapcraft snap $SNAPCRAFT_TARGET_ARGS --output "$SNAP_PATH")
displayName: Prepare for publish
- script: |
set -e
SNAP_ROOT="$(pwd)/.build/linux/snap/$(VSCODE_ARCH)"
SNAP_EXTRACTED_PATH=$(find $SNAP_ROOT -maxdepth 1 -type d -name 'code-*')
SNAP_PATH=$(find $SNAP_ROOT -maxdepth 1 -type f -name '*.snap')
# SBOM tool doesn't like recursive symlinks
sudo find $SNAP_EXTRACTED_PATH -type l -delete
echo "##vso[task.setvariable variable=SNAP_EXTRACTED_PATH]$SNAP_EXTRACTED_PATH"
echo "##vso[task.setvariable variable=SNAP_PATH]$SNAP_PATH"
target:
container: host
displayName: Find host snap path & prepare for SBOM
- task: 1ES.PublishPipelineArtifact@1
inputs:
targetPath: $(SNAP_PATH)
artifactName: vscode_client_linux_$(VSCODE_ARCH)_snap
sbomBuildDropPath: $(SNAP_EXTRACTED_PATH)
sbomPackageName: "VS Code Linux $(VSCODE_ARCH) SNAP"
sbomPackageVersion: $(Build.SourceVersion)
displayName: Publish snap package
================================================
FILE: build/azure-pipelines/linux/verify-glibc-requirements.sh
================================================
#!/usr/bin/env bash
set -e
TRIPLE="x86_64-linux-gnu"
if [ "$VSCODE_ARCH" == "arm64" ]; then
TRIPLE="aarch64-linux-gnu"
elif [ "$VSCODE_ARCH" == "armhf" ]; then
TRIPLE="arm-rpi-linux-gnueabihf"
fi
# Get all files with .node extension from server folder
files=$(find $SEARCH_PATH -name "*.node" -not -path "*prebuilds*" -not -path "*extensions/node_modules/@parcel/watcher*" -o -type f -executable -name "node")
echo "Verifying requirements for files: $files"
for file in $files; do
glibc_version="$EXPECTED_GLIBC_VERSION"
glibcxx_version="$EXPECTED_GLIBCXX_VERSION"
while IFS= read -r line; do
if [[ $line == *"GLIBC_"* ]]; then
version=$(echo "$line" | awk '{if ($5 ~ /^[0-9a-fA-F]+$/) print $6; else print $5}' | tr -d '()')
version=${version#*_}
if [[ $(printf "%s\n%s" "$version" "$glibc_version" | sort -V | tail -n1) == "$version" ]]; then
glibc_version=$version
fi
elif [[ $line == *"GLIBCXX_"* ]]; then
version=$(echo "$line" | awk '{if ($5 ~ /^[0-9a-fA-F]+$/) print $6; else print $5}' | tr -d '()')
version=${version#*_}
if [[ $(printf "%s\n%s" "$version" "$glibcxx_version" | sort -V | tail -n1) == "$version" ]]; then
glibcxx_version=$version
fi
fi
done < <("$PWD/.build/sysroots/$TRIPLE/$TRIPLE/bin/objdump" -T "$file")
if [[ "$glibc_version" != "$EXPECTED_GLIBC_VERSION" ]]; then
echo "Error: File $file has dependency on GLIBC > $EXPECTED_GLIBC_VERSION, found $glibc_version"
exit 1
fi
if [[ "$glibcxx_version" != "$EXPECTED_GLIBCXX_VERSION" ]]; then
echo "Error: File $file has dependency on GLIBCXX > $EXPECTED_GLIBCXX_VERSION, found $glibcxx_version"
exit 1
fi
done
================================================
FILE: build/azure-pipelines/linux/xvfb.init
================================================
#!/bin/bash
#
# /etc/rc.d/init.d/xvfbd
#
# chkconfig: 345 95 28
# description: Starts/Stops X Virtual Framebuffer server
# processname: Xvfb
#
### BEGIN INIT INFO
# Provides: xvfb
# Required-Start: $remote_fs $syslog
# Required-Stop: $remote_fs $syslog
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: Start xvfb at boot time
# Description: Enable xvfb provided by daemon.
### END INIT INFO
[ "${NETWORKING}" = "no" ] && exit 0
PROG="/usr/bin/Xvfb"
PROG_OPTIONS=":10 -ac -screen 0 1024x768x24"
PROG_OUTPUT="/tmp/Xvfb.out"
case "$1" in
start)
echo "Starting : X Virtual Frame Buffer "
$PROG $PROG_OPTIONS>>$PROG_OUTPUT 2>&1 &
disown -ar
;;
stop)
echo "Shutting down : X Virtual Frame Buffer"
killproc $PROG
RETVAL=$?
[ $RETVAL -eq 0 ] && /bin/rm -f /var/lock/subsys/Xvfb
/var/run/Xvfb.pid
echo
;;
restart|reload)
$0 stop
$0 start
RETVAL=$?
;;
status)
status Xvfb
RETVAL=$?
;;
*)
echo $"Usage: $0 (start|stop|restart|reload|status)"
exit 1
esac
exit $RETVAL
================================================
FILE: build/azure-pipelines/oss/product-build-pr-cache-linux.yml
================================================
steps:
- checkout: self
fetchDepth: 1
retryCountOnTaskFailure: 3
- task: NodeTool@0
inputs:
versionSource: fromFile
versionFilePath: .nvmrc
nodejsMirror: https://github.com/joaomoreno/node-mirror/releases/download
- script: node build/setup-npm-registry.js $NPM_REGISTRY
condition: and(succeeded(), ne(variables['NPM_REGISTRY'], 'none'))
displayName: Setup NPM Registry
- script: mkdir -p .build && node build/azure-pipelines/common/computeNodeModulesCacheKey.js linux $VSCODE_ARCH > .build/packagelockhash
displayName: Prepare node_modules cache key
- task: Cache@2
inputs:
key: '"node_modules" | .build/packagelockhash'
path: .build/node_modules_cache
cacheHitVar: NODE_MODULES_RESTORED
displayName: Restore node_modules cache
- script: tar -xzf .build/node_modules_cache/cache.tgz
condition: and(succeeded(), eq(variables.NODE_MODULES_RESTORED, 'true'))
displayName: Extract node_modules cache
- script: |
set -e
# Set the private NPM registry to the global npmrc file
# so that authentication works for subfolders like build/, remote/, extensions/ etc
# which does not have their own .npmrc file
npm config set registry "$NPM_REGISTRY"
echo "##vso[task.setvariable variable=NPMRC_PATH]$(npm config get userconfig)"
condition: and(succeeded(), ne(variables.NODE_MODULES_RESTORED, 'true'), ne(variables['NPM_REGISTRY'], 'none'))
displayName: Setup NPM
- task: npmAuthenticate@0
inputs:
workingFile: $(NPMRC_PATH)
condition: and(succeeded(), ne(variables.NODE_MODULES_RESTORED, 'true'), ne(variables['NPM_REGISTRY'], 'none'))
displayName: Setup NPM Authentication
- script: |
set -e
./build/azure-pipelines/linux/apt-retry.sh sudo apt-get update
./build/azure-pipelines/linux/apt-retry.sh sudo apt-get install -y libkrb5-dev
displayName: Setup system services
condition: and(succeeded(), ne(variables.NODE_MODULES_RESTORED, 'true'))
- script: |
set -e
for i in {1..5}; do # try 5 times
npm ci && break
if [ $i -eq 5 ]; then
echo "Npm install failed too many times" >&2
exit 1
fi
echo "Npm install failed $i, trying again..."
done
env:
ELECTRON_SKIP_BINARY_DOWNLOAD: 1
PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD: 1
GITHUB_TOKEN: "$(github-distro-mixin-password)"
displayName: Install dependencies
condition: and(succeeded(), ne(variables.NODE_MODULES_RESTORED, 'true'))
- script: |
set -e
node build/azure-pipelines/common/listNodeModules.js .build/node_modules_list.txt
mkdir -p .build/node_modules_cache
tar -czf .build/node_modules_cache/cache.tgz --files-from .build/node_modules_list.txt
condition: and(succeeded(), ne(variables.NODE_MODULES_RESTORED, 'true'))
displayName: Create node_modules archive
================================================
FILE: build/azure-pipelines/oss/product-build-pr-cache-win32.yml
================================================
steps:
- checkout: self
fetchDepth: 1
retryCountOnTaskFailure: 3
- task: NodeTool@0
inputs:
versionSource: fromFile
versionFilePath: .nvmrc
nodejsMirror: https://github.com/joaomoreno/node-mirror/releases/download
- powershell: node build/setup-npm-registry.js $env:NPM_REGISTRY
condition: and(succeeded(), ne(variables['NPM_REGISTRY'], 'none'))
displayName: Setup NPM Registry
- pwsh: |
mkdir .build -ea 0
node build/azure-pipelines/common/computeNodeModulesCacheKey.js win32 $(VSCODE_ARCH) > .build/packagelockhash
displayName: Prepare node_modules cache key
- task: Cache@2
inputs:
key: '"node_modules" | .build/packagelockhash'
path: .build/node_modules_cache
cacheHitVar: NODE_MODULES_RESTORED
displayName: Restore node_modules cache
- powershell: 7z.exe x .build/node_modules_cache/cache.7z -aoa
condition: and(succeeded(), eq(variables.NODE_MODULES_RESTORED, 'true'))
displayName: Extract node_modules cache
- powershell: |
. build/azure-pipelines/win32/exec.ps1
$ErrorActionPreference = "Stop"
# Set the private NPM registry to the global npmrc file
# so that authentication works for subfolders like build/, remote/, extensions/ etc
# which does not have their own .npmrc file
exec { npm config set registry "$env:NPM_REGISTRY" }
$NpmrcPath = (npm config get userconfig)
echo "##vso[task.setvariable variable=NPMRC_PATH]$NpmrcPath"
condition: and(succeeded(), ne(variables.NODE_MODULES_RESTORED, 'true'), ne(variables['NPM_REGISTRY'], 'none'))
displayName: Setup NPM
- task: npmAuthenticate@0
inputs:
workingFile: $(NPMRC_PATH)
condition: and(succeeded(), ne(variables.NODE_MODULES_RESTORED, 'true'), ne(variables['NPM_REGISTRY'], 'none'))
displayName: Setup NPM Authentication
- powershell: |
. build/azure-pipelines/win32/exec.ps1
$ErrorActionPreference = "Stop"
exec { npm ci }
env:
npm_config_arch: $(VSCODE_ARCH)
npm_config_foreground_scripts: "true"
ELECTRON_SKIP_BINARY_DOWNLOAD: 1
PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD: 1
GITHUB_TOKEN: "$(github-distro-mixin-password)"
retryCountOnTaskFailure: 5
displayName: Install dependencies
condition: and(succeeded(), ne(variables.NODE_MODULES_RESTORED, 'true'))
- powershell: |
. build/azure-pipelines/win32/exec.ps1
$ErrorActionPreference = "Stop"
exec { node build/azure-pipelines/common/listNodeModules.js .build/node_modules_list.txt }
exec { mkdir -Force .build/node_modules_cache }
exec { 7z.exe a .build/node_modules_cache/cache.7z -mx3 `@.build/node_modules_list.txt }
condition: and(succeeded(), ne(variables.NODE_MODULES_RESTORED, 'true'))
displayName: Create node_modules archive
================================================
FILE: build/azure-pipelines/product-build-pr.yml
================================================
trigger:
- main
- release/*
pr:
branches:
include: ["main", "release/*"]
variables:
- name: Codeql.SkipTaskAutoInjection
value: true
- name: skipComponentGovernanceDetection
value: true
- name: NPM_REGISTRY
value: "none"
- name: CARGO_REGISTRY
value: "none"
- name: VSCODE_CIBUILD
value: ${{ in(variables['Build.Reason'], 'IndividualCI', 'BatchedCI') }}
- name: VSCODE_QUALITY
value: oss
- name: VSCODE_STEP_ON_IT
value: false
jobs:
- ${{ if ne(variables['VSCODE_CIBUILD'], true) }}:
- job: Compile
displayName: Compile & Hygiene
pool: 1es-oss-ubuntu-22.04-x64
timeoutInMinutes: 30
variables:
VSCODE_ARCH: x64
steps:
- template: product-compile.yml@self
parameters:
VSCODE_QUALITY: ${{ variables.VSCODE_QUALITY }}
- job: Linuxx64UnitTest
displayName: Linux (Unit Tests)
pool: 1es-oss-ubuntu-22.04-x64
timeoutInMinutes: 30
variables:
VSCODE_ARCH: x64
NPM_ARCH: x64
DISPLAY: ":10"
steps:
- template: linux/product-build-linux.yml@self
parameters:
VSCODE_ARCH: x64
VSCODE_QUALITY: ${{ variables.VSCODE_QUALITY }}
VSCODE_CIBUILD: ${{ variables.VSCODE_CIBUILD }}
VSCODE_RUN_UNIT_TESTS: true
VSCODE_RUN_INTEGRATION_TESTS: false
VSCODE_RUN_SMOKE_TESTS: false
- job: Linuxx64IntegrationTest
displayName: Linux (Integration Tests)
pool: 1es-oss-ubuntu-22.04-x64
timeoutInMinutes: 30
variables:
VSCODE_ARCH: x64
NPM_ARCH: x64
DISPLAY: ":10"
steps:
- template: linux/product-build-linux.yml@self
parameters:
VSCODE_ARCH: x64
VSCODE_QUALITY: ${{ variables.VSCODE_QUALITY }}
VSCODE_CIBUILD: ${{ variables.VSCODE_CIBUILD }}
VSCODE_RUN_UNIT_TESTS: false
VSCODE_RUN_INTEGRATION_TESTS: true
VSCODE_RUN_SMOKE_TESTS: false
- job: Linuxx64SmokeTest
displayName: Linux (Smoke Tests)
pool: 1es-oss-ubuntu-22.04-x64
timeoutInMinutes: 30
variables:
VSCODE_ARCH: x64
NPM_ARCH: x64
DISPLAY: ":10"
steps:
- template: linux/product-build-linux.yml@self
parameters:
VSCODE_ARCH: x64
VSCODE_QUALITY: ${{ variables.VSCODE_QUALITY }}
VSCODE_CIBUILD: ${{ variables.VSCODE_CIBUILD }}
VSCODE_RUN_UNIT_TESTS: false
VSCODE_RUN_INTEGRATION_TESTS: false
VSCODE_RUN_SMOKE_TESTS: true
- job: LinuxCLI
displayName: Linux (CLI)
pool: 1es-oss-ubuntu-22.04-x64
timeoutInMinutes: 30
steps:
- template: cli/test.yml@self
- job: Windowsx64UnitTests
displayName: Windows (Unit Tests)
pool: 1es-oss-windows-2022-x64
timeoutInMinutes: 30
variables:
VSCODE_ARCH: x64
NPM_ARCH: x64
steps:
- template: win32/product-build-win32.yml@self
parameters:
VSCODE_QUALITY: ${{ variables.VSCODE_QUALITY }}
VSCODE_ARCH: x64
VSCODE_CIBUILD: ${{ variables.VSCODE_CIBUILD }}
VSCODE_RUN_UNIT_TESTS: true
VSCODE_RUN_INTEGRATION_TESTS: false
VSCODE_RUN_SMOKE_TESTS: false
- job: Windowsx64IntegrationTests
displayName: Windows (Integration Tests)
pool: 1es-oss-windows-2022-x64
timeoutInMinutes: 60
variables:
VSCODE_ARCH: x64
NPM_ARCH: x64
steps:
- template: win32/product-build-win32.yml@self
parameters:
VSCODE_QUALITY: ${{ variables.VSCODE_QUALITY }}
VSCODE_ARCH: x64
VSCODE_CIBUILD: ${{ variables.VSCODE_CIBUILD }}
VSCODE_RUN_UNIT_TESTS: false
VSCODE_RUN_INTEGRATION_TESTS: true
VSCODE_RUN_SMOKE_TESTS: false
# - job: Windowsx64SmokeTests
# displayName: Windows (Smoke Tests)
# pool: 1es-oss-windows-2022-x64
# timeoutInMinutes: 30
# variables:
# VSCODE_ARCH: x64
# NPM_ARCH: x64
# steps:
# - template: win32/product-build-win32.yml@self
# parameters:
# VSCODE_QUALITY: ${{ variables.VSCODE_QUALITY }}
# VSCODE_ARCH: x64
# VSCODE_RUN_UNIT_TESTS: false
# VSCODE_RUN_INTEGRATION_TESTS: false
# VSCODE_RUN_SMOKE_TESTS: true
- ${{ if eq(variables['VSCODE_CIBUILD'], true) }}:
- job: Linuxx64MaintainNodeModulesCache
displayName: Linux (Maintain node_modules cache)
pool: 1es-oss-ubuntu-22.04-x64
timeoutInMinutes: 30
variables:
VSCODE_ARCH: x64
steps:
- template: oss/product-build-pr-cache-linux.yml@self
- job: Windowsx64MaintainNodeModulesCache
displayName: Windows (Maintain node_modules cache)
pool: 1es-oss-windows-2022-x64
timeoutInMinutes: 30
variables:
VSCODE_ARCH: x64
steps:
- template: oss/product-build-pr-cache-win32.yml@self
# - job: macOSUnitTest
# displayName: macOS (Unit Tests)
# pool:
# vmImage: macOS-11
# timeoutInMinutes: 60
# variables:
# BUILDSECMON_OPT_IN: true
# VSCODE_ARCH: x64
# steps:
# - template: darwin/product-build-darwin.yml@self
# parameters:
# VSCODE_QUALITY: ${{ variables.VSCODE_QUALITY }}
# VSCODE_RUN_UNIT_TESTS: true
# VSCODE_RUN_INTEGRATION_TESTS: false
# VSCODE_RUN_SMOKE_TESTS: false
# - job: macOSIntegrationTest
# displayName: macOS (Integration Tests)
# pool:
# vmImage: macOS-11
# timeoutInMinutes: 60
# variables:
# BUILDSECMON_OPT_IN: true
# VSCODE_ARCH: x64
# steps:
# - template: darwin/product-build-darwin.yml@self
# parameters:
# VSCODE_QUALITY: ${{ variables.VSCODE_QUALITY }}
# VSCODE_RUN_UNIT_TESTS: false
# VSCODE_RUN_INTEGRATION_TESTS: true
# VSCODE_RUN_SMOKE_TESTS: false
# - job: macOSSmokeTest
# displayName: macOS (Smoke Tests)
# pool:
# vmImage: macOS-11
# timeoutInMinutes: 60
# variables:
# BUILDSECMON_OPT_IN: true
# VSCODE_ARCH: x64
# steps:
# - template: darwin/product-build-darwin.yml@self
# parameters:
# VSCODE_QUALITY: ${{ variables.VSCODE_QUALITY }}
# VSCODE_RUN_UNIT_TESTS: false
# VSCODE_RUN_INTEGRATION_TESTS: false
# VSCODE_RUN_SMOKE_TESTS: true
================================================
FILE: build/azure-pipelines/product-build.yml
================================================
pr: none
schedules:
- cron: "0 5 * * Mon-Fri"
displayName: Mon-Fri at 7:00
branches:
include:
- main
trigger:
batch: true
branches:
include: ["main", "release/*"]
parameters:
- name: VSCODE_QUALITY
displayName: Quality
type: string
default: insider
values:
- exploration
- insider
- stable
- name: NPM_REGISTRY
displayName: "Custom NPM Registry"
type: string
default: 'https://pkgs.dev.azure.com/monacotools/Monaco/_packaging/vscode/npm/registry/'
- name: CARGO_REGISTRY
displayName: "Custom Cargo Registry"
type: string
default: 'sparse+https://pkgs.dev.azure.com/monacotools/Monaco/_packaging/vscode/Cargo/index/'
- name: VSCODE_BUILD_WIN32
displayName: "🎯 Windows x64"
type: boolean
default: true
- name: VSCODE_BUILD_WIN32_ARM64
displayName: "🎯 Windows arm64"
type: boolean
default: true
- name: VSCODE_BUILD_LINUX
displayName: "🎯 Linux x64"
type: boolean
default: true
- name: VSCODE_BUILD_LINUX_ARM64
displayName: "🎯 Linux arm64"
type: boolean
default: true
- name: VSCODE_BUILD_LINUX_ARMHF
displayName: "🎯 Linux armhf"
type: boolean
default: true
- name: VSCODE_BUILD_ALPINE
displayName: "🎯 Alpine x64"
type: boolean
default: true
- name: VSCODE_BUILD_ALPINE_ARM64
displayName: "🎯 Alpine arm64"
type: boolean
default: true
- name: VSCODE_BUILD_MACOS
displayName: "🎯 macOS x64"
type: boolean
default: true
- name: VSCODE_BUILD_MACOS_ARM64
displayName: "🎯 macOS arm64"
type: boolean
default: true
- name: VSCODE_BUILD_MACOS_UNIVERSAL
displayName: "🎯 macOS universal"
type: boolean
default: true
- name: VSCODE_BUILD_WEB
displayName: "🎯 Web"
type: boolean
default: true
- name: VSCODE_PUBLISH
displayName: "Publish to builds.code.visualstudio.com"
type: boolean
default: true
- name: VSCODE_RELEASE
displayName: "Release build if successful"
type: boolean
default: false
- name: VSCODE_COMPILE_ONLY
displayName: "Run Compile stage exclusively"
type: boolean
default: false
- name: VSCODE_STEP_ON_IT
displayName: "Skip tests"
type: boolean
default: false
variables:
- name: VSCODE_PRIVATE_BUILD
value: ${{ ne(variables['Build.Repository.Uri'], 'https://github.com/microsoft/vscode.git') }}
- name: NPM_REGISTRY
${{ if in(variables['Build.Reason'], 'IndividualCI', 'BatchedCI') }}: # disable terrapin when in VSCODE_CIBUILD
value: none
${{ else }}:
value: ${{ parameters.NPM_REGISTRY }}
- name: CARGO_REGISTRY
value: ${{ parameters.CARGO_REGISTRY }}
- name: VSCODE_QUALITY
value: ${{ parameters.VSCODE_QUALITY }}
- name: VSCODE_BUILD_STAGE_WINDOWS
value: ${{ or(eq(parameters.VSCODE_BUILD_WIN32, true), eq(parameters.VSCODE_BUILD_WIN32_ARM64, true)) }}
- name: VSCODE_BUILD_STAGE_LINUX
value: ${{ or(eq(parameters.VSCODE_BUILD_LINUX, true), eq(parameters.VSCODE_BUILD_LINUX_ARMHF, true), eq(parameters.VSCODE_BUILD_LINUX_ARM64, true)) }}
- name: VSCODE_BUILD_STAGE_ALPINE
value: ${{ or(eq(parameters.VSCODE_BUILD_ALPINE, true), eq(parameters.VSCODE_BUILD_ALPINE_ARM64, true)) }}
- name: VSCODE_BUILD_STAGE_MACOS
value: ${{ or(eq(parameters.VSCODE_BUILD_MACOS, true), eq(parameters.VSCODE_BUILD_MACOS_ARM64, true)) }}
- name: VSCODE_BUILD_STAGE_WEB
value: ${{ eq(parameters.VSCODE_BUILD_WEB, true) }}
- name: VSCODE_CIBUILD
value: ${{ in(variables['Build.Reason'], 'IndividualCI', 'BatchedCI') }}
- name: VSCODE_PUBLISH
value: ${{ and(eq(parameters.VSCODE_PUBLISH, true), eq(variables.VSCODE_CIBUILD, false), eq(parameters.VSCODE_COMPILE_ONLY, false)) }}
- name: VSCODE_SCHEDULEDBUILD
value: ${{ eq(variables['Build.Reason'], 'Schedule') }}
- name: VSCODE_STEP_ON_IT
value: ${{ eq(parameters.VSCODE_STEP_ON_IT, true) }}
- name: VSCODE_BUILD_MACOS_UNIVERSAL
value: ${{ and(eq(parameters.VSCODE_BUILD_MACOS, true), eq(parameters.VSCODE_BUILD_MACOS_ARM64, true), eq(parameters.VSCODE_BUILD_MACOS_UNIVERSAL, true)) }}
- name: VSCODE_STAGING_BLOB_STORAGE_ACCOUNT_NAME
value: vscodeesrp
- name: PRSS_CDN_URL
value: https://vscode.download.prss.microsoft.com/dbazure/download
- name: VSCODE_ESRP_SERVICE_CONNECTION_ID
value: fe07e6ce-6ffb-4df9-8d27-d129523a3f3e
- name: VSCODE_ESRP_TENANT_ID
value: 975f013f-7f24-47e8-a7d3-abc4752bf346
- name: VSCODE_ESRP_CLIENT_ID
value: 4ac7ed59-b5e9-4f66-9c30-8d1afa72d32d
- name: ESRP_TENANT_ID
value: 975f013f-7f24-47e8-a7d3-abc4752bf346
- name: ESRP_CLIENT_ID
value: c24324f7-e65f-4c45-8702-ed2d4c35df99
- name: AZURE_DOCUMENTDB_ENDPOINT
value: https://vscode.documents.azure.com/
- name: VSCODE_MIXIN_REPO
value: microsoft/vscode-distro
- name: skipComponentGovernanceDetection
value: true
- name: ComponentDetection.Timeout
value: 600
- name: Codeql.SkipTaskAutoInjection
value: true
- name: ARTIFACT_PREFIX
value: ''
name: "$(Date:yyyyMMdd).$(Rev:r) (${{ parameters.VSCODE_QUALITY }})"
resources:
pipelines:
- pipeline: vscode-7pm-kick-off
source: 'VS Code 7PM Kick-Off'
trigger: true
repositories:
- repository: 1ESPipelines
type: git
name: 1ESPipelineTemplates/1ESPipelineTemplates
ref: refs/tags/release
extends:
template: v1/1ES.Official.PipelineTemplate.yml@1esPipelines
parameters:
sdl:
tsa:
enabled: true
configFile: $(Build.SourcesDirectory)/build/azure-pipelines/config/tsaoptions.json
binskim:
analyzeTargetGlob: '+:file|$(Agent.BuildDirectory)/VSCode-*/**/*.exe;+:file|$(Agent.BuildDirectory)/VSCode-*/**/*.node;+:file|$(Agent.BuildDirectory)/VSCode-*/**/*.dll;-:file|$(Build.SourcesDirectory)/.build/**/system-setup/VSCodeSetup*.exe;-:file|$(Build.SourcesDirectory)/.build/**/user-setup/VSCodeUserSetup*.exe'
codeql:
runSourceLanguagesInSourceAnalysis: true
compiled:
enabled: false
justificationForDisabling: "CodeQL breaks ESRP CodeSign on macOS (ICM #520035761, githubcustomers/microsoft-codeql-support#198)"
credscan:
suppressionsFile: $(Build.SourcesDirectory)/build/azure-pipelines/config/CredScanSuppressions.json
eslint:
enabled: true
enableExclusions: true
exclusionsFilePath: $(Build.SourcesDirectory)/.eslint-ignore
sourceAnalysisPool: 1es-windows-2022-x64
createAdoIssuesForJustificationsForDisablement: false
containers:
snapcraft:
image: vscodehub.azurecr.io/vscode-linux-build-agent:snapcraft-x64
ubuntu-2004-arm64:
image: onebranch.azurecr.io/linux/ubuntu-2004-arm64:latest
stages:
- stage: Compile
jobs:
- job: Compile
timeoutInMinutes: 90
pool:
name: AcesShared
os: macOS
variables:
VSCODE_ARCH: arm64
steps:
- template: build/azure-pipelines/product-compile.yml@self
parameters:
VSCODE_QUALITY: ${{ variables.VSCODE_QUALITY }}
- ${{ if or(eq(parameters.VSCODE_BUILD_LINUX, true),eq(parameters.VSCODE_BUILD_LINUX_ARMHF, true),eq(parameters.VSCODE_BUILD_LINUX_ARM64, true),eq(parameters.VSCODE_BUILD_ALPINE, true),eq(parameters.VSCODE_BUILD_ALPINE_ARM64, true),eq(parameters.VSCODE_BUILD_MACOS, true),eq(parameters.VSCODE_BUILD_MACOS_ARM64, true),eq(parameters.VSCODE_BUILD_WIN32, true),eq(parameters.VSCODE_BUILD_WIN32_ARM64, true)) }}:
- stage: CompileCLI
dependsOn: []
jobs:
- ${{ if eq(parameters.VSCODE_BUILD_LINUX, true) }}:
- job: CLILinuxX64
pool:
name: 1es-ubuntu-22.04-x64
os: linux
steps:
- template: build/azure-pipelines/linux/cli-build-linux.yml@self
parameters:
VSCODE_CHECK_ONLY: ${{ variables.VSCODE_CIBUILD }}
VSCODE_QUALITY: ${{ variables.VSCODE_QUALITY }}
VSCODE_BUILD_LINUX: ${{ parameters.VSCODE_BUILD_LINUX }}
- ${{ if and(eq(variables['VSCODE_CIBUILD'], false), or(eq(parameters.VSCODE_BUILD_LINUX_ARMHF, true), eq(parameters.VSCODE_BUILD_LINUX_ARM64, true))) }}:
- job: CLILinuxGnuARM
pool:
name: 1es-ubuntu-22.04-x64
os: linux
steps:
- template: build/azure-pipelines/linux/cli-build-linux.yml@self
parameters:
VSCODE_QUALITY: ${{ variables.VSCODE_QUALITY }}
VSCODE_BUILD_LINUX_ARMHF: ${{ parameters.VSCODE_BUILD_LINUX_ARMHF }}
VSCODE_BUILD_LINUX_ARM64: ${{ parameters.VSCODE_BUILD_LINUX_ARM64 }}
- ${{ if and(eq(variables['VSCODE_CIBUILD'], false), eq(parameters.VSCODE_BUILD_ALPINE, true)) }}:
- job: CLIAlpineX64
pool:
name: 1es-ubuntu-22.04-x64
os: linux
steps:
- template: build/azure-pipelines/alpine/cli-build-alpine.yml@self
parameters:
VSCODE_QUALITY: ${{ variables.VSCODE_QUALITY }}
VSCODE_BUILD_ALPINE: ${{ parameters.VSCODE_BUILD_ALPINE }}
- ${{ if and(eq(variables['VSCODE_CIBUILD'], false), eq(parameters.VSCODE_BUILD_ALPINE_ARM64, true)) }}:
- job: CLIAlpineARM64
pool:
name: 1es-mariner-2.0-arm64
os: linux
hostArchitecture: arm64
container: ubuntu-2004-arm64
templateContext:
authenticatedContainerRegistries:
- registry: onebranch.azurecr.io
tenant: AME
identity: 1ESPipelineIdentity
steps:
- template: build/azure-pipelines/alpine/cli-build-alpine.yml@self
parameters:
VSCODE_QUALITY: ${{ variables.VSCODE_QUALITY }}
VSCODE_BUILD_ALPINE_ARM64: ${{ parameters.VSCODE_BUILD_ALPINE_ARM64 }}
- ${{ if eq(parameters.VSCODE_BUILD_MACOS, true) }}:
- job: CLIMacOSX64
pool:
name: Azure Pipelines
image: macOS-13
os: macOS
variables:
# todo@connor4312 to diagnose build flakes
- name: MSRUSTUP_LOG
value: debug
steps:
- template: build/azure-pipelines/darwin/cli-build-darwin.yml@self
parameters:
VSCODE_CHECK_ONLY: ${{ variables.VSCODE_CIBUILD }}
VSCODE_QUALITY: ${{ variables.VSCODE_QUALITY }}
VSCODE_BUILD_MACOS: ${{ parameters.VSCODE_BUILD_MACOS }}
- ${{ if and(eq(variables['VSCODE_CIBUILD'], false), eq(parameters.VSCODE_BUILD_MACOS_ARM64, true)) }}:
- job: CLIMacOSARM64
pool:
name: Azure Pipelines
image: macOS-13
os: macOS
variables:
# todo@connor4312 to diagnose build flakes
- name: MSRUSTUP_LOG
value: debug
steps:
- template: build/azure-pipelines/darwin/cli-build-darwin.yml@self
parameters:
VSCODE_QUALITY: ${{ variables.VSCODE_QUALITY }}
VSCODE_BUILD_MACOS_ARM64: ${{ parameters.VSCODE_BUILD_MACOS_ARM64 }}
- ${{ if eq(parameters.VSCODE_BUILD_WIN32, true) }}:
- job: CLIWindowsX64
pool:
name: 1es-windows-2019-x64
os: windows
steps:
- template: build/azure-pipelines/win32/cli-build-win32.yml@self
parameters:
VSCODE_CHECK_ONLY: ${{ variables.VSCODE_CIBUILD }}
VSCODE_QUALITY: ${{ variables.VSCODE_QUALITY }}
VSCODE_BUILD_WIN32: ${{ parameters.VSCODE_BUILD_WIN32 }}
- ${{ if and(eq(variables['VSCODE_CIBUILD'], false), eq(parameters.VSCODE_BUILD_WIN32_ARM64, true)) }}:
- job: CLIWindowsARM64
pool:
name: 1es-windows-2019-x64
os: windows
steps:
- template: build/azure-pipelines/win32/cli-build-win32.yml@self
parameters:
VSCODE_QUALITY: ${{ variables.VSCODE_QUALITY }}
VSCODE_BUILD_WIN32_ARM64: ${{ parameters.VSCODE_BUILD_WIN32_ARM64 }}
- ${{ if and(eq(variables['VSCODE_CIBUILD'], false), eq(parameters.VSCODE_COMPILE_ONLY, false)) }}:
- stage: APIScan
dependsOn: []
pool:
name: 1es-windows-2019-x64
os: windows
jobs:
- job: WindowsAPIScan
steps:
- template: build/azure-pipelines/win32/sdl-scan-win32.yml@self
parameters:
VSCODE_ARCH: x64
VSCODE_QUALITY: ${{ variables.VSCODE_QUALITY }}
- ${{ if and(eq(parameters.VSCODE_COMPILE_ONLY, false), eq(variables['VSCODE_BUILD_STAGE_WINDOWS'], true)) }}:
- stage: Windows
dependsOn:
- Compile
- ${{ if or(eq(parameters.VSCODE_BUILD_LINUX, true),eq(parameters.VSCODE_BUILD_LINUX_ARMHF, true),eq(parameters.VSCODE_BUILD_LINUX_ARM64, true),eq(parameters.VSCODE_BUILD_ALPINE, true),eq(parameters.VSCODE_BUILD_ALPINE_ARM64, true),eq(parameters.VSCODE_BUILD_MACOS, true),eq(parameters.VSCODE_BUILD_MACOS_ARM64, true),eq(parameters.VSCODE_BUILD_WIN32, true),eq(parameters.VSCODE_BUILD_WIN32_ARM64, true)) }}:
- CompileCLI
pool:
name: 1es-windows-2019-x64
os: windows
jobs:
- ${{ if eq(variables['VSCODE_CIBUILD'], true) }}:
- job: WindowsUnitTests
displayName: Unit Tests
timeoutInMinutes: 60
variables:
VSCODE_ARCH: x64
steps:
- template: build/azure-pipelines/win32/product-build-win32.yml@self
parameters:
VSCODE_QUALITY: ${{ variables.VSCODE_QUALITY }}
VSCODE_ARCH: x64
VSCODE_CIBUILD: ${{ variables.VSCODE_CIBUILD }}
VSCODE_RUN_UNIT_TESTS: true
VSCODE_RUN_INTEGRATION_TESTS: false
VSCODE_RUN_SMOKE_TESTS: false
- job: WindowsIntegrationTests
displayName: Integration Tests
timeoutInMinutes: 60
variables:
VSCODE_ARCH: x64
steps:
- template: build/azure-pipelines/win32/product-build-win32.yml@self
parameters:
VSCODE_QUALITY: ${{ variables.VSCODE_QUALITY }}
VSCODE_ARCH: x64
VSCODE_CIBUILD: ${{ variables.VSCODE_CIBUILD }}
VSCODE_RUN_UNIT_TESTS: false
VSCODE_RUN_INTEGRATION_TESTS: true
VSCODE_RUN_SMOKE_TESTS: false
- job: WindowsSmokeTests
displayName: Smoke Tests
timeoutInMinutes: 60
variables:
VSCODE_ARCH: x64
steps:
- template: build/azure-pipelines/win32/product-build-win32.yml@self
parameters:
VSCODE_QUALITY: ${{ variables.VSCODE_QUALITY }}
VSCODE_ARCH: x64
VSCODE_CIBUILD: ${{ variables.VSCODE_CIBUILD }}
VSCODE_RUN_UNIT_TESTS: false
VSCODE_RUN_INTEGRATION_TESTS: false
VSCODE_RUN_SMOKE_TESTS: true
- ${{ if and(eq(variables['VSCODE_CIBUILD'], false), eq(parameters.VSCODE_BUILD_WIN32, true)) }}:
- job: Windows
timeoutInMinutes: 120
variables:
VSCODE_ARCH: x64
templateContext:
sdl:
suppression:
suppressionFile: $(Build.SourcesDirectory)\.config\guardian\.gdnsuppress
steps:
- template: build/azure-pipelines/win32/product-build-win32.yml@self
parameters:
VSCODE_QUALITY: ${{ variables.VSCODE_QUALITY }}
VSCODE_ARCH: x64
VSCODE_CIBUILD: ${{ variables.VSCODE_CIBUILD }}
VSCODE_RUN_UNIT_TESTS: ${{ eq(parameters.VSCODE_STEP_ON_IT, false) }}
VSCODE_RUN_INTEGRATION_TESTS: ${{ eq(parameters.VSCODE_STEP_ON_IT, false) }}
VSCODE_RUN_SMOKE_TESTS: ${{ eq(parameters.VSCODE_STEP_ON_IT, false) }}
- job: WindowsCLISign
timeoutInMinutes: 90
steps:
- template: build/azure-pipelines/win32/product-build-win32-cli-sign.yml@self
parameters:
VSCODE_QUALITY: ${{ variables.VSCODE_QUALITY }}
VSCODE_BUILD_WIN32: ${{ parameters.VSCODE_BUILD_WIN32 }}
VSCODE_BUILD_WIN32_ARM64: ${{ parameters.VSCODE_BUILD_WIN32_ARM64 }}
- ${{ if and(eq(variables['VSCODE_CIBUILD'], false), eq(parameters.VSCODE_BUILD_WIN32_ARM64, true)) }}:
- job: WindowsARM64
timeoutInMinutes: 90
variables:
VSCODE_ARCH: arm64
templateContext:
sdl:
suppression:
suppressionFile: $(Build.SourcesDirectory)\.config\guardian\.gdnsuppress
steps:
- template: build/azure-pipelines/win32/product-build-win32.yml@self
parameters:
VSCODE_QUALITY: ${{ variables.VSCODE_QUALITY }}
VSCODE_ARCH: arm64
VSCODE_CIBUILD: ${{ variables.VSCODE_CIBUILD }}
VSCODE_RUN_UNIT_TESTS: false
VSCODE_RUN_INTEGRATION_TESTS: false
VSCODE_RUN_SMOKE_TESTS: false
- ${{ if and(eq(parameters.VSCODE_COMPILE_ONLY, false), eq(variables['VSCODE_BUILD_STAGE_LINUX'], true)) }}:
- stage: Linux
dependsOn:
- Compile
- ${{ if or(eq(parameters.VSCODE_BUILD_LINUX, true),eq(parameters.VSCODE_BUILD_LINUX_ARMHF, true),eq(parameters.VSCODE_BUILD_LINUX_ARM64, true),eq(parameters.VSCODE_BUILD_ALPINE, true),eq(parameters.VSCODE_BUILD_ALPINE_ARM64, true),eq(parameters.VSCODE_BUILD_MACOS, true),eq(parameters.VSCODE_BUILD_MACOS_ARM64, true),eq(parameters.VSCODE_BUILD_WIN32, true),eq(parameters.VSCODE_BUILD_WIN32_ARM64, true)) }}:
- CompileCLI
pool:
name: 1es-ubuntu-22.04-x64
os: linux
jobs:
- ${{ if eq(variables['VSCODE_CIBUILD'], true) }}:
- job: Linuxx64UnitTest
displayName: Unit Tests
variables:
VSCODE_ARCH: x64
NPM_ARCH: x64
DISPLAY: ":10"
steps:
- template: build/azure-pipelines/linux/product-build-linux.yml@self
parameters:
VSCODE_ARCH: x64
VSCODE_QUALITY: ${{ variables.VSCODE_QUALITY }}
VSCODE_CIBUILD: ${{ variables.VSCODE_CIBUILD }}
VSCODE_RUN_UNIT_TESTS: true
VSCODE_RUN_INTEGRATION_TESTS: false
VSCODE_RUN_SMOKE_TESTS: false
- job: Linuxx64IntegrationTest
displayName: Integration Tests
variables:
VSCODE_ARCH: x64
NPM_ARCH: x64
DISPLAY: ":10"
steps:
- template: build/azure-pipelines/linux/product-build-linux.yml@self
parameters:
VSCODE_ARCH: x64
VSCODE_QUALITY: ${{ variables.VSCODE_QUALITY }}
VSCODE_CIBUILD: ${{ variables.VSCODE_CIBUILD }}
VSCODE_RUN_UNIT_TESTS: false
VSCODE_RUN_INTEGRATION_TESTS: true
VSCODE_RUN_SMOKE_TESTS: false
- job: Linuxx64SmokeTest
displayName: Smoke Tests
variables:
VSCODE_ARCH: x64
NPM_ARCH: x64
DISPLAY: ":10"
steps:
- template: build/azure-pipelines/linux/product-build-linux.yml@self
parameters:
VSCODE_ARCH: x64
VSCODE_QUALITY: ${{ variables.VSCODE_QUALITY }}
VSCODE_CIBUILD: ${{ variables.VSCODE_CIBUILD }}
VSCODE_RUN_UNIT_TESTS: false
VSCODE_RUN_INTEGRATION_TESTS: false
VSCODE_RUN_SMOKE_TESTS: true
- ${{ if and(eq(variables['VSCODE_CIBUILD'], false), eq(parameters.VSCODE_BUILD_LINUX, true)) }}:
- job: Linuxx64
timeoutInMinutes: 90
variables:
VSCODE_ARCH: x64
NPM_ARCH: x64
DISPLAY: ":10"
steps:
- template: build/azure-pipelines/linux/product-build-linux.yml@self
parameters:
VSCODE_ARCH: x64
VSCODE_QUALITY: ${{ variables.VSCODE_QUALITY }}
VSCODE_CIBUILD: ${{ variables.VSCODE_CIBUILD }}
VSCODE_RUN_UNIT_TESTS: ${{ eq(parameters.VSCODE_STEP_ON_IT, false) }}
VSCODE_RUN_INTEGRATION_TESTS: ${{ eq(parameters.VSCODE_STEP_ON_IT, false) }}
VSCODE_RUN_SMOKE_TESTS: ${{ eq(parameters.VSCODE_STEP_ON_IT, false) }}
- ${{ if and(eq(variables['VSCODE_CIBUILD'], false), eq(parameters.VSCODE_BUILD_LINUX, true)) }}:
- job: LinuxSnap
dependsOn:
- Linuxx64
container: snapcraft
variables:
VSCODE_ARCH: x64
templateContext:
authenticatedContainerRegistries:
- registry: onebranch.azurecr.io
tenant: AME
identity: 1ESPipelineIdentity
steps:
- template: build/azure-pipelines/linux/snap-build-linux.yml@self
- ${{ if and(eq(variables['VSCODE_CIBUILD'], false), eq(parameters.VSCODE_BUILD_LINUX_ARMHF, true)) }}:
- job: LinuxArmhf
variables:
VSCODE_ARCH: armhf
NPM_ARCH: arm
steps:
- template: build/azure-pipelines/linux/product-build-linux.yml@self
parameters:
VSCODE_ARCH: armhf
VSCODE_QUALITY: ${{ variables.VSCODE_QUALITY }}
VSCODE_CIBUILD: ${{ variables.VSCODE_CIBUILD }}
VSCODE_RUN_UNIT_TESTS: false
VSCODE_RUN_INTEGRATION_TESTS: false
VSCODE_RUN_SMOKE_TESTS: false
- ${{ if and(eq(variables['VSCODE_CIBUILD'], false), eq(parameters.VSCODE_BUILD_LINUX_ARM64, true)) }}:
- job: LinuxArm64
variables:
VSCODE_ARCH: arm64
NPM_ARCH: arm64
steps:
- template: build/azure-pipelines/linux/product-build-linux.yml@self
parameters:
VSCODE_ARCH: arm64
VSCODE_QUALITY: ${{ variables.VSCODE_QUALITY }}
VSCODE_CIBUILD: ${{ variables.VSCODE_CIBUILD }}
VSCODE_RUN_UNIT_TESTS: false
VSCODE_RUN_INTEGRATION_TESTS: false
VSCODE_RUN_SMOKE_TESTS: false
- ${{ if and(eq(variables['VSCODE_CIBUILD'], false), eq(parameters.VSCODE_COMPILE_ONLY, false), eq(variables['VSCODE_BUILD_STAGE_ALPINE'], true)) }}:
- stage: Alpine
dependsOn:
- Compile
- ${{ if or(eq(parameters.VSCODE_BUILD_LINUX, true),eq(parameters.VSCODE_BUILD_LINUX_ARMHF, true),eq(parameters.VSCODE_BUILD_LINUX_ARM64, true),eq(parameters.VSCODE_BUILD_ALPINE, true),eq(parameters.VSCODE_BUILD_ALPINE_ARM64, true),eq(parameters.VSCODE_BUILD_MACOS, true),eq(parameters.VSCODE_BUILD_MACOS_ARM64, true),eq(parameters.VSCODE_BUILD_WIN32, true),eq(parameters.VSCODE_BUILD_WIN32_ARM64, true)) }}:
- CompileCLI
pool:
name: 1es-ubuntu-22.04-x64
os: linux
jobs:
- ${{ if eq(parameters.VSCODE_BUILD_ALPINE, true) }}:
- job: LinuxAlpine
variables:
VSCODE_ARCH: x64
NPM_ARCH: x64
steps:
- template: build/azure-pipelines/alpine/product-build-alpine.yml@self
- ${{ if eq(parameters.VSCODE_BUILD_ALPINE_ARM64, true) }}:
- job: LinuxAlpineArm64
timeoutInMinutes: 120
variables:
VSCODE_ARCH: arm64
NPM_ARCH: arm64
steps:
- template: build/azure-pipelines/alpine/product-build-alpine.yml@self
- ${{ if and(eq(parameters.VSCODE_COMPILE_ONLY, false), eq(variables['VSCODE_BUILD_STAGE_MACOS'], true)) }}:
- stage: macOS
dependsOn:
- Compile
- ${{ if or(eq(parameters.VSCODE_BUILD_LINUX, true),eq(parameters.VSCODE_BUILD_LINUX_ARMHF, true),eq(parameters.VSCODE_BUILD_LINUX_ARM64, true),eq(parameters.VSCODE_BUILD_ALPINE, true),eq(parameters.VSCODE_BUILD_ALPINE_ARM64, true),eq(parameters.VSCODE_BUILD_MACOS, true),eq(parameters.VSCODE_BUILD_MACOS_ARM64, true),eq(parameters.VSCODE_BUILD_WIN32, true),eq(parameters.VSCODE_BUILD_WIN32_ARM64, true)) }}:
- CompileCLI
pool:
name: Azure Pipelines
image: macOS-13
os: macOS
variables:
BUILDSECMON_OPT_IN: true
jobs:
- ${{ if eq(variables['VSCODE_CIBUILD'], true) }}:
- job: macOSUnitTest
displayName: Unit Tests
timeoutInMinutes: 90
variables:
VSCODE_ARCH: x64
steps:
- template: build/azure-pipelines/darwin/product-build-darwin.yml@self
parameters:
VSCODE_QUALITY: ${{ variables.VSCODE_QUALITY }}
VSCODE_CIBUILD: ${{ variables.VSCODE_CIBUILD }}
VSCODE_RUN_UNIT_TESTS: true
VSCODE_RUN_INTEGRATION_TESTS: false
VSCODE_RUN_SMOKE_TESTS: false
- job: macOSIntegrationTest
displayName: Integration Tests
timeoutInMinutes: 90
variables:
VSCODE_ARCH: x64
steps:
- template: build/azure-pipelines/darwin/product-build-darwin.yml@self
parameters:
VSCODE_QUALITY: ${{ variables.VSCODE_QUALITY }}
VSCODE_CIBUILD: ${{ variables.VSCODE_CIBUILD }}
VSCODE_RUN_UNIT_TESTS: false
VSCODE_RUN_INTEGRATION_TESTS: true
VSCODE_RUN_SMOKE_TESTS: false
- job: macOSSmokeTest
displayName: Smoke Tests
timeoutInMinutes: 90
variables:
VSCODE_ARCH: x64
steps:
- template: build/azure-pipelines/darwin/product-build-darwin.yml@self
parameters:
VSCODE_QUALITY: ${{ variables.VSCODE_QUALITY }}
VSCODE_CIBUILD: ${{ variables.VSCODE_CIBUILD }}
VSCODE_RUN_UNIT_TESTS: false
VSCODE_RUN_INTEGRATION_TESTS: false
VSCODE_RUN_SMOKE_TESTS: true
- ${{ if and(eq(variables['VSCODE_CIBUILD'], false), eq(parameters.VSCODE_BUILD_MACOS, true)) }}:
- job: macOS
timeoutInMinutes: 90
variables:
VSCODE_ARCH: x64
steps:
- template: build/azure-pipelines/darwin/product-build-darwin.yml@self
parameters:
VSCODE_QUALITY: ${{ variables.VSCODE_QUALITY }}
VSCODE_CIBUILD: ${{ variables.VSCODE_CIBUILD }}
VSCODE_RUN_UNIT_TESTS: false
VSCODE_RUN_INTEGRATION_TESTS: false
VSCODE_RUN_SMOKE_TESTS: false
- ${{ if eq(parameters.VSCODE_STEP_ON_IT, false) }}:
- job: macOSTest
timeoutInMinutes: 90
variables:
VSCODE_ARCH: x64
steps:
- template: build/azure-pipelines/darwin/product-build-darwin.yml@self
parameters:
VSCODE_QUALITY: ${{ variables.VSCODE_QUALITY }}
VSCODE_CIBUILD: ${{ variables.VSCODE_CIBUILD }}
VSCODE_RUN_UNIT_TESTS: ${{ eq(parameters.VSCODE_STEP_ON_IT, false) }}
VSCODE_RUN_INTEGRATION_TESTS: ${{ eq(parameters.VSCODE_STEP_ON_IT, false) }}
VSCODE_RUN_SMOKE_TESTS: ${{ eq(parameters.VSCODE_STEP_ON_IT, false) }}
- job: macOSSign
dependsOn:
- macOS
timeoutInMinutes: 90
variables:
VSCODE_ARCH: x64
steps:
- template: build/azure-pipelines/darwin/product-build-darwin-sign.yml@self
- job: macOSCLISign
timeoutInMinutes: 90
steps:
- template: build/azure-pipelines/darwin/product-build-darwin-cli-sign.yml@self
parameters:
VSCODE_QUALITY: ${{ variables.VSCODE_QUALITY }}
VSCODE_BUILD_MACOS: ${{ parameters.VSCODE_BUILD_MACOS }}
VSCODE_BUILD_MACOS_ARM64: ${{ parameters.VSCODE_BUILD_MACOS_ARM64 }}
- ${{ if and(eq(variables['VSCODE_CIBUILD'], false), eq(parameters.VSCODE_BUILD_MACOS_ARM64, true)) }}:
- job: macOSARM64
timeoutInMinutes: 90
variables:
VSCODE_ARCH: arm64
steps:
- template: build/azure-pipelines/darwin/product-build-darwin.yml@self
parameters:
VSCODE_QUALITY: ${{ variables.VSCODE_QUALITY }}
VSCODE_CIBUILD: ${{ variables.VSCODE_CIBUILD }}
VSCODE_RUN_UNIT_TESTS: false
VSCODE_RUN_INTEGRATION_TESTS: false
VSCODE_RUN_SMOKE_TESTS: false
- job: macOSARM64Sign
dependsOn:
- macOSARM64
timeoutInMinutes: 90
variables:
VSCODE_ARCH: arm64
steps:
- template: build/azure-pipelines/darwin/product-build-darwin-sign.yml@self
- ${{ if and(eq(variables['VSCODE_CIBUILD'], false), eq(variables['VSCODE_BUILD_MACOS_UNIVERSAL'], true)) }}:
- job: macOSUniversal
dependsOn:
- macOS
- macOSARM64
timeoutInMinutes: 90
variables:
VSCODE_ARCH: universal
steps:
- template: build/azure-pipelines/darwin/product-build-darwin-universal.yml@self
- job: macOSUniversalSign
dependsOn:
- macOSUniversal
timeoutInMinutes: 90
variables:
VSCODE_ARCH: universal
steps:
- template: build/azure-pipelines/darwin/product-build-darwin-sign.yml@self
- ${{ if and(eq(variables['VSCODE_CIBUILD'], false), eq(parameters.VSCODE_COMPILE_ONLY, false), eq(variables['VSCODE_BUILD_STAGE_WEB'], true)) }}:
- stage: Web
dependsOn:
- Compile
pool:
name: 1es-ubuntu-22.04-x64
os: linux
jobs:
- ${{ if eq(parameters.VSCODE_BUILD_WEB, true) }}:
- job: Web
variables:
VSCODE_ARCH: x64
steps:
- template: build/azure-pipelines/web/product-build-web.yml@self
- ${{ if eq(variables['VSCODE_PUBLISH'], 'true') }}:
- stage: Publish
dependsOn: []
pool:
name: 1es-windows-2019-x64
os: windows
variables:
- name: BUILDS_API_URL
value: $(System.CollectionUri)$(System.TeamProject)/_apis/build/builds/$(Build.BuildId)/
jobs:
- job: PublishBuild
timeoutInMinutes: 180
displayName: Publish Build
steps:
- template: build/azure-pipelines/product-publish.yml@self
- ${{ if and(parameters.VSCODE_RELEASE, eq(variables['VSCODE_PRIVATE_BUILD'], false)) }}:
- stage: ApproveRelease
dependsOn: [] # run in parallel to compile stage
pool:
name: 1es-ubuntu-22.04-x64
os: linux
jobs:
- job: ApproveRelease
displayName: "Approve Release"
variables:
- group: VSCodePeerApproval
- name: skipComponentGovernanceDetection
value: true
- ${{ if or(and(parameters.VSCODE_RELEASE, eq(variables['VSCODE_PRIVATE_BUILD'], false)), and(in(parameters.VSCODE_QUALITY, 'insider', 'exploration'), eq(variables['VSCODE_SCHEDULEDBUILD'], true))) }}:
- stage: Release
dependsOn:
- Publish
- ${{ if and(parameters.VSCODE_RELEASE, eq(variables['VSCODE_PRIVATE_BUILD'], false)) }}:
- ApproveRelease
pool:
name: 1es-ubuntu-22.04-x64
os: linux
jobs:
- job: ReleaseBuild
displayName: Release Build
steps:
- template: build/azure-pipelines/product-release.yml@self
parameters:
VSCODE_RELEASE: ${{ parameters.VSCODE_RELEASE }}
================================================
FILE: build/azure-pipelines/product-compile.yml
================================================
parameters:
- name: VSCODE_QUALITY
type: string
steps:
- task: NodeTool@0
inputs:
versionSource: fromFile
versionFilePath: .nvmrc
nodejsMirror: https://github.com/joaomoreno/node-mirror/releases/download
- ${{ if ne(parameters.VSCODE_QUALITY, 'oss') }}:
- template: ./distro/download-distro.yml@self
- task: AzureKeyVault@2
displayName: "Azure Key Vault: Get Secrets"
inputs:
azureSubscription: vscode
KeyVaultName: vscode-build-secrets
SecretsFilter: "github-distro-mixin-password"
- script: node build/setup-npm-registry.js $NPM_REGISTRY
condition: and(succeeded(), ne(variables['NPM_REGISTRY'], 'none'))
displayName: Setup NPM Registry
- script: mkdir -p .build && node build/azure-pipelines/common/computeNodeModulesCacheKey.js compile $VSCODE_ARCH > .build/packagelockhash
displayName: Prepare node_modules cache key
- task: Cache@2
inputs:
key: '"node_modules" | .build/packagelockhash'
path: .build/node_modules_cache
cacheHitVar: NODE_MODULES_RESTORED
displayName: Restore node_modules cache
- script: tar -xzf .build/node_modules_cache/cache.tgz
condition: and(succeeded(), eq(variables.NODE_MODULES_RESTORED, 'true'))
displayName: Extract node_modules cache
- script: |
set -e
# Set the private NPM registry to the global npmrc file
# so that authentication works for subfolders like build/, remote/, extensions/ etc
# which does not have their own .npmrc file
npm config set registry "$NPM_REGISTRY"
echo "##vso[task.setvariable variable=NPMRC_PATH]$(npm config get userconfig)"
condition: and(succeeded(), ne(variables.NODE_MODULES_RESTORED, 'true'), ne(variables['NPM_REGISTRY'], 'none'))
displayName: Setup NPM
- task: npmAuthenticate@0
inputs:
workingFile: $(NPMRC_PATH)
condition: and(succeeded(), ne(variables.NODE_MODULES_RESTORED, 'true'), ne(variables['NPM_REGISTRY'], 'none'))
displayName: Setup NPM Authentication
- ${{ if eq(parameters.VSCODE_QUALITY, 'oss') }}:
- script: sudo apt update -y && sudo apt install -y build-essential pkg-config libx11-dev libx11-xcb-dev libxkbfile-dev libnotify-bin libkrb5-dev
displayName: Install build tools
condition: and(succeeded(), ne(variables.NODE_MODULES_RESTORED, 'true'))
- script: |
set -e
for i in {1..5}; do # try 5 times
npm ci && break
if [ $i -eq 5 ]; then
echo "Npm install failed too many times" >&2
exit 1
fi
echo "Npm install failed $i, trying again..."
done
env:
ELECTRON_SKIP_BINARY_DOWNLOAD: 1
PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD: 1
GITHUB_TOKEN: "$(github-distro-mixin-password)"
displayName: Install dependencies
condition: and(succeeded(), ne(variables.NODE_MODULES_RESTORED, 'true'))
- ${{ if ne(parameters.VSCODE_QUALITY, 'oss') }}:
- script: node build/azure-pipelines/distro/mixin-npm
condition: and(succeeded(), ne(variables.NODE_MODULES_RESTORED, 'true'))
displayName: Mixin distro node modules
- script: |
set -e
node build/azure-pipelines/common/listNodeModules.js .build/node_modules_list.txt
mkdir -p .build/node_modules_cache
tar -czf .build/node_modules_cache/cache.tgz --files-from .build/node_modules_list.txt
condition: and(succeeded(), ne(variables.NODE_MODULES_RESTORED, 'true'))
displayName: Create node_modules archive
- ${{ if eq(parameters.VSCODE_QUALITY, 'oss') }}:
- script: npm run compile
workingDirectory: build
displayName: Compile /build/ folder
- script: .github/workflows/check-clean-git-state.sh
displayName: Check /build/ folder
- ${{ if ne(parameters.VSCODE_QUALITY, 'oss') }}:
- script: node build/azure-pipelines/distro/mixin-quality
displayName: Mixin distro quality
- template: common/install-builtin-extensions.yml@self
- ${{ if eq(parameters.VSCODE_QUALITY, 'oss') }}:
- script: npm exec -- npm-run-all -lp core-ci-pr extensions-ci-pr hygiene eslint valid-layers-check property-init-order-check vscode-dts-compile-check tsec-compile-check
env:
GITHUB_TOKEN: "$(github-distro-mixin-password)"
displayName: Compile & Hygiene (OSS)
- ${{ else }}:
- script: npm exec -- npm-run-all -lp core-ci extensions-ci hygiene eslint valid-layers-check property-init-order-check vscode-dts-compile-check tsec-compile-check
env:
GITHUB_TOKEN: "$(github-distro-mixin-password)"
displayName: Compile & Hygiene (non-OSS)
- ${{ if ne(parameters.VSCODE_QUALITY, 'oss') }}:
- script: |
set -e
npm run compile
displayName: Compile smoke test suites (non-OSS)
workingDirectory: test/smoke
condition: and(succeeded(), eq(variables['VSCODE_STEP_ON_IT'], 'false'))
- script: |
set -e
npm run compile
displayName: Compile integration test suites (non-OSS)
workingDirectory: test/integration/browser
condition: and(succeeded(), eq(variables['VSCODE_STEP_ON_IT'], 'false'))
- task: AzureCLI@2
displayName: Fetch secrets
inputs:
azureSubscription: vscode
scriptType: pscore
scriptLocation: inlineScript
addSpnToEnvironment: true
inlineScript: |
Write-Host "##vso[task.setvariable variable=AZURE_TENANT_ID]$env:tenantId"
Write-Host "##vso[task.setvariable variable=AZURE_CLIENT_ID]$env:servicePrincipalId"
Write-Host "##vso[task.setvariable variable=AZURE_ID_TOKEN;issecret=true]$env:idToken"
- script: |
set -e
AZURE_STORAGE_ACCOUNT="vscodeweb" \
AZURE_TENANT_ID="$(AZURE_TENANT_ID)" \
AZURE_CLIENT_ID="$(AZURE_CLIENT_ID)" \
AZURE_ID_TOKEN="$(AZURE_ID_TOKEN)" \
node build/azure-pipelines/upload-sourcemaps
displayName: Upload sourcemaps to Azure
- script: ./build/azure-pipelines/common/extract-telemetry.sh
displayName: Generate lists of telemetry events
- script: tar -cz --exclude='.build/node_modules_cache' --exclude='.build/node_modules_list.txt' --exclude='.build/distro' -f $(Build.ArtifactStagingDirectory)/compilation.tar.gz $(ls -d .build out-* test/integration/browser/out test/smoke/out test/automation/out 2>/dev/null)
displayName: Compress compilation artifact
- task: 1ES.PublishPipelineArtifact@1
inputs:
targetPath: $(Build.ArtifactStagingDirectory)/compilation.tar.gz
artifactName: Compilation
sbomEnabled: false
displayName: Publish compilation artifact
- script: npm run download-builtin-extensions-cg
env:
GITHUB_TOKEN: "$(github-distro-mixin-password)"
displayName: Download component details of built-in extensions
- task: ms.vss-governance-buildtask.governance-build-task-component-detection.ComponentGovernanceComponentDetection@0
displayName: "Component Detection"
inputs:
sourceScanPath: $(Build.SourcesDirectory)
alertWarningLevel: Medium
continueOnError: true
================================================
FILE: build/azure-pipelines/product-npm-package-validate.yml
================================================
trigger: none
pr:
branches:
include: ["main"]
paths:
include: ["package.json", "package-lock.json"]
variables:
- name: NPM_REGISTRY
value: "https://pkgs.dev.azure.com/monacotools/Monaco/_packaging/vscode/npm/registry/"
- name: VSCODE_CIBUILD
value: ${{ in(variables['Build.Reason'], 'IndividualCI', 'BatchedCI') }}
- name: VSCODE_QUALITY
value: oss
jobs:
- ${{ if ne(variables['VSCODE_CIBUILD'], true) }}:
- job: ValidateNpmPackage
displayName: Valiate NPM package against Terrapin
pool:
name: 1es-ubuntu-22.04-x64
os: linux
timeoutInMinutes: 40000
continueOnError: true
variables:
VSCODE_ARCH: x64
steps:
- task: NodeTool@0
inputs:
versionSource: fromFile
versionFilePath: .nvmrc
nodejsMirror: https://github.com/joaomoreno/node-mirror/releases/download
- script: node build/setup-npm-registry.js $NPM_REGISTRY
condition: and(succeeded(), ne(variables['NPM_REGISTRY'], 'none'))
displayName: Setup NPM Registry
- script: |
set -e
# Set the private NPM registry to the global npmrc file
# so that authentication works for subfolders like build/, remote/, extensions/ etc
# which does not have their own .npmrc file
echo "NPMRC Path: $(npm config get userconfig)"
echo "NPM Registry: $(npm config get registry)"
npm config set registry "$NPM_REGISTRY"
echo "##vso[task.setvariable variable=NPMRC_PATH]$(npm config get userconfig)"
condition: and(succeeded(), ne(variables['NPM_REGISTRY'], 'none'))
displayName: Setup NPM
- task: npmAuthenticate@0
inputs:
workingFile: $(NPMRC_PATH)
condition: and(succeeded(), ne(variables['NPM_REGISTRY'], 'none'))
displayName: Setup NPM Authentication
- script: sudo apt update -y && sudo apt install -y build-essential pkg-config libx11-dev libx11-xcb-dev libxkbfile-dev libnotify-bin libkrb5-dev
displayName: Install build tools
condition: succeeded()
- script: |
set -e
for attempt in {1..6}; do
if [ $attempt -gt 1 ]; then
echo "Attempt $attempt: Waiting for 1 hour before retrying..."
sleep 3600
fi
echo "Attempt $attempt: Running npm ci"
if npm i --ignore-scripts; then
if node build/npm/postinstall.js; then
echo "npm i succeeded on attempt $attempt"
exit 0
else
echo "node build/npm/postinstall.js failed on attempt $attempt"
fi
else
echo "npm i failed on attempt $attempt"
fi
done
echo "npm i failed after 6 attempts"
exit 1
env:
npm_command: 'install --ignore-scripts'
ELECTRON_SKIP_BINARY_DOWNLOAD: 1
PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD: 1
GITHUB_TOKEN: "$(github-distro-mixin-password)"
displayName: Install dependencies with retries
timeoutInMinutes: 400
- script: .github/workflows/check-clean-git-state.sh
displayName: Check clean git state
================================================
FILE: build/azure-pipelines/product-publish.yml
================================================
steps:
- task: NodeTool@0
inputs:
versionSource: fromFile
versionFilePath: .nvmrc
nodejsMirror: https://github.com/joaomoreno/node-mirror/releases/download
- task: AzureKeyVault@2
displayName: "Azure Key Vault: Get Secrets"
inputs:
azureSubscription: vscode
KeyVaultName: vscode-build-secrets
SecretsFilter: "github-distro-mixin-password"
- task: AzureKeyVault@2
displayName: "Azure Key Vault: Get ESRP Secrets"
inputs:
azureSubscription: vscode-esrp
KeyVaultName: vscode-esrp
SecretsFilter: esrp-auth,esrp-sign
# allow-any-unicode-next-line
- pwsh: Write-Host "##vso[build.addbuildtag]🚀"
displayName: Add build tag
- pwsh: |
npm ci
workingDirectory: build
env:
GITHUB_TOKEN: "$(github-distro-mixin-password)"
displayName: Install build dependencies
- download: current
patterns: "**/artifacts_processed_*.txt"
displayName: Download all artifacts_processed text files
- task: AzureCLI@2
displayName: Fetch secrets
inputs:
azureSubscription: vscode
scriptType: pscore
scriptLocation: inlineScript
addSpnToEnvironment: true
inlineScript: |
Write-Host "##vso[task.setvariable variable=AZURE_TENANT_ID]$env:tenantId"
Write-Host "##vso[task.setvariable variable=AZURE_CLIENT_ID]$env:servicePrincipalId"
Write-Host "##vso[task.setvariable variable=AZURE_ID_TOKEN;issecret=true]$env:idToken"
- pwsh: |
. build/azure-pipelines/win32/exec.ps1
if (Test-Path "$(Pipeline.Workspace)/artifacts_processed_*/artifacts_processed_*.txt") {
Write-Host "Artifacts already processed so a build must have already been created."
return
}
$VERSION = node -p "require('./package.json').version"
Write-Host "Creating build with version: $VERSION"
exec { node build/azure-pipelines/common/createBuild.js $VERSION }
env:
AZURE_TENANT_ID: "$(AZURE_TENANT_ID)"
AZURE_CLIENT_ID: "$(AZURE_CLIENT_ID)"
AZURE_ID_TOKEN: "$(AZURE_ID_TOKEN)"
displayName: Create build if it hasn't been created before
- pwsh: |
$publishAuthTokens = (node build/azure-pipelines/common/getPublishAuthTokens)
Write-Host "##vso[task.setvariable variable=PUBLISH_AUTH_TOKENS;issecret=true]$publishAuthTokens"
env:
AZURE_TENANT_ID: "$(AZURE_TENANT_ID)"
AZURE_CLIENT_ID: "$(AZURE_CLIENT_ID)"
AZURE_ID_TOKEN: "$(AZURE_ID_TOKEN)"
displayName: Get publish auth tokens
- pwsh: node build/azure-pipelines/common/publish.js
env:
GITHUB_TOKEN: "$(github-distro-mixin-password)"
AZURE_TENANT_ID: "$(AZURE_TENANT_ID)"
AZURE_CLIENT_ID: "$(AZURE_CLIENT_ID)"
AZURE_ID_TOKEN: "$(AZURE_ID_TOKEN)"
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
PUBLISH_AUTH_TOKENS: "$(PUBLISH_AUTH_TOKENS)"
RELEASE_TENANT_ID: "$(ESRP_TENANT_ID)"
RELEASE_CLIENT_ID: "$(ESRP_CLIENT_ID)"
RELEASE_AUTH_CERT: "$(esrp-auth)"
RELEASE_REQUEST_SIGNING_CERT: "$(esrp-sign)"
displayName: Process artifacts
retryCountOnTaskFailure: 3
- task: 1ES.PublishPipelineArtifact@1
inputs:
targetPath: $(Pipeline.Workspace)/artifacts_processed_$(System.StageAttempt)/artifacts_processed_$(System.StageAttempt).txt
artifactName: artifacts_processed_$(System.StageAttempt)
sbomEnabled: false
displayName: Publish the artifacts processed for this stage attempt
condition: always()
- pwsh: |
$ErrorActionPreference = 'Stop'
# Determine which stages we need to watch
$stages = @(
if ($env:VSCODE_BUILD_STAGE_WINDOWS -eq 'True') { 'Windows' }
if ($env:VSCODE_BUILD_STAGE_LINUX -eq 'True') { 'Linux' }
if ($env:VSCODE_BUILD_STAGE_ALPINE -eq 'True') { 'Alpine' }
if ($env:VSCODE_BUILD_STAGE_MACOS -eq 'True') { 'macOS' }
if ($env:VSCODE_BUILD_STAGE_WEB -eq 'True') { 'Web' }
)
Write-Host "Stages to check: $stages"
# Get the timeline and see if it says the other stage completed
$timeline = Invoke-RestMethod "$($env:BUILDS_API_URL)timeline?api-version=6.0" -Headers @{
Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"
} -MaximumRetryCount 5 -RetryIntervalSec 1
$failedStages = @()
foreach ($stage in $stages) {
$didStageFail = $timeline.records | Where-Object {
$_.name -eq $stage -and $_.type -eq 'stage' -and $_.result -ne 'succeeded' -and $_.result -ne 'succeededWithIssues'
}
if($didStageFail) {
$failedStages += $stage
Write-Host "'$stage' failed!"
Write-Host $didStageFail
} else {
Write-Host "'$stage' did not fail."
}
}
if ($failedStages.Length) {
throw "Failed stages: $($failedStages -join ', '). This stage will now fail so that it is easier to retry failed jobs."
}
env:
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
displayName: Determine if stage should succeed
================================================
FILE: build/azure-pipelines/product-release.yml
================================================
parameters:
- name: VSCODE_RELEASE
type: boolean
steps:
- task: NodeTool@0
inputs:
versionSource: fromFile
versionFilePath: .nvmrc
nodejsMirror: https://github.com/joaomoreno/node-mirror/releases/download
- task: AzureCLI@2
displayName: Fetch secrets
inputs:
azureSubscription: vscode
scriptType: pscore
scriptLocation: inlineScript
addSpnToEnvironment: true
inlineScript: |
Write-Host "##vso[task.setvariable variable=AZURE_TENANT_ID]$env:tenantId"
Write-Host "##vso[task.setvariable variable=AZURE_CLIENT_ID]$env:servicePrincipalId"
Write-Host "##vso[task.setvariable variable=AZURE_ID_TOKEN;issecret=true]$env:idToken"
- script: npm ci
workingDirectory: build
displayName: Install /build dependencies
- script: |
set -e
AZURE_TENANT_ID="$(AZURE_TENANT_ID)" \
AZURE_CLIENT_ID="$(AZURE_CLIENT_ID)" \
AZURE_ID_TOKEN="$(AZURE_ID_TOKEN)" \
node build/azure-pipelines/common/releaseBuild.js ${{ parameters.VSCODE_RELEASE }}
displayName: Release build
================================================
FILE: build/azure-pipelines/publish-types/check-version.js
================================================
"use strict";
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
Object.defineProperty(exports, "__esModule", { value: true });
const child_process_1 = __importDefault(require("child_process"));
let tag = '';
try {
tag = child_process_1.default
.execSync('git describe --tags `git rev-list --tags --max-count=1`')
.toString()
.trim();
if (!isValidTag(tag)) {
throw Error(`Invalid tag ${tag}`);
}
}
catch (err) {
console.error(err);
console.error('Failed to update types');
process.exit(1);
}
function isValidTag(t) {
if (t.split('.').length !== 3) {
return false;
}
const [major, minor, bug] = t.split('.');
// Only release for tags like 1.34.0
if (bug !== '0') {
return false;
}
if (isNaN(parseInt(major, 10)) || isNaN(parseInt(minor, 10))) {
return false;
}
return true;
}
//# sourceMappingURL=check-version.js.map
================================================
FILE: build/azure-pipelines/publish-types/check-version.ts
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
import cp from 'child_process';
let tag = '';
try {
tag = cp
.execSync('git describe --tags `git rev-list --tags --max-count=1`')
.toString()
.trim();
if (!isValidTag(tag)) {
throw Error(`Invalid tag ${tag}`);
}
} catch (err) {
console.error(err);
console.error('Failed to update types');
process.exit(1);
}
function isValidTag(t: string) {
if (t.split('.').length !== 3) {
return false;
}
const [major, minor, bug] = t.split('.');
// Only release for tags like 1.34.0
if (bug !== '0') {
return false;
}
if (isNaN(parseInt(major, 10)) || isNaN(parseInt(minor, 10))) {
return false;
}
return true;
}
================================================
FILE: build/azure-pipelines/publish-types/publish-types.yml
================================================
# Publish @types/vscode for each release
trigger:
branches:
include: ["refs/tags/*"]
pr: none
pool:
vmImage: ubuntu-latest
steps:
- task: NodeTool@0
inputs:
versionSource: fromFile
versionFilePath: .nvmrc
nodejsMirror: https://github.com/joaomoreno/node-mirror/releases/download
- bash: |
TAG_VERSION=$(git describe --tags `git rev-list --tags --max-count=1`)
CHANNEL="C1C14HJ2F"
if [ "$TAG_VERSION" == "1.999.0" ]; then
MESSAGE=". Someone pushed 1.999.0 tag. Please delete it ASAP from remote and local."
curl -X POST -H "Authorization: Bearer $(SLACK_TOKEN)" \
-H 'Content-type: application/json; charset=utf-8' \
--data '{"channel":"'"$CHANNEL"'", "link_names": true, "text":"'"$MESSAGE"'"}' \
https://slack.com/api/chat.postMessage
exit 1
fi
displayName: Check 1.999.0 tag
- bash: |
# Install build dependencies
(cd build && npm ci)
node build/azure-pipelines/publish-types/check-version.js
displayName: Check version
- bash: |
git config --global user.email "vscode@microsoft.com"
git config --global user.name "VSCode"
git clone https://$(GITHUB_TOKEN)@github.com/DefinitelyTyped/DefinitelyTyped.git --depth=1
node build/azure-pipelines/publish-types/update-types.js
TAG_VERSION=$(git describe --tags `git rev-list --tags --max-count=1`)
cd DefinitelyTyped
git diff --color | cat
git add -A
git status
git checkout -b "vscode-types-$TAG_VERSION"
git commit -m "VS Code $TAG_VERSION Extension API"
git push origin "vscode-types-$TAG_VERSION"
displayName: Push update to DefinitelyTyped
- bash: |
TAG_VERSION=$(git describe --tags `git rev-list --tags --max-count=1`)
CHANNEL="C1C14HJ2F"
MESSAGE="DefinitelyTyped/DefinitelyTyped#vscode-types-$TAG_VERSION created. Endgame champion, please open this link, examine changes and create a PR:"
LINK="https://github.com/DefinitelyTyped/DefinitelyTyped/compare/vscode-types-$TAG_VERSION?quick_pull=1&body=Updating%20VS%20Code%20Extension%20API.%20See%20https%3A%2F%2Fgithub.com%2Fmicrosoft%2Fvscode%2Fissues%2F70175%20for%20details."
MESSAGE2="[@jrieken, @kmaetzel, @egamma]. Please review and merge PR to publish @types/vscode."
curl -X POST -H "Authorization: Bearer $(SLACK_TOKEN)" \
-H 'Content-type: application/json; charset=utf-8' \
--data '{"channel":"'"$CHANNEL"'", "link_names": true, "text":"'"$MESSAGE"'"}' \
https://slack.com/api/chat.postMessage
curl -X POST -H "Authorization: Bearer $(SLACK_TOKEN)" \
-H 'Content-type: application/json; charset=utf-8' \
--data '{"channel":"'"$CHANNEL"'", "link_names": true, "text":"'"$LINK"'"}' \
https://slack.com/api/chat.postMessage
curl -X POST -H "Authorization: Bearer $(SLACK_TOKEN)" \
-H 'Content-type: application/json; charset=utf-8' \
--data '{"channel":"'"$CHANNEL"'", "link_names": true, "text":"'"$MESSAGE2"'"}' \
https://slack.com/api/chat.postMessage
displayName: Send message linking to changes on Slack
================================================
FILE: build/azure-pipelines/publish-types/update-types.js
================================================
"use strict";
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
Object.defineProperty(exports, "__esModule", { value: true });
const fs_1 = __importDefault(require("fs"));
const child_process_1 = __importDefault(require("child_process"));
const path_1 = __importDefault(require("path"));
let tag = '';
try {
tag = child_process_1.default
.execSync('git describe --tags `git rev-list --tags --max-count=1`')
.toString()
.trim();
const dtsUri = `https://raw.githubusercontent.com/microsoft/vscode/${tag}/src/vscode-dts/vscode.d.ts`;
const outPath = path_1.default.resolve(process.cwd(), 'DefinitelyTyped/types/vscode/index.d.ts');
child_process_1.default.execSync(`curl ${dtsUri} --output ${outPath}`);
updateDTSFile(outPath, tag);
console.log(`Done updating vscode.d.ts at ${outPath}`);
}
catch (err) {
console.error(err);
console.error('Failed to update types');
process.exit(1);
}
function updateDTSFile(outPath, tag) {
const oldContent = fs_1.default.readFileSync(outPath, 'utf-8');
const newContent = getNewFileContent(oldContent, tag);
fs_1.default.writeFileSync(outPath, newContent);
}
function repeat(str, times) {
const result = new Array(times);
for (let i = 0; i < times; i++) {
result[i] = str;
}
return result.join('');
}
function convertTabsToSpaces(str) {
return str.replace(/\t/gm, value => repeat(' ', value.length));
}
function getNewFileContent(content, tag) {
const oldheader = [
`/*---------------------------------------------------------------------------------------------`,
` * Copyright (c) Microsoft Corporation. All rights reserved.`,
` * Licensed under the MIT License. See License.txt in the project root for license information.`,
` *--------------------------------------------------------------------------------------------*/`
].join('\n');
return convertTabsToSpaces(getNewFileHeader(tag) + content.slice(oldheader.length));
}
function getNewFileHeader(tag) {
const [major, minor] = tag.split('.');
const shorttag = `${major}.${minor}`;
const header = [
`// Type definitions for Visual Studio Code ${shorttag}`,
`// Project: https://github.com/microsoft/vscode`,
`// Definitions by: Visual Studio Code Team, Microsoft `,
`// Definitions: https://github.com/DefinitelyTyped/DefinitelyTyped`,
``,
`/*---------------------------------------------------------------------------------------------`,
` * Copyright (c) Microsoft Corporation. All rights reserved.`,
` * Licensed under the MIT License.`,
` * See https://github.com/microsoft/vscode/blob/main/LICENSE.txt for license information.`,
` *--------------------------------------------------------------------------------------------*/`,
``,
`/**`,
` * Type Definition for Visual Studio Code ${shorttag} Extension API`,
` * See https://code.visualstudio.com/api for more information`,
` */`
].join('\n');
return header;
}
//# sourceMappingURL=update-types.js.map
================================================
FILE: build/azure-pipelines/publish-types/update-types.ts
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
import fs from 'fs';
import cp from 'child_process';
import path from 'path';
let tag = '';
try {
tag = cp
.execSync('git describe --tags `git rev-list --tags --max-count=1`')
.toString()
.trim();
const dtsUri = `https://raw.githubusercontent.com/microsoft/vscode/${tag}/src/vscode-dts/vscode.d.ts`;
const outPath = path.resolve(process.cwd(), 'DefinitelyTyped/types/vscode/index.d.ts');
cp.execSync(`curl ${dtsUri} --output ${outPath}`);
updateDTSFile(outPath, tag);
console.log(`Done updating vscode.d.ts at ${outPath}`);
} catch (err) {
console.error(err);
console.error('Failed to update types');
process.exit(1);
}
function updateDTSFile(outPath: string, tag: string) {
const oldContent = fs.readFileSync(outPath, 'utf-8');
const newContent = getNewFileContent(oldContent, tag);
fs.writeFileSync(outPath, newContent);
}
function repeat(str: string, times: number): string {
const result = new Array(times);
for (let i = 0; i < times; i++) {
result[i] = str;
}
return result.join('');
}
function convertTabsToSpaces(str: string): string {
return str.replace(/\t/gm, value => repeat(' ', value.length));
}
function getNewFileContent(content: string, tag: string) {
const oldheader = [
`/*---------------------------------------------------------------------------------------------`,
` * Copyright (c) Microsoft Corporation. All rights reserved.`,
` * Licensed under the MIT License. See License.txt in the project root for license information.`,
` *--------------------------------------------------------------------------------------------*/`
].join('\n');
return convertTabsToSpaces(getNewFileHeader(tag) + content.slice(oldheader.length));
}
function getNewFileHeader(tag: string) {
const [major, minor] = tag.split('.');
const shorttag = `${major}.${minor}`;
const header = [
`// Type definitions for Visual Studio Code ${shorttag}`,
`// Project: https://github.com/microsoft/vscode`,
`// Definitions by: Visual Studio Code Team, Microsoft `,
`// Definitions: https://github.com/DefinitelyTyped/DefinitelyTyped`,
``,
`/*---------------------------------------------------------------------------------------------`,
` * Copyright (c) Microsoft Corporation. All rights reserved.`,
` * Licensed under the MIT License.`,
` * See https://github.com/microsoft/vscode/blob/main/LICENSE.txt for license information.`,
` *--------------------------------------------------------------------------------------------*/`,
``,
`/**`,
` * Type Definition for Visual Studio Code ${shorttag} Extension API`,
` * See https://code.visualstudio.com/api for more information`,
` */`
].join('\n');
return header;
}
================================================
FILE: build/azure-pipelines/upload-cdn.js
================================================
"use strict";
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
Object.defineProperty(exports, "__esModule", { value: true });
const event_stream_1 = __importDefault(require("event-stream"));
const vinyl_1 = __importDefault(require("vinyl"));
const vinyl_fs_1 = __importDefault(require("vinyl-fs"));
const gulp_filter_1 = __importDefault(require("gulp-filter"));
const gulp_gzip_1 = __importDefault(require("gulp-gzip"));
const mime_1 = __importDefault(require("mime"));
const identity_1 = require("@azure/identity");
const azure = require('gulp-azure-storage');
const commit = process.env['BUILD_SOURCEVERSION'];
const credential = new identity_1.ClientAssertionCredential(process.env['AZURE_TENANT_ID'], process.env['AZURE_CLIENT_ID'], () => Promise.resolve(process.env['AZURE_ID_TOKEN']));
mime_1.default.define({
'application/typescript': ['ts'],
'application/json': ['code-snippets'],
});
// From default AFD configuration
const MimeTypesToCompress = new Set([
'application/eot',
'application/font',
'application/font-sfnt',
'application/javascript',
'application/json',
'application/opentype',
'application/otf',
'application/pkcs7-mime',
'application/truetype',
'application/ttf',
'application/typescript',
'application/vnd.ms-fontobject',
'application/xhtml+xml',
'application/xml',
'application/xml+rss',
'application/x-font-opentype',
'application/x-font-truetype',
'application/x-font-ttf',
'application/x-httpd-cgi',
'application/x-javascript',
'application/x-mpegurl',
'application/x-opentype',
'application/x-otf',
'application/x-perl',
'application/x-ttf',
'font/eot',
'font/ttf',
'font/otf',
'font/opentype',
'image/svg+xml',
'text/css',
'text/csv',
'text/html',
'text/javascript',
'text/js',
'text/markdown',
'text/plain',
'text/richtext',
'text/tab-separated-values',
'text/xml',
'text/x-script',
'text/x-component',
'text/x-java-source'
]);
function wait(stream) {
return new Promise((c, e) => {
stream.on('end', () => c());
stream.on('error', (err) => e(err));
});
}
async function main() {
const files = [];
const options = (compressed) => ({
account: process.env.AZURE_STORAGE_ACCOUNT,
credential,
container: '$web',
prefix: `${process.env.VSCODE_QUALITY}/${commit}/`,
contentSettings: {
contentEncoding: compressed ? 'gzip' : undefined,
cacheControl: 'max-age=31536000, public'
}
});
const all = vinyl_fs_1.default.src('**', { cwd: '../vscode-web', base: '../vscode-web', dot: true })
.pipe((0, gulp_filter_1.default)(f => !f.isDirectory()));
const compressed = all
.pipe((0, gulp_filter_1.default)(f => MimeTypesToCompress.has(mime_1.default.lookup(f.path))))
.pipe((0, gulp_gzip_1.default)({ append: false }))
.pipe(azure.upload(options(true)));
const uncompressed = all
.pipe((0, gulp_filter_1.default)(f => !MimeTypesToCompress.has(mime_1.default.lookup(f.path))))
.pipe(azure.upload(options(false)));
const out = event_stream_1.default.merge(compressed, uncompressed)
.pipe(event_stream_1.default.through(function (f) {
console.log('Uploaded:', f.relative);
files.push(f.relative);
this.emit('data', f);
}));
console.log(`Uploading files to CDN...`); // debug
await wait(out);
const listing = new vinyl_1.default({
path: 'files.txt',
contents: Buffer.from(files.join('\n')),
stat: { mode: 0o666 }
});
const filesOut = event_stream_1.default.readArray([listing])
.pipe((0, gulp_gzip_1.default)({ append: false }))
.pipe(azure.upload(options(true)));
console.log(`Uploading: files.txt (${files.length} files)`); // debug
await wait(filesOut);
}
main().catch(err => {
console.error(err);
process.exit(1);
});
//# sourceMappingURL=upload-cdn.js.map
================================================
FILE: build/azure-pipelines/upload-cdn.ts
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
import es from 'event-stream';
import Vinyl from 'vinyl';
import vfs from 'vinyl-fs';
import filter from 'gulp-filter';
import gzip from 'gulp-gzip';
import mime from 'mime';
import { ClientAssertionCredential } from '@azure/identity';
const azure = require('gulp-azure-storage');
const commit = process.env['BUILD_SOURCEVERSION'];
const credential = new ClientAssertionCredential(process.env['AZURE_TENANT_ID']!, process.env['AZURE_CLIENT_ID']!, () => Promise.resolve(process.env['AZURE_ID_TOKEN']!));
mime.define({
'application/typescript': ['ts'],
'application/json': ['code-snippets'],
});
// From default AFD configuration
const MimeTypesToCompress = new Set([
'application/eot',
'application/font',
'application/font-sfnt',
'application/javascript',
'application/json',
'application/opentype',
'application/otf',
'application/pkcs7-mime',
'application/truetype',
'application/ttf',
'application/typescript',
'application/vnd.ms-fontobject',
'application/xhtml+xml',
'application/xml',
'application/xml+rss',
'application/x-font-opentype',
'application/x-font-truetype',
'application/x-font-ttf',
'application/x-httpd-cgi',
'application/x-javascript',
'application/x-mpegurl',
'application/x-opentype',
'application/x-otf',
'application/x-perl',
'application/x-ttf',
'font/eot',
'font/ttf',
'font/otf',
'font/opentype',
'image/svg+xml',
'text/css',
'text/csv',
'text/html',
'text/javascript',
'text/js',
'text/markdown',
'text/plain',
'text/richtext',
'text/tab-separated-values',
'text/xml',
'text/x-script',
'text/x-component',
'text/x-java-source'
]);
function wait(stream: es.ThroughStream): Promise {
return new Promise((c, e) => {
stream.on('end', () => c());
stream.on('error', (err: any) => e(err));
});
}
async function main(): Promise {
const files: string[] = [];
const options = (compressed: boolean) => ({
account: process.env.AZURE_STORAGE_ACCOUNT,
credential,
container: '$web',
prefix: `${process.env.VSCODE_QUALITY}/${commit}/`,
contentSettings: {
contentEncoding: compressed ? 'gzip' : undefined,
cacheControl: 'max-age=31536000, public'
}
});
const all = vfs.src('**', { cwd: '../vscode-web', base: '../vscode-web', dot: true })
.pipe(filter(f => !f.isDirectory()));
const compressed = all
.pipe(filter(f => MimeTypesToCompress.has(mime.lookup(f.path))))
.pipe(gzip({ append: false }))
.pipe(azure.upload(options(true)));
const uncompressed = all
.pipe(filter(f => !MimeTypesToCompress.has(mime.lookup(f.path))))
.pipe(azure.upload(options(false)));
const out = es.merge(compressed, uncompressed)
.pipe(es.through(function (f) {
console.log('Uploaded:', f.relative);
files.push(f.relative);
this.emit('data', f);
}));
console.log(`Uploading files to CDN...`); // debug
await wait(out);
const listing = new Vinyl({
path: 'files.txt',
contents: Buffer.from(files.join('\n')),
stat: { mode: 0o666 } as any
});
const filesOut = es.readArray([listing])
.pipe(gzip({ append: false }))
.pipe(azure.upload(options(true)));
console.log(`Uploading: files.txt (${files.length} files)`); // debug
await wait(filesOut);
}
main().catch(err => {
console.error(err);
process.exit(1);
});
================================================
FILE: build/azure-pipelines/upload-nlsmetadata.js
================================================
"use strict";
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
Object.defineProperty(exports, "__esModule", { value: true });
const event_stream_1 = __importDefault(require("event-stream"));
const vinyl_fs_1 = __importDefault(require("vinyl-fs"));
const gulp_merge_json_1 = __importDefault(require("gulp-merge-json"));
const gulp_gzip_1 = __importDefault(require("gulp-gzip"));
const identity_1 = require("@azure/identity");
const path = require("path");
const fs_1 = require("fs");
const azure = require('gulp-azure-storage');
const commit = process.env['BUILD_SOURCEVERSION'];
const credential = new identity_1.ClientAssertionCredential(process.env['AZURE_TENANT_ID'], process.env['AZURE_CLIENT_ID'], () => Promise.resolve(process.env['AZURE_ID_TOKEN']));
function main() {
return new Promise((c, e) => {
const combinedMetadataJson = event_stream_1.default.merge(
// vscode: we are not using `out-build/nls.metadata.json` here because
// it includes metadata for translators for `keys`. but for our purpose
// we want only the `keys` and `messages` as `string`.
event_stream_1.default.merge(vinyl_fs_1.default.src('out-build/nls.keys.json', { base: 'out-build' }), vinyl_fs_1.default.src('out-build/nls.messages.json', { base: 'out-build' }))
.pipe((0, gulp_merge_json_1.default)({
fileName: 'vscode.json',
jsonSpace: '',
concatArrays: true,
edit: (parsedJson, file) => {
if (file.base === 'out-build') {
if (file.basename === 'nls.keys.json') {
return { keys: parsedJson };
}
else {
return { messages: parsedJson };
}
}
}
})),
// extensions
vinyl_fs_1.default.src('.build/extensions/**/nls.metadata.json', { base: '.build/extensions' }), vinyl_fs_1.default.src('.build/extensions/**/nls.metadata.header.json', { base: '.build/extensions' }), vinyl_fs_1.default.src('.build/extensions/**/package.nls.json', { base: '.build/extensions' })).pipe((0, gulp_merge_json_1.default)({
fileName: 'combined.nls.metadata.json',
jsonSpace: '',
concatArrays: true,
edit: (parsedJson, file) => {
if (file.basename === 'vscode.json') {
return { vscode: parsedJson };
}
// Handle extensions and follow the same structure as the Core nls file.
switch (file.basename) {
case 'package.nls.json':
// put package.nls.json content in Core NlsMetadata format
// language packs use the key "package" to specify that
// translations are for the package.json file
parsedJson = {
messages: {
package: Object.values(parsedJson)
},
keys: {
package: Object.keys(parsedJson)
},
bundles: {
main: ['package']
}
};
break;
case 'nls.metadata.header.json':
parsedJson = { header: parsedJson };
break;
case 'nls.metadata.json': {
// put nls.metadata.json content in Core NlsMetadata format
const modules = Object.keys(parsedJson);
const json = {
keys: {},
messages: {},
bundles: {
main: []
}
};
for (const module of modules) {
json.messages[module] = parsedJson[module].messages;
json.keys[module] = parsedJson[module].keys;
json.bundles.main.push(module);
}
parsedJson = json;
break;
}
}
// Get extension id and use that as the key
const folderPath = path.join(file.base, file.relative.split('/')[0]);
const manifest = (0, fs_1.readFileSync)(path.join(folderPath, 'package.json'), 'utf-8');
const manifestJson = JSON.parse(manifest);
const key = manifestJson.publisher + '.' + manifestJson.name;
return { [key]: parsedJson };
},
}));
const nlsMessagesJs = vinyl_fs_1.default.src('out-build/nls.messages.js', { base: 'out-build' });
event_stream_1.default.merge(combinedMetadataJson, nlsMessagesJs)
.pipe((0, gulp_gzip_1.default)({ append: false }))
.pipe(vinyl_fs_1.default.dest('./nlsMetadata'))
.pipe(event_stream_1.default.through(function (data) {
console.log(`Uploading ${data.path}`);
// trigger artifact upload
console.log(`##vso[artifact.upload containerfolder=nlsmetadata;artifactname=${data.basename}]${data.path}`);
this.emit('data', data);
}))
.pipe(azure.upload({
account: process.env.AZURE_STORAGE_ACCOUNT,
credential,
container: '$web',
prefix: `nlsmetadata/${commit}/`,
contentSettings: {
contentEncoding: 'gzip',
cacheControl: 'max-age=31536000, public'
}
}))
.on('end', () => c())
.on('error', (err) => e(err));
});
}
main().catch(err => {
console.error(err);
process.exit(1);
});
//# sourceMappingURL=upload-nlsmetadata.js.map
================================================
FILE: build/azure-pipelines/upload-nlsmetadata.ts
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
import es from 'event-stream';
import Vinyl from 'vinyl';
import vfs from 'vinyl-fs';
import merge from 'gulp-merge-json';
import gzip from 'gulp-gzip';
import { ClientAssertionCredential } from '@azure/identity';
import path = require('path');
import { readFileSync } from 'fs';
const azure = require('gulp-azure-storage');
const commit = process.env['BUILD_SOURCEVERSION'];
const credential = new ClientAssertionCredential(process.env['AZURE_TENANT_ID']!, process.env['AZURE_CLIENT_ID']!, () => Promise.resolve(process.env['AZURE_ID_TOKEN']!));
interface NlsMetadata {
keys: { [module: string]: string };
messages: { [module: string]: string };
bundles: { [bundle: string]: string[] };
}
function main(): Promise {
return new Promise((c, e) => {
const combinedMetadataJson = es.merge(
// vscode: we are not using `out-build/nls.metadata.json` here because
// it includes metadata for translators for `keys`. but for our purpose
// we want only the `keys` and `messages` as `string`.
es.merge(
vfs.src('out-build/nls.keys.json', { base: 'out-build' }),
vfs.src('out-build/nls.messages.json', { base: 'out-build' }))
.pipe(merge({
fileName: 'vscode.json',
jsonSpace: '',
concatArrays: true,
edit: (parsedJson, file) => {
if (file.base === 'out-build') {
if (file.basename === 'nls.keys.json') {
return { keys: parsedJson };
} else {
return { messages: parsedJson };
}
}
}
})),
// extensions
vfs.src('.build/extensions/**/nls.metadata.json', { base: '.build/extensions' }),
vfs.src('.build/extensions/**/nls.metadata.header.json', { base: '.build/extensions' }),
vfs.src('.build/extensions/**/package.nls.json', { base: '.build/extensions' })
).pipe(merge({
fileName: 'combined.nls.metadata.json',
jsonSpace: '',
concatArrays: true,
edit: (parsedJson, file) => {
if (file.basename === 'vscode.json') {
return { vscode: parsedJson };
}
// Handle extensions and follow the same structure as the Core nls file.
switch (file.basename) {
case 'package.nls.json':
// put package.nls.json content in Core NlsMetadata format
// language packs use the key "package" to specify that
// translations are for the package.json file
parsedJson = {
messages: {
package: Object.values(parsedJson)
},
keys: {
package: Object.keys(parsedJson)
},
bundles: {
main: ['package']
}
};
break;
case 'nls.metadata.header.json':
parsedJson = { header: parsedJson };
break;
case 'nls.metadata.json': {
// put nls.metadata.json content in Core NlsMetadata format
const modules = Object.keys(parsedJson);
const json: NlsMetadata = {
keys: {},
messages: {},
bundles: {
main: []
}
};
for (const module of modules) {
json.messages[module] = parsedJson[module].messages;
json.keys[module] = parsedJson[module].keys;
json.bundles.main.push(module);
}
parsedJson = json;
break;
}
}
// Get extension id and use that as the key
const folderPath = path.join(file.base, file.relative.split('/')[0]);
const manifest = readFileSync(path.join(folderPath, 'package.json'), 'utf-8');
const manifestJson = JSON.parse(manifest);
const key = manifestJson.publisher + '.' + manifestJson.name;
return { [key]: parsedJson };
},
}));
const nlsMessagesJs = vfs.src('out-build/nls.messages.js', { base: 'out-build' });
es.merge(combinedMetadataJson, nlsMessagesJs)
.pipe(gzip({ append: false }))
.pipe(vfs.dest('./nlsMetadata'))
.pipe(es.through(function (data: Vinyl) {
console.log(`Uploading ${data.path}`);
// trigger artifact upload
console.log(`##vso[artifact.upload containerfolder=nlsmetadata;artifactname=${data.basename}]${data.path}`);
this.emit('data', data);
}))
.pipe(azure.upload({
account: process.env.AZURE_STORAGE_ACCOUNT,
credential,
container: '$web',
prefix: `nlsmetadata/${commit}/`,
contentSettings: {
contentEncoding: 'gzip',
cacheControl: 'max-age=31536000, public'
}
}))
.on('end', () => c())
.on('error', (err: any) => e(err));
});
}
main().catch(err => {
console.error(err);
process.exit(1);
});
================================================
FILE: build/azure-pipelines/upload-sourcemaps.js
================================================
"use strict";
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
if (k2 === undefined) k2 = k;
var desc = Object.getOwnPropertyDescriptor(m, k);
if (!desc || ("get" in desc ? !m.__esModule : desc.writable || desc.configurable)) {
desc = { enumerable: true, get: function() { return m[k]; } };
}
Object.defineProperty(o, k2, desc);
}) : (function(o, m, k, k2) {
if (k2 === undefined) k2 = k;
o[k2] = m[k];
}));
var __setModuleDefault = (this && this.__setModuleDefault) || (Object.create ? (function(o, v) {
Object.defineProperty(o, "default", { enumerable: true, value: v });
}) : function(o, v) {
o["default"] = v;
});
var __importStar = (this && this.__importStar) || (function () {
var ownKeys = function(o) {
ownKeys = Object.getOwnPropertyNames || function (o) {
var ar = [];
for (var k in o) if (Object.prototype.hasOwnProperty.call(o, k)) ar[ar.length] = k;
return ar;
};
return ownKeys(o);
};
return function (mod) {
if (mod && mod.__esModule) return mod;
var result = {};
if (mod != null) for (var k = ownKeys(mod), i = 0; i < k.length; i++) if (k[i] !== "default") __createBinding(result, mod, k[i]);
__setModuleDefault(result, mod);
return result;
};
})();
var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
Object.defineProperty(exports, "__esModule", { value: true });
const path_1 = __importDefault(require("path"));
const event_stream_1 = __importDefault(require("event-stream"));
const vinyl_fs_1 = __importDefault(require("vinyl-fs"));
const util = __importStar(require("../lib/util"));
const dependencies_1 = require("../lib/dependencies");
const identity_1 = require("@azure/identity");
const azure = require('gulp-azure-storage');
const root = path_1.default.dirname(path_1.default.dirname(__dirname));
const commit = process.env['BUILD_SOURCEVERSION'];
const credential = new identity_1.ClientAssertionCredential(process.env['AZURE_TENANT_ID'], process.env['AZURE_CLIENT_ID'], () => Promise.resolve(process.env['AZURE_ID_TOKEN']));
// optionally allow to pass in explicit base/maps to upload
const [, , base, maps] = process.argv;
function src(base, maps = `${base}/**/*.map`) {
return vinyl_fs_1.default.src(maps, { base })
.pipe(event_stream_1.default.mapSync((f) => {
f.path = `${f.base}/core/${f.relative}`;
return f;
}));
}
function main() {
const sources = [];
// vscode client maps (default)
if (!base) {
const vs = src('out-vscode-min'); // client source-maps only
sources.push(vs);
const productionDependencies = (0, dependencies_1.getProductionDependencies)(root);
const productionDependenciesSrc = productionDependencies.map((d) => path_1.default.relative(root, d)).map((d) => `./${d}/**/*.map`);
const nodeModules = vinyl_fs_1.default.src(productionDependenciesSrc, { base: '.' })
.pipe(util.cleanNodeModules(path_1.default.join(root, 'build', '.moduleignore')))
.pipe(util.cleanNodeModules(path_1.default.join(root, 'build', `.moduleignore.${process.platform}`)));
sources.push(nodeModules);
const extensionsOut = vinyl_fs_1.default.src(['.build/extensions/**/*.js.map', '!**/node_modules/**'], { base: '.build' });
sources.push(extensionsOut);
}
// specific client base/maps
else {
sources.push(src(base, maps));
}
return new Promise((c, e) => {
event_stream_1.default.merge(...sources)
.pipe(event_stream_1.default.through(function (data) {
console.log('Uploading Sourcemap', data.relative); // debug
this.emit('data', data);
}))
.pipe(azure.upload({
account: process.env.AZURE_STORAGE_ACCOUNT,
credential,
container: '$web',
prefix: `sourcemaps/${commit}/`
}))
.on('end', () => c())
.on('error', (err) => e(err));
});
}
main().catch(err => {
console.error(err);
process.exit(1);
});
//# sourceMappingURL=upload-sourcemaps.js.map
================================================
FILE: build/azure-pipelines/upload-sourcemaps.ts
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
import path from 'path';
import es from 'event-stream';
import Vinyl from 'vinyl';
import vfs from 'vinyl-fs';
import * as util from '../lib/util';
import { getProductionDependencies } from '../lib/dependencies';
import { ClientAssertionCredential } from '@azure/identity';
const azure = require('gulp-azure-storage');
const root = path.dirname(path.dirname(__dirname));
const commit = process.env['BUILD_SOURCEVERSION'];
const credential = new ClientAssertionCredential(process.env['AZURE_TENANT_ID']!, process.env['AZURE_CLIENT_ID']!, () => Promise.resolve(process.env['AZURE_ID_TOKEN']!));
// optionally allow to pass in explicit base/maps to upload
const [, , base, maps] = process.argv;
function src(base: string, maps = `${base}/**/*.map`) {
return vfs.src(maps, { base })
.pipe(es.mapSync((f: Vinyl) => {
f.path = `${f.base}/core/${f.relative}`;
return f;
}));
}
function main(): Promise {
const sources: any[] = [];
// vscode client maps (default)
if (!base) {
const vs = src('out-vscode-min'); // client source-maps only
sources.push(vs);
const productionDependencies = getProductionDependencies(root);
const productionDependenciesSrc = productionDependencies.map((d: string) => path.relative(root, d)).map((d: string) => `./${d}/**/*.map`);
const nodeModules = vfs.src(productionDependenciesSrc, { base: '.' })
.pipe(util.cleanNodeModules(path.join(root, 'build', '.moduleignore')))
.pipe(util.cleanNodeModules(path.join(root, 'build', `.moduleignore.${process.platform}`)));
sources.push(nodeModules);
const extensionsOut = vfs.src(['.build/extensions/**/*.js.map', '!**/node_modules/**'], { base: '.build' });
sources.push(extensionsOut);
}
// specific client base/maps
else {
sources.push(src(base, maps));
}
return new Promise((c, e) => {
es.merge(...sources)
.pipe(es.through(function (data: Vinyl) {
console.log('Uploading Sourcemap', data.relative); // debug
this.emit('data', data);
}))
.pipe(azure.upload({
account: process.env.AZURE_STORAGE_ACCOUNT,
credential,
container: '$web',
prefix: `sourcemaps/${commit}/`
}))
.on('end', () => c())
.on('error', (err: any) => e(err));
});
}
main().catch(err => {
console.error(err);
process.exit(1);
});
================================================
FILE: build/azure-pipelines/web/product-build-web.yml
================================================
steps:
- task: NodeTool@0
inputs:
versionSource: fromFile
versionFilePath: .nvmrc
nodejsMirror: https://github.com/joaomoreno/node-mirror/releases/download
- template: ../distro/download-distro.yml@self
- task: AzureKeyVault@2
displayName: "Azure Key Vault: Get Secrets"
inputs:
azureSubscription: vscode
KeyVaultName: vscode-build-secrets
SecretsFilter: "github-distro-mixin-password"
- task: DownloadPipelineArtifact@2
inputs:
artifact: Compilation
path: $(Build.ArtifactStagingDirectory)
displayName: Download compilation output
- script: tar -xzf $(Build.ArtifactStagingDirectory)/compilation.tar.gz
displayName: Extract compilation output
- script: node build/setup-npm-registry.js $NPM_REGISTRY
condition: and(succeeded(), ne(variables['NPM_REGISTRY'], 'none'))
displayName: Setup NPM Registry
- script: mkdir -p .build && node build/azure-pipelines/common/computeNodeModulesCacheKey.js web > .build/packagelockhash
displayName: Prepare node_modules cache key
- task: Cache@2
inputs:
key: '"node_modules" | .build/packagelockhash'
path: .build/node_modules_cache
cacheHitVar: NODE_MODULES_RESTORED
displayName: Restore node_modules cache
- script: tar -xzf .build/node_modules_cache/cache.tgz
condition: and(succeeded(), eq(variables.NODE_MODULES_RESTORED, 'true'))
displayName: Extract node_modules cache
- script: |
set -e
# Set the private NPM registry to the global npmrc file
# so that authentication works for subfolders like build/, remote/, extensions/ etc
# which does not have their own .npmrc file
npm config set registry "$NPM_REGISTRY"
echo "##vso[task.setvariable variable=NPMRC_PATH]$(npm config get userconfig)"
condition: and(succeeded(), ne(variables.NODE_MODULES_RESTORED, 'true'), ne(variables['NPM_REGISTRY'], 'none'))
displayName: Setup NPM
- task: npmAuthenticate@0
inputs:
workingFile: $(NPMRC_PATH)
condition: and(succeeded(), ne(variables.NODE_MODULES_RESTORED, 'true'), ne(variables['NPM_REGISTRY'], 'none'))
displayName: Setup NPM Authentication
- script: |
set -e
./build/azure-pipelines/linux/apt-retry.sh sudo apt-get update
./build/azure-pipelines/linux/apt-retry.sh sudo apt-get install -y libkrb5-dev
displayName: Setup system services
condition: and(succeeded(), ne(variables.NODE_MODULES_RESTORED, 'true'))
- script: |
set -e
for i in {1..5}; do # try 5 times
npm ci && break
if [ $i -eq 5 ]; then
echo "Npm install failed too many times" >&2
exit 1
fi
echo "Npm install failed $i, trying again..."
done
env:
ELECTRON_SKIP_BINARY_DOWNLOAD: 1
PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD: 1
GITHUB_TOKEN: "$(github-distro-mixin-password)"
displayName: Install dependencies
condition: and(succeeded(), ne(variables.NODE_MODULES_RESTORED, 'true'))
- script: node build/azure-pipelines/distro/mixin-npm
condition: and(succeeded(), ne(variables.NODE_MODULES_RESTORED, 'true'))
displayName: Mixin distro node modules
- script: |
set -e
node build/azure-pipelines/common/listNodeModules.js .build/node_modules_list.txt
mkdir -p .build/node_modules_cache
tar -czf .build/node_modules_cache/cache.tgz --files-from .build/node_modules_list.txt
condition: and(succeeded(), ne(variables.NODE_MODULES_RESTORED, 'true'))
displayName: Create node_modules archive
- script: node build/azure-pipelines/distro/mixin-quality
displayName: Mixin distro quality
- template: ../common/install-builtin-extensions.yml@self
- script: |
set -e
npm run gulp vscode-web-min-ci
ARCHIVE_PATH=".build/web/vscode-web.tar.gz"
mkdir -p $(dirname $ARCHIVE_PATH)
tar --owner=0 --group=0 -czf $ARCHIVE_PATH -C .. vscode-web
echo "##vso[task.setvariable variable=WEB_PATH]$ARCHIVE_PATH"
env:
GITHUB_TOKEN: "$(github-distro-mixin-password)"
displayName: Build
- task: AzureCLI@2
displayName: Fetch secrets from Azure
inputs:
azureSubscription: vscode
scriptType: pscore
scriptLocation: inlineScript
addSpnToEnvironment: true
inlineScript: |
Write-Host "##vso[task.setvariable variable=AZURE_TENANT_ID]$env:tenantId"
Write-Host "##vso[task.setvariable variable=AZURE_CLIENT_ID]$env:servicePrincipalId"
Write-Host "##vso[task.setvariable variable=AZURE_ID_TOKEN;issecret=true]$env:idToken"
- script: |
set -e
AZURE_STORAGE_ACCOUNT="vscodeweb" \
AZURE_TENANT_ID="$(AZURE_TENANT_ID)" \
AZURE_CLIENT_ID="$(AZURE_CLIENT_ID)" \
AZURE_ID_TOKEN="$(AZURE_ID_TOKEN)" \
node build/azure-pipelines/upload-cdn
displayName: Upload to CDN
- script: |
set -e
AZURE_STORAGE_ACCOUNT="vscodeweb" \
AZURE_TENANT_ID="$(AZURE_TENANT_ID)" \
AZURE_CLIENT_ID="$(AZURE_CLIENT_ID)" \
AZURE_ID_TOKEN="$(AZURE_ID_TOKEN)" \
node build/azure-pipelines/upload-sourcemaps out-vscode-web-min out-vscode-web-min/vs/workbench/workbench.web.main.js.map
displayName: Upload sourcemaps (Web Main)
- script: |
set -e
AZURE_STORAGE_ACCOUNT="vscodeweb" \
AZURE_TENANT_ID="$(AZURE_TENANT_ID)" \
AZURE_CLIENT_ID="$(AZURE_CLIENT_ID)" \
AZURE_ID_TOKEN="$(AZURE_ID_TOKEN)" \
node build/azure-pipelines/upload-sourcemaps out-vscode-web-min out-vscode-web-min/vs/workbench/workbench.web.main.internal.js.map
displayName: Upload sourcemaps (Web Internal)
- script: |
set -e
AZURE_STORAGE_ACCOUNT="vscodeweb" \
AZURE_TENANT_ID="$(AZURE_TENANT_ID)" \
AZURE_CLIENT_ID="$(AZURE_CLIENT_ID)" \
AZURE_ID_TOKEN="$(AZURE_ID_TOKEN)" \
node build/azure-pipelines/upload-nlsmetadata
displayName: Upload NLS Metadata
- script: echo "##vso[task.setvariable variable=ARTIFACT_PREFIX]attempt$(System.JobAttempt)_"
condition: and(succeededOrFailed(), notIn(variables['Agent.JobStatus'], 'Succeeded', 'SucceededWithIssues'))
displayName: Generate artifact prefix
- task: 1ES.PublishPipelineArtifact@1
inputs:
targetPath: $(WEB_PATH)
artifactName: $(ARTIFACT_PREFIX)vscode_web_linux_standalone_archive-unsigned
sbomBuildDropPath: $(Agent.BuildDirectory)/vscode-web
sbomPackageName: "VS Code Web"
sbomPackageVersion: $(Build.SourceVersion)
condition: and(succeededOrFailed(), ne(variables['WEB_PATH'], ''))
displayName: Publish web archive
================================================
FILE: build/azure-pipelines/win32/cli-build-win32.yml
================================================
parameters:
- name: VSCODE_BUILD_WIN32
type: boolean
default: false
- name: VSCODE_BUILD_WIN32_ARM64
type: boolean
default: false
- name: VSCODE_CHECK_ONLY
type: boolean
default: false
- name: VSCODE_QUALITY
type: string
steps:
- task: NodeTool@0
inputs:
versionSource: fromFile
versionFilePath: .nvmrc
nodejsMirror: https://github.com/joaomoreno/node-mirror/releases/download
- ${{ if ne(parameters.VSCODE_QUALITY, 'oss') }}:
- template: ../cli/cli-apply-patches.yml@self
- task: Npm@1
displayName: Download openssl prebuilt
inputs:
command: custom
customCommand: pack @vscode-internal/openssl-prebuilt@0.0.11
customRegistry: useFeed
customFeed: "Monaco/openssl-prebuilt"
workingDir: $(Build.ArtifactStagingDirectory)
- powershell: |
mkdir $(Build.ArtifactStagingDirectory)/openssl
tar -xvzf $(Build.ArtifactStagingDirectory)/vscode-internal-openssl-prebuilt-0.0.11.tgz --strip-components=1 --directory=$(Build.ArtifactStagingDirectory)/openssl
displayName: Extract openssl prebuilt
- template: ../cli/install-rust-win32.yml@self
parameters:
targets:
- ${{ if eq(parameters.VSCODE_BUILD_WIN32, true) }}:
- x86_64-pc-windows-msvc
- ${{ if eq(parameters.VSCODE_BUILD_WIN32_ARM64, true) }}:
- aarch64-pc-windows-msvc
- ${{ if eq(parameters.VSCODE_BUILD_WIN32, true) }}:
- template: ../cli/cli-compile.yml@self
parameters:
VSCODE_QUALITY: ${{ parameters.VSCODE_QUALITY }}
VSCODE_CLI_TARGET: x86_64-pc-windows-msvc
VSCODE_CLI_ARTIFACT: unsigned_vscode_cli_win32_x64_cli
VSCODE_CHECK_ONLY: ${{ parameters.VSCODE_CHECK_ONLY }}
VSCODE_CLI_ENV:
OPENSSL_LIB_DIR: $(Build.ArtifactStagingDirectory)/openssl/x64-windows-static/lib
OPENSSL_INCLUDE_DIR: $(Build.ArtifactStagingDirectory)/openssl/x64-windows-static/include
RUSTFLAGS: "-Ctarget-feature=+crt-static -Clink-args=/guard:cf -Clink-args=/CETCOMPAT"
CFLAGS: "/guard:cf /Qspectre"
- ${{ if eq(parameters.VSCODE_BUILD_WIN32_ARM64, true) }}:
- template: ../cli/cli-compile.yml@self
parameters:
VSCODE_QUALITY: ${{ parameters.VSCODE_QUALITY }}
VSCODE_CLI_TARGET: aarch64-pc-windows-msvc
VSCODE_CLI_ARTIFACT: unsigned_vscode_cli_win32_arm64_cli
VSCODE_CHECK_ONLY: ${{ parameters.VSCODE_CHECK_ONLY }}
VSCODE_CLI_ENV:
OPENSSL_LIB_DIR: $(Build.ArtifactStagingDirectory)/openssl/arm64-windows-static/lib
OPENSSL_INCLUDE_DIR: $(Build.ArtifactStagingDirectory)/openssl/arm64-windows-static/include
RUSTFLAGS: "-C target-feature=+crt-static -Clink-args=/guard:cf -Clink-args=/CETCOMPAT:NO"
CFLAGS: "/guard:cf /Qspectre"
- ${{ if not(parameters.VSCODE_CHECK_ONLY) }}:
- ${{ if eq(parameters.VSCODE_BUILD_WIN32_ARM64, true) }}:
- task: 1ES.PublishPipelineArtifact@1
inputs:
targetPath: $(Build.ArtifactStagingDirectory)/unsigned_vscode_cli_win32_arm64_cli.zip
artifactName: unsigned_vscode_cli_win32_arm64_cli
sbomBuildDropPath: $(Build.ArtifactStagingDirectory)/cli
sbomPackageName: "VS Code Windows arm64 CLI (unsigned)"
sbomPackageVersion: $(Build.SourceVersion)
displayName: Publish unsigned_vscode_cli_win32_arm64_cli artifact
- ${{ if eq(parameters.VSCODE_BUILD_WIN32, true) }}:
- task: 1ES.PublishPipelineArtifact@1
inputs:
targetPath: $(Build.ArtifactStagingDirectory)/unsigned_vscode_cli_win32_x64_cli.zip
artifactName: unsigned_vscode_cli_win32_x64_cli
sbomBuildDropPath: $(Build.ArtifactStagingDirectory)/cli
sbomPackageName: "VS Code Windows x64 CLI (unsigned)"
sbomPackageVersion: $(Build.SourceVersion)
displayName: Publish unsigned_vscode_cli_win32_x64_cli artifact
================================================
FILE: build/azure-pipelines/win32/exec.ps1
================================================
# Taken from psake https://github.com/psake/psake
<#
.SYNOPSIS
This is a helper function that runs a scriptblock and checks the PS variable $lastexitcode
to see if an error occcured. If an error is detected then an exception is thrown.
This function allows you to run command-line programs without having to
explicitly check the $lastexitcode variable.
.EXAMPLE
exec { svn info $repository_trunk } "Error executing SVN. Please verify SVN command-line client is installed"
#>
function Exec
{
[CmdletBinding()]
param(
[Parameter(Position=0,Mandatory=1)][scriptblock]$cmd,
[Parameter(Position=1,Mandatory=0)][string]$errorMessage = ($msgs.error_bad_command -f $cmd)
)
& $cmd
if ($lastexitcode -ne 0) {
throw ("Exec: " + $errorMessage)
}
}
================================================
FILE: build/azure-pipelines/win32/import-esrp-auth-cert.ps1
================================================
param ($CertBase64)
$ErrorActionPreference = "Stop"
$CertBytes = [System.Convert]::FromBase64String($CertBase64)
$CertCollection = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2Collection
$CertCollection.Import($CertBytes, $null, [System.Security.Cryptography.X509Certificates.X509KeyStorageFlags]::Exportable -bxor [System.Security.Cryptography.X509Certificates.X509KeyStorageFlags]::PersistKeySet)
$CertStore = New-Object System.Security.Cryptography.X509Certificates.X509Store("My","LocalMachine")
$CertStore.Open("ReadWrite")
$CertStore.AddRange($CertCollection)
$CertStore.Close()
$ESRPAuthCertificateSubjectName = $CertCollection[0].Subject
Write-Output ("##vso[task.setvariable variable=ESRPAuthCertificateSubjectName;]$ESRPAuthCertificateSubjectName")
================================================
FILE: build/azure-pipelines/win32/listprocesses.bat
================================================
echo "------------------------------------"
tasklist /V
echo "------------------------------------"
================================================
FILE: build/azure-pipelines/win32/product-build-win32-cli-sign.yml
================================================
parameters:
- name: VSCODE_BUILD_WIN32
type: boolean
- name: VSCODE_BUILD_WIN32_ARM64
type: boolean
- name: VSCODE_QUALITY
type: string
steps:
- task: NodeTool@0
displayName: "Use Node.js"
inputs:
versionSource: fromFile
versionFilePath: .nvmrc
nodejsMirror: https://github.com/joaomoreno/node-mirror/releases/download
- ${{ if ne(parameters.VSCODE_QUALITY, 'oss') }}:
- task: AzureKeyVault@2
displayName: "Azure Key Vault: Get Secrets"
inputs:
azureSubscription: vscode
KeyVaultName: vscode-build-secrets
SecretsFilter: "github-distro-mixin-password"
- powershell: node build/setup-npm-registry.js $env:NPM_REGISTRY build
condition: and(succeeded(), ne(variables['NPM_REGISTRY'], 'none'))
displayName: Setup NPM Registry
- powershell: |
. build/azure-pipelines/win32/exec.ps1
$ErrorActionPreference = "Stop"
# Set the private NPM registry to the global npmrc file
# so that authentication works for subfolders like build/, remote/, extensions/ etc
# which does not have their own .npmrc file
exec { npm config set registry "$env:NPM_REGISTRY" }
$NpmrcPath = (npm config get userconfig)
echo "##vso[task.setvariable variable=NPMRC_PATH]$NpmrcPath"
condition: and(succeeded(), ne(variables['NPM_REGISTRY'], 'none'))
displayName: Setup NPM
- task: npmAuthenticate@0
inputs:
workingFile: $(NPMRC_PATH)
condition: and(succeeded(), ne(variables['NPM_REGISTRY'], 'none'))
displayName: Setup NPM Authentication
- powershell: |
. azure-pipelines/win32/exec.ps1
$ErrorActionPreference = "Stop"
exec { npm ci }
workingDirectory: build
env:
GITHUB_TOKEN: "$(github-distro-mixin-password)"
retryCountOnTaskFailure: 5
displayName: Install build dependencies
- template: ../cli/cli-win32-sign.yml@self
parameters:
VSCODE_CLI_ARTIFACTS:
- ${{ if eq(parameters.VSCODE_BUILD_WIN32, true) }}:
- unsigned_vscode_cli_win32_x64_cli
- ${{ if eq(parameters.VSCODE_BUILD_WIN32_ARM64, true) }}:
- unsigned_vscode_cli_win32_arm64_cli
================================================
FILE: build/azure-pipelines/win32/product-build-win32-test.yml
================================================
parameters:
- name: VSCODE_QUALITY
type: string
- name: VSCODE_ARCH
type: string
- name: VSCODE_RUN_UNIT_TESTS
type: boolean
- name: VSCODE_RUN_INTEGRATION_TESTS
type: boolean
- name: VSCODE_RUN_SMOKE_TESTS
type: boolean
- name: PUBLISH_TASK_NAME
type: string
default: PublishPipelineArtifact@0
steps:
- powershell: npm exec -- npm-run-all -lp "electron $(VSCODE_ARCH)" "playwright-install"
env:
GITHUB_TOKEN: "$(github-distro-mixin-password)"
displayName: Download Electron and Playwright
retryCountOnTaskFailure: 3
- ${{ if eq(parameters.VSCODE_RUN_UNIT_TESTS, true) }}:
- ${{ if eq(parameters.VSCODE_QUALITY, 'oss') }}:
- powershell: .\scripts\test.bat --tfs "Unit Tests"
displayName: Run unit tests (Electron)
timeoutInMinutes: 15
- powershell: npm run test-node
displayName: Run unit tests (node.js)
timeoutInMinutes: 15
- powershell: node test/unit/browser/index.js --browser chromium --tfs "Browser Unit Tests"
displayName: Run unit tests (Browser, Chromium)
timeoutInMinutes: 20
- ${{ if ne(parameters.VSCODE_QUALITY, 'oss') }}:
- powershell: .\scripts\test.bat --build --tfs "Unit Tests"
displayName: Run unit tests (Electron)
timeoutInMinutes: 15
- powershell: npm run test-node -- --build
displayName: Run unit tests (node.js)
timeoutInMinutes: 15
- powershell: npm run test-browser-no-install -- --build --browser chromium --tfs "Browser Unit Tests"
displayName: Run unit tests (Browser, Chromium)
timeoutInMinutes: 20
- ${{ if eq(parameters.VSCODE_RUN_INTEGRATION_TESTS, true) }}:
- powershell: |
. build/azure-pipelines/win32/exec.ps1
$ErrorActionPreference = "Stop"
exec { npm run gulp `
compile-extension:configuration-editing `
compile-extension:css-language-features-server `
compile-extension:emmet `
compile-extension:git `
compile-extension:github-authentication `
compile-extension:html-language-features-server `
compile-extension:ipynb `
compile-extension:notebook-renderers `
compile-extension:json-language-features-server `
compile-extension:markdown-language-features `
compile-extension-media `
compile-extension:microsoft-authentication `
compile-extension:typescript-language-features `
compile-extension:vscode-api-tests `
compile-extension:vscode-colorize-tests `
compile-extension:vscode-colorize-perf-tests `
compile-extension:vscode-test-resolver `
}
displayName: Build integration tests
- powershell: .\build\azure-pipelines\win32\listprocesses.bat
displayName: Diagnostics before integration test runs
continueOnError: true
condition: succeededOrFailed()
- ${{ if eq(parameters.VSCODE_QUALITY, 'oss') }}:
- powershell: .\scripts\test-integration.bat --tfs "Integration Tests"
displayName: Run integration tests (Electron)
timeoutInMinutes: 20
- powershell: .\scripts\test-web-integration.bat --browser chromium
displayName: Run integration tests (Browser, Chromium)
timeoutInMinutes: 20
- powershell: .\scripts\test-remote-integration.bat
displayName: Run integration tests (Remote)
timeoutInMinutes: 20
- ${{ if ne(parameters.VSCODE_QUALITY, 'oss') }}:
- powershell: |
# Figure out the full absolute path of the product we just built
# including the remote server and configure the integration tests
# to run with these builds instead of running out of sources.
. build/azure-pipelines/win32/exec.ps1
$ErrorActionPreference = "Stop"
$AppRoot = "$(agent.builddirectory)\VSCode-win32-$(VSCODE_ARCH)"
$AppProductJson = Get-Content -Raw -Path "$AppRoot\resources\app\product.json" | ConvertFrom-Json
$AppNameShort = $AppProductJson.nameShort
$env:INTEGRATION_TEST_ELECTRON_PATH = "$AppRoot\$AppNameShort.exe"
$env:VSCODE_REMOTE_SERVER_PATH = "$(agent.builddirectory)\vscode-server-win32-$(VSCODE_ARCH)"
exec { .\scripts\test-integration.bat --build --tfs "Integration Tests" }
displayName: Run integration tests (Electron)
timeoutInMinutes: 20
- powershell: |
. build/azure-pipelines/win32/exec.ps1
$ErrorActionPreference = "Stop"
$env:VSCODE_REMOTE_SERVER_PATH = "$(agent.builddirectory)\vscode-server-win32-$(VSCODE_ARCH)-web"
exec { .\scripts\test-web-integration.bat --browser firefox }
displayName: Run integration tests (Browser, Firefox)
timeoutInMinutes: 20
- powershell: |
. build/azure-pipelines/win32/exec.ps1
$ErrorActionPreference = "Stop"
$AppRoot = "$(agent.builddirectory)\VSCode-win32-$(VSCODE_ARCH)"
$AppProductJson = Get-Content -Raw -Path "$AppRoot\resources\app\product.json" | ConvertFrom-Json
$AppNameShort = $AppProductJson.nameShort
$env:INTEGRATION_TEST_ELECTRON_PATH = "$AppRoot\$AppNameShort.exe"
$env:VSCODE_REMOTE_SERVER_PATH = "$(agent.builddirectory)\vscode-server-win32-$(VSCODE_ARCH)"
exec { .\scripts\test-remote-integration.bat }
displayName: Run integration tests (Remote)
timeoutInMinutes: 20
- powershell: .\build\azure-pipelines\win32\listprocesses.bat
displayName: Diagnostics after integration test runs
continueOnError: true
condition: succeededOrFailed()
- ${{ if eq(parameters.VSCODE_RUN_SMOKE_TESTS, true) }}:
- powershell: .\build\azure-pipelines\win32\listprocesses.bat
displayName: Diagnostics before smoke test run
continueOnError: true
condition: succeededOrFailed()
- ${{ if eq(parameters.VSCODE_QUALITY, 'oss') }}:
- powershell: npm run compile
workingDirectory: test/smoke
displayName: Compile smoke tests
- powershell: npm run gulp compile-extension-media
displayName: Build extensions for smoke tests
- powershell: npm run smoketest-no-compile -- --tracing
displayName: Run smoke tests (Electron)
timeoutInMinutes: 20
- ${{ if ne(parameters.VSCODE_QUALITY, 'oss') }}:
- powershell: npm run smoketest-no-compile -- --tracing --build "$(agent.builddirectory)\VSCode-win32-$(VSCODE_ARCH)"
displayName: Run smoke tests (Electron)
timeoutInMinutes: 20
- powershell: npm run smoketest-no-compile -- --web --tracing --headless
env:
VSCODE_REMOTE_SERVER_PATH: $(agent.builddirectory)\vscode-server-win32-$(VSCODE_ARCH)-web
displayName: Run smoke tests (Browser, Chromium)
timeoutInMinutes: 20
- powershell: npm run gulp compile-extension:vscode-test-resolver
displayName: Compile test resolver extension
timeoutInMinutes: 20
- powershell: npm run smoketest-no-compile -- --tracing --remote --build "$(agent.builddirectory)\VSCode-win32-$(VSCODE_ARCH)"
env:
VSCODE_REMOTE_SERVER_PATH: $(agent.builddirectory)\vscode-server-win32-$(VSCODE_ARCH)
displayName: Run smoke tests (Remote)
timeoutInMinutes: 20
- powershell: .\build\azure-pipelines\win32\listprocesses.bat
displayName: Diagnostics after smoke test run
continueOnError: true
condition: succeededOrFailed()
- ${{ if or(eq(parameters.VSCODE_RUN_INTEGRATION_TESTS, true), eq(parameters.VSCODE_RUN_SMOKE_TESTS, true)) }}:
- task: ${{ parameters.PUBLISH_TASK_NAME }}
inputs:
targetPath: .build\crashes
${{ if and(eq(parameters.VSCODE_RUN_INTEGRATION_TESTS, true), eq(parameters.VSCODE_RUN_SMOKE_TESTS, false)) }}:
artifactName: crash-dump-windows-$(VSCODE_ARCH)-integration-$(System.JobAttempt)
${{ elseif and(eq(parameters.VSCODE_RUN_INTEGRATION_TESTS, false), eq(parameters.VSCODE_RUN_SMOKE_TESTS, true)) }}:
artifactName: crash-dump-windows-$(VSCODE_ARCH)-smoke-$(System.JobAttempt)
${{ else }}:
artifactName: crash-dump-windows-$(VSCODE_ARCH)-$(System.JobAttempt)
sbomEnabled: false
displayName: "Publish Crash Reports"
continueOnError: true
condition: failed()
# In order to properly symbolify above crash reports
# (if any), we need the compiled native modules too
- task: ${{ parameters.PUBLISH_TASK_NAME }}
inputs:
targetPath: node_modules
${{ if and(eq(parameters.VSCODE_RUN_INTEGRATION_TESTS, true), eq(parameters.VSCODE_RUN_SMOKE_TESTS, false)) }}:
artifactName: node-modules-windows-$(VSCODE_ARCH)-integration-$(System.JobAttempt)
${{ elseif and(eq(parameters.VSCODE_RUN_INTEGRATION_TESTS, false), eq(parameters.VSCODE_RUN_SMOKE_TESTS, true)) }}:
artifactName: node-modules-windows-$(VSCODE_ARCH)-smoke-$(System.JobAttempt)
${{ else }}:
artifactName: node-modules-windows-$(VSCODE_ARCH)-$(System.JobAttempt)
sbomEnabled: false
displayName: "Publish Node Modules"
continueOnError: true
condition: failed()
- task: ${{ parameters.PUBLISH_TASK_NAME }}
inputs:
targetPath: .build\logs
${{ if and(eq(parameters.VSCODE_RUN_INTEGRATION_TESTS, true), eq(parameters.VSCODE_RUN_SMOKE_TESTS, false)) }}:
artifactName: logs-windows-$(VSCODE_ARCH)-integration-$(System.JobAttempt)
${{ elseif and(eq(parameters.VSCODE_RUN_INTEGRATION_TESTS, false), eq(parameters.VSCODE_RUN_SMOKE_TESTS, true)) }}:
artifactName: logs-windows-$(VSCODE_ARCH)-smoke-$(System.JobAttempt)
${{ else }}:
artifactName: logs-windows-$(VSCODE_ARCH)-$(System.JobAttempt)
sbomEnabled: false
displayName: "Publish Log Files"
continueOnError: true
condition: succeededOrFailed()
- task: PublishTestResults@2
displayName: Publish Tests Results
inputs:
testResultsFiles: "*-results.xml"
searchFolder: "$(Build.ArtifactStagingDirectory)/test-results"
condition: succeededOrFailed()
================================================
FILE: build/azure-pipelines/win32/product-build-win32.yml
================================================
parameters:
- name: VSCODE_QUALITY
type: string
- name: VSCODE_ARCH
type: string
- name: VSCODE_CIBUILD
type: boolean
- name: VSCODE_RUN_UNIT_TESTS
type: boolean
- name: VSCODE_RUN_INTEGRATION_TESTS
type: boolean
- name: VSCODE_RUN_SMOKE_TESTS
type: boolean
steps:
- ${{ if eq(parameters.VSCODE_QUALITY, 'oss') }}:
- checkout: self
fetchDepth: 1
retryCountOnTaskFailure: 3
- task: NodeTool@0
inputs:
versionSource: fromFile
versionFilePath: .nvmrc
nodejsMirror: https://github.com/joaomoreno/node-mirror/releases/download
- task: UsePythonVersion@0
inputs:
versionSpec: "3.x"
addToPath: true
- ${{ if ne(parameters.VSCODE_QUALITY, 'oss') }}:
- template: ../distro/download-distro.yml@self
- task: AzureKeyVault@2
displayName: "Azure Key Vault: Get Secrets"
inputs:
azureSubscription: vscode
KeyVaultName: vscode-build-secrets
SecretsFilter: "github-distro-mixin-password"
- task: DownloadPipelineArtifact@2
inputs:
artifact: Compilation
path: $(Build.ArtifactStagingDirectory)
displayName: Download compilation output
- task: ExtractFiles@1
displayName: Extract compilation output
inputs:
archiveFilePatterns: "$(Build.ArtifactStagingDirectory)/compilation.tar.gz"
cleanDestinationFolder: false
- powershell: node build/setup-npm-registry.js $env:NPM_REGISTRY
condition: and(succeeded(), ne(variables['NPM_REGISTRY'], 'none'))
displayName: Setup NPM Registry
- pwsh: |
mkdir .build -ea 0
node build/azure-pipelines/common/computeNodeModulesCacheKey.js win32 $(VSCODE_ARCH) > .build/packagelockhash
displayName: Prepare node_modules cache key
- task: Cache@2
inputs:
key: '"node_modules" | .build/packagelockhash'
path: .build/node_modules_cache
cacheHitVar: NODE_MODULES_RESTORED
displayName: Restore node_modules cache
- powershell: 7z.exe x .build/node_modules_cache/cache.7z -aoa
condition: and(succeeded(), eq(variables.NODE_MODULES_RESTORED, 'true'))
displayName: Extract node_modules cache
- powershell: |
. build/azure-pipelines/win32/exec.ps1
$ErrorActionPreference = "Stop"
# Set the private NPM registry to the global npmrc file
# so that authentication works for subfolders like build/, remote/, extensions/ etc
# which does not have their own .npmrc file
exec { npm config set registry "$env:NPM_REGISTRY" }
$NpmrcPath = (npm config get userconfig)
echo "##vso[task.setvariable variable=NPMRC_PATH]$NpmrcPath"
condition: and(succeeded(), ne(variables.NODE_MODULES_RESTORED, 'true'), ne(variables['NPM_REGISTRY'], 'none'))
displayName: Setup NPM
- task: npmAuthenticate@0
inputs:
workingFile: $(NPMRC_PATH)
condition: and(succeeded(), ne(variables.NODE_MODULES_RESTORED, 'true'), ne(variables['NPM_REGISTRY'], 'none'))
displayName: Setup NPM Authentication
- powershell: |
. build/azure-pipelines/win32/exec.ps1
$ErrorActionPreference = "Stop"
exec { npm ci }
env:
npm_config_arch: $(VSCODE_ARCH)
npm_config_foreground_scripts: "true"
ELECTRON_SKIP_BINARY_DOWNLOAD: 1
PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD: 1
GITHUB_TOKEN: "$(github-distro-mixin-password)"
retryCountOnTaskFailure: 5
displayName: Install dependencies
condition: and(succeeded(), ne(variables.NODE_MODULES_RESTORED, 'true'))
- ${{ if ne(parameters.VSCODE_QUALITY, 'oss') }}:
- powershell: node build/azure-pipelines/distro/mixin-npm
condition: and(succeeded(), ne(variables.NODE_MODULES_RESTORED, 'true'))
displayName: Mixin distro node modules
- powershell: |
. build/azure-pipelines/win32/exec.ps1
$ErrorActionPreference = "Stop"
exec { node build/azure-pipelines/common/listNodeModules.js .build/node_modules_list.txt }
exec { mkdir -Force .build/node_modules_cache }
exec { 7z.exe a .build/node_modules_cache/cache.7z -mx3 `@.build/node_modules_list.txt }
condition: and(succeeded(), ne(variables.NODE_MODULES_RESTORED, 'true'))
displayName: Create node_modules archive
- ${{ if ne(parameters.VSCODE_QUALITY, 'oss') }}:
- powershell: node build/azure-pipelines/distro/mixin-quality
displayName: Mixin distro quality
- template: ../common/install-builtin-extensions.yml@self
- ${{ if and(ne(parameters.VSCODE_CIBUILD, true), ne(parameters.VSCODE_QUALITY, 'oss')) }}:
- powershell: node build\lib\policies win32
displayName: Generate Group Policy definitions
retryCountOnTaskFailure: 3
- ${{ if eq(parameters.VSCODE_QUALITY, 'oss') }}:
- powershell: npm run gulp "transpile-client-esbuild" "transpile-extensions"
env:
GITHUB_TOKEN: "$(github-distro-mixin-password)"
displayName: Transpile client and extensions
- ${{ else }}:
- ${{ if and(ne(parameters.VSCODE_CIBUILD, true), eq(parameters.VSCODE_QUALITY, 'insider')) }}:
- powershell: node build/win32/explorer-appx-fetcher .build/win32/appx
displayName: Download Explorer Sparse Package
- powershell: |
. build/azure-pipelines/win32/exec.ps1
$ErrorActionPreference = "Stop"
exec { npm run gulp "vscode-win32-$(VSCODE_ARCH)-min-ci" }
exec { npm run gulp "vscode-win32-$(VSCODE_ARCH)-inno-updater" }
echo "##vso[task.setvariable variable=BUILT_CLIENT]true"
echo "##vso[task.setvariable variable=CodeSigningFolderPath]$(Agent.BuildDirectory)/VSCode-win32-$(VSCODE_ARCH)"
env:
GITHUB_TOKEN: "$(github-distro-mixin-password)"
displayName: Build client
- powershell: |
. build/azure-pipelines/win32/exec.ps1
$ErrorActionPreference = "Stop"
exec { npm run gulp "vscode-reh-win32-$(VSCODE_ARCH)-min-ci" }
mv ..\vscode-reh-win32-$(VSCODE_ARCH) ..\vscode-server-win32-$(VSCODE_ARCH) # TODO@joaomoreno
echo "##vso[task.setvariable variable=BUILT_SERVER]true"
echo "##vso[task.setvariable variable=CodeSigningFolderPath]$(CodeSigningFolderPath),$(Agent.BuildDirectory)/vscode-server-win32-$(VSCODE_ARCH)"
env:
GITHUB_TOKEN: "$(github-distro-mixin-password)"
displayName: Build server
- powershell: |
. build/azure-pipelines/win32/exec.ps1
$ErrorActionPreference = "Stop"
exec { npm run gulp "vscode-reh-web-win32-$(VSCODE_ARCH)-min-ci" }
mv ..\vscode-reh-web-win32-$(VSCODE_ARCH) ..\vscode-server-win32-$(VSCODE_ARCH)-web # TODO@joaomoreno
echo "##vso[task.setvariable variable=BUILT_WEB]true"
env:
GITHUB_TOKEN: "$(github-distro-mixin-password)"
displayName: Build server (web)
- ${{ if or(eq(parameters.VSCODE_RUN_UNIT_TESTS, true), eq(parameters.VSCODE_RUN_INTEGRATION_TESTS, true), eq(parameters.VSCODE_RUN_SMOKE_TESTS, true)) }}:
- template: product-build-win32-test.yml@self
parameters:
VSCODE_QUALITY: ${{ parameters.VSCODE_QUALITY }}
VSCODE_ARCH: ${{ parameters.VSCODE_ARCH }}
VSCODE_RUN_UNIT_TESTS: ${{ parameters.VSCODE_RUN_UNIT_TESTS }}
VSCODE_RUN_INTEGRATION_TESTS: ${{ parameters.VSCODE_RUN_INTEGRATION_TESTS }}
VSCODE_RUN_SMOKE_TESTS: ${{ parameters.VSCODE_RUN_SMOKE_TESTS }}
${{ if ne(parameters.VSCODE_QUALITY, 'oss') }}:
PUBLISH_TASK_NAME: 1ES.PublishPipelineArtifact@1
- ${{ if ne(parameters.VSCODE_CIBUILD, true) }}:
- ${{ if ne(parameters.VSCODE_QUALITY, 'oss') }}:
- task: DownloadPipelineArtifact@2
inputs:
artifact: unsigned_vscode_cli_win32_$(VSCODE_ARCH)_cli
patterns: "**"
path: $(Build.ArtifactStagingDirectory)/cli
displayName: Download VS Code CLI
- powershell: |
. build/azure-pipelines/win32/exec.ps1
$ErrorActionPreference = "Stop"
$ArtifactName = (gci -Path "$(Build.ArtifactStagingDirectory)/cli" | Select-Object -last 1).FullName
Expand-Archive -Path $ArtifactName -DestinationPath "$(Build.ArtifactStagingDirectory)/cli"
$AppProductJson = Get-Content -Raw -Path "$(Agent.BuildDirectory)\VSCode-win32-$(VSCODE_ARCH)\resources\app\product.json" | ConvertFrom-Json
$CliAppName = $AppProductJson.tunnelApplicationName
$AppName = $AppProductJson.applicationName
Move-Item -Path "$(Build.ArtifactStagingDirectory)/cli/$AppName.exe" -Destination "$(Agent.BuildDirectory)/VSCode-win32-$(VSCODE_ARCH)/bin/$CliAppName.exe"
displayName: Move VS Code CLI
- task: UseDotNet@2
inputs:
version: 6.x
- task: EsrpCodeSigning@5
inputs:
UseMSIAuthentication: true
ConnectedServiceName: vscode-esrp
AppRegistrationClientId: $(ESRP_CLIENT_ID)
AppRegistrationTenantId: $(ESRP_TENANT_ID)
AuthAKVName: vscode-esrp
AuthSignCertName: esrp-sign
FolderPath: .
Pattern: noop
displayName: 'Install ESRP Tooling'
- powershell: |
. build/azure-pipelines/win32/exec.ps1
$ErrorActionPreference = "Stop"
$EsrpCodeSigningTool = (gci -directory -filter EsrpCodeSigning_* $(Agent.RootDirectory)\_tasks | Select-Object -last 1).FullName
$Version = (gci -directory $EsrpCodeSigningTool | Select-Object -last 1).FullName
echo "##vso[task.setvariable variable=EsrpCliDllPath]$Version\net6.0\esrpcli.dll"
displayName: Find ESRP CLI
- powershell: node build\azure-pipelines\common\sign $env:EsrpCliDllPath sign-windows $(CodeSigningFolderPath) '*.dll,*.exe,*.node'
env:
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
displayName: Codesign executables and shared libraries
- powershell: node build\azure-pipelines\common\sign $env:EsrpCliDllPath sign-windows-appx $(CodeSigningFolderPath) '*.ps1'
env:
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
displayName: Codesign Powershell scripts
- ${{ if eq(parameters.VSCODE_QUALITY, 'insider') }}:
- powershell: node build\azure-pipelines\common\sign $env:EsrpCliDllPath sign-windows-appx $(CodeSigningFolderPath) '*.appx'
env:
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
displayName: Codesign context menu appx package
- ${{ if ne(parameters.VSCODE_QUALITY, 'oss') }}:
- powershell: |
. build/azure-pipelines/win32/exec.ps1
$ErrorActionPreference = "Stop"
$PackageJson = Get-Content -Raw -Path ..\VSCode-win32-$(VSCODE_ARCH)\resources\app\package.json | ConvertFrom-Json
$Version = $PackageJson.version
echo "##vso[task.setvariable variable=VSCODE_VERSION]$Version"
condition: succeededOrFailed()
displayName: Get product version
- powershell: |
. build/azure-pipelines/win32/exec.ps1
$ErrorActionPreference = "Stop"
$ArchivePath = ".build\win32-$(VSCODE_ARCH)\VSCode-win32-$(VSCODE_ARCH)-$(VSCODE_VERSION).zip"
New-Item -ItemType Directory -Path .build\win32-$(VSCODE_ARCH) -Force
exec { 7z.exe a -tzip $ArchivePath ..\VSCode-win32-$(VSCODE_ARCH)\* "-xr!CodeSignSummary*.md" }
echo "##vso[task.setvariable variable=CLIENT_PATH]$ArchivePath"
echo "Listing archive contents"
7z.exe l $ArchivePath
condition: and(succeededOrFailed(), eq(variables['BUILT_CLIENT'], 'true'))
displayName: Package client
- powershell: |
. build/azure-pipelines/win32/exec.ps1
$ErrorActionPreference = "Stop"
$ArchivePath = ".build\win32-$(VSCODE_ARCH)\vscode-server-win32-$(VSCODE_ARCH).zip"
New-Item -ItemType Directory -Path .build\win32-$(VSCODE_ARCH) -Force
exec { 7z.exe a -tzip $ArchivePath ..\vscode-server-win32-$(VSCODE_ARCH) }
echo "##vso[task.setvariable variable=SERVER_PATH]$ArchivePath"
echo "Listing archive contents"
7z.exe l $ArchivePath
condition: and(succeededOrFailed(), eq(variables['BUILT_SERVER'], 'true'))
displayName: Package server
- powershell: |
. build/azure-pipelines/win32/exec.ps1
$ErrorActionPreference = "Stop"
$ArchivePath = ".build\win32-$(VSCODE_ARCH)\vscode-server-win32-$(VSCODE_ARCH)-web.zip"
New-Item -ItemType Directory -Path .build\win32-$(VSCODE_ARCH) -Force
exec { 7z.exe a -tzip $ArchivePath ..\vscode-server-win32-$(VSCODE_ARCH)-web }
echo "##vso[task.setvariable variable=WEB_PATH]$ArchivePath"
echo "Listing archive contents"
7z.exe l $ArchivePath
condition: and(succeededOrFailed(), eq(variables['BUILT_WEB'], 'true'))
displayName: Package server (web)
- powershell: |
. build/azure-pipelines/win32/exec.ps1
$ErrorActionPreference = "Stop"
exec { npm exec -- npm-run-all -lp "gulp vscode-win32-$(VSCODE_ARCH)-system-setup -- --sign" "gulp vscode-win32-$(VSCODE_ARCH)-user-setup -- --sign" }
$SystemSetupPath = ".build\win32-$(VSCODE_ARCH)\system-setup\VSCodeSetup-$(VSCODE_ARCH)-$(VSCODE_VERSION).exe"
$UserSetupPath = ".build\win32-$(VSCODE_ARCH)\user-setup\VSCodeUserSetup-$(VSCODE_ARCH)-$(VSCODE_VERSION).exe"
mv .build\win32-$(VSCODE_ARCH)\system-setup\VSCodeSetup.exe $SystemSetupPath
mv .build\win32-$(VSCODE_ARCH)\user-setup\VSCodeSetup.exe $UserSetupPath
echo "##vso[task.setvariable variable=SYSTEM_SETUP_PATH]$SystemSetupPath"
echo "##vso[task.setvariable variable=USER_SETUP_PATH]$UserSetupPath"
env:
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
displayName: Build setup packages (system, user)
- powershell: echo "##vso[task.setvariable variable=ARTIFACT_PREFIX]attempt$(System.JobAttempt)_"
condition: and(succeededOrFailed(), notIn(variables['Agent.JobStatus'], 'Succeeded', 'SucceededWithIssues'))
displayName: Generate artifact prefix
- task: 1ES.PublishPipelineArtifact@1
inputs:
targetPath: $(CLIENT_PATH)
artifactName: $(ARTIFACT_PREFIX)vscode_client_win32_$(VSCODE_ARCH)_archive
sbomBuildDropPath: $(Agent.BuildDirectory)/VSCode-win32-$(VSCODE_ARCH)
sbomPackageName: "VS Code Windows $(VSCODE_ARCH)"
sbomPackageVersion: $(Build.SourceVersion)
condition: and(succeededOrFailed(), ne(variables['CLIENT_PATH'], ''))
displayName: Publish archive
- task: 1ES.PublishPipelineArtifact@1
inputs:
targetPath: $(SERVER_PATH)
artifactName: $(ARTIFACT_PREFIX)vscode_server_win32_$(VSCODE_ARCH)_archive
sbomBuildDropPath: $(Agent.BuildDirectory)/vscode-server-win32-$(VSCODE_ARCH)
sbomPackageName: "VS Code Windows $(VSCODE_ARCH) Server"
sbomPackageVersion: $(Build.SourceVersion)
condition: and(succeededOrFailed(), ne(variables['SERVER_PATH'], ''))
displayName: Publish server archive
- task: 1ES.PublishPipelineArtifact@1
inputs:
targetPath: $(WEB_PATH)
artifactName: $(ARTIFACT_PREFIX)vscode_web_win32_$(VSCODE_ARCH)_archive
sbomBuildDropPath: $(Agent.BuildDirectory)/vscode-server-win32-$(VSCODE_ARCH)-web
sbomPackageName: "VS Code Windows $(VSCODE_ARCH) Web"
sbomPackageVersion: $(Build.SourceVersion)
condition: and(succeededOrFailed(), ne(variables['WEB_PATH'], ''))
displayName: Publish web server archive
- task: 1ES.PublishPipelineArtifact@1
inputs:
targetPath: $(SYSTEM_SETUP_PATH)
artifactName: $(ARTIFACT_PREFIX)vscode_client_win32_$(VSCODE_ARCH)_setup
sbomBuildDropPath: $(Agent.BuildDirectory)/VSCode-win32-$(VSCODE_ARCH)
sbomPackageName: "VS Code Windows $(VSCODE_ARCH) System Setup"
sbomPackageVersion: $(Build.SourceVersion)
condition: and(succeededOrFailed(), ne(variables['SYSTEM_SETUP_PATH'], ''))
displayName: Publish system setup
- task: 1ES.PublishPipelineArtifact@1
inputs:
targetPath: $(USER_SETUP_PATH)
artifactName: $(ARTIFACT_PREFIX)vscode_client_win32_$(VSCODE_ARCH)_user-setup
sbomBuildDropPath: $(Agent.BuildDirectory)/VSCode-win32-$(VSCODE_ARCH)
sbomPackageName: "VS Code Windows $(VSCODE_ARCH) User Setup"
sbomPackageVersion: $(Build.SourceVersion)
condition: and(succeededOrFailed(), ne(variables['USER_SETUP_PATH'], ''))
displayName: Publish user setup
================================================
FILE: build/azure-pipelines/win32/retry.ps1
================================================
function Retry
{
[CmdletBinding()]
param(
[Parameter(Position=0,Mandatory=1)][scriptblock]$cmd
)
$retry = 0
while ($retry++ -lt 5) {
try {
& $cmd
return
} catch {
# noop
}
}
throw "Max retries reached"
}
================================================
FILE: build/azure-pipelines/win32/sdl-scan-win32.yml
================================================
parameters:
- name: VSCODE_ARCH
type: string
- name: VSCODE_QUALITY
type: string
steps:
- task: NodeTool@0
inputs:
versionSource: fromFile
versionFilePath: .nvmrc
nodejsMirror: https://github.com/joaomoreno/node-mirror/releases/download
- task: UsePythonVersion@0
inputs:
versionSpec: "3.x"
addToPath: true
- template: ../distro/download-distro.yml@self
- task: AzureKeyVault@2
displayName: "Azure Key Vault: Get Secrets"
inputs:
azureSubscription: vscode
KeyVaultName: vscode-build-secrets
SecretsFilter: "github-distro-mixin-password"
- powershell: node build/setup-npm-registry.js $env:NPM_REGISTRY
condition: and(succeeded(), ne(variables['NPM_REGISTRY'], 'none'))
displayName: Setup NPM Registry
- powershell: |
. build/azure-pipelines/win32/exec.ps1
$ErrorActionPreference = "Stop"
# Set the private NPM registry to the global npmrc file
# so that authentication works for subfolders like build/, remote/, extensions/ etc
# which does not have their own .npmrc file
exec { npm config set registry "$env:NPM_REGISTRY" }
$NpmrcPath = (npm config get userconfig)
echo "##vso[task.setvariable variable=NPMRC_PATH]$NpmrcPath"
condition: and(succeeded(), ne(variables['NPM_REGISTRY'], 'none'))
displayName: Setup NPM
- task: npmAuthenticate@0
inputs:
workingFile: $(NPMRC_PATH)
condition: and(succeeded(), ne(variables['NPM_REGISTRY'], 'none'))
displayName: Setup NPM Authentication
- pwsh: |
$includes = @'
{
'target_defaults': {
'conditions': [
['OS=="win"', {
'msvs_settings': {
'VCCLCompilerTool': {
'AdditionalOptions': [
'/Zi',
'/FS'
],
},
'VCLinkerTool': {
'AdditionalOptions': [
'/profile'
]
}
}
}]
]
}
}
'@
if (!(Test-Path "~/.gyp")) {
mkdir "~/.gyp"
}
echo $includes > "~/.gyp/include.gypi"
displayName: Create include.gypi
- powershell: |
. build/azure-pipelines/win32/exec.ps1
$ErrorActionPreference = "Stop"
exec { npm ci }
env:
npm_config_arch: ${{ parameters.VSCODE_ARCH }}
npm_config_foreground_scripts: "true"
ELECTRON_SKIP_BINARY_DOWNLOAD: 1
PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD: 1
GITHUB_TOKEN: "$(github-distro-mixin-password)"
retryCountOnTaskFailure: 5
displayName: Install dependencies
- script: node build/azure-pipelines/distro/mixin-npm
displayName: Mixin distro node modules
- script: node build/azure-pipelines/distro/mixin-quality
displayName: Mixin distro quality
env:
VSCODE_QUALITY: ${{ parameters.VSCODE_QUALITY }}
- powershell: npm run compile
displayName: Compile
- powershell: npm run gulp "vscode-symbols-win32-${{ parameters.VSCODE_ARCH }}"
env:
GITHUB_TOKEN: "$(github-distro-mixin-password)"
displayName: Download Symbols
- powershell: |
Get-ChildItem '$(Agent.BuildDirectory)\scanbin' -Recurse -Filter "*.exe"
Get-ChildItem '$(Agent.BuildDirectory)\scanbin' -Recurse -Filter "*.dll"
Get-ChildItem '$(Agent.BuildDirectory)\scanbin' -Recurse -Filter "*.node"
Get-ChildItem '$(Agent.BuildDirectory)\scanbin' -Recurse -Filter "*.pdb"
displayName: List files
- task: CopyFiles@2
displayName: 'Collect Symbols for API Scan'
inputs:
SourceFolder: $(Agent.BuildDirectory)
Contents: 'scanbin\**\*.pdb'
TargetFolder: '$(Agent.BuildDirectory)\symbols'
flattenFolders: true
condition: succeeded()
- task: APIScan@2
inputs:
softwareFolder: $(Agent.BuildDirectory)\scanbin
softwareName: 'vscode-client'
softwareVersionNum: '1'
symbolsFolder: 'srv*https://symweb.azurefd.net;$(Agent.BuildDirectory)\symbols'
isLargeApp: false
toolVersion: 'Latest'
azureSubscription: 'vscode-apiscan'
displayName: Run ApiScan
condition: succeeded()
env:
AzureServicesAuthConnectionString: RunAs=App;AppId=c0940da5-8bd3-4dd3-8af1-40774b50edbd;TenantId=72f988bf-86f1-41af-91ab-2d7cd011db47;ServiceConnectionId=3e55d992-b60d-414d-9071-e4fad359c748;
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
- task: PublishSecurityAnalysisLogs@3
inputs:
ArtifactName: CodeAnalysisLogs
ArtifactType: Container
PublishProcessedResults: false
AllTools: true
# TSA Upload
- task: securedevelopmentteam.vss-secure-development-tools.build-task-uploadtotsa.TSAUpload@2
displayName: TSA Upload
continueOnError: true
inputs:
GdnPublishTsaOnboard: true
GdnPublishTsaConfigFile: '$(Build.SourcesDirectory)/build/azure-pipelines/config/tsaoptions.json'
================================================
FILE: build/buildfile.js
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
/**
* @param {string} name
* @returns {import('./lib/bundle').IEntryPoint}
*/
function createModuleDescription(name) {
return {
name
};
}
exports.workerEditor = createModuleDescription('vs/editor/common/services/editorWebWorkerMain');
exports.workerExtensionHost = createModuleDescription('vs/workbench/api/worker/extensionHostWorkerMain');
exports.workerNotebook = createModuleDescription('vs/workbench/contrib/notebook/common/services/notebookWebWorkerMain');
exports.workerLanguageDetection = createModuleDescription('vs/workbench/services/languageDetection/browser/languageDetectionWebWorkerMain');
exports.workerLocalFileSearch = createModuleDescription('vs/workbench/services/search/worker/localFileSearchMain');
exports.workerProfileAnalysis = createModuleDescription('vs/platform/profiling/electron-sandbox/profileAnalysisWorkerMain');
exports.workerOutputLinks = createModuleDescription('vs/workbench/contrib/output/common/outputLinkComputerMain');
exports.workerBackgroundTokenization = createModuleDescription('vs/workbench/services/textMate/browser/backgroundTokenization/worker/textMateTokenizationWorker.workerMain');
exports.workbenchDesktop = [
createModuleDescription('vs/workbench/contrib/debug/node/telemetryApp'),
createModuleDescription('vs/platform/files/node/watcher/watcherMain'),
createModuleDescription('vs/platform/terminal/node/ptyHostMain'),
createModuleDescription('vs/workbench/api/node/extensionHostProcess'),
createModuleDescription('vs/workbench/workbench.desktop.main')
];
exports.workbenchWeb = createModuleDescription('vs/workbench/workbench.web.main');
exports.keyboardMaps = [
createModuleDescription('vs/workbench/services/keybinding/browser/keyboardLayouts/layout.contribution.linux'),
createModuleDescription('vs/workbench/services/keybinding/browser/keyboardLayouts/layout.contribution.darwin'),
createModuleDescription('vs/workbench/services/keybinding/browser/keyboardLayouts/layout.contribution.win')
];
exports.code = [
// 'vs/code/electron-main/main' is not included here because it comes in via ./src/main.js
// 'vs/code/node/cli' is not included here because it comes in via ./src/cli.js
createModuleDescription('vs/code/node/cliProcessMain'),
createModuleDescription('vs/code/electron-utility/sharedProcess/sharedProcessMain'),
createModuleDescription('vs/code/electron-sandbox/processExplorer/processExplorerMain'),
createModuleDescription('vs/code/electron-sandbox/workbench/workbench'),
createModuleDescription('vs/code/electron-sandbox/processExplorer/processExplorer')
];
exports.codeWeb = createModuleDescription('vs/code/browser/workbench/workbench');
exports.codeServer = [
// 'vs/server/node/server.main' is not included here because it gets inlined via ./src/server-main.js
// 'vs/server/node/server.cli' is not included here because it gets inlined via ./src/server-cli.js
createModuleDescription('vs/workbench/api/node/extensionHostProcess'),
createModuleDescription('vs/platform/files/node/watcher/watcherMain'),
createModuleDescription('vs/platform/terminal/node/ptyHostMain')
];
exports.entrypoint = createModuleDescription;
================================================
FILE: build/builtin/.eslintrc
================================================
{
"env": {
"node": true,
"es6": true,
"browser": true
},
"rules": {
"no-console": 0,
"no-cond-assign": 0,
"no-unused-vars": 1,
"no-extra-semi": "warn",
"semi": "warn"
},
"extends": "eslint:recommended",
"parserOptions": {
"ecmaFeatures": {
"experimentalObjectRestSpread": true
}
}
}
================================================
FILE: build/builtin/browser-main.js
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
//@ts-check
const fs = require('fs');
const path = require('path');
const os = require('os');
const { ipcRenderer } = require('electron');
const builtInExtensionsPath = path.join(__dirname, '..', '..', 'product.json');
const controlFilePath = path.join(os.homedir(), '.vscode-oss-dev', 'extensions', 'control.json');
/**
* @param {string} filePath
*/
function readJson(filePath) {
return JSON.parse(fs.readFileSync(filePath, { encoding: 'utf8' }));
}
/**
* @param {string} filePath
* @param {any} obj
*/
function writeJson(filePath, obj) {
fs.writeFileSync(filePath, JSON.stringify(obj, null, 2));
}
/**
* @param {HTMLFormElement} form
* @param {string} id
* @param {string} title
* @param {string} value
* @param {boolean} checked
*/
function renderOption(form, id, title, value, checked) {
const input = document.createElement('input');
input.type = 'radio';
input.id = id;
input.name = 'choice';
input.value = value;
input.checked = !!checked;
form.appendChild(input);
const label = document.createElement('label');
label.setAttribute('for', id);
label.textContent = title;
form.appendChild(label);
return input;
}
/**
* @param {HTMLElement} el
* @param {any} state
*/
function render(el, state) {
/**
* @param {any} state
*/
function setState(state) {
try {
writeJson(controlFilePath, state.control);
} catch (err) {
console.error(err);
}
el.innerHTML = '';
render(el, state);
}
const ul = document.createElement('ul');
const { builtin, control } = state;
for (const ext of builtin) {
const controlState = control[ext.name] || 'marketplace';
const li = document.createElement('li');
ul.appendChild(li);
const name = document.createElement('code');
name.textContent = ext.name;
li.appendChild(name);
const form = document.createElement('form');
li.appendChild(form);
const marketplaceInput = renderOption(form, `marketplace-${ext.name}`, 'Marketplace', 'marketplace', controlState === 'marketplace');
marketplaceInput.onchange = function () {
control[ext.name] = 'marketplace';
setState({ builtin, control });
};
const disabledInput = renderOption(form, `disabled-${ext.name}`, 'Disabled', 'disabled', controlState === 'disabled');
disabledInput.onchange = function () {
control[ext.name] = 'disabled';
setState({ builtin, control });
};
let local = undefined;
if (controlState !== 'marketplace' && controlState !== 'disabled') {
local = controlState;
}
const localInput = renderOption(form, `local-${ext.name}`, 'Local', 'local', !!local);
localInput.onchange = async function () {
const result = await ipcRenderer.invoke('pickdir');
if (result) {
control[ext.name] = result;
setState({ builtin, control });
}
};
if (local) {
const localSpan = document.createElement('code');
localSpan.className = 'local';
localSpan.textContent = local;
form.appendChild(localSpan);
}
}
el.appendChild(ul);
}
function main() {
const el = document.getElementById('extensions');
const builtin = readJson(builtInExtensionsPath).builtInExtensions;
let control;
try {
control = readJson(controlFilePath);
} catch (err) {
control = {};
}
if (el) {
render(el, { builtin, control });
}
}
window.onload = main;
================================================
FILE: build/builtin/index.html
================================================
Manage Built-in Extensions
Built-in Extensions
================================================
FILE: build/builtin/main.js
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
// @ts-check
const { app, BrowserWindow, ipcMain, dialog } = require('electron');
const url = require('url');
const path = require('path');
let window = null;
ipcMain.handle('pickdir', async () => {
const result = await dialog.showOpenDialog(window, {
title: 'Choose Folder',
properties: ['openDirectory']
});
if (result.canceled || result.filePaths.length < 1) {
return undefined;
}
return result.filePaths[0];
});
app.once('ready', () => {
window = new BrowserWindow({
width: 800,
height: 600,
webPreferences: {
nodeIntegration: true,
contextIsolation: false,
enableWebSQL: false
}
});
window.setMenuBarVisibility(false);
window.loadURL(url.format({ pathname: path.join(__dirname, 'index.html'), protocol: 'file:', slashes: true }));
// window.webContents.openDevTools();
window.once('closed', () => window = null);
});
app.on('window-all-closed', () => app.quit());
================================================
FILE: build/builtin/package.json
================================================
{
"name": "builtin",
"version": "0.1.0",
"main": "main.js"
}
================================================
FILE: build/checksums/electron.txt
================================================
c9b82c9f381742e839fea00aeb14f24519bcaf38a0f4eed25532191701f9535b *chromedriver-v34.3.2-darwin-arm64.zip
d556c1e2b06f1bf131e83c2fb981de755c28e1083a884d257eb964815be16b0c *chromedriver-v34.3.2-darwin-x64.zip
1cabad4f3303ac2ff172a9f22185f64944dbaa6fc68271609077158eaefdee35 *chromedriver-v34.3.2-linux-arm64.zip
4213ce52c72ef414179b5c5c22ae8423847ff030d438296bd6c2aac763930a7b *chromedriver-v34.3.2-linux-armv7l.zip
3c64c08221fdfc0f4be60ea8b1b126f2ecca45f60001b63778522f711022c6ea *chromedriver-v34.3.2-linux-x64.zip
e8388734d88e011cb6cd79795431de9206820749219d80565ee49d90501d2bf3 *chromedriver-v34.3.2-mas-arm64.zip
3ad1dd37bd6e0bb37e8503898db7aedd56bd5213e6d6760b05c3d11f4625062b *chromedriver-v34.3.2-mas-x64.zip
d567b481a0f5d88e84bba7718f89fb08f56363bfc4cb5914e1c2086358a5c252 *chromedriver-v34.3.2-win32-arm64.zip
df6732e9dc61cb20a3c0b2a2de453aac7e2bd54e7cbff43512afa614852c15fa *chromedriver-v34.3.2-win32-ia32.zip
dda0765c8d064924632e18cd152014ecd767f3808fc51c8249a053bfb7ca70a2 *chromedriver-v34.3.2-win32-x64.zip
1945f15caff98f2e0f1ee539c483d352fb8d4d0c13f342caa7abe247676d828c *electron-api.json
c078bbf727b3c3026f60e07a0f4643b85c06c581b54be017d0a6c284ba6772d3 *electron-v34.3.2-darwin-arm64-dsym-snapshot.zip
35f587754d6a3272606258386bf73688d63dd53c7e572d3a7cbaae6f3f60bdae *electron-v34.3.2-darwin-arm64-dsym.zip
08b14ee02c98353de3c738120dfd017322666e82b914a7f6de9b9888dcc5c0f0 *electron-v34.3.2-darwin-arm64-symbols.zip
2a4aa7e8fa30f229e465ebd18d3e4722e2b41529dc51a68a954d333a7e556ffe *electron-v34.3.2-darwin-arm64.zip
1509ccdeb80024f5e3edd5ecf804b4cef4e47ea2bd74e33ef0b39044b0ccf892 *electron-v34.3.2-darwin-x64-dsym-snapshot.zip
3bbe5d587c3f582ed8c126b0fb635cc02ad9a14d077b04892fe6f862092445b0 *electron-v34.3.2-darwin-x64-dsym.zip
fa7ece82e6ecaf1c94ed341e8ebff98e64687c68fe113f52cd9a21400302e22f *electron-v34.3.2-darwin-x64-symbols.zip
23938c62257a65a863ed7aa7c7966ba5f257a7d3dc16c78293e826962cc39c5c *electron-v34.3.2-darwin-x64.zip
0547eecf8ab538d74fa854f591ce8b888a3dbb339256d2db3038e7bb2c6dd929 *electron-v34.3.2-linux-arm64-debug.zip
676d0dc2b1c1c85c8b2abbb8cd5376ee22ecdb910493b910d9ae5a998532136a *electron-v34.3.2-linux-arm64-symbols.zip
774e4ccb39d553e5487994a9f8c60774a90f08cdb049ff65f3963fc27c969ff2 *electron-v34.3.2-linux-arm64.zip
0547eecf8ab538d74fa854f591ce8b888a3dbb339256d2db3038e7bb2c6dd929 *electron-v34.3.2-linux-armv7l-debug.zip
ba33bf53fcb35dea568a2795f5b23ecf46c218abe8258946611c72a1f42f716c *electron-v34.3.2-linux-armv7l-symbols.zip
73ae92c8fffb351d3a455569cf57ce9a3f676f42bf61939c613c607fe0fc3bfb *electron-v34.3.2-linux-armv7l.zip
e61a9a69dd7ea6f2687708a8e83516670cdea53c728226e598e2f6f1fad5b77b *electron-v34.3.2-linux-x64-debug.zip
f1a04df7fe67dd1cd29e7b87871525458d2eb24c0cf3b5835a1c56974707562a *electron-v34.3.2-linux-x64-symbols.zip
7b74c0c4fae82e27c7e9cbca13e9763e046113dba8737d3e27de9a0b300ac87e *electron-v34.3.2-linux-x64.zip
8571a6aa83e00925ceb39fdc5a45a9f6b9aa3d92fd84951c6f252ed723aea4ae *electron-v34.3.2-mas-arm64-dsym-snapshot.zip
477410c6f9a6c5eeaedf376058a02c2996fc0a334aa40eeec7d3734c09347f4d *electron-v34.3.2-mas-arm64-dsym.zip
c2e62dcd6630cb51b2d8e2869e74e47d29bda785521cea6e82e546d0fc58aabb *electron-v34.3.2-mas-arm64-symbols.zip
a1698e8546a062fd59b7f8e5507a7f3220fb00b347f2377de83fc9a07f7f3507 *electron-v34.3.2-mas-arm64.zip
741a24ac230a3651dca81d211f9f00b835c428a5ed0c5f67d370d4e88b62f8d6 *electron-v34.3.2-mas-x64-dsym-snapshot.zip
aeff97ec9e5c9e173ac89e38acd94476025c5640d5f27be1e8c2abd54398bab3 *electron-v34.3.2-mas-x64-dsym.zip
9f14b66b1d612ac66697288e8763171c388f7f200854871a5f0ab464a6a921c2 *electron-v34.3.2-mas-x64-symbols.zip
c979d7e7175f1e8e03ca187997d4c156b878189fc3611b347fadebcb16f3e027 *electron-v34.3.2-mas-x64.zip
f43c700641e8220205dd356952e32718d113cf530520c4ed7209b59851eac266 *electron-v34.3.2-win32-arm64-pdb.zip
3ba6e01c99bffac6b5dd3fd6f122ecdb571cf6f675dc5498c65050bd7a382ef8 *electron-v34.3.2-win32-arm64-symbols.zip
c23f84aabb09c24cd2ae759a547fdba4206af19a3bb0f4554a91cd9528648ad0 *electron-v34.3.2-win32-arm64-toolchain-profile.zip
9b9cb65d75a16782088b492f9ef3bb4d27525012b819c12bf29bd27e159d749b *electron-v34.3.2-win32-arm64.zip
1006e7af4c149114b5ebc3497617aaa6cd1bb0b131e0a225fd73709ff308f9c5 *electron-v34.3.2-win32-ia32-pdb.zip
1ecb6430cd04454f08f557c9579163f3552144bfcc0b67b768dad8868b5b891d *electron-v34.3.2-win32-ia32-symbols.zip
c23f84aabb09c24cd2ae759a547fdba4206af19a3bb0f4554a91cd9528648ad0 *electron-v34.3.2-win32-ia32-toolchain-profile.zip
d004fd5f853754001fafaec33e383d1950b30c935ee71b297ec1c9e084355e9b *electron-v34.3.2-win32-ia32.zip
4e0721552fd2f09e9466e88089af8b965f1bfbc4ae00a59aaf6245b1d1efabfd *electron-v34.3.2-win32-x64-pdb.zip
9dea812a7e7cd0fb18e5fed9a99db5531959a068c24d3c0ecedceb644cd3ffa0 *electron-v34.3.2-win32-x64-symbols.zip
c23f84aabb09c24cd2ae759a547fdba4206af19a3bb0f4554a91cd9528648ad0 *electron-v34.3.2-win32-x64-toolchain-profile.zip
1785e161420fb90d2331c26e50bba3413cae9625b7db3c8524ea02ade631efba *electron-v34.3.2-win32-x64.zip
722b304c31ddac58b0083d94a241c5276464f04bd8ea4fcbfd33051d197be103 *electron.d.ts
31ce159b2e47d1de5bc907d8e1c89726b0f2ba530ec2e9d7a8e5c723b1ccf6e0 *ffmpeg-v34.3.2-darwin-arm64.zip
565539bac64a6ee9cf6f188070f520210a1507341718f5dc388ac7c454b1e1d5 *ffmpeg-v34.3.2-darwin-x64.zip
6006ea0f46ab229feb2685be086b0fafd65981e2939dd2218a078459c75ab527 *ffmpeg-v34.3.2-linux-arm64.zip
9404ce2e85df7c40f809f2cf62c7af607de299839fe6b7ae978c3015500abcc8 *ffmpeg-v34.3.2-linux-armv7l.zip
79aec96898b7e2462826780ee0b52b9ab299dc662af333e128a34fd5ddae87f1 *ffmpeg-v34.3.2-linux-x64.zip
9190743c78210574faf5d5ecb82a00f8fa15e5f2253378cb925a99ca9d39961b *ffmpeg-v34.3.2-mas-arm64.zip
48915adcb1a6342efeda896035101300f0432c0304cfb38f2378e98c6309ebae *ffmpeg-v34.3.2-mas-x64.zip
745d5ef786de6d4a720475079836e2fda7b501cfcd255819485a47de5b24b74e *ffmpeg-v34.3.2-win32-arm64.zip
d0d86d60978439dc8ae4a723d4e4c1f853891d596bfd84033440a232fa762e2f *ffmpeg-v34.3.2-win32-ia32.zip
4441539fd8c9cbe79880ff1bade9bdc0c3744c33d7409130af6404e57ee401ff *ffmpeg-v34.3.2-win32-x64.zip
39edd1eeefe881aa75af0e438204e0b1c6e6724e34fa5819109276331c0c2c9a *hunspell_dictionaries.zip
20dd417536e5f4ebc01f480221284c0673729c27b082bc04e2922f16cd571719 *libcxx-objects-v34.3.2-linux-arm64.zip
7e53c5779c04f895f8282c0450ec4a63074d15a0e910e41006cfea133d0288af *libcxx-objects-v34.3.2-linux-armv7l.zip
92e2283c924ab523ffec3ea22513beaab6417f7fc3d570f57d51a1e1ceb7f510 *libcxx-objects-v34.3.2-linux-x64.zip
9bf3c6e8ad68f523fe086fada4148dd04e5bb0b9290d104873e66f2584a5cf50 *libcxx_headers.zip
34e4b44f9c5e08b557a2caed55456ce7690abab910196a783a2a47b58d2b9ac9 *libcxxabi_headers.zip
11f67635e6659f9188198e4086c51b89890b61a22f6c17c99eff35595ee8f51d *mksnapshot-v34.3.2-darwin-arm64.zip
c0add9ef4ac27c73fa043d04b4c9635fd3fd9f5c16d7a03e961864ba05252813 *mksnapshot-v34.3.2-darwin-x64.zip
6262adf86a340d8d28059937b01ef1117b93212e945fddbceea5c18d7e7c64f0 *mksnapshot-v34.3.2-linux-arm64-x64.zip
f7db8ebe91a1cc8d24ef6aad12949a18d8e4975ac296e3e5e9ecd88c9bccb143 *mksnapshot-v34.3.2-linux-armv7l-x64.zip
6642038e86bda362980ff1c8973a404e2b02efdd87de9e35b650fc1e743833da *mksnapshot-v34.3.2-linux-x64.zip
15883bf8e8cd737c3682d1e719d7cbac92f50b525681aac324dca876861dfc7d *mksnapshot-v34.3.2-mas-arm64.zip
4da23a950bfcc377ef21c37d496017ab4c36da03f3b41049ac114042c42608ce *mksnapshot-v34.3.2-mas-x64.zip
fab59573d3c2f9bdf31146a1896d24ac0c51f736aad86d2f3c7ecef13c05a7fd *mksnapshot-v34.3.2-win32-arm64-x64.zip
66f25e07c6f8d5d2009577a129440255a3baf63c929a5b60b2e77cd52e46105b *mksnapshot-v34.3.2-win32-ia32.zip
8168bfbf61882cfac80aed1e71e364e1c7f2fccd11eac298e6abade8b46894ea *mksnapshot-v34.3.2-win32-x64.zip
================================================
FILE: build/checksums/nodejs.txt
================================================
1f15b7ed18a580af31cf32bc126572292d820f547bf55bf9cdce08041a24e1d9 node-v20.18.3-darwin-arm64.tar.gz
ba668f64df9239843fefcef095ee539f5ac5aa1b0fc15a71f1ecca16abedec7a node-v20.18.3-darwin-x64.tar.gz
93a9df19238adfaa289f4784041d03edaf2fdd89fbb247faffca2fe4a1000703 node-v20.18.3-linux-arm64.tar.gz
8a84eb34287db6a273066934d7195e429f57b91686b62fc19497210204a2b3de node-v20.18.3-linux-armv7l.tar.gz
9fc3952da39b20d1fcfdb777b198cc035485afbbb1004b4df93f35245d61151e node-v20.18.3-linux-x64.tar.gz
4258e333f4b95060681d61bffa762542a8068547d3dffebe57c575b38d380dda win-arm64/node.exe
528a9aa64888a2a3ba71c6aea89434dd5ab5cb3caa9f0f31345cf5facf685ab0 win-x64/node.exe
================================================
FILE: build/checksums/vscode-sysroot.txt
================================================
0de422a81683cf9e8cf875dbd1e0c27545ac3c775b2d53015daf3ca2b31d3f15 aarch64-linux-gnu-glibc-2.28.tar.gz
7aea163f7fad8cc50000c86b5108be880121d35e2f55d016ef8c96bbe54129eb arm-rpi-linux-gnueabihf-glibc-2.28.tar.gz
dbb927408393041664a020661f2641c9785741be3d29b050b9dac58980967784 x86_64-linux-gnu-glibc-2.28.tar.gz
================================================
FILE: build/darwin/create-universal-app.js
================================================
"use strict";
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
Object.defineProperty(exports, "__esModule", { value: true });
const path_1 = __importDefault(require("path"));
const fs_1 = __importDefault(require("fs"));
const minimatch_1 = __importDefault(require("minimatch"));
const vscode_universal_bundler_1 = require("vscode-universal-bundler");
const root = path_1.default.dirname(path_1.default.dirname(__dirname));
async function main(buildDir) {
const arch = process.env['VSCODE_ARCH'];
if (!buildDir) {
throw new Error('Build dir not provided');
}
const product = JSON.parse(fs_1.default.readFileSync(path_1.default.join(root, 'product.json'), 'utf8'));
const appName = product.nameLong + '.app';
const x64AppPath = path_1.default.join(buildDir, 'VSCode-darwin-x64', appName);
const arm64AppPath = path_1.default.join(buildDir, 'VSCode-darwin-arm64', appName);
const asarRelativePath = path_1.default.join('Contents', 'Resources', 'app', 'node_modules.asar');
const outAppPath = path_1.default.join(buildDir, `VSCode-darwin-${arch}`, appName);
const productJsonPath = path_1.default.resolve(outAppPath, 'Contents', 'Resources', 'app', 'product.json');
const filesToSkip = [
'**/CodeResources',
'**/Credits.rtf',
'**/policies/{*.mobileconfig,**/*.plist}',
// TODO: Should we consider expanding this to other files in this area?
'**/node_modules/@parcel/node-addon-api/nothing.target.mk'
];
await (0, vscode_universal_bundler_1.makeUniversalApp)({
x64AppPath,
arm64AppPath,
asarPath: asarRelativePath,
outAppPath,
force: true,
mergeASARs: true,
x64ArchFiles: '*/kerberos.node',
filesToSkipComparison: (file) => {
for (const expected of filesToSkip) {
if ((0, minimatch_1.default)(file, expected)) {
return true;
}
}
return false;
}
});
const productJson = JSON.parse(fs_1.default.readFileSync(productJsonPath, 'utf8'));
Object.assign(productJson, {
darwinUniversalAssetId: 'darwin-universal'
});
fs_1.default.writeFileSync(productJsonPath, JSON.stringify(productJson, null, '\t'));
}
if (require.main === module) {
main(process.argv[2]).catch(err => {
console.error(err);
process.exit(1);
});
}
//# sourceMappingURL=create-universal-app.js.map
================================================
FILE: build/darwin/create-universal-app.ts
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
import path from 'path';
import fs from 'fs';
import minimatch from 'minimatch';
import { makeUniversalApp } from 'vscode-universal-bundler';
const root = path.dirname(path.dirname(__dirname));
async function main(buildDir?: string) {
const arch = process.env['VSCODE_ARCH'];
if (!buildDir) {
throw new Error('Build dir not provided');
}
const product = JSON.parse(fs.readFileSync(path.join(root, 'product.json'), 'utf8'));
const appName = product.nameLong + '.app';
const x64AppPath = path.join(buildDir, 'VSCode-darwin-x64', appName);
const arm64AppPath = path.join(buildDir, 'VSCode-darwin-arm64', appName);
const asarRelativePath = path.join('Contents', 'Resources', 'app', 'node_modules.asar');
const outAppPath = path.join(buildDir, `VSCode-darwin-${arch}`, appName);
const productJsonPath = path.resolve(outAppPath, 'Contents', 'Resources', 'app', 'product.json');
const filesToSkip = [
'**/CodeResources',
'**/Credits.rtf',
'**/policies/{*.mobileconfig,**/*.plist}',
// TODO: Should we consider expanding this to other files in this area?
'**/node_modules/@parcel/node-addon-api/nothing.target.mk'
];
await makeUniversalApp({
x64AppPath,
arm64AppPath,
asarPath: asarRelativePath,
outAppPath,
force: true,
mergeASARs: true,
x64ArchFiles: '*/kerberos.node',
filesToSkipComparison: (file: string) => {
for (const expected of filesToSkip) {
if (minimatch(file, expected)) {
return true;
}
}
return false;
}
});
const productJson = JSON.parse(fs.readFileSync(productJsonPath, 'utf8'));
Object.assign(productJson, {
darwinUniversalAssetId: 'darwin-universal'
});
fs.writeFileSync(productJsonPath, JSON.stringify(productJson, null, '\t'));
}
if (require.main === module) {
main(process.argv[2]).catch(err => {
console.error(err);
process.exit(1);
});
}
================================================
FILE: build/darwin/sign.js
================================================
"use strict";
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
Object.defineProperty(exports, "__esModule", { value: true });
const fs_1 = __importDefault(require("fs"));
const path_1 = __importDefault(require("path"));
const electron_osx_sign_1 = __importDefault(require("electron-osx-sign"));
const cross_spawn_promise_1 = require("@malept/cross-spawn-promise");
const root = path_1.default.dirname(path_1.default.dirname(__dirname));
function getElectronVersion() {
const npmrc = fs_1.default.readFileSync(path_1.default.join(root, '.npmrc'), 'utf8');
const target = /^target="(.*)"$/m.exec(npmrc)[1];
return target;
}
async function main(buildDir) {
const tempDir = process.env['AGENT_TEMPDIRECTORY'];
const arch = process.env['VSCODE_ARCH'];
const identity = process.env['CODESIGN_IDENTITY'];
if (!buildDir) {
throw new Error('$AGENT_BUILDDIRECTORY not set');
}
if (!tempDir) {
throw new Error('$AGENT_TEMPDIRECTORY not set');
}
const product = JSON.parse(fs_1.default.readFileSync(path_1.default.join(root, 'product.json'), 'utf8'));
const baseDir = path_1.default.dirname(__dirname);
const appRoot = path_1.default.join(buildDir, `VSCode-darwin-${arch}`);
const appName = product.nameLong + '.app';
const appFrameworkPath = path_1.default.join(appRoot, appName, 'Contents', 'Frameworks');
const helperAppBaseName = product.nameShort;
const gpuHelperAppName = helperAppBaseName + ' Helper (GPU).app';
const rendererHelperAppName = helperAppBaseName + ' Helper (Renderer).app';
const pluginHelperAppName = helperAppBaseName + ' Helper (Plugin).app';
const infoPlistPath = path_1.default.resolve(appRoot, appName, 'Contents', 'Info.plist');
const defaultOpts = {
app: path_1.default.join(appRoot, appName),
platform: 'darwin',
entitlements: path_1.default.join(baseDir, 'azure-pipelines', 'darwin', 'app-entitlements.plist'),
'entitlements-inherit': path_1.default.join(baseDir, 'azure-pipelines', 'darwin', 'app-entitlements.plist'),
hardenedRuntime: true,
'pre-auto-entitlements': false,
'pre-embed-provisioning-profile': false,
keychain: path_1.default.join(tempDir, 'buildagent.keychain'),
version: getElectronVersion(),
identity,
'gatekeeper-assess': false
};
const appOpts = {
...defaultOpts,
// TODO(deepak1556): Incorrectly declared type in electron-osx-sign
ignore: (filePath) => {
return filePath.includes(gpuHelperAppName) ||
filePath.includes(rendererHelperAppName) ||
filePath.includes(pluginHelperAppName);
}
};
const gpuHelperOpts = {
...defaultOpts,
app: path_1.default.join(appFrameworkPath, gpuHelperAppName),
entitlements: path_1.default.join(baseDir, 'azure-pipelines', 'darwin', 'helper-gpu-entitlements.plist'),
'entitlements-inherit': path_1.default.join(baseDir, 'azure-pipelines', 'darwin', 'helper-gpu-entitlements.plist'),
};
const rendererHelperOpts = {
...defaultOpts,
app: path_1.default.join(appFrameworkPath, rendererHelperAppName),
entitlements: path_1.default.join(baseDir, 'azure-pipelines', 'darwin', 'helper-renderer-entitlements.plist'),
'entitlements-inherit': path_1.default.join(baseDir, 'azure-pipelines', 'darwin', 'helper-renderer-entitlements.plist'),
};
const pluginHelperOpts = {
...defaultOpts,
app: path_1.default.join(appFrameworkPath, pluginHelperAppName),
entitlements: path_1.default.join(baseDir, 'azure-pipelines', 'darwin', 'helper-plugin-entitlements.plist'),
'entitlements-inherit': path_1.default.join(baseDir, 'azure-pipelines', 'darwin', 'helper-plugin-entitlements.plist'),
};
// Only overwrite plist entries for x64 and arm64 builds,
// universal will get its copy from the x64 build.
if (arch !== 'universal') {
await (0, cross_spawn_promise_1.spawn)('plutil', [
'-insert',
'NSAppleEventsUsageDescription',
'-string',
'An application in Visual Studio Code wants to use AppleScript.',
`${infoPlistPath}`
]);
await (0, cross_spawn_promise_1.spawn)('plutil', [
'-replace',
'NSMicrophoneUsageDescription',
'-string',
'An application in Visual Studio Code wants to use the Microphone.',
`${infoPlistPath}`
]);
await (0, cross_spawn_promise_1.spawn)('plutil', [
'-replace',
'NSCameraUsageDescription',
'-string',
'An application in Visual Studio Code wants to use the Camera.',
`${infoPlistPath}`
]);
}
await electron_osx_sign_1.default.signAsync(gpuHelperOpts);
await electron_osx_sign_1.default.signAsync(rendererHelperOpts);
await electron_osx_sign_1.default.signAsync(pluginHelperOpts);
await electron_osx_sign_1.default.signAsync(appOpts);
}
if (require.main === module) {
main(process.argv[2]).catch(err => {
console.error(err);
process.exit(1);
});
}
//# sourceMappingURL=sign.js.map
================================================
FILE: build/darwin/sign.ts
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
import fs from 'fs';
import path from 'path';
import codesign from 'electron-osx-sign';
import { spawn } from '@malept/cross-spawn-promise';
const root = path.dirname(path.dirname(__dirname));
function getElectronVersion(): string {
const npmrc = fs.readFileSync(path.join(root, '.npmrc'), 'utf8');
const target = /^target="(.*)"$/m.exec(npmrc)![1];
return target;
}
async function main(buildDir?: string): Promise {
const tempDir = process.env['AGENT_TEMPDIRECTORY'];
const arch = process.env['VSCODE_ARCH'];
const identity = process.env['CODESIGN_IDENTITY'];
if (!buildDir) {
throw new Error('$AGENT_BUILDDIRECTORY not set');
}
if (!tempDir) {
throw new Error('$AGENT_TEMPDIRECTORY not set');
}
const product = JSON.parse(fs.readFileSync(path.join(root, 'product.json'), 'utf8'));
const baseDir = path.dirname(__dirname);
const appRoot = path.join(buildDir, `VSCode-darwin-${arch}`);
const appName = product.nameLong + '.app';
const appFrameworkPath = path.join(appRoot, appName, 'Contents', 'Frameworks');
const helperAppBaseName = product.nameShort;
const gpuHelperAppName = helperAppBaseName + ' Helper (GPU).app';
const rendererHelperAppName = helperAppBaseName + ' Helper (Renderer).app';
const pluginHelperAppName = helperAppBaseName + ' Helper (Plugin).app';
const infoPlistPath = path.resolve(appRoot, appName, 'Contents', 'Info.plist');
const defaultOpts: codesign.SignOptions = {
app: path.join(appRoot, appName),
platform: 'darwin',
entitlements: path.join(baseDir, 'azure-pipelines', 'darwin', 'app-entitlements.plist'),
'entitlements-inherit': path.join(baseDir, 'azure-pipelines', 'darwin', 'app-entitlements.plist'),
hardenedRuntime: true,
'pre-auto-entitlements': false,
'pre-embed-provisioning-profile': false,
keychain: path.join(tempDir, 'buildagent.keychain'),
version: getElectronVersion(),
identity,
'gatekeeper-assess': false
};
const appOpts = {
...defaultOpts,
// TODO(deepak1556): Incorrectly declared type in electron-osx-sign
ignore: (filePath: string) => {
return filePath.includes(gpuHelperAppName) ||
filePath.includes(rendererHelperAppName) ||
filePath.includes(pluginHelperAppName);
}
};
const gpuHelperOpts: codesign.SignOptions = {
...defaultOpts,
app: path.join(appFrameworkPath, gpuHelperAppName),
entitlements: path.join(baseDir, 'azure-pipelines', 'darwin', 'helper-gpu-entitlements.plist'),
'entitlements-inherit': path.join(baseDir, 'azure-pipelines', 'darwin', 'helper-gpu-entitlements.plist'),
};
const rendererHelperOpts: codesign.SignOptions = {
...defaultOpts,
app: path.join(appFrameworkPath, rendererHelperAppName),
entitlements: path.join(baseDir, 'azure-pipelines', 'darwin', 'helper-renderer-entitlements.plist'),
'entitlements-inherit': path.join(baseDir, 'azure-pipelines', 'darwin', 'helper-renderer-entitlements.plist'),
};
const pluginHelperOpts: codesign.SignOptions = {
...defaultOpts,
app: path.join(appFrameworkPath, pluginHelperAppName),
entitlements: path.join(baseDir, 'azure-pipelines', 'darwin', 'helper-plugin-entitlements.plist'),
'entitlements-inherit': path.join(baseDir, 'azure-pipelines', 'darwin', 'helper-plugin-entitlements.plist'),
};
// Only overwrite plist entries for x64 and arm64 builds,
// universal will get its copy from the x64 build.
if (arch !== 'universal') {
await spawn('plutil', [
'-insert',
'NSAppleEventsUsageDescription',
'-string',
'An application in Visual Studio Code wants to use AppleScript.',
`${infoPlistPath}`
]);
await spawn('plutil', [
'-replace',
'NSMicrophoneUsageDescription',
'-string',
'An application in Visual Studio Code wants to use the Microphone.',
`${infoPlistPath}`
]);
await spawn('plutil', [
'-replace',
'NSCameraUsageDescription',
'-string',
'An application in Visual Studio Code wants to use the Camera.',
`${infoPlistPath}`
]);
}
await codesign.signAsync(gpuHelperOpts);
await codesign.signAsync(rendererHelperOpts);
await codesign.signAsync(pluginHelperOpts);
await codesign.signAsync(appOpts as any);
}
if (require.main === module) {
main(process.argv[2]).catch(err => {
console.error(err);
process.exit(1);
});
}
================================================
FILE: build/darwin/verify-macho.js
================================================
"use strict";
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
Object.defineProperty(exports, "__esModule", { value: true });
const assert_1 = __importDefault(require("assert"));
const path_1 = __importDefault(require("path"));
const promises_1 = require("fs/promises");
const cross_spawn_promise_1 = require("@malept/cross-spawn-promise");
const MACHO_PREFIX = 'Mach-O ';
const MACHO_64_MAGIC_LE = 0xfeedfacf;
const MACHO_UNIVERSAL_MAGIC_LE = 0xbebafeca;
const MACHO_ARM64_CPU_TYPE = new Set([
0x0c000001,
0x0100000c,
]);
const MACHO_X86_64_CPU_TYPE = new Set([
0x07000001,
0x01000007,
]);
async function read(file, buf, offset, length, position) {
let filehandle;
try {
filehandle = await (0, promises_1.open)(file);
await filehandle.read(buf, offset, length, position);
}
finally {
await filehandle?.close();
}
}
async function checkMachOFiles(appPath, arch) {
const visited = new Set();
const invalidFiles = [];
const header = Buffer.alloc(8);
const file_header_entry_size = 20;
const checkx86_64Arch = (arch === 'x64');
const checkArm64Arch = (arch === 'arm64');
const checkUniversalArch = (arch === 'universal');
const traverse = async (p) => {
p = await (0, promises_1.realpath)(p);
if (visited.has(p)) {
return;
}
visited.add(p);
const info = await (0, promises_1.stat)(p);
if (info.isSymbolicLink()) {
return;
}
if (info.isFile()) {
let fileOutput = '';
try {
fileOutput = await (0, cross_spawn_promise_1.spawn)('file', ['--brief', '--no-pad', p]);
}
catch (e) {
if (e instanceof cross_spawn_promise_1.ExitCodeError) {
/* silently accept error codes from "file" */
}
else {
throw e;
}
}
if (fileOutput.startsWith(MACHO_PREFIX)) {
console.log(`Verifying architecture of ${p}`);
read(p, header, 0, 8, 0).then(_ => {
const header_magic = header.readUInt32LE();
if (header_magic === MACHO_64_MAGIC_LE) {
const cpu_type = header.readUInt32LE(4);
if (checkUniversalArch) {
invalidFiles.push(p);
}
else if (checkArm64Arch && !MACHO_ARM64_CPU_TYPE.has(cpu_type)) {
invalidFiles.push(p);
}
else if (checkx86_64Arch && !MACHO_X86_64_CPU_TYPE.has(cpu_type)) {
invalidFiles.push(p);
}
}
else if (header_magic === MACHO_UNIVERSAL_MAGIC_LE) {
const num_binaries = header.readUInt32BE(4);
assert_1.default.equal(num_binaries, 2);
const file_entries_size = file_header_entry_size * num_binaries;
const file_entries = Buffer.alloc(file_entries_size);
read(p, file_entries, 0, file_entries_size, 8).then(_ => {
for (let i = 0; i < num_binaries; i++) {
const cpu_type = file_entries.readUInt32LE(file_header_entry_size * i);
if (!MACHO_ARM64_CPU_TYPE.has(cpu_type) && !MACHO_X86_64_CPU_TYPE.has(cpu_type)) {
invalidFiles.push(p);
}
}
});
}
});
}
}
if (info.isDirectory()) {
for (const child of await (0, promises_1.readdir)(p)) {
await traverse(path_1.default.resolve(p, child));
}
}
};
await traverse(appPath);
return invalidFiles;
}
const archToCheck = process.argv[2];
(0, assert_1.default)(process.env['APP_PATH'], 'APP_PATH not set');
(0, assert_1.default)(archToCheck === 'x64' || archToCheck === 'arm64' || archToCheck === 'universal', `Invalid architecture ${archToCheck} to check`);
checkMachOFiles(process.env['APP_PATH'], archToCheck).then(invalidFiles => {
if (invalidFiles.length > 0) {
console.error('\x1b[31mThe following files are built for the wrong architecture:\x1b[0m');
for (const file of invalidFiles) {
console.error(`\x1b[31m${file}\x1b[0m`);
}
process.exit(1);
}
else {
console.log('\x1b[32mAll files are valid\x1b[0m');
}
}).catch(err => {
console.error(err);
process.exit(1);
});
//# sourceMappingURL=verify-macho.js.map
================================================
FILE: build/darwin/verify-macho.ts
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
import assert from 'assert';
import path from 'path';
import { open, stat, readdir, realpath } from 'fs/promises';
import { spawn, ExitCodeError } from '@malept/cross-spawn-promise';
const MACHO_PREFIX = 'Mach-O ';
const MACHO_64_MAGIC_LE = 0xfeedfacf;
const MACHO_UNIVERSAL_MAGIC_LE = 0xbebafeca;
const MACHO_ARM64_CPU_TYPE = new Set([
0x0c000001,
0x0100000c,
]);
const MACHO_X86_64_CPU_TYPE = new Set([
0x07000001,
0x01000007,
]);
async function read(file: string, buf: Buffer, offset: number, length: number, position: number) {
let filehandle;
try {
filehandle = await open(file);
await filehandle.read(buf, offset, length, position);
} finally {
await filehandle?.close();
}
}
async function checkMachOFiles(appPath: string, arch: string) {
const visited = new Set();
const invalidFiles: string[] = [];
const header = Buffer.alloc(8);
const file_header_entry_size = 20;
const checkx86_64Arch = (arch === 'x64');
const checkArm64Arch = (arch === 'arm64');
const checkUniversalArch = (arch === 'universal');
const traverse = async (p: string) => {
p = await realpath(p);
if (visited.has(p)) {
return;
}
visited.add(p);
const info = await stat(p);
if (info.isSymbolicLink()) {
return;
}
if (info.isFile()) {
let fileOutput = '';
try {
fileOutput = await spawn('file', ['--brief', '--no-pad', p]);
} catch (e) {
if (e instanceof ExitCodeError) {
/* silently accept error codes from "file" */
} else {
throw e;
}
}
if (fileOutput.startsWith(MACHO_PREFIX)) {
console.log(`Verifying architecture of ${p}`);
read(p, header, 0, 8, 0).then(_ => {
const header_magic = header.readUInt32LE();
if (header_magic === MACHO_64_MAGIC_LE) {
const cpu_type = header.readUInt32LE(4);
if (checkUniversalArch) {
invalidFiles.push(p);
} else if (checkArm64Arch && !MACHO_ARM64_CPU_TYPE.has(cpu_type)) {
invalidFiles.push(p);
} else if (checkx86_64Arch && !MACHO_X86_64_CPU_TYPE.has(cpu_type)) {
invalidFiles.push(p);
}
} else if (header_magic === MACHO_UNIVERSAL_MAGIC_LE) {
const num_binaries = header.readUInt32BE(4);
assert.equal(num_binaries, 2);
const file_entries_size = file_header_entry_size * num_binaries;
const file_entries = Buffer.alloc(file_entries_size);
read(p, file_entries, 0, file_entries_size, 8).then(_ => {
for (let i = 0; i < num_binaries; i++) {
const cpu_type = file_entries.readUInt32LE(file_header_entry_size * i);
if (!MACHO_ARM64_CPU_TYPE.has(cpu_type) && !MACHO_X86_64_CPU_TYPE.has(cpu_type)) {
invalidFiles.push(p);
}
}
});
}
});
}
}
if (info.isDirectory()) {
for (const child of await readdir(p)) {
await traverse(path.resolve(p, child));
}
}
};
await traverse(appPath);
return invalidFiles;
}
const archToCheck = process.argv[2];
assert(process.env['APP_PATH'], 'APP_PATH not set');
assert(archToCheck === 'x64' || archToCheck === 'arm64' || archToCheck === 'universal', `Invalid architecture ${archToCheck} to check`);
checkMachOFiles(process.env['APP_PATH'], archToCheck).then(invalidFiles => {
if (invalidFiles.length > 0) {
console.error('\x1b[31mThe following files are built for the wrong architecture:\x1b[0m');
for (const file of invalidFiles) {
console.error(`\x1b[31m${file}\x1b[0m`);
}
process.exit(1);
} else {
console.log('\x1b[32mAll files are valid\x1b[0m');
}
}).catch(err => {
console.error(err);
process.exit(1);
});
================================================
FILE: build/eslint.js
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
const es = require('event-stream');
const vfs = require('vinyl-fs');
const { eslintFilter } = require('./filters');
function eslint() {
const eslint = require('./gulp-eslint');
return vfs
.src(eslintFilter, { base: '.', follow: true, allowEmpty: true })
.pipe(
eslint((results) => {
if (results.warningCount > 0 || results.errorCount > 0) {
throw new Error('eslint failed with warnings and/or errors');
}
})
).pipe(es.through(function () { /* noop, important for the stream to end */ }));
}
if (require.main === module) {
eslint().on('error', (err) => {
console.error();
console.error(err);
process.exit(1);
});
}
================================================
FILE: build/filters.js
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
/**
* Hygiene works by creating cascading subsets of all our files and
* passing them through a sequence of checks. Here are the current subsets,
* named according to the checks performed on them. Each subset contains
* the following one, as described in mathematical notation:
*
* all ⊃ eol ⊇ indentation ⊃ copyright ⊃ typescript
*/
const { readFileSync } = require('fs');
const { join } = require('path');
module.exports.all = [
'*',
'build/**/*',
'extensions/**/*',
'scripts/**/*',
'src/**/*',
'test/**/*',
'!cli/**/*',
'!out*/**',
'!test/**/out/**',
'!**/node_modules/**',
];
module.exports.unicodeFilter = [
'**',
'!**/ThirdPartyNotices.txt',
'!**/ThirdPartyNotices.cli.txt',
'!**/LICENSE.{txt,rtf}',
'!LICENSES.chromium.html',
'!**/LICENSE',
'!**/*.{dll,exe,png,bmp,jpg,scpt,cur,ttf,woff,eot,template,ico,icns,opus,wasm}',
'!**/test/**',
'!**/*.test.ts',
'!**/*.{d.ts,json,md}',
'!**/*.mp3',
'!build/win32/**',
'!extensions/markdown-language-features/notebook-out/*.js',
'!extensions/markdown-math/notebook-out/**',
'!extensions/ipynb/notebook-out/**',
'!extensions/notebook-renderers/renderer-out/**',
'!extensions/php-language-features/src/features/phpGlobalFunctions.ts',
'!extensions/terminal-suggest/src/completions/upstream/**',
'!extensions/typescript-language-features/test-workspace/**',
'!extensions/vscode-api-tests/testWorkspace/**',
'!extensions/vscode-api-tests/testWorkspace2/**',
'!extensions/**/dist/**',
'!extensions/**/out/**',
'!extensions/**/snippets/**',
'!extensions/**/colorize-fixtures/**',
'!extensions/terminal-suggest/src/shell/fishBuiltinsCache.ts',
'!src/vs/base/browser/dompurify/**',
'!src/vs/workbench/services/keybinding/browser/keyboardLayouts/**',
];
module.exports.indentationFilter = [
'**',
// except specific files
'!**/ThirdPartyNotices.txt',
'!**/ThirdPartyNotices.cli.txt',
'!**/LICENSE.{txt,rtf}',
'!LICENSES.chromium.html',
'!**/LICENSE',
'!**/*.mp3',
'!src/vs/loader.js',
'!src/vs/base/browser/dompurify/*',
'!src/vs/base/common/marked/marked.js',
'!src/vs/base/common/semver/semver.js',
'!src/vs/base/node/terminateProcess.sh',
'!src/vs/base/node/cpuUsage.sh',
'!src/vs/editor/common/languages/highlights/*.scm',
'!src/vs/editor/common/languages/injections/*.scm',
'!test/unit/assert.js',
'!resources/linux/snap/electron-launch',
'!build/ext.js',
'!build/npm/gyp/patches/gyp_spectre_mitigation_support.patch',
'!product.overrides.json',
// except specific folders
'!test/automation/out/**',
'!test/monaco/out/**',
'!test/smoke/out/**',
'!extensions/terminal-suggest/src/shell/zshBuiltinsCache.ts',
'!extensions/terminal-suggest/src/shell/fishBuiltinsCache.ts',
'!extensions/terminal-suggest/src/completions/upstream/**',
'!extensions/typescript-language-features/test-workspace/**',
'!extensions/typescript-language-features/resources/walkthroughs/**',
'!extensions/typescript-language-features/package-manager/node-maintainer/**',
'!extensions/markdown-math/notebook-out/**',
'!extensions/ipynb/notebook-out/**',
'!extensions/vscode-api-tests/testWorkspace/**',
'!extensions/vscode-api-tests/testWorkspace2/**',
'!build/monaco/**',
'!build/win32/**',
// except multiple specific files
'!**/package.json',
'!**/package-lock.json',
// except multiple specific folders
'!**/codicon/**',
'!**/fixtures/**',
'!**/lib/**',
'!extensions/**/dist/**',
'!extensions/**/out/**',
'!extensions/**/snippets/**',
'!extensions/**/syntaxes/**',
'!extensions/**/themes/**',
'!extensions/**/colorize-fixtures/**',
// except specific file types
'!src/vs/*/**/*.d.ts',
'!src/typings/**/*.d.ts',
'!extensions/**/*.d.ts',
'!**/*.{svg,exe,png,bmp,jpg,scpt,bat,cmd,cur,ttf,woff,eot,md,ps1,psm1,template,yaml,yml,d.ts.recipe,ico,icns,plist,opus,admx,adml,wasm}',
'!build/{lib,download,linux,darwin}/**/*.js',
'!build/**/*.sh',
'!build/azure-pipelines/**/*.js',
'!build/azure-pipelines/**/*.config',
'!**/Dockerfile',
'!**/Dockerfile.*',
'!**/*.Dockerfile',
'!**/*.dockerfile',
// except for built files
'!extensions/markdown-language-features/media/*.js',
'!extensions/markdown-language-features/notebook-out/*.js',
'!extensions/markdown-math/notebook-out/*.js',
'!extensions/ipynb/notebook-out/**',
'!extensions/notebook-renderers/renderer-out/*.js',
'!extensions/simple-browser/media/*.js',
];
module.exports.copyrightFilter = [
'**',
'!**/*.desktop',
'!**/*.json',
'!**/*.html',
'!**/*.template',
'!**/*.md',
'!**/*.bat',
'!**/*.cmd',
'!**/*.ico',
'!**/*.opus',
'!**/*.mp3',
'!**/*.icns',
'!**/*.xml',
'!**/*.sh',
'!**/*.zsh',
'!**/*.fish',
'!**/*.txt',
'!**/*.xpm',
'!**/*.opts',
'!**/*.disabled',
'!**/*.code-workspace',
'!**/*.js.map',
'!**/*.wasm',
'!build/**/*.init',
'!build/linux/libcxx-fetcher.*',
'!resources/linux/snap/snapcraft.yaml',
'!resources/win32/bin/code.js',
'!resources/completions/**',
'!extensions/configuration-editing/build/inline-allOf.ts',
'!extensions/markdown-language-features/media/highlight.css',
'!extensions/markdown-math/notebook-out/**',
'!extensions/ipynb/notebook-out/**',
'!extensions/simple-browser/media/codicon.css',
'!extensions/terminal-suggest/src/completions/upstream/**',
'!extensions/typescript-language-features/node-maintainer/**',
'!extensions/html-language-features/server/src/modes/typescript/*',
'!extensions/*/server/bin/*',
];
module.exports.tsFormattingFilter = [
'src/**/*.ts',
'test/**/*.ts',
'extensions/**/*.ts',
'!src/vs/*/**/*.d.ts',
'!src/typings/**/*.d.ts',
'!extensions/**/*.d.ts',
'!**/fixtures/**',
'!**/typings/**',
'!**/node_modules/**',
'!extensions/**/colorize-fixtures/**',
'!extensions/vscode-api-tests/testWorkspace/**',
'!extensions/vscode-api-tests/testWorkspace2/**',
'!extensions/**/*.test.ts',
'!extensions/html-language-features/server/lib/jquery.d.ts',
'!extensions/terminal-suggest/src/shell/zshBuiltinsCache.ts',
'!extensions/terminal-suggest/src/shell/fishBuiltinsCache.ts',
];
module.exports.eslintFilter = [
'**/*.js',
'**/*.cjs',
'**/*.mjs',
'**/*.ts',
'.eslint-plugin-local/**/*.ts',
...readFileSync(join(__dirname, '..', '.eslint-ignore'))
.toString()
.split(/\r\n|\n/)
.filter(line => line && !line.startsWith('#'))
.map(line => line.startsWith('!') ? line.slice(1) : `!${line}`)
];
module.exports.stylelintFilter = [
'src/**/*.css'
];
================================================
FILE: build/gulp-eslint.js
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
'use strict';
const { ESLint } = require('eslint');
const { Transform } = require('stream');
const { relative } = require('path');
const fancyLog = require('fancy-log');
/**
* @param {Function} action - A function to handle all ESLint results
* @returns {stream} gulp file stream
*/
function eslint(action) {
const linter = new ESLint({});
const formatter = linter.loadFormatter('compact');
const results = [];
results.errorCount = 0;
results.warningCount = 0;
return transform(
async (file, enc, cb) => {
const filePath = relative(process.cwd(), file.path);
if (file.isNull()) {
cb(null, file);
return;
}
if (file.isStream()) {
cb(new Error('vinyl files with Stream contents are not supported'));
return;
}
try {
// TODO: Should this be checked?
if (await linter.isPathIgnored(filePath)) {
cb(null, file);
return;
}
const result = (await linter.lintText(file.contents.toString(), { filePath }))[0];
results.push(result);
results.errorCount += result.errorCount;
results.warningCount += result.warningCount;
const message = (await formatter).format([result]);
if (message) {
fancyLog(message);
}
cb(null, file);
} catch (error) {
cb(error);
}
},
(done) => {
try {
action(results);
done();
} catch (error) {
done(error);
}
});
}
function transform(transform, flush) {
return new Transform({
objectMode: true,
transform,
flush
});
}
module.exports = eslint;
================================================
FILE: build/gulpfile.cli.js
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
'use strict';
const es = require('event-stream');
const gulp = require('gulp');
const path = require('path');
const fancyLog = require('fancy-log');
const ansiColors = require('ansi-colors');
const cp = require('child_process');
const { tmpdir } = require('os');
const { promises: fs, existsSync, mkdirSync, rmSync } = require('fs');
const task = require('./lib/task');
const watcher = require('./lib/watch');
const { debounce } = require('./lib/util');
const createReporter = require('./lib/reporter').createReporter;
const root = 'cli';
const rootAbs = path.resolve(__dirname, '..', root);
const src = `${root}/src`;
const platformOpensslDirName =
process.platform === 'win32' ? (
process.arch === 'arm64'
? 'arm64-windows-static-md'
: 'x64-windows-static-md')
: process.platform === 'darwin' ? (
process.arch === 'arm64'
? 'arm64-osx'
: 'x64-osx')
: (process.arch === 'arm64'
? 'arm64-linux'
: process.arch === 'arm'
? 'arm-linux'
: 'x64-linux');
const platformOpensslDir = path.join(rootAbs, 'openssl', 'package', 'out', platformOpensslDirName);
const hasLocalRust = (() => {
/** @type boolean | undefined */
let result = undefined;
return () => {
if (result !== undefined) {
return result;
}
try {
const r = cp.spawnSync('cargo', ['--version']);
result = r.status === 0;
} catch (e) {
result = false;
}
return result;
};
})();
const debounceEsStream = (fn, duration = 100) => {
let handle = undefined;
let pending = [];
const sendAll = (pending) => (event, ...args) => {
for (const stream of pending) {
pending.emit(event, ...args);
}
};
return es.map(function (_, callback) {
console.log('defer');
if (handle !== undefined) {
clearTimeout(handle);
}
handle = setTimeout(() => {
handle = undefined;
const previous = pending;
pending = [];
fn()
.on('error', sendAll('error'))
.on('data', sendAll('data'))
.on('end', sendAll('end'));
}, duration);
pending.push(this);
});
};
const compileFromSources = (callback) => {
const proc = cp.spawn('cargo', ['--color', 'always', 'build'], {
cwd: root,
stdio: ['ignore', 'pipe', 'pipe'],
env: existsSync(platformOpensslDir) ? { OPENSSL_DIR: platformOpensslDir, ...process.env } : process.env
});
/** @type Buffer[] */
const stdoutErr = [];
proc.stdout.on('data', d => stdoutErr.push(d));
proc.stderr.on('data', d => stdoutErr.push(d));
proc.on('error', callback);
proc.on('exit', code => {
if (code !== 0) {
callback(Buffer.concat(stdoutErr).toString());
} else {
callback();
}
});
};
const acquireBuiltOpenSSL = (callback) => {
const untar = require('gulp-untar');
const gunzip = require('gulp-gunzip');
const dir = path.join(tmpdir(), 'vscode-openssl-download');
mkdirSync(dir, { recursive: true });
cp.spawnSync(
process.platform === 'win32' ? 'npm.cmd' : 'npm',
['pack', '@vscode/openssl-prebuilt'],
{ stdio: ['ignore', 'ignore', 'inherit'], cwd: dir }
);
gulp.src('*.tgz', { cwd: dir })
.pipe(gunzip())
.pipe(untar())
.pipe(gulp.dest(`${root}/openssl`))
.on('error', callback)
.on('end', () => {
rmSync(dir, { recursive: true, force: true });
callback();
});
};
const compileWithOpenSSLCheck = (/** @type import('./lib/reporter').IReporter */ reporter) => es.map((_, callback) => {
compileFromSources(err => {
if (!err) {
// no-op
} else if (err.toString().includes('Could not find directory of OpenSSL installation') && !existsSync(platformOpensslDir)) {
fancyLog(ansiColors.yellow(`[cli]`), 'OpenSSL libraries not found, acquiring prebuilt bits...');
acquireBuiltOpenSSL(err => {
if (err) {
callback(err);
} else {
compileFromSources(err => {
if (err) {
reporter(err.toString());
}
callback(null, '');
});
}
});
} else {
reporter(err.toString());
}
callback(null, '');
});
});
const warnIfRustNotInstalled = () => {
if (!hasLocalRust()) {
fancyLog(ansiColors.yellow(`[cli]`), 'No local Rust install detected, compilation may fail.');
fancyLog(ansiColors.yellow(`[cli]`), 'Get rust from: https://rustup.rs/');
}
};
const compileCliTask = task.define('compile-cli', () => {
warnIfRustNotInstalled();
const reporter = createReporter('cli');
return gulp.src(`${root}/Cargo.toml`)
.pipe(compileWithOpenSSLCheck(reporter))
.pipe(reporter.end(true));
});
const watchCliTask = task.define('watch-cli', () => {
warnIfRustNotInstalled();
return watcher(`${src}/**`, { read: false })
.pipe(debounce(compileCliTask));
});
gulp.task(compileCliTask);
gulp.task(watchCliTask);
================================================
FILE: build/gulpfile.compile.js
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
//@ts-check
'use strict';
const gulp = require('gulp');
const util = require('./lib/util');
const date = require('./lib/date');
const task = require('./lib/task');
const compilation = require('./lib/compilation');
/**
* @param {boolean} disableMangle
*/
function makeCompileBuildTask(disableMangle) {
return task.series(
util.rimraf('out-build'),
date.writeISODate('out-build'),
compilation.compileApiProposalNamesTask,
compilation.compileTask('src', 'out-build', true, { disableMangle })
);
}
// Local/PR compile, including nls and inline sources in sourcemaps, minification, no mangling
const compileBuildWithoutManglingTask = task.define('compile-build-without-mangling', makeCompileBuildTask(true));
gulp.task(compileBuildWithoutManglingTask);
exports.compileBuildWithoutManglingTask = compileBuildWithoutManglingTask;
// CI compile, including nls and inline sources in sourcemaps, mangling, minification, for build
const compileBuildWithManglingTask = task.define('compile-build-with-mangling', makeCompileBuildTask(false));
gulp.task(compileBuildWithManglingTask);
exports.compileBuildWithManglingTask = compileBuildWithManglingTask;
================================================
FILE: build/gulpfile.editor.js
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
//@ts-check
const gulp = require('gulp');
const path = require('path');
const util = require('./lib/util');
const { getVersion } = require('./lib/getVersion');
const task = require('./lib/task');
const es = require('event-stream');
const File = require('vinyl');
const i18n = require('./lib/i18n');
const standalone = require('./lib/standalone');
const cp = require('child_process');
const compilation = require('./lib/compilation');
const monacoapi = require('./lib/monaco-api');
const fs = require('fs');
const filter = require('gulp-filter');
const root = path.dirname(__dirname);
const sha1 = getVersion(root);
const semver = require('./monaco/package.json').version;
const headerVersion = semver + '(' + sha1 + ')';
const BUNDLED_FILE_HEADER = [
'/*!-----------------------------------------------------------',
' * Copyright (c) Microsoft Corporation. All rights reserved.',
' * Version: ' + headerVersion,
' * Released under the MIT license',
' * https://github.com/microsoft/vscode/blob/main/LICENSE.txt',
' *-----------------------------------------------------------*/',
''
].join('\n');
const extractEditorSrcTask = task.define('extract-editor-src', () => {
const apiusages = monacoapi.execute().usageContent;
const extrausages = fs.readFileSync(path.join(root, 'build', 'monaco', 'monaco.usage.recipe')).toString();
standalone.extractEditor({
sourcesRoot: path.join(root, 'src'),
entryPoints: [
'vs/editor/editor.main',
'vs/editor/editor.worker.start',
'vs/editor/common/services/editorWebWorkerMain',
],
inlineEntryPoints: [
apiusages,
extrausages
],
typings: [],
shakeLevel: 2, // 0-Files, 1-InnerFile, 2-ClassMembers
importIgnorePattern: /\.css$/,
destRoot: path.join(root, 'out-editor-src'),
tsOutDir: '../out-monaco-editor-core/esm/vs',
redirects: {
'@vscode/tree-sitter-wasm': '../node_modules/@vscode/tree-sitter-wasm/wasm/web-tree-sitter',
}
});
});
const compileEditorESMTask = task.define('compile-editor-esm', () => {
const src = 'out-editor-src';
const out = 'out-monaco-editor-core/esm';
const compile = compilation.createCompile(src, { build: true, emitError: true, transpileOnly: false, preserveEnglish: true });
const srcPipe = gulp.src(`${src}/**`, { base: `${src}` });
return (
srcPipe
.pipe(compile())
.pipe(i18n.processNlsFiles({
out,
fileHeader: BUNDLED_FILE_HEADER,
languages: i18n.defaultLanguages,
}))
.pipe(filter(['**', '!**/inlineEntryPoint*', '!**/tsconfig.json', '!**/loader.js']))
.pipe(gulp.dest(out))
);
});
/**
* @param {string} contents
*/
function toExternalDTS(contents) {
const lines = contents.split(/\r\n|\r|\n/);
let killNextCloseCurlyBrace = false;
for (let i = 0; i < lines.length; i++) {
const line = lines[i];
if (killNextCloseCurlyBrace) {
if ('}' === line) {
lines[i] = '';
killNextCloseCurlyBrace = false;
continue;
}
if (line.indexOf(' ') === 0) {
lines[i] = line.substr(4);
} else if (line.charAt(0) === '\t') {
lines[i] = line.substr(1);
}
continue;
}
if ('declare namespace monaco {' === line) {
lines[i] = '';
killNextCloseCurlyBrace = true;
continue;
}
if (line.indexOf('declare namespace monaco.') === 0) {
lines[i] = line.replace('declare namespace monaco.', 'export namespace ');
}
if (line.indexOf('declare let MonacoEnvironment') === 0) {
lines[i] = `declare global {\n let MonacoEnvironment: Environment | undefined;\n}`;
}
if (line.indexOf('\tMonacoEnvironment?') === 0) {
lines[i] = ` MonacoEnvironment?: Environment | undefined;`;
}
}
return lines.join('\n').replace(/\n\n\n+/g, '\n\n');
}
const finalEditorResourcesTask = task.define('final-editor-resources', () => {
return es.merge(
// other assets
es.merge(
gulp.src('build/monaco/LICENSE'),
gulp.src('build/monaco/ThirdPartyNotices.txt'),
gulp.src('src/vs/monaco.d.ts')
).pipe(gulp.dest('out-monaco-editor-core')),
// place the .d.ts in the esm folder
gulp.src('src/vs/monaco.d.ts')
.pipe(es.through(function (data) {
this.emit('data', new File({
path: data.path.replace(/monaco\.d\.ts/, 'editor.api.d.ts'),
base: data.base,
contents: Buffer.from(toExternalDTS(data.contents.toString()))
}));
}))
.pipe(gulp.dest('out-monaco-editor-core/esm/vs/editor')),
// package.json
gulp.src('build/monaco/package.json')
.pipe(es.through(function (data) {
const json = JSON.parse(data.contents.toString());
json.private = false;
data.contents = Buffer.from(JSON.stringify(json, null, ' '));
this.emit('data', data);
}))
.pipe(gulp.dest('out-monaco-editor-core')),
// version.txt
gulp.src('build/monaco/version.txt')
.pipe(es.through(function (data) {
data.contents = Buffer.from(`monaco-editor-core: https://github.com/microsoft/vscode/tree/${sha1}`);
this.emit('data', data);
}))
.pipe(gulp.dest('out-monaco-editor-core')),
// README.md
gulp.src('build/monaco/README-npm.md')
.pipe(es.through(function (data) {
this.emit('data', new File({
path: data.path.replace(/README-npm\.md/, 'README.md'),
base: data.base,
contents: data.contents
}));
}))
.pipe(gulp.dest('out-monaco-editor-core')),
);
});
gulp.task('extract-editor-src',
task.series(
util.rimraf('out-editor-src'),
extractEditorSrcTask
)
);
gulp.task('editor-distro',
task.series(
task.parallel(
util.rimraf('out-editor-src'),
util.rimraf('out-monaco-editor-core'),
),
extractEditorSrcTask,
compileEditorESMTask,
finalEditorResourcesTask
)
);
gulp.task('monacodts', task.define('monacodts', () => {
const result = monacoapi.execute();
fs.writeFileSync(result.filePath, result.content);
fs.writeFileSync(path.join(root, 'src/vs/editor/common/standalone/standaloneEnums.ts'), result.enums);
return Promise.resolve(true);
}));
//#region monaco type checking
/**
* @param {boolean} watch
*/
function createTscCompileTask(watch) {
return () => {
const createReporter = require('./lib/reporter').createReporter;
return new Promise((resolve, reject) => {
const args = ['./node_modules/.bin/tsc', '-p', './src/tsconfig.monaco.json', '--noEmit'];
if (watch) {
args.push('-w');
}
const child = cp.spawn(`node`, args, {
cwd: path.join(__dirname, '..'),
// stdio: [null, 'pipe', 'inherit']
});
const errors = [];
const reporter = createReporter('monaco');
/** @type {NodeJS.ReadWriteStream | undefined} */
let report;
const magic = /[\u001b\u009b][[()#;?]*(?:[0-9]{1,4}(?:;[0-9]{0,4})*)?[0-9A-ORZcf-nqry=><]/g; // https://stackoverflow.com/questions/25245716/remove-all-ansi-colors-styles-from-strings
child.stdout.on('data', data => {
let str = String(data);
str = str.replace(magic, '').trim();
if (str.indexOf('Starting compilation') >= 0 || str.indexOf('File change detected') >= 0) {
errors.length = 0;
report = reporter.end(false);
} else if (str.indexOf('Compilation complete') >= 0) {
// @ts-ignore
report.end();
} else if (str) {
const match = /(.*\(\d+,\d+\): )(.*: )(.*)/.exec(str);
if (match) {
// trying to massage the message so that it matches the gulp-tsb error messages
// e.g. src/vs/base/common/strings.ts(663,5): error TS2322: Type '1234' is not assignable to type 'string'.
const fullpath = path.join(root, match[1]);
const message = match[3];
reporter(fullpath + message);
} else {
reporter(str);
}
}
});
child.on('exit', resolve);
child.on('error', reject);
});
};
}
const monacoTypecheckWatchTask = task.define('monaco-typecheck-watch', createTscCompileTask(true));
exports.monacoTypecheckWatchTask = monacoTypecheckWatchTask;
const monacoTypecheckTask = task.define('monaco-typecheck', createTscCompileTask(false));
exports.monacoTypecheckTask = monacoTypecheckTask;
//#endregion
================================================
FILE: build/gulpfile.extensions.js
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
// Increase max listeners for event emitters
require('events').EventEmitter.defaultMaxListeners = 100;
const gulp = require('gulp');
const path = require('path');
const nodeUtil = require('util');
const es = require('event-stream');
const filter = require('gulp-filter');
const util = require('./lib/util');
const { getVersion } = require('./lib/getVersion');
const task = require('./lib/task');
const watcher = require('./lib/watch');
const createReporter = require('./lib/reporter').createReporter;
const glob = require('glob');
const root = path.dirname(__dirname);
const commit = getVersion(root);
const plumber = require('gulp-plumber');
const ext = require('./lib/extensions');
// To save 250ms for each gulp startup, we are caching the result here
// const compilations = glob.sync('**/tsconfig.json', {
// cwd: extensionsPath,
// ignore: ['**/out/**', '**/node_modules/**']
// });
const compilations = [
'extensions/configuration-editing/tsconfig.json',
'extensions/css-language-features/client/tsconfig.json',
'extensions/css-language-features/server/tsconfig.json',
'extensions/debug-auto-launch/tsconfig.json',
'extensions/debug-server-ready/tsconfig.json',
'extensions/emmet/tsconfig.json',
'extensions/extension-editing/tsconfig.json',
'extensions/git/tsconfig.json',
'extensions/git-base/tsconfig.json',
'extensions/github/tsconfig.json',
'extensions/github-authentication/tsconfig.json',
'extensions/grunt/tsconfig.json',
'extensions/gulp/tsconfig.json',
'extensions/html-language-features/client/tsconfig.json',
'extensions/html-language-features/server/tsconfig.json',
'extensions/ipynb/tsconfig.json',
'extensions/jake/tsconfig.json',
'extensions/json-language-features/client/tsconfig.json',
'extensions/json-language-features/server/tsconfig.json',
'extensions/markdown-language-features/preview-src/tsconfig.json',
'extensions/markdown-language-features/tsconfig.json',
'extensions/markdown-math/tsconfig.json',
'extensions/media-preview/tsconfig.json',
'extensions/merge-conflict/tsconfig.json',
'extensions/terminal-suggest/tsconfig.json',
'extensions/microsoft-authentication/tsconfig.json',
'extensions/notebook-renderers/tsconfig.json',
'extensions/npm/tsconfig.json',
'extensions/php-language-features/tsconfig.json',
'extensions/references-view/tsconfig.json',
'extensions/search-result/tsconfig.json',
'extensions/simple-browser/tsconfig.json',
'extensions/tunnel-forwarding/tsconfig.json',
'extensions/typescript-language-features/test-workspace/tsconfig.json',
'extensions/typescript-language-features/web/tsconfig.json',
'extensions/typescript-language-features/tsconfig.json',
'extensions/vscode-api-tests/tsconfig.json',
'extensions/vscode-colorize-tests/tsconfig.json',
'extensions/vscode-colorize-perf-tests/tsconfig.json',
'extensions/vscode-test-resolver/tsconfig.json',
'.vscode/extensions/vscode-selfhost-test-provider/tsconfig.json',
'.vscode/extensions/vscode-selfhost-import-aid/tsconfig.json',
];
const getBaseUrl = out => `https://main.vscode-cdn.net/sourcemaps/${commit}/${out}`;
const tasks = compilations.map(function (tsconfigFile) {
const absolutePath = path.join(root, tsconfigFile);
const relativeDirname = path.dirname(tsconfigFile.replace(/^(.*\/)?extensions\//i, ''));
const overrideOptions = {};
overrideOptions.sourceMap = true;
const name = relativeDirname.replace(/\//g, '-');
const srcRoot = path.dirname(tsconfigFile);
const srcBase = path.join(srcRoot, 'src');
const src = path.join(srcBase, '**');
const srcOpts = { cwd: root, base: srcBase, dot: true };
const out = path.join(srcRoot, 'out');
const baseUrl = getBaseUrl(out);
let headerId, headerOut;
const index = relativeDirname.indexOf('/');
if (index < 0) {
headerId = 'vscode.' + relativeDirname;
headerOut = 'out';
} else {
headerId = 'vscode.' + relativeDirname.substr(0, index);
headerOut = relativeDirname.substr(index + 1) + '/out';
}
function createPipeline(build, emitError, transpileOnly) {
const tsb = require('./lib/tsb');
const sourcemaps = require('gulp-sourcemaps');
const reporter = createReporter('extensions');
overrideOptions.inlineSources = Boolean(build);
overrideOptions.base = path.dirname(absolutePath);
const compilation = tsb.create(absolutePath, overrideOptions, { verbose: false, transpileOnly, transpileOnlyIncludesDts: transpileOnly, transpileWithSwc: true }, err => reporter(err.toString()));
const pipeline = function () {
const input = es.through();
const tsFilter = filter(['**/*.ts', '!**/lib/lib*.d.ts', '!**/node_modules/**'], { restore: true, dot: true });
const output = input
.pipe(plumber({
errorHandler: function (err) {
if (err && !err.__reporter__) {
reporter(err);
}
}
}))
.pipe(tsFilter)
.pipe(util.loadSourcemaps())
.pipe(compilation())
.pipe(build ? util.stripSourceMappingURL() : es.through())
.pipe(sourcemaps.write('.', {
sourceMappingURL: !build ? null : f => `${baseUrl}/${f.relative}.map`,
addComment: !!build,
includeContent: !!build,
// note: trailing slash is important, else the source URLs in V8's file coverage are incorrect
sourceRoot: '../src/',
}))
.pipe(tsFilter.restore)
.pipe(reporter.end(emitError));
return es.duplex(input, output);
};
// add src-stream for project files
pipeline.tsProjectSrc = () => {
return compilation.src(srcOpts);
};
return pipeline;
}
const cleanTask = task.define(`clean-extension-${name}`, util.rimraf(out));
const transpileTask = task.define(`transpile-extension:${name}`, task.series(cleanTask, () => {
const pipeline = createPipeline(false, true, true);
const nonts = gulp.src(src, srcOpts).pipe(filter(['**', '!**/*.ts']));
const input = es.merge(nonts, pipeline.tsProjectSrc());
return input
.pipe(pipeline())
.pipe(gulp.dest(out));
}));
const compileTask = task.define(`compile-extension:${name}`, task.series(cleanTask, () => {
const pipeline = createPipeline(false, true);
const nonts = gulp.src(src, srcOpts).pipe(filter(['**', '!**/*.ts']));
const input = es.merge(nonts, pipeline.tsProjectSrc());
return input
.pipe(pipeline())
.pipe(gulp.dest(out));
}));
const watchTask = task.define(`watch-extension:${name}`, task.series(cleanTask, () => {
const pipeline = createPipeline(false);
const nonts = gulp.src(src, srcOpts).pipe(filter(['**', '!**/*.ts']));
const input = es.merge(nonts, pipeline.tsProjectSrc());
const watchInput = watcher(src, { ...srcOpts, ...{ readDelay: 200 } });
return watchInput
.pipe(util.incremental(pipeline, input))
.pipe(gulp.dest(out));
}));
const compileBuildTask = task.define(`compile-build-extension-${name}`, task.series(cleanTask, () => {
const pipeline = createPipeline(true, true);
const nonts = gulp.src(src, srcOpts).pipe(filter(['**', '!**/*.ts']));
const input = es.merge(nonts, pipeline.tsProjectSrc());
return input
.pipe(pipeline())
.pipe(gulp.dest(out));
}));
// Tasks
gulp.task(transpileTask);
gulp.task(compileTask);
gulp.task(watchTask);
return { transpileTask, compileTask, watchTask, compileBuildTask };
});
const transpileExtensionsTask = task.define('transpile-extensions', task.parallel(...tasks.map(t => t.transpileTask)));
gulp.task(transpileExtensionsTask);
const compileExtensionsTask = task.define('compile-extensions', task.parallel(...tasks.map(t => t.compileTask)));
gulp.task(compileExtensionsTask);
exports.compileExtensionsTask = compileExtensionsTask;
const watchExtensionsTask = task.define('watch-extensions', task.parallel(...tasks.map(t => t.watchTask)));
gulp.task(watchExtensionsTask);
exports.watchExtensionsTask = watchExtensionsTask;
const compileExtensionsBuildLegacyTask = task.define('compile-extensions-build-legacy', task.parallel(...tasks.map(t => t.compileBuildTask)));
gulp.task(compileExtensionsBuildLegacyTask);
//#region Extension media
const compileExtensionMediaTask = task.define('compile-extension-media', () => ext.buildExtensionMedia(false));
gulp.task(compileExtensionMediaTask);
exports.compileExtensionMediaTask = compileExtensionMediaTask;
const watchExtensionMedia = task.define('watch-extension-media', () => ext.buildExtensionMedia(true));
gulp.task(watchExtensionMedia);
exports.watchExtensionMedia = watchExtensionMedia;
const compileExtensionMediaBuildTask = task.define('compile-extension-media-build', () => ext.buildExtensionMedia(false, '.build/extensions'));
gulp.task(compileExtensionMediaBuildTask);
exports.compileExtensionMediaBuildTask = compileExtensionMediaBuildTask;
//#endregion
//#region Azure Pipelines
/**
* Cleans the build directory for extensions
*/
const cleanExtensionsBuildTask = task.define('clean-extensions-build', util.rimraf('.build/extensions'));
exports.cleanExtensionsBuildTask = cleanExtensionsBuildTask;
/**
* brings in the marketplace extensions for the build
*/
const bundleMarketplaceExtensionsBuildTask = task.define('bundle-marketplace-extensions-build', () => ext.packageMarketplaceExtensionsStream(false).pipe(gulp.dest('.build')));
/**
* Compiles the non-native extensions for the build
* @note this does not clean the directory ahead of it. See {@link cleanExtensionsBuildTask} for that.
*/
const compileNonNativeExtensionsBuildTask = task.define('compile-non-native-extensions-build', task.series(
bundleMarketplaceExtensionsBuildTask,
task.define('bundle-non-native-extensions-build', () => ext.packageNonNativeLocalExtensionsStream().pipe(gulp.dest('.build')))
));
gulp.task(compileNonNativeExtensionsBuildTask);
exports.compileNonNativeExtensionsBuildTask = compileNonNativeExtensionsBuildTask;
/**
* Compiles the native extensions for the build
* @note this does not clean the directory ahead of it. See {@link cleanExtensionsBuildTask} for that.
*/
const compileNativeExtensionsBuildTask = task.define('compile-native-extensions-build', () => ext.packageNativeLocalExtensionsStream().pipe(gulp.dest('.build')));
gulp.task(compileNativeExtensionsBuildTask);
exports.compileNativeExtensionsBuildTask = compileNativeExtensionsBuildTask;
/**
* Compiles the extensions for the build.
* This is essentially a helper task that combines {@link cleanExtensionsBuildTask}, {@link compileNonNativeExtensionsBuildTask} and {@link compileNativeExtensionsBuildTask}
*/
const compileAllExtensionsBuildTask = task.define('compile-extensions-build', task.series(
cleanExtensionsBuildTask,
bundleMarketplaceExtensionsBuildTask,
task.define('bundle-extensions-build', () => ext.packageAllLocalExtensionsStream(false, false).pipe(gulp.dest('.build'))),
));
gulp.task(compileAllExtensionsBuildTask);
exports.compileAllExtensionsBuildTask = compileAllExtensionsBuildTask;
// This task is run in the compilation stage of the CI pipeline. We only compile the non-native extensions since those can be fully built regardless of platform.
// This defers the native extensions to the platform specific stage of the CI pipeline.
gulp.task(task.define('extensions-ci', task.series(compileNonNativeExtensionsBuildTask, compileExtensionMediaBuildTask)));
const compileExtensionsBuildPullRequestTask = task.define('compile-extensions-build-pr', task.series(
cleanExtensionsBuildTask,
bundleMarketplaceExtensionsBuildTask,
task.define('bundle-extensions-build-pr', () => ext.packageAllLocalExtensionsStream(false, true).pipe(gulp.dest('.build'))),
));
gulp.task(compileExtensionsBuildPullRequestTask);
// This task is run in the compilation stage of the PR pipeline. We compile all extensions in it to verify compilation.
gulp.task(task.define('extensions-ci-pr', task.series(compileExtensionsBuildPullRequestTask, compileExtensionMediaBuildTask)));
//#endregion
const compileWebExtensionsTask = task.define('compile-web', () => buildWebExtensions(false));
gulp.task(compileWebExtensionsTask);
exports.compileWebExtensionsTask = compileWebExtensionsTask;
const watchWebExtensionsTask = task.define('watch-web', () => buildWebExtensions(true));
gulp.task(watchWebExtensionsTask);
exports.watchWebExtensionsTask = watchWebExtensionsTask;
/**
* @param {boolean} isWatch
*/
async function buildWebExtensions(isWatch) {
const extensionsPath = path.join(root, 'extensions');
const webpackConfigLocations = await nodeUtil.promisify(glob)(
path.join(extensionsPath, '**', 'extension-browser.webpack.config.js'),
{ ignore: ['**/node_modules'] }
);
return ext.webpackExtensions('packaging web extension', isWatch, webpackConfigLocations.map(configPath => ({ configPath })));
}
================================================
FILE: build/gulpfile.hygiene.js
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
const gulp = require('gulp');
const es = require('event-stream');
const path = require('path');
const task = require('./lib/task');
const { hygiene } = require('./hygiene');
/**
* @param {string} actualPath
*/
function checkPackageJSON(actualPath) {
const actual = require(path.join(__dirname, '..', actualPath));
const rootPackageJSON = require('../package.json');
const checkIncluded = (set1, set2) => {
for (const depName in set1) {
const depVersion = set1[depName];
const rootDepVersion = set2[depName];
if (!rootDepVersion) {
// missing in root is allowed
continue;
}
if (depVersion !== rootDepVersion) {
this.emit(
'error',
`The dependency ${depName} in '${actualPath}' (${depVersion}) is different than in the root package.json (${rootDepVersion})`
);
}
}
};
checkIncluded(actual.dependencies, rootPackageJSON.dependencies);
checkIncluded(actual.devDependencies, rootPackageJSON.devDependencies);
}
const checkPackageJSONTask = task.define('check-package-json', () => {
return gulp.src('package.json').pipe(
es.through(function () {
checkPackageJSON.call(this, 'remote/package.json');
checkPackageJSON.call(this, 'remote/web/package.json');
checkPackageJSON.call(this, 'build/package.json');
})
);
});
gulp.task(checkPackageJSONTask);
const hygieneTask = task.define('hygiene', task.series(checkPackageJSONTask, () => hygiene(undefined, false)));
gulp.task(hygieneTask);
================================================
FILE: build/gulpfile.js
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
'use strict';
// Increase max listeners for event emitters
require('events').EventEmitter.defaultMaxListeners = 100;
const gulp = require('gulp');
const util = require('./lib/util');
const task = require('./lib/task');
const { transpileClientSWC, transpileTask, compileTask, watchTask, compileApiProposalNamesTask, watchApiProposalNamesTask } = require('./lib/compilation');
const { monacoTypecheckTask/* , monacoTypecheckWatchTask */ } = require('./gulpfile.editor');
const { compileExtensionsTask, watchExtensionsTask, compileExtensionMediaTask } = require('./gulpfile.extensions');
// API proposal names
gulp.task(compileApiProposalNamesTask);
gulp.task(watchApiProposalNamesTask);
// SWC Client Transpile
const transpileClientSWCTask = task.define('transpile-client-esbuild', task.series(util.rimraf('out'), transpileTask('src', 'out', true)));
gulp.task(transpileClientSWCTask);
// Transpile only
const transpileClientTask = task.define('transpile-client', task.series(util.rimraf('out'), transpileTask('src', 'out')));
gulp.task(transpileClientTask);
// Fast compile for development time
const compileClientTask = task.define('compile-client', task.series(util.rimraf('out'), compileApiProposalNamesTask, compileTask('src', 'out', false)));
gulp.task(compileClientTask);
const watchClientTask = task.define('watch-client', task.series(util.rimraf('out'), task.parallel(watchTask('out', false), watchApiProposalNamesTask)));
gulp.task(watchClientTask);
// All
const _compileTask = task.define('compile', task.parallel(monacoTypecheckTask, compileClientTask, compileExtensionsTask, compileExtensionMediaTask));
gulp.task(_compileTask);
gulp.task(task.define('watch', task.parallel(/* monacoTypecheckWatchTask, */ watchClientTask, watchExtensionsTask)));
// Default
gulp.task('default', _compileTask);
process.on('unhandledRejection', (reason, p) => {
console.log('Unhandled Rejection at: Promise', p, 'reason:', reason);
process.exit(1);
});
// Load all the gulpfiles only if running tasks other than the editor tasks
require('glob').sync('gulpfile.*.js', { cwd: __dirname })
.forEach(f => require(`./${f}`));
================================================
FILE: build/gulpfile.reh.js
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
'use strict';
const gulp = require('gulp');
const path = require('path');
const es = require('event-stream');
const util = require('./lib/util');
const { getVersion } = require('./lib/getVersion');
const task = require('./lib/task');
const optimize = require('./lib/optimize');
const { inlineMeta } = require('./lib/inlineMeta');
const product = require('../product.json');
const rename = require('gulp-rename');
const replace = require('gulp-replace');
const filter = require('gulp-filter');
const { getProductionDependencies } = require('./lib/dependencies');
const { readISODate } = require('./lib/date');
const vfs = require('vinyl-fs');
const packageJson = require('../package.json');
const flatmap = require('gulp-flatmap');
const gunzip = require('gulp-gunzip');
const File = require('vinyl');
const fs = require('fs');
const glob = require('glob');
const { compileBuildWithManglingTask } = require('./gulpfile.compile');
const { cleanExtensionsBuildTask, compileNonNativeExtensionsBuildTask, compileNativeExtensionsBuildTask, compileExtensionMediaBuildTask } = require('./gulpfile.extensions');
const { vscodeWebResourceIncludes, createVSCodeWebFileContentMapper } = require('./gulpfile.vscode.web');
const cp = require('child_process');
const log = require('fancy-log');
const buildfile = require('./buildfile');
const REPO_ROOT = path.dirname(__dirname);
const commit = getVersion(REPO_ROOT);
const BUILD_ROOT = path.dirname(REPO_ROOT);
const REMOTE_FOLDER = path.join(REPO_ROOT, 'remote');
// Targets
const BUILD_TARGETS = [
{ platform: 'win32', arch: 'x64' },
{ platform: 'win32', arch: 'arm64' },
{ platform: 'darwin', arch: 'x64' },
{ platform: 'darwin', arch: 'arm64' },
{ platform: 'linux', arch: 'x64' },
{ platform: 'linux', arch: 'armhf' },
{ platform: 'linux', arch: 'arm64' },
{ platform: 'alpine', arch: 'arm64' },
// legacy: we use to ship only one alpine so it was put in the arch, but now we ship
// multiple alpine images and moved to a better model (alpine as the platform)
{ platform: 'linux', arch: 'alpine' },
];
const serverResourceIncludes = [
// NLS
'out-build/nls.messages.json',
// Process monitor
'out-build/vs/base/node/cpuUsage.sh',
'out-build/vs/base/node/ps.sh',
// External Terminal
'out-build/vs/workbench/contrib/externalTerminal/**/*.scpt',
// Terminal shell integration
'out-build/vs/workbench/contrib/terminal/common/scripts/shellIntegration.ps1',
'out-build/vs/workbench/contrib/terminal/common/scripts/CodeTabExpansion.psm1',
'out-build/vs/workbench/contrib/terminal/common/scripts/GitTabExpansion.psm1',
'out-build/vs/workbench/contrib/terminal/common/scripts/shellIntegration-bash.sh',
'out-build/vs/workbench/contrib/terminal/common/scripts/shellIntegration-env.zsh',
'out-build/vs/workbench/contrib/terminal/common/scripts/shellIntegration-profile.zsh',
'out-build/vs/workbench/contrib/terminal/common/scripts/shellIntegration-rc.zsh',
'out-build/vs/workbench/contrib/terminal/common/scripts/shellIntegration-login.zsh',
'out-build/vs/workbench/contrib/terminal/common/scripts/shellIntegration.fish',
];
const serverResourceExcludes = [
'!out-build/vs/**/{electron-sandbox,electron-main,electron-utility}/**',
'!out-build/vs/editor/standalone/**',
'!out-build/vs/workbench/**/*-tb.png',
'!**/test/**'
];
const serverResources = [
...serverResourceIncludes,
...serverResourceExcludes
];
const serverWithWebResourceIncludes = [
...serverResourceIncludes,
'out-build/vs/code/browser/workbench/*.html',
...vscodeWebResourceIncludes
];
const serverWithWebResourceExcludes = [
...serverResourceExcludes,
'!out-build/vs/code/**/*-dev.html'
];
const serverWithWebResources = [
...serverWithWebResourceIncludes,
...serverWithWebResourceExcludes
];
const serverEntryPoints = buildfile.codeServer;
const webEntryPoints = [
buildfile.workerEditor,
buildfile.workerExtensionHost,
buildfile.workerNotebook,
buildfile.workerLanguageDetection,
buildfile.workerLocalFileSearch,
buildfile.workerOutputLinks,
buildfile.workerBackgroundTokenization,
buildfile.keyboardMaps,
buildfile.codeWeb
].flat();
const serverWithWebEntryPoints = [
// Include all of server
...serverEntryPoints,
// Include all of web
...webEntryPoints,
].flat();
const bootstrapEntryPoints = [
'out-build/server-main.js',
'out-build/server-cli.js',
'out-build/bootstrap-fork.js'
];
function getNodeVersion() {
const npmrc = fs.readFileSync(path.join(REPO_ROOT, 'remote', '.npmrc'), 'utf8');
const nodeVersion = /^target="(.*)"$/m.exec(npmrc)[1];
const internalNodeVersion = /^ms_build_id="(.*)"$/m.exec(npmrc)[1];
return { nodeVersion, internalNodeVersion };
}
function getNodeChecksum(expectedName) {
const nodeJsChecksums = fs.readFileSync(path.join(REPO_ROOT, 'build', 'checksums', 'nodejs.txt'), 'utf8');
for (const line of nodeJsChecksums.split('\n')) {
const [checksum, name] = line.split(/\s+/);
if (name === expectedName) {
return checksum;
}
}
return undefined;
}
function extractAlpinefromDocker(nodeVersion, platform, arch) {
const imageName = arch === 'arm64' ? 'arm64v8/node' : 'node';
log(`Downloading node.js ${nodeVersion} ${platform} ${arch} from docker image ${imageName}`);
const contents = cp.execSync(`docker run --rm ${imageName}:${nodeVersion}-alpine /bin/sh -c 'cat \`which node\`'`, { maxBuffer: 100 * 1024 * 1024, encoding: 'buffer' });
return es.readArray([new File({ path: 'node', contents, stat: { mode: parseInt('755', 8) } })]);
}
const { nodeVersion, internalNodeVersion } = getNodeVersion();
BUILD_TARGETS.forEach(({ platform, arch }) => {
gulp.task(task.define(`node-${platform}-${arch}`, () => {
const nodePath = path.join('.build', 'node', `v${nodeVersion}`, `${platform}-${arch}`);
if (!fs.existsSync(nodePath)) {
util.rimraf(nodePath);
return nodejs(platform, arch)
.pipe(vfs.dest(nodePath));
}
return Promise.resolve(null);
}));
});
const defaultNodeTask = gulp.task(`node-${process.platform}-${process.arch}`);
if (defaultNodeTask) {
gulp.task(task.define('node', defaultNodeTask));
}
function nodejs(platform, arch) {
const { fetchUrls, fetchGithub } = require('./lib/fetch');
const untar = require('gulp-untar');
if (arch === 'armhf') {
arch = 'armv7l';
} else if (arch === 'alpine') {
platform = 'alpine';
arch = 'x64';
}
log(`Downloading node.js ${nodeVersion} ${platform} ${arch} from ${product.nodejsRepository}...`);
const glibcPrefix = process.env['VSCODE_NODE_GLIBC'] ?? '';
let expectedName;
switch (platform) {
case 'win32':
expectedName = product.nodejsRepository !== 'https://nodejs.org' ?
`win-${arch}-node.exe` : `win-${arch}/node.exe`;
break;
case 'darwin':
expectedName = `node-v${nodeVersion}-${platform}-${arch}.tar.gz`;
break;
case 'linux':
expectedName = `node-v${nodeVersion}${glibcPrefix}-${platform}-${arch}.tar.gz`;
break;
case 'alpine':
expectedName = `node-v${nodeVersion}-linux-${arch}-musl.tar.gz`;
break;
}
const checksumSha256 = getNodeChecksum(expectedName);
if (checksumSha256) {
log(`Using SHA256 checksum for checking integrity: ${checksumSha256}`);
} else {
log.warn(`Unable to verify integrity of downloaded node.js binary because no SHA256 checksum was found!`);
}
switch (platform) {
case 'win32':
return (product.nodejsRepository !== 'https://nodejs.org' ?
fetchGithub(product.nodejsRepository, { version: `${nodeVersion}-${internalNodeVersion}`, name: expectedName, checksumSha256 }) :
fetchUrls(`/dist/v${nodeVersion}/win-${arch}/node.exe`, { base: 'https://nodejs.org', checksumSha256 }))
.pipe(rename('node.exe'));
case 'darwin':
case 'linux':
return (product.nodejsRepository !== 'https://nodejs.org' ?
fetchGithub(product.nodejsRepository, { version: `${nodeVersion}-${internalNodeVersion}`, name: expectedName, checksumSha256 }) :
fetchUrls(`/dist/v${nodeVersion}/node-v${nodeVersion}-${platform}-${arch}.tar.gz`, { base: 'https://nodejs.org', checksumSha256 })
).pipe(flatmap(stream => stream.pipe(gunzip()).pipe(untar())))
.pipe(filter('**/node'))
.pipe(util.setExecutableBit('**'))
.pipe(rename('node'));
case 'alpine':
return product.nodejsRepository !== 'https://nodejs.org' ?
fetchGithub(product.nodejsRepository, { version: `${nodeVersion}-${internalNodeVersion}`, name: expectedName, checksumSha256 })
.pipe(flatmap(stream => stream.pipe(gunzip()).pipe(untar())))
.pipe(filter('**/node'))
.pipe(util.setExecutableBit('**'))
.pipe(rename('node'))
: extractAlpinefromDocker(nodeVersion, platform, arch);
}
}
function packageTask(type, platform, arch, sourceFolderName, destinationFolderName) {
const destination = path.join(BUILD_ROOT, destinationFolderName);
return () => {
const json = require('gulp-json-editor');
const src = gulp.src(sourceFolderName + '/**', { base: '.' })
.pipe(rename(function (path) { path.dirname = path.dirname.replace(new RegExp('^' + sourceFolderName), 'out'); }))
.pipe(util.setExecutableBit(['**/*.sh']))
.pipe(filter(['**', '!**/*.js.map']));
const workspaceExtensionPoints = ['debuggers', 'jsonValidation'];
const isUIExtension = (manifest) => {
switch (manifest.extensionKind) {
case 'ui': return true;
case 'workspace': return false;
default: {
if (manifest.main) {
return false;
}
if (manifest.contributes && Object.keys(manifest.contributes).some(key => workspaceExtensionPoints.indexOf(key) !== -1)) {
return false;
}
// Default is UI Extension
return true;
}
}
};
const localWorkspaceExtensions = glob.sync('extensions/*/package.json')
.filter((extensionPath) => {
if (type === 'reh-web') {
return true; // web: ship all extensions for now
}
// Skip shipping UI extensions because the client side will have them anyways
// and they'd just increase the download without being used
const manifest = JSON.parse(fs.readFileSync(path.join(REPO_ROOT, extensionPath)).toString());
return !isUIExtension(manifest);
}).map((extensionPath) => path.basename(path.dirname(extensionPath)))
.filter(name => name !== 'vscode-api-tests' && name !== 'vscode-test-resolver'); // Do not ship the test extensions
const marketplaceExtensions = JSON.parse(fs.readFileSync(path.join(REPO_ROOT, 'product.json'), 'utf8')).builtInExtensions
.filter(entry => !entry.platforms || new Set(entry.platforms).has(platform))
.filter(entry => !entry.clientOnly)
.map(entry => entry.name);
const extensionPaths = [...localWorkspaceExtensions, ...marketplaceExtensions]
.map(name => `.build/extensions/${name}/**`);
const extensions = gulp.src(extensionPaths, { base: '.build', dot: true });
const extensionsCommonDependencies = gulp.src('.build/extensions/node_modules/**', { base: '.build', dot: true });
const sources = es.merge(src, extensions, extensionsCommonDependencies)
.pipe(filter(['**', '!**/*.js.map'], { dot: true }));
let version = packageJson.version;
const quality = product.quality;
if (quality && quality !== 'stable') {
version += '-' + quality;
}
const name = product.nameShort;
let packageJsonContents;
const packageJsonStream = gulp.src(['remote/package.json'], { base: 'remote' })
.pipe(json({ name, version, dependencies: undefined, optionalDependencies: undefined, type: 'module' }))
.pipe(es.through(function (file) {
packageJsonContents = file.contents.toString();
this.emit('data', file);
}));
let productJsonContents;
const productJsonStream = gulp.src(['product.json'], { base: '.' })
.pipe(json({ commit, date: readISODate('out-build'), version }))
.pipe(es.through(function (file) {
productJsonContents = file.contents.toString();
this.emit('data', file);
}));
const license = gulp.src(['remote/LICENSE'], { base: 'remote', allowEmpty: true });
const jsFilter = util.filter(data => !data.isDirectory() && /\.js$/.test(data.path));
const productionDependencies = getProductionDependencies(REMOTE_FOLDER);
const dependenciesSrc = productionDependencies.map(d => path.relative(REPO_ROOT, d)).map(d => [`${d}/**`, `!${d}/**/{test,tests}/**`, `!${d}/.bin/**`]).flat();
const deps = gulp.src(dependenciesSrc, { base: 'remote', dot: true })
// filter out unnecessary files, no source maps in server build
.pipe(filter(['**', '!**/package-lock.json', '!**/*.js.map']))
.pipe(util.cleanNodeModules(path.join(__dirname, '.moduleignore')))
.pipe(util.cleanNodeModules(path.join(__dirname, `.moduleignore.${process.platform}`)))
.pipe(jsFilter)
.pipe(util.stripSourceMappingURL())
.pipe(jsFilter.restore);
const nodePath = `.build/node/v${nodeVersion}/${platform}-${arch}`;
const node = gulp.src(`${nodePath}/**`, { base: nodePath, dot: true });
let web = [];
if (type === 'reh-web') {
web = [
'resources/server/favicon.ico',
'resources/server/code-192.png',
'resources/server/code-512.png',
'resources/server/manifest.json'
].map(resource => gulp.src(resource, { base: '.' }).pipe(rename(resource)));
}
const all = es.merge(
packageJsonStream,
productJsonStream,
license,
sources,
deps,
node,
...web
);
let result = all
.pipe(util.skipDirectories())
.pipe(util.fixWin32DirectoryPermissions());
if (platform === 'win32') {
result = es.merge(result,
gulp.src('resources/server/bin/remote-cli/code.cmd', { base: '.' })
.pipe(replace('@@VERSION@@', version))
.pipe(replace('@@COMMIT@@', commit))
.pipe(replace('@@APPNAME@@', product.applicationName))
.pipe(rename(`bin/remote-cli/${product.applicationName}.cmd`)),
gulp.src('resources/server/bin/helpers/browser.cmd', { base: '.' })
.pipe(replace('@@VERSION@@', version))
.pipe(replace('@@COMMIT@@', commit))
.pipe(replace('@@APPNAME@@', product.applicationName))
.pipe(rename(`bin/helpers/browser.cmd`)),
gulp.src('resources/server/bin/code-server.cmd', { base: '.' })
.pipe(rename(`bin/${product.serverApplicationName}.cmd`)),
);
} else if (platform === 'linux' || platform === 'alpine' || platform === 'darwin') {
result = es.merge(result,
gulp.src(`resources/server/bin/remote-cli/${platform === 'darwin' ? 'code-darwin.sh' : 'code-linux.sh'}`, { base: '.' })
.pipe(replace('@@VERSION@@', version))
.pipe(replace('@@COMMIT@@', commit))
.pipe(replace('@@APPNAME@@', product.applicationName))
.pipe(rename(`bin/remote-cli/${product.applicationName}`))
.pipe(util.setExecutableBit()),
gulp.src(`resources/server/bin/helpers/${platform === 'darwin' ? 'browser-darwin.sh' : 'browser-linux.sh'}`, { base: '.' })
.pipe(replace('@@VERSION@@', version))
.pipe(replace('@@COMMIT@@', commit))
.pipe(replace('@@APPNAME@@', product.applicationName))
.pipe(rename(`bin/helpers/browser.sh`))
.pipe(util.setExecutableBit()),
gulp.src(`resources/server/bin/${platform === 'darwin' ? 'code-server-darwin.sh' : 'code-server-linux.sh'}`, { base: '.' })
.pipe(rename(`bin/${product.serverApplicationName}`))
.pipe(util.setExecutableBit())
);
}
if (platform === 'linux' || platform === 'alpine') {
result = es.merge(result,
gulp.src(`resources/server/bin/helpers/check-requirements-linux.sh`, { base: '.' })
.pipe(rename(`bin/helpers/check-requirements.sh`))
.pipe(util.setExecutableBit())
);
}
result = inlineMeta(result, {
targetPaths: bootstrapEntryPoints,
packageJsonFn: () => packageJsonContents,
productJsonFn: () => productJsonContents
});
return result.pipe(vfs.dest(destination));
};
}
/**
* @param {object} product The parsed product.json file contents
*/
function tweakProductForServerWeb(product) {
const result = { ...product };
delete result.webEndpointUrlTemplate;
return result;
}
['reh', 'reh-web'].forEach(type => {
const bundleTask = task.define(`bundle-vscode-${type}`, task.series(
util.rimraf(`out-vscode-${type}`),
optimize.bundleTask(
{
out: `out-vscode-${type}`,
esm: {
src: 'out-build',
entryPoints: [
...(type === 'reh' ? serverEntryPoints : serverWithWebEntryPoints),
...bootstrapEntryPoints
],
resources: type === 'reh' ? serverResources : serverWithWebResources,
fileContentMapper: createVSCodeWebFileContentMapper('.build/extensions', type === 'reh-web' ? tweakProductForServerWeb(product) : product)
}
}
)
));
const minifyTask = task.define(`minify-vscode-${type}`, task.series(
bundleTask,
util.rimraf(`out-vscode-${type}-min`),
optimize.minifyTask(`out-vscode-${type}`, `https://main.vscode-cdn.net/sourcemaps/${commit}/core`)
));
gulp.task(minifyTask);
BUILD_TARGETS.forEach(buildTarget => {
const dashed = (str) => (str ? `-${str}` : ``);
const platform = buildTarget.platform;
const arch = buildTarget.arch;
['', 'min'].forEach(minified => {
const sourceFolderName = `out-vscode-${type}${dashed(minified)}`;
const destinationFolderName = `vscode-${type}${dashed(platform)}${dashed(arch)}`;
const serverTaskCI = task.define(`vscode-${type}${dashed(platform)}${dashed(arch)}${dashed(minified)}-ci`, task.series(
compileNativeExtensionsBuildTask,
gulp.task(`node-${platform}-${arch}`),
util.rimraf(path.join(BUILD_ROOT, destinationFolderName)),
packageTask(type, platform, arch, sourceFolderName, destinationFolderName)
));
gulp.task(serverTaskCI);
const serverTask = task.define(`vscode-${type}${dashed(platform)}${dashed(arch)}${dashed(minified)}`, task.series(
compileBuildWithManglingTask,
cleanExtensionsBuildTask,
compileNonNativeExtensionsBuildTask,
compileExtensionMediaBuildTask,
minified ? minifyTask : bundleTask,
serverTaskCI
));
gulp.task(serverTask);
});
});
});
================================================
FILE: build/gulpfile.scan.js
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
'use strict';
const gulp = require('gulp');
const path = require('path');
const task = require('./lib/task');
const util = require('./lib/util');
const electron = require('@vscode/gulp-electron');
const { config } = require('./lib/electron');
const filter = require('gulp-filter');
const deps = require('./lib/dependencies');
const { existsSync, readdirSync } = require('fs');
const root = path.dirname(__dirname);
const BUILD_TARGETS = [
{ platform: 'win32', arch: 'x64' },
{ platform: 'win32', arch: 'arm64' },
{ platform: 'darwin', arch: null, opts: { stats: true } },
{ platform: 'linux', arch: 'x64' },
{ platform: 'linux', arch: 'armhf' },
{ platform: 'linux', arch: 'arm64' },
];
// The following files do not have PDBs downloaded for them during the download symbols process.
const excludedCheckList = ['d3dcompiler_47.dll'];
BUILD_TARGETS.forEach(buildTarget => {
const dashed = (/** @type {string | null} */ str) => (str ? `-${str}` : ``);
const platform = buildTarget.platform;
const arch = buildTarget.arch;
const destinationExe = path.join(path.dirname(root), 'scanbin', `VSCode${dashed(platform)}${dashed(arch)}`, 'bin');
const destinationPdb = path.join(path.dirname(root), 'scanbin', `VSCode${dashed(platform)}${dashed(arch)}`, 'pdb');
const tasks = [];
// removal tasks
tasks.push(util.rimraf(destinationExe), util.rimraf(destinationPdb));
// electron
tasks.push(() => electron.dest(destinationExe, { ...config, platform, arch: arch === 'armhf' ? 'arm' : arch }));
// pdbs for windows
if (platform === 'win32') {
tasks.push(
() => electron.dest(destinationPdb, { ...config, platform, arch: arch === 'armhf' ? 'arm' : arch, pdbs: true }),
() => confirmPdbsExist(destinationExe, destinationPdb)
);
}
if (platform === 'linux') {
tasks.push(
() => electron.dest(destinationPdb, { ...config, platform, arch: arch === 'armhf' ? 'arm' : arch, symbols: true })
);
}
// node modules
tasks.push(
nodeModules(destinationExe, destinationPdb, platform)
);
const setupSymbolsTask = task.define(`vscode-symbols${dashed(platform)}${dashed(arch)}`,
task.series(...tasks)
);
gulp.task(setupSymbolsTask);
});
function getProductionDependencySources() {
const productionDependencies = deps.getProductionDependencies(root);
return productionDependencies.map(d => path.relative(root, d)).map(d => [`${d}/**`, `!${d}/**/{test,tests}/**`]).flat();
}
function nodeModules(destinationExe, destinationPdb, platform) {
const exe = () => {
return gulp.src(getProductionDependencySources(), { base: '.', dot: true })
.pipe(filter([
'**/*.node',
// Exclude these paths.
// We don't build the prebuilt node files so we don't scan them
'!**/prebuilds/**/*.node',
// These are 3rd party modules that we should ignore
'!**/@parcel/watcher/**/*']))
.pipe(gulp.dest(destinationExe));
};
if (platform === 'win32') {
const pdb = () => {
return gulp.src(getProductionDependencySources(), { base: '.', dot: true })
.pipe(filter(['**/*.pdb']))
.pipe(gulp.dest(destinationPdb));
};
return gulp.parallel(exe, pdb);
}
if (platform === 'linux') {
const pdb = () => {
return gulp.src(getProductionDependencySources(), { base: '.', dot: true })
.pipe(filter(['**/*.sym']))
.pipe(gulp.dest(destinationPdb));
};
return gulp.parallel(exe, pdb);
}
return exe;
}
function confirmPdbsExist(destinationExe, destinationPdb) {
readdirSync(destinationExe).forEach(file => {
if (excludedCheckList.includes(file)) {
return;
}
if (file.endsWith('.dll') || file.endsWith('.exe')) {
const pdb = `${file}.pdb`;
if (!existsSync(path.join(destinationPdb, pdb))) {
throw new Error(`Missing pdb file for ${file}. Tried searching for ${pdb} in ${destinationPdb}.`);
}
}
});
return Promise.resolve();
}
================================================
FILE: build/gulpfile.vscode.js
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
'use strict';
const gulp = require('gulp');
const fs = require('fs');
const path = require('path');
const es = require('event-stream');
const vfs = require('vinyl-fs');
const rename = require('gulp-rename');
const replace = require('gulp-replace');
const filter = require('gulp-filter');
const util = require('./lib/util');
const { getVersion } = require('./lib/getVersion');
const { readISODate } = require('./lib/date');
const task = require('./lib/task');
const buildfile = require('./buildfile');
const optimize = require('./lib/optimize');
const { inlineMeta } = require('./lib/inlineMeta');
const root = path.dirname(__dirname);
const commit = getVersion(root);
const packageJson = require('../package.json');
const product = require('../product.json');
const crypto = require('crypto');
const i18n = require('./lib/i18n');
const { getProductionDependencies } = require('./lib/dependencies');
const { config } = require('./lib/electron');
const createAsar = require('./lib/asar').createAsar;
const minimist = require('minimist');
const { compileBuildWithoutManglingTask, compileBuildWithManglingTask } = require('./gulpfile.compile');
const { compileNonNativeExtensionsBuildTask, compileNativeExtensionsBuildTask, compileAllExtensionsBuildTask, compileExtensionMediaBuildTask, cleanExtensionsBuildTask } = require('./gulpfile.extensions');
const { promisify } = require('util');
const glob = promisify(require('glob'));
const rcedit = promisify(require('rcedit'));
// Build
const vscodeEntryPoints = [
buildfile.workerEditor,
buildfile.workerExtensionHost,
buildfile.workerNotebook,
buildfile.workerLanguageDetection,
buildfile.workerLocalFileSearch,
buildfile.workerProfileAnalysis,
buildfile.workerOutputLinks,
buildfile.workerBackgroundTokenization,
buildfile.workbenchDesktop,
buildfile.code
].flat();
const vscodeResourceIncludes = [
// NLS
'out-build/nls.messages.json',
'out-build/nls.keys.json',
// Workbench
'out-build/vs/code/electron-sandbox/workbench/workbench.html',
// Electron Preload
'out-build/vs/base/parts/sandbox/electron-sandbox/preload.js',
'out-build/vs/base/parts/sandbox/electron-sandbox/preload-aux.js',
// Node Scripts
'out-build/vs/base/node/{terminateProcess.sh,cpuUsage.sh,ps.sh}',
// Touchbar
'out-build/vs/workbench/browser/parts/editor/media/*.png',
'out-build/vs/workbench/contrib/debug/browser/media/*.png',
// External Terminal
'out-build/vs/workbench/contrib/externalTerminal/**/*.scpt',
// Terminal shell integration
'out-build/vs/workbench/contrib/terminal/common/scripts/*.fish',
'out-build/vs/workbench/contrib/terminal/common/scripts/*.ps1',
'out-build/vs/workbench/contrib/terminal/common/scripts/*.psm1',
'out-build/vs/workbench/contrib/terminal/common/scripts/*.sh',
'out-build/vs/workbench/contrib/terminal/common/scripts/*.zsh',
// Accessibility Signals
'out-build/vs/platform/accessibilitySignal/browser/media/*.mp3',
// Welcome
'out-build/vs/workbench/contrib/welcomeGettingStarted/common/media/**/*.{svg,png}',
// Extensions
'out-build/vs/workbench/contrib/extensions/browser/media/{theme-icon.png,language-icon.svg}',
'out-build/vs/workbench/services/extensionManagement/common/media/*.{svg,png}',
// Webview
'out-build/vs/workbench/contrib/webview/browser/pre/*.{js,html}',
// Extension Host Worker
'out-build/vs/workbench/services/extensions/worker/webWorkerExtensionHostIframe.html',
// Process Explorer
'out-build/vs/code/electron-sandbox/processExplorer/processExplorer.html',
// Tree Sitter highlights
'out-build/vs/editor/common/languages/highlights/*.scm',
// Tree Sitter injection queries
'out-build/vs/editor/common/languages/injections/*.scm',
];
const vscodeResources = [
// Includes
...vscodeResourceIncludes,
// Excludes
'!out-build/vs/code/browser/**',
'!out-build/vs/editor/standalone/**',
'!out-build/vs/code/**/*-dev.html',
'!out-build/vs/workbench/contrib/issue/**/*-dev.html',
'!**/test/**'
];
const bootstrapEntryPoints = [
'out-build/main.js',
'out-build/cli.js',
'out-build/bootstrap-fork.js'
];
const bundleVSCodeTask = task.define('bundle-vscode', task.series(
util.rimraf('out-vscode'),
// Optimize: bundles source files automatically based on
// import statements based on the passed in entry points.
// In addition, concat window related bootstrap files into
// a single file.
optimize.bundleTask(
{
out: 'out-vscode',
esm: {
src: 'out-build',
entryPoints: [
...vscodeEntryPoints,
...bootstrapEntryPoints
],
resources: vscodeResources,
fileContentMapper: filePath => {
if (
filePath.endsWith('vs/code/electron-sandbox/workbench/workbench.js') ||
filePath.endsWith('vs/code/electron-sandbox/processExplorer/processExplorer.js')) {
return async (content) => {
const bootstrapWindowContent = await fs.promises.readFile(path.join(root, 'out-build', 'bootstrap-window.js'), 'utf-8');
return `${bootstrapWindowContent}\n${content}`; // prepend bootstrap-window.js content to entry points that are Electron windows
};
}
return undefined;
},
skipTSBoilerplateRemoval: entryPoint =>
entryPoint === 'vs/code/electron-sandbox/workbench/workbench' ||
entryPoint === 'vs/code/electron-sandbox/processExplorer/processExplorer',
}
}
)
));
gulp.task(bundleVSCodeTask);
const sourceMappingURLBase = `https://main.vscode-cdn.net/sourcemaps/${commit}`;
const minifyVSCodeTask = task.define('minify-vscode', task.series(
bundleVSCodeTask,
util.rimraf('out-vscode-min'),
optimize.minifyTask('out-vscode', `${sourceMappingURLBase}/core`)
));
gulp.task(minifyVSCodeTask);
const coreCI = task.define('core-ci', task.series(
gulp.task('compile-build-with-mangling'),
task.parallel(
gulp.task('minify-vscode'),
gulp.task('minify-vscode-reh'),
gulp.task('minify-vscode-reh-web'),
)
));
gulp.task(coreCI);
const coreCIPR = task.define('core-ci-pr', task.series(
gulp.task('compile-build-without-mangling'),
task.parallel(
gulp.task('minify-vscode'),
gulp.task('minify-vscode-reh'),
gulp.task('minify-vscode-reh-web'),
)
));
gulp.task(coreCIPR);
/**
* Compute checksums for some files.
*
* @param {string} out The out folder to read the file from.
* @param {string[]} filenames The paths to compute a checksum for.
* @return {Object} A map of paths to checksums.
*/
function computeChecksums(out, filenames) {
const result = {};
filenames.forEach(function (filename) {
const fullPath = path.join(process.cwd(), out, filename);
result[filename] = computeChecksum(fullPath);
});
return result;
}
/**
* Compute checksum for a file.
*
* @param {string} filename The absolute path to a filename.
* @return {string} The checksum for `filename`.
*/
function computeChecksum(filename) {
const contents = fs.readFileSync(filename);
const hash = crypto
.createHash('sha256')
.update(contents)
.digest('base64')
.replace(/=+$/, '');
return hash;
}
function packageTask(platform, arch, sourceFolderName, destinationFolderName, opts) {
opts = opts || {};
const destination = path.join(path.dirname(root), destinationFolderName);
platform = platform || process.platform;
return () => {
const electron = require('@vscode/gulp-electron');
const json = require('gulp-json-editor');
const out = sourceFolderName;
const checksums = computeChecksums(out, [
'vs/base/parts/sandbox/electron-sandbox/preload.js',
'vs/workbench/workbench.desktop.main.js',
'vs/workbench/workbench.desktop.main.css',
'vs/workbench/api/node/extensionHostProcess.js',
'vs/code/electron-sandbox/workbench/workbench.html',
'vs/code/electron-sandbox/workbench/workbench.js'
]);
const src = gulp.src(out + '/**', { base: '.' })
.pipe(rename(function (path) { path.dirname = path.dirname.replace(new RegExp('^' + out), 'out'); }))
.pipe(util.setExecutableBit(['**/*.sh']));
const platformSpecificBuiltInExtensionsExclusions = product.builtInExtensions.filter(ext => {
if (!ext.platforms) {
return false;
}
const set = new Set(ext.platforms);
return !set.has(platform);
}).map(ext => `!.build/extensions/${ext.name}/**`);
const extensions = gulp.src(['.build/extensions/**', ...platformSpecificBuiltInExtensionsExclusions], { base: '.build', dot: true });
const sources = es.merge(src, extensions)
.pipe(filter(['**', '!**/*.js.map'], { dot: true }));
let version = packageJson.version;
const quality = product.quality;
if (quality && quality !== 'stable') {
version += '-' + quality;
}
const name = product.nameShort;
const packageJsonUpdates = { name, version };
if (platform === 'linux') {
packageJsonUpdates.desktopName = `${product.applicationName}.desktop`;
}
let packageJsonContents;
const packageJsonStream = gulp.src(['package.json'], { base: '.' })
.pipe(json(packageJsonUpdates))
.pipe(es.through(function (file) {
packageJsonContents = file.contents.toString();
this.emit('data', file);
}));
let productJsonContents;
const productJsonStream = gulp.src(['product.json'], { base: '.' })
.pipe(json({ commit, date: readISODate('out-build'), checksums, version }))
.pipe(es.through(function (file) {
productJsonContents = file.contents.toString();
this.emit('data', file);
}));
const license = gulp.src([product.licenseFileName, 'ThirdPartyNotices.txt', 'licenses/**'], { base: '.', allowEmpty: true });
// TODO the API should be copied to `out` during compile, not here
const api = gulp.src('src/vscode-dts/vscode.d.ts').pipe(rename('out/vscode-dts/vscode.d.ts'));
const telemetry = gulp.src('.build/telemetry/**', { base: '.build/telemetry', dot: true });
const jsFilter = util.filter(data => !data.isDirectory() && /\.js$/.test(data.path));
const root = path.resolve(path.join(__dirname, '..'));
const productionDependencies = getProductionDependencies(root);
const dependenciesSrc = productionDependencies.map(d => path.relative(root, d)).map(d => [`${d}/**`, `!${d}/**/{test,tests}/**`]).flat().concat('!**/*.mk');
const deps = gulp.src(dependenciesSrc, { base: '.', dot: true })
.pipe(filter(['**', `!**/${config.version}/**`, '!**/bin/darwin-arm64-87/**', '!**/package-lock.json', '!**/yarn.lock', '!**/*.js.map']))
.pipe(util.cleanNodeModules(path.join(__dirname, '.moduleignore')))
.pipe(util.cleanNodeModules(path.join(__dirname, `.moduleignore.${process.platform}`)))
.pipe(jsFilter)
.pipe(util.rewriteSourceMappingURL(sourceMappingURLBase))
.pipe(jsFilter.restore)
.pipe(createAsar(path.join(process.cwd(), 'node_modules'), [
'**/*.node',
'**/@vscode/ripgrep/bin/*',
'**/node-pty/build/Release/*',
'**/node-pty/build/Release/conpty/*',
'**/node-pty/lib/worker/conoutSocketWorker.js',
'**/node-pty/lib/shared/conout.js',
'**/*.wasm',
'**/@vscode/vsce-sign/bin/*',
], [
'**/*.mk',
'!node_modules/vsda/**' // stay compatible with extensions that depend on us shipping `vsda` into ASAR
], [
'node_modules/vsda/**' // retain copy of `vsda` in node_modules for internal use
], 'node_modules.asar'));
let all = es.merge(
packageJsonStream,
productJsonStream,
license,
api,
telemetry,
sources,
deps
);
if (platform === 'win32') {
all = es.merge(all, gulp.src([
'resources/win32/bower.ico',
'resources/win32/c.ico',
'resources/win32/config.ico',
'resources/win32/cpp.ico',
'resources/win32/csharp.ico',
'resources/win32/css.ico',
'resources/win32/default.ico',
'resources/win32/go.ico',
'resources/win32/html.ico',
'resources/win32/jade.ico',
'resources/win32/java.ico',
'resources/win32/javascript.ico',
'resources/win32/json.ico',
'resources/win32/less.ico',
'resources/win32/markdown.ico',
'resources/win32/php.ico',
'resources/win32/powershell.ico',
'resources/win32/python.ico',
'resources/win32/react.ico',
'resources/win32/ruby.ico',
'resources/win32/sass.ico',
'resources/win32/shell.ico',
'resources/win32/sql.ico',
'resources/win32/typescript.ico',
'resources/win32/vue.ico',
'resources/win32/xml.ico',
'resources/win32/yaml.ico',
'resources/win32/code_70x70.png',
'resources/win32/code_150x150.png'
], { base: '.' }));
} else if (platform === 'linux') {
all = es.merge(all, gulp.src('resources/linux/code.png', { base: '.' }));
} else if (platform === 'darwin') {
const shortcut = gulp.src('resources/darwin/bin/code.sh')
.pipe(replace('@@APPNAME@@', product.applicationName))
.pipe(rename('bin/code'));
const policyDest = gulp.src('.build/policies/darwin/**', { base: '.build/policies/darwin' })
.pipe(rename(f => f.dirname = `policies/${f.dirname}`));
all = es.merge(all, shortcut, policyDest);
}
let result = all
.pipe(util.skipDirectories())
.pipe(util.fixWin32DirectoryPermissions())
.pipe(filter(['**', '!**/.github/**'], { dot: true })) // https://github.com/microsoft/vscode/issues/116523
.pipe(electron({ ...config, platform, arch: arch === 'armhf' ? 'arm' : arch, ffmpegChromium: false }))
.pipe(filter(['**', '!LICENSE', '!version'], { dot: true }));
if (platform === 'linux') {
result = es.merge(result, gulp.src('resources/completions/bash/code', { base: '.' })
.pipe(replace('@@APPNAME@@', product.applicationName))
.pipe(rename(function (f) { f.basename = product.applicationName; })));
result = es.merge(result, gulp.src('resources/completions/zsh/_code', { base: '.' })
.pipe(replace('@@APPNAME@@', product.applicationName))
.pipe(rename(function (f) { f.basename = '_' + product.applicationName; })));
}
if (platform === 'win32') {
result = es.merge(result, gulp.src('resources/win32/bin/code.js', { base: 'resources/win32', allowEmpty: true }));
result = es.merge(result, gulp.src('resources/win32/bin/code.cmd', { base: 'resources/win32' })
.pipe(replace('@@NAME@@', product.nameShort))
.pipe(rename(function (f) { f.basename = product.applicationName; })));
result = es.merge(result, gulp.src('resources/win32/bin/code.sh', { base: 'resources/win32' })
.pipe(replace('@@NAME@@', product.nameShort))
.pipe(replace('@@PRODNAME@@', product.nameLong))
.pipe(replace('@@VERSION@@', version))
.pipe(replace('@@COMMIT@@', commit))
.pipe(replace('@@APPNAME@@', product.applicationName))
.pipe(replace('@@SERVERDATAFOLDER@@', product.serverDataFolderName || '.vscode-remote'))
.pipe(replace('@@QUALITY@@', quality))
.pipe(rename(function (f) { f.basename = product.applicationName; f.extname = ''; })));
result = es.merge(result, gulp.src('resources/win32/VisualElementsManifest.xml', { base: 'resources/win32' })
.pipe(rename(product.nameShort + '.VisualElementsManifest.xml')));
result = es.merge(result, gulp.src('.build/policies/win32/**', { base: '.build/policies/win32' })
.pipe(rename(f => f.dirname = `policies/${f.dirname}`)));
if (quality === 'insider') {
result = es.merge(result, gulp.src('.build/win32/appx/**', { base: '.build/win32' }));
}
} else if (platform === 'linux') {
result = es.merge(result, gulp.src('resources/linux/bin/code.sh', { base: '.' })
.pipe(replace('@@PRODNAME@@', product.nameLong))
.pipe(replace('@@APPNAME@@', product.applicationName))
.pipe(rename('bin/' + product.applicationName)));
}
result = inlineMeta(result, {
targetPaths: bootstrapEntryPoints,
packageJsonFn: () => packageJsonContents,
productJsonFn: () => productJsonContents
});
return result.pipe(vfs.dest(destination));
};
}
function patchWin32DependenciesTask(destinationFolderName) {
const cwd = path.join(path.dirname(root), destinationFolderName);
return async () => {
const deps = await glob('**/*.node', { cwd, ignore: 'extensions/node_modules/@parcel/watcher/**' });
const packageJson = JSON.parse(await fs.promises.readFile(path.join(cwd, 'resources', 'app', 'package.json'), 'utf8'));
const product = JSON.parse(await fs.promises.readFile(path.join(cwd, 'resources', 'app', 'product.json'), 'utf8'));
const baseVersion = packageJson.version.replace(/-.*$/, '');
await Promise.all(deps.map(async dep => {
const basename = path.basename(dep);
await rcedit(path.join(cwd, dep), {
'file-version': baseVersion,
'version-string': {
'CompanyName': 'Microsoft Corporation',
'FileDescription': product.nameLong,
'FileVersion': packageJson.version,
'InternalName': basename,
'LegalCopyright': 'Copyright (C) 2022 Microsoft. All rights reserved',
'OriginalFilename': basename,
'ProductName': product.nameLong,
'ProductVersion': packageJson.version,
}
});
}));
};
}
const buildRoot = path.dirname(root);
const BUILD_TARGETS = [
{ platform: 'win32', arch: 'x64' },
{ platform: 'win32', arch: 'arm64' },
{ platform: 'darwin', arch: 'x64', opts: { stats: true } },
{ platform: 'darwin', arch: 'arm64', opts: { stats: true } },
{ platform: 'linux', arch: 'x64' },
{ platform: 'linux', arch: 'armhf' },
{ platform: 'linux', arch: 'arm64' },
];
BUILD_TARGETS.forEach(buildTarget => {
const dashed = (str) => (str ? `-${str}` : ``);
const platform = buildTarget.platform;
const arch = buildTarget.arch;
const opts = buildTarget.opts;
const [vscode, vscodeMin] = ['', 'min'].map(minified => {
const sourceFolderName = `out-vscode${dashed(minified)}`;
const destinationFolderName = `VSCode${dashed(platform)}${dashed(arch)}`;
const tasks = [
compileNativeExtensionsBuildTask,
util.rimraf(path.join(buildRoot, destinationFolderName)),
packageTask(platform, arch, sourceFolderName, destinationFolderName, opts)
];
if (platform === 'win32') {
tasks.push(patchWin32DependenciesTask(destinationFolderName));
}
const vscodeTaskCI = task.define(`vscode${dashed(platform)}${dashed(arch)}${dashed(minified)}-ci`, task.series(...tasks));
gulp.task(vscodeTaskCI);
const vscodeTask = task.define(`vscode${dashed(platform)}${dashed(arch)}${dashed(minified)}`, task.series(
minified ? compileBuildWithManglingTask : compileBuildWithoutManglingTask,
cleanExtensionsBuildTask,
compileNonNativeExtensionsBuildTask,
compileExtensionMediaBuildTask,
minified ? minifyVSCodeTask : bundleVSCodeTask,
vscodeTaskCI
));
gulp.task(vscodeTask);
return vscodeTask;
});
if (process.platform === platform && process.arch === arch) {
gulp.task(task.define('vscode', task.series(vscode)));
gulp.task(task.define('vscode-min', task.series(vscodeMin)));
}
});
// #region nls
const innoSetupConfig = {
'zh-cn': { codePage: 'CP936', defaultInfo: { name: 'Simplified Chinese', id: '$0804', } },
'zh-tw': { codePage: 'CP950', defaultInfo: { name: 'Traditional Chinese', id: '$0404' } },
'ko': { codePage: 'CP949', defaultInfo: { name: 'Korean', id: '$0412' } },
'ja': { codePage: 'CP932' },
'de': { codePage: 'CP1252' },
'fr': { codePage: 'CP1252' },
'es': { codePage: 'CP1252' },
'ru': { codePage: 'CP1251' },
'it': { codePage: 'CP1252' },
'pt-br': { codePage: 'CP1252' },
'hu': { codePage: 'CP1250' },
'tr': { codePage: 'CP1254' }
};
gulp.task(task.define(
'vscode-translations-export',
task.series(
coreCI,
compileAllExtensionsBuildTask,
function () {
const pathToMetadata = './out-build/nls.metadata.json';
const pathToExtensions = '.build/extensions/*';
const pathToSetup = 'build/win32/i18n/messages.en.isl';
return es.merge(
gulp.src(pathToMetadata).pipe(i18n.createXlfFilesForCoreBundle()),
gulp.src(pathToSetup).pipe(i18n.createXlfFilesForIsl()),
gulp.src(pathToExtensions).pipe(i18n.createXlfFilesForExtensions())
).pipe(vfs.dest('../vscode-translations-export'));
}
)
));
gulp.task('vscode-translations-import', function () {
const options = minimist(process.argv.slice(2), {
string: 'location',
default: {
location: '../vscode-translations-import'
}
});
return es.merge([...i18n.defaultLanguages, ...i18n.extraLanguages].map(language => {
const id = language.id;
return gulp.src(`${options.location}/${id}/vscode-setup/messages.xlf`)
.pipe(i18n.prepareIslFiles(language, innoSetupConfig[language.id]))
.pipe(vfs.dest(`./build/win32/i18n`));
}));
});
// #endregion
================================================
FILE: build/gulpfile.vscode.linux.js
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
'use strict';
const gulp = require('gulp');
const replace = require('gulp-replace');
const rename = require('gulp-rename');
const es = require('event-stream');
const vfs = require('vinyl-fs');
const { rimraf } = require('./lib/util');
const { getVersion } = require('./lib/getVersion');
const task = require('./lib/task');
const packageJson = require('../package.json');
const product = require('../product.json');
const dependenciesGenerator = require('./linux/dependencies-generator');
const debianRecommendedDependencies = require('./linux/debian/dep-lists').recommendedDeps;
const path = require('path');
const cp = require('child_process');
const util = require('util');
const exec = util.promisify(cp.exec);
const root = path.dirname(__dirname);
const commit = getVersion(root);
const linuxPackageRevision = Math.floor(new Date().getTime() / 1000);
/**
* @param {string} arch
*/
function getDebPackageArch(arch) {
return { x64: 'amd64', armhf: 'armhf', arm64: 'arm64' }[arch];
}
function prepareDebPackage(arch) {
const binaryDir = '../VSCode-linux-' + arch;
const debArch = getDebPackageArch(arch);
const destination = '.build/linux/deb/' + debArch + '/' + product.applicationName + '-' + debArch;
return async function () {
const dependencies = await dependenciesGenerator.getDependencies('deb', binaryDir, product.applicationName, debArch);
const desktop = gulp.src('resources/linux/code.desktop', { base: '.' })
.pipe(rename('usr/share/applications/' + product.applicationName + '.desktop'));
const desktopUrlHandler = gulp.src('resources/linux/code-url-handler.desktop', { base: '.' })
.pipe(rename('usr/share/applications/' + product.applicationName + '-url-handler.desktop'));
const desktops = es.merge(desktop, desktopUrlHandler)
.pipe(replace('@@NAME_LONG@@', product.nameLong))
.pipe(replace('@@NAME_SHORT@@', product.nameShort))
.pipe(replace('@@NAME@@', product.applicationName))
.pipe(replace('@@EXEC@@', `/usr/share/${product.applicationName}/${product.applicationName}`))
.pipe(replace('@@ICON@@', product.linuxIconName))
.pipe(replace('@@URLPROTOCOL@@', product.urlProtocol));
const appdata = gulp.src('resources/linux/code.appdata.xml', { base: '.' })
.pipe(replace('@@NAME_LONG@@', product.nameLong))
.pipe(replace('@@NAME@@', product.applicationName))
.pipe(replace('@@LICENSE@@', product.licenseName))
.pipe(rename('usr/share/appdata/' + product.applicationName + '.appdata.xml'));
const workspaceMime = gulp.src('resources/linux/code-workspace.xml', { base: '.' })
.pipe(replace('@@NAME_LONG@@', product.nameLong))
.pipe(replace('@@NAME@@', product.applicationName))
.pipe(rename('usr/share/mime/packages/' + product.applicationName + '-workspace.xml'));
const icon = gulp.src('resources/linux/code.png', { base: '.' })
.pipe(rename('usr/share/pixmaps/' + product.linuxIconName + '.png'));
const bash_completion = gulp.src('resources/completions/bash/code')
.pipe(replace('@@APPNAME@@', product.applicationName))
.pipe(rename('usr/share/bash-completion/completions/' + product.applicationName));
const zsh_completion = gulp.src('resources/completions/zsh/_code')
.pipe(replace('@@APPNAME@@', product.applicationName))
.pipe(rename('usr/share/zsh/vendor-completions/_' + product.applicationName));
const code = gulp.src(binaryDir + '/**/*', { base: binaryDir })
.pipe(rename(function (p) { p.dirname = 'usr/share/' + product.applicationName + '/' + p.dirname; }));
let size = 0;
const control = code.pipe(es.through(
function (f) { size += f.isDirectory() ? 4096 : f.contents.length; },
function () {
const that = this;
gulp.src('resources/linux/debian/control.template', { base: '.' })
.pipe(replace('@@NAME@@', product.applicationName))
.pipe(replace('@@VERSION@@', packageJson.version + '-' + linuxPackageRevision))
.pipe(replace('@@ARCHITECTURE@@', debArch))
.pipe(replace('@@DEPENDS@@', dependencies.join(', ')))
.pipe(replace('@@RECOMMENDS@@', debianRecommendedDependencies.join(', ')))
.pipe(replace('@@INSTALLEDSIZE@@', Math.ceil(size / 1024)))
.pipe(rename('DEBIAN/control'))
.pipe(es.through(function (f) { that.emit('data', f); }, function () { that.emit('end'); }));
}));
const prerm = gulp.src('resources/linux/debian/prerm.template', { base: '.' })
.pipe(replace('@@NAME@@', product.applicationName))
.pipe(rename('DEBIAN/prerm'));
const postrm = gulp.src('resources/linux/debian/postrm.template', { base: '.' })
.pipe(replace('@@NAME@@', product.applicationName))
.pipe(rename('DEBIAN/postrm'));
const postinst = gulp.src('resources/linux/debian/postinst.template', { base: '.' })
.pipe(replace('@@NAME@@', product.applicationName))
.pipe(rename('DEBIAN/postinst'));
const templates = gulp.src('resources/linux/debian/templates.template', { base: '.' })
.pipe(replace('@@NAME@@', product.applicationName))
.pipe(rename('DEBIAN/templates'));
const all = es.merge(control, templates, postinst, postrm, prerm, desktops, appdata, workspaceMime, icon, bash_completion, zsh_completion, code);
return all.pipe(vfs.dest(destination));
};
}
/**
* @param {string} arch
*/
function buildDebPackage(arch) {
const debArch = getDebPackageArch(arch);
const cwd = `.build/linux/deb/${debArch}`;
return async () => {
await exec(`chmod 755 ${product.applicationName}-${debArch}/DEBIAN/postinst ${product.applicationName}-${debArch}/DEBIAN/prerm ${product.applicationName}-${debArch}/DEBIAN/postrm`, { cwd });
await exec('mkdir -p deb', { cwd });
await exec(`fakeroot dpkg-deb -Zxz -b ${product.applicationName}-${debArch} deb`, { cwd });
};
}
/**
* @param {string} rpmArch
*/
function getRpmBuildPath(rpmArch) {
return '.build/linux/rpm/' + rpmArch + '/rpmbuild';
}
/**
* @param {string} arch
*/
function getRpmPackageArch(arch) {
return { x64: 'x86_64', armhf: 'armv7hl', arm64: 'aarch64' }[arch];
}
/**
* @param {string} arch
*/
function prepareRpmPackage(arch) {
const binaryDir = '../VSCode-linux-' + arch;
const rpmArch = getRpmPackageArch(arch);
const stripBinary = process.env['STRIP'] ?? '/usr/bin/strip';
return async function () {
const dependencies = await dependenciesGenerator.getDependencies('rpm', binaryDir, product.applicationName, rpmArch);
const desktop = gulp.src('resources/linux/code.desktop', { base: '.' })
.pipe(rename('BUILD/usr/share/applications/' + product.applicationName + '.desktop'));
const desktopUrlHandler = gulp.src('resources/linux/code-url-handler.desktop', { base: '.' })
.pipe(rename('BUILD/usr/share/applications/' + product.applicationName + '-url-handler.desktop'));
const desktops = es.merge(desktop, desktopUrlHandler)
.pipe(replace('@@NAME_LONG@@', product.nameLong))
.pipe(replace('@@NAME_SHORT@@', product.nameShort))
.pipe(replace('@@NAME@@', product.applicationName))
.pipe(replace('@@EXEC@@', `/usr/share/${product.applicationName}/${product.applicationName}`))
.pipe(replace('@@ICON@@', product.linuxIconName))
.pipe(replace('@@URLPROTOCOL@@', product.urlProtocol));
const appdata = gulp.src('resources/linux/code.appdata.xml', { base: '.' })
.pipe(replace('@@NAME_LONG@@', product.nameLong))
.pipe(replace('@@NAME@@', product.applicationName))
.pipe(replace('@@LICENSE@@', product.licenseName))
.pipe(rename('BUILD/usr/share/appdata/' + product.applicationName + '.appdata.xml'));
const workspaceMime = gulp.src('resources/linux/code-workspace.xml', { base: '.' })
.pipe(replace('@@NAME_LONG@@', product.nameLong))
.pipe(replace('@@NAME@@', product.applicationName))
.pipe(rename('BUILD/usr/share/mime/packages/' + product.applicationName + '-workspace.xml'));
const icon = gulp.src('resources/linux/code.png', { base: '.' })
.pipe(rename('BUILD/usr/share/pixmaps/' + product.linuxIconName + '.png'));
const bash_completion = gulp.src('resources/completions/bash/code')
.pipe(replace('@@APPNAME@@', product.applicationName))
.pipe(rename('BUILD/usr/share/bash-completion/completions/' + product.applicationName));
const zsh_completion = gulp.src('resources/completions/zsh/_code')
.pipe(replace('@@APPNAME@@', product.applicationName))
.pipe(rename('BUILD/usr/share/zsh/site-functions/_' + product.applicationName));
const code = gulp.src(binaryDir + '/**/*', { base: binaryDir })
.pipe(rename(function (p) { p.dirname = 'BUILD/usr/share/' + product.applicationName + '/' + p.dirname; }));
const spec = gulp.src('resources/linux/rpm/code.spec.template', { base: '.' })
.pipe(replace('@@NAME@@', product.applicationName))
.pipe(replace('@@NAME_LONG@@', product.nameLong))
.pipe(replace('@@ICON@@', product.linuxIconName))
.pipe(replace('@@VERSION@@', packageJson.version))
.pipe(replace('@@RELEASE@@', linuxPackageRevision))
.pipe(replace('@@ARCHITECTURE@@', rpmArch))
.pipe(replace('@@LICENSE@@', product.licenseName))
.pipe(replace('@@QUALITY@@', product.quality || '@@QUALITY@@'))
.pipe(replace('@@UPDATEURL@@', product.updateUrl || '@@UPDATEURL@@'))
.pipe(replace('@@DEPENDENCIES@@', dependencies.join(', ')))
.pipe(replace('@@STRIP@@', stripBinary))
.pipe(rename('SPECS/' + product.applicationName + '.spec'));
const specIcon = gulp.src('resources/linux/rpm/code.xpm', { base: '.' })
.pipe(rename('SOURCES/' + product.applicationName + '.xpm'));
const all = es.merge(code, desktops, appdata, workspaceMime, icon, bash_completion, zsh_completion, spec, specIcon);
return all.pipe(vfs.dest(getRpmBuildPath(rpmArch)));
};
}
/**
* @param {string} arch
*/
function buildRpmPackage(arch) {
const rpmArch = getRpmPackageArch(arch);
const rpmBuildPath = getRpmBuildPath(rpmArch);
const rpmOut = `${rpmBuildPath}/RPMS/${rpmArch}`;
const destination = `.build/linux/rpm/${rpmArch}`;
return async () => {
await exec(`mkdir -p ${destination}`);
await exec(`HOME="$(pwd)/${destination}" rpmbuild -bb ${rpmBuildPath}/SPECS/${product.applicationName}.spec --target=${rpmArch}`);
await exec(`cp "${rpmOut}/$(ls ${rpmOut})" ${destination}/`);
};
}
/**
* @param {string} arch
*/
function getSnapBuildPath(arch) {
return `.build/linux/snap/${arch}/${product.applicationName}-${arch}`;
}
/**
* @param {string} arch
*/
function prepareSnapPackage(arch) {
const binaryDir = '../VSCode-linux-' + arch;
const destination = getSnapBuildPath(arch);
return function () {
// A desktop file that is placed in snap/gui will be placed into meta/gui verbatim.
const desktop = gulp.src('resources/linux/code.desktop', { base: '.' })
.pipe(rename(`snap/gui/${product.applicationName}.desktop`));
// A desktop file that is placed in snap/gui will be placed into meta/gui verbatim.
const desktopUrlHandler = gulp.src('resources/linux/code-url-handler.desktop', { base: '.' })
.pipe(rename(`snap/gui/${product.applicationName}-url-handler.desktop`));
const desktops = es.merge(desktop, desktopUrlHandler)
.pipe(replace('@@NAME_LONG@@', product.nameLong))
.pipe(replace('@@NAME_SHORT@@', product.nameShort))
.pipe(replace('@@NAME@@', product.applicationName))
.pipe(replace('@@EXEC@@', `${product.applicationName} --force-user-env`))
.pipe(replace('@@ICON@@', `\${SNAP}/meta/gui/${product.linuxIconName}.png`))
.pipe(replace('@@URLPROTOCOL@@', product.urlProtocol));
// An icon that is placed in snap/gui will be placed into meta/gui verbatim.
const icon = gulp.src('resources/linux/code.png', { base: '.' })
.pipe(rename(`snap/gui/${product.linuxIconName}.png`));
const code = gulp.src(binaryDir + '/**/*', { base: binaryDir })
.pipe(rename(function (p) { p.dirname = `usr/share/${product.applicationName}/${p.dirname}`; }));
const snapcraft = gulp.src('resources/linux/snap/snapcraft.yaml', { base: '.' })
.pipe(replace('@@NAME@@', product.applicationName))
.pipe(replace('@@VERSION@@', commit.substr(0, 8)))
// Possible run-on values https://snapcraft.io/docs/architectures
.pipe(replace('@@ARCHITECTURE@@', arch === 'x64' ? 'amd64' : arch))
.pipe(rename('snap/snapcraft.yaml'));
const electronLaunch = gulp.src('resources/linux/snap/electron-launch', { base: '.' })
.pipe(rename('electron-launch'));
const all = es.merge(desktops, icon, code, snapcraft, electronLaunch);
return all.pipe(vfs.dest(destination));
};
}
/**
* @param {string} arch
*/
function buildSnapPackage(arch) {
const cwd = getSnapBuildPath(arch);
return () => exec('snapcraft', { cwd });
}
const BUILD_TARGETS = [
{ arch: 'x64' },
{ arch: 'armhf' },
{ arch: 'arm64' },
];
BUILD_TARGETS.forEach(({ arch }) => {
const debArch = getDebPackageArch(arch);
const prepareDebTask = task.define(`vscode-linux-${arch}-prepare-deb`, task.series(rimraf(`.build/linux/deb/${debArch}`), prepareDebPackage(arch)));
gulp.task(prepareDebTask);
const buildDebTask = task.define(`vscode-linux-${arch}-build-deb`, buildDebPackage(arch));
gulp.task(buildDebTask);
const rpmArch = getRpmPackageArch(arch);
const prepareRpmTask = task.define(`vscode-linux-${arch}-prepare-rpm`, task.series(rimraf(`.build/linux/rpm/${rpmArch}`), prepareRpmPackage(arch)));
gulp.task(prepareRpmTask);
const buildRpmTask = task.define(`vscode-linux-${arch}-build-rpm`, buildRpmPackage(arch));
gulp.task(buildRpmTask);
const prepareSnapTask = task.define(`vscode-linux-${arch}-prepare-snap`, task.series(rimraf(`.build/linux/snap/${arch}`), prepareSnapPackage(arch)));
gulp.task(prepareSnapTask);
const buildSnapTask = task.define(`vscode-linux-${arch}-build-snap`, task.series(prepareSnapTask, buildSnapPackage(arch)));
gulp.task(buildSnapTask);
});
================================================
FILE: build/gulpfile.vscode.web.js
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
'use strict';
const gulp = require('gulp');
const path = require('path');
const es = require('event-stream');
const util = require('./lib/util');
const { getVersion } = require('./lib/getVersion');
const task = require('./lib/task');
const optimize = require('./lib/optimize');
const { readISODate } = require('./lib/date');
const product = require('../product.json');
const rename = require('gulp-rename');
const filter = require('gulp-filter');
const { getProductionDependencies } = require('./lib/dependencies');
const vfs = require('vinyl-fs');
const packageJson = require('../package.json');
const { compileBuildWithManglingTask } = require('./gulpfile.compile');
const extensions = require('./lib/extensions');
const VinylFile = require('vinyl');
const REPO_ROOT = path.dirname(__dirname);
const BUILD_ROOT = path.dirname(REPO_ROOT);
const WEB_FOLDER = path.join(REPO_ROOT, 'remote', 'web');
const commit = getVersion(REPO_ROOT);
const quality = product.quality;
const version = (quality && quality !== 'stable') ? `${packageJson.version}-${quality}` : packageJson.version;
const vscodeWebResourceIncludes = [
// NLS
'out-build/nls.messages.js',
// Accessibility Signals
'out-build/vs/platform/accessibilitySignal/browser/media/*.mp3',
// Welcome
'out-build/vs/workbench/contrib/welcomeGettingStarted/common/media/**/*.{svg,png}',
// Extensions
'out-build/vs/workbench/contrib/extensions/browser/media/{theme-icon.png,language-icon.svg}',
'out-build/vs/workbench/services/extensionManagement/common/media/*.{svg,png}',
// Webview
'out-build/vs/workbench/contrib/webview/browser/pre/*.{js,html}',
// Tree Sitter highlights
'out-build/vs/editor/common/languages/highlights/*.scm',
// Tree Sitter injections
'out-build/vs/editor/common/languages/injections/*.scm',
// Extension Host Worker
'out-build/vs/workbench/services/extensions/worker/webWorkerExtensionHostIframe.html',
];
exports.vscodeWebResourceIncludes = vscodeWebResourceIncludes;
const vscodeWebResources = [
// Includes
...vscodeWebResourceIncludes,
// Excludes
'!out-build/vs/**/{node,electron-sandbox,electron-main,electron-utility}/**',
'!out-build/vs/editor/standalone/**',
'!out-build/vs/workbench/**/*-tb.png',
'!out-build/vs/code/**/*-dev.html',
'!**/test/**'
];
const buildfile = require('./buildfile');
const vscodeWebEntryPoints = [
buildfile.workerEditor,
buildfile.workerExtensionHost,
buildfile.workerNotebook,
buildfile.workerLanguageDetection,
buildfile.workerLocalFileSearch,
buildfile.workerOutputLinks,
buildfile.workerBackgroundTokenization,
buildfile.keyboardMaps,
buildfile.workbenchWeb,
buildfile.entrypoint('vs/workbench/workbench.web.main.internal') // TODO@esm remove line when we stop supporting web-amd-esm-bridge
].flat();
/**
* @param extensionsRoot {string} The location where extension will be read from
* @param {object} product The parsed product.json file contents
*/
const createVSCodeWebFileContentMapper = (extensionsRoot, product) => {
return path => {
if (path.endsWith('vs/platform/product/common/product.js')) {
return content => {
const productConfiguration = JSON.stringify({
...product,
version,
commit,
date: readISODate('out-build')
});
return content.replace('/*BUILD->INSERT_PRODUCT_CONFIGURATION*/', () => productConfiguration.substr(1, productConfiguration.length - 2) /* without { and }*/);
};
} else if (path.endsWith('vs/workbench/services/extensionManagement/browser/builtinExtensionsScannerService.js')) {
return content => {
const builtinExtensions = JSON.stringify(extensions.scanBuiltinExtensions(extensionsRoot));
return content.replace('/*BUILD->INSERT_BUILTIN_EXTENSIONS*/', () => builtinExtensions.substr(1, builtinExtensions.length - 2) /* without [ and ]*/);
};
}
return undefined;
};
};
exports.createVSCodeWebFileContentMapper = createVSCodeWebFileContentMapper;
const bundleVSCodeWebTask = task.define('bundle-vscode-web', task.series(
util.rimraf('out-vscode-web'),
optimize.bundleTask(
{
out: 'out-vscode-web',
esm: {
src: 'out-build',
entryPoints: vscodeWebEntryPoints,
resources: vscodeWebResources,
fileContentMapper: createVSCodeWebFileContentMapper('.build/web/extensions', product)
}
}
)
));
const minifyVSCodeWebTask = task.define('minify-vscode-web', task.series(
bundleVSCodeWebTask,
util.rimraf('out-vscode-web-min'),
optimize.minifyTask('out-vscode-web', `https://main.vscode-cdn.net/sourcemaps/${commit}/core`)
));
gulp.task(minifyVSCodeWebTask);
function packageTask(sourceFolderName, destinationFolderName) {
const destination = path.join(BUILD_ROOT, destinationFolderName);
return () => {
const json = require('gulp-json-editor');
const src = gulp.src(sourceFolderName + '/**', { base: '.' })
.pipe(rename(function (path) { path.dirname = path.dirname.replace(new RegExp('^' + sourceFolderName), 'out'); }));
const extensions = gulp.src('.build/web/extensions/**', { base: '.build/web', dot: true });
const loader = gulp.src('build/loader.min', { base: 'build', dot: true }).pipe(rename('out/vs/loader.js')); // TODO@esm remove line when we stop supporting web-amd-esm-bridge
const sources = es.merge(src, extensions, loader)
.pipe(filter(['**', '!**/*.js.map'], { dot: true }))
// TODO@esm remove me once we stop supporting our web-esm-bridge
.pipe(es.through(function (file) {
if (file.relative === 'out/vs/workbench/workbench.web.main.internal.css') {
this.emit('data', new VinylFile({
contents: file.contents,
path: file.path.replace('workbench.web.main.internal.css', 'workbench.web.main.css'),
base: file.base
}));
}
this.emit('data', file);
}));
const name = product.nameShort;
const packageJsonStream = gulp.src(['remote/web/package.json'], { base: 'remote/web' })
.pipe(json({ name, version }));
const license = gulp.src(['remote/LICENSE'], { base: 'remote', allowEmpty: true });
const productionDependencies = getProductionDependencies(WEB_FOLDER);
const dependenciesSrc = productionDependencies.map(d => path.relative(REPO_ROOT, d)).map(d => [`${d}/**`, `!${d}/**/{test,tests}/**`, `!${d}/.bin/**`]).flat();
const deps = gulp.src(dependenciesSrc, { base: 'remote/web', dot: true })
.pipe(filter(['**', '!**/package-lock.json']))
.pipe(util.cleanNodeModules(path.join(__dirname, '.webignore')));
const favicon = gulp.src('resources/server/favicon.ico', { base: 'resources/server' });
const manifest = gulp.src('resources/server/manifest.json', { base: 'resources/server' });
const pwaicons = es.merge(
gulp.src('resources/server/code-192.png', { base: 'resources/server' }),
gulp.src('resources/server/code-512.png', { base: 'resources/server' })
);
const all = es.merge(
packageJsonStream,
license,
sources,
deps,
favicon,
manifest,
pwaicons
);
const result = all
.pipe(util.skipDirectories())
.pipe(util.fixWin32DirectoryPermissions());
return result.pipe(vfs.dest(destination));
};
}
const compileWebExtensionsBuildTask = task.define('compile-web-extensions-build', task.series(
task.define('clean-web-extensions-build', util.rimraf('.build/web/extensions')),
task.define('bundle-web-extensions-build', () => extensions.packageAllLocalExtensionsStream(true, false).pipe(gulp.dest('.build/web'))),
task.define('bundle-marketplace-web-extensions-build', () => extensions.packageMarketplaceExtensionsStream(true).pipe(gulp.dest('.build/web'))),
task.define('bundle-web-extension-media-build', () => extensions.buildExtensionMedia(false, '.build/web/extensions')),
));
gulp.task(compileWebExtensionsBuildTask);
const dashed = (/** @type {string} */ str) => (str ? `-${str}` : ``);
['', 'min'].forEach(minified => {
const sourceFolderName = `out-vscode-web${dashed(minified)}`;
const destinationFolderName = `vscode-web`;
const vscodeWebTaskCI = task.define(`vscode-web${dashed(minified)}-ci`, task.series(
compileWebExtensionsBuildTask,
minified ? minifyVSCodeWebTask : bundleVSCodeWebTask,
util.rimraf(path.join(BUILD_ROOT, destinationFolderName)),
packageTask(sourceFolderName, destinationFolderName)
));
gulp.task(vscodeWebTaskCI);
const vscodeWebTask = task.define(`vscode-web${dashed(minified)}`, task.series(
compileBuildWithManglingTask,
vscodeWebTaskCI
));
gulp.task(vscodeWebTask);
});
================================================
FILE: build/gulpfile.vscode.win32.js
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
'use strict';
const gulp = require('gulp');
const path = require('path');
const fs = require('fs');
const assert = require('assert');
const cp = require('child_process');
const util = require('./lib/util');
const task = require('./lib/task');
const pkg = require('../package.json');
const product = require('../product.json');
const vfs = require('vinyl-fs');
const rcedit = require('rcedit');
const repoPath = path.dirname(__dirname);
const buildPath = (/** @type {string} */ arch) => path.join(path.dirname(repoPath), `VSCode-win32-${arch}`);
const setupDir = (/** @type {string} */ arch, /** @type {string} */ target) => path.join(repoPath, '.build', `win32-${arch}`, `${target}-setup`);
const issPath = path.join(__dirname, 'win32', 'code.iss');
const innoSetupPath = path.join(path.dirname(path.dirname(require.resolve('innosetup'))), 'bin', 'ISCC.exe');
const signWin32Path = path.join(repoPath, 'build', 'azure-pipelines', 'common', 'sign-win32');
function packageInnoSetup(iss, options, cb) {
options = options || {};
const definitions = options.definitions || {};
if (process.argv.some(arg => arg === '--debug-inno')) {
definitions['Debug'] = 'true';
}
if (process.argv.some(arg => arg === '--sign')) {
definitions['Sign'] = 'true';
}
const keys = Object.keys(definitions);
keys.forEach(key => assert(typeof definitions[key] === 'string', `Missing value for '${key}' in Inno Setup package step`));
const defs = keys.map(key => `/d${key}=${definitions[key]}`);
const args = [
iss,
...defs,
`/sesrp=node ${signWin32Path} $f`
];
cp.spawn(innoSetupPath, args, { stdio: ['ignore', 'inherit', 'inherit'] })
.on('error', cb)
.on('exit', code => {
if (code === 0) {
cb(null);
} else {
cb(new Error(`InnoSetup returned exit code: ${code}`));
}
});
}
/**
* @param {string} arch
* @param {string} target
*/
function buildWin32Setup(arch, target) {
if (target !== 'system' && target !== 'user') {
throw new Error('Invalid setup target');
}
return cb => {
const x64AppId = target === 'system' ? product.win32x64AppId : product.win32x64UserAppId;
const arm64AppId = target === 'system' ? product.win32arm64AppId : product.win32arm64UserAppId;
const sourcePath = buildPath(arch);
const outputPath = setupDir(arch, target);
fs.mkdirSync(outputPath, { recursive: true });
const originalProductJsonPath = path.join(sourcePath, 'resources/app/product.json');
const productJsonPath = path.join(outputPath, 'product.json');
const productJson = JSON.parse(fs.readFileSync(originalProductJsonPath, 'utf8'));
productJson['target'] = target;
fs.writeFileSync(productJsonPath, JSON.stringify(productJson, undefined, '\t'));
const quality = product.quality || 'dev';
const definitions = {
NameLong: product.nameLong,
NameShort: product.nameShort,
DirName: product.win32DirName,
Version: pkg.version,
RawVersion: pkg.version.replace(/-\w+$/, ''),
NameVersion: product.win32NameVersion + (target === 'user' ? ' (User)' : ''),
ExeBasename: product.nameShort,
RegValueName: product.win32RegValueName,
ShellNameShort: product.win32ShellNameShort,
AppMutex: product.win32MutexName,
TunnelMutex: product.win32TunnelMutex,
TunnelServiceMutex: product.win32TunnelServiceMutex,
TunnelApplicationName: product.tunnelApplicationName,
ApplicationName: product.applicationName,
Arch: arch,
AppId: { 'x64': x64AppId, 'arm64': arm64AppId }[arch],
IncompatibleTargetAppId: { 'x64': product.win32x64AppId, 'arm64': product.win32arm64AppId }[arch],
AppUserId: product.win32AppUserModelId,
ArchitecturesAllowed: { 'x64': 'x64', 'arm64': 'arm64' }[arch],
ArchitecturesInstallIn64BitMode: { 'x64': 'x64', 'arm64': 'arm64' }[arch],
SourceDir: sourcePath,
RepoDir: repoPath,
OutputDir: outputPath,
InstallTarget: target,
ProductJsonPath: productJsonPath,
Quality: quality
};
if (quality === 'insider') {
definitions['AppxPackage'] = `code_insiders_explorer_${arch}.appx`;
definitions['AppxPackageFullname'] = `Microsoft.${product.win32RegValueName}_1.0.0.0_neutral__8wekyb3d8bbwe`;
}
packageInnoSetup(issPath, { definitions }, cb);
};
}
/**
* @param {string} arch
* @param {string} target
*/
function defineWin32SetupTasks(arch, target) {
const cleanTask = util.rimraf(setupDir(arch, target));
gulp.task(task.define(`vscode-win32-${arch}-${target}-setup`, task.series(cleanTask, buildWin32Setup(arch, target))));
}
defineWin32SetupTasks('x64', 'system');
defineWin32SetupTasks('arm64', 'system');
defineWin32SetupTasks('x64', 'user');
defineWin32SetupTasks('arm64', 'user');
/**
* @param {string} arch
*/
function copyInnoUpdater(arch) {
return () => {
return gulp.src('build/win32/{inno_updater.exe,vcruntime140.dll}', { base: 'build/win32' })
.pipe(vfs.dest(path.join(buildPath(arch), 'tools')));
};
}
/**
* @param {string} executablePath
*/
function updateIcon(executablePath) {
return cb => {
const icon = path.join(repoPath, 'resources', 'win32', 'code.ico');
rcedit(executablePath, { icon }, cb);
};
}
gulp.task(task.define('vscode-win32-x64-inno-updater', task.series(copyInnoUpdater('x64'), updateIcon(path.join(buildPath('x64'), 'tools', 'inno_updater.exe')))));
gulp.task(task.define('vscode-win32-arm64-inno-updater', task.series(copyInnoUpdater('arm64'), updateIcon(path.join(buildPath('arm64'), 'tools', 'inno_updater.exe')))));
================================================
FILE: build/hygiene.js
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
const filter = require('gulp-filter');
const es = require('event-stream');
const VinylFile = require('vinyl');
const vfs = require('vinyl-fs');
const path = require('path');
const fs = require('fs');
const pall = require('p-all');
const { all, copyrightFilter, unicodeFilter, indentationFilter, tsFormattingFilter, eslintFilter, stylelintFilter } = require('./filters');
const copyrightHeaderLines = [
'/*---------------------------------------------------------------------------------------------',
' * Copyright (c) Microsoft Corporation. All rights reserved.',
' * Licensed under the MIT License. See License.txt in the project root for license information.',
' *--------------------------------------------------------------------------------------------*/',
];
function hygiene(some, linting = true) {
const eslint = require('./gulp-eslint');
const gulpstylelint = require('./stylelint');
const formatter = require('./lib/formatter');
let errorCount = 0;
const productJson = es.through(function (file) {
const product = JSON.parse(file.contents.toString('utf8'));
if (product.extensionsGallery) {
console.error(`product.json: Contains 'extensionsGallery'`);
errorCount++;
}
this.emit('data', file);
});
const unicode = es.through(function (file) {
const lines = file.contents.toString('utf8').split(/\r\n|\r|\n/);
file.__lines = lines;
const allowInComments = lines.some(line => /allow-any-unicode-comment-file/.test(line));
let skipNext = false;
lines.forEach((line, i) => {
if (/allow-any-unicode-next-line/.test(line)) {
skipNext = true;
return;
}
if (skipNext) {
skipNext = false;
return;
}
// If unicode is allowed in comments, trim the comment from the line
if (allowInComments) {
if (line.match(/\s+(\*)/)) { // Naive multi-line comment check
line = '';
} else {
const index = line.indexOf('\/\/');
line = index === -1 ? line : line.substring(0, index);
}
}
// Please do not add symbols that resemble ASCII letters!
// eslint-disable-next-line no-misleading-character-class
const m = /([^\t\n\r\x20-\x7E⊃⊇✔︎✓🎯⚠️🛑🔴🚗🚙🚕🎉✨❗⇧⌥⌘×÷¦⋯…↑↓￫→←↔⟷·•●◆▼⟪⟫┌└├⏎↩√φ]+)/g.exec(line);
if (m) {
console.error(
file.relative + `(${i + 1},${m.index + 1}): Unexpected unicode character: "${m[0]}" (charCode: ${m[0].charCodeAt(0)}). To suppress, use // allow-any-unicode-next-line`
);
errorCount++;
}
});
this.emit('data', file);
});
const indentation = es.through(function (file) {
const lines = file.__lines || file.contents.toString('utf8').split(/\r\n|\r|\n/);
file.__lines = lines;
lines.forEach((line, i) => {
if (/^\s*$/.test(line)) {
// empty or whitespace lines are OK
} else if (/^[\t]*[^\s]/.test(line)) {
// good indent
} else if (/^[\t]* \*/.test(line)) {
// block comment using an extra space
} else {
console.error(
file.relative + '(' + (i + 1) + ',1): Bad whitespace indentation'
);
errorCount++;
}
});
this.emit('data', file);
});
const copyrights = es.through(function (file) {
const lines = file.__lines;
for (let i = 0; i < copyrightHeaderLines.length; i++) {
if (lines[i] !== copyrightHeaderLines[i]) {
console.error(file.relative + ': Missing or bad copyright statement');
errorCount++;
break;
}
}
this.emit('data', file);
});
const formatting = es.map(function (file, cb) {
try {
const rawInput = file.contents.toString('utf8');
const rawOutput = formatter.format(file.path, rawInput);
const original = rawInput.replace(/\r\n/gm, '\n');
const formatted = rawOutput.replace(/\r\n/gm, '\n');
if (original !== formatted) {
console.error(
`File not formatted. Run the 'Format Document' command to fix it:`,
file.relative
);
errorCount++;
}
cb(null, file);
} catch (err) {
cb(err);
}
});
let input;
if (Array.isArray(some) || typeof some === 'string' || !some) {
const options = { base: '.', follow: true, allowEmpty: true };
if (some) {
input = vfs.src(some, options).pipe(filter(all)); // split this up to not unnecessarily filter all a second time
} else {
input = vfs.src(all, options);
}
} else {
input = some;
}
const productJsonFilter = filter('product.json', { restore: true });
const snapshotFilter = filter(['**', '!**/*.snap', '!**/*.snap.actual']);
const yarnLockFilter = filter(['**', '!**/yarn.lock']);
const unicodeFilterStream = filter(unicodeFilter, { restore: true });
const result = input
.pipe(filter((f) => !f.stat.isDirectory()))
.pipe(snapshotFilter)
.pipe(yarnLockFilter)
.pipe(productJsonFilter)
.pipe(process.env['BUILD_SOURCEVERSION'] ? es.through() : productJson)
.pipe(productJsonFilter.restore)
.pipe(unicodeFilterStream)
.pipe(unicode)
.pipe(unicodeFilterStream.restore)
.pipe(filter(indentationFilter))
.pipe(indentation)
.pipe(filter(copyrightFilter))
.pipe(copyrights);
const streams = [
result.pipe(filter(tsFormattingFilter)).pipe(formatting)
];
if (linting) {
streams.push(
result
.pipe(filter(eslintFilter))
.pipe(
eslint((results) => {
errorCount += results.warningCount;
errorCount += results.errorCount;
})
)
);
streams.push(
result.pipe(filter(stylelintFilter)).pipe(gulpstylelint(((message, isError) => {
if (isError) {
console.error(message);
errorCount++;
} else {
console.warn(message);
}
})))
);
}
let count = 0;
return es.merge(...streams).pipe(
es.through(
function (data) {
count++;
if (process.env['TRAVIS'] && count % 10 === 0) {
process.stdout.write('.');
}
this.emit('data', data);
},
function () {
process.stdout.write('\n');
if (errorCount > 0) {
this.emit(
'error',
'Hygiene failed with ' +
errorCount +
` errors. Check 'build / gulpfile.hygiene.js'.`
);
} else {
this.emit('end');
}
}
)
);
}
module.exports.hygiene = hygiene;
function createGitIndexVinyls(paths) {
const cp = require('child_process');
const repositoryPath = process.cwd();
const fns = paths.map((relativePath) => () =>
new Promise((c, e) => {
const fullPath = path.join(repositoryPath, relativePath);
fs.stat(fullPath, (err, stat) => {
if (err && err.code === 'ENOENT') {
// ignore deletions
return c(null);
} else if (err) {
return e(err);
}
cp.exec(
process.platform === 'win32' ? `git show :${relativePath}` : `git show ':${relativePath}'`,
{ maxBuffer: stat.size, encoding: 'buffer' },
(err, out) => {
if (err) {
return e(err);
}
c(
new VinylFile({
path: fullPath,
base: repositoryPath,
contents: out,
stat,
})
);
}
);
});
})
);
return pall(fns, { concurrency: 4 }).then((r) => r.filter((p) => !!p));
}
// Void - NO PRE COMMIT HOOKS!!!! for now... - Void team
// // this allows us to run hygiene as a git pre-commit hook
// if (require.main === module) {
// const cp = require('child_process');
// process.on('unhandledRejection', (reason, p) => {
// console.log('Unhandled Rejection at: Promise', p, 'reason:', reason);
// process.exit(1);
// });
// if (process.argv.length > 2) {
// hygiene(process.argv.slice(2)).on('error', (err) => {
// console.error();
// console.error(err);
// process.exit(1);
// });
// } else {
// cp.exec(
// 'git diff --cached --name-only',
// { maxBuffer: 2000 * 1024 },
// (err, out) => {
// if (err) {
// console.error();
// console.error(err);
// process.exit(1);
// }
// const some = out.split(/\r?\n/).filter((l) => !!l);
// if (some.length > 0) {
// console.log('Reading git index versions...');
// createGitIndexVinyls(some)
// .then(
// (vinyls) =>
// new Promise((c, e) =>
// hygiene(es.readArray(vinyls).pipe(filter(all)))
// .on('end', () => c())
// .on('error', e)
// )
// )
// .catch((err) => {
// console.error();
// console.error(err);
// process.exit(1);
// });
// }
// }
// );
// }
// }
================================================
FILE: build/lib/asar.js
================================================
"use strict";
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
Object.defineProperty(exports, "__esModule", { value: true });
exports.createAsar = createAsar;
const path_1 = __importDefault(require("path"));
const event_stream_1 = __importDefault(require("event-stream"));
const pickle = require('chromium-pickle-js');
const Filesystem = require('asar/lib/filesystem');
const vinyl_1 = __importDefault(require("vinyl"));
const minimatch_1 = __importDefault(require("minimatch"));
function createAsar(folderPath, unpackGlobs, skipGlobs, duplicateGlobs, destFilename) {
const shouldUnpackFile = (file) => {
for (let i = 0; i < unpackGlobs.length; i++) {
if ((0, minimatch_1.default)(file.relative, unpackGlobs[i])) {
return true;
}
}
return false;
};
const shouldSkipFile = (file) => {
for (const skipGlob of skipGlobs) {
if ((0, minimatch_1.default)(file.relative, skipGlob)) {
return true;
}
}
return false;
};
// Files that should be duplicated between
// node_modules.asar and node_modules
const shouldDuplicateFile = (file) => {
for (const duplicateGlob of duplicateGlobs) {
if ((0, minimatch_1.default)(file.relative, duplicateGlob)) {
return true;
}
}
return false;
};
const filesystem = new Filesystem(folderPath);
const out = [];
// Keep track of pending inserts
let pendingInserts = 0;
let onFileInserted = () => { pendingInserts--; };
// Do not insert twice the same directory
const seenDir = {};
const insertDirectoryRecursive = (dir) => {
if (seenDir[dir]) {
return;
}
let lastSlash = dir.lastIndexOf('/');
if (lastSlash === -1) {
lastSlash = dir.lastIndexOf('\\');
}
if (lastSlash !== -1) {
insertDirectoryRecursive(dir.substring(0, lastSlash));
}
seenDir[dir] = true;
filesystem.insertDirectory(dir);
};
const insertDirectoryForFile = (file) => {
let lastSlash = file.lastIndexOf('/');
if (lastSlash === -1) {
lastSlash = file.lastIndexOf('\\');
}
if (lastSlash !== -1) {
insertDirectoryRecursive(file.substring(0, lastSlash));
}
};
const insertFile = (relativePath, stat, shouldUnpack) => {
insertDirectoryForFile(relativePath);
pendingInserts++;
// Do not pass `onFileInserted` directly because it gets overwritten below.
// Create a closure capturing `onFileInserted`.
filesystem.insertFile(relativePath, shouldUnpack, { stat: stat }, {}).then(() => onFileInserted(), () => onFileInserted());
};
return event_stream_1.default.through(function (file) {
if (file.stat.isDirectory()) {
return;
}
if (!file.stat.isFile()) {
throw new Error(`unknown item in stream!`);
}
if (shouldSkipFile(file)) {
this.queue(new vinyl_1.default({
base: '.',
path: file.path,
stat: file.stat,
contents: file.contents
}));
return;
}
if (shouldDuplicateFile(file)) {
this.queue(new vinyl_1.default({
base: '.',
path: file.path,
stat: file.stat,
contents: file.contents
}));
}
const shouldUnpack = shouldUnpackFile(file);
insertFile(file.relative, { size: file.contents.length, mode: file.stat.mode }, shouldUnpack);
if (shouldUnpack) {
// The file goes outside of xx.asar, in a folder xx.asar.unpacked
const relative = path_1.default.relative(folderPath, file.path);
this.queue(new vinyl_1.default({
base: '.',
path: path_1.default.join(destFilename + '.unpacked', relative),
stat: file.stat,
contents: file.contents
}));
}
else {
// The file goes inside of xx.asar
out.push(file.contents);
}
}, function () {
const finish = () => {
{
const headerPickle = pickle.createEmpty();
headerPickle.writeString(JSON.stringify(filesystem.header));
const headerBuf = headerPickle.toBuffer();
const sizePickle = pickle.createEmpty();
sizePickle.writeUInt32(headerBuf.length);
const sizeBuf = sizePickle.toBuffer();
out.unshift(headerBuf);
out.unshift(sizeBuf);
}
const contents = Buffer.concat(out);
out.length = 0;
this.queue(new vinyl_1.default({
base: '.',
path: destFilename,
contents: contents
}));
this.queue(null);
};
// Call finish() only when all file inserts have finished...
if (pendingInserts === 0) {
finish();
}
else {
onFileInserted = () => {
pendingInserts--;
if (pendingInserts === 0) {
finish();
}
};
}
});
}
//# sourceMappingURL=asar.js.map
================================================
FILE: build/lib/asar.ts
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
import path from 'path';
import es from 'event-stream';
const pickle = require('chromium-pickle-js');
const Filesystem = require('asar/lib/filesystem');
import VinylFile from 'vinyl';
import minimatch from 'minimatch';
declare class AsarFilesystem {
readonly header: unknown;
constructor(src: string);
insertDirectory(path: string, shouldUnpack?: boolean): unknown;
insertFile(path: string, shouldUnpack: boolean, file: { stat: { size: number; mode: number } }, options: {}): Promise;
}
export function createAsar(folderPath: string, unpackGlobs: string[], skipGlobs: string[], duplicateGlobs: string[], destFilename: string): NodeJS.ReadWriteStream {
const shouldUnpackFile = (file: VinylFile): boolean => {
for (let i = 0; i < unpackGlobs.length; i++) {
if (minimatch(file.relative, unpackGlobs[i])) {
return true;
}
}
return false;
};
const shouldSkipFile = (file: VinylFile): boolean => {
for (const skipGlob of skipGlobs) {
if (minimatch(file.relative, skipGlob)) {
return true;
}
}
return false;
};
// Files that should be duplicated between
// node_modules.asar and node_modules
const shouldDuplicateFile = (file: VinylFile): boolean => {
for (const duplicateGlob of duplicateGlobs) {
if (minimatch(file.relative, duplicateGlob)) {
return true;
}
}
return false;
};
const filesystem = new Filesystem(folderPath);
const out: Buffer[] = [];
// Keep track of pending inserts
let pendingInserts = 0;
let onFileInserted = () => { pendingInserts--; };
// Do not insert twice the same directory
const seenDir: { [key: string]: boolean } = {};
const insertDirectoryRecursive = (dir: string) => {
if (seenDir[dir]) {
return;
}
let lastSlash = dir.lastIndexOf('/');
if (lastSlash === -1) {
lastSlash = dir.lastIndexOf('\\');
}
if (lastSlash !== -1) {
insertDirectoryRecursive(dir.substring(0, lastSlash));
}
seenDir[dir] = true;
filesystem.insertDirectory(dir);
};
const insertDirectoryForFile = (file: string) => {
let lastSlash = file.lastIndexOf('/');
if (lastSlash === -1) {
lastSlash = file.lastIndexOf('\\');
}
if (lastSlash !== -1) {
insertDirectoryRecursive(file.substring(0, lastSlash));
}
};
const insertFile = (relativePath: string, stat: { size: number; mode: number }, shouldUnpack: boolean) => {
insertDirectoryForFile(relativePath);
pendingInserts++;
// Do not pass `onFileInserted` directly because it gets overwritten below.
// Create a closure capturing `onFileInserted`.
filesystem.insertFile(relativePath, shouldUnpack, { stat: stat }, {}).then(() => onFileInserted(), () => onFileInserted());
};
return es.through(function (file) {
if (file.stat.isDirectory()) {
return;
}
if (!file.stat.isFile()) {
throw new Error(`unknown item in stream!`);
}
if (shouldSkipFile(file)) {
this.queue(new VinylFile({
base: '.',
path: file.path,
stat: file.stat,
contents: file.contents
}));
return;
}
if (shouldDuplicateFile(file)) {
this.queue(new VinylFile({
base: '.',
path: file.path,
stat: file.stat,
contents: file.contents
}));
}
const shouldUnpack = shouldUnpackFile(file);
insertFile(file.relative, { size: file.contents.length, mode: file.stat.mode }, shouldUnpack);
if (shouldUnpack) {
// The file goes outside of xx.asar, in a folder xx.asar.unpacked
const relative = path.relative(folderPath, file.path);
this.queue(new VinylFile({
base: '.',
path: path.join(destFilename + '.unpacked', relative),
stat: file.stat,
contents: file.contents
}));
} else {
// The file goes inside of xx.asar
out.push(file.contents);
}
}, function () {
const finish = () => {
{
const headerPickle = pickle.createEmpty();
headerPickle.writeString(JSON.stringify(filesystem.header));
const headerBuf = headerPickle.toBuffer();
const sizePickle = pickle.createEmpty();
sizePickle.writeUInt32(headerBuf.length);
const sizeBuf = sizePickle.toBuffer();
out.unshift(headerBuf);
out.unshift(sizeBuf);
}
const contents = Buffer.concat(out);
out.length = 0;
this.queue(new VinylFile({
base: '.',
path: destFilename,
contents: contents
}));
this.queue(null);
};
// Call finish() only when all file inserts have finished...
if (pendingInserts === 0) {
finish();
} else {
onFileInserted = () => {
pendingInserts--;
if (pendingInserts === 0) {
finish();
}
};
}
});
}
================================================
FILE: build/lib/builtInExtensions.js
================================================
"use strict";
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
if (k2 === undefined) k2 = k;
var desc = Object.getOwnPropertyDescriptor(m, k);
if (!desc || ("get" in desc ? !m.__esModule : desc.writable || desc.configurable)) {
desc = { enumerable: true, get: function() { return m[k]; } };
}
Object.defineProperty(o, k2, desc);
}) : (function(o, m, k, k2) {
if (k2 === undefined) k2 = k;
o[k2] = m[k];
}));
var __setModuleDefault = (this && this.__setModuleDefault) || (Object.create ? (function(o, v) {
Object.defineProperty(o, "default", { enumerable: true, value: v });
}) : function(o, v) {
o["default"] = v;
});
var __importStar = (this && this.__importStar) || (function () {
var ownKeys = function(o) {
ownKeys = Object.getOwnPropertyNames || function (o) {
var ar = [];
for (var k in o) if (Object.prototype.hasOwnProperty.call(o, k)) ar[ar.length] = k;
return ar;
};
return ownKeys(o);
};
return function (mod) {
if (mod && mod.__esModule) return mod;
var result = {};
if (mod != null) for (var k = ownKeys(mod), i = 0; i < k.length; i++) if (k[i] !== "default") __createBinding(result, mod, k[i]);
__setModuleDefault(result, mod);
return result;
};
})();
var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
Object.defineProperty(exports, "__esModule", { value: true });
exports.getExtensionStream = getExtensionStream;
exports.getBuiltInExtensions = getBuiltInExtensions;
const fs_1 = __importDefault(require("fs"));
const path_1 = __importDefault(require("path"));
const os_1 = __importDefault(require("os"));
const rimraf_1 = __importDefault(require("rimraf"));
const event_stream_1 = __importDefault(require("event-stream"));
const gulp_rename_1 = __importDefault(require("gulp-rename"));
const vinyl_fs_1 = __importDefault(require("vinyl-fs"));
const ext = __importStar(require("./extensions"));
const fancy_log_1 = __importDefault(require("fancy-log"));
const ansi_colors_1 = __importDefault(require("ansi-colors"));
const root = path_1.default.dirname(path_1.default.dirname(__dirname));
const productjson = JSON.parse(fs_1.default.readFileSync(path_1.default.join(__dirname, '../../product.json'), 'utf8'));
const builtInExtensions = productjson.builtInExtensions || [];
const webBuiltInExtensions = productjson.webBuiltInExtensions || [];
const controlFilePath = path_1.default.join(os_1.default.homedir(), '.vscode-oss-dev', 'extensions', 'control.json');
const ENABLE_LOGGING = !process.env['VSCODE_BUILD_BUILTIN_EXTENSIONS_SILENCE_PLEASE'];
function log(...messages) {
if (ENABLE_LOGGING) {
(0, fancy_log_1.default)(...messages);
}
}
function getExtensionPath(extension) {
return path_1.default.join(root, '.build', 'builtInExtensions', extension.name);
}
function isUpToDate(extension) {
const packagePath = path_1.default.join(getExtensionPath(extension), 'package.json');
if (!fs_1.default.existsSync(packagePath)) {
return false;
}
const packageContents = fs_1.default.readFileSync(packagePath, { encoding: 'utf8' });
try {
const diskVersion = JSON.parse(packageContents).version;
return (diskVersion === extension.version);
}
catch (err) {
return false;
}
}
function getExtensionDownloadStream(extension) {
let input;
if (extension.vsix) {
input = ext.fromVsix(path_1.default.join(root, extension.vsix), extension);
}
else if (productjson.extensionsGallery?.serviceUrl) {
input = ext.fromMarketplace(productjson.extensionsGallery.serviceUrl, extension);
}
else {
input = ext.fromGithub(extension);
}
return input.pipe((0, gulp_rename_1.default)(p => p.dirname = `${extension.name}/${p.dirname}`));
}
function getExtensionStream(extension) {
// if the extension exists on disk, use those files instead of downloading anew
if (isUpToDate(extension)) {
log('[extensions]', `${extension.name}@${extension.version} up to date`, ansi_colors_1.default.green('✔︎'));
return vinyl_fs_1.default.src(['**'], { cwd: getExtensionPath(extension), dot: true })
.pipe((0, gulp_rename_1.default)(p => p.dirname = `${extension.name}/${p.dirname}`));
}
return getExtensionDownloadStream(extension);
}
function syncMarketplaceExtension(extension) {
const galleryServiceUrl = productjson.extensionsGallery?.serviceUrl;
const source = ansi_colors_1.default.blue(galleryServiceUrl ? '[marketplace]' : '[github]');
if (isUpToDate(extension)) {
log(source, `${extension.name}@${extension.version}`, ansi_colors_1.default.green('✔︎'));
return event_stream_1.default.readArray([]);
}
rimraf_1.default.sync(getExtensionPath(extension));
return getExtensionDownloadStream(extension)
.pipe(vinyl_fs_1.default.dest('.build/builtInExtensions'))
.on('end', () => log(source, extension.name, ansi_colors_1.default.green('✔︎')));
}
function syncExtension(extension, controlState) {
if (extension.platforms) {
const platforms = new Set(extension.platforms);
if (!platforms.has(process.platform)) {
log(ansi_colors_1.default.gray('[skip]'), `${extension.name}@${extension.version}: Platform '${process.platform}' not supported: [${extension.platforms}]`, ansi_colors_1.default.green('✔︎'));
return event_stream_1.default.readArray([]);
}
}
switch (controlState) {
case 'disabled':
log(ansi_colors_1.default.blue('[disabled]'), ansi_colors_1.default.gray(extension.name));
return event_stream_1.default.readArray([]);
case 'marketplace':
return syncMarketplaceExtension(extension);
default:
if (!fs_1.default.existsSync(controlState)) {
log(ansi_colors_1.default.red(`Error: Built-in extension '${extension.name}' is configured to run from '${controlState}' but that path does not exist.`));
return event_stream_1.default.readArray([]);
}
else if (!fs_1.default.existsSync(path_1.default.join(controlState, 'package.json'))) {
log(ansi_colors_1.default.red(`Error: Built-in extension '${extension.name}' is configured to run from '${controlState}' but there is no 'package.json' file in that directory.`));
return event_stream_1.default.readArray([]);
}
log(ansi_colors_1.default.blue('[local]'), `${extension.name}: ${ansi_colors_1.default.cyan(controlState)}`, ansi_colors_1.default.green('✔︎'));
return event_stream_1.default.readArray([]);
}
}
function readControlFile() {
try {
return JSON.parse(fs_1.default.readFileSync(controlFilePath, 'utf8'));
}
catch (err) {
return {};
}
}
function writeControlFile(control) {
fs_1.default.mkdirSync(path_1.default.dirname(controlFilePath), { recursive: true });
fs_1.default.writeFileSync(controlFilePath, JSON.stringify(control, null, 2));
}
function getBuiltInExtensions() {
log('Synchronizing built-in extensions...');
log(`You can manage built-in extensions with the ${ansi_colors_1.default.cyan('--builtin')} flag`);
const control = readControlFile();
const streams = [];
for (const extension of [...builtInExtensions, ...webBuiltInExtensions]) {
const controlState = control[extension.name] || 'marketplace';
control[extension.name] = controlState;
streams.push(syncExtension(extension, controlState));
}
writeControlFile(control);
return new Promise((resolve, reject) => {
event_stream_1.default.merge(streams)
.on('error', reject)
.on('end', resolve);
});
}
if (require.main === module) {
getBuiltInExtensions().then(() => process.exit(0)).catch(err => {
console.error(err);
process.exit(1);
});
}
//# sourceMappingURL=builtInExtensions.js.map
================================================
FILE: build/lib/builtInExtensions.ts
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
import fs from 'fs';
import path from 'path';
import os from 'os';
import rimraf from 'rimraf';
import es from 'event-stream';
import rename from 'gulp-rename';
import vfs from 'vinyl-fs';
import * as ext from './extensions';
import fancyLog from 'fancy-log';
import ansiColors from 'ansi-colors';
import { Stream } from 'stream';
export interface IExtensionDefinition {
name: string;
version: string;
sha256: string;
repo: string;
platforms?: string[];
vsix?: string;
metadata: {
id: string;
publisherId: {
publisherId: string;
publisherName: string;
displayName: string;
flags: string;
};
publisherDisplayName: string;
};
}
const root = path.dirname(path.dirname(__dirname));
const productjson = JSON.parse(fs.readFileSync(path.join(__dirname, '../../product.json'), 'utf8'));
const builtInExtensions = productjson.builtInExtensions || [];
const webBuiltInExtensions = productjson.webBuiltInExtensions || [];
const controlFilePath = path.join(os.homedir(), '.vscode-oss-dev', 'extensions', 'control.json');
const ENABLE_LOGGING = !process.env['VSCODE_BUILD_BUILTIN_EXTENSIONS_SILENCE_PLEASE'];
function log(...messages: string[]): void {
if (ENABLE_LOGGING) {
fancyLog(...messages);
}
}
function getExtensionPath(extension: IExtensionDefinition): string {
return path.join(root, '.build', 'builtInExtensions', extension.name);
}
function isUpToDate(extension: IExtensionDefinition): boolean {
const packagePath = path.join(getExtensionPath(extension), 'package.json');
if (!fs.existsSync(packagePath)) {
return false;
}
const packageContents = fs.readFileSync(packagePath, { encoding: 'utf8' });
try {
const diskVersion = JSON.parse(packageContents).version;
return (diskVersion === extension.version);
} catch (err) {
return false;
}
}
function getExtensionDownloadStream(extension: IExtensionDefinition) {
let input: Stream;
if (extension.vsix) {
input = ext.fromVsix(path.join(root, extension.vsix), extension);
} else if (productjson.extensionsGallery?.serviceUrl) {
input = ext.fromMarketplace(productjson.extensionsGallery.serviceUrl, extension);
} else {
input = ext.fromGithub(extension);
}
return input.pipe(rename(p => p.dirname = `${extension.name}/${p.dirname}`));
}
export function getExtensionStream(extension: IExtensionDefinition) {
// if the extension exists on disk, use those files instead of downloading anew
if (isUpToDate(extension)) {
log('[extensions]', `${extension.name}@${extension.version} up to date`, ansiColors.green('✔︎'));
return vfs.src(['**'], { cwd: getExtensionPath(extension), dot: true })
.pipe(rename(p => p.dirname = `${extension.name}/${p.dirname}`));
}
return getExtensionDownloadStream(extension);
}
function syncMarketplaceExtension(extension: IExtensionDefinition): Stream {
const galleryServiceUrl = productjson.extensionsGallery?.serviceUrl;
const source = ansiColors.blue(galleryServiceUrl ? '[marketplace]' : '[github]');
if (isUpToDate(extension)) {
log(source, `${extension.name}@${extension.version}`, ansiColors.green('✔︎'));
return es.readArray([]);
}
rimraf.sync(getExtensionPath(extension));
return getExtensionDownloadStream(extension)
.pipe(vfs.dest('.build/builtInExtensions'))
.on('end', () => log(source, extension.name, ansiColors.green('✔︎')));
}
function syncExtension(extension: IExtensionDefinition, controlState: 'disabled' | 'marketplace'): Stream {
if (extension.platforms) {
const platforms = new Set(extension.platforms);
if (!platforms.has(process.platform)) {
log(ansiColors.gray('[skip]'), `${extension.name}@${extension.version}: Platform '${process.platform}' not supported: [${extension.platforms}]`, ansiColors.green('✔︎'));
return es.readArray([]);
}
}
switch (controlState) {
case 'disabled':
log(ansiColors.blue('[disabled]'), ansiColors.gray(extension.name));
return es.readArray([]);
case 'marketplace':
return syncMarketplaceExtension(extension);
default:
if (!fs.existsSync(controlState)) {
log(ansiColors.red(`Error: Built-in extension '${extension.name}' is configured to run from '${controlState}' but that path does not exist.`));
return es.readArray([]);
} else if (!fs.existsSync(path.join(controlState, 'package.json'))) {
log(ansiColors.red(`Error: Built-in extension '${extension.name}' is configured to run from '${controlState}' but there is no 'package.json' file in that directory.`));
return es.readArray([]);
}
log(ansiColors.blue('[local]'), `${extension.name}: ${ansiColors.cyan(controlState)}`, ansiColors.green('✔︎'));
return es.readArray([]);
}
}
interface IControlFile {
[name: string]: 'disabled' | 'marketplace';
}
function readControlFile(): IControlFile {
try {
return JSON.parse(fs.readFileSync(controlFilePath, 'utf8'));
} catch (err) {
return {};
}
}
function writeControlFile(control: IControlFile): void {
fs.mkdirSync(path.dirname(controlFilePath), { recursive: true });
fs.writeFileSync(controlFilePath, JSON.stringify(control, null, 2));
}
export function getBuiltInExtensions(): Promise {
log('Synchronizing built-in extensions...');
log(`You can manage built-in extensions with the ${ansiColors.cyan('--builtin')} flag`);
const control = readControlFile();
const streams: Stream[] = [];
for (const extension of [...builtInExtensions, ...webBuiltInExtensions]) {
const controlState = control[extension.name] || 'marketplace';
control[extension.name] = controlState;
streams.push(syncExtension(extension, controlState));
}
writeControlFile(control);
return new Promise((resolve, reject) => {
es.merge(streams)
.on('error', reject)
.on('end', resolve);
});
}
if (require.main === module) {
getBuiltInExtensions().then(() => process.exit(0)).catch(err => {
console.error(err);
process.exit(1);
});
}
================================================
FILE: build/lib/builtInExtensionsCG.js
================================================
"use strict";
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
Object.defineProperty(exports, "__esModule", { value: true });
const fs_1 = __importDefault(require("fs"));
const path_1 = __importDefault(require("path"));
const url_1 = __importDefault(require("url"));
const ansi_colors_1 = __importDefault(require("ansi-colors"));
const root = path_1.default.dirname(path_1.default.dirname(__dirname));
const rootCG = path_1.default.join(root, 'extensionsCG');
const productjson = JSON.parse(fs_1.default.readFileSync(path_1.default.join(__dirname, '../../product.json'), 'utf8'));
const builtInExtensions = productjson.builtInExtensions || [];
const webBuiltInExtensions = productjson.webBuiltInExtensions || [];
const token = process.env['GITHUB_TOKEN'];
const contentBasePath = 'raw.githubusercontent.com';
const contentFileNames = ['package.json', 'package-lock.json'];
async function downloadExtensionDetails(extension) {
const extensionLabel = `${extension.name}@${extension.version}`;
const repository = url_1.default.parse(extension.repo).path.substr(1);
const repositoryContentBaseUrl = `https://${token ? `${token}@` : ''}${contentBasePath}/${repository}/v${extension.version}`;
async function getContent(fileName) {
try {
const response = await fetch(`${repositoryContentBaseUrl}/${fileName}`);
if (response.ok) {
return { fileName, body: Buffer.from(await response.arrayBuffer()) };
}
else if (response.status === 404) {
return { fileName, body: undefined };
}
else {
return { fileName, body: null };
}
}
catch (e) {
return { fileName, body: null };
}
}
const promises = contentFileNames.map(getContent);
console.log(extensionLabel);
const results = await Promise.all(promises);
for (const result of results) {
if (result.body) {
const extensionFolder = path_1.default.join(rootCG, extension.name);
fs_1.default.mkdirSync(extensionFolder, { recursive: true });
fs_1.default.writeFileSync(path_1.default.join(extensionFolder, result.fileName), result.body);
console.log(` - ${result.fileName} ${ansi_colors_1.default.green('✔︎')}`);
}
else if (result.body === undefined) {
console.log(` - ${result.fileName} ${ansi_colors_1.default.yellow('⚠️')}`);
}
else {
console.log(` - ${result.fileName} ${ansi_colors_1.default.red('🛑')}`);
}
}
// Validation
if (!results.find(r => r.fileName === 'package.json')?.body) {
// throw new Error(`The "package.json" file could not be found for the built-in extension - ${extensionLabel}`);
}
if (!results.find(r => r.fileName === 'package-lock.json')?.body) {
// throw new Error(`The "package-lock.json" could not be found for the built-in extension - ${extensionLabel}`);
}
}
async function main() {
for (const extension of [...builtInExtensions, ...webBuiltInExtensions]) {
await downloadExtensionDetails(extension);
}
}
main().then(() => {
console.log(`Built-in extensions component data downloaded ${ansi_colors_1.default.green('✔︎')}`);
process.exit(0);
}, err => {
console.log(`Built-in extensions component data could not be downloaded ${ansi_colors_1.default.red('🛑')}`);
console.error(err);
process.exit(1);
});
//# sourceMappingURL=builtInExtensionsCG.js.map
================================================
FILE: build/lib/builtInExtensionsCG.ts
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
import fs from 'fs';
import path from 'path';
import url from 'url';
import ansiColors from 'ansi-colors';
import { IExtensionDefinition } from './builtInExtensions';
const root = path.dirname(path.dirname(__dirname));
const rootCG = path.join(root, 'extensionsCG');
const productjson = JSON.parse(fs.readFileSync(path.join(__dirname, '../../product.json'), 'utf8'));
const builtInExtensions = productjson.builtInExtensions || [];
const webBuiltInExtensions = productjson.webBuiltInExtensions || [];
const token = process.env['GITHUB_TOKEN'];
const contentBasePath = 'raw.githubusercontent.com';
const contentFileNames = ['package.json', 'package-lock.json'];
async function downloadExtensionDetails(extension: IExtensionDefinition): Promise {
const extensionLabel = `${extension.name}@${extension.version}`;
const repository = url.parse(extension.repo).path!.substr(1);
const repositoryContentBaseUrl = `https://${token ? `${token}@` : ''}${contentBasePath}/${repository}/v${extension.version}`;
async function getContent(fileName: string): Promise<{ fileName: string; body: Buffer | undefined | null }> {
try {
const response = await fetch(`${repositoryContentBaseUrl}/${fileName}`);
if (response.ok) {
return { fileName, body: Buffer.from(await response.arrayBuffer()) };
} else if (response.status === 404) {
return { fileName, body: undefined };
} else {
return { fileName, body: null };
}
} catch (e) {
return { fileName, body: null };
}
}
const promises = contentFileNames.map(getContent);
console.log(extensionLabel);
const results = await Promise.all(promises);
for (const result of results) {
if (result.body) {
const extensionFolder = path.join(rootCG, extension.name);
fs.mkdirSync(extensionFolder, { recursive: true });
fs.writeFileSync(path.join(extensionFolder, result.fileName), result.body);
console.log(` - ${result.fileName} ${ansiColors.green('✔︎')}`);
} else if (result.body === undefined) {
console.log(` - ${result.fileName} ${ansiColors.yellow('⚠️')}`);
} else {
console.log(` - ${result.fileName} ${ansiColors.red('🛑')}`);
}
}
// Validation
if (!results.find(r => r.fileName === 'package.json')?.body) {
// throw new Error(`The "package.json" file could not be found for the built-in extension - ${extensionLabel}`);
}
if (!results.find(r => r.fileName === 'package-lock.json')?.body) {
// throw new Error(`The "package-lock.json" could not be found for the built-in extension - ${extensionLabel}`);
}
}
async function main(): Promise {
for (const extension of [...builtInExtensions, ...webBuiltInExtensions]) {
await downloadExtensionDetails(extension);
}
}
main().then(() => {
console.log(`Built-in extensions component data downloaded ${ansiColors.green('✔︎')}`);
process.exit(0);
}, err => {
console.log(`Built-in extensions component data could not be downloaded ${ansiColors.red('🛑')}`);
console.error(err);
process.exit(1);
});
================================================
FILE: build/lib/bundle.js
================================================
"use strict";
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
Object.defineProperty(exports, "__esModule", { value: true });
exports.removeAllTSBoilerplate = removeAllTSBoilerplate;
function removeAllTSBoilerplate(source) {
const seen = new Array(BOILERPLATE.length).fill(true, 0, BOILERPLATE.length);
return removeDuplicateTSBoilerplate(source, seen);
}
// Taken from typescript compiler => emitFiles
const BOILERPLATE = [
{ start: /^var __extends/, end: /^}\)\(\);$/ },
{ start: /^var __assign/, end: /^};$/ },
{ start: /^var __decorate/, end: /^};$/ },
{ start: /^var __metadata/, end: /^};$/ },
{ start: /^var __param/, end: /^};$/ },
{ start: /^var __awaiter/, end: /^};$/ },
{ start: /^var __generator/, end: /^};$/ },
{ start: /^var __createBinding/, end: /^}\)\);$/ },
{ start: /^var __setModuleDefault/, end: /^}\);$/ },
{ start: /^var __importStar/, end: /^};$/ },
{ start: /^var __addDisposableResource/, end: /^};$/ },
{ start: /^var __disposeResources/, end: /^}\);$/ },
];
function removeDuplicateTSBoilerplate(source, SEEN_BOILERPLATE = []) {
const lines = source.split(/\r\n|\n|\r/);
const newLines = [];
let IS_REMOVING_BOILERPLATE = false, END_BOILERPLATE;
for (let i = 0; i < lines.length; i++) {
const line = lines[i];
if (IS_REMOVING_BOILERPLATE) {
newLines.push('');
if (END_BOILERPLATE.test(line)) {
IS_REMOVING_BOILERPLATE = false;
}
}
else {
for (let j = 0; j < BOILERPLATE.length; j++) {
const boilerplate = BOILERPLATE[j];
if (boilerplate.start.test(line)) {
if (SEEN_BOILERPLATE[j]) {
IS_REMOVING_BOILERPLATE = true;
END_BOILERPLATE = boilerplate.end;
}
else {
SEEN_BOILERPLATE[j] = true;
}
}
}
if (IS_REMOVING_BOILERPLATE) {
newLines.push('');
}
else {
newLines.push(line);
}
}
}
return newLines.join('\n');
}
//# sourceMappingURL=bundle.js.map
================================================
FILE: build/lib/bundle.ts
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
export interface IEntryPoint {
name: string;
include?: string[];
dest?: string;
}
export function removeAllTSBoilerplate(source: string) {
const seen = new Array(BOILERPLATE.length).fill(true, 0, BOILERPLATE.length);
return removeDuplicateTSBoilerplate(source, seen);
}
// Taken from typescript compiler => emitFiles
const BOILERPLATE = [
{ start: /^var __extends/, end: /^}\)\(\);$/ },
{ start: /^var __assign/, end: /^};$/ },
{ start: /^var __decorate/, end: /^};$/ },
{ start: /^var __metadata/, end: /^};$/ },
{ start: /^var __param/, end: /^};$/ },
{ start: /^var __awaiter/, end: /^};$/ },
{ start: /^var __generator/, end: /^};$/ },
{ start: /^var __createBinding/, end: /^}\)\);$/ },
{ start: /^var __setModuleDefault/, end: /^}\);$/ },
{ start: /^var __importStar/, end: /^};$/ },
{ start: /^var __addDisposableResource/, end: /^};$/ },
{ start: /^var __disposeResources/, end: /^}\);$/ },
];
function removeDuplicateTSBoilerplate(source: string, SEEN_BOILERPLATE: boolean[] = []): string {
const lines = source.split(/\r\n|\n|\r/);
const newLines: string[] = [];
let IS_REMOVING_BOILERPLATE = false, END_BOILERPLATE: RegExp;
for (let i = 0; i < lines.length; i++) {
const line = lines[i];
if (IS_REMOVING_BOILERPLATE) {
newLines.push('');
if (END_BOILERPLATE!.test(line)) {
IS_REMOVING_BOILERPLATE = false;
}
} else {
for (let j = 0; j < BOILERPLATE.length; j++) {
const boilerplate = BOILERPLATE[j];
if (boilerplate.start.test(line)) {
if (SEEN_BOILERPLATE[j]) {
IS_REMOVING_BOILERPLATE = true;
END_BOILERPLATE = boilerplate.end;
} else {
SEEN_BOILERPLATE[j] = true;
}
}
}
if (IS_REMOVING_BOILERPLATE) {
newLines.push('');
} else {
newLines.push(line);
}
}
}
return newLines.join('\n');
}
================================================
FILE: build/lib/compilation.js
================================================
"use strict";
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
if (k2 === undefined) k2 = k;
var desc = Object.getOwnPropertyDescriptor(m, k);
if (!desc || ("get" in desc ? !m.__esModule : desc.writable || desc.configurable)) {
desc = { enumerable: true, get: function() { return m[k]; } };
}
Object.defineProperty(o, k2, desc);
}) : (function(o, m, k, k2) {
if (k2 === undefined) k2 = k;
o[k2] = m[k];
}));
var __setModuleDefault = (this && this.__setModuleDefault) || (Object.create ? (function(o, v) {
Object.defineProperty(o, "default", { enumerable: true, value: v });
}) : function(o, v) {
o["default"] = v;
});
var __importStar = (this && this.__importStar) || (function () {
var ownKeys = function(o) {
ownKeys = Object.getOwnPropertyNames || function (o) {
var ar = [];
for (var k in o) if (Object.prototype.hasOwnProperty.call(o, k)) ar[ar.length] = k;
return ar;
};
return ownKeys(o);
};
return function (mod) {
if (mod && mod.__esModule) return mod;
var result = {};
if (mod != null) for (var k = ownKeys(mod), i = 0; i < k.length; i++) if (k[i] !== "default") __createBinding(result, mod, k[i]);
__setModuleDefault(result, mod);
return result;
};
})();
var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
Object.defineProperty(exports, "__esModule", { value: true });
exports.watchApiProposalNamesTask = exports.compileApiProposalNamesTask = void 0;
exports.createCompile = createCompile;
exports.transpileTask = transpileTask;
exports.compileTask = compileTask;
exports.watchTask = watchTask;
const event_stream_1 = __importDefault(require("event-stream"));
const fs_1 = __importDefault(require("fs"));
const gulp_1 = __importDefault(require("gulp"));
const path_1 = __importDefault(require("path"));
const monacodts = __importStar(require("./monaco-api"));
const nls = __importStar(require("./nls"));
const reporter_1 = require("./reporter");
const util = __importStar(require("./util"));
const fancy_log_1 = __importDefault(require("fancy-log"));
const ansi_colors_1 = __importDefault(require("ansi-colors"));
const os_1 = __importDefault(require("os"));
const vinyl_1 = __importDefault(require("vinyl"));
const task = __importStar(require("./task"));
const index_1 = require("./mangle/index");
const postcss_1 = require("./postcss");
const ts = require("typescript");
const watch = require('./watch');
// --- gulp-tsb: compile and transpile --------------------------------
const reporter = (0, reporter_1.createReporter)();
function getTypeScriptCompilerOptions(src) {
const rootDir = path_1.default.join(__dirname, `../../${src}`);
const options = {};
options.verbose = false;
options.sourceMap = true;
if (process.env['VSCODE_NO_SOURCEMAP']) { // To be used by developers in a hurry
options.sourceMap = false;
}
options.rootDir = rootDir;
options.baseUrl = rootDir;
options.sourceRoot = util.toFileUri(rootDir);
options.newLine = /\r\n/.test(fs_1.default.readFileSync(__filename, 'utf8')) ? 0 : 1;
return options;
}
function createCompile(src, { build, emitError, transpileOnly, preserveEnglish }) {
const tsb = require('./tsb');
const sourcemaps = require('gulp-sourcemaps');
const projectPath = path_1.default.join(__dirname, '../../', src, 'tsconfig.json');
const overrideOptions = { ...getTypeScriptCompilerOptions(src), inlineSources: Boolean(build) };
if (!build) {
overrideOptions.inlineSourceMap = true;
}
const compilation = tsb.create(projectPath, overrideOptions, {
verbose: false,
transpileOnly: Boolean(transpileOnly),
transpileWithSwc: typeof transpileOnly !== 'boolean' && transpileOnly.esbuild
}, err => reporter(err));
function pipeline(token) {
const bom = require('gulp-bom');
const tsFilter = util.filter(data => /\.ts$/.test(data.path));
const isUtf8Test = (f) => /(\/|\\)test(\/|\\).*utf8/.test(f.path);
const isRuntimeJs = (f) => f.path.endsWith('.js') && !f.path.includes('fixtures');
const isCSS = (f) => f.path.endsWith('.css') && !f.path.includes('fixtures');
const noDeclarationsFilter = util.filter(data => !(/\.d\.ts$/.test(data.path)));
const postcssNesting = require('postcss-nesting');
const input = event_stream_1.default.through();
const output = input
.pipe(util.$if(isUtf8Test, bom())) // this is required to preserve BOM in test files that loose it otherwise
.pipe(util.$if(!build && isRuntimeJs, util.appendOwnPathSourceURL()))
.pipe(util.$if(isCSS, (0, postcss_1.gulpPostcss)([postcssNesting()], err => reporter(String(err)))))
.pipe(tsFilter)
.pipe(util.loadSourcemaps())
.pipe(compilation(token))
.pipe(noDeclarationsFilter)
.pipe(util.$if(build, nls.nls({ preserveEnglish })))
.pipe(noDeclarationsFilter.restore)
.pipe(util.$if(!transpileOnly, sourcemaps.write('.', {
addComment: false,
includeContent: !!build,
sourceRoot: overrideOptions.sourceRoot
})))
.pipe(tsFilter.restore)
.pipe(reporter.end(!!emitError));
return event_stream_1.default.duplex(input, output);
}
pipeline.tsProjectSrc = () => {
return compilation.src({ base: src });
};
pipeline.projectPath = projectPath;
return pipeline;
}
function transpileTask(src, out, esbuild) {
const task = () => {
const transpile = createCompile(src, { build: false, emitError: true, transpileOnly: { esbuild }, preserveEnglish: false });
const srcPipe = gulp_1.default.src(`${src}/**`, { base: `${src}` });
return srcPipe
.pipe(transpile())
.pipe(gulp_1.default.dest(out));
};
task.taskName = `transpile-${path_1.default.basename(src)}`;
return task;
}
function compileTask(src, out, build, options = {}) {
const task = () => {
if (os_1.default.totalmem() < 4_000_000_000) {
throw new Error('compilation requires 4GB of RAM');
}
const compile = createCompile(src, { build, emitError: true, transpileOnly: false, preserveEnglish: !!options.preserveEnglish });
const srcPipe = gulp_1.default.src(`${src}/**`, { base: `${src}` });
const generator = new MonacoGenerator(false);
if (src === 'src') {
generator.execute();
}
// mangle: TypeScript to TypeScript
let mangleStream = event_stream_1.default.through();
if (build && !options.disableMangle) {
let ts2tsMangler = new index_1.Mangler(compile.projectPath, (...data) => (0, fancy_log_1.default)(ansi_colors_1.default.blue('[mangler]'), ...data), { mangleExports: true, manglePrivateFields: true });
const newContentsByFileName = ts2tsMangler.computeNewFileContents(new Set(['saveState']));
mangleStream = event_stream_1.default.through(async function write(data) {
const tsNormalPath = ts.normalizePath(data.path);
const newContents = (await newContentsByFileName).get(tsNormalPath);
if (newContents !== undefined) {
data.contents = Buffer.from(newContents.out);
data.sourceMap = newContents.sourceMap && JSON.parse(newContents.sourceMap);
}
this.push(data);
}, async function end() {
// free resources
(await newContentsByFileName).clear();
this.push(null);
ts2tsMangler = undefined;
});
}
return srcPipe
.pipe(mangleStream)
.pipe(generator.stream)
.pipe(compile())
.pipe(gulp_1.default.dest(out));
};
task.taskName = `compile-${path_1.default.basename(src)}`;
return task;
}
function watchTask(out, build, srcPath = 'src') {
const task = () => {
const compile = createCompile(srcPath, { build, emitError: false, transpileOnly: false, preserveEnglish: false });
const src = gulp_1.default.src(`${srcPath}/**`, { base: srcPath });
const watchSrc = watch(`${srcPath}/**`, { base: srcPath, readDelay: 200 });
const generator = new MonacoGenerator(true);
generator.execute();
return watchSrc
.pipe(generator.stream)
.pipe(util.incremental(compile, src, true))
.pipe(gulp_1.default.dest(out));
};
task.taskName = `watch-${path_1.default.basename(out)}`;
return task;
}
const REPO_SRC_FOLDER = path_1.default.join(__dirname, '../../src');
class MonacoGenerator {
_isWatch;
stream;
_watchedFiles;
_fsProvider;
_declarationResolver;
constructor(isWatch) {
this._isWatch = isWatch;
this.stream = event_stream_1.default.through();
this._watchedFiles = {};
const onWillReadFile = (moduleId, filePath) => {
if (!this._isWatch) {
return;
}
if (this._watchedFiles[filePath]) {
return;
}
this._watchedFiles[filePath] = true;
fs_1.default.watchFile(filePath, () => {
this._declarationResolver.invalidateCache(moduleId);
this._executeSoon();
});
};
this._fsProvider = new class extends monacodts.FSProvider {
readFileSync(moduleId, filePath) {
onWillReadFile(moduleId, filePath);
return super.readFileSync(moduleId, filePath);
}
};
this._declarationResolver = new monacodts.DeclarationResolver(this._fsProvider);
if (this._isWatch) {
fs_1.default.watchFile(monacodts.RECIPE_PATH, () => {
this._executeSoon();
});
}
}
_executeSoonTimer = null;
_executeSoon() {
if (this._executeSoonTimer !== null) {
clearTimeout(this._executeSoonTimer);
this._executeSoonTimer = null;
}
this._executeSoonTimer = setTimeout(() => {
this._executeSoonTimer = null;
this.execute();
}, 20);
}
_run() {
const r = monacodts.run3(this._declarationResolver);
if (!r && !this._isWatch) {
// The build must always be able to generate the monaco.d.ts
throw new Error(`monaco.d.ts generation error - Cannot continue`);
}
return r;
}
_log(message, ...rest) {
(0, fancy_log_1.default)(ansi_colors_1.default.cyan('[monaco.d.ts]'), message, ...rest);
}
execute() {
const startTime = Date.now();
const result = this._run();
if (!result) {
// nothing really changed
return;
}
if (result.isTheSame) {
return;
}
fs_1.default.writeFileSync(result.filePath, result.content);
fs_1.default.writeFileSync(path_1.default.join(REPO_SRC_FOLDER, 'vs/editor/common/standalone/standaloneEnums.ts'), result.enums);
this._log(`monaco.d.ts is changed - total time took ${Date.now() - startTime} ms`);
if (!this._isWatch) {
this.stream.emit('error', 'monaco.d.ts is no longer up to date. Please run gulp watch and commit the new file.');
}
}
}
function generateApiProposalNames() {
let eol;
try {
const src = fs_1.default.readFileSync('src/vs/platform/extensions/common/extensionsApiProposals.ts', 'utf-8');
const match = /\r?\n/m.exec(src);
eol = match ? match[0] : os_1.default.EOL;
}
catch {
eol = os_1.default.EOL;
}
const pattern = /vscode\.proposed\.([a-zA-Z\d]+)\.d\.ts$/;
const versionPattern = /^\s*\/\/\s*version\s*:\s*(\d+)\s*$/mi;
const proposals = new Map();
const input = event_stream_1.default.through();
const output = input
.pipe(util.filter((f) => pattern.test(f.path)))
.pipe(event_stream_1.default.through((f) => {
const name = path_1.default.basename(f.path);
const match = pattern.exec(name);
if (!match) {
return;
}
const proposalName = match[1];
const contents = f.contents.toString('utf8');
const versionMatch = versionPattern.exec(contents);
const version = versionMatch ? versionMatch[1] : undefined;
proposals.set(proposalName, {
proposal: `https://raw.githubusercontent.com/microsoft/vscode/main/src/vscode-dts/vscode.proposed.${proposalName}.d.ts`,
version: version ? parseInt(version) : undefined
});
}, function () {
const names = [...proposals.keys()].sort();
const contents = [
'/*---------------------------------------------------------------------------------------------',
' * Copyright (c) Microsoft Corporation. All rights reserved.',
' * Licensed under the MIT License. See License.txt in the project root for license information.',
' *--------------------------------------------------------------------------------------------*/',
'',
'// THIS IS A GENERATED FILE. DO NOT EDIT DIRECTLY.',
'',
'const _allApiProposals = {',
`${names.map(proposalName => {
const proposal = proposals.get(proposalName);
return `\t${proposalName}: {${eol}\t\tproposal: '${proposal.proposal}',${eol}${proposal.version ? `\t\tversion: ${proposal.version}${eol}` : ''}\t}`;
}).join(`,${eol}`)}`,
'};',
'export const allApiProposals = Object.freeze<{ [proposalName: string]: Readonly<{ proposal: string; version?: number }> }>(_allApiProposals);',
'export type ApiProposalName = keyof typeof _allApiProposals;',
'',
].join(eol);
this.emit('data', new vinyl_1.default({
path: 'vs/platform/extensions/common/extensionsApiProposals.ts',
contents: Buffer.from(contents)
}));
this.emit('end');
}));
return event_stream_1.default.duplex(input, output);
}
const apiProposalNamesReporter = (0, reporter_1.createReporter)('api-proposal-names');
exports.compileApiProposalNamesTask = task.define('compile-api-proposal-names', () => {
return gulp_1.default.src('src/vscode-dts/**')
.pipe(generateApiProposalNames())
.pipe(gulp_1.default.dest('src'))
.pipe(apiProposalNamesReporter.end(true));
});
exports.watchApiProposalNamesTask = task.define('watch-api-proposal-names', () => {
const task = () => gulp_1.default.src('src/vscode-dts/**')
.pipe(generateApiProposalNames())
.pipe(apiProposalNamesReporter.end(true));
return watch('src/vscode-dts/**', { readDelay: 200 })
.pipe(util.debounce(task))
.pipe(gulp_1.default.dest('src'));
});
//# sourceMappingURL=compilation.js.map
================================================
FILE: build/lib/compilation.ts
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
import es from 'event-stream';
import fs from 'fs';
import gulp from 'gulp';
import path from 'path';
import * as monacodts from './monaco-api';
import * as nls from './nls';
import { createReporter } from './reporter';
import * as util from './util';
import fancyLog from 'fancy-log';
import ansiColors from 'ansi-colors';
import os from 'os';
import File from 'vinyl';
import * as task from './task';
import { Mangler } from './mangle/index';
import { RawSourceMap } from 'source-map';
import { gulpPostcss } from './postcss';
import ts = require('typescript');
const watch = require('./watch');
// --- gulp-tsb: compile and transpile --------------------------------
const reporter = createReporter();
function getTypeScriptCompilerOptions(src: string): ts.CompilerOptions {
const rootDir = path.join(__dirname, `../../${src}`);
const options: ts.CompilerOptions = {};
options.verbose = false;
options.sourceMap = true;
if (process.env['VSCODE_NO_SOURCEMAP']) { // To be used by developers in a hurry
options.sourceMap = false;
}
options.rootDir = rootDir;
options.baseUrl = rootDir;
options.sourceRoot = util.toFileUri(rootDir);
options.newLine = /\r\n/.test(fs.readFileSync(__filename, 'utf8')) ? 0 : 1;
return options;
}
interface ICompileTaskOptions {
readonly build: boolean;
readonly emitError: boolean;
readonly transpileOnly: boolean | { esbuild: boolean };
readonly preserveEnglish: boolean;
}
export function createCompile(src: string, { build, emitError, transpileOnly, preserveEnglish }: ICompileTaskOptions) {
const tsb = require('./tsb') as typeof import('./tsb');
const sourcemaps = require('gulp-sourcemaps') as typeof import('gulp-sourcemaps');
const projectPath = path.join(__dirname, '../../', src, 'tsconfig.json');
const overrideOptions = { ...getTypeScriptCompilerOptions(src), inlineSources: Boolean(build) };
if (!build) {
overrideOptions.inlineSourceMap = true;
}
const compilation = tsb.create(projectPath, overrideOptions, {
verbose: false,
transpileOnly: Boolean(transpileOnly),
transpileWithSwc: typeof transpileOnly !== 'boolean' && transpileOnly.esbuild
}, err => reporter(err));
function pipeline(token?: util.ICancellationToken) {
const bom = require('gulp-bom') as typeof import('gulp-bom');
const tsFilter = util.filter(data => /\.ts$/.test(data.path));
const isUtf8Test = (f: File) => /(\/|\\)test(\/|\\).*utf8/.test(f.path);
const isRuntimeJs = (f: File) => f.path.endsWith('.js') && !f.path.includes('fixtures');
const isCSS = (f: File) => f.path.endsWith('.css') && !f.path.includes('fixtures');
const noDeclarationsFilter = util.filter(data => !(/\.d\.ts$/.test(data.path)));
const postcssNesting = require('postcss-nesting');
const input = es.through();
const output = input
.pipe(util.$if(isUtf8Test, bom())) // this is required to preserve BOM in test files that loose it otherwise
.pipe(util.$if(!build && isRuntimeJs, util.appendOwnPathSourceURL()))
.pipe(util.$if(isCSS, gulpPostcss([postcssNesting()], err => reporter(String(err)))))
.pipe(tsFilter)
.pipe(util.loadSourcemaps())
.pipe(compilation(token))
.pipe(noDeclarationsFilter)
.pipe(util.$if(build, nls.nls({ preserveEnglish })))
.pipe(noDeclarationsFilter.restore)
.pipe(util.$if(!transpileOnly, sourcemaps.write('.', {
addComment: false,
includeContent: !!build,
sourceRoot: overrideOptions.sourceRoot
})))
.pipe(tsFilter.restore)
.pipe(reporter.end(!!emitError));
return es.duplex(input, output);
}
pipeline.tsProjectSrc = () => {
return compilation.src({ base: src });
};
pipeline.projectPath = projectPath;
return pipeline;
}
export function transpileTask(src: string, out: string, esbuild: boolean): task.StreamTask {
const task = () => {
const transpile = createCompile(src, { build: false, emitError: true, transpileOnly: { esbuild }, preserveEnglish: false });
const srcPipe = gulp.src(`${src}/**`, { base: `${src}` });
return srcPipe
.pipe(transpile())
.pipe(gulp.dest(out));
};
task.taskName = `transpile-${path.basename(src)}`;
return task;
}
export function compileTask(src: string, out: string, build: boolean, options: { disableMangle?: boolean; preserveEnglish?: boolean } = {}): task.StreamTask {
const task = () => {
if (os.totalmem() < 4_000_000_000) {
throw new Error('compilation requires 4GB of RAM');
}
const compile = createCompile(src, { build, emitError: true, transpileOnly: false, preserveEnglish: !!options.preserveEnglish });
const srcPipe = gulp.src(`${src}/**`, { base: `${src}` });
const generator = new MonacoGenerator(false);
if (src === 'src') {
generator.execute();
}
// mangle: TypeScript to TypeScript
let mangleStream = es.through();
if (build && !options.disableMangle) {
let ts2tsMangler = new Mangler(compile.projectPath, (...data) => fancyLog(ansiColors.blue('[mangler]'), ...data), { mangleExports: true, manglePrivateFields: true });
const newContentsByFileName = ts2tsMangler.computeNewFileContents(new Set(['saveState']));
mangleStream = es.through(async function write(data: File & { sourceMap?: RawSourceMap }) {
type TypeScriptExt = typeof ts & { normalizePath(path: string): string };
const tsNormalPath = (ts).normalizePath(data.path);
const newContents = (await newContentsByFileName).get(tsNormalPath);
if (newContents !== undefined) {
data.contents = Buffer.from(newContents.out);
data.sourceMap = newContents.sourceMap && JSON.parse(newContents.sourceMap);
}
this.push(data);
}, async function end() {
// free resources
(await newContentsByFileName).clear();
this.push(null);
(ts2tsMangler) = undefined;
});
}
return srcPipe
.pipe(mangleStream)
.pipe(generator.stream)
.pipe(compile())
.pipe(gulp.dest(out));
};
task.taskName = `compile-${path.basename(src)}`;
return task;
}
export function watchTask(out: string, build: boolean, srcPath: string = 'src'): task.StreamTask {
const task = () => {
const compile = createCompile(srcPath, { build, emitError: false, transpileOnly: false, preserveEnglish: false });
const src = gulp.src(`${srcPath}/**`, { base: srcPath });
const watchSrc = watch(`${srcPath}/**`, { base: srcPath, readDelay: 200 });
const generator = new MonacoGenerator(true);
generator.execute();
return watchSrc
.pipe(generator.stream)
.pipe(util.incremental(compile, src, true))
.pipe(gulp.dest(out));
};
task.taskName = `watch-${path.basename(out)}`;
return task;
}
const REPO_SRC_FOLDER = path.join(__dirname, '../../src');
class MonacoGenerator {
private readonly _isWatch: boolean;
public readonly stream: NodeJS.ReadWriteStream;
private readonly _watchedFiles: { [filePath: string]: boolean };
private readonly _fsProvider: monacodts.FSProvider;
private readonly _declarationResolver: monacodts.DeclarationResolver;
constructor(isWatch: boolean) {
this._isWatch = isWatch;
this.stream = es.through();
this._watchedFiles = {};
const onWillReadFile = (moduleId: string, filePath: string) => {
if (!this._isWatch) {
return;
}
if (this._watchedFiles[filePath]) {
return;
}
this._watchedFiles[filePath] = true;
fs.watchFile(filePath, () => {
this._declarationResolver.invalidateCache(moduleId);
this._executeSoon();
});
};
this._fsProvider = new class extends monacodts.FSProvider {
public readFileSync(moduleId: string, filePath: string): Buffer {
onWillReadFile(moduleId, filePath);
return super.readFileSync(moduleId, filePath);
}
};
this._declarationResolver = new monacodts.DeclarationResolver(this._fsProvider);
if (this._isWatch) {
fs.watchFile(monacodts.RECIPE_PATH, () => {
this._executeSoon();
});
}
}
private _executeSoonTimer: NodeJS.Timeout | null = null;
private _executeSoon(): void {
if (this._executeSoonTimer !== null) {
clearTimeout(this._executeSoonTimer);
this._executeSoonTimer = null;
}
this._executeSoonTimer = setTimeout(() => {
this._executeSoonTimer = null;
this.execute();
}, 20);
}
private _run(): monacodts.IMonacoDeclarationResult | null {
const r = monacodts.run3(this._declarationResolver);
if (!r && !this._isWatch) {
// The build must always be able to generate the monaco.d.ts
throw new Error(`monaco.d.ts generation error - Cannot continue`);
}
return r;
}
private _log(message: any, ...rest: any[]): void {
fancyLog(ansiColors.cyan('[monaco.d.ts]'), message, ...rest);
}
public execute(): void {
const startTime = Date.now();
const result = this._run();
if (!result) {
// nothing really changed
return;
}
if (result.isTheSame) {
return;
}
fs.writeFileSync(result.filePath, result.content);
fs.writeFileSync(path.join(REPO_SRC_FOLDER, 'vs/editor/common/standalone/standaloneEnums.ts'), result.enums);
this._log(`monaco.d.ts is changed - total time took ${Date.now() - startTime} ms`);
if (!this._isWatch) {
this.stream.emit('error', 'monaco.d.ts is no longer up to date. Please run gulp watch and commit the new file.');
}
}
}
function generateApiProposalNames() {
let eol: string;
try {
const src = fs.readFileSync('src/vs/platform/extensions/common/extensionsApiProposals.ts', 'utf-8');
const match = /\r?\n/m.exec(src);
eol = match ? match[0] : os.EOL;
} catch {
eol = os.EOL;
}
const pattern = /vscode\.proposed\.([a-zA-Z\d]+)\.d\.ts$/;
const versionPattern = /^\s*\/\/\s*version\s*:\s*(\d+)\s*$/mi;
const proposals = new Map();
const input = es.through();
const output = input
.pipe(util.filter((f: File) => pattern.test(f.path)))
.pipe(es.through((f: File) => {
const name = path.basename(f.path);
const match = pattern.exec(name);
if (!match) {
return;
}
const proposalName = match[1];
const contents = f.contents.toString('utf8');
const versionMatch = versionPattern.exec(contents);
const version = versionMatch ? versionMatch[1] : undefined;
proposals.set(proposalName, {
proposal: `https://raw.githubusercontent.com/microsoft/vscode/main/src/vscode-dts/vscode.proposed.${proposalName}.d.ts`,
version: version ? parseInt(version) : undefined
});
}, function () {
const names = [...proposals.keys()].sort();
const contents = [
'/*---------------------------------------------------------------------------------------------',
' * Copyright (c) Microsoft Corporation. All rights reserved.',
' * Licensed under the MIT License. See License.txt in the project root for license information.',
' *--------------------------------------------------------------------------------------------*/',
'',
'// THIS IS A GENERATED FILE. DO NOT EDIT DIRECTLY.',
'',
'const _allApiProposals = {',
`${names.map(proposalName => {
const proposal = proposals.get(proposalName)!;
return `\t${proposalName}: {${eol}\t\tproposal: '${proposal.proposal}',${eol}${proposal.version ? `\t\tversion: ${proposal.version}${eol}` : ''}\t}`;
}).join(`,${eol}`)}`,
'};',
'export const allApiProposals = Object.freeze<{ [proposalName: string]: Readonly<{ proposal: string; version?: number }> }>(_allApiProposals);',
'export type ApiProposalName = keyof typeof _allApiProposals;',
'',
].join(eol);
this.emit('data', new File({
path: 'vs/platform/extensions/common/extensionsApiProposals.ts',
contents: Buffer.from(contents)
}));
this.emit('end');
}));
return es.duplex(input, output);
}
const apiProposalNamesReporter = createReporter('api-proposal-names');
export const compileApiProposalNamesTask = task.define('compile-api-proposal-names', () => {
return gulp.src('src/vscode-dts/**')
.pipe(generateApiProposalNames())
.pipe(gulp.dest('src'))
.pipe(apiProposalNamesReporter.end(true));
});
export const watchApiProposalNamesTask = task.define('watch-api-proposal-names', () => {
const task = () => gulp.src('src/vscode-dts/**')
.pipe(generateApiProposalNames())
.pipe(apiProposalNamesReporter.end(true));
return watch('src/vscode-dts/**', { readDelay: 200 })
.pipe(util.debounce(task))
.pipe(gulp.dest('src'));
});
================================================
FILE: build/lib/date.js
================================================
"use strict";
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
Object.defineProperty(exports, "__esModule", { value: true });
exports.writeISODate = writeISODate;
exports.readISODate = readISODate;
const path_1 = __importDefault(require("path"));
const fs_1 = __importDefault(require("fs"));
const root = path_1.default.join(__dirname, '..', '..');
/**
* Writes a `outDir/date` file with the contents of the build
* so that other tasks during the build process can use it and
* all use the same date.
*/
function writeISODate(outDir) {
const result = () => new Promise((resolve, _) => {
const outDirectory = path_1.default.join(root, outDir);
fs_1.default.mkdirSync(outDirectory, { recursive: true });
const date = new Date().toISOString();
fs_1.default.writeFileSync(path_1.default.join(outDirectory, 'date'), date, 'utf8');
resolve();
});
result.taskName = 'build-date-file';
return result;
}
function readISODate(outDir) {
const outDirectory = path_1.default.join(root, outDir);
return fs_1.default.readFileSync(path_1.default.join(outDirectory, 'date'), 'utf8');
}
//# sourceMappingURL=date.js.map
================================================
FILE: build/lib/date.ts
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
import path from 'path';
import fs from 'fs';
const root = path.join(__dirname, '..', '..');
/**
* Writes a `outDir/date` file with the contents of the build
* so that other tasks during the build process can use it and
* all use the same date.
*/
export function writeISODate(outDir: string) {
const result = () => new Promise((resolve, _) => {
const outDirectory = path.join(root, outDir);
fs.mkdirSync(outDirectory, { recursive: true });
const date = new Date().toISOString();
fs.writeFileSync(path.join(outDirectory, 'date'), date, 'utf8');
resolve();
});
result.taskName = 'build-date-file';
return result;
}
export function readISODate(outDir: string): string {
const outDirectory = path.join(root, outDir);
return fs.readFileSync(path.join(outDirectory, 'date'), 'utf8');
}
================================================
FILE: build/lib/dependencies.js
================================================
"use strict";
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
Object.defineProperty(exports, "__esModule", { value: true });
exports.getProductionDependencies = getProductionDependencies;
const fs_1 = __importDefault(require("fs"));
const path_1 = __importDefault(require("path"));
const child_process_1 = __importDefault(require("child_process"));
const root = fs_1.default.realpathSync(path_1.default.dirname(path_1.default.dirname(__dirname)));
function getNpmProductionDependencies(folder) {
let raw;
try {
raw = child_process_1.default.execSync('npm ls --all --omit=dev --parseable', { cwd: folder, encoding: 'utf8', env: { ...process.env, NODE_ENV: 'production' }, stdio: [null, null, null] });
}
catch (err) {
const regex = /^npm ERR! .*$/gm;
let match;
while (match = regex.exec(err.message)) {
if (/ELSPROBLEMS/.test(match[0])) {
continue;
}
else if (/invalid: xterm/.test(match[0])) {
continue;
}
else if (/A complete log of this run/.test(match[0])) {
continue;
}
else {
throw err;
}
}
raw = err.stdout;
}
return raw.split(/\r?\n/).filter(line => {
return !!line.trim() && path_1.default.relative(root, line) !== path_1.default.relative(root, folder);
});
}
function getProductionDependencies(folderPath) {
const result = getNpmProductionDependencies(folderPath);
// Account for distro npm dependencies
const realFolderPath = fs_1.default.realpathSync(folderPath);
const relativeFolderPath = path_1.default.relative(root, realFolderPath);
const distroFolderPath = `${root}/.build/distro/npm/${relativeFolderPath}`;
if (fs_1.default.existsSync(distroFolderPath)) {
result.push(...getNpmProductionDependencies(distroFolderPath));
}
return [...new Set(result)];
}
if (require.main === module) {
console.log(JSON.stringify(getProductionDependencies(root), null, ' '));
}
//# sourceMappingURL=dependencies.js.map
================================================
FILE: build/lib/dependencies.ts
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
import fs from 'fs';
import path from 'path';
import cp from 'child_process';
const root = fs.realpathSync(path.dirname(path.dirname(__dirname)));
function getNpmProductionDependencies(folder: string): string[] {
let raw: string;
try {
raw = cp.execSync('npm ls --all --omit=dev --parseable', { cwd: folder, encoding: 'utf8', env: { ...process.env, NODE_ENV: 'production' }, stdio: [null, null, null] });
} catch (err) {
const regex = /^npm ERR! .*$/gm;
let match: RegExpExecArray | null;
while (match = regex.exec(err.message)) {
if (/ELSPROBLEMS/.test(match[0])) {
continue;
} else if (/invalid: xterm/.test(match[0])) {
continue;
} else if (/A complete log of this run/.test(match[0])) {
continue;
} else {
throw err;
}
}
raw = err.stdout;
}
return raw.split(/\r?\n/).filter(line => {
return !!line.trim() && path.relative(root, line) !== path.relative(root, folder);
});
}
export function getProductionDependencies(folderPath: string): string[] {
const result = getNpmProductionDependencies(folderPath);
// Account for distro npm dependencies
const realFolderPath = fs.realpathSync(folderPath);
const relativeFolderPath = path.relative(root, realFolderPath);
const distroFolderPath = `${root}/.build/distro/npm/${relativeFolderPath}`;
if (fs.existsSync(distroFolderPath)) {
result.push(...getNpmProductionDependencies(distroFolderPath));
}
return [...new Set(result)];
}
if (require.main === module) {
console.log(JSON.stringify(getProductionDependencies(root), null, ' '));
}
================================================
FILE: build/lib/electron.js
================================================
"use strict";
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
if (k2 === undefined) k2 = k;
var desc = Object.getOwnPropertyDescriptor(m, k);
if (!desc || ("get" in desc ? !m.__esModule : desc.writable || desc.configurable)) {
desc = { enumerable: true, get: function() { return m[k]; } };
}
Object.defineProperty(o, k2, desc);
}) : (function(o, m, k, k2) {
if (k2 === undefined) k2 = k;
o[k2] = m[k];
}));
var __setModuleDefault = (this && this.__setModuleDefault) || (Object.create ? (function(o, v) {
Object.defineProperty(o, "default", { enumerable: true, value: v });
}) : function(o, v) {
o["default"] = v;
});
var __importStar = (this && this.__importStar) || (function () {
var ownKeys = function(o) {
ownKeys = Object.getOwnPropertyNames || function (o) {
var ar = [];
for (var k in o) if (Object.prototype.hasOwnProperty.call(o, k)) ar[ar.length] = k;
return ar;
};
return ownKeys(o);
};
return function (mod) {
if (mod && mod.__esModule) return mod;
var result = {};
if (mod != null) for (var k = ownKeys(mod), i = 0; i < k.length; i++) if (k[i] !== "default") __createBinding(result, mod, k[i]);
__setModuleDefault(result, mod);
return result;
};
})();
var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
Object.defineProperty(exports, "__esModule", { value: true });
exports.config = void 0;
const fs_1 = __importDefault(require("fs"));
const path_1 = __importDefault(require("path"));
const vinyl_fs_1 = __importDefault(require("vinyl-fs"));
const gulp_filter_1 = __importDefault(require("gulp-filter"));
const util = __importStar(require("./util"));
const getVersion_1 = require("./getVersion");
function isDocumentSuffix(str) {
return str === 'document' || str === 'script' || str === 'file' || str === 'source code';
}
const root = path_1.default.dirname(path_1.default.dirname(__dirname));
const product = JSON.parse(fs_1.default.readFileSync(path_1.default.join(root, 'product.json'), 'utf8'));
const commit = (0, getVersion_1.getVersion)(root);
function createTemplate(input) {
return (params) => {
return input.replace(/<%=\s*([^\s]+)\s*%>/g, (match, key) => {
return params[key] || match;
});
};
}
const darwinCreditsTemplate = product.darwinCredits && createTemplate(fs_1.default.readFileSync(path_1.default.join(root, product.darwinCredits), 'utf8'));
/**
* Generate a `DarwinDocumentType` given a list of file extensions, an icon name, and an optional suffix or file type name.
* @param extensions A list of file extensions, such as `['bat', 'cmd']`
* @param icon A sentence-cased file type name that matches the lowercase name of a darwin icon resource.
* For example, `'HTML'` instead of `'html'`, or `'Java'` instead of `'java'`.
* This parameter is lowercased before it is used to reference an icon file.
* @param nameOrSuffix An optional suffix or a string to use as the file type. If a suffix is provided,
* it is used with the icon parameter to generate a file type string. If nothing is provided,
* `'document'` is used with the icon parameter to generate file type string.
*
* For example, if you call `darwinBundleDocumentType(..., 'HTML')`, the resulting file type is `"HTML document"`,
* and the `'html'` darwin icon is used.
*
* If you call `darwinBundleDocumentType(..., 'Javascript', 'file')`, the resulting file type is `"Javascript file"`.
* and the `'javascript'` darwin icon is used.
*
* If you call `darwinBundleDocumentType(..., 'bat', 'Windows command script')`, the file type is `"Windows command script"`,
* and the `'bat'` darwin icon is used.
*/
function darwinBundleDocumentType(extensions, icon, nameOrSuffix, utis) {
// If given a suffix, generate a name from it. If not given anything, default to 'document'
if (isDocumentSuffix(nameOrSuffix) || !nameOrSuffix) {
nameOrSuffix = icon.charAt(0).toUpperCase() + icon.slice(1) + ' ' + (nameOrSuffix ?? 'document');
}
return {
name: nameOrSuffix,
role: 'Editor',
ostypes: ['TEXT', 'utxt', 'TUTX', '****'],
extensions,
iconFile: 'resources/darwin/' + icon.toLowerCase() + '.icns',
utis
};
}
/**
* Generate several `DarwinDocumentType`s with unique names and a shared icon.
* @param types A map of file type names to their associated file extensions.
* @param icon A darwin icon resource to use. For example, `'HTML'` would refer to `resources/darwin/html.icns`
*
* Examples:
* ```
* darwinBundleDocumentTypes({ 'C header file': 'h', 'C source code': 'c' },'c')
* darwinBundleDocumentTypes({ 'React source code': ['jsx', 'tsx'] }, 'react')
* ```
*/
function darwinBundleDocumentTypes(types, icon) {
return Object.keys(types).map((name) => {
const extensions = types[name];
return {
name,
role: 'Editor',
ostypes: ['TEXT', 'utxt', 'TUTX', '****'],
extensions: Array.isArray(extensions) ? extensions : [extensions],
iconFile: 'resources/darwin/' + icon + '.icns'
};
});
}
const { electronVersion, msBuildId } = util.getElectronVersion();
exports.config = {
version: electronVersion,
tag: product.electronRepository ? `v${electronVersion}-${msBuildId}` : undefined,
productAppName: product.nameLong,
companyName: 'Microsoft Corporation',
copyright: 'Copyright (C) 2024 Microsoft. All rights reserved',
darwinIcon: 'resources/darwin/code.icns',
darwinBundleIdentifier: product.darwinBundleIdentifier,
darwinApplicationCategoryType: 'public.app-category.developer-tools',
darwinHelpBookFolder: 'VS Code HelpBook',
darwinHelpBookName: 'VS Code HelpBook',
darwinBundleDocumentTypes: [
...darwinBundleDocumentTypes({ 'C header file': 'h', 'C source code': 'c' }, 'c'),
...darwinBundleDocumentTypes({ 'Git configuration file': ['gitattributes', 'gitconfig', 'gitignore'] }, 'config'),
...darwinBundleDocumentTypes({ 'HTML template document': ['asp', 'aspx', 'cshtml', 'jshtm', 'jsp', 'phtml', 'shtml'] }, 'html'),
darwinBundleDocumentType(['bat', 'cmd'], 'bat', 'Windows command script'),
darwinBundleDocumentType(['bowerrc'], 'Bower'),
darwinBundleDocumentType(['config', 'editorconfig', 'ini', 'cfg'], 'config', 'Configuration file'),
darwinBundleDocumentType(['hh', 'hpp', 'hxx', 'h++'], 'cpp', 'C++ header file'),
darwinBundleDocumentType(['cc', 'cpp', 'cxx', 'c++'], 'cpp', 'C++ source code'),
darwinBundleDocumentType(['m'], 'default', 'Objective-C source code'),
darwinBundleDocumentType(['mm'], 'cpp', 'Objective-C++ source code'),
darwinBundleDocumentType(['cs', 'csx'], 'csharp', 'C# source code'),
darwinBundleDocumentType(['css'], 'css', 'CSS'),
darwinBundleDocumentType(['go'], 'go', 'Go source code'),
darwinBundleDocumentType(['htm', 'html', 'xhtml'], 'HTML'),
darwinBundleDocumentType(['jade'], 'Jade'),
darwinBundleDocumentType(['jav', 'java'], 'Java'),
darwinBundleDocumentType(['js', 'jscsrc', 'jshintrc', 'mjs', 'cjs'], 'Javascript', 'file'),
darwinBundleDocumentType(['json'], 'JSON'),
darwinBundleDocumentType(['less'], 'Less'),
darwinBundleDocumentType(['markdown', 'md', 'mdoc', 'mdown', 'mdtext', 'mdtxt', 'mdwn', 'mkd', 'mkdn'], 'Markdown'),
darwinBundleDocumentType(['php'], 'PHP', 'source code'),
darwinBundleDocumentType(['ps1', 'psd1', 'psm1'], 'Powershell', 'script'),
darwinBundleDocumentType(['py', 'pyi'], 'Python', 'script'),
darwinBundleDocumentType(['gemspec', 'rb', 'erb'], 'Ruby', 'source code'),
darwinBundleDocumentType(['scss', 'sass'], 'SASS', 'file'),
darwinBundleDocumentType(['sql'], 'SQL', 'script'),
darwinBundleDocumentType(['ts'], 'TypeScript', 'file'),
darwinBundleDocumentType(['tsx', 'jsx'], 'React', 'source code'),
darwinBundleDocumentType(['vue'], 'Vue', 'source code'),
darwinBundleDocumentType(['ascx', 'csproj', 'dtd', 'plist', 'wxi', 'wxl', 'wxs', 'xml', 'xaml'], 'XML'),
darwinBundleDocumentType(['eyaml', 'eyml', 'yaml', 'yml'], 'YAML'),
darwinBundleDocumentType([
'bash', 'bash_login', 'bash_logout', 'bash_profile', 'bashrc',
'profile', 'rhistory', 'rprofile', 'sh', 'zlogin', 'zlogout',
'zprofile', 'zsh', 'zshenv', 'zshrc'
], 'Shell', 'script'),
// Default icon with specified names
...darwinBundleDocumentTypes({
'Clojure source code': ['clj', 'cljs', 'cljx', 'clojure'],
'VS Code workspace file': 'code-workspace',
'CoffeeScript source code': 'coffee',
'Comma Separated Values': 'csv',
'CMake script': 'cmake',
'Dart script': 'dart',
'Diff file': 'diff',
'Dockerfile': 'dockerfile',
'Gradle file': 'gradle',
'Groovy script': 'groovy',
'Makefile': ['makefile', 'mk'],
'Lua script': 'lua',
'Pug document': 'pug',
'Jupyter': 'ipynb',
'Lockfile': 'lock',
'Log file': 'log',
'Plain Text File': 'txt',
'Xcode project file': 'xcodeproj',
'Xcode workspace file': 'xcworkspace',
'Visual Basic script': 'vb',
'R source code': 'r',
'Rust source code': 'rs',
'Restructured Text document': 'rst',
'LaTeX document': ['tex', 'cls'],
'F# source code': 'fs',
'F# signature file': 'fsi',
'F# script': ['fsx', 'fsscript'],
'SVG document': ['svg'],
'TOML document': 'toml',
'Swift source code': 'swift',
}, 'default'),
// Default icon with default name
darwinBundleDocumentType([
'containerfile', 'ctp', 'dot', 'edn', 'handlebars', 'hbs', 'ml', 'mli',
'pl', 'pl6', 'pm', 'pm6', 'pod', 'pp', 'properties', 'psgi', 'rt', 't'
], 'default', product.nameLong + ' document'),
// Folder support ()
darwinBundleDocumentType([], 'default', 'Folder', ['public.folder'])
],
darwinBundleURLTypes: [{
role: 'Viewer',
name: product.nameLong,
urlSchemes: [product.urlProtocol]
}],
darwinForceDarkModeSupport: true,
darwinCredits: darwinCreditsTemplate ? Buffer.from(darwinCreditsTemplate({ commit: commit, date: new Date().toISOString() })) : undefined,
linuxExecutableName: product.applicationName,
winIcon: 'resources/win32/code.ico',
token: process.env['GITHUB_TOKEN'],
repo: product.electronRepository || undefined,
validateChecksum: true,
checksumFile: path_1.default.join(root, 'build', 'checksums', 'electron.txt'),
};
function getElectron(arch) {
return () => {
const electron = require('@vscode/gulp-electron');
const json = require('gulp-json-editor');
const electronOpts = {
...exports.config,
platform: process.platform,
arch: arch === 'armhf' ? 'arm' : arch,
ffmpegChromium: false,
keepDefaultApp: true
};
return vinyl_fs_1.default.src('package.json')
.pipe(json({ name: product.nameShort }))
.pipe(electron(electronOpts))
.pipe((0, gulp_filter_1.default)(['**', '!**/app/package.json']))
.pipe(vinyl_fs_1.default.dest('.build/electron'));
};
}
async function main(arch = process.arch) {
const version = electronVersion;
const electronPath = path_1.default.join(root, '.build', 'electron');
const versionFile = path_1.default.join(electronPath, 'version');
const isUpToDate = fs_1.default.existsSync(versionFile) && fs_1.default.readFileSync(versionFile, 'utf8') === `${version}`;
if (!isUpToDate) {
await util.rimraf(electronPath)();
await util.streamToPromise(getElectron(arch)());
}
}
if (require.main === module) {
main(process.argv[2]).catch(err => {
console.error(err);
process.exit(1);
});
}
//# sourceMappingURL=electron.js.map
================================================
FILE: build/lib/electron.ts
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
import fs from 'fs';
import path from 'path';
import vfs from 'vinyl-fs';
import filter from 'gulp-filter';
import * as util from './util';
import { getVersion } from './getVersion';
type DarwinDocumentSuffix = 'document' | 'script' | 'file' | 'source code';
type DarwinDocumentType = {
name: string;
role: string;
ostypes: string[];
extensions: string[];
iconFile: string;
utis?: string[];
};
function isDocumentSuffix(str?: string): str is DarwinDocumentSuffix {
return str === 'document' || str === 'script' || str === 'file' || str === 'source code';
}
const root = path.dirname(path.dirname(__dirname));
const product = JSON.parse(fs.readFileSync(path.join(root, 'product.json'), 'utf8'));
const commit = getVersion(root);
function createTemplate(input: string): (params: Record) => string {
return (params: Record) => {
return input.replace(/<%=\s*([^\s]+)\s*%>/g, (match, key) => {
return params[key] || match;
});
};
}
const darwinCreditsTemplate = product.darwinCredits && createTemplate(fs.readFileSync(path.join(root, product.darwinCredits), 'utf8'));
/**
* Generate a `DarwinDocumentType` given a list of file extensions, an icon name, and an optional suffix or file type name.
* @param extensions A list of file extensions, such as `['bat', 'cmd']`
* @param icon A sentence-cased file type name that matches the lowercase name of a darwin icon resource.
* For example, `'HTML'` instead of `'html'`, or `'Java'` instead of `'java'`.
* This parameter is lowercased before it is used to reference an icon file.
* @param nameOrSuffix An optional suffix or a string to use as the file type. If a suffix is provided,
* it is used with the icon parameter to generate a file type string. If nothing is provided,
* `'document'` is used with the icon parameter to generate file type string.
*
* For example, if you call `darwinBundleDocumentType(..., 'HTML')`, the resulting file type is `"HTML document"`,
* and the `'html'` darwin icon is used.
*
* If you call `darwinBundleDocumentType(..., 'Javascript', 'file')`, the resulting file type is `"Javascript file"`.
* and the `'javascript'` darwin icon is used.
*
* If you call `darwinBundleDocumentType(..., 'bat', 'Windows command script')`, the file type is `"Windows command script"`,
* and the `'bat'` darwin icon is used.
*/
function darwinBundleDocumentType(extensions: string[], icon: string, nameOrSuffix?: string | DarwinDocumentSuffix, utis?: string[]): DarwinDocumentType {
// If given a suffix, generate a name from it. If not given anything, default to 'document'
if (isDocumentSuffix(nameOrSuffix) || !nameOrSuffix) {
nameOrSuffix = icon.charAt(0).toUpperCase() + icon.slice(1) + ' ' + (nameOrSuffix ?? 'document');
}
return {
name: nameOrSuffix,
role: 'Editor',
ostypes: ['TEXT', 'utxt', 'TUTX', '****'],
extensions,
iconFile: 'resources/darwin/' + icon.toLowerCase() + '.icns',
utis
};
}
/**
* Generate several `DarwinDocumentType`s with unique names and a shared icon.
* @param types A map of file type names to their associated file extensions.
* @param icon A darwin icon resource to use. For example, `'HTML'` would refer to `resources/darwin/html.icns`
*
* Examples:
* ```
* darwinBundleDocumentTypes({ 'C header file': 'h', 'C source code': 'c' },'c')
* darwinBundleDocumentTypes({ 'React source code': ['jsx', 'tsx'] }, 'react')
* ```
*/
function darwinBundleDocumentTypes(types: { [name: string]: string | string[] }, icon: string): DarwinDocumentType[] {
return Object.keys(types).map((name: string): DarwinDocumentType => {
const extensions = types[name];
return {
name,
role: 'Editor',
ostypes: ['TEXT', 'utxt', 'TUTX', '****'],
extensions: Array.isArray(extensions) ? extensions : [extensions],
iconFile: 'resources/darwin/' + icon + '.icns'
};
});
}
const { electronVersion, msBuildId } = util.getElectronVersion();
export const config = {
version: electronVersion,
tag: product.electronRepository ? `v${electronVersion}-${msBuildId}` : undefined,
productAppName: product.nameLong,
companyName: 'Microsoft Corporation',
copyright: 'Copyright (C) 2024 Microsoft. All rights reserved',
darwinIcon: 'resources/darwin/code.icns',
darwinBundleIdentifier: product.darwinBundleIdentifier,
darwinApplicationCategoryType: 'public.app-category.developer-tools',
darwinHelpBookFolder: 'VS Code HelpBook',
darwinHelpBookName: 'VS Code HelpBook',
darwinBundleDocumentTypes: [
...darwinBundleDocumentTypes({ 'C header file': 'h', 'C source code': 'c' }, 'c'),
...darwinBundleDocumentTypes({ 'Git configuration file': ['gitattributes', 'gitconfig', 'gitignore'] }, 'config'),
...darwinBundleDocumentTypes({ 'HTML template document': ['asp', 'aspx', 'cshtml', 'jshtm', 'jsp', 'phtml', 'shtml'] }, 'html'),
darwinBundleDocumentType(['bat', 'cmd'], 'bat', 'Windows command script'),
darwinBundleDocumentType(['bowerrc'], 'Bower'),
darwinBundleDocumentType(['config', 'editorconfig', 'ini', 'cfg'], 'config', 'Configuration file'),
darwinBundleDocumentType(['hh', 'hpp', 'hxx', 'h++'], 'cpp', 'C++ header file'),
darwinBundleDocumentType(['cc', 'cpp', 'cxx', 'c++'], 'cpp', 'C++ source code'),
darwinBundleDocumentType(['m'], 'default', 'Objective-C source code'),
darwinBundleDocumentType(['mm'], 'cpp', 'Objective-C++ source code'),
darwinBundleDocumentType(['cs', 'csx'], 'csharp', 'C# source code'),
darwinBundleDocumentType(['css'], 'css', 'CSS'),
darwinBundleDocumentType(['go'], 'go', 'Go source code'),
darwinBundleDocumentType(['htm', 'html', 'xhtml'], 'HTML'),
darwinBundleDocumentType(['jade'], 'Jade'),
darwinBundleDocumentType(['jav', 'java'], 'Java'),
darwinBundleDocumentType(['js', 'jscsrc', 'jshintrc', 'mjs', 'cjs'], 'Javascript', 'file'),
darwinBundleDocumentType(['json'], 'JSON'),
darwinBundleDocumentType(['less'], 'Less'),
darwinBundleDocumentType(['markdown', 'md', 'mdoc', 'mdown', 'mdtext', 'mdtxt', 'mdwn', 'mkd', 'mkdn'], 'Markdown'),
darwinBundleDocumentType(['php'], 'PHP', 'source code'),
darwinBundleDocumentType(['ps1', 'psd1', 'psm1'], 'Powershell', 'script'),
darwinBundleDocumentType(['py', 'pyi'], 'Python', 'script'),
darwinBundleDocumentType(['gemspec', 'rb', 'erb'], 'Ruby', 'source code'),
darwinBundleDocumentType(['scss', 'sass'], 'SASS', 'file'),
darwinBundleDocumentType(['sql'], 'SQL', 'script'),
darwinBundleDocumentType(['ts'], 'TypeScript', 'file'),
darwinBundleDocumentType(['tsx', 'jsx'], 'React', 'source code'),
darwinBundleDocumentType(['vue'], 'Vue', 'source code'),
darwinBundleDocumentType(['ascx', 'csproj', 'dtd', 'plist', 'wxi', 'wxl', 'wxs', 'xml', 'xaml'], 'XML'),
darwinBundleDocumentType(['eyaml', 'eyml', 'yaml', 'yml'], 'YAML'),
darwinBundleDocumentType([
'bash', 'bash_login', 'bash_logout', 'bash_profile', 'bashrc',
'profile', 'rhistory', 'rprofile', 'sh', 'zlogin', 'zlogout',
'zprofile', 'zsh', 'zshenv', 'zshrc'
], 'Shell', 'script'),
// Default icon with specified names
...darwinBundleDocumentTypes({
'Clojure source code': ['clj', 'cljs', 'cljx', 'clojure'],
'VS Code workspace file': 'code-workspace',
'CoffeeScript source code': 'coffee',
'Comma Separated Values': 'csv',
'CMake script': 'cmake',
'Dart script': 'dart',
'Diff file': 'diff',
'Dockerfile': 'dockerfile',
'Gradle file': 'gradle',
'Groovy script': 'groovy',
'Makefile': ['makefile', 'mk'],
'Lua script': 'lua',
'Pug document': 'pug',
'Jupyter': 'ipynb',
'Lockfile': 'lock',
'Log file': 'log',
'Plain Text File': 'txt',
'Xcode project file': 'xcodeproj',
'Xcode workspace file': 'xcworkspace',
'Visual Basic script': 'vb',
'R source code': 'r',
'Rust source code': 'rs',
'Restructured Text document': 'rst',
'LaTeX document': ['tex', 'cls'],
'F# source code': 'fs',
'F# signature file': 'fsi',
'F# script': ['fsx', 'fsscript'],
'SVG document': ['svg'],
'TOML document': 'toml',
'Swift source code': 'swift',
}, 'default'),
// Default icon with default name
darwinBundleDocumentType([
'containerfile', 'ctp', 'dot', 'edn', 'handlebars', 'hbs', 'ml', 'mli',
'pl', 'pl6', 'pm', 'pm6', 'pod', 'pp', 'properties', 'psgi', 'rt', 't'
], 'default', product.nameLong + ' document'),
// Folder support ()
darwinBundleDocumentType([], 'default', 'Folder', ['public.folder'])
],
darwinBundleURLTypes: [{
role: 'Viewer',
name: product.nameLong,
urlSchemes: [product.urlProtocol]
}],
darwinForceDarkModeSupport: true,
darwinCredits: darwinCreditsTemplate ? Buffer.from(darwinCreditsTemplate({ commit: commit, date: new Date().toISOString() })) : undefined,
linuxExecutableName: product.applicationName,
winIcon: 'resources/win32/code.ico',
token: process.env['GITHUB_TOKEN'],
repo: product.electronRepository || undefined,
validateChecksum: true,
checksumFile: path.join(root, 'build', 'checksums', 'electron.txt'),
};
function getElectron(arch: string): () => NodeJS.ReadWriteStream {
return () => {
const electron = require('@vscode/gulp-electron');
const json = require('gulp-json-editor') as typeof import('gulp-json-editor');
const electronOpts = {
...config,
platform: process.platform,
arch: arch === 'armhf' ? 'arm' : arch,
ffmpegChromium: false,
keepDefaultApp: true
};
return vfs.src('package.json')
.pipe(json({ name: product.nameShort }))
.pipe(electron(electronOpts))
.pipe(filter(['**', '!**/app/package.json']))
.pipe(vfs.dest('.build/electron'));
};
}
async function main(arch: string = process.arch): Promise {
const version = electronVersion;
const electronPath = path.join(root, '.build', 'electron');
const versionFile = path.join(electronPath, 'version');
const isUpToDate = fs.existsSync(versionFile) && fs.readFileSync(versionFile, 'utf8') === `${version}`;
if (!isUpToDate) {
await util.rimraf(electronPath)();
await util.streamToPromise(getElectron(arch)());
}
}
if (require.main === module) {
main(process.argv[2]).catch(err => {
console.error(err);
process.exit(1);
});
}
================================================
FILE: build/lib/extensions.js
================================================
"use strict";
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
if (k2 === undefined) k2 = k;
var desc = Object.getOwnPropertyDescriptor(m, k);
if (!desc || ("get" in desc ? !m.__esModule : desc.writable || desc.configurable)) {
desc = { enumerable: true, get: function() { return m[k]; } };
}
Object.defineProperty(o, k2, desc);
}) : (function(o, m, k, k2) {
if (k2 === undefined) k2 = k;
o[k2] = m[k];
}));
var __setModuleDefault = (this && this.__setModuleDefault) || (Object.create ? (function(o, v) {
Object.defineProperty(o, "default", { enumerable: true, value: v });
}) : function(o, v) {
o["default"] = v;
});
var __importStar = (this && this.__importStar) || (function () {
var ownKeys = function(o) {
ownKeys = Object.getOwnPropertyNames || function (o) {
var ar = [];
for (var k in o) if (Object.prototype.hasOwnProperty.call(o, k)) ar[ar.length] = k;
return ar;
};
return ownKeys(o);
};
return function (mod) {
if (mod && mod.__esModule) return mod;
var result = {};
if (mod != null) for (var k = ownKeys(mod), i = 0; i < k.length; i++) if (k[i] !== "default") __createBinding(result, mod, k[i]);
__setModuleDefault(result, mod);
return result;
};
})();
var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
Object.defineProperty(exports, "__esModule", { value: true });
exports.fromMarketplace = fromMarketplace;
exports.fromVsix = fromVsix;
exports.fromGithub = fromGithub;
exports.packageNonNativeLocalExtensionsStream = packageNonNativeLocalExtensionsStream;
exports.packageNativeLocalExtensionsStream = packageNativeLocalExtensionsStream;
exports.packageAllLocalExtensionsStream = packageAllLocalExtensionsStream;
exports.packageMarketplaceExtensionsStream = packageMarketplaceExtensionsStream;
exports.scanBuiltinExtensions = scanBuiltinExtensions;
exports.translatePackageJSON = translatePackageJSON;
exports.webpackExtensions = webpackExtensions;
exports.buildExtensionMedia = buildExtensionMedia;
const event_stream_1 = __importDefault(require("event-stream"));
const fs_1 = __importDefault(require("fs"));
const child_process_1 = __importDefault(require("child_process"));
const glob_1 = __importDefault(require("glob"));
const gulp_1 = __importDefault(require("gulp"));
const path_1 = __importDefault(require("path"));
const crypto_1 = __importDefault(require("crypto"));
const vinyl_1 = __importDefault(require("vinyl"));
const stats_1 = require("./stats");
const util2 = __importStar(require("./util"));
const vzip = require('gulp-vinyl-zip');
const gulp_filter_1 = __importDefault(require("gulp-filter"));
const gulp_rename_1 = __importDefault(require("gulp-rename"));
const fancy_log_1 = __importDefault(require("fancy-log"));
const ansi_colors_1 = __importDefault(require("ansi-colors"));
const gulp_buffer_1 = __importDefault(require("gulp-buffer"));
const jsoncParser = __importStar(require("jsonc-parser"));
const dependencies_1 = require("./dependencies");
const builtInExtensions_1 = require("./builtInExtensions");
const getVersion_1 = require("./getVersion");
const fetch_1 = require("./fetch");
const root = path_1.default.dirname(path_1.default.dirname(__dirname));
const commit = (0, getVersion_1.getVersion)(root);
const sourceMappingURLBase = `https://main.vscode-cdn.net/sourcemaps/${commit}`;
function minifyExtensionResources(input) {
const jsonFilter = (0, gulp_filter_1.default)(['**/*.json', '**/*.code-snippets'], { restore: true });
return input
.pipe(jsonFilter)
.pipe((0, gulp_buffer_1.default)())
.pipe(event_stream_1.default.mapSync((f) => {
const errors = [];
const value = jsoncParser.parse(f.contents.toString('utf8'), errors, { allowTrailingComma: true });
if (errors.length === 0) {
// file parsed OK => just stringify to drop whitespace and comments
f.contents = Buffer.from(JSON.stringify(value));
}
return f;
}))
.pipe(jsonFilter.restore);
}
function updateExtensionPackageJSON(input, update) {
const packageJsonFilter = (0, gulp_filter_1.default)('extensions/*/package.json', { restore: true });
return input
.pipe(packageJsonFilter)
.pipe((0, gulp_buffer_1.default)())
.pipe(event_stream_1.default.mapSync((f) => {
const data = JSON.parse(f.contents.toString('utf8'));
f.contents = Buffer.from(JSON.stringify(update(data)));
return f;
}))
.pipe(packageJsonFilter.restore);
}
function fromLocal(extensionPath, forWeb, disableMangle) {
const webpackConfigFileName = forWeb ? 'extension-browser.webpack.config.js' : 'extension.webpack.config.js';
const isWebPacked = fs_1.default.existsSync(path_1.default.join(extensionPath, webpackConfigFileName));
let input = isWebPacked
? fromLocalWebpack(extensionPath, webpackConfigFileName, disableMangle)
: fromLocalNormal(extensionPath);
if (isWebPacked) {
input = updateExtensionPackageJSON(input, (data) => {
delete data.scripts;
delete data.dependencies;
delete data.devDependencies;
if (data.main) {
data.main = data.main.replace('/out/', '/dist/');
}
return data;
});
}
return input;
}
function fromLocalWebpack(extensionPath, webpackConfigFileName, disableMangle) {
const vsce = require('@vscode/vsce');
const webpack = require('webpack');
const webpackGulp = require('webpack-stream');
const result = event_stream_1.default.through();
const packagedDependencies = [];
const packageJsonConfig = require(path_1.default.join(extensionPath, 'package.json'));
if (packageJsonConfig.dependencies) {
const webpackRootConfig = require(path_1.default.join(extensionPath, webpackConfigFileName));
for (const key in webpackRootConfig.externals) {
if (key in packageJsonConfig.dependencies) {
packagedDependencies.push(key);
}
}
}
// TODO: add prune support based on packagedDependencies to vsce.PackageManager.Npm similar
// to vsce.PackageManager.Yarn.
// A static analysis showed there are no webpack externals that are dependencies of the current
// local extensions so we can use the vsce.PackageManager.None config to ignore dependencies list
// as a temporary workaround.
vsce.listFiles({ cwd: extensionPath, packageManager: vsce.PackageManager.None, packagedDependencies }).then(fileNames => {
const files = fileNames
.map(fileName => path_1.default.join(extensionPath, fileName))
.map(filePath => new vinyl_1.default({
path: filePath,
stat: fs_1.default.statSync(filePath),
base: extensionPath,
contents: fs_1.default.createReadStream(filePath)
}));
// check for a webpack configuration files, then invoke webpack
// and merge its output with the files stream.
const webpackConfigLocations = glob_1.default.sync(path_1.default.join(extensionPath, '**', webpackConfigFileName), { ignore: ['**/node_modules'] });
const webpackStreams = webpackConfigLocations.flatMap(webpackConfigPath => {
const webpackDone = (err, stats) => {
(0, fancy_log_1.default)(`Bundled extension: ${ansi_colors_1.default.yellow(path_1.default.join(path_1.default.basename(extensionPath), path_1.default.relative(extensionPath, webpackConfigPath)))}...`);
if (err) {
result.emit('error', err);
}
const { compilation } = stats;
if (compilation.errors.length > 0) {
result.emit('error', compilation.errors.join('\n'));
}
if (compilation.warnings.length > 0) {
result.emit('error', compilation.warnings.join('\n'));
}
};
const exportedConfig = require(webpackConfigPath);
return (Array.isArray(exportedConfig) ? exportedConfig : [exportedConfig]).map(config => {
const webpackConfig = {
...config,
...{ mode: 'production' }
};
if (disableMangle) {
if (Array.isArray(config.module.rules)) {
for (const rule of config.module.rules) {
if (Array.isArray(rule.use)) {
for (const use of rule.use) {
if (String(use.loader).endsWith('mangle-loader.js')) {
use.options.disabled = true;
}
}
}
}
}
}
const relativeOutputPath = path_1.default.relative(extensionPath, webpackConfig.output.path);
return webpackGulp(webpackConfig, webpack, webpackDone)
.pipe(event_stream_1.default.through(function (data) {
data.stat = data.stat || {};
data.base = extensionPath;
this.emit('data', data);
}))
.pipe(event_stream_1.default.through(function (data) {
// source map handling:
// * rewrite sourceMappingURL
// * save to disk so that upload-task picks this up
if (path_1.default.extname(data.basename) === '.js') {
const contents = data.contents.toString('utf8');
data.contents = Buffer.from(contents.replace(/\n\/\/# sourceMappingURL=(.*)$/gm, function (_m, g1) {
return `\n//# sourceMappingURL=${sourceMappingURLBase}/extensions/${path_1.default.basename(extensionPath)}/${relativeOutputPath}/${g1}`;
}), 'utf8');
}
this.emit('data', data);
}));
});
});
event_stream_1.default.merge(...webpackStreams, event_stream_1.default.readArray(files))
// .pipe(es.through(function (data) {
// // debug
// console.log('out', data.path, data.contents.length);
// this.emit('data', data);
// }))
.pipe(result);
}).catch(err => {
console.error(extensionPath);
console.error(packagedDependencies);
result.emit('error', err);
});
return result.pipe((0, stats_1.createStatsStream)(path_1.default.basename(extensionPath)));
}
function fromLocalNormal(extensionPath) {
const vsce = require('@vscode/vsce');
const result = event_stream_1.default.through();
vsce.listFiles({ cwd: extensionPath, packageManager: vsce.PackageManager.Npm })
.then(fileNames => {
const files = fileNames
.map(fileName => path_1.default.join(extensionPath, fileName))
.map(filePath => new vinyl_1.default({
path: filePath,
stat: fs_1.default.statSync(filePath),
base: extensionPath,
contents: fs_1.default.createReadStream(filePath)
}));
event_stream_1.default.readArray(files).pipe(result);
})
.catch(err => result.emit('error', err));
return result.pipe((0, stats_1.createStatsStream)(path_1.default.basename(extensionPath)));
}
const userAgent = 'VSCode Build';
const baseHeaders = {
'X-Market-Client-Id': 'VSCode Build',
'User-Agent': userAgent,
'X-Market-User-Id': '291C1CD0-051A-4123-9B4B-30D60EF52EE2',
};
function fromMarketplace(serviceUrl, { name: extensionName, version, sha256, metadata }) {
const json = require('gulp-json-editor');
const [publisher, name] = extensionName.split('.');
const url = `${serviceUrl}/publishers/${publisher}/vsextensions/${name}/${version}/vspackage`;
(0, fancy_log_1.default)('Downloading extension:', ansi_colors_1.default.yellow(`${extensionName}@${version}`), '...');
const packageJsonFilter = (0, gulp_filter_1.default)('package.json', { restore: true });
return (0, fetch_1.fetchUrls)('', {
base: url,
nodeFetchOptions: {
headers: baseHeaders
},
checksumSha256: sha256
})
.pipe(vzip.src())
.pipe((0, gulp_filter_1.default)('extension/**'))
.pipe((0, gulp_rename_1.default)(p => p.dirname = p.dirname.replace(/^extension\/?/, '')))
.pipe(packageJsonFilter)
.pipe((0, gulp_buffer_1.default)())
.pipe(json({ __metadata: metadata }))
.pipe(packageJsonFilter.restore);
}
function fromVsix(vsixPath, { name: extensionName, version, sha256, metadata }) {
const json = require('gulp-json-editor');
(0, fancy_log_1.default)('Using local VSIX for extension:', ansi_colors_1.default.yellow(`${extensionName}@${version}`), '...');
const packageJsonFilter = (0, gulp_filter_1.default)('package.json', { restore: true });
return gulp_1.default.src(vsixPath)
.pipe((0, gulp_buffer_1.default)())
.pipe(event_stream_1.default.mapSync((f) => {
const hash = crypto_1.default.createHash('sha256');
hash.update(f.contents);
const checksum = hash.digest('hex');
if (checksum !== sha256) {
throw new Error(`Checksum mismatch for ${vsixPath} (expected ${sha256}, actual ${checksum}))`);
}
return f;
}))
.pipe(vzip.src())
.pipe((0, gulp_filter_1.default)('extension/**'))
.pipe((0, gulp_rename_1.default)(p => p.dirname = p.dirname.replace(/^extension\/?/, '')))
.pipe(packageJsonFilter)
.pipe((0, gulp_buffer_1.default)())
.pipe(json({ __metadata: metadata }))
.pipe(packageJsonFilter.restore);
}
function fromGithub({ name, version, repo, sha256, metadata }) {
const json = require('gulp-json-editor');
(0, fancy_log_1.default)('Downloading extension from GH:', ansi_colors_1.default.yellow(`${name}@${version}`), '...');
const packageJsonFilter = (0, gulp_filter_1.default)('package.json', { restore: true });
return (0, fetch_1.fetchGithub)(new URL(repo).pathname, {
version,
name: name => name.endsWith('.vsix'),
checksumSha256: sha256
})
.pipe((0, gulp_buffer_1.default)())
.pipe(vzip.src())
.pipe((0, gulp_filter_1.default)('extension/**'))
.pipe((0, gulp_rename_1.default)(p => p.dirname = p.dirname.replace(/^extension\/?/, '')))
.pipe(packageJsonFilter)
.pipe((0, gulp_buffer_1.default)())
.pipe(json({ __metadata: metadata }))
.pipe(packageJsonFilter.restore);
}
/**
* All extensions that are known to have some native component and thus must be built on the
* platform that is being built.
*/
const nativeExtensions = [
'microsoft-authentication',
];
const excludedExtensions = [
'vscode-api-tests',
'vscode-colorize-tests',
'vscode-colorize-perf-tests',
'vscode-test-resolver',
'ms-vscode.node-debug',
'ms-vscode.node-debug2',
];
const marketplaceWebExtensionsExclude = new Set([
'ms-vscode.node-debug',
'ms-vscode.node-debug2',
'ms-vscode.js-debug-companion',
'ms-vscode.js-debug',
'ms-vscode.vscode-js-profile-table'
]);
const productJson = JSON.parse(fs_1.default.readFileSync(path_1.default.join(__dirname, '../../product.json'), 'utf8'));
const builtInExtensions = productJson.builtInExtensions || [];
const webBuiltInExtensions = productJson.webBuiltInExtensions || [];
/**
* Loosely based on `getExtensionKind` from `src/vs/workbench/services/extensions/common/extensionManifestPropertiesService.ts`
*/
function isWebExtension(manifest) {
if (Boolean(manifest.browser)) {
return true;
}
if (Boolean(manifest.main)) {
return false;
}
// neither browser nor main
if (typeof manifest.extensionKind !== 'undefined') {
const extensionKind = Array.isArray(manifest.extensionKind) ? manifest.extensionKind : [manifest.extensionKind];
if (extensionKind.indexOf('web') >= 0) {
return true;
}
}
if (typeof manifest.contributes !== 'undefined') {
for (const id of ['debuggers', 'terminal', 'typescriptServerPlugins']) {
if (manifest.contributes.hasOwnProperty(id)) {
return false;
}
}
}
return true;
}
/**
* Package local extensions that are known to not have native dependencies. Mutually exclusive to {@link packageNativeLocalExtensionsStream}.
* @param forWeb build the extensions that have web targets
* @param disableMangle disable the mangler
* @returns a stream
*/
function packageNonNativeLocalExtensionsStream(forWeb, disableMangle) {
return doPackageLocalExtensionsStream(forWeb, disableMangle, false);
}
/**
* Package local extensions that are known to have native dependencies. Mutually exclusive to {@link packageNonNativeLocalExtensionsStream}.
* @note it's possible that the extension does not have native dependencies for the current platform, especially if building for the web,
* but we simplify the logic here by having a flat list of extensions (See {@link nativeExtensions}) that are known to have native
* dependencies on some platform and thus should be packaged on the platform that they are building for.
* @param forWeb build the extensions that have web targets
* @param disableMangle disable the mangler
* @returns a stream
*/
function packageNativeLocalExtensionsStream(forWeb, disableMangle) {
return doPackageLocalExtensionsStream(forWeb, disableMangle, true);
}
/**
* Package all the local extensions... both those that are known to have native dependencies and those that are not.
* @param forWeb build the extensions that have web targets
* @param disableMangle disable the mangler
* @returns a stream
*/
function packageAllLocalExtensionsStream(forWeb, disableMangle) {
return event_stream_1.default.merge([
packageNonNativeLocalExtensionsStream(forWeb, disableMangle),
packageNativeLocalExtensionsStream(forWeb, disableMangle)
]);
}
/**
* @param forWeb build the extensions that have web targets
* @param disableMangle disable the mangler
* @param native build the extensions that are marked as having native dependencies
*/
function doPackageLocalExtensionsStream(forWeb, disableMangle, native) {
const nativeExtensionsSet = new Set(nativeExtensions);
const localExtensionsDescriptions = (glob_1.default.sync('extensions/*/package.json')
.map(manifestPath => {
const absoluteManifestPath = path_1.default.join(root, manifestPath);
const extensionPath = path_1.default.dirname(path_1.default.join(root, manifestPath));
const extensionName = path_1.default.basename(extensionPath);
return { name: extensionName, path: extensionPath, manifestPath: absoluteManifestPath };
})
.filter(({ name }) => native ? nativeExtensionsSet.has(name) : !nativeExtensionsSet.has(name))
.filter(({ name }) => excludedExtensions.indexOf(name) === -1)
.filter(({ name }) => builtInExtensions.every(b => b.name !== name))
.filter(({ manifestPath }) => (forWeb ? isWebExtension(require(manifestPath)) : true)));
const localExtensionsStream = minifyExtensionResources(event_stream_1.default.merge(...localExtensionsDescriptions.map(extension => {
return fromLocal(extension.path, forWeb, disableMangle)
.pipe((0, gulp_rename_1.default)(p => p.dirname = `extensions/${extension.name}/${p.dirname}`));
})));
let result;
if (forWeb) {
result = localExtensionsStream;
}
else {
// also include shared production node modules
const productionDependencies = (0, dependencies_1.getProductionDependencies)('extensions/');
const dependenciesSrc = productionDependencies.map(d => path_1.default.relative(root, d)).map(d => [`${d}/**`, `!${d}/**/{test,tests}/**`]).flat();
result = event_stream_1.default.merge(localExtensionsStream, gulp_1.default.src(dependenciesSrc, { base: '.' })
.pipe(util2.cleanNodeModules(path_1.default.join(root, 'build', '.moduleignore')))
.pipe(util2.cleanNodeModules(path_1.default.join(root, 'build', `.moduleignore.${process.platform}`))));
}
return (result
.pipe(util2.setExecutableBit(['**/*.sh'])));
}
function packageMarketplaceExtensionsStream(forWeb) {
const marketplaceExtensionsDescriptions = [
...builtInExtensions.filter(({ name }) => (forWeb ? !marketplaceWebExtensionsExclude.has(name) : true)),
...(forWeb ? webBuiltInExtensions : [])
];
const marketplaceExtensionsStream = minifyExtensionResources(event_stream_1.default.merge(...marketplaceExtensionsDescriptions
.map(extension => {
const src = (0, builtInExtensions_1.getExtensionStream)(extension).pipe((0, gulp_rename_1.default)(p => p.dirname = `extensions/${p.dirname}`));
return updateExtensionPackageJSON(src, (data) => {
delete data.scripts;
delete data.dependencies;
delete data.devDependencies;
return data;
});
})));
return (marketplaceExtensionsStream
.pipe(util2.setExecutableBit(['**/*.sh'])));
}
function scanBuiltinExtensions(extensionsRoot, exclude = []) {
const scannedExtensions = [];
try {
const extensionsFolders = fs_1.default.readdirSync(extensionsRoot);
for (const extensionFolder of extensionsFolders) {
if (exclude.indexOf(extensionFolder) >= 0) {
continue;
}
const packageJSONPath = path_1.default.join(extensionsRoot, extensionFolder, 'package.json');
if (!fs_1.default.existsSync(packageJSONPath)) {
continue;
}
const packageJSON = JSON.parse(fs_1.default.readFileSync(packageJSONPath).toString('utf8'));
if (!isWebExtension(packageJSON)) {
continue;
}
const children = fs_1.default.readdirSync(path_1.default.join(extensionsRoot, extensionFolder));
const packageNLSPath = children.filter(child => child === 'package.nls.json')[0];
const packageNLS = packageNLSPath ? JSON.parse(fs_1.default.readFileSync(path_1.default.join(extensionsRoot, extensionFolder, packageNLSPath)).toString()) : undefined;
const readme = children.filter(child => /^readme(\.txt|\.md|)$/i.test(child))[0];
const changelog = children.filter(child => /^changelog(\.txt|\.md|)$/i.test(child))[0];
scannedExtensions.push({
extensionPath: extensionFolder,
packageJSON,
packageNLS,
readmePath: readme ? path_1.default.join(extensionFolder, readme) : undefined,
changelogPath: changelog ? path_1.default.join(extensionFolder, changelog) : undefined,
});
}
return scannedExtensions;
}
catch (ex) {
return scannedExtensions;
}
}
function translatePackageJSON(packageJSON, packageNLSPath) {
const CharCode_PC = '%'.charCodeAt(0);
const packageNls = JSON.parse(fs_1.default.readFileSync(packageNLSPath).toString());
const translate = (obj) => {
for (const key in obj) {
const val = obj[key];
if (Array.isArray(val)) {
val.forEach(translate);
}
else if (val && typeof val === 'object') {
translate(val);
}
else if (typeof val === 'string' && val.charCodeAt(0) === CharCode_PC && val.charCodeAt(val.length - 1) === CharCode_PC) {
const translated = packageNls[val.substr(1, val.length - 2)];
if (translated) {
obj[key] = typeof translated === 'string' ? translated : (typeof translated.message === 'string' ? translated.message : val);
}
}
}
};
translate(packageJSON);
return packageJSON;
}
const extensionsPath = path_1.default.join(root, 'extensions');
// Additional projects to run esbuild on. These typically build code for webviews
const esbuildMediaScripts = [
'markdown-language-features/esbuild-notebook.js',
'markdown-language-features/esbuild-preview.js',
'markdown-math/esbuild.js',
'notebook-renderers/esbuild.js',
'ipynb/esbuild.js',
'simple-browser/esbuild-preview.js',
];
async function webpackExtensions(taskName, isWatch, webpackConfigLocations) {
const webpack = require('webpack');
const webpackConfigs = [];
for (const { configPath, outputRoot } of webpackConfigLocations) {
const configOrFnOrArray = require(configPath);
function addConfig(configOrFnOrArray) {
for (const configOrFn of Array.isArray(configOrFnOrArray) ? configOrFnOrArray : [configOrFnOrArray]) {
const config = typeof configOrFn === 'function' ? configOrFn({}, {}) : configOrFn;
if (outputRoot) {
config.output.path = path_1.default.join(outputRoot, path_1.default.relative(path_1.default.dirname(configPath), config.output.path));
}
webpackConfigs.push(config);
}
}
addConfig(configOrFnOrArray);
}
function reporter(fullStats) {
if (Array.isArray(fullStats.children)) {
for (const stats of fullStats.children) {
const outputPath = stats.outputPath;
if (outputPath) {
const relativePath = path_1.default.relative(extensionsPath, outputPath).replace(/\\/g, '/');
const match = relativePath.match(/[^\/]+(\/server|\/client)?/);
(0, fancy_log_1.default)(`Finished ${ansi_colors_1.default.green(taskName)} ${ansi_colors_1.default.cyan(match[0])} with ${stats.errors.length} errors.`);
}
if (Array.isArray(stats.errors)) {
stats.errors.forEach((error) => {
fancy_log_1.default.error(error);
});
}
if (Array.isArray(stats.warnings)) {
stats.warnings.forEach((warning) => {
fancy_log_1.default.warn(warning);
});
}
}
}
}
return new Promise((resolve, reject) => {
if (isWatch) {
webpack(webpackConfigs).watch({}, (err, stats) => {
if (err) {
reject();
}
else {
reporter(stats?.toJson());
}
});
}
else {
webpack(webpackConfigs).run((err, stats) => {
if (err) {
fancy_log_1.default.error(err);
reject();
}
else {
reporter(stats?.toJson());
resolve();
}
});
}
});
}
async function esbuildExtensions(taskName, isWatch, scripts) {
function reporter(stdError, script) {
const matches = (stdError || '').match(/\> (.+): error: (.+)?/g);
(0, fancy_log_1.default)(`Finished ${ansi_colors_1.default.green(taskName)} ${script} with ${matches ? matches.length : 0} errors.`);
for (const match of matches || []) {
fancy_log_1.default.error(match);
}
}
const tasks = scripts.map(({ script, outputRoot }) => {
return new Promise((resolve, reject) => {
const args = [script];
if (isWatch) {
args.push('--watch');
}
if (outputRoot) {
args.push('--outputRoot', outputRoot);
}
const proc = child_process_1.default.execFile(process.argv[0], args, {}, (error, _stdout, stderr) => {
if (error) {
return reject(error);
}
reporter(stderr, script);
return resolve();
});
proc.stdout.on('data', (data) => {
(0, fancy_log_1.default)(`${ansi_colors_1.default.green(taskName)}: ${data.toString('utf8')}`);
});
});
});
return Promise.all(tasks);
}
async function buildExtensionMedia(isWatch, outputRoot) {
return esbuildExtensions('esbuilding extension media', isWatch, esbuildMediaScripts.map(p => ({
script: path_1.default.join(extensionsPath, p),
outputRoot: outputRoot ? path_1.default.join(root, outputRoot, path_1.default.dirname(p)) : undefined
})));
}
//# sourceMappingURL=extensions.js.map
================================================
FILE: build/lib/extensions.ts
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
import es from 'event-stream';
import fs from 'fs';
import cp from 'child_process';
import glob from 'glob';
import gulp from 'gulp';
import path from 'path';
import crypto from 'crypto';
import { Stream } from 'stream';
import File from 'vinyl';
import { createStatsStream } from './stats';
import * as util2 from './util';
const vzip = require('gulp-vinyl-zip');
import filter from 'gulp-filter';
import rename from 'gulp-rename';
import fancyLog from 'fancy-log';
import ansiColors from 'ansi-colors';
import buffer from 'gulp-buffer';
import * as jsoncParser from 'jsonc-parser';
import webpack from 'webpack';
import { getProductionDependencies } from './dependencies';
import { IExtensionDefinition, getExtensionStream } from './builtInExtensions';
import { getVersion } from './getVersion';
import { fetchUrls, fetchGithub } from './fetch';
const root = path.dirname(path.dirname(__dirname));
const commit = getVersion(root);
const sourceMappingURLBase = `https://main.vscode-cdn.net/sourcemaps/${commit}`;
function minifyExtensionResources(input: Stream): Stream {
const jsonFilter = filter(['**/*.json', '**/*.code-snippets'], { restore: true });
return input
.pipe(jsonFilter)
.pipe(buffer())
.pipe(es.mapSync((f: File) => {
const errors: jsoncParser.ParseError[] = [];
const value = jsoncParser.parse(f.contents.toString('utf8'), errors, { allowTrailingComma: true });
if (errors.length === 0) {
// file parsed OK => just stringify to drop whitespace and comments
f.contents = Buffer.from(JSON.stringify(value));
}
return f;
}))
.pipe(jsonFilter.restore);
}
function updateExtensionPackageJSON(input: Stream, update: (data: any) => any): Stream {
const packageJsonFilter = filter('extensions/*/package.json', { restore: true });
return input
.pipe(packageJsonFilter)
.pipe(buffer())
.pipe(es.mapSync((f: File) => {
const data = JSON.parse(f.contents.toString('utf8'));
f.contents = Buffer.from(JSON.stringify(update(data)));
return f;
}))
.pipe(packageJsonFilter.restore);
}
function fromLocal(extensionPath: string, forWeb: boolean, disableMangle: boolean): Stream {
const webpackConfigFileName = forWeb ? 'extension-browser.webpack.config.js' : 'extension.webpack.config.js';
const isWebPacked = fs.existsSync(path.join(extensionPath, webpackConfigFileName));
let input = isWebPacked
? fromLocalWebpack(extensionPath, webpackConfigFileName, disableMangle)
: fromLocalNormal(extensionPath);
if (isWebPacked) {
input = updateExtensionPackageJSON(input, (data: any) => {
delete data.scripts;
delete data.dependencies;
delete data.devDependencies;
if (data.main) {
data.main = data.main.replace('/out/', '/dist/');
}
return data;
});
}
return input;
}
function fromLocalWebpack(extensionPath: string, webpackConfigFileName: string, disableMangle: boolean): Stream {
const vsce = require('@vscode/vsce') as typeof import('@vscode/vsce');
const webpack = require('webpack');
const webpackGulp = require('webpack-stream');
const result = es.through();
const packagedDependencies: string[] = [];
const packageJsonConfig = require(path.join(extensionPath, 'package.json'));
if (packageJsonConfig.dependencies) {
const webpackRootConfig = require(path.join(extensionPath, webpackConfigFileName));
for (const key in webpackRootConfig.externals) {
if (key in packageJsonConfig.dependencies) {
packagedDependencies.push(key);
}
}
}
// TODO: add prune support based on packagedDependencies to vsce.PackageManager.Npm similar
// to vsce.PackageManager.Yarn.
// A static analysis showed there are no webpack externals that are dependencies of the current
// local extensions so we can use the vsce.PackageManager.None config to ignore dependencies list
// as a temporary workaround.
vsce.listFiles({ cwd: extensionPath, packageManager: vsce.PackageManager.None, packagedDependencies }).then(fileNames => {
const files = fileNames
.map(fileName => path.join(extensionPath, fileName))
.map(filePath => new File({
path: filePath,
stat: fs.statSync(filePath),
base: extensionPath,
contents: fs.createReadStream(filePath) as any
}));
// check for a webpack configuration files, then invoke webpack
// and merge its output with the files stream.
const webpackConfigLocations = (glob.sync(
path.join(extensionPath, '**', webpackConfigFileName),
{ ignore: ['**/node_modules'] }
));
const webpackStreams = webpackConfigLocations.flatMap(webpackConfigPath => {
const webpackDone = (err: any, stats: any) => {
fancyLog(`Bundled extension: ${ansiColors.yellow(path.join(path.basename(extensionPath), path.relative(extensionPath, webpackConfigPath)))}...`);
if (err) {
result.emit('error', err);
}
const { compilation } = stats;
if (compilation.errors.length > 0) {
result.emit('error', compilation.errors.join('\n'));
}
if (compilation.warnings.length > 0) {
result.emit('error', compilation.warnings.join('\n'));
}
};
const exportedConfig = require(webpackConfigPath);
return (Array.isArray(exportedConfig) ? exportedConfig : [exportedConfig]).map(config => {
const webpackConfig = {
...config,
...{ mode: 'production' }
};
if (disableMangle) {
if (Array.isArray(config.module.rules)) {
for (const rule of config.module.rules) {
if (Array.isArray(rule.use)) {
for (const use of rule.use) {
if (String(use.loader).endsWith('mangle-loader.js')) {
use.options.disabled = true;
}
}
}
}
}
}
const relativeOutputPath = path.relative(extensionPath, webpackConfig.output.path);
return webpackGulp(webpackConfig, webpack, webpackDone)
.pipe(es.through(function (data) {
data.stat = data.stat || {};
data.base = extensionPath;
this.emit('data', data);
}))
.pipe(es.through(function (data: File) {
// source map handling:
// * rewrite sourceMappingURL
// * save to disk so that upload-task picks this up
if (path.extname(data.basename) === '.js') {
const contents = (data.contents).toString('utf8');
data.contents = Buffer.from(contents.replace(/\n\/\/# sourceMappingURL=(.*)$/gm, function (_m, g1) {
return `\n//# sourceMappingURL=${sourceMappingURLBase}/extensions/${path.basename(extensionPath)}/${relativeOutputPath}/${g1}`;
}), 'utf8');
}
this.emit('data', data);
}));
});
});
es.merge(...webpackStreams, es.readArray(files))
// .pipe(es.through(function (data) {
// // debug
// console.log('out', data.path, data.contents.length);
// this.emit('data', data);
// }))
.pipe(result);
}).catch(err => {
console.error(extensionPath);
console.error(packagedDependencies);
result.emit('error', err);
});
return result.pipe(createStatsStream(path.basename(extensionPath)));
}
function fromLocalNormal(extensionPath: string): Stream {
const vsce = require('@vscode/vsce') as typeof import('@vscode/vsce');
const result = es.through();
vsce.listFiles({ cwd: extensionPath, packageManager: vsce.PackageManager.Npm })
.then(fileNames => {
const files = fileNames
.map(fileName => path.join(extensionPath, fileName))
.map(filePath => new File({
path: filePath,
stat: fs.statSync(filePath),
base: extensionPath,
contents: fs.createReadStream(filePath) as any
}));
es.readArray(files).pipe(result);
})
.catch(err => result.emit('error', err));
return result.pipe(createStatsStream(path.basename(extensionPath)));
}
const userAgent = 'VSCode Build';
const baseHeaders = {
'X-Market-Client-Id': 'VSCode Build',
'User-Agent': userAgent,
'X-Market-User-Id': '291C1CD0-051A-4123-9B4B-30D60EF52EE2',
};
export function fromMarketplace(serviceUrl: string, { name: extensionName, version, sha256, metadata }: IExtensionDefinition): Stream {
const json = require('gulp-json-editor') as typeof import('gulp-json-editor');
const [publisher, name] = extensionName.split('.');
const url = `${serviceUrl}/publishers/${publisher}/vsextensions/${name}/${version}/vspackage`;
fancyLog('Downloading extension:', ansiColors.yellow(`${extensionName}@${version}`), '...');
const packageJsonFilter = filter('package.json', { restore: true });
return fetchUrls('', {
base: url,
nodeFetchOptions: {
headers: baseHeaders
},
checksumSha256: sha256
})
.pipe(vzip.src())
.pipe(filter('extension/**'))
.pipe(rename(p => p.dirname = p.dirname!.replace(/^extension\/?/, '')))
.pipe(packageJsonFilter)
.pipe(buffer())
.pipe(json({ __metadata: metadata }))
.pipe(packageJsonFilter.restore);
}
export function fromVsix(vsixPath: string, { name: extensionName, version, sha256, metadata }: IExtensionDefinition): Stream {
const json = require('gulp-json-editor') as typeof import('gulp-json-editor');
fancyLog('Using local VSIX for extension:', ansiColors.yellow(`${extensionName}@${version}`), '...');
const packageJsonFilter = filter('package.json', { restore: true });
return gulp.src(vsixPath)
.pipe(buffer())
.pipe(es.mapSync((f: File) => {
const hash = crypto.createHash('sha256');
hash.update(f.contents as Buffer);
const checksum = hash.digest('hex');
if (checksum !== sha256) {
throw new Error(`Checksum mismatch for ${vsixPath} (expected ${sha256}, actual ${checksum}))`);
}
return f;
}))
.pipe(vzip.src())
.pipe(filter('extension/**'))
.pipe(rename(p => p.dirname = p.dirname!.replace(/^extension\/?/, '')))
.pipe(packageJsonFilter)
.pipe(buffer())
.pipe(json({ __metadata: metadata }))
.pipe(packageJsonFilter.restore);
}
export function fromGithub({ name, version, repo, sha256, metadata }: IExtensionDefinition): Stream {
const json = require('gulp-json-editor') as typeof import('gulp-json-editor');
fancyLog('Downloading extension from GH:', ansiColors.yellow(`${name}@${version}`), '...');
const packageJsonFilter = filter('package.json', { restore: true });
return fetchGithub(new URL(repo).pathname, {
version,
name: name => name.endsWith('.vsix'),
checksumSha256: sha256
})
.pipe(buffer())
.pipe(vzip.src())
.pipe(filter('extension/**'))
.pipe(rename(p => p.dirname = p.dirname!.replace(/^extension\/?/, '')))
.pipe(packageJsonFilter)
.pipe(buffer())
.pipe(json({ __metadata: metadata }))
.pipe(packageJsonFilter.restore);
}
/**
* All extensions that are known to have some native component and thus must be built on the
* platform that is being built.
*/
const nativeExtensions = [
'microsoft-authentication',
];
const excludedExtensions = [
'vscode-api-tests',
'vscode-colorize-tests',
'vscode-colorize-perf-tests',
'vscode-test-resolver',
'ms-vscode.node-debug',
'ms-vscode.node-debug2',
];
const marketplaceWebExtensionsExclude = new Set([
'ms-vscode.node-debug',
'ms-vscode.node-debug2',
'ms-vscode.js-debug-companion',
'ms-vscode.js-debug',
'ms-vscode.vscode-js-profile-table'
]);
const productJson = JSON.parse(fs.readFileSync(path.join(__dirname, '../../product.json'), 'utf8'));
const builtInExtensions: IExtensionDefinition[] = productJson.builtInExtensions || [];
const webBuiltInExtensions: IExtensionDefinition[] = productJson.webBuiltInExtensions || [];
type ExtensionKind = 'ui' | 'workspace' | 'web';
interface IExtensionManifest {
main?: string;
browser?: string;
extensionKind?: ExtensionKind | ExtensionKind[];
extensionPack?: string[];
extensionDependencies?: string[];
contributes?: { [id: string]: any };
}
/**
* Loosely based on `getExtensionKind` from `src/vs/workbench/services/extensions/common/extensionManifestPropertiesService.ts`
*/
function isWebExtension(manifest: IExtensionManifest): boolean {
if (Boolean(manifest.browser)) {
return true;
}
if (Boolean(manifest.main)) {
return false;
}
// neither browser nor main
if (typeof manifest.extensionKind !== 'undefined') {
const extensionKind = Array.isArray(manifest.extensionKind) ? manifest.extensionKind : [manifest.extensionKind];
if (extensionKind.indexOf('web') >= 0) {
return true;
}
}
if (typeof manifest.contributes !== 'undefined') {
for (const id of ['debuggers', 'terminal', 'typescriptServerPlugins']) {
if (manifest.contributes.hasOwnProperty(id)) {
return false;
}
}
}
return true;
}
/**
* Package local extensions that are known to not have native dependencies. Mutually exclusive to {@link packageNativeLocalExtensionsStream}.
* @param forWeb build the extensions that have web targets
* @param disableMangle disable the mangler
* @returns a stream
*/
export function packageNonNativeLocalExtensionsStream(forWeb: boolean, disableMangle: boolean): Stream {
return doPackageLocalExtensionsStream(forWeb, disableMangle, false);
}
/**
* Package local extensions that are known to have native dependencies. Mutually exclusive to {@link packageNonNativeLocalExtensionsStream}.
* @note it's possible that the extension does not have native dependencies for the current platform, especially if building for the web,
* but we simplify the logic here by having a flat list of extensions (See {@link nativeExtensions}) that are known to have native
* dependencies on some platform and thus should be packaged on the platform that they are building for.
* @param forWeb build the extensions that have web targets
* @param disableMangle disable the mangler
* @returns a stream
*/
export function packageNativeLocalExtensionsStream(forWeb: boolean, disableMangle: boolean): Stream {
return doPackageLocalExtensionsStream(forWeb, disableMangle, true);
}
/**
* Package all the local extensions... both those that are known to have native dependencies and those that are not.
* @param forWeb build the extensions that have web targets
* @param disableMangle disable the mangler
* @returns a stream
*/
export function packageAllLocalExtensionsStream(forWeb: boolean, disableMangle: boolean): Stream {
return es.merge([
packageNonNativeLocalExtensionsStream(forWeb, disableMangle),
packageNativeLocalExtensionsStream(forWeb, disableMangle)
]);
}
/**
* @param forWeb build the extensions that have web targets
* @param disableMangle disable the mangler
* @param native build the extensions that are marked as having native dependencies
*/
function doPackageLocalExtensionsStream(forWeb: boolean, disableMangle: boolean, native: boolean): Stream {
const nativeExtensionsSet = new Set(nativeExtensions);
const localExtensionsDescriptions = (
(glob.sync('extensions/*/package.json'))
.map(manifestPath => {
const absoluteManifestPath = path.join(root, manifestPath);
const extensionPath = path.dirname(path.join(root, manifestPath));
const extensionName = path.basename(extensionPath);
return { name: extensionName, path: extensionPath, manifestPath: absoluteManifestPath };
})
.filter(({ name }) => native ? nativeExtensionsSet.has(name) : !nativeExtensionsSet.has(name))
.filter(({ name }) => excludedExtensions.indexOf(name) === -1)
.filter(({ name }) => builtInExtensions.every(b => b.name !== name))
.filter(({ manifestPath }) => (forWeb ? isWebExtension(require(manifestPath)) : true))
);
const localExtensionsStream = minifyExtensionResources(
es.merge(
...localExtensionsDescriptions.map(extension => {
return fromLocal(extension.path, forWeb, disableMangle)
.pipe(rename(p => p.dirname = `extensions/${extension.name}/${p.dirname}`));
})
)
);
let result: Stream;
if (forWeb) {
result = localExtensionsStream;
} else {
// also include shared production node modules
const productionDependencies = getProductionDependencies('extensions/');
const dependenciesSrc = productionDependencies.map(d => path.relative(root, d)).map(d => [`${d}/**`, `!${d}/**/{test,tests}/**`]).flat();
result = es.merge(
localExtensionsStream,
gulp.src(dependenciesSrc, { base: '.' })
.pipe(util2.cleanNodeModules(path.join(root, 'build', '.moduleignore')))
.pipe(util2.cleanNodeModules(path.join(root, 'build', `.moduleignore.${process.platform}`))));
}
return (
result
.pipe(util2.setExecutableBit(['**/*.sh']))
);
}
export function packageMarketplaceExtensionsStream(forWeb: boolean): Stream {
const marketplaceExtensionsDescriptions = [
...builtInExtensions.filter(({ name }) => (forWeb ? !marketplaceWebExtensionsExclude.has(name) : true)),
...(forWeb ? webBuiltInExtensions : [])
];
const marketplaceExtensionsStream = minifyExtensionResources(
es.merge(
...marketplaceExtensionsDescriptions
.map(extension => {
const src = getExtensionStream(extension).pipe(rename(p => p.dirname = `extensions/${p.dirname}`));
return updateExtensionPackageJSON(src, (data: any) => {
delete data.scripts;
delete data.dependencies;
delete data.devDependencies;
return data;
});
})
)
);
return (
marketplaceExtensionsStream
.pipe(util2.setExecutableBit(['**/*.sh']))
);
}
export interface IScannedBuiltinExtension {
extensionPath: string;
packageJSON: any;
packageNLS?: any;
readmePath?: string;
changelogPath?: string;
}
export function scanBuiltinExtensions(extensionsRoot: string, exclude: string[] = []): IScannedBuiltinExtension[] {
const scannedExtensions: IScannedBuiltinExtension[] = [];
try {
const extensionsFolders = fs.readdirSync(extensionsRoot);
for (const extensionFolder of extensionsFolders) {
if (exclude.indexOf(extensionFolder) >= 0) {
continue;
}
const packageJSONPath = path.join(extensionsRoot, extensionFolder, 'package.json');
if (!fs.existsSync(packageJSONPath)) {
continue;
}
const packageJSON = JSON.parse(fs.readFileSync(packageJSONPath).toString('utf8'));
if (!isWebExtension(packageJSON)) {
continue;
}
const children = fs.readdirSync(path.join(extensionsRoot, extensionFolder));
const packageNLSPath = children.filter(child => child === 'package.nls.json')[0];
const packageNLS = packageNLSPath ? JSON.parse(fs.readFileSync(path.join(extensionsRoot, extensionFolder, packageNLSPath)).toString()) : undefined;
const readme = children.filter(child => /^readme(\.txt|\.md|)$/i.test(child))[0];
const changelog = children.filter(child => /^changelog(\.txt|\.md|)$/i.test(child))[0];
scannedExtensions.push({
extensionPath: extensionFolder,
packageJSON,
packageNLS,
readmePath: readme ? path.join(extensionFolder, readme) : undefined,
changelogPath: changelog ? path.join(extensionFolder, changelog) : undefined,
});
}
return scannedExtensions;
} catch (ex) {
return scannedExtensions;
}
}
export function translatePackageJSON(packageJSON: string, packageNLSPath: string) {
interface NLSFormat {
[key: string]: string | { message: string; comment: string[] };
}
const CharCode_PC = '%'.charCodeAt(0);
const packageNls: NLSFormat = JSON.parse(fs.readFileSync(packageNLSPath).toString());
const translate = (obj: any) => {
for (const key in obj) {
const val = obj[key];
if (Array.isArray(val)) {
val.forEach(translate);
} else if (val && typeof val === 'object') {
translate(val);
} else if (typeof val === 'string' && val.charCodeAt(0) === CharCode_PC && val.charCodeAt(val.length - 1) === CharCode_PC) {
const translated = packageNls[val.substr(1, val.length - 2)];
if (translated) {
obj[key] = typeof translated === 'string' ? translated : (typeof translated.message === 'string' ? translated.message : val);
}
}
}
};
translate(packageJSON);
return packageJSON;
}
const extensionsPath = path.join(root, 'extensions');
// Additional projects to run esbuild on. These typically build code for webviews
const esbuildMediaScripts = [
'markdown-language-features/esbuild-notebook.js',
'markdown-language-features/esbuild-preview.js',
'markdown-math/esbuild.js',
'notebook-renderers/esbuild.js',
'ipynb/esbuild.js',
'simple-browser/esbuild-preview.js',
];
export async function webpackExtensions(taskName: string, isWatch: boolean, webpackConfigLocations: { configPath: string; outputRoot?: string }[]) {
const webpack = require('webpack') as typeof import('webpack');
const webpackConfigs: webpack.Configuration[] = [];
for (const { configPath, outputRoot } of webpackConfigLocations) {
const configOrFnOrArray = require(configPath);
function addConfig(configOrFnOrArray: webpack.Configuration | ((env: unknown, args: unknown) => webpack.Configuration) | webpack.Configuration[]) {
for (const configOrFn of Array.isArray(configOrFnOrArray) ? configOrFnOrArray : [configOrFnOrArray]) {
const config = typeof configOrFn === 'function' ? configOrFn({}, {}) : configOrFn;
if (outputRoot) {
config.output!.path = path.join(outputRoot, path.relative(path.dirname(configPath), config.output!.path!));
}
webpackConfigs.push(config);
}
}
addConfig(configOrFnOrArray);
}
function reporter(fullStats: any) {
if (Array.isArray(fullStats.children)) {
for (const stats of fullStats.children) {
const outputPath = stats.outputPath;
if (outputPath) {
const relativePath = path.relative(extensionsPath, outputPath).replace(/\\/g, '/');
const match = relativePath.match(/[^\/]+(\/server|\/client)?/);
fancyLog(`Finished ${ansiColors.green(taskName)} ${ansiColors.cyan(match![0])} with ${stats.errors.length} errors.`);
}
if (Array.isArray(stats.errors)) {
stats.errors.forEach((error: any) => {
fancyLog.error(error);
});
}
if (Array.isArray(stats.warnings)) {
stats.warnings.forEach((warning: any) => {
fancyLog.warn(warning);
});
}
}
}
}
return new Promise((resolve, reject) => {
if (isWatch) {
webpack(webpackConfigs).watch({}, (err, stats) => {
if (err) {
reject();
} else {
reporter(stats?.toJson());
}
});
} else {
webpack(webpackConfigs).run((err, stats) => {
if (err) {
fancyLog.error(err);
reject();
} else {
reporter(stats?.toJson());
resolve();
}
});
}
});
}
async function esbuildExtensions(taskName: string, isWatch: boolean, scripts: { script: string; outputRoot?: string }[]) {
function reporter(stdError: string, script: string) {
const matches = (stdError || '').match(/\> (.+): error: (.+)?/g);
fancyLog(`Finished ${ansiColors.green(taskName)} ${script} with ${matches ? matches.length : 0} errors.`);
for (const match of matches || []) {
fancyLog.error(match);
}
}
const tasks = scripts.map(({ script, outputRoot }) => {
return new Promise((resolve, reject) => {
const args = [script];
if (isWatch) {
args.push('--watch');
}
if (outputRoot) {
args.push('--outputRoot', outputRoot);
}
const proc = cp.execFile(process.argv[0], args, {}, (error, _stdout, stderr) => {
if (error) {
return reject(error);
}
reporter(stderr, script);
return resolve();
});
proc.stdout!.on('data', (data) => {
fancyLog(`${ansiColors.green(taskName)}: ${data.toString('utf8')}`);
});
});
});
return Promise.all(tasks);
}
export async function buildExtensionMedia(isWatch: boolean, outputRoot?: string) {
return esbuildExtensions('esbuilding extension media', isWatch, esbuildMediaScripts.map(p => ({
script: path.join(extensionsPath, p),
outputRoot: outputRoot ? path.join(root, outputRoot, path.dirname(p)) : undefined
})));
}
================================================
FILE: build/lib/fetch.js
================================================
"use strict";
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
Object.defineProperty(exports, "__esModule", { value: true });
exports.fetchUrls = fetchUrls;
exports.fetchUrl = fetchUrl;
exports.fetchGithub = fetchGithub;
const event_stream_1 = __importDefault(require("event-stream"));
const vinyl_1 = __importDefault(require("vinyl"));
const fancy_log_1 = __importDefault(require("fancy-log"));
const ansi_colors_1 = __importDefault(require("ansi-colors"));
const crypto_1 = __importDefault(require("crypto"));
const through2_1 = __importDefault(require("through2"));
function fetchUrls(urls, options) {
if (options === undefined) {
options = {};
}
if (typeof options.base !== 'string' && options.base !== null) {
options.base = '/';
}
if (!Array.isArray(urls)) {
urls = [urls];
}
return event_stream_1.default.readArray(urls).pipe(event_stream_1.default.map((data, cb) => {
const url = [options.base, data].join('');
fetchUrl(url, options).then(file => {
cb(undefined, file);
}, error => {
cb(error);
});
}));
}
async function fetchUrl(url, options, retries = 10, retryDelay = 1000) {
const verbose = !!options.verbose || !!process.env['CI'] || !!process.env['BUILD_ARTIFACTSTAGINGDIRECTORY'];
try {
let startTime = 0;
if (verbose) {
(0, fancy_log_1.default)(`Start fetching ${ansi_colors_1.default.magenta(url)}${retries !== 10 ? ` (${10 - retries} retry)` : ''}`);
startTime = new Date().getTime();
}
const controller = new AbortController();
const timeout = setTimeout(() => controller.abort(), 30 * 1000);
try {
const response = await fetch(url, {
...options.nodeFetchOptions,
signal: controller.signal /* Typings issue with lib.dom.d.ts */
});
if (verbose) {
(0, fancy_log_1.default)(`Fetch completed: Status ${response.status}. Took ${ansi_colors_1.default.magenta(`${new Date().getTime() - startTime} ms`)}`);
}
if (response.ok && (response.status >= 200 && response.status < 300)) {
const contents = Buffer.from(await response.arrayBuffer());
if (options.checksumSha256) {
const actualSHA256Checksum = crypto_1.default.createHash('sha256').update(contents).digest('hex');
if (actualSHA256Checksum !== options.checksumSha256) {
throw new Error(`Checksum mismatch for ${ansi_colors_1.default.cyan(url)} (expected ${options.checksumSha256}, actual ${actualSHA256Checksum}))`);
}
else if (verbose) {
(0, fancy_log_1.default)(`Verified SHA256 checksums match for ${ansi_colors_1.default.cyan(url)}`);
}
}
else if (verbose) {
(0, fancy_log_1.default)(`Skipping checksum verification for ${ansi_colors_1.default.cyan(url)} because no expected checksum was provided`);
}
if (verbose) {
(0, fancy_log_1.default)(`Fetched response body buffer: ${ansi_colors_1.default.magenta(`${contents.byteLength} bytes`)}`);
}
return new vinyl_1.default({
cwd: '/',
base: options.base,
path: url,
contents
});
}
let err = `Request ${ansi_colors_1.default.magenta(url)} failed with status code: ${response.status}`;
if (response.status === 403) {
err += ' (you may be rate limited)';
}
throw new Error(err);
}
finally {
clearTimeout(timeout);
}
}
catch (e) {
if (verbose) {
(0, fancy_log_1.default)(`Fetching ${ansi_colors_1.default.cyan(url)} failed: ${e}`);
}
if (retries > 0) {
await new Promise(resolve => setTimeout(resolve, retryDelay));
return fetchUrl(url, options, retries - 1, retryDelay);
}
throw e;
}
}
const ghApiHeaders = {
Accept: 'application/vnd.github.v3+json',
'User-Agent': 'VSCode Build',
};
if (process.env.GITHUB_TOKEN) {
ghApiHeaders.Authorization = 'Basic ' + Buffer.from(process.env.GITHUB_TOKEN).toString('base64');
}
const ghDownloadHeaders = {
...ghApiHeaders,
Accept: 'application/octet-stream',
};
/**
* @param repo for example `Microsoft/vscode`
* @param version for example `16.17.1` - must be a valid releases tag
* @param assetName for example (name) => name === `win-x64-node.exe` - must be an asset that exists
* @returns a stream with the asset as file
*/
function fetchGithub(repo, options) {
return fetchUrls(`/repos/${repo.replace(/^\/|\/$/g, '')}/releases/tags/v${options.version}`, {
base: 'https://api.github.com',
verbose: options.verbose,
nodeFetchOptions: { headers: ghApiHeaders }
}).pipe(through2_1.default.obj(async function (file, _enc, callback) {
const assetFilter = typeof options.name === 'string' ? (name) => name === options.name : options.name;
const asset = JSON.parse(file.contents.toString()).assets.find((a) => assetFilter(a.name));
if (!asset) {
return callback(new Error(`Could not find asset in release of ${repo} @ ${options.version}`));
}
try {
callback(null, await fetchUrl(asset.url, {
nodeFetchOptions: { headers: ghDownloadHeaders },
verbose: options.verbose,
checksumSha256: options.checksumSha256
}));
}
catch (error) {
callback(error);
}
}));
}
//# sourceMappingURL=fetch.js.map
================================================
FILE: build/lib/fetch.ts
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
import es from 'event-stream';
import VinylFile from 'vinyl';
import log from 'fancy-log';
import ansiColors from 'ansi-colors';
import crypto from 'crypto';
import through2 from 'through2';
import { Stream } from 'stream';
export interface IFetchOptions {
base?: string;
nodeFetchOptions?: RequestInit;
verbose?: boolean;
checksumSha256?: string;
}
export function fetchUrls(urls: string[] | string, options: IFetchOptions): es.ThroughStream {
if (options === undefined) {
options = {};
}
if (typeof options.base !== 'string' && options.base !== null) {
options.base = '/';
}
if (!Array.isArray(urls)) {
urls = [urls];
}
return es.readArray(urls).pipe(es.map((data: string, cb) => {
const url = [options.base, data].join('');
fetchUrl(url, options).then(file => {
cb(undefined, file);
}, error => {
cb(error);
});
}));
}
export async function fetchUrl(url: string, options: IFetchOptions, retries = 10, retryDelay = 1000): Promise {
const verbose = !!options.verbose || !!process.env['CI'] || !!process.env['BUILD_ARTIFACTSTAGINGDIRECTORY'];
try {
let startTime = 0;
if (verbose) {
log(`Start fetching ${ansiColors.magenta(url)}${retries !== 10 ? ` (${10 - retries} retry)` : ''}`);
startTime = new Date().getTime();
}
const controller = new AbortController();
const timeout = setTimeout(() => controller.abort(), 30 * 1000);
try {
const response = await fetch(url, {
...options.nodeFetchOptions,
signal: controller.signal as any /* Typings issue with lib.dom.d.ts */
});
if (verbose) {
log(`Fetch completed: Status ${response.status}. Took ${ansiColors.magenta(`${new Date().getTime() - startTime} ms`)}`);
}
if (response.ok && (response.status >= 200 && response.status < 300)) {
const contents = Buffer.from(await response.arrayBuffer());
if (options.checksumSha256) {
const actualSHA256Checksum = crypto.createHash('sha256').update(contents).digest('hex');
if (actualSHA256Checksum !== options.checksumSha256) {
throw new Error(`Checksum mismatch for ${ansiColors.cyan(url)} (expected ${options.checksumSha256}, actual ${actualSHA256Checksum}))`);
} else if (verbose) {
log(`Verified SHA256 checksums match for ${ansiColors.cyan(url)}`);
}
} else if (verbose) {
log(`Skipping checksum verification for ${ansiColors.cyan(url)} because no expected checksum was provided`);
}
if (verbose) {
log(`Fetched response body buffer: ${ansiColors.magenta(`${(contents as Buffer).byteLength} bytes`)}`);
}
return new VinylFile({
cwd: '/',
base: options.base,
path: url,
contents
});
}
let err = `Request ${ansiColors.magenta(url)} failed with status code: ${response.status}`;
if (response.status === 403) {
err += ' (you may be rate limited)';
}
throw new Error(err);
} finally {
clearTimeout(timeout);
}
} catch (e) {
if (verbose) {
log(`Fetching ${ansiColors.cyan(url)} failed: ${e}`);
}
if (retries > 0) {
await new Promise(resolve => setTimeout(resolve, retryDelay));
return fetchUrl(url, options, retries - 1, retryDelay);
}
throw e;
}
}
const ghApiHeaders: Record = {
Accept: 'application/vnd.github.v3+json',
'User-Agent': 'VSCode Build',
};
if (process.env.GITHUB_TOKEN) {
ghApiHeaders.Authorization = 'Basic ' + Buffer.from(process.env.GITHUB_TOKEN).toString('base64');
}
const ghDownloadHeaders = {
...ghApiHeaders,
Accept: 'application/octet-stream',
};
export interface IGitHubAssetOptions {
version: string;
name: string | ((name: string) => boolean);
checksumSha256?: string;
verbose?: boolean;
}
/**
* @param repo for example `Microsoft/vscode`
* @param version for example `16.17.1` - must be a valid releases tag
* @param assetName for example (name) => name === `win-x64-node.exe` - must be an asset that exists
* @returns a stream with the asset as file
*/
export function fetchGithub(repo: string, options: IGitHubAssetOptions): Stream {
return fetchUrls(`/repos/${repo.replace(/^\/|\/$/g, '')}/releases/tags/v${options.version}`, {
base: 'https://api.github.com',
verbose: options.verbose,
nodeFetchOptions: { headers: ghApiHeaders }
}).pipe(through2.obj(async function (file, _enc, callback) {
const assetFilter = typeof options.name === 'string' ? (name: string) => name === options.name : options.name;
const asset = JSON.parse(file.contents.toString()).assets.find((a: { name: string }) => assetFilter(a.name));
if (!asset) {
return callback(new Error(`Could not find asset in release of ${repo} @ ${options.version}`));
}
try {
callback(null, await fetchUrl(asset.url, {
nodeFetchOptions: { headers: ghDownloadHeaders },
verbose: options.verbose,
checksumSha256: options.checksumSha256
}));
} catch (error) {
callback(error);
}
}));
}
================================================
FILE: build/lib/formatter.js
================================================
"use strict";
var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
Object.defineProperty(exports, "__esModule", { value: true });
exports.format = format;
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
const fs_1 = __importDefault(require("fs"));
const path_1 = __importDefault(require("path"));
const typescript_1 = __importDefault(require("typescript"));
class LanguageServiceHost {
files = {};
addFile(fileName, text) {
this.files[fileName] = typescript_1.default.ScriptSnapshot.fromString(text);
}
fileExists(path) {
return !!this.files[path];
}
readFile(path) {
return this.files[path]?.getText(0, this.files[path].getLength());
}
// for ts.LanguageServiceHost
getCompilationSettings = () => typescript_1.default.getDefaultCompilerOptions();
getScriptFileNames = () => Object.keys(this.files);
getScriptVersion = (_fileName) => '0';
getScriptSnapshot = (fileName) => this.files[fileName];
getCurrentDirectory = () => process.cwd();
getDefaultLibFileName = (options) => typescript_1.default.getDefaultLibFilePath(options);
}
const defaults = {
baseIndentSize: 0,
indentSize: 4,
tabSize: 4,
indentStyle: typescript_1.default.IndentStyle.Smart,
newLineCharacter: '\r\n',
convertTabsToSpaces: false,
insertSpaceAfterCommaDelimiter: true,
insertSpaceAfterSemicolonInForStatements: true,
insertSpaceBeforeAndAfterBinaryOperators: true,
insertSpaceAfterConstructor: false,
insertSpaceAfterKeywordsInControlFlowStatements: true,
insertSpaceAfterFunctionKeywordForAnonymousFunctions: false,
insertSpaceAfterOpeningAndBeforeClosingNonemptyParenthesis: false,
insertSpaceAfterOpeningAndBeforeClosingNonemptyBrackets: false,
insertSpaceAfterOpeningAndBeforeClosingNonemptyBraces: true,
insertSpaceAfterOpeningAndBeforeClosingTemplateStringBraces: false,
insertSpaceAfterOpeningAndBeforeClosingJsxExpressionBraces: false,
insertSpaceAfterTypeAssertion: false,
insertSpaceBeforeFunctionParenthesis: false,
placeOpenBraceOnNewLineForFunctions: false,
placeOpenBraceOnNewLineForControlBlocks: false,
insertSpaceBeforeTypeAnnotation: false,
};
const getOverrides = (() => {
let value;
return () => {
value ??= JSON.parse(fs_1.default.readFileSync(path_1.default.join(__dirname, '..', '..', 'tsfmt.json'), 'utf8'));
return value;
};
})();
function format(fileName, text) {
const host = new LanguageServiceHost();
host.addFile(fileName, text);
const languageService = typescript_1.default.createLanguageService(host);
const edits = languageService.getFormattingEditsForDocument(fileName, { ...defaults, ...getOverrides() });
edits
.sort((a, b) => a.span.start - b.span.start)
.reverse()
.forEach(edit => {
const head = text.slice(0, edit.span.start);
const tail = text.slice(edit.span.start + edit.span.length);
text = `${head}${edit.newText}${tail}`;
});
return text;
}
//# sourceMappingURL=formatter.js.map
================================================
FILE: build/lib/formatter.ts
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
import fs from 'fs';
import path from 'path';
import ts from 'typescript';
class LanguageServiceHost implements ts.LanguageServiceHost {
files: ts.MapLike = {};
addFile(fileName: string, text: string) {
this.files[fileName] = ts.ScriptSnapshot.fromString(text);
}
fileExists(path: string): boolean {
return !!this.files[path];
}
readFile(path: string): string | undefined {
return this.files[path]?.getText(0, this.files[path]!.getLength());
}
// for ts.LanguageServiceHost
getCompilationSettings = () => ts.getDefaultCompilerOptions();
getScriptFileNames = () => Object.keys(this.files);
getScriptVersion = (_fileName: string) => '0';
getScriptSnapshot = (fileName: string) => this.files[fileName];
getCurrentDirectory = () => process.cwd();
getDefaultLibFileName = (options: ts.CompilerOptions) => ts.getDefaultLibFilePath(options);
}
const defaults: ts.FormatCodeSettings = {
baseIndentSize: 0,
indentSize: 4,
tabSize: 4,
indentStyle: ts.IndentStyle.Smart,
newLineCharacter: '\r\n',
convertTabsToSpaces: false,
insertSpaceAfterCommaDelimiter: true,
insertSpaceAfterSemicolonInForStatements: true,
insertSpaceBeforeAndAfterBinaryOperators: true,
insertSpaceAfterConstructor: false,
insertSpaceAfterKeywordsInControlFlowStatements: true,
insertSpaceAfterFunctionKeywordForAnonymousFunctions: false,
insertSpaceAfterOpeningAndBeforeClosingNonemptyParenthesis: false,
insertSpaceAfterOpeningAndBeforeClosingNonemptyBrackets: false,
insertSpaceAfterOpeningAndBeforeClosingNonemptyBraces: true,
insertSpaceAfterOpeningAndBeforeClosingTemplateStringBraces: false,
insertSpaceAfterOpeningAndBeforeClosingJsxExpressionBraces: false,
insertSpaceAfterTypeAssertion: false,
insertSpaceBeforeFunctionParenthesis: false,
placeOpenBraceOnNewLineForFunctions: false,
placeOpenBraceOnNewLineForControlBlocks: false,
insertSpaceBeforeTypeAnnotation: false,
};
const getOverrides = (() => {
let value: ts.FormatCodeSettings | undefined;
return () => {
value ??= JSON.parse(fs.readFileSync(path.join(__dirname, '..', '..', 'tsfmt.json'), 'utf8'));
return value;
};
})();
export function format(fileName: string, text: string) {
const host = new LanguageServiceHost();
host.addFile(fileName, text);
const languageService = ts.createLanguageService(host);
const edits = languageService.getFormattingEditsForDocument(fileName, { ...defaults, ...getOverrides() });
edits
.sort((a, b) => a.span.start - b.span.start)
.reverse()
.forEach(edit => {
const head = text.slice(0, edit.span.start);
const tail = text.slice(edit.span.start + edit.span.length);
text = `${head}${edit.newText}${tail}`;
});
return text;
}
================================================
FILE: build/lib/getVersion.js
================================================
"use strict";
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
if (k2 === undefined) k2 = k;
var desc = Object.getOwnPropertyDescriptor(m, k);
if (!desc || ("get" in desc ? !m.__esModule : desc.writable || desc.configurable)) {
desc = { enumerable: true, get: function() { return m[k]; } };
}
Object.defineProperty(o, k2, desc);
}) : (function(o, m, k, k2) {
if (k2 === undefined) k2 = k;
o[k2] = m[k];
}));
var __setModuleDefault = (this && this.__setModuleDefault) || (Object.create ? (function(o, v) {
Object.defineProperty(o, "default", { enumerable: true, value: v });
}) : function(o, v) {
o["default"] = v;
});
var __importStar = (this && this.__importStar) || (function () {
var ownKeys = function(o) {
ownKeys = Object.getOwnPropertyNames || function (o) {
var ar = [];
for (var k in o) if (Object.prototype.hasOwnProperty.call(o, k)) ar[ar.length] = k;
return ar;
};
return ownKeys(o);
};
return function (mod) {
if (mod && mod.__esModule) return mod;
var result = {};
if (mod != null) for (var k = ownKeys(mod), i = 0; i < k.length; i++) if (k[i] !== "default") __createBinding(result, mod, k[i]);
__setModuleDefault(result, mod);
return result;
};
})();
Object.defineProperty(exports, "__esModule", { value: true });
exports.getVersion = getVersion;
const git = __importStar(require("./git"));
function getVersion(root) {
let version = process.env['BUILD_SOURCEVERSION'];
if (!version || !/^[0-9a-f]{40}$/i.test(version.trim())) {
version = git.getVersion(root);
}
return version;
}
//# sourceMappingURL=getVersion.js.map
================================================
FILE: build/lib/getVersion.ts
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
import * as git from './git';
export function getVersion(root: string): string | undefined {
let version = process.env['BUILD_SOURCEVERSION'];
if (!version || !/^[0-9a-f]{40}$/i.test(version.trim())) {
version = git.getVersion(root);
}
return version;
}
================================================
FILE: build/lib/git.js
================================================
"use strict";
var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
Object.defineProperty(exports, "__esModule", { value: true });
exports.getVersion = getVersion;
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
const path_1 = __importDefault(require("path"));
const fs_1 = __importDefault(require("fs"));
/**
* Returns the sha1 commit version of a repository or undefined in case of failure.
*/
function getVersion(repo) {
const git = path_1.default.join(repo, '.git');
const headPath = path_1.default.join(git, 'HEAD');
let head;
try {
head = fs_1.default.readFileSync(headPath, 'utf8').trim();
}
catch (e) {
return undefined;
}
if (/^[0-9a-f]{40}$/i.test(head)) {
return head;
}
const refMatch = /^ref: (.*)$/.exec(head);
if (!refMatch) {
return undefined;
}
const ref = refMatch[1];
const refPath = path_1.default.join(git, ref);
try {
return fs_1.default.readFileSync(refPath, 'utf8').trim();
}
catch (e) {
// noop
}
const packedRefsPath = path_1.default.join(git, 'packed-refs');
let refsRaw;
try {
refsRaw = fs_1.default.readFileSync(packedRefsPath, 'utf8').trim();
}
catch (e) {
return undefined;
}
const refsRegex = /^([0-9a-f]{40})\s+(.+)$/gm;
let refsMatch;
const refs = {};
while (refsMatch = refsRegex.exec(refsRaw)) {
refs[refsMatch[2]] = refsMatch[1];
}
return refs[ref];
}
//# sourceMappingURL=git.js.map
================================================
FILE: build/lib/git.ts
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
import path from 'path';
import fs from 'fs';
/**
* Returns the sha1 commit version of a repository or undefined in case of failure.
*/
export function getVersion(repo: string): string | undefined {
const git = path.join(repo, '.git');
const headPath = path.join(git, 'HEAD');
let head: string;
try {
head = fs.readFileSync(headPath, 'utf8').trim();
} catch (e) {
return undefined;
}
if (/^[0-9a-f]{40}$/i.test(head)) {
return head;
}
const refMatch = /^ref: (.*)$/.exec(head);
if (!refMatch) {
return undefined;
}
const ref = refMatch[1];
const refPath = path.join(git, ref);
try {
return fs.readFileSync(refPath, 'utf8').trim();
} catch (e) {
// noop
}
const packedRefsPath = path.join(git, 'packed-refs');
let refsRaw: string;
try {
refsRaw = fs.readFileSync(packedRefsPath, 'utf8').trim();
} catch (e) {
return undefined;
}
const refsRegex = /^([0-9a-f]{40})\s+(.+)$/gm;
let refsMatch: RegExpExecArray | null;
const refs: { [ref: string]: string } = {};
while (refsMatch = refsRegex.exec(refsRaw)) {
refs[refsMatch[2]] = refsMatch[1];
}
return refs[ref];
}
================================================
FILE: build/lib/i18n.js
================================================
"use strict";
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
Object.defineProperty(exports, "__esModule", { value: true });
exports.EXTERNAL_EXTENSIONS = exports.XLF = exports.Line = exports.extraLanguages = exports.defaultLanguages = void 0;
exports.processNlsFiles = processNlsFiles;
exports.getResource = getResource;
exports.createXlfFilesForCoreBundle = createXlfFilesForCoreBundle;
exports.createXlfFilesForExtensions = createXlfFilesForExtensions;
exports.createXlfFilesForIsl = createXlfFilesForIsl;
exports.prepareI18nPackFiles = prepareI18nPackFiles;
exports.prepareIslFiles = prepareIslFiles;
const path_1 = __importDefault(require("path"));
const fs_1 = __importDefault(require("fs"));
const event_stream_1 = require("event-stream");
const gulp_merge_json_1 = __importDefault(require("gulp-merge-json"));
const vinyl_1 = __importDefault(require("vinyl"));
const xml2js_1 = __importDefault(require("xml2js"));
const gulp_1 = __importDefault(require("gulp"));
const fancy_log_1 = __importDefault(require("fancy-log"));
const ansi_colors_1 = __importDefault(require("ansi-colors"));
const iconv_lite_umd_1 = __importDefault(require("@vscode/iconv-lite-umd"));
const l10n_dev_1 = require("@vscode/l10n-dev");
const REPO_ROOT_PATH = path_1.default.join(__dirname, '../..');
function log(message, ...rest) {
(0, fancy_log_1.default)(ansi_colors_1.default.green('[i18n]'), message, ...rest);
}
exports.defaultLanguages = [
{ id: 'zh-tw', folderName: 'cht', translationId: 'zh-hant' },
{ id: 'zh-cn', folderName: 'chs', translationId: 'zh-hans' },
{ id: 'ja', folderName: 'jpn' },
{ id: 'ko', folderName: 'kor' },
{ id: 'de', folderName: 'deu' },
{ id: 'fr', folderName: 'fra' },
{ id: 'es', folderName: 'esn' },
{ id: 'ru', folderName: 'rus' },
{ id: 'it', folderName: 'ita' }
];
// languages requested by the community to non-stable builds
exports.extraLanguages = [
{ id: 'pt-br', folderName: 'ptb' },
{ id: 'hu', folderName: 'hun' },
{ id: 'tr', folderName: 'trk' }
];
var LocalizeInfo;
(function (LocalizeInfo) {
function is(value) {
const candidate = value;
return candidate && typeof candidate.key === 'string' && (candidate.comment === undefined || (Array.isArray(candidate.comment) && candidate.comment.every(element => typeof element === 'string')));
}
LocalizeInfo.is = is;
})(LocalizeInfo || (LocalizeInfo = {}));
var BundledFormat;
(function (BundledFormat) {
function is(value) {
if (value === undefined) {
return false;
}
const candidate = value;
const length = Object.keys(value).length;
return length === 3 && !!candidate.keys && !!candidate.messages && !!candidate.bundles;
}
BundledFormat.is = is;
})(BundledFormat || (BundledFormat = {}));
var NLSKeysFormat;
(function (NLSKeysFormat) {
function is(value) {
if (value === undefined) {
return false;
}
const candidate = value;
return Array.isArray(candidate) && Array.isArray(candidate[1]);
}
NLSKeysFormat.is = is;
})(NLSKeysFormat || (NLSKeysFormat = {}));
class Line {
buffer = [];
constructor(indent = 0) {
if (indent > 0) {
this.buffer.push(new Array(indent + 1).join(' '));
}
}
append(value) {
this.buffer.push(value);
return this;
}
toString() {
return this.buffer.join('');
}
}
exports.Line = Line;
class TextModel {
_lines;
constructor(contents) {
this._lines = contents.split(/\r\n|\r|\n/);
}
get lines() {
return this._lines;
}
}
class XLF {
project;
buffer;
files;
numberOfMessages;
constructor(project) {
this.project = project;
this.buffer = [];
this.files = Object.create(null);
this.numberOfMessages = 0;
}
toString() {
this.appendHeader();
const files = Object.keys(this.files).sort();
for (const file of files) {
this.appendNewLine(``, 2);
const items = this.files[file].sort((a, b) => {
return a.id < b.id ? -1 : a.id > b.id ? 1 : 0;
});
for (const item of items) {
this.addStringItem(file, item);
}
this.appendNewLine('');
}
this.appendFooter();
return this.buffer.join('\r\n');
}
addFile(original, keys, messages) {
if (keys.length === 0) {
console.log('No keys in ' + original);
return;
}
if (keys.length !== messages.length) {
throw new Error(`Unmatching keys(${keys.length}) and messages(${messages.length}).`);
}
this.numberOfMessages += keys.length;
this.files[original] = [];
const existingKeys = new Set();
for (let i = 0; i < keys.length; i++) {
const key = keys[i];
let realKey;
let comment;
if (typeof key === 'string') {
realKey = key;
comment = undefined;
}
else if (LocalizeInfo.is(key)) {
realKey = key.key;
if (key.comment && key.comment.length > 0) {
comment = key.comment.map(comment => encodeEntities(comment)).join('\r\n');
}
}
if (!realKey || existingKeys.has(realKey)) {
continue;
}
existingKeys.add(realKey);
const message = encodeEntities(messages[i]);
this.files[original].push({ id: realKey, message: message, comment: comment });
}
}
addStringItem(file, item) {
if (!item.id || item.message === undefined || item.message === null) {
throw new Error(`No item ID or value specified: ${JSON.stringify(item)}. File: ${file}`);
}
if (item.message.length === 0) {
log(`Item with id ${item.id} in file ${file} has an empty message.`);
}
this.appendNewLine(``, 4);
this.appendNewLine(`${item.message}`, 6);
if (item.comment) {
this.appendNewLine(`${item.comment}`, 6);
}
this.appendNewLine('', 4);
}
appendHeader() {
this.appendNewLine('', 0);
this.appendNewLine('', 0);
}
appendFooter() {
this.appendNewLine('', 0);
}
appendNewLine(content, indent) {
const line = new Line(indent);
line.append(content);
this.buffer.push(line.toString());
}
static parse = function (xlfString) {
return new Promise((resolve, reject) => {
const parser = new xml2js_1.default.Parser();
const files = [];
parser.parseString(xlfString, function (err, result) {
if (err) {
reject(new Error(`XLF parsing error: Failed to parse XLIFF string. ${err}`));
}
const fileNodes = result['xliff']['file'];
if (!fileNodes) {
reject(new Error(`XLF parsing error: XLIFF file does not contain "xliff" or "file" node(s) required for parsing.`));
}
fileNodes.forEach((file) => {
const name = file.$.original;
if (!name) {
reject(new Error(`XLF parsing error: XLIFF file node does not contain original attribute to determine the original location of the resource file.`));
}
const language = file.$['target-language'];
if (!language) {
reject(new Error(`XLF parsing error: XLIFF file node does not contain target-language attribute to determine translated language.`));
}
const messages = {};
const transUnits = file.body[0]['trans-unit'];
if (transUnits) {
transUnits.forEach((unit) => {
const key = unit.$.id;
if (!unit.target) {
return; // No translation available
}
let val = unit.target[0];
if (typeof val !== 'string') {
// We allow empty source values so support them for translations as well.
val = val._ ? val._ : '';
}
if (!key) {
reject(new Error(`XLF parsing error: trans-unit ${JSON.stringify(unit, undefined, 0)} defined in file ${name} is missing the ID attribute.`));
return;
}
messages[key] = decodeEntities(val);
});
files.push({ messages, name, language: language.toLowerCase() });
}
});
resolve(files);
});
});
};
}
exports.XLF = XLF;
function sortLanguages(languages) {
return languages.sort((a, b) => {
return a.id < b.id ? -1 : (a.id > b.id ? 1 : 0);
});
}
function stripComments(content) {
// Copied from stripComments.js
//
// First group matches a double quoted string
// Second group matches a single quoted string
// Third group matches a multi line comment
// Forth group matches a single line comment
// Fifth group matches a trailing comma
const regexp = /("[^"\\]*(?:\\.[^"\\]*)*")|('[^'\\]*(?:\\.[^'\\]*)*')|(\/\*[^\/\*]*(?:(?:\*|\/)[^\/\*]*)*?\*\/)|(\/{2,}.*?(?:(?:\r?\n)|$))|(,\s*[}\]])/g;
const result = content.replace(regexp, (match, _m1, _m2, m3, m4, m5) => {
// Only one of m1, m2, m3, m4, m5 matches
if (m3) {
// A block comment. Replace with nothing
return '';
}
else if (m4) {
// Since m4 is a single line comment is is at least of length 2 (e.g. //)
// If it ends in \r?\n then keep it.
const length = m4.length;
if (m4[length - 1] === '\n') {
return m4[length - 2] === '\r' ? '\r\n' : '\n';
}
else {
return '';
}
}
else if (m5) {
// Remove the trailing comma
return match.substring(1);
}
else {
// We match a string
return match;
}
});
return result;
}
function processCoreBundleFormat(base, fileHeader, languages, json, emitter) {
const languageDirectory = path_1.default.join(REPO_ROOT_PATH, '..', 'vscode-loc', 'i18n');
if (!fs_1.default.existsSync(languageDirectory)) {
log(`No VS Code localization repository found. Looking at ${languageDirectory}`);
log(`To bundle translations please check out the vscode-loc repository as a sibling of the vscode repository.`);
}
const sortedLanguages = sortLanguages(languages);
sortedLanguages.forEach((language) => {
if (process.env['VSCODE_BUILD_VERBOSE']) {
log(`Generating nls bundles for: ${language.id}`);
}
const languageFolderName = language.translationId || language.id;
const i18nFile = path_1.default.join(languageDirectory, `vscode-language-pack-${languageFolderName}`, 'translations', 'main.i18n.json');
let allMessages;
if (fs_1.default.existsSync(i18nFile)) {
const content = stripComments(fs_1.default.readFileSync(i18nFile, 'utf8'));
allMessages = JSON.parse(content);
}
let nlsIndex = 0;
const nlsResult = [];
for (const [moduleId, nlsKeys] of json) {
const moduleTranslations = allMessages?.contents[moduleId];
for (const nlsKey of nlsKeys) {
nlsResult.push(moduleTranslations?.[nlsKey]); // pushing `undefined` is fine, as we keep english strings as fallback for monaco editor in the build
nlsIndex++;
}
}
emitter.queue(new vinyl_1.default({
contents: Buffer.from(`${fileHeader}
globalThis._VSCODE_NLS_MESSAGES=${JSON.stringify(nlsResult)};
globalThis._VSCODE_NLS_LANGUAGE=${JSON.stringify(language.id)};`),
base,
path: `${base}/nls.messages.${language.id}.js`
}));
});
}
function processNlsFiles(opts) {
return (0, event_stream_1.through)(function (file) {
const fileName = path_1.default.basename(file.path);
if (fileName === 'nls.keys.json') {
try {
const contents = file.contents.toString('utf8');
const json = JSON.parse(contents);
if (NLSKeysFormat.is(json)) {
processCoreBundleFormat(file.base, opts.fileHeader, opts.languages, json, this);
}
}
catch (error) {
this.emit('error', `Failed to read component file: ${error}`);
}
}
this.queue(file);
});
}
const editorProject = 'vscode-editor', workbenchProject = 'vscode-workbench', extensionsProject = 'vscode-extensions', setupProject = 'vscode-setup', serverProject = 'vscode-server';
function getResource(sourceFile) {
let resource;
if (/^vs\/platform/.test(sourceFile)) {
return { name: 'vs/platform', project: editorProject };
}
else if (/^vs\/editor\/contrib/.test(sourceFile)) {
return { name: 'vs/editor/contrib', project: editorProject };
}
else if (/^vs\/editor/.test(sourceFile)) {
return { name: 'vs/editor', project: editorProject };
}
else if (/^vs\/base/.test(sourceFile)) {
return { name: 'vs/base', project: editorProject };
}
else if (/^vs\/code/.test(sourceFile)) {
return { name: 'vs/code', project: workbenchProject };
}
else if (/^vs\/server/.test(sourceFile)) {
return { name: 'vs/server', project: serverProject };
}
else if (/^vs\/workbench\/contrib/.test(sourceFile)) {
resource = sourceFile.split('/', 4).join('/');
return { name: resource, project: workbenchProject };
}
else if (/^vs\/workbench\/services/.test(sourceFile)) {
resource = sourceFile.split('/', 4).join('/');
return { name: resource, project: workbenchProject };
}
else if (/^vs\/workbench/.test(sourceFile)) {
return { name: 'vs/workbench', project: workbenchProject };
}
throw new Error(`Could not identify the XLF bundle for ${sourceFile}`);
}
function createXlfFilesForCoreBundle() {
return (0, event_stream_1.through)(function (file) {
const basename = path_1.default.basename(file.path);
if (basename === 'nls.metadata.json') {
if (file.isBuffer()) {
const xlfs = Object.create(null);
const json = JSON.parse(file.contents.toString('utf8'));
for (const coreModule in json.keys) {
const projectResource = getResource(coreModule);
const resource = projectResource.name;
const project = projectResource.project;
const keys = json.keys[coreModule];
const messages = json.messages[coreModule];
if (keys.length !== messages.length) {
this.emit('error', `There is a mismatch between keys and messages in ${file.relative} for module ${coreModule}`);
return;
}
else {
let xlf = xlfs[resource];
if (!xlf) {
xlf = new XLF(project);
xlfs[resource] = xlf;
}
xlf.addFile(`src/${coreModule}`, keys, messages);
}
}
for (const resource in xlfs) {
const xlf = xlfs[resource];
const filePath = `${xlf.project}/${resource.replace(/\//g, '_')}.xlf`;
const xlfFile = new vinyl_1.default({
path: filePath,
contents: Buffer.from(xlf.toString(), 'utf8')
});
this.queue(xlfFile);
}
}
else {
this.emit('error', new Error(`File ${file.relative} is not using a buffer content`));
return;
}
}
else {
this.emit('error', new Error(`File ${file.relative} is not a core meta data file.`));
return;
}
});
}
function createL10nBundleForExtension(extensionFolderName, prefixWithBuildFolder) {
const prefix = prefixWithBuildFolder ? '.build/' : '';
return gulp_1.default
.src([
// For source code of extensions
`${prefix}extensions/${extensionFolderName}/{src,client,server}/**/*.{ts,tsx}`,
// // For any dependencies pulled in (think vscode-css-languageservice or @vscode/emmet-helper)
`${prefix}extensions/${extensionFolderName}/**/node_modules/{@vscode,vscode-*}/**/*.{js,jsx}`,
// // For any dependencies pulled in that bundle @vscode/l10n. They needed to export the bundle
`${prefix}extensions/${extensionFolderName}/**/bundle.l10n.json`,
])
.pipe((0, event_stream_1.map)(function (data, callback) {
const file = data;
if (!file.isBuffer()) {
// Not a buffer so we drop it
callback();
return;
}
const extension = path_1.default.extname(file.relative);
if (extension !== '.json') {
const contents = file.contents.toString('utf8');
(0, l10n_dev_1.getL10nJson)([{ contents, extension }])
.then((json) => {
callback(undefined, new vinyl_1.default({
path: `extensions/${extensionFolderName}/bundle.l10n.json`,
contents: Buffer.from(JSON.stringify(json), 'utf8')
}));
})
.catch((err) => {
callback(new Error(`File ${file.relative} threw an error when parsing: ${err}`));
});
// signal pause?
return false;
}
// for bundle.l10n.jsons
let bundleJson;
try {
bundleJson = JSON.parse(file.contents.toString('utf8'));
}
catch (err) {
callback(new Error(`File ${file.relative} threw an error when parsing: ${err}`));
return;
}
// some validation of the bundle.l10n.json format
for (const key in bundleJson) {
if (typeof bundleJson[key] !== 'string' &&
(typeof bundleJson[key].message !== 'string' || !Array.isArray(bundleJson[key].comment))) {
callback(new Error(`Invalid bundle.l10n.json file. The value for key ${key} is not in the expected format.`));
return;
}
}
callback(undefined, file);
}))
.pipe((0, gulp_merge_json_1.default)({
fileName: `extensions/${extensionFolderName}/bundle.l10n.json`,
jsonSpace: '',
concatArrays: true
}));
}
exports.EXTERNAL_EXTENSIONS = [
'ms-vscode.js-debug',
'ms-vscode.js-debug-companion',
'ms-vscode.vscode-js-profile-table',
];
function createXlfFilesForExtensions() {
let counter = 0;
let folderStreamEnded = false;
let folderStreamEndEmitted = false;
return (0, event_stream_1.through)(function (extensionFolder) {
const folderStream = this;
const stat = fs_1.default.statSync(extensionFolder.path);
if (!stat.isDirectory()) {
return;
}
const extensionFolderName = path_1.default.basename(extensionFolder.path);
if (extensionFolderName === 'node_modules') {
return;
}
// Get extension id and use that as the id
const manifest = fs_1.default.readFileSync(path_1.default.join(extensionFolder.path, 'package.json'), 'utf-8');
const manifestJson = JSON.parse(manifest);
const extensionId = manifestJson.publisher + '.' + manifestJson.name;
counter++;
let _l10nMap;
function getL10nMap() {
if (!_l10nMap) {
_l10nMap = new Map();
}
return _l10nMap;
}
(0, event_stream_1.merge)(gulp_1.default.src([`.build/extensions/${extensionFolderName}/package.nls.json`, `.build/extensions/${extensionFolderName}/**/nls.metadata.json`], { allowEmpty: true }), createL10nBundleForExtension(extensionFolderName, exports.EXTERNAL_EXTENSIONS.includes(extensionId))).pipe((0, event_stream_1.through)(function (file) {
if (file.isBuffer()) {
const buffer = file.contents;
const basename = path_1.default.basename(file.path);
if (basename === 'package.nls.json') {
const json = JSON.parse(buffer.toString('utf8'));
getL10nMap().set(`extensions/${extensionId}/package`, json);
}
else if (basename === 'nls.metadata.json') {
const json = JSON.parse(buffer.toString('utf8'));
const relPath = path_1.default.relative(`.build/extensions/${extensionFolderName}`, path_1.default.dirname(file.path));
for (const file in json) {
const fileContent = json[file];
const info = Object.create(null);
for (let i = 0; i < fileContent.messages.length; i++) {
const message = fileContent.messages[i];
const { key, comment } = LocalizeInfo.is(fileContent.keys[i])
? fileContent.keys[i]
: { key: fileContent.keys[i], comment: undefined };
info[key] = comment ? { message, comment } : message;
}
getL10nMap().set(`extensions/${extensionId}/${relPath}/${file}`, info);
}
}
else if (basename === 'bundle.l10n.json') {
const json = JSON.parse(buffer.toString('utf8'));
getL10nMap().set(`extensions/${extensionId}/bundle`, json);
}
else {
this.emit('error', new Error(`${file.path} is not a valid extension nls file`));
return;
}
}
}, function () {
if (_l10nMap?.size > 0) {
const xlfFile = new vinyl_1.default({
path: path_1.default.join(extensionsProject, extensionId + '.xlf'),
contents: Buffer.from((0, l10n_dev_1.getL10nXlf)(_l10nMap), 'utf8')
});
folderStream.queue(xlfFile);
}
this.queue(null);
counter--;
if (counter === 0 && folderStreamEnded && !folderStreamEndEmitted) {
folderStreamEndEmitted = true;
folderStream.queue(null);
}
}));
}, function () {
folderStreamEnded = true;
if (counter === 0) {
folderStreamEndEmitted = true;
this.queue(null);
}
});
}
function createXlfFilesForIsl() {
return (0, event_stream_1.through)(function (file) {
let projectName, resourceFile;
if (path_1.default.basename(file.path) === 'messages.en.isl') {
projectName = setupProject;
resourceFile = 'messages.xlf';
}
else {
throw new Error(`Unknown input file ${file.path}`);
}
const xlf = new XLF(projectName), keys = [], messages = [];
const model = new TextModel(file.contents.toString());
let inMessageSection = false;
model.lines.forEach(line => {
if (line.length === 0) {
return;
}
const firstChar = line.charAt(0);
switch (firstChar) {
case ';':
// Comment line;
return;
case '[':
inMessageSection = '[Messages]' === line || '[CustomMessages]' === line;
return;
}
if (!inMessageSection) {
return;
}
const sections = line.split('=');
if (sections.length !== 2) {
throw new Error(`Badly formatted message found: ${line}`);
}
else {
const key = sections[0];
const value = sections[1];
if (key.length > 0 && value.length > 0) {
keys.push(key);
messages.push(value);
}
}
});
const originalPath = file.path.substring(file.cwd.length + 1, file.path.split('.')[0].length).replace(/\\/g, '/');
xlf.addFile(originalPath, keys, messages);
// Emit only upon all ISL files combined into single XLF instance
const newFilePath = path_1.default.join(projectName, resourceFile);
const xlfFile = new vinyl_1.default({ path: newFilePath, contents: Buffer.from(xlf.toString(), 'utf-8') });
this.queue(xlfFile);
});
}
function createI18nFile(name, messages) {
const result = Object.create(null);
result[''] = [
'--------------------------------------------------------------------------------------------',
'Copyright (c) Microsoft Corporation. All rights reserved.',
'Licensed under the MIT License. See License.txt in the project root for license information.',
'--------------------------------------------------------------------------------------------',
'Do not edit this file. It is machine generated.'
];
for (const key of Object.keys(messages)) {
result[key] = messages[key];
}
let content = JSON.stringify(result, null, '\t');
if (process.platform === 'win32') {
content = content.replace(/\n/g, '\r\n');
}
return new vinyl_1.default({
path: path_1.default.join(name + '.i18n.json'),
contents: Buffer.from(content, 'utf8')
});
}
const i18nPackVersion = '1.0.0';
function getRecordFromL10nJsonFormat(l10nJsonFormat) {
const record = {};
for (const key of Object.keys(l10nJsonFormat).sort()) {
const value = l10nJsonFormat[key];
record[key] = typeof value === 'string' ? value : value.message;
}
return record;
}
function prepareI18nPackFiles(resultingTranslationPaths) {
const parsePromises = [];
const mainPack = { version: i18nPackVersion, contents: {} };
const extensionsPacks = {};
const errors = [];
return (0, event_stream_1.through)(function (xlf) {
let project = path_1.default.basename(path_1.default.dirname(path_1.default.dirname(xlf.relative)));
// strip `-new` since vscode-extensions-loc uses the `-new` suffix to indicate that it's from the new loc pipeline
const resource = path_1.default.basename(path_1.default.basename(xlf.relative, '.xlf'), '-new');
if (exports.EXTERNAL_EXTENSIONS.find(e => e === resource)) {
project = extensionsProject;
}
const contents = xlf.contents.toString();
log(`Found ${project}: ${resource}`);
const parsePromise = (0, l10n_dev_1.getL10nFilesFromXlf)(contents);
parsePromises.push(parsePromise);
parsePromise.then(resolvedFiles => {
resolvedFiles.forEach(file => {
const path = file.name;
const firstSlash = path.indexOf('/');
if (project === extensionsProject) {
// resource will be the extension id
let extPack = extensionsPacks[resource];
if (!extPack) {
extPack = extensionsPacks[resource] = { version: i18nPackVersion, contents: {} };
}
// remove 'extensions/extensionId/' segment
const secondSlash = path.indexOf('/', firstSlash + 1);
extPack.contents[path.substring(secondSlash + 1)] = getRecordFromL10nJsonFormat(file.messages);
}
else {
mainPack.contents[path.substring(firstSlash + 1)] = getRecordFromL10nJsonFormat(file.messages);
}
});
}).catch(reason => {
errors.push(reason);
});
}, function () {
Promise.all(parsePromises)
.then(() => {
if (errors.length > 0) {
throw errors;
}
const translatedMainFile = createI18nFile('./main', mainPack);
resultingTranslationPaths.push({ id: 'vscode', resourceName: 'main.i18n.json' });
this.queue(translatedMainFile);
for (const extensionId in extensionsPacks) {
const translatedExtFile = createI18nFile(`extensions/${extensionId}`, extensionsPacks[extensionId]);
this.queue(translatedExtFile);
resultingTranslationPaths.push({ id: extensionId, resourceName: `extensions/${extensionId}.i18n.json` });
}
this.queue(null);
})
.catch((reason) => {
this.emit('error', reason);
});
});
}
function prepareIslFiles(language, innoSetupConfig) {
const parsePromises = [];
return (0, event_stream_1.through)(function (xlf) {
const stream = this;
const parsePromise = XLF.parse(xlf.contents.toString());
parsePromises.push(parsePromise);
parsePromise.then(resolvedFiles => {
resolvedFiles.forEach(file => {
const translatedFile = createIslFile(file.name, file.messages, language, innoSetupConfig);
stream.queue(translatedFile);
});
}).catch(reason => {
this.emit('error', reason);
});
}, function () {
Promise.all(parsePromises)
.then(() => { this.queue(null); })
.catch(reason => {
this.emit('error', reason);
});
});
}
function createIslFile(name, messages, language, innoSetup) {
const content = [];
let originalContent;
if (path_1.default.basename(name) === 'Default') {
originalContent = new TextModel(fs_1.default.readFileSync(name + '.isl', 'utf8'));
}
else {
originalContent = new TextModel(fs_1.default.readFileSync(name + '.en.isl', 'utf8'));
}
originalContent.lines.forEach(line => {
if (line.length > 0) {
const firstChar = line.charAt(0);
if (firstChar === '[' || firstChar === ';') {
content.push(line);
}
else {
const sections = line.split('=');
const key = sections[0];
let translated = line;
if (key) {
const translatedMessage = messages[key];
if (translatedMessage) {
translated = `${key}=${translatedMessage}`;
}
}
content.push(translated);
}
}
});
const basename = path_1.default.basename(name);
const filePath = `${basename}.${language.id}.isl`;
const encoded = iconv_lite_umd_1.default.encode(Buffer.from(content.join('\r\n'), 'utf8').toString(), innoSetup.codePage);
return new vinyl_1.default({
path: filePath,
contents: Buffer.from(encoded),
});
}
function encodeEntities(value) {
const result = [];
for (let i = 0; i < value.length; i++) {
const ch = value[i];
switch (ch) {
case '<':
result.push('<');
break;
case '>':
result.push('>');
break;
case '&':
result.push('&');
break;
default:
result.push(ch);
}
}
return result.join('');
}
function decodeEntities(value) {
return value.replace(/</g, '<').replace(/>/g, '>').replace(/&/g, '&');
}
//# sourceMappingURL=i18n.js.map
================================================
FILE: build/lib/i18n.resources.json
================================================
{
"editor": [
{
"name": "vs/platform",
"project": "vscode-editor"
},
{
"name": "vs/editor/contrib",
"project": "vscode-editor"
},
{
"name": "vs/editor",
"project": "vscode-editor"
},
{
"name": "vs/base",
"project": "vscode-editor"
}
],
"workbench": [
{
"name": "vs/code",
"project": "vscode-workbench"
},
{
"name": "vs/workbench",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/api/common",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/bulkEdit",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/cli",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/codeEditor",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/callHierarchy",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/typeHierarchy",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/codeActions",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/commands",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/markdown",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/comments",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/debug",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/dialogs",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/multiDiffEditor",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/emmet",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/services/assignment",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/services/auxiliaryWindow",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/extensions",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/externalTerminal",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/files",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/folding",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/html",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/issue",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/inlayHints",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/interactive",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/languageStatus",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/limitIndicator",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/keybindings",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/markers",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/mergeEditor",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/localization",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/logs",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/output",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/performance",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/preferences",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/notebook",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/inlineChat",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/chat",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/quickaccess",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/userData",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/remote",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/relauncher",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/sash",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/scm",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/search",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/searchEditor",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/snippets",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/format",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/tags",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/speech",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/surveys",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/tasks",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/testing",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/terminal",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/terminalContrib",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/themes",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/trust",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/update",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/url",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/watermark",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/webview",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/webviewPanel",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/workspace",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/workspaces",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/customEditor",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/externalUriOpener",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/welcomeGettingStarted",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/welcomePage",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/welcomeViews",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/welcomeWalkthrough",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/welcomeDialog",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/outline",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/userDataSync",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/editSessions",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/views",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/languageDetection",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/accessibilitySignals",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/deprecatedExtensionMigrator",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/bracketPairColorizer2Telemetry",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/scrollLocking",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/remoteTunnel",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/services/actions",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/services/authToken",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/services/backup",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/services/bulkEdit",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/services/clipboard",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/services/commands",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/services/configuration",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/services/configurationResolver",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/services/dialogs",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/services/editor",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/services/extensions",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/services/extensionManagement",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/services/files",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/services/filesConfiguration",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/services/history",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/services/hover",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/services/log",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/services/integrity",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/services/keybinding",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/services/lifecycle",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/services/language",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/services/progress",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/services/remote",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/services/search",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/services/suggest",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/services/textfile",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/services/themes",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/services/textMate",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/services/workingCopy",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/services/workspaces",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/services/decorations",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/services/label",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/services/preferences",
"project": "vscode-preferences"
},
{
"name": "vs/workbench/services/notification",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/services/userData",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/services/userDataSync",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/services/editSessions",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/services/views",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/timeline",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/localHistory",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/services/authentication",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/services/extensionRecommendations",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/services/gettingStarted",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/services/host",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/userDataProfile",
"project": "vscode-profiles"
},
{
"name": "vs/workbench/services/userDataProfile",
"project": "vscode-profiles"
},
{
"name": "vs/workbench/services/localization",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/share",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/accessibility",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/services/issue",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/services/secrets",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/accountEntitlements",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/authentication",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/replNotebook",
"project": "vscode-workbench"
},
{
"name": "vs/workbench/contrib/list",
"project": "vscode-workbench"
}
]
}
================================================
FILE: build/lib/i18n.ts
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
import path from 'path';
import fs from 'fs';
import { map, merge, through, ThroughStream } from 'event-stream';
import jsonMerge from 'gulp-merge-json';
import File from 'vinyl';
import xml2js from 'xml2js';
import gulp from 'gulp';
import fancyLog from 'fancy-log';
import ansiColors from 'ansi-colors';
import iconv from '@vscode/iconv-lite-umd';
import { l10nJsonFormat, getL10nXlf, l10nJsonDetails, getL10nFilesFromXlf, getL10nJson } from '@vscode/l10n-dev';
const REPO_ROOT_PATH = path.join(__dirname, '../..');
function log(message: any, ...rest: any[]): void {
fancyLog(ansiColors.green('[i18n]'), message, ...rest);
}
export interface Language {
id: string; // language id, e.g. zh-tw, de
translationId?: string; // language id used in translation tools, e.g. zh-hant, de (optional, if not set, the id is used)
folderName?: string; // language specific folder name, e.g. cht, deu (optional, if not set, the id is used)
}
export interface InnoSetup {
codePage: string; //code page for encoding (http://www.jrsoftware.org/ishelp/index.php?topic=langoptionssection)
}
export const defaultLanguages: Language[] = [
{ id: 'zh-tw', folderName: 'cht', translationId: 'zh-hant' },
{ id: 'zh-cn', folderName: 'chs', translationId: 'zh-hans' },
{ id: 'ja', folderName: 'jpn' },
{ id: 'ko', folderName: 'kor' },
{ id: 'de', folderName: 'deu' },
{ id: 'fr', folderName: 'fra' },
{ id: 'es', folderName: 'esn' },
{ id: 'ru', folderName: 'rus' },
{ id: 'it', folderName: 'ita' }
];
// languages requested by the community to non-stable builds
export const extraLanguages: Language[] = [
{ id: 'pt-br', folderName: 'ptb' },
{ id: 'hu', folderName: 'hun' },
{ id: 'tr', folderName: 'trk' }
];
interface Item {
id: string;
message: string;
comment?: string;
}
export interface Resource {
name: string;
project: string;
}
interface LocalizeInfo {
key: string;
comment: string[];
}
module LocalizeInfo {
export function is(value: any): value is LocalizeInfo {
const candidate = value as LocalizeInfo;
return candidate && typeof candidate.key === 'string' && (candidate.comment === undefined || (Array.isArray(candidate.comment) && candidate.comment.every(element => typeof element === 'string')));
}
}
interface BundledFormat {
keys: Record;
messages: Record;
bundles: Record;
}
module BundledFormat {
export function is(value: any): value is BundledFormat {
if (value === undefined) {
return false;
}
const candidate = value as BundledFormat;
const length = Object.keys(value).length;
return length === 3 && !!candidate.keys && !!candidate.messages && !!candidate.bundles;
}
}
type NLSKeysFormat = [string /* module ID */, string[] /* keys */];
module NLSKeysFormat {
export function is(value: any): value is NLSKeysFormat {
if (value === undefined) {
return false;
}
const candidate = value as NLSKeysFormat;
return Array.isArray(candidate) && Array.isArray(candidate[1]);
}
}
interface BundledExtensionFormat {
[key: string]: {
messages: string[];
keys: (string | LocalizeInfo)[];
};
}
interface I18nFormat {
version: string;
contents: {
[module: string]: {
[messageKey: string]: string;
};
};
}
export class Line {
private buffer: string[] = [];
constructor(indent: number = 0) {
if (indent > 0) {
this.buffer.push(new Array(indent + 1).join(' '));
}
}
public append(value: string): Line {
this.buffer.push(value);
return this;
}
public toString(): string {
return this.buffer.join('');
}
}
class TextModel {
private _lines: string[];
constructor(contents: string) {
this._lines = contents.split(/\r\n|\r|\n/);
}
public get lines(): string[] {
return this._lines;
}
}
export class XLF {
private buffer: string[];
private files: Record;
public numberOfMessages: number;
constructor(public project: string) {
this.buffer = [];
this.files = Object.create(null);
this.numberOfMessages = 0;
}
public toString(): string {
this.appendHeader();
const files = Object.keys(this.files).sort();
for (const file of files) {
this.appendNewLine(``, 2);
const items = this.files[file].sort((a: Item, b: Item) => {
return a.id < b.id ? -1 : a.id > b.id ? 1 : 0;
});
for (const item of items) {
this.addStringItem(file, item);
}
this.appendNewLine('');
}
this.appendFooter();
return this.buffer.join('\r\n');
}
public addFile(original: string, keys: (string | LocalizeInfo)[], messages: string[]) {
if (keys.length === 0) {
console.log('No keys in ' + original);
return;
}
if (keys.length !== messages.length) {
throw new Error(`Unmatching keys(${keys.length}) and messages(${messages.length}).`);
}
this.numberOfMessages += keys.length;
this.files[original] = [];
const existingKeys = new Set();
for (let i = 0; i < keys.length; i++) {
const key = keys[i];
let realKey: string | undefined;
let comment: string | undefined;
if (typeof key === 'string') {
realKey = key;
comment = undefined;
} else if (LocalizeInfo.is(key)) {
realKey = key.key;
if (key.comment && key.comment.length > 0) {
comment = key.comment.map(comment => encodeEntities(comment)).join('\r\n');
}
}
if (!realKey || existingKeys.has(realKey)) {
continue;
}
existingKeys.add(realKey);
const message: string = encodeEntities(messages[i]);
this.files[original].push({ id: realKey, message: message, comment: comment });
}
}
private addStringItem(file: string, item: Item): void {
if (!item.id || item.message === undefined || item.message === null) {
throw new Error(`No item ID or value specified: ${JSON.stringify(item)}. File: ${file}`);
}
if (item.message.length === 0) {
log(`Item with id ${item.id} in file ${file} has an empty message.`);
}
this.appendNewLine(``, 4);
this.appendNewLine(`${item.message}`, 6);
if (item.comment) {
this.appendNewLine(`${item.comment}`, 6);
}
this.appendNewLine('', 4);
}
private appendHeader(): void {
this.appendNewLine('', 0);
this.appendNewLine('', 0);
}
private appendFooter(): void {
this.appendNewLine('', 0);
}
private appendNewLine(content: string, indent?: number): void {
const line = new Line(indent);
line.append(content);
this.buffer.push(line.toString());
}
static parse = function (xlfString: string): Promise {
return new Promise((resolve, reject) => {
const parser = new xml2js.Parser();
const files: { messages: Record; name: string; language: string }[] = [];
parser.parseString(xlfString, function (err: any, result: any) {
if (err) {
reject(new Error(`XLF parsing error: Failed to parse XLIFF string. ${err}`));
}
const fileNodes: any[] = result['xliff']['file'];
if (!fileNodes) {
reject(new Error(`XLF parsing error: XLIFF file does not contain "xliff" or "file" node(s) required for parsing.`));
}
fileNodes.forEach((file) => {
const name = file.$.original;
if (!name) {
reject(new Error(`XLF parsing error: XLIFF file node does not contain original attribute to determine the original location of the resource file.`));
}
const language = file.$['target-language'];
if (!language) {
reject(new Error(`XLF parsing error: XLIFF file node does not contain target-language attribute to determine translated language.`));
}
const messages: Record = {};
const transUnits = file.body[0]['trans-unit'];
if (transUnits) {
transUnits.forEach((unit: any) => {
const key = unit.$.id;
if (!unit.target) {
return; // No translation available
}
let val = unit.target[0];
if (typeof val !== 'string') {
// We allow empty source values so support them for translations as well.
val = val._ ? val._ : '';
}
if (!key) {
reject(new Error(`XLF parsing error: trans-unit ${JSON.stringify(unit, undefined, 0)} defined in file ${name} is missing the ID attribute.`));
return;
}
messages[key] = decodeEntities(val);
});
files.push({ messages, name, language: language.toLowerCase() });
}
});
resolve(files);
});
});
};
}
function sortLanguages(languages: Language[]): Language[] {
return languages.sort((a: Language, b: Language): number => {
return a.id < b.id ? -1 : (a.id > b.id ? 1 : 0);
});
}
function stripComments(content: string): string {
// Copied from stripComments.js
//
// First group matches a double quoted string
// Second group matches a single quoted string
// Third group matches a multi line comment
// Forth group matches a single line comment
// Fifth group matches a trailing comma
const regexp = /("[^"\\]*(?:\\.[^"\\]*)*")|('[^'\\]*(?:\\.[^'\\]*)*')|(\/\*[^\/\*]*(?:(?:\*|\/)[^\/\*]*)*?\*\/)|(\/{2,}.*?(?:(?:\r?\n)|$))|(,\s*[}\]])/g;
const result = content.replace(regexp, (match, _m1: string, _m2: string, m3: string, m4: string, m5: string) => {
// Only one of m1, m2, m3, m4, m5 matches
if (m3) {
// A block comment. Replace with nothing
return '';
} else if (m4) {
// Since m4 is a single line comment is is at least of length 2 (e.g. //)
// If it ends in \r?\n then keep it.
const length = m4.length;
if (m4[length - 1] === '\n') {
return m4[length - 2] === '\r' ? '\r\n' : '\n';
} else {
return '';
}
} else if (m5) {
// Remove the trailing comma
return match.substring(1);
} else {
// We match a string
return match;
}
});
return result;
}
function processCoreBundleFormat(base: string, fileHeader: string, languages: Language[], json: NLSKeysFormat, emitter: ThroughStream) {
const languageDirectory = path.join(REPO_ROOT_PATH, '..', 'vscode-loc', 'i18n');
if (!fs.existsSync(languageDirectory)) {
log(`No VS Code localization repository found. Looking at ${languageDirectory}`);
log(`To bundle translations please check out the vscode-loc repository as a sibling of the vscode repository.`);
}
const sortedLanguages = sortLanguages(languages);
sortedLanguages.forEach((language) => {
if (process.env['VSCODE_BUILD_VERBOSE']) {
log(`Generating nls bundles for: ${language.id}`);
}
const languageFolderName = language.translationId || language.id;
const i18nFile = path.join(languageDirectory, `vscode-language-pack-${languageFolderName}`, 'translations', 'main.i18n.json');
let allMessages: I18nFormat | undefined;
if (fs.existsSync(i18nFile)) {
const content = stripComments(fs.readFileSync(i18nFile, 'utf8'));
allMessages = JSON.parse(content);
}
let nlsIndex = 0;
const nlsResult: Array = [];
for (const [moduleId, nlsKeys] of json) {
const moduleTranslations = allMessages?.contents[moduleId];
for (const nlsKey of nlsKeys) {
nlsResult.push(moduleTranslations?.[nlsKey]); // pushing `undefined` is fine, as we keep english strings as fallback for monaco editor in the build
nlsIndex++;
}
}
emitter.queue(new File({
contents: Buffer.from(`${fileHeader}
globalThis._VSCODE_NLS_MESSAGES=${JSON.stringify(nlsResult)};
globalThis._VSCODE_NLS_LANGUAGE=${JSON.stringify(language.id)};`),
base,
path: `${base}/nls.messages.${language.id}.js`
}));
});
}
export function processNlsFiles(opts: { out: string; fileHeader: string; languages: Language[] }): ThroughStream {
return through(function (this: ThroughStream, file: File) {
const fileName = path.basename(file.path);
if (fileName === 'nls.keys.json') {
try {
const contents = file.contents.toString('utf8');
const json = JSON.parse(contents);
if (NLSKeysFormat.is(json)) {
processCoreBundleFormat(file.base, opts.fileHeader, opts.languages, json, this);
}
} catch (error) {
this.emit('error', `Failed to read component file: ${error}`);
}
}
this.queue(file);
});
}
const editorProject: string = 'vscode-editor',
workbenchProject: string = 'vscode-workbench',
extensionsProject: string = 'vscode-extensions',
setupProject: string = 'vscode-setup',
serverProject: string = 'vscode-server';
export function getResource(sourceFile: string): Resource {
let resource: string;
if (/^vs\/platform/.test(sourceFile)) {
return { name: 'vs/platform', project: editorProject };
} else if (/^vs\/editor\/contrib/.test(sourceFile)) {
return { name: 'vs/editor/contrib', project: editorProject };
} else if (/^vs\/editor/.test(sourceFile)) {
return { name: 'vs/editor', project: editorProject };
} else if (/^vs\/base/.test(sourceFile)) {
return { name: 'vs/base', project: editorProject };
} else if (/^vs\/code/.test(sourceFile)) {
return { name: 'vs/code', project: workbenchProject };
} else if (/^vs\/server/.test(sourceFile)) {
return { name: 'vs/server', project: serverProject };
} else if (/^vs\/workbench\/contrib/.test(sourceFile)) {
resource = sourceFile.split('/', 4).join('/');
return { name: resource, project: workbenchProject };
} else if (/^vs\/workbench\/services/.test(sourceFile)) {
resource = sourceFile.split('/', 4).join('/');
return { name: resource, project: workbenchProject };
} else if (/^vs\/workbench/.test(sourceFile)) {
return { name: 'vs/workbench', project: workbenchProject };
}
throw new Error(`Could not identify the XLF bundle for ${sourceFile}`);
}
export function createXlfFilesForCoreBundle(): ThroughStream {
return through(function (this: ThroughStream, file: File) {
const basename = path.basename(file.path);
if (basename === 'nls.metadata.json') {
if (file.isBuffer()) {
const xlfs: Record = Object.create(null);
const json: BundledFormat = JSON.parse((file.contents as Buffer).toString('utf8'));
for (const coreModule in json.keys) {
const projectResource = getResource(coreModule);
const resource = projectResource.name;
const project = projectResource.project;
const keys = json.keys[coreModule];
const messages = json.messages[coreModule];
if (keys.length !== messages.length) {
this.emit('error', `There is a mismatch between keys and messages in ${file.relative} for module ${coreModule}`);
return;
} else {
let xlf = xlfs[resource];
if (!xlf) {
xlf = new XLF(project);
xlfs[resource] = xlf;
}
xlf.addFile(`src/${coreModule}`, keys, messages);
}
}
for (const resource in xlfs) {
const xlf = xlfs[resource];
const filePath = `${xlf.project}/${resource.replace(/\//g, '_')}.xlf`;
const xlfFile = new File({
path: filePath,
contents: Buffer.from(xlf.toString(), 'utf8')
});
this.queue(xlfFile);
}
} else {
this.emit('error', new Error(`File ${file.relative} is not using a buffer content`));
return;
}
} else {
this.emit('error', new Error(`File ${file.relative} is not a core meta data file.`));
return;
}
});
}
function createL10nBundleForExtension(extensionFolderName: string, prefixWithBuildFolder: boolean): NodeJS.ReadWriteStream {
const prefix = prefixWithBuildFolder ? '.build/' : '';
return gulp
.src([
// For source code of extensions
`${prefix}extensions/${extensionFolderName}/{src,client,server}/**/*.{ts,tsx}`,
// // For any dependencies pulled in (think vscode-css-languageservice or @vscode/emmet-helper)
`${prefix}extensions/${extensionFolderName}/**/node_modules/{@vscode,vscode-*}/**/*.{js,jsx}`,
// // For any dependencies pulled in that bundle @vscode/l10n. They needed to export the bundle
`${prefix}extensions/${extensionFolderName}/**/bundle.l10n.json`,
])
.pipe(map(function (data, callback) {
const file = data as File;
if (!file.isBuffer()) {
// Not a buffer so we drop it
callback();
return;
}
const extension = path.extname(file.relative);
if (extension !== '.json') {
const contents = file.contents.toString('utf8');
getL10nJson([{ contents, extension }])
.then((json) => {
callback(undefined, new File({
path: `extensions/${extensionFolderName}/bundle.l10n.json`,
contents: Buffer.from(JSON.stringify(json), 'utf8')
}));
})
.catch((err) => {
callback(new Error(`File ${file.relative} threw an error when parsing: ${err}`));
});
// signal pause?
return false;
}
// for bundle.l10n.jsons
let bundleJson;
try {
bundleJson = JSON.parse(file.contents.toString('utf8'));
} catch (err) {
callback(new Error(`File ${file.relative} threw an error when parsing: ${err}`));
return;
}
// some validation of the bundle.l10n.json format
for (const key in bundleJson) {
if (
typeof bundleJson[key] !== 'string' &&
(typeof bundleJson[key].message !== 'string' || !Array.isArray(bundleJson[key].comment))
) {
callback(new Error(`Invalid bundle.l10n.json file. The value for key ${key} is not in the expected format.`));
return;
}
}
callback(undefined, file);
}))
.pipe(jsonMerge({
fileName: `extensions/${extensionFolderName}/bundle.l10n.json`,
jsonSpace: '',
concatArrays: true
}));
}
export const EXTERNAL_EXTENSIONS = [
'ms-vscode.js-debug',
'ms-vscode.js-debug-companion',
'ms-vscode.vscode-js-profile-table',
];
export function createXlfFilesForExtensions(): ThroughStream {
let counter: number = 0;
let folderStreamEnded: boolean = false;
let folderStreamEndEmitted: boolean = false;
return through(function (this: ThroughStream, extensionFolder: File) {
const folderStream = this;
const stat = fs.statSync(extensionFolder.path);
if (!stat.isDirectory()) {
return;
}
const extensionFolderName = path.basename(extensionFolder.path);
if (extensionFolderName === 'node_modules') {
return;
}
// Get extension id and use that as the id
const manifest = fs.readFileSync(path.join(extensionFolder.path, 'package.json'), 'utf-8');
const manifestJson = JSON.parse(manifest);
const extensionId = manifestJson.publisher + '.' + manifestJson.name;
counter++;
let _l10nMap: Map;
function getL10nMap() {
if (!_l10nMap) {
_l10nMap = new Map();
}
return _l10nMap;
}
merge(
gulp.src([`.build/extensions/${extensionFolderName}/package.nls.json`, `.build/extensions/${extensionFolderName}/**/nls.metadata.json`], { allowEmpty: true }),
createL10nBundleForExtension(extensionFolderName, EXTERNAL_EXTENSIONS.includes(extensionId))
).pipe(through(function (file: File) {
if (file.isBuffer()) {
const buffer: Buffer = file.contents as Buffer;
const basename = path.basename(file.path);
if (basename === 'package.nls.json') {
const json: l10nJsonFormat = JSON.parse(buffer.toString('utf8'));
getL10nMap().set(`extensions/${extensionId}/package`, json);
} else if (basename === 'nls.metadata.json') {
const json: BundledExtensionFormat = JSON.parse(buffer.toString('utf8'));
const relPath = path.relative(`.build/extensions/${extensionFolderName}`, path.dirname(file.path));
for (const file in json) {
const fileContent = json[file];
const info: l10nJsonFormat = Object.create(null);
for (let i = 0; i < fileContent.messages.length; i++) {
const message = fileContent.messages[i];
const { key, comment } = LocalizeInfo.is(fileContent.keys[i])
? fileContent.keys[i] as LocalizeInfo
: { key: fileContent.keys[i] as string, comment: undefined };
info[key] = comment ? { message, comment } : message;
}
getL10nMap().set(`extensions/${extensionId}/${relPath}/${file}`, info);
}
} else if (basename === 'bundle.l10n.json') {
const json: l10nJsonFormat = JSON.parse(buffer.toString('utf8'));
getL10nMap().set(`extensions/${extensionId}/bundle`, json);
} else {
this.emit('error', new Error(`${file.path} is not a valid extension nls file`));
return;
}
}
}, function () {
if (_l10nMap?.size > 0) {
const xlfFile = new File({
path: path.join(extensionsProject, extensionId + '.xlf'),
contents: Buffer.from(getL10nXlf(_l10nMap), 'utf8')
});
folderStream.queue(xlfFile);
}
this.queue(null);
counter--;
if (counter === 0 && folderStreamEnded && !folderStreamEndEmitted) {
folderStreamEndEmitted = true;
folderStream.queue(null);
}
}));
}, function () {
folderStreamEnded = true;
if (counter === 0) {
folderStreamEndEmitted = true;
this.queue(null);
}
});
}
export function createXlfFilesForIsl(): ThroughStream {
return through(function (this: ThroughStream, file: File) {
let projectName: string,
resourceFile: string;
if (path.basename(file.path) === 'messages.en.isl') {
projectName = setupProject;
resourceFile = 'messages.xlf';
} else {
throw new Error(`Unknown input file ${file.path}`);
}
const xlf = new XLF(projectName),
keys: string[] = [],
messages: string[] = [];
const model = new TextModel(file.contents.toString());
let inMessageSection = false;
model.lines.forEach(line => {
if (line.length === 0) {
return;
}
const firstChar = line.charAt(0);
switch (firstChar) {
case ';':
// Comment line;
return;
case '[':
inMessageSection = '[Messages]' === line || '[CustomMessages]' === line;
return;
}
if (!inMessageSection) {
return;
}
const sections: string[] = line.split('=');
if (sections.length !== 2) {
throw new Error(`Badly formatted message found: ${line}`);
} else {
const key = sections[0];
const value = sections[1];
if (key.length > 0 && value.length > 0) {
keys.push(key);
messages.push(value);
}
}
});
const originalPath = file.path.substring(file.cwd.length + 1, file.path.split('.')[0].length).replace(/\\/g, '/');
xlf.addFile(originalPath, keys, messages);
// Emit only upon all ISL files combined into single XLF instance
const newFilePath = path.join(projectName, resourceFile);
const xlfFile = new File({ path: newFilePath, contents: Buffer.from(xlf.toString(), 'utf-8') });
this.queue(xlfFile);
});
}
function createI18nFile(name: string, messages: any): File {
const result = Object.create(null);
result[''] = [
'--------------------------------------------------------------------------------------------',
'Copyright (c) Microsoft Corporation. All rights reserved.',
'Licensed under the MIT License. See License.txt in the project root for license information.',
'--------------------------------------------------------------------------------------------',
'Do not edit this file. It is machine generated.'
];
for (const key of Object.keys(messages)) {
result[key] = messages[key];
}
let content = JSON.stringify(result, null, '\t');
if (process.platform === 'win32') {
content = content.replace(/\n/g, '\r\n');
}
return new File({
path: path.join(name + '.i18n.json'),
contents: Buffer.from(content, 'utf8')
});
}
interface I18nPack {
version: string;
contents: {
[path: string]: Record;
};
}
const i18nPackVersion = '1.0.0';
export interface TranslationPath {
id: string;
resourceName: string;
}
function getRecordFromL10nJsonFormat(l10nJsonFormat: l10nJsonFormat): Record {
const record: Record = {};
for (const key of Object.keys(l10nJsonFormat).sort()) {
const value = l10nJsonFormat[key];
record[key] = typeof value === 'string' ? value : value.message;
}
return record;
}
export function prepareI18nPackFiles(resultingTranslationPaths: TranslationPath[]): NodeJS.ReadWriteStream {
const parsePromises: Promise[] = [];
const mainPack: I18nPack = { version: i18nPackVersion, contents: {} };
const extensionsPacks: Record = {};
const errors: any[] = [];
return through(function (this: ThroughStream, xlf: File) {
let project = path.basename(path.dirname(path.dirname(xlf.relative)));
// strip `-new` since vscode-extensions-loc uses the `-new` suffix to indicate that it's from the new loc pipeline
const resource = path.basename(path.basename(xlf.relative, '.xlf'), '-new');
if (EXTERNAL_EXTENSIONS.find(e => e === resource)) {
project = extensionsProject;
}
const contents = xlf.contents.toString();
log(`Found ${project}: ${resource}`);
const parsePromise = getL10nFilesFromXlf(contents);
parsePromises.push(parsePromise);
parsePromise.then(
resolvedFiles => {
resolvedFiles.forEach(file => {
const path = file.name;
const firstSlash = path.indexOf('/');
if (project === extensionsProject) {
// resource will be the extension id
let extPack = extensionsPacks[resource];
if (!extPack) {
extPack = extensionsPacks[resource] = { version: i18nPackVersion, contents: {} };
}
// remove 'extensions/extensionId/' segment
const secondSlash = path.indexOf('/', firstSlash + 1);
extPack.contents[path.substring(secondSlash + 1)] = getRecordFromL10nJsonFormat(file.messages);
} else {
mainPack.contents[path.substring(firstSlash + 1)] = getRecordFromL10nJsonFormat(file.messages);
}
});
}
).catch(reason => {
errors.push(reason);
});
}, function () {
Promise.all(parsePromises)
.then(() => {
if (errors.length > 0) {
throw errors;
}
const translatedMainFile = createI18nFile('./main', mainPack);
resultingTranslationPaths.push({ id: 'vscode', resourceName: 'main.i18n.json' });
this.queue(translatedMainFile);
for (const extensionId in extensionsPacks) {
const translatedExtFile = createI18nFile(`extensions/${extensionId}`, extensionsPacks[extensionId]);
this.queue(translatedExtFile);
resultingTranslationPaths.push({ id: extensionId, resourceName: `extensions/${extensionId}.i18n.json` });
}
this.queue(null);
})
.catch((reason) => {
this.emit('error', reason);
});
});
}
export function prepareIslFiles(language: Language, innoSetupConfig: InnoSetup): ThroughStream {
const parsePromises: Promise[] = [];
return through(function (this: ThroughStream, xlf: File) {
const stream = this;
const parsePromise = XLF.parse(xlf.contents.toString());
parsePromises.push(parsePromise);
parsePromise.then(
resolvedFiles => {
resolvedFiles.forEach(file => {
const translatedFile = createIslFile(file.name, file.messages, language, innoSetupConfig);
stream.queue(translatedFile);
});
}
).catch(reason => {
this.emit('error', reason);
});
}, function () {
Promise.all(parsePromises)
.then(() => { this.queue(null); })
.catch(reason => {
this.emit('error', reason);
});
});
}
function createIslFile(name: string, messages: l10nJsonFormat, language: Language, innoSetup: InnoSetup): File {
const content: string[] = [];
let originalContent: TextModel;
if (path.basename(name) === 'Default') {
originalContent = new TextModel(fs.readFileSync(name + '.isl', 'utf8'));
} else {
originalContent = new TextModel(fs.readFileSync(name + '.en.isl', 'utf8'));
}
originalContent.lines.forEach(line => {
if (line.length > 0) {
const firstChar = line.charAt(0);
if (firstChar === '[' || firstChar === ';') {
content.push(line);
} else {
const sections: string[] = line.split('=');
const key = sections[0];
let translated = line;
if (key) {
const translatedMessage = messages[key];
if (translatedMessage) {
translated = `${key}=${translatedMessage}`;
}
}
content.push(translated);
}
}
});
const basename = path.basename(name);
const filePath = `${basename}.${language.id}.isl`;
const encoded = iconv.encode(Buffer.from(content.join('\r\n'), 'utf8').toString(), innoSetup.codePage);
return new File({
path: filePath,
contents: Buffer.from(encoded),
});
}
function encodeEntities(value: string): string {
const result: string[] = [];
for (let i = 0; i < value.length; i++) {
const ch = value[i];
switch (ch) {
case '<':
result.push('<');
break;
case '>':
result.push('>');
break;
case '&':
result.push('&');
break;
default:
result.push(ch);
}
}
return result.join('');
}
function decodeEntities(value: string): string {
return value.replace(/</g, '<').replace(/>/g, '>').replace(/&/g, '&');
}
================================================
FILE: build/lib/inlineMeta.js
================================================
"use strict";
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
Object.defineProperty(exports, "__esModule", { value: true });
exports.inlineMeta = inlineMeta;
const event_stream_1 = __importDefault(require("event-stream"));
const path_1 = require("path");
const packageJsonMarkerId = 'BUILD_INSERT_PACKAGE_CONFIGURATION';
// TODO in order to inline `product.json`, more work is
// needed to ensure that we cover all cases where modifications
// are done to the product configuration during build. There are
// at least 2 more changes that kick in very late:
// - a `darwinUniversalAssetId` is added in`create-universal-app.ts`
// - a `target` is added in `gulpfile.vscode.win32.js`
// const productJsonMarkerId = 'BUILD_INSERT_PRODUCT_CONFIGURATION';
function inlineMeta(result, ctx) {
return result.pipe(event_stream_1.default.through(function (file) {
if (matchesFile(file, ctx)) {
let content = file.contents.toString();
let markerFound = false;
const packageMarker = `${packageJsonMarkerId}:"${packageJsonMarkerId}"`; // this needs to be the format after esbuild has processed the file (e.g. double quotes)
if (content.includes(packageMarker)) {
content = content.replace(packageMarker, JSON.stringify(JSON.parse(ctx.packageJsonFn())).slice(1, -1) /* trim braces */);
markerFound = true;
}
// const productMarker = `${productJsonMarkerId}:"${productJsonMarkerId}"`; // this needs to be the format after esbuild has processed the file (e.g. double quotes)
// if (content.includes(productMarker)) {
// content = content.replace(productMarker, JSON.stringify(JSON.parse(ctx.productJsonFn())).slice(1, -1) /* trim braces */);
// markerFound = true;
// }
if (markerFound) {
file.contents = Buffer.from(content);
}
}
this.emit('data', file);
}));
}
function matchesFile(file, ctx) {
for (const targetPath of ctx.targetPaths) {
if (file.basename === (0, path_1.basename)(targetPath)) { // TODO would be nicer to figure out root relative path to not match on false positives
return true;
}
}
return false;
}
//# sourceMappingURL=inlineMeta.js.map
================================================
FILE: build/lib/inlineMeta.ts
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
import es from 'event-stream';
import { basename } from 'path';
import File from 'vinyl';
export interface IInlineMetaContext {
readonly targetPaths: string[];
readonly packageJsonFn: () => string;
readonly productJsonFn: () => string;
}
const packageJsonMarkerId = 'BUILD_INSERT_PACKAGE_CONFIGURATION';
// TODO in order to inline `product.json`, more work is
// needed to ensure that we cover all cases where modifications
// are done to the product configuration during build. There are
// at least 2 more changes that kick in very late:
// - a `darwinUniversalAssetId` is added in`create-universal-app.ts`
// - a `target` is added in `gulpfile.vscode.win32.js`
// const productJsonMarkerId = 'BUILD_INSERT_PRODUCT_CONFIGURATION';
export function inlineMeta(result: NodeJS.ReadWriteStream, ctx: IInlineMetaContext): NodeJS.ReadWriteStream {
return result.pipe(es.through(function (file: File) {
if (matchesFile(file, ctx)) {
let content = file.contents.toString();
let markerFound = false;
const packageMarker = `${packageJsonMarkerId}:"${packageJsonMarkerId}"`; // this needs to be the format after esbuild has processed the file (e.g. double quotes)
if (content.includes(packageMarker)) {
content = content.replace(packageMarker, JSON.stringify(JSON.parse(ctx.packageJsonFn())).slice(1, -1) /* trim braces */);
markerFound = true;
}
// const productMarker = `${productJsonMarkerId}:"${productJsonMarkerId}"`; // this needs to be the format after esbuild has processed the file (e.g. double quotes)
// if (content.includes(productMarker)) {
// content = content.replace(productMarker, JSON.stringify(JSON.parse(ctx.productJsonFn())).slice(1, -1) /* trim braces */);
// markerFound = true;
// }
if (markerFound) {
file.contents = Buffer.from(content);
}
}
this.emit('data', file);
}));
}
function matchesFile(file: File, ctx: IInlineMetaContext): boolean {
for (const targetPath of ctx.targetPaths) {
if (file.basename === basename(targetPath)) { // TODO would be nicer to figure out root relative path to not match on false positives
return true;
}
}
return false;
}
================================================
FILE: build/lib/layersChecker.js
================================================
"use strict";
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
Object.defineProperty(exports, "__esModule", { value: true });
const typescript_1 = __importDefault(require("typescript"));
const fs_1 = require("fs");
const path_1 = require("path");
const minimatch_1 = require("minimatch");
//
// #############################################################################################
//
// A custom typescript checker for the specific task of detecting the use of certain types in a
// layer that does not allow such use. For example:
// - using DOM globals in common/node/electron-main layer (e.g. HTMLElement)
// - using node.js globals in common/browser layer (e.g. process)
//
// Make changes to below RULES to lift certain files from these checks only if absolutely needed
//
// #############################################################################################
//
// Types we assume are present in all implementations of JS VMs (node.js, browsers)
// Feel free to add more core types as you see needed if present in node.js and browsers
const CORE_TYPES = [
'setTimeout',
'clearTimeout',
'setInterval',
'clearInterval',
'console',
'Console',
'Error',
'ErrorConstructor',
'String',
'TextDecoder',
'TextEncoder',
'self',
'queueMicrotask',
'Array',
'Uint8Array',
'Uint16Array',
'Uint32Array',
'Int8Array',
'Int16Array',
'Int32Array',
'Float32Array',
'Float64Array',
'Uint8ClampedArray',
'BigUint64Array',
'BigInt64Array',
'btoa',
'atob',
'AbortController',
'AbortSignal',
'MessageChannel',
'MessagePort',
'URL',
'URLSearchParams',
'ReadonlyArray',
'Event',
'EventTarget',
'BroadcastChannel',
'performance',
'Blob',
'crypto',
'File',
'fetch',
'RequestInit',
'Headers',
'Request',
'Response',
'Body',
'__type',
'__global',
'Performance',
'PerformanceMark',
'PerformanceObserver',
'ImportMeta',
'structuredClone',
// webcrypto has been available since Node.js 19, but still live in dom.d.ts
'Crypto',
'SubtleCrypto',
'JsonWebKey',
'MessageEvent',
];
// Types that are defined in a common layer but are known to be only
// available in native environments should not be allowed in browser
const NATIVE_TYPES = [
'NativeParsedArgs',
'INativeEnvironmentService',
'AbstractNativeEnvironmentService',
'INativeWindowConfiguration',
'ICommonNativeHostService',
'INativeHostService',
'IMainProcessService'
];
const RULES = [
// Tests: skip
{
target: '**/vs/**/test/**',
skip: true // -> skip all test files
},
// Common: vs/base/common/platform.ts
{
target: '**/vs/base/common/platform.ts',
allowedTypes: [
...CORE_TYPES,
// Safe access to postMessage() and friends
'MessageEvent',
],
disallowedTypes: NATIVE_TYPES,
disallowedDefinitions: [
'lib.dom.d.ts', // no DOM
'@types/node' // no node.js
]
},
// Common: vs/base/common/async.ts
{
target: '**/vs/base/common/async.ts',
allowedTypes: [
...CORE_TYPES,
// Safe access to requestIdleCallback & cancelIdleCallback
'requestIdleCallback',
'cancelIdleCallback'
],
disallowedTypes: NATIVE_TYPES,
disallowedDefinitions: [
'lib.dom.d.ts', // no DOM
'@types/node' // no node.js
]
},
// Common: vs/base/common/performance.ts
{
target: '**/vs/base/common/performance.ts',
allowedTypes: [
...CORE_TYPES,
// Safe access to Performance
'Performance',
'PerformanceEntry',
'PerformanceTiming'
],
disallowedTypes: NATIVE_TYPES,
disallowedDefinitions: [
'lib.dom.d.ts', // no DOM
'@types/node' // no node.js
]
},
// Common: vs/platform/environment/common/*
{
target: '**/vs/platform/environment/common/*.ts',
allowedTypes: CORE_TYPES,
disallowedTypes: [ /* Ignore native types that are defined from here */],
disallowedDefinitions: [
'lib.dom.d.ts', // no DOM
'@types/node' // no node.js
]
},
// Common: vs/platform/window/common/window.ts
{
target: '**/vs/platform/window/common/window.ts',
allowedTypes: CORE_TYPES,
disallowedTypes: [ /* Ignore native types that are defined from here */],
disallowedDefinitions: [
'lib.dom.d.ts', // no DOM
'@types/node' // no node.js
]
},
// Common: vs/platform/native/common/native.ts
{
target: '**/vs/platform/native/common/native.ts',
allowedTypes: CORE_TYPES,
disallowedTypes: [ /* Ignore native types that are defined from here */],
disallowedDefinitions: [
'lib.dom.d.ts', // no DOM
'@types/node' // no node.js
]
},
// Common: vs/platform/native/common/nativeHostService.ts
{
target: '**/vs/platform/native/common/nativeHostService.ts',
allowedTypes: CORE_TYPES,
disallowedTypes: [ /* Ignore native types that are defined from here */],
disallowedDefinitions: [
'lib.dom.d.ts', // no DOM
'@types/node' // no node.js
]
},
// Common: vs/workbench/api/common/extHostExtensionService.ts
{
target: '**/vs/workbench/api/common/extHostExtensionService.ts',
allowedTypes: [
...CORE_TYPES,
// Safe access to global
'global'
],
disallowedTypes: NATIVE_TYPES,
disallowedDefinitions: [
'lib.dom.d.ts', // no DOM
'@types/node' // no node.js
]
},
// Common: vs/base/parts/sandbox/electron-sandbox/preload.ts
{
target: '**/vs/base/parts/sandbox/electron-sandbox/preload.ts',
allowedTypes: [
...CORE_TYPES,
// Safe access to a very small subset of node.js
'process',
'NodeJS'
],
disallowedTypes: NATIVE_TYPES,
disallowedDefinitions: [
'@types/node' // no node.js
]
},
// Common
{
target: '**/vs/**/common/**',
allowedTypes: CORE_TYPES,
disallowedTypes: NATIVE_TYPES,
disallowedDefinitions: [
'lib.dom.d.ts', // no DOM
'@types/node' // no node.js
]
},
// Browser
{
target: '**/vs/**/browser/**',
allowedTypes: CORE_TYPES,
disallowedTypes: NATIVE_TYPES,
allowedDefinitions: [
'@types/node/stream/consumers.d.ts' // node.js started to duplicate types from lib.dom.d.ts so we have to account for that
],
disallowedDefinitions: [
'@types/node' // no node.js
]
},
// Browser (editor contrib)
{
target: '**/src/vs/editor/contrib/**',
allowedTypes: CORE_TYPES,
disallowedTypes: NATIVE_TYPES,
disallowedDefinitions: [
'@types/node' // no node.js
]
},
// node.js
{
target: '**/vs/**/node/**',
allowedTypes: CORE_TYPES,
disallowedDefinitions: [
'lib.dom.d.ts' // no DOM
]
},
// Electron (sandbox)
{
target: '**/vs/**/electron-sandbox/**',
allowedTypes: CORE_TYPES,
disallowedDefinitions: [
'@types/node' // no node.js
]
},
// Electron (utility)
{
target: '**/vs/**/electron-utility/**',
allowedTypes: [
...CORE_TYPES,
// --> types from electron.d.ts that duplicate from lib.dom.d.ts
'Event',
'Request'
],
disallowedTypes: [
'ipcMain' // not allowed, use validatedIpcMain instead
],
disallowedDefinitions: [
'lib.dom.d.ts' // no DOM
]
},
// Electron (main)
{
target: '**/vs/**/electron-main/**',
allowedTypes: [
...CORE_TYPES,
// --> types from electron.d.ts that duplicate from lib.dom.d.ts
'Event',
'Request'
],
disallowedTypes: [
'ipcMain' // not allowed, use validatedIpcMain instead
],
disallowedDefinitions: [
'lib.dom.d.ts' // no DOM
]
}
];
const TS_CONFIG_PATH = (0, path_1.join)(__dirname, '../../', 'src', 'tsconfig.json');
let hasErrors = false;
function checkFile(program, sourceFile, rule) {
checkNode(sourceFile);
function checkNode(node) {
if (node.kind !== typescript_1.default.SyntaxKind.Identifier) {
return typescript_1.default.forEachChild(node, checkNode); // recurse down
}
const checker = program.getTypeChecker();
const symbol = checker.getSymbolAtLocation(node);
if (!symbol) {
return;
}
let _parentSymbol = symbol;
while (_parentSymbol.parent) {
_parentSymbol = _parentSymbol.parent;
}
const parentSymbol = _parentSymbol;
const text = parentSymbol.getName();
if (rule.allowedTypes?.some(allowed => allowed === text)) {
return; // override
}
if (rule.disallowedTypes?.some(disallowed => disallowed === text)) {
const { line, character } = sourceFile.getLineAndCharacterOfPosition(node.getStart());
console.log(`[build/lib/layersChecker.ts]: Reference to type '${text}' violates layer '${rule.target}' (${sourceFile.fileName} (${line + 1},${character + 1}). Learn more about our source code organization at https://github.com/microsoft/vscode/wiki/Source-Code-Organization.`);
hasErrors = true;
return;
}
const declarations = symbol.declarations;
if (Array.isArray(declarations)) {
DeclarationLoop: for (const declaration of declarations) {
if (declaration) {
const parent = declaration.parent;
if (parent) {
const parentSourceFile = parent.getSourceFile();
if (parentSourceFile) {
const definitionFileName = parentSourceFile.fileName;
if (rule.allowedDefinitions) {
for (const allowedDefinition of rule.allowedDefinitions) {
if (definitionFileName.indexOf(allowedDefinition) >= 0) {
continue DeclarationLoop;
}
}
}
if (rule.disallowedDefinitions) {
for (const disallowedDefinition of rule.disallowedDefinitions) {
if (definitionFileName.indexOf(disallowedDefinition) >= 0) {
const { line, character } = sourceFile.getLineAndCharacterOfPosition(node.getStart());
console.log(`[build/lib/layersChecker.ts]: Reference to symbol '${text}' from '${disallowedDefinition}' violates layer '${rule.target}' (${sourceFile.fileName} (${line + 1},${character + 1}) Learn more about our source code organization at https://github.com/microsoft/vscode/wiki/Source-Code-Organization.`);
hasErrors = true;
return;
}
}
}
}
}
}
}
}
}
}
function createProgram(tsconfigPath) {
const tsConfig = typescript_1.default.readConfigFile(tsconfigPath, typescript_1.default.sys.readFile);
const configHostParser = { fileExists: fs_1.existsSync, readDirectory: typescript_1.default.sys.readDirectory, readFile: file => (0, fs_1.readFileSync)(file, 'utf8'), useCaseSensitiveFileNames: process.platform === 'linux' };
const tsConfigParsed = typescript_1.default.parseJsonConfigFileContent(tsConfig.config, configHostParser, (0, path_1.resolve)((0, path_1.dirname)(tsconfigPath)), { noEmit: true });
const compilerHost = typescript_1.default.createCompilerHost(tsConfigParsed.options, true);
return typescript_1.default.createProgram(tsConfigParsed.fileNames, tsConfigParsed.options, compilerHost);
}
//
// Create program and start checking
//
const program = createProgram(TS_CONFIG_PATH);
for (const sourceFile of program.getSourceFiles()) {
for (const rule of RULES) {
if ((0, minimatch_1.match)([sourceFile.fileName], rule.target).length > 0) {
if (!rule.skip) {
checkFile(program, sourceFile, rule);
}
break;
}
}
}
if (hasErrors) {
process.exit(1);
}
//# sourceMappingURL=layersChecker.js.map
================================================
FILE: build/lib/layersChecker.ts
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
import ts from 'typescript';
import { readFileSync, existsSync } from 'fs';
import { resolve, dirname, join } from 'path';
import { match } from 'minimatch';
//
// #############################################################################################
//
// A custom typescript checker for the specific task of detecting the use of certain types in a
// layer that does not allow such use. For example:
// - using DOM globals in common/node/electron-main layer (e.g. HTMLElement)
// - using node.js globals in common/browser layer (e.g. process)
//
// Make changes to below RULES to lift certain files from these checks only if absolutely needed
//
// #############################################################################################
//
// Types we assume are present in all implementations of JS VMs (node.js, browsers)
// Feel free to add more core types as you see needed if present in node.js and browsers
const CORE_TYPES = [
'setTimeout',
'clearTimeout',
'setInterval',
'clearInterval',
'console',
'Console',
'Error',
'ErrorConstructor',
'String',
'TextDecoder',
'TextEncoder',
'self',
'queueMicrotask',
'Array',
'Uint8Array',
'Uint16Array',
'Uint32Array',
'Int8Array',
'Int16Array',
'Int32Array',
'Float32Array',
'Float64Array',
'Uint8ClampedArray',
'BigUint64Array',
'BigInt64Array',
'btoa',
'atob',
'AbortController',
'AbortSignal',
'MessageChannel',
'MessagePort',
'URL',
'URLSearchParams',
'ReadonlyArray',
'Event',
'EventTarget',
'BroadcastChannel',
'performance',
'Blob',
'crypto',
'File',
'fetch',
'RequestInit',
'Headers',
'Request',
'Response',
'Body',
'__type',
'__global',
'Performance',
'PerformanceMark',
'PerformanceObserver',
'ImportMeta',
'structuredClone',
// webcrypto has been available since Node.js 19, but still live in dom.d.ts
'Crypto',
'SubtleCrypto',
'JsonWebKey',
'MessageEvent',
];
// Types that are defined in a common layer but are known to be only
// available in native environments should not be allowed in browser
const NATIVE_TYPES = [
'NativeParsedArgs',
'INativeEnvironmentService',
'AbstractNativeEnvironmentService',
'INativeWindowConfiguration',
'ICommonNativeHostService',
'INativeHostService',
'IMainProcessService'
];
const RULES: IRule[] = [
// Tests: skip
{
target: '**/vs/**/test/**',
skip: true // -> skip all test files
},
// Common: vs/base/common/platform.ts
{
target: '**/vs/base/common/platform.ts',
allowedTypes: [
...CORE_TYPES,
// Safe access to postMessage() and friends
'MessageEvent',
],
disallowedTypes: NATIVE_TYPES,
disallowedDefinitions: [
'lib.dom.d.ts', // no DOM
'@types/node' // no node.js
]
},
// Common: vs/base/common/async.ts
{
target: '**/vs/base/common/async.ts',
allowedTypes: [
...CORE_TYPES,
// Safe access to requestIdleCallback & cancelIdleCallback
'requestIdleCallback',
'cancelIdleCallback'
],
disallowedTypes: NATIVE_TYPES,
disallowedDefinitions: [
'lib.dom.d.ts', // no DOM
'@types/node' // no node.js
]
},
// Common: vs/base/common/performance.ts
{
target: '**/vs/base/common/performance.ts',
allowedTypes: [
...CORE_TYPES,
// Safe access to Performance
'Performance',
'PerformanceEntry',
'PerformanceTiming'
],
disallowedTypes: NATIVE_TYPES,
disallowedDefinitions: [
'lib.dom.d.ts', // no DOM
'@types/node' // no node.js
]
},
// Common: vs/platform/environment/common/*
{
target: '**/vs/platform/environment/common/*.ts',
allowedTypes: CORE_TYPES,
disallowedTypes: [/* Ignore native types that are defined from here */],
disallowedDefinitions: [
'lib.dom.d.ts', // no DOM
'@types/node' // no node.js
]
},
// Common: vs/platform/window/common/window.ts
{
target: '**/vs/platform/window/common/window.ts',
allowedTypes: CORE_TYPES,
disallowedTypes: [/* Ignore native types that are defined from here */],
disallowedDefinitions: [
'lib.dom.d.ts', // no DOM
'@types/node' // no node.js
]
},
// Common: vs/platform/native/common/native.ts
{
target: '**/vs/platform/native/common/native.ts',
allowedTypes: CORE_TYPES,
disallowedTypes: [/* Ignore native types that are defined from here */],
disallowedDefinitions: [
'lib.dom.d.ts', // no DOM
'@types/node' // no node.js
]
},
// Common: vs/platform/native/common/nativeHostService.ts
{
target: '**/vs/platform/native/common/nativeHostService.ts',
allowedTypes: CORE_TYPES,
disallowedTypes: [/* Ignore native types that are defined from here */],
disallowedDefinitions: [
'lib.dom.d.ts', // no DOM
'@types/node' // no node.js
]
},
// Common: vs/workbench/api/common/extHostExtensionService.ts
{
target: '**/vs/workbench/api/common/extHostExtensionService.ts',
allowedTypes: [
...CORE_TYPES,
// Safe access to global
'global'
],
disallowedTypes: NATIVE_TYPES,
disallowedDefinitions: [
'lib.dom.d.ts', // no DOM
'@types/node' // no node.js
]
},
// Common: vs/base/parts/sandbox/electron-sandbox/preload.ts
{
target: '**/vs/base/parts/sandbox/electron-sandbox/preload.ts',
allowedTypes: [
...CORE_TYPES,
// Safe access to a very small subset of node.js
'process',
'NodeJS'
],
disallowedTypes: NATIVE_TYPES,
disallowedDefinitions: [
'@types/node' // no node.js
]
},
// Common
{
target: '**/vs/**/common/**',
allowedTypes: CORE_TYPES,
disallowedTypes: NATIVE_TYPES,
disallowedDefinitions: [
'lib.dom.d.ts', // no DOM
'@types/node' // no node.js
]
},
// Browser
{
target: '**/vs/**/browser/**',
allowedTypes: CORE_TYPES,
disallowedTypes: NATIVE_TYPES,
allowedDefinitions: [
'@types/node/stream/consumers.d.ts' // node.js started to duplicate types from lib.dom.d.ts so we have to account for that
],
disallowedDefinitions: [
'@types/node' // no node.js
]
},
// Browser (editor contrib)
{
target: '**/src/vs/editor/contrib/**',
allowedTypes: CORE_TYPES,
disallowedTypes: NATIVE_TYPES,
disallowedDefinitions: [
'@types/node' // no node.js
]
},
// node.js
{
target: '**/vs/**/node/**',
allowedTypes: CORE_TYPES,
disallowedDefinitions: [
'lib.dom.d.ts' // no DOM
]
},
// Electron (sandbox)
{
target: '**/vs/**/electron-sandbox/**',
allowedTypes: CORE_TYPES,
disallowedDefinitions: [
'@types/node' // no node.js
]
},
// Electron (utility)
{
target: '**/vs/**/electron-utility/**',
allowedTypes: [
...CORE_TYPES,
// --> types from electron.d.ts that duplicate from lib.dom.d.ts
'Event',
'Request'
],
disallowedTypes: [
'ipcMain' // not allowed, use validatedIpcMain instead
],
disallowedDefinitions: [
'lib.dom.d.ts' // no DOM
]
},
// Electron (main)
{
target: '**/vs/**/electron-main/**',
allowedTypes: [
...CORE_TYPES,
// --> types from electron.d.ts that duplicate from lib.dom.d.ts
'Event',
'Request'
],
disallowedTypes: [
'ipcMain' // not allowed, use validatedIpcMain instead
],
disallowedDefinitions: [
'lib.dom.d.ts' // no DOM
]
}
];
const TS_CONFIG_PATH = join(__dirname, '../../', 'src', 'tsconfig.json');
interface IRule {
target: string;
skip?: boolean;
allowedTypes?: string[];
allowedDefinitions?: string[];
disallowedDefinitions?: string[];
disallowedTypes?: string[];
}
let hasErrors = false;
function checkFile(program: ts.Program, sourceFile: ts.SourceFile, rule: IRule) {
checkNode(sourceFile);
function checkNode(node: ts.Node): void {
if (node.kind !== ts.SyntaxKind.Identifier) {
return ts.forEachChild(node, checkNode); // recurse down
}
const checker = program.getTypeChecker();
const symbol = checker.getSymbolAtLocation(node);
if (!symbol) {
return;
}
let _parentSymbol: any = symbol;
while (_parentSymbol.parent) {
_parentSymbol = _parentSymbol.parent;
}
const parentSymbol = _parentSymbol as ts.Symbol;
const text = parentSymbol.getName();
if (rule.allowedTypes?.some(allowed => allowed === text)) {
return; // override
}
if (rule.disallowedTypes?.some(disallowed => disallowed === text)) {
const { line, character } = sourceFile.getLineAndCharacterOfPosition(node.getStart());
console.log(`[build/lib/layersChecker.ts]: Reference to type '${text}' violates layer '${rule.target}' (${sourceFile.fileName} (${line + 1},${character + 1}). Learn more about our source code organization at https://github.com/microsoft/vscode/wiki/Source-Code-Organization.`);
hasErrors = true;
return;
}
const declarations = symbol.declarations;
if (Array.isArray(declarations)) {
DeclarationLoop: for (const declaration of declarations) {
if (declaration) {
const parent = declaration.parent;
if (parent) {
const parentSourceFile = parent.getSourceFile();
if (parentSourceFile) {
const definitionFileName = parentSourceFile.fileName;
if (rule.allowedDefinitions) {
for (const allowedDefinition of rule.allowedDefinitions) {
if (definitionFileName.indexOf(allowedDefinition) >= 0) {
continue DeclarationLoop;
}
}
}
if (rule.disallowedDefinitions) {
for (const disallowedDefinition of rule.disallowedDefinitions) {
if (definitionFileName.indexOf(disallowedDefinition) >= 0) {
const { line, character } = sourceFile.getLineAndCharacterOfPosition(node.getStart());
console.log(`[build/lib/layersChecker.ts]: Reference to symbol '${text}' from '${disallowedDefinition}' violates layer '${rule.target}' (${sourceFile.fileName} (${line + 1},${character + 1}) Learn more about our source code organization at https://github.com/microsoft/vscode/wiki/Source-Code-Organization.`);
hasErrors = true;
return;
}
}
}
}
}
}
}
}
}
}
function createProgram(tsconfigPath: string): ts.Program {
const tsConfig = ts.readConfigFile(tsconfigPath, ts.sys.readFile);
const configHostParser: ts.ParseConfigHost = { fileExists: existsSync, readDirectory: ts.sys.readDirectory, readFile: file => readFileSync(file, 'utf8'), useCaseSensitiveFileNames: process.platform === 'linux' };
const tsConfigParsed = ts.parseJsonConfigFileContent(tsConfig.config, configHostParser, resolve(dirname(tsconfigPath)), { noEmit: true });
const compilerHost = ts.createCompilerHost(tsConfigParsed.options, true);
return ts.createProgram(tsConfigParsed.fileNames, tsConfigParsed.options, compilerHost);
}
//
// Create program and start checking
//
const program = createProgram(TS_CONFIG_PATH);
for (const sourceFile of program.getSourceFiles()) {
for (const rule of RULES) {
if (match([sourceFile.fileName], rule.target).length > 0) {
if (!rule.skip) {
checkFile(program, sourceFile, rule);
}
break;
}
}
}
if (hasErrors) {
process.exit(1);
}
================================================
FILE: build/lib/mangle/index.js
================================================
"use strict";
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
Object.defineProperty(exports, "__esModule", { value: true });
exports.Mangler = void 0;
const node_v8_1 = __importDefault(require("node:v8"));
const fs_1 = __importDefault(require("fs"));
const path_1 = __importDefault(require("path"));
const process_1 = require("process");
const source_map_1 = require("source-map");
const typescript_1 = __importDefault(require("typescript"));
const url_1 = require("url");
const workerpool_1 = __importDefault(require("workerpool"));
const staticLanguageServiceHost_1 = require("./staticLanguageServiceHost");
const buildfile = require('../../buildfile');
class ShortIdent {
prefix;
static _keywords = new Set(['await', 'break', 'case', 'catch', 'class', 'const', 'continue', 'debugger',
'default', 'delete', 'do', 'else', 'export', 'extends', 'false', 'finally', 'for', 'function', 'if',
'import', 'in', 'instanceof', 'let', 'new', 'null', 'return', 'static', 'super', 'switch', 'this', 'throw',
'true', 'try', 'typeof', 'var', 'void', 'while', 'with', 'yield']);
static _alphabet = 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890$_'.split('');
_value = 0;
constructor(prefix) {
this.prefix = prefix;
}
next(isNameTaken) {
const candidate = this.prefix + ShortIdent.convert(this._value);
this._value++;
if (ShortIdent._keywords.has(candidate) || /^[_0-9]/.test(candidate) || isNameTaken?.(candidate)) {
// try again
return this.next(isNameTaken);
}
return candidate;
}
static convert(n) {
const base = this._alphabet.length;
let result = '';
do {
const rest = n % base;
result += this._alphabet[rest];
n = (n / base) | 0;
} while (n > 0);
return result;
}
}
var FieldType;
(function (FieldType) {
FieldType[FieldType["Public"] = 0] = "Public";
FieldType[FieldType["Protected"] = 1] = "Protected";
FieldType[FieldType["Private"] = 2] = "Private";
})(FieldType || (FieldType = {}));
class ClassData {
fileName;
node;
fields = new Map();
replacements;
parent;
children;
constructor(fileName, node) {
// analyse all fields (properties and methods). Find usages of all protected and
// private ones and keep track of all public ones (to prevent naming collisions)
this.fileName = fileName;
this.node = node;
const candidates = [];
for (const member of node.members) {
if (typescript_1.default.isMethodDeclaration(member)) {
// method `foo() {}`
candidates.push(member);
}
else if (typescript_1.default.isPropertyDeclaration(member)) {
// property `foo = 234`
candidates.push(member);
}
else if (typescript_1.default.isGetAccessor(member)) {
// getter: `get foo() { ... }`
candidates.push(member);
}
else if (typescript_1.default.isSetAccessor(member)) {
// setter: `set foo() { ... }`
candidates.push(member);
}
else if (typescript_1.default.isConstructorDeclaration(member)) {
// constructor-prop:`constructor(private foo) {}`
for (const param of member.parameters) {
if (hasModifier(param, typescript_1.default.SyntaxKind.PrivateKeyword)
|| hasModifier(param, typescript_1.default.SyntaxKind.ProtectedKeyword)
|| hasModifier(param, typescript_1.default.SyntaxKind.PublicKeyword)
|| hasModifier(param, typescript_1.default.SyntaxKind.ReadonlyKeyword)) {
candidates.push(param);
}
}
}
}
for (const member of candidates) {
const ident = ClassData._getMemberName(member);
if (!ident) {
continue;
}
const type = ClassData._getFieldType(member);
this.fields.set(ident, { type, pos: member.name.getStart() });
}
}
static _getMemberName(node) {
if (!node.name) {
return undefined;
}
const { name } = node;
let ident = name.getText();
if (name.kind === typescript_1.default.SyntaxKind.ComputedPropertyName) {
if (name.expression.kind !== typescript_1.default.SyntaxKind.StringLiteral) {
// unsupported: [Symbol.foo] or [abc + 'field']
return;
}
// ['foo']
ident = name.expression.getText().slice(1, -1);
}
return ident;
}
static _getFieldType(node) {
if (hasModifier(node, typescript_1.default.SyntaxKind.PrivateKeyword)) {
return 2 /* FieldType.Private */;
}
else if (hasModifier(node, typescript_1.default.SyntaxKind.ProtectedKeyword)) {
return 1 /* FieldType.Protected */;
}
else {
return 0 /* FieldType.Public */;
}
}
static _shouldMangle(type) {
return type === 2 /* FieldType.Private */
|| type === 1 /* FieldType.Protected */;
}
static makeImplicitPublicActuallyPublic(data, reportViolation) {
// TS-HACK
// A subtype can make an inherited protected field public. To prevent accidential
// mangling of public fields we mark the original (protected) fields as public...
for (const [name, info] of data.fields) {
if (info.type !== 0 /* FieldType.Public */) {
continue;
}
let parent = data.parent;
while (parent) {
if (parent.fields.get(name)?.type === 1 /* FieldType.Protected */) {
const parentPos = parent.node.getSourceFile().getLineAndCharacterOfPosition(parent.fields.get(name).pos);
const infoPos = data.node.getSourceFile().getLineAndCharacterOfPosition(info.pos);
reportViolation(name, `'${name}' from ${parent.fileName}:${parentPos.line + 1}`, `${data.fileName}:${infoPos.line + 1}`);
parent.fields.get(name).type = 0 /* FieldType.Public */;
}
parent = parent.parent;
}
}
}
static fillInReplacement(data) {
if (data.replacements) {
// already done
return;
}
// fill in parents first
if (data.parent) {
ClassData.fillInReplacement(data.parent);
}
data.replacements = new Map();
const isNameTaken = (name) => {
// locally taken
if (data._isNameTaken(name)) {
return true;
}
// parents
let parent = data.parent;
while (parent) {
if (parent._isNameTaken(name)) {
return true;
}
parent = parent.parent;
}
// children
if (data.children) {
const stack = [...data.children];
while (stack.length) {
const node = stack.pop();
if (node._isNameTaken(name)) {
return true;
}
if (node.children) {
stack.push(...node.children);
}
}
}
return false;
};
const identPool = new ShortIdent('');
for (const [name, info] of data.fields) {
if (ClassData._shouldMangle(info.type)) {
const shortName = identPool.next(isNameTaken);
data.replacements.set(name, shortName);
}
}
}
// a name is taken when a field that doesn't get mangled exists or
// when the name is already in use for replacement
_isNameTaken(name) {
if (this.fields.has(name) && !ClassData._shouldMangle(this.fields.get(name).type)) {
// public field
return true;
}
if (this.replacements) {
for (const shortName of this.replacements.values()) {
if (shortName === name) {
// replaced already (happens wih super types)
return true;
}
}
}
if (isNameTakenInFile(this.node, name)) {
return true;
}
return false;
}
lookupShortName(name) {
let value = this.replacements.get(name);
let parent = this.parent;
while (parent) {
if (parent.replacements.has(name) && parent.fields.get(name)?.type === 1 /* FieldType.Protected */) {
value = parent.replacements.get(name) ?? value;
}
parent = parent.parent;
}
return value;
}
// --- parent chaining
addChild(child) {
this.children ??= [];
this.children.push(child);
child.parent = this;
}
}
function isNameTakenInFile(node, name) {
const identifiers = node.getSourceFile().identifiers;
if (identifiers instanceof Map) {
if (identifiers.has(name)) {
return true;
}
}
return false;
}
const skippedExportMangledFiles = [
// Build
'css.build',
// Monaco
'editorCommon',
'editorOptions',
'editorZoom',
'standaloneEditor',
'standaloneEnums',
'standaloneLanguages',
// Generated
'extensionsApiProposals',
// Module passed around as type
'pfs',
// entry points
...[
buildfile.workerEditor,
buildfile.workerExtensionHost,
buildfile.workerNotebook,
buildfile.workerLanguageDetection,
buildfile.workerLocalFileSearch,
buildfile.workerProfileAnalysis,
buildfile.workerOutputLinks,
buildfile.workerBackgroundTokenization,
buildfile.workbenchDesktop,
buildfile.workbenchWeb,
buildfile.code,
buildfile.codeWeb
].flat().map(x => x.name),
];
const skippedExportMangledProjects = [
// Test projects
'vscode-api-tests',
// These projects use webpack to dynamically rewrite imports, which messes up our mangling
'configuration-editing',
'microsoft-authentication',
'github-authentication',
'html-language-features/server',
];
const skippedExportMangledSymbols = [
// Don't mangle extension entry points
'activate',
'deactivate',
];
class DeclarationData {
fileName;
node;
replacementName;
constructor(fileName, node, fileIdents) {
this.fileName = fileName;
this.node = node;
// Todo: generate replacement names based on usage count, with more used names getting shorter identifiers
this.replacementName = fileIdents.next();
}
getLocations(service) {
if (typescript_1.default.isVariableDeclaration(this.node)) {
// If the const aliases any types, we need to rename those too
const definitionResult = service.getDefinitionAndBoundSpan(this.fileName, this.node.name.getStart());
if (definitionResult?.definitions && definitionResult.definitions.length > 1) {
return definitionResult.definitions.map(x => ({ fileName: x.fileName, offset: x.textSpan.start }));
}
}
return [{
fileName: this.fileName,
offset: this.node.name.getStart()
}];
}
shouldMangle(newName) {
const currentName = this.node.name.getText();
if (currentName.startsWith('$') || skippedExportMangledSymbols.includes(currentName)) {
return false;
}
// New name is longer the existing one :'(
if (newName.length >= currentName.length) {
return false;
}
// Don't mangle functions we've explicitly opted out
if (this.node.getFullText().includes('@skipMangle')) {
return false;
}
return true;
}
}
/**
* TypeScript2TypeScript transformer that mangles all private and protected fields
*
* 1. Collect all class fields (properties, methods)
* 2. Collect all sub and super-type relations between classes
* 3. Compute replacement names for each field
* 4. Lookup rename locations for these fields
* 5. Prepare and apply edits
*/
class Mangler {
projectPath;
log;
config;
allClassDataByKey = new Map();
allExportedSymbols = new Set();
renameWorkerPool;
constructor(projectPath, log = () => { }, config) {
this.projectPath = projectPath;
this.log = log;
this.config = config;
this.renameWorkerPool = workerpool_1.default.pool(path_1.default.join(__dirname, 'renameWorker.js'), {
maxWorkers: 4,
minWorkers: 'max'
});
}
async computeNewFileContents(strictImplicitPublicHandling) {
const service = typescript_1.default.createLanguageService(new staticLanguageServiceHost_1.StaticLanguageServiceHost(this.projectPath));
// STEP:
// - Find all classes and their field info.
// - Find exported symbols.
const fileIdents = new ShortIdent('$');
const visit = (node) => {
if (this.config.manglePrivateFields) {
if (typescript_1.default.isClassDeclaration(node) || typescript_1.default.isClassExpression(node)) {
const anchor = node.name ?? node;
const key = `${node.getSourceFile().fileName}|${anchor.getStart()}`;
if (this.allClassDataByKey.has(key)) {
throw new Error('DUPE?');
}
this.allClassDataByKey.set(key, new ClassData(node.getSourceFile().fileName, node));
}
}
if (this.config.mangleExports) {
// Find exported classes, functions, and vars
if ((
// Exported class
typescript_1.default.isClassDeclaration(node)
&& hasModifier(node, typescript_1.default.SyntaxKind.ExportKeyword)
&& node.name) || (
// Exported function
typescript_1.default.isFunctionDeclaration(node)
&& typescript_1.default.isSourceFile(node.parent)
&& hasModifier(node, typescript_1.default.SyntaxKind.ExportKeyword)
&& node.name && node.body // On named function and not on the overload
) || (
// Exported variable
typescript_1.default.isVariableDeclaration(node)
&& hasModifier(node.parent.parent, typescript_1.default.SyntaxKind.ExportKeyword) // Variable statement is exported
&& typescript_1.default.isSourceFile(node.parent.parent.parent))
// Disabled for now because we need to figure out how to handle
// enums that are used in monaco or extHost interfaces.
/* || (
// Exported enum
ts.isEnumDeclaration(node)
&& ts.isSourceFile(node.parent)
&& hasModifier(node, ts.SyntaxKind.ExportKeyword)
&& !hasModifier(node, ts.SyntaxKind.ConstKeyword) // Don't bother mangling const enums because these are inlined
&& node.name
*/
) {
if (isInAmbientContext(node)) {
return;
}
this.allExportedSymbols.add(new DeclarationData(node.getSourceFile().fileName, node, fileIdents));
}
}
typescript_1.default.forEachChild(node, visit);
};
for (const file of service.getProgram().getSourceFiles()) {
if (!file.isDeclarationFile) {
typescript_1.default.forEachChild(file, visit);
}
}
this.log(`Done collecting. Classes: ${this.allClassDataByKey.size}. Exported symbols: ${this.allExportedSymbols.size}`);
// STEP: connect sub and super-types
const setupParents = (data) => {
const extendsClause = data.node.heritageClauses?.find(h => h.token === typescript_1.default.SyntaxKind.ExtendsKeyword);
if (!extendsClause) {
// no EXTENDS-clause
return;
}
const info = service.getDefinitionAtPosition(data.fileName, extendsClause.types[0].expression.getEnd());
if (!info || info.length === 0) {
// throw new Error('SUPER type not found');
return;
}
if (info.length !== 1) {
// inherits from declared/library type
return;
}
const [definition] = info;
const key = `${definition.fileName}|${definition.textSpan.start}`;
const parent = this.allClassDataByKey.get(key);
if (!parent) {
// throw new Error(`SUPER type not found: ${key}`);
return;
}
parent.addChild(data);
};
for (const data of this.allClassDataByKey.values()) {
setupParents(data);
}
// STEP: make implicit public (actually protected) field really public
const violations = new Map();
let violationsCauseFailure = false;
for (const data of this.allClassDataByKey.values()) {
ClassData.makeImplicitPublicActuallyPublic(data, (name, what, why) => {
const arr = violations.get(what);
if (arr) {
arr.push(why);
}
else {
violations.set(what, [why]);
}
if (strictImplicitPublicHandling && !strictImplicitPublicHandling.has(name)) {
violationsCauseFailure = true;
}
});
}
for (const [why, whys] of violations) {
this.log(`WARN: ${why} became PUBLIC because of: ${whys.join(' , ')}`);
}
if (violationsCauseFailure) {
const message = 'Protected fields have been made PUBLIC. This hurts minification and is therefore not allowed. Review the WARN messages further above';
this.log(`ERROR: ${message}`);
throw new Error(message);
}
// STEP: compute replacement names for each class
for (const data of this.allClassDataByKey.values()) {
ClassData.fillInReplacement(data);
}
this.log(`Done creating class replacements`);
// STEP: prepare rename edits
this.log(`Starting prepare rename edits`);
const editsByFile = new Map();
const appendEdit = (fileName, edit) => {
const edits = editsByFile.get(fileName);
if (!edits) {
editsByFile.set(fileName, [edit]);
}
else {
edits.push(edit);
}
};
const appendRename = (newText, loc) => {
appendEdit(loc.fileName, {
newText: (loc.prefixText || '') + newText + (loc.suffixText || ''),
offset: loc.textSpan.start,
length: loc.textSpan.length
});
};
const renameResults = [];
const queueRename = (fileName, pos, newName) => {
renameResults.push(Promise.resolve(this.renameWorkerPool.exec('findRenameLocations', [this.projectPath, fileName, pos]))
.then((locations) => ({ newName, locations })));
};
for (const data of this.allClassDataByKey.values()) {
if (hasModifier(data.node, typescript_1.default.SyntaxKind.DeclareKeyword)) {
continue;
}
fields: for (const [name, info] of data.fields) {
if (!ClassData._shouldMangle(info.type)) {
continue fields;
}
// TS-HACK: protected became public via 'some' child
// and because of that we might need to ignore this now
let parent = data.parent;
while (parent) {
if (parent.fields.get(name)?.type === 0 /* FieldType.Public */) {
continue fields;
}
parent = parent.parent;
}
const newName = data.lookupShortName(name);
queueRename(data.fileName, info.pos, newName);
}
}
for (const data of this.allExportedSymbols.values()) {
if (data.fileName.endsWith('.d.ts')
|| skippedExportMangledProjects.some(proj => data.fileName.includes(proj))
|| skippedExportMangledFiles.some(file => data.fileName.endsWith(file + '.ts'))) {
continue;
}
if (!data.shouldMangle(data.replacementName)) {
continue;
}
const newText = data.replacementName;
for (const { fileName, offset } of data.getLocations(service)) {
queueRename(fileName, offset, newText);
}
}
await Promise.all(renameResults).then((result) => {
for (const { newName, locations } of result) {
for (const loc of locations) {
appendRename(newName, loc);
}
}
});
await this.renameWorkerPool.terminate();
this.log(`Done preparing edits: ${editsByFile.size} files`);
// STEP: apply all rename edits (per file)
const result = new Map();
let savedBytes = 0;
for (const item of service.getProgram().getSourceFiles()) {
const { mapRoot, sourceRoot } = service.getProgram().getCompilerOptions();
const projectDir = path_1.default.dirname(this.projectPath);
const sourceMapRoot = mapRoot ?? (0, url_1.pathToFileURL)(sourceRoot ?? projectDir).toString();
// source maps
let generator;
let newFullText;
const edits = editsByFile.get(item.fileName);
if (!edits) {
// just copy
newFullText = item.getFullText();
}
else {
// source map generator
const relativeFileName = normalize(path_1.default.relative(projectDir, item.fileName));
const mappingsByLine = new Map();
// apply renames
edits.sort((a, b) => b.offset - a.offset);
const characters = item.getFullText().split('');
let lastEdit;
for (const edit of edits) {
if (lastEdit && lastEdit.offset === edit.offset) {
//
if (lastEdit.length !== edit.length || lastEdit.newText !== edit.newText) {
this.log('ERROR: Overlapping edit', item.fileName, edit.offset, edits);
throw new Error('OVERLAPPING edit');
}
else {
continue;
}
}
lastEdit = edit;
const mangledName = characters.splice(edit.offset, edit.length, edit.newText).join('');
savedBytes += mangledName.length - edit.newText.length;
// source maps
const pos = item.getLineAndCharacterOfPosition(edit.offset);
let mappings = mappingsByLine.get(pos.line);
if (!mappings) {
mappings = [];
mappingsByLine.set(pos.line, mappings);
}
mappings.unshift({
source: relativeFileName,
original: { line: pos.line + 1, column: pos.character },
generated: { line: pos.line + 1, column: pos.character },
name: mangledName
}, {
source: relativeFileName,
original: { line: pos.line + 1, column: pos.character + edit.length },
generated: { line: pos.line + 1, column: pos.character + edit.newText.length },
});
}
// source map generation, make sure to get mappings per line correct
generator = new source_map_1.SourceMapGenerator({ file: path_1.default.basename(item.fileName), sourceRoot: sourceMapRoot });
generator.setSourceContent(relativeFileName, item.getFullText());
for (const [, mappings] of mappingsByLine) {
let lineDelta = 0;
for (const mapping of mappings) {
generator.addMapping({
...mapping,
generated: { line: mapping.generated.line, column: mapping.generated.column - lineDelta }
});
lineDelta += mapping.original.column - mapping.generated.column;
}
}
newFullText = characters.join('');
}
result.set(item.fileName, { out: newFullText, sourceMap: generator?.toString() });
}
service.dispose();
this.renameWorkerPool.terminate();
this.log(`Done: ${savedBytes / 1000}kb saved, memory-usage: ${JSON.stringify(node_v8_1.default.getHeapStatistics())}`);
return result;
}
}
exports.Mangler = Mangler;
// --- ast utils
function hasModifier(node, kind) {
const modifiers = typescript_1.default.canHaveModifiers(node) ? typescript_1.default.getModifiers(node) : undefined;
return Boolean(modifiers?.find(mode => mode.kind === kind));
}
function isInAmbientContext(node) {
for (let p = node.parent; p; p = p.parent) {
if (typescript_1.default.isModuleDeclaration(p)) {
return true;
}
}
return false;
}
function normalize(path) {
return path.replace(/\\/g, '/');
}
async function _run() {
const root = path_1.default.join(__dirname, '..', '..', '..');
const projectBase = path_1.default.join(root, 'src');
const projectPath = path_1.default.join(projectBase, 'tsconfig.json');
const newProjectBase = path_1.default.join(path_1.default.dirname(projectBase), path_1.default.basename(projectBase) + '2');
fs_1.default.cpSync(projectBase, newProjectBase, { recursive: true });
const mangler = new Mangler(projectPath, console.log, {
mangleExports: true,
manglePrivateFields: true,
});
for (const [fileName, contents] of await mangler.computeNewFileContents(new Set(['saveState']))) {
const newFilePath = path_1.default.join(newProjectBase, path_1.default.relative(projectBase, fileName));
await fs_1.default.promises.mkdir(path_1.default.dirname(newFilePath), { recursive: true });
await fs_1.default.promises.writeFile(newFilePath, contents.out);
if (contents.sourceMap) {
await fs_1.default.promises.writeFile(newFilePath + '.map', contents.sourceMap);
}
}
}
if (__filename === process_1.argv[1]) {
_run();
}
//# sourceMappingURL=index.js.map
================================================
FILE: build/lib/mangle/index.ts
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
import v8 from 'node:v8';
import fs from 'fs';
import path from 'path';
import { argv } from 'process';
import { Mapping, SourceMapGenerator } from 'source-map';
import ts from 'typescript';
import { pathToFileURL } from 'url';
import workerpool from 'workerpool';
import { StaticLanguageServiceHost } from './staticLanguageServiceHost';
const buildfile = require('../../buildfile');
class ShortIdent {
private static _keywords = new Set(['await', 'break', 'case', 'catch', 'class', 'const', 'continue', 'debugger',
'default', 'delete', 'do', 'else', 'export', 'extends', 'false', 'finally', 'for', 'function', 'if',
'import', 'in', 'instanceof', 'let', 'new', 'null', 'return', 'static', 'super', 'switch', 'this', 'throw',
'true', 'try', 'typeof', 'var', 'void', 'while', 'with', 'yield']);
private static _alphabet = 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890$_'.split('');
private _value = 0;
constructor(
private readonly prefix: string
) { }
next(isNameTaken?: (name: string) => boolean): string {
const candidate = this.prefix + ShortIdent.convert(this._value);
this._value++;
if (ShortIdent._keywords.has(candidate) || /^[_0-9]/.test(candidate) || isNameTaken?.(candidate)) {
// try again
return this.next(isNameTaken);
}
return candidate;
}
private static convert(n: number): string {
const base = this._alphabet.length;
let result = '';
do {
const rest = n % base;
result += this._alphabet[rest];
n = (n / base) | 0;
} while (n > 0);
return result;
}
}
const enum FieldType {
Public,
Protected,
Private
}
class ClassData {
fields = new Map();
private replacements: Map | undefined;
parent: ClassData | undefined;
children: ClassData[] | undefined;
constructor(
readonly fileName: string,
readonly node: ts.ClassDeclaration | ts.ClassExpression,
) {
// analyse all fields (properties and methods). Find usages of all protected and
// private ones and keep track of all public ones (to prevent naming collisions)
const candidates: (ts.NamedDeclaration)[] = [];
for (const member of node.members) {
if (ts.isMethodDeclaration(member)) {
// method `foo() {}`
candidates.push(member);
} else if (ts.isPropertyDeclaration(member)) {
// property `foo = 234`
candidates.push(member);
} else if (ts.isGetAccessor(member)) {
// getter: `get foo() { ... }`
candidates.push(member);
} else if (ts.isSetAccessor(member)) {
// setter: `set foo() { ... }`
candidates.push(member);
} else if (ts.isConstructorDeclaration(member)) {
// constructor-prop:`constructor(private foo) {}`
for (const param of member.parameters) {
if (hasModifier(param, ts.SyntaxKind.PrivateKeyword)
|| hasModifier(param, ts.SyntaxKind.ProtectedKeyword)
|| hasModifier(param, ts.SyntaxKind.PublicKeyword)
|| hasModifier(param, ts.SyntaxKind.ReadonlyKeyword)
) {
candidates.push(param);
}
}
}
}
for (const member of candidates) {
const ident = ClassData._getMemberName(member);
if (!ident) {
continue;
}
const type = ClassData._getFieldType(member);
this.fields.set(ident, { type, pos: member.name!.getStart() });
}
}
private static _getMemberName(node: ts.NamedDeclaration): string | undefined {
if (!node.name) {
return undefined;
}
const { name } = node;
let ident = name.getText();
if (name.kind === ts.SyntaxKind.ComputedPropertyName) {
if (name.expression.kind !== ts.SyntaxKind.StringLiteral) {
// unsupported: [Symbol.foo] or [abc + 'field']
return;
}
// ['foo']
ident = name.expression.getText().slice(1, -1);
}
return ident;
}
private static _getFieldType(node: ts.Node): FieldType {
if (hasModifier(node, ts.SyntaxKind.PrivateKeyword)) {
return FieldType.Private;
} else if (hasModifier(node, ts.SyntaxKind.ProtectedKeyword)) {
return FieldType.Protected;
} else {
return FieldType.Public;
}
}
static _shouldMangle(type: FieldType): boolean {
return type === FieldType.Private
|| type === FieldType.Protected
;
}
static makeImplicitPublicActuallyPublic(data: ClassData, reportViolation: (name: string, what: string, why: string) => void): void {
// TS-HACK
// A subtype can make an inherited protected field public. To prevent accidential
// mangling of public fields we mark the original (protected) fields as public...
for (const [name, info] of data.fields) {
if (info.type !== FieldType.Public) {
continue;
}
let parent: ClassData | undefined = data.parent;
while (parent) {
if (parent.fields.get(name)?.type === FieldType.Protected) {
const parentPos = parent.node.getSourceFile().getLineAndCharacterOfPosition(parent.fields.get(name)!.pos);
const infoPos = data.node.getSourceFile().getLineAndCharacterOfPosition(info.pos);
reportViolation(name, `'${name}' from ${parent.fileName}:${parentPos.line + 1}`, `${data.fileName}:${infoPos.line + 1}`);
parent.fields.get(name)!.type = FieldType.Public;
}
parent = parent.parent;
}
}
}
static fillInReplacement(data: ClassData) {
if (data.replacements) {
// already done
return;
}
// fill in parents first
if (data.parent) {
ClassData.fillInReplacement(data.parent);
}
data.replacements = new Map();
const isNameTaken = (name: string) => {
// locally taken
if (data._isNameTaken(name)) {
return true;
}
// parents
let parent: ClassData | undefined = data.parent;
while (parent) {
if (parent._isNameTaken(name)) {
return true;
}
parent = parent.parent;
}
// children
if (data.children) {
const stack = [...data.children];
while (stack.length) {
const node = stack.pop()!;
if (node._isNameTaken(name)) {
return true;
}
if (node.children) {
stack.push(...node.children);
}
}
}
return false;
};
const identPool = new ShortIdent('');
for (const [name, info] of data.fields) {
if (ClassData._shouldMangle(info.type)) {
const shortName = identPool.next(isNameTaken);
data.replacements.set(name, shortName);
}
}
}
// a name is taken when a field that doesn't get mangled exists or
// when the name is already in use for replacement
private _isNameTaken(name: string) {
if (this.fields.has(name) && !ClassData._shouldMangle(this.fields.get(name)!.type)) {
// public field
return true;
}
if (this.replacements) {
for (const shortName of this.replacements.values()) {
if (shortName === name) {
// replaced already (happens wih super types)
return true;
}
}
}
if (isNameTakenInFile(this.node, name)) {
return true;
}
return false;
}
lookupShortName(name: string): string {
let value = this.replacements!.get(name)!;
let parent = this.parent;
while (parent) {
if (parent.replacements!.has(name) && parent.fields.get(name)?.type === FieldType.Protected) {
value = parent.replacements!.get(name)! ?? value;
}
parent = parent.parent;
}
return value;
}
// --- parent chaining
addChild(child: ClassData) {
this.children ??= [];
this.children.push(child);
child.parent = this;
}
}
function isNameTakenInFile(node: ts.Node, name: string): boolean {
const identifiers = (node.getSourceFile()).identifiers;
if (identifiers instanceof Map) {
if (identifiers.has(name)) {
return true;
}
}
return false;
}
const skippedExportMangledFiles = [
// Build
'css.build',
// Monaco
'editorCommon',
'editorOptions',
'editorZoom',
'standaloneEditor',
'standaloneEnums',
'standaloneLanguages',
// Generated
'extensionsApiProposals',
// Module passed around as type
'pfs',
// entry points
...[
buildfile.workerEditor,
buildfile.workerExtensionHost,
buildfile.workerNotebook,
buildfile.workerLanguageDetection,
buildfile.workerLocalFileSearch,
buildfile.workerProfileAnalysis,
buildfile.workerOutputLinks,
buildfile.workerBackgroundTokenization,
buildfile.workbenchDesktop,
buildfile.workbenchWeb,
buildfile.code,
buildfile.codeWeb
].flat().map(x => x.name),
];
const skippedExportMangledProjects = [
// Test projects
'vscode-api-tests',
// These projects use webpack to dynamically rewrite imports, which messes up our mangling
'configuration-editing',
'microsoft-authentication',
'github-authentication',
'html-language-features/server',
];
const skippedExportMangledSymbols = [
// Don't mangle extension entry points
'activate',
'deactivate',
];
class DeclarationData {
readonly replacementName: string;
constructor(
readonly fileName: string,
readonly node: ts.FunctionDeclaration | ts.ClassDeclaration | ts.EnumDeclaration | ts.VariableDeclaration,
fileIdents: ShortIdent,
) {
// Todo: generate replacement names based on usage count, with more used names getting shorter identifiers
this.replacementName = fileIdents.next();
}
getLocations(service: ts.LanguageService): Iterable<{ fileName: string; offset: number }> {
if (ts.isVariableDeclaration(this.node)) {
// If the const aliases any types, we need to rename those too
const definitionResult = service.getDefinitionAndBoundSpan(this.fileName, this.node.name.getStart());
if (definitionResult?.definitions && definitionResult.definitions.length > 1) {
return definitionResult.definitions.map(x => ({ fileName: x.fileName, offset: x.textSpan.start }));
}
}
return [{
fileName: this.fileName,
offset: this.node.name!.getStart()
}];
}
shouldMangle(newName: string): boolean {
const currentName = this.node.name!.getText();
if (currentName.startsWith('$') || skippedExportMangledSymbols.includes(currentName)) {
return false;
}
// New name is longer the existing one :'(
if (newName.length >= currentName.length) {
return false;
}
// Don't mangle functions we've explicitly opted out
if (this.node.getFullText().includes('@skipMangle')) {
return false;
}
return true;
}
}
export interface MangleOutput {
out: string;
sourceMap?: string;
}
/**
* TypeScript2TypeScript transformer that mangles all private and protected fields
*
* 1. Collect all class fields (properties, methods)
* 2. Collect all sub and super-type relations between classes
* 3. Compute replacement names for each field
* 4. Lookup rename locations for these fields
* 5. Prepare and apply edits
*/
export class Mangler {
private readonly allClassDataByKey = new Map();
private readonly allExportedSymbols = new Set();
private readonly renameWorkerPool: workerpool.WorkerPool;
constructor(
private readonly projectPath: string,
private readonly log: typeof console.log = () => { },
private readonly config: { readonly manglePrivateFields: boolean; readonly mangleExports: boolean },
) {
this.renameWorkerPool = workerpool.pool(path.join(__dirname, 'renameWorker.js'), {
maxWorkers: 4,
minWorkers: 'max'
});
}
async computeNewFileContents(strictImplicitPublicHandling?: Set): Promise> {
const service = ts.createLanguageService(new StaticLanguageServiceHost(this.projectPath));
// STEP:
// - Find all classes and their field info.
// - Find exported symbols.
const fileIdents = new ShortIdent('$');
const visit = (node: ts.Node): void => {
if (this.config.manglePrivateFields) {
if (ts.isClassDeclaration(node) || ts.isClassExpression(node)) {
const anchor = node.name ?? node;
const key = `${node.getSourceFile().fileName}|${anchor.getStart()}`;
if (this.allClassDataByKey.has(key)) {
throw new Error('DUPE?');
}
this.allClassDataByKey.set(key, new ClassData(node.getSourceFile().fileName, node));
}
}
if (this.config.mangleExports) {
// Find exported classes, functions, and vars
if (
(
// Exported class
ts.isClassDeclaration(node)
&& hasModifier(node, ts.SyntaxKind.ExportKeyword)
&& node.name
) || (
// Exported function
ts.isFunctionDeclaration(node)
&& ts.isSourceFile(node.parent)
&& hasModifier(node, ts.SyntaxKind.ExportKeyword)
&& node.name && node.body // On named function and not on the overload
) || (
// Exported variable
ts.isVariableDeclaration(node)
&& hasModifier(node.parent.parent, ts.SyntaxKind.ExportKeyword) // Variable statement is exported
&& ts.isSourceFile(node.parent.parent.parent)
)
// Disabled for now because we need to figure out how to handle
// enums that are used in monaco or extHost interfaces.
/* || (
// Exported enum
ts.isEnumDeclaration(node)
&& ts.isSourceFile(node.parent)
&& hasModifier(node, ts.SyntaxKind.ExportKeyword)
&& !hasModifier(node, ts.SyntaxKind.ConstKeyword) // Don't bother mangling const enums because these are inlined
&& node.name
*/
) {
if (isInAmbientContext(node)) {
return;
}
this.allExportedSymbols.add(new DeclarationData(node.getSourceFile().fileName, node, fileIdents));
}
}
ts.forEachChild(node, visit);
};
for (const file of service.getProgram()!.getSourceFiles()) {
if (!file.isDeclarationFile) {
ts.forEachChild(file, visit);
}
}
this.log(`Done collecting. Classes: ${this.allClassDataByKey.size}. Exported symbols: ${this.allExportedSymbols.size}`);
// STEP: connect sub and super-types
const setupParents = (data: ClassData) => {
const extendsClause = data.node.heritageClauses?.find(h => h.token === ts.SyntaxKind.ExtendsKeyword);
if (!extendsClause) {
// no EXTENDS-clause
return;
}
const info = service.getDefinitionAtPosition(data.fileName, extendsClause.types[0].expression.getEnd());
if (!info || info.length === 0) {
// throw new Error('SUPER type not found');
return;
}
if (info.length !== 1) {
// inherits from declared/library type
return;
}
const [definition] = info;
const key = `${definition.fileName}|${definition.textSpan.start}`;
const parent = this.allClassDataByKey.get(key);
if (!parent) {
// throw new Error(`SUPER type not found: ${key}`);
return;
}
parent.addChild(data);
};
for (const data of this.allClassDataByKey.values()) {
setupParents(data);
}
// STEP: make implicit public (actually protected) field really public
const violations = new Map();
let violationsCauseFailure = false;
for (const data of this.allClassDataByKey.values()) {
ClassData.makeImplicitPublicActuallyPublic(data, (name: string, what, why) => {
const arr = violations.get(what);
if (arr) {
arr.push(why);
} else {
violations.set(what, [why]);
}
if (strictImplicitPublicHandling && !strictImplicitPublicHandling.has(name)) {
violationsCauseFailure = true;
}
});
}
for (const [why, whys] of violations) {
this.log(`WARN: ${why} became PUBLIC because of: ${whys.join(' , ')}`);
}
if (violationsCauseFailure) {
const message = 'Protected fields have been made PUBLIC. This hurts minification and is therefore not allowed. Review the WARN messages further above';
this.log(`ERROR: ${message}`);
throw new Error(message);
}
// STEP: compute replacement names for each class
for (const data of this.allClassDataByKey.values()) {
ClassData.fillInReplacement(data);
}
this.log(`Done creating class replacements`);
// STEP: prepare rename edits
this.log(`Starting prepare rename edits`);
type Edit = { newText: string; offset: number; length: number };
const editsByFile = new Map();
const appendEdit = (fileName: string, edit: Edit) => {
const edits = editsByFile.get(fileName);
if (!edits) {
editsByFile.set(fileName, [edit]);
} else {
edits.push(edit);
}
};
const appendRename = (newText: string, loc: ts.RenameLocation) => {
appendEdit(loc.fileName, {
newText: (loc.prefixText || '') + newText + (loc.suffixText || ''),
offset: loc.textSpan.start,
length: loc.textSpan.length
});
};
type RenameFn = (projectName: string, fileName: string, pos: number) => ts.RenameLocation[];
const renameResults: Array> = [];
const queueRename = (fileName: string, pos: number, newName: string) => {
renameResults.push(Promise.resolve(this.renameWorkerPool.exec('findRenameLocations', [this.projectPath, fileName, pos]))
.then((locations) => ({ newName, locations })));
};
for (const data of this.allClassDataByKey.values()) {
if (hasModifier(data.node, ts.SyntaxKind.DeclareKeyword)) {
continue;
}
fields: for (const [name, info] of data.fields) {
if (!ClassData._shouldMangle(info.type)) {
continue fields;
}
// TS-HACK: protected became public via 'some' child
// and because of that we might need to ignore this now
let parent = data.parent;
while (parent) {
if (parent.fields.get(name)?.type === FieldType.Public) {
continue fields;
}
parent = parent.parent;
}
const newName = data.lookupShortName(name);
queueRename(data.fileName, info.pos, newName);
}
}
for (const data of this.allExportedSymbols.values()) {
if (data.fileName.endsWith('.d.ts')
|| skippedExportMangledProjects.some(proj => data.fileName.includes(proj))
|| skippedExportMangledFiles.some(file => data.fileName.endsWith(file + '.ts'))
) {
continue;
}
if (!data.shouldMangle(data.replacementName)) {
continue;
}
const newText = data.replacementName;
for (const { fileName, offset } of data.getLocations(service)) {
queueRename(fileName, offset, newText);
}
}
await Promise.all(renameResults).then((result) => {
for (const { newName, locations } of result) {
for (const loc of locations) {
appendRename(newName, loc);
}
}
});
await this.renameWorkerPool.terminate();
this.log(`Done preparing edits: ${editsByFile.size} files`);
// STEP: apply all rename edits (per file)
const result = new Map();
let savedBytes = 0;
for (const item of service.getProgram()!.getSourceFiles()) {
const { mapRoot, sourceRoot } = service.getProgram()!.getCompilerOptions();
const projectDir = path.dirname(this.projectPath);
const sourceMapRoot = mapRoot ?? pathToFileURL(sourceRoot ?? projectDir).toString();
// source maps
let generator: SourceMapGenerator | undefined;
let newFullText: string;
const edits = editsByFile.get(item.fileName);
if (!edits) {
// just copy
newFullText = item.getFullText();
} else {
// source map generator
const relativeFileName = normalize(path.relative(projectDir, item.fileName));
const mappingsByLine = new Map();
// apply renames
edits.sort((a, b) => b.offset - a.offset);
const characters = item.getFullText().split('');
let lastEdit: Edit | undefined;
for (const edit of edits) {
if (lastEdit && lastEdit.offset === edit.offset) {
//
if (lastEdit.length !== edit.length || lastEdit.newText !== edit.newText) {
this.log('ERROR: Overlapping edit', item.fileName, edit.offset, edits);
throw new Error('OVERLAPPING edit');
} else {
continue;
}
}
lastEdit = edit;
const mangledName = characters.splice(edit.offset, edit.length, edit.newText).join('');
savedBytes += mangledName.length - edit.newText.length;
// source maps
const pos = item.getLineAndCharacterOfPosition(edit.offset);
let mappings = mappingsByLine.get(pos.line);
if (!mappings) {
mappings = [];
mappingsByLine.set(pos.line, mappings);
}
mappings.unshift({
source: relativeFileName,
original: { line: pos.line + 1, column: pos.character },
generated: { line: pos.line + 1, column: pos.character },
name: mangledName
}, {
source: relativeFileName,
original: { line: pos.line + 1, column: pos.character + edit.length },
generated: { line: pos.line + 1, column: pos.character + edit.newText.length },
});
}
// source map generation, make sure to get mappings per line correct
generator = new SourceMapGenerator({ file: path.basename(item.fileName), sourceRoot: sourceMapRoot });
generator.setSourceContent(relativeFileName, item.getFullText());
for (const [, mappings] of mappingsByLine) {
let lineDelta = 0;
for (const mapping of mappings) {
generator.addMapping({
...mapping,
generated: { line: mapping.generated.line, column: mapping.generated.column - lineDelta }
});
lineDelta += mapping.original.column - mapping.generated.column;
}
}
newFullText = characters.join('');
}
result.set(item.fileName, { out: newFullText, sourceMap: generator?.toString() });
}
service.dispose();
this.renameWorkerPool.terminate();
this.log(`Done: ${savedBytes / 1000}kb saved, memory-usage: ${JSON.stringify(v8.getHeapStatistics())}`);
return result;
}
}
// --- ast utils
function hasModifier(node: ts.Node, kind: ts.SyntaxKind) {
const modifiers = ts.canHaveModifiers(node) ? ts.getModifiers(node) : undefined;
return Boolean(modifiers?.find(mode => mode.kind === kind));
}
function isInAmbientContext(node: ts.Node): boolean {
for (let p = node.parent; p; p = p.parent) {
if (ts.isModuleDeclaration(p)) {
return true;
}
}
return false;
}
function normalize(path: string): string {
return path.replace(/\\/g, '/');
}
async function _run() {
const root = path.join(__dirname, '..', '..', '..');
const projectBase = path.join(root, 'src');
const projectPath = path.join(projectBase, 'tsconfig.json');
const newProjectBase = path.join(path.dirname(projectBase), path.basename(projectBase) + '2');
fs.cpSync(projectBase, newProjectBase, { recursive: true });
const mangler = new Mangler(projectPath, console.log, {
mangleExports: true,
manglePrivateFields: true,
});
for (const [fileName, contents] of await mangler.computeNewFileContents(new Set(['saveState']))) {
const newFilePath = path.join(newProjectBase, path.relative(projectBase, fileName));
await fs.promises.mkdir(path.dirname(newFilePath), { recursive: true });
await fs.promises.writeFile(newFilePath, contents.out);
if (contents.sourceMap) {
await fs.promises.writeFile(newFilePath + '.map', contents.sourceMap);
}
}
}
if (__filename === argv[1]) {
_run();
}
================================================
FILE: build/lib/mangle/renameWorker.js
================================================
"use strict";
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
Object.defineProperty(exports, "__esModule", { value: true });
const typescript_1 = __importDefault(require("typescript"));
const workerpool_1 = __importDefault(require("workerpool"));
const staticLanguageServiceHost_1 = require("./staticLanguageServiceHost");
let service;
function findRenameLocations(projectPath, fileName, position) {
if (!service) {
service = typescript_1.default.createLanguageService(new staticLanguageServiceHost_1.StaticLanguageServiceHost(projectPath));
}
return service.findRenameLocations(fileName, position, false, false, {
providePrefixAndSuffixTextForRename: true,
}) ?? [];
}
workerpool_1.default.worker({
findRenameLocations
});
//# sourceMappingURL=renameWorker.js.map
================================================
FILE: build/lib/mangle/renameWorker.ts
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
import ts from 'typescript';
import workerpool from 'workerpool';
import { StaticLanguageServiceHost } from './staticLanguageServiceHost';
let service: ts.LanguageService | undefined;
function findRenameLocations(
projectPath: string,
fileName: string,
position: number,
): readonly ts.RenameLocation[] {
if (!service) {
service = ts.createLanguageService(new StaticLanguageServiceHost(projectPath));
}
return service.findRenameLocations(fileName, position, false, false, {
providePrefixAndSuffixTextForRename: true,
}) ?? [];
}
workerpool.worker({
findRenameLocations
});
================================================
FILE: build/lib/mangle/staticLanguageServiceHost.js
================================================
"use strict";
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
Object.defineProperty(exports, "__esModule", { value: true });
exports.StaticLanguageServiceHost = void 0;
const typescript_1 = __importDefault(require("typescript"));
const path_1 = __importDefault(require("path"));
class StaticLanguageServiceHost {
projectPath;
_cmdLine;
_scriptSnapshots = new Map();
constructor(projectPath) {
this.projectPath = projectPath;
const existingOptions = {};
const parsed = typescript_1.default.readConfigFile(projectPath, typescript_1.default.sys.readFile);
if (parsed.error) {
throw parsed.error;
}
this._cmdLine = typescript_1.default.parseJsonConfigFileContent(parsed.config, typescript_1.default.sys, path_1.default.dirname(projectPath), existingOptions);
if (this._cmdLine.errors.length > 0) {
throw parsed.error;
}
}
getCompilationSettings() {
return this._cmdLine.options;
}
getScriptFileNames() {
return this._cmdLine.fileNames;
}
getScriptVersion(_fileName) {
return '1';
}
getProjectVersion() {
return '1';
}
getScriptSnapshot(fileName) {
let result = this._scriptSnapshots.get(fileName);
if (result === undefined) {
const content = typescript_1.default.sys.readFile(fileName);
if (content === undefined) {
return undefined;
}
result = typescript_1.default.ScriptSnapshot.fromString(content);
this._scriptSnapshots.set(fileName, result);
}
return result;
}
getCurrentDirectory() {
return path_1.default.dirname(this.projectPath);
}
getDefaultLibFileName(options) {
return typescript_1.default.getDefaultLibFilePath(options);
}
directoryExists = typescript_1.default.sys.directoryExists;
getDirectories = typescript_1.default.sys.getDirectories;
fileExists = typescript_1.default.sys.fileExists;
readFile = typescript_1.default.sys.readFile;
readDirectory = typescript_1.default.sys.readDirectory;
// this is necessary to make source references work.
realpath = typescript_1.default.sys.realpath;
}
exports.StaticLanguageServiceHost = StaticLanguageServiceHost;
//# sourceMappingURL=staticLanguageServiceHost.js.map
================================================
FILE: build/lib/mangle/staticLanguageServiceHost.ts
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
import ts from 'typescript';
import path from 'path';
export class StaticLanguageServiceHost implements ts.LanguageServiceHost {
private readonly _cmdLine: ts.ParsedCommandLine;
private readonly _scriptSnapshots: Map = new Map();
constructor(readonly projectPath: string) {
const existingOptions: Partial = {};
const parsed = ts.readConfigFile(projectPath, ts.sys.readFile);
if (parsed.error) {
throw parsed.error;
}
this._cmdLine = ts.parseJsonConfigFileContent(parsed.config, ts.sys, path.dirname(projectPath), existingOptions);
if (this._cmdLine.errors.length > 0) {
throw parsed.error;
}
}
getCompilationSettings(): ts.CompilerOptions {
return this._cmdLine.options;
}
getScriptFileNames(): string[] {
return this._cmdLine.fileNames;
}
getScriptVersion(_fileName: string): string {
return '1';
}
getProjectVersion(): string {
return '1';
}
getScriptSnapshot(fileName: string): ts.IScriptSnapshot | undefined {
let result: ts.IScriptSnapshot | undefined = this._scriptSnapshots.get(fileName);
if (result === undefined) {
const content = ts.sys.readFile(fileName);
if (content === undefined) {
return undefined;
}
result = ts.ScriptSnapshot.fromString(content);
this._scriptSnapshots.set(fileName, result);
}
return result;
}
getCurrentDirectory(): string {
return path.dirname(this.projectPath);
}
getDefaultLibFileName(options: ts.CompilerOptions): string {
return ts.getDefaultLibFilePath(options);
}
directoryExists = ts.sys.directoryExists;
getDirectories = ts.sys.getDirectories;
fileExists = ts.sys.fileExists;
readFile = ts.sys.readFile;
readDirectory = ts.sys.readDirectory;
// this is necessary to make source references work.
realpath = ts.sys.realpath;
}
================================================
FILE: build/lib/monaco-api.js
================================================
"use strict";
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
Object.defineProperty(exports, "__esModule", { value: true });
exports.DeclarationResolver = exports.FSProvider = exports.RECIPE_PATH = void 0;
exports.run3 = run3;
exports.execute = execute;
const fs_1 = __importDefault(require("fs"));
const path_1 = __importDefault(require("path"));
const fancy_log_1 = __importDefault(require("fancy-log"));
const ansi_colors_1 = __importDefault(require("ansi-colors"));
const dtsv = '3';
const tsfmt = require('../../tsfmt.json');
const SRC = path_1.default.join(__dirname, '../../src');
exports.RECIPE_PATH = path_1.default.join(__dirname, '../monaco/monaco.d.ts.recipe');
const DECLARATION_PATH = path_1.default.join(__dirname, '../../src/vs/monaco.d.ts');
function logErr(message, ...rest) {
(0, fancy_log_1.default)(ansi_colors_1.default.yellow(`[monaco.d.ts]`), message, ...rest);
}
function isDeclaration(ts, a) {
return (a.kind === ts.SyntaxKind.InterfaceDeclaration
|| a.kind === ts.SyntaxKind.EnumDeclaration
|| a.kind === ts.SyntaxKind.ClassDeclaration
|| a.kind === ts.SyntaxKind.TypeAliasDeclaration
|| a.kind === ts.SyntaxKind.FunctionDeclaration
|| a.kind === ts.SyntaxKind.ModuleDeclaration);
}
function visitTopLevelDeclarations(ts, sourceFile, visitor) {
let stop = false;
const visit = (node) => {
if (stop) {
return;
}
switch (node.kind) {
case ts.SyntaxKind.InterfaceDeclaration:
case ts.SyntaxKind.EnumDeclaration:
case ts.SyntaxKind.ClassDeclaration:
case ts.SyntaxKind.VariableStatement:
case ts.SyntaxKind.TypeAliasDeclaration:
case ts.SyntaxKind.FunctionDeclaration:
case ts.SyntaxKind.ModuleDeclaration:
stop = visitor(node);
}
if (stop) {
return;
}
ts.forEachChild(node, visit);
};
visit(sourceFile);
}
function getAllTopLevelDeclarations(ts, sourceFile) {
const all = [];
visitTopLevelDeclarations(ts, sourceFile, (node) => {
if (node.kind === ts.SyntaxKind.InterfaceDeclaration || node.kind === ts.SyntaxKind.ClassDeclaration || node.kind === ts.SyntaxKind.ModuleDeclaration) {
const interfaceDeclaration = node;
const triviaStart = interfaceDeclaration.pos;
const triviaEnd = interfaceDeclaration.name.pos;
const triviaText = getNodeText(sourceFile, { pos: triviaStart, end: triviaEnd });
if (triviaText.indexOf('@internal') === -1) {
all.push(node);
}
}
else {
const nodeText = getNodeText(sourceFile, node);
if (nodeText.indexOf('@internal') === -1) {
all.push(node);
}
}
return false /*continue*/;
});
return all;
}
function getTopLevelDeclaration(ts, sourceFile, typeName) {
let result = null;
visitTopLevelDeclarations(ts, sourceFile, (node) => {
if (isDeclaration(ts, node) && node.name) {
if (node.name.text === typeName) {
result = node;
return true /*stop*/;
}
return false /*continue*/;
}
// node is ts.VariableStatement
if (getNodeText(sourceFile, node).indexOf(typeName) >= 0) {
result = node;
return true /*stop*/;
}
return false /*continue*/;
});
return result;
}
function getNodeText(sourceFile, node) {
return sourceFile.getFullText().substring(node.pos, node.end);
}
function hasModifier(modifiers, kind) {
if (modifiers) {
for (let i = 0; i < modifiers.length; i++) {
const mod = modifiers[i];
if (mod.kind === kind) {
return true;
}
}
}
return false;
}
function isStatic(ts, member) {
if (ts.canHaveModifiers(member)) {
return hasModifier(ts.getModifiers(member), ts.SyntaxKind.StaticKeyword);
}
return false;
}
function isDefaultExport(ts, declaration) {
return (hasModifier(declaration.modifiers, ts.SyntaxKind.DefaultKeyword)
&& hasModifier(declaration.modifiers, ts.SyntaxKind.ExportKeyword));
}
function getMassagedTopLevelDeclarationText(ts, sourceFile, declaration, importName, usage, enums) {
let result = getNodeText(sourceFile, declaration);
if (declaration.kind === ts.SyntaxKind.InterfaceDeclaration || declaration.kind === ts.SyntaxKind.ClassDeclaration) {
const interfaceDeclaration = declaration;
const staticTypeName = (isDefaultExport(ts, interfaceDeclaration)
? `${importName}.default`
: `${importName}.${declaration.name.text}`);
let instanceTypeName = staticTypeName;
const typeParametersCnt = (interfaceDeclaration.typeParameters ? interfaceDeclaration.typeParameters.length : 0);
if (typeParametersCnt > 0) {
const arr = [];
for (let i = 0; i < typeParametersCnt; i++) {
arr.push('any');
}
instanceTypeName = `${instanceTypeName}<${arr.join(',')}>`;
}
const members = interfaceDeclaration.members;
members.forEach((member) => {
try {
const memberText = getNodeText(sourceFile, member);
if (memberText.indexOf('@internal') >= 0 || memberText.indexOf('private') >= 0) {
result = result.replace(memberText, '');
}
else {
const memberName = member.name.text;
const memberAccess = (memberName.indexOf('.') >= 0 ? `['${memberName}']` : `.${memberName}`);
if (isStatic(ts, member)) {
usage.push(`a = ${staticTypeName}${memberAccess};`);
}
else {
usage.push(`a = (<${instanceTypeName}>b)${memberAccess};`);
}
}
}
catch (err) {
// life..
}
});
}
result = result.replace(/export default /g, 'export ');
result = result.replace(/export declare /g, 'export ');
result = result.replace(/declare /g, '');
const lines = result.split(/\r\n|\r|\n/);
for (let i = 0; i < lines.length; i++) {
if (/\s*\*/.test(lines[i])) {
// very likely a comment
continue;
}
lines[i] = lines[i].replace(/"/g, '\'');
}
result = lines.join('\n');
if (declaration.kind === ts.SyntaxKind.EnumDeclaration) {
result = result.replace(/const enum/, 'enum');
enums.push({
enumName: declaration.name.getText(sourceFile),
text: result
});
}
return result;
}
function format(ts, text, endl) {
const REALLY_FORMAT = false;
text = preformat(text, endl);
if (!REALLY_FORMAT) {
return text;
}
// Parse the source text
const sourceFile = ts.createSourceFile('file.ts', text, ts.ScriptTarget.Latest, /*setParentPointers*/ true);
// Get the formatting edits on the input sources
const edits = ts.formatting.formatDocument(sourceFile, getRuleProvider(tsfmt), tsfmt);
// Apply the edits on the input code
return applyEdits(text, edits);
function countParensCurly(text) {
let cnt = 0;
for (let i = 0; i < text.length; i++) {
if (text.charAt(i) === '(' || text.charAt(i) === '{') {
cnt++;
}
if (text.charAt(i) === ')' || text.charAt(i) === '}') {
cnt--;
}
}
return cnt;
}
function repeatStr(s, cnt) {
let r = '';
for (let i = 0; i < cnt; i++) {
r += s;
}
return r;
}
function preformat(text, endl) {
const lines = text.split(endl);
let inComment = false;
let inCommentDeltaIndent = 0;
let indent = 0;
for (let i = 0; i < lines.length; i++) {
let line = lines[i].replace(/\s$/, '');
let repeat = false;
let lineIndent = 0;
do {
repeat = false;
if (line.substring(0, 4) === ' ') {
line = line.substring(4);
lineIndent++;
repeat = true;
}
if (line.charAt(0) === '\t') {
line = line.substring(1);
lineIndent++;
repeat = true;
}
} while (repeat);
if (line.length === 0) {
continue;
}
if (inComment) {
if (/\*\//.test(line)) {
inComment = false;
}
lines[i] = repeatStr('\t', lineIndent + inCommentDeltaIndent) + line;
continue;
}
if (/\/\*/.test(line)) {
inComment = true;
inCommentDeltaIndent = indent - lineIndent;
lines[i] = repeatStr('\t', indent) + line;
continue;
}
const cnt = countParensCurly(line);
let shouldUnindentAfter = false;
let shouldUnindentBefore = false;
if (cnt < 0) {
if (/[({]/.test(line)) {
shouldUnindentAfter = true;
}
else {
shouldUnindentBefore = true;
}
}
else if (cnt === 0) {
shouldUnindentBefore = /^\}/.test(line);
}
let shouldIndentAfter = false;
if (cnt > 0) {
shouldIndentAfter = true;
}
else if (cnt === 0) {
shouldIndentAfter = /{$/.test(line);
}
if (shouldUnindentBefore) {
indent--;
}
lines[i] = repeatStr('\t', indent) + line;
if (shouldUnindentAfter) {
indent--;
}
if (shouldIndentAfter) {
indent++;
}
}
return lines.join(endl);
}
function getRuleProvider(options) {
// Share this between multiple formatters using the same options.
// This represents the bulk of the space the formatter uses.
return ts.formatting.getFormatContext(options);
}
function applyEdits(text, edits) {
// Apply edits in reverse on the existing text
let result = text;
for (let i = edits.length - 1; i >= 0; i--) {
const change = edits[i];
const head = result.slice(0, change.span.start);
const tail = result.slice(change.span.start + change.span.length);
result = head + change.newText + tail;
}
return result;
}
}
function createReplacerFromDirectives(directives) {
return (str) => {
for (let i = 0; i < directives.length; i++) {
str = str.replace(directives[i][0], directives[i][1]);
}
return str;
};
}
function createReplacer(data) {
data = data || '';
const rawDirectives = data.split(';');
const directives = [];
rawDirectives.forEach((rawDirective) => {
if (rawDirective.length === 0) {
return;
}
const pieces = rawDirective.split('=>');
let findStr = pieces[0];
const replaceStr = pieces[1];
findStr = findStr.replace(/[\-\\\{\}\*\+\?\|\^\$\.\,\[\]\(\)\#\s]/g, '\\$&');
findStr = '\\b' + findStr + '\\b';
directives.push([new RegExp(findStr, 'g'), replaceStr]);
});
return createReplacerFromDirectives(directives);
}
function generateDeclarationFile(ts, recipe, sourceFileGetter) {
const endl = /\r\n/.test(recipe) ? '\r\n' : '\n';
const lines = recipe.split(endl);
const result = [];
let usageCounter = 0;
const usageImports = [];
const usage = [];
let failed = false;
usage.push(`var a: any;`);
usage.push(`var b: any;`);
const generateUsageImport = (moduleId) => {
const importName = 'm' + (++usageCounter);
usageImports.push(`import * as ${importName} from './${moduleId.replace(/\.d\.ts$/, '')}';`);
return importName;
};
const enums = [];
let version = null;
lines.forEach(line => {
if (failed) {
return;
}
const m0 = line.match(/^\/\/dtsv=(\d+)$/);
if (m0) {
version = m0[1];
}
const m1 = line.match(/^\s*#include\(([^;)]*)(;[^)]*)?\)\:(.*)$/);
if (m1) {
const moduleId = m1[1];
const sourceFile = sourceFileGetter(moduleId);
if (!sourceFile) {
logErr(`While handling ${line}`);
logErr(`Cannot find ${moduleId}`);
failed = true;
return;
}
const importName = generateUsageImport(moduleId);
const replacer = createReplacer(m1[2]);
const typeNames = m1[3].split(/,/);
typeNames.forEach((typeName) => {
typeName = typeName.trim();
if (typeName.length === 0) {
return;
}
const declaration = getTopLevelDeclaration(ts, sourceFile, typeName);
if (!declaration) {
logErr(`While handling ${line}`);
logErr(`Cannot find ${typeName}`);
failed = true;
return;
}
result.push(replacer(getMassagedTopLevelDeclarationText(ts, sourceFile, declaration, importName, usage, enums)));
});
return;
}
const m2 = line.match(/^\s*#includeAll\(([^;)]*)(;[^)]*)?\)\:(.*)$/);
if (m2) {
const moduleId = m2[1];
const sourceFile = sourceFileGetter(moduleId);
if (!sourceFile) {
logErr(`While handling ${line}`);
logErr(`Cannot find ${moduleId}`);
failed = true;
return;
}
const importName = generateUsageImport(moduleId);
const replacer = createReplacer(m2[2]);
const typeNames = m2[3].split(/,/);
const typesToExcludeMap = {};
const typesToExcludeArr = [];
typeNames.forEach((typeName) => {
typeName = typeName.trim();
if (typeName.length === 0) {
return;
}
typesToExcludeMap[typeName] = true;
typesToExcludeArr.push(typeName);
});
getAllTopLevelDeclarations(ts, sourceFile).forEach((declaration) => {
if (isDeclaration(ts, declaration) && declaration.name) {
if (typesToExcludeMap[declaration.name.text]) {
return;
}
}
else {
// node is ts.VariableStatement
const nodeText = getNodeText(sourceFile, declaration);
for (let i = 0; i < typesToExcludeArr.length; i++) {
if (nodeText.indexOf(typesToExcludeArr[i]) >= 0) {
return;
}
}
}
result.push(replacer(getMassagedTopLevelDeclarationText(ts, sourceFile, declaration, importName, usage, enums)));
});
return;
}
result.push(line);
});
if (failed) {
return null;
}
if (version !== dtsv) {
if (!version) {
logErr(`gulp watch restart required. 'monaco.d.ts.recipe' is written before versioning was introduced.`);
}
else {
logErr(`gulp watch restart required. 'monaco.d.ts.recipe' v${version} does not match runtime v${dtsv}.`);
}
return null;
}
let resultTxt = result.join(endl);
resultTxt = resultTxt.replace(/\bURI\b/g, 'Uri');
resultTxt = resultTxt.replace(/\bEvent {
if (e1.enumName < e2.enumName) {
return -1;
}
if (e1.enumName > e2.enumName) {
return 1;
}
return 0;
});
let resultEnums = [
'/*---------------------------------------------------------------------------------------------',
' * Copyright (c) Microsoft Corporation. All rights reserved.',
' * Licensed under the MIT License. See License.txt in the project root for license information.',
' *--------------------------------------------------------------------------------------------*/',
'',
'// THIS IS A GENERATED FILE. DO NOT EDIT DIRECTLY.',
''
].concat(enums.map(e => e.text)).join(endl);
resultEnums = resultEnums.split(/\r\n|\n|\r/).join(endl);
resultEnums = format(ts, resultEnums, endl);
resultEnums = resultEnums.split(/\r\n|\n|\r/).join(endl);
return {
result: resultTxt,
usageContent: `${usageImports.join('\n')}\n\n${usage.join('\n')}`,
enums: resultEnums
};
}
function _run(ts, sourceFileGetter) {
const recipe = fs_1.default.readFileSync(exports.RECIPE_PATH).toString();
const t = generateDeclarationFile(ts, recipe, sourceFileGetter);
if (!t) {
return null;
}
const result = t.result;
const usageContent = t.usageContent;
const enums = t.enums;
const currentContent = fs_1.default.readFileSync(DECLARATION_PATH).toString();
const one = currentContent.replace(/\r\n/gm, '\n');
const other = result.replace(/\r\n/gm, '\n');
const isTheSame = (one === other);
return {
content: result,
usageContent: usageContent,
enums: enums,
filePath: DECLARATION_PATH,
isTheSame
};
}
class FSProvider {
existsSync(filePath) {
return fs_1.default.existsSync(filePath);
}
statSync(filePath) {
return fs_1.default.statSync(filePath);
}
readFileSync(_moduleId, filePath) {
return fs_1.default.readFileSync(filePath);
}
}
exports.FSProvider = FSProvider;
class CacheEntry {
sourceFile;
mtime;
constructor(sourceFile, mtime) {
this.sourceFile = sourceFile;
this.mtime = mtime;
}
}
class DeclarationResolver {
_fsProvider;
ts;
_sourceFileCache;
constructor(_fsProvider) {
this._fsProvider = _fsProvider;
this.ts = require('typescript');
this._sourceFileCache = Object.create(null);
}
invalidateCache(moduleId) {
this._sourceFileCache[moduleId] = null;
}
getDeclarationSourceFile(moduleId) {
if (this._sourceFileCache[moduleId]) {
// Since we cannot trust file watching to invalidate the cache, check also the mtime
const fileName = this._getFileName(moduleId);
const mtime = this._fsProvider.statSync(fileName).mtime.getTime();
if (this._sourceFileCache[moduleId].mtime !== mtime) {
this._sourceFileCache[moduleId] = null;
}
}
if (!this._sourceFileCache[moduleId]) {
this._sourceFileCache[moduleId] = this._getDeclarationSourceFile(moduleId);
}
return this._sourceFileCache[moduleId] ? this._sourceFileCache[moduleId].sourceFile : null;
}
_getFileName(moduleId) {
if (/\.d\.ts$/.test(moduleId)) {
return path_1.default.join(SRC, moduleId);
}
return path_1.default.join(SRC, `${moduleId}.ts`);
}
_getDeclarationSourceFile(moduleId) {
const fileName = this._getFileName(moduleId);
if (!this._fsProvider.existsSync(fileName)) {
return null;
}
const mtime = this._fsProvider.statSync(fileName).mtime.getTime();
if (/\.d\.ts$/.test(moduleId)) {
// const mtime = this._fsProvider.statFileSync()
const fileContents = this._fsProvider.readFileSync(moduleId, fileName).toString();
return new CacheEntry(this.ts.createSourceFile(fileName, fileContents, this.ts.ScriptTarget.ES5), mtime);
}
const fileContents = this._fsProvider.readFileSync(moduleId, fileName).toString();
const fileMap = {
'file.ts': fileContents
};
const service = this.ts.createLanguageService(new TypeScriptLanguageServiceHost(this.ts, {}, fileMap, {}));
const text = service.getEmitOutput('file.ts', true, true).outputFiles[0].text;
return new CacheEntry(this.ts.createSourceFile(fileName, text, this.ts.ScriptTarget.ES5), mtime);
}
}
exports.DeclarationResolver = DeclarationResolver;
function run3(resolver) {
const sourceFileGetter = (moduleId) => resolver.getDeclarationSourceFile(moduleId);
return _run(resolver.ts, sourceFileGetter);
}
class TypeScriptLanguageServiceHost {
_ts;
_libs;
_files;
_compilerOptions;
constructor(ts, libs, files, compilerOptions) {
this._ts = ts;
this._libs = libs;
this._files = files;
this._compilerOptions = compilerOptions;
}
// --- language service host ---------------
getCompilationSettings() {
return this._compilerOptions;
}
getScriptFileNames() {
return ([]
.concat(Object.keys(this._libs))
.concat(Object.keys(this._files)));
}
getScriptVersion(_fileName) {
return '1';
}
getProjectVersion() {
return '1';
}
getScriptSnapshot(fileName) {
if (this._files.hasOwnProperty(fileName)) {
return this._ts.ScriptSnapshot.fromString(this._files[fileName]);
}
else if (this._libs.hasOwnProperty(fileName)) {
return this._ts.ScriptSnapshot.fromString(this._libs[fileName]);
}
else {
return this._ts.ScriptSnapshot.fromString('');
}
}
getScriptKind(_fileName) {
return this._ts.ScriptKind.TS;
}
getCurrentDirectory() {
return '';
}
getDefaultLibFileName(_options) {
return 'defaultLib:es5';
}
isDefaultLibFileName(fileName) {
return fileName === this.getDefaultLibFileName(this._compilerOptions);
}
readFile(path, _encoding) {
return this._files[path] || this._libs[path];
}
fileExists(path) {
return path in this._files || path in this._libs;
}
}
function execute() {
const r = run3(new DeclarationResolver(new FSProvider()));
if (!r) {
throw new Error(`monaco.d.ts generation error - Cannot continue`);
}
return r;
}
//# sourceMappingURL=monaco-api.js.map
================================================
FILE: build/lib/monaco-api.ts
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
import fs from 'fs';
import type * as ts from 'typescript';
import path from 'path';
import fancyLog from 'fancy-log';
import ansiColors from 'ansi-colors';
const dtsv = '3';
const tsfmt = require('../../tsfmt.json');
const SRC = path.join(__dirname, '../../src');
export const RECIPE_PATH = path.join(__dirname, '../monaco/monaco.d.ts.recipe');
const DECLARATION_PATH = path.join(__dirname, '../../src/vs/monaco.d.ts');
function logErr(message: any, ...rest: any[]): void {
fancyLog(ansiColors.yellow(`[monaco.d.ts]`), message, ...rest);
}
type SourceFileGetter = (moduleId: string) => ts.SourceFile | null;
type TSTopLevelDeclaration = ts.InterfaceDeclaration | ts.EnumDeclaration | ts.ClassDeclaration | ts.TypeAliasDeclaration | ts.FunctionDeclaration | ts.ModuleDeclaration;
type TSTopLevelDeclare = TSTopLevelDeclaration | ts.VariableStatement;
function isDeclaration(ts: typeof import('typescript'), a: TSTopLevelDeclare): a is TSTopLevelDeclaration {
return (
a.kind === ts.SyntaxKind.InterfaceDeclaration
|| a.kind === ts.SyntaxKind.EnumDeclaration
|| a.kind === ts.SyntaxKind.ClassDeclaration
|| a.kind === ts.SyntaxKind.TypeAliasDeclaration
|| a.kind === ts.SyntaxKind.FunctionDeclaration
|| a.kind === ts.SyntaxKind.ModuleDeclaration
);
}
function visitTopLevelDeclarations(ts: typeof import('typescript'), sourceFile: ts.SourceFile, visitor: (node: TSTopLevelDeclare) => boolean): void {
let stop = false;
const visit = (node: ts.Node): void => {
if (stop) {
return;
}
switch (node.kind) {
case ts.SyntaxKind.InterfaceDeclaration:
case ts.SyntaxKind.EnumDeclaration:
case ts.SyntaxKind.ClassDeclaration:
case ts.SyntaxKind.VariableStatement:
case ts.SyntaxKind.TypeAliasDeclaration:
case ts.SyntaxKind.FunctionDeclaration:
case ts.SyntaxKind.ModuleDeclaration:
stop = visitor(node);
}
if (stop) {
return;
}
ts.forEachChild(node, visit);
};
visit(sourceFile);
}
function getAllTopLevelDeclarations(ts: typeof import('typescript'), sourceFile: ts.SourceFile): TSTopLevelDeclare[] {
const all: TSTopLevelDeclare[] = [];
visitTopLevelDeclarations(ts, sourceFile, (node) => {
if (node.kind === ts.SyntaxKind.InterfaceDeclaration || node.kind === ts.SyntaxKind.ClassDeclaration || node.kind === ts.SyntaxKind.ModuleDeclaration) {
const interfaceDeclaration = node;
const triviaStart = interfaceDeclaration.pos;
const triviaEnd = interfaceDeclaration.name.pos;
const triviaText = getNodeText(sourceFile, { pos: triviaStart, end: triviaEnd });
if (triviaText.indexOf('@internal') === -1) {
all.push(node);
}
} else {
const nodeText = getNodeText(sourceFile, node);
if (nodeText.indexOf('@internal') === -1) {
all.push(node);
}
}
return false /*continue*/;
});
return all;
}
function getTopLevelDeclaration(ts: typeof import('typescript'), sourceFile: ts.SourceFile, typeName: string): TSTopLevelDeclare | null {
let result: TSTopLevelDeclare | null = null;
visitTopLevelDeclarations(ts, sourceFile, (node) => {
if (isDeclaration(ts, node) && node.name) {
if (node.name.text === typeName) {
result = node;
return true /*stop*/;
}
return false /*continue*/;
}
// node is ts.VariableStatement
if (getNodeText(sourceFile, node).indexOf(typeName) >= 0) {
result = node;
return true /*stop*/;
}
return false /*continue*/;
});
return result;
}
function getNodeText(sourceFile: ts.SourceFile, node: { pos: number; end: number }): string {
return sourceFile.getFullText().substring(node.pos, node.end);
}
function hasModifier(modifiers: readonly ts.ModifierLike[] | undefined, kind: ts.SyntaxKind): boolean {
if (modifiers) {
for (let i = 0; i < modifiers.length; i++) {
const mod = modifiers[i];
if (mod.kind === kind) {
return true;
}
}
}
return false;
}
function isStatic(ts: typeof import('typescript'), member: ts.ClassElement | ts.TypeElement): boolean {
if (ts.canHaveModifiers(member)) {
return hasModifier(ts.getModifiers(member), ts.SyntaxKind.StaticKeyword);
}
return false;
}
function isDefaultExport(ts: typeof import('typescript'), declaration: ts.InterfaceDeclaration | ts.ClassDeclaration): boolean {
return (
hasModifier(declaration.modifiers, ts.SyntaxKind.DefaultKeyword)
&& hasModifier(declaration.modifiers, ts.SyntaxKind.ExportKeyword)
);
}
function getMassagedTopLevelDeclarationText(ts: typeof import('typescript'), sourceFile: ts.SourceFile, declaration: TSTopLevelDeclare, importName: string, usage: string[], enums: IEnumEntry[]): string {
let result = getNodeText(sourceFile, declaration);
if (declaration.kind === ts.SyntaxKind.InterfaceDeclaration || declaration.kind === ts.SyntaxKind.ClassDeclaration) {
const interfaceDeclaration = declaration;
const staticTypeName = (
isDefaultExport(ts, interfaceDeclaration)
? `${importName}.default`
: `${importName}.${declaration.name!.text}`
);
let instanceTypeName = staticTypeName;
const typeParametersCnt = (interfaceDeclaration.typeParameters ? interfaceDeclaration.typeParameters.length : 0);
if (typeParametersCnt > 0) {
const arr: string[] = [];
for (let i = 0; i < typeParametersCnt; i++) {
arr.push('any');
}
instanceTypeName = `${instanceTypeName}<${arr.join(',')}>`;
}
const members: ts.NodeArray = interfaceDeclaration.members;
members.forEach((member) => {
try {
const memberText = getNodeText(sourceFile, member);
if (memberText.indexOf('@internal') >= 0 || memberText.indexOf('private') >= 0) {
result = result.replace(memberText, '');
} else {
const memberName = (member.name).text;
const memberAccess = (memberName.indexOf('.') >= 0 ? `['${memberName}']` : `.${memberName}`);
if (isStatic(ts, member)) {
usage.push(`a = ${staticTypeName}${memberAccess};`);
} else {
usage.push(`a = (<${instanceTypeName}>b)${memberAccess};`);
}
}
} catch (err) {
// life..
}
});
}
result = result.replace(/export default /g, 'export ');
result = result.replace(/export declare /g, 'export ');
result = result.replace(/declare /g, '');
const lines = result.split(/\r\n|\r|\n/);
for (let i = 0; i < lines.length; i++) {
if (/\s*\*/.test(lines[i])) {
// very likely a comment
continue;
}
lines[i] = lines[i].replace(/"/g, '\'');
}
result = lines.join('\n');
if (declaration.kind === ts.SyntaxKind.EnumDeclaration) {
result = result.replace(/const enum/, 'enum');
enums.push({
enumName: declaration.name.getText(sourceFile),
text: result
});
}
return result;
}
function format(ts: typeof import('typescript'), text: string, endl: string): string {
const REALLY_FORMAT = false;
text = preformat(text, endl);
if (!REALLY_FORMAT) {
return text;
}
// Parse the source text
const sourceFile = ts.createSourceFile('file.ts', text, ts.ScriptTarget.Latest, /*setParentPointers*/ true);
// Get the formatting edits on the input sources
const edits = (ts).formatting.formatDocument(sourceFile, getRuleProvider(tsfmt), tsfmt);
// Apply the edits on the input code
return applyEdits(text, edits);
function countParensCurly(text: string): number {
let cnt = 0;
for (let i = 0; i < text.length; i++) {
if (text.charAt(i) === '(' || text.charAt(i) === '{') {
cnt++;
}
if (text.charAt(i) === ')' || text.charAt(i) === '}') {
cnt--;
}
}
return cnt;
}
function repeatStr(s: string, cnt: number): string {
let r = '';
for (let i = 0; i < cnt; i++) {
r += s;
}
return r;
}
function preformat(text: string, endl: string): string {
const lines = text.split(endl);
let inComment = false;
let inCommentDeltaIndent = 0;
let indent = 0;
for (let i = 0; i < lines.length; i++) {
let line = lines[i].replace(/\s$/, '');
let repeat = false;
let lineIndent = 0;
do {
repeat = false;
if (line.substring(0, 4) === ' ') {
line = line.substring(4);
lineIndent++;
repeat = true;
}
if (line.charAt(0) === '\t') {
line = line.substring(1);
lineIndent++;
repeat = true;
}
} while (repeat);
if (line.length === 0) {
continue;
}
if (inComment) {
if (/\*\//.test(line)) {
inComment = false;
}
lines[i] = repeatStr('\t', lineIndent + inCommentDeltaIndent) + line;
continue;
}
if (/\/\*/.test(line)) {
inComment = true;
inCommentDeltaIndent = indent - lineIndent;
lines[i] = repeatStr('\t', indent) + line;
continue;
}
const cnt = countParensCurly(line);
let shouldUnindentAfter = false;
let shouldUnindentBefore = false;
if (cnt < 0) {
if (/[({]/.test(line)) {
shouldUnindentAfter = true;
} else {
shouldUnindentBefore = true;
}
} else if (cnt === 0) {
shouldUnindentBefore = /^\}/.test(line);
}
let shouldIndentAfter = false;
if (cnt > 0) {
shouldIndentAfter = true;
} else if (cnt === 0) {
shouldIndentAfter = /{$/.test(line);
}
if (shouldUnindentBefore) {
indent--;
}
lines[i] = repeatStr('\t', indent) + line;
if (shouldUnindentAfter) {
indent--;
}
if (shouldIndentAfter) {
indent++;
}
}
return lines.join(endl);
}
function getRuleProvider(options: ts.FormatCodeSettings) {
// Share this between multiple formatters using the same options.
// This represents the bulk of the space the formatter uses.
return (ts as any).formatting.getFormatContext(options);
}
function applyEdits(text: string, edits: ts.TextChange[]): string {
// Apply edits in reverse on the existing text
let result = text;
for (let i = edits.length - 1; i >= 0; i--) {
const change = edits[i];
const head = result.slice(0, change.span.start);
const tail = result.slice(change.span.start + change.span.length);
result = head + change.newText + tail;
}
return result;
}
}
function createReplacerFromDirectives(directives: [RegExp, string][]): (str: string) => string {
return (str: string) => {
for (let i = 0; i < directives.length; i++) {
str = str.replace(directives[i][0], directives[i][1]);
}
return str;
};
}
function createReplacer(data: string): (str: string) => string {
data = data || '';
const rawDirectives = data.split(';');
const directives: [RegExp, string][] = [];
rawDirectives.forEach((rawDirective) => {
if (rawDirective.length === 0) {
return;
}
const pieces = rawDirective.split('=>');
let findStr = pieces[0];
const replaceStr = pieces[1];
findStr = findStr.replace(/[\-\\\{\}\*\+\?\|\^\$\.\,\[\]\(\)\#\s]/g, '\\$&');
findStr = '\\b' + findStr + '\\b';
directives.push([new RegExp(findStr, 'g'), replaceStr]);
});
return createReplacerFromDirectives(directives);
}
interface ITempResult {
result: string;
usageContent: string;
enums: string;
}
interface IEnumEntry {
enumName: string;
text: string;
}
function generateDeclarationFile(ts: typeof import('typescript'), recipe: string, sourceFileGetter: SourceFileGetter): ITempResult | null {
const endl = /\r\n/.test(recipe) ? '\r\n' : '\n';
const lines = recipe.split(endl);
const result: string[] = [];
let usageCounter = 0;
const usageImports: string[] = [];
const usage: string[] = [];
let failed = false;
usage.push(`var a: any;`);
usage.push(`var b: any;`);
const generateUsageImport = (moduleId: string) => {
const importName = 'm' + (++usageCounter);
usageImports.push(`import * as ${importName} from './${moduleId.replace(/\.d\.ts$/, '')}';`);
return importName;
};
const enums: IEnumEntry[] = [];
let version: string | null = null;
lines.forEach(line => {
if (failed) {
return;
}
const m0 = line.match(/^\/\/dtsv=(\d+)$/);
if (m0) {
version = m0[1];
}
const m1 = line.match(/^\s*#include\(([^;)]*)(;[^)]*)?\)\:(.*)$/);
if (m1) {
const moduleId = m1[1];
const sourceFile = sourceFileGetter(moduleId);
if (!sourceFile) {
logErr(`While handling ${line}`);
logErr(`Cannot find ${moduleId}`);
failed = true;
return;
}
const importName = generateUsageImport(moduleId);
const replacer = createReplacer(m1[2]);
const typeNames = m1[3].split(/,/);
typeNames.forEach((typeName) => {
typeName = typeName.trim();
if (typeName.length === 0) {
return;
}
const declaration = getTopLevelDeclaration(ts, sourceFile, typeName);
if (!declaration) {
logErr(`While handling ${line}`);
logErr(`Cannot find ${typeName}`);
failed = true;
return;
}
result.push(replacer(getMassagedTopLevelDeclarationText(ts, sourceFile, declaration, importName, usage, enums)));
});
return;
}
const m2 = line.match(/^\s*#includeAll\(([^;)]*)(;[^)]*)?\)\:(.*)$/);
if (m2) {
const moduleId = m2[1];
const sourceFile = sourceFileGetter(moduleId);
if (!sourceFile) {
logErr(`While handling ${line}`);
logErr(`Cannot find ${moduleId}`);
failed = true;
return;
}
const importName = generateUsageImport(moduleId);
const replacer = createReplacer(m2[2]);
const typeNames = m2[3].split(/,/);
const typesToExcludeMap: { [typeName: string]: boolean } = {};
const typesToExcludeArr: string[] = [];
typeNames.forEach((typeName) => {
typeName = typeName.trim();
if (typeName.length === 0) {
return;
}
typesToExcludeMap[typeName] = true;
typesToExcludeArr.push(typeName);
});
getAllTopLevelDeclarations(ts, sourceFile).forEach((declaration) => {
if (isDeclaration(ts, declaration) && declaration.name) {
if (typesToExcludeMap[declaration.name.text]) {
return;
}
} else {
// node is ts.VariableStatement
const nodeText = getNodeText(sourceFile, declaration);
for (let i = 0; i < typesToExcludeArr.length; i++) {
if (nodeText.indexOf(typesToExcludeArr[i]) >= 0) {
return;
}
}
}
result.push(replacer(getMassagedTopLevelDeclarationText(ts, sourceFile, declaration, importName, usage, enums)));
});
return;
}
result.push(line);
});
if (failed) {
return null;
}
if (version !== dtsv) {
if (!version) {
logErr(`gulp watch restart required. 'monaco.d.ts.recipe' is written before versioning was introduced.`);
} else {
logErr(`gulp watch restart required. 'monaco.d.ts.recipe' v${version} does not match runtime v${dtsv}.`);
}
return null;
}
let resultTxt = result.join(endl);
resultTxt = resultTxt.replace(/\bURI\b/g, 'Uri');
resultTxt = resultTxt.replace(/\bEvent {
if (e1.enumName < e2.enumName) {
return -1;
}
if (e1.enumName > e2.enumName) {
return 1;
}
return 0;
});
let resultEnums = [
'/*---------------------------------------------------------------------------------------------',
' * Copyright (c) Microsoft Corporation. All rights reserved.',
' * Licensed under the MIT License. See License.txt in the project root for license information.',
' *--------------------------------------------------------------------------------------------*/',
'',
'// THIS IS A GENERATED FILE. DO NOT EDIT DIRECTLY.',
''
].concat(enums.map(e => e.text)).join(endl);
resultEnums = resultEnums.split(/\r\n|\n|\r/).join(endl);
resultEnums = format(ts, resultEnums, endl);
resultEnums = resultEnums.split(/\r\n|\n|\r/).join(endl);
return {
result: resultTxt,
usageContent: `${usageImports.join('\n')}\n\n${usage.join('\n')}`,
enums: resultEnums
};
}
export interface IMonacoDeclarationResult {
content: string;
usageContent: string;
enums: string;
filePath: string;
isTheSame: boolean;
}
function _run(ts: typeof import('typescript'), sourceFileGetter: SourceFileGetter): IMonacoDeclarationResult | null {
const recipe = fs.readFileSync(RECIPE_PATH).toString();
const t = generateDeclarationFile(ts, recipe, sourceFileGetter);
if (!t) {
return null;
}
const result = t.result;
const usageContent = t.usageContent;
const enums = t.enums;
const currentContent = fs.readFileSync(DECLARATION_PATH).toString();
const one = currentContent.replace(/\r\n/gm, '\n');
const other = result.replace(/\r\n/gm, '\n');
const isTheSame = (one === other);
return {
content: result,
usageContent: usageContent,
enums: enums,
filePath: DECLARATION_PATH,
isTheSame
};
}
export class FSProvider {
public existsSync(filePath: string): boolean {
return fs.existsSync(filePath);
}
public statSync(filePath: string): fs.Stats {
return fs.statSync(filePath);
}
public readFileSync(_moduleId: string, filePath: string): Buffer {
return fs.readFileSync(filePath);
}
}
class CacheEntry {
constructor(
public readonly sourceFile: ts.SourceFile,
public readonly mtime: number
) { }
}
export class DeclarationResolver {
public readonly ts: typeof import('typescript');
private _sourceFileCache: { [moduleId: string]: CacheEntry | null };
constructor(private readonly _fsProvider: FSProvider) {
this.ts = require('typescript') as typeof import('typescript');
this._sourceFileCache = Object.create(null);
}
public invalidateCache(moduleId: string): void {
this._sourceFileCache[moduleId] = null;
}
public getDeclarationSourceFile(moduleId: string): ts.SourceFile | null {
if (this._sourceFileCache[moduleId]) {
// Since we cannot trust file watching to invalidate the cache, check also the mtime
const fileName = this._getFileName(moduleId);
const mtime = this._fsProvider.statSync(fileName).mtime.getTime();
if (this._sourceFileCache[moduleId]!.mtime !== mtime) {
this._sourceFileCache[moduleId] = null;
}
}
if (!this._sourceFileCache[moduleId]) {
this._sourceFileCache[moduleId] = this._getDeclarationSourceFile(moduleId);
}
return this._sourceFileCache[moduleId] ? this._sourceFileCache[moduleId]!.sourceFile : null;
}
private _getFileName(moduleId: string): string {
if (/\.d\.ts$/.test(moduleId)) {
return path.join(SRC, moduleId);
}
return path.join(SRC, `${moduleId}.ts`);
}
private _getDeclarationSourceFile(moduleId: string): CacheEntry | null {
const fileName = this._getFileName(moduleId);
if (!this._fsProvider.existsSync(fileName)) {
return null;
}
const mtime = this._fsProvider.statSync(fileName).mtime.getTime();
if (/\.d\.ts$/.test(moduleId)) {
// const mtime = this._fsProvider.statFileSync()
const fileContents = this._fsProvider.readFileSync(moduleId, fileName).toString();
return new CacheEntry(
this.ts.createSourceFile(fileName, fileContents, this.ts.ScriptTarget.ES5),
mtime
);
}
const fileContents = this._fsProvider.readFileSync(moduleId, fileName).toString();
const fileMap: IFileMap = {
'file.ts': fileContents
};
const service = this.ts.createLanguageService(new TypeScriptLanguageServiceHost(this.ts, {}, fileMap, {}));
const text = service.getEmitOutput('file.ts', true, true).outputFiles[0].text;
return new CacheEntry(
this.ts.createSourceFile(fileName, text, this.ts.ScriptTarget.ES5),
mtime
);
}
}
export function run3(resolver: DeclarationResolver): IMonacoDeclarationResult | null {
const sourceFileGetter = (moduleId: string) => resolver.getDeclarationSourceFile(moduleId);
return _run(resolver.ts, sourceFileGetter);
}
interface ILibMap { [libName: string]: string }
interface IFileMap { [fileName: string]: string }
class TypeScriptLanguageServiceHost implements ts.LanguageServiceHost {
private readonly _ts: typeof import('typescript');
private readonly _libs: ILibMap;
private readonly _files: IFileMap;
private readonly _compilerOptions: ts.CompilerOptions;
constructor(ts: typeof import('typescript'), libs: ILibMap, files: IFileMap, compilerOptions: ts.CompilerOptions) {
this._ts = ts;
this._libs = libs;
this._files = files;
this._compilerOptions = compilerOptions;
}
// --- language service host ---------------
getCompilationSettings(): ts.CompilerOptions {
return this._compilerOptions;
}
getScriptFileNames(): string[] {
return (
([] as string[])
.concat(Object.keys(this._libs))
.concat(Object.keys(this._files))
);
}
getScriptVersion(_fileName: string): string {
return '1';
}
getProjectVersion(): string {
return '1';
}
getScriptSnapshot(fileName: string): ts.IScriptSnapshot {
if (this._files.hasOwnProperty(fileName)) {
return this._ts.ScriptSnapshot.fromString(this._files[fileName]);
} else if (this._libs.hasOwnProperty(fileName)) {
return this._ts.ScriptSnapshot.fromString(this._libs[fileName]);
} else {
return this._ts.ScriptSnapshot.fromString('');
}
}
getScriptKind(_fileName: string): ts.ScriptKind {
return this._ts.ScriptKind.TS;
}
getCurrentDirectory(): string {
return '';
}
getDefaultLibFileName(_options: ts.CompilerOptions): string {
return 'defaultLib:es5';
}
isDefaultLibFileName(fileName: string): boolean {
return fileName === this.getDefaultLibFileName(this._compilerOptions);
}
readFile(path: string, _encoding?: string): string | undefined {
return this._files[path] || this._libs[path];
}
fileExists(path: string): boolean {
return path in this._files || path in this._libs;
}
}
export function execute(): IMonacoDeclarationResult {
const r = run3(new DeclarationResolver(new FSProvider()));
if (!r) {
throw new Error(`monaco.d.ts generation error - Cannot continue`);
}
return r;
}
================================================
FILE: build/lib/nls.js
================================================
"use strict";
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
Object.defineProperty(exports, "__esModule", { value: true });
exports.nls = nls;
const lazy_js_1 = __importDefault(require("lazy.js"));
const event_stream_1 = require("event-stream");
const vinyl_1 = __importDefault(require("vinyl"));
const source_map_1 = __importDefault(require("source-map"));
const path_1 = __importDefault(require("path"));
const gulp_sort_1 = __importDefault(require("gulp-sort"));
var CollectStepResult;
(function (CollectStepResult) {
CollectStepResult[CollectStepResult["Yes"] = 0] = "Yes";
CollectStepResult[CollectStepResult["YesAndRecurse"] = 1] = "YesAndRecurse";
CollectStepResult[CollectStepResult["No"] = 2] = "No";
CollectStepResult[CollectStepResult["NoAndRecurse"] = 3] = "NoAndRecurse";
})(CollectStepResult || (CollectStepResult = {}));
function collect(ts, node, fn) {
const result = [];
function loop(node) {
const stepResult = fn(node);
if (stepResult === CollectStepResult.Yes || stepResult === CollectStepResult.YesAndRecurse) {
result.push(node);
}
if (stepResult === CollectStepResult.YesAndRecurse || stepResult === CollectStepResult.NoAndRecurse) {
ts.forEachChild(node, loop);
}
}
loop(node);
return result;
}
function clone(object) {
const result = {};
for (const id in object) {
result[id] = object[id];
}
return result;
}
/**
* Returns a stream containing the patched JavaScript and source maps.
*/
function nls(options) {
let base;
const input = (0, event_stream_1.through)();
const output = input
.pipe((0, gulp_sort_1.default)()) // IMPORTANT: to ensure stable NLS metadata generation, we must sort the files because NLS messages are globally extracted and indexed across all files
.pipe((0, event_stream_1.through)(function (f) {
if (!f.sourceMap) {
return this.emit('error', new Error(`File ${f.relative} does not have sourcemaps.`));
}
let source = f.sourceMap.sources[0];
if (!source) {
return this.emit('error', new Error(`File ${f.relative} does not have a source in the source map.`));
}
const root = f.sourceMap.sourceRoot;
if (root) {
source = path_1.default.join(root, source);
}
const typescript = f.sourceMap.sourcesContent[0];
if (!typescript) {
return this.emit('error', new Error(`File ${f.relative} does not have the original content in the source map.`));
}
base = f.base;
this.emit('data', _nls.patchFile(f, typescript, options));
}, function () {
for (const file of [
new vinyl_1.default({
contents: Buffer.from(JSON.stringify({
keys: _nls.moduleToNLSKeys,
messages: _nls.moduleToNLSMessages,
}, null, '\t')),
base,
path: `${base}/nls.metadata.json`
}),
new vinyl_1.default({
contents: Buffer.from(JSON.stringify(_nls.allNLSMessages)),
base,
path: `${base}/nls.messages.json`
}),
new vinyl_1.default({
contents: Buffer.from(JSON.stringify(_nls.allNLSModulesAndKeys)),
base,
path: `${base}/nls.keys.json`
}),
new vinyl_1.default({
contents: Buffer.from(`/*---------------------------------------------------------
* Copyright (C) Microsoft Corporation. All rights reserved.
*--------------------------------------------------------*/
globalThis._VSCODE_NLS_MESSAGES=${JSON.stringify(_nls.allNLSMessages)};`),
base,
path: `${base}/nls.messages.js`
})
]) {
this.emit('data', file);
}
this.emit('end');
}));
return (0, event_stream_1.duplex)(input, output);
}
function isImportNode(ts, node) {
return node.kind === ts.SyntaxKind.ImportDeclaration || node.kind === ts.SyntaxKind.ImportEqualsDeclaration;
}
var _nls;
(function (_nls) {
_nls.moduleToNLSKeys = {};
_nls.moduleToNLSMessages = {};
_nls.allNLSMessages = [];
_nls.allNLSModulesAndKeys = [];
let allNLSMessagesIndex = 0;
function fileFrom(file, contents, path = file.path) {
return new vinyl_1.default({
contents: Buffer.from(contents),
base: file.base,
cwd: file.cwd,
path: path
});
}
function mappedPositionFrom(source, lc) {
return { source, line: lc.line + 1, column: lc.character };
}
function lcFrom(position) {
return { line: position.line - 1, character: position.column };
}
class SingleFileServiceHost {
options;
filename;
file;
lib;
constructor(ts, options, filename, contents) {
this.options = options;
this.filename = filename;
this.file = ts.ScriptSnapshot.fromString(contents);
this.lib = ts.ScriptSnapshot.fromString('');
}
getCompilationSettings = () => this.options;
getScriptFileNames = () => [this.filename];
getScriptVersion = () => '1';
getScriptSnapshot = (name) => name === this.filename ? this.file : this.lib;
getCurrentDirectory = () => '';
getDefaultLibFileName = () => 'lib.d.ts';
readFile(path, _encoding) {
if (path === this.filename) {
return this.file.getText(0, this.file.getLength());
}
return undefined;
}
fileExists(path) {
return path === this.filename;
}
}
function isCallExpressionWithinTextSpanCollectStep(ts, textSpan, node) {
if (!ts.textSpanContainsTextSpan({ start: node.pos, length: node.end - node.pos }, textSpan)) {
return CollectStepResult.No;
}
return node.kind === ts.SyntaxKind.CallExpression ? CollectStepResult.YesAndRecurse : CollectStepResult.NoAndRecurse;
}
function analyze(ts, contents, functionName, options = {}) {
const filename = 'file.ts';
const serviceHost = new SingleFileServiceHost(ts, Object.assign(clone(options), { noResolve: true }), filename, contents);
const service = ts.createLanguageService(serviceHost);
const sourceFile = ts.createSourceFile(filename, contents, ts.ScriptTarget.ES5, true);
// all imports
const imports = (0, lazy_js_1.default)(collect(ts, sourceFile, n => isImportNode(ts, n) ? CollectStepResult.YesAndRecurse : CollectStepResult.NoAndRecurse));
// import nls = require('vs/nls');
const importEqualsDeclarations = imports
.filter(n => n.kind === ts.SyntaxKind.ImportEqualsDeclaration)
.map(n => n)
.filter(d => d.moduleReference.kind === ts.SyntaxKind.ExternalModuleReference)
.filter(d => d.moduleReference.expression.getText().endsWith(`/nls.js'`));
// import ... from 'vs/nls';
const importDeclarations = imports
.filter(n => n.kind === ts.SyntaxKind.ImportDeclaration)
.map(n => n)
.filter(d => d.moduleSpecifier.kind === ts.SyntaxKind.StringLiteral)
.filter(d => d.moduleSpecifier.getText().endsWith(`/nls.js'`))
.filter(d => !!d.importClause && !!d.importClause.namedBindings);
// `nls.localize(...)` calls
const nlsLocalizeCallExpressions = importDeclarations
.filter(d => !!(d.importClause && d.importClause.namedBindings && d.importClause.namedBindings.kind === ts.SyntaxKind.NamespaceImport))
.map(d => d.importClause.namedBindings.name)
.concat(importEqualsDeclarations.map(d => d.name))
// find read-only references to `nls`
.map(n => service.getReferencesAtPosition(filename, n.pos + 1))
.flatten()
.filter(r => !r.isWriteAccess)
// find the deepest call expressions AST nodes that contain those references
.map(r => collect(ts, sourceFile, n => isCallExpressionWithinTextSpanCollectStep(ts, r.textSpan, n)))
.map(a => (0, lazy_js_1.default)(a).last())
.filter(n => !!n)
.map(n => n)
// only `localize` calls
.filter(n => n.expression.kind === ts.SyntaxKind.PropertyAccessExpression && n.expression.name.getText() === functionName);
// `localize` named imports
const allLocalizeImportDeclarations = importDeclarations
.filter(d => !!(d.importClause && d.importClause.namedBindings && d.importClause.namedBindings.kind === ts.SyntaxKind.NamedImports))
.map(d => [].concat(d.importClause.namedBindings.elements))
.flatten();
// `localize` read-only references
const localizeReferences = allLocalizeImportDeclarations
.filter(d => d.name.getText() === functionName)
.map(n => service.getReferencesAtPosition(filename, n.pos + 1))
.flatten()
.filter(r => !r.isWriteAccess);
// custom named `localize` read-only references
const namedLocalizeReferences = allLocalizeImportDeclarations
.filter(d => d.propertyName && d.propertyName.getText() === functionName)
.map(n => service.getReferencesAtPosition(filename, n.name.pos + 1))
.flatten()
.filter(r => !r.isWriteAccess);
// find the deepest call expressions AST nodes that contain those references
const localizeCallExpressions = localizeReferences
.concat(namedLocalizeReferences)
.map(r => collect(ts, sourceFile, n => isCallExpressionWithinTextSpanCollectStep(ts, r.textSpan, n)))
.map(a => (0, lazy_js_1.default)(a).last())
.filter(n => !!n)
.map(n => n);
// collect everything
const localizeCalls = nlsLocalizeCallExpressions
.concat(localizeCallExpressions)
.map(e => e.arguments)
.filter(a => a.length > 1)
.sort((a, b) => a[0].getStart() - b[0].getStart())
.map(a => ({
keySpan: { start: ts.getLineAndCharacterOfPosition(sourceFile, a[0].getStart()), end: ts.getLineAndCharacterOfPosition(sourceFile, a[0].getEnd()) },
key: a[0].getText(),
valueSpan: { start: ts.getLineAndCharacterOfPosition(sourceFile, a[1].getStart()), end: ts.getLineAndCharacterOfPosition(sourceFile, a[1].getEnd()) },
value: a[1].getText()
}));
return {
localizeCalls: localizeCalls.toArray()
};
}
class TextModel {
lines;
lineEndings;
constructor(contents) {
const regex = /\r\n|\r|\n/g;
let index = 0;
let match;
this.lines = [];
this.lineEndings = [];
while (match = regex.exec(contents)) {
this.lines.push(contents.substring(index, match.index));
this.lineEndings.push(match[0]);
index = regex.lastIndex;
}
if (contents.length > 0) {
this.lines.push(contents.substring(index, contents.length));
this.lineEndings.push('');
}
}
get(index) {
return this.lines[index];
}
set(index, line) {
this.lines[index] = line;
}
get lineCount() {
return this.lines.length;
}
/**
* Applies patch(es) to the model.
* Multiple patches must be ordered.
* Does not support patches spanning multiple lines.
*/
apply(patch) {
const startLineNumber = patch.span.start.line;
const endLineNumber = patch.span.end.line;
const startLine = this.lines[startLineNumber] || '';
const endLine = this.lines[endLineNumber] || '';
this.lines[startLineNumber] = [
startLine.substring(0, patch.span.start.character),
patch.content,
endLine.substring(patch.span.end.character)
].join('');
for (let i = startLineNumber + 1; i <= endLineNumber; i++) {
this.lines[i] = '';
}
}
toString() {
return (0, lazy_js_1.default)(this.lines).zip(this.lineEndings)
.flatten().toArray().join('');
}
}
function patchJavascript(patches, contents) {
const model = new TextModel(contents);
// patch the localize calls
(0, lazy_js_1.default)(patches).reverse().each(p => model.apply(p));
return model.toString();
}
function patchSourcemap(patches, rsm, smc) {
const smg = new source_map_1.default.SourceMapGenerator({
file: rsm.file,
sourceRoot: rsm.sourceRoot
});
patches = patches.reverse();
let currentLine = -1;
let currentLineDiff = 0;
let source = null;
smc.eachMapping(m => {
const patch = patches[patches.length - 1];
const original = { line: m.originalLine, column: m.originalColumn };
const generated = { line: m.generatedLine, column: m.generatedColumn };
if (currentLine !== generated.line) {
currentLineDiff = 0;
}
currentLine = generated.line;
generated.column += currentLineDiff;
if (patch && m.generatedLine - 1 === patch.span.end.line && m.generatedColumn === patch.span.end.character) {
const originalLength = patch.span.end.character - patch.span.start.character;
const modifiedLength = patch.content.length;
const lengthDiff = modifiedLength - originalLength;
currentLineDiff += lengthDiff;
generated.column += lengthDiff;
patches.pop();
}
source = rsm.sourceRoot ? path_1.default.relative(rsm.sourceRoot, m.source) : m.source;
source = source.replace(/\\/g, '/');
smg.addMapping({ source, name: m.name, original, generated });
}, null, source_map_1.default.SourceMapConsumer.GENERATED_ORDER);
if (source) {
smg.setSourceContent(source, smc.sourceContentFor(source));
}
return JSON.parse(smg.toString());
}
function parseLocalizeKeyOrValue(sourceExpression) {
// sourceValue can be "foo", 'foo', `foo` or { .... }
// in its evalulated form
// we want to return either the string or the object
// eslint-disable-next-line no-eval
return eval(`(${sourceExpression})`);
}
function patch(ts, typescript, javascript, sourcemap, options) {
const { localizeCalls } = analyze(ts, typescript, 'localize');
const { localizeCalls: localize2Calls } = analyze(ts, typescript, 'localize2');
if (localizeCalls.length === 0 && localize2Calls.length === 0) {
return { javascript, sourcemap };
}
const nlsKeys = localizeCalls.map(lc => parseLocalizeKeyOrValue(lc.key)).concat(localize2Calls.map(lc => parseLocalizeKeyOrValue(lc.key)));
const nlsMessages = localizeCalls.map(lc => parseLocalizeKeyOrValue(lc.value)).concat(localize2Calls.map(lc => parseLocalizeKeyOrValue(lc.value)));
const smc = new source_map_1.default.SourceMapConsumer(sourcemap);
const positionFrom = mappedPositionFrom.bind(null, sourcemap.sources[0]);
// build patches
const toPatch = (c) => {
const start = lcFrom(smc.generatedPositionFor(positionFrom(c.range.start)));
const end = lcFrom(smc.generatedPositionFor(positionFrom(c.range.end)));
return { span: { start, end }, content: c.content };
};
const localizePatches = (0, lazy_js_1.default)(localizeCalls)
.map(lc => (options.preserveEnglish ? [
{ range: lc.keySpan, content: `${allNLSMessagesIndex++}` } // localize('key', "message") => localize(, "message")
] : [
{ range: lc.keySpan, content: `${allNLSMessagesIndex++}` }, // localize('key', "message") => localize(, null)
{ range: lc.valueSpan, content: 'null' }
]))
.flatten()
.map(toPatch);
const localize2Patches = (0, lazy_js_1.default)(localize2Calls)
.map(lc => ({ range: lc.keySpan, content: `${allNLSMessagesIndex++}` } // localize2('key', "message") => localize(, "message")
))
.map(toPatch);
// Sort patches by their start position
const patches = localizePatches.concat(localize2Patches).toArray().sort((a, b) => {
if (a.span.start.line < b.span.start.line) {
return -1;
}
else if (a.span.start.line > b.span.start.line) {
return 1;
}
else if (a.span.start.character < b.span.start.character) {
return -1;
}
else if (a.span.start.character > b.span.start.character) {
return 1;
}
else {
return 0;
}
});
javascript = patchJavascript(patches, javascript);
sourcemap = patchSourcemap(patches, sourcemap, smc);
return { javascript, sourcemap, nlsKeys, nlsMessages };
}
function patchFile(javascriptFile, typescript, options) {
const ts = require('typescript');
// hack?
const moduleId = javascriptFile.relative
.replace(/\.js$/, '')
.replace(/\\/g, '/');
const { javascript, sourcemap, nlsKeys, nlsMessages } = patch(ts, typescript, javascriptFile.contents.toString(), javascriptFile.sourceMap, options);
const result = fileFrom(javascriptFile, javascript);
result.sourceMap = sourcemap;
if (nlsKeys) {
_nls.moduleToNLSKeys[moduleId] = nlsKeys;
_nls.allNLSModulesAndKeys.push([moduleId, nlsKeys.map(nlsKey => typeof nlsKey === 'string' ? nlsKey : nlsKey.key)]);
}
if (nlsMessages) {
_nls.moduleToNLSMessages[moduleId] = nlsMessages;
_nls.allNLSMessages.push(...nlsMessages);
}
return result;
}
_nls.patchFile = patchFile;
})(_nls || (_nls = {}));
//# sourceMappingURL=nls.js.map
================================================
FILE: build/lib/nls.ts
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
import type * as ts from 'typescript';
import lazy from 'lazy.js';
import { duplex, through } from 'event-stream';
import File from 'vinyl';
import sm from 'source-map';
import path from 'path';
import sort from 'gulp-sort';
declare class FileSourceMap extends File {
public sourceMap: sm.RawSourceMap;
}
enum CollectStepResult {
Yes,
YesAndRecurse,
No,
NoAndRecurse
}
function collect(ts: typeof import('typescript'), node: ts.Node, fn: (node: ts.Node) => CollectStepResult): ts.Node[] {
const result: ts.Node[] = [];
function loop(node: ts.Node) {
const stepResult = fn(node);
if (stepResult === CollectStepResult.Yes || stepResult === CollectStepResult.YesAndRecurse) {
result.push(node);
}
if (stepResult === CollectStepResult.YesAndRecurse || stepResult === CollectStepResult.NoAndRecurse) {
ts.forEachChild(node, loop);
}
}
loop(node);
return result;
}
function clone(object: T): T {
const result = {} as any as T;
for (const id in object) {
result[id] = object[id];
}
return result;
}
/**
* Returns a stream containing the patched JavaScript and source maps.
*/
export function nls(options: { preserveEnglish: boolean }): NodeJS.ReadWriteStream {
let base: string;
const input = through();
const output = input
.pipe(sort()) // IMPORTANT: to ensure stable NLS metadata generation, we must sort the files because NLS messages are globally extracted and indexed across all files
.pipe(through(function (f: FileSourceMap) {
if (!f.sourceMap) {
return this.emit('error', new Error(`File ${f.relative} does not have sourcemaps.`));
}
let source = f.sourceMap.sources[0];
if (!source) {
return this.emit('error', new Error(`File ${f.relative} does not have a source in the source map.`));
}
const root = f.sourceMap.sourceRoot;
if (root) {
source = path.join(root, source);
}
const typescript = f.sourceMap.sourcesContent![0];
if (!typescript) {
return this.emit('error', new Error(`File ${f.relative} does not have the original content in the source map.`));
}
base = f.base;
this.emit('data', _nls.patchFile(f, typescript, options));
}, function () {
for (const file of [
new File({
contents: Buffer.from(JSON.stringify({
keys: _nls.moduleToNLSKeys,
messages: _nls.moduleToNLSMessages,
}, null, '\t')),
base,
path: `${base}/nls.metadata.json`
}),
new File({
contents: Buffer.from(JSON.stringify(_nls.allNLSMessages)),
base,
path: `${base}/nls.messages.json`
}),
new File({
contents: Buffer.from(JSON.stringify(_nls.allNLSModulesAndKeys)),
base,
path: `${base}/nls.keys.json`
}),
new File({
contents: Buffer.from(`/*---------------------------------------------------------
* Copyright (C) Microsoft Corporation. All rights reserved.
*--------------------------------------------------------*/
globalThis._VSCODE_NLS_MESSAGES=${JSON.stringify(_nls.allNLSMessages)};`),
base,
path: `${base}/nls.messages.js`
})
]) {
this.emit('data', file);
}
this.emit('end');
}));
return duplex(input, output);
}
function isImportNode(ts: typeof import('typescript'), node: ts.Node): boolean {
return node.kind === ts.SyntaxKind.ImportDeclaration || node.kind === ts.SyntaxKind.ImportEqualsDeclaration;
}
module _nls {
export const moduleToNLSKeys: { [name: string /* module ID */]: ILocalizeKey[] /* keys */ } = {};
export const moduleToNLSMessages: { [name: string /* module ID */]: string[] /* messages */ } = {};
export const allNLSMessages: string[] = [];
export const allNLSModulesAndKeys: Array<[string /* module ID */, string[] /* keys */]> = [];
let allNLSMessagesIndex = 0;
type ILocalizeKey = string | { key: string }; // key might contain metadata for translators and then is not just a string
interface INlsPatchResult {
javascript: string;
sourcemap: sm.RawSourceMap;
nlsMessages?: string[];
nlsKeys?: ILocalizeKey[];
}
interface ISpan {
start: ts.LineAndCharacter;
end: ts.LineAndCharacter;
}
interface ILocalizeCall {
keySpan: ISpan;
key: string;
valueSpan: ISpan;
value: string;
}
interface ILocalizeAnalysisResult {
localizeCalls: ILocalizeCall[];
}
interface IPatch {
span: ISpan;
content: string;
}
function fileFrom(file: File, contents: string, path: string = file.path) {
return new File({
contents: Buffer.from(contents),
base: file.base,
cwd: file.cwd,
path: path
});
}
function mappedPositionFrom(source: string, lc: ts.LineAndCharacter): sm.MappedPosition {
return { source, line: lc.line + 1, column: lc.character };
}
function lcFrom(position: sm.Position): ts.LineAndCharacter {
return { line: position.line - 1, character: position.column };
}
class SingleFileServiceHost implements ts.LanguageServiceHost {
private file: ts.IScriptSnapshot;
private lib: ts.IScriptSnapshot;
constructor(ts: typeof import('typescript'), private options: ts.CompilerOptions, private filename: string, contents: string) {
this.file = ts.ScriptSnapshot.fromString(contents);
this.lib = ts.ScriptSnapshot.fromString('');
}
getCompilationSettings = () => this.options;
getScriptFileNames = () => [this.filename];
getScriptVersion = () => '1';
getScriptSnapshot = (name: string) => name === this.filename ? this.file : this.lib;
getCurrentDirectory = () => '';
getDefaultLibFileName = () => 'lib.d.ts';
readFile(path: string, _encoding?: string): string | undefined {
if (path === this.filename) {
return this.file.getText(0, this.file.getLength());
}
return undefined;
}
fileExists(path: string): boolean {
return path === this.filename;
}
}
function isCallExpressionWithinTextSpanCollectStep(ts: typeof import('typescript'), textSpan: ts.TextSpan, node: ts.Node): CollectStepResult {
if (!ts.textSpanContainsTextSpan({ start: node.pos, length: node.end - node.pos }, textSpan)) {
return CollectStepResult.No;
}
return node.kind === ts.SyntaxKind.CallExpression ? CollectStepResult.YesAndRecurse : CollectStepResult.NoAndRecurse;
}
function analyze(
ts: typeof import('typescript'),
contents: string,
functionName: 'localize' | 'localize2',
options: ts.CompilerOptions = {}
): ILocalizeAnalysisResult {
const filename = 'file.ts';
const serviceHost = new SingleFileServiceHost(ts, Object.assign(clone(options), { noResolve: true }), filename, contents);
const service = ts.createLanguageService(serviceHost);
const sourceFile = ts.createSourceFile(filename, contents, ts.ScriptTarget.ES5, true);
// all imports
const imports = lazy(collect(ts, sourceFile, n => isImportNode(ts, n) ? CollectStepResult.YesAndRecurse : CollectStepResult.NoAndRecurse));
// import nls = require('vs/nls');
const importEqualsDeclarations = imports
.filter(n => n.kind === ts.SyntaxKind.ImportEqualsDeclaration)
.map(n => n)
.filter(d => d.moduleReference.kind === ts.SyntaxKind.ExternalModuleReference)
.filter(d => (d.moduleReference).expression.getText().endsWith(`/nls.js'`));
// import ... from 'vs/nls';
const importDeclarations = imports
.filter(n => n.kind === ts.SyntaxKind.ImportDeclaration)
.map(n => n)
.filter(d => d.moduleSpecifier.kind === ts.SyntaxKind.StringLiteral)
.filter(d => d.moduleSpecifier.getText().endsWith(`/nls.js'`))
.filter(d => !!d.importClause && !!d.importClause.namedBindings);
// `nls.localize(...)` calls
const nlsLocalizeCallExpressions = importDeclarations
.filter(d => !!(d.importClause && d.importClause.namedBindings && d.importClause.namedBindings.kind === ts.SyntaxKind.NamespaceImport))
.map(d => (d.importClause!.namedBindings).name)
.concat(importEqualsDeclarations.map(d => d.name))
// find read-only references to `nls`
.map(n => service.getReferencesAtPosition(filename, n.pos + 1))
.flatten()
.filter(r => !r.isWriteAccess)
// find the deepest call expressions AST nodes that contain those references
.map(r => collect(ts, sourceFile, n => isCallExpressionWithinTextSpanCollectStep(ts, r.textSpan, n)))
.map(a => lazy(a).last())
.filter(n => !!n)
.map(n => n)
// only `localize` calls
.filter(n => n.expression.kind === ts.SyntaxKind.PropertyAccessExpression && (n.expression).name.getText() === functionName);
// `localize` named imports
const allLocalizeImportDeclarations = importDeclarations
.filter(d => !!(d.importClause && d.importClause.namedBindings && d.importClause.namedBindings.kind === ts.SyntaxKind.NamedImports))
.map(d => ([] as any[]).concat((d.importClause!.namedBindings!).elements))
.flatten();
// `localize` read-only references
const localizeReferences = allLocalizeImportDeclarations
.filter(d => d.name.getText() === functionName)
.map(n => service.getReferencesAtPosition(filename, n.pos + 1))
.flatten()
.filter(r => !r.isWriteAccess);
// custom named `localize` read-only references
const namedLocalizeReferences = allLocalizeImportDeclarations
.filter(d => d.propertyName && d.propertyName.getText() === functionName)
.map(n => service.getReferencesAtPosition(filename, n.name.pos + 1))
.flatten()
.filter(r => !r.isWriteAccess);
// find the deepest call expressions AST nodes that contain those references
const localizeCallExpressions = localizeReferences
.concat(namedLocalizeReferences)
.map(r => collect(ts, sourceFile, n => isCallExpressionWithinTextSpanCollectStep(ts, r.textSpan, n)))
.map(a => lazy(a).last())
.filter(n => !!n)
.map(n => n);
// collect everything
const localizeCalls = nlsLocalizeCallExpressions
.concat(localizeCallExpressions)
.map(e => e.arguments)
.filter(a => a.length > 1)
.sort((a, b) => a[0].getStart() - b[0].getStart())
.map(a => ({
keySpan: { start: ts.getLineAndCharacterOfPosition(sourceFile, a[0].getStart()), end: ts.getLineAndCharacterOfPosition(sourceFile, a[0].getEnd()) },
key: a[0].getText(),
valueSpan: { start: ts.getLineAndCharacterOfPosition(sourceFile, a[1].getStart()), end: ts.getLineAndCharacterOfPosition(sourceFile, a[1].getEnd()) },
value: a[1].getText()
}));
return {
localizeCalls: localizeCalls.toArray()
};
}
class TextModel {
private lines: string[];
private lineEndings: string[];
constructor(contents: string) {
const regex = /\r\n|\r|\n/g;
let index = 0;
let match: RegExpExecArray | null;
this.lines = [];
this.lineEndings = [];
while (match = regex.exec(contents)) {
this.lines.push(contents.substring(index, match.index));
this.lineEndings.push(match[0]);
index = regex.lastIndex;
}
if (contents.length > 0) {
this.lines.push(contents.substring(index, contents.length));
this.lineEndings.push('');
}
}
public get(index: number): string {
return this.lines[index];
}
public set(index: number, line: string): void {
this.lines[index] = line;
}
public get lineCount(): number {
return this.lines.length;
}
/**
* Applies patch(es) to the model.
* Multiple patches must be ordered.
* Does not support patches spanning multiple lines.
*/
public apply(patch: IPatch): void {
const startLineNumber = patch.span.start.line;
const endLineNumber = patch.span.end.line;
const startLine = this.lines[startLineNumber] || '';
const endLine = this.lines[endLineNumber] || '';
this.lines[startLineNumber] = [
startLine.substring(0, patch.span.start.character),
patch.content,
endLine.substring(patch.span.end.character)
].join('');
for (let i = startLineNumber + 1; i <= endLineNumber; i++) {
this.lines[i] = '';
}
}
public toString(): string {
return lazy(this.lines).zip(this.lineEndings)
.flatten().toArray().join('');
}
}
function patchJavascript(patches: IPatch[], contents: string): string {
const model = new TextModel(contents);
// patch the localize calls
lazy(patches).reverse().each(p => model.apply(p));
return model.toString();
}
function patchSourcemap(patches: IPatch[], rsm: sm.RawSourceMap, smc: sm.SourceMapConsumer): sm.RawSourceMap {
const smg = new sm.SourceMapGenerator({
file: rsm.file,
sourceRoot: rsm.sourceRoot
});
patches = patches.reverse();
let currentLine = -1;
let currentLineDiff = 0;
let source: string | null = null;
smc.eachMapping(m => {
const patch = patches[patches.length - 1];
const original = { line: m.originalLine, column: m.originalColumn };
const generated = { line: m.generatedLine, column: m.generatedColumn };
if (currentLine !== generated.line) {
currentLineDiff = 0;
}
currentLine = generated.line;
generated.column += currentLineDiff;
if (patch && m.generatedLine - 1 === patch.span.end.line && m.generatedColumn === patch.span.end.character) {
const originalLength = patch.span.end.character - patch.span.start.character;
const modifiedLength = patch.content.length;
const lengthDiff = modifiedLength - originalLength;
currentLineDiff += lengthDiff;
generated.column += lengthDiff;
patches.pop();
}
source = rsm.sourceRoot ? path.relative(rsm.sourceRoot, m.source) : m.source;
source = source.replace(/\\/g, '/');
smg.addMapping({ source, name: m.name, original, generated });
}, null, sm.SourceMapConsumer.GENERATED_ORDER);
if (source) {
smg.setSourceContent(source, smc.sourceContentFor(source));
}
return JSON.parse(smg.toString());
}
function parseLocalizeKeyOrValue(sourceExpression: string) {
// sourceValue can be "foo", 'foo', `foo` or { .... }
// in its evalulated form
// we want to return either the string or the object
// eslint-disable-next-line no-eval
return eval(`(${sourceExpression})`);
}
function patch(ts: typeof import('typescript'), typescript: string, javascript: string, sourcemap: sm.RawSourceMap, options: { preserveEnglish: boolean }): INlsPatchResult {
const { localizeCalls } = analyze(ts, typescript, 'localize');
const { localizeCalls: localize2Calls } = analyze(ts, typescript, 'localize2');
if (localizeCalls.length === 0 && localize2Calls.length === 0) {
return { javascript, sourcemap };
}
const nlsKeys = localizeCalls.map(lc => parseLocalizeKeyOrValue(lc.key)).concat(localize2Calls.map(lc => parseLocalizeKeyOrValue(lc.key)));
const nlsMessages = localizeCalls.map(lc => parseLocalizeKeyOrValue(lc.value)).concat(localize2Calls.map(lc => parseLocalizeKeyOrValue(lc.value)));
const smc = new sm.SourceMapConsumer(sourcemap);
const positionFrom = mappedPositionFrom.bind(null, sourcemap.sources[0]);
// build patches
const toPatch = (c: { range: ISpan; content: string }): IPatch => {
const start = lcFrom(smc.generatedPositionFor(positionFrom(c.range.start)));
const end = lcFrom(smc.generatedPositionFor(positionFrom(c.range.end)));
return { span: { start, end }, content: c.content };
};
const localizePatches = lazy(localizeCalls)
.map(lc => (
options.preserveEnglish ? [
{ range: lc.keySpan, content: `${allNLSMessagesIndex++}` } // localize('key', "message") => localize(, "message")
] : [
{ range: lc.keySpan, content: `${allNLSMessagesIndex++}` }, // localize('key', "message") => localize(, null)
{ range: lc.valueSpan, content: 'null' }
]))
.flatten()
.map(toPatch);
const localize2Patches = lazy(localize2Calls)
.map(lc => (
{ range: lc.keySpan, content: `${allNLSMessagesIndex++}` } // localize2('key', "message") => localize(, "message")
))
.map(toPatch);
// Sort patches by their start position
const patches = localizePatches.concat(localize2Patches).toArray().sort((a, b) => {
if (a.span.start.line < b.span.start.line) {
return -1;
} else if (a.span.start.line > b.span.start.line) {
return 1;
} else if (a.span.start.character < b.span.start.character) {
return -1;
} else if (a.span.start.character > b.span.start.character) {
return 1;
} else {
return 0;
}
});
javascript = patchJavascript(patches, javascript);
sourcemap = patchSourcemap(patches, sourcemap, smc);
return { javascript, sourcemap, nlsKeys, nlsMessages };
}
export function patchFile(javascriptFile: File, typescript: string, options: { preserveEnglish: boolean }): File {
const ts = require('typescript') as typeof import('typescript');
// hack?
const moduleId = javascriptFile.relative
.replace(/\.js$/, '')
.replace(/\\/g, '/');
const { javascript, sourcemap, nlsKeys, nlsMessages } = patch(
ts,
typescript,
javascriptFile.contents.toString(),
(javascriptFile).sourceMap,
options
);
const result = fileFrom(javascriptFile, javascript);
(result).sourceMap = sourcemap;
if (nlsKeys) {
moduleToNLSKeys[moduleId] = nlsKeys;
allNLSModulesAndKeys.push([moduleId, nlsKeys.map(nlsKey => typeof nlsKey === 'string' ? nlsKey : nlsKey.key)]);
}
if (nlsMessages) {
moduleToNLSMessages[moduleId] = nlsMessages;
allNLSMessages.push(...nlsMessages);
}
return result;
}
}
================================================
FILE: build/lib/node.js
================================================
"use strict";
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
Object.defineProperty(exports, "__esModule", { value: true });
const path_1 = __importDefault(require("path"));
const fs_1 = __importDefault(require("fs"));
const root = path_1.default.dirname(path_1.default.dirname(__dirname));
const npmrcPath = path_1.default.join(root, 'remote', '.npmrc');
const npmrc = fs_1.default.readFileSync(npmrcPath, 'utf8');
const version = /^target="(.*)"$/m.exec(npmrc)[1];
const platform = process.platform;
const arch = process.arch;
const node = platform === 'win32' ? 'node.exe' : 'node';
const nodePath = path_1.default.join(root, '.build', 'node', `v${version}`, `${platform}-${arch}`, node);
console.log(nodePath);
//# sourceMappingURL=node.js.map
================================================
FILE: build/lib/node.ts
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
import path from 'path';
import fs from 'fs';
const root = path.dirname(path.dirname(__dirname));
const npmrcPath = path.join(root, 'remote', '.npmrc');
const npmrc = fs.readFileSync(npmrcPath, 'utf8');
const version = /^target="(.*)"$/m.exec(npmrc)![1];
const platform = process.platform;
const arch = process.arch;
const node = platform === 'win32' ? 'node.exe' : 'node';
const nodePath = path.join(root, '.build', 'node', `v${version}`, `${platform}-${arch}`, node);
console.log(nodePath);
================================================
FILE: build/lib/optimize.js
================================================
"use strict";
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
if (k2 === undefined) k2 = k;
var desc = Object.getOwnPropertyDescriptor(m, k);
if (!desc || ("get" in desc ? !m.__esModule : desc.writable || desc.configurable)) {
desc = { enumerable: true, get: function() { return m[k]; } };
}
Object.defineProperty(o, k2, desc);
}) : (function(o, m, k, k2) {
if (k2 === undefined) k2 = k;
o[k2] = m[k];
}));
var __setModuleDefault = (this && this.__setModuleDefault) || (Object.create ? (function(o, v) {
Object.defineProperty(o, "default", { enumerable: true, value: v });
}) : function(o, v) {
o["default"] = v;
});
var __importStar = (this && this.__importStar) || (function () {
var ownKeys = function(o) {
ownKeys = Object.getOwnPropertyNames || function (o) {
var ar = [];
for (var k in o) if (Object.prototype.hasOwnProperty.call(o, k)) ar[ar.length] = k;
return ar;
};
return ownKeys(o);
};
return function (mod) {
if (mod && mod.__esModule) return mod;
var result = {};
if (mod != null) for (var k = ownKeys(mod), i = 0; i < k.length; i++) if (k[i] !== "default") __createBinding(result, mod, k[i]);
__setModuleDefault(result, mod);
return result;
};
})();
var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
Object.defineProperty(exports, "__esModule", { value: true });
exports.bundleTask = bundleTask;
exports.minifyTask = minifyTask;
const event_stream_1 = __importDefault(require("event-stream"));
const gulp_1 = __importDefault(require("gulp"));
const gulp_filter_1 = __importDefault(require("gulp-filter"));
const path_1 = __importDefault(require("path"));
const fs_1 = __importDefault(require("fs"));
const pump_1 = __importDefault(require("pump"));
const vinyl_1 = __importDefault(require("vinyl"));
const bundle = __importStar(require("./bundle"));
const postcss_1 = require("./postcss");
const esbuild_1 = __importDefault(require("esbuild"));
const gulp_sourcemaps_1 = __importDefault(require("gulp-sourcemaps"));
const fancy_log_1 = __importDefault(require("fancy-log"));
const ansi_colors_1 = __importDefault(require("ansi-colors"));
const REPO_ROOT_PATH = path_1.default.join(__dirname, '../..');
const DEFAULT_FILE_HEADER = [
'/*!--------------------------------------------------------',
' * Copyright (C) Microsoft Corporation. All rights reserved.',
' *--------------------------------------------------------*/'
].join('\n');
function bundleESMTask(opts) {
const resourcesStream = event_stream_1.default.through(); // this stream will contain the resources
const bundlesStream = event_stream_1.default.through(); // this stream will contain the bundled files
const entryPoints = opts.entryPoints.map(entryPoint => {
if (typeof entryPoint === 'string') {
return { name: path_1.default.parse(entryPoint).name };
}
return entryPoint;
});
const bundleAsync = async () => {
const files = [];
const tasks = [];
for (const entryPoint of entryPoints) {
(0, fancy_log_1.default)(`Bundled entry point: ${ansi_colors_1.default.yellow(entryPoint.name)}...`);
// support for 'dest' via esbuild#in/out
const dest = entryPoint.dest?.replace(/\.[^/.]+$/, '') ?? entryPoint.name;
// banner contents
const banner = {
js: DEFAULT_FILE_HEADER,
css: DEFAULT_FILE_HEADER
};
// TS Boilerplate
if (!opts.skipTSBoilerplateRemoval?.(entryPoint.name)) {
const tslibPath = path_1.default.join(require.resolve('tslib'), '../tslib.es6.js');
banner.js += await fs_1.default.promises.readFile(tslibPath, 'utf-8');
}
const contentsMapper = {
name: 'contents-mapper',
setup(build) {
build.onLoad({ filter: /\.js$/ }, async ({ path }) => {
const contents = await fs_1.default.promises.readFile(path, 'utf-8');
// TS Boilerplate
let newContents;
if (!opts.skipTSBoilerplateRemoval?.(entryPoint.name)) {
newContents = bundle.removeAllTSBoilerplate(contents);
}
else {
newContents = contents;
}
// File Content Mapper
const mapper = opts.fileContentMapper?.(path.replace(/\\/g, '/'));
if (mapper) {
newContents = await mapper(newContents);
}
return { contents: newContents };
});
}
};
const externalOverride = {
name: 'external-override',
setup(build) {
// We inline selected modules that are we depend on on startup without
// a conditional `await import(...)` by hooking into the resolution.
build.onResolve({ filter: /^minimist$/ }, () => {
return { path: path_1.default.join(REPO_ROOT_PATH, 'node_modules', 'minimist', 'index.js'), external: false };
});
},
};
const task = esbuild_1.default.build({
bundle: true,
packages: 'external', // "external all the things", see https://esbuild.github.io/api/#packages
platform: 'neutral', // makes esm
format: 'esm',
sourcemap: 'external',
plugins: [contentsMapper, externalOverride],
target: ['es2022'],
loader: {
'.ttf': 'file',
'.svg': 'file',
'.png': 'file',
'.sh': 'file',
},
assetNames: 'media/[name]', // moves media assets into a sub-folder "media"
banner: entryPoint.name === 'vs/workbench/workbench.web.main' ? undefined : banner, // TODO@esm remove line when we stop supporting web-amd-esm-bridge
entryPoints: [
{
in: path_1.default.join(REPO_ROOT_PATH, opts.src, `${entryPoint.name}.js`),
out: dest,
}
],
outdir: path_1.default.join(REPO_ROOT_PATH, opts.src),
write: false, // enables res.outputFiles
metafile: true, // enables res.metafile
// minify: NOT enabled because we have a separate minify task that takes care of the TSLib banner as well
}).then(res => {
for (const file of res.outputFiles) {
let sourceMapFile = undefined;
if (file.path.endsWith('.js')) {
sourceMapFile = res.outputFiles.find(f => f.path === `${file.path}.map`);
}
const fileProps = {
contents: Buffer.from(file.contents),
sourceMap: sourceMapFile ? JSON.parse(sourceMapFile.text) : undefined, // support gulp-sourcemaps
path: file.path,
base: path_1.default.join(REPO_ROOT_PATH, opts.src)
};
files.push(new vinyl_1.default(fileProps));
}
});
tasks.push(task);
}
await Promise.all(tasks);
return { files };
};
bundleAsync().then((output) => {
// bundle output (JS, CSS, SVG...)
event_stream_1.default.readArray(output.files).pipe(bundlesStream);
// forward all resources
gulp_1.default.src(opts.resources ?? [], { base: `${opts.src}`, allowEmpty: true }).pipe(resourcesStream);
});
const result = event_stream_1.default.merge(bundlesStream, resourcesStream);
return result
.pipe(gulp_sourcemaps_1.default.write('./', {
sourceRoot: undefined,
addComment: true,
includeContent: true
}));
}
function bundleTask(opts) {
return function () {
return bundleESMTask(opts.esm).pipe(gulp_1.default.dest(opts.out));
};
}
function minifyTask(src, sourceMapBaseUrl) {
const sourceMappingURL = sourceMapBaseUrl ? ((f) => `${sourceMapBaseUrl}/${f.relative}.map`) : undefined;
return cb => {
const cssnano = require('cssnano');
const svgmin = require('gulp-svgmin');
const jsFilter = (0, gulp_filter_1.default)('**/*.js', { restore: true });
const cssFilter = (0, gulp_filter_1.default)('**/*.css', { restore: true });
const svgFilter = (0, gulp_filter_1.default)('**/*.svg', { restore: true });
(0, pump_1.default)(gulp_1.default.src([src + '/**', '!' + src + '/**/*.map']), jsFilter, gulp_sourcemaps_1.default.init({ loadMaps: true }), event_stream_1.default.map((f, cb) => {
esbuild_1.default.build({
entryPoints: [f.path],
minify: true,
sourcemap: 'external',
outdir: '.',
packages: 'external', // "external all the things", see https://esbuild.github.io/api/#packages
platform: 'neutral', // makes esm
target: ['es2022'],
write: false
}).then(res => {
const jsFile = res.outputFiles.find(f => /\.js$/.test(f.path));
const sourceMapFile = res.outputFiles.find(f => /\.js\.map$/.test(f.path));
const contents = Buffer.from(jsFile.contents);
const unicodeMatch = contents.toString().match(/[^\x00-\xFF]+/g);
if (unicodeMatch) {
cb(new Error(`Found non-ascii character ${unicodeMatch[0]} in the minified output of ${f.path}. Non-ASCII characters in the output can cause performance problems when loading. Please review if you have introduced a regular expression that esbuild is not automatically converting and convert it to using unicode escape sequences.`));
}
else {
f.contents = contents;
f.sourceMap = JSON.parse(sourceMapFile.text);
cb(undefined, f);
}
}, cb);
}), jsFilter.restore, cssFilter, (0, postcss_1.gulpPostcss)([cssnano({ preset: 'default' })]), cssFilter.restore, svgFilter, svgmin(), svgFilter.restore, gulp_sourcemaps_1.default.write('./', {
sourceMappingURL,
sourceRoot: undefined,
includeContent: true,
addComment: true
}), gulp_1.default.dest(src + '-min'), (err) => cb(err));
};
}
//# sourceMappingURL=optimize.js.map
================================================
FILE: build/lib/optimize.ts
================================================
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
import es from 'event-stream';
import gulp from 'gulp';
import filter from 'gulp-filter';
import path from 'path';
import fs from 'fs';
import pump from 'pump';
import VinylFile from 'vinyl';
import * as bundle from './bundle';
import { gulpPostcss } from './postcss';
import esbuild from 'esbuild';
import sourcemaps from 'gulp-sourcemaps';
import fancyLog from 'fancy-log';
import ansiColors from 'ansi-colors';
const REPO_ROOT_PATH = path.join(__dirname, '../..');
export interface IBundleESMTaskOpts {
/**
* The folder to read files from.
*/
src: string;
/**
* The entry points to bundle.
*/
entryPoints: Array;
/**
* Other resources to consider (svg, etc.)
*/
resources?: string[];
/**
* File contents interceptor for a given path.
*/
fileContentMapper?: (path: string) => ((contents: string) => Promise | string) | undefined;
/**
* Allows to skip the removal of TS boilerplate. Use this when
* the entry point is small and the overhead of removing the
* boilerplate makes the file larger in the end.
*/
skipTSBoilerplateRemoval?: (entryPointName: string) => boolean;
}
const DEFAULT_FILE_HEADER = [
'/*!--------------------------------------------------------',
' * Copyright (C) Microsoft Corporation. All rights reserved.',
' *--------------------------------------------------------*/'
].join('\n');
function bundleESMTask(opts: IBundleESMTaskOpts): NodeJS.ReadWriteStream {
const resourcesStream = es.through(); // this stream will contain the resources
const bundlesStream = es.through(); // this stream will contain the bundled files
const entryPoints = opts.entryPoints.map(entryPoint => {
if (typeof entryPoint === 'string') {
return { name: path.parse(entryPoint).name };
}
return entryPoint;
});
const bundleAsync = async () => {
const files: VinylFile[] = [];
const tasks: Promise[] = [];
for (const entryPoint of entryPoints) {
fancyLog(`Bundled entry point: ${ansiColors.yellow(entryPoint.name)}...`);
// support for 'dest' via esbuild#in/out
const dest = entryPoint.dest?.replace(/\.[^/.]+$/, '') ?? entryPoint.name;
// banner contents
const banner = {
js: DEFAULT_FILE_HEADER,
css: DEFAULT_FILE_HEADER
};
// TS Boilerplate
if (!opts.skipTSBoilerplateRemoval?.(entryPoint.name)) {
const tslibPath = path.join(require.resolve('tslib'), '../tslib.es6.js');
banner.js += await fs.promises.readFile(tslibPath, 'utf-8');
}
const contentsMapper: esbuild.Plugin = {
name: 'contents-mapper',
setup(build) {
build.onLoad({ filter: /\.js$/ }, async ({ path }) => {
const contents = await fs.promises.readFile(path, 'utf-8');
// TS Boilerplate
let newContents: string;
if (!opts.skipTSBoilerplateRemoval?.(entryPoint.name)) {
newContents = bundle.removeAllTSBoilerplate(contents);
} else {
newContents = contents;
}
// File Content Mapper
const mapper = opts.fileContentMapper?.(path.replace(/\\/g, '/'));
if (mapper) {
newContents = await mapper(newContents);
}
return { contents: newContents };
});
}
};
const externalOverride: esbuild.Plugin = {
name: 'external-override',
setup(build) {
// We inline selected modules that are we depend on on startup without
// a conditional `await import(...)` by hooking into the resolution.
build.onResolve({ filter: /^minimist$/ }, () => {
return { path: path.join(REPO_ROOT_PATH, 'node_modules', 'minimist', 'index.js'), external: false };
});
},
};
const task = esbuild.build({
bundle: true,
packages: 'external', // "external all the things", see https://esbuild.github.io/api/#packages
platform: 'neutral', // makes esm
format: 'esm',
sourcemap: 'external',
plugins: [contentsMapper, externalOverride],
target: ['es2022'],
loader: {
'.ttf': 'file',
'.svg': 'file',
'.png': 'file',
'.sh': 'file',
},
assetNames: 'media/[name]', // moves media assets into a sub-folder "media"
banner: entryPoint.name === 'vs/workbench/workbench.web.main' ? undefined : banner, // TODO@esm remove line when we stop supporting web-amd-esm-bridge
entryPoints: [
{
in: path.join(REPO_ROOT_PATH, opts.src, `${entryPoint.name}.js`),
out: dest,
}
],
outdir: path.join(REPO_ROOT_PATH, opts.src),
write: false, // enables res.outputFiles
metafile: true, // enables res.metafile
// minify: NOT enabled because we have a separate minify task that takes care of the TSLib banner as well
}).then(res => {
for (const file of res.outputFiles) {
let sourceMapFile: esbuild.OutputFile | undefined = undefined;
if (file.path.endsWith('.js')) {
sourceMapFile = res.outputFiles.find(f => f.path === `${file.path}.map`);
}
const fileProps = {
contents: Buffer.from(file.contents),
sourceMap: sourceMapFile ? JSON.parse(sourceMapFile.text) : undefined, // support gulp-sourcemaps
path: file.path,
base: path.join(REPO_ROOT_PATH, opts.src)
};
files.push(new VinylFile(fileProps));
}
});
tasks.push(task);
}
await Promise.all(tasks);
return { files };
};
bundleAsync().then((output) => {
// bundle output (JS, CSS, SVG...)
es.readArray(output.files).pipe(bundlesStream);
// forward all resources
gulp.src(opts.resources ?? [], { base: `${opts.src}`, allowEmpty: true }).pipe(resourcesStream);
});
const result = es.merge(
bundlesStream,
resourcesStream
);
return result
.pipe(sourcemaps.write('./', {
sourceRoot: undefined,
addComment: true,
includeContent: true
}));
}
export interface IBundleESMTaskOpts {
/**
* Destination folder for the bundled files.
*/
out: string;
/**
* Bundle ESM modules (using esbuild).
*/
esm: IBundleESMTaskOpts;
}
export function bundleTask(opts: IBundleESMTaskOpts): () => NodeJS.ReadWriteStream {
return function () {
return bundleESMTask(opts.esm).pipe(gulp.dest(opts.out));
};
}
export function minifyTask(src: string, sourceMapBaseUrl?: string): (cb: any) => void {
const sourceMappingURL = sourceMapBaseUrl ? ((f: any) => `${sourceMapBaseUrl}/${f.relative}.map`) : undefined;
return cb => {
const cssnano = require('cssnano') as typeof import('cssnano');
const svgmin = require('gulp-svgmin') as typeof import('gulp-svgmin');
const jsFilter = filter('**/*.js', { restore: true });
const cssFilter = filter('**/*.css', { restore: true });
const svgFilter = filter('**/*.svg', { restore: true });
pump(
gulp.src([src + '/**', '!' + src + '/**/*.map']),
jsFilter,
sourcemaps.init({ loadMaps: true }),
es.map((f: any, cb) => {
esbuild.build({
entryPoints: [f.path],
minify: true,
sourcemap: 'external',
outdir: '.',
packages: 'external', // "external all the things", see https://esbuild.github.io/api/#packages
platform: 'neutral', // makes esm
target: ['es2022'],
write: false
}).then(res => {
const jsFile = res.outputFiles.find(f => /\.js$/.test(f.path))!;
const sourceMapFile = res.outputFiles.find(f => /\.js\.map$/.test(f.path))!;
const contents = Buffer.from(jsFile.contents);
const unicodeMatch = contents.toString().match(/[^\x00-\xFF]+/g);
if (unicodeMatch) {
cb(new Error(`Found non-ascii character ${unicodeMatch[0]} in the minified output of ${f.path}. Non-ASCII characters in the output can cause performance problems when loading. Please review if you have introduced a regular expression that esbuild is not automatically converting and convert it to using unicode escape sequences.`));
} else {
f.contents = contents;
f.sourceMap = JSON.parse(sourceMapFile.text);
cb(undefined, f);
}
}, cb);
}),
jsFilter.restore,
cssFilter,
gulpPostcss([cssnano({ preset: 'default' })]),
cssFilter.restore,
svgFilter,
svgmin(),
svgFilter.restore,
sourcemaps.write('./', {
sourceMappingURL,
sourceRoot: undefined,
includeContent: true,
addComment: true
} as any),
gulp.dest(src + '-min'),
(err: any) => cb(err));
};
}
================================================
FILE: build/lib/policies.js
================================================
"use strict";
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
Object.defineProperty(exports, "__esModule", { value: true });
const child_process_1 = require("child_process");
const fs_1 = require("fs");
const path_1 = __importDefault(require("path"));
const byline_1 = __importDefault(require("byline"));
const ripgrep_1 = require("@vscode/ripgrep");
const tree_sitter_1 = __importDefault(require("tree-sitter"));
const { typescript } = require('tree-sitter-typescript');
const product = require('../../product.json');
const packageJson = require('../../package.json');
function isNlsString(value) {
return value ? typeof value !== 'string' : false;
}
function isStringArray(value) {
return !value.some(s => isNlsString(s));
}
function isNlsStringArray(value) {
return value.every(s => isNlsString(s));
}
var PolicyType;
(function (PolicyType) {
PolicyType["Boolean"] = "boolean";
PolicyType["Number"] = "number";
PolicyType["Object"] = "object";
PolicyType["String"] = "string";
PolicyType["StringEnum"] = "stringEnum";
})(PolicyType || (PolicyType = {}));
function renderADMLString(prefix, moduleName, nlsString, translations) {
let value;
if (translations) {
const moduleTranslations = translations[moduleName];
if (moduleTranslations) {
value = moduleTranslations[nlsString.nlsKey];
}
}
if (!value) {
value = nlsString.value;
}
return `${value}`;
}
function renderProfileString(_prefix, moduleName, nlsString, translations) {
let value;
if (translations) {
const moduleTranslations = translations[moduleName];
if (moduleTranslations) {
value = moduleTranslations[nlsString.nlsKey];
}
}
if (!value) {
value = nlsString.value;
}
return value;
}
class BasePolicy {
type;
name;
category;
minimumVersion;
description;
moduleName;
constructor(type, name, category, minimumVersion, description, moduleName) {
this.type = type;
this.name = name;
this.category = category;
this.minimumVersion = minimumVersion;
this.description = description;
this.moduleName = moduleName;
}
renderADMLString(nlsString, translations) {
return renderADMLString(this.name, this.moduleName, nlsString, translations);
}
renderADMX(regKey) {
return [
``,
` `,
` `,
` `,
...this.renderADMXElements(),
` `,
``
];
}
renderADMLStrings(translations) {
return [
`${this.name}`,
this.renderADMLString(this.description, translations)
];
}
renderADMLPresentation() {
return `${this.renderADMLPresentationContents()}`;
}
renderProfile() {
return [`${this.name}`, this.renderProfileValue()];
}
renderProfileManifest(translations) {
return `
${this.renderProfileManifestValue(translations)}
`;
}
}
class BooleanPolicy extends BasePolicy {
static from(name, category, minimumVersion, description, moduleName, settingNode) {
const type = getStringProperty(moduleName, settingNode, 'type');
if (type !== 'boolean') {
return undefined;
}
return new BooleanPolicy(name, category, minimumVersion, description, moduleName);
}
constructor(name, category, minimumVersion, description, moduleName) {
super(PolicyType.Boolean, name, category, minimumVersion, description, moduleName);
}
renderADMXElements() {
return [
``,
`
