[ { "path": ".clang-format", "content": "---\nLanguage: Cpp\n# BasedOnStyle: Google\nAccessModifierOffset: -1\nAlignAfterOpenBracket: Align\nAlignArrayOfStructures: None\nAlignConsecutiveMacros: None\nAlignConsecutiveAssignments: None\nAlignConsecutiveBitFields: None\nAlignConsecutiveDeclarations: None\nAlignEscapedNewlines: Left\nAlignOperands: Align\nAlignTrailingComments: true\nAllowAllArgumentsOnNextLine: true\nAllowAllParametersOfDeclarationOnNextLine: true\nAllowShortEnumsOnASingleLine: true\nAllowShortBlocksOnASingleLine: Never\nAllowShortCaseLabelsOnASingleLine: false\nAllowShortFunctionsOnASingleLine: All\nAllowShortLambdasOnASingleLine: Inline\nAllowShortIfStatementsOnASingleLine: WithoutElse\nAllowShortLoopsOnASingleLine: true\nAlwaysBreakAfterDefinitionReturnType: None\nAlwaysBreakAfterReturnType: None\nAlwaysBreakBeforeMultilineStrings: true\nAlwaysBreakTemplateDeclarations: Yes\nAttributeMacros:\n - __capability\nBinPackArguments: true\nBinPackParameters: true\nBraceWrapping:\n AfterCaseLabel: false\n AfterClass: false\n AfterControlStatement: Never\n AfterEnum: false\n AfterFunction: false\n AfterNamespace: false\n AfterObjCDeclaration: false\n AfterStruct: false\n AfterUnion: false\n AfterExternBlock: false\n BeforeCatch: false\n BeforeElse: false\n BeforeLambdaBody: false\n BeforeWhile: false\n IndentBraces: false\n SplitEmptyFunction: true\n SplitEmptyRecord: true\n SplitEmptyNamespace: true\nBreakBeforeBinaryOperators: None\nBreakBeforeConceptDeclarations: true\nBreakBeforeBraces: Attach\nBreakBeforeInheritanceComma: false\nBreakInheritanceList: BeforeColon\nBreakBeforeTernaryOperators: true\nBreakConstructorInitializersBeforeComma: false\nBreakConstructorInitializers: BeforeColon\nBreakAfterJavaFieldAnnotations: false\nBreakStringLiterals: true\nColumnLimit: 100\nCommentPragmas: '^ IWYU pragma:'\nQualifierAlignment: Leave\nCompactNamespaces: false\nConstructorInitializerIndentWidth: 4\nContinuationIndentWidth: 4\nCpp11BracedListStyle: true\nDeriveLineEnding: true\nDerivePointerAlignment: true\nDisableFormat: false\nEmptyLineAfterAccessModifier: Never\nEmptyLineBeforeAccessModifier: LogicalBlock\nExperimentalAutoDetectBinPacking: false\nPackConstructorInitializers: NextLine\nBasedOnStyle: ''\nConstructorInitializerAllOnOneLineOrOnePerLine: false\nAllowAllConstructorInitializersOnNextLine: true\nFixNamespaceComments: true\nForEachMacros:\n - foreach\n - Q_FOREACH\n - BOOST_FOREACH\nIfMacros:\n - KJ_IF_MAYBE\nIncludeBlocks: Regroup\nIncludeCategories:\n - Regex: '^'\n Priority: 2\n SortPriority: 0\n CaseSensitive: false\n - Regex: '^<.*\\.h>'\n Priority: 1\n SortPriority: 0\n CaseSensitive: false\n - Regex: '^<.*'\n Priority: 2\n SortPriority: 0\n CaseSensitive: false\n - Regex: '.*'\n Priority: 3\n SortPriority: 0\n CaseSensitive: false\nIncludeIsMainRegex: '([-_](test|unittest))?$'\nIncludeIsMainSourceRegex: ''\nIndentAccessModifiers: false\nIndentCaseLabels: true\nIndentCaseBlocks: false\nIndentGotoLabels: true\nIndentPPDirectives: None\nIndentExternBlock: AfterExternBlock\nIndentRequires: false\nIndentWidth: 2\nIndentWrappedFunctionNames: false\nInsertTrailingCommas: None\nJavaScriptQuotes: Leave\nJavaScriptWrapImports: true\nKeepEmptyLinesAtTheStartOfBlocks: false\nLambdaBodyIndentation: Signature\nMacroBlockBegin: ''\nMacroBlockEnd: ''\nMaxEmptyLinesToKeep: 1\nNamespaceIndentation: None\nObjCBinPackProtocolList: Never\nObjCBlockIndentWidth: 2\nObjCBreakBeforeNestedBlockParam: true\nObjCSpaceAfterProperty: false\nObjCSpaceBeforeProtocolList: true\nPenaltyBreakAssignment: 2\nPenaltyBreakBeforeFirstCallParameter: 1\nPenaltyBreakComment: 300\nPenaltyBreakFirstLessLess: 120\nPenaltyBreakString: 1000\nPenaltyBreakTemplateDeclaration: 10\nPenaltyExcessCharacter: 1000000\nPenaltyReturnTypeOnItsOwnLine: 200\nPenaltyIndentedWhitespace: 0\nPointerAlignment: Left\nPPIndentWidth: -1\nRawStringFormats:\n - Language: Cpp\n Delimiters:\n - cc\n - CC\n - cpp\n - Cpp\n - CPP\n - 'c++'\n - 'C++'\n CanonicalDelimiter: ''\n BasedOnStyle: google\n - Language: TextProto\n Delimiters:\n - pb\n - PB\n - proto\n - PROTO\n EnclosingFunctions:\n - EqualsProto\n - EquivToProto\n - PARSE_PARTIAL_TEXT_PROTO\n - PARSE_TEST_PROTO\n - PARSE_TEXT_PROTO\n - ParseTextOrDie\n - ParseTextProtoOrDie\n - ParseTestProto\n - ParsePartialTestProto\n CanonicalDelimiter: pb\n BasedOnStyle: google\nReferenceAlignment: Pointer\nReflowComments: false\nShortNamespaceLines: 1\nSortIncludes: CaseSensitive\nSortJavaStaticImport: Before\nSortUsingDeclarations: true\nSpaceAfterCStyleCast: false\nSpaceAfterLogicalNot: false\nSpaceAfterTemplateKeyword: true\nSpaceBeforeAssignmentOperators: true\nSpaceBeforeCaseColon: false\nSpaceBeforeCpp11BracedList: false\nSpaceBeforeCtorInitializerColon: true\nSpaceBeforeInheritanceColon: true\nSpaceBeforeParens: ControlStatements\nSpaceAroundPointerQualifiers: Default\nSpaceBeforeRangeBasedForLoopColon: true\nSpaceInEmptyBlock: false\nSpaceInEmptyParentheses: false\nSpacesBeforeTrailingComments: 2\nSpacesInAngles: Never\nSpacesInConditionalStatement: false\nSpacesInContainerLiterals: true\nSpacesInCStyleCastParentheses: false\nSpacesInLineCommentPrefix:\n Minimum: 1\n Maximum: -1\nSpacesInParentheses: false\nSpacesInSquareBrackets: false\nSpaceBeforeSquareBrackets: false\nBitFieldColonSpacing: Both\nStandard: Auto\nStatementAttributeLikeMacros:\n - Q_EMIT\nStatementMacros:\n - Q_UNUSED\n - QT_REQUIRE_VERSION\nTabWidth: 8\nUseCRLF: false\nUseTab: Never\nWhitespaceSensitiveMacros:\n - STRINGIZE\n - PP_STRINGIZE\n - BOOST_PP_STRINGIZE\n - NS_SWIFT_NAME\n - CF_SWIFT_NAME\n...\n" }, { "path": ".clang-tidy", "content": "Checks: 'modernize-*,-modernize-use-nodiscard,-modernize-concat-nested-namespaces,-modernize-make-*,-modernize-use-auto,-modernize-raw-string-literal,-modernize-avoid-c-arrays,-modernize-use-trailing-return-type,google-*,-google-default-arguments,-clang-diagnostic-#pragma-messages,readability-identifier-naming'\nCheckOptions:\n - { key: readability-identifier-naming.ClassCase, value: CamelCase }\n - { key: readability-identifier-naming.StructCase, value: CamelCase }\n - { key: readability-identifier-naming.TypeAliasCase, value: CamelCase }\n - { key: readability-identifier-naming.TypedefCase, value: CamelCase }\n - { key: readability-identifier-naming.TypeTemplateParameterCase, value: CamelCase }\n - { key: readability-identifier-naming.MemberCase, value: lower_case }\n - { key: readability-identifier-naming.PrivateMemberSuffix, value: '_' }\n - { key: readability-identifier-naming.ProtectedMemberSuffix, value: '_' }\n - { key: readability-identifier-naming.EnumCase, value: CamelCase }\n - { key: readability-identifier-naming.EnumConstant, value: CamelCase }\n - { key: readability-identifier-naming.EnumConstantPrefix, value: k }\n - { key: readability-identifier-naming.GlobalConstantCase, value: CamelCase }\n - { key: readability-identifier-naming.GlobalConstantPrefix, value: k }\n - { key: readability-identifier-naming.StaticConstantCase, value: CamelCase }\n - { key: readability-identifier-naming.StaticConstantPrefix, value: k }\n - { key: readability-identifier-naming.ConstexprVariableCase, value: CamelCase }\n - { key: readability-identifier-naming.ConstexprVariablePrefix, value: k }\n - { key: readability-identifier-naming.FunctionCase, value: CamelCase }\n - { key: readability-identifier-naming.NamespaceCase, value: lower_case }\n" }, { "path": ".editorconfig", "content": "root = true\n\n[*]\ncharset=utf-8\nindent_style = space\nindent_size = 2\ninsert_final_newline = true\n\n[*.py]\nindent_style = space\nindent_size = 4\n" }, { "path": ".gitattributes", "content": "* text=auto\n\n*.c text eol=lf\n*.h text eol=lf\n*.cc text eol=lf\n*.cuh text eol=lf\n*.cu text eol=lf\n*.py text eol=lf\n*.txt text eol=lf\n*.R text eol=lf\n*.scala text eol=lf\n*.java text eol=lf\n\n*.sh text eol=lf\n\n*.rst text eol=lf\n*.md text eol=lf\n*.csv text eol=lf" }, { "path": ".github/FUNDING.yml", "content": "open_collective: xgboost\ncustom: https://xgboost.ai/sponsors\n" }, { "path": ".github/ISSUE_TEMPLATE.md", "content": "Thanks for participating in the XGBoost community! The issue tracker is used for actionable items such as feature proposals discussion, roadmaps, and bug tracking.\n\nIssues that are inactive for a period of time may get closed. We adopt this policy so that we won't lose track of actionable issues that may fall at the bottom of the pile. Feel free to reopen a new one if you feel there is an additional problem that needs attention when an old one gets closed.\n\nFor bug reports, to help the developer act on the issues, please include a description of your environment, preferably a minimum script to reproduce the problem.\n\nFor feature proposals, list clear, small actionable items so we can track the progress of the change.\n" }, { "path": ".github/dependabot.yml", "content": "version: 2\nupdates:\n - package-ecosystem: \"github-actions\"\n directory: /\n schedule:\n interval: \"weekly\"\n groups:\n github-actions:\n patterns:\n - \"*\"\n" }, { "path": ".github/lock.yml", "content": "# Configuration for lock-threads - https://github.com/dessant/lock-threads\n\n# Number of days of inactivity before a closed issue or pull request is locked\ndaysUntilLock: 90\n\n# Issues and pull requests with these labels will not be locked. Set to `[]` to disable\nexemptLabels:\n - feature-request\n\n# Label to add before locking, such as `outdated`. Set to `false` to disable\nlockLabel: false\n\n# Comment to post before locking. Set to `false` to disable\nlockComment: false\n\n# Assign `resolved` as the reason for locking. Set to `false` to disable\nsetLockReason: true\n\n# Limit to only `issues` or `pulls`\n# only: issues\n\n# Optionally, specify configuration settings just for `issues` or `pulls`\n# issues:\n# exemptLabels:\n# - help-wanted\n# lockLabel: outdated\n\n# pulls:\n# daysUntilLock: 30\n\n# Repository to extend settings from\n# _extends: repo\n" }, { "path": ".github/runs-on.yml", "content": "# Custom images with CUDA toolkit installed\n# See ops/packer for instructions for building the images\nimages:\n linux-amd64:\n platform: \"linux\"\n arch: \"x64\"\n owner: \"492475357299\" # XGBooost CI\n name: \"xgboost-ci-runs-on-linux-amd64-*\"\n linux-arm64:\n platform: \"linux\"\n arch: \"arm64\"\n owner: \"492475357299\" # XGBooost CI\n name: \"xgboost-ci-runs-on-linux-arm64-*\"\n windows-amd64:\n platform: \"windows\"\n arch: \"x64\"\n owner: \"492475357299\" # XGBooost CI\n name: \"xgboost-ci-runs-on-windows-*\"\n\nrunners:\n linux-amd64-cpu:\n cpu: 16\n family: [\"c7i-flex\", \"c7i\", \"c7a\", \"c5\", \"c5a\"]\n image: linux-amd64\n spot: \"false\"\n linux-amd64-gpu:\n family: [\"g4dn.xlarge\"]\n image: linux-amd64\n spot: \"false\"\n linux-amd64-mgpu:\n family: [\"g4dn.12xlarge\"]\n image: linux-amd64\n spot: \"false\"\n linux-arm64-cpu:\n cpu: 16\n family: [\"c6g\", \"c7g\"]\n image: linux-arm64\n spot: \"false\"\n linux-arm64-gpu:\n family: [\"g5g.xlarge\"]\n image: linux-arm64\n spot: \"false\"\n windows-gpu:\n family: [\"g4dn.2xlarge\"]\n image: windows-amd64\n spot: \"false\"\n windows-cpu:\n cpu: 32\n family: [\"c7i-flex\", \"c7i\", \"c7a\", \"c5\", \"c5a\"]\n image: windows-amd64\n spot: \"false\"\n" }, { "path": ".github/workflows/cccl_nightly.yml", "content": "name: Test XGBoost with latest CCCL and RMM\n\non:\n workflow_dispatch: # Can be manually triggered\n schedule:\n - cron: \"0 7 * * *\" # Run once daily\n\npermissions:\n contents: read\n\ndefaults:\n run:\n shell: bash -l {0}\n\nconcurrency:\n group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}\n cancel-in-progress: true\n\nenv:\n GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}\n\njobs:\n get-latest-cccl-version:\n name: Query the latest version of CCCL\n runs-on: ubuntu-latest\n outputs:\n cccl_version: ${{ steps.query_version_step.outputs.cccl_version }}\n steps:\n - uses: actions/checkout@v6.0.2\n - id: query_version_step\n name: Query version\n run: |\n source ops/pipeline/query-latest-cccl.sh\n echo \"cccl_version=${CCCL_VERSION}\" >> \"$GITHUB_OUTPUT\"\n\n test-latest-cccl:\n name: Test building XGBoost with latest CCCL (RC included)\n runs-on:\n - runs-on=${{ github.run_id }}\n - runner=linux-amd64-cpu\n - tag=nightly-cccl\n needs: get-latest-cccl-version\n steps:\n - uses: actions/checkout@v6.0.2\n with:\n submodules: \"true\"\n - name: Log into Docker registry (AWS ECR)\n run: bash ops/pipeline/login-docker-registry.sh\n - name: Build XGBoost with latest CCCL\n run: |\n bash ops/pipeline/nightly-test-cccl.sh ${{ needs.get-latest-cccl-version.outputs.cccl_version }}\n\n get-latest-rmm-version:\n name: Query the latest version of RMM\n runs-on: ubuntu-latest\n outputs:\n rmm_version: ${{ steps.query_version_step.outputs.rmm_version }}\n steps:\n - uses: actions/checkout@v6.0.2\n - id: query_version_step\n name: Query version\n run: |\n source ops/pipeline/query-latest-rmm.sh\n echo \"rmm_version=${RMM_VERSION}\" >> \"$GITHUB_OUTPUT\"\n\n test-latest-rmm:\n name: Test building XGBoost with latest nightly version of RMM\n # This job uses the stable CCCL used by RMM and rest of RAPIDS\n runs-on:\n - runs-on=${{ github.run_id }}\n - runner=linux-amd64-cpu\n - tag=nightly-rmm\n needs: get-latest-rmm-version\n steps:\n - uses: actions/checkout@v6.0.2\n with:\n submodules: \"true\"\n - name: Log into Docker registry (AWS ECR)\n run: bash ops/pipeline/login-docker-registry.sh\n - name: Build XGBoost with latest RMM\n run: |\n bash ops/pipeline/nightly-test-rmm.sh ${{ needs.get-latest-rmm-version.outputs.rmm_version }}\n" }, { "path": ".github/workflows/freebsd.yml", "content": "name: FreeBSD\n\non:\n push:\n branches:\n - master\n - 'release_*'\n pull_request:\n\npermissions:\n contents: read # to fetch code (actions/checkout)\n\nconcurrency:\n group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}\n cancel-in-progress: true\n\njobs:\n test:\n runs-on: ubuntu-latest\n timeout-minutes: 20\n name: A job to run test in FreeBSD\n steps:\n - uses: actions/checkout@v6.0.2\n with:\n submodules: 'true'\n - name: Test in FreeBSD\n id: test\n uses: vmactions/freebsd-vm@v1.4.3\n with:\n usesh: true\n prepare: |\n pkg install -y cmake git ninja googletest bash\n run: |\n bash ops/pipeline/test-freebsd.sh\n" }, { "path": ".github/workflows/jvm_tests.yml", "content": "name: JVM Tests\n\non:\n push:\n branches:\n - master\n - 'release_*'\n pull_request:\n\npermissions:\n contents: read # to fetch code (actions/checkout)\n\nconcurrency:\n group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}\n cancel-in-progress: true\n\nenv:\n BRANCH_NAME: >-\n ${{ github.event.pull_request.number && 'PR-' }}${{ github.event.pull_request.number || github.ref_name }}\n\njobs:\n ci-configure:\n name: Configure variables for CI\n runs-on:\n - runs-on=${{ github.run_id }}\n - runner=linux-amd64-cpu\n - tag=jvm-tests-ci-configure\n steps:\n - name: Login to Amazon ECR\n id: login-ecr\n uses: aws-actions/amazon-ecr-login@v2.0.2\n with:\n mask-password: 'false'\n registries: '492475357299'\n - uses: actions/checkout@v6.0.2\n - name: Get image tag\n id: get-image-tag\n run: |\n source ops/pipeline/get-image-tag.sh\n echo \"Using image tag $IMAGE_TAG\"\n echo \"image_tag=$IMAGE_TAG\" >> \"$GITHUB_OUTPUT\"\n outputs:\n docker_registry: ${{ steps.login-ecr.outputs.registry }}\n docker_username: ${{ steps.login-ecr.outputs.docker_username_492475357299_dkr_ecr_us_west_2_amazonaws_com }}\n docker_password: ${{ steps.login-ecr.outputs.docker_password_492475357299_dkr_ecr_us_west_2_amazonaws_com }}\n image_tag: ${{ steps.get-image-tag.outputs.image_tag }}\n\n build-jvm-manylinux2014:\n name: >-\n Build libxgboost4j.so targeting glibc 2.17\n (arch ${{ matrix.arch }}, runner ${{ matrix.runner }})\n runs-on:\n - runs-on\n - runner=${{ matrix.runner }}\n - run-id=${{ github.run_id }}\n - tag=jvm-tests-build-jvm-manylinux2014-${{ matrix.arch }}\n strategy:\n fail-fast: false\n matrix:\n include:\n - arch: aarch64\n runner: linux-arm64-cpu\n - arch: x86_64\n runner: linux-amd64-cpu\n steps:\n - uses: actions/checkout@v6.0.2\n with:\n submodules: \"true\"\n - name: Log into Docker registry (AWS ECR)\n run: bash ops/pipeline/login-docker-registry.sh\n - run: bash ops/pipeline/build-jvm-manylinux2014.sh ${{ matrix.arch }}\n\n build-jvm-gpu:\n name: Build libxgboost4j.so with CUDA\n needs: ci-configure\n runs-on:\n - runs-on=${{ github.run_id }}\n - runner=linux-amd64-cpu\n - tag=jvm-tests-build-jvm-gpu\n - extras=s3-cache\n container:\n image: ${{ needs.ci-configure.outputs.docker_registry }}/xgb-ci.jvm_gpu_build:${{ needs.ci-configure.outputs.image_tag }}\n credentials:\n username: ${{ needs.ci-configure.outputs.docker_username }}\n password: ${{ needs.ci-configure.outputs.docker_password }}\n steps:\n - uses: runs-on/action@v2\n - uses: actions/checkout@v6.0.2\n with:\n submodules: \"true\"\n - uses: dmlc/xgboost-devops/actions/sccache@main\n - name: Build libxgboost4j.so with CUDA\n run: bash ops/pipeline/build-jvm-gpu.sh\n - run: sccache --show-stats\n - name: Stash files\n run: |\n python3 ops/pipeline/manage-artifacts.py upload \\\n --s3-bucket ${{ env.RUNS_ON_S3_BUCKET_CACHE }} \\\n --prefix cache/${{ github.run_id }}/build-jvm-gpu \\\n lib/libxgboost4j.so\n\n build-jvm-mac:\n name: \"Build libxgboost4j.dylib for ${{ matrix.description }}\"\n runs-on: ${{ matrix.runner }}\n strategy:\n fail-fast: false\n matrix:\n include:\n - description: \"MacOS (Apple Silicon)\"\n libname: libxgboost4j_m1.dylib\n runner: macos-14\n - description: \"MacOS (Intel)\"\n libname: libxgboost4j_intel.dylib\n runner: macos-15-intel\n steps:\n - uses: actions/checkout@v6.0.2\n with:\n submodules: \"true\"\n - uses: dmlc/xgboost-devops/actions/sccache@main\n with:\n cache-key-prefix: ${{ github.job }}-${{ matrix.runner }}\n - run: bash ops/pipeline/build-jvm-macos.sh\n - run: sccache --show-stats\n - name: Upload libxgboost4j.dylib\n if: github.ref == 'refs/heads/master' || contains(github.ref, 'refs/heads/release_')\n run: |\n mv -v lib/libxgboost4j.dylib ${{ matrix.libname }}\n python3 ops/pipeline/manage-artifacts.py upload \\\n --s3-bucket xgboost-nightly-builds \\\n --prefix ${{ env.BRANCH_NAME }}/${{ github.sha }} --make-public \\\n ${{ matrix.libname }}\n env:\n AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID_IAM_S3_UPLOADER }}\n AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY_IAM_S3_UPLOADER }}\n\n build-test-jvm-packages-linux:\n name: Build and test JVM packages (Linux, Scala ${{ matrix.scala_version }})\n needs: ci-configure\n runs-on:\n - runs-on=${{ github.run_id }}\n - runner=linux-amd64-cpu\n - tag=jvm-tests-build-test-jvm-packages-scala${{ matrix.scala_version }}\n strategy:\n fail-fast: false\n matrix:\n scala_version: [\"2.12\", \"2.13\"]\n container:\n image: ${{ needs.ci-configure.outputs.docker_registry }}/xgb-ci.jvm:${{ needs.ci-configure.outputs.image_tag }}\n credentials:\n username: ${{ needs.ci-configure.outputs.docker_username }}\n password: ${{ needs.ci-configure.outputs.docker_password }}\n env:\n SCALA_VERSION: ${{ matrix.scala_version }}\n steps:\n - uses: runs-on/action@v2\n - uses: actions/checkout@v6.0.2\n with:\n submodules: \"true\"\n - name: Build and test JVM packages (Scala ${{ matrix.scala_version }})\n run: bash ops/pipeline/build-test-jvm-packages.sh\n - name: Stash files\n run: |\n python3 ops/pipeline/manage-artifacts.py upload \\\n --s3-bucket ${{ env.RUNS_ON_S3_BUCKET_CACHE }} \\\n --prefix cache/${{ github.run_id }}/build-test-jvm-packages \\\n lib/libxgboost4j.so\n if: matrix.scala_version == '2.13'\n\n build-test-jvm-packages-other-os:\n name: Build and test JVM packages (${{ matrix.os }})\n timeout-minutes: 30\n runs-on: ${{ matrix.os }}\n strategy:\n fail-fast: false\n matrix:\n os: [windows-latest, macos-15-intel]\n steps:\n - uses: actions/checkout@v6.0.2\n with:\n submodules: 'true'\n - uses: actions/setup-java@v5\n with:\n distribution: 'temurin'\n java-version: '8'\n - uses: dmlc/xgboost-devops/actions/miniforge-setup@main\n with:\n environment-name: minimal\n environment-file: ops/conda_env/minimal.yml\n - name: Cache Maven packages\n uses: actions/cache@v5.0.3\n with:\n path: ~/.m2\n key: ${{ runner.os }}-m2-${{ hashFiles('/jvm-packages/pom.xml') }}\n restore-keys: ${{ runner.os }}-m2-${{ hashFiles('/jvm-packages/pom.xml') }}\n - name: Test XGBoost4J (Core) on macos\n if: matrix.os == 'macos-15-intel'\n run: |\n cd jvm-packages\n mvn test -B -pl :xgboost4j_2.12 -Duse.openmp=OFF\n - name: Test XGBoost4J (Core) on windows\n if: matrix.os == 'windows-latest'\n run: |\n cd jvm-packages\n mvn test -B -pl :xgboost4j_2.12\n - name: Publish artifact xgboost4j.dll to S3\n run: |\n python ops/pipeline/manage-artifacts.py upload `\n --s3-bucket xgboost-nightly-builds `\n --prefix ${{ env.BRANCH_NAME }}/${{ github.sha }} --make-public `\n lib/xgboost4j.dll\n if: |\n (github.ref == 'refs/heads/master' || contains(github.ref, 'refs/heads/release_')) &&\n matrix.os == 'windows-latest'\n env:\n AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID_IAM_S3_UPLOADER }}\n AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY_IAM_S3_UPLOADER }}\n\n test-jvm-packages-gpu:\n name: Test JVM packages with CUDA (Scala ${{ matrix.scala_version }})\n needs: [ci-configure, build-jvm-gpu]\n runs-on:\n - runs-on=${{ github.run_id }}\n - runner=linux-amd64-mgpu\n - tag=jvm-tests-test-jvm-packages-gpu-scala${{ matrix.scala_version }}\n strategy:\n fail-fast: false\n matrix:\n scala_version: [\"2.12\", \"2.13\"]\n container:\n image: ${{ needs.ci-configure.outputs.docker_registry }}/xgb-ci.jvm_gpu_build:${{ needs.ci-configure.outputs.image_tag }}\n credentials:\n username: ${{ needs.ci-configure.outputs.docker_username }}\n password: ${{ needs.ci-configure.outputs.docker_password }}\n options: --gpus all --shm-size=4g --privileged\n env:\n SCALA_VERSION: ${{ matrix.scala_version }}\n USE_CUDA: \"1\"\n SKIP_NATIVE_BUILD: \"1\"\n steps:\n - uses: actions/checkout@v6.0.2\n with:\n submodules: \"true\"\n - name: Unstash files\n run: |\n python3 ops/pipeline/manage-artifacts.py download \\\n --s3-bucket ${{ env.RUNS_ON_S3_BUCKET_CACHE }} \\\n --prefix cache/${{ github.run_id }}/build-jvm-gpu \\\n --dest-dir lib \\\n libxgboost4j.so\n - name: Test JVM packages with CUDA\n run: bash ops/pipeline/build-test-jvm-packages.sh\n\n deploy-jvm-packages:\n name: Deploy JVM packages to S3 (${{ matrix.variant.name }}, Scala ${{ matrix.scala_version }})\n needs: [ci-configure, build-jvm-gpu, build-test-jvm-packages-linux, test-jvm-packages-gpu]\n runs-on:\n - runs-on=${{ github.run_id }}\n - runner=linux-amd64-cpu\n - tag=jvm-tests-deploy-jvm-packages-${{ matrix.variant.name }}-scala${{ matrix.scala_version }}\n strategy:\n fail-fast: false\n matrix:\n variant:\n - name: cpu\n image_repo: xgb-ci.jvm\n artifact_from: build-test-jvm-packages\n - name: gpu\n image_repo: xgb-ci.jvm_gpu_build\n artifact_from: build-jvm-gpu\n scala_version: ['2.12', '2.13']\n container:\n image: ${{ needs.ci-configure.outputs.docker_registry }}/${{ matrix.variant.image_repo }}:${{ needs.ci-configure.outputs.image_tag }}\n credentials:\n username: ${{ needs.ci-configure.outputs.docker_username }}\n password: ${{ needs.ci-configure.outputs.docker_password }}\n steps:\n - uses: actions/checkout@v6.0.2\n with:\n submodules: \"true\"\n - name: Unstash files\n run: |\n python3 ops/pipeline/manage-artifacts.py download \\\n --s3-bucket ${{ env.RUNS_ON_S3_BUCKET_CACHE }} \\\n --prefix cache/${{ github.run_id }}/${{ matrix.variant.artifact_from }} \\\n --dest-dir lib \\\n libxgboost4j.so\n ls -lh lib/libxgboost4j.so\n - name: Deploy JVM packages to S3\n run: bash ops/pipeline/deploy-jvm-packages.sh ${{ matrix.variant.name }} ${{ matrix.scala_version }}\n if: github.ref == 'refs/heads/master' || startsWith(github.ref, 'refs/heads/release_')\n" }, { "path": ".github/workflows/lint.yml", "content": "name: XGBoost CI (Lint)\n\non:\n push:\n branches:\n - master\n - 'release_*'\n pull_request:\n\npermissions:\n contents: read # to fetch code (actions/checkout)\n\nconcurrency:\n group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}\n cancel-in-progress: true\n\nenv:\n BRANCH_NAME: >-\n ${{ github.event.pull_request.number && 'PR-' }}${{ github.event.pull_request.number || github.ref_name }}\n\njobs:\n clang-tidy:\n name: Run clang-tidy\n runs-on:\n - runs-on=${{ github.run_id }}\n - runner=linux-amd64-cpu\n - tag=lint-clang-tidy\n steps:\n - uses: actions/checkout@v6.0.2\n with:\n submodules: \"true\"\n - name: Log into Docker registry (AWS ECR)\n run: bash ops/pipeline/login-docker-registry.sh\n - run: bash ops/pipeline/run-clang-tidy.sh\n\n python-mypy:\n runs-on: ubuntu-latest\n name: Type checks for the Python package\n steps:\n - uses: actions/checkout@v6.0.2\n with:\n submodules: 'true'\n - uses: dmlc/xgboost-devops/actions/miniforge-setup@main\n with:\n environment-name: python_lint\n environment-file: ops/conda_env/python_lint.yml\n - name: Run mypy\n shell: bash -el {0}\n run: |\n python ops/script/type_check_python.py\n" }, { "path": ".github/workflows/main.yml", "content": "name: XGBoost CI\n\non:\n push:\n branches:\n - master\n - 'release_*'\n pull_request:\n\npermissions:\n contents: read # to fetch code (actions/checkout)\n\nconcurrency:\n group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}\n cancel-in-progress: true\n\nenv:\n BRANCH_NAME: >-\n ${{ github.event.pull_request.number && 'PR-' }}${{ github.event.pull_request.number || github.ref_name }}\n\njobs:\n ci-configure:\n name: Configure variables for CI\n runs-on:\n - runs-on=${{ github.run_id }}\n - runner=linux-amd64-cpu\n - tag=main-ci-configure\n steps:\n - name: Login to Amazon ECR\n id: login-ecr\n uses: aws-actions/amazon-ecr-login@v2.0.2\n with:\n mask-password: 'false'\n registries: '492475357299'\n - uses: actions/checkout@v6.0.2\n - name: Get image tag\n id: get-image-tag\n run: |\n source ops/pipeline/get-image-tag.sh\n echo \"Using image tag $IMAGE_TAG\"\n echo \"image_tag=$IMAGE_TAG\" >> \"$GITHUB_OUTPUT\"\n outputs:\n docker_registry: ${{ steps.login-ecr.outputs.registry }}\n docker_username: ${{ steps.login-ecr.outputs.docker_username_492475357299_dkr_ecr_us_west_2_amazonaws_com }}\n docker_password: ${{ steps.login-ecr.outputs.docker_password_492475357299_dkr_ecr_us_west_2_amazonaws_com }}\n image_tag: ${{ steps.get-image-tag.outputs.image_tag }}\n\n build-cpu:\n name: Build CPU (${{ matrix.variant }})\n needs: ci-configure\n runs-on:\n - runs-on=${{ github.run_id }}\n - runner=linux-amd64-cpu\n - tag=main-build-cpu-${{ matrix.variant }}\n strategy:\n fail-fast: false\n matrix:\n include:\n - variant: default\n build_suite: cpu\n # Default build doesn't need privileged mode\n # Using --init as harmless default (proper signal handling)\n container_options: \"--init\"\n - variant: sanitizer\n build_suite: cpu-sanitizer\n # Sanitizer needs privileged for: sysctl vm.mmap_rnd_bits=28\n # See https://github.com/google/sanitizers/issues/1614\n container_options: \"--privileged\"\n container:\n image: ${{ needs.ci-configure.outputs.docker_registry }}/xgb-ci.cpu:${{ needs.ci-configure.outputs.image_tag }}\n credentials:\n username: ${{ needs.ci-configure.outputs.docker_username }}\n password: ${{ needs.ci-configure.outputs.docker_password }}\n options: ${{ matrix.container_options }}\n steps:\n - uses: actions/checkout@v6.0.2\n with:\n submodules: \"true\"\n # Remove default build config to ensure CMake-configured header is used\n - name: Remove default build config\n run: rm -fv dmlc-core/include/dmlc/build_config_default.h\n - name: Configure the system for sanitizers\n if: matrix.variant == 'sanitizer'\n run: |\n echo \"ASAN_SYMBOLIZER_PATH=/usr/bin/llvm-symbolizer\" >> $GITHUB_ENV\n echo \"ASAN_OPTIONS=symbolize=1\" >> $GITHUB_ENV\n echo \"UBSAN_OPTIONS=print_stacktrace=1:log_path=ubsan_error.log\" >> $GITHUB_ENV\n # Work around https://github.com/google/sanitizers/issues/1614\n sysctl vm.mmap_rnd_bits=28\n - uses: dmlc/xgboost-devops/actions/sccache@main\n with:\n cache-key-prefix: ${{ github.job }}-${{ matrix.build_suite }}\n - name: Build and test\n run: bash ops/pipeline/build-cpu.sh ${{ matrix.build_suite }}\n - run: sccache --show-stats\n\n build-cuda:\n name: Build CUDA ${{ matrix.cuda_version }} (${{ matrix.arch }})\n needs: ci-configure\n runs-on:\n - runs-on=${{ github.run_id }}\n - runner=${{ matrix.runner }}\n - tag=main-build-cuda${{ matrix.cuda_version }}-${{ matrix.arch }}\n - extras=s3-cache\n strategy:\n fail-fast: false\n matrix:\n include:\n # CUDA 12\n - cuda_version: 12\n arch: aarch64\n runner: linux-arm64-cpu\n image_repo: xgb-ci.gpu_build_rockylinux8_aarch64\n use_rmm: 0\n use_federated: 1\n - cuda_version: 12\n arch: x86_64\n runner: linux-amd64-cpu\n image_repo: xgb-ci.gpu_build_rockylinux8\n use_rmm: 0\n use_federated: 1\n # CUDA 13\n - cuda_version: 13\n arch: aarch64\n runner: linux-arm64-cpu\n image_repo: xgb-ci.gpu_build_cuda13_rockylinux8_aarch64\n use_rmm: 0\n use_federated: 0\n - cuda_version: 13\n arch: x86_64\n runner: linux-amd64-cpu\n image_repo: xgb-ci.gpu_build_cuda13_rockylinux8\n use_rmm: 0\n use_federated: 0\n container:\n image: ${{ needs.ci-configure.outputs.docker_registry }}/${{ matrix.image_repo }}:${{ needs.ci-configure.outputs.image_tag }}\n credentials:\n username: ${{ needs.ci-configure.outputs.docker_username }}\n password: ${{ needs.ci-configure.outputs.docker_password }}\n steps:\n - uses: runs-on/action@v2\n - uses: actions/checkout@v6.0.2\n with:\n submodules: \"true\"\n - uses: dmlc/xgboost-devops/actions/sccache@main\n with:\n cache-key-prefix: ${{ github.job }}-${{ matrix.cuda_version }}\n - run: >-\n bash ops/pipeline/build-cuda.sh\n --cuda-version ${{ matrix.cuda_version }}\n --use-rmm ${{ matrix.use_rmm }}\n --use-federated ${{ matrix.use_federated }}\n - run: sccache --show-stats\n - name: Stash files\n run: |\n python3 ops/pipeline/manage-artifacts.py upload \\\n --s3-bucket ${{ env.RUNS_ON_S3_BUCKET_CACHE }} \\\n --prefix cache/${{ github.run_id }}/build-cuda${{ matrix.cuda_version }}-${{ matrix.arch }} \\\n build/testxgboost python-package/dist/*.whl\n\n audit-cuda-wheel:\n name: Audit CUDA ${{ matrix.cuda_version }} wheel for manylinux_2_28_${{ matrix.arch }}\n needs: [ci-configure, build-cuda]\n runs-on:\n - runs-on=${{ github.run_id }}\n - runner=${{ matrix.runner }}\n - tag=main-audit-cuda${{ matrix.cuda_version }}-wheel-${{ matrix.arch }}\n strategy:\n fail-fast: false\n matrix:\n include:\n # CUDA 12\n - cuda_version: 12\n cuda_variant_flag: \"\"\n arch: aarch64\n runner: linux-arm64-cpu\n - cuda_version: 12\n cuda_variant_flag: \"\"\n arch: x86_64\n runner: linux-amd64-cpu\n # CUDA 13\n - cuda_version: 13\n cuda_variant_flag: \"--cuda-variant cuda13\"\n arch: aarch64\n runner: linux-arm64-cpu\n - cuda_version: 13\n cuda_variant_flag: \"--cuda-variant cuda13\"\n arch: x86_64\n runner: linux-amd64-cpu\n container:\n image: ${{ needs.ci-configure.outputs.docker_registry }}/xgb-ci.manylinux_2_28_${{ matrix.arch }}:${{ needs.ci-configure.outputs.image_tag }}\n credentials:\n username: ${{ needs.ci-configure.outputs.docker_username }}\n password: ${{ needs.ci-configure.outputs.docker_password }}\n steps:\n - uses: actions/checkout@v6.0.2\n - name: Pick Python\n run: |\n export PATH=/opt/python/cp310-cp310/bin/:$PATH\n echo ${PATH} >> $GITHUB_PATH\n - name: Install dependencies\n run: |\n pip install awscli wheel auditwheel pydistcheck\n - name: Unstash raw wheel\n run: |\n mkdir -p python-package/dist\n python3 ops/pipeline/manage-artifacts.py download \\\n --s3-bucket ${{ env.RUNS_ON_S3_BUCKET_CACHE }} \\\n --prefix cache/${{ github.run_id }}/build-cuda${{ matrix.cuda_version }}-${{ matrix.arch }} \\\n --dest-dir python-package/dist \\\n *.whl\n - run: bash ops/pipeline/audit-cuda-wheel.sh ${{ matrix.arch }} ${{ matrix.cuda_variant_flag }}\n - name: Stash files\n run: |\n python3 ops/pipeline/manage-artifacts.py upload \\\n --s3-bucket ${{ env.RUNS_ON_S3_BUCKET_CACHE }} \\\n --prefix cache/${{ github.run_id }}/audit-cuda${{ matrix.cuda_version }}-wheel-${{ matrix.arch }} \\\n python-package/dist/*.whl\n\n build-cuda-with-rmm:\n name: Build CUDA with RMM\n needs: ci-configure\n runs-on:\n - runs-on=${{ github.run_id }}\n - runner=linux-amd64-cpu\n - tag=main-build-cuda-with-rmm\n - extras=s3-cache\n container:\n image: ${{ needs.ci-configure.outputs.docker_registry }}/xgb-ci.gpu_build_rockylinux8:${{ needs.ci-configure.outputs.image_tag }}\n credentials:\n username: ${{ needs.ci-configure.outputs.docker_username }}\n password: ${{ needs.ci-configure.outputs.docker_password }}\n steps:\n - uses: runs-on/action@v2\n - uses: actions/checkout@v6.0.2\n with:\n submodules: \"true\"\n - uses: dmlc/xgboost-devops/actions/sccache@main\n - run: >-\n bash ops/pipeline/build-cuda.sh\n --cuda-version 12\n --use-rmm 1\n --use-federated 1\n - run: sccache --show-stats\n - name: Stash files\n run: |\n python3 ops/pipeline/manage-artifacts.py upload \\\n --s3-bucket ${{ env.RUNS_ON_S3_BUCKET_CACHE }} \\\n --prefix cache/${{ github.run_id }}/build-cuda-with-rmm \\\n build/testxgboost\n\n build-python-wheels-cpu:\n name: Build CPU wheel (xgboost-cpu) for ${{ matrix.manylinux_target }}_${{ matrix.arch }}\n runs-on:\n - runs-on\n - runner=${{ matrix.runner }}\n - run-id=${{ github.run_id }}\n - tag=main-build-python-wheels-cpu-${{ matrix.manylinux_target }}-${{ matrix.arch }}\n strategy:\n fail-fast: false\n matrix:\n include:\n - manylinux_target: manylinux_2_28\n arch: aarch64\n runner: linux-arm64-cpu\n - manylinux_target: manylinux_2_28\n arch: x86_64\n runner: linux-amd64-cpu\n steps:\n - uses: actions/checkout@v6.0.2\n with:\n submodules: \"true\"\n - name: Log into Docker registry (AWS ECR)\n run: bash ops/pipeline/login-docker-registry.sh\n - run: |\n bash ops/pipeline/build-python-wheels-cpu.sh \\\n ${{ matrix.manylinux_target }} ${{ matrix.arch }}\n\n build-gpu-rpkg:\n name: Build GPU-enabled R package\n needs: ci-configure\n runs-on:\n - runs-on=${{ github.run_id }}\n - runner=linux-amd64-cpu\n - tag=main-build-gpu-rpkg\n - extras=s3-cache\n container:\n image: ${{ needs.ci-configure.outputs.docker_registry }}/xgb-ci.gpu_build_r_rockylinux8:${{ needs.ci-configure.outputs.image_tag }}\n credentials:\n username: ${{ needs.ci-configure.outputs.docker_username }}\n password: ${{ needs.ci-configure.outputs.docker_password }}\n steps:\n - uses: runs-on/action@v2\n - name: Trust git cloning project sources\n run: |\n git config --global --add safe.directory \"${GITHUB_WORKSPACE}\"\n - uses: actions/checkout@v6.0.2\n with:\n submodules: \"true\"\n - uses: dmlc/xgboost-devops/actions/sccache@main\n with:\n cache-key-prefix: ${{ github.job }}\n - run: bash ops/pipeline/build-gpu-rpkg.sh\n - run: sccache --show-stats\n - name: Upload R package tarball\n run: |\n python3 ops/pipeline/manage-artifacts.py upload \\\n --s3-bucket xgboost-nightly-builds \\\n --prefix ${BRANCH_NAME}/${GITHUB_SHA} --make-public \\\n xgboost_r_gpu_linux.tar.gz\n if: github.ref == 'refs/heads/master' || startsWith(github.ref, 'refs/heads/release_')\n\n test-cpp-gpu:\n name: >-\n Google Test (C++) CUDA ${{ matrix.cuda_version }}\n (${{ matrix.suite }}, ${{ matrix.runner }})\n needs: [ci-configure, build-cuda, build-cuda-with-rmm]\n runs-on:\n - runs-on=${{ github.run_id }}\n - runner=${{ matrix.runner }}\n - tag=main-test-cpp-gpu-cuda${{ matrix.cuda_version }}-${{ matrix.suite }}-${{ matrix.arch }}\n - extras=s3-cache\n timeout-minutes: 30\n strategy:\n fail-fast: false\n matrix:\n include:\n # CUDA 12 tests\n # Note: --gpus all provides GPU access; --privileged is not needed for basic GPU tests\n - cuda_version: 12\n suite: gpu\n arch: x86_64\n runner: linux-amd64-gpu\n image_repo: xgb-ci.gpu\n artifact_from: build-cuda12-x86_64\n container_options: \"--gpus all --privileged\"\n test_args: \"\"\n - cuda_version: 12\n suite: gpu\n arch: aarch64\n runner: linux-arm64-gpu\n image_repo: xgb-ci.gpu_aarch64\n artifact_from: build-cuda12-aarch64\n container_options: \"--gpus all --privileged\"\n test_args: \"\"\n - cuda_version: 12\n suite: gpu-rmm\n arch: x86_64\n runner: linux-amd64-gpu\n image_repo: xgb-ci.gpu\n artifact_from: build-cuda-with-rmm\n container_options: \"--gpus all --privileged\"\n test_args: \"--use-rmm-pool\"\n - cuda_version: 12\n suite: mgpu\n arch: x86_64\n runner: linux-amd64-mgpu\n image_repo: xgb-ci.gpu\n artifact_from: build-cuda12-x86_64\n # mgpu needs --shm-size for NCCL shared memory communication\n container_options: \"--gpus all --shm-size=4g --privileged\"\n test_args: \"--gtest_filter=*MGPU*\"\n # CUDA 13 tests\n - cuda_version: 13\n suite: gpu\n arch: x86_64\n runner: linux-amd64-gpu\n image_repo: xgb-ci.gpu_build_cuda13_rockylinux8\n artifact_from: build-cuda13-x86_64\n container_options: \"--gpus all --privileged\"\n test_args: \"\"\n - cuda_version: 13\n suite: gpu\n arch: aarch64\n runner: linux-arm64-gpu\n image_repo: xgb-ci.gpu_build_cuda13_rockylinux8_aarch64\n artifact_from: build-cuda13-aarch64\n container_options: \"--gpus all --privileged\"\n test_args: \"\"\n container:\n image: ${{ needs.ci-configure.outputs.docker_registry }}/${{ matrix.image_repo }}:${{ needs.ci-configure.outputs.image_tag }}\n credentials:\n username: ${{ needs.ci-configure.outputs.docker_username }}\n password: ${{ needs.ci-configure.outputs.docker_password }}\n options: ${{ matrix.container_options }}\n steps:\n - uses: runs-on/action@v2\n - uses: actions/checkout@v6.0.2\n - name: Unstash gtest\n run: |\n python3 ops/pipeline/manage-artifacts.py download \\\n --s3-bucket ${{ env.RUNS_ON_S3_BUCKET_CACHE }} \\\n --prefix cache/${{ github.run_id }}/${{ matrix.artifact_from }} \\\n --dest-dir build \\\n testxgboost\n chmod +x build/testxgboost\n - name: Run Google Tests (${{ matrix.suite }})\n run: build/testxgboost ${{ matrix.test_args }}\n\n test-python-wheel-gpu:\n name: Python tests CUDA ${{ matrix.cuda_version }} (${{ matrix.description }})\n needs: [ci-configure, audit-cuda-wheel]\n runs-on:\n - runs-on=${{ github.run_id }}\n - runner=${{ matrix.runner }}\n - tag=main-test-python-wheel-cuda${{ matrix.cuda_version }}-${{ matrix.description }}\n - extras=s3-cache\n timeout-minutes: 60\n strategy:\n fail-fast: false\n matrix:\n include:\n # CUDA 12 tests\n - cuda_version: 12\n description: GPU-x86_64-CUDA-12\n image_repo: xgb-ci.gpu\n suite: gpu\n runner: linux-amd64-gpu\n artifact_from: audit-cuda12-wheel-x86_64\n container_options: \"--gpus all --privileged\"\n - cuda_version: 12\n description: Multi-GPU-x86_64-CUDA-12\n image_repo: xgb-ci.gpu\n suite: mgpu\n runner: linux-amd64-mgpu\n artifact_from: audit-cuda12-wheel-x86_64\n # mgpu needs --shm-size for NCCL shared memory communication\n container_options: \"--gpus all --shm-size=4g --privileged\"\n # CUDA 12 aarch64 tests\n - cuda_version: 12\n description: GPU-arm64-CUDA-12\n image_repo: xgb-ci.gpu_aarch64\n suite: gpu-arm64\n runner: linux-arm64-gpu\n artifact_from: audit-cuda12-wheel-aarch64\n container_options: \"--gpus all --privileged\"\n # CUDA 13 tests\n - cuda_version: 13\n description: GPU-x86_64-CUDA-13\n image_repo: xgb-ci.gpu_build_cuda13_rockylinux8\n suite: gpu\n runner: linux-amd64-gpu\n artifact_from: audit-cuda13-wheel-x86_64\n container_options: \"--gpus all --privileged\"\n - cuda_version: 13\n description: GPU-arm64-CUDA-13\n image_repo: xgb-ci.gpu_build_cuda13_rockylinux8_aarch64\n suite: gpu-arm64\n runner: linux-arm64-gpu\n artifact_from: audit-cuda13-wheel-aarch64\n container_options: \"--gpus all --privileged\"\n container:\n image: ${{ needs.ci-configure.outputs.docker_registry }}/${{ matrix.image_repo }}:${{ needs.ci-configure.outputs.image_tag }}\n credentials:\n username: ${{ needs.ci-configure.outputs.docker_username }}\n password: ${{ needs.ci-configure.outputs.docker_password }}\n options: ${{ matrix.container_options }}\n steps:\n - uses: runs-on/action@v2\n - uses: actions/checkout@v6.0.2\n - name: Unstash Python wheel\n run: |\n python3 ops/pipeline/manage-artifacts.py download \\\n --s3-bucket ${{ env.RUNS_ON_S3_BUCKET_CACHE }} \\\n --prefix cache/${{ github.run_id }}/${{ matrix.artifact_from }} \\\n --dest-dir wheelhouse \\\n *.whl\n - name: Run Python tests (${{ matrix.description }})\n run: >-\n bash ops/pipeline/test-python-wheel.sh\n --suite ${{ matrix.suite }}\n --cuda-version ${{ matrix.cuda_version }}\n # Train a model for cross-platform testing (only for CUDA 12 x86_64)\n - name: Train cross-platform test model\n if: matrix.cuda_version == 12 && matrix.suite == 'gpu'\n shell: bash -l {0}\n run: |\n source activate gpu_test\n python tests/cross-platform/test_cross_platform_model.py \\\n --train --model-path cross_platform_model.ubj\n - name: Upload cross-platform model artifact\n if: matrix.cuda_version == 12 && matrix.suite == 'gpu'\n uses: actions/upload-artifact@v7.0.0\n with:\n name: cross-platform-model\n path: |\n cross_platform_model.ubj\n cross_platform_model.pkl\n retention-days: 1\n\n test-python-wheel-cpu:\n name: Python tests CPU (${{ matrix.description }})\n needs: [ci-configure, audit-cuda-wheel]\n runs-on:\n - runs-on=${{ github.run_id }}\n - runner=${{ matrix.runner }}\n - tag=main-test-python-wheel-cpu-${{ matrix.description }}\n - extras=s3-cache\n timeout-minutes: 60\n strategy:\n fail-fast: false\n # Uses the wheel from cuda12 for tests.\n matrix:\n include:\n - description: CPU-amd64\n image_repo: xgb-ci.cpu\n suite: cpu\n runner: linux-amd64-cpu\n artifact_from: audit-cuda12-wheel-x86_64\n container_options: \"--init\"\n - description: CPU-arm64\n image_repo: xgb-ci.cpu_aarch64\n suite: cpu-arm64\n runner: linux-arm64-cpu\n artifact_from: audit-cuda12-wheel-aarch64\n container_options: \"--init\"\n container:\n image: ${{ needs.ci-configure.outputs.docker_registry }}/${{ matrix.image_repo }}:${{ needs.ci-configure.outputs.image_tag }}\n credentials:\n username: ${{ needs.ci-configure.outputs.docker_username }}\n password: ${{ needs.ci-configure.outputs.docker_password }}\n options: ${{ matrix.container_options }}\n steps:\n - uses: runs-on/action@v2\n - uses: actions/checkout@v6.0.2\n - name: Unstash Python wheel\n run: |\n python3 ops/pipeline/manage-artifacts.py download \\\n --s3-bucket ${{ env.RUNS_ON_S3_BUCKET_CACHE }} \\\n --prefix cache/${{ github.run_id }}/${{ matrix.artifact_from }} \\\n --dest-dir wheelhouse \\\n *.whl\n - name: Run Python tests (${{ matrix.description }})\n run: bash ops/pipeline/test-python-wheel.sh --suite ${{ matrix.suite }}\n\n python-wheels-macos:\n name: Build macOS wheel (${{ matrix.platform_id }})\n runs-on: ${{ matrix.os }}\n defaults:\n run:\n shell: bash -l {0}\n strategy:\n fail-fast: false\n matrix:\n include:\n - os: macos-15-intel\n platform_id: macosx_x86_64\n - os: macos-14\n platform_id: macosx_arm64\n steps:\n - uses: actions/checkout@v6.0.2\n with:\n submodules: 'true'\n - name: Set up homebrew\n uses: Homebrew/actions/setup-homebrew@13341b4d5e459a98bbe0b122b12c11bf90518cc8\n - name: Install libomp\n run: brew install libomp\n - uses: dmlc/xgboost-devops/actions/miniforge-setup@main\n with:\n environment-name: minimal\n environment-file: ops/conda_env/minimal.yml\n - name: Build wheels\n run: bash ops/pipeline/build-python-wheels-macos.sh ${{ matrix.platform_id }} ${{ github.sha }}\n - name: Verify wheel can be installed\n run: |\n python -m pip install -vvv wheelhouse/*.whl\n - name: Upload wheel artifact\n uses: actions/upload-artifact@v7.0.0\n with:\n name: python-wheel-${{ matrix.platform_id }}\n path: wheelhouse/*.whl\n retention-days: 1\n - name: Upload Python wheel to S3\n if: github.ref == 'refs/heads/master' || contains(github.ref, 'refs/heads/release_')\n run: |\n python ops/pipeline/manage-artifacts.py upload \\\n --s3-bucket xgboost-nightly-builds \\\n --prefix ${{ env.BRANCH_NAME }}/${{ github.sha }} --make-public \\\n wheelhouse/*.whl\n env:\n AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID_IAM_S3_UPLOADER }}\n AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY_IAM_S3_UPLOADER }}\n\n test-cross-platform-inference:\n name: Cross-platform inference test (macOS Apple Silicon)\n needs: [test-python-wheel-gpu, python-wheels-macos]\n runs-on: ${{ matrix.os }}\n strategy:\n fail-fast: false\n matrix:\n include:\n - os: macos-15-intel\n platform_id: macosx_x86_64\n - os: macos-14\n platform_id: macosx_arm64\n defaults:\n run:\n shell: bash -l {0}\n steps:\n - uses: actions/checkout@v6.0.2\n - uses: dmlc/xgboost-devops/actions/miniforge-setup@main\n with:\n environment-name: macos_test\n environment-file: ops/conda_env/minimal.yml\n - run: conda install scikit-learn numpy -y\n - name: Download macOS wheel artifact\n uses: actions/download-artifact@v8.0.1\n with:\n name: python-wheel-${{ matrix.platform_id }}\n path: wheelhouse\n - name: Install XGBoost wheel\n run: |\n python -m pip install -v wheelhouse/*.whl\n - name: Download cross-platform model artifact\n uses: actions/download-artifact@v8.0.1\n with:\n name: cross-platform-model\n path: .\n - name: Run cross-platform inference test\n run: |\n python tests/cross-platform/test_cross_platform_model.py \\\n --inference --model-path cross_platform_model.ubj\n" }, { "path": ".github/workflows/misc.yml", "content": "name: Miscellaneous\n\non:\n push:\n branches:\n - master\n - 'release_*'\n pull_request:\n\npermissions:\n contents: read # to fetch code (actions/checkout)\n\nconcurrency:\n group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}\n cancel-in-progress: true\n\nenv:\n BRANCH_NAME: >-\n ${{ github.event.pull_request.number && 'PR-' }}${{ github.event.pull_request.number || github.ref_name }}\n\njobs:\n ci-configure:\n name: Configure variables for CI\n runs-on:\n - runs-on=${{ github.run_id }}\n - runner=linux-amd64-cpu\n - tag=misc-ci-configure\n steps:\n - name: Login to Amazon ECR\n id: login-ecr\n uses: aws-actions/amazon-ecr-login@v2.0.2\n with:\n mask-password: 'false'\n registries: '492475357299'\n - uses: actions/checkout@v6.0.2\n - name: Get image tag\n id: get-image-tag\n run: |\n source ops/pipeline/get-image-tag.sh\n echo \"Using image tag $IMAGE_TAG\"\n echo \"image_tag=$IMAGE_TAG\" >> \"$GITHUB_OUTPUT\"\n outputs:\n docker_registry: ${{ steps.login-ecr.outputs.registry }}\n docker_username: ${{ steps.login-ecr.outputs.docker_username_492475357299_dkr_ecr_us_west_2_amazonaws_com }}\n docker_password: ${{ steps.login-ecr.outputs.docker_password_492475357299_dkr_ecr_us_west_2_amazonaws_com }}\n image_tag: ${{ steps.get-image-tag.outputs.image_tag }}\n\n gtest-cpu-nonomp:\n name: Test Google C++ unittest (CPU Non-OMP)\n runs-on: ubuntu-latest\n defaults:\n run:\n shell: bash -l {0}\n steps:\n - uses: actions/checkout@v6.0.2\n with:\n submodules: 'true'\n - uses: dmlc/xgboost-devops/actions/miniforge-setup@main\n with:\n environment-name: cpp_test\n environment-file: ops/conda_env/cpp_test.yml\n - uses: dmlc/xgboost-devops/actions/sccache@main\n - name: Build and test XGBoost\n run: bash ops/pipeline/build-cpu.sh cpu-nonomp\n - run: sccache --show-stats\n\n c-api-demo:\n name: Test installing XGBoost lib + building the C API demo\n runs-on: ubuntu-latest\n defaults:\n run:\n shell: bash -l {0}\n steps:\n - uses: actions/checkout@v6.0.2\n with:\n submodules: 'true'\n - uses: dmlc/xgboost-devops/actions/miniforge-setup@main\n with:\n environment-name: cpp_test\n environment-file: ops/conda_env/cpp_test.yml\n - uses: dmlc/xgboost-devops/actions/sccache@main\n - name: Build and run C API demo with shared\n run: bash ops/pipeline/test-c-api-demo.sh\n\n build-i386:\n name: Build 32-bit (i386)\n runs-on:\n - runs-on=${{ github.run_id }}\n - runner=linux-amd64-cpu\n - tag=misc-build-i386\n steps:\n - uses: actions/checkout@v6.0.2\n with:\n submodules: \"true\"\n - name: Log into Docker registry (AWS ECR)\n run: bash ops/pipeline/login-docker-registry.sh\n - run: bash ops/pipeline/test-cpp-i386.sh\n\n build-jvm-docs:\n name: Build docs for JVM packages\n needs: ci-configure\n runs-on:\n - runs-on=${{ github.run_id }}\n - runner=linux-amd64-cpu\n - tag=misc-build-jvm-docs\n container:\n image: ${{ needs.ci-configure.outputs.docker_registry }}/xgb-ci.jvm_gpu_build:${{ needs.ci-configure.outputs.image_tag }}\n credentials:\n username: ${{ needs.ci-configure.outputs.docker_username }}\n password: ${{ needs.ci-configure.outputs.docker_password }}\n steps:\n - uses: runs-on/action@v2\n - uses: actions/checkout@v6.0.2\n with:\n submodules: \"true\"\n - uses: dmlc/xgboost-devops/actions/sccache@main\n - name: Build libxgboost4j.so with CUDA\n run: bash ops/pipeline/build-jvm-gpu.sh\n - run: sccache --show-stats\n - name: Build JVM docs\n run: bash ops/pipeline/build-jvm-doc.sh\n - name: Upload JVM doc\n run: |\n python3 ops/pipeline/manage-artifacts.py upload \\\n --s3-bucket xgboost-docs \\\n --prefix ${BRANCH_NAME}/${{ github.event.pull_request.head.sha || github.sha }} --make-public \\\n jvm-packages/${BRANCH_NAME}.tar.bz2\n\n build-r-docs:\n name: Build docs for the R package\n needs: ci-configure\n runs-on:\n - runs-on=${{ github.run_id }}\n - runner=linux-amd64-cpu\n - tag=misc-build-r-docs\n container:\n image: ${{ needs.ci-configure.outputs.docker_registry }}/xgb-ci.cpu_build_r_doc:${{ needs.ci-configure.outputs.image_tag }}\n credentials:\n username: ${{ needs.ci-configure.outputs.docker_username }}\n password: ${{ needs.ci-configure.outputs.docker_password }}\n steps:\n - uses: actions/checkout@v6.0.2\n with:\n submodules: \"true\"\n - uses: dmlc/xgboost-devops/actions/sccache@main\n - run: bash ops/script/setup_r_sccache.sh\n - name: Build R docs\n run: bash ops/pipeline/build-r-docs.sh\n - name: Upload R doc\n run: |\n python3 ops/pipeline/manage-artifacts.py upload \\\n --s3-bucket xgboost-docs \\\n --prefix ${BRANCH_NAME}/${{ github.event.pull_request.head.sha || github.sha }} --make-public \\\n r-docs-${BRANCH_NAME}.tar.bz2\n\n trigger-rtd-build:\n name: Trigger Read The Docs build\n needs: [build-jvm-docs, build-r-docs]\n runs-on:\n - runs-on=${{ github.run_id }}\n - runner=linux-amd64-cpu\n - tag=misc-trigger-rtd-build\n steps:\n - uses: actions/checkout@v6.0.2\n - name: Trigger RTD\n run: bash ops/pipeline/trigger-rtd.sh\n" }, { "path": ".github/workflows/pre-commit.yml", "content": "name: XGBoost CI (Pre-commit)\n\non:\n push:\n branches:\n - master\n - 'release_*'\n pull_request:\n\npermissions:\n contents: read\n\njobs:\n pre-commit:\n name: Run pre-commit\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v6.0.2\n with:\n fetch-depth: 0\n - uses: actions/setup-python@v6.2.0\n with:\n python-version: \"3.11\"\n - name: Install pre-commit\n run: python -m pip install pre-commit\n - name: Run pre-commit on updated files\n shell: bash\n run: |\n if [ \"${{ github.event_name }}\" = \"pull_request\" ]; then\n FROM_REF=\"${{ github.event.pull_request.base.sha }}\"\n TO_REF=\"${{ github.event.pull_request.head.sha }}\"\n else\n FROM_REF=\"${{ github.event.before }}\"\n TO_REF=\"${{ github.sha }}\"\n fi\n\n if [ \"${FROM_REF}\" = \"0000000000000000000000000000000000000000\" ]; then\n FROM_REF=\"$(git hash-object -t tree /dev/null)\"\n fi\n\n pre-commit run --from-ref \"${FROM_REF}\" --to-ref \"${TO_REF}\" --show-diff-on-failure\n" }, { "path": ".github/workflows/python_tests.yml", "content": "name: Python tests\n\non:\n push:\n branches:\n - master\n - 'release_*'\n pull_request:\n\npermissions:\n contents: read # to fetch code (actions/checkout)\n\ndefaults:\n run:\n shell: bash -l {0}\n\nconcurrency:\n group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}\n cancel-in-progress: true\n\njobs:\n python-sdist-test:\n runs-on: ${{ matrix.os }}\n name: Test installing Python XGBoost from the source distribution (${{ matrix.os }})\n strategy:\n fail-fast: false\n matrix:\n os: [macos-15-intel, windows-latest, ubuntu-latest]\n steps:\n - uses: actions/checkout@v6.0.2\n with:\n submodules: 'true'\n - uses: dmlc/xgboost-devops/actions/miniforge-setup@main\n with:\n environment-name: sdist_test\n environment-file: ops/conda_env/sdist_test.yml\n - name: Install extra package for MacOS\n run: |\n mamba install -c conda-forge llvm-openmp\n if: matrix.os == 'macos-15-intel'\n - name: Build and install XGBoost\n run: bash ops/pipeline/test-python-sdist.sh\n\n python-tests-on-macos:\n name: Test XGBoost Python package on macos-15-intel\n runs-on: macos-15-intel\n timeout-minutes: 60\n steps:\n - uses: actions/checkout@v6.0.2\n with:\n submodules: 'true'\n - uses: dmlc/xgboost-devops/actions/miniforge-setup@main\n with:\n environment-name: macos_cpu_test\n environment-file: ops/conda_env/macos_cpu_test.yml\n - run: bash ops/pipeline/test-python-macos.sh\n\n python-system-installation-on-ubuntu:\n name: Test XGBoost Python package System Installation on Ubuntu\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v6.0.2\n with:\n submodules: 'true'\n - name: Set up Python 3.10\n uses: actions/setup-python@v6.2.0\n with:\n python-version: \"3.10\"\n - run: bash ops/pipeline/test-python-with-sysprefix.sh\n" }, { "path": ".github/workflows/python_wheels_variants.yml", "content": "name: Build Python wheels using Wheel Variant prototype (WheelNext)\n\non:\n push:\n branches:\n - master\n - 'release_*'\n pull_request:\n\npermissions:\n contents: read # to fetch code (actions/checkout)\n\ndefaults:\n run:\n shell: bash -l {0}\n\nconcurrency:\n group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}\n cancel-in-progress: true\n\nenv:\n BRANCH_NAME: >-\n ${{ github.event.pull_request.number && 'PR-' }}${{ github.event.pull_request.number || github.ref_name }}\n\njobs:\n ecr-login:\n name: Login to AWS ECR\n runs-on:\n - runs-on=${{ github.run_id }}\n - runner=linux-amd64-cpu\n - tag=python-wheels-variants-ecr-login\n steps:\n - name: Login to Amazon ECR\n id: login-ecr\n uses: aws-actions/amazon-ecr-login@v2.0.2\n with:\n mask-password: 'false'\n registries: '492475357299'\n outputs:\n docker_registry: ${{ steps.login-ecr.outputs.registry }}\n docker_username: ${{ steps.login-ecr.outputs.docker_username_492475357299_dkr_ecr_us_west_2_amazonaws_com }}\n docker_password: ${{ steps.login-ecr.outputs.docker_password_492475357299_dkr_ecr_us_west_2_amazonaws_com }}\n\n build-variant-wheels:\n name: Build raw wheel for variant\n needs: ecr-login\n runs-on:\n - runs-on=${{ github.run_id }}\n - runner=linux-amd64-cpu\n - tag=python-wheels-variants-build\n container:\n image: ${{ needs.ecr-login.outputs.docker_registry }}/xgb-ci.gpu_build_rockylinux8:main\n credentials:\n username: ${{ needs.ecr-login.outputs.docker_username }}\n password: ${{ needs.ecr-login.outputs.docker_password }}\n steps:\n - uses: actions/checkout@v6.0.2\n with:\n submodules: \"true\"\n - uses: dmlc/xgboost-devops/actions/sccache@main\n with:\n cache-key-prefix: build-variant-wheels\n - run: bash ops/pipeline/build-variant-wheels.sh\n - run: sccache --show-stats\n - name: Stash files\n run: |\n python3 ops/pipeline/manage-artifacts.py upload \\\n --s3-bucket ${{ env.RUNS_ON_S3_BUCKET_CACHE }} \\\n --prefix cache/${{ github.run_id }}/build-variant-wheels \\\n python-package/dist/*.whl\n\n audit-variant-wheel:\n name: Audit variant wheel for manylinux_2_28_x86_64\n needs: [ecr-login, build-variant-wheels]\n runs-on:\n - runs-on=${{ github.run_id }}\n - runner=linux-amd64-cpu\n - tag=python-wheels-variants-audit\n container:\n image: ${{ needs.ecr-login.outputs.docker_registry }}/xgb-ci.manylinux_2_28_x86_64:main\n credentials:\n username: ${{ needs.ecr-login.outputs.docker_username }}\n password: ${{ needs.ecr-login.outputs.docker_password }}\n steps:\n - uses: actions/checkout@v6.0.2\n - name: Pick Python\n run: |\n export PATH=/opt/python/cp310-cp310/bin/:$PATH\n echo ${PATH} >> $GITHUB_PATH\n - name: Install dependencies\n run: |\n pip install awscli wheel auditwheel pydistcheck\n - name: Unstash raw wheel\n run: |\n mkdir -p python-package/dist\n python3 ops/pipeline/manage-artifacts.py download \\\n --s3-bucket ${{ env.RUNS_ON_S3_BUCKET_CACHE }} \\\n --prefix cache/${{ github.run_id }}/build-variant-wheels \\\n --dest-dir python-package/dist \\\n *.whl\n - name: Audit wheel\n run: |\n WHEEL_TAG=manylinux_2_28_x86_64\n echo \"--- Audit binary wheel to ensure it's compliant with ${WHEEL_TAG} standard\"\n auditwheel repair --only-plat --plat ${WHEEL_TAG} python-package/dist/*.whl\n python3 -m wheel tags --python-tag py3 --abi-tag none --platform ${WHEEL_TAG} --remove \\\n wheelhouse/*.whl\n mv -v wheelhouse/*.whl python-package/dist/\n - name: Stash files\n run: |\n python3 ops/pipeline/manage-artifacts.py upload \\\n --s3-bucket ${{ env.RUNS_ON_S3_BUCKET_CACHE }} \\\n --prefix cache/${{ github.run_id }}/audit-variant-wheel \\\n python-package/dist/*.whl\n\n convert-variant-wheel:\n name: Convert to variant wheel (WheelNext)\n needs: [ecr-login, audit-variant-wheel]\n runs-on:\n - runs-on=${{ github.run_id }}\n - runner=linux-amd64-cpu\n - tag=python-wheels-variants-convert\n container:\n image: ${{ needs.ecr-login.outputs.docker_registry }}/xgb-ci.gpu_build_rockylinux8:main\n credentials:\n username: ${{ needs.ecr-login.outputs.docker_username }}\n password: ${{ needs.ecr-login.outputs.docker_password }}\n steps:\n - uses: actions/checkout@v6.0.2\n with:\n submodules: \"true\"\n - name: Unstash audited wheel\n run: |\n mkdir -p python-package/dist\n python3 ops/pipeline/manage-artifacts.py download \\\n --s3-bucket ${{ env.RUNS_ON_S3_BUCKET_CACHE }} \\\n --prefix cache/${{ github.run_id }}/audit-variant-wheel \\\n --dest-dir python-package/dist \\\n *.whl\n - name: Convert to variant wheel\n run: bash ops/pipeline/build-variant-wheels-impl.sh\n" }, { "path": ".github/workflows/python_wheels_winarm64.yml", "content": "name: Build Python wheels targeting Windows ARM64\n\non:\n push:\n branches:\n - master\n - 'release_*'\n pull_request:\n\npermissions:\n contents: read # to fetch code (actions/checkout)\n\ndefaults:\n run:\n shell: pwsh\n\nconcurrency:\n group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}\n cancel-in-progress: true\n\nenv:\n BRANCH_NAME: >-\n ${{ github.event.pull_request.number && 'PR-' }}${{ github.event.pull_request.number || github.ref_name }}\n\njobs:\n python-wheels-Win-ARM64:\n name: Build wheel for Windows ARM64\n runs-on: windows-11-arm\n steps:\n - uses: actions/checkout@v6.0.2\n with:\n submodules: 'true'\n\n - name: Setup Python\n uses: actions/setup-python@v6.2.0\n with:\n python-version: '3.11'\n\n - name: Install build dependencies\n run: |\n python -m pip install --upgrade pip\n python -m pip install wheel setuptools awscli packaging\n\n - uses: dmlc/xgboost-devops/actions/msvc-dev-env@main\n - uses: dmlc/xgboost-devops/actions/sccache@main\n\n - name: Build XGBoost for Win-ARM64\n run: |\n mkdir build\n cd build\n cmake .. -G\"Ninja\" -DCMAKE_BUILD_TYPE=Release `\n -DCMAKE_C_COMPILER_LAUNCHER=sccache -DCMAKE_CXX_COMPILER_LAUNCHER=sccache\n cmake --build . -v\n - run: sccache --show-stats\n\n - name: Build Python wheel xgboost for Win-ARM64\n run: |\n cd python-package\n mkdir -p wheelhouse\n pip wheel --no-deps -v . --wheel-dir wheelhouse/\n $wheelFile = Get-ChildItem wheelhouse/*.whl | Select-Object -First 1 -ExpandProperty FullName\n python -m wheel tags --python-tag py3 --abi-tag none --platform win_arm64 --remove $wheelFile\n\n - name: Upload Python wheel xgboost\n if: github.ref == 'refs/heads/master' || contains(github.ref, 'refs/heads/release_')\n run: |\n $wheelFile = Get-ChildItem python-package/wheelhouse/*.whl | Select-Object -First 1 -ExpandProperty FullName\n python ops/pipeline/manage-artifacts.py upload `\n --s3-bucket xgboost-nightly-builds `\n --prefix ${{ env.BRANCH_NAME }}/${{ github.sha }} --make-public `\n $wheelFile\n env:\n AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID_IAM_S3_UPLOADER }}\n AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY_IAM_S3_UPLOADER }}\n\n - name: Clean up\n run: |\n $wheelFile = Get-ChildItem python-package/wheelhouse/*.whl | Select-Object -First 1 -ExpandProperty FullName\n Remove-Item -Path $wheelFile -Verbose\n\n - name: Build Python wheel xgboost-cpu for Win-ARM64\n run: |\n # Patch to rename pkg to xgboost-cpu\n python ops/script/pypi_variants.py --use-suffix=cpu --require-nccl-dep=na\n cd python-package\n pip wheel --no-deps -v . --wheel-dir wheelhouse/\n $wheelFile = Get-ChildItem wheelhouse/*.whl | Select-Object -First 1 -ExpandProperty FullName\n python -m wheel tags --python-tag py3 --abi-tag none --platform win_arm64 --remove $wheelFile\n\n - name: Upload Python wheel xgboost-cpu\n if: github.ref == 'refs/heads/master' || contains(github.ref, 'refs/heads/release_')\n run: |\n $wheelFile = Get-ChildItem python-package/wheelhouse/*.whl | Select-Object -First 1 -ExpandProperty FullName\n python ops/pipeline/manage-artifacts.py upload `\n --s3-bucket xgboost-nightly-builds `\n --prefix ${{ env.BRANCH_NAME }}/${{ github.sha }} --make-public `\n $wheelFile\n env:\n AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID_IAM_S3_UPLOADER }}\n AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY_IAM_S3_UPLOADER }}\n" }, { "path": ".github/workflows/r_nold.yml", "content": "# Run expensive R tests with the help of rhub. Only triggered by a pull request review\n# See discussion at https://github.com/dmlc/xgboost/pull/6378\n\nname: XGBoost-R-noLD\n\non:\n pull_request_review_comment:\n types: [created]\n\npermissions:\n contents: read # to fetch code (actions/checkout)\n\nconcurrency:\n group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}\n cancel-in-progress: true\n\njobs:\n test-R-noLD:\n if: github.event.comment.body == '/gha run r-nold-test' && contains('OWNER,MEMBER,COLLABORATOR', github.event.comment.author_association)\n timeout-minutes: 120\n runs-on: ubuntu-latest\n container:\n image: rhub/debian-gcc-devel-nold\n steps:\n - name: Install git and system packages\n shell: bash\n run: |\n apt update && apt install libcurl4-openssl-dev libssl-dev libssh2-1-dev libgit2-dev libglpk-dev libxml2-dev libharfbuzz-dev libfribidi-dev git -y\n - uses: actions/checkout@v6.0.2\n with:\n submodules: 'true'\n - name: Install dependencies\n shell: bash -l {0}\n run: |\n /tmp/R-devel/bin/Rscript -e \"source('./R-package/tests/helper_scripts/install_deps.R')\"\n - name: Run R tests\n shell: bash\n run: |\n cd R-package && \\\n /tmp/R-devel/bin/R CMD INSTALL . && \\\n /tmp/R-devel/bin/R -q -e \"library(testthat); setwd('tests'); source('testthat.R')\"\n" }, { "path": ".github/workflows/r_tests.yml", "content": "name: R Tests\n\non:\n push:\n branches:\n - master\n - 'release_*'\n pull_request:\n\nenv:\n GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }}\n\npermissions:\n contents: read # to fetch code (actions/checkout)\n\nconcurrency:\n group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}\n cancel-in-progress: true\n\njobs:\n test-Rpkg:\n runs-on: ${{ matrix.os }}\n name: Test R on OS ${{ matrix.os }}, R ${{ matrix.r }}, Compiler ${{ matrix.compiler }}, Build ${{ matrix.build }}\n strategy:\n fail-fast: false\n matrix:\n include:\n - os: windows-latest\n r: release\n compiler: mingw\n build: autotools\n - os: ubuntu-latest\n r: release\n compiler: none\n build: cmake\n env:\n R_REMOTES_NO_ERRORS_FROM_WARNINGS: true\n steps:\n - name: Install system dependencies\n run: |\n sudo apt update\n sudo apt install libcurl4-openssl-dev libssl-dev libssh2-1-dev libgit2-dev libglpk-dev libxml2-dev libharfbuzz-dev libfribidi-dev librsvg2-dev librsvg2-2\n if: matrix.os == 'ubuntu-latest'\n - uses: actions/checkout@v6.0.2\n with:\n submodules: 'true'\n - uses: r-lib/actions/setup-r@v2\n with:\n r-version: ${{ matrix.r }}\n - uses: actions/setup-python@v6.2.0\n with:\n python-version: \"3.10\"\n architecture: 'x64'\n - uses: r-lib/actions/setup-tinytex@v2\n - uses: dmlc/xgboost-devops/actions/sccache@main\n if: matrix.os == 'ubuntu-latest'\n - run: bash ops/script/setup_r_sccache.sh\n if: matrix.os == 'ubuntu-latest'\n - name: Install dependencies\n shell: Rscript {0}\n run: |\n source(\"./R-package/tests/helper_scripts/install_deps.R\")\n - name: Test R\n run: |\n python ops/script/test_r_package.py --compiler='${{ matrix.compiler }}' --build-tool=\"${{ matrix.build }}\" --task=check\n if: matrix.compiler != 'none'\n - name: Test R\n run: |\n python ops/script/test_r_package.py --build-tool=\"${{ matrix.build }}\" --task=check\n if: matrix.compiler == 'none'\n - run: sccache --show-stats\n if: matrix.os == 'ubuntu-latest'\n\n test-R-on-Debian:\n name: Test R package on Debian\n runs-on: ubuntu-latest\n container:\n image: rhub/ubuntu-release # rhub uses ubuntu for debian tests.\n steps:\n - name: Install system dependencies\n run: |\n # Must run before checkout to have the latest git installed.\n apt update && apt install libcurl4-openssl-dev libssl-dev libssh2-1-dev libgit2-dev libglpk-dev libxml2-dev libharfbuzz-dev libfribidi-dev git librsvg2-dev librsvg2-2 pandoc -y\n - name: Trust git cloning project sources\n run: |\n git config --global --add safe.directory \"${GITHUB_WORKSPACE}\"\n - uses: actions/checkout@v6.0.2\n with:\n submodules: 'true'\n - uses: dmlc/xgboost-devops/actions/sccache@main\n - run: bash ops/script/setup_r_sccache.sh\n - name: Install dependencies\n shell: bash -l {0}\n run: |\n Rscript -e \"source('./R-package/tests/helper_scripts/install_deps.R')\"\n - name: Test R\n shell: bash -l {0}\n run: |\n python3 ops/script/test_r_package.py --r=/opt/R/release/bin/R --build-tool=autotools --task=check\n - uses: dorny/paths-filter@v4\n # Run the document check if there are changes in the R package.\n id: changes\n with:\n filters: |\n r_package:\n - 'R-package/**'\n - name: Run document check\n if: steps.changes.outputs.r_package == 'true'\n run: |\n python3 ops/script/test_r_package.py --r=/opt/R/release/bin/R --task=doc\n - name: Run lintr\n run: |\n # Prevent the linter from checking generated R scripts.\n if [ -d ./xgboost.Rcheck ] ; then\n rm -rf ./xgboost.Rcheck\n fi\n MAKEFLAGS=\"-j$(nproc)\" R CMD INSTALL R-package/\n Rscript ops/script/lint_r.R $(pwd)\n - run: sccache --show-stats\n" }, { "path": ".github/workflows/scorecards.yml", "content": "name: Scorecards supply-chain security\non:\n # Only the default branch is supported.\n branch_protection_rule:\n schedule:\n - cron: '17 2 * * 6'\n push:\n branches: [ \"master\" ]\n\n# Declare default permissions as read only.\npermissions: read-all\n\njobs:\n analysis:\n name: Scorecards analysis\n runs-on: ubuntu-latest\n permissions:\n # Needed to upload the results to code-scanning dashboard.\n security-events: write\n # Used to receive a badge.\n id-token: write\n\n steps:\n - name: \"Checkout code\"\n uses: actions/checkout@v6.0.2\n with:\n persist-credentials: false\n\n - name: \"Run analysis\"\n uses: ossf/scorecard-action@4eaacf0543bb3f2c246792bd56e8cdeffafb205a # v2.4.3\n with:\n results_file: results.sarif\n results_format: sarif\n\n # Publish the results for public repositories to enable scorecard badges. For more details, see\n # https://github.com/ossf/scorecard-action#publishing-results.\n # For private repositories, `publish_results` will automatically be set to `false`, regardless\n # of the value entered here.\n publish_results: true\n\n # Upload the results as artifacts (optional). Commenting out will disable uploads of run results in SARIF\n # format to the repository Actions tab.\n - name: \"Upload artifact\"\n uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v4.4.3\n with:\n name: SARIF file\n path: results.sarif\n retention-days: 5\n\n # Upload the results to GitHub's code scanning dashboard.\n - name: \"Upload to code-scanning\"\n uses: github/codeql-action/upload-sarif@83a02f7883b12e0e4e1a146174f5e2292a01e601 # v2.16.4\n with:\n sarif_file: results.sarif\n" }, { "path": ".github/workflows/sycl_tests.yml", "content": "name: XGBoost CI (oneAPI)\n\non:\n push:\n branches:\n - master\n - 'release_*'\n pull_request:\n\npermissions:\n contents: read # to fetch code (actions/checkout)\n\ndefaults:\n run:\n shell: bash -l {0}\n\nconcurrency:\n group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}\n cancel-in-progress: true\n\nenv:\n BRANCH_NAME: >-\n ${{ github.event.pull_request.number && 'PR-' }}${{ github.event.pull_request.number || github.ref_name }}\n\njobs:\n gtest-cpu-sycl:\n name: Test Google C++ unittest (CPU SYCL)\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v6.0.2\n with:\n submodules: 'true'\n - uses: dmlc/xgboost-devops/actions/miniforge-setup@main\n with:\n environment-name: linux_sycl_test\n environment-file: ops/conda_env/linux_sycl_test.yml\n - name: Run gtest\n run: bash ops/pipeline/build-test-sycl.sh gtest\n\n python-sycl-tests-on-ubuntu:\n name: Test XGBoost Python package with SYCL\n runs-on: ubuntu-latest\n timeout-minutes: 90\n steps:\n - uses: actions/checkout@v6.0.2\n with:\n submodules: 'true'\n - uses: dmlc/xgboost-devops/actions/miniforge-setup@main\n with:\n environment-name: linux_sycl_test\n environment-file: ops/conda_env/linux_sycl_test.yml\n - name: Test Python package\n run: bash ops/pipeline/build-test-sycl.sh pytest\n" }, { "path": ".github/workflows/windows.yml", "content": "name: Windows\n\non:\n push:\n branches:\n - master\n - 'release_*'\n pull_request:\n\npermissions:\n contents: read # to fetch code (actions/checkout)\n\nconcurrency:\n group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}\n cancel-in-progress: true\n\ndefaults:\n run:\n shell: powershell\n\nenv:\n BRANCH_NAME: >-\n ${{ github.event.pull_request.number && 'PR-' }}${{ github.event.pull_request.number || github.ref_name }}\n\njobs:\n build-win64-gpu:\n name: Build XGBoost for Windows with CUDA\n runs-on:\n - runs-on=${{ github.run_id }}\n - runner=windows-cpu\n - tag=windows-build-win64-gpu\n steps:\n - uses: actions/checkout@v6.0.2\n with:\n submodules: \"true\"\n - uses: dmlc/xgboost-devops/actions/msvc-dev-env@main\n - uses: dmlc/xgboost-devops/actions/sccache@main\n - run: ops/pipeline/build-win64.ps1 -variant gpu\n - run: sccache --show-stats\n - name: Stash files\n shell: powershell\n run: |\n conda activate\n python ops/pipeline/manage-artifacts.py upload `\n --s3-bucket ${{ env.RUNS_ON_S3_BUCKET_CACHE }} `\n --prefix cache/${{ github.run_id }}/build-win64-gpu `\n build/testxgboost.exe `\n (Get-ChildItem python-package/dist/*.whl | Select-Object -Expand FullName)\n\n build-win64-cpu:\n name: Build XGBoost for Windows (minimal)\n runs-on:\n - runs-on=${{ github.run_id }}\n - runner=windows-cpu\n - tag=windows-build-win64-cpu\n steps:\n - uses: actions/checkout@v6.0.2\n with:\n submodules: \"true\"\n - uses: dmlc/xgboost-devops/actions/msvc-dev-env@main\n - uses: dmlc/xgboost-devops/actions/sccache@main\n - run: ops/pipeline/build-win64.ps1 -variant cpu\n - run: sccache --show-stats\n\n test-win64-gpu:\n name: Test XGBoost on Windows\n needs: build-win64-gpu\n runs-on:\n - runs-on=${{ github.run_id }}\n - runner=windows-gpu\n - tag=windows-test-win64-gpu\n steps:\n - uses: actions/checkout@v6.0.2\n with:\n submodules: \"true\"\n - name: Unstash files\n shell: powershell\n run: |\n conda activate\n python ops/pipeline/manage-artifacts.py download `\n --s3-bucket ${{ env.RUNS_ON_S3_BUCKET_CACHE }} `\n --prefix cache/${{ github.run_id }}/build-win64-gpu `\n --dest-dir build `\n *.whl testxgboost.exe\n New-Item -ItemType Directory -Path python-package/dist/ -Force\n Move-Item -Path (Get-ChildItem build/*.whl | Select-Object -Expand FullName) `\n -Destination python-package/dist/\n - run: powershell ops/pipeline/test-win64-gpu.ps1\n" }, { "path": ".gitignore", "content": "# Compiled Object files\n*.slo\n*.lo\n*.o\n*.page\n# Compiled Dynamic libraries\n*.so\n*.dylib\n*.page\n# Compiled Static libraries\n*.lai\n*.la\n*.a\n*~\n*.Rcheck\n*.rds\n*.tar.gz\n*.tar.bz2\n*conf\n*buffer\n*.model\n*pyc\n*.train\n*.test\n*.tar\n*group\n*rar\n*vali\n*sdf\nRelease\n*exe\n*exp\nipch\n*.filters\n*.user\n*log\nrmm_log.txt\nDebug\n*suo\n.Rhistory\n*.dll\n*i386\n*x64\n*dump\n*save\n*csv\n.Rproj.user\n*.cpage.col\n*.cpage\n*.Rproj\n./xgboost.mpi\n./xgboost.mock\n*.bak\n#.Rbuildignore\nR-package.Rproj\nR-package/build/*\n*.cache*\n.mypy_cache/\ndoxygen\n\n# java\njava/xgboost4j/target\njava/xgboost4j/tmp\njava/xgboost4j-demo/target\njava/xgboost4j-demo/data/\njava/xgboost4j-demo/tmp/\njava/xgboost4j-demo/model/\nnb-configuration*\n*.xml.versionsBackup\n\n# Eclipse\n.project\n.cproject\n.classpath\n.pydevproject\n.settings/\n/build\n/build-gpu\n/xgboost\n*.data\nbuild_plugin\nrecommonmark/\ntags\nTAGS\n*.class\ntarget\n*.swp\n\n# cpp tests and gcov generated files\n*.gcov\n*.gcda\n*.gcno\n*.ubj\nbuild_tests\n/tests/cpp/xgboost_test\n\n.DS_Store\nlib/\n\n# spark\nmetastore_db\n\n/include/xgboost/build_config.h\n\n# files from R-package source install\n**/config.status\nR-package/config.h\nR-package/src/Makevars\n*.lib\n\n# Visual Studio\n.vs/\nCMakeSettings.json\n*.ilk\n*.pdb\n\n# IntelliJ/CLion\n.idea\n*.iml\n/cmake-build-debug/\n\n# GDB\n.gdb_history\n\n# Python joblib.Memory used in pytest.\ncachedir/\n\n# Files from local Dask work\ndask-worker-space/\n\n# Jupyter notebook checkpoints\n.ipynb_checkpoints/\n\n# credentials and key material\nconfig\ncredentials\ncredentials.csv\n*.env\n*.pem\n*.pub\n*.rdp\n*_rsa\n\n# Visual Studio code + extensions\n.vscode\n.metals\n.bloop\n\n# python tests\n*.bin\ndemo/**/*.txt\n*.dmatrix\n.hypothesis\n__MACOSX/\nmodel*.json\n/tests/python/models/models/\n\n# R tests\n*.htm\n*.html\n*.libsvm\n*.rds\nRplots.pdf\n*.zip\n\n# nsys\n*.nsys-rep\nrmm_log.dev*" }, { "path": ".gitmodules", "content": "[submodule \"dmlc-core\"]\n\tpath = dmlc-core\n\turl = https://github.com/dmlc/dmlc-core\n\tbranch = main\n[submodule \"gputreeshap\"]\n\tpath = gputreeshap\n\turl = https://github.com/rapidsai/gputreeshap.git\n" }, { "path": ".pre-commit-config.yaml", "content": "repos:\n - repo: https://github.com/pre-commit/pre-commit-hooks\n rev: v4.6.0\n hooks:\n - id: trailing-whitespace\n - id: end-of-file-fixer\n - id: check-merge-conflict\n - id: check-case-conflict\n - id: check-yaml\n - id: check-toml\n - id: check-json\n - id: check-executables-have-shebangs\n - id: check-shebang-scripts-are-executable\n - id: mixed-line-ending\n - repo: https://github.com/astral-sh/ruff-pre-commit\n rev: v0.14.14\n hooks:\n - id: ruff-check\n args:\n - --fix\n - --select=I\n files: \\.py$\n exclude: (dmlc-core|gputreeshap)\n - id: ruff-format\n files: \\.py$\n exclude: (dmlc-core|gputreeshap)\n - repo: https://github.com/pre-commit/mirrors-clang-format\n rev: v21.1.8\n hooks:\n - id: clang-format\n args: [--style=file]\n files: \\.(cc|c|cpp|h|cu|hpp)$\n exclude: (dmlc-core|gputreeshap)\n - repo: local\n hooks:\n - id: lint-cpp\n name: C++ lint (cpplint)\n entry: python ops/script/lint_cpp.py\n language: python\n types: [file]\n files: \\.(cc|c|cpp|h|cu|hpp)$\n exclude: (dmlc-core|gputreeshap)\n additional_dependencies:\n - cpplint==1.6.1\n - id: lint-cmake\n name: CMake lint (cmakelint)\n entry: cmakelint --linelength=120 --filter=-convention/filename,-package/stdargs,-readability/wonkycase\n language: python\n types: [file]\n files: (CMakeLists\\.txt$|^cmake/.*\\.cmake$)\n exclude: (dmlc-core|gputreeshap)\n additional_dependencies:\n - cmakelint\n - id: pylint\n name: Python lint (pylint)\n entry: pylint\n language: python\n types: [file]\n files: \\.py$\n exclude: (dmlc-core|gputreeshap|^ops/|^tests/)\n args:\n - --rcfile=python-package/pyproject.toml\n additional_dependencies:\n - pylint\n" }, { "path": ".readthedocs.yaml", "content": "# .readthedocs.yaml\n# Read the Docs configuration file\n# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details\n\n# Required\nversion: 2\n\nsubmodules:\n include: all\n\n# Set the version of Python and other tools you might need\nbuild:\n os: ubuntu-22.04\n tools:\n python: \"3.10\"\n apt_packages:\n - graphviz\n - cmake\n - g++\n - doxygen\n - ninja-build\n\n# Build documentation in the docs/ directory with Sphinx\nsphinx:\n configuration: doc/conf.py\n\n# If using Sphinx, optionally build your docs in additional formats such as PDF\nformats:\n - pdf\n\n# Optionally declare the Python requirements required to build your docs\npython:\n install:\n - requirements: doc/requirements.txt\n" }, { "path": "CITATION", "content": "@inproceedings{Chen:2016:XST:2939672.2939785,\n author = {Chen, Tianqi and Guestrin, Carlos},\n title = {{XGBoost}: A Scalable Tree Boosting System},\n booktitle = {Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining},\n series = {KDD '16},\n year = {2016},\n isbn = {978-1-4503-4232-2},\n location = {San Francisco, California, USA},\n pages = {785--794},\n numpages = {10},\n url = {http://doi.acm.org/10.1145/2939672.2939785},\n doi = {10.1145/2939672.2939785},\n acmid = {2939785},\n publisher = {ACM},\n address = {New York, NY, USA},\n keywords = {large-scale machine learning},\n}\n" }, { "path": "CMakeLists.txt", "content": "cmake_minimum_required(VERSION 3.18 FATAL_ERROR)\n\nif(PLUGIN_SYCL)\n string(REPLACE \" -isystem ${CONDA_PREFIX}/include\" \"\" CMAKE_CXX_FLAGS \"${CMAKE_CXX_FLAGS}\")\nendif()\n\nproject(xgboost LANGUAGES CXX C VERSION 3.3.0)\ninclude(cmake/Utils.cmake)\nlist(APPEND CMAKE_MODULE_PATH \"${xgboost_SOURCE_DIR}/cmake/modules\")\n\n# These policies are already set from 3.18 but we still need to set the policy\n# default variables here for lower minimum versions in the submodules\nset(CMAKE_POLICY_DEFAULT_CMP0063 NEW)\nset(CMAKE_POLICY_DEFAULT_CMP0069 NEW)\nset(CMAKE_POLICY_DEFAULT_CMP0076 NEW)\nset(CMAKE_POLICY_DEFAULT_CMP0077 NEW)\nset(CMAKE_POLICY_DEFAULT_CMP0079 NEW)\n\nmessage(STATUS \"CMake version ${CMAKE_VERSION}\")\n\n# Check compiler versions\n# Use recent compilers to ensure that std::filesystem is available\nif(MSVC)\n if(MSVC_VERSION LESS 1920)\n message(FATAL_ERROR \"Need Visual Studio 2019 or newer to build XGBoost\")\n endif()\nelseif(CMAKE_CXX_COMPILER_ID STREQUAL \"GNU\")\n if(CMAKE_CXX_COMPILER_VERSION VERSION_LESS \"8.1\")\n message(FATAL_ERROR \"Need GCC 8.1 or newer to build XGBoost\")\n endif()\nelseif(CMAKE_CXX_COMPILER_ID STREQUAL \"AppleClang\")\n if(CMAKE_CXX_COMPILER_VERSION VERSION_LESS \"11.0\")\n message(FATAL_ERROR \"Need Xcode 11.0 (AppleClang 11.0) or newer to build XGBoost\")\n endif()\nelseif(CMAKE_CXX_COMPILER_ID STREQUAL \"Clang\")\n if(CMAKE_CXX_COMPILER_VERSION VERSION_LESS \"9.0\")\n message(FATAL_ERROR \"Need Clang 9.0 or newer to build XGBoost\")\n endif()\nendif()\n\ninclude(${xgboost_SOURCE_DIR}/cmake/PrefetchIntrinsics.cmake)\nfind_prefetch_intrinsics()\ninclude(${xgboost_SOURCE_DIR}/cmake/Version.cmake)\nwrite_version()\nset_default_configuration_release()\n\n#-- Options\ninclude(CMakeDependentOption)\n\n## User options\noption(BUILD_C_DOC \"Build documentation for C APIs using Doxygen.\" OFF)\noption(USE_OPENMP \"Build with OpenMP support.\" ON)\noption(BUILD_STATIC_LIB \"Build static library\" OFF)\noption(BUILD_DEPRECATED_CLI \"Build the deprecated command line interface\" OFF)\noption(FORCE_SHARED_CRT \"Build with dynamic CRT on Windows (/MD)\" OFF)\noption(BUILD_WITH_GIT_HASH \"Add a short git hash to the build info.\" OFF)\noption(BUILD_WITH_SYSTEM_DMLC \"Use system package for dmlc.\" OFF)\n## Bindings\noption(JVM_BINDINGS \"Build JVM bindings\" OFF)\noption(R_LIB \"Build shared library for R package\" OFF)\n## Dev\noption(USE_DEBUG_OUTPUT \"Dump internal training results like gradients and predictions to stdout.\nShould only be used for debugging.\" OFF)\noption(FORCE_COLORED_OUTPUT \"Force colored output from compilers, useful when ninja is used instead of make.\" OFF)\noption(ENABLE_ALL_WARNINGS \"Enable all compiler warnings. Only effective for GCC/Clang\" OFF)\noption(LOG_CAPI_INVOCATION \"Log all C API invocations for debugging\" OFF)\noption(GOOGLE_TEST \"Build google tests\" OFF)\noption(USE_DMLC_GTEST \"Use google tests bundled with dmlc-core submodule\" OFF)\noption(USE_DEVICE_DEBUG \"Generate CUDA device debug info.\" OFF)\noption(USE_NVTX \"Build with cuda profiling annotations. Developers only.\" OFF)\nset(NVTX_HEADER_DIR \"\" CACHE PATH \"Path to the stand-alone nvtx header\")\noption(HIDE_CXX_SYMBOLS \"Build shared library and hide all C++ symbols\" OFF)\noption(KEEP_BUILD_ARTIFACTS_IN_BINARY_DIR \"Output build artifacts in CMake binary dir\" OFF)\n## CUDA\noption(USE_CUDA \"Build with GPU acceleration\" OFF)\noption(USE_NCCL \"Build with NCCL to enable distributed GPU support.\" OFF)\noption(USE_NVCOMP \"Build with nvcomp to enable sparse data compression. (experimental)\" OFF)\n# This is specifically designed for PyPI binary release and should be disabled for most of the cases.\noption(USE_DLOPEN_NCCL \"Whether to load nccl dynamically.\" OFF)\noption(BUILD_WITH_SHARED_NCCL \"Build with shared NCCL library.\" OFF)\n\nif(USE_CUDA)\n if(NOT DEFINED CMAKE_CUDA_ARCHITECTURES AND NOT DEFINED ENV{CUDAARCHS})\n set(GPU_COMPUTE_VER \"\" CACHE STRING\n \"Semicolon separated list of compute versions to be built against, e.g. '35;61'\")\n else()\n # Clear any cached values from previous runs\n unset(GPU_COMPUTE_VER)\n unset(GPU_COMPUTE_VER CACHE)\n endif()\nendif()\n\n# CUDA device LTO was introduced in CMake v3.25 and requires host LTO to also be enabled but can still\n# be explicitly disabled allowing for LTO on host only, host and device, or neither, but device-only LTO\n# is not a supproted configuration\ncmake_dependent_option(USE_CUDA_LTO\n \"Enable link-time optimization for CUDA device code\"\n \"${CMAKE_INTERPROCEDURAL_OPTIMIZATION}\"\n \"CMAKE_VERSION VERSION_GREATER_EQUAL 3.25;USE_CUDA;CMAKE_INTERPROCEDURAL_OPTIMIZATION\"\n OFF)\n## Sanitizers\noption(USE_SANITIZER \"Use santizer flags\" OFF)\noption(SANITIZER_PATH \"Path to sanitizes.\")\nset(ENABLED_SANITIZERS \"address\" \"leak\" CACHE STRING\n \"Semicolon separated list of sanitizer names. E.g 'address;leak'. Supported sanitizers are\naddress, leak, undefined and thread.\")\n## Plugins\noption(PLUGIN_RMM \"Build with RAPIDS Memory Manager (RMM)\" OFF)\noption(PLUGIN_FEDERATED \"Build with Federated Learning\" OFF)\n## TODO: 1. Add check if DPC++ compiler is used for building\noption(PLUGIN_SYCL \"SYCL plugin\" OFF)\noption(ADD_PKGCONFIG \"Add xgboost.pc into system.\" ON)\n\n#-- Checks for building XGBoost\nif(USE_DEBUG_OUTPUT AND (NOT (CMAKE_BUILD_TYPE MATCHES Debug)))\n message(SEND_ERROR \"Do not enable `USE_DEBUG_OUTPUT' with release build.\")\nendif()\nif(USE_NVTX AND (NOT USE_CUDA))\n message(SEND_ERROR \"`USE_NVTX` must be enabled with `USE_CUDA` flag.\")\nendif()\nif(USE_NVTX)\n if(CMAKE_VERSION VERSION_LESS \"3.25.0\")\n # CUDA:nvtx3 target is added in 3.25\n message(\"cmake >= 3.25 is required for NVTX.\")\n endif()\nendif()\nif(USE_NCCL AND (NOT USE_CUDA))\n message(SEND_ERROR \"`USE_NCCL` must be enabled with `USE_CUDA` flag.\")\nendif()\nif(USE_NVCOMP AND (NOT USE_CUDA))\n message(SEND_ERROR \"`USE_NVCOMP` must be enabled with `USE_CUDA` flag.\")\nendif()\nif(USE_DEVICE_DEBUG AND (NOT USE_CUDA))\n message(SEND_ERROR \"`USE_DEVICE_DEBUG` must be enabled with `USE_CUDA` flag.\")\nendif()\nif(BUILD_WITH_SHARED_NCCL AND (NOT USE_NCCL))\n message(SEND_ERROR \"Build XGBoost with -DUSE_NCCL=ON to enable BUILD_WITH_SHARED_NCCL.\")\nendif()\nif(USE_DLOPEN_NCCL AND (NOT USE_NCCL))\n message(SEND_ERROR \"Build XGBoost with -DUSE_NCCL=ON to enable USE_DLOPEN_NCCL.\")\nendif()\nif(USE_DLOPEN_NCCL AND (NOT (CMAKE_SYSTEM_NAME STREQUAL \"Linux\")))\n message(SEND_ERROR \"`USE_DLOPEN_NCCL` supports only Linux at the moment.\")\nendif()\nif(JVM_BINDINGS AND R_LIB)\n message(SEND_ERROR \"`R_LIB' is not compatible with `JVM_BINDINGS' as they both have customized configurations.\")\nendif()\nif(R_LIB AND GOOGLE_TEST)\n message(\n WARNING\n \"Some C++ tests will fail with `R_LIB` enabled, as R package redirects some functions to R runtime implementation.\"\n )\nendif()\nif(R_LIB AND USE_NCCL)\n message(SEND_ERROR \"`R_LIB` doesn't support distributed computing with NCCL yet.\")\nendif()\nif(PLUGIN_RMM AND NOT (USE_CUDA))\n message(SEND_ERROR \"`PLUGIN_RMM` must be enabled with `USE_CUDA` flag.\")\nendif()\nif(PLUGIN_RMM AND NOT ((CMAKE_CXX_COMPILER_ID STREQUAL \"Clang\") OR (CMAKE_CXX_COMPILER_ID STREQUAL \"GNU\")))\n message(SEND_ERROR \"`PLUGIN_RMM` must be used with GCC or Clang compiler.\")\nendif()\nif(PLUGIN_RMM AND NOT (CMAKE_SYSTEM_NAME STREQUAL \"Linux\"))\n message(SEND_ERROR \"`PLUGIN_RMM` must be used with Linux.\")\nendif()\nif(ENABLE_ALL_WARNINGS)\n if((NOT CMAKE_CXX_COMPILER_ID MATCHES \"Clang\") AND (NOT CMAKE_CXX_COMPILER_ID STREQUAL \"GNU\"))\n message(SEND_ERROR \"ENABLE_ALL_WARNINGS is only available for Clang and GCC.\")\n endif()\nendif()\nif(BUILD_STATIC_LIB AND (R_LIB OR JVM_BINDINGS))\n message(SEND_ERROR \"Cannot build a static library libxgboost.a when R or JVM packages are enabled.\")\nendif()\nif(PLUGIN_FEDERATED)\n if(CMAKE_CROSSCOMPILING)\n message(SEND_ERROR \"Cannot cross compile with federated learning support\")\n endif()\n if(BUILD_STATIC_LIB)\n message(SEND_ERROR \"Cannot build static lib with federated learning support\")\n endif()\n if(R_LIB OR JVM_BINDINGS)\n message(SEND_ERROR \"Cannot enable federated learning support when R or JVM packages are enabled.\")\n endif()\n if(WIN32)\n message(SEND_ERROR \"Federated learning not supported for Windows platform\")\n endif()\nendif()\n\n#-- Removed options\nif(USE_AVX)\n message(SEND_ERROR \"The option `USE_AVX` is deprecated as experimental AVX features have been removed from XGBoost.\")\nendif()\nif(PLUGIN_LZ4)\n message(SEND_ERROR \"The option `PLUGIN_LZ4` is removed from XGBoost.\")\nendif()\nif(RABIT_BUILD_MPI)\n message(SEND_ERROR \"The option `RABIT_BUILD_MPI` has been removed from XGBoost.\")\nendif()\nif(USE_S3)\n message(SEND_ERROR \"The option `USE_S3` has been removed from XGBoost\")\nendif()\nif(USE_AZURE)\n message(SEND_ERROR \"The option `USE_AZURE` has been removed from XGBoost\")\nendif()\nif(USE_HDFS)\n message(SEND_ERROR \"The option `USE_HDFS` has been removed from XGBoost\")\nendif()\nif(PLUGIN_DENSE_PARSER)\n message(SEND_ERROR \"The option `PLUGIN_DENSE_PARSER` has been removed from XGBoost.\")\nendif()\nif(BUILD_DEPRECATED_CLI)\n message(SEND_ERROR \"The option `BUILD_DEPRECATED_CLI` is removed from XGBoost.\")\nendif()\n\n#-- Sanitizer\nif(USE_SANITIZER)\n include(cmake/Sanitizer.cmake)\n enable_sanitizers(\"${ENABLED_SANITIZERS}\")\nendif()\n\nif(USE_CUDA)\n set(USE_OPENMP ON CACHE BOOL \"CUDA requires OpenMP\" FORCE)\n # `export CXX=' is ignored by CMake CUDA.\n if(NOT DEFINED CMAKE_CUDA_HOST_COMPILER AND NOT DEFINED ENV{CUDAHOSTCXX})\n set(CMAKE_CUDA_HOST_COMPILER ${CMAKE_CXX_COMPILER} CACHE FILEPATH\n \"The compiler executable to use when compiling host code for CUDA or HIP language files.\")\n mark_as_advanced(CMAKE_CUDA_HOST_COMPILER)\n message(STATUS \"Configured CUDA host compiler: ${CMAKE_CUDA_HOST_COMPILER}\")\n endif()\n\n if(NOT DEFINED CMAKE_CUDA_RUNTIME_LIBRARY)\n set(CMAKE_CUDA_RUNTIME_LIBRARY Static)\n endif()\n\n enable_language(CUDA)\n if(${CMAKE_CUDA_COMPILER_VERSION} VERSION_LESS 12.9)\n message(FATAL_ERROR \"CUDA version must be at least 12.9!\")\n endif()\n if(DEFINED GPU_COMPUTE_VER)\n compute_cmake_cuda_archs(\"${GPU_COMPUTE_VER}\")\n endif()\n\n find_package(CUDAToolkit 12.8 REQUIRED)\nendif()\n\nif(USE_NVCOMP)\n find_package(nvcomp REQUIRED)\nendif()\n\n\nif(FORCE_COLORED_OUTPUT AND (CMAKE_GENERATOR STREQUAL \"Ninja\") AND\n ((CMAKE_CXX_COMPILER_ID STREQUAL \"GNU\") OR\n (CMAKE_CXX_COMPILER_ID STREQUAL \"Clang\")))\n set(CMAKE_CXX_FLAGS \"${CMAKE_CXX_FLAGS} -fdiagnostics-color=always\")\nendif()\n\nif(NOT (CMAKE_SYSTEM_NAME STREQUAL \"Emscripten\"))\n find_package(Threads REQUIRED)\nendif()\n\n# -- OpenMP\ninclude(cmake/FindOpenMPMacOS.cmake)\nif(USE_OPENMP)\n if(APPLE)\n find_openmp_macos()\n else()\n find_package(OpenMP REQUIRED C CXX)\n endif()\nendif()\n\n# Add for IBM i\nif(${CMAKE_SYSTEM_NAME} MATCHES \"OS400\")\n set(CMAKE_CXX_FLAGS \"${CMAKE_CXX_FLAGS} -pthread\")\n set(CMAKE_CXX_ARCHIVE_CREATE \" -X64 qc \")\nendif()\n\nif(USE_NCCL)\n find_package(Nccl REQUIRED)\nendif()\n\nif(MSVC)\n if(FORCE_SHARED_CRT)\n message(STATUS \"XGBoost: Using dynamically linked MSVC runtime...\")\n set(CMAKE_MSVC_RUNTIME_LIBRARY \"MultiThreaded$<$:Debug>DLL\")\n else()\n message(STATUS \"XGBoost: Using statically linked MSVC runtime...\")\n set(CMAKE_MSVC_RUNTIME_LIBRARY \"MultiThreaded$<$:Debug>\")\n endif()\nendif()\n\n# dmlc-core\nset(DMLC_FORCE_SHARED_CRT ${FORCE_SHARED_CRT})\nif(BUILD_WITH_SYSTEM_DMLC)\n find_package(dmlc REQUIRED)\nelse()\n set(DMLC_FORCE_SHARED_CRT ${FORCE_SHARED_CRT})\n add_subdirectory(${xgboost_SOURCE_DIR}/dmlc-core)\nendif()\n\nif(MSVC)\n if(TARGET dmlc_unit_tests)\n target_compile_options(\n dmlc_unit_tests PRIVATE\n -D_CRT_SECURE_NO_WARNINGS -D_CRT_SECURE_NO_DEPRECATE\n )\n endif()\nendif()\n\n# core xgboost\nadd_subdirectory(${xgboost_SOURCE_DIR}/src)\n# dmlc-core\nif(BUIILD_WTIH_SYSTEM_DMLC)\n target_link_libraries(objxgboost PUBLIC ${dmlc-LIBRARIES})\nelse()\n target_link_libraries(objxgboost PUBLIC dmlc)\nendif()\n\n# Link -lstdc++fs for GCC 8.x\nif(CMAKE_CXX_COMPILER_ID STREQUAL \"GNU\" AND CMAKE_CXX_COMPILER_VERSION VERSION_LESS \"9.0\")\n target_link_libraries(objxgboost PUBLIC stdc++fs)\nendif()\n\n# Exports some R specific definitions and objects\nif(R_LIB)\n add_subdirectory(${xgboost_SOURCE_DIR}/R-package)\nendif()\n\n# This creates its own shared library `xgboost4j'.\nif(JVM_BINDINGS)\n add_subdirectory(${xgboost_SOURCE_DIR}/jvm-packages)\nendif()\n\n# Plugin\nadd_subdirectory(${xgboost_SOURCE_DIR}/plugin)\n\nif(PLUGIN_RMM)\n find_package(rmm REQUIRED)\n\n # Patch the rmm targets so they reference the static cudart\n # Remove this patch once RMM stops specifying cudart requirement\n # (since RMM is a header-only library, it should not specify cudart in its CMake config)\n get_target_property(rmm_link_libs rmm::rmm INTERFACE_LINK_LIBRARIES)\n list(REMOVE_ITEM rmm_link_libs CUDA::cudart)\n list(APPEND rmm_link_libs CUDA::cudart_static)\n set_target_properties(rmm::rmm PROPERTIES INTERFACE_LINK_LIBRARIES \"${rmm_link_libs}\")\n\n # Pick up patched CCCL from RMM\nelseif(USE_CUDA)\n # If using CUDA and not RMM, search for CCCL.\n # Use HINTS to prioritize CCCL from the same CUDA toolkit as nvcc (e.g., in conda environments).\n # CUDAToolkit_TARGET_DIR points to the target-specific directory (e.g., targets/x86_64-linux).\n find_package(CCCL CONFIG\n HINTS ${CUDAToolkit_TARGET_DIR}/lib/cmake ${CUDAToolkit_LIBRARY_DIR}/cmake)\n if(CCCL_FOUND)\n message(STATUS \"Standalone CCCL found.\")\n else()\n message(STATUS \"Standalone CCCL not found. Attempting to use CCCL from CUDA Toolkit...\")\n find_package(CCCL CONFIG\n HINTS ${CUDAToolkit_TARGET_DIR}/lib/cmake ${CUDAToolkit_LIBRARY_DIR}/cmake)\n if(NOT CCCL_FOUND)\n message(STATUS \"Could not locate CCCL from CUDA Toolkit. Using Thrust and CUB from CUDA Toolkit...\")\n find_package(libcudacxx CONFIG REQUIRED\n HINTS ${CUDAToolkit_TARGET_DIR}/lib/cmake ${CUDAToolkit_LIBRARY_DIR}/cmake)\n find_package(CUB CONFIG REQUIRED\n HINTS ${CUDAToolkit_TARGET_DIR}/lib/cmake ${CUDAToolkit_LIBRARY_DIR}/cmake)\n find_package(Thrust CONFIG REQUIRED\n HINTS ${CUDAToolkit_TARGET_DIR}/lib/cmake ${CUDAToolkit_LIBRARY_DIR}/cmake)\n thrust_create_target(Thrust HOST CPP DEVICE CUDA)\n add_library(CCCL::CCCL INTERFACE IMPORTED GLOBAL)\n target_link_libraries(CCCL::CCCL INTERFACE libcudacxx::libcudacxx CUB::CUB Thrust)\n endif()\n endif()\n # Define guard macros to prevent windows.h from conflicting with winsock2.h\n if(WIN32)\n target_compile_definitions(CCCL::CCCL INTERFACE NOMINMAX WIN32_LEAN_AND_MEAN _WINSOCKAPI_)\n endif()\nendif()\n\nif(PLUGIN_SYCL)\n set(CMAKE_CXX_LINK_EXECUTABLE\n \"icpx -qopenmp -o \")\n set(CMAKE_CXX_CREATE_SHARED_LIBRARY\n \"icpx -shared -qopenmp \\\n , \\\n -o \")\nendif()\n\n#-- library\nif(BUILD_STATIC_LIB)\n add_library(xgboost STATIC)\nelse()\n # Provide versioned shared object\n add_library(xgboost SHARED)\n set_target_properties(xgboost PROPERTIES\n VERSION ${PROJECT_VERSION}\n SOVERSION ${PROJECT_VERSION_MAJOR})\nendif()\ntarget_link_libraries(xgboost PRIVATE objxgboost)\ntarget_include_directories(xgboost\n INTERFACE\n $/include>\n $)\n#-- End shared library\n\n# Common setup for all targets\nforeach(target xgboost objxgboost)\n xgboost_target_properties(${target})\n xgboost_target_link_libraries(${target})\n xgboost_target_defs(${target})\nendforeach()\nif(NOT BUILD_WITH_SYSTEM_DMLC)\n xgboost_target_properties(dmlc)\n xgboost_target_link_libraries(dmlc)\n xgboost_target_defs(dmlc)\nendif()\n\nif(JVM_BINDINGS)\n xgboost_target_properties(xgboost4j)\n xgboost_target_link_libraries(xgboost4j)\n xgboost_target_defs(xgboost4j)\nendif()\n\nif(USE_OPENMP AND APPLE AND NOT BUILD_STATIC_LIB)\n patch_openmp_path_macos(xgboost libxgboost)\nendif()\n\nif(KEEP_BUILD_ARTIFACTS_IN_BINARY_DIR)\n set_output_directory(xgboost ${xgboost_BINARY_DIR}/lib)\nelse()\n set_output_directory(xgboost ${xgboost_SOURCE_DIR}/lib)\nendif()\n\n#-- Installing XGBoost\nif(R_LIB)\n include(cmake/RPackageInstallTargetSetup.cmake)\n set_target_properties(xgboost PROPERTIES PREFIX \"\")\n if(APPLE)\n set_target_properties(xgboost PROPERTIES SUFFIX \".so\")\n endif()\n setup_rpackage_install_target(xgboost \"${CMAKE_CURRENT_BINARY_DIR}/R-package-install\")\n set(CMAKE_INSTALL_PREFIX \"${CMAKE_CURRENT_BINARY_DIR}/dummy_inst\")\nendif()\nif(MINGW)\n set_target_properties(xgboost PROPERTIES PREFIX \"\")\nendif()\n\nif(BUILD_C_DOC)\n include(cmake/Doc.cmake)\n run_doxygen()\nendif()\n\ninclude(CPack)\n\ninclude(GNUInstallDirs)\n# Install all headers. Please note that currently the C++ headers does not form an \"API\".\ninstall(DIRECTORY ${xgboost_SOURCE_DIR}/include/xgboost\n DESTINATION ${CMAKE_INSTALL_INCLUDEDIR})\n\n# Install libraries. If `xgboost` is a static lib, specify `objxgboost` also, to avoid the\n# following error:\n#\n# > install(EXPORT ...) includes target \"xgboost\" which requires target \"objxgboost\" that is not\n# > in any export set.\n#\n# https://github.com/dmlc/xgboost/issues/6085\nif(BUILD_STATIC_LIB)\n set(INSTALL_TARGETS xgboost objxgboost dmlc)\nelse()\n set(INSTALL_TARGETS xgboost)\nendif()\n\ninstall(TARGETS ${INSTALL_TARGETS}\n EXPORT XGBoostTargets\n ARCHIVE DESTINATION ${CMAKE_INSTALL_LIBDIR}\n LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR}\n RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR}\n INCLUDES DESTINATION ${LIBLEGACY_INCLUDE_DIRS})\ninstall(EXPORT XGBoostTargets\n FILE XGBoostTargets.cmake\n NAMESPACE xgboost::\n DESTINATION ${CMAKE_INSTALL_LIBDIR}/cmake/xgboost)\n\ninclude(CMakePackageConfigHelpers)\nconfigure_package_config_file(\n ${CMAKE_CURRENT_LIST_DIR}/cmake/xgboost-config.cmake.in\n ${CMAKE_CURRENT_BINARY_DIR}/cmake/xgboost-config.cmake\n INSTALL_DESTINATION ${CMAKE_INSTALL_LIBDIR}/cmake/xgboost)\nwrite_basic_package_version_file(\n ${CMAKE_BINARY_DIR}/cmake/xgboost-config-version.cmake\n VERSION ${XGBOOST_VERSION}\n COMPATIBILITY AnyNewerVersion)\ninstall(\n FILES\n ${CMAKE_CURRENT_BINARY_DIR}/cmake/xgboost-config.cmake\n ${CMAKE_BINARY_DIR}/cmake/xgboost-config-version.cmake\n DESTINATION ${CMAKE_INSTALL_LIBDIR}/cmake/xgboost)\n\n#-- Test\nif(GOOGLE_TEST)\n enable_testing()\n # Unittests.\n add_executable(testxgboost)\n target_link_libraries(testxgboost PRIVATE objxgboost)\n xgboost_target_properties(testxgboost)\n xgboost_target_link_libraries(testxgboost)\n xgboost_target_defs(testxgboost)\n\n add_subdirectory(${xgboost_SOURCE_DIR}/tests/cpp)\n\n add_test(\n NAME TestXGBoostLib\n COMMAND testxgboost\n WORKING_DIRECTORY ${xgboost_BINARY_DIR})\nendif()\n\n# Add xgboost.pc\nif(ADD_PKGCONFIG)\n configure_file(${xgboost_SOURCE_DIR}/cmake/xgboost.pc.in ${xgboost_BINARY_DIR}/xgboost.pc @ONLY)\n\n install(\n FILES ${xgboost_BINARY_DIR}/xgboost.pc\n DESTINATION ${CMAKE_INSTALL_LIBDIR}/pkgconfig)\nendif()\n" }, { "path": "CONTRIBUTORS.md", "content": "Contributors of DMLC/XGBoost\n============================\nXGBoost has been developed and used by a group of active community. Everyone is more than welcomed to is a great way to make the project better and more accessible to more users.\n\nProject Management Committee(PMC) \n----------\nThe Project Management Committee(PMC) consists group of active committers that moderate the discussion, manage the project release, and proposes new committer/PMC members. \n\n* [Tianqi Chen](https://github.com/tqchen), University of Washington\n - Tianqi is a Ph.D. student working on large-scale machine learning. He is the creator of the project.\n* [Michael Benesty](https://github.com/pommedeterresautee)\n - Michael is a lawyer and data scientist in France. He is the creator of XGBoost interactive analysis module in R.\n* [Yuan Tang](https://github.com/terrytangyuan), Red Hat\n - Yuan is a Senior Principal Software Engineer at Red Hat AI. He contributed mostly in R and Python packages.\n* [Nan Zhu](https://github.com/CodingCat), Uber\n - Nan is a software engineer in Uber. He contributed mostly in JVM packages.\n* [Jiaming Yuan](https://github.com/trivialfis)\n - Jiaming contributed to the GPU algorithms. He has also introduced new abstractions to improve the quality of the C++ codebase.\n* [Hyunsu Cho](http://hyunsu-cho.io/), NVIDIA\n - Hyunsu is the maintainer of the XGBoost Python package. He also manages the Jenkins continuous integration system (https://xgboost-ci.net/). He is the initial author of the CPU 'hist' updater.\n* [Rory Mitchell](https://github.com/RAMitchell), University of Waikato\n - Rory is a Ph.D. student at University of Waikato. He is the original creator of the GPU training algorithms. He improved the CMake build system and continuous integration. \n* [Hongliang Liu](https://github.com/phunterlau)\n\n\nCommitters\n----------\nCommitters are people who have made substantial contribution to the project and granted write access to the project.\n\n* [Tong He](https://github.com/hetong007), Amazon AI\n - Tong is an applied scientist in Amazon AI. He is the maintainer of XGBoost R package.\n* [Vadim Khotilovich](https://github.com/khotilov)\n - Vadim contributes many improvements in R and core packages.\n* [Bing Xu](https://github.com/antinucleon)\n - Bing is the original creator of XGBoost Python package and currently the maintainer of [XGBoost.jl](https://github.com/antinucleon/XGBoost.jl).\n* [Sergei Lebedev](https://github.com/superbobry), Criteo\n - Sergei is a software engineer in Criteo. He contributed mostly in JVM packages.\n* [Scott Lundberg](http://scottlundberg.com/), University of Washington\n - Scott is a Ph.D. student at University of Washington. He is the creator of SHAP, a unified approach to explain the output of machine learning models such as decision tree ensembles. He also helps maintain the XGBoost Julia package.\n* [Egor Smirnov](https://github.com/SmirnovEgorRu), Intel\n - Egor has led a major effort to improve the performance of XGBoost on multi-core CPUs.\n\n\nBecome a Committer\n------------------\nXGBoost is an open source project and we are actively looking for new committers who are willing to help maintaining and lead the project.\nCommitters come from contributors who:\n* Made substantial contribution to the project.\n* Willing to spent time on maintaining and lead the project.\n\nNew committers will be proposed by current committer members, with support from more than two of current committers.\n\nList of Contributors\n--------------------\n* [Full List of Contributors](https://github.com/dmlc/xgboost/graphs/contributors)\n - To contributors: please add your name to the list when you submit a patch to the project:)\n* [Kailong Chen](https://github.com/kalenhaha)\n - Kailong is an early contributor of XGBoost, he is creator of ranking objectives in XGBoost.\n* [Skipper Seabold](https://github.com/jseabold)\n - Skipper is the major contributor to the scikit-learn module of XGBoost.\n* [Zygmunt Zając](https://github.com/zygmuntz)\n - Zygmunt is the master behind the early stopping feature frequently used by Kagglers.\n* [Ajinkya Kale](https://github.com/ajkl)\n* [Boliang Chen](https://github.com/cblsjtu)\n* [Yangqing Men](https://github.com/yanqingmen)\n - Yangqing is the creator of XGBoost java package.\n* [Engpeng Yao](https://github.com/yepyao)\n* [Giulio](https://github.com/giuliohome)\n - Giulio is the creator of Windows project of XGBoost\n* [Jamie Hall](https://github.com/nerdcha)\n - Jamie is the initial creator of XGBoost scikit-learn module.\n* [Yen-Ying Lee](https://github.com/white1033)\n* [Masaaki Horikoshi](https://github.com/sinhrks)\n - Masaaki is the initial creator of XGBoost Python plotting module.\n* [daiyl0320](https://github.com/daiyl0320)\n - daiyl0320 contributed patch to XGBoost distributed version more robust, and scales stably on TB scale datasets.\n* [Huayi Zhang](https://github.com/irachex)\n* [Johan Manders](https://github.com/johanmanders)\n* [yoori](https://github.com/yoori)\n* [Mathias Müller](https://github.com/far0n)\n* [Sam Thomson](https://github.com/sammthomson)\n* [ganesh-krishnan](https://github.com/ganesh-krishnan)\n* [Damien Carol](https://github.com/damiencarol)\n* [Alex Bain](https://github.com/convexquad)\n* [Baltazar Bieniek](https://github.com/bbieniek)\n* [Adam Pocock](https://github.com/Craigacp)\n* [Gideon Whitehead](https://github.com/gaw89)\n* [Yi-Lin Juang](https://github.com/frankyjuang)\n* [Andrew Hannigan](https://github.com/andrewhannigan)\n* [Andy Adinets](https://github.com/canonizer)\n* [Henry Gouk](https://github.com/henrygouk)\n* [Pierre de Sahb](https://github.com/pdesahb)\n* [liuliang01](https://github.com/liuliang01)\n - liuliang01 added support for the qid column for LIBSVM input format. This makes ranking task easier in distributed setting.\n* [Andrew Thia](https://github.com/BlueTea88)\n - Andrew Thia implemented feature interaction constraints\n* [Wei Tian](https://github.com/weitian)\n* [Chen Qin](https://github.com/chenqin)\n* [Sam Wilkinson](https://samwilkinson.io)\n* [Matthew Jones](https://github.com/mt-jones)\n* [Jiaxiang Li](https://github.com/JiaxiangBU)\n* [Bryan Woods](https://github.com/bryan-woods)\n - Bryan added support for cross-validation for the ranking objective\n* [Haoda Fu](https://github.com/fuhaoda)\n* [Evan Kepner](https://github.com/EvanKepner)\n - Evan Kepner added support for os.PathLike file paths in Python\n" }, { "path": "LICENSE", "content": " Apache License\n Version 2.0, January 2004\n http://www.apache.org/licenses/\n\n TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION\n\n 1. Definitions.\n\n \"License\" shall mean the terms and conditions for use, reproduction,\n and distribution as defined by Sections 1 through 9 of this document.\n\n \"Licensor\" shall mean the copyright owner or entity authorized by\n the copyright owner that is granting the License.\n\n \"Legal Entity\" shall mean the union of the acting entity and all\n other entities that control, are controlled by, or are under common\n control with that entity. For the purposes of this definition,\n \"control\" means (i) the power, direct or indirect, to cause the\n direction or management of such entity, whether by contract or\n otherwise, or (ii) ownership of fifty percent (50%) or more of the\n outstanding shares, or (iii) beneficial ownership of such entity.\n\n \"You\" (or \"Your\") shall mean an individual or Legal Entity\n exercising permissions granted by this License.\n\n \"Source\" form shall mean the preferred form for making modifications,\n including but not limited to software source code, documentation\n source, and configuration files.\n\n \"Object\" form shall mean any form resulting from mechanical\n transformation or translation of a Source form, including but\n not limited to compiled object code, generated documentation,\n and conversions to other media types.\n\n \"Work\" shall mean the work of authorship, whether in Source or\n Object form, made available under the License, as indicated by a\n copyright notice that is included in or attached to the work\n (an example is provided in the Appendix below).\n\n \"Derivative Works\" shall mean any work, whether in Source or Object\n form, that is based on (or derived from) the Work and for which the\n editorial revisions, annotations, elaborations, or other modifications\n represent, as a whole, an original work of authorship. For the purposes\n of this License, Derivative Works shall not include works that remain\n separable from, or merely link (or bind by name) to the interfaces of,\n the Work and Derivative Works thereof.\n\n \"Contribution\" shall mean any work of authorship, including\n the original version of the Work and any modifications or additions\n to that Work or Derivative Works thereof, that is intentionally\n submitted to Licensor for inclusion in the Work by the copyright owner\n or by an individual or Legal Entity authorized to submit on behalf of\n the copyright owner. For the purposes of this definition, \"submitted\"\n means any form of electronic, verbal, or written communication sent\n to the Licensor or its representatives, including but not limited to\n communication on electronic mailing lists, source code control systems,\n and issue tracking systems that are managed by, or on behalf of, the\n Licensor for the purpose of discussing and improving the Work, but\n excluding communication that is conspicuously marked or otherwise\n designated in writing by the copyright owner as \"Not a Contribution.\"\n\n \"Contributor\" shall mean Licensor and any individual or Legal Entity\n on behalf of whom a Contribution has been received by Licensor and\n subsequently incorporated within the Work.\n\n 2. Grant of Copyright License. Subject to the terms and conditions of\n this License, each Contributor hereby grants to You a perpetual,\n worldwide, non-exclusive, no-charge, royalty-free, irrevocable\n copyright license to reproduce, prepare Derivative Works of,\n publicly display, publicly perform, sublicense, and distribute the\n Work and such Derivative Works in Source or Object form.\n\n 3. Grant of Patent License. Subject to the terms and conditions of\n this License, each Contributor hereby grants to You a perpetual,\n worldwide, non-exclusive, no-charge, royalty-free, irrevocable\n (except as stated in this section) patent license to make, have made,\n use, offer to sell, sell, import, and otherwise transfer the Work,\n where such license applies only to those patent claims licensable\n by such Contributor that are necessarily infringed by their\n Contribution(s) alone or by combination of their Contribution(s)\n with the Work to which such Contribution(s) was submitted. If You\n institute patent litigation against any entity (including a\n cross-claim or counterclaim in a lawsuit) alleging that the Work\n or a Contribution incorporated within the Work constitutes direct\n or contributory patent infringement, then any patent licenses\n granted to You under this License for that Work shall terminate\n as of the date such litigation is filed.\n\n 4. Redistribution. You may reproduce and distribute copies of the\n Work or Derivative Works thereof in any medium, with or without\n modifications, and in Source or Object form, provided that You\n meet the following conditions:\n\n (a) You must give any other recipients of the Work or\n Derivative Works a copy of this License; and\n\n (b) You must cause any modified files to carry prominent notices\n stating that You changed the files; and\n\n (c) You must retain, in the Source form of any Derivative Works\n that You distribute, all copyright, patent, trademark, and\n attribution notices from the Source form of the Work,\n excluding those notices that do not pertain to any part of\n the Derivative Works; and\n\n (d) If the Work includes a \"NOTICE\" text file as part of its\n distribution, then any Derivative Works that You distribute must\n include a readable copy of the attribution notices contained\n within such NOTICE file, excluding those notices that do not\n pertain to any part of the Derivative Works, in at least one\n of the following places: within a NOTICE text file distributed\n as part of the Derivative Works; within the Source form or\n documentation, if provided along with the Derivative Works; or,\n within a display generated by the Derivative Works, if and\n wherever such third-party notices normally appear. The contents\n of the NOTICE file are for informational purposes only and\n do not modify the License. You may add Your own attribution\n notices within Derivative Works that You distribute, alongside\n or as an addendum to the NOTICE text from the Work, provided\n that such additional attribution notices cannot be construed\n as modifying the License.\n\n You may add Your own copyright statement to Your modifications and\n may provide additional or different license terms and conditions\n for use, reproduction, or distribution of Your modifications, or\n for any such Derivative Works as a whole, provided Your use,\n reproduction, and distribution of the Work otherwise complies with\n the conditions stated in this License.\n\n 5. Submission of Contributions. Unless You explicitly state otherwise,\n any Contribution intentionally submitted for inclusion in the Work\n by You to the Licensor shall be under the terms and conditions of\n this License, without any additional terms or conditions.\n Notwithstanding the above, nothing herein shall supersede or modify\n the terms of any separate license agreement you may have executed\n with Licensor regarding such Contributions.\n\n 6. Trademarks. This License does not grant permission to use the trade\n names, trademarks, service marks, or product names of the Licensor,\n except as required for reasonable and customary use in describing the\n origin of the Work and reproducing the content of the NOTICE file.\n\n 7. Disclaimer of Warranty. Unless required by applicable law or\n agreed to in writing, Licensor provides the Work (and each\n Contributor provides its Contributions) on an \"AS IS\" BASIS,\n WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or\n implied, including, without limitation, any warranties or conditions\n of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A\n PARTICULAR PURPOSE. You are solely responsible for determining the\n appropriateness of using or redistributing the Work and assume any\n risks associated with Your exercise of permissions under this License.\n\n 8. Limitation of Liability. In no event and under no legal theory,\n whether in tort (including negligence), contract, or otherwise,\n unless required by applicable law (such as deliberate and grossly\n negligent acts) or agreed to in writing, shall any Contributor be\n liable to You for damages, including any direct, indirect, special,\n incidental, or consequential damages of any character arising as a\n result of this License or out of the use or inability to use the\n Work (including but not limited to damages for loss of goodwill,\n work stoppage, computer failure or malfunction, or any and all\n other commercial damages or losses), even if such Contributor\n has been advised of the possibility of such damages.\n\n 9. Accepting Warranty or Additional Liability. While redistributing\n the Work or Derivative Works thereof, You may choose to offer,\n and charge a fee for, acceptance of support, warranty, indemnity,\n or other liability obligations and/or rights consistent with this\n License. However, in accepting such obligations, You may act only\n on Your own behalf and on Your sole responsibility, not on behalf\n of any other Contributor, and only if You agree to indemnify,\n defend, and hold each Contributor harmless for any liability\n incurred by, or claims asserted against, such Contributor by reason\n of your accepting any such warranty or additional liability.\n\n END OF TERMS AND CONDITIONS\n\n APPENDIX: How to apply the Apache License to your work.\n\n To apply the Apache License to your work, attach the following\n boilerplate notice, with the fields enclosed by brackets \"{}\"\n replaced with your own identifying information. (Don't include\n the brackets!) The text should be enclosed in the appropriate\n comment syntax for the file format. We also recommend that a\n file or class name and description of purpose be included on the\n same \"printed page\" as the copyright notice for easier\n identification within third-party archives.\n\n Copyright (c) 2019 by Contributors\n\n Licensed under the Apache License, Version 2.0 (the \"License\");\n you may not use this file except in compliance with the License.\n You may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\n Unless required by applicable law or agreed to in writing, software\n distributed under the License is distributed on an \"AS IS\" BASIS,\n WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n See the License for the specific language governing permissions and\n limitations under the License.\n" }, { "path": "NEWS.md", "content": "XGBoost Change Log\n==================\n\n**Starting from 2.1.0, release note is recorded in the documentation.**\n\nThis file records the changes in xgboost library in reverse chronological order.\n\n## 2.0.0 (2023 Aug 16)\n\nWe are excited to announce the release of XGBoost 2.0. This note will begin by covering some overall changes and then highlight specific updates to the package.\n\n### Initial work on multi-target trees with vector-leaf outputs\nWe have been working on vector-leaf tree models for multi-target regression, multi-label classification, and multi-class classification in version 2.0. Previously, XGBoost would build a separate model for each target. However, with this new feature that's still being developed, XGBoost can build one tree for all targets. The feature has multiple benefits and trade-offs compared to the existing approach. It can help prevent overfitting, produce smaller models, and build trees that consider the correlation between targets. In addition, users can combine vector leaf and scalar leaf trees during a training session using a callback. Please note that the feature is still a working in progress, and many parts are not yet available. See #9043 for the current status. Related PRs: (#8538, #8697, #8902, #8884, #8895, #8898, #8612, #8652, #8698, #8908, #8928, #8968, #8616, #8922, #8890, #8872, #8889, #9509) Please note that, only the `hist` (default) tree method on CPU can be used for building vector leaf trees at the moment.\n\n### New `device` parameter.\n\nA new `device` parameter is set to replace the existing `gpu_id`, `gpu_hist`, `gpu_predictor`, `cpu_predictor`, `gpu_coord_descent`, and the PySpark specific parameter `use_gpu`. Onward, users need only the `device` parameter to select which device to run along with the ordinal of the device. For more information, please see our document page (https://xgboost.readthedocs.io/en/stable/parameter.html#general-parameters) . For example, with `device=\"cuda\", tree_method=\"hist\"`, XGBoost will run the `hist` tree method on GPU. (#9363, #8528, #8604, #9354, #9274, #9243, #8896, #9129, #9362, #9402, #9385, #9398, #9390, #9386, #9412, #9507, #9536). The old behavior of ``gpu_hist`` is preserved but deprecated. In addition, the `predictor` parameter is removed.\n\n\n### `hist` is now the default tree method\nStarting from 2.0, the `hist` tree method will be the default. In previous versions, XGBoost chooses `approx` or `exact` depending on the input data and training environment. The new default can help XGBoost train models more efficiently and consistently. (#9320, #9353)\n\n### GPU-based approx tree method\nThere's initial support for using the `approx` tree method on GPU. The performance of the `approx` is not yet well optimized but is feature complete except for the JVM packages. It can be accessed through the use of the parameter combination `device=\"cuda\", tree_method=\"approx\"`. (#9414, #9399, #9478). Please note that the Scala-based Spark interface is not yet supported.\n\n### Optimize and bound the size of the histogram on CPU, to control memory footprint\n\nXGBoost has a new parameter `max_cached_hist_node` for users to limit the CPU cache size for histograms. It can help prevent XGBoost from caching histograms too aggressively. Without the cache, performance is likely to decrease. However, the size of the cache grows exponentially with the depth of the tree. The limit can be crucial when growing deep trees. In most cases, users need not configure this parameter as it does not affect the model's accuracy. (#9455, #9441, #9440, #9427, #9400).\n\nAlong with the cache limit, XGBoost also reduces the memory usage of the `hist` and `approx` tree method on distributed systems by cutting the size of the cache by half. (#9433)\n\n### Improved external memory support\nThere is some exciting development around external memory support in XGBoost. It's still an experimental feature, but the performance has been significantly improved with the default `hist` tree method. We replaced the old file IO logic with memory map. In addition to performance, we have reduced CPU memory usage and added extensive documentation. Beginning from 2.0.0, we encourage users to try it with the `hist` tree method when the memory saving by `QuantileDMatrix` is not sufficient. (#9361, #9317, #9282, #9315, #8457)\n\n### Learning to rank\nWe created a brand-new implementation for the learning-to-rank task. With the latest version, XGBoost gained a set of new features for ranking task including:\n\n- A new parameter `lambdarank_pair_method` for choosing the pair construction strategy.\n- A new parameter `lambdarank_num_pair_per_sample` for controlling the number of samples for each group.\n- An experimental implementation of unbiased learning-to-rank, which can be accessed using the `lambdarank_unbiased` parameter.\n- Support for custom gain function with `NDCG` using the `ndcg_exp_gain` parameter.\n- Deterministic GPU computation for all objectives and metrics.\n- `NDCG` is now the default objective function.\n- Improved performance of metrics using caches.\n- Support scikit-learn utilities for `XGBRanker`.\n- Extensive documentation on how learning-to-rank works with XGBoost.\n\nFor more information, please see the [tutorial](https://xgboost.readthedocs.io/en/latest/tutorials/learning_to_rank.html). Related PRs: (#8771, #8692, #8783, #8789, #8790, #8859, #8887, #8893, #8906, #8931, #9075, #9015, #9381, #9336, #8822, #9222, #8984, #8785, #8786, #8768)\n\n### Automatically estimated intercept\n\nIn the previous version, `base_score` was a constant that could be set as a training parameter. In the new version, XGBoost can automatically estimate this parameter based on input labels for optimal accuracy. (#8539, #8498, #8272, #8793, #8607)\n\n### Quantile regression\nThe XGBoost algorithm now supports quantile regression, which involves minimizing the quantile loss (also called \"pinball loss\"). Furthermore, XGBoost allows for training with multiple target quantiles simultaneously with one tree per quantile. (#8775, #8761, #8760, #8758, #8750)\n\n### L1 and Quantile regression now supports learning rate\nBoth objectives use adaptive trees due to the lack of proper Hessian values. In the new version, XGBoost can scale the leaf value with the learning rate accordingly. (#8866)\n\n### Export cut value\n\nUsing the Python or the C package, users can export the quantile values (not to be confused with quantile regression) used for the `hist` tree method. (#9356)\n\n### column-based split and federated learning\nWe made progress on column-based split for federated learning. In 2.0, both `approx`, `hist`, and `hist` with vector leaf can work with column-based data split, along with support for vertical federated learning. Work on GPU support is still on-going, stay tuned. (#8576, #8468, #8442, #8847, #8811, #8985, #8623, #8568, #8828, #8932, #9081, #9102, #9103, #9124, #9120, #9367, #9370, #9343, #9171, #9346, #9270, #9244, #8494, #8434, #8742, #8804, #8710, #8676, #9020, #9002, #9058, #9037, #9018, #9295, #9006, #9300, #8765, #9365, #9060)\n\n### PySpark\nAfter the initial introduction of the PySpark interface, it has gained some new features and optimizations in 2.0.\n\n- GPU-based prediction. (#9292, #9542)\n- Optimization for data initialization by avoiding the stack operation. (#9088)\n- Support predict feature contribution. (#8633)\n- Python typing support. (#9156, #9172, #9079, #8375)\n- `use_gpu` is deprecated. The `device` parameter is preferred.\n- Update eval_metric validation to support list of strings (#8826)\n- Improved logs for training (#9449)\n- Maintenance, including refactoring and document updates (#8324, #8465, #8605, #9202, #9460, #9302, #8385, #8630, #8525, #8496)\n- Fix for GPU setup. (#9495)\n\n### Other General New Features\nHere's a list of new features that don't have their own section and yet are general to all language bindings.\n\n- Use array interface for CSC matrix. This helps XGBoost to use a consistent number of threads and align the interface of the CSC matrix with other interfaces. In addition, memory usage is likely to decrease with CSC input thanks to on-the-fly type conversion. (#8672)\n- CUDA compute 90 is now part of the default build.. (#9397)\n\n### Other General Optimization\nThese optimizations are general to all language bindings. For language-specific optimization, please visit the corresponding sections.\n\n- Performance for input with `array_interface` on CPU (like `numpy`) is significantly improved. (#9090)\n- Some optimization with CUDA for data initialization. (#9199, #9209, #9144)\n- Use the latest thrust policy to prevent synchronizing GPU devices. (#9212)\n- XGBoost now uses a per-thread CUDA stream, which prevents synchronization with other streams. (#9416, #9396, #9413)\n\n### Notable breaking change\n\nOther than the aforementioned change with the `device` parameter, here's a list of breaking changes affecting all packages.\n\n- Users must specify the format for text input (#9077). However, we suggest using third-party data structures such as `numpy.ndarray` instead of relying on text inputs. See https://github.com/dmlc/xgboost/issues/9472 for more info.\n\n### Notable bug fixes\n\nSome noteworthy bug fixes that are not related to specific language bindings are listed in this section.\n\n- Some language environments use a different thread to perform garbage collection, which breaks the thread-local cache used in XGBoost. XGBoost 2.0 implements a new thread-safe cache using a light weight lock to replace the thread-local cache. (#8851)\n- Fix model IO by clearing the prediction cache. (#8904)\n- `inf` is checked during data construction. (#8911)\n- Preserve order of saved updaters configuration. Usually, this is not an issue unless the `updater` parameter is used instead of the `tree_method` parameter (#9355)\n- Fix GPU memory allocation issue with categorical splits. (#9529)\n- Handle escape sequence like `\\t\\n` in feature names for JSON model dump. (#9474)\n- Normalize file path for model IO and text input. This handles short paths on Windows and paths that contain `~` on Unix (#9463). In addition, all path inputs are required to be encoded in UTF-8 (#9448, #9443)\n- Fix integer overflow on H100. (#9380)\n- Fix weighted sketching on GPU with categorical features. (#9341)\n- Fix metric serialization. The bug might cause some of the metrics to be dropped during evaluation. (#9405)\n- Fixes compilation errors on MSVC x86 targets (#8823)\n- Pick up the dmlc-core fix for the CSV parser. (#8897)\n\n\n### Documentation\nAside from documents for new features, we have many smaller updates to improve user experience, from troubleshooting guides to typo fixes.\n\n- Explain CPU/GPU interop. (#8450)\n- Guide to troubleshoot NCCL errors. (#8943, #9206)\n- Add a note for rabit port selection. (#8879)\n- How to build the docs using conda (#9276)\n- Explain how to obtain reproducible results on distributed systems. (#8903)\n\n* Fixes and small updates to document and demonstration scripts. (#8626, #8436, #8995, #8907, #8923, #8926, #9358, #9232, #9201, #9469, #9462, #9458, #8543, #8597, #8401, #8784, #9213, #9098, #9008, #9223, #9333, #9434, #9435, #9415, #8773, #8752, #9291, #9549)\n\n### Python package\n* New Features and Improvements\n- Support primitive types of pyarrow-backed pandas dataframe. (#8653)\n- Warning messages emitted by XGBoost are now emitted using Python warnings. (#9387)\n- User can now format the value printed near the bars on the `plot_importance` plot (#8540)\n- XGBoost has improved half-type support (float16) with pandas, cupy, and cuDF. With GPU input, the handling is through CUDA `__half` type, and no data copy is made. (#8487, #9207, #8481)\n- Support `Series` and Python primitive types in `inplace_predict` and `QuantileDMatrix` (#8547, #8542)\n- Support all pandas' nullable integer types. (#8480)\n- Custom metric with the scikit-learn interface now supports `sample_weight`. (#8706)\n- Enable Installation of Python Package with System lib in a Virtual Environment (#9349)\n- Raise if expected workers are not alive in `xgboost.dask.train` (#9421)\n\n* Optimization\n- Cache transformed data in `QuantileDMatrix` for efficiency. (#8666, #9445)\n- Take datatable as row-major input. (#8472)\n- Remove unnecessary conversions between data structures (#8546)\n\n* Adopt modern Python packaging conventions (PEP 517, PEP 518, PEP 621)\n- XGBoost adopted the modern Python packaging conventions. The old setup script `setup.py` is now replaced with the new configuration file `pyproject.toml`. Along with this, XGBoost now supports Python 3.11. (#9021, #9112, #9114, #9115) Consult the latest documentation for the updated instructions to build and install XGBoost.\n\n* Fixes\n- `DataIter` now accepts only keyword arguments. (#9431)\n- Fix empty DMatrix with categorical features. (#8739)\n- Convert ``DaskXGBClassifier.classes_`` to an array (#8452)\n- Define `best_iteration` only if early stopping is used to be consistent with documented behavior. (#9403)\n- Make feature validation immutable. (#9388)\n\n* Breaking changes\n- Discussed in the new `device` parameter section, the `predictor` parameter is now removed. (#9129)\n- Remove support for single-string feature info. Feature type and names should be a sequence of strings (#9401)\n- Remove parameters in the `save_model` call for the scikit-learn interface. (#8963)\n- Remove the `ntree_limit` in the python package. This has been deprecated in previous versions. (#8345)\n\n* Maintenance including formatting and refactoring along with type hints.\n- More consistent use of `black` and `isort` for code formatting (#8420, #8748, #8867)\n- Improved type support. Most of the type changes happen in the PySpark module; here, we list the remaining changes. (#8444, #8617, #9197, #9005)\n- Set `enable_categorical` to True in predict. (#8592)\n- Some refactoring and updates for tests (#8395, #8372, #8557, #8379, #8702, #9459, #9316, #8446, #8695, #8409, #8993, #9480)\n\n* Documentation\n- Add introduction and notes for the sklearn interface. (#8948)\n- Demo for using dask for hyper-parameter optimization. (#8891)\n- Document all supported Python input types. (#8643)\n- Other documentation updates (#8944, #9304)\n\n### R package\n- Use the new data consumption interface for CSR and CSC. This provides better control for the number of threads and improves performance. (#8455, #8673)\n- Accept multiple evaluation metrics during training. (#8657)\n- Fix integer inputs with `NA`. (#9522)\n- Some refactoring for the R package (#8545, #8430, #8614, #8624, #8613, #9457, #8689, #8563, #9461, #8647, #8564, #8565, #8736, #8610, #8609, #8599, #8704, #9456, #9450, #9476, #9477, #9481). Special thanks to @jameslamb.\n- Document updates (#8886, #9323, #9437, #8998)\n\n### JVM packages\nFollowing are changes specific to various JVM-based packages.\n\n- Stop using Rabit in prediction (#9054)\n- Set feature_names and feature_types in jvm-packages. This is to prepare support for categorical features (#9364)\n- Scala 2.13 support. (#9099)\n- Change training stage from `ResultStage` to `ShuffleMapStage` (#9423)\n- Automatically set the max/min direction for the best score during early stopping. (#9404)\n* Revised support for `flink` (#9046)\n\n* Breaking changes\n- Scala-based tracker is removed. (#9078, #9045)\n- Change `DeviceQuantileDmatrix` into `QuantileDMatrix` (#8461)\n\n* Maintenance (#9253, #9166, #9395, #9389, #9224, #9233, #9351, #9479)\n\n* CI bot PRs\nWe employed GitHub dependent bot to help us keep the dependencies up-to-date for JVM packages. With the help from the bot, we have cleared up all the dependencies that are lagging behind (#8501, #8507).\n\nHere's a list of dependency update PRs including those made by dependent bots (#8456, #8560, #8571, #8561, #8562, #8600, #8594, #8524, #8509, #8548, #8549, #8533, #8521, #8534, #8532, #8516, #8503, #8531, #8530, #8518, #8512, #8515, #8517, #8506, #8504, #8502, #8629, #8815, #8813, #8814, #8877, #8876, #8875, #8874, #8873, #9049, #9070, #9073, #9039, #9083, #8917, #8952, #8980, #8973, #8962, #9252, #9208, #9131, #9136, #9219, #9160, #9158, #9163, #9184, #9192, #9265, #9268, #8882, #8837, #8662, #8661, #8390, #9056, #8508, #8925, #8920, #9149, #9230, #9097, #8648, #9203, #8593).\n\n### Maintenance\nMaintenance work includes refactoring, fixing small issues that don't affect end users. (#9256, #8627, #8756, #8735, #8966, #8864, #8747, #8892, #9057, #8921, #8949, #8941, #8942, #9108, #9125, #9155, #9153, #9176, #9447, #9444, #9436, #9438, #9430, #9200, #9210, #9055, #9014, #9004, #8999, #9154, #9148, #9283, #9246, #8888, #8900, #8871, #8861, #8858, #8791, #8807, #8751, #8703, #8696, #8693, #8677, #8686, #8665, #8660, #8386, #8371, #8410, #8578, #8574, #8483, #8443, #8454, #8733)\n\n### CI\n- Build pip wheel with RMM support (#9383)\n- Other CI updates including updating dependencies and work on the CI infrastructure. (#9464, #9428, #8767, #9394, #9278, #9214, #9234, #9205, #9034, #9104, #8878, #9294, #8625, #8806, #8741, #8707, #8381, #8382, #8388, #8402, #8397, #8445, #8602, #8628, #8583, #8460, #9544)\n\n## 1.7.6 (2023 Jun 16)\n\nThis is a patch release for bug fixes. The CRAN package for the R binding is kept at 1.7.5.\n\n### Bug Fixes\n* Fix distributed training with mixed dense and sparse partitions. (#9272)\n* Fix monotone constraints on CPU with large trees. (#9122)\n* [spark] Make the spark model have the same UID as its estimator (#9022)\n* Optimize prediction with `QuantileDMatrix`. (#9096)\n\n### Document\n* Improve doxygen (#8959)\n* Update the cuDF pip index URL. (#9106)\n\n### Maintenance\n* Fix tests with pandas 2.0. (#9014)\n\n## 1.7.5 (2023 Mar 30)\nThis is a patch release for bug fixes.\n\n* C++ requirement is updated to C++-17, along with which, CUDA 11.8 is used as the default CTK. (#8860, #8855, #8853)\n* Fix import for pyspark ranker. (#8692)\n* Fix Windows binary wheel to be compatible with Poetry (#8991)\n* Fix GPU hist with column sampling. (#8850)\n* Make sure iterative DMatrix is properly initialized. (#8997)\n* [R] Update link in document. (#8998)\n\n## 1.7.4 (2023 Feb 16)\nThis is a patch release for bug fixes.\n\n* [R] Fix OpenMP detection on macOS. (#8684)\n* [Python] Make sure input numpy array is aligned. (#8690)\n* Fix feature interaction with column sampling in gpu_hist evaluator. (#8754)\n* Fix GPU L1 error. (#8749)\n* [PySpark] Fix feature types param (#8772)\n* Fix ranking with quantile dmatrix and group weight. (#8762)\n\n## 1.7.3 (2023 Jan 6)\nThis is a patch release for bug fixes.\n\n* [Breaking] XGBoost Sklearn estimator method `get_params` no longer returns internally configured values. (#8634)\n* Fix linalg iterator, which may crash the L1 error. (#8603)\n* Fix loading pickled GPU model with a CPU-only XGBoost build. (#8632)\n* Fix inference with unseen categories with categorical features. (#8591, #8602)\n* CI fixes. (#8620, #8631, #8579)\n\n## v1.7.2 (2022 Dec 8)\nThis is a patch release for bug fixes.\n\n* Work with newer thrust and libcudacxx (#8432)\n* Support null value in CUDA array interface namespace. (#8486)\n* Use `getsockname` instead of `SO_DOMAIN` on AIX. (#8437)\n* [pyspark] Make QDM optional based on a cuDF check (#8471)\n* [pyspark] sort qid for SparkRanker. (#8497)\n* [dask] Properly await async method client.wait_for_workers. (#8558)\n\n* [R] Fix CRAN test notes. (#8428)\n\n* [doc] Fix outdated document [skip ci]. (#8527)\n* [CI] Fix github action mismatched glibcxx. (#8551)\n\n## v1.7.1 (2022 Nov 3)\nThis is a patch release to incorporate the following hotfix:\n\n* Add back xgboost.rabit for backwards compatibility (#8411)\n\n\n## v1.7.0 (2022 Oct 20)\n\nWe are excited to announce the feature packed XGBoost 1.7 release. The release note will walk through some of the major new features first, then make a summary for other improvements and language-binding-specific changes.\n\n### PySpark\n\nXGBoost 1.7 features initial support for PySpark integration. The new interface is adapted from the existing PySpark XGBoost interface developed by databricks with additional features like `QuantileDMatrix` and the rapidsai plugin (GPU pipeline) support. The new Spark XGBoost Python estimators not only benefit from PySpark ml facilities for powerful distributed computing but also enjoy the rest of the Python ecosystem. Users can define a custom objective, callbacks, and metrics in Python and use them with this interface on distributed clusters. The support is labeled as experimental with more features to come in future releases. For a brief introduction please visit the tutorial on XGBoost's [document page](https://xgboost.readthedocs.io/en/latest/tutorials/spark_estimator.html). (#8355, #8344, #8335, #8284, #8271, #8283, #8250, #8231, #8219, #8245, #8217, #8200, #8173, #8172, #8145, #8117, #8131, #8088, #8082, #8085, #8066, #8068, #8067, #8020, #8385)\n\nDue to its initial support status, the new interface has some limitations; categorical features and multi-output models are not yet supported.\n\n### Development of categorical data support\nMore progress on the experimental support for categorical features. In 1.7, XGBoost can handle missing values in categorical features and features a new parameter `max_cat_threshold`, which limits the number of categories that can be used in the split evaluation. The parameter is enabled when the partitioning algorithm is used and helps prevent over-fitting. Also, the sklearn interface can now accept the `feature_types` parameter to use data types other than dataframe for categorical features. (#8280, #7821, #8285, #8080, #7948, #7858, #7853, #8212, #7957, #7937, #7934)\n\n\n### Experimental support for federated learning and new communication collective\n\nAn exciting addition to XGBoost is the experimental federated learning support. The federated learning is implemented with a gRPC federated server that aggregates allreduce calls, and federated clients that train on local data and use existing tree methods (approx, hist, gpu_hist). Currently, this only supports horizontal federated learning (samples are split across participants, and each participant has all the features and labels). Future plans include vertical federated learning (features split across participants), and stronger privacy guarantees with homomorphic encryption and differential privacy. See [Demo with NVFlare integration](demo/nvflare/README.md) for example usage with nvflare.\n\nAs part of the work, XGBoost 1.7 has replaced the old rabit module with the new collective module as the network communication interface with added support for runtime backend selection. In previous versions, the backend is defined at compile time and can not be changed once built. In this new release, users can choose between `rabit` and `federated.` (#8029, #8351, #8350, #8342, #8340, #8325, #8279, #8181, #8027, #7958, #7831, #7879, #8257, #8316, #8242, #8057, #8203, #8038, #7965, #7930, #7911)\n\nThe feature is available in the public PyPI binary package for testing.\n\n### Quantile DMatrix\nBefore 1.7, XGBoost has an internal data structure called `DeviceQuantileDMatrix` (and its distributed version). We now extend its support to CPU and renamed it to `QuantileDMatrix`. This data structure is used for optimizing memory usage for the `hist` and `gpu_hist` tree methods. The new feature helps reduce CPU memory usage significantly, especially for dense data. The new `QuantileDMatrix` can be initialized from both CPU and GPU data, and regardless of where the data comes from, the constructed instance can be used by both the CPU algorithm and GPU algorithm including training and prediction (with some overhead of conversion if the device of data and training algorithm doesn't match). Also, a new parameter `ref` is added to `QuantileDMatrix`, which can be used to construct validation/test datasets. Lastly, it's set as default in the scikit-learn interface when a supported tree method is specified by users. (#7889, #7923, #8136, #8215, #8284, #8268, #8220, #8346, #8327, #8130, #8116, #8103, #8094, #8086, #7898, #8060, #8019, #8045, #7901, #7912, #7922)\n\n### Mean absolute error\nThe mean absolute error is a new member of the collection of objectives in XGBoost. It's noteworthy since MAE has zero hessian value, which is unusual to XGBoost as XGBoost relies on Newton optimization. Without valid Hessian values, the convergence speed can be slow. As part of the support for MAE, we added line searches into the XGBoost training algorithm to overcome the difficulty of training without valid Hessian values. In the future, we will extend the line search to other objectives where it's appropriate for faster convergence speed. (#8343, #8107, #7812, #8380)\n\n### XGBoost on Browser\nWith the help of the [pyodide](https://github.com/pyodide/pyodide) project, you can now run XGBoost on browsers. (#7954, #8369)\n\n### Experimental IPv6 Support for Dask\n\nWith the growing adaption of the new internet protocol, XGBoost joined the club. In the latest release, the Dask interface can be used on IPv6 clusters, see XGBoost's Dask tutorial for details. (#8225, #8234)\n\n### Optimizations\nWe have new optimizations for both the `hist` and `gpu_hist` tree methods to make XGBoost's training even more efficient.\n\n* Hist\nHist now supports optional by-column histogram build, which is automatically configured based on various conditions of input data. This helps the XGBoost CPU hist algorithm to scale better with different shapes of training datasets. (#8233, #8259). Also, the build histogram kernel now can better utilize CPU registers (#8218)\n\n* GPU Hist\nGPU hist performance is significantly improved for wide datasets. GPU hist now supports batched node build, which reduces kernel latency and increases throughput. The improvement is particularly significant when growing deep trees with the default ``depthwise`` policy. (#7919, #8073, #8051, #8118, #7867, #7964, #8026)\n\n### Breaking Changes\nBreaking changes made in the 1.7 release are summarized below.\n- The `grow_local_histmaker` updater is removed. This updater is rarely used in practice and has no test. We decided to remove it and focus have XGBoot focus on other more efficient algorithms. (#7992, #8091)\n- Single precision histogram is removed due to its lack of accuracy caused by significant floating point error. In some cases the error can be difficult to detect due to log-scale operations, which makes the parameter dangerous to use. (#7892, #7828)\n- Deprecated CUDA architectures are no longer supported in the release binaries. (#7774)\n- As part of the federated learning development, the `rabit` module is replaced with the new `collective` module. It's a drop-in replacement with added runtime backend selection, see the federated learning section for more details (#8257)\n\n### General new features and improvements\nBefore diving into package-specific changes, some general new features other than those listed at the beginning are summarized here.\n* Users of `DMatrix` and `QuantileDMatrix` can get the data from XGBoost. In previous versions, only getters for meta info like labels are available. The new method is available in Python (`DMatrix::get_data`) and C. (#8269, #8323)\n* In previous versions, the GPU histogram tree method may generate phantom gradient for missing values due to floating point error. We fixed such an error in this release and XGBoost is much better equated to handle floating point errors when training on GPU. (#8274, #8246)\n* Parameter validation is no longer experimental. (#8206)\n* C pointer parameters and JSON parameters are vigorously checked. (#8254, #8254)\n* Improved handling of JSON model input. (#7953, #7918)\n* Support IBM i OS (#7920, #8178)\n\n### Fixes\nSome noteworthy bug fixes that are not related to specific language binding are listed in this section.\n* Rename misspelled config parameter for pseudo-Huber (#7904)\n* Fix feature weights with nested column sampling. (#8100)\n* Fix loading DMatrix binary in distributed env. (#8149)\n* Force auc.cc to be statically linked for unusual compiler platforms. (#8039)\n* New logic for detecting libomp on macos (#8384).\n\n### Python Package\n* Python 3.8 is now the minimum required Python version. (#8071)\n* More progress on type hint support. Except for the new PySpark interface, the XGBoost module is fully typed. (#7742, #7945, #8302, #7914, #8052)\n* XGBoost now validates the feature names in `inplace_predict`, which also affects the predict function in scikit-learn estimators as it uses `inplace_predict` internally. (#8359)\n* Users can now get the data from `DMatrix` using `DMatrix::get_data` or `QuantileDMatrix::get_data`.\n* Show `libxgboost.so` path in build info. (#7893)\n* Raise import error when using the sklearn module while scikit-learn is missing. (#8049)\n* Use `config_context` in the sklearn interface. (#8141)\n* Validate features for inplace prediction. (#8359)\n* Pandas dataframe handling is refactored to reduce data fragmentation. (#7843)\n* Support more pandas nullable types (#8262)\n* Remove pyarrow workaround. (#7884)\n\n* Binary wheel size\nWe aim to enable as many features as possible in XGBoost's default binary distribution on PyPI (package installed with pip), but there's a upper limit on the size of the binary wheel. In 1.7, XGBoost reduces the size of the wheel by pruning unused CUDA architectures. (#8179, #8152, #8150)\n\n* Fixes\n Some noteworthy fixes are listed here:\n - Fix the Dask interface with the latest cupy. (#8210)\n - Check cuDF lazily to avoid potential errors with cuda-python. (#8084)\n* Fix potential error in DMatrix constructor on 32-bit platform. (#8369)\n\n* Maintenance work\n - Linter script is moved from dmlc-core to XGBoost with added support for formatting, mypy, and parallel run, along with some fixes (#7967, #8101, #8216)\n - We now require the use of `isort` and `black` for selected files. (#8137, #8096)\n - Code cleanups. (#7827)\n - Deprecate `use_label_encoder` in XGBClassifier. The label encoder has already been deprecated and removed in the previous version. These changes only affect the indicator parameter (#7822)\n - Remove the use of distutils. (#7770)\n - Refactor and fixes for tests (#8077, #8064, #8078, #8076, #8013, #8010, #8244, #7833)\n\n* Documents\n - [dask] Fix potential error in demo. (#8079)\n - Improved documentation for the ranker. (#8356, #8347)\n - Indicate lack of py-xgboost-gpu on Windows (#8127)\n - Clarification for feature importance. (#8151)\n - Simplify Python getting started example (#8153)\n\n### R Package\nWe summarize improvements for the R package briefly here:\n* Feature info including names and types are now passed to DMatrix in preparation for categorical feature support. (#804)\n* XGBoost 1.7 can now gracefully load old R models from RDS for better compatibility with 3-party tuning libraries (#7864)\n* The R package now can be built with parallel compilation, along with fixes for warnings in CRAN tests. (#8330)\n* Emit error early if DiagrammeR is missing (#8037)\n* Fix R package Windows build. (#8065)\n\n### JVM Packages\nThe consistency between JVM packages and other language bindings is greatly improved in 1.7, improvements range from model serialization format to the default value of hyper-parameters.\n\n* Java package now supports feature names and feature types for DMatrix in preparation for categorical feature support. (#7966)\n* Models trained by the JVM packages can now be safely used with other language bindings. (#7896, #7907)\n* Users can specify the model format when saving models with a stream. (#7940, #7955)\n* The default value for training parameters is now sourced from XGBoost directly, which helps JVM packages be consistent with other packages. (#7938)\n* Set the correct objective if the user doesn't explicitly set it (#7781)\n* Auto-detection of MUSL is replaced by system properties (#7921)\n* Improved error message for launching tracker. (#7952, #7968)\n* Fix a race condition in parameter configuration. (#8025)\n* [Breaking] ` timeoutRequestWorkers` is now removed. With the support for barrier mode, this parameter is no longer needed. (#7839)\n* Dependencies updates. (#7791, #8157, #7801, #8240)\n\n### Documents\n- Document for the C interface is greatly improved and is now displayed at the [sphinx document page](https://xgboost.readthedocs.io/en/latest/c.html). Thanks to the breathe project, you can view the C API just like the Python API. (#8300)\n- We now avoid having XGBoost internal text parser in demos and recommend users use dedicated libraries for loading data whenever it's feasible. (#7753)\n- Python survival training demos are now displayed at [sphinx gallery](https://xgboost.readthedocs.io/en/latest/python/survival-examples/index.html). (#8328)\n- Some typos, links, format, and grammar fixes. (#7800, #7832, #7861, #8099, #8163, #8166, #8229, #8028, #8214, #7777, #7905, #8270, #8309, d70e59fef, #7806)\n- Updated winning solution under readme.md (#7862)\n- New security policy. (#8360)\n- GPU document is overhauled as we consider CUDA support to be feature-complete. (#8378)\n\n### Maintenance\n* Code refactoring and cleanups. (#7850, #7826, #7910, #8332, #8204)\n* Reduce compiler warnings. (#7768, #7916, #8046, #8059, #7974, #8031, #8022)\n* Compiler workarounds. (#8211, #8314, #8226, #8093)\n* Dependencies update. (#8001, #7876, #7973, #8298, #7816)\n* Remove warnings emitted in previous versions. (#7815)\n* Small fixes occurred during development. (#8008)\n\n### CI and Tests\n* We overhauled the CI infrastructure to reduce the CI cost and lift the maintenance burdens. Jenkins is replaced with buildkite for better automation, with which, finer control of test runs is implemented to reduce overall cost. Also, we refactored some of the existing tests to reduce their runtime, drooped the size of docker images, and removed multi-GPU C++ tests. Lastly, `pytest-timeout` is added as an optional dependency for running Python tests to keep the test time in check. (#7772, #8291, #8286, #8276, #8306, #8287, #8243, #8313, #8235, #8288, #8303, #8142, #8092, #8333, #8312, #8348)\n* New documents for how to reproduce the CI environment (#7971, #8297)\n* Improved automation for JVM release. (#7882)\n* GitHub Action security-related updates. (#8263, #8267, #8360)\n* Other fixes and maintenance work. (#8154, #7848, #8069, #7943)\n* Small updates and fixes to GitHub action pipelines. (#8364, #8321, #8241, #7950, #8011)\n\n## v1.6.1 (2022 May 9)\nThis is a patch release for bug fixes and Spark barrier mode support. The R package is unchanged.\n\n### Experimental support for categorical data\n- Fix segfault when the number of samples is smaller than the number of categories. (https://github.com/dmlc/xgboost/pull/7853)\n- Enable partition-based split for all model types. (https://github.com/dmlc/xgboost/pull/7857)\n\n### JVM packages\nWe replaced the old parallelism tracker with spark barrier mode to improve the robustness of the JVM package and fix the GPU training pipeline.\n- Fix GPU training pipeline quantile synchronization. (#7823, #7834)\n- Use barrier model in spark package. (https://github.com/dmlc/xgboost/pull/7836, https://github.com/dmlc/xgboost/pull/7840, https://github.com/dmlc/xgboost/pull/7845, https://github.com/dmlc/xgboost/pull/7846)\n- Fix shared object loading on some platforms. (https://github.com/dmlc/xgboost/pull/7844)\n\n## v1.6.0 (2022 Apr 16)\n\nAfter a long period of development, XGBoost v1.6.0 is packed with many new features and\nimprovements. We summarize them in the following sections starting with an introduction to\nsome major new features, then moving on to language binding specific changes including new\nfeatures and notable bug fixes for that binding.\n\n### Development of categorical data support\nThis version of XGBoost features new improvements and full coverage of experimental\ncategorical data support in Python and C package with tree model. Both `hist`, `approx`\nand `gpu_hist` now support training with categorical data. Also, partition-based\ncategorical split is introduced in this release. This split type is first available in\nLightGBM in the context of gradient boosting. The previous XGBoost release supported one-hot split where the splitting criteria is of form `x \\in {c}`, i.e. the categorical feature `x` is tested against a single candidate. The new release allows for more expressive conditions: `x \\in S` where the categorical feature `x` is tested against multiple candidates. Moreover, it is now possible to use any tree algorithms (`hist`, `approx`, `gpu_hist`) when creating categorical splits. For more\ninformation, please see our tutorial on [categorical\ndata](https://xgboost.readthedocs.io/en/latest/tutorials/categorical.html), along with\nexamples linked on that page. (#7380, #7708, #7695, #7330, #7307, #7322, #7705,\n#7652, #7592, #7666, #7576, #7569, #7529, #7575, #7393, #7465, #7385, #7371, #7745, #7810)\n\nIn the future, we will continue to improve categorical data support with new features and\noptimizations. Also, we are looking forward to bringing the feature beyond Python binding,\ncontributions and feedback are welcomed! Lastly, as a result of experimental status, the\nbehavior might be subject to change, especially the default value of related\nhyper-parameters.\n\n### Experimental support for multi-output model\n\nXGBoost 1.6 features initial support for the multi-output model, which includes\nmulti-output regression and multi-label classification. Along with this, the XGBoost\nclassifier has proper support for base margin without to need for the user to flatten the\ninput. In this initial support, XGBoost builds one model for each target similar to the\nsklearn meta estimator, for more details, please see our [quick\nintroduction](https://xgboost.readthedocs.io/en/latest/tutorials/multioutput.html).\n\n(#7365, #7736, #7607, #7574, #7521, #7514, #7456, #7453, #7455, #7434, #7429, #7405, #7381)\n\n### External memory support\nExternal memory support for both approx and hist tree method is considered feature\ncomplete in XGBoost 1.6. Building upon the iterator-based interface introduced in the\nprevious version, now both `hist` and `approx` iterates over each batch of data during\ntraining and prediction. In previous versions, `hist` concatenates all the batches into\nan internal representation, which is removed in this version. As a result, users can\nexpect higher scalability in terms of data size but might experience lower performance due\nto disk IO. (#7531, #7320, #7638, #7372)\n\n### Rewritten approx\n\nThe `approx` tree method is rewritten based on the existing `hist` tree method. The\nrewrite closes the feature gap between `approx` and `hist` and improves the performance.\nNow the behavior of `approx` should be more aligned with `hist` and `gpu_hist`. Here is a\nlist of user-visible changes:\n\n- Supports both `max_leaves` and `max_depth`.\n- Supports `grow_policy`.\n- Supports monotonic constraint.\n- Supports feature weights.\n- Use `max_bin` to replace `sketch_eps`.\n- Supports categorical data.\n- Faster performance for many of the datasets.\n- Improved performance and robustness for distributed training.\n- Supports prediction cache.\n- Significantly better performance for external memory when `depthwise` policy is used.\n\n### New serialization format\nBased on the existing JSON serialization format, we introduce UBJSON support as a more\nefficient alternative. Both formats will be available in the future and we plan to\ngradually [phase out](https://github.com/dmlc/xgboost/issues/7547) support for the old\nbinary model format. Users can opt to use the different formats in the serialization\nfunction by providing the file extension `json` or `ubj`. Also, the `save_raw` function in\nall supported languages bindings gains a new parameter for exporting the model in different\nformats, available options are `json`, `ubj`, and `deprecated`, see document for the\nlanguage binding you are using for details. Lastly, the default internal serialization\nformat is set to UBJSON, which affects Python pickle and R RDS. (#7572, #7570, #7358,\n#7571, #7556, #7549, #7416)\n\n### General new features and improvements\nAside from the major new features mentioned above, some others are summarized here:\n\n* Users can now access the build information of XGBoost binary in Python and C\n interface. (#7399, #7553)\n* Auto-configuration of `seed_per_iteration` is removed, now distributed training should\n generate closer results to single node training when sampling is used. (#7009)\n* A new parameter `huber_slope` is introduced for the `Pseudo-Huber` objective.\n* During source build, XGBoost can choose cub in the system path automatically. (#7579)\n* XGBoost now honors the CPU counts from CFS, which is usually set in docker\n environments. (#7654, #7704)\n* The metric `aucpr` is rewritten for better performance and GPU support. (#7297, #7368)\n* Metric calculation is now performed in double precision. (#7364)\n* XGBoost no longer mutates the global OpenMP thread limit. (#7537, #7519, #7608, #7590,\n #7589, #7588, #7687)\n* The default behavior of `max_leave` and `max_depth` is now unified (#7302, #7551).\n* CUDA fat binary is now compressed. (#7601)\n* Deterministic result for evaluation metric and linear model. In previous versions of\n XGBoost, evaluation results might differ slightly for each run due to parallel reduction\n for floating-point values, which is now addressed. (#7362, #7303, #7316, #7349)\n* XGBoost now uses double for GPU Hist node sum, which improves the accuracy of\n `gpu_hist`. (#7507)\n\n### Performance improvements\nMost of the performance improvements are integrated into other refactors during feature\ndevelopments. The `approx` should see significant performance gain for many datasets as\nmentioned in the previous section, while the `hist` tree method also enjoys improved\nperformance with the removal of the internal `pruner` along with some other\nrefactoring. Lastly, `gpu_hist` no longer synchronizes the device during training. (#7737)\n\n### General bug fixes\nThis section lists bug fixes that are not specific to any language binding.\n* The `num_parallel_tree` is now a model parameter instead of a training hyper-parameter,\n which fixes model IO with random forest. (#7751)\n* Fixes in CMake script for exporting configuration. (#7730)\n* XGBoost can now handle unsorted sparse input. This includes text file formats like\n libsvm and scipy sparse matrix where column index might not be sorted. (#7731)\n* Fix tree param feature type, this affects inputs with the number of columns greater than\n the maximum value of int32. (#7565)\n* Fix external memory with gpu_hist and subsampling. (#7481)\n* Check the number of trees in inplace predict, this avoids a potential segfault when an\n incorrect value for `iteration_range` is provided. (#7409)\n* Fix non-stable result in cox regression (#7756)\n\n### Changes in the Python package\nOther than the changes in Dask, the XGBoost Python package gained some new features and\nimprovements along with small bug fixes.\n\n* Python 3.7 is required as the lowest Python version. (#7682)\n* Pre-built binary wheel for Apple Silicon. (#7621, #7612, #7747) Apple Silicon users will\n now be able to run `pip install xgboost` to install XGBoost.\n* MacOS users no longer need to install `libomp` from Homebrew, as the XGBoost wheel now\n bundles `libomp.dylib` library.\n* There are new parameters for users to specify the custom metric with new\n behavior. XGBoost can now output transformed prediction values when a custom objective is\n not supplied. See our explanation in the\n [tutorial](https://xgboost.readthedocs.io/en/latest/tutorials/custom_metric_obj.html#reverse-link-function)\n for details.\n* For the sklearn interface, following the estimator guideline from scikit-learn, all\n parameters in `fit` that are not related to input data are moved into the constructor\n and can be set by `set_params`. (#6751, #7420, #7375, #7369)\n* Apache arrow format is now supported, which can bring better performance to users'\n pipeline (#7512)\n* Pandas nullable types are now supported (#7760)\n* A new function `get_group` is introduced for `DMatrix` to allow users to get the group\n information in the custom objective function. (#7564)\n* More training parameters are exposed in the sklearn interface instead of relying on the\n `**kwargs`. (#7629)\n* A new attribute `feature_names_in_` is defined for all sklearn estimators like\n `XGBRegressor` to follow the convention of sklearn. (#7526)\n* More work on Python type hint. (#7432, #7348, #7338, #7513, #7707)\n* Support the latest pandas Index type. (#7595)\n* Fix for Feature shape mismatch error on s390x platform (#7715)\n* Fix using feature names for constraints with multiple groups (#7711)\n* We clarified the behavior of the callback function when it contains mutable\n states. (#7685)\n* Lastly, there are some code cleanups and maintenance work. (#7585, #7426, #7634, #7665,\n #7667, #7377, #7360, #7498, #7438, #7667, #7752, #7749, #7751)\n\n### Changes in the Dask interface\n* Dask module now supports user-supplied host IP and port address of scheduler node.\n Please see [introduction](https://xgboost.readthedocs.io/en/latest/tutorials/dask.html#troubleshooting) and\n [API document](https://xgboost.readthedocs.io/en/latest/python/python_api.html#optional-dask-configuration)\n for reference. (#7645, #7581)\n* Internal `DMatrix` construction in dask now honers thread configuration. (#7337)\n* A fix for `nthread` configuration using the Dask sklearn interface. (#7633)\n* The Dask interface can now handle empty partitions. An empty partition is different\n from an empty worker, the latter refers to the case when a worker has no partition of an\n input dataset, while the former refers to some partitions on a worker that has zero\n sizes. (#7644, #7510)\n* Scipy sparse matrix is supported as Dask array partition. (#7457)\n* Dask interface is no longer considered experimental. (#7509)\n\n### Changes in the R package\nThis section summarizes the new features, improvements, and bug fixes to the R package.\n\n* `load.raw` can optionally construct a booster as return. (#7686)\n* Fix parsing decision stump, which affects both transforming text representation to data\n table and plotting. (#7689)\n* Implement feature weights. (#7660)\n* Some improvements for complying the CRAN release policy. (#7672, #7661, #7763)\n* Support CSR data for predictions (#7615)\n* Document update (#7263, #7606)\n* New maintainer for the CRAN package (#7691, #7649)\n* Handle non-standard installation of toolchain on macos (#7759)\n\n### Changes in JVM-packages\nSome new features for JVM-packages are introduced for a more integrated GPU pipeline and\nbetter compatibility with musl-based Linux. Aside from this, we have a few notable bug\nfixes.\n\n* User can specify the tracker IP address for training, which helps running XGBoost on\n restricted network environments. (#7808)\n* Add support for detecting musl-based Linux (#7624)\n* Add `DeviceQuantileDMatrix` to Scala binding (#7459)\n* Add Rapids plugin support, now more of the JVM pipeline can be accelerated by RAPIDS (#7491, #7779, #7793, #7806)\n* The setters for CPU and GPU are more aligned (#7692, #7798)\n* Control logging for early stopping (#7326)\n* Do not repartition when nWorker = 1 (#7676)\n* Fix the prediction issue for `multi:softmax` (#7694)\n* Fix for serialization of custom objective and eval (#7274)\n* Update documentation about Python tracker (#7396)\n* Remove jackson from dependency, which fixes CVE-2020-36518. (#7791)\n* Some refactoring to the training pipeline for better compatibility between CPU and\n GPU. (#7440, #7401, #7789, #7784)\n* Maintenance work. (#7550, #7335, #7641, #7523, #6792, #4676)\n\n### Deprecation\nOther than the changes in the Python package and serialization, we removed some deprecated\nfeatures in previous releases. Also, as mentioned in the previous section, we plan to\nphase out the old binary format in future releases.\n\n* Remove old warning in 1.3 (#7279)\n* Remove label encoder deprecated in 1.3. (#7357)\n* Remove old callback deprecated in 1.3. (#7280)\n* Pre-built binary will no longer support deprecated CUDA architectures including sm35 and\n sm50. Users can continue to use these platforms with source build. (#7767)\n\n### Documentation\nThis section lists some of the general changes to XGBoost's document, for language binding\nspecific change please visit related sections.\n\n* Document is overhauled to use the new RTD theme, along with integration of Python\n examples using Sphinx gallery. Also, we replaced most of the hard-coded URLs with sphinx\n references. (#7347, #7346, #7468, #7522, #7530)\n* Small update along with fixes for broken links, typos, etc. (#7684, #7324, #7334, #7655,\n #7628, #7623, #7487, #7532, #7500, #7341, #7648, #7311)\n* Update document for GPU. [skip ci] (#7403)\n* Document the status of RTD hosting. (#7353)\n* Update document for building from source. (#7664)\n* Add note about CRAN release [skip ci] (#7395)\n\n### Maintenance\nThis is a summary of maintenance work that is not specific to any language binding.\n\n* Add CMake option to use /MD runtime (#7277)\n* Add clang-format configuration. (#7383)\n* Code cleanups (#7539, #7536, #7466, #7499, #7533, #7735, #7722, #7668, #7304, #7293,\n #7321, #7356, #7345, #7387, #7577, #7548, #7469, #7680, #7433, #7398)\n* Improved tests with better coverage and latest dependency (#7573, #7446, #7650, #7520,\n #7373, #7723, #7611, #7771)\n* Improved automation of the release process. (#7278, #7332, #7470)\n* Compiler workarounds (#7673)\n* Change shebang used in CLI demo. (#7389)\n* Update affiliation (#7289)\n\n### CI\nSome fixes and update to XGBoost's CI infrastructure. (#7739, #7701, #7382, #7662, #7646,\n#7582, #7407, #7417, #7475, #7474, #7479, #7472, #7626)\n\n\n## v1.5.0 (2021 Oct 11)\n\nThis release comes with many exciting new features and optimizations, along with some bug\nfixes. We will describe the experimental categorical data support and the external memory\ninterface independently. Package-specific new features will be listed in respective\nsections.\n\n### Development on categorical data support\nIn version 1.3, XGBoost introduced an experimental feature for handling categorical data\nnatively, without one-hot encoding. XGBoost can fit categorical splits in decision\ntrees. (Currently, the generated splits will be of form `x \\in {v}`, where the input is\ncompared to a single category value. A future version of XGBoost will generate splits that\ncompare the input against a list of multiple category values.)\n\nMost of the other features, including prediction, SHAP value computation, feature\nimportance, and model plotting were revised to natively handle categorical splits. Also,\nall Python interfaces including native interface with and without quantized `DMatrix`,\nscikit-learn interface, and Dask interface now accept categorical data with a wide range\nof data structures support including numpy/cupy array and cuDF/pandas/modin dataframe. In\npractice, the following are required for enabling categorical data support during\ntraining:\n\n - Use Python package.\n - Use `gpu_hist` to train the model.\n - Use JSON model file format for saving the model.\n\nOnce the model is trained, it can be used with most of the features that are available on\nthe Python package. For a quick introduction, see\nhttps://xgboost.readthedocs.io/en/latest/tutorials/categorical.html\n\nRelated PRs: (#7011, #7001, #7042, #7041, #7047, #7043, #7036, #7054, #7053, #7065, #7213, #7228, #7220, #7221, #7231, #7306)\n\n* Next steps\n\n\t- Revise the CPU training algorithm to handle categorical data natively and generate categorical splits\n\t- Extend the CPU and GPU algorithms to generate categorical splits of form `x \\in S`\n\twhere the input is compared with multiple category values. split. (#7081)\n\n### External memory\nThis release features a brand-new interface and implementation for external memory (also\nknown as out-of-core training). (#6901, #7064, #7088, #7089, #7087, #7092, #7070,\n#7216). The new implementation leverages the data iterator interface, which is currently\nused to create `DeviceQuantileDMatrix`. For a quick introduction, see\nhttps://xgboost.readthedocs.io/en/latest/tutorials/external_memory.html#data-iterator\n. During the development of this new interface, `lz4` compression is removed. (#7076).\nPlease note that external memory support is still experimental and not ready for\nproduction use yet. All future development will focus on this new interface and users are\nadvised to migrate. (You are using the old interface if you are using a URL suffix to use\nexternal memory.)\n\n### New features in Python package\n* Support numpy array interface and all numeric types from numpy in `DMatrix`\n construction and `inplace_predict` (#6998, #7003). Now XGBoost no longer makes data\n copy when input is numpy array view.\n* The early stopping callback in Python has a new `min_delta` parameter to control the\n stopping behavior (#7137)\n* Python package now supports calculating feature scores for the linear model, which is\n also available on R package. (#7048)\n* Python interface now supports configuring constraints using feature names instead of\n feature indices.\n* Typehint support for more Python code including scikit-learn interface and rabit\n module. (#6799, #7240)\n* Add tutorial for XGBoost-Ray (#6884)\n\n### New features in R package\n* In 1.4 we have a new prediction function in the C API which is used by the Python\n package. This release revises the R package to use the new prediction function as well.\n A new parameter `iteration_range` for the predict function is available, which can be\n used for specifying the range of trees for running prediction. (#6819, #7126)\n* R package now supports the `nthread` parameter in `DMatrix` construction. (#7127)\n\n### New features in JVM packages\n* Support GPU dataframe and `DeviceQuantileDMatrix` (#7195). Constructing `DMatrix`\n with GPU data structures and the interface for quantized `DMatrix` were first\n introduced in the Python package and are now available in the xgboost4j package.\n* JVM packages now support saving and getting early stopping attributes. (#7095) Here is a\n quick [example](https://github.com/dmlc/xgboost/jvm-packages/xgboost4j-example/src/main/java/ml/dmlc/xgboost4j/java/example/EarlyStopping.java \"example\") in JAVA (#7252).\n\n### General new features\n* We now have a pre-built binary package for R on Windows with GPU support. (#7185)\n* CUDA compute capability 86 is now part of the default CMake build configuration with\n newly added support for CUDA 11.4. (#7131, #7182, #7254)\n* XGBoost can be compiled using system CUB provided by CUDA 11.x installation. (#7232)\n\n### Optimizations\nThe performance for both `hist` and `gpu_hist` has been significantly improved in 1.5\nwith the following optimizations:\n* GPU multi-class model training now supports prediction cache. (#6860)\n* GPU histogram building is sped up and the overall training time is 2-3 times faster on\n large datasets (#7180, #7198). In addition, we removed the parameter `deterministic_histogram` and now\n the GPU algorithm is always deterministic.\n* CPU hist has an optimized procedure for data sampling (#6922)\n* More performance optimization in regression and binary classification objectives on\n CPU (#7206)\n* Tree model dump is now performed in parallel (#7040)\n\n### Breaking changes\n* `n_gpus` was deprecated in 1.0 release and is now removed.\n* Feature grouping in CPU hist tree method is removed, which was disabled long\n ago. (#7018)\n* C API for Quantile DMatrix is changed to be consistent with the new external memory\n implementation. (#7082)\n\n### Notable general bug fixes\n* XGBoost no long changes global CUDA device ordinal when `gpu_id` is specified (#6891,\n #6987)\n* Fix `gamma` negative likelihood evaluation metric. (#7275)\n* Fix integer value of `verbose_eal` for `xgboost.cv` function in Python. (#7291)\n* Remove extra sync in CPU hist for dense data, which can lead to incorrect tree node\n statistics. (#7120, #7128)\n* Fix a bug in GPU hist when data size is larger than `UINT32_MAX` with missing\n values. (#7026)\n* Fix a thread safety issue in prediction with the `softmax` objective. (#7104)\n* Fix a thread safety issue in CPU SHAP value computation. (#7050) Please note that all\n prediction functions in Python are thread-safe.\n* Fix model slicing. (#7149, #7078)\n* Workaround a bug in old GCC which can lead to segfault during construction of\n DMatrix. (#7161)\n* Fix histogram truncation in GPU hist, which can lead to slightly-off results. (#7181)\n* Fix loading GPU linear model pickle files on CPU-only machine. (#7154)\n* Check input value is duplicated when CPU quantile queue is full (#7091)\n* Fix parameter loading with training continuation. (#7121)\n* Fix CMake interface for exposing C library by specifying dependencies. (#7099)\n* Callback and early stopping are explicitly disabled for the scikit-learn interface\n random forest estimator. (#7236)\n* Fix compilation error on x86 (32-bit machine) (#6964)\n* Fix CPU memory usage with extremely sparse datasets (#7255)\n* Fix a bug in GPU multi-class AUC implementation with weighted data (#7300)\n\n### Python package\nOther than the items mentioned in the previous sections, there are some Python-specific\nimprovements.\n* Change development release postfix to `dev` (#6988)\n* Fix early stopping behavior with MAPE metric (#7061)\n* Fixed incorrect feature mismatch error message (#6949)\n* Add predictor to skl constructor. (#7000, #7159)\n* Re-enable feature validation in predict proba. (#7177)\n* scikit learn interface regression estimator now can pass the scikit-learn estimator\n check and is fully compatible with scikit-learn utilities. `__sklearn_is_fitted__` is\n implemented as part of the changes (#7130, #7230)\n* Conform the latest pylint. (#7071, #7241)\n* Support latest panda range index in DMatrix construction. (#7074)\n* Fix DMatrix construction from pandas series. (#7243)\n* Fix typo and grammatical mistake in error message (#7134)\n* [dask] disable work stealing explicitly for training tasks (#6794)\n* [dask] Set dataframe index in predict. (#6944)\n* [dask] Fix prediction on df with latest dask. (#6969)\n* [dask] Fix dask predict on `DaskDMatrix` with `iteration_range`. (#7005)\n* [dask] Disallow importing non-dask estimators from xgboost.dask (#7133)\n\n### R package\nImprovements other than new features on R package:\n* Optimization for updating R handles in-place (#6903)\n* Removed the magrittr dependency. (#6855, #6906, #6928)\n* The R package now hides all C++ symbols to avoid conflicts. (#7245)\n* Other maintenance including code cleanups, document updates. (#6863, #6915, #6930, #6966, #6967)\n\n### JVM packages\nImprovements other than new features on JVM packages:\n* Constructors with implicit missing value are deprecated due to confusing behaviors. (#7225)\n* Reduce scala-compiler, scalatest dependency scopes (#6730)\n* Making the Java library loader emit helpful error messages on missing dependencies. (#6926)\n* JVM packages now use the Python tracker in XGBoost instead of dmlc. The one in XGBoost\n is shared between JVM packages and Python Dask and enjoys better maintenance (#7132)\n* Fix \"key not found: train\" error (#6842)\n* Fix model loading from stream (#7067)\n\n### General document improvements\n* Overhaul the installation documents. (#6877)\n* A few demos are added for AFT with dask (#6853), callback with dask (#6995), inference\n in C (#7151), `process_type`. (#7135)\n* Fix PDF format of document. (#7143)\n* Clarify the behavior of `use_rmm`. (#6808)\n* Clarify prediction function. (#6813)\n* Improve tutorial on feature interactions (#7219)\n* Add small example for dask sklearn interface. (#6970)\n* Update Python intro. (#7235)\n* Some fixes/updates (#6810, #6856, #6935, #6948, #6976, #7084, #7097, #7170, #7173, #7174, #7226, #6979, #6809, #6796, #6979)\n\n### Maintenance\n* Some refactoring around CPU hist, which lead to better performance but are listed under general maintenance tasks:\n - Extract evaluate splits from CPU hist. (#7079)\n - Merge lossgude and depthwise strategies for CPU hist (#7007)\n - Simplify sparse and dense CPU hist kernels (#7029)\n - Extract histogram builder from CPU Hist. (#7152)\n\n* Others\n - Fix `gpu_id` with custom objective. (#7015)\n - Fix typos in AUC. (#6795)\n - Use constexpr in `dh::CopyIf`. (#6828)\n - Update dmlc-core. (#6862)\n - Bump version to 1.5.0 snapshot in master. (#6875)\n - Relax shotgun test. (#6900)\n - Guard against index error in prediction. (#6982)\n - Hide symbols in CI build + hide symbols for C and CUDA (#6798)\n - Persist data in dask test. (#7077)\n - Fix typo in arguments of PartitionBuilder::Init (#7113)\n - Fix typo in src/common/hist.cc BuildHistKernel (#7116)\n - Use upstream URI in distributed quantile tests. (#7129)\n - Include cpack (#7160)\n - Remove synchronization in monitor. (#7164)\n - Remove unused code. (#7175)\n - Fix building on CUDA 11.0. (#7187)\n - Better error message for `ncclUnhandledCudaError`. (#7190)\n - Add noexcept to JSON objects. (#7205)\n - Improve wording for warning (#7248)\n - Fix typo in release script. [skip ci] (#7238)\n - Relax shotgun test. (#6918)\n - Relax test for decision stump in distributed environment. (#6919)\n -\t[dask] speed up tests (#7020)\n\n### CI\n* [CI] Rotate access keys for uploading MacOS artifacts from Travis CI (#7253)\n* Reduce Travis environment setup time. (#6912)\n* Restore R cache on github action. (#6985)\n* [CI] Remove stray build artifact to avoid error in artifact packaging (#6994)\n* [CI] Move appveyor tests to action (#6986)\n* Remove appveyor badge. [skip ci] (#7035)\n* [CI] Configure RAPIDS, dask, modin (#7033)\n* Test on s390x. (#7038)\n* [CI] Upgrade to CMake 3.14 (#7060)\n* [CI] Update R cache. (#7102)\n* [CI] Pin libomp to 11.1.0 (#7107)\n* [CI] Upgrade build image to CentOS 7 + GCC 8; require CUDA 10.1 and later (#7141)\n* [dask] Work around segfault in prediction. (#7112)\n* [dask] Remove the workaround for segfault. (#7146)\n* [CI] Fix hanging Python setup in Windows CI (#7186)\n* [CI] Clean up in beginning of each task in Win CI (#7189)\n* Fix travis. (#7237)\n\n### Acknowledgement\n* **Contributors**: Adam Pocock (@Craigacp), Jeff H (@JeffHCross), Johan Hansson (@JohanWork), Jose Manuel Llorens (@JoseLlorensRipolles), Benjamin Szőke (@Livius90), @ReeceGoding, @ShvetsKS, Robert Zabel (@ZabelTech), Ali (@ali5h), Andrew Ziem (@az0), Andy Adinets (@canonizer), @david-cortes, Daniel Saxton (@dsaxton), Emil Sadek (@esadek), @farfarawayzyt, Gil Forsyth (@gforsyth), @giladmaya, @graue70, Philip Hyunsu Cho (@hcho3), James Lamb (@jameslamb), José Morales (@jmoralez), Kai Fricke (@krfricke), Christian Lorentzen (@lorentzenchr), Mads R. B. Kristensen (@madsbk), Anton Kostin (@masguit42), Martin Petříček (@mpetricek-corp), @naveenkb, Taewoo Kim (@oOTWK), Viktor Szathmáry (@phraktle), Robert Maynard (@robertmaynard), TP Boudreau (@tpboudreau), Jiaming Yuan (@trivialfis), Paul Taylor (@trxcllnt), @vslaykovsky, Bobby Wang (@wbo4958),\n* **Reviewers**: Nan Zhu (@CodingCat), Adam Pocock (@Craigacp), Jose Manuel Llorens (@JoseLlorensRipolles), Kodi Arfer (@Kodiologist), Benjamin Szőke (@Livius90), Mark Guryanov (@MarkGuryanov), Rory Mitchell (@RAMitchell), @ReeceGoding, @ShvetsKS, Egor Smirnov (@SmirnovEgorRu), Andrew Ziem (@az0), @candalfigomoro, Andy Adinets (@canonizer), Dante Gama Dessavre (@dantegd), @david-cortes, Daniel Saxton (@dsaxton), @farfarawayzyt, Gil Forsyth (@gforsyth), Harutaka Kawamura (@harupy), Philip Hyunsu Cho (@hcho3), @jakirkham, James Lamb (@jameslamb), José Morales (@jmoralez), James Bourbeau (@jrbourbeau), Christian Lorentzen (@lorentzenchr), Martin Petříček (@mpetricek-corp), Nikolay Petrov (@napetrov), @naveenkb, Viktor Szathmáry (@phraktle), Robin Teuwens (@rteuwens), Yuan Tang (@terrytangyuan), TP Boudreau (@tpboudreau), Jiaming Yuan (@trivialfis), @vkuzmin-uber, Bobby Wang (@wbo4958), William Hicks (@wphicks)\n\n\n## v1.4.2 (2021.05.13)\nThis is a patch release for Python package with following fixes:\n\n* Handle the latest version of cupy.ndarray in inplace_predict. (#6933)\n* Ensure output array from predict_leaf is (n_samples, ) when there's only 1 tree. 1.4.0 outputs (n_samples, 1). (#6889)\n* Fix empty dataset handling with multi-class AUC. (#6947)\n* Handle object type from pandas in inplace_predict. (#6927)\n\n\n## v1.4.1 (2021.04.20)\nThis is a bug fix release.\n\n* Fix GPU implementation of AUC on some large datasets. (#6866)\n\n## v1.4.0 (2021.04.12)\n\n### Introduction of pre-built binary package for R, with GPU support\nStarting with release 1.4.0, users now have the option of installing `{xgboost}` without\nhaving to build it from the source. This is particularly advantageous for users who want\nto take advantage of the GPU algorithm (`gpu_hist`), as previously they'd have to build\n`{xgboost}` from the source using CMake and NVCC. Now installing `{xgboost}` with GPU\nsupport is as easy as: `R CMD INSTALL ./xgboost_r_gpu_linux.tar.gz`. (#6827)\n\nSee the instructions at https://xgboost.readthedocs.io/en/latest/build.html\n\n### Improvements on prediction functions\nXGBoost has many prediction types including shap value computation and inplace prediction.\nIn 1.4 we overhauled the underlying prediction functions for C API and Python API with an\nunified interface. (#6777, #6693, #6653, #6662, #6648, #6668, #6804)\n* Starting with 1.4, sklearn interface prediction will use inplace predict by default when\n input data is supported.\n* Users can use inplace predict with `dart` booster and enable GPU acceleration just\n like `gbtree`.\n* Also all prediction functions with tree models are now thread-safe. Inplace predict is\n improved with `base_margin` support.\n* A new set of C predict functions are exposed in the public interface.\n* A user-visible change is a newly added parameter called `strict_shape`. See\n https://xgboost.readthedocs.io/en/latest/prediction.html for more details.\n\n\n### Improvement on Dask interface\n* Starting with 1.4, the Dask interface is considered to be feature-complete, which means\n all of the models found in the single node Python interface are now supported in Dask,\n including but not limited to ranking and random forest. Also, the prediction function\n is significantly faster and supports shap value computation.\n - Most of the parameters found in single node sklearn interface are supported by\n Dask interface. (#6471, #6591)\n - Implements learning to rank. On the Dask interface, we use the newly added support of\n query ID to enable group structure. (#6576)\n - The Dask interface has Python type hints support. (#6519)\n - All models can be safely pickled. (#6651)\n - Random forest estimators are now supported. (#6602)\n - Shap value computation is now supported. (#6575, #6645, #6614)\n - Evaluation result is printed on the scheduler process. (#6609)\n - `DaskDMatrix` (and device quantile dmatrix) now accepts all meta-information. (#6601)\n\n* Prediction optimization. We enhanced and speeded up the prediction function for the\n Dask interface. See the latest Dask tutorial page in our document for an overview of\n how you can optimize it even further. (#6650, #6645, #6648, #6668)\n\n* Bug fixes\n - If you are using the latest Dask and distributed where `distributed.MultiLock` is\n present, XGBoost supports training multiple models on the same cluster in\n parallel. (#6743)\n - A bug fix for when using `dask.client` to launch async task, XGBoost might use a\n different client object internally. (#6722)\n\n* Other improvements on documents, blogs, tutorials, and demos. (#6389, #6366, #6687,\n #6699, #6532, #6501)\n\n### Python package\nWith changes from Dask and general improvement on prediction, we have made some\nenhancements on the general Python interface and IO for booster information. Starting\nfrom 1.4, booster feature names and types can be saved into the JSON model. Also some\nmodel attributes like `best_iteration`, `best_score` are restored upon model load. On\nsklearn interface, some attributes are now implemented as Python object property with\nbetter documents.\n\n* Breaking change: All `data` parameters in prediction functions are renamed to `X`\n for better compliance to sklearn estimator interface guidelines.\n* Breaking change: XGBoost used to generate some pseudo feature names with `DMatrix`\n when inputs like `np.ndarray` don't have column names. The procedure is removed to\n avoid conflict with other inputs. (#6605)\n* Early stopping with training continuation is now supported. (#6506)\n* Optional import for Dask and cuDF are now lazy. (#6522)\n* As mentioned in the prediction improvement summary, the sklearn interface uses inplace\n prediction whenever possible. (#6718)\n* Booster information like feature names and feature types are now saved into the JSON\n model file. (#6605)\n* All `DMatrix` interfaces including `DeviceQuantileDMatrix` and counterparts in Dask\n interface (as mentioned in the Dask changes summary) now accept all the meta-information\n like `group` and `qid` in their constructor for better consistency. (#6601)\n* Booster attributes are restored upon model load so users don't have to call `attr`\n manually. (#6593)\n* On sklearn interface, all models accept `base_margin` for evaluation datasets. (#6591)\n* Improvements over the setup script including smaller sdist size and faster installation\n if the C++ library is already built (#6611, #6694, #6565).\n\n* Bug fixes for Python package:\n - Don't validate feature when number of rows is 0. (#6472)\n - Move metric configuration into booster. (#6504)\n - Calling XGBModel.fit() should clear the Booster by default (#6562)\n - Support `_estimator_type`. (#6582)\n - [dask, sklearn] Fix predict proba. (#6566, #6817)\n - Restore unknown data support. (#6595)\n - Fix learning rate scheduler with cv. (#6720)\n - Fixes small typo in sklearn documentation (#6717)\n - [python-package] Fix class Booster: feature_types = None (#6705)\n - Fix divide by 0 in feature importance when no split is found. (#6676)\n\n\n### JVM package\n* [jvm-packages] fix early stopping doesn't work even without custom_eval setting (#6738)\n* fix potential TaskFailedListener's callback won't be called (#6612)\n* [jvm] Add ability to load booster direct from byte array (#6655)\n* [jvm-packages] JVM library loader extensions (#6630)\n\n### R package\n* R documentation: Make construction of DMatrix consistent.\n* Fix R documentation for xgb.train. (#6764)\n\n### ROC-AUC\nWe re-implemented the ROC-AUC metric in XGBoost. The new implementation supports\nmulti-class classification and has better support for learning to rank tasks that are not\nbinary. Also, it has a better-defined average on distributed environments with additional\nhandling for invalid datasets. (#6749, #6747, #6797)\n\n### Global configuration.\nStarting from 1.4, XGBoost's Python, R and C interfaces support a new global configuration\nmodel where users can specify some global parameters. Currently, supported parameters are\n`verbosity` and `use_rmm`. The latter is experimental, see rmm plugin demo and\nrelated README file for details. (#6414, #6656)\n\n### Other New features.\n* Better handling for input data types that support `__array_interface__`. For some\n data types including GPU inputs and `scipy.sparse.csr_matrix`, XGBoost employs\n `__array_interface__` for processing the underlying data. Starting from 1.4, XGBoost\n can accept arbitrary array strides (which means column-major is supported) without\n making data copies, potentially reducing a significant amount of memory consumption.\n Also version 3 of `__cuda_array_interface__` is now supported. (#6776, #6765, #6459,\n #6675)\n* Improved parameter validation, now feeding XGBoost with parameters that contain\n whitespace will trigger an error. (#6769)\n* For Python and R packages, file paths containing the home indicator `~` are supported.\n* As mentioned in the Python changes summary, the JSON model can now save feature\n information of the trained booster. The JSON schema is updated accordingly. (#6605)\n* Development of categorical data support is continued. Newly added weighted data support\n and `dart` booster support. (#6508, #6693)\n* As mentioned in Dask change summary, ranking now supports the `qid` parameter for\n query groups. (#6576)\n* `DMatrix.slice` can now consume a numpy array. (#6368)\n\n### Other breaking changes\n* Aside from the feature name generation, there are 2 breaking changes:\n - Drop saving binary format for memory snapshot. (#6513, #6640)\n - Change default evaluation metric for binary:logitraw objective to logloss (#6647)\n\n### CPU Optimization\n* Aside from the general changes on predict function, some optimizations are applied on\n CPU implementation. (#6683, #6550, #6696, #6700)\n* Also performance for sampling initialization in `hist` is improved. (#6410)\n\n### Notable fixes in the core library\nThese fixes do not reside in particular language bindings:\n* Fixes for gamma regression. This includes checking for invalid input values, fixes for\n gamma deviance metric, and better floating point guard for gamma negative log-likelihood\n metric. (#6778, #6537, #6761)\n* Random forest with `gpu_hist` might generate low accuracy in previous versions. (#6755)\n* Fix a bug in GPU sketching when data size exceeds limit of 32-bit integer. (#6826)\n* Memory consumption fix for row-major adapters (#6779)\n* Don't estimate sketch batch size when rmm is used. (#6807) (#6830)\n* Fix in-place predict with missing value. (#6787)\n* Re-introduce double buffer in UpdatePosition, to fix perf regression in gpu_hist (#6757)\n* Pass correct split_type to GPU predictor (#6491)\n* Fix DMatrix feature names/types IO. (#6507)\n* Use view for `SparsePage` exclusively to avoid some data access races. (#6590)\n* Check for invalid data. (#6742)\n* Fix relocatable include in CMakeList (#6734) (#6737)\n* Fix DMatrix slice with feature types. (#6689)\n\n### Other deprecation notices:\n\n* This release will be the last release to support CUDA 10.0. (#6642)\n\n* Starting in the next release, the Python package will require Pip 19.3+ due to the use\n of manylinux2014 tag. Also, CentOS 6, RHEL 6 and other old distributions will not be\n supported.\n\n### Known issue:\n\nMacOS build of the JVM packages doesn't support multi-threading out of the box. To enable\nmulti-threading with JVM packages, MacOS users will need to build the JVM packages from\nthe source. See https://xgboost.readthedocs.io/en/latest/jvm/index.html#installation-from-source\n\n\n### Doc\n* Dedicated page for `tree_method` parameter is added. (#6564, #6633)\n* [doc] Add FLAML as a fast tuning tool for XGBoost (#6770)\n* Add document for tests directory. [skip ci] (#6760)\n* Fix doc string of config.py to use correct `versionadded` (#6458)\n* Update demo for prediction. (#6789)\n* [Doc] Document that AUCPR is for binary classification/ranking (#5899)\n* Update the C API comments (#6457)\n* Fix document. [skip ci] (#6669)\n\n### Maintenance: Testing, continuous integration\n* Use CPU input for test_boost_from_prediction. (#6818)\n* [CI] Upload xgboost4j.dll to S3 (#6781)\n* Update dmlc-core submodule (#6745)\n* [CI] Use manylinux2010_x86_64 container to vendor libgomp (#6485)\n* Add conda-forge badge (#6502)\n* Fix merge conflict. (#6512)\n* [CI] Split up main.yml, add mypy. (#6515)\n* [Breaking] Upgrade cuDF and RMM to 0.18 nightlies; require RMM 0.18+ for RMM plugin (#6510)\n* \"featue_map\" typo changed to \"feature_map\" (#6540)\n* Add script for generating release tarball. (#6544)\n* Add credentials to .gitignore (#6559)\n* Remove warnings in tests. (#6554)\n* Update dmlc-core submodule and conform to new API (#6431)\n* Suppress hypothesis health check for dask client. (#6589)\n* Fix pylint. (#6714)\n* [CI] Clear R package cache (#6746)\n* Exclude dmlc test on github action. (#6625)\n* Tests for regression metrics with weights. (#6729)\n* Add helper script and doc for releasing pip package. (#6613)\n* Support pylint 2.7.0 (#6726)\n* Remove R cache in github action. (#6695)\n* [CI] Do not mix up stashed executable built for ARM and x86_64 platforms (#6646)\n* [CI] Add ARM64 test to Jenkins pipeline (#6643)\n* Disable s390x and arm64 tests on travis for now. (#6641)\n* Move sdist test to action. (#6635)\n* [dask] Rework base margin test. (#6627)\n\n\n### Maintenance: Refactor code for legibility and maintainability\n* Improve OpenMP exception handling (#6680)\n* Improve string view to reduce string allocation. (#6644)\n* Simplify Span checks. (#6685)\n* Use generic dispatching routine for array interface. (#6672)\n\n\n## v1.3.0 (2020.12.08)\n\n### XGBoost4J-Spark: Exceptions should cancel jobs gracefully instead of killing SparkContext (#6019).\n* By default, exceptions in XGBoost4J-Spark causes the whole SparkContext to shut down, necessitating the restart of the Spark cluster. This behavior is often a major inconvenience.\n* Starting from 1.3.0 release, XGBoost adds a new parameter `killSparkContextOnWorkerFailure` to optionally prevent killing SparkContext. If this parameter is set, exceptions will gracefully cancel training jobs instead of killing SparkContext.\n\n### GPUTreeSHAP: GPU acceleration of the TreeSHAP algorithm (#6038, #6064, #6087, #6099, #6163, #6281, #6332)\n* [SHAP (SHapley Additive exPlanations)](https://github.com/slundberg/shap) is a game theoretic approach to explain predictions of machine learning models. It computes feature importance scores for individual examples, establishing how each feature influences a particular prediction. TreeSHAP is an optimized SHAP algorithm specifically designed for decision tree ensembles.\n* Starting with 1.3.0 release, it is now possible to leverage CUDA-capable GPUs to accelerate the TreeSHAP algorithm. Check out [the demo notebook](https://github.com/dmlc/xgboost/blob/master/demo/gpu_acceleration/shap.ipynb).\n* The CUDA implementation of the TreeSHAP algorithm is hosted at [rapidsai/GPUTreeSHAP](https://github.com/rapidsai/gputreeshap). XGBoost imports it as a Git submodule.\n\n### New style Python callback API (#6199, #6270, #6320, #6348, #6376, #6399, #6441)\n* The XGBoost Python package now offers a re-designed callback API. The new callback API lets you design various extensions of training in idomatic Python. In addition, the new callback API allows you to use early stopping with the native Dask API (`xgboost.dask`). Check out [the tutorial](https://xgboost.readthedocs.io/en/release_1.3.0/python/callbacks.html) and [the demo](https://github.com/dmlc/xgboost/blob/master/demo/guide-python/callbacks.py).\n\n### Enable the use of `DeviceQuantileDMatrix` / `DaskDeviceQuantileDMatrix` with large data (#6201, #6229, #6234).\n* `DeviceQuantileDMatrix` can achieve memory saving by avoiding extra copies of the training data, and the saving is bigger for large data. Unfortunately, large data with more than 2^31 elements was triggering integer overflow bugs in CUB and Thrust. Tracking issue: #6228.\n* This release contains a series of work-arounds to allow the use of `DeviceQuantileDMatrix` with large data:\n - Loop over `copy_if` (#6201)\n - Loop over `thrust::reduce` (#6229)\n - Implement the inclusive scan algorithm in-house, to handle large offsets (#6234)\n\n### Support slicing of tree models (#6302)\n* Accessing the best iteration of a model after the application of early stopping used to be error-prone, need to manually pass the `ntree_limit` argument to the `predict()` function.\n* Now we provide a simple interface to slice tree models by specifying a range of boosting rounds. The tree ensemble can be split into multiple sub-ensembles via the slicing interface. Check out [an example](https://xgboost.readthedocs.io/en/release_1.3.0/python/model.html).\n* In addition, the early stopping callback now supports `save_best` option. When enabled, XGBoost will save (persist) the model at the best boosting round and discard the trees that were fit subsequent to the best round.\n\n### Weighted subsampling of features (columns) (#5962)\n* It is now possible to sample features (columns) via weighted subsampling, in which features with higher weights are more likely to be selected in the sample. Weighted subsampling allows you to encode domain knowledge by emphasizing a particular set of features in the choice of tree splits. In addition, you can prevent particular features from being used in any splits, by assigning them zero weights.\n* Check out [the demo](https://github.com/dmlc/xgboost/blob/master/demo/guide-python/feature_weights.py).\n\n### Improved integration with Dask\n* Support reverse-proxy environment such as Google Kubernetes Engine (#6343, #6475)\n* An XGBoost training job will no longer use all available workers. Instead, it will only use the workers that contain input data (#6343).\n* The new callback API works well with the Dask training API.\n* The `predict()` and `fit()` function of `DaskXGBClassifier` and `DaskXGBRegressor` now accept a base margin (#6155).\n* Support more meta data in the Dask API (#6130, #6132, #6333).\n* Allow passing extra keyword arguments as `kwargs` in `predict()` (#6117)\n* Fix typo in dask interface: `sample_weights` -> `sample_weight` (#6240)\n* Allow empty data matrix in AFT survival, as Dask may produce empty partitions (#6379)\n* Speed up prediction by overlapping prediction jobs in all workers (#6412)\n\n### Experimental support for direct splits with categorical features (#6028, #6128, #6137, #6140, #6164, #6165, #6166, #6179, #6194, #6219)\n* Currently, XGBoost requires users to one-hot-encode categorical variables. This has adverse performance implications, as the creation of many dummy variables results into higher memory consumption and may require fitting deeper trees to achieve equivalent model accuracy.\n* The 1.3.0 release of XGBoost contains an experimental support for direct handling of categorical variables in test nodes. Each test node will have the condition of form `feature_value \\in match_set`, where the `match_set` on the right hand side contains one or more matching categories. The matching categories in `match_set` represent the condition for traversing to the right child node. Currently, XGBoost will only generate categorical splits with only a single matching category (\"one-vs-rest split\"). In a future release, we plan to remove this restriction and produce splits with multiple matching categories in `match_set`.\n* The categorical split requires the use of JSON model serialization. The legacy binary serialization method cannot be used to save (persist) models with categorical splits.\n* Note. This feature is currently highly experimental. Use it at your own risk. See the detailed list of limitations at [#5949](https://github.com/dmlc/xgboost/pull/5949).\n\n### Experimental plugin for RAPIDS Memory Manager (#5873, #6131, #6146, #6150, #6182)\n* RAPIDS Memory Manager library ([rapidsai/rmm](https://github.com/rapidsai/rmm)) provides a collection of efficient memory allocators for NVIDIA GPUs. It is now possible to use XGBoost with memory allocators provided by RMM, by enabling the RMM integration plugin. With this plugin, XGBoost is now able to share a common GPU memory pool with other applications using RMM, such as the RAPIDS data science packages.\n* See [the demo](https://github.com/dmlc/xgboost/blob/master/demo/rmm_plugin/README.md) for a working example, as well as directions for building XGBoost with the RMM plugin.\n* The plugin will be soon considered non-experimental, once #6297 is resolved.\n\n### Experimental plugin for oneAPI programming model (#5825)\n* oneAPI is a programming interface developed by Intel aimed at providing one programming model for many types of hardware such as CPU, GPU, FGPA and other hardware accelerators.\n* XGBoost now includes an experimental plugin for using oneAPI for the predictor and objective functions. The plugin is hosted in the directory `plugin/updater_oneapi`.\n* Roadmap: #5442\n\n### Pickling the XGBoost model will now trigger JSON serialization (#6027)\n* The pickle will now contain the JSON string representation of the XGBoost model, as well as related configuration.\n\n### Performance improvements\n* Various performance improvement on multi-core CPUs\n - Optimize DMatrix build time by up to 3.7x. (#5877)\n - CPU predict performance improvement, by up to 3.6x. (#6127)\n - Optimize CPU sketch allreduce for sparse data (#6009)\n - Thread local memory allocation for BuildHist, leading to speedup up to 1.7x. (#6358)\n - Disable hyperthreading for DMatrix creation (#6386). This speeds up DMatrix creation by up to 2x.\n - Simple fix for static shedule in predict (#6357)\n* Unify thread configuration, to make it easy to utilize all CPU cores (#6186)\n* [jvm-packages] Clean the way deterministic paritioning is computed (#6033)\n* Speed up JSON serialization by implementing an intrusive pointer class (#6129). It leads to 1.5x-2x performance boost.\n\n### API additions\n* [R] Add SHAP summary plot using ggplot2 (#5882)\n* Modin DataFrame can now be used as input (#6055)\n* [jvm-packages] Add `getNumFeature` method (#6075)\n* Add MAPE metric (#6119)\n* Implement GPU predict leaf. (#6187)\n* Enable cuDF/cuPy inputs in `XGBClassifier` (#6269)\n* Document tree method for feature weights. (#6312)\n* Add `fail_on_invalid_gpu_id` parameter, which will cause XGBoost to terminate upon seeing an invalid value of `gpu_id` (#6342)\n\n### Breaking: the default evaluation metric for classification is changed to `logloss` / `mlogloss` (#6183)\n* The default metric used to be accuracy, and it is not statistically consistent to perform early stopping with the accuracy metric when we are really optimizing the log loss for the `binary:logistic` objective.\n* For statistical consistency, the default metric for classification has been changed to `logloss`. Users may choose to preserve the old behavior by explicitly specifying `eval_metric`.\n\n### Breaking: `skmaker` is now removed (#5971)\n* The `skmaker` updater has not been documented nor tested.\n\n### Breaking: the JSON model format no longer stores the leaf child count (#6094).\n* The leaf child count field has been deprecated and is not used anywhere in the XGBoost codebase.\n\n### Breaking: XGBoost now requires MacOS 10.14 (Mojave) and later.\n* Homebrew has dropped support for MacOS 10.13 (High Sierra), so we are not able to install the OpenMP runtime (`libomp`) from Homebrew on MacOS 10.13. Please use MacOS 10.14 (Mojave) or later.\n\n### Deprecation notices\n* The use of `LabelEncoder` in `XGBClassifier` is now deprecated and will be removed in the next minor release (#6269). The deprecation is necessary to support multiple types of inputs, such as cuDF data frames or cuPy arrays.\n* The use of certain positional arguments in the Python interface is deprecated (#6365). Users will use deprecation warnings for the use of position arguments for certain function parameters. New code should use keyword arguments as much as possible. We have not yet decided when we will fully require the use of keyword arguments.\n\n### Bug-fixes\n* On big-endian arch, swap the byte order in the binary serializer to enable loading models that were produced by a little-endian machine (#5813).\n* [jvm-packages] Fix deterministic partitioning with dataset containing Double.NaN (#5996)\n* Limit tree depth for GPU hist to 31 to prevent integer overflow (#6045)\n* [jvm-packages] Set `maxBins` to 256 to align with the default value in the C++ code (#6066)\n* [R] Fix CRAN check (#6077)\n* Add back support for `scipy.sparse.coo_matrix` (#6162)\n* Handle duplicated values in sketching. (#6178)\n* Catch all standard exceptions in C API. (#6220)\n* Fix linear GPU input (#6255)\n* Fix inplace prediction interval. (#6259)\n* [R] allow `xgb.plot.importance()` calls to fill a grid (#6294)\n* Lazy import dask libraries. (#6309)\n* Deterministic data partitioning for external memory (#6317)\n* Avoid resetting seed for every configuration. (#6349)\n* Fix label errors in graph visualization (#6369)\n* [jvm-packages] fix potential unit test suites aborted issue due to race condition (#6373)\n* [R] Fix warnings from `R check --as-cran` (#6374)\n* [R] Fix a crash that occurs with noLD R (#6378)\n* [R] Do not convert continuous labels to factors (#6380)\n* [R] remove uses of `exists()` (#6387)\n* Propagate parameters to the underlying `Booster` handle from `XGBClassifier.set_param` / `XGBRegressor.set_param`. (#6416)\n* [R] Fix R package installation via CMake (#6423)\n* Enforce row-major order in cuPy array (#6459)\n* Fix filtering callable objects in the parameters passed to the scikit-learn API. (#6466)\n\n### Maintenance: Testing, continuous integration, build system\n* [CI] Improve JVM test in GitHub Actions (#5930)\n* Refactor plotting test so that it can run independently (#6040)\n* [CI] Cancel builds on subsequent pushes (#6011)\n* Fix Dask Pytest fixture (#6024)\n* [CI] Migrate linters to GitHub Actions (#6035)\n* [CI] Remove win2016 JVM test from GitHub Actions (#6042)\n* Fix CMake build with `BUILD_STATIC_LIB` option (#6090)\n* Don't link imported target in CMake (#6093)\n* Work around a compiler bug in MacOS AppleClang 11 (#6103)\n* [CI] Fix CTest by running it in a correct directory (#6104)\n* [R] Check warnings explicitly for model compatibility tests (#6114)\n* [jvm-packages] add xgboost4j-gpu/xgboost4j-spark-gpu module to facilitate release (#6136)\n* [CI] Time GPU tests. (#6141)\n* [R] remove warning in configure.ac (#6152)\n* [CI] Upgrade cuDF and RMM to 0.16 nightlies; upgrade to Ubuntu 18.04 (#6157)\n* [CI] Test C API demo (#6159)\n* Option for generating device debug info. (#6168)\n* Update `.gitignore` (#6175, #6193, #6346)\n* Hide C++ symbols from dmlc-core (#6188)\n* [CI] Added arm64 job in Travis-CI (#6200)\n* [CI] Fix Docker build for CUDA 11 (#6202)\n* [CI] Move non-OpenMP gtest to GitHub Actions (#6210)\n* [jvm-packages] Fix up build for xgboost4j-gpu, xgboost4j-spark-gpu (#6216)\n* Add more tests for categorical data support (#6219)\n* [dask] Test for data initializaton. (#6226)\n* Bump junit from 4.11 to 4.13.1 in /jvm-packages/xgboost4j (#6230)\n* Bump junit from 4.11 to 4.13.1 in /jvm-packages/xgboost4j-gpu (#6233)\n* [CI] Reduce testing load with RMM (#6249)\n* [CI] Build a Python wheel for aarch64 platform (#6253)\n* [CI] Time the CPU tests on Jenkins. (#6257)\n* [CI] Skip Dask tests on ARM. (#6267)\n* Fix a typo in `is_arm()` in testing.py (#6271)\n* [CI] replace `egrep` with `grep -E` (#6287)\n* Support unity build. (#6295)\n* [CI] Mark flaky tests as XFAIL (#6299)\n* [CI] Use separate Docker cache for each CUDA version (#6305)\n* Added `USE_NCCL_LIB_PATH` option to enable user to set `NCCL_LIBRARY` during build (#6310)\n* Fix flaky data initialization test. (#6318)\n* Add a badge for GitHub Actions (#6321)\n* Optional `find_package` for sanitizers. (#6329)\n* Use pytest conventions consistently in Python tests (#6337)\n* Fix missing space in warning message (#6340)\n* Update `custom_metric_obj.rst` (#6367)\n* [CI] Run R check with `--as-cran` flag on GitHub Actions (#6371)\n* [CI] Remove R check from Jenkins (#6372)\n* Mark GPU external memory test as XFAIL. (#6381)\n* [CI] Add noLD R test (#6382)\n* Fix MPI build. (#6403)\n* [CI] Upgrade to MacOS Mojave image (#6406)\n* Fix flaky sparse page dmatrix test. (#6417)\n* [CI] Upgrade cuDF and RMM to 0.17 nightlies (#6434)\n* [CI] Fix CentOS 6 Docker images (#6467)\n* [CI] Vendor libgomp in the manylinux Python wheel (#6461)\n* [CI] Hot fix for libgomp vendoring (#6482)\n\n### Maintenance: Clean up and merge the Rabit submodule (#6023, #6095, #6096, #6105, #6110, #6262, #6275, #6290)\n* The Rabit submodule is now maintained as part of the XGBoost codebase.\n* Tests for Rabit are now part of the test suites of XGBoost.\n* Rabit can now be built on the Windows platform.\n* We made various code re-formatting for the C++ code with clang-tidy.\n* Public headers of XGBoost no longer depend on Rabit headers.\n* Unused CMake targets for Rabit were removed.\n* Single-point model recovery has been dropped and removed from Rabit, simplifying the Rabit code greatly. The single-point model recovery feature has not been adequately maintained over the years.\n* We removed the parts of Rabit that were not useful for XGBoost.\n\n### Maintenance: Refactor code for legibility and maintainability\n* Unify CPU hist sketching (#5880)\n* [R] fix uses of 1:length(x) and other small things (#5992)\n* Unify evaluation functions. (#6037)\n* Make binary bin search reusable. (#6058)\n* Unify set index data. (#6062)\n* [R] Remove `stringi` dependency (#6109)\n* Merge extract cuts into QuantileContainer. (#6125)\n* Reduce C++ compiler warnings (#6197, #6198, #6213, #6286, #6325)\n* Cleanup Python code. (#6223)\n* Small cleanup to evaluator. (#6400)\n\n### Usability Improvements, Documentation\n* [jvm-packages] add example to handle missing value other than 0 (#5677)\n* Add DMatrix usage examples to the C API demo (#5854)\n* List `DaskDeviceQuantileDMatrix` in the doc. (#5975)\n* Update Python custom objective demo. (#5981)\n* Update the JSON model schema to document more objective functions. (#5982)\n* [Python] Fix warning when `missing` field is not used. (#5969)\n* Fix typo in tracker logging (#5994)\n* Move a warning about empty dataset, so that it's shown for all objectives and metrics (#5998)\n* Fix the instructions for installing the nightly build. (#6004)\n* [Doc] Add dtreeviz as a showcase example of integration with 3rd-party software (#6013)\n* [jvm-packages] [doc] Update install doc for JVM packages (#6051)\n* Fix typo in `xgboost.callback.early_stop` docstring (#6071)\n* Add cache suffix to the files used in the external memory demo. (#6088)\n* [Doc] Document the parameter `kill_spark_context_on_worker_failure` (#6097)\n* Fix link to the demo for custom objectives (#6100)\n* Update Dask doc. (#6108)\n* Validate weights are positive values. (#6115)\n* Document the updated CMake version requirement. (#6123)\n* Add demo for `DaskDeviceQuantileDMatrix`. (#6156)\n* Cosmetic fixes in `faq.rst` (#6161)\n* Fix error message. (#6176)\n* [Doc] Add list of winning solutions in data science competitions using XGBoost (#6177)\n* Fix a comment in demo to use correct reference (#6190)\n* Update the list of winning solutions using XGBoost (#6192)\n* Consistent style for build status badge (#6203)\n* [Doc] Add info on GPU compiler (#6204)\n* Update the list of winning solutions (#6222, #6254)\n* Add link to XGBoost's Twitter handle (#6244)\n* Fix minor typos in XGBClassifier methods' docstrings (#6247)\n* Add sponsors link to FUNDING.yml (#6252)\n* Group CLI demo into subdirectory. (#6258)\n* Reduce warning messages from `gbtree`. (#6273)\n* Create a tutorial for using the C API in a C/C++ application (#6285)\n* Update plugin instructions for CMake build (#6289)\n* [doc] make Dask distributed example copy-pastable (#6345)\n* [Python] Add option to use `libxgboost.so` from the system path (#6362)\n* Fixed few grammatical mistakes in doc (#6393)\n* Fix broken link in CLI doc (#6396)\n* Improve documentation for the Dask API (#6413)\n* Revise misleading exception information: no such param of `allow_non_zero_missing` (#6418)\n* Fix CLI ranking demo. (#6439)\n* Fix broken links. (#6455)\n\n### Acknowledgement\n**Contributors**: Nan Zhu (@CodingCat), @FelixYBW, Jack Dunn (@JackDunnNZ), Jean Lescut-Muller (@JeanLescut), Boris Feld (@Lothiraldan), Nikhil Choudhary (@Nikhil1O1), Rory Mitchell (@RAMitchell), @ShvetsKS, Anthony D'Amato (@Totoketchup), @Wittty-Panda, neko (@akiyamaneko), Alexander Gugel (@alexanderGugel), @dependabot[bot], DIVYA CHAUHAN (@divya661), Daniel Steinberg (@dstein64), Akira Funahashi (@funasoul), Philip Hyunsu Cho (@hcho3), Tong He (@hetong007), Hristo Iliev (@hiliev), Honza Sterba (@honzasterba), @hzy001, Igor Moura (@igormp), @jameskrach, James Lamb (@jameslamb), Naveed Ahmed Saleem Janvekar (@janvekarnaveed), Kyle Nicholson (@kylejn27), lacrosse91 (@lacrosse91), Christian Lorentzen (@lorentzenchr), Manikya Bardhan (@manikyabard), @nabokovas, John Quitto-Graham (@nvidia-johnq), @odidev, Qi Zhang (@qzhang90), Sergio Gavilán (@sgavil), Tanuja Kirthi Doddapaneni (@tanuja3), Cuong Duong (@tcuongd), Yuan Tang (@terrytangyuan), Jiaming Yuan (@trivialfis), vcarpani (@vcarpani), Vladislav Epifanov (@vepifanov), Vitalie Spinu (@vspinu), Bobby Wang (@wbo4958), Zeno Gantner (@zenogantner), zhang_jf (@zuston)\n\n**Reviewers**: Nan Zhu (@CodingCat), John Zedlewski (@JohnZed), Rory Mitchell (@RAMitchell), @ShvetsKS, Egor Smirnov (@SmirnovEgorRu), Anthony D'Amato (@Totoketchup), @Wittty-Panda, Alexander Gugel (@alexanderGugel), Codecov Comments Bot (@codecov-commenter), Codecov (@codecov-io), DIVYA CHAUHAN (@divya661), Devin Robison (@drobison00), Geoffrey Blake (@geoffreyblake), Mark Harris (@harrism), Philip Hyunsu Cho (@hcho3), Honza Sterba (@honzasterba), Igor Moura (@igormp), @jakirkham, @jameskrach, James Lamb (@jameslamb), Janakarajan Natarajan (@janaknat), Jake Hemstad (@jrhemstad), Keith Kraus (@kkraus14), Kyle Nicholson (@kylejn27), Christian Lorentzen (@lorentzenchr), Michael Mayer (@mayer79), Nikolay Petrov (@napetrov), @odidev, PSEUDOTENSOR / Jonathan McKinney (@pseudotensor), Qi Zhang (@qzhang90), Sergio Gavilán (@sgavil), Scott Lundberg (@slundberg), Cuong Duong (@tcuongd), Yuan Tang (@terrytangyuan), Jiaming Yuan (@trivialfis), vcarpani (@vcarpani), Vladislav Epifanov (@vepifanov), Vincent Nijs (@vnijs), Vitalie Spinu (@vspinu), Bobby Wang (@wbo4958), William Hicks (@wphicks)\n\n## v1.2.0 (2020.08.22)\n\n### XGBoost4J-Spark now supports the GPU algorithm (#5171)\n* Now XGBoost4J-Spark is able to leverage NVIDIA GPU hardware to speed up training.\n* There is on-going work for accelerating the rest of the data pipeline with NVIDIA GPUs (#5950, #5972).\n\n### XGBoost now supports CUDA 11 (#5808)\n* It is now possible to build XGBoost with CUDA 11. Note that we do not yet distribute pre-built binaries built with CUDA 11; all current distributions use CUDA 10.0.\n\n### Better guidance for persisting XGBoost models in an R environment (#5940, #5964)\n* Users are strongly encouraged to use `xgb.save()` and `xgb.save.raw()` instead of `saveRDS()`. This is so that the persisted models can be accessed with future releases of XGBoost.\n* The previous release (1.1.0) had problems loading models that were saved with `saveRDS()`. This release adds a compatibility layer to restore access to the old RDS files. Note that this is meant to be a temporary measure; users are advised to stop using `saveRDS()` and migrate to `xgb.save()` and `xgb.save.raw()`.\n\n### New objectives and metrics\n* The pseudo-Huber loss `reg:pseudohubererror` is added (#5647). The corresponding metric is `mphe`. Right now, the slope is hard-coded to 1.\n* The Accelerated Failure Time objective for survival analysis (`survival:aft`) is now accelerated on GPUs (#5714, #5716). The survival metrics `aft-nloglik` and `interval-regression-accuracy` are also accelerated on GPUs.\n\n### Improved integration with scikit-learn\n* Added `n_features_in_` attribute to the scikit-learn interface to store the number of features used (#5780). This is useful for integrating with some scikit-learn features such as `StackingClassifier`. See [this link](https://scikit-learn-enhancement-proposals.readthedocs.io/en/latest/slep010/proposal.html) for more details.\n* `XGBoostError` now inherits `ValueError`, which conforms scikit-learn's exception requirement (#5696).\n\n### Improved integration with Dask\n* The XGBoost Dask API now exposes an asynchronous interface (#5862). See [the document](https://xgboost.readthedocs.io/en/latest/tutorials/dask.html#working-with-asyncio) for details.\n* Zero-copy ingestion of GPU arrays via `DaskDeviceQuantileDMatrix` (#5623, #5799, #5800, #5803, #5837, #5874, #5901): Previously, the Dask interface had to make 2 data copies: one for concatenating the Dask partition/block into a single block and another for internal representation. To save memory, we introduce `DaskDeviceQuantileDMatrix`. As long as Dask partitions are resident in the GPU memory, `DaskDeviceQuantileDMatrix` is able to ingest them directly without making copies. This matrix type wraps `DeviceQuantileDMatrix`.\n* The prediction function now returns GPU Series type if the input is from Dask-cuDF (#5710). This is to preserve the input data type.\n\n### Robust handling of external data types (#5689, #5893)\n- As we support more and more external data types, the handling logic has proliferated all over the code base and became hard to keep track. It also became unclear how missing values and threads are handled. We refactored the Python package code to collect all data handling logic to a central location, and now we have an explicit list of of all supported data types.\n\n### Improvements in GPU-side data matrix (`DeviceQuantileDMatrix`)\n* The GPU-side data matrix now implements its own quantile sketching logic, so that data don't have to be transported back to the main memory (#5700, #5747, #5760, #5846, #5870, #5898). The GK sketching algorithm is also now better documented.\n - Now we can load extremely sparse dataset like URL, although performance is still sub-optimal.\n* The GPU-side data matrix now exposes an iterative interface (#5783), so that users are able to construct a matrix from a data iterator. See the [Python demo](https://github.com/dmlc/xgboost/blob/release_1.2.0/demo/guide-python/data_iterator.py).\n\n### New language binding: Swift (#5728)\n* Visit https://github.com/kongzii/SwiftXGBoost for more details.\n\n### Robust model serialization with JSON (#5772, #5804, #5831, #5857, #5934)\n* We continue efforts from the 1.0.0 release to adopt JSON as the format to save and load models robustly.\n* JSON model IO is significantly faster and produces smaller model files.\n* Round-trip reproducibility is guaranteed, via the introduction of an efficient float-to-string conversion algorithm known as [the Ryū algorithm](https://dl.acm.org/doi/10.1145/3192366.3192369). The conversion is locale-independent, producing consistent numeric representation regardless of the locale setting of the user's machine.\n* We fixed an issue in loading large JSON files to memory.\n* It is now possible to load a JSON file from a remote source such as S3.\n\n### Performance improvements\n* CPU hist tree method optimization\n - Skip missing lookup in hist row partitioning if data is dense. (#5644)\n - Specialize training procedures for CPU hist tree method on distributed environment. (#5557)\n - Add single point histogram for CPU hist. Previously gradient histogram for CPU hist is hard coded to be 64 bit, now users can specify the parameter `single_precision_histogram` to use 32 bit histogram instead for faster training performance. (#5624, #5811)\n* GPU hist tree method optimization\n - Removed some unnecessary synchronizations and better memory allocation pattern. (#5707)\n - Optimize GPU Hist for wide dataset. Previously for wide dataset the atomic operation is performed on global memory, now it can run on shared memory for faster histogram building. But there's a known small regression on GeForce cards with dense data. (#5795, #5926, #5948, #5631)\n\n### API additions\n* Support passing fmap to importance plot (#5719). Now importance plot can show actual names of features instead of default ones.\n* Support 64bit seed. (#5643)\n* A new C API `XGBoosterGetNumFeature` is added for getting number of features in booster (#5856).\n* Feature names and feature types are now stored in C++ core and saved in binary DMatrix (#5858).\n\n### Breaking: The `predict()` method of `DaskXGBClassifier` now produces class predictions (#5986). Use `predict_proba()` to obtain probability predictions.\n* Previously, `DaskXGBClassifier.predict()` produced probability predictions. This is inconsistent with the behavior of other scikit-learn classifiers, where `predict()` returns class predictions. We make a breaking change in 1.2.0 release so that `DaskXGBClassifier.predict()` now correctly produces class predictions and thus behave like other scikit-learn classifiers. Furthermore, we introduce the `predict_proba()` method for obtaining probability predictions, again to be in line with other scikit-learn classifiers.\n\n### Breaking: Custom evaluation metric now receives raw prediction (#5954)\n* Previously, the custom evaluation metric received a transformed prediction result when used with a classifier. Now the custom metric will receive a raw (untransformed) prediction and will need to transform the prediction itself. See [demo/guide-python/custom\\_softmax.py](https://github.com/dmlc/xgboost/blob/release_1.2.0/demo/guide-python/custom_softmax.py) for an example.\n* This change is to make the custom metric behave consistently with the custom objective, which already receives raw prediction (#5564).\n\n### Breaking: XGBoost4J-Spark now requires Spark 3.0 and Scala 2.12 (#5836, #5890)\n* Starting with version 3.0, Spark can manage GPU resources and allocate them among executors.\n* Spark 3.0 dropped support for Scala 2.11 and now only supports Scala 2.12. Thus, XGBoost4J-Spark also only supports Scala 2.12.\n\n### Breaking: XGBoost Python package now requires Python 3.6 and later (#5715)\n* Python 3.6 has many useful features such as f-strings.\n\n### Breaking: XGBoost now adopts the C++14 standard (#5664)\n* Make sure to use a sufficiently modern C++ compiler that supports C++14, such as Visual Studio 2017, GCC 5.0+, and Clang 3.4+.\n\n### Bug-fixes\n* Fix a data race in the prediction function (#5853). As a byproduct, the prediction function now uses a thread-local data store and became thread-safe.\n* Restore capability to run prediction when the test input has fewer features than the training data (#5955). This capability is necessary to support predicting with LIBSVM inputs. The previous release (1.1) had broken this capability, so we restore it in this version with better tests.\n* Fix OpenMP build with CMake for R package, to support CMake 3.13 (#5895).\n* Fix Windows 2016 build (#5902, #5918).\n* Fix edge cases in scikit-learn interface with Pandas input by disabling feature validation. (#5953)\n* [R] Enable weighted learning to rank (#5945)\n* [R] Fix early stopping with custom objective (#5923)\n* Fix NDK Build (#5886)\n* Add missing explicit template specializations for greater portability (#5921)\n* Handle empty rows in data iterators correctly (#5929). This bug affects file loader and JVM data frames.\n* Fix `IsDense` (#5702)\n* [jvm-packages] Fix wrong method name `setAllowZeroForMissingValue` (#5740)\n* Fix shape inference for Dask predict (#5989)\n\n### Usability Improvements, Documentation\n* [Doc] Document that CUDA 10.0 is required (#5872)\n* Refactored command line interface (CLI). Now CLI is able to handle user errors and output basic document. (#5574)\n* Better error handling in Python: use `raise from` syntax to preserve full stacktrace (#5787).\n* The JSON model dump now has a formal schema (#5660, #5818). The benefit is to prevent `dump_model()` function from breaking. See [this document](https://xgboost.readthedocs.io/en/latest/tutorials/saving_model.html#difference-between-saving-model-and-dumping-model) to understand the difference between saving and dumping models.\n* Add a reference to the GPU external memory paper (#5684)\n* Document more objective parameters in the R package (#5682)\n* Document the existence of pre-built binary wheels for MacOS (#5711)\n* Remove `max.depth` in the R gblinear example. (#5753)\n* Added conda environment file for building docs (#5773)\n* Mention dask blog post in the doc, which introduces using Dask with GPU and some internal workings. (#5789)\n* Fix rendering of Markdown docs (#5821)\n* Document new objectives and metrics available on GPUs (#5909)\n* Better message when no GPU is found. (#5594)\n* Remove the use of `silent` parameter from R demos. (#5675)\n* Don't use masked array in array interface. (#5730)\n* Update affiliation of @terrytangyuan: Ant Financial -> Ant Group (#5827)\n* Move dask tutorial closer other distributed tutorials (#5613)\n* Update XGBoost + Dask overview documentation (#5961)\n* Show `n_estimators` in the docstring of the scikit-learn interface (#6041)\n* Fix a type in a doctring of the scikit-learn interface (#5980)\n\n### Maintenance: testing, continuous integration, build system\n* [CI] Remove CUDA 9.0 from CI (#5674, #5745)\n* Require CUDA 10.0+ in CMake build (#5718)\n* [R] Remove dependency on gendef for Visual Studio builds (fixes #5608) (#5764). This enables building XGBoost with GPU support with R 4.x.\n* [R-package] Reduce duplication in configure.ac (#5693)\n* Bump com.esotericsoftware to 4.0.2 (#5690)\n* Migrate some tests from AppVeyor to GitHub Actions to speed up the tests. (#5911, #5917, #5919, #5922, #5928)\n* Reduce cost of the Jenkins CI server (#5884, #5904, #5892). We now enforce a daily budget via an automated monitor. We also dramatically reduced the workload for the Windows platform, since the cloud VM cost is vastly greater for Windows.\n* [R] Set up automated R linter (#5944)\n* [R] replace uses of T and F with TRUE and FALSE (#5778)\n* Update Docker container 'CPU' (#5956)\n* Simplify CMake build with modern CMake techniques (#5871)\n* Use `hypothesis` package for testing (#5759, #5835, #5849).\n* Define `_CRT_SECURE_NO_WARNINGS` to remove unneeded warnings in MSVC (#5434)\n* Run all Python demos in CI, to ensure that they don't break (#5651)\n* Enhance nvtx support (#5636). Now we can use unified timer between CPU and GPU. Also CMake is able to find nvtx automatically.\n* Speed up python test. (#5752)\n* Add helper for generating batches of data. (#5756)\n* Add c-api-demo to .gitignore (#5855)\n* Add option to enable all compiler warnings in GCC/Clang (#5897)\n* Make Python model compatibility test runnable locally (#5941)\n* Add cupy to Windows CI (#5797)\n* [CI] Fix cuDF install; merge 'gpu' and 'cudf' test suite (#5814)\n* Update rabit submodule (#5680, #5876)\n* Force colored output for Ninja build. (#5959)\n* [CI] Assign larger /dev/shm to NCCL (#5966)\n* Add missing Pytest marks to AsyncIO unit test (#5968)\n* [CI] Use latest cuDF and dask-cudf (#6048)\n* Add CMake flag to log C API invocations, to aid debugging (#5925)\n* Fix a unit test on CLI, to handle RC versions (#6050)\n* [CI] Use mgpu machine to run gpu hist unit tests (#6050)\n* [CI] Build GPU-enabled JAR artifact and deploy to xgboost-maven-repo (#6050)\n\n### Maintenance: Refactor code for legibility and maintainability\n* Remove dead code in DMatrix initialization. (#5635)\n* Catch dmlc error by ref. (#5678)\n* Refactor the `gpu_hist` split evaluation in preparation for batched nodes enumeration. (#5610)\n* Remove column major specialization. (#5755)\n* Remove unused imports in Python (#5776)\n* Avoid including `c_api.h` in header files. (#5782)\n* Remove unweighted GK quantile, which is unused. (#5816)\n* Add Python binding for rabit ops. (#5743)\n* Implement `Empty` method for host device vector. (#5781)\n* Remove print (#5867)\n* Enforce tree order in JSON (#5974)\n\n### Acknowledgement\n**Contributors**: Nan Zhu (@CodingCat), @LionOrCatThatIsTheQuestion, Dmitry Mottl (@Mottl), Rory Mitchell (@RAMitchell), @ShvetsKS, Alex Wozniakowski (@a-wozniakowski), Alexander Gugel (@alexanderGugel), @anttisaukko, @boxdot, Andy Adinets (@canonizer), Ram Rachum (@cool-RR), Elliot Hershberg (@elliothershberg), Jason E. Aten, Ph.D. (@glycerine), Philip Hyunsu Cho (@hcho3), @jameskrach, James Lamb (@jameslamb), James Bourbeau (@jrbourbeau), Peter Jung (@kongzii), Lorenz Walthert (@lorenzwalthert), Oleksandr Kuvshynov (@okuvshynov), Rong Ou (@rongou), Shaochen Shi (@shishaochen), Yuan Tang (@terrytangyuan), Jiaming Yuan (@trivialfis), Bobby Wang (@wbo4958), Zhang Zhang (@zhangzhang10)\n\n**Reviewers**: Nan Zhu (@CodingCat), @LionOrCatThatIsTheQuestion, Hao Yang (@QuantHao), Rory Mitchell (@RAMitchell), @ShvetsKS, Egor Smirnov (@SmirnovEgorRu), Alex Wozniakowski (@a-wozniakowski), Amit Kumar (@aktech), Avinash Barnwal (@avinashbarnwal), @boxdot, Andy Adinets (@canonizer), Chandra Shekhar Reddy (@chandrureddy), Ram Rachum (@cool-RR), Cristiano Goncalves (@cristianogoncalves), Elliot Hershberg (@elliothershberg), Jason E. Aten, Ph.D. (@glycerine), Philip Hyunsu Cho (@hcho3), Tong He (@hetong007), James Lamb (@jameslamb), James Bourbeau (@jrbourbeau), Lee Drake (@leedrake5), DougM (@mengdong), Oleksandr Kuvshynov (@okuvshynov), RongOu (@rongou), Shaochen Shi (@shishaochen), Xu Xiao (@sperlingxx), Yuan Tang (@terrytangyuan), Theodore Vasiloudis (@thvasilo), Jiaming Yuan (@trivialfis), Bobby Wang (@wbo4958), Zhang Zhang (@zhangzhang10)\n\n## v1.1.1 (2020.06.06)\nThis patch release applies the following patches to 1.1.0 release:\n\n* CPU performance improvement in the PyPI wheels (#5720)\n* Fix loading old model (#5724)\n* Install pkg-config file (#5744)\n\n## v1.1.0 (2020.05.17)\n\n### Better performance on multi-core CPUs (#5244, #5334, #5522)\n* Poor performance scaling of the `hist` algorithm for multi-core CPUs has been under investigation (#3810). #5244 concludes the ongoing effort to improve performance scaling on multi-CPUs, in particular Intel CPUs. Roadmap: #5104\n* #5334 makes steps toward reducing memory consumption for the `hist` tree method on CPU.\n* #5522 optimizes random number generation for data sampling.\n\n### Deterministic GPU algorithm for regression and classification (#5361)\n* GPU algorithm for regression and classification tasks is now deterministic.\n* Roadmap: #5023. Currently only single-GPU training is deterministic. Distributed training with multiple GPUs is not yet deterministic.\n\n### Improve external memory support on GPUs (#5093, #5365)\n* Starting from 1.0.0 release, we added support for external memory on GPUs to enable training with larger datasets. Gradient-based sampling (#5093) speeds up the external memory algorithm by intelligently sampling a subset of the training data to copy into the GPU memory. [Learn more about out-of-core GPU gradient boosting.](https://arxiv.org/abs/2005.09148)\n* GPU-side data sketching now works with data from external memory (#5365).\n\n### Parameter validation: detection of unused or incorrect parameters (#5477, #5569, #5508)\n* Mis-spelled training parameter is a common user mistake. In previous versions of XGBoost, mis-spelled parameters were silently ignored. Starting with 1.0.0 release, XGBoost will produce a warning message if there is any unused training parameters. The 1.1.0 release makes parameter validation available to the scikit-learn interface (#5477) and the R binding (#5569).\n\n### Thread-safe, in-place prediction method (#5389, #5512)\n* Previously, the prediction method was not thread-safe (#5339). This release adds a new API function `inplace_predict()` that is thread-safe. It is now possible to serve concurrent requests for prediction using a shared model object.\n* It is now possible to compute prediction in-place for selected data formats (`numpy.ndarray` / `scipy.sparse.csr_matrix` / `cupy.ndarray` / `cudf.DataFrame` / `pd.DataFrame`) without creating a `DMatrix` object.\n\n### Addition of Accelerated Failure Time objective for survival analysis (#4763, #5473, #5486, #5552, #5553)\n* Survival analysis (regression) models the time it takes for an event of interest to occur. The target label is potentially censored, i.e. the label is a range rather than a single number. We added a new objective `survival:aft` to support survival analysis. Also added is the new API to specify the ranged labels. Check out [the tutorial](https://xgboost.readthedocs.io/en/release_1.1.0/tutorials/aft_survival_analysis.html) and the [demos](https://github.com/dmlc/xgboost/tree/release_1.1.0/demo/aft_survival).\n* GPU support is work in progress (#5714).\n\n### Improved installation experience on Mac OSX (#5597, #5602, #5606, #5701)\n* It only takes two commands to install the XGBoost Python package: `brew install libomp` followed by `pip install xgboost`. The installed XGBoost will use all CPU cores. Even better, starting with this release, we distribute pre-compiled binary wheels targeting Mac OSX. Now the install command `pip install xgboost` finishes instantly, as it no longer compiles the C++ source of XGBoost. The last three Mac versions (High Sierra, Mojave, Catalina) are supported.\n* R package: the 1.1.0 release fixes the error `Initializing libomp.dylib, but found libomp.dylib already initialized` (#5701)\n\n### Ranking metrics are now accelerated on GPUs (#5380, #5387, #5398)\n\n### GPU-side data matrix to ingest data directly from other GPU libraries (#5420, #5465)\n* Previously, data on GPU memory had to be copied back to the main memory before it could be used by XGBoost. Starting with 1.1.0 release, XGBoost provides a dedicated interface (`DeviceQuantileDMatrix`) so that it can ingest data from GPU memory directly. The result is that XGBoost interoperates better with GPU-accelerated data science libraries, such as cuDF, cuPy, and PyTorch.\n* Set device in device dmatrix. (#5596)\n\n### Robust model serialization with JSON (#5123, #5217)\n* We continue efforts from the 1.0.0 release to adopt JSON as the format to save and load models robustly. Refer to the release note for 1.0.0 to learn more.\n* It is now possible to store internal configuration of the trained model (`Booster`) object in R as a JSON string (#5123, #5217).\n\n### Improved integration with Dask\n* Pass through `verbose` parameter for dask fit (#5413)\n* Use `DMLC_TASK_ID`. (#5415)\n* Order the prediction result. (#5416)\n* Honor `nthreads` from dask worker. (#5414)\n* Enable grid searching with scikit-learn. (#5417)\n* Check non-equal when setting threads. (#5421)\n* Accept other inputs for prediction. (#5428)\n* Fix missing value for scikit-learn interface. (#5435)\n\n### XGBoost4J-Spark: Check number of columns in the data iterator (#5202, #5303)\n* Before, the native layer in XGBoost did not know the number of columns (features) ahead of time and had to guess the number of columns by counting the feature index when ingesting data. This method has a failure more in distributed setting: if the training data is highly sparse, some features may be completely missing in one or more worker partitions. Thus, one or more workers may deduce an incorrect data shape, leading to crashes or silently wrong models.\n* Enforce correct data shape by passing the number of columns explicitly from the JVM layer into the native layer.\n\n### Major refactoring of the `DMatrix` class\n* Continued from 1.0.0 release.\n* Remove update prediction cache from predictors. (#5312)\n* Predict on Ellpack. (#5327)\n* Partial rewrite EllpackPage (#5352)\n* Use ellpack for prediction only when sparsepage doesn't exist. (#5504)\n* RFC: #4354, Roadmap: #5143\n\n### Breaking: XGBoost Python package now requires Pip 19.0 and higher (#5589)\n* Your Linux machine may have an old version of Pip and may attempt to install a source package, leading to long installation time. This is because we are now using `manylinux2010` tag in the binary wheel release. Ensure you have Pip 19.0 or newer by running `python3 -m pip -V` to check the version. Upgrade Pip with command\n```\npython3 -m pip install --upgrade pip\n```\nUpgrading to latest pip allows us to depend on newer versions of system libraries. [TensorFlow](https://www.tensorflow.org/install/pip) also requires Pip 19.0+.\n\n### Breaking: GPU algorithm now requires CUDA 10.0 and higher (#5649)\n* CUDA 10.0 is necessary to make the GPU algorithm deterministic (#5361).\n\n### Breaking: `silent` parameter is now removed (#5476)\n* Please use `verbosity` instead.\n\n### Breaking: Set `output_margin` to True for custom objectives (#5564)\n* Now both R and Python interface custom objectives get un-transformed (raw) prediction outputs.\n\n### Breaking: `Makefile` is now removed. We use CMake exclusively to build XGBoost (#5513)\n* Exception: the R package uses Autotools, as the CRAN ecosystem did not yet adopt CMake widely.\n\n### Breaking: `distcol` updater is now removed (#5507)\n* The `distcol` updater has been long broken, and currently we lack resources to implement a working implementation from scratch.\n\n### Deprecation notices\n* **Python 3.5**. This release is the last release to support Python 3.5. The following release (1.2.0) will require Python 3.6.\n* **Scala 2.11**. Currently XGBoost4J supports Scala 2.11. However, if a future release of XGBoost adopts Spark 3, it will not support Scala 2.11, as Spark 3 requires Scala 2.12+. We do not yet know which XGBoost release will adopt Spark 3.\n\n### Known limitations\n* (Python package) When early stopping is activated with `early_stopping_rounds` at training time, the prediction method (`xgb.predict()`) behaves in a surprising way. If XGBoost runs for M rounds and chooses iteration N (N < M) as the best iteration, then the prediction method will use M trees by default. To use the best iteration (N trees), users will need to manually take the best iteration field `bst.best_iteration` and pass it as the `ntree_limit` argument to `xgb.predict()`. See #5209 and #4052 for additional context.\n* GPU ranking objective is currently not deterministic (#5561).\n* When training parameter `reg_lambda` is set to zero, some leaf nodes may be assigned a NaN value. (See [discussion](https://discuss.xgboost.ai/t/still-getting-unexplained-nans-new-replication-code/1383/9).) For now, please set `reg_lambda` to a nonzero value.\n\n### Community and Governance\n* The XGBoost Project Management Committee (PMC) is pleased to announce a new committer: Egor Smirnov (@SmirnovEgorRu). He has led a major initiative to improve the performance of XGBoost on multi-core CPUs.\n\n### Bug-fixes\n* Improved compatibility with scikit-learn (#5255, #5505, #5538)\n* Remove f-string, since it's not supported by Python 3.5 (#5330). Note that Python 3.5 support is deprecated and schedule to be dropped in the upcoming release (1.2.0).\n* Fix the pruner so that it doesn't prune the same branch twice (#5335)\n* Enforce only major version in JSON model schema (#5336). Any major revision of the model schema would bump up the major version.\n* Fix a small typo in sklearn.py that broke multiple eval metrics (#5341)\n* Restore loading model from a memory buffer (#5360)\n* Define lazy isinstance for Python compat (#5364)\n* [R] fixed uses of `class()` (#5426)\n* Force compressed buffer to be 4 bytes aligned, to keep cuda-memcheck happy (#5441)\n* Remove warning for calling host function (`std::max`) on a GPU device (#5453)\n* Fix uninitialized value bug in xgboost callback (#5463)\n* Fix model dump in CLI (#5485)\n* Fix out-of-bound array access in `WQSummary::SetPrune()` (#5493)\n* Ensure that configured `dmlc/build_config.h` is picked up by Rabit and XGBoost, to fix build on Alpine (#5514)\n* Fix a misspelled method, made in a git merge (#5509)\n* Fix a bug in binary model serialization (#5532)\n* Fix CLI model IO (#5535)\n* Don't use `uint` for threads (#5542)\n* Fix R interaction constraints to handle more than 100000 features (#5543)\n* [jvm-packages] XGBoost Spark should deal with NaN when parsing evaluation output (#5546)\n* GPU-side data sketching is now aware of query groups in learning-to-rank data (#5551)\n* Fix DMatrix slicing for newly added fields (#5552)\n* Fix configuration status with loading binary model (#5562)\n* Fix build when OpenMP is disabled (#5566)\n* R compatibility patches (#5577, #5600)\n* gpu\\_hist performance fixes (#5558)\n* Don't set seed on CLI interface (#5563)\n* [R] When serializing model, preserve model attributes related to early stopping (#5573)\n* Avoid rabit calls in learner configuration (#5581)\n* Hide C++ symbols in libxgboost.so when building Python wheel (#5590). This fixes apache/incubator-tvm#4953.\n* Fix compilation on Mac OSX High Sierra (10.13) (#5597)\n* Fix build on big endian CPUs (#5617)\n* Resolve crash due to use of `vector::iterator` (#5642)\n* Validation JSON model dump using JSON schema (#5660)\n\n### Performance improvements\n* Wide dataset quantile performance improvement (#5306)\n* Reduce memory usage of GPU-side data sketching (#5407)\n* Reduce span check overhead (#5464)\n* Serialise booster after training to free up GPU memory (#5484)\n* Use the maximum amount of GPU shared memory available to speed up the histogram kernel (#5491)\n* Use non-synchronising scan in Thrust (#5560)\n* Use `cudaDeviceGetAttribute()` instead of `cudaGetDeviceProperties()` for speed (#5570)\n\n### API changes\n* Support importing data from a Pandas SparseArray (#5431)\n* `HostDeviceVector` (vector shared between CPU and GPU memory) now exposes `HostSpan` interface, to enable access on the CPU side with bound check (#5459)\n* Accept other gradient types for `SplitEntry` (#5467)\n\n### Usability Improvements, Documentation\n* Add `JVM_CHECK_CALL` to prevent C++ exceptions from leaking into the JVM layer (#5199)\n* Updated Windows build docs (#5283)\n* Update affiliation of @hcho3 (#5292)\n* Display Sponsor button, link to OpenCollective (#5325)\n* Update docs for GPU external memory (#5332)\n* Add link to GPU documentation (#5437)\n* Small updates to GPU documentation (#5483)\n* Edits on tutorial for XGBoost job on Kubernetes (#5487)\n* Add reference to GPU external memory (#5490)\n* Fix typos (#5346, #5371, #5384, #5399, #5482, #5515)\n* Update Python doc (#5517)\n* Add Neptune and Optuna to list of examples (#5528)\n* Raise error if the number of data weights doesn't match the number of data sets (#5540)\n* Add a note about GPU ranking (#5572)\n* Clarify meaning of `training` parameter in the C API function `XGBoosterPredict()` (#5604)\n* Better error handling for situations where existing trees cannot be modified (#5406, #5418). This feature is enabled when `process_type` is set to `update`.\n\n### Maintenance: testing, continuous integration, build system\n* Add C++ test coverage for data sketching (#5251)\n* Ignore gdb\\_history (#5257)\n* Rewrite setup.py. (#5271, #5280)\n* Use `scikit-learn` in extra dependencies (#5310)\n* Add CMake option to build static library (#5397)\n* [R] changed FindLibR to take advantage of CMake cache (#5427)\n* [R] fixed inconsistency in R -e calls in FindLibR.cmake (#5438)\n* Refactor tests with data generator (#5439)\n* Resolve failing Travis CI (#5445)\n* Update dmlc-core. (#5466)\n* [CI] Use clang-tidy 10 (#5469)\n* De-duplicate code for checking maximum number of nodes (#5497)\n* [CI] Use Ubuntu 18.04 LTS in JVM CI, because 19.04 is EOL (#5537)\n* [jvm-packages] [CI] Create a Maven repository to host SNAPSHOT JARs (#5533)\n* [jvm-packages] [CI] Publish XGBoost4J JARs with Scala 2.11 and 2.12 (#5539)\n* [CI] Use Vault repository to re-gain access to devtoolset-4 (#5589)\n\n### Maintenance: Refactor code for legibility and maintainability\n* Move prediction cache to Learner (#5220, #5302)\n* Remove SimpleCSRSource (#5315)\n* Refactor SparsePageSource, delete cache files after use (#5321)\n* Remove unnecessary DMatrix methods (#5324)\n* Split up `LearnerImpl` (#5350)\n* Move segment sorter to common (#5378)\n* Move thread local entry into Learner (#5396)\n* Split up test helpers header (#5455)\n* Requires setting leaf stat when expanding tree (#5501)\n* Purge device\\_helpers.cuh (#5534)\n* Use thrust functions instead of custom functions (#5544)\n\n### Acknowledgement\n**Contributors**: Nan Zhu (@CodingCat), Rory Mitchell (@RAMitchell), @ShvetsKS, Egor Smirnov (@SmirnovEgorRu), Andrew Kane (@ankane), Avinash Barnwal (@avinashbarnwal), Bart Broere (@bartbroere), Andy Adinets (@canonizer), Chen Qin (@chenqin), Daiki Katsuragawa (@daikikatsuragawa), David Díaz Vico (@daviddiazvico), Darius Kharazi (@dkharazi), Darby Payne (@dpayne), Jason E. Aten, Ph.D. (@glycerine), Philip Hyunsu Cho (@hcho3), James Lamb (@jameslamb), Jan Borchmann (@jborchma), Kamil A. Kaczmarek (@kamil-kaczmarek), Melissa Kohl (@mjkohl32), Nicolas Scozzaro (@nscozzaro), Paul Kaefer (@paulkaefer), Rong Ou (@rongou), Samrat Pandiri (@samratp), Sriram Chandramouli (@sriramch), Yuan Tang (@terrytangyuan), Jiaming Yuan (@trivialfis), Liang-Chi Hsieh (@viirya), Bobby Wang (@wbo4958), Zhang Zhang (@zhangzhang10),\n\n**Reviewers**: Nan Zhu (@CodingCat), @LeZhengThu, Rory Mitchell (@RAMitchell), @ShvetsKS, Egor Smirnov (@SmirnovEgorRu), Steve Bronder (@SteveBronder), Nikita Titov (@StrikerRUS), Andrew Kane (@ankane), Avinash Barnwal (@avinashbarnwal), @brydag, Andy Adinets (@canonizer), Chandra Shekhar Reddy (@chandrureddy), Chen Qin (@chenqin), Codecov (@codecov-io), David Díaz Vico (@daviddiazvico), Darby Payne (@dpayne), Jason E. Aten, Ph.D. (@glycerine), Philip Hyunsu Cho (@hcho3), James Lamb (@jameslamb), @johnny-cat, Mu Li (@mli), Mate Soos (@msoos), @rnyak, Rong Ou (@rongou), Sriram Chandramouli (@sriramch), Toby Dylan Hocking (@tdhock), Yuan Tang (@terrytangyuan), Oleksandr Pryimak (@trams), Jiaming Yuan (@trivialfis), Liang-Chi Hsieh (@viirya), Bobby Wang (@wbo4958),\n\n## v1.0.2 (2020.03.03)\nThis patch release applies the following patches to 1.0.0 release:\n\n* Fix a small typo in sklearn.py that broke multiple eval metrics (#5341)\n* Restore loading model from buffer (#5360)\n* Use type name for data type check (#5364)\n\n## v1.0.1 (2020.02.21)\nThis release is identical to the 1.0.0 release, except that it fixes a small bug that rendered 1.0.0 incompatible with Python 3.5. See #5328.\n\n## v1.0.0 (2020.02.19)\nThis release marks a major milestone for the XGBoost project.\n\n### Apache-style governance, contribution policy, and semantic versioning (#4646, #4659)\n* Starting with 1.0.0 release, the XGBoost Project is adopting Apache-style governance. The full community guideline is [available in the doc website](https://xgboost.readthedocs.io/en/release_1.0.0/contrib/community.html). Note that we now have Project Management Committee (PMC) who would steward the project on the long-term basis. The PMC is also entrusted to run and fund the project's continuous integration (CI) infrastructure (https://xgboost-ci.net).\n* We also adopt the [semantic versioning](https://semver.org/). See [our release versioning policy](https://xgboost.readthedocs.io/en/release_1.0.0/contrib/release.html).\n\n### Better performance scaling for multi-core CPUs (#4502, #4529, #4716, #4851, #5008, #5107, #5138, #5156)\n* Poor performance scaling of the `hist` algorithm for multi-core CPUs has been under investigation (#3810). Previous effort #4529 was replaced with a series of pull requests (#5107, #5138, #5156) aimed at achieving the same performance benefits while keeping the C++ codebase legible. The latest performance benchmark results show [up to 5x speedup on Intel CPUs with many cores](https://github.com/dmlc/xgboost/pull/5156#issuecomment-580024413). Note: #5244, which concludes the effort, will become part of the upcoming release 1.1.0.\n\n### Improved installation experience on Mac OSX (#4672, #5074, #5080, #5146, #5240)\n* It used to be quite complicated to install XGBoost on Mac OSX. XGBoost uses OpenMP to distribute work among multiple CPU cores, and Mac's default C++ compiler (Apple Clang) does not come with OpenMP. Existing work-around (using another C++ compiler) was complex and prone to fail with cryptic diagnosis (#4933, #4949, #4969).\n* Now it only takes two commands to install XGBoost: `brew install libomp` followed by `pip install xgboost`. The installed XGBoost will use all CPU cores.\n* Even better, XGBoost is now available from Homebrew: `brew install xgboost`. See Homebrew/homebrew-core#50467.\n* Previously, if you installed the XGBoost R package using the command `install.packages('xgboost')`, it could only use a single CPU core and you would experience slow training performance. With 1.0.0 release, the R package will use all CPU cores out of box.\n\n### Distributed XGBoost now available on Kubernetes (#4621, #4939)\n* Check out the [tutorial for setting up distributed XGBoost on a Kubernetes cluster](https://xgboost.readthedocs.io/en/release_1.0.0/tutorials/kubernetes.html).\n\n### Ruby binding for XGBoost (#4856)\n\n### New Native Dask interface for multi-GPU and multi-node scaling (#4473, #4507, #4617, #4819, #4907, #4914, #4941, #4942, #4951, #4973, #5048, #5077, #5144, #5270)\n* XGBoost now integrates seamlessly with [Dask](https://dask.org/), a lightweight distributed framework for data processing. Together with the first-class support for cuDF data frames (see below), it is now easier than ever to create end-to-end data pipeline running on one or more NVIDIA GPUs.\n* Multi-GPU training with Dask is now up to 20% faster than the previous release (#4914, #4951).\n\n### First-class support for cuDF data frames and cuPy arrays (#4737, #4745, #4794, #4850, #4891, #4902, #4918, #4927, #4928, #5053, #5189, #5194, #5206, #5219, #5225)\n* [cuDF](https://github.com/rapidsai/cudf) is a data frame library for loading and processing tabular data on NVIDIA GPUs. It provides a Pandas-like API.\n* [cuPy](https://github.com/cupy/cupy) implements a NumPy-compatible multi-dimensional array on NVIDIA GPUs.\n* Now users can keep the data on the GPU memory throughout the end-to-end data pipeline, obviating the need for copying data between the main memory and GPU memory.\n* XGBoost can accept any data structure that exposes `__array_interface__` signature, opening way to support other columar formats that are compatible with Apache Arrow.\n\n### [Feature interaction constraint](https://xgboost.readthedocs.io/en/release_1.0.0/tutorials/feature_interaction_constraint.html) is now available with `approx` and `gpu_hist` algorithms (#4534, #4587, #4596, #5034).\n\n### Learning to rank is now GPU accelerated (#4873, #5004, #5129)\n* Supported ranking objectives: NDGC, Map, Pairwise.\n* [Up to 2x improved training performance on GPUs](https://devblogs.nvidia.com/learning-to-rank-with-xgboost-and-gpu/).\n\n### Enable `gamma` parameter for GPU training (#4874, #4953)\n* The `gamma` parameter specifies the minimum loss reduction required to add a new split in a tree. A larger value for `gamma` has the effect of pre-pruning the tree, by making harder to add splits.\n\n### External memory for GPU training (#4486, #4526, #4747, #4833, #4879, #5014)\n* It is now possible to use NVIDIA GPUs even when the size of training data exceeds the available GPU memory. Note that the external memory support for GPU is still experimental. #5093 will further improve performance and will become part of the upcoming release 1.1.0.\n* RFC for enabling external memory with GPU algorithms: #4357\n\n### Improve Scikit-Learn interface (#4558, #4842, #4929, #5049, #5151, #5130, #5227)\n* Many users of XGBoost enjoy the convenience and breadth of Scikit-Learn ecosystem. In this release, we revise the Scikit-Learn API of XGBoost (`XGBRegressor`, `XGBClassifier`, and `XGBRanker`) to achieve feature parity with the traditional XGBoost interface (`xgboost.train()`).\n* Insert check to validate data shapes.\n* Produce an error message if `eval_set` is not a tuple. An error message is better than silently crashing.\n* Allow using `numpy.RandomState` object.\n* Add `n_jobs` as an alias of `nthread`.\n* Roadmap: #5152\n\n### XGBoost4J-Spark: Redesigning checkpointing mechanism\n* RFC is available at #4786\n* Clean up checkpoint file after a successful training job (#4754): The current implementation in XGBoost4J-Spark does not clean up the checkpoint file after a successful training job. If the user runs another job with the same checkpointing directory, she will get a wrong model because the second job will re-use the checkpoint file left over from the first job. To prevent this scenario, we propose to always clean up the checkpoint file after every successful training job.\n* Avoid Multiple Jobs for Checkpointing (#5082): The current method for checkpoint is to collect the booster produced at the last iteration of each checkpoint internal to Driver and persist it in HDFS. The major issue with this approach is that it needs to re-perform the data preparation for training if the user did not choose to cache the training dataset. To avoid re-performing data prep, we build external-memory checkpointing in the XGBoost4J layer as well.\n* Enable deterministic repartitioning when checkpoint is enabled (#4807): Distributed algorithm for gradient boosting assumes a fixed partition of the training data between multiple iterations. In previous versions, there was no guarantee that data partition would stay the same, especially when a worker goes down and some data had to recovered from previous checkpoint. In this release, we make data partition deterministic by using the data hash value of each data row in computing the partition.\n\n### XGBoost4J-Spark: handle errors thrown by the native code (#4560)\n* All core logic of XGBoost is written in C++, so XGBoost4J-Spark internally uses the C++ code via Java Native Interface (JNI). #4560 adds a proper error handling for any errors or exceptions arising from the C++ code, so that the XGBoost Spark application can be torn down in an orderly fashion.\n\n### XGBoost4J-Spark: Refine method to count the number of alive cores (#4858)\n* The `SparkParallelismTracker` class ensures that sufficient number of executor cores are alive. To that end, it is important to query the number of alive cores reliably.\n\n### XGBoost4J: Add `BigDenseMatrix` to store more than `Integer.MAX_VALUE` elements (#4383)\n\n### Robust model serialization with JSON (#4632, #4708, #4739, #4868, #4936, #4945, #4974, #5086, #5087, #5089, #5091, #5094, #5110, #5111, #5112, #5120, #5137, #5218, #5222, #5236, #5245, #5248, #5281)\n* In this release, we introduce an experimental support of using [JSON](https://www.json.org/json-en.html) for serializing (saving/loading) XGBoost models and related hyperparameters for training. We would like to eventually replace the old binary format with JSON, since it is an open format and parsers are available in many programming languages and platforms. See [the documentation for model I/O using JSON](https://xgboost.readthedocs.io/en/release_1.0.0/tutorials/saving_model.html). #3980 explains why JSON was chosen over other alternatives.\n* To maximize interoperability and compatibility of the serialized models, we now split serialization into two parts (#4855):\n 1. Model, e.g. decision trees and strictly related metadata like `num_features`.\n 2. Internal configuration, consisting of training parameters and other configurable parameters. For example, `max_delta_step`, `tree_method`, `objective`, `predictor`, `gpu_id`.\n\n Previously, users often ran into issues where the model file produced by one machine could not load or run on another machine. For example, models trained using a machine with an NVIDIA GPU could not run on another machine without a GPU (#5291, #5234). The reason is that the old binary format saved some internal configuration that were not universally applicable to all machines, e.g. `predictor='gpu_predictor'`.\n\n Now, model saving function (`Booster.save_model()` in Python) will save only the model, without internal configuration. This will guarantee that your model file would be used anywhere. Internal configuration will be serialized in limited circumstances such as:\n * Multiple nodes in a distributed system exchange model details over the network.\n * Model checkpointing, to recover from possible crashes.\n\n This work proved to be useful for parameter validation as well (see below).\n* Starting with 1.0.0 release, we will use semantic versioning to indicate whether the model produced by one version of XGBoost would be compatible with another version of XGBoost. Any change in the major version indicates a breaking change in the serialization format.\n* We now provide a robust method to save and load scikit-learn related attributes (#5245). Previously, we used Python pickle to save Python attributes related to `XGBClassifier`, `XGBRegressor`, and `XGBRanker` objects. The attributes are necessary to properly interact with scikit-learn. See #4639 for more details. The use of pickling hampered interoperability, as a pickle from one machine may not necessarily work on another machine. Starting with this release, we use an alternative method to serialize the scikit-learn related attributes. The use of Python pickle is now discouraged (#5236, #5281).\n\n### Parameter validation: detection of unused or incorrect parameters (#4553, #4577, #4738, #4801, #4961, #5101, #5157, #5167, #5256)\n* Mis-spelled training parameter is a common user mistake. In previous versions of XGBoost, mis-spelled parameters were silently ignored. Starting with 1.0.0 release, XGBoost will produce a warning message if there is any unused training parameters. Currently, parameter validation is available to R users and Python XGBoost API users. We are working to extend its support to scikit-learn users.\n* Configuration steps now have well-defined semantics (#4542, #4738), so we know exactly where and how the internal configurable parameters are changed.\n* The user can now use `save_config()` function to inspect all (used) training parameters. This is helpful for debugging model performance.\n\n### Allow individual workers to recover from faults (#4808, #4966)\n* Status quo: if a worker fails, all workers are shut down and restarted, and learning resumes from the last checkpoint. This involves requesting resources from the scheduler (e.g. Spark) and shuffling all the data again from scratch. Both of these operations can be quite costly and block training for extended periods of time, especially if the training data is big and the number of worker nodes is in the hundreds.\n* The proposed solution is to recover the single node that failed, instead of shutting down all workers. The rest of the clusters wait until the single failed worker is bootstrapped and catches up with the rest.\n* See roadmap at #4753. Note that this is work in progress. In particular, the feature is not yet available from XGBoost4J-Spark.\n\n### Accurate prediction for DART models\n* Use DART tree weights when computing SHAPs (#5050)\n* Don't drop trees during DART prediction by default (#5115)\n* Fix DART prediction in R (#5204)\n\n### Make external memory more robust\n* Fix issues with training with external memory on cpu (#4487)\n* Fix crash with approx tree method on cpu (#4510)\n* Fix external memory race in `exact` (#4980). Note: `dmlc::ThreadedIter` is not actually thread-safe. We would like to re-design it in the long term.\n\n### Major refactoring of the `DMatrix` class (#4686, #4744, #4748, #5044, #5092, #5108, #5188, #5198)\n* Goal 1: improve performance and reduce memory consumption. Right now, if the user trains a model with a NumPy array as training data, the array gets copies 2-3 times before training begins. We'd like to reduce duplication of the data matrix.\n* Goal 2: Expose a common interface to external data, unify the way DMatrix objects are constructed and simplify the process of adding new external data sources. This work is essential for ingesting cuPy arrays.\n* Goal 3: Handle missing values consistently.\n* RFC: #4354, Roadmap: #5143\n* This work is also relevant to external memory support on GPUs.\n\n### Breaking: XGBoost Python package now requires Python 3.5 or newer (#5021, #5274)\n* Python 3.4 has reached its end-of-life on March 16, 2019, so we now require Python 3.5 or newer.\n\n### Breaking: GPU algorithm now requires CUDA 9.0 and higher (#4527, #4580)\n\n### Breaking: `n_gpus` parameter removed; multi-GPU training now requires a distributed framework (#4579, #4749, #4773, #4810, #4867, #4908)\n* #4531 proposed removing support for single-process multi-GPU training. Contributors would focus on multi-GPU support through distributed frameworks such as Dask and Spark, where the framework would be expected to assign a worker process for each GPU independently. By delegating GPU management and data movement to the distributed framework, we can greatly simplify the core XGBoost codebase, make multi-GPU training more robust, and reduce burden for future development.\n\n### Breaking: Some deprecated features have been removed\n* ``gpu_exact`` training method (#4527, #4742, #4777). Use ``gpu_hist`` instead.\n* ``learning_rates`` parameter in Python (#5155). Use the callback API instead.\n* ``num_roots`` (#5059, #5165), since the current training code always uses a single root node.\n* GPU-specific objectives (#4690), such as `gpu:reg:linear`. Use objectives without `gpu:` prefix; GPU will be used automatically if your machine has one.\n\n### Breaking: the C API function `XGBoosterPredict()` now asks for an extra parameter `training`.\n\n### Breaking: We now use CMake exclusively to build XGBoost. `Makefile` is being sunset.\n* Exception: the R package uses Autotools, as the CRAN ecosystem did not yet adopt CMake widely.\n\n### Performance improvements\n* Smarter choice of histogram construction for distributed `gpu_hist` (#4519)\n* Optimizations for quantization on device (#4572)\n* Introduce caching memory allocator to avoid latency associated with GPU memory allocation (#4554, #4615)\n* Optimize the initialization stage of the CPU `hist` algorithm for sparse datasets (#4625)\n* Prevent unnecessary data copies from GPU memory to the host (#4795)\n* Improve operation efficiency for single prediction (#5016)\n* Group builder modified for incremental building, to speed up building large `DMatrix` (#5098)\n\n### Bug-fixes\n* Eliminate `FutureWarning: Series.base is deprecated` (#4337)\n* Ensure pandas DataFrame column names are treated as strings in type error message (#4481)\n* [jvm-packages] Add back `reg:linear` for scala, as it is only deprecated and not meant to be removed yet (#4490)\n* Fix library loading for Cygwin users (#4499)\n* Fix prediction from loaded pickle (#4516)\n* Enforce exclusion between `pred_interactions=True` and `pred_interactions=True` (#4522)\n* Do not return dangling reference to local `std::string` (#4543)\n* Set the appropriate device before freeing device memory (#4566)\n* Mark `SparsePageDmatrix` destructor default. (#4568)\n* Choose the appropriate tree method only when the tree method is 'auto' (#4571)\n* Fix `benchmark_tree.py` (#4593)\n* [jvm-packages] Fix silly bug in feature scoring (#4604)\n* Fix GPU predictor when the test data matrix has different number of features than the training data matrix used to train the model (#4613)\n* Fix external memory for get column batches. (#4622)\n* [R] Use built-in label when xgb.DMatrix is given to xgb.cv() (#4631)\n* Fix early stopping in the Python package (#4638)\n* Fix AUC error in distributed mode caused by imbalanced dataset (#4645, #4798)\n* [jvm-packages] Expose `setMissing` method in `XGBoostClassificationModel` / `XGBoostRegressionModel` (#4643)\n* Remove initializing stringstream reference. (#4788)\n* [R] `xgb.get.handle` now checks all class listed of `object` (#4800)\n* Do not use `gpu_predictor` unless data comes from GPU (#4836)\n* Fix data loading (#4862)\n* Workaround `isnan` across different environments. (#4883)\n* [jvm-packages] Handle Long-type parameter (#4885)\n* Don't `set_params` at the end of `set_state` (#4947). Ensure that the model does not change after pickling and unpickling multiple times.\n* C++ exceptions should not crash OpenMP loops (#4960)\n* Fix `usegpu` flag in DART. (#4984)\n* Run training with empty `DMatrix` (#4990, #5159)\n* Ensure that no two processes can use the same GPU (#4990)\n* Fix repeated split and 0 cover nodes (#5010)\n* Reset histogram hit counter between multiple data batches (#5035)\n* Fix `feature_name` crated from int64index dataframe. (#5081)\n* Don't use 0 for \"fresh leaf\" (#5084)\n* Throw error when user attempts to use multi-GPU training and XGBoost has not been compiled with NCCL (#5170)\n* Fix metric name loading (#5122)\n* Quick fix for memory leak in CPU `hist` algorithm (#5153)\n* Fix wrapping GPU ID and prevent data copying (#5160)\n* Fix signature of Span constructor (#5166)\n* Lazy initialization of device vector, so that XGBoost compiled with CUDA can run on a machine without any GPU (#5173)\n* Model loading should not change system locale (#5314)\n* Distributed training jobs would sometimes hang; revert Rabit to fix this regression (dmlc/rabit#132, #5237)\n\n### API changes\n* Add support for cross-validation using query ID (#4474)\n* Enable feature importance property for DART model (#4525)\n* Add `rmsle` metric and `reg:squaredlogerror` objective (#4541)\n* All objective and evaluation metrics are now exposed to JVM packages (#4560)\n* `dump_model()` and `get_dump()` now support exporting in GraphViz language (#4602)\n* Support metrics `ndcg-` and `map-` (#4635)\n* [jvm-packages] Allow chaining prediction (transform) in XGBoost4J-Spark (#4667)\n* [jvm-packages] Add option to bypass missing value check in the Spark layer (#4805). Only use this option if you know what you are doing.\n* [jvm-packages] Add public group getter (#4838)\n* `XGDMatrixSetGroup` C API is now deprecated (#4864). Use `XGDMatrixSetUIntInfo` instead.\n* [R] Added new `train_folds` parameter to `xgb.cv()` (#5114)\n* Ingest meta information from Pandas DataFrame, such as data weights (#5216)\n\n### Maintenance: Refactor code for legibility and maintainability\n* De-duplicate GPU parameters (#4454)\n* Simplify INI-style config reader using C++11 STL (#4478, #4521)\n* Refactor histogram building code for `gpu_hist` (#4528)\n* Overload device memory allocator, to enable instrumentation for compiling memory usage statistics (#4532)\n* Refactor out row partitioning logic from `gpu_hist` (#4554)\n* Remove an unused variable (#4588)\n* Implement tree model dump with code generator, to de-duplicate code for generating dumps in 3 different formats (#4602)\n* Remove `RowSet` class which is no longer being used (#4697)\n* Remove some unused functions as reported by cppcheck (#4743)\n* Mimic CUDA assert output in Span check (#4762)\n* [jvm-packages] Refactor `XGBoost.scala` to put all params processing in one place (#4815)\n* Add some comments for GPU row partitioner (#4832)\n* Span: use `size_t' for index_type, add `front' and `back'. (#4935)\n* Remove dead code in `exact` algorithm (#5034, #5105)\n* Unify integer types used for row and column indices (#5034)\n* Extract feature interaction constraint from `SplitEvaluator` class. (#5034)\n* [Breaking] De-duplicate paramters and docstrings in the constructors of Scikit-Learn models (#5130)\n* Remove benchmark code from GPU tests (#5141)\n* Clean up Python 2 compatibility code. (#5161)\n* Extensible binary serialization format for `DMatrix::MetaInfo` (#5187). This will be useful for implementing censored labels for survival analysis applications.\n* Cleanup clang-tidy warnings. (#5247)\n\n### Maintenance: testing, continuous integration, build system\n* Use `yaml.safe_load` instead of `yaml.load`. (#4537)\n* Ensure GCC is at least 5.x (#4538)\n* Remove all mention of `reg:linear` from tests (#4544)\n* [jvm-packages] Upgrade to Scala 2.12 (#4574)\n* [jvm-packages] Update kryo dependency to 2.22 (#4575)\n* [CI] Specify account ID when logging into ECR Docker registry (#4584)\n* Use Sphinx 2.1+ to compile documentation (#4609)\n* Make Pandas optional for running Python unit tests (#4620)\n* Fix spark tests on machines with many cores (#4634)\n* [jvm-packages] Update local dev build process (#4640)\n* Add optional dependencies to setup.py (#4655)\n* [jvm-packages] Fix maven warnings (#4664)\n* Remove extraneous files from the R package, to comply with CRAN policy (#4699)\n* Remove VC-2013 support, since it is not C++11 compliant (#4701)\n* [CI] Fix broken installation of Pandas (#4704, #4722)\n* [jvm-packages] Clean up temporary files afer running tests (#4706)\n* Specify version macro in CMake. (#4730)\n* Include dmlc-tracker into XGBoost Python package (#4731)\n* [CI] Use long key ID for Ubuntu repository fingerprints. (#4783)\n* Remove plugin, CUDA related code in automake & autoconf files (#4789)\n* Skip related tests when scikit-learn is not installed. (#4791)\n* Ignore vscode and clion files (#4866)\n* Use bundled Google Test by default (#4900)\n* [CI] Raise timeout threshold in Jenkins (#4938)\n* Copy CMake parameter from dmlc-core. (#4948)\n* Set correct file permission. (#4964)\n* [CI] Update lint configuration to support latest pylint convention (#4971)\n* [CI] Upload nightly builds to S3 (#4976, #4979)\n* Add asan.so.5 to cmake script. (#4999)\n* [CI] Fix Travis tests. (#5062)\n* [CI] Locate vcomp140.dll from System32 directory (#5078)\n* Implement training observer to dump internal states of objects (#5088). This will be useful for debugging.\n* Fix visual studio output library directories (#5119)\n* [jvm-packages] Comply with scala style convention + fix broken unit test (#5134)\n* [CI] Repair download URL for Maven 3.6.1 (#5139)\n* Don't use modernize-use-trailing-return-type in clang-tidy. (#5169)\n* Explicitly use UTF-8 codepage when using MSVC (#5197)\n* Add CMake option to run Undefined Behavior Sanitizer (UBSan) (#5211)\n* Make some GPU tests deterministic (#5229)\n* [R] Robust endian detection in CRAN xgboost build (#5232)\n* Support FreeBSD (#5233)\n* Make `pip install xgboost*.tar.gz` work by fixing build-python.sh (#5241)\n* Fix compilation error due to 64-bit integer narrowing to `size_t` (#5250)\n* Remove use of `std::cout` from R package, to comply with CRAN policy (#5261)\n* Update DMLC-Core submodule (#4674, #4688, #4726, #4924)\n* Update Rabit submodule (#4560, #4667, #4718, #4808, #4966, #5237)\n\n### Usability Improvements, Documentation\n* Add Random Forest API to Python API doc (#4500)\n* Fix Python demo and doc. (#4545)\n* Remove doc about not supporting CUDA 10.1 (#4578)\n* Address some sphinx warnings and errors, add doc for building doc. (#4589)\n* Add instruction to run formatting checks locally (#4591)\n* Fix docstring for `XGBModel.predict()` (#4592)\n* Doc and demo for customized metric and objective (#4598, #4608)\n* Add to documentation how to run tests locally (#4610)\n* Empty evaluation list in early stopping should produce meaningful error message (#4633)\n* Fixed year to 2019 in conf.py, helpers.h and LICENSE (#4661)\n* Minor updates to links and grammar (#4673)\n* Remove `silent` in doc (#4689)\n* Remove old Python trouble shooting doc (#4729)\n* Add `os.PathLike` support for file paths to DMatrix and Booster Python classes (#4757)\n* Update XGBoost4J-Spark doc (#4804)\n* Regular formatting for evaluation metrics (#4803)\n* [jvm-packages] Refine documentation for handling missing values in XGBoost4J-Spark (#4805)\n* Monitor for distributed environment (#4829). This is useful for identifying performance bottleneck.\n* Add check for length of weights and produce a good error message (#4872)\n* Fix DMatrix doc (#4884)\n* Export C++ headers in CMake installation (#4897)\n* Update license year in README.md to 2019 (#4940)\n* Fix incorrectly displayed Note in the doc (#4943)\n* Follow PEP 257 Docstring Conventions (#4959)\n* Document minimum version required for Google Test (#5001)\n* Add better error message for invalid feature names (#5024)\n* Some guidelines on device memory usage (#5038)\n* [doc] Some notes for external memory. (#5065)\n* Update document for `tree_method` (#5106)\n* Update demo for ranking. (#5154)\n* Add new lines for Spark XGBoost missing values section (#5180)\n* Fix simple typo: utilty -> utility (#5182)\n* Update R doc by roxygen2 (#5201)\n* [R] Direct user to use `set.seed()` instead of setting `seed` parameter (#5125)\n* Add Optuna badge to `README.md` (#5208)\n* Fix compilation error in `c-api-demo.c` (#5215)\n\n### Acknowledgement\n**Contributors**: Nan Zhu (@CodingCat), Crissman Loomis (@Crissman), Cyprien Ricque (@Cyprien-Ricque), Evan Kepner (@EvanKepner), K.O. (@Hi-king), KaiJin Ji (@KerryJi), Peter Badida (@KeyWeeUsr), Kodi Arfer (@Kodiologist), Rory Mitchell (@RAMitchell), Egor Smirnov (@SmirnovEgorRu), Jacob Kim (@TheJacobKim), Vibhu Jawa (@VibhuJawa), Marcos (@astrowonk), Andy Adinets (@canonizer), Chen Qin (@chenqin), Christopher Cowden (@cowden), @cpfarrell, @david-cortes, Liangcai Li (@firestarman), @fuhaoda, Philip Hyunsu Cho (@hcho3), @here-nagini, Tong He (@hetong007), Michal Kurka (@michalkurka), Honza Sterba (@honzasterba), @iblumin, @koertkuipers, mattn (@mattn), Mingjie Tang (@merlintang), OrdoAbChao (@mglowacki100), Matthew Jones (@mt-jones), mitama (@nigimitama), Nathan Moore (@nmoorenz), Daniel Stahl (@phillyfan1138), Michaël Benesty (@pommedeterresautee), Rong Ou (@rongou), Sebastian (@sfahnens), Xu Xiao (@sperlingxx), @sriramch, Sean Owen (@srowen), Stephanie Yang (@stpyang), Yuan Tang (@terrytangyuan), Mathew Wicks (@thesuperzapper), Tim Gates (@timgates42), TinkleG (@tinkle1129), Oleksandr Pryimak (@trams), Jiaming Yuan (@trivialfis), Matvey Turkov (@turk0v), Bobby Wang (@wbo4958), yage (@yage99), @yellowdolphin\n\n**Reviewers**: Nan Zhu (@CodingCat), Crissman Loomis (@Crissman), Cyprien Ricque (@Cyprien-Ricque), Evan Kepner (@EvanKepner), John Zedlewski (@JohnZed), KOLANICH (@KOLANICH), KaiJin Ji (@KerryJi), Kodi Arfer (@Kodiologist), Rory Mitchell (@RAMitchell), Egor Smirnov (@SmirnovEgorRu), Nikita Titov (@StrikerRUS), Jacob Kim (@TheJacobKim), Vibhu Jawa (@VibhuJawa), Andrew Kane (@ankane), Arno Candel (@arnocandel), Marcos (@astrowonk), Bryan Woods (@bryan-woods), Andy Adinets (@canonizer), Chen Qin (@chenqin), Thomas Franke (@coding-komek), Peter (@codingforfun), @cpfarrell, Joshua Patterson (@datametrician), @fuhaoda, Philip Hyunsu Cho (@hcho3), Tong He (@hetong007), Honza Sterba (@honzasterba), @iblumin, @jakirkham, Vadim Khotilovich (@khotilov), Keith Kraus (@kkraus14), @koertkuipers, @melonki, Mingjie Tang (@merlintang), OrdoAbChao (@mglowacki100), Daniel Mahler (@mhlr), Matthew Rocklin (@mrocklin), Matthew Jones (@mt-jones), Michaël Benesty (@pommedeterresautee), PSEUDOTENSOR / Jonathan McKinney (@pseudotensor), Rong Ou (@rongou), Vladimir (@sh1ng), Scott Lundberg (@slundberg), Xu Xiao (@sperlingxx), @sriramch, Pasha Stetsenko (@st-pasha), Stephanie Yang (@stpyang), Yuan Tang (@terrytangyuan), Mathew Wicks (@thesuperzapper), Theodore Vasiloudis (@thvasilo), TinkleG (@tinkle1129), Oleksandr Pryimak (@trams), Jiaming Yuan (@trivialfis), Bobby Wang (@wbo4958), yage (@yage99), @yellowdolphin, Yin Lou (@yinlou)\n\n## v0.90 (2019.05.18)\n\n### XGBoost Python package drops Python 2.x (#4379, #4381)\nPython 2.x is reaching its end-of-life at the end of this year. [Many scientific Python packages are now moving to drop Python 2.x](https://python3statement.github.io/).\n\n### XGBoost4J-Spark now requires Spark 2.4.x (#4377)\n* Spark 2.3 is reaching its end-of-life soon. See discussion at #4389.\n* **Consistent handling of missing values** (#4309, #4349, #4411): Many users had reported issue with inconsistent predictions between XGBoost4J-Spark and the Python XGBoost package. The issue was caused by Spark mis-handling non-zero missing values (NaN, -1, 999 etc). We now alert the user whenever Spark doesn't handle missing values correctly (#4309, #4349). See [the tutorial for dealing with missing values in XGBoost4J-Spark](https://xgboost.readthedocs.io/en/release_0.90/jvm/xgboost4j_spark_tutorial.html#dealing-with-missing-values). This fix also depends on the availability of Spark 2.4.x.\n\n### Roadmap: better performance scaling for multi-core CPUs (#4310)\n* Poor performance scaling of the `hist` algorithm for multi-core CPUs has been under investigation (#3810). #4310 optimizes quantile sketches and other pre-processing tasks. Special thanks to @SmirnovEgorRu.\n\n### Roadmap: Harden distributed training (#4250)\n* Make distributed training in XGBoost more robust by hardening [Rabit](https://github.com/dmlc/rabit), which implements [the AllReduce primitive](https://en.wikipedia.org/wiki/Reduce_%28parallel_pattern%29). In particular, improve test coverage on mechanisms for fault tolerance and recovery. Special thanks to @chenqin.\n\n### New feature: Multi-class metric functions for GPUs (#4368)\n* Metrics for multi-class classification have been ported to GPU: `merror`, `mlogloss`. Special thanks to @trivialfis.\n* With supported metrics, XGBoost will select the correct devices based on your system and `n_gpus` parameter.\n\n### New feature: Scikit-learn-like random forest API (#4148, #4255, #4258)\n* XGBoost Python package now offers `XGBRFClassifier` and `XGBRFRegressor` API to train random forests. See [the tutorial](https://xgboost.readthedocs.io/en/release_0.90/tutorials/rf.html). Special thanks to @canonizer\n\n### New feature: use external memory in GPU predictor (#4284, #4396, #4438, #4457)\n* It is now possible to make predictions on GPU when the input is read from external memory. This is useful when you want to make predictions with big dataset that does not fit into the GPU memory. Special thanks to @rongou, @canonizer, @sriramch.\n\n ```python\n dtest = xgboost.DMatrix('test_data.libsvm#dtest.cache')\n bst.set_param('predictor', 'gpu_predictor')\n bst.predict(dtest)\n ```\n\n* Coming soon: GPU training (`gpu_hist`) with external memory\n\n### New feature: XGBoost can now handle comments in LIBSVM files (#4430)\n* Special thanks to @trivialfis and @hcho3\n\n### New feature: Embed XGBoost in your C/C++ applications using CMake (#4323, #4333, #4453)\n* It is now easier than ever to embed XGBoost in your C/C++ applications. In your CMakeLists.txt, add `xgboost::xgboost` as a linked library:\n\n ```cmake\n find_package(xgboost REQUIRED)\n add_executable(api-demo c-api-demo.c)\n target_link_libraries(api-demo xgboost::xgboost)\n ```\n\n [XGBoost C API documentation is available.](https://xgboost.readthedocs.io/en/release_0.90/dev) Special thanks to @trivialfis\n\n### Performance improvements\n* Use feature interaction constraints to narrow split search space (#4341, #4428)\n* Additional optimizations for `gpu_hist` (#4248, #4283)\n* Reduce OpenMP thread launches in `gpu_hist` (#4343)\n* Additional optimizations for multi-node multi-GPU random forests. (#4238)\n* Allocate unique prediction buffer for each input matrix, to avoid re-sizing GPU array (#4275)\n* Remove various synchronisations from CUDA API calls (#4205)\n* XGBoost4J-Spark\n - Allow the user to control whether to cache partitioned training data, to potentially reduce execution time (#4268)\n\n### Bug-fixes\n* Fix node reuse in `hist` (#4404)\n* Fix GPU histogram allocation (#4347)\n* Fix matrix attributes not sliced (#4311)\n* Revise AUC and AUCPR metrics now work with weighted ranking task (#4216, #4436)\n* Fix timer invocation for InitDataOnce() in `gpu_hist` (#4206)\n* Fix R-devel errors (#4251)\n* Make gradient update in GPU linear updater thread-safe (#4259)\n* Prevent out-of-range access in column matrix (#4231)\n* Don't store DMatrix handle in Python object until it's initialized, to improve exception safety (#4317)\n* XGBoost4J-Spark\n - Fix non-deterministic order within a zipped partition on prediction (#4388)\n - Remove race condition on tracker shutdown (#4224)\n - Allow set the parameter `maxLeaves`. (#4226)\n - Allow partial evaluation of dataframe before prediction (#4407)\n - Automatically set `maximize_evaluation_metrics` if not explicitly given (#4446)\n\n### API changes\n* Deprecate `reg:linear` in favor of `reg:squarederror`. (#4267, #4427)\n* Add attribute getter and setter to the Booster object in XGBoost4J (#4336)\n\n### Maintenance: Refactor C++ code for legibility and maintainability\n* Fix clang-tidy warnings. (#4149)\n* Remove deprecated C APIs. (#4266)\n* Use Monitor class to time functions in `hist`. (#4273)\n* Retire DVec class in favour of c++20 style span for device memory. (#4293)\n* Improve HostDeviceVector exception safety (#4301)\n\n### Maintenance: testing, continuous integration, build system\n* **Major refactor of CMakeLists.txt** (#4323, #4333, #4453): adopt modern CMake and export XGBoost as a target\n* **Major improvement in Jenkins CI pipeline** (#4234)\n - Migrate all Linux tests to Jenkins (#4401)\n - Builds and tests are now de-coupled, to test an artifact against multiple versions of CUDA, JDK, and other dependencies (#4401)\n - Add Windows GPU to Jenkins CI pipeline (#4463, #4469)\n* Support CUDA 10.1 (#4223, #4232, #4265, #4468)\n* Python wheels are now built with CUDA 9.0, so that JIT is not required on Volta architecture (#4459)\n* Integrate with NVTX CUDA profiler (#4205)\n* Add a test for cpu predictor using external memory (#4308)\n* Refactor tests to get rid of duplication (#4358)\n* Remove test dependency on `craigcitro/r-travis`, since it's deprecated (#4353)\n* Add files from local R build to `.gitignore` (#4346)\n* Make XGBoost4J compatible with Java 9+ by revising NativeLibLoader (#4351)\n* Jenkins build for CUDA 10.0 (#4281)\n* Remove remaining `silent` and `debug_verbose` in Python tests (#4299)\n* Use all cores to build XGBoost4J lib on linux (#4304)\n* Upgrade Jenkins Linux build environment to GCC 5.3.1, CMake 3.6.0 (#4306)\n* Make CMakeLists.txt compatible with CMake 3.3 (#4420)\n* Add OpenMP option in CMakeLists.txt (#4339)\n* Get rid of a few trivial compiler warnings (#4312)\n* Add external Docker build cache, to speed up builds on Jenkins CI (#4331, #4334, #4458)\n* Fix Windows tests (#4403)\n* Fix a broken python test (#4395)\n* Use a fixed seed to split data in XGBoost4J-Spark tests, for reproducibility (#4417)\n* Add additional Python tests to test training under constraints (#4426)\n* Enable building with shared NCCL. (#4447)\n\n### Usability Improvements, Documentation\n* Document limitation of one-split-at-a-time Greedy tree learning heuristic (#4233)\n* Update build doc: PyPI wheel now support multi-GPU (#4219)\n* Fix docs for `num_parallel_tree` (#4221)\n* Fix document about `colsample_by*` parameter (#4340)\n* Make the train and test input with same colnames. (#4329)\n* Update R contribute link. (#4236)\n* Fix travis R tests (#4277)\n* Log version number in crash log in XGBoost4J-Spark (#4271, #4303)\n* Allow supression of Rabit output in Booster::train in XGBoost4J (#4262)\n* Add tutorial on handling missing values in XGBoost4J-Spark (#4425)\n* Fix typos (#4345, #4393, #4432, #4435)\n* Added language classifier in setup.py (#4327)\n* Added Travis CI badge (#4344)\n* Add BentoML to use case section (#4400)\n* Remove subtly sexist remark (#4418)\n* Add R vignette about parsing JSON dumps (#4439)\n\n### Acknowledgement\n**Contributors**: Nan Zhu (@CodingCat), Adam Pocock (@Craigacp), Daniel Hen (@Daniel8hen), Jiaxiang Li (@JiaxiangBU), Rory Mitchell (@RAMitchell), Egor Smirnov (@SmirnovEgorRu), Andy Adinets (@canonizer), Jonas (@elcombato), Harry Braviner (@harrybraviner), Philip Hyunsu Cho (@hcho3), Tong He (@hetong007), James Lamb (@jameslamb), Jean-Francois Zinque (@jeffzi), Yang Yang (@jokerkeny), Mayank Suman (@mayanksuman), jess (@monkeywithacupcake), Hajime Morrita (@omo), Ravi Kalia (@project-delphi), @ras44, Rong Ou (@rongou), Shaochen Shi (@shishaochen), Xu Xiao (@sperlingxx), @sriramch, Jiaming Yuan (@trivialfis), Christopher Suchanek (@wsuchy), Bozhao (@yubozhao)\n\n**Reviewers**: Nan Zhu (@CodingCat), Adam Pocock (@Craigacp), Daniel Hen (@Daniel8hen), Jiaxiang Li (@JiaxiangBU), Laurae (@Laurae2), Rory Mitchell (@RAMitchell), Egor Smirnov (@SmirnovEgorRu), @alois-bissuel, Andy Adinets (@canonizer), Chen Qin (@chenqin), Harry Braviner (@harrybraviner), Philip Hyunsu Cho (@hcho3), Tong He (@hetong007), @jakirkham, James Lamb (@jameslamb), Julien Schueller (@jschueller), Mayank Suman (@mayanksuman), Hajime Morrita (@omo), Rong Ou (@rongou), Sara Robinson (@sararob), Shaochen Shi (@shishaochen), Xu Xiao (@sperlingxx), @sriramch, Sean Owen (@srowen), Sergei Lebedev (@superbobry), Yuan (Terry) Tang (@terrytangyuan), Theodore Vasiloudis (@thvasilo), Matthew Tovbin (@tovbinm), Jiaming Yuan (@trivialfis), Xin Yin (@xydrolase)\n\n## v0.82 (2019.03.03)\nThis release is packed with many new features and bug fixes.\n\n### Roadmap: better performance scaling for multi-core CPUs (#3957)\n* Poor performance scaling of the `hist` algorithm for multi-core CPUs has been under investigation (#3810). #3957 marks an important step toward better performance scaling, by using software pre-fetching and replacing STL vectors with C-style arrays. Special thanks to @Laurae2 and @SmirnovEgorRu.\n* See #3810 for latest progress on this roadmap.\n\n### New feature: Distributed Fast Histogram Algorithm (`hist`) (#4011, #4102, #4140, #4128)\n* It is now possible to run the `hist` algorithm in distributed setting. Special thanks to @CodingCat. The benefits include:\n 1. Faster local computation via feature binning\n 2. Support for monotonic constraints and feature interaction constraints\n 3. Simpler codebase than `approx`, allowing for future improvement\n* Depth-wise tree growing is now performed in a separate code path, so that cross-node syncronization is performed only once per level.\n\n### New feature: Multi-Node, Multi-GPU training (#4095)\n* Distributed training is now able to utilize clusters equipped with NVIDIA GPUs. In particular, the rabit AllReduce layer will communicate GPU device information. Special thanks to @mt-jones, @RAMitchell, @rongou, @trivialfis, @canonizer, and @jeffdk.\n* Resource management systems will be able to assign a rank for each GPU in the cluster.\n* In Dask, users will be able to construct a collection of XGBoost processes over an inhomogeneous device cluster (i.e. workers with different number and/or kinds of GPUs).\n\n### New feature: Multiple validation datasets in XGBoost4J-Spark (#3904, #3910)\n* You can now track the performance of the model during training with multiple evaluation datasets. By specifying `eval_sets` or call `setEvalSets` over a `XGBoostClassifier` or `XGBoostRegressor`, you can pass in multiple evaluation datasets typed as a `Map` from `String` to `DataFrame`. Special thanks to @CodingCat.\n* See the usage of multiple validation datasets [here](https://github.com/dmlc/xgboost/blob/0c1d5f1120c0a159f2567b267f0ec4ffadee00d0/jvm-packages/xgboost4j-example/src/main/scala/ml/dmlc/xgboost4j/scala/example/spark/SparkTraining.scala#L66-L78)\n\n### New feature: Additional metric functions for GPUs (#3952)\n* Element-wise metrics have been ported to GPU: `rmse`, `mae`, `logloss`, `poisson-nloglik`, `gamma-deviance`, `gamma-nloglik`, `error`, `tweedie-nloglik`. Special thanks to @trivialfis and @RAMitchell.\n* With supported metrics, XGBoost will select the correct devices based on your system and `n_gpus` parameter.\n\n### New feature: Column sampling at individual nodes (splits) (#3971)\n* Columns (features) can now be sampled at individual tree nodes, in addition to per-tree and per-level sampling. To enable per-node sampling, set `colsample_bynode` parameter, which represents the fraction of columns sampled at each node. This parameter is set to 1.0 by default (i.e. no sampling per node). Special thanks to @canonizer.\n* The `colsample_bynode` parameter works cumulatively with other `colsample_by*` parameters: for example, `{'colsample_bynode':0.5, 'colsample_bytree':0.5}` with 100 columns will give 25 features to choose from at each split.\n\n### Major API change: consistent logging level via `verbosity` (#3982, #4002, #4138)\n* XGBoost now allows fine-grained control over logging. You can set `verbosity` to 0 (silent), 1 (warning), 2 (info), and 3 (debug). This is useful for controlling the amount of logging outputs. Special thanks to @trivialfis.\n* Parameters `silent` and `debug_verbose` are now deprecated.\n* Note: Sometimes XGBoost tries to change configurations based on heuristics, which is displayed as warning message. If there's unexpected behaviour, please try to increase value of verbosity.\n\n### Major bug fix: external memory (#4040, #4193)\n* Clarify object ownership in multi-threaded prefetcher, to avoid memory error.\n* Correctly merge two column batches (which uses [CSC layout](https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_column_(CSC_or_CCS))).\n* Add unit tests for external memory.\n* Special thanks to @trivialfis and @hcho3.\n\n### Major bug fix: early stopping fixed in XGBoost4J and XGBoost4J-Spark (#3928, #4176)\n* Early stopping in XGBoost4J and XGBoost4J-Spark is now consistent with its counterpart in the Python package. Training stops if the current iteration is `earlyStoppingSteps` away from the best iteration. If there are multiple evaluation sets, only the last one is used to determinate early stop.\n* See the updated documentation [here](https://xgboost.readthedocs.io/en/release_0.82/jvm/xgboost4j_spark_tutorial.html#early-stopping)\n* Special thanks to @CodingCat, @yanboliang, and @mingyang.\n\n### Major bug fix: infrequent features should not crash distributed training (#4045)\n* For infrequently occuring features, some partitions may not get any instance. This scenario used to crash distributed training due to mal-formed ranges. The problem has now been fixed.\n* In practice, one-hot-encoded categorical variables tend to produce rare features, particularly when the cardinality is high.\n* Special thanks to @CodingCat.\n\n### Performance improvements\n* Faster, more space-efficient radix sorting in `gpu_hist` (#3895)\n* Subtraction trick in histogram calculation in `gpu_hist` (#3945)\n* More performant re-partition in XGBoost4J-Spark (#4049)\n\n### Bug-fixes\n* Fix semantics of `gpu_id` when running multiple XGBoost processes on a multi-GPU machine (#3851)\n* Fix page storage path for external memory on Windows (#3869)\n* Fix configuration setup so that DART utilizes GPU (#4024)\n* Eliminate NAN values from SHAP prediction (#3943)\n* Prevent empty quantile sketches in `hist` (#4155)\n* Enable running objectives with 0 GPU (#3878)\n* Parameters are no longer dependent on system locale (#3891, #3907)\n* Use consistent data type in the GPU coordinate descent code (#3917)\n* Remove undefined behavior in the CLI config parser on the ARM platform (#3976)\n* Initialize counters in GPU AllReduce (#3987)\n* Prevent deadlocks in GPU AllReduce (#4113)\n* Load correct values from sliced NumPy arrays (#4147, #4165)\n* Fix incorrect GPU device selection (#4161)\n* Make feature binning logic in `hist` aware of query groups when running a ranking task (#4115). For ranking task, query groups are weighted, not individual instances.\n* Generate correct C++ exception type for `LOG(FATAL)` macro (#4159)\n* Python package\n - Python package should run on system without `PATH` environment variable (#3845)\n - Fix `coef_` and `intercept_` signature to be compatible with `sklearn.RFECV` (#3873)\n - Use UTF-8 encoding in Python package README, to support non-English locale (#3867)\n - Add AUC-PR to list of metrics to maximize for early stopping (#3936)\n - Allow loading pickles without `self.booster` attribute, for backward compatibility (#3938, #3944)\n - White-list DART for feature importances (#4073)\n - Update usage of [h2oai/datatable](https://github.com/h2oai/datatable) (#4123)\n* XGBoost4J-Spark\n - Address scalability issue in prediction (#4033)\n - Enforce the use of per-group weights for ranking task (#4118)\n - Fix vector size of `rawPredictionCol` in `XGBoostClassificationModel` (#3932)\n - More robust error handling in Spark tracker (#4046, #4108)\n - Fix return type of `setEvalSets` (#4105)\n - Return correct value of `getMaxLeaves` (#4114)\n\n### API changes\n* Add experimental parameter `single_precision_histogram` to use single-precision histograms for the `gpu_hist` algorithm (#3965)\n* Python package\n - Add option to select type of feature importances in the scikit-learn inferface (#3876)\n - Add `trees_to_df()` method to dump decision trees as Pandas data frame (#4153)\n - Add options to control node shapes in the GraphViz plotting function (#3859)\n - Add `xgb_model` option to `XGBClassifier`, to load previously saved model (#4092)\n - Passing lists into `DMatrix` is now deprecated (#3970)\n* XGBoost4J\n - Support multiple feature importance features (#3801)\n\n### Maintenance: Refactor C++ code for legibility and maintainability\n* Refactor `hist` algorithm code and add unit tests (#3836)\n* Minor refactoring of split evaluator in `gpu_hist` (#3889)\n* Removed unused leaf vector field in the tree model (#3989)\n* Simplify the tree representation by combining `TreeModel` and `RegTree` classes (#3995)\n* Simplify and harden tree expansion code (#4008, #4015)\n* De-duplicate parameter classes in the linear model algorithms (#4013)\n* Robust handling of ranges with C++20 span in `gpu_exact` and `gpu_coord_descent` (#4020, #4029)\n* Simplify tree training code (#3825). Also use Span class for robust handling of ranges.\n\n### Maintenance: testing, continuous integration, build system\n* Disallow `std::regex` since it's not supported by GCC 4.8.x (#3870)\n* Add multi-GPU tests for coordinate descent algorithm for linear models (#3893, #3974)\n* Enforce naming style in Python lint (#3896)\n* Refactor Python tests (#3897, #3901): Use pytest exclusively, display full trace upon failure\n* Address `DeprecationWarning` when using Python collections (#3909)\n* Use correct group for maven site plugin (#3937)\n* Jenkins CI is now using on-demand EC2 instances exclusively, due to unreliability of Spot instances (#3948)\n* Better GPU performance logging (#3945)\n* Fix GPU tests on machines with only 1 GPU (#4053)\n* Eliminate CRAN check warnings and notes (#3988)\n* Add unit tests for tree serialization (#3989)\n* Add unit tests for tree fitting functions in `hist` (#4155)\n* Add a unit test for `gpu_exact` algorithm (#4020)\n* Correct JVM CMake GPU flag (#4071)\n* Fix failing Travis CI on Mac (#4086)\n* Speed up Jenkins by not compiling CMake (#4099)\n* Analyze C++ and CUDA code using clang-tidy, as part of Jenkins CI pipeline (#4034)\n* Fix broken R test: Install Homebrew GCC (#4142)\n* Check for empty datasets in GPU unit tests (#4151)\n* Fix Windows compilation (#4139)\n* Comply with latest convention of cpplint (#4157)\n* Fix a unit test in `gpu_hist` (#4158)\n* Speed up data generation in Python tests (#4164)\n\n### Usability Improvements\n* Add link to [InfoWorld 2019 Technology of the Year Award](https://www.infoworld.com/article/3336072/application-development/infoworlds-2019-technology-of-the-year-award-winners.html) (#4116)\n* Remove outdated AWS YARN tutorial (#3885)\n* Document current limitation in number of features (#3886)\n* Remove unnecessary warning when `gblinear` is selected (#3888)\n* Document limitation of CSV parser: header not supported (#3934)\n* Log training parameters in XGBoost4J-Spark (#4091)\n* Clarify early stopping behavior in the scikit-learn interface (#3967)\n* Clarify behavior of `max_depth` parameter (#4078)\n* Revise Python docstrings for ranking task (#4121). In particular, weights must be per-group in learning-to-rank setting.\n* Document parameter `num_parallel_tree` (#4022)\n* Add Jenkins status badge (#4090)\n* Warn users against using internal functions of `Booster` object (#4066)\n* Reformat `benchmark_tree.py` to comply with Python style convention (#4126)\n* Clarify a comment in `objectiveTrait` (#4174)\n* Fix typos and broken links in documentation (#3890, #3872, #3902, #3919, #3975, #4027, #4156, #4167)\n\n### Acknowledgement\n**Contributors** (in no particular order): Jiaming Yuan (@trivialfis), Hyunsu Cho (@hcho3), Nan Zhu (@CodingCat), Rory Mitchell (@RAMitchell), Yanbo Liang (@yanboliang), Andy Adinets (@canonizer), Tong He (@hetong007), Yuan Tang (@terrytangyuan)\n\n**First-time Contributors** (in no particular order): Jelle Zijlstra (@JelleZijlstra), Jiacheng Xu (@jiachengxu), @ajing, Kashif Rasul (@kashif), @theycallhimavi, Joey Gao (@pjgao), Prabakaran Kumaresshan (@nixphix), Huafeng Wang (@huafengw), @lyxthe, Sam Wilkinson (@scwilkinson), Tatsuhito Kato (@stabacov), Shayak Banerjee (@shayakbanerjee), Kodi Arfer (@Kodiologist), @KyleLi1985, Egor Smirnov (@SmirnovEgorRu), @tmitanitky, Pasha Stetsenko (@st-pasha), Kenichi Nagahara (@keni-chi), Abhai Kollara Dilip (@abhaikollara), Patrick Ford (@pford221), @hshujuan, Matthew Jones (@mt-jones), Thejaswi Rao (@teju85), Adam November (@anovember)\n\n**First-time Reviewers** (in no particular order): Mingyang Hu (@mingyang), Theodore Vasiloudis (@thvasilo), Jakub Troszok (@troszok), Rong Ou (@rongou), @Denisevi4, Matthew Jones (@mt-jones), Jeff Kaplan (@jeffdk)\n\n## v0.81 (2018.11.04)\n### New feature: feature interaction constraints\n* Users are now able to control which features (independent variables) are allowed to interact by specifying feature interaction constraints (#3466).\n* [Tutorial](https://xgboost.readthedocs.io/en/release_0.81/tutorials/feature_interaction_constraint.html) is available, as well as [R](https://github.com/dmlc/xgboost/blob/9254c58e4dfff6a59dc0829a2ceb02e45ed17cd0/R-package/demo/interaction_constraints.R) and [Python](https://github.com/dmlc/xgboost/blob/9254c58e4dfff6a59dc0829a2ceb02e45ed17cd0/tests/python/test_interaction_constraints.py) examples.\n\n### New feature: learning to rank using scikit-learn interface\n* Learning to rank task is now available for the scikit-learn interface of the Python package (#3560, #3848). It is now possible to integrate the XGBoost ranking model into the scikit-learn learning pipeline.\n* Examples of using `XGBRanker` class is found at [demo/rank/rank_sklearn.py](https://github.com/dmlc/xgboost/blob/24a268a2e3cb17302db3d72da8f04016b7d352d9/demo/rank/rank_sklearn.py).\n\n### New feature: R interface for SHAP interactions\n* SHAP (SHapley Additive exPlanations) is a unified approach to explain the output of any machine learning model. Previously, this feature was only available from the Python package; now it is available from the R package as well (#3636).\n\n### New feature: GPU predictor now use multiple GPUs to predict\n* GPU predictor is now able to utilize multiple GPUs at once to accelerate prediction (#3738)\n\n### New feature: Scale distributed XGBoost to large-scale clusters\n* Fix OS file descriptor limit assertion error on large cluster (#3835, dmlc/rabit#73) by replacing `select()` based AllReduce/Broadcast with `poll()` based implementation.\n* Mitigate tracker \"thundering herd\" issue on large cluster. Add exponential backoff retry when workers connect to tracker.\n* With this change, we were able to scale to 1.5k executors on a 12 billion row dataset after some tweaks here and there.\n\n### New feature: Additional objective functions for GPUs\n* New objective functions ported to GPU: `hinge`, `multi:softmax`, `multi:softprob`, `count:poisson`, `reg:gamma`, `\"reg:tweedie`.\n* With supported objectives, XGBoost will select the correct devices based on your system and `n_gpus` parameter.\n\n### Major bug fix: learning to rank with XGBoost4J-Spark\n* Previously, `repartitionForData` would shuffle data and lose ordering necessary for ranking task.\n* To fix this issue, data points within each RDD partition is explicitly group by their group (query session) IDs (#3654). Also handle empty RDD partition carefully (#3750).\n\n### Major bug fix: early stopping fixed in XGBoost4J-Spark\n* Earlier implementation of early stopping had incorrect semantics and didn't let users to specify direction for optimizing (maximize / minimize)\n* A parameter `maximize_evaluation_metrics` is defined so as to tell whether a metric should be maximized or minimized as part of early stopping criteria (#3808). Also early stopping now has correct semantics.\n\n### API changes\n* Column sampling by level (`colsample_bylevel`) is now functional for `hist` algorithm (#3635, #3862)\n* GPU tag `gpu:` for regression objectives are now deprecated. XGBoost will select the correct devices automatically (#3643)\n* Add `disable_default_eval_metric` parameter to disable default metric (#3606)\n* Experimental AVX support for gradient computation is removed (#3752)\n* XGBoost4J-Spark\n - Add `rank:ndcg` and `rank:map` to supported objectives (#3697)\n* Python package\n - Add `callbacks` argument to `fit()` function of sciki-learn API (#3682)\n - Add `XGBRanker` to scikit-learn interface (#3560, #3848)\n - Add `validate_features` argument to `predict()` function of scikit-learn API (#3653)\n - Allow scikit-learn grid search over parameters specified as keyword arguments (#3791)\n - Add `coef_` and `intercept_` as properties of scikit-learn wrapper (#3855). Some scikit-learn functions expect these properties.\n\n### Performance improvements\n* Address very high GPU memory usage for large data (#3635)\n* Fix performance regression within `EvaluateSplits()` of `gpu_hist` algorithm. (#3680)\n\n### Bug-fixes\n* Fix a problem in GPU quantile sketch with tiny instance weights. (#3628)\n* Fix copy constructor for `HostDeviceVectorImpl` to prevent dangling pointers (#3657)\n* Fix a bug in partitioned file loading (#3673)\n* Fixed an uninitialized pointer in `gpu_hist` (#3703)\n* Reshared data among GPUs when number of GPUs is changed (#3721)\n* Add back `max_delta_step` to split evaluation (#3668)\n* Do not round up integer thresholds for integer features in JSON dump (#3717)\n* Use `dmlc::TemporaryDirectory` to handle temporaries in cross-platform way (#3783)\n* Fix accuracy problem with `gpu_hist` when `min_child_weight` and `lambda` are set to 0 (#3793)\n* Make sure that `tree_method` parameter is recognized and not silently ignored (#3849)\n* XGBoost4J-Spark\n - Make sure `thresholds` are considered when executing `predict()` method (#3577)\n - Avoid losing precision when computing probabilities by converting to `Double` early (#3576)\n - `getTreeLimit()` should return `Int` (#3602)\n - Fix checkpoint serialization on HDFS (#3614)\n - Throw `ControlThrowable` instead of `InterruptedException` so that it is properly re-thrown (#3632)\n - Remove extraneous output to stdout (#3665)\n - Allow specification of task type for custom objectives and evaluations (#3646)\n - Fix distributed updater check (#3739)\n - Fix issue when spark job execution thread cannot return before we execute `first()` (#3758)\n* Python package\n - Fix accessing `DMatrix.handle` before it is set (#3599)\n - `XGBClassifier.predict()` should return margin scores when `output_margin` is set to true (#3651)\n - Early stopping callback should maximize metric of form `NDCG@n-` (#3685)\n - Preserve feature names when slicing `DMatrix` (#3766)\n* R package\n - Replace `nround` with `nrounds` to match actual parameter (#3592)\n - Amend `xgb.createFolds` to handle classes of a single element (#3630)\n - Fix buggy random generator and make `colsample_bytree` functional (#3781)\n\n### Maintenance: testing, continuous integration, build system\n* Add sanitizers tests to Travis CI (#3557)\n* Add NumPy, Matplotlib, Graphviz as requirements for doc build (#3669)\n* Comply with CRAN submission policy (#3660, #3728)\n* Remove copy-paste error in JVM test suite (#3692)\n* Disable flaky tests in `R-package/tests/testthat/test_update.R` (#3723)\n* Make Python tests compatible with scikit-learn 0.20 release (#3731)\n* Separate out restricted and unrestricted tasks, so that pull requests don't build downloadable artifacts (#3736)\n* Add multi-GPU unit test environment (#3741)\n* Allow plug-ins to be built by CMake (#3752)\n* Test wheel compatibility on CPU containers for pull requests (#3762)\n* Fix broken doc build due to Matplotlib 3.0 release (#3764)\n* Produce `xgboost.so` for XGBoost-R on Mac OSX, so that `make install` works (#3767)\n* Retry Jenkins CI tests up to 3 times to improve reliability (#3769, #3769, #3775, #3776, #3777)\n* Add basic unit tests for `gpu_hist` algorithm (#3785)\n* Fix Python environment for distributed unit tests (#3806)\n* Test wheels on CUDA 10.0 container for compatibility (#3838)\n* Fix JVM doc build (#3853)\n\n### Maintenance: Refactor C++ code for legibility and maintainability\n* Merge generic device helper functions into `GPUSet` class (#3626)\n* Re-factor column sampling logic into `ColumnSampler` class (#3635, #3637)\n* Replace `std::vector` with `HostDeviceVector` in `MetaInfo` and `SparsePage` (#3446)\n* Simplify `DMatrix` class (#3395)\n* De-duplicate CPU/GPU code using `Transform` class (#3643, #3751)\n* Remove obsoleted `QuantileHistMaker` class (#3761)\n* Remove obsoleted `NoConstraint` class (#3792)\n\n### Other Features\n* C++20-compliant Span class for safe pointer indexing (#3548, #3588)\n* Add helper functions to manipulate multiple GPU devices (#3693)\n* XGBoost4J-Spark\n - Allow specifying host ip from the `xgboost-tracker.properties file` (#3833). This comes in handy when `hosts` files doesn't correctly define localhost.\n\n### Usability Improvements\n* Add reference to GitHub repository in `pom.xml` of JVM packages (#3589)\n* Add R demo of multi-class classification (#3695)\n* Document JSON dump functionality (#3600, #3603)\n* Document CUDA requirement and lack of external memory for GPU algorithms (#3624)\n* Document LambdaMART objectives, both pairwise and listwise (#3672)\n* Document `aucpr` evaluation metric (#3687)\n* Document gblinear parameters: `feature_selector` and `top_k` (#3780)\n* Add instructions for using MinGW-built XGBoost with Python. (#3774)\n* Removed nonexistent parameter `use_buffer` from documentation (#3610)\n* Update Python API doc to include all classes and members (#3619, #3682)\n* Fix typos and broken links in documentation (#3618, #3640, #3676, #3713, #3759, #3784, #3843, #3852)\n* Binary classification demo should produce LIBSVM with 0-based indexing (#3652)\n* Process data once for Python and CLI examples of learning to rank (#3666)\n* Include full text of Apache 2.0 license in the repository (#3698)\n* Save predictor parameters in model file (#3856)\n* JVM packages\n - Let users specify feature names when calling `getModelDump` and `getFeatureScore` (#3733)\n - Warn the user about the lack of over-the-wire encryption (#3667)\n - Fix errors in examples (#3719)\n - Document choice of trackers (#3831)\n - Document that vanilla Apache Spark is required (#3854)\n* Python package\n - Document that custom objective can't contain colon (:) (#3601)\n - Show a better error message for failed library loading (#3690)\n - Document that feature importance is unavailable for non-tree learners (#3765)\n - Document behavior of `get_fscore()` for zero-importance features (#3763)\n - Recommend pickling as the way to save `XGBClassifier` / `XGBRegressor` / `XGBRanker` (#3829)\n* R package\n - Enlarge variable importance plot to make it more visible (#3820)\n\n### BREAKING CHANGES\n* External memory page files have changed, breaking backwards compatibility for temporary storage used during external memory training. This only affects external memory users upgrading their xgboost version - we recommend clearing all `*.page` files before resuming training. Model serialization is unaffected.\n\n### Known issues\n* Quantile sketcher fails to produce any quantile for some edge cases (#2943)\n* The `hist` algorithm leaks memory when used with learning rate decay callback (#3579)\n* Using custom evaluation function together with early stopping causes assertion failure in XGBoost4J-Spark (#3595)\n* Early stopping doesn't work with `gblinear` learner (#3789)\n* Label and weight vectors are not reshared upon the change in number of GPUs (#3794). To get around this issue, delete the `DMatrix` object and re-load.\n* The `DMatrix` Python objects are initialized with incorrect values when given array slices (#3841)\n* The `gpu_id` parameter is broken and not yet properly supported (#3850)\n\n### Acknowledgement\n**Contributors** (in no particular order): Hyunsu Cho (@hcho3), Jiaming Yuan (@trivialfis), Nan Zhu (@CodingCat), Rory Mitchell (@RAMitchell), Andy Adinets (@canonizer), Vadim Khotilovich (@khotilov), Sergei Lebedev (@superbobry)\n\n**First-time Contributors** (in no particular order): Matthew Tovbin (@tovbinm), Jakob Richter (@jakob-r), Grace Lam (@grace-lam), Grant W Schneider (@grantschneider), Andrew Thia (@BlueTea88), Sergei Chipiga (@schipiga), Joseph Bradley (@jkbradley), Chen Qin (@chenqin), Jerry Lin (@linjer), Dmitriy Rybalko (@rdtft), Michael Mui (@mmui), Takahiro Kojima (@515hikaru), Bruce Zhao (@BruceZhaoR), Wei Tian (@weitian), Saumya Bhatnagar (@Sam1301), Juzer Shakir (@JuzerShakir), Zhao Hang (@cleghom), Jonathan Friedman (@jontonsoup), Bruno Tremblay (@meztez), Boris Filippov (@frenzykryger), @Shiki-H, @mrgutkun, @gorogm, @htgeis, @jakehoare, @zengxy, @KOLANICH\n\n**First-time Reviewers** (in no particular order): Nikita Titov (@StrikerRUS), Xiangrui Meng (@mengxr), Nirmal Borah (@Nirmal-Neel)\n\n\n## v0.80 (2018.08.13)\n* **JVM packages received a major upgrade**: To consolidate the APIs and improve the user experience, we refactored the design of XGBoost4J-Spark in a significant manner. (#3387)\n - Consolidated APIs: It is now much easier to integrate XGBoost models into a Spark ML pipeline. Users can control behaviors like output leaf prediction results by setting corresponding column names. Training is now more consistent with other Estimators in Spark MLLIB: there is now one single method `fit()` to train decision trees.\n - Better user experience: we refactored the parameters relevant modules in XGBoost4J-Spark to provide both camel-case (Spark ML style) and underscore (XGBoost style) parameters\n - A brand-new tutorial is [available](https://xgboost.readthedocs.io/en/release_0.80/jvm/xgboost4j_spark_tutorial.html) for XGBoost4J-Spark.\n - Latest API documentation is now hosted at https://xgboost.readthedocs.io/.\n* XGBoost documentation now keeps track of multiple versions:\n - Latest master: https://xgboost.readthedocs.io/en/latest\n - 0.80 stable: https://xgboost.readthedocs.io/en/release_0.80\n - 0.72 stable: https://xgboost.readthedocs.io/en/release_0.72\n* Support for per-group weights in ranking objective (#3379)\n* Fix inaccurate decimal parsing (#3546)\n* New functionality\n - Query ID column support in LIBSVM data files (#2749). This is convenient for performing ranking task in distributed setting.\n - Hinge loss for binary classification (`binary:hinge`) (#3477)\n - Ability to specify delimiter and instance weight column for CSV files (#3546)\n - Ability to use 1-based indexing instead of 0-based (#3546)\n* GPU support\n - Quantile sketch, binning, and index compression are now performed on GPU, eliminating PCIe transfer for 'gpu_hist' algorithm (#3319, #3393)\n - Upgrade to NCCL2 for multi-GPU training (#3404).\n - Use shared memory atomics for faster training (#3384).\n - Dynamically allocate GPU memory, to prevent large allocations for deep trees (#3519)\n - Fix memory copy bug for large files (#3472)\n* Python package\n - Importing data from Python datatable (#3272)\n - Pre-built binary wheels available for 64-bit Linux and Windows (#3424, #3443)\n - Add new importance measures 'total_gain', 'total_cover' (#3498)\n - Sklearn API now supports saving and loading models (#3192)\n - Arbitrary cross validation fold indices (#3353)\n - `predict()` function in Sklearn API uses `best_ntree_limit` if available, to make early stopping easier to use (#3445)\n - Informational messages are now directed to Python's `print()` rather than standard output (#3438). This way, messages appear inside Jupyter notebooks.\n* R package\n - Oracle Solaris support, per CRAN policy (#3372)\n* JVM packages\n - Single-instance prediction (#3464)\n - Pre-built JARs are now available from Maven Central (#3401)\n - Add NULL pointer check (#3021)\n - Consider `spark.task.cpus` when controlling parallelism (#3530)\n - Handle missing values in prediction (#3529)\n - Eliminate outputs of `System.out` (#3572)\n* Refactored C++ DMatrix class for simplicity and de-duplication (#3301)\n* Refactored C++ histogram facilities (#3564)\n* Refactored constraints / regularization mechanism for split finding (#3335, #3429). Users may specify an elastic net (L2 + L1 regularization) on leaf weights as well as monotonic constraints on test nodes. The refactor will be useful for a future addition of feature interaction constraints.\n* Statically link `libstdc++` for MinGW32 (#3430)\n* Enable loading from `group`, `base_margin` and `weight` (see [here](http://xgboost.readthedocs.io/en/latest/tutorials/input_format.html#auxiliary-files-for-additional-information)) for Python, R, and JVM packages (#3431)\n* Fix model saving for `count:possion` so that `max_delta_step` doesn't get truncated (#3515)\n* Fix loading of sparse CSC matrix (#3553)\n* Fix incorrect handling of `base_score` parameter for Tweedie regression (#3295)\n\n## v0.72.1 (2018.07.08)\nThis version is only applicable for the Python package. The content is identical to that of v0.72.\n\n## v0.72 (2018.06.01)\n* Starting with this release, we plan to make a new release every two months. See #3252 for more details.\n* Fix a pathological behavior (near-zero second-order gradients) in multiclass objective (#3304)\n* Tree dumps now use high precision in storing floating-point values (#3298)\n* Submodules `rabit` and `dmlc-core` have been brought up to date, bringing bug fixes (#3330, #3221).\n* GPU support\n - Continuous integration tests for GPU code (#3294, #3309)\n - GPU accelerated coordinate descent algorithm (#3178)\n - Abstract 1D vector class now works with multiple GPUs (#3287)\n - Generate PTX code for most recent architecture (#3316)\n - Fix a memory bug on NVIDIA K80 cards (#3293)\n - Address performance instability for single-GPU, multi-core machines (#3324)\n* Python package\n - FreeBSD support (#3247)\n - Validation of feature names in `Booster.predict()` is now optional (#3323)\n* Updated Sklearn API\n - Validation sets now support instance weights (#2354)\n - `XGBClassifier.predict_proba()` should not support `output_margin` option. (#3343) See BREAKING CHANGES below.\n* R package:\n - Better handling of NULL in `print.xgb.Booster()` (#3338)\n - Comply with CRAN policy by removing compiler warning suppression (#3329)\n - Updated CRAN submission\n* JVM packages\n - JVM packages will now use the same versioning scheme as other packages (#3253)\n - Update Spark to 2.3 (#3254)\n - Add scripts to cross-build and deploy artifacts (#3276, #3307)\n - Fix a compilation error for Scala 2.10 (#3332)\n* BREAKING CHANGES\n - `XGBClassifier.predict_proba()` no longer accepts parameter `output_margin`. The parameter makes no sense for `predict_proba()` because the method is to predict class probabilities, not raw margin scores.\n\n## v0.71 (2018.04.11)\n* This is a minor release, mainly motivated by issues concerning `pip install`, e.g. #2426, #3189, #3118, and #3194.\n With this release, users of Linux and MacOS will be able to run `pip install` for the most part.\n* Refactored linear booster class (`gblinear`), so as to support multiple coordinate descent updaters (#3103, #3134). See BREAKING CHANGES below.\n* Fix slow training for multiclass classification with high number of classes (#3109)\n* Fix a corner case in approximate quantile sketch (#3167). Applicable for 'hist' and 'gpu_hist' algorithms\n* Fix memory leak in DMatrix (#3182)\n* New functionality\n - Better linear booster class (#3103, #3134)\n - Pairwise SHAP interaction effects (#3043)\n - Cox loss (#3043)\n - AUC-PR metric for ranking task (#3172)\n - Monotonic constraints for 'hist' algorithm (#3085)\n* GPU support\n - Create an abstract 1D vector class that moves data seamlessly between the main and GPU memory (#2935, #3116, #3068). This eliminates unnecessary PCIe data transfer during training time.\n - Fix minor bugs (#3051, #3217)\n - Fix compatibility error for CUDA 9.1 (#3218)\n* Python package:\n - Correctly handle parameter `verbose_eval=0` (#3115)\n* R package:\n - Eliminate segmentation fault on 32-bit Windows platform (#2994)\n* JVM packages\n - Fix a memory bug involving double-freeing Booster objects (#3005, #3011)\n - Handle empty partition in predict (#3014)\n - Update docs and unify terminology (#3024)\n - Delete cache files after job finishes (#3022)\n - Compatibility fixes for latest Spark versions (#3062, #3093)\n* BREAKING CHANGES: Updated linear modelling algorithms. In particular L1/L2 regularisation penalties are now normalised to number of training examples. This makes the implementation consistent with sklearn/glmnet. L2 regularisation has also been removed from the intercept. To produce linear models with the old regularisation behaviour, the alpha/lambda regularisation parameters can be manually scaled by dividing them by the number of training examples.\n\n## v0.7 (2017.12.30)\n* **This version represents a major change from the last release (v0.6), which was released one year and half ago.**\n* Updated Sklearn API\n - Add compatibility layer for scikit-learn v0.18: `sklearn.cross_validation` now deprecated\n - Updated to allow use of all XGBoost parameters via `**kwargs`.\n - Updated `nthread` to `n_jobs` and `seed` to `random_state` (as per Sklearn convention); `nthread` and `seed` are now marked as deprecated\n - Updated to allow choice of Booster (`gbtree`, `gblinear`, or `dart`)\n - `XGBRegressor` now supports instance weights (specify `sample_weight` parameter)\n - Pass `n_jobs` parameter to the `DMatrix` constructor\n - Add `xgb_model` parameter to `fit` method, to allow continuation of training\n* Refactored gbm to allow more friendly cache strategy\n - Specialized some prediction routine\n* Robust `DMatrix` construction from a sparse matrix\n* Faster construction of `DMatrix` from 2D NumPy matrices: elide copies, use of multiple threads\n* Automatically remove nan from input data when it is sparse.\n - This can solve some of user reported problem of istart != hist.size\n* Fix the single-instance prediction function to obtain correct predictions\n* Minor fixes\n - Thread local variable is upgraded so it is automatically freed at thread exit.\n - Fix saving and loading `count::poisson` models\n - Fix CalcDCG to use base-2 logarithm\n - Messages are now written to stderr instead of stdout\n - Keep built-in evaluations while using customized evaluation functions\n - Use `bst_float` consistently to minimize type conversion\n - Copy the base margin when slicing `DMatrix`\n - Evaluation metrics are now saved to the model file\n - Use `int32_t` explicitly when serializing version\n - In distributed training, synchronize the number of features after loading a data matrix.\n* Migrate to C++11\n - The current master version now requires C++11 enabled compiled(g++4.8 or higher)\n* Predictor interface was factored out (in a manner similar to the updater interface).\n* Makefile support for Solaris and ARM\n* Test code coverage using Codecov\n* Add CPP tests\n* Add `Dockerfile` and `Jenkinsfile` to support continuous integration for GPU code\n* New functionality\n - Ability to adjust tree model's statistics to a new dataset without changing tree structures.\n - Ability to extract feature contributions from individual predictions, as described in [here](http://blog.datadive.net/interpreting-random-forests/) and [here](https://arxiv.org/abs/1706.06060).\n - Faster, histogram-based tree algorithm (`tree_method='hist'`) .\n - GPU/CUDA accelerated tree algorithms (`tree_method='gpu_hist'` or `'gpu_exact'`), including the GPU-based predictor.\n - Monotonic constraints: when other features are fixed, force the prediction to be monotonic increasing with respect to a certain specified feature.\n - Faster gradient calculation using AVX SIMD\n - Ability to export models in JSON format\n - Support for Tweedie regression\n - Additional dropout options for DART: binomial+1, epsilon\n - Ability to update an existing model in-place: this is useful for many applications, such as determining feature importance\n* Python package:\n - New parameters:\n - `learning_rates` in `cv()`\n - `shuffle` in `mknfold()`\n - `max_features` and `show_values` in `plot_importance()`\n - `sample_weight` in `XGBRegressor.fit()`\n - Support binary wheel builds\n - Fix `MultiIndex` detection to support Pandas 0.21.0 and higher\n - Support metrics and evaluation sets whose names contain `-`\n - Support feature maps when plotting trees\n - Compatibility fix for Python 2.6\n - Call `print_evaluation` callback at last iteration\n - Use appropriate integer types when calling native code, to prevent truncation and memory error\n - Fix shared library loading on Mac OS X\n* R package:\n - New parameters:\n - `silent` in `xgb.DMatrix()`\n - `use_int_id` in `xgb.model.dt.tree()`\n - `predcontrib` in `predict()`\n - `monotone_constraints` in `xgb.train()`\n - Default value of the `save_period` parameter in `xgboost()` changed to NULL (consistent with `xgb.train()`).\n - It's possible to custom-build the R package with GPU acceleration support.\n - Enable JVM build for Mac OS X and Windows\n - Integration with AppVeyor CI\n - Improved safety for garbage collection\n - Store numeric attributes with higher precision\n - Easier installation for devel version\n - Improved `xgb.plot.tree()`\n - Various minor fixes to improve user experience and robustness\n - Register native code to pass CRAN check\n - Updated CRAN submission\n* JVM packages\n - Add Spark pipeline persistence API\n - Fix data persistence: loss evaluation on test data had wrongly used caches for training data.\n - Clean external cache after training\n - Implement early stopping\n - Enable training of multiple models by distinguishing stage IDs\n - Better Spark integration: support RDD / dataframe / dataset, integrate with Spark ML package\n - XGBoost4j now supports ranking task\n - Support training with missing data\n - Refactor JVM package to separate regression and classification models to be consistent with other machine learning libraries\n - Support XGBoost4j compilation on Windows\n - Parameter tuning tool\n - Publish source code for XGBoost4j to maven local repo\n - Scala implementation of the Rabit tracker (drop-in replacement for the Java implementation)\n - Better exception handling for the Rabit tracker\n - Persist `num_class`, number of classes (for classification task)\n - `XGBoostModel` now holds `BoosterParams`\n - libxgboost4j is now part of CMake build\n - Release `DMatrix` when no longer needed, to conserve memory\n - Expose `baseMargin`, to allow initialization of boosting with predictions from an external model\n - Support instance weights\n - Use `SparkParallelismTracker` to prevent jobs from hanging forever\n - Expose train-time evaluation metrics via `XGBoostModel.summary`\n - Option to specify `host-ip` explicitly in the Rabit tracker\n* Documentation\n - Better math notation for gradient boosting\n - Updated build instructions for Mac OS X\n - Template for GitHub issues\n - Add `CITATION` file for citing XGBoost in scientific writing\n - Fix dropdown menu in xgboost.readthedocs.io\n - Document `updater_seq` parameter\n - Style fixes for Python documentation\n - Links to additional examples and tutorials\n - Clarify installation requirements\n* Changes that break backward compatibility\n - [#1519](https://github.com/dmlc/xgboost/pull/1519) XGBoost-spark no longer contains APIs for DMatrix; use the public booster interface instead.\n - [#2476](https://github.com/dmlc/xgboost/pull/2476) `XGBoostModel.predict()` now has a different signature\n\n\n## v0.6 (2016.07.29)\n* Version 0.5 is skipped due to major improvements in the core\n* Major refactor of core library.\n - Goal: more flexible and modular code as a portable library.\n - Switch to use of c++11 standard code.\n - Random number generator defaults to ```std::mt19937```.\n - Share the data loading pipeline and logging module from dmlc-core.\n - Enable registry pattern to allow optionally plugin of objective, metric, tree constructor, data loader.\n - Future plugin modules can be put into xgboost/plugin and register back to the library.\n - Remove most of the raw pointers to smart ptrs, for RAII safety.\n* Add official option to approximate algorithm `tree_method` to parameter.\n - Change default behavior to switch to prefer faster algorithm.\n - User will get a message when approximate algorithm is chosen.\n* Change library name to libxgboost.so\n* Backward compatiblity\n - The binary buffer file is not backward compatible with previous version.\n - The model file is backward compatible on 64 bit platforms.\n* The model file is compatible between 64/32 bit platforms(not yet tested).\n* External memory version and other advanced features will be exposed to R library as well on linux.\n - Previously some of the features are blocked due to C++11 and threading limits.\n - The windows version is still blocked due to Rtools do not support ```std::thread```.\n* rabit and dmlc-core are maintained through git submodule\n - Anyone can open PR to update these dependencies now.\n* Improvements\n - Rabit and xgboost libs are not thread-safe and use thread local PRNGs\n - This could fix some of the previous problem which runs xgboost on multiple threads.\n* JVM Package\n - Enable xgboost4j for java and scala\n - XGBoost distributed now runs on Flink and Spark.\n* Support model attributes listing for meta data.\n - https://github.com/dmlc/xgboost/pull/1198\n - https://github.com/dmlc/xgboost/pull/1166\n* Support callback API\n - https://github.com/dmlc/xgboost/issues/892\n - https://github.com/dmlc/xgboost/pull/1211\n - https://github.com/dmlc/xgboost/pull/1264\n* Support new booster DART(dropout in tree boosting)\n - https://github.com/dmlc/xgboost/pull/1220\n* Add CMake build system\n - https://github.com/dmlc/xgboost/pull/1314\n\n## v0.47 (2016.01.14)\n\n* Changes in R library\n - fixed possible problem of poisson regression.\n - switched from 0 to NA for missing values.\n - exposed access to additional model parameters.\n* Changes in Python library\n - throws exception instead of crash terminal when a parameter error happens.\n - has importance plot and tree plot functions.\n - accepts different learning rates for each boosting round.\n - allows model training continuation from previously saved model.\n - allows early stopping in CV.\n - allows feval to return a list of tuples.\n - allows eval_metric to handle additional format.\n - improved compatibility in sklearn module.\n - additional parameters added for sklearn wrapper.\n - added pip installation functionality.\n - supports more Pandas DataFrame dtypes.\n - added best_ntree_limit attribute, in addition to best_score and best_iteration.\n* Java api is ready for use\n* Added more test cases and continuous integration to make each build more robust.\n\n## v0.4 (2015.05.11)\n\n* Distributed version of xgboost that runs on YARN, scales to billions of examples\n* Direct save/load data and model from/to S3 and HDFS\n* Feature importance visualization in R module, by Michael Benesty\n* Predict leaf index\n* Poisson regression for counts data\n* Early stopping option in training\n* Native save load support in R and python\n - xgboost models now can be saved using save/load in R\n - xgboost python model is now pickable\n* sklearn wrapper is supported in python module\n* Experimental External memory version\n\n\n## v0.3 (2014.09.07)\n\n* Faster tree construction module\n - Allows subsample columns during tree construction via ```bst:col_samplebytree=ratio```\n* Support for boosting from initial predictions\n* Experimental version of LambdaRank\n* Linear booster is now parallelized, using parallel coordinated descent.\n* Add [Code Guide](src/README.md) for customizing objective function and evaluation\n* Add R module\n\n\n## v0.2x (2014.05.20)\n\n* Python module\n* Weighted samples instances\n* Initial version of pairwise rank\n\n\n## v0.1 (2014.03.26)\n\n* Initial release\n" }, { "path": "R-package/.Rbuildignore", "content": "\\.o$\n\\.so$\n\\.dll$\n^.*\\.Rproj$\n^\\.Rproj\\.user$\nREADME.md\n^doc$\n^Meta$\n^_pkgdown\\.yml$\n^docs$\n^pkgdown$\n" }, { "path": "R-package/.gitignore", "content": "docs\n" }, { "path": "R-package/CMakeLists.txt", "content": "find_package(LibR REQUIRED)\nmessage(STATUS \"LIBR_CORE_LIBRARY \" ${LIBR_CORE_LIBRARY})\n\nfile(\n GLOB_RECURSE R_SOURCES\n ${CMAKE_CURRENT_LIST_DIR}/src/*.cc\n ${CMAKE_CURRENT_LIST_DIR}/src/*.c\n)\n\n# Use object library to expose symbols\nadd_library(xgboost-r OBJECT ${R_SOURCES})\n\nif(ENABLE_ALL_WARNINGS)\n target_compile_options(xgboost-r PRIVATE -Wall -Wextra)\nendif()\n\nif(MSVC)\n # https://github.com/lightgbm-org/LightGBM/pull/6061\n # MSVC doesn't work with anonymous types in structs. (R complex)\n #\n # syntax error: missing ';' before identifier 'private_data_c'\n #\n target_compile_definitions(xgboost-r PRIVATE -DR_LEGACY_RCOMPLEX)\nendif()\n\ntarget_compile_definitions(\n xgboost-r PUBLIC\n -DXGBOOST_STRICT_R_MODE=1\n -DDMLC_LOG_BEFORE_THROW=0\n -DDMLC_DISABLE_STDIN=1\n -DDMLC_LOG_CUSTOMIZE=1\n)\n\ntarget_include_directories(\n xgboost-r PRIVATE\n ${LIBR_INCLUDE_DIRS}\n ${PROJECT_SOURCE_DIR}/include\n ${PROJECT_SOURCE_DIR}/dmlc-core/include\n)\n\ntarget_link_libraries(xgboost-r PUBLIC ${LIBR_CORE_LIBRARY})\n\nif(USE_OPENMP)\n find_package(OpenMP REQUIRED)\n target_link_libraries(xgboost-r PUBLIC OpenMP::OpenMP_CXX OpenMP::OpenMP_C)\nendif()\n\nset_target_properties(\n xgboost-r PROPERTIES\n CXX_STANDARD 17\n CXX_STANDARD_REQUIRED ON\n POSITION_INDEPENDENT_CODE ON\n)\n\n# Get compilation and link flags of xgboost-r and propagate to objxgboost\ntarget_link_libraries(objxgboost PUBLIC xgboost-r)\n\n# Add all objects of xgboost-r to objxgboost\ntarget_sources(objxgboost INTERFACE $)\n\nset(LIBR_HOME \"${LIBR_HOME}\" PARENT_SCOPE)\nset(LIBR_EXECUTABLE \"${LIBR_EXECUTABLE}\" PARENT_SCOPE)\n" }, { "path": "R-package/DESCRIPTION", "content": "Package: xgboost\nType: Package\nTitle: Extreme Gradient Boosting\nVersion: 3.3.0.0\nDate: 2026-02-10\nAuthors@R: c(\n person(\"Tianqi\", \"Chen\", role = c(\"aut\"),\n email = \"tianqi.tchen@gmail.com\"),\n person(\"Tong\", \"He\", role = c(\"aut\"),\n email = \"hetong007@gmail.com\"),\n person(\"Michael\", \"Benesty\", role = c(\"aut\"),\n email = \"michael@benesty.fr\"),\n person(\"Vadim\", \"Khotilovich\", role = c(\"aut\"),\n email = \"khotilovich@gmail.com\"),\n person(\"Yuan\", \"Tang\", role = c(\"aut\"),\n email = \"terrytangyuan@gmail.com\",\n comment = c(ORCID = \"0000-0001-5243-233X\")),\n person(\"Hyunsu\", \"Cho\", role = c(\"aut\"),\n email = \"chohyu01@cs.washington.edu\"),\n person(\"Kailong\", \"Chen\", role = c(\"aut\")),\n person(\"Rory\", \"Mitchell\", role = c(\"aut\")),\n person(\"Ignacio\", \"Cano\", role = c(\"aut\")),\n person(\"Tianyi\", \"Zhou\", role = c(\"aut\")),\n person(\"Mu\", \"Li\", role = c(\"aut\")),\n person(\"Junyuan\", \"Xie\", role = c(\"aut\")),\n person(\"Min\", \"Lin\", role = c(\"aut\")),\n person(\"Yifeng\", \"Geng\", role = c(\"aut\")),\n person(\"Yutian\", \"Li\", role = c(\"aut\")),\n person(\"Jiaming\", \"Yuan\", role = c(\"aut\", \"cre\"),\n email = \"jm.yuan@outlook.com\"),\n person(\"David\", \"Cortes\", role = c(\"aut\")),\n person(\"XGBoost contributors\", role = c(\"cph\"),\n comment = \"base XGBoost implementation\")\n )\nMaintainer: Jiaming Yuan \nDescription: Extreme Gradient Boosting, which is an efficient implementation\n of the gradient boosting framework from Chen & Guestrin (2016) .\n This package is its R interface. The package includes efficient linear\n model solver and tree learning algorithms. The package can automatically\n do parallel computation on a single machine which could be more than 10\n times faster than existing gradient boosting packages. It supports\n various objective functions, including regression, classification and ranking.\n The package is made to be extensible, so that users are also allowed to define\n their own objectives easily.\nLicense: Apache License (== 2.0) | file LICENSE\nURL: https://github.com/dmlc/xgboost\nBugReports: https://github.com/dmlc/xgboost/issues\nNeedsCompilation: yes\nVignetteBuilder: knitr\nSuggests:\n knitr,\n rmarkdown,\n ggplot2 (>= 1.0.1),\n DiagrammeR (>= 0.9.0),\n DiagrammeRsvg,\n rsvg,\n htmlwidgets,\n Ckmeans.1d.dp (>= 3.3.1),\n vcd (>= 1.3),\n testthat,\n igraph (>= 1.0.1),\n float,\n titanic,\n RhpcBLASctl,\n survival\nDepends:\n R (>= 4.3.0)\nImports:\n Matrix (>= 1.1-0),\n methods,\n data.table (>= 1.9.6),\n jsonlite (>= 1.0)\nRoxygen: list(markdown = TRUE)\nRoxygenNote: 7.3.3\nEncoding: UTF-8\nSystemRequirements: GNU make, C++17\n" }, { "path": "R-package/NAMESPACE", "content": "# Generated by roxygen2: do not edit by hand\n\nS3method(\"[\",xgb.Booster)\nS3method(\"[\",xgb.DMatrix)\nS3method(\"dimnames<-\",xgb.DMatrix)\nS3method(coef,xgb.Booster)\nS3method(dim,xgb.DMatrix)\nS3method(dimnames,xgb.DMatrix)\nS3method(getinfo,xgb.Booster)\nS3method(getinfo,xgb.DMatrix)\nS3method(length,xgb.Booster)\nS3method(predict,xgb.Booster)\nS3method(predict,xgboost)\nS3method(print,xgb.Booster)\nS3method(print,xgb.DMatrix)\nS3method(print,xgb.cv.synchronous)\nS3method(print,xgboost)\nS3method(setinfo,xgb.Booster)\nS3method(setinfo,xgb.DMatrix)\nS3method(variable.names,xgb.Booster)\nexport(\"xgb.attr<-\")\nexport(\"xgb.attributes<-\")\nexport(\"xgb.config<-\")\nexport(\"xgb.model.parameters<-\")\nexport(getinfo)\nexport(setinfo)\nexport(xgb.Callback)\nexport(xgb.DMatrix)\nexport(xgb.DMatrix.hasinfo)\nexport(xgb.DMatrix.save)\nexport(xgb.DataBatch)\nexport(xgb.DataIter)\nexport(xgb.ExtMemDMatrix)\nexport(xgb.QuantileDMatrix)\nexport(xgb.QuantileDMatrix.from_iterator)\nexport(xgb.attr)\nexport(xgb.attributes)\nexport(xgb.cb.cv.predict)\nexport(xgb.cb.early.stop)\nexport(xgb.cb.evaluation.log)\nexport(xgb.cb.gblinear.history)\nexport(xgb.cb.print.evaluation)\nexport(xgb.cb.reset.parameters)\nexport(xgb.cb.save.model)\nexport(xgb.config)\nexport(xgb.copy.Booster)\nexport(xgb.create.features)\nexport(xgb.cv)\nexport(xgb.dump)\nexport(xgb.gblinear.history)\nexport(xgb.get.DMatrix.data)\nexport(xgb.get.DMatrix.num.non.missing)\nexport(xgb.get.DMatrix.qcut)\nexport(xgb.get.config)\nexport(xgb.get.num.boosted.rounds)\nexport(xgb.ggplot.deepness)\nexport(xgb.ggplot.importance)\nexport(xgb.ggplot.shap.summary)\nexport(xgb.importance)\nexport(xgb.is.same.Booster)\nexport(xgb.load)\nexport(xgb.load.raw)\nexport(xgb.model.dt.tree)\nexport(xgb.params)\nexport(xgb.plot.deepness)\nexport(xgb.plot.importance)\nexport(xgb.plot.multi.trees)\nexport(xgb.plot.shap)\nexport(xgb.plot.shap.summary)\nexport(xgb.plot.tree)\nexport(xgb.save)\nexport(xgb.save.raw)\nexport(xgb.set.config)\nexport(xgb.slice.Booster)\nexport(xgb.slice.DMatrix)\nexport(xgb.train)\nexport(xgboost)\nimport(methods)\nimportClassesFrom(Matrix,CsparseMatrix)\nimportClassesFrom(Matrix,dgCMatrix)\nimportClassesFrom(Matrix,dgRMatrix)\nimportFrom(Matrix,sparse.model.matrix)\nimportFrom(data.table,\":=\")\nimportFrom(data.table,as.data.table)\nimportFrom(data.table,data.table)\nimportFrom(data.table,is.data.table)\nimportFrom(data.table,rbindlist)\nimportFrom(data.table,setkey)\nimportFrom(data.table,setkeyv)\nimportFrom(data.table,setnames)\nimportFrom(grDevices,rgb)\nimportFrom(graphics,barplot)\nimportFrom(graphics,grid)\nimportFrom(graphics,lines)\nimportFrom(graphics,par)\nimportFrom(graphics,points)\nimportFrom(graphics,title)\nimportFrom(jsonlite,fromJSON)\nimportFrom(jsonlite,toJSON)\nimportFrom(methods,new)\nimportFrom(stats,coef)\nimportFrom(stats,median)\nimportFrom(stats,predict)\nimportFrom(stats,sd)\nimportFrom(stats,variable.names)\nimportFrom(utils,hasName)\nimportFrom(utils,head)\nimportFrom(utils,object.size)\nimportFrom(utils,str)\nimportFrom(utils,tail)\nuseDynLib(xgboost, .registration = TRUE)\n" }, { "path": "R-package/R/callbacks.R", "content": ".reserved_cb_names <- c(\"names\", \"class\", \"call\", \"params\", \"niter\", \"nfeatures\", \"folds\")\n\n#' XGBoost Callback Constructor\n#'\n#' Constructor for defining the structure of callback functions that can be executed\n#' at different stages of model training (before / after training, before / after each boosting\n#' iteration).\n#'\n#' @details\n#' Arguments that will be passed to the supplied functions are as follows:\n#' - env The same environment that is passed under argument `env`.\n#'\n#' It may be modified by the functions in order to e.g. keep tracking of what happens\n#' across iterations or similar.\n#'\n#' This environment is only used by the functions supplied to the callback, and will\n#' not be kept after the model fitting function terminates (see parameter `f_after_training`).\n#'\n#' - model The booster object when using [xgb.train()], or the folds when using [xgb.cv()].\n#'\n#' For [xgb.cv()], folds are a list with a structure as follows:\n#' - `dtrain`: The training data for the fold (as an `xgb.DMatrix` object).\n#' - `bst`: Rhe `xgb.Booster` object for the fold.\n#' - `evals`: A list containing two DMatrices, with names `train` and `test`\n#' (`test` is the held-out data for the fold).\n#' - `index`: The indices of the hold-out data for that fold (base-1 indexing),\n#' from which the `test` entry in `evals` was obtained.\n#'\n#' This object should **not** be in-place modified in ways that conflict with the\n#' training (e.g. resetting the parameters for a training update in a way that resets\n#' the number of rounds to zero in order to overwrite rounds).\n#'\n#' Note that any R attributes that are assigned to the booster during the callback functions,\n#' will not be kept thereafter as the booster object variable is not re-assigned during\n#' training. It is however possible to set C-level attributes of the booster through\n#' [xgb.attr()] or [xgb.attributes()], which should remain available for the rest\n#' of the iterations and after the training is done.\n#'\n#' For keeping variables across iterations, it's recommended to use `env` instead.\n#' - data The data to which the model is being fit, as an `xgb.DMatrix` object.\n#'\n#' Note that, for [xgb.cv()], this will be the full data, while data for the specific\n#' folds can be found in the `model` object.\n#' - evals The evaluation data, as passed under argument `evals` to [xgb.train()].\n#'\n#' For [xgb.cv()], this will always be `NULL`.\n#' - begin_iteration Index of the first boosting iteration that will be executed (base-1 indexing).\n#'\n#' This will typically be '1', but when using training continuation, depending on the\n#' parameters for updates, boosting rounds will be continued from where the previous\n#' model ended, in which case this will be larger than 1.\n#'\n#' - end_iteration Index of the last boostign iteration that will be executed\n#' (base-1 indexing, inclusive of this end).\n#'\n#' It should match with argument `nrounds` passed to [xgb.train()] or [xgb.cv()].\n#'\n#' Note that boosting might be interrupted before reaching this last iteration, for\n#' example by using the early stopping callback [xgb.cb.early.stop()].\n#' - iteration Index of the iteration number that is being executed (first iteration\n#' will be the same as parameter `begin_iteration`, then next one will add +1, and so on).\n#'\n#' - iter_feval Evaluation metrics for `evals` that were supplied, either\n#' determined by the objective, or by parameter `custom_metric`.\n#'\n#' For [xgb.train()], this will be a named vector with one entry per element in\n#' `evals`, where the names are determined as 'evals name' + '-' + 'metric name' - for\n#' example, if `evals` contains an entry named \"tr\" and the metric is \"rmse\",\n#' this will be a one-element vector with name \"tr-rmse\".\n#'\n#' For [xgb.cv()], this will be a 2d matrix with dimensions `[length(evals), nfolds]`,\n#' where the row names will follow the same naming logic as the one-dimensional vector\n#' that is passed in [xgb.train()].\n#'\n#' Note that, internally, the built-in callbacks such as [xgb.cb.print.evaluation] summarize\n#' this table by calculating the row-wise means and standard deviations.\n#'\n#' - final_feval The evaluation results after the last boosting round is executed\n#' (same format as `iter_feval`, and will be the exact same input as passed under\n#' `iter_feval` to the last round that is executed during model fitting).\n#'\n#' - prev_cb_res Result from a previous run of a callback sharing the same name\n#' (as given by parameter `cb_name`) when conducting training continuation, if there\n#' was any in the booster R attributes.\n#'\n#' Sometimes, one might want to append the new results to the previous one, and this will\n#' be done automatically by the built-in callbacks such as [xgb.cb.evaluation.log],\n#' which will append the new rows to the previous table.\n#'\n#' If no such previous callback result is available (which it never will when fitting\n#' a model from start instead of updating an existing model), this will be `NULL`.\n#'\n#' For [xgb.cv()], which doesn't support training continuation, this will always be `NULL`.\n#'\n#' The following names (`cb_name` values) are reserved for internal callbacks:\n#' - print_evaluation\n#' - evaluation_log\n#' - reset_parameters\n#' - early_stop\n#' - save_model\n#' - cv_predict\n#' - gblinear_history\n#'\n#' The following names are reserved for other non-callback attributes:\n#' - names\n#' - class\n#' - call\n#' - params\n#' - niter\n#' - nfeatures\n#' - folds\n#'\n#' When using the built-in early stopping callback ([xgb.cb.early.stop]), said callback\n#' will always be executed before the others, as it sets some booster C-level attributes\n#' that other callbacks might also use. Otherwise, the order of execution will match with\n#' the order in which the callbacks are passed to the model fitting function.\n#'\n#' @param cb_name Name for the callback.\n#'\n#' If the callback produces some non-NULL result (from executing the function passed under\n#' `f_after_training`), that result will be added as an R attribute to the resulting booster\n#' (or as a named element in the result of CV), with the attribute name specified here.\n#'\n#' Names of callbacks must be unique - i.e. there cannot be two callbacks with the same name.\n#' @param env An environment object that will be passed to the different functions in the callback.\n#' Note that this environment will not be shared with other callbacks.\n#' @param f_before_training A function that will be executed before the training has started.\n#'\n#' If passing `NULL` for this or for the other function inputs, then no function will be executed.\n#'\n#' If passing a function, it will be called with parameters supplied as non-named arguments\n#' matching the function signatures that are shown in the default value for each function argument.\n#' @param f_before_iter A function that will be executed before each boosting round.\n#'\n#' This function can signal whether the training should be finalized or not, by outputting\n#' a value that evaluates to `TRUE` - i.e. if the output from the function provided here at\n#' a given round is `TRUE`, then training will be stopped before the current iteration happens.\n#'\n#' Return values of `NULL` will be interpreted as `FALSE`.\n#' @param f_after_iter A function that will be executed after each boosting round.\n#'\n#' This function can signal whether the training should be finalized or not, by outputting\n#' a value that evaluates to `TRUE` - i.e. if the output from the function provided here at\n#' a given round is `TRUE`, then training will be stopped at that round.\n#'\n#' Return values of `NULL` will be interpreted as `FALSE`.\n#' @param f_after_training A function that will be executed after training is finished.\n#'\n#' This function can optionally output something non-NULL, which will become part of the R\n#' attributes of the booster (assuming one passes `keep_extra_attributes=TRUE` to [xgb.train()])\n#' under the name supplied for parameter `cb_name` imn the case of [xgb.train()]; or a part\n#' of the named elements in the result of [xgb.cv()].\n#' @return An `xgb.Callback` object, which can be passed to [xgb.train()] or [xgb.cv()].\n#'\n#' @seealso Built-in callbacks:\n#' - [xgb.cb.print.evaluation]\n#' - [xgb.cb.evaluation.log]\n#' - [xgb.cb.reset.parameters]\n#' - [xgb.cb.early.stop]\n#' - [xgb.cb.save.model]\n#' - [xgb.cb.cv.predict]\n#' - [xgb.cb.gblinear.history]\n#\n#' @examples\n#' # Example constructing a custom callback that calculates\n#' # squared error on the training data (no separate test set),\n#' # and outputs the per-iteration results.\n#' ssq_callback <- xgb.Callback(\n#' cb_name = \"ssq\",\n#' f_before_training = function(env, model, data, evals,\n#' begin_iteration, end_iteration) {\n#' # A vector to keep track of a number at each iteration\n#' env$logs <- rep(NA_real_, end_iteration - begin_iteration + 1)\n#' },\n#' f_after_iter = function(env, model, data, evals, iteration, iter_feval) {\n#' # This calculates the sum of squared errors on the training data.\n#' # Note that this can be better done by passing an 'evals' entry,\n#' # but this demonstrates a way in which callbacks can be structured.\n#' pred <- predict(model, data)\n#' err <- pred - getinfo(data, \"label\")\n#' sq_err <- sum(err^2)\n#' env$logs[iteration] <- sq_err\n#' cat(\n#' sprintf(\n#' \"Squared error at iteration %d: %.2f\\n\",\n#' iteration, sq_err\n#' )\n#' )\n#'\n#' # A return value of 'TRUE' here would signal to finalize the training\n#' return(FALSE)\n#' },\n#' f_after_training = function(env, model, data, evals, iteration,\n#' final_feval, prev_cb_res) {\n#' return(env$logs)\n#' }\n#' )\n#'\n#' data(mtcars)\n#'\n#' y <- mtcars$mpg\n#' x <- as.matrix(mtcars[, -1])\n#'\n#' dm <- xgb.DMatrix(x, label = y, nthread = 1)\n#' model <- xgb.train(\n#' data = dm,\n#' params = xgb.params(objective = \"reg:squarederror\", nthread = 1),\n#' nrounds = 5,\n#' callbacks = list(ssq_callback)\n#' )\n#'\n#' # Result from 'f_after_iter' will be available as an attribute\n#' attributes(model)$ssq\n#' @export\nxgb.Callback <- function(\n cb_name = \"custom_callback\",\n env = new.env(),\n f_before_training = function(env, model, data, evals, begin_iteration, end_iteration) NULL,\n f_before_iter = function(env, model, data, evals, iteration) NULL,\n f_after_iter = function(env, model, data, evals, iteration, iter_feval) NULL,\n f_after_training = function(env, model, data, evals, iteration, final_feval, prev_cb_res) NULL\n) {\n stopifnot(is.null(f_before_training) || is.function(f_before_training))\n stopifnot(is.null(f_before_iter) || is.function(f_before_iter))\n stopifnot(is.null(f_after_iter) || is.function(f_after_iter))\n stopifnot(is.null(f_after_training) || is.function(f_after_training))\n stopifnot(is.character(cb_name) && length(cb_name) == 1)\n\n if (cb_name %in% .reserved_cb_names) {\n stop(\"Cannot use reserved callback name '\", cb_name, \"'.\")\n }\n\n out <- list(\n cb_name = cb_name,\n env = env,\n f_before_training = f_before_training,\n f_before_iter = f_before_iter,\n f_after_iter = f_after_iter,\n f_after_training = f_after_training\n )\n class(out) <- \"xgb.Callback\"\n return(out)\n}\n\n.execute.cb.before.training <- function(\n callbacks,\n model,\n data,\n evals,\n begin_iteration,\n end_iteration\n) {\n for (callback in callbacks) {\n if (!is.null(callback$f_before_training)) {\n callback$f_before_training(\n callback$env,\n model,\n data,\n evals,\n begin_iteration,\n end_iteration\n )\n }\n }\n}\n\n.execute.cb.before.iter <- function(\n callbacks,\n model,\n data,\n evals,\n iteration\n) {\n if (!length(callbacks)) {\n return(FALSE)\n }\n out <- sapply(callbacks, function(cb) {\n if (is.null(cb$f_before_iter)) {\n return(FALSE)\n }\n should_stop <- cb$f_before_iter(\n cb$env,\n model,\n data,\n evals,\n iteration\n )\n if (!NROW(should_stop)) {\n should_stop <- FALSE\n } else if (NROW(should_stop) > 1) {\n should_stop <- head(as.logical(should_stop), 1)\n }\n return(should_stop)\n })\n return(any(out))\n}\n\n.execute.cb.after.iter <- function(\n callbacks,\n model,\n data,\n evals,\n iteration,\n iter_feval\n) {\n if (!length(callbacks)) {\n return(FALSE)\n }\n out <- sapply(callbacks, function(cb) {\n if (is.null(cb$f_after_iter)) {\n return(FALSE)\n }\n should_stop <- cb$f_after_iter(\n cb$env,\n model,\n data,\n evals,\n iteration,\n iter_feval\n )\n if (!NROW(should_stop)) {\n should_stop <- FALSE\n } else if (NROW(should_stop) > 1) {\n should_stop <- head(as.logical(should_stop), 1)\n }\n return(should_stop)\n })\n return(any(out))\n}\n\n.execute.cb.after.training <- function(\n callbacks,\n model,\n data,\n evals,\n iteration,\n final_feval,\n prev_cb_res\n) {\n if (!length(callbacks)) {\n return(NULL)\n }\n old_cb_res <- attributes(model)\n out <- lapply(callbacks, function(cb) {\n if (is.null(cb$f_after_training)) {\n return(NULL)\n } else {\n return(\n cb$f_after_training(\n cb$env,\n model,\n data,\n evals,\n iteration,\n final_feval,\n getElement(old_cb_res, cb$cb_name)\n )\n )\n }\n })\n names(out) <- sapply(callbacks, function(cb) cb$cb_name)\n if (NROW(out)) {\n out <- out[!sapply(out, is.null)]\n }\n return(out)\n}\n\n.summarize.feval <- function(iter_feval, showsd) {\n if (NCOL(iter_feval) > 1L && showsd) {\n stdev <- apply(iter_feval, 1, sd)\n } else {\n stdev <- NULL\n }\n if (NCOL(iter_feval) > 1L) {\n iter_feval <- rowMeans(iter_feval)\n }\n return(list(feval = iter_feval, stdev = stdev))\n}\n\n.print.evaluation <- function(iter_feval, showsd, iteration) {\n tmp <- .summarize.feval(iter_feval, showsd)\n msg <- .format_eval_string(iteration, tmp$feval, tmp$stdev)\n cat(msg, '\\n')\n}\n\n# Format the evaluation metric string\n.format_eval_string <- function(iter, eval_res, eval_err = NULL) {\n if (length(eval_res) == 0)\n stop('no evaluation results')\n enames <- names(eval_res)\n if (is.null(enames))\n stop('evaluation results must have names')\n iter <- sprintf('[%d]\\t', iter)\n if (!is.null(eval_err)) {\n if (length(eval_res) != length(eval_err))\n stop('eval_res & eval_err lengths mismatch')\n # Note: UTF-8 code for plus/minus sign is U+00B1\n res <- paste0(sprintf(\"%s:%f\\U00B1%f\", enames, eval_res, eval_err), collapse = '\\t')\n } else {\n res <- paste0(sprintf(\"%s:%f\", enames, eval_res), collapse = '\\t')\n }\n return(paste0(iter, res))\n}\n\n#' Callback for printing the result of evaluation\n#'\n#' @description\n#' The callback function prints the result of evaluation at every `period` iterations.\n#' The initial and the last iteration's evaluations are always printed.\n#'\n#' Does not leave any attribute in the booster (see [xgb.cb.evaluation.log] for that).\n#'\n#' @param period Results would be printed every number of periods.\n#' @param showsd Whether standard deviations should be printed (when available).\n#' @return An `xgb.Callback` object, which can be passed to [xgb.train()] or [xgb.cv()].\n#' @seealso [xgb.Callback]\n#' @export\nxgb.cb.print.evaluation <- function(period = 1, showsd = TRUE) {\n if (length(period) != 1 || period != floor(period) || period < 1) {\n stop(\"'period' must be a positive integer.\")\n }\n\n xgb.Callback(\n cb_name = \"print_evaluation\",\n env = as.environment(list(period = period, showsd = showsd, is_first_call = TRUE)),\n f_before_training = NULL,\n f_before_iter = NULL,\n f_after_iter = function(env, model, data, evals, iteration, iter_feval) {\n if (is.null(iter_feval)) {\n return(FALSE)\n }\n if (env$is_first_call || (iteration - 1) %% env$period == 0) {\n .print.evaluation(iter_feval, env$showsd, iteration)\n env$last_printed_iter <- iteration\n }\n env$is_first_call <- FALSE\n return(FALSE)\n },\n f_after_training = function(env, model, data, evals, iteration, final_feval, prev_cb_res) {\n if (is.null(final_feval)) {\n return(NULL)\n }\n if (is.null(env$last_printed_iter) || iteration > env$last_printed_iter) {\n .print.evaluation(final_feval, env$showsd, iteration)\n }\n }\n )\n}\n\n#' Callback for logging the evaluation history\n#'\n#' @details This callback creates a table with per-iteration evaluation metrics (see parameters\n#' `evals` and `custom_metric` in [xgb.train()]).\n#'\n#' Note: in the column names of the final data.table, the dash '-' character is replaced with\n#' the underscore '_' in order to make the column names more like regular R identifiers.\n#'\n#' @return An `xgb.Callback` object, which can be passed to [xgb.train()] or [xgb.cv()].\n#' @seealso [xgb.cb.print.evaluation]\n#' @export\nxgb.cb.evaluation.log <- function() {\n xgb.Callback(\n cb_name = \"evaluation_log\",\n f_before_training = function(env, model, data, evals, begin_iteration, end_iteration) {\n env$evaluation_log <- vector(\"list\", end_iteration - begin_iteration + 1)\n env$next_log <- 1\n },\n f_before_iter = NULL,\n f_after_iter = function(env, model, data, evals, iteration, iter_feval) {\n tmp <- .summarize.feval(iter_feval, TRUE)\n env$evaluation_log[[env$next_log]] <- list(iter = iteration, metrics = tmp$feval, sds = tmp$stdev)\n env$next_log <- env$next_log + 1\n return(FALSE)\n },\n f_after_training = function(env, model, data, evals, iteration, final_feval, prev_cb_res) {\n if (!NROW(env$evaluation_log)) {\n return(prev_cb_res)\n }\n # in case of early stopping\n if (env$next_log <= length(env$evaluation_log)) {\n env$evaluation_log <- head(env$evaluation_log, env$next_log - 1)\n }\n\n iters <- data.frame(iter = sapply(env$evaluation_log, function(x) x$iter))\n metrics <- do.call(rbind, lapply(env$evaluation_log, function(x) x$metrics))\n mnames <- gsub(\"-\", \"_\", names(env$evaluation_log[[1]]$metrics), fixed = TRUE)\n colnames(metrics) <- mnames\n has_sds <- !is.null(env$evaluation_log[[1]]$sds)\n if (has_sds) {\n sds <- do.call(rbind, lapply(env$evaluation_log, function(x) x$sds))\n colnames(sds) <- mnames\n metrics <- lapply(\n mnames,\n function(metric) {\n out <- cbind(metrics[, metric], sds[, metric])\n colnames(out) <- paste0(metric, c(\"_mean\", \"_std\"))\n return(out)\n }\n )\n metrics <- do.call(cbind, metrics)\n }\n evaluation_log <- cbind(iters, metrics)\n\n if (!is.null(prev_cb_res)) {\n if (!is.data.table(prev_cb_res)) {\n prev_cb_res <- data.table::as.data.table(prev_cb_res)\n }\n prev_take <- prev_cb_res[prev_cb_res$iter < min(evaluation_log$iter)]\n if (nrow(prev_take)) {\n evaluation_log <- rbind(prev_cb_res, evaluation_log)\n }\n }\n evaluation_log <- data.table::as.data.table(evaluation_log)\n return(evaluation_log)\n }\n )\n}\n\n#' Callback for resetting booster parameters at each iteration\n#'\n#' @details\n#' Note that when training is resumed from some previous model, and a function is used to\n#' reset a parameter value, the `nrounds` argument in this function would be the\n#' the number of boosting rounds in the current training.\n#'\n#' Does not leave any attribute in the booster.\n#'\n#' @param new_params List of parameters needed to be reset.\n#' Each element's value must be either a vector of values of length `nrounds`\n#' to be set at each iteration,\n#' or a function of two parameters `learning_rates(iteration, nrounds)`\n#' which returns a new parameter value by using the current iteration number\n#' and the total number of boosting rounds.\n#' @return An `xgb.Callback` object, which can be passed to [xgb.train()] or [xgb.cv()].\n#' @export\nxgb.cb.reset.parameters <- function(new_params) {\n stopifnot(is.list(new_params))\n pnames <- gsub(\".\", \"_\", names(new_params), fixed = TRUE)\n not_allowed <- pnames %in%\n c('num_class', 'num_output_group', 'size_leaf_vector', 'updater_seq')\n if (any(not_allowed))\n stop('Parameters ', paste(pnames[not_allowed]), \" cannot be changed during boosting.\")\n\n xgb.Callback(\n cb_name = \"reset_parameters\",\n env = as.environment(list(new_params = new_params)),\n f_before_training = function(env, model, data, evals, begin_iteration, end_iteration) {\n env$end_iteration <- end_iteration\n\n pnames <- gsub(\".\", \"_\", names(env$new_params), fixed = TRUE)\n for (n in pnames) {\n p <- env$new_params[[n]]\n if (is.function(p)) {\n if (length(formals(p)) != 2)\n stop(\"Parameter '\", n, \"' is a function but not of two arguments\")\n } else if (is.numeric(p) || is.character(p)) {\n if (length(p) != env$end_iteration)\n stop(\"Length of '\", n, \"' has to be equal to 'nrounds'\")\n } else {\n stop(\"Parameter '\", n, \"' is not a function or a vector\")\n }\n }\n },\n f_before_iter = function(env, model, data, evals, iteration) {\n params <- lapply(env$new_params, function(p) {\n if (is.function(p)) {\n return(p(iteration, env$end_iteration))\n } else {\n return(p[iteration])\n }\n })\n\n if (inherits(model, \"xgb.Booster\")) {\n xgb.model.parameters(model) <- params\n } else {\n for (fd in model) {\n xgb.model.parameters(fd$bst) <- params\n }\n }\n return(FALSE)\n },\n f_after_iter = NULL,\n f_after_training = NULL\n )\n}\n\n#' Callback to activate early stopping\n#'\n#' @description\n#' This callback function determines the condition for early stopping.\n#'\n#' The following attributes are assigned to the booster's object:\n#' - `best_score` the evaluation score at the best iteration\n#' - `best_iteration` at which boosting iteration the best score has occurred\n#' (0-based index for interoperability of binary models)\n#'\n#' The same values are also stored as R attributes as a result of the callback, plus an additional\n#' attribute `stopped_by_max_rounds` which indicates whether an early stopping by the `stopping_rounds`\n#' condition occurred. Note that the `best_iteration` that is stored under R attributes will follow\n#' base-1 indexing, so it will be larger by '1' than the C-level 'best_iteration' that is accessed\n#' through [xgb.attr()] or [xgb.attributes()].\n#'\n#' At least one dataset is required in `evals` for early stopping to work.\n#'\n#' @param stopping_rounds The number of rounds with no improvement in\n#' the evaluation metric in order to stop the training.\n#' @param maximize Whether to maximize the evaluation metric.\n#' @param metric_name The name of an evaluation column to use as a criteria for early\n#' stopping. If not set, the last column would be used.\n#' Let's say the test data in `evals` was labelled as `dtest`,\n#' and one wants to use the AUC in test data for early stopping regardless of where\n#' it is in the `evals`, then one of the following would need to be set:\n#' `metric_name = 'dtest-auc'` or `metric_name = 'dtest_auc'`.\n#' All dash '-' characters in metric names are considered equivalent to '_'.\n#' @param verbose Whether to print the early stopping information.\n#'\n#' @param save_best Whether training should return the best model or the last model. If\n#' set to `TRUE`, it will only keep the boosting rounds up to the detected best\n#' iteration, discarding the ones that come after. This parameter is not supported by\n#' the `xgb.cv` function and the `gblinear` booster yet.\n#' @return An `xgb.Callback` object, which can be passed to [xgb.train()] or [xgb.cv()].\n#' @export\nxgb.cb.early.stop <- function(\n stopping_rounds,\n maximize = FALSE,\n metric_name = NULL,\n verbose = TRUE,\n save_best = FALSE\n) {\n if (!is.null(metric_name)) {\n stopifnot(is.character(metric_name))\n stopifnot(length(metric_name) == 1L)\n }\n\n xgb.Callback(\n cb_name = \"early_stop\",\n env = as.environment(\n list(\n checked_evnames = FALSE,\n stopping_rounds = stopping_rounds,\n maximize = maximize,\n metric_name = metric_name,\n verbose = verbose,\n save_best = save_best,\n stopped_by_max_rounds = FALSE\n )\n ),\n f_before_training = function(env, model, data, evals, begin_iteration, end_iteration) {\n if (inherits(model, \"xgb.Booster\") && !length(evals)) {\n stop(\"For early stopping, 'evals' must have at least one element\")\n }\n if (!inherits(model, \"xgb.Booster\") && save_best) {\n stop(\"'save_best' must be set to FALSE when using early stopping in 'xgb.cv'.\")\n }\n env$begin_iteration <- begin_iteration\n return(NULL)\n },\n f_before_iter = function(env, model, data, evals, iteration) NULL,\n f_after_iter = function(env, model, data, evals, iteration, iter_feval) {\n sds <- NULL\n if (NCOL(iter_feval) > 1) {\n tmp <- .summarize.feval(iter_feval, TRUE)\n iter_feval <- tmp$feval\n sds <- tmp$stdev\n }\n\n if (!env$checked_evnames) {\n\n eval_names <- gsub('-', '_', names(iter_feval), fixed = TRUE)\n if (!is.null(env$metric_name)) {\n env$metric_idx <- which(gsub('-', '_', env$metric_name, fixed = TRUE) == eval_names)\n if (length(env$metric_idx) == 0)\n stop(\"'metric_name' for early stopping is not one of the following:\\n\",\n paste(eval_names, collapse = ' '), '\\n')\n }\n\n if (is.null(env$metric_name)) {\n if (NROW(iter_feval) == 1) {\n env$metric_idx <- 1L\n } else {\n env$metric_idx <- length(eval_names)\n if (env$verbose)\n cat('Multiple eval metrics are present. Will use ',\n eval_names[env$metric_idx], ' for early stopping.\\n', sep = '')\n }\n }\n\n env$metric_name <- eval_names[env$metric_idx]\n\n # maximize is usually NULL when not set in xgb.train and built-in metrics\n if (is.null(env$maximize))\n env$maximize <- grepl('(_auc|_aupr|_map|_ndcg|_pre)', env$metric_name)\n\n if (env$verbose)\n cat(\"Will train until \", env$metric_name, \" hasn't improved in \",\n env$stopping_rounds, \" rounds.\\n\\n\", sep = '')\n\n env$best_iteration <- env$begin_iteration\n if (env$maximize) {\n env$best_score <- -Inf\n } else {\n env$best_score <- Inf\n }\n\n if (inherits(model, \"xgb.Booster\")) {\n best_score <- xgb.attr(model, 'best_score')\n if (NROW(best_score)) env$best_score <- as.numeric(best_score)\n best_iteration <- xgb.attr(model, 'best_iteration')\n if (NROW(best_iteration)) env$best_iteration <- as.numeric(best_iteration) + 1\n }\n\n env$checked_evnames <- TRUE\n }\n\n score <- iter_feval[env$metric_idx]\n if ((env$maximize && score > env$best_score) ||\n (!env$maximize && score < env$best_score)) {\n\n env$best_score <- score\n env$best_iteration <- iteration\n # save the property to attributes, so they will occur in checkpoint\n if (inherits(model, \"xgb.Booster\")) {\n xgb.attributes(model) <- list(\n best_iteration = env$best_iteration - 1, # convert to 0-based index\n best_score = env$best_score\n )\n }\n } else if (iteration - env$best_iteration >= env$stopping_rounds) {\n if (env$verbose) {\n best_msg <- .format_eval_string(iteration, iter_feval, sds)\n cat(\"Stopping. Best iteration:\\n\", best_msg, \"\\n\\n\", sep = '')\n }\n env$stopped_by_max_rounds <- TRUE\n return(TRUE)\n }\n return(FALSE)\n },\n f_after_training = function(env, model, data, evals, iteration, final_feval, prev_cb_res) {\n if (inherits(model, \"xgb.Booster\") && env$save_best && env$best_iteration < iteration) {\n # Note: it loses the attributes after being sliced,\n # so they have to be re-assigned afterwards.\n prev_attr <- xgb.attributes(model)\n if (NROW(prev_attr)) {\n suppressWarnings({\n prev_attr <- within(prev_attr, rm(\"best_score\", \"best_iteration\"))\n })\n }\n .Call(XGBoosterSliceAndReplace_R, xgb.get.handle(model), 0L, env$best_iteration, 1L)\n if (NROW(prev_attr)) {\n xgb.attributes(model) <- prev_attr\n }\n }\n attrs_set <- list(best_iteration = env$best_iteration - 1, best_score = env$best_score)\n if (inherits(model, \"xgb.Booster\")) {\n xgb.attributes(model) <- attrs_set\n } else {\n for (fd in model) {\n xgb.attributes(fd$bst) <- attrs_set # to use in the cv.predict callback\n }\n }\n return(\n list(\n best_iteration = env$best_iteration,\n best_score = env$best_score,\n stopped_by_max_rounds = env$stopped_by_max_rounds\n )\n )\n }\n )\n}\n\n.save.model.w.formatted.name <- function(model, save_name, iteration) {\n # Note: this throws a warning if the name doesn't have anything to format through 'sprintf'\n suppressWarnings({\n save_name <- sprintf(save_name, iteration)\n })\n xgb.save(model, save_name)\n}\n\n#' Callback for saving a model file\n#'\n#' @description\n#' This callback function allows to save an xgb-model file, either periodically\n#' after each `save_period`'s or at the end.\n#'\n#' Does not leave any attribute in the booster.\n#'\n#' @param save_period Save the model to disk after every `save_period` iterations;\n#' 0 means save the model at the end.\n#' @param save_name The name or path for the saved model file.\n#' It can contain a [sprintf()] formatting specifier to include the integer\n#' iteration number in the file name. E.g., with `save_name = 'xgboost_%04d.model'`,\n#' the file saved at iteration 50 would be named \"xgboost_0050.model\".\n#' @return An `xgb.Callback` object, which can be passed to [xgb.train()],\n#' but **not** to [xgb.cv()].\n#' @export\nxgb.cb.save.model <- function(save_period = 0, save_name = \"xgboost.ubj\") {\n if (save_period < 0) {\n stop(\"'save_period' cannot be negative\")\n }\n if (!is.character(save_name) || length(save_name) != 1L) {\n stop(\"'save_name' must be a single character refering to file name.\")\n }\n\n xgb.Callback(\n cb_name = \"save_model\",\n env = as.environment(list(save_period = save_period, save_name = save_name, last_save = 0)),\n f_before_training = function(env, model, data, evals, begin_iteration, end_iteration) {\n env$begin_iteration <- begin_iteration\n },\n f_before_iter = NULL,\n f_after_iter = function(env, model, data, evals, iteration, iter_feval) {\n if (env$save_period > 0 && (iteration - env$begin_iteration) %% env$save_period == 0) {\n .save.model.w.formatted.name(model, env$save_name, iteration)\n env$last_save <- iteration\n }\n return(FALSE)\n },\n f_after_training = function(env, model, data, evals, iteration, final_feval, prev_cb_res) {\n if (env$save_period == 0 && iteration > env$last_save) {\n .save.model.w.formatted.name(model, env$save_name, iteration)\n }\n }\n )\n}\n\n#' Callback for returning cross-validation based predictions\n#'\n#' This callback function saves predictions for all of the test folds,\n#' and also allows to save the folds' models.\n#'\n#' @details\n#' Predictions are saved inside of the `pred` element, which is either a vector or a matrix,\n#' depending on the number of prediction outputs per data row. The order of predictions corresponds\n#' to the order of rows in the original dataset. Note that when a custom `folds` list is\n#' provided in [xgb.cv()], the predictions would only be returned properly when this list is a\n#' non-overlapping list of k sets of indices, as in a standard k-fold CV. The predictions would not be\n#' meaningful when user-provided folds have overlapping indices as in, e.g., random sampling splits.\n#' When some of the indices in the training dataset are not included into user-provided `folds`,\n#' their prediction value would be `NA`.\n#'\n#' @param save_models A flag for whether to save the folds' models.\n#' @param outputmargin Whether to save margin predictions (same effect as passing this\n#' parameter to [predict.xgb.Booster]).\n#' @return An `xgb.Callback` object, which can be passed to [xgb.cv()],\n#' but **not** to [xgb.train()].\n#' @export\nxgb.cb.cv.predict <- function(save_models = FALSE, outputmargin = FALSE) {\n xgb.Callback(\n cb_name = \"cv_predict\",\n env = as.environment(list(save_models = save_models, outputmargin = outputmargin)),\n f_before_training = function(env, model, data, evals, begin_iteration, end_iteration) {\n if (inherits(model, \"xgb.Booster\")) {\n stop(\"'cv.predict' callback is only for 'xgb.cv'.\")\n }\n },\n f_before_iter = NULL,\n f_after_iter = NULL,\n f_after_training = function(env, model, data, evals, iteration, final_feval, prev_cb_res) {\n pred <- NULL\n for (fd in model) {\n pr <- predict(\n fd$bst,\n fd$evals[[2L]],\n outputmargin = env$outputmargin\n )\n if (is.null(pred)) {\n if (NCOL(pr) > 1L) {\n pred <- matrix(NA_real_, nrow(data), ncol(pr))\n } else {\n pred <- matrix(NA_real_, nrow(data))\n }\n }\n if (is.matrix(pred)) {\n pred[fd$index, ] <- pr\n } else {\n pred[fd$index] <- pr\n }\n }\n out <- list(pred = pred)\n if (env$save_models) {\n out$models <- lapply(model, function(fd) fd$bst)\n }\n return(out)\n }\n )\n}\n\n.list2mat <- function(coef_list, sparse) {\n if (sparse) {\n coef_mat <- methods::new(\"dgRMatrix\")\n coef_mat@p <- as.integer(c(0, cumsum(sapply(coef_list, function(x) length(x@x)))))\n coef_mat@j <- as.integer(unlist(lapply(coef_list, slot, \"i\")) - 1L)\n coef_mat@x <- unlist(lapply(coef_list, slot, \"x\"))\n coef_mat@Dim <- as.integer(c(length(coef_list), length(coef_list[[1L]])))\n # Note: function 'xgb.gblinear.history' might later on try to slice by columns\n coef_mat <- methods::as(coef_mat, \"CsparseMatrix\")\n return(coef_mat)\n } else {\n return(unname(do.call(rbind, coef_list)))\n }\n}\n\n.extract.coef <- function(model, sparse) {\n coefs <- .internal.coef.xgb.Booster(model, add_names = FALSE)\n if (NCOL(coefs) > 1L) {\n coefs <- as.vector(coefs)\n }\n if (sparse) {\n coefs <- methods::as(coefs, \"sparseVector\")\n }\n return(coefs)\n}\n\n#' Callback for collecting coefficients history of a gblinear booster\n#'\n#' @details\n#' To keep things fast and simple, gblinear booster does not internally store the history of linear\n#' model coefficients at each boosting iteration. This callback provides a workaround for storing\n#' the coefficients' path, by extracting them after each training iteration.\n#'\n#' This callback will construct a matrix where rows are boosting iterations and columns are\n#' feature coefficients (same order as when calling [coef.xgb.Booster], with the intercept\n#' corresponding to the first column).\n#'\n#' When there is more than one coefficient per feature (e.g. multi-class classification),\n#' the result will be reshaped into a vector where coefficients are arranged first by features and\n#' then by class (e.g. first 1 through N coefficients will be for the first class, then\n#' coefficients N+1 through 2N for the second class, and so on).\n#'\n#' If the result has only one coefficient per feature in the data, then the resulting matrix\n#' will have column names matching with the feature names, otherwise (when there's more than\n#' one coefficient per feature) the names will be composed as 'column name' + ':' + 'class index'\n#' (so e.g. column 'c1' for class '0' will be named 'c1:0').\n#'\n#' With [xgb.train()], the output is either a dense or a sparse matrix.\n#' With with [xgb.cv()], it is a list (one element per each fold) of such matrices.\n#'\n#' Function [xgb.gblinear.history] provides an easy way to retrieve the\n#' outputs from this callback.\n#'\n#' @param sparse When set to `FALSE`/`TRUE`, a dense/sparse matrix is used to store the result.\n#' Sparse format is useful when one expects only a subset of coefficients to be non-zero,\n#' when using the \"thrifty\" feature selector with fairly small number of top features\n#' selected per iteration.\n#' @return An `xgb.Callback` object, which can be passed to [xgb.train()] or [xgb.cv()].\n#' @seealso [xgb.gblinear.history], [coef.xgb.Booster].\n#' @examples\n#' #### Binary classification:\n#'\n#' ## Keep the number of threads to 1 for examples\n#' nthread <- 1\n#' data.table::setDTthreads(nthread)\n#'\n#' # In the iris dataset, it is hard to linearly separate Versicolor class from the rest\n#' # without considering the 2nd order interactions:\n#' x <- model.matrix(Species ~ .^2, iris)[, -1]\n#' colnames(x)\n#' dtrain <- xgb.DMatrix(\n#' scale(x),\n#' label = 1 * (iris$Species == \"versicolor\"),\n#' nthread = nthread\n#' )\n#' param <- xgb.params(\n#' booster = \"gblinear\",\n#' objective = \"reg:logistic\",\n#' eval_metric = \"auc\",\n#' reg_lambda = 0.0003,\n#' reg_alpha = 0.0003,\n#' nthread = nthread\n#' )\n#'\n#' # For 'shotgun', which is a default linear updater, using high learning_rate values may result in\n#' # unstable behaviour in some datasets. With this simple dataset, however, the high learning\n#' # rate does not break the convergence, but allows us to illustrate the typical pattern of\n#' # \"stochastic explosion\" behaviour of this lock-free algorithm at early boosting iterations.\n#' bst <- xgb.train(\n#' c(param, list(learning_rate = 1.)),\n#' dtrain,\n#' evals = list(tr = dtrain),\n#' nrounds = 200,\n#' callbacks = list(xgb.cb.gblinear.history())\n#' )\n#'\n#' # Extract the coefficients' path and plot them vs boosting iteration number:\n#' coef_path <- xgb.gblinear.history(bst)\n#' matplot(coef_path, type = \"l\")\n#'\n#' # With the deterministic coordinate descent updater, it is safer to use higher learning rates.\n#' # Will try the classical componentwise boosting which selects a single best feature per round:\n#' bst <- xgb.train(\n#' c(\n#' param,\n#' xgb.params(\n#' learning_rate = 0.8,\n#' updater = \"coord_descent\",\n#' feature_selector = \"thrifty\",\n#' top_k = 1\n#' )\n#' ),\n#' dtrain,\n#' evals = list(tr = dtrain),\n#' nrounds = 200,\n#' callbacks = list(xgb.cb.gblinear.history())\n#' )\n#' matplot(xgb.gblinear.history(bst), type = \"l\")\n#' # Componentwise boosting is known to have similar effect to Lasso regularization.\n#' # Try experimenting with various values of top_k, learning_rate, nrounds,\n#' # as well as different feature_selectors.\n#'\n#' # For xgb.cv:\n#' bst <- xgb.cv(\n#' c(\n#' param,\n#' xgb.params(\n#' learning_rate = 0.8,\n#' updater = \"coord_descent\",\n#' feature_selector = \"thrifty\",\n#' top_k = 1\n#' )\n#' ),\n#' dtrain,\n#' nfold = 5,\n#' nrounds = 100,\n#' callbacks = list(xgb.cb.gblinear.history())\n#' )\n#' # coefficients in the CV fold #3\n#' matplot(xgb.gblinear.history(bst)[[3]], type = \"l\")\n#'\n#'\n#' #### Multiclass classification:\n#' dtrain <- xgb.DMatrix(scale(x), label = as.numeric(iris$Species) - 1, nthread = nthread)\n#'\n#' param <- xgb.params(\n#' booster = \"gblinear\",\n#' objective = \"multi:softprob\",\n#' num_class = 3,\n#' reg_lambda = 0.0003,\n#' reg_alpha = 0.0003,\n#' nthread = nthread\n#' )\n#'\n#' # For the default linear updater 'shotgun' it sometimes is helpful\n#' # to use smaller learning_rate to reduce instability\n#' bst <- xgb.train(\n#' c(param, list(learning_rate = 0.5)),\n#' dtrain,\n#' evals = list(tr = dtrain),\n#' nrounds = 50,\n#' callbacks = list(xgb.cb.gblinear.history())\n#' )\n#'\n#' # Will plot the coefficient paths separately for each class:\n#' matplot(xgb.gblinear.history(bst, class_index = 0), type = \"l\")\n#' matplot(xgb.gblinear.history(bst, class_index = 1), type = \"l\")\n#' matplot(xgb.gblinear.history(bst, class_index = 2), type = \"l\")\n#'\n#' # CV:\n#' bst <- xgb.cv(\n#' c(param, list(learning_rate = 0.5)),\n#' dtrain,\n#' nfold = 5,\n#' nrounds = 70,\n#' callbacks = list(xgb.cb.gblinear.history(FALSE))\n#' )\n#' # 1st fold of 1st class\n#' matplot(xgb.gblinear.history(bst, class_index = 0)[[1]], type = \"l\")\n#'\n#' @export\nxgb.cb.gblinear.history <- function(sparse = FALSE) {\n xgb.Callback(\n cb_name = \"gblinear_history\",\n env = as.environment(list(sparse = sparse)),\n f_before_training = function(env, model, data, evals, begin_iteration, end_iteration) {\n if (!inherits(model, \"xgb.Booster\")) {\n model <- model[[1L]]$bst\n }\n if (xgb.booster_type(model) != \"gblinear\") {\n stop(\"Callback 'xgb.cb.gblinear.history' is only for booster='gblinear'.\")\n }\n env$coef_hist <- vector(\"list\", end_iteration - begin_iteration + 1)\n env$next_idx <- 1\n },\n f_before_iter = NULL,\n f_after_iter = function(env, model, data, evals, iteration, iter_feval) {\n if (inherits(model, \"xgb.Booster\")) {\n coef_this <- .extract.coef(model, env$sparse)\n } else {\n coef_this <- lapply(model, function(fd) .extract.coef(fd$bst, env$sparse))\n }\n env$coef_hist[[env$next_idx]] <- coef_this\n env$next_idx <- env$next_idx + 1\n return(FALSE)\n },\n f_after_training = function(env, model, data, evals, iteration, final_feval, prev_cb_res) {\n # in case of early stopping\n if (env$next_idx <= length(env$coef_hist)) {\n env$coef_hist <- head(env$coef_hist, env$next_idx - 1)\n }\n\n is_booster <- inherits(model, \"xgb.Booster\")\n if (is_booster) {\n out <- .list2mat(env$coef_hist, env$sparse)\n } else {\n out <- lapply(\n X = lapply(\n X = seq_along(env$coef_hist[[1]]),\n FUN = function(i) lapply(env$coef_hist, \"[[\", i)\n ),\n FUN = .list2mat,\n env$sparse\n )\n }\n if (!is.null(prev_cb_res)) {\n if (is_booster) {\n out <- rbind(prev_cb_res, out)\n } else {\n # Note: this case should never be encountered, since training cannot\n # be continued from the result of xgb.cv, but this code should in\n # theory do the job if the situation were to be encountered.\n out <- lapply(\n out,\n function(lst) {\n lapply(\n seq_along(lst),\n function(i) rbind(prev_cb_res[[i]], lst[[i]])\n )\n }\n )\n }\n }\n feature_names <- getinfo(data, \"feature_name\")\n if (!NROW(feature_names)) {\n feature_names <- paste0(\"V\", seq(1L, ncol(data)))\n }\n expected_ncols <- length(feature_names) + 1\n if (is_booster) {\n mat_ncols <- ncol(out)\n } else {\n mat_ncols <- ncol(out[[1L]])\n }\n if (mat_ncols %% expected_ncols == 0) {\n feature_names <- c(\"(Intercept)\", feature_names)\n n_rep <- mat_ncols / expected_ncols\n if (n_rep > 1) {\n feature_names <- unlist(\n lapply(\n seq(1, n_rep),\n function(cl) paste(feature_names, cl - 1, sep = \":\")\n )\n )\n }\n if (is_booster) {\n colnames(out) <- feature_names\n } else {\n out <- lapply(\n out,\n function(mat) {\n colnames(mat) <- feature_names\n return(mat)\n }\n )\n }\n }\n return(out)\n }\n )\n}\n\n#' Extract gblinear coefficients history\n#'\n#' A helper function to extract the matrix of linear coefficients' history\n#' from a gblinear model created while using the [xgb.cb.gblinear.history]\n#' callback (which must be added manually as by default it is not used).\n#'\n#' @details\n#' Note that this is an R-specific function that relies on R attributes that\n#' are not saved when using XGBoost's own serialization functions like [xgb.load()]\n#' or [xgb.load.raw()].\n#'\n#' In order for a serialized model to be accepted by this function, one must use R\n#' serializers such as [saveRDS()].\n#' @param model Either an `xgb.Booster` or a result of [xgb.cv()], trained\n#' using the [xgb.cb.gblinear.history] callback, but **not** a booster\n#' loaded from [xgb.load()] or [xgb.load.raw()].\n#' @param class_index zero-based class index to extract the coefficients for only that\n#' specific class in a multinomial multiclass model. When it is `NULL`, all the\n#' coefficients are returned. Has no effect in non-multiclass models.\n#'\n#' @return\n#' For an [xgb.train()] result, a matrix (either dense or sparse) with the columns\n#' corresponding to iteration's coefficients and the rows corresponding to boosting iterations.\n#'\n#' For an [xgb.cv()] result, a list of such matrices is returned with the elements\n#' corresponding to CV folds.\n#'\n#' When there is more than one coefficient per feature (e.g. multi-class classification)\n#' and `class_index` is not provided,\n#' the result will be reshaped into a vector where coefficients are arranged first by features and\n#' then by class (e.g. first 1 through N coefficients will be for the first class, then\n#' coefficients N+1 through 2N for the second class, and so on).\n#' @seealso [xgb.cb.gblinear.history], [coef.xgb.Booster].\n#' @export\nxgb.gblinear.history <- function(model, class_index = NULL) {\n\n if (!(inherits(model, \"xgb.Booster\") ||\n inherits(model, \"xgb.cv.synchronous\")))\n stop(\"model must be an object of either xgb.Booster or xgb.cv.synchronous class\")\n is_cv <- inherits(model, \"xgb.cv.synchronous\")\n\n if (!is_cv) {\n coef_path <- getElement(attributes(model), \"gblinear_history\")\n } else {\n coef_path <- getElement(model, \"gblinear_history\")\n }\n if (is.null(coef_path)) {\n stop(\"model must be trained while using the xgb.cb.gblinear.history() callback\")\n }\n\n if (!is_cv) {\n num_class <- xgb.num_class(model)\n num_feat <- xgb.num_feature(model)\n } else {\n # in case of CV, the object is expected to have this info\n if (model$params$booster != \"gblinear\")\n stop(\"It does not appear to be a gblinear model\")\n num_class <- NVL(model$params$num_class, 1)\n num_feat <- model$nfeatures\n if (is.null(num_feat))\n stop(\"This xgb.cv result does not have nfeatures info\")\n }\n\n if (!is.null(class_index) &&\n num_class > 1 &&\n (class_index[1] < 0 || class_index[1] >= num_class))\n stop(\"class_index has to be within [0,\", num_class - 1, \"]\")\n\n if (!is.null(class_index) && num_class > 1) {\n seq_take <- seq(1 + class_index * (num_feat + 1), (class_index + 1) * (num_feat + 1))\n coef_path <- if (is.list(coef_path)) {\n lapply(coef_path, function(x) x[, seq_take])\n } else {\n coef_path <- coef_path[, seq_take]\n }\n }\n return(coef_path)\n}\n\n.callbacks.only.train <- \"save_model\"\n.callbacks.only.cv <- \"cv_predict\"\n\n.process.callbacks <- function(callbacks, is_cv) {\n if (inherits(callbacks, \"xgb.Callback\")) {\n callbacks <- list(callbacks)\n }\n if (!is.list(callbacks)) {\n stop(\"'callbacks' must be a list.\")\n }\n cb_names <- character()\n if (length(callbacks)) {\n is_callback <- sapply(callbacks, inherits, \"xgb.Callback\")\n if (!all(is_callback)) {\n stop(\"Entries in 'callbacks' must be 'xgb.Callback' objects.\")\n }\n cb_names <- sapply(callbacks, function(cb) cb$cb_name)\n if (length(cb_names) != length(callbacks)) {\n stop(\"Passed invalid callback(s).\")\n }\n if (anyDuplicated(cb_names) > 0) {\n stop(\"Callbacks must have unique names.\")\n }\n if (is_cv) {\n if (any(.callbacks.only.train %in% cb_names)) {\n stop(\n \"Passed callback(s) not supported for 'xgb.cv': \",\n paste(intersect(.callbacks.only.train, cb_names), collapse = \", \")\n )\n }\n } else {\n if (any(.callbacks.only.cv %in% cb_names)) {\n stop(\n \"Passed callback(s) not supported for 'xgb.train': \",\n paste(intersect(.callbacks.only.cv, cb_names), collapse = \", \")\n )\n }\n }\n # Early stopping callback needs to be executed before the others\n if (\"early_stop\" %in% cb_names) {\n mask <- cb_names == \"early_stop\"\n callbacks <- c(list(callbacks[[which(mask)]]), callbacks[!mask])\n }\n }\n return(list(callbacks = callbacks, cb_names = cb_names))\n}\n\n# Note: don't try to use functions like 'append', as they will\n# merge the elements of the different callbacks into a single list.\nadd.callback <- function(callbacks, cb, as_first_elt = FALSE) {\n if (!as_first_elt) {\n callbacks[[length(callbacks) + 1]] <- cb\n return(callbacks)\n } else {\n if (!length(callbacks)) {\n return(list(cb))\n }\n new_cb <- vector(\"list\", length(callbacks) + 1)\n new_cb[[1]] <- cb\n new_cb[seq(2, length(new_cb))] <- callbacks\n return(new_cb)\n }\n}\n\nhas.callbacks <- function(callbacks, cb_name) {\n cb_names <- sapply(callbacks, function(cb) cb$name)\n return(cb_name %in% cb_names)\n}\n" }, { "path": "R-package/R/utils.R", "content": "#\n# This file is for the low level reusable utility functions\n# that are not supposed to be visible to a user.\n#\n\n#\n# General helper utilities ----------------------------------------------------\n#\n\n# SQL-style NVL shortcut.\nNVL <- function(x, val) {\n if (is.null(x))\n return(val)\n if (is.vector(x)) {\n x[is.na(x)] <- val\n return(x)\n }\n if (typeof(x) == 'closure')\n return(x)\n stop(\"typeof(x) == \", typeof(x), \" is not supported by NVL\")\n}\n\n# List of classification and ranking objectives\n.CLASSIFICATION_OBJECTIVES <- function() {\n return(c('binary:logistic', 'binary:logitraw', 'binary:hinge', 'multi:softmax',\n 'multi:softprob', 'rank:pairwise', 'rank:ndcg', 'rank:map'))\n}\n\n.RANKING_OBJECTIVES <- function() {\n return(c('rank:pairwise', 'rank:ndcg', 'rank:map'))\n}\n\n.OBJECTIVES_NON_DEFAULT_MODE <- function() {\n return(c(\"reg:logistic\", \"binary:logitraw\", \"multi:softmax\"))\n}\n\n.BINARY_CLASSIF_OBJECTIVES <- function() {\n return(c(\"binary:logistic\", \"binary:hinge\"))\n}\n\n.MULTICLASS_CLASSIF_OBJECTIVES <- function() {\n return(\"multi:softprob\")\n}\n\n.SURVIVAL_RIGHT_CENSORING_OBJECTIVES <- function() { # nolint\n return(c(\"survival:cox\", \"survival:aft\"))\n}\n\n.SURVIVAL_ALL_CENSORING_OBJECTIVES <- function() { # nolint\n return(\"survival:aft\")\n}\n\n.REGRESSION_OBJECTIVES <- function() {\n return(c(\n \"reg:squarederror\", \"reg:squaredlogerror\", \"reg:logistic\", \"reg:pseudohubererror\",\n \"reg:absoluteerror\", \"reg:quantileerror\", \"reg:expectileerror\", \"count:poisson\",\n \"reg:gamma\", \"reg:tweedie\"\n ))\n}\n\n.MULTI_TARGET_OBJECTIVES <- function() {\n return(c(\n \"reg:squarederror\", \"reg:squaredlogerror\", \"reg:logistic\", \"reg:pseudohubererror\",\n \"reg:quantileerror\", \"reg:gamma\"\n ))\n}\n\n\n#\n# Low-level functions for boosting --------------------------------------------\n#\n\n# Merges booster params with whatever is provided in ...\n# plus runs some checks\ncheck.booster.params <- function(params) {\n if (!identical(class(params), \"list\"))\n stop(\"params must be a list\")\n\n # in R interface, allow for '.' instead of '_' in parameter names\n names(params) <- gsub(\".\", \"_\", names(params), fixed = TRUE)\n\n # providing a parameter multiple times makes sense only for 'eval_metric'\n name_freqs <- table(names(params))\n multi_names <- setdiff(names(name_freqs[name_freqs > 1]), 'eval_metric')\n if (length(multi_names) > 0) {\n warning(\"The following parameters were provided multiple times:\\n\\t\",\n paste(multi_names, collapse = ', '), \"\\n Only the last value for each of them will be used.\\n\")\n # While xgboost internals would choose the last value for a multiple-times parameter,\n # enforce it here in R as well (b/c multi-parameters might be used further in R code,\n # and R takes the 1st value when multiple elements with the same name are present in a list).\n for (n in multi_names) {\n del_idx <- which(n == names(params))\n del_idx <- del_idx[-length(del_idx)]\n params[[del_idx]] <- NULL\n }\n }\n\n # for multiclass, expect num_class to be set\n if (typeof(params[['objective']]) == \"character\" &&\n startsWith(NVL(params[['objective']], 'x'), 'multi:') &&\n as.numeric(NVL(params[['num_class']], 0)) < 2) {\n stop(\"'num_class' > 1 parameter must be set for multiclass classification\")\n }\n\n # monotone_constraints parser\n if (!is.null(params[['monotone_constraints']]) &&\n typeof(params[['monotone_constraints']]) != \"character\") {\n vec2str <- paste(params[['monotone_constraints']], collapse = ',')\n vec2str <- paste0('(', vec2str, ')')\n params[['monotone_constraints']] <- vec2str\n }\n\n # interaction constraints parser (convert from list of column indices to string)\n if (!is.null(params[['interaction_constraints']]) &&\n typeof(params[['interaction_constraints']]) != \"character\") {\n # check input class\n if (!identical(class(params[['interaction_constraints']]), 'list')) stop('interaction_constraints should be class list')\n if (!all(unique(sapply(params[['interaction_constraints']], class)) %in% c('numeric', 'integer'))) {\n stop('interaction_constraints should be a list of numeric/integer vectors')\n }\n\n # recast parameter as string\n interaction_constraints <- sapply(params[['interaction_constraints']], function(x) paste0('[', paste(x, collapse = ','), ']'))\n params[['interaction_constraints']] <- paste0('[', paste(interaction_constraints, collapse = ','), ']')\n }\n\n # for evaluation metrics, should generate multiple entries per metric\n if (NROW(params[['eval_metric']]) > 1) {\n eval_metrics <- as.list(params[[\"eval_metric\"]])\n names(eval_metrics) <- rep(\"eval_metric\", length(eval_metrics))\n params_without_ev_metrics <- within(params, rm(\"eval_metric\"))\n params <- c(params_without_ev_metrics, eval_metrics)\n }\n return(params)\n}\n\n\n# Performs some checks related to custom objective function.\ncheck.custom.obj <- function(params, objective) {\n if (!is.null(params[['objective']]) && !is.null(objective))\n stop(\"Setting objectives in 'params' and 'objective' at the same time is not allowed\")\n\n if (!is.null(objective) && typeof(objective) != 'closure') {\n if (is.character(objective)) {\n msg <- paste(\n \"Argument 'objective' is only for custom objectives.\",\n \"For built-in objectives, pass the objective under 'params'.\",\n sep = \" \"\n )\n error_on_deprecated <- getOption(\"xgboost.strict_mode\", default = FALSE)\n if (error_on_deprecated) {\n stop(msg)\n } else {\n warning(msg, \" This warning will become an error in a future version.\")\n }\n params$objective <- objective\n return(list(params = params, objective = NULL))\n }\n stop(\"'objective' must be a function\")\n }\n\n # handle the case when custom objective function was provided through params\n if (!is.null(params[['objective']]) &&\n typeof(params$objective) == 'closure') {\n objective <- params$objective\n params$objective <- NULL\n }\n return(list(params = params, objective = objective))\n}\n\n# Performs some checks related to custom evaluation function.\ncheck.custom.eval <- function(params, custom_metric, maximize, early_stopping_rounds, callbacks) {\n if (!is.null(params[['eval_metric']]) && !is.null(custom_metric))\n stop(\"Setting evaluation metrics in 'params' and 'custom_metric' at the same time is not allowed\")\n\n if (!is.null(custom_metric) && typeof(custom_metric) != 'closure')\n stop(\"'custom_metric' must be a function\")\n\n # handle a situation when custom eval function was provided through params\n if (!is.null(params[['eval_metric']]) &&\n typeof(params$eval_metric) == 'closure') {\n custom_metric <- params$eval_metric\n params$eval_metric <- NULL\n }\n\n # require maximize to be set when custom metric and early stopping are used together\n if (!is.null(custom_metric) &&\n is.null(maximize) && (\n !is.null(early_stopping_rounds) ||\n has.callbacks(callbacks, \"early_stop\")))\n stop(\"Please set 'maximize' to indicate whether the evaluation metric needs to be maximized or not\")\n\n return(list(params = params, custom_metric = custom_metric))\n}\n\n\n# Update a booster handle for an iteration with dtrain data\nxgb.iter.update <- function(bst, dtrain, iter, objective) {\n if (!inherits(dtrain, \"xgb.DMatrix\")) {\n stop(\"dtrain must be of xgb.DMatrix class\")\n }\n handle <- xgb.get.handle(bst)\n\n if (is.null(objective)) {\n .Call(XGBoosterUpdateOneIter_R, handle, as.integer(iter), dtrain)\n } else {\n pred <- predict(\n bst,\n dtrain,\n outputmargin = TRUE,\n training = TRUE\n )\n gpair <- objective(pred, dtrain)\n n_samples <- dim(dtrain)[1L]\n grad <- gpair$grad\n hess <- gpair$hess\n\n if ((is.matrix(grad) && dim(grad)[1L] != n_samples) ||\n (is.vector(grad) && length(grad) != n_samples) ||\n (is.vector(grad) != is.vector(hess))) {\n warning(paste(\n \"Since 2.1.0, the shape of the gradient and hessian is required to be \",\n \"(n_samples, n_targets) or (n_samples, n_classes). Will reshape assuming \",\n \"column-major order.\",\n sep = \"\"\n ))\n grad <- matrix(grad, nrow = n_samples)\n hess <- matrix(hess, nrow = n_samples)\n }\n\n .Call(\n XGBoosterTrainOneIter_R, handle, dtrain, iter, grad, hess\n )\n }\n return(TRUE)\n}\n\n\n# Evaluate one iteration.\n# Returns a named vector of evaluation metrics\n# with the names in a 'datasetname-metricname' format.\nxgb.iter.eval <- function(bst, evals, iter, custom_metric) {\n handle <- xgb.get.handle(bst)\n\n if (length(evals) == 0)\n return(NULL)\n\n evnames <- names(evals)\n if (is.null(custom_metric)) {\n msg <- .Call(XGBoosterEvalOneIter_R, handle, as.integer(iter), evals, as.list(evnames))\n mat <- matrix(strsplit(msg, '\\\\s+|:')[[1]][-1], nrow = 2)\n res <- structure(as.numeric(mat[2, ]), names = mat[1, ])\n } else {\n res <- sapply(seq_along(evals), function(j) {\n w <- evals[[j]]\n ## predict using all trees\n preds <- predict(bst, w, outputmargin = TRUE, iterationrange = \"all\")\n eval_res <- custom_metric(preds, w)\n out <- eval_res$value\n names(out) <- paste0(evnames[j], \"-\", eval_res$metric)\n out\n })\n }\n return(res)\n}\n\n\n#\n# Helper functions for cross validation ---------------------------------------\n#\n\n# Possibly convert the labels into factors, depending on the objective.\n# The labels are converted into factors only when the given objective refers to the classification\n# or ranking tasks.\nconvert.labels <- function(labels, objective_name) {\n if (objective_name %in% .CLASSIFICATION_OBJECTIVES()) {\n return(as.factor(labels))\n } else {\n return(labels)\n }\n}\n\n# Generates random (stratified if needed) CV folds\ngenerate.cv.folds <- function(nfold, nrows, stratified, label, group, params) {\n if (NROW(group)) {\n if (stratified) {\n warning(\n paste0(\n \"Stratified splitting is not supported when using 'group' attribute.\",\n \" Will use unstratified splitting.\"\n )\n )\n }\n return(generate.group.folds(nfold, group))\n }\n objective <- params$objective\n if (stratified && !is.character(objective)) {\n warning(\"Will use unstratified splitting (custom objective used)\")\n stratified <- FALSE\n }\n # cannot stratify if label is NULL\n if (stratified && is.null(label)) {\n warning(\"Will use unstratified splitting (no 'labels' available)\")\n stratified <- FALSE\n }\n\n # cannot do it for rank\n if (is.character(objective) && strtrim(objective, 5) == 'rank:') {\n stop(\"\\n\\tAutomatic generation of CV-folds is not implemented for ranking without 'group' field!\\n\",\n \"\\tConsider providing pre-computed CV-folds through the 'folds=' parameter.\\n\")\n }\n # shuffle\n rnd_idx <- sample.int(nrows)\n if (stratified && length(label) == length(rnd_idx)) {\n y <- label[rnd_idx]\n # - For classification, need to convert y labels to factor before making the folds,\n # and then do stratification by factor levels.\n # - For regression, leave y numeric and do stratification by quantiles.\n if (is.character(objective)) {\n y <- convert.labels(y, objective)\n }\n folds <- xgb.createFolds(y = y, k = nfold)\n } else {\n # make simple non-stratified folds\n kstep <- length(rnd_idx) %/% nfold\n folds <- list()\n for (i in seq_len(nfold - 1)) {\n folds[[i]] <- rnd_idx[seq_len(kstep)]\n rnd_idx <- rnd_idx[-seq_len(kstep)]\n }\n folds[[nfold]] <- rnd_idx\n }\n return(folds)\n}\n\ngenerate.group.folds <- function(nfold, group) {\n ngroups <- length(group) - 1\n if (ngroups < nfold) {\n stop(\"DMatrix has fewer groups than folds.\")\n }\n seq_groups <- seq_len(ngroups)\n indices <- lapply(seq_groups, function(gr) seq(group[gr] + 1, group[gr + 1]))\n assignments <- base::split(seq_groups, as.integer(seq_groups %% nfold))\n assignments <- unname(assignments)\n\n out <- vector(\"list\", nfold)\n randomized_groups <- sample(ngroups)\n for (idx in seq_len(nfold)) {\n groups_idx_test <- randomized_groups[assignments[[idx]]]\n groups_test <- indices[groups_idx_test]\n idx_test <- unlist(groups_test)\n attributes(idx_test)$group_test <- lengths(groups_test)\n attributes(idx_test)$group_train <- lengths(indices[-groups_idx_test])\n out[[idx]] <- idx_test\n }\n return(out)\n}\n\n# Creates CV folds stratified by the values of y.\n# It was borrowed from caret::createFolds and simplified\n# by always returning an unnamed list of fold indices.\nxgb.createFolds <- function(y, k) {\n if (is.numeric(y)) {\n ## Group the numeric data based on their magnitudes\n ## and sample within those groups.\n\n ## When the number of samples is low, we may have\n ## issues further slicing the numeric data into\n ## groups. The number of groups will depend on the\n ## ratio of the number of folds to the sample size.\n ## At most, we will use quantiles. If the sample\n ## is too small, we just do regular unstratified\n ## CV\n cuts <- floor(length(y) / k)\n if (cuts < 2) cuts <- 2\n if (cuts > 5) cuts <- 5\n y <- cut(y,\n unique(stats::quantile(y, probs = seq(0, 1, length = cuts))),\n include.lowest = TRUE)\n }\n\n if (k < length(y)) {\n ## reset levels so that the possible levels and\n ## the levels in the vector are the same\n y <- factor(as.character(y))\n numInClass <- table(y)\n foldVector <- vector(mode = \"integer\", length(y))\n\n ## For each class, balance the fold allocation as far\n ## as possible, then resample the remainder.\n ## The final assignment of folds is also randomized.\n for (i in seq_along(numInClass)) {\n ## create a vector of integers from 1:k as many times as possible without\n ## going over the number of samples in the class. Note that if the number\n ## of samples in a class is less than k, nothing is produced here.\n seqVector <- rep(seq_len(k), numInClass[i] %/% k)\n ## add enough random integers to get length(seqVector) == numInClass[i]\n if (numInClass[i] %% k > 0) seqVector <- c(seqVector, sample.int(k, numInClass[i] %% k))\n ## shuffle the integers for fold assignment and assign to this classes's data\n ## seqVector[sample.int(length(seqVector))] is used to handle length(seqVector) == 1\n foldVector[y == dimnames(numInClass)$y[i]] <- seqVector[sample.int(length(seqVector))]\n }\n } else {\n foldVector <- seq(along = y)\n }\n\n out <- split(seq(along = y), foldVector)\n names(out) <- NULL\n out\n}\n\n#' Model Serialization and Compatibility\n#'\n#' @description\n#' When it comes to serializing XGBoost models, it's possible to use R serializers such as\n#' [save()] or [saveRDS()] to serialize an XGBoost model object, but XGBoost also provides\n#' its own serializers with better compatibility guarantees, which allow loading\n#' said models in other language bindings of XGBoost.\n#'\n#' Note that an `xgb.Booster` object (**as produced by [xgb.train()]**, see rest of the doc\n#' for objects produced by [xgboost()]), outside of its core components, might also keep:\n#' - Additional model configuration (accessible through [xgb.config()]), which includes\n#' model fitting parameters like `max_depth` and runtime parameters like `nthread`.\n#' These are not necessarily useful for prediction/importance/plotting.\n#' - Additional R specific attributes - e.g. results of callbacks, such as evaluation logs,\n#' which are kept as a `data.table` object, accessible through\n#' `attributes(model)$evaluation_log` if present.\n#'\n#' The first one (configurations) does not have the same compatibility guarantees as\n#' the model itself, including attributes that are set and accessed through\n#' [xgb.attributes()] - that is, such configuration might be lost after loading the\n#' booster in a different XGBoost version, regardless of the serializer that was used.\n#' These are saved when using [saveRDS()], but will be discarded if loaded into an\n#' incompatible XGBoost version. They are not saved when using XGBoost's\n#' serializers from its public interface including [xgb.save()] and [xgb.save.raw()].\n#'\n#' The second ones (R attributes) are not part of the standard XGBoost model structure,\n#' and thus are not saved when using XGBoost's own serializers. These attributes are\n#' only used for informational purposes, such as keeping track of evaluation metrics as\n#' the model was fit, or saving the R call that produced the model, but are otherwise\n#' not used for prediction / importance / plotting / etc.\n#' These R attributes are only preserved when using R's serializers.\n#'\n#' In addition to the regular `xgb.Booster` objects produced by [xgb.train()], the\n#' function [xgboost()] produces objects with a different subclass `xgboost` (which\n#' inherits from `xgb.Booster`), which keeps other additional metadata as R attributes\n#' such as class names in classification problems, and which has a dedicated `predict`\n#' method that uses different defaults and takes different argument names. XGBoost's\n#' own serializers can work with this `xgboost` class, but as they do not keep R\n#' attributes, the resulting object, when deserialized, is downcasted to the regular\n#' `xgb.Booster` class (i.e. it loses the metadata, and the resulting object will use\n#' [predict.xgb.Booster()] instead of [predict.xgboost()]) - for these `xgboost` objects,\n#' `saveRDS` might thus be a better option if the extra functionalities are needed.\n#'\n#' Note that XGBoost models in R starting from version `2.1.0` and onwards, and\n#' XGBoost models before version `2.1.0`; have a very different R object structure and\n#' are incompatible with each other. Hence, models that were saved with R serializers\n#' like [saveRDS()] or [save()] before version `2.1.0` will not work with latter\n#' `xgboost` versions and vice versa. Be aware that the structure of R model objects\n#' could in theory change again in the future, so XGBoost's serializers should be\n#' preferred for long-term storage.\n#'\n#' Furthermore, note that model objects from XGBoost might not be serializable with third-party\n#' R packages like `qs` or `qs2`.\n#'\n#' @details\n#' Use [xgb.save()] to save the XGBoost model as a stand-alone file. You may opt into\n#' the JSON format by specifying the JSON extension. To read the model back, use\n#' [xgb.load()].\n#'\n#' Use [xgb.save.raw()] to save the XGBoost model as a sequence (vector) of raw bytes\n#' in a future-proof manner. Future releases of XGBoost will be able to read the raw bytes and\n#' re-construct the corresponding model. To read the model back, use [xgb.load.raw()].\n#' The [xgb.save.raw()] function is useful if you would like to persist the XGBoost model\n#' as part of another R object.\n#'\n#' Use [saveRDS()] if you require the R-specific attributes that a booster might have, such\n#' as evaluation logs or the model class `xgboost` instead of `xgb.Booster`, but note that\n#' future compatibility of such objects is outside XGBoost's control as it relies on R's\n#' serialization format (see e.g. the details section in [serialize] and [save()] from base R).\n#'\n#' For more details and explanation about model persistence and archival, consult the page\n#' \\url{https://xgboost.readthedocs.io/en/latest/tutorials/saving_model.html}.\n#'\n#' @examples\n#' \\dontshow{RhpcBLASctl::omp_set_num_threads(1)}\n#' data(agaricus.train, package = \"xgboost\")\n#'\n#' bst <- xgb.train(\n#' data = xgb.DMatrix(agaricus.train$data, label = agaricus.train$label, nthread = 1),\n#' nrounds = 2,\n#' params = xgb.params(\n#' max_depth = 2,\n#' nthread = 2,\n#' objective = \"binary:logistic\"\n#' )\n#' )\n#'\n#' # Save as a stand-alone file; load it with xgb.load()\n#' fname <- file.path(tempdir(), \"xgb_model.ubj\")\n#' xgb.save(bst, fname)\n#' bst2 <- xgb.load(fname)\n#'\n#' # Save as a stand-alone file (JSON); load it with xgb.load()\n#' fname <- file.path(tempdir(), \"xgb_model.json\")\n#' xgb.save(bst, fname)\n#' bst2 <- xgb.load(fname)\n#'\n#' # Save as a raw byte vector; load it with xgb.load.raw()\n#' xgb_bytes <- xgb.save.raw(bst)\n#' bst2 <- xgb.load.raw(xgb_bytes)\n#'\n#' # Persist XGBoost model as part of another R object\n#' obj <- list(xgb_model_bytes = xgb.save.raw(bst), description = \"My first XGBoost model\")\n#' # Persist the R object. Here, saveRDS() is okay, since it doesn't persist\n#' # xgb.Booster directly. What's being persisted is the future-proof byte representation\n#' # as given by xgb.save.raw().\n#' fname <- file.path(tempdir(), \"my_object.Rds\")\n#' saveRDS(obj, fname)\n#' # Read back the R object\n#' obj2 <- readRDS(fname)\n#' # Re-construct xgb.Booster object from the bytes\n#' bst2 <- xgb.load.raw(obj2$xgb_model_bytes)\n#'\n#' @name a-compatibility-note-for-saveRDS-save\nNULL\n\n#' @name xgboost-options\n#' @title XGBoost Options\n#' @description XGBoost offers an \\link[base:options]{option setting} for controlling the behavior\n#' of deprecated and removed function arguments.\n#'\n#' Some of the arguments in functions like [xgb.train()] or [predict.xgb.Booster()] been renamed\n#' from how they were in previous versions, or have been removed.\n#'\n#' In order to make the transition to newer XGBoost versions easier, some of these parameters are\n#' still accepted but issue a warning when using them. \\bold{Note that these warnings will become\n#' errors in the future!!} - this is just a temporary workaround to make the transition easier.\n#'\n#' One can optionally use 'strict mode' to turn these warnings into errors, in order to ensure\n#' that code calling xgboost will still work once those are removed in future releases.\n#'\n#' Currently, the only supported option is `xgboost.strict_mode`, which can be set to `TRUE` or\n#' `FALSE` (default).\n#'\n#' In addition to an R option, it can also be enabled through by setting environment variable\n#' `XGB_STRICT_MODE=1`. If set, this environment variable will take precedence over the option.\n#' @examples\n#' options(\"xgboost.strict_mode\" = FALSE)\n#' options(\"xgboost.strict_mode\" = TRUE)\n#' Sys.setenv(\"XGB_STRICT_MODE\" = \"1\")\n#' Sys.setenv(\"XGB_STRICT_MODE\" = \"0\")\nNULL\n\nget.strict.mode.option <- function() {\n env_var_option <- Sys.getenv(\"XGB_STRICT_MODE\")\n if (!nchar(env_var_option)) {\n return(getOption(\"xgboost.strict_mode\", default = FALSE))\n }\n return(tolower(as.character(env_var_option)) %in% c(\"1\", \"true\", \"t\", \"yes\", \"y\"))\n}\n\n# Lookup table for the deprecated parameters bookkeeping\ndeprecated_train_params <- list(\n renamed = list(\n 'print.every.n' = 'print_every_n',\n 'early.stop.round' = 'early_stopping_rounds',\n 'training.data' = 'data',\n 'dtrain' = 'data',\n 'watchlist' = 'evals',\n 'feval' = 'custom_metric'\n ),\n removed = character()\n)\ndeprecated_cv_params <- deprecated_train_params\ndeprecated_cv_params$removed <- 'label'\ndeprecated_xgboost_params <- list(\n renamed = list(\n 'data' = 'x',\n 'label' = 'y',\n 'eta' = 'learning_rate',\n 'gamma' = 'min_split_loss',\n 'lambda' = 'reg_lambda',\n 'alpha' = 'reg_alpha',\n 'min.split.loss' = 'min_split_loss',\n 'reg.lambda' = 'reg_lambda',\n 'reg.alpha' = 'reg_alpha',\n 'watchlist' = 'evals'\n ),\n removed = c(\n 'params',\n 'save_period',\n 'save_name',\n 'xgb_model',\n 'callbacks',\n 'missing',\n 'maximize'\n )\n)\ndeprecated_dttree_params <- list(\n renamed = list('n_first_tree' = 'trees'),\n removed = c(\"feature_names\", \"text\")\n)\ndeprecated_plotimp_params <- list(\n renamed = list(\n 'plot.height' = 'plot_height',\n 'plot.width' = 'plot_width'\n ),\n removed = character()\n)\ndeprecated_multitrees_params <- list(\n renamed = c(\n deprecated_plotimp_params$renamed,\n list('features.keep' = 'features_keep')\n ),\n removed = \"feature_names\"\n)\ndeprecated_dump_params <- list(\n renamed = list('with.stats' = 'with_stats'),\n removed = character()\n)\ndeprecated_plottree_params <- c(\n renamed = list(\n deprecated_plotimp_params$renamed,\n deprecated_dump_params$renamed,\n list('trees' = 'tree_idx')\n ),\n removed = c(\"show_node_id\", \"feature_names\")\n)\ndeprecated_predict_params <- list(\n renamed = list(\"ntreelimit\" = \"iterationrange\"),\n removed = \"reshape\"\n)\ndeprecated_dmatrix_params <- list(\n renamed = character(),\n removed = \"info\"\n)\n\n# These got moved from 'info' to function arguments\nargs_previous_dmatrix_info <- c(\"label\", \"weight\", \"base_margin\", \"group\")\n\n# Checks the dot-parameters for deprecated names\n# (including partial matching), gives a deprecation warning,\n# and sets new parameters to the old parameters' values within its parent frame.\n# WARNING: has side-effects\ncheck.deprecation <- function(\n deprecated_list,\n fn_call,\n ...,\n env = parent.frame(),\n allow_unrecognized = FALSE\n) {\n params <- list(...)\n if (length(params) == 0) {\n return(NULL)\n }\n error_on_deprecated <- get.strict.mode.option()\n throw_err_or_depr_msg <- function(...) {\n if (error_on_deprecated) {\n stop(...)\n } else {\n warning(..., \" This warning will become an error in a future version.\")\n }\n }\n\n if (is.null(names(params)) || min(nchar(names(params))) == 0L) {\n throw_err_or_depr_msg(\"Passed invalid positional arguments\")\n }\n list_renamed <- deprecated_list$renamed\n list_removed <- deprecated_list$removed\n has_params_arg <-\n length(list_renamed) == length(deprecated_train_params$renamed) &&\n list_renamed[[1L]] == deprecated_train_params$renamed[[1L]]\n is_dmatrix_constructor <-\n length(list_removed) == length(deprecated_dmatrix_params$removed) &&\n list_removed[[1L]] == deprecated_dmatrix_params$removed[[1L]]\n all_match <- pmatch(names(params), names(list_renamed))\n # throw error on unrecognized parameters\n if (!allow_unrecognized && anyNA(all_match)) {\n\n names_unrecognized <- names(params)[is.na(all_match)]\n # make it informative if they match something that goes under 'params'\n if (has_params_arg) {\n names_params <- formalArgs(xgb.params)\n names_params <- c(names_params, gsub(\"_\", \".\", names_params, fixed = TRUE))\n names_under_params <- intersect(names_unrecognized, names_params)\n if (length(names_under_params)) {\n if (error_on_deprecated) {\n stop(\n \"Passed invalid function arguments: \",\n paste(head(names_under_params), collapse = \", \"),\n \". These should be passed as a list to argument 'params'.\"\n )\n } else {\n warning(\n \"Passed invalid function arguments: \",\n paste(head(names_under_params), collapse = \", \"),\n \". These should be passed as a list to argument 'params'.\",\n \" Conversion from argument to 'params' entry will be done automatically, but this \",\n \"behavior will become an error in a future version.\"\n )\n if (any(names_under_params %in% names(env[[\"params\"]]))) {\n repeteated_params <- intersect(names_under_params, names(env[[\"params\"]]))\n stop(\n \"Passed entries as both function argument(s) and as elements under 'params': \",\n paste(head(repeteated_params), collapse = \", \")\n )\n } else {\n env[[\"params\"]] <- c(env[[\"params\"]], params[names_under_params])\n }\n }\n names_unrecognized <- setdiff(names_unrecognized, names_under_params)\n }\n } else if (is_dmatrix_constructor && NROW(params$info)) {\n # same thing for the earlier 'info' in 'xgb.DMatrix'\n throw_err_or_depr_msg(\n \"Passed invalid argument 'info' - entries on it should be passed as direct arguments.\"\n )\n entries_info <- names(params$info)\n if (length(setdiff(entries_info, args_previous_dmatrix_info))) {\n stop(\n \"Passed unrecognized entries under info: \",\n paste(setdiff(entries_info, args_previous_dmatrix_info) |> head(), collapse = \", \")\n )\n }\n for (entry_name in entries_info) {\n if (!is.null(env[[entry_name]])) {\n stop(\"Passed entry under both 'info' and function argument(s): \", entry_name)\n }\n env[[entry_name]] <- params$info[[entry_name]]\n }\n names_unrecognized <- setdiff(names_unrecognized, \"info\")\n }\n\n # check for parameters that were removed from a previous version\n names_removed <- intersect(names_unrecognized, list_removed)\n if (length(names_removed)) {\n throw_err_or_depr_msg(\n \"Parameter(s) have been removed from this function: \",\n paste(names_removed, collapse = \", \"), \".\"\n )\n names_unrecognized <- setdiff(names_unrecognized, list_removed)\n }\n\n # otherwise throw a generic error\n if (length(names_unrecognized)) {\n throw_err_or_depr_msg(\n \"Passed unrecognized parameters: \",\n paste(head(names_unrecognized), collapse = \", \"), \".\"\n )\n }\n\n } else {\n\n names_removed <- intersect(names(params)[is.na(all_match)], list_removed)\n if (length(names_removed)) {\n throw_err_or_depr_msg(\n \"Parameter(s) have been removed from this function: \",\n paste(names_removed, collapse = \", \"), \".\"\n )\n }\n\n }\n\n matched_params <- list_renamed[all_match[!is.na(all_match)]]\n idx_orig <- seq_along(params)[!is.na(all_match)]\n function_args_passed <- names(as.list(fn_call))[-1L]\n for (idx in seq_along(matched_params)) {\n match_old <- names(matched_params)[[idx]]\n match_new <- matched_params[[idx]]\n throw_err_or_depr_msg(\n \"Parameter '\", match_old, \"' has been renamed to '\",\n match_new, \"'.\"\n )\n if (match_new %in% function_args_passed) {\n stop(\"Passed both '\", match_new, \"' and '\", match_old, \"'.\")\n }\n env[[match_new]] <- params[[idx_orig[idx]]]\n }\n}\n" }, { "path": "R-package/R/xgb.Booster.R", "content": "# Construct an internal XGBoost Booster and get its current number of rounds.\n# internal utility function\n# Note: the number of rounds in the C booster gets reset to zero when changing\n# key booster parameters like 'process_type=update', but in some cases, when\n# replacing previous iterations, it needs to make a check that the new number\n# of iterations doesn't exceed the previous ones, hence it keeps track of the\n# current number of iterations before resetting the parameters in order to\n# perform the check later on.\nxgb.Booster <- function(params, cachelist, modelfile) {\n if (typeof(cachelist) != \"list\" ||\n !all(vapply(cachelist, inherits, logical(1), what = 'xgb.DMatrix'))) {\n stop(\"cachelist must be a list of xgb.DMatrix objects\")\n }\n ## Load existing model, dispatch for on disk model file and in memory buffer\n if (!is.null(modelfile)) {\n if (is.character(modelfile)) {\n ## A filename\n bst <- .Call(XGBoosterCreate_R, cachelist)\n modelfile <- path.expand(modelfile)\n .Call(XGBoosterLoadModel_R, xgb.get.handle(bst), enc2utf8(modelfile[1]))\n niter <- xgb.get.num.boosted.rounds(bst)\n if (length(params) > 0) {\n xgb.model.parameters(bst) <- params\n }\n return(list(bst = bst, niter = niter))\n } else if (is.raw(modelfile)) {\n ## A memory buffer\n bst <- xgb.load.raw(modelfile)\n niter <- xgb.get.num.boosted.rounds(bst)\n xgb.model.parameters(bst) <- params\n return(list(bst = bst, niter = niter))\n } else if (inherits(modelfile, \"xgb.Booster\")) {\n ## A booster object\n bst <- .Call(XGDuplicate_R, modelfile)\n niter <- xgb.get.num.boosted.rounds(bst)\n xgb.model.parameters(bst) <- params\n return(list(bst = bst, niter = niter))\n } else {\n stop(\"modelfile must be either character filename, or raw booster dump, or xgb.Booster object\")\n }\n }\n ## Create new model\n bst <- .Call(XGBoosterCreate_R, cachelist)\n if (length(params) > 0) {\n xgb.model.parameters(bst) <- params\n }\n return(list(bst = bst, niter = 0L))\n}\n\n# Check whether xgb.Booster handle is null\n# internal utility function\nis.null.handle <- function(handle) {\n if (is.null(handle)) return(TRUE)\n\n if (!inherits(handle, \"externalptr\"))\n stop(\"argument type must be 'externalptr'\")\n\n return(.Call(XGCheckNullPtr_R, handle))\n}\n\n# Return a verified to be valid handle out of xgb.Booster\n# internal utility function\nxgb.get.handle <- function(object) {\n if (inherits(object, \"xgb.Booster\")) {\n handle <- object$ptr\n if (is.null(handle) || !inherits(handle, \"externalptr\")) {\n stop(\"'xgb.Booster' object is corrupted or is from an incompatible XGBoost version.\")\n }\n } else {\n stop(\"argument must be an 'xgb.Booster' object.\")\n }\n if (is.null.handle(handle)) {\n stop(\"invalid 'xgb.Booster' (blank 'externalptr').\")\n }\n return(handle)\n}\n\n#' Predict method for XGBoost model\n#'\n#' Predict values on data based on XGBoost model.\n#'\n#' @param object Object of class `xgb.Booster`.\n#' @param newdata Takes `data.frame`, `matrix`, `dgCMatrix`, `dgRMatrix`, `dsparseVector`,\n#' local data file, or `xgb.DMatrix`.\n#'\n#' For single-row predictions on sparse data, it is recommended to use CSR format. If passing\n#' a sparse vector, it will take it as a row vector.\n#'\n#' Note that, for repeated predictions on the same data, one might want to create a DMatrix to\n#' pass here instead of passing R types like matrices or data frames, as predictions will be\n#' faster on DMatrix.\n#'\n#' If `newdata` is a `data.frame`, be aware that:\n#' - Columns will be converted to numeric if they aren't already, which could potentially make\n#' the operation slower than in an equivalent `matrix` object.\n#' - The order of the columns must match with that of the data from which the model was fitted\n#' (i.e. columns will not be referenced by their names, just by their order in the data),\n#' unless passing `validate_features = TRUE` (which is not the default).\n#' - If the model was fitted to data with categorical columns, these columns must be of\n#' `factor` type here, and must use the same encoding (i.e. have the same levels).\n#' - If `newdata` contains any `factor` columns, they will be converted to base-0\n#' encoding (same as during DMatrix creation) - hence, one should not pass a `factor`\n#' under a column which during training had a different type.\n#' - Any columns with type other than `factor` will be interpreted as numeric.\n#' @param missing Float value that represents missing values in data\n#' (e.g., 0 or some other extreme value).\n#'\n#' This parameter is not used when `newdata` is an `xgb.DMatrix` - in such cases,\n#' should pass this as an argument to the DMatrix constructor instead.\n#' @param outputmargin Whether the prediction should be returned in the form of\n#' original untransformed sum of predictions from boosting iterations' results.\n#' E.g., setting `outputmargin = TRUE` for logistic regression would return log-odds\n#' instead of probabilities.\n#' @param predleaf Whether to predict per-tree leaf indices.\n#' @param predcontrib Whether to return feature contributions to individual predictions (see Details).\n#' @param approxcontrib Whether to use a fast approximation for feature contributions (see Details).\n#' @param predinteraction Whether to return contributions of feature interactions to individual predictions (see Details).\n#' @param training Whether the prediction result is used for training. When enabled,\n#' XGBoost uses the training prediction path instead of inplace prediction.\n#' @param iterationrange Sequence of rounds/iterations from the model to use for prediction, specified by passing\n#' a two-dimensional vector with the start and end numbers in the sequence (same format as R's `seq` - i.e.\n#' base-1 indexing, and inclusive of both ends).\n#'\n#' For example, passing `c(1,20)` will predict using the first twenty iterations, while passing `c(1,1)` will\n#' predict using only the first one.\n#'\n#' If passing `NULL`, will either stop at the best iteration if the model used early stopping, or use all\n#' of the iterations (rounds) otherwise.\n#'\n#' If passing \"all\", will use all of the rounds regardless of whether the model had early stopping or not.\n#'\n#' Not applicable to `gblinear` booster.\n#' @param strict_shape Whether to always return an array with the same dimensions for the given prediction mode\n#' regardless of the model type - meaning that, for example, both a multi-class and a binary classification\n#' model would generate output arrays with the same number of dimensions, with the 'class' dimension having\n#' size equal to '1' for the binary model.\n#'\n#' If passing `FALSE` (the default), dimensions will be simplified according to the model type, so that a\n#' binary classification model for example would not have a redundant dimension for 'class'.\n#'\n#' See documentation for the return type for the exact shape of the output arrays for each prediction mode.\n#' @param avoid_transpose Whether to output the resulting predictions in the same memory layout in which they\n#' are generated by the core XGBoost library, without transposing them to match the expected output shape.\n#'\n#' Internally, XGBoost uses row-major order for the predictions it generates, while R arrays use column-major\n#' order, hence the result needs to be transposed in order to have the expected shape when represented as\n#' an R array or matrix, which might be a slow operation.\n#'\n#' If passing `TRUE`, then the result will have dimensions in reverse order - for example, rows\n#' will be the last dimensions instead of the first dimension.\n#' @param base_margin Base margin used for boosting from existing model (raw score that gets added to\n#' all observations independently of the trees in the model).\n#'\n#' If supplied, should be either a vector with length equal to the number of rows in `newdata`\n#' (for objectives which produces a single score per observation), or a matrix with number of\n#' rows matching to the number rows in `newdata` and number of columns matching to the number\n#' of scores estimated by the model (e.g. number of classes for multi-class classification).\n#'\n#' Note that, if `newdata` is an `xgb.DMatrix` object, this argument will\n#' be ignored as it needs to be added to the DMatrix instead (e.g. by passing it as\n#' an argument in its constructor, or by calling [setinfo.xgb.DMatrix()].\n#' @param validate_features When `TRUE`, validate that the Booster's and newdata's\n#' feature_names match (only applicable when both `object` and `newdata` have feature names).\n#'\n#' If the column names differ and `newdata` is not an `xgb.DMatrix`, will try to reorder\n#' the columns in `newdata` to match with the booster's.\n#'\n#' If the booster has feature types and `newdata` is either an `xgb.DMatrix` or\n#' `data.frame`, will additionally verify that categorical columns are of the\n#' correct type in `newdata`, throwing an error if they do not match.\n#'\n#' If passing `FALSE`, it is assumed that the feature names and types are the same,\n#' and come in the same order as in the training data.\n#'\n#' Note that this check might add some sizable latency to the predictions, so it's\n#' recommended to disable it for performance-sensitive applications.\n#' @param ... Not used.\n#'\n#' @details\n#' Note that `iterationrange` would currently do nothing for predictions from \"gblinear\",\n#' since \"gblinear\" doesn't keep its boosting history.\n#'\n#' One possible practical applications of the `predleaf` option is to use the model\n#' as a generator of new features which capture non-linearity and interactions,\n#' e.g., as implemented in [xgb.create.features()].\n#'\n#' Setting `predcontrib = TRUE` allows to calculate contributions of each feature to\n#' individual predictions. For \"gblinear\" booster, feature contributions are simply linear terms\n#' (feature_beta * feature_value). For \"gbtree\" booster, feature contributions are SHAP\n#' values (Lundberg 2017) that sum to the difference between the expected output\n#' of the model and the current prediction (where the hessian weights are used to compute the expectations).\n#' Setting `approxcontrib = TRUE` approximates these values following the idea explained\n#' in \\url{http://blog.datadive.net/interpreting-random-forests/}.\n#'\n#' With `predinteraction = TRUE`, SHAP values of contributions of interaction of each pair of features\n#' are computed. Note that this operation might be rather expensive in terms of compute and memory.\n#' Since it quadratically depends on the number of features, it is recommended to perform selection\n#' of the most important features first. See below about the format of the returned results.\n#'\n#' The `predict()` method uses as many threads as defined in `xgb.Booster` object (all by default).\n#' If you want to change their number, assign a new number to `nthread` using [xgb.model.parameters<-()].\n#' Note that converting a matrix to [xgb.DMatrix()] uses multiple threads too.\n#'\n#' @return\n#' A numeric vector or array, with corresponding dimensions depending on the prediction mode and on\n#' parameter `strict_shape` as follows:\n#'\n#' If passing `strict_shape=FALSE`:\\itemize{\n#' \\item For regression or binary classification: a vector of length `nrows`.\n#' \\item For multi-class and multi-target objectives: a matrix of dimensions `[nrows, ngroups]`.\n#'\n#' Note that objective variant `multi:softmax` defaults towards predicting most likely class (a vector\n#' `nrows`) instead of per-class probabilities.\n#' \\item For `predleaf`: a matrix with one column per tree.\n#'\n#' For multi-class / multi-target, they will be arranged so that columns in the output will have\n#' the leafs from one group followed by leafs of the other group (e.g. order will be `group1:feat1`,\n#' `group1:feat2`, ..., `group2:feat1`, `group2:feat2`, ...).\n#'\n#' If there is more than one parallel tree (e.g. random forests), the parallel trees will be the\n#' last grouping in the resulting order, which will still be 2D.\n#' \\item For `predcontrib`: when not multi-class / multi-target, a matrix with dimensions\n#' `[nrows, nfeats+1]`. The last \"+ 1\" column corresponds to the baseline value.\n#'\n#' For multi-class and multi-target objectives, will be an array with dimensions `[nrows, ngroups, nfeats+1]`.\n#'\n#' The contribution values are on the scale of untransformed margin (e.g., for binary classification,\n#' the values are log-odds deviations from the baseline).\n#' \\item For `predinteraction`: when not multi-class / multi-target, the output is a 3D array of\n#' dimensions `[nrows, nfeats+1, nfeats+1]`. The off-diagonal (in the last two dimensions)\n#' elements represent different feature interaction contributions. The array is symmetric w.r.t. the last\n#' two dimensions. The \"+ 1\" columns corresponds to the baselines. Summing this array along the last\n#' dimension should produce practically the same result as `predcontrib = TRUE`.\n#'\n#' For multi-class and multi-target, will be a 4D array with dimensions `[nrows, ngroups, nfeats+1, nfeats+1]`\n#' }\n#'\n#' If passing `strict_shape=TRUE`, the result is always a matrix (if 2D) or array (if 3D or higher):\n#' - For normal predictions, the dimension is `[nrows, ngroups]`.\n#' - For `predcontrib=TRUE`, the dimension is `[nrows, ngroups, nfeats+1]`.\n#' - For `predinteraction=TRUE`, the dimension is `[nrows, ngroups, nfeats+1, nfeats+1]`.\n#' - For `predleaf=TRUE`, the dimension is `[nrows, niter, ngroups, num_parallel_tree]`.\n#'\n#' If passing `avoid_transpose=TRUE`, then the dimensions in all cases will be in reverse order - for\n#' example, for `predinteraction`, they will be `[nfeats+1, nfeats+1, ngroups, nrows]`\n#' instead of `[nrows, ngroups, nfeats+1, nfeats+1]`.\n#' @seealso [xgb.train()]\n#' @references\n#' 1. Scott M. Lundberg, Su-In Lee, \"A Unified Approach to Interpreting Model Predictions\",\n#' NIPS Proceedings 2017, \\url{https://arxiv.org/abs/1705.07874}\n#' 2. Scott M. Lundberg, Su-In Lee, \"Consistent feature attribution for tree ensembles\",\n#' \\url{https://arxiv.org/abs/1706.06060}\n#'\n#' @examples\n#' \\dontshow{RhpcBLASctl::omp_set_num_threads(1)}\n#' ## binary classification:\n#'\n#' data(agaricus.train, package = \"xgboost\")\n#' data(agaricus.test, package = \"xgboost\")\n#'\n#' ## Keep the number of threads to 2 for examples\n#' nthread <- 2\n#' data.table::setDTthreads(nthread)\n#'\n#' train <- agaricus.train\n#' test <- agaricus.test\n#'\n#' bst <- xgb.train(\n#' data = xgb.DMatrix(train$data, label = train$label, nthread = 1),\n#' nrounds = 5,\n#' params = xgb.params(\n#' max_depth = 2,\n#' nthread = nthread,\n#' objective = \"binary:logistic\"\n#' )\n#' )\n#'\n#' # use all trees by default\n#' pred <- predict(bst, test$data)\n#' # use only the 1st tree\n#' pred1 <- predict(bst, test$data, iterationrange = c(1, 1))\n#'\n#' # Predicting tree leafs:\n#' # the result is an nsamples X ntrees matrix\n#' pred_leaf <- predict(bst, test$data, predleaf = TRUE)\n#' str(pred_leaf)\n#'\n#' # Predicting feature contributions to predictions:\n#' # the result is an nsamples X (nfeatures + 1) matrix\n#' pred_contr <- predict(bst, test$data, predcontrib = TRUE)\n#' str(pred_contr)\n#' # verify that contributions' sums are equal to log-odds of predictions (up to float precision):\n#' summary(rowSums(pred_contr) - qlogis(pred))\n#' # for the 1st record, let's inspect its features that had non-zero contribution to prediction:\n#' contr1 <- pred_contr[1,]\n#' contr1 <- contr1[-length(contr1)] # drop intercept\n#' contr1 <- contr1[contr1 != 0] # drop non-contributing features\n#' contr1 <- contr1[order(abs(contr1))] # order by contribution magnitude\n#' old_mar <- par(\"mar\")\n#' par(mar = old_mar + c(0,7,0,0))\n#' barplot(contr1, horiz = TRUE, las = 2, xlab = \"contribution to prediction in log-odds\")\n#' par(mar = old_mar)\n#'\n#'\n#' ## multiclass classification in iris dataset:\n#'\n#' lb <- as.numeric(iris$Species) - 1\n#' num_class <- 3\n#'\n#' set.seed(11)\n#'\n#' bst <- xgb.train(\n#' data = xgb.DMatrix(as.matrix(iris[, -5], nthread = 1), label = lb),\n#' nrounds = 10,\n#' params = xgb.params(\n#' max_depth = 4,\n#' nthread = 2,\n#' subsample = 0.5,\n#' objective = \"multi:softprob\",\n#' num_class = num_class\n#' )\n#' )\n#'\n#' # predict for softmax returns num_class probability numbers per case:\n#' pred <- predict(bst, as.matrix(iris[, -5]))\n#' str(pred)\n#' # convert the probabilities to softmax labels\n#' pred_labels <- max.col(pred) - 1\n#' # the following should result in the same error as seen in the last iteration\n#' sum(pred_labels != lb) / length(lb)\n#'\n#' # compare with predictions from softmax:\n#' set.seed(11)\n#'\n#' bst <- xgb.train(\n#' data = xgb.DMatrix(as.matrix(iris[, -5], nthread = 1), label = lb),\n#' nrounds = 10,\n#' params = xgb.params(\n#' max_depth = 4,\n#' nthread = 2,\n#' subsample = 0.5,\n#' objective = \"multi:softmax\",\n#' num_class = num_class\n#' )\n#' )\n#'\n#' pred <- predict(bst, as.matrix(iris[, -5]))\n#' str(pred)\n#' all.equal(pred, pred_labels)\n#' # prediction from using only 5 iterations should result\n#' # in the same error as seen in iteration 5:\n#' pred5 <- predict(bst, as.matrix(iris[, -5]), iterationrange = c(1, 5))\n#' sum(pred5 != lb) / length(lb)\n#'\n#' @export\npredict.xgb.Booster <- function(object, newdata, missing = NA, outputmargin = FALSE,\n predleaf = FALSE, predcontrib = FALSE, approxcontrib = FALSE, predinteraction = FALSE,\n training = FALSE, iterationrange = NULL, strict_shape = FALSE, avoid_transpose = FALSE,\n validate_features = FALSE, base_margin = NULL, ...) {\n check.deprecation(deprecated_predict_params, match.call(), ..., allow_unrecognized = TRUE)\n if (validate_features) {\n newdata <- validate.features(object, newdata)\n }\n is_dmatrix <- inherits(newdata, \"xgb.DMatrix\")\n if (is_dmatrix && !is.null(base_margin)) {\n stop(\n \"'base_margin' is not supported when passing 'xgb.DMatrix' as input.\",\n \" Should be passed as argument to 'xgb.DMatrix' constructor.\"\n )\n }\n if (is_dmatrix) {\n rnames <- NULL\n } else {\n rnames <- row.names(newdata)\n }\n\n use_as_df <- FALSE\n use_as_dense_matrix <- FALSE\n use_as_csr_matrix <- FALSE\n n_row <- NULL\n if (!is_dmatrix) {\n\n inplace_predict_supported <- !predcontrib && !predinteraction && !predleaf\n if (inplace_predict_supported) {\n booster_type <- xgb.booster_type(object)\n if (booster_type == \"gblinear\" || training) {\n inplace_predict_supported <- FALSE\n }\n }\n if (inplace_predict_supported) {\n\n if (is.matrix(newdata)) {\n use_as_dense_matrix <- TRUE\n } else if (is.data.frame(newdata)) {\n # note: since here it turns it into a non-data-frame list,\n # needs to keep track of the number of rows it had for later\n n_row <- nrow(newdata)\n newdata <- lapply(\n newdata,\n function(x) {\n if (is.factor(x)) {\n return(as.numeric(x) - 1)\n } else {\n return(as.numeric(x))\n }\n }\n )\n use_as_df <- TRUE\n } else if (inherits(newdata, \"dgRMatrix\")) {\n use_as_csr_matrix <- TRUE\n csr_data <- list(newdata@p, newdata@j, newdata@x, ncol(newdata))\n } else if (inherits(newdata, \"dsparseVector\")) {\n use_as_csr_matrix <- TRUE\n n_row <- 1L\n i <- newdata@i - 1L\n if (storage.mode(i) != \"integer\") {\n storage.mode(i) <- \"integer\"\n }\n csr_data <- list(c(0L, length(i)), i, newdata@x, length(newdata))\n }\n\n }\n\n } # if (!is_dmatrix)\n\n if (!is_dmatrix && !use_as_dense_matrix && !use_as_csr_matrix && !use_as_df) {\n nthread <- xgb.nthread(object)\n newdata <- xgb.DMatrix(\n newdata,\n missing = missing,\n base_margin = base_margin,\n nthread = NVL(nthread, -1)\n )\n is_dmatrix <- TRUE\n }\n\n if (is.null(n_row)) {\n n_row <- nrow(newdata)\n }\n\n\n if (!is.null(iterationrange)) {\n if (is.character(iterationrange)) {\n stopifnot(iterationrange == \"all\")\n iterationrange <- c(0, 0)\n } else {\n iterationrange[1] <- iterationrange[1] - 1 # base-0 indexing\n }\n } else {\n ## no limit is supplied, use best\n best_iteration <- xgb.best_iteration(object)\n if (is.null(best_iteration)) {\n iterationrange <- c(0, 0)\n } else {\n iterationrange <- c(0, as.integer(best_iteration) + 1L)\n }\n }\n ## Handle the 0 length values.\n box <- function(val) {\n if (length(val) == 0) {\n cval <- vector(, 1)\n cval[0] <- val\n return(cval)\n }\n return(val)\n }\n\n args <- list(\n training = box(training),\n strict_shape = as.logical(strict_shape),\n iteration_begin = box(as.integer(iterationrange[1])),\n iteration_end = box(as.integer(iterationrange[2])),\n type = box(as.integer(0))\n )\n\n set_type <- function(type) {\n if (args$type != 0) {\n stop(\"One type of prediction at a time.\")\n }\n return(box(as.integer(type)))\n }\n if (outputmargin) {\n args$type <- set_type(1)\n }\n if (predcontrib) {\n args$type <- set_type(if (approxcontrib) 3 else 2)\n }\n if (predinteraction) {\n args$type <- set_type(if (approxcontrib) 5 else 4)\n }\n if (predleaf) {\n args$type <- set_type(6)\n }\n\n json_conf <- jsonlite::toJSON(args, auto_unbox = TRUE)\n if (is_dmatrix) {\n arr <- .Call(\n XGBoosterPredictFromDMatrix_R, xgb.get.handle(object), newdata, json_conf\n )\n } else if (use_as_dense_matrix) {\n arr <- .Call(\n XGBoosterPredictFromDense_R, xgb.get.handle(object), newdata, missing, json_conf, base_margin\n )\n } else if (use_as_csr_matrix) {\n arr <- .Call(\n XGBoosterPredictFromCSR_R, xgb.get.handle(object), csr_data, missing, json_conf, base_margin\n )\n } else if (use_as_df) {\n arr <- .Call(\n XGBoosterPredictFromColumnar_R, xgb.get.handle(object), newdata, missing, json_conf, base_margin\n )\n }\n\n ## Needed regardless of whether strict shape is being used.\n if ((predcontrib || predinteraction) && !is.null(colnames(newdata))) {\n cnames <- c(colnames(newdata), \"(Intercept)\")\n dim_names <- vector(mode = \"list\", length = length(dim(arr)))\n dim_names[[1L]] <- cnames\n if (predinteraction) dim_names[[2L]] <- cnames\n .Call(XGSetArrayDimNamesInplace_R, arr, dim_names)\n }\n\n if (NROW(rnames)) {\n if (is.null(dim(arr))) {\n .Call(XGSetVectorNamesInplace_R, arr, rnames)\n } else {\n dim_names <- dimnames(arr)\n if (is.null(dim_names)) {\n dim_names <- vector(mode = \"list\", length = length(dim(arr)))\n }\n dim_names[[length(dim_names)]] <- rnames\n .Call(XGSetArrayDimNamesInplace_R, arr, dim_names)\n }\n }\n\n if (!avoid_transpose && is.array(arr)) {\n arr <- aperm(arr)\n }\n\n return(arr)\n}\n\nvalidate.features <- function(bst, newdata) {\n if (is.character(newdata)) {\n # this will be encountered when passing file paths\n return(newdata)\n }\n if (inherits(newdata, \"sparseVector\")) {\n # in this case, newdata won't have metadata\n return(newdata)\n }\n if (is.vector(newdata)) {\n newdata <- as.matrix(newdata)\n }\n\n booster_names <- getinfo(bst, \"feature_name\")\n checked_names <- FALSE\n if (NROW(booster_names)) {\n\n try_reorder <- FALSE\n if (inherits(newdata, \"xgb.DMatrix\")) {\n curr_names <- getinfo(newdata, \"feature_name\")\n } else {\n curr_names <- colnames(newdata)\n try_reorder <- TRUE\n }\n\n if (NROW(curr_names)) {\n checked_names <- TRUE\n\n if (length(curr_names) != length(booster_names) || any(curr_names != booster_names)) {\n\n if (!try_reorder) {\n stop(\"Feature names in 'newdata' do not match with booster's.\")\n } else {\n if (inherits(newdata, \"data.table\")) {\n newdata <- newdata[, booster_names, with = FALSE]\n } else {\n newdata <- newdata[, booster_names, drop = FALSE]\n }\n }\n\n }\n\n } # if (NROW(curr_names)) {\n\n } # if (NROW(booster_names)) {\n\n if (inherits(newdata, c(\"data.frame\", \"xgb.DMatrix\"))) {\n\n booster_types <- getinfo(bst, \"feature_type\")\n if (!NROW(booster_types)) {\n # Note: types in the booster are optional. Other interfaces\n # might not even save it as booster attributes for example,\n # even if the model uses categorical features.\n return(newdata)\n }\n if (inherits(newdata, \"xgb.DMatrix\")) {\n curr_types <- getinfo(newdata, \"feature_type\")\n if (length(curr_types) != length(booster_types) || any(curr_types != booster_types)) {\n stop(\"Feature types in 'newdata' do not match with booster's.\")\n }\n }\n if (inherits(newdata, \"data.frame\")) {\n is_factor <- sapply(newdata, is.factor)\n if (any(is_factor != (booster_types == \"c\"))) {\n stop(\n paste0(\n \"Feature types in 'newdata' do not match with booster's for same columns (by \",\n ifelse(checked_names, \"name\", \"position\"),\n \").\"\n )\n )\n }\n }\n\n }\n\n return(newdata)\n}\n\n\n#' Accessors for serializable attributes of a model\n#'\n#' These methods allow to manipulate the key-value attribute strings of an XGBoost model.\n#'\n#' @details\n#' The primary purpose of XGBoost model attributes is to store some meta data about the model.\n#' Note that they are a separate concept from the object attributes in R.\n#' Specifically, they refer to key-value strings that can be attached to an XGBoost model,\n#' stored together with the model's binary representation, and accessed later\n#' (from R or any other interface).\n#' In contrast, any R attribute assigned to an R object of `xgb.Booster` class\n#' would not be saved by [xgb.save()] because an XGBoost model is an external memory object\n#' and its serialization is handled externally.\n#' Also, setting an attribute that has the same name as one of XGBoost's parameters wouldn't\n#' change the value of that parameter for a model.\n#' Use [xgb.model.parameters<-()] to set or change model parameters.\n#'\n#' The `xgb.attributes<-` setter either updates the existing or adds one or several attributes,\n#' but it doesn't delete the other existing attributes.\n#'\n#' Important: since this modifies the booster's C object, semantics for assignment here\n#' will differ from R's, as any object reference to the same booster will be modified\n#' too, while assignment of R attributes through `attributes(model)$ <- `\n#' will follow the usual copy-on-write R semantics (see [xgb.copy.Booster()] for an\n#' example of these behaviors).\n#'\n#' @param object Object of class `xgb.Booster`. **Will be modified in-place** when assigning to it.\n#' @param name A non-empty character string specifying which attribute is to be accessed.\n#' @param value For `xgb.attr<-`, a value of an attribute; for `xgb.attributes<-`,\n#' it is a list (or an object coercible to a list) with the names of attributes to set\n#' and the elements corresponding to attribute values.\n#' Non-character values are converted to character.\n#' When an attribute value is not a scalar, only the first index is used.\n#' Use `NULL` to remove an attribute.\n#' @return\n#' - `xgb.attr()` returns either a string value of an attribute\n#' or `NULL` if an attribute wasn't stored in a model.\n#' - `xgb.attributes()` returns a list of all attributes stored in a model\n#' or `NULL` if a model has no stored attributes.\n#'\n#' @examples\n#' data(agaricus.train, package = \"xgboost\")\n#' train <- agaricus.train\n#'\n#' bst <- xgb.train(\n#' data = xgb.DMatrix(train$data, label = train$label, nthread = 1),\n#' nrounds = 2,\n#' params = xgb.params(\n#' max_depth = 2,\n#' nthread = 2,\n#' objective = \"binary:logistic\"\n#' )\n#' )\n#'\n#' xgb.attr(bst, \"my_attribute\") <- \"my attribute value\"\n#' print(xgb.attr(bst, \"my_attribute\"))\n#' xgb.attributes(bst) <- list(a = 123, b = \"abc\")\n#'\n#' fname <- file.path(tempdir(), \"xgb.ubj\")\n#' xgb.save(bst, fname)\n#' bst1 <- xgb.load(fname)\n#' print(xgb.attr(bst1, \"my_attribute\"))\n#' print(xgb.attributes(bst1))\n#'\n#' # deletion:\n#' xgb.attr(bst1, \"my_attribute\") <- NULL\n#' print(xgb.attributes(bst1))\n#' xgb.attributes(bst1) <- list(a = NULL, b = NULL)\n#' print(xgb.attributes(bst1))\n#'\n#' @rdname xgb.attr\n#' @export\nxgb.attr <- function(object, name) {\n if (is.null(name) || nchar(as.character(name[1])) == 0) stop(\"invalid attribute name\")\n handle <- xgb.get.handle(object)\n out <- .Call(XGBoosterGetAttr_R, handle, as.character(name[1]))\n if (!NROW(out) || !nchar(out)) {\n return(NULL)\n }\n if (!is.null(out)) {\n if (name %in% c(\"best_iteration\", \"best_score\")) {\n out <- as.numeric(out)\n }\n }\n return(out)\n}\n\n#' @rdname xgb.attr\n#' @export\n`xgb.attr<-` <- function(object, name, value) {\n name <- as.character(name[1])\n if (!NROW(name) || !nchar(name)) stop(\"invalid attribute name\")\n handle <- xgb.get.handle(object)\n\n if (!is.null(value)) {\n # Coerce the elements to be scalar strings.\n # Q: should we warn user about non-scalar elements?\n if (is.numeric(value[1])) {\n value <- format(value[1], digits = 17)\n } else {\n value <- as.character(value[1])\n }\n }\n .Call(XGBoosterSetAttr_R, handle, name, value)\n return(object)\n}\n\n#' @rdname xgb.attr\n#' @export\nxgb.attributes <- function(object) {\n handle <- xgb.get.handle(object)\n attr_names <- .Call(XGBoosterGetAttrNames_R, handle)\n if (!NROW(attr_names)) return(list())\n out <- lapply(attr_names, function(name) xgb.attr(object, name))\n names(out) <- attr_names\n return(out)\n}\n\n#' @rdname xgb.attr\n#' @export\n`xgb.attributes<-` <- function(object, value) {\n a <- as.list(value)\n if (is.null(names(a)) || any(nchar(names(a)) == 0)) {\n stop(\"attribute names cannot be empty strings\")\n }\n for (i in seq_along(a)) {\n xgb.attr(object, names(a[i])) <- a[[i]]\n }\n return(object)\n}\n\n#' Accessors for model parameters as JSON string\n#'\n#' @details\n#' Note that assignment is performed in-place on the booster C object, which unlike assignment\n#' of R attributes, doesn't follow typical copy-on-write semantics for assignment - i.e. all references\n#' to the same booster will also get updated.\n#'\n#' See [xgb.copy.Booster()] for an example of this behavior.\n#'\n#' @param object Object of class `xgb.Booster`.**Will be modified in-place** when assigning to it.\n#' @param value A list.\n#' @return Parameters as a list.\n#' @examples\n#' data(agaricus.train, package = \"xgboost\")\n#'\n#' ## Keep the number of threads to 1 for examples\n#' nthread <- 1\n#' data.table::setDTthreads(nthread)\n#' train <- agaricus.train\n#'\n#' bst <- xgb.train(\n#' data = xgb.DMatrix(train$data, label = train$label, nthread = 1),\n#' nrounds = 2,\n#' params = xgb.params(\n#' max_depth = 2,\n#' nthread = nthread,\n#' objective = \"binary:logistic\"\n#' )\n#' )\n#'\n#' config <- xgb.config(bst)\n#'\n#' @rdname xgb.config\n#' @export\nxgb.config <- function(object) {\n handle <- xgb.get.handle(object)\n return(jsonlite::fromJSON(.Call(XGBoosterSaveJsonConfig_R, handle)))\n}\n\n#' @rdname xgb.config\n#' @export\n`xgb.config<-` <- function(object, value) {\n handle <- xgb.get.handle(object)\n .Call(\n XGBoosterLoadJsonConfig_R,\n handle,\n jsonlite::toJSON(value, auto_unbox = TRUE, null = \"null\")\n )\n return(object)\n}\n\n#' Accessors for model parameters\n#'\n#' Only the setter for XGBoost parameters is currently implemented.\n#'\n#' @details\n#' Just like [xgb.attr()], this function will make in-place modifications\n#' on the booster object which do not follow typical R assignment semantics - that is,\n#' all references to the same booster will also be updated, unlike assingment of R\n#' attributes which follow copy-on-write semantics.\n#'\n#' See [xgb.copy.Booster()] for an example of this behavior.\n#'\n#' Be aware that setting parameters of a fitted booster related to training continuation / updates\n#' will reset its number of rounds indicator to zero.\n#' @param object Object of class `xgb.Booster`. **Will be modified in-place**.\n#' @param value A list (or an object coercible to a list) with the names of parameters to set\n#' and the elements corresponding to parameter values.\n#' @return The same booster `object`, which gets modified in-place.\n#' @examples\n#' data(agaricus.train, package = \"xgboost\")\n#'\n#' train <- agaricus.train\n#'\n#' bst <- xgb.train(\n#' data = xgb.DMatrix(train$data, label = train$label, nthread = 1),\n#' nrounds = 2,\n#' params = xgb.params(\n#' max_depth = 2,\n#' learning_rate = 1,\n#' nthread = 2,\n#' objective = \"binary:logistic\"\n#' )\n#' )\n#'\n#' xgb.model.parameters(bst) <- list(learning_rate = 0.1)\n#'\n#' @rdname xgb.model.parameters\n#' @export\n`xgb.model.parameters<-` <- function(object, value) {\n if (length(value) == 0) return(object)\n p <- as.list(value)\n if (is.null(names(p)) || any(nchar(names(p)) == 0)) {\n stop(\"parameter names cannot be empty strings\")\n }\n names(p) <- gsub(\".\", \"_\", names(p), fixed = TRUE)\n p <- lapply(p, function(x) {\n if (is.vector(x) && length(x) == 1) {\n return(as.character(x)[1])\n } else {\n return(jsonlite::toJSON(x, auto_unbox = TRUE))\n }\n })\n handle <- xgb.get.handle(object)\n for (i in seq_along(p)) {\n .Call(XGBoosterSetParam_R, handle, names(p[i]), p[[i]])\n }\n return(object)\n}\n\n#' @rdname getinfo\n#' @export\ngetinfo.xgb.Booster <- function(object, name) {\n name <- as.character(head(name, 1L))\n allowed_fields <- c(\"feature_name\", \"feature_type\")\n if (!(name %in% allowed_fields)) {\n stop(\"getinfo: name must be one of the following: \", paste(allowed_fields, collapse = \", \"))\n }\n handle <- xgb.get.handle(object)\n out <- .Call(\n XGBoosterGetStrFeatureInfo_R,\n handle,\n name\n )\n if (!NROW(out)) {\n return(NULL)\n }\n return(out)\n}\n\n#' @rdname getinfo\n#' @export\nsetinfo.xgb.Booster <- function(object, name, info) {\n name <- as.character(head(name, 1L))\n allowed_fields <- c(\"feature_name\", \"feature_type\")\n if (!(name %in% allowed_fields)) {\n stop(\"setinfo: unknown info name \", name)\n }\n info <- as.character(info)\n handle <- xgb.get.handle(object)\n .Call(\n XGBoosterSetStrFeatureInfo_R,\n handle,\n name,\n info\n )\n return(TRUE)\n}\n\n#' Get number of boosting in a fitted booster\n#'\n#' @param model,x A fitted `xgb.Booster` model.\n#' @return The number of rounds saved in the model as an integer.\n#' @details Note that setting booster parameters related to training\n#' continuation / updates through [xgb.model.parameters<-()] will reset the\n#' number of rounds to zero.\n#' @export\n#' @rdname xgb.get.num.boosted.rounds\nxgb.get.num.boosted.rounds <- function(model) {\n return(.Call(XGBoosterBoostedRounds_R, xgb.get.handle(model)))\n}\n\n#' @rdname xgb.get.num.boosted.rounds\n#' @export\nlength.xgb.Booster <- function(x) {\n return(xgb.get.num.boosted.rounds(x))\n}\n\n#' Slice Booster by Rounds\n#'\n#' Creates a new booster including only a selected range of rounds / iterations\n#' from an existing booster, as given by the sequence `seq(start, end, step)`.\n#'\n#' @details\n#' Note that any R attributes that the booster might have, will not be copied into\n#' the resulting object.\n#'\n#' @param model,x A fitted `xgb.Booster` object, which is to be sliced by taking only a subset\n#' of its rounds / iterations.\n#' @param start Start of the slice (base-1 and inclusive, like R's [seq()]).\n#' @param end End of the slice (base-1 and inclusive, like R's [seq()]).\n#' Passing a value of zero here is equivalent to passing the full number of rounds in the\n#' booster object.\n#' @param step Step size of the slice. Passing '1' will take every round in the sequence defined by\n#' `(start, end)`, while passing '2' will take every second value, and so on.\n#' @return A sliced booster object containing only the requested rounds.\n#' @examples\n#' data(mtcars)\n#'\n#' y <- mtcars$mpg\n#' x <- as.matrix(mtcars[, -1])\n#'\n#' dm <- xgb.DMatrix(x, label = y, nthread = 1)\n#' model <- xgb.train(data = dm, params = xgb.params(nthread = 1), nrounds = 5)\n#' model_slice <- xgb.slice.Booster(model, 1, 3)\n#' # Prediction for first three rounds\n#' predict(model, x, predleaf = TRUE)[, 1:3]\n#'\n#' # The new model has only those rounds, so\n#' # a full prediction from it is equivalent\n#' predict(model_slice, x, predleaf = TRUE)\n#' @export\n#' @rdname xgb.slice.Booster\nxgb.slice.Booster <- function(model, start, end = xgb.get.num.boosted.rounds(model), step = 1L) {\n # This makes the slice mimic the behavior of R's 'seq',\n # which truncates on the end of the slice when the step\n # doesn't reach it.\n if (end > start && step > 1) {\n d <- (end - start + 1) / step\n if (d != floor(d)) {\n end <- start + step * ceiling(d) - 1\n }\n }\n return(\n .Call(\n XGBoosterSlice_R,\n xgb.get.handle(model),\n start - 1,\n end,\n step\n )\n )\n}\n\n#' @export\n#' @rdname xgb.slice.Booster\n#' @param i The indices - must be an increasing sequence as generated by e.g. `seq(...)`.\n`[.xgb.Booster` <- function(x, i) {\n if (missing(i)) {\n return(xgb.slice.Booster(x, 1, 0))\n }\n if (length(i) == 1) {\n return(xgb.slice.Booster(x, i, i))\n }\n steps <- diff(i)\n if (any(steps < 0)) {\n stop(\"Can only slice booster with ascending sequences.\")\n }\n if (length(unique(steps)) > 1) {\n stop(\"Can only slice booster with fixed-step sequences.\")\n }\n return(xgb.slice.Booster(x, i[1L], i[length(i)], steps[1L]))\n}\n\n#' Get Features Names from Booster\n#'\n#' @description\n#' Returns the feature / variable / column names from a fitted\n#' booster object, which are set automatically during the call to [xgb.train()]\n#' from the DMatrix names, or which can be set manually through [setinfo()].\n#'\n#' If the object doesn't have feature names, will return `NULL`.\n#'\n#' It is equivalent to calling `getinfo(object, \"feature_name\")`.\n#' @param object An `xgb.Booster` object.\n#' @param ... Not used.\n#' @export\nvariable.names.xgb.Booster <- function(object, ...) {\n return(getinfo(object, \"feature_name\"))\n}\n\nxgb.nthread <- function(bst) {\n config <- xgb.config(bst)\n out <- strtoi(config$learner$generic_param$nthread)\n return(out)\n}\n\nxgb.booster_type <- function(bst) {\n config <- xgb.config(bst)\n out <- config$learner$learner_train_param$booster\n return(out)\n}\n\nxgb.num_class <- function(bst) {\n config <- xgb.config(bst)\n out <- strtoi(config$learner$learner_model_param$num_class)\n return(out)\n}\n\nxgb.feature_names <- function(bst) {\n return(getinfo(bst, \"feature_name\"))\n}\n\nxgb.feature_types <- function(bst) {\n return(getinfo(bst, \"feature_type\"))\n}\n\nxgb.num_feature <- function(bst) {\n handle <- xgb.get.handle(bst)\n return(.Call(XGBoosterGetNumFeature_R, handle))\n}\n\nxgb.best_iteration <- function(bst) {\n out <- xgb.attr(bst, \"best_iteration\")\n if (!NROW(out) || !nchar(out)) {\n out <- NULL\n }\n return(out)\n}\n\nxgb.has_categ_features <- function(bst) {\n return(\"c\" %in% xgb.feature_types(bst))\n}\n\n#' Extract coefficients from linear booster\n#'\n#' @description\n#' Extracts the coefficients from a 'gblinear' booster object,\n#' as produced by [xgb.train()] when using parameter `booster=\"gblinear\"`.\n#'\n#' Note: this function will error out if passing a booster model\n#' which is not of \"gblinear\" type.\n#'\n#' @param object A fitted booster of 'gblinear' type.\n#' @param ... Not used.\n#' @return The extracted coefficients:\n#' - If there is only one coefficient per column in the data, will be returned as a\n#' vector, potentially containing the feature names if available, with the intercept\n#' as first column.\n#' - If there is more than one coefficient per column in the data (e.g. when using\n#' `objective=\"multi:softmax\"`), will be returned as a matrix with dimensions equal\n#' to `[num_features, num_cols]`, with the intercepts as first row. Note that the column\n#' (classes in multi-class classification) dimension will not be named.\n#'\n#' The intercept returned here will include the 'base_score' parameter (unlike the 'bias'\n#' or the last coefficient in the model dump, which doesn't have 'base_score' added to it),\n#' hence one should get the same values from calling `predict(..., outputmargin = TRUE)` and\n#' from performing a matrix multiplication with `model.matrix(~., ...)`.\n#'\n#' Be aware that the coefficients are obtained by first converting them to strings and\n#' back, so there will always be some very small lose of precision compared to the actual\n#' coefficients as used by [predict.xgb.Booster].\n#' @examples\n#' library(xgboost)\n#'\n#' data(mtcars)\n#'\n#' y <- mtcars[, 1]\n#' x <- as.matrix(mtcars[, -1])\n#'\n#' dm <- xgb.DMatrix(data = x, label = y, nthread = 1)\n#' params <- xgb.params(booster = \"gblinear\", nthread = 1)\n#' model <- xgb.train(data = dm, params = params, nrounds = 2)\n#' coef(model)\n#' @export\ncoef.xgb.Booster <- function(object, ...) {\n return(.internal.coef.xgb.Booster(object, add_names = TRUE))\n}\n\n.internal.coef.xgb.Booster <- function(object, add_names = TRUE) {\n booster_type <- xgb.booster_type(object)\n if (booster_type != \"gblinear\") {\n stop(\"Coefficients are not defined for Booster type \", booster_type)\n }\n model_json <- jsonlite::fromJSON(\n rawToChar(xgb.save.raw(object, raw_format = \"json\"))\n )\n num_feature <- as.numeric(model_json$learner$learner_model_param$num_feature)\n\n weights <- model_json$learner$gradient_booster$model$weights\n n_cols <- length(weights) / (num_feature + 1)\n if (n_cols != floor(n_cols) || n_cols < 1) {\n stop(\"Internal error: could not determine shape of coefficients.\")\n }\n sep <- num_feature * n_cols\n coefs <- weights[seq(1, sep)]\n intercepts <- weights[seq(sep + 1, length(weights))]\n base_score <- jsonlite::fromJSON(\n model_json$learner$learner_model_param$base_score\n )\n intercepts <- intercepts + as.numeric(base_score)\n\n if (add_names) {\n feature_names <- xgb.feature_names(object)\n if (!NROW(feature_names)) {\n # This mimics the default naming in R which names columns as \"V1..N\"\n # when names are needed but not available\n feature_names <- paste0(\"V\", seq(1L, num_feature))\n }\n feature_names <- c(\"(Intercept)\", feature_names)\n }\n if (n_cols == 1L) {\n out <- c(intercepts, coefs)\n if (add_names) {\n .Call(XGSetVectorNamesInplace_R, out, feature_names)\n }\n } else {\n coefs <- matrix(coefs, nrow = num_feature, byrow = TRUE)\n dim(intercepts) <- c(1L, n_cols)\n out <- rbind(intercepts, coefs)\n out_names <- vector(mode = \"list\", length = 2)\n if (add_names) {\n out_names[[1L]] <- feature_names\n }\n if (inherits(object, \"xgboost\")) {\n metadata <- attributes(object)$metadata\n if (NROW(metadata$y_levels)) {\n out_names[[2L]] <- metadata$y_levels\n } else if (NROW(metadata$y_names)) {\n out_names[[2L]] <- metadata$y_names\n }\n }\n .Call(XGSetArrayDimNamesInplace_R, out, out_names)\n }\n return(out)\n}\n\n#' Deep-copies a Booster Object\n#'\n#' Creates a deep copy of an 'xgb.Booster' object, such that the\n#' C object pointer contained will be a different object, and hence functions\n#' like [xgb.attr()] will not affect the object from which it was copied.\n#'\n#' @param model An 'xgb.Booster' object.\n#' @return A deep copy of `model` - it will be identical in every way, but C-level\n#' functions called on that copy will not affect the `model` variable.\n#' @examples\n#' library(xgboost)\n#'\n#' data(mtcars)\n#'\n#' y <- mtcars$mpg\n#' x <- mtcars[, -1]\n#'\n#' dm <- xgb.DMatrix(x, label = y, nthread = 1)\n#'\n#' model <- xgb.train(\n#' data = dm,\n#' params = xgb.params(nthread = 1),\n#' nrounds = 3\n#' )\n#'\n#' # Set an arbitrary attribute kept at the C level\n#' xgb.attr(model, \"my_attr\") <- 100\n#' print(xgb.attr(model, \"my_attr\"))\n#'\n#' # Just assigning to a new variable will not create\n#' # a deep copy - C object pointer is shared, and in-place\n#' # modifications will affect both objects\n#' model_shallow_copy <- model\n#' xgb.attr(model_shallow_copy, \"my_attr\") <- 333\n#' # 'model' was also affected by this change:\n#' print(xgb.attr(model, \"my_attr\"))\n#'\n#' model_deep_copy <- xgb.copy.Booster(model)\n#' xgb.attr(model_deep_copy, \"my_attr\") <- 444\n#' # 'model' was NOT affected by this change\n#' # (keeps previous value that was assigned before)\n#' print(xgb.attr(model, \"my_attr\"))\n#'\n#' # Verify that the new object was actually modified\n#' print(xgb.attr(model_deep_copy, \"my_attr\"))\n#' @export\nxgb.copy.Booster <- function(model) {\n if (!inherits(model, \"xgb.Booster\")) {\n stop(\"'model' must be an 'xgb.Booster' object.\")\n }\n return(.Call(XGDuplicate_R, model))\n}\n\nxgb.reset.Booster <- function(model) {\n if (!inherits(model, \"xgb.Booster\")) {\n stop(\"'model' must be an 'xgb.Booster' object.\")\n }\n .Call(XGBoosterReset_R, xgb.get.handle(model))\n return(model)\n}\n\n#' Check if two boosters share the same C object\n#'\n#' Checks whether two booster objects refer to the same underlying C object.\n#'\n#' @details\n#' As booster objects (as returned by e.g. [xgb.train()]) contain an R 'externalptr'\n#' object, they don't follow typical copy-on-write semantics of other R objects - that is, if\n#' one assigns a booster to a different variable and modifies that new variable through in-place\n#' methods like [xgb.attr<-()], the modification will be applied to both the old and the new\n#' variable, unlike typical R assignments which would only modify the latter.\n#'\n#' This function allows checking whether two booster objects share the same 'externalptr',\n#' regardless of the R attributes that they might have.\n#'\n#' In order to duplicate a booster in such a way that the copy wouldn't share the same\n#' 'externalptr', one can use function [xgb.copy.Booster()].\n#' @param obj1 Booster model to compare with `obj2`.\n#' @param obj2 Booster model to compare with `obj1`.\n#' @return Either `TRUE` or `FALSE` according to whether the two boosters share the\n#' underlying C object.\n#' @seealso [xgb.copy.Booster()]\n#' @examples\n#' library(xgboost)\n#'\n#' data(mtcars)\n#'\n#' y <- mtcars$mpg\n#' x <- as.matrix(mtcars[, -1])\n#'\n#' model <- xgb.train(\n#' params = xgb.params(nthread = 1),\n#' data = xgb.DMatrix(x, label = y, nthread = 1),\n#' nrounds = 3\n#' )\n#'\n#' model_shallow_copy <- model\n#' xgb.is.same.Booster(model, model_shallow_copy) # same C object\n#'\n#' model_deep_copy <- xgb.copy.Booster(model)\n#' xgb.is.same.Booster(model, model_deep_copy) # different C objects\n#'\n#' # In-place assignments modify all references,\n#' # but not full/deep copies of the booster\n#' xgb.attr(model_shallow_copy, \"my_attr\") <- 111\n#' xgb.attr(model, \"my_attr\") # gets modified\n#' xgb.attr(model_deep_copy, \"my_attr\") # doesn't get modified\n#' @export\nxgb.is.same.Booster <- function(obj1, obj2) {\n if (!inherits(obj1, \"xgb.Booster\") || !inherits(obj2, \"xgb.Booster\")) {\n stop(\"'xgb.is.same.Booster' is only applicable to 'xgb.Booster' objects.\")\n }\n return(\n .Call(\n XGPointerEqComparison_R,\n xgb.get.handle(obj1),\n xgb.get.handle(obj2)\n )\n )\n}\n\n#' @title Print xgb.Booster\n#' @description Print information about `xgb.Booster`.\n#' @param x An `xgb.Booster` object.\n#' @param ... Not used.\n#' @return The same `x` object, returned invisibly\n#' @examples\n#' data(agaricus.train, package = \"xgboost\")\n#' train <- agaricus.train\n#'\n#' bst <- xgb.train(\n#' data = xgb.DMatrix(train$data, label = train$label, nthread = 1),\n#' nrounds = 2,\n#' params = xgb.params(\n#' max_depth = 2,\n#' nthread = 2,\n#' objective = \"binary:logistic\"\n#' )\n#' )\n#'\n#' attr(bst, \"myattr\") <- \"memo\"\n#'\n#' print(bst)\n#' @method print xgb.Booster\n#' @export\nprint.xgb.Booster <- function(x, ...) {\n # this lets it error out when the object comes from an earlier R XGBoost version\n handle <- xgb.get.handle(x)\n cat('##### xgb.Booster\\n')\n\n R_attrs <- attributes(x)\n if (!is.null(R_attrs$call)) {\n cat('call:\\n ')\n print(R_attrs$call)\n }\n\n cat('# of features:', xgb.num_feature(x), '\\n')\n cat('# of rounds: ', xgb.get.num.boosted.rounds(x), '\\n')\n\n attr_names <- .Call(XGBoosterGetAttrNames_R, handle)\n if (NROW(attr_names)) {\n cat('xgb.attributes:\\n')\n cat(\" \", paste(attr_names, collapse = \", \"), \"\\n\")\n }\n\n additional_attr <- setdiff(names(R_attrs), .reserved_cb_names)\n if (NROW(additional_attr)) {\n cat(\"callbacks:\\n \", paste(additional_attr, collapse = \", \"), \"\\n\")\n }\n\n if (!is.null(R_attrs$evaluation_log)) {\n cat('evaluation_log:\\n')\n print(R_attrs$evaluation_log, row.names = FALSE, topn = 2)\n }\n\n return(invisible(x))\n}\n" }, { "path": "R-package/R/xgb.DMatrix.R", "content": "#' Construct xgb.DMatrix object\n#'\n#' Construct an 'xgb.DMatrix' object from a given data source, which can then be passed to functions\n#' such as [xgb.train()] or [predict()].\n#'\n#' Function `xgb.QuantileDMatrix()` will construct a DMatrix with quantization for the histogram\n#' method already applied to it, which can be used to reduce memory usage (compared to using a\n#' a regular DMatrix first and then creating a quantization out of it) when using the histogram\n#' method (`tree_method = \"hist\"`, which is the default algorithm), but is not usable for the\n#' sorted-indices method (`tree_method = \"exact\"`), nor for the approximate method\n#' (`tree_method = \"approx\"`).\n#'\n#' @param data Data from which to create a DMatrix, which can then be used for fitting models or\n#' for getting predictions out of a fitted model.\n#'\n#' Supported input types are as follows:\n#' - `matrix` objects, with types `numeric`, `integer`, or `logical`.\n#' - `data.frame` objects, with columns of types `numeric`, `integer`, `logical`, or `factor`\n#'\n#' Note that xgboost uses base-0 encoding for categorical types, hence `factor` types (which use base-1\n#' encoding') will be converted inside the function call. Be aware that the encoding used for `factor`\n#' types is not kept as part of the model, so in subsequent calls to `predict`, it is the user's\n#' responsibility to ensure that factor columns have the same levels as the ones from which the DMatrix\n#' was constructed.\n#'\n#' Other column types are not supported.\n#' - CSR matrices, as class `dgRMatrix` from package `Matrix`.\n#' - CSC matrices, as class `dgCMatrix` from package `Matrix`.\n#'\n#' These are **not** supported by `xgb.QuantileDMatrix`.\n#' - XGBoost's own binary format for DMatrices, as produced by [xgb.DMatrix.save()].\n#' - Single-row CSR matrices, as class `dsparseVector` from package `Matrix`, which is interpreted\n#' as a single row (only when making predictions from a fitted model).\n#'\n#' @param label Label of the training data. For classification problems, should be passed encoded as\n#' integers with numeration starting at zero.\n#' @param weight Weight for each instance.\n#'\n#' Note that, for ranking task, weights are per-group. In ranking task, one weight\n#' is assigned to each group (not each data point). This is because we\n#' only care about the relative ordering of data points within each group,\n#' so it doesn't make sense to assign weights to individual data points.\n#' @param base_margin Base margin used for boosting from existing model.\n#'\n#' In the case of multi-output models, one can also pass multi-dimensional base_margin.\n#' @param missing A float value to represents missing values in data (not used when creating DMatrix\n#' from text files). It is useful to change when a zero, infinite, or some other\n#' extreme value represents missing values in data.\n#' @param silent whether to suppress printing an informational message after loading from a file.\n#' @param feature_names Set names for features. Overrides column names in data frame and matrix.\n#'\n#' Note: columns are not referenced by name when calling `predict`, so the column order there\n#' must be the same as in the DMatrix construction, regardless of the column names.\n#' @param feature_types Set types for features.\n#'\n#' If `data` is a `data.frame` and passing `feature_types` is not supplied,\n#' feature types will be deduced automatically from the column types.\n#'\n#' Otherwise, one can pass a character vector with the same length as number of columns in `data`,\n#' with the following possible values:\n#' - \"c\", which represents categorical columns.\n#' - \"q\", which represents numeric columns.\n#' - \"int\", which represents integer columns.\n#' - \"i\", which represents logical (boolean) columns.\n#'\n#' Note that, while categorical types are treated differently from the rest for model fitting\n#' purposes, the other types do not influence the generated model, but have effects in other\n#' functionalities such as feature importances.\n#'\n#' **Important**: Categorical features, if specified manually through `feature_types`, must\n#' be encoded as integers with numeration starting at zero, and the same encoding needs to be\n#' applied when passing data to [predict()]. Even if passing `factor` types, the encoding will\n#' not be saved, so make sure that `factor` columns passed to `predict` have the same `levels`.\n#' @param nthread Number of threads used for creating DMatrix.\n#' @param group Group size for all ranking group.\n#' @param qid Query ID for data samples, used for ranking.\n#' @param label_lower_bound Lower bound for survival training.\n#' @param label_upper_bound Upper bound for survival training.\n#' @param feature_weights Set feature weights for column sampling.\n#' @param data_split_mode Not used yet. This parameter is for distributed training, which is not yet available for the R package.\n#' @inheritParams xgb.train\n#' @return An 'xgb.DMatrix' object. If calling `xgb.QuantileDMatrix`, it will have additional\n#' subclass `xgb.QuantileDMatrix`.\n#'\n#' @details\n#' Note that DMatrix objects are not serializable through R functions such as [saveRDS()] or [save()].\n#' If a DMatrix gets serialized and then de-serialized (for example, when saving data in an R session or caching\n#' chunks in an Rmd file), the resulting object will not be usable anymore and will need to be reconstructed\n#' from the original source of data.\n#'\n#' @examples\n#' data(agaricus.train, package = \"xgboost\")\n#'\n#' ## Keep the number of threads to 1 for examples\n#' nthread <- 1\n#' data.table::setDTthreads(nthread)\n#' dtrain <- with(\n#' agaricus.train, xgb.DMatrix(data, label = label, nthread = nthread)\n#' )\n#' fname <- file.path(tempdir(), \"xgb.DMatrix.data\")\n#' xgb.DMatrix.save(dtrain, fname)\n#' dtrain <- xgb.DMatrix(fname, nthread = 1)\n#' @export\n#' @rdname xgb.DMatrix\nxgb.DMatrix <- function(\n data,\n label = NULL,\n weight = NULL,\n base_margin = NULL,\n missing = NA,\n silent = FALSE,\n feature_names = colnames(data),\n feature_types = NULL,\n nthread = NULL,\n group = NULL,\n qid = NULL,\n label_lower_bound = NULL,\n label_upper_bound = NULL,\n feature_weights = NULL,\n data_split_mode = \"row\",\n ...\n) {\n check.deprecation(deprecated_dmatrix_params, match.call(), ...)\n if (!is.null(group) && !is.null(qid)) {\n stop(\"Either one of 'group' or 'qid' should be NULL\")\n }\n if (data_split_mode != \"row\") {\n stop(\"'data_split_mode' is not supported yet.\")\n }\n nthread <- as.integer(NVL(nthread, -1L))\n if (typeof(data) == \"character\") {\n if (length(data) > 1) {\n stop(\n \"'data' has class 'character' and length \", length(data),\n \".\\n 'data' accepts either a numeric matrix or a single filename.\"\n )\n }\n data <- path.expand(data)\n if (data_split_mode == \"row\") {\n data_split_mode <- 0L\n } else if (data_split_mode == \"col\") {\n data_split_mode <- 1L\n } else {\n stop(\"Passed invalid 'data_split_mode': \", data_split_mode)\n }\n handle <- .Call(XGDMatrixCreateFromURI_R, data, as.integer(silent), data_split_mode)\n } else if (is.matrix(data)) {\n handle <- .Call(\n XGDMatrixCreateFromMat_R, data, missing, nthread\n )\n } else if (inherits(data, \"dgCMatrix\")) {\n handle <- .Call(\n XGDMatrixCreateFromCSC_R,\n data@p,\n data@i,\n data@x,\n nrow(data),\n missing,\n nthread\n )\n } else if (inherits(data, \"dgRMatrix\")) {\n handle <- .Call(\n XGDMatrixCreateFromCSR_R,\n data@p,\n data@j,\n data@x,\n ncol(data),\n missing,\n nthread\n )\n } else if (inherits(data, \"dsparseVector\")) {\n indptr <- c(0L, as.integer(length(data@i)))\n ind <- as.integer(data@i) - 1L\n handle <- .Call(\n XGDMatrixCreateFromCSR_R,\n indptr,\n ind,\n data@x,\n length(data),\n missing,\n nthread\n )\n } else if (is.data.frame(data)) {\n tmp <- .process.df.for.dmatrix(data, feature_types)\n feature_types <- tmp$feature_types\n handle <- .Call(\n XGDMatrixCreateFromDF_R, tmp$lst, missing, nthread\n )\n rm(tmp)\n } else {\n stop(\"xgb.DMatrix does not support construction from \", typeof(data))\n }\n\n dmat <- handle\n attributes(dmat) <- list(\n class = \"xgb.DMatrix\",\n fields = new.env()\n )\n .set.dmatrix.fields(\n dmat = dmat,\n label = label,\n weight = weight,\n base_margin = base_margin,\n feature_names = feature_names,\n feature_types = feature_types,\n group = group,\n qid = qid,\n label_lower_bound = label_lower_bound,\n label_upper_bound = label_upper_bound,\n feature_weights = feature_weights\n )\n\n return(dmat)\n}\n\n.process.df.for.dmatrix <- function(df, feature_types) {\n if (!nrow(df) || !ncol(df)) {\n stop(\"'data' is an empty data.frame.\")\n }\n if (!is.null(feature_types)) {\n if (!is.character(feature_types) || length(feature_types) != ncol(df)) {\n stop(\n \"'feature_types' must be a character vector with one entry per column in 'data'.\"\n )\n }\n } else {\n feature_types <- sapply(df, function(col) {\n if (is.factor(col)) {\n return(\"c\")\n } else if (is.integer(col)) {\n return(\"int\")\n } else if (is.logical(col)) {\n return(\"i\")\n } else {\n if (!is.numeric(col)) {\n stop(\"Invalid type in dataframe.\")\n }\n return(\"float\")\n }\n })\n }\n\n lst <- lapply(df, function(col) {\n is_factor <- is.factor(col)\n col <- as.numeric(col)\n if (is_factor) {\n col <- col - 1\n }\n return(col)\n })\n\n return(list(lst = lst, feature_types = feature_types))\n}\n\n.set.dmatrix.fields <- function(\n dmat,\n label,\n weight,\n base_margin,\n feature_names,\n feature_types,\n group,\n qid,\n label_lower_bound,\n label_upper_bound,\n feature_weights\n) {\n if (!is.null(label)) {\n setinfo(dmat, \"label\", label)\n }\n if (!is.null(weight)) {\n setinfo(dmat, \"weight\", weight)\n }\n if (!is.null(base_margin)) {\n setinfo(dmat, \"base_margin\", base_margin)\n }\n if (!is.null(feature_names)) {\n setinfo(dmat, \"feature_name\", feature_names)\n }\n if (!is.null(feature_types)) {\n setinfo(dmat, \"feature_type\", feature_types)\n }\n if (!is.null(group)) {\n setinfo(dmat, \"group\", group)\n }\n if (!is.null(qid)) {\n setinfo(dmat, \"qid\", qid)\n }\n if (!is.null(label_lower_bound)) {\n setinfo(dmat, \"label_lower_bound\", label_lower_bound)\n }\n if (!is.null(label_upper_bound)) {\n setinfo(dmat, \"label_upper_bound\", label_upper_bound)\n }\n if (!is.null(feature_weights)) {\n setinfo(dmat, \"feature_weights\", feature_weights)\n }\n}\n\n#' @param ref The training dataset that provides quantile information, needed when creating\n#' validation/test dataset with [xgb.QuantileDMatrix()]. Supplying the training DMatrix\n#' as a reference means that the same quantisation applied to the training data is\n#' applied to the validation/test data\n#' @param max_bin The number of histogram bin, should be consistent with the training parameter\n#' `max_bin`.\n#'\n#' This is only supported when constructing a QuantileDMatrix.\n#' @export\n#' @rdname xgb.DMatrix\nxgb.QuantileDMatrix <- function(\n data,\n label = NULL,\n weight = NULL,\n base_margin = NULL,\n missing = NA,\n feature_names = colnames(data),\n feature_types = NULL,\n nthread = NULL,\n group = NULL,\n qid = NULL,\n label_lower_bound = NULL,\n label_upper_bound = NULL,\n feature_weights = NULL,\n ref = NULL,\n max_bin = NULL\n) {\n nthread <- as.integer(NVL(nthread, -1L))\n if (!is.null(ref) && !inherits(ref, \"xgb.DMatrix\")) {\n stop(\"'ref' must be an xgb.DMatrix object.\")\n }\n\n # Note: when passing an integer matrix, it won't get casted to numeric.\n # Since 'int' values as understood by languages like C cannot have missing values,\n # R represents missingness there by assigning them a value equal to the minimum\n # integer. The 'missing' value here is set before the data, so in case of integers,\n # need to make the conversion manually beforehand.\n if (is.matrix(data) && storage.mode(data) %in% c(\"integer\", \"logical\") && is.na(missing)) {\n missing <- .Call(XGGetRNAIntAsDouble)\n }\n\n iterator_env <- as.environment(\n list(\n data = data,\n label = label,\n weight = weight,\n base_margin = base_margin,\n missing = missing,\n feature_names = feature_names,\n feature_types = feature_types,\n group = group,\n qid = qid,\n label_lower_bound = label_lower_bound,\n label_upper_bound = label_upper_bound,\n feature_weights = feature_weights\n )\n )\n data_iterator <- .single.data.iterator(iterator_env)\n\n env_keep_alive <- new.env()\n env_keep_alive$keepalive <- NULL\n\n # Note: the ProxyDMatrix has its finalizer assigned in the R externalptr\n # object, but that finalizer will only be called once the object is\n # garbage-collected, which doesn't happen immediately after it goes out\n # of scope, hence this piece of code to tigger its destruction earlier\n # and free memory right away.\n proxy_handle <- .make.proxy.handle()\n on.exit({\n .Call(XGDMatrixFree_R, proxy_handle)\n })\n iterator_next <- function() {\n return(xgb.ProxyDMatrix(proxy_handle, data_iterator, env_keep_alive))\n }\n iterator_reset <- function() {\n env_keep_alive$keepalive <- NULL\n return(data_iterator$f_reset(iterator_env))\n }\n calling_env <- environment()\n\n dmat <- .Call(\n XGQuantileDMatrixCreateFromCallback_R,\n iterator_next,\n iterator_reset,\n calling_env,\n proxy_handle,\n nthread,\n missing,\n max_bin,\n ref\n )\n attributes(dmat) <- list(\n class = c(\"xgb.DMatrix\", \"xgb.QuantileDMatrix\"),\n fields = attributes(proxy_handle)$fields\n )\n return(dmat)\n}\n\n#' XGBoost Data Iterator\n#'\n#' @description\n#' Interface to create a custom data iterator in order to construct a DMatrix\n#' from external memory.\n#'\n#' This function is responsible for generating an R object structure containing callback\n#' functions and an environment shared with them.\n#'\n#' The output structure from this function is then meant to be passed to [xgb.ExtMemDMatrix()],\n#' which will consume the data and create a DMatrix from it by executing the callback functions.\n#'\n#' For more information, and for a usage example, see the documentation for [xgb.ExtMemDMatrix()].\n#'\n#' @param env An R environment to pass to the callback functions supplied here, which can be\n#' used to keep track of variables to determine how to handle the batches.\n#'\n#' For example, one might want to keep track of an iteration number in this environment in order\n#' to know which part of the data to pass next.\n#' @param f_next `function(env)` which is responsible for:\n#' - Accessing or retrieving the next batch of data in the iterator.\n#' - Supplying this data by calling function [xgb.DataBatch()] on it and returning the result.\n#' - Keeping track of where in the iterator batch it is or will go next, which can for example\n#' be done by modifiying variables in the `env` variable that is passed here.\n#' - Signaling whether there are more batches to be consumed or not, by returning `NULL`\n#' when the stream of data ends (all batches in the iterator have been consumed), or the result from\n#' calling [xgb.DataBatch()] when there are more batches in the line to be consumed.\n#' @param f_reset `function(env)` which is responsible for reseting the data iterator\n#' (i.e. taking it back to the first batch, called before and after the sequence of batches\n#' has been consumed).\n#'\n#' Note that, after resetting the iterator, the batches will be accessed again, so the same data\n#' (and in the same order) must be passed in subsequent iterations.\n#' @return An `xgb.DataIter` object, containing the same inputs supplied here, which can then\n#' be passed to [xgb.ExtMemDMatrix()].\n#' @seealso [xgb.ExtMemDMatrix()], [xgb.DataBatch()].\n#' @export\nxgb.DataIter <- function(env = new.env(), f_next, f_reset) {\n if (!is.function(f_next)) {\n stop(\"'f_next' must be a function.\")\n }\n if (!is.function(f_reset)) {\n stop(\"'f_reset' must be a function.\")\n }\n out <- list(\n env = env,\n f_next = f_next,\n f_reset = f_reset\n )\n class(out) <- \"xgb.DataIter\"\n return(out)\n}\n\n.qdm.single.fnext <- function(env) {\n curr_iter <- env[[\"iter\"]]\n if (curr_iter >= 1L) {\n return(NULL)\n }\n\n on.exit({\n env[[\"iter\"]] <- curr_iter + 1L\n })\n return(\n xgb.DataBatch(\n data = env[[\"data\"]],\n label = env[[\"label\"]],\n weight = env[[\"weight\"]],\n base_margin = env[[\"base_margin\"]],\n feature_names = env[[\"feature_names\"]],\n feature_types = env[[\"feature_types\"]],\n group = env[[\"group\"]],\n qid = env[[\"qid\"]],\n label_lower_bound = env[[\"label_lower_bound\"]],\n label_upper_bound = env[[\"label_upper_bound\"]],\n feature_weights = env[[\"feature_weights\"]]\n )\n )\n}\n\n.qdm.single.freset <- function(env) {\n env[[\"iter\"]] <- 0L\n return(invisible(NULL))\n}\n\n.single.data.iterator <- function(env) {\n env[[\"iter\"]] <- 0L\n return(xgb.DataIter(env, .qdm.single.fnext, .qdm.single.freset))\n}\n\n# Only for internal usage\n.make.proxy.handle <- function() {\n out <- .Call(XGProxyDMatrixCreate_R)\n attributes(out) <- list(\n class = c(\"xgb.DMatrix\", \"xgb.ProxyDMatrix\"),\n fields = new.env()\n )\n return(out)\n}\n\n#' Structure for Data Batches\n#'\n#' @description\n#' Helper function to supply data in batches of a data iterator when\n#' constructing a DMatrix from external memory through [xgb.ExtMemDMatrix()]\n#' or through [xgb.QuantileDMatrix.from_iterator()].\n#'\n#' This function is **only** meant to be called inside of a callback function (which\n#' is passed as argument to function [xgb.DataIter()] to construct a data iterator)\n#' when constructing a DMatrix through external memory - otherwise, one should call\n#' [xgb.DMatrix()] or [xgb.QuantileDMatrix()].\n#'\n#' The object that results from calling this function directly is **not** like\n#' an `xgb.DMatrix` - i.e. cannot be used to train a model, nor to get predictions - only\n#' possible usage is to supply data to an iterator, from which a DMatrix is then constructed.\n#'\n#' For more information and for example usage, see the documentation for [xgb.ExtMemDMatrix()].\n#' @inheritParams xgb.DMatrix\n#' @param data Batch of data belonging to this batch.\n#'\n#' Note that not all of the input types supported by [xgb.DMatrix()] are possible\n#' to pass here. Supported types are:\n#' - `matrix`, with types `numeric`, `integer`, and `logical`. Note that for types\n#' `integer` and `logical`, missing values might not be automatically recognized as\n#' as such - see the documentation for parameter `missing` in [xgb.ExtMemDMatrix()]\n#' for details on this.\n#' - `data.frame`, with the same types as supported by 'xgb.DMatrix' and same\n#' conversions applied to it. See the documentation for parameter `data` in\n#' [xgb.DMatrix()] for details on it.\n#' - CSR matrices, as class `dgRMatrix` from package \"Matrix\".\n#' @return An object of class `xgb.DataBatch`, which is just a list containing the\n#' data and parameters passed here. It does **not** inherit from `xgb.DMatrix`.\n#' @seealso [xgb.DataIter()], [xgb.ExtMemDMatrix()].\n#' @export\nxgb.DataBatch <- function(\n data,\n label = NULL,\n weight = NULL,\n base_margin = NULL,\n feature_names = colnames(data),\n feature_types = NULL,\n group = NULL,\n qid = NULL,\n label_lower_bound = NULL,\n label_upper_bound = NULL,\n feature_weights = NULL\n) {\n stopifnot(inherits(data, c(\"matrix\", \"data.frame\", \"dgRMatrix\")))\n out <- list(\n data = data,\n label = label,\n weight = weight,\n base_margin = base_margin,\n feature_names = feature_names,\n feature_types = feature_types,\n group = group,\n qid = qid,\n label_lower_bound = label_lower_bound,\n label_upper_bound = label_upper_bound,\n feature_weights = feature_weights\n )\n class(out) <- \"xgb.DataBatch\"\n return(out)\n}\n\n# This is only for internal usage, class is not exposed to the user.\nxgb.ProxyDMatrix <- function(proxy_handle, data_iterator, env_keep_alive) {\n env_keep_alive$keepalive <- NULL\n lst <- data_iterator$f_next(data_iterator$env)\n if (is.null(lst)) {\n return(0L)\n }\n if (!inherits(lst, \"xgb.DataBatch\")) {\n stop(\"DataIter 'f_next' must return either NULL or the result from calling 'xgb.DataBatch'.\")\n }\n\n if (!is.null(lst$group) && !is.null(lst$qid)) {\n stop(\"Either one of 'group' or 'qid' should be NULL\")\n }\n if (is.data.frame(lst$data)) {\n data <- lst$data\n lst$data <- NULL\n tmp <- .process.df.for.dmatrix(data, lst$feature_types)\n lst$feature_types <- tmp$feature_types\n data <- NULL\n env_keep_alive$keepalive <- tmp\n .Call(XGProxyDMatrixSetDataColumnar_R, proxy_handle, tmp$lst)\n } else if (is.matrix(lst$data)) {\n env_keep_alive$keepalive <- lst\n .Call(XGProxyDMatrixSetDataDense_R, proxy_handle, lst$data)\n } else if (inherits(lst$data, \"dgRMatrix\")) {\n tmp <- list(p = lst$data@p, j = lst$data@j, x = lst$data@x, ncol = ncol(lst$data))\n env_keep_alive$keepalive <- tmp\n .Call(XGProxyDMatrixSetDataCSR_R, proxy_handle, tmp)\n } else {\n stop(\"'data' has unsupported type.\")\n }\n\n .set.dmatrix.fields(\n dmat = proxy_handle,\n label = lst$label,\n weight = lst$weight,\n base_margin = lst$base_margin,\n feature_names = lst$feature_names,\n feature_types = lst$feature_types,\n group = lst$group,\n qid = lst$qid,\n label_lower_bound = lst$label_lower_bound,\n label_upper_bound = lst$label_upper_bound,\n feature_weights = lst$feature_weights\n )\n\n return(1L)\n}\n\n#' DMatrix from External Data\n#'\n#' @description\n#' Create a special type of XGBoost 'DMatrix' object from external data\n#' supplied by an [xgb.DataIter()] object, potentially passed in batches from a\n#' bigger set that might not fit entirely in memory.\n#'\n#' The data supplied by the iterator is accessed on-demand as needed, multiple times,\n#' without being concatenated, but note that fields like 'label' **will** be\n#' concatenated from multiple calls to the data iterator.\n#'\n#' For more information, see the guide 'Using XGBoost External Memory Version':\n#' \\url{https://xgboost.readthedocs.io/en/stable/tutorials/external_memory.html}\n#' @details\n#' Be aware that construction of external data DMatrices \\bold{will cache data on disk}\n#' in a compressed format, under the path supplied in `cache_prefix`.\n#'\n#' External data is not supported for the exact tree method.\n#' @inheritParams xgb.DMatrix\n#' @param data_iterator A data iterator structure as returned by [xgb.DataIter()],\n#' which includes an environment shared between function calls, and functions to access\n#' the data in batches on-demand.\n#' @param cache_prefix The path of cache file, caller must initialize all the directories in this path.\n#' @param missing A float value to represents missing values in data.\n#'\n#' Note that, while functions like [xgb.DMatrix()] can take a generic `NA` and interpret it\n#' correctly for different types like `numeric` and `integer`, if an `NA` value is passed here,\n#' it will not be adapted for different input types.\n#'\n#' For example, in R `integer` types, missing values are represented by integer number `-2147483648`\n#' (since machine 'integer' types do not have an inherent 'NA' value) - hence, if one passes `NA`,\n#' which is interpreted as a floating-point NaN by [xgb.ExtMemDMatrix()] and by\n#' [xgb.QuantileDMatrix.from_iterator()], these integer missing values will not be treated as missing.\n#' This should not pose any problem for `numeric` types, since they do have an inheret NaN value.\n#' @return An 'xgb.DMatrix' object, with subclass 'xgb.ExtMemDMatrix', in which the data is not\n#' held internally but accessed through the iterator when needed.\n#' @seealso [xgb.DataIter()], [xgb.DataBatch()], [xgb.QuantileDMatrix.from_iterator()]\n#' @examples\n#' data(mtcars)\n#'\n#' # This custom environment will be passed to the iterator\n#' # functions at each call. It is up to the user to keep\n#' # track of the iteration number in this environment.\n#' iterator_env <- as.environment(\n#' list(\n#' iter = 0,\n#' x = mtcars[, -1],\n#' y = mtcars[, 1]\n#' )\n#' )\n#'\n#' # Data is passed in two batches.\n#' # In this example, batches are obtained by subsetting the 'x' variable.\n#' # This is not advantageous to do, since the data is already loaded in memory\n#' # and can be passed in full in one go, but there can be situations in which\n#' # only a subset of the data will fit in the computer's memory, and it can\n#' # be loaded in batches that are accessed one-at-a-time only.\n#' iterator_next <- function(iterator_env) {\n#' curr_iter <- iterator_env[[\"iter\"]]\n#' if (curr_iter >= 2) {\n#' # there are only two batches, so this signals end of the stream\n#' return(NULL)\n#' }\n#'\n#' if (curr_iter == 0) {\n#' x_batch <- iterator_env[[\"x\"]][1:16, ]\n#' y_batch <- iterator_env[[\"y\"]][1:16]\n#' } else {\n#' x_batch <- iterator_env[[\"x\"]][17:32, ]\n#' y_batch <- iterator_env[[\"y\"]][17:32]\n#' }\n#' on.exit({\n#' iterator_env[[\"iter\"]] <- curr_iter + 1\n#' })\n#'\n#' # Function 'xgb.DataBatch' must be called manually\n#' # at each batch with all the appropriate attributes,\n#' # such as feature names and feature types.\n#' return(xgb.DataBatch(data = x_batch, label = y_batch))\n#' }\n#'\n#' # This moves the iterator back to its beginning\n#' iterator_reset <- function(iterator_env) {\n#' iterator_env[[\"iter\"]] <- 0\n#' }\n#'\n#' data_iterator <- xgb.DataIter(\n#' env = iterator_env,\n#' f_next = iterator_next,\n#' f_reset = iterator_reset\n#' )\n#' cache_prefix <- tempdir()\n#'\n#' # DMatrix will be constructed from the iterator's batches\n#' dm <- xgb.ExtMemDMatrix(data_iterator, cache_prefix, nthread = 1)\n#'\n#' # After construction, can be used as a regular DMatrix\n#' params <- xgb.params(nthread = 1, objective = \"reg:squarederror\")\n#' model <- xgb.train(data = dm, nrounds = 2, params = params)\n#'\n#' # Predictions can also be called on it, and should be the same\n#' # as if the data were passed differently.\n#' pred_dm <- predict(model, dm)\n#' pred_mat <- predict(model, as.matrix(mtcars[, -1]))\n#' @export\nxgb.ExtMemDMatrix <- function(\n data_iterator,\n cache_prefix = tempdir(),\n missing = NA,\n nthread = NULL\n) {\n stopifnot(inherits(data_iterator, \"xgb.DataIter\"))\n stopifnot(is.character(cache_prefix))\n\n cache_prefix <- path.expand(cache_prefix)\n nthread <- as.integer(NVL(nthread, -1L))\n\n # The purpose of this environment is to keep data alive (protected from the\n # garbage collector) after setting the data in the proxy dmatrix. The data\n # held here (under name 'keepalive') should be unset (leaving it unprotected\n # for garbage collection) before the start of each data iteration batch and\n # during each iterator reset.\n env_keep_alive <- new.env()\n env_keep_alive$keepalive <- NULL\n\n proxy_handle <- .make.proxy.handle()\n on.exit({\n .Call(XGDMatrixFree_R, proxy_handle)\n })\n iterator_next <- function() {\n return(xgb.ProxyDMatrix(proxy_handle, data_iterator, env_keep_alive))\n }\n iterator_reset <- function() {\n env_keep_alive$keepalive <- NULL\n return(data_iterator$f_reset(data_iterator$env))\n }\n calling_env <- environment()\n\n dmat <- .Call(\n XGDMatrixCreateFromCallback_R,\n iterator_next,\n iterator_reset,\n calling_env,\n proxy_handle,\n nthread,\n missing,\n cache_prefix\n )\n\n attributes(dmat) <- list(\n class = c(\"xgb.DMatrix\", \"xgb.ExtMemDMatrix\"),\n fields = attributes(proxy_handle)$fields\n )\n return(dmat)\n}\n\n\n#' QuantileDMatrix from External Data\n#'\n#' @description\n#' Create an `xgb.QuantileDMatrix` object (exact same class as would be returned by\n#' calling function [xgb.QuantileDMatrix()], with the same advantages and limitations) from\n#' external data supplied by [xgb.DataIter()], potentially passed in batches from\n#' a bigger set that might not fit entirely in memory, same way as [xgb.ExtMemDMatrix()].\n#'\n#' Note that, while external data will only be loaded through the iterator (thus the full data\n#' might not be held entirely in-memory), the quantized representation of the data will get\n#' created in-memory, being concatenated from multiple calls to the data iterator. The quantized\n#' version is typically lighter than the original data, so there might be cases in which this\n#' representation could potentially fit in memory even if the full data does not.\n#'\n#' For more information, see the guide 'Using XGBoost External Memory Version':\n#' \\url{https://xgboost.readthedocs.io/en/stable/tutorials/external_memory.html}\n#' @inheritParams xgb.ExtMemDMatrix\n#' @inheritParams xgb.QuantileDMatrix\n#' @return An 'xgb.DMatrix' object, with subclass 'xgb.QuantileDMatrix'.\n#' @seealso [xgb.DataIter()], [xgb.DataBatch()], [xgb.ExtMemDMatrix()],\n#' [xgb.QuantileDMatrix()]\n#' @export\nxgb.QuantileDMatrix.from_iterator <- function( # nolint\n data_iterator,\n missing = NA,\n nthread = NULL,\n ref = NULL,\n max_bin = NULL\n) {\n stopifnot(inherits(data_iterator, \"xgb.DataIter\"))\n if (!is.null(ref) && !inherits(ref, \"xgb.DMatrix\")) {\n stop(\"'ref' must be an xgb.DMatrix object.\")\n }\n\n nthread <- as.integer(NVL(nthread, -1L))\n\n env_keep_alive <- new.env()\n env_keep_alive$keepalive <- NULL\n proxy_handle <- .make.proxy.handle()\n on.exit({\n .Call(XGDMatrixFree_R, proxy_handle)\n })\n iterator_next <- function() {\n return(xgb.ProxyDMatrix(proxy_handle, data_iterator, env_keep_alive))\n }\n iterator_reset <- function() {\n env_keep_alive$keepalive <- NULL\n return(data_iterator$f_reset(data_iterator$env))\n }\n calling_env <- environment()\n\n dmat <- .Call(\n XGQuantileDMatrixCreateFromCallback_R,\n iterator_next,\n iterator_reset,\n calling_env,\n proxy_handle,\n nthread,\n missing,\n max_bin,\n ref\n )\n\n attributes(dmat) <- list(\n class = c(\"xgb.DMatrix\", \"xgb.QuantileDMatrix\"),\n fields = attributes(proxy_handle)$fields\n )\n return(dmat)\n}\n\n#' Check whether DMatrix object has a field\n#'\n#' Checks whether an xgb.DMatrix object has a given field assigned to\n#' it, such as weights, labels, etc.\n#' @param object The DMatrix object to check for the given `info` field.\n#' @param info The field to check for presence or absence in `object`.\n#' @seealso [xgb.DMatrix()], [getinfo.xgb.DMatrix()], [setinfo.xgb.DMatrix()]\n#' @examples\n#' x <- matrix(1:10, nrow = 5)\n#' dm <- xgb.DMatrix(x, nthread = 1)\n#'\n#' # 'dm' so far does not have any fields set\n#' xgb.DMatrix.hasinfo(dm, \"label\")\n#'\n#' # Fields can be added after construction\n#' setinfo(dm, \"label\", 1:5)\n#' xgb.DMatrix.hasinfo(dm, \"label\")\n#' @export\nxgb.DMatrix.hasinfo <- function(object, info) {\n if (!inherits(object, \"xgb.DMatrix\")) {\n stop(\"Object is not an 'xgb.DMatrix'.\")\n }\n if (.Call(XGCheckNullPtr_R, object)) {\n warning(\"xgb.DMatrix object is invalid. Must be constructed again.\")\n return(FALSE)\n }\n return(NVL(attr(object, \"fields\")[[info]], FALSE))\n}\n\n\n#' Dimensions of xgb.DMatrix\n#'\n#' Returns a vector of numbers of rows and of columns in an `xgb.DMatrix`.\n#'\n#' @param x Object of class `xgb.DMatrix`\n#'\n#' @details\n#' Note: since [nrow()] and [ncol()] internally use [dim()], they can also\n#' be directly used with an `xgb.DMatrix` object.\n#'\n#' @examples\n#' data(agaricus.train, package = \"xgboost\")\n#'\n#' train <- agaricus.train\n#' dtrain <- xgb.DMatrix(train$data, label = train$label, nthread = 2)\n#'\n#' stopifnot(nrow(dtrain) == nrow(train$data))\n#' stopifnot(ncol(dtrain) == ncol(train$data))\n#' stopifnot(all(dim(dtrain) == dim(train$data)))\n#'\n#' @export\ndim.xgb.DMatrix <- function(x) {\n c(.Call(XGDMatrixNumRow_R, x), .Call(XGDMatrixNumCol_R, x))\n}\n\n\n#' Handling of column names of `xgb.DMatrix`\n#'\n#' Only column names are supported for `xgb.DMatrix`, thus setting of\n#' row names would have no effect and returned row names would be `NULL`.\n#'\n#' @param x Object of class `xgb.DMatrix`.\n#' @param value A list of two elements: the first one is ignored\n#' and the second one is column names\n#'\n#' @details\n#' Generic [dimnames()] methods are used by [colnames()].\n#' Since row names are irrelevant, it is recommended to use [colnames()] directly.\n#'\n#' @examples\n#' data(agaricus.train, package = \"xgboost\")\n#'\n#' train <- agaricus.train\n#' dtrain <- xgb.DMatrix(train$data, label = train$label, nthread = 2)\n#' dimnames(dtrain)\n#' colnames(dtrain)\n#' colnames(dtrain) <- make.names(1:ncol(train$data))\n#' print(dtrain, verbose = TRUE)\n#'\n#' @rdname dimnames.xgb.DMatrix\n#' @export\ndimnames.xgb.DMatrix <- function(x) {\n fn <- getinfo(x, \"feature_name\")\n ## row names is null.\n list(NULL, fn)\n}\n\n#' @rdname dimnames.xgb.DMatrix\n#' @export\n`dimnames<-.xgb.DMatrix` <- function(x, value) {\n if (!is.list(value) || length(value) != 2L)\n stop(\"invalid 'dimnames' given: must be a list of two elements\")\n if (!is.null(value[[1L]]))\n stop(\"xgb.DMatrix does not have rownames\")\n if (is.null(value[[2]])) {\n setinfo(x, \"feature_name\", NULL)\n return(x)\n }\n if (ncol(x) != length(value[[2]])) {\n stop(\"can't assign \", length(value[[2]]), \" colnames to a \", ncol(x), \" column xgb.DMatrix\")\n }\n setinfo(x, \"feature_name\", value[[2]])\n x\n}\n\n\n#' Get or set information of xgb.DMatrix and xgb.Booster objects\n#'\n#' @param object Object of class `xgb.DMatrix` or `xgb.Booster`.\n#' @param name The name of the information field to get (see details).\n#' @return For `getinfo()`, will return the requested field. For `setinfo()`,\n#' will always return value `TRUE` if it succeeds.\n#' @details\n#' The `name` field can be one of the following for `xgb.DMatrix`:\n#' - label\n#' - weight\n#' - base_margin\n#' - label_lower_bound\n#' - label_upper_bound\n#' - group\n#' - feature_type\n#' - feature_name\n#' - nrow\n#'\n#' See the documentation for [xgb.DMatrix()] for more information about these fields.\n#'\n#' For `xgb.Booster`, can be one of the following:\n#' - `feature_type`\n#' - `feature_name`\n#'\n#' Note that, while 'qid' cannot be retrieved, it is possible to get the equivalent 'group'\n#' for a DMatrix that had 'qid' assigned.\n#'\n#' **Important**: when calling [setinfo()], the objects are modified in-place. See\n#' [xgb.copy.Booster()] for an idea of this in-place assignment works.\n#' @examples\n#' data(agaricus.train, package = \"xgboost\")\n#'\n#' dtrain <- with(agaricus.train, xgb.DMatrix(data, label = label, nthread = 2))\n#'\n#' labels <- getinfo(dtrain, \"label\")\n#' setinfo(dtrain, \"label\", 1 - labels)\n#'\n#' labels2 <- getinfo(dtrain, \"label\")\n#' stopifnot(all(labels2 == 1 - labels))\n#' @rdname getinfo\n#' @export\ngetinfo <- function(object, name) UseMethod(\"getinfo\")\n\n#' @rdname getinfo\n#' @export\ngetinfo.xgb.DMatrix <- function(object, name) {\n allowed_int_fields <- 'group'\n allowed_float_fields <- c(\n 'label', 'weight', 'base_margin',\n 'label_lower_bound', 'label_upper_bound'\n )\n allowed_str_fields <- c(\"feature_type\", \"feature_name\")\n allowed_fields <- c(allowed_float_fields, allowed_int_fields, allowed_str_fields, 'nrow')\n\n if (typeof(name) != \"character\" ||\n length(name) != 1 ||\n !name %in% allowed_fields) {\n stop(\"getinfo: name must be one of the following\\n\",\n paste(paste0(\"'\", allowed_fields, \"'\"), collapse = \", \"))\n }\n if (name == \"nrow\") {\n ret <- nrow(object)\n } else if (name %in% allowed_str_fields) {\n ret <- .Call(XGDMatrixGetStrFeatureInfo_R, object, name)\n } else if (name %in% allowed_float_fields) {\n ret <- .Call(XGDMatrixGetFloatInfo_R, object, name)\n if (length(ret) > nrow(object)) {\n ret <- matrix(ret, nrow = nrow(object), byrow = TRUE)\n }\n } else if (name %in% allowed_int_fields) {\n if (name == \"group\") {\n name <- \"group_ptr\"\n }\n ret <- .Call(XGDMatrixGetUIntInfo_R, object, name)\n if (length(ret) > nrow(object)) {\n ret <- matrix(ret, nrow = nrow(object), byrow = TRUE)\n }\n }\n if (length(ret) == 0) return(NULL)\n return(ret)\n}\n\n#' @rdname getinfo\n#' @param info The specific field of information to set.\n#'\n#' @details\n#' See the documentation for [xgb.DMatrix()] for possible fields that can be set\n#' (which correspond to arguments in that function).\n#'\n#' Note that the following fields are allowed in the construction of an `xgb.DMatrix`\n#' but **are not** allowed here:\n#' - data\n#' - missing\n#' - silent\n#' - nthread\n#'\n#' @examples\n#' data(agaricus.train, package = \"xgboost\")\n#'\n#' dtrain <- with(agaricus.train, xgb.DMatrix(data, label = label, nthread = 2))\n#'\n#' labels <- getinfo(dtrain, \"label\")\n#' setinfo(dtrain, \"label\", 1 - labels)\n#'\n#' labels2 <- getinfo(dtrain, \"label\")\n#' stopifnot(all.equal(labels2, 1 - labels))\n#' @export\nsetinfo <- function(object, name, info) UseMethod(\"setinfo\")\n\n#' @rdname getinfo\n#' @export\nsetinfo.xgb.DMatrix <- function(object, name, info) {\n .internal.setinfo.xgb.DMatrix(object, name, info)\n attr(object, \"fields\")[[name]] <- TRUE\n return(TRUE)\n}\n\n.internal.setinfo.xgb.DMatrix <- function(object, name, info) {\n if (name == \"label\") {\n if (NROW(info) != nrow(object))\n stop(\"The length of labels must equal to the number of rows in the input data\")\n if (is.factor(info)) {\n stop(\"'label' must be a numeric variable.\")\n }\n .Call(XGDMatrixSetInfo_R, object, name, info)\n return(TRUE)\n }\n if (name == \"label_lower_bound\") {\n if (NROW(info) != nrow(object))\n stop(\"The length of lower-bound labels must equal to the number of rows in the input data\")\n .Call(XGDMatrixSetInfo_R, object, name, info)\n return(TRUE)\n }\n if (name == \"label_upper_bound\") {\n if (NROW(info) != nrow(object))\n stop(\"The length of upper-bound labels must equal to the number of rows in the input data\")\n .Call(XGDMatrixSetInfo_R, object, name, info)\n return(TRUE)\n }\n if (name == \"weight\") {\n .Call(XGDMatrixSetInfo_R, object, name, info)\n return(TRUE)\n }\n if (name == \"base_margin\") {\n .Call(XGDMatrixSetInfo_R, object, name, info)\n return(TRUE)\n }\n if (name == \"group\") {\n if (sum(info) != nrow(object))\n stop(\"The sum of groups must equal to the number of rows in the input data\")\n .Call(XGDMatrixSetInfo_R, object, name, info)\n return(TRUE)\n }\n if (name == \"qid\") {\n if (NROW(info) != nrow(object))\n stop(\"The length of qid assignments must equal to the number of rows in the input data\")\n .Call(XGDMatrixSetInfo_R, object, name, info)\n return(TRUE)\n }\n if (name == \"feature_weights\") {\n if (NROW(info) != ncol(object)) {\n stop(\"The number of feature weights must equal to the number of columns in the input data\")\n }\n .Call(XGDMatrixSetInfo_R, object, name, info)\n return(TRUE)\n }\n\n set_feat_info <- function(name) {\n msg <- sprintf(\n \"The number of %s must equal to the number of columns in the input data. %s vs. %s\",\n name,\n length(info),\n ncol(object)\n )\n if (!is.null(info)) {\n info <- as.list(info)\n if (length(info) != ncol(object)) {\n stop(msg)\n }\n }\n .Call(XGDMatrixSetStrFeatureInfo_R, object, name, info)\n }\n if (name == \"feature_name\") {\n set_feat_info(\"feature_name\")\n return(TRUE)\n }\n if (name == \"feature_type\") {\n set_feat_info(\"feature_type\")\n return(TRUE)\n }\n stop(\"setinfo: unknown info name \", name)\n}\n\n#' Get Quantile Cuts from DMatrix\n#'\n#' @description\n#' Get the quantile cuts (a.k.a. borders) from an `xgb.DMatrix`\n#' that has been quantized for the histogram method (`tree_method = \"hist\"`).\n#'\n#' These cuts are used in order to assign observations to bins - i.e. these are ordered\n#' boundaries which are used to determine assignment condition `border_low < x < border_high`.\n#' As such, the first and last bin will be outside of the range of the data, so as to include\n#' all of the observations there.\n#'\n#' If a given column has 'n' bins, then there will be 'n+1' cuts / borders for that column,\n#' which will be output in sorted order from lowest to highest.\n#'\n#' Different columns can have different numbers of bins according to their range.\n#' @param dmat An `xgb.DMatrix` object, as returned by [xgb.DMatrix()].\n#' @param output Output format for the quantile cuts. Possible options are:\n#' - \"list\"` will return the output as a list with one entry per column, where\n#' each column will have a numeric vector with the cuts. The list will be named if\n#' `dmat` has column names assigned to it.\n#' - `\"arrays\"` will return a list with entries `indptr` (base-0 indexing) and\n#' `data`. Here, the cuts for column 'i' are obtained by slicing 'data' from entries\n#' ` indptr[i]+1` to `indptr[i+1]`.\n#' @return The quantile cuts, in the format specified by parameter `output`.\n#' @examples\n#' data(mtcars)\n#'\n#' y <- mtcars$mpg\n#' x <- as.matrix(mtcars[, -1])\n#' dm <- xgb.DMatrix(x, label = y, nthread = 1)\n#'\n#' # DMatrix is not quantized right away, but will be once a hist model is generated\n#' model <- xgb.train(\n#' data = dm,\n#' params = xgb.params(tree_method = \"hist\", max_bin = 8, nthread = 1),\n#' nrounds = 3\n#' )\n#'\n#' # Now can get the quantile cuts\n#' xgb.get.DMatrix.qcut(dm)\n#' @export\nxgb.get.DMatrix.qcut <- function(dmat, output = c(\"list\", \"arrays\")) { # nolint\n stopifnot(inherits(dmat, \"xgb.DMatrix\"))\n output <- head(output, 1L)\n stopifnot(output %in% c(\"list\", \"arrays\"))\n res <- .Call(XGDMatrixGetQuantileCut_R, dmat)\n if (output == \"arrays\") {\n return(res)\n } else {\n feature_names <- getinfo(dmat, \"feature_name\")\n ncols <- length(res$indptr) - 1\n out <- lapply(\n seq(1, ncols),\n function(col) {\n st <- res$indptr[col]\n end <- res$indptr[col + 1]\n if (end <= st) {\n return(numeric())\n }\n return(res$data[seq(1 + st, end)])\n }\n )\n if (NROW(feature_names)) {\n names(out) <- feature_names\n }\n return(out)\n }\n}\n\n#' Get Number of Non-Missing Entries in DMatrix\n#'\n#' @param dmat An `xgb.DMatrix` object, as returned by [xgb.DMatrix()].\n#' @return The number of non-missing entries in the DMatrix.\n#' @export\nxgb.get.DMatrix.num.non.missing <- function(dmat) { # nolint\n stopifnot(inherits(dmat, \"xgb.DMatrix\"))\n return(.Call(XGDMatrixNumNonMissing_R, dmat))\n}\n\n#' Get DMatrix Data\n#'\n#' @param dmat An `xgb.DMatrix` object, as returned by [xgb.DMatrix()].\n#' @return The data held in the DMatrix, as a sparse CSR matrix (class `dgRMatrix`\n#' from package `Matrix`). If it had feature names, these will be added as column names\n#' in the output.\n#' @export\nxgb.get.DMatrix.data <- function(dmat) {\n stopifnot(inherits(dmat, \"xgb.DMatrix\"))\n res <- .Call(XGDMatrixGetDataAsCSR_R, dmat)\n out <- methods::new(\"dgRMatrix\")\n nrows <- as.integer(length(res$indptr) - 1)\n out@p <- res$indptr\n out@j <- res$indices\n out@x <- res$data\n out@Dim <- as.integer(c(nrows, res$ncols))\n\n feature_names <- getinfo(dmat, \"feature_name\")\n dim_names <- list(NULL, NULL)\n if (NROW(feature_names)) {\n dim_names[[2L]] <- feature_names\n }\n out@Dimnames <- dim_names\n return(out)\n}\n\n#' Slice DMatrix\n#'\n#' Get a new DMatrix containing the specified rows of original xgb.DMatrix object.\n#'\n#' @param object Object of class `xgb.DMatrix`.\n#' @param idxset An integer vector of indices of rows needed (base-1 indexing).\n#' @param allow_groups Whether to allow slicing an `xgb.DMatrix` with `group` (or\n#' equivalently `qid`) field. Note that in such case, the result will not have\n#' the groups anymore - they need to be set manually through [setinfo()].\n#' @param colset Currently not used (columns subsetting is not available).\n#'\n#' @examples\n#' data(agaricus.train, package = \"xgboost\")\n#'\n#' dtrain <- with(agaricus.train, xgb.DMatrix(data, label = label, nthread = 2))\n#'\n#' dsub <- xgb.slice.DMatrix(dtrain, 1:42)\n#' labels1 <- getinfo(dsub, \"label\")\n#'\n#' dsub <- dtrain[1:42, ]\n#' labels2 <- getinfo(dsub, \"label\")\n#' all.equal(labels1, labels2)\n#'\n#' @rdname xgb.slice.DMatrix\n#' @export\nxgb.slice.DMatrix <- function(object, idxset, allow_groups = FALSE) {\n if (!inherits(object, \"xgb.DMatrix\")) {\n stop(\"object must be xgb.DMatrix\")\n }\n ret <- .Call(XGDMatrixSliceDMatrix_R, object, idxset, allow_groups)\n\n attr_list <- attributes(object)\n nr <- nrow(object)\n len <- sapply(attr_list, NROW)\n ind <- which(len == nr)\n if (length(ind) > 0) {\n nms <- names(attr_list)[ind]\n for (i in seq_along(ind)) {\n obj_attr <- attr(object, nms[i])\n if (NCOL(obj_attr) > 1) {\n attr(ret, nms[i]) <- obj_attr[idxset, ]\n } else {\n attr(ret, nms[i]) <- obj_attr[idxset]\n }\n }\n }\n\n out <- structure(ret, class = \"xgb.DMatrix\")\n parent_fields <- as.list(attributes(object)$fields)\n if (NROW(parent_fields)) {\n child_fields <- parent_fields[!(names(parent_fields) %in% c(\"group\", \"qid\"))]\n child_fields <- as.environment(child_fields)\n attributes(out)$fields <- child_fields\n }\n return(out)\n}\n\n#' @rdname xgb.slice.DMatrix\n#' @export\n`[.xgb.DMatrix` <- function(object, idxset, colset = NULL) {\n xgb.slice.DMatrix(object, idxset)\n}\n\n\n#' Print xgb.DMatrix\n#'\n#' Print information about xgb.DMatrix.\n#' Currently it displays dimensions and presence of info-fields and colnames.\n#'\n#' @param x An xgb.DMatrix object.\n#' @param verbose Whether to print colnames (when present).\n#' @param ... Not currently used.\n#'\n#' @examples\n#' data(agaricus.train, package = \"xgboost\")\n#'\n#' dtrain <- with(agaricus.train, xgb.DMatrix(data, label = label, nthread = 2))\n#' dtrain\n#'\n#' print(dtrain, verbose = TRUE)\n#'\n#' @method print xgb.DMatrix\n#' @export\nprint.xgb.DMatrix <- function(x, verbose = FALSE, ...) {\n if (.Call(XGCheckNullPtr_R, x)) {\n cat(\"INVALID xgb.DMatrix object. Must be constructed anew.\\n\")\n return(invisible(x))\n }\n class_print <- if (inherits(x, \"xgb.QuantileDMatrix\")) {\n \"xgb.QuantileDMatrix\"\n } else if (inherits(x, \"xgb.ExtMemDMatrix\")) {\n \"xgb.ExtMemDMatrix\"\n } else if (inherits(x, \"xgb.ProxyDMatrix\")) {\n \"xgb.ProxyDMatrix\"\n } else {\n \"xgb.DMatrix\"\n }\n\n cat(class_print, ' dim:', nrow(x), 'x', ncol(x), ' info: ')\n infos <- names(attributes(x)$fields)\n infos <- infos[infos != \"feature_name\"]\n if (!NROW(infos)) infos <- \"NA\"\n infos <- infos[order(infos)]\n infos <- paste(infos, collapse = \", \")\n cat(infos)\n cnames <- colnames(x)\n cat(' colnames:')\n if (verbose && !is.null(cnames)) {\n cat(\"\\n'\")\n cat(cnames, sep = \"','\")\n cat(\"'\")\n } else {\n if (is.null(cnames)) cat(' no')\n else cat(' yes')\n }\n cat(\"\\n\")\n invisible(x)\n}\n" }, { "path": "R-package/R/xgb.DMatrix.save.R", "content": "#' Save xgb.DMatrix object to binary file\n#'\n#' Save xgb.DMatrix object to binary file\n#'\n#' @param dmatrix the `xgb.DMatrix` object\n#' @param fname the name of the file to write.\n#'\n#' @examples\n#' \\dontshow{RhpcBLASctl::omp_set_num_threads(1)}\n#' data(agaricus.train, package = \"xgboost\")\n#'\n#' dtrain <- with(agaricus.train, xgb.DMatrix(data, label = label, nthread = 2))\n#' fname <- file.path(tempdir(), \"xgb.DMatrix.data\")\n#' xgb.DMatrix.save(dtrain, fname)\n#' dtrain <- xgb.DMatrix(fname, nthread = 1)\n#' @export\nxgb.DMatrix.save <- function(dmatrix, fname) {\n if (typeof(fname) != \"character\")\n stop(\"fname must be character\")\n if (!inherits(dmatrix, \"xgb.DMatrix\"))\n stop(\"dmatrix must be xgb.DMatrix\")\n\n fname <- path.expand(fname)\n .Call(XGDMatrixSaveBinary_R, dmatrix, fname[1], 0L)\n return(TRUE)\n}\n" }, { "path": "R-package/R/xgb.config.R", "content": "#' Set and get global configuration\n#'\n#' Global configuration consists of a collection of parameters that can be applied in the global\n#' scope. See \\url{https://xgboost.readthedocs.io/en/stable/parameter.html} for the full list of\n#' parameters supported in the global configuration. Use `xgb.set.config()` to update the\n#' values of one or more global-scope parameters. Use `xgb.get.config()` to fetch the current\n#' values of all global-scope parameters (listed in\n#' \\url{https://xgboost.readthedocs.io/en/stable/parameter.html}).\n#'\n#' @details\n#' Note that serialization-related functions might use a globally-configured number of threads,\n#' which is managed by the system's OpenMP (OMP) configuration instead. Typically, XGBoost methods\n#' accept an `nthreads` parameter, but some methods like [readRDS()] might get executed before such\n#' parameter can be supplied.\n#'\n#' The number of OMP threads can in turn be configured for example through an environment variable\n#' `OMP_NUM_THREADS` (needs to be set before R is started), or through `RhpcBLASctl::omp_set_num_threads`.\n#' @rdname xgbConfig\n#' @name xgb.set.config, xgb.get.config\n#' @export xgb.set.config xgb.get.config\n#' @param ... List of parameters to be set, as keyword arguments\n#' @return\n#' `xgb.set.config()` returns `TRUE` to signal success. `xgb.get.config()` returns\n#' a list containing all global-scope parameters and their values.\n#'\n#' @examples\n#' # Set verbosity level to silent (0)\n#' xgb.set.config(verbosity = 0)\n#' # Now global verbosity level is 0\n#' config <- xgb.get.config()\n#' print(config$verbosity)\n#' # Set verbosity level to warning (1)\n#' xgb.set.config(verbosity = 1)\n#' # Now global verbosity level is 1\n#' config <- xgb.get.config()\n#' print(config$verbosity)\nxgb.set.config <- function(...) {\n new_config <- list(...)\n .Call(XGBSetGlobalConfig_R, jsonlite::toJSON(new_config, auto_unbox = TRUE))\n return(TRUE)\n}\n\n#' @rdname xgbConfig\nxgb.get.config <- function() {\n config <- .Call(XGBGetGlobalConfig_R)\n return(jsonlite::fromJSON(config))\n}\n" }, { "path": "R-package/R/xgb.create.features.R", "content": "#' Create new features from a previously learned model\n#'\n#' May improve the learning by adding new features to the training data based on the\n#' decision trees from a previously learned model.\n#'\n#' @details\n#' This is the function inspired from the paragraph 3.1 of the paper:\n#'\n#' **Practical Lessons from Predicting Clicks on Ads at Facebook**\n#'\n#' *(Xinran He, Junfeng Pan, Ou Jin, Tianbing Xu, Bo Liu, Tao Xu, Yan, xin Shi, Antoine Atallah, Ralf Herbrich, Stuart Bowers,\n#' Joaquin Quinonero Candela)*\n#'\n#' International Workshop on Data Mining for Online Advertising (ADKDD) - August 24, 2014\n#'\n#' \\url{https://research.facebook.com/publications/practical-lessons-from-predicting-clicks-on-ads-at-facebook/}.\n#'\n#' Extract explaining the method:\n#'\n#' \"We found that boosted decision trees are a powerful and very\n#' convenient way to implement non-linear and tuple transformations\n#' of the kind we just described. We treat each individual\n#' tree as a categorical feature that takes as value the\n#' index of the leaf an instance ends up falling in. We use\n#' 1-of-K coding of this type of features.\n#'\n#' For example, consider the boosted tree model in Figure 1 with 2 subtrees,\n#' where the first subtree has 3 leafs and the second 2 leafs. If an\n#' instance ends up in leaf 2 in the first subtree and leaf 1 in\n#' second subtree, the overall input to the linear classifier will\n#' be the binary vector `[0, 1, 0, 1, 0]`, where the first 3 entries\n#' correspond to the leaves of the first subtree and last 2 to\n#' those of the second subtree.\n#'\n#' ...\n#'\n#' We can understand boosted decision tree\n#' based transformation as a supervised feature encoding that\n#' converts a real-valued vector into a compact binary-valued\n#' vector. A traversal from root node to a leaf node represents\n#' a rule on certain features.\"\n#'\n#' @param model Decision tree boosting model learned on the original data.\n#' @param data Original data (usually provided as a `dgCMatrix` matrix).\n#'\n#' @return A `dgCMatrix` matrix including both the original data and the new features.\n#'\n#' @examples\n#' data(agaricus.train, package = \"xgboost\")\n#' data(agaricus.test, package = \"xgboost\")\n#'\n#' dtrain <- with(agaricus.train, xgb.DMatrix(data, label = label, nthread = 2))\n#' dtest <- with(agaricus.test, xgb.DMatrix(data, label = label, nthread = 2))\n#'\n#' param <- list(max_depth = 2, learning_rate = 1, objective = 'binary:logistic', nthread = 1)\n#' nrounds = 4\n#'\n#' bst <- xgb.train(params = param, data = dtrain, nrounds = nrounds)\n#'\n#' # Model accuracy without new features\n#' accuracy.before <- sum((predict(bst, agaricus.test$data) >= 0.5) == agaricus.test$label) /\n#' length(agaricus.test$label)\n#'\n#' # Convert previous features to one hot encoding\n#' new.features.train <- xgb.create.features(model = bst, agaricus.train$data)\n#' new.features.test <- xgb.create.features(model = bst, agaricus.test$data)\n#'\n#' # learning with new features\n#' new.dtrain <- xgb.DMatrix(\n#' data = new.features.train, label = agaricus.train$label, nthread = 1\n#' )\n#' new.dtest <- xgb.DMatrix(\n#' data = new.features.test, label = agaricus.test$label, nthread = 1\n#' )\n#' bst <- xgb.train(params = param, data = new.dtrain, nrounds = nrounds)\n#'\n#' # Model accuracy with new features\n#' accuracy.after <- sum((predict(bst, new.dtest) >= 0.5) == agaricus.test$label) /\n#' length(agaricus.test$label)\n#'\n#' # Here the accuracy was already good and is now perfect.\n#' cat(paste(\"The accuracy was\", accuracy.before, \"before adding leaf features and it is now\",\n#' accuracy.after, \"!\\n\"))\n#'\n#' @export\nxgb.create.features <- function(model, data) {\n pred_with_leaf <- predict.xgb.Booster(model, data, predleaf = TRUE)\n cols <- lapply(as.data.frame(pred_with_leaf), factor)\n cbind(data, sparse.model.matrix(~ . -1, cols)) # nolint\n}\n" }, { "path": "R-package/R/xgb.cv.R", "content": "#' Cross Validation\n#'\n#' The cross validation function of xgboost.\n#'\n#' @inheritParams xgb.train\n#' @param data An `xgb.DMatrix` object, with corresponding fields like `label` or bounds as required\n#' for model training by the objective.\n#'\n#' Note that only the basic `xgb.DMatrix` class is supported - variants such as `xgb.QuantileDMatrix`\n#' or `xgb.ExtMemDMatrix` are not supported here.\n#' @param nfold The original dataset is randomly partitioned into `nfold` equal size subsamples.\n#' @param prediction A logical value indicating whether to return the test fold predictions\n#' from each CV model. This parameter engages the [xgb.cb.cv.predict()] callback.\n#' @param showsd Logical value whether to show standard deviation of cross validation.\n#' @param metrics List of evaluation metrics to be used in cross validation,\n#' when it is not specified, the evaluation metric is chosen according to objective function.\n#' Possible options are:\n#' - `error`: Binary classification error rate\n#' - `rmse`: Root mean square error\n#' - `logloss`: Negative log-likelihood function\n#' - `mae`: Mean absolute error\n#' - `mape`: Mean absolute percentage error\n#' - `auc`: Area under curve\n#' - `aucpr`: Area under PR curve\n#' - `merror`: Exact matching error used to evaluate multi-class classification\n#' @param stratified Logical flag indicating whether sampling of folds should be stratified\n#' by the values of outcome labels. For real-valued labels in regression objectives,\n#' stratification will be done by discretizing the labels into up to 5 buckets beforehand.\n#'\n#' If passing \"auto\", will be set to `TRUE` if the objective in `params` is a classification\n#' objective (from XGBoost's built-in objectives, doesn't apply to custom ones), and to\n#' `FALSE` otherwise.\n#'\n#' This parameter is ignored when `data` has a `group` field - in such case, the splitting\n#' will be based on whole groups (note that this might make the folds have different sizes).\n#'\n#' Value `TRUE` here is **not** supported for custom objectives.\n#' @param folds List with pre-defined CV folds (each element must be a vector of test fold's indices).\n#' When folds are supplied, the `nfold` and `stratified` parameters are ignored.\n#'\n#' If `data` has a `group` field and the objective requires this field, each fold (list element)\n#' must additionally have two attributes (retrievable through `attributes`) named `group_test`\n#' and `group_train`, which should hold the `group` to assign through [setinfo.xgb.DMatrix()] to\n#' the resulting DMatrices.\n#' @param train_folds List specifying which indices to use for training. If `NULL`\n#' (the default) all indices not specified in `folds` will be used for training.\n#'\n#' This is not supported when `data` has `group` field.\n#' @param callbacks A list of callback functions to perform various task during boosting.\n#' See [xgb.Callback()]. Some of the callbacks are automatically created depending on the\n#' parameters' values. User can provide either existing or their own callback methods in order\n#' to customize the training process.\n#' @details\n#' The original sample is randomly partitioned into `nfold` equal size subsamples.\n#'\n#' Of the `nfold` subsamples, a single subsample is retained as the validation data for testing the model,\n#' and the remaining `nfold - 1` subsamples are used as training data.\n#'\n#' The cross-validation process is then repeated `nrounds` times, with each of the\n#' `nfold` subsamples used exactly once as the validation data.\n#'\n#' All observations are used for both training and validation.\n#'\n#' Adapted from \\url{https://en.wikipedia.org/wiki/Cross-validation_\\%28statistics\\%29}\n#'\n#' @return\n#' An object of class 'xgb.cv.synchronous' with the following elements:\n#' - `call`: Function call.\n#' - `params`: Parameters that were passed to the xgboost library. Note that it does not\n#' capture parameters changed by the [xgb.cb.reset.parameters()] callback.\n#' - `evaluation_log`: Evaluation history stored as a `data.table` with the\n#' first column corresponding to iteration number and the rest corresponding to the\n#' CV-based evaluation means and standard deviations for the training and test CV-sets.\n#' It is created by the [xgb.cb.evaluation.log()] callback.\n#' - `niter`: Number of boosting iterations.\n#' - `nfeatures`: Number of features in training data.\n#' - `folds`: The list of CV folds' indices - either those passed through the `folds`\n#' parameter or randomly generated.\n#'\n#' Plus other potential elements that are the result of callbacks, such as a list `cv_predict` with\n#' a sub-element `pred` when passing `prediction = TRUE`, which is added by the [xgb.cb.cv.predict()]\n#' callback (note that one can also pass it manually under `callbacks` with different settings,\n#' such as saving also the models created during cross validation); or a list `early_stop` which\n#' will contain elements such as `best_iteration` when using the early stopping callback ([xgb.cb.early.stop()]).\n#'\n#' @examples\n#' data(agaricus.train, package = \"xgboost\")\n#'\n#' dtrain <- with(agaricus.train, xgb.DMatrix(data, label = label, nthread = 2))\n#'\n#' cv <- xgb.cv(\n#' data = dtrain,\n#' nrounds = 20,\n#' early_stopping_rounds = 1,\n#' params = xgb.params(\n#' nthread = 2,\n#' max_depth = 3,\n#' objective = \"binary:logistic\"\n#' ),\n#' nfold = 5,\n#' metrics = list(\"rmse\",\"auc\"),\n#' prediction = TRUE\n#' )\n#' print(cv)\n#' print(cv, verbose = TRUE)\n#'\n#' # Callbacks might add additional attributes, separated by the name of the callback\n#' cv$early_stop$best_iteration\n#' head(cv$cv_predict$pred)\n#' @export\nxgb.cv <- function(params = xgb.params(), data, nrounds, nfold,\n prediction = FALSE, showsd = TRUE, metrics = list(),\n objective = NULL, custom_metric = NULL, stratified = \"auto\",\n folds = NULL, train_folds = NULL, verbose = TRUE, print_every_n = 1L,\n early_stopping_rounds = NULL, maximize = NULL, callbacks = list(), ...) {\n check.deprecation(deprecated_cv_params, match.call(), ...)\n\n stopifnot(inherits(data, \"xgb.DMatrix\"))\n\n if (inherits(data, \"xgb.DMatrix\") && .Call(XGCheckNullPtr_R, data)) {\n stop(\"'data' is an invalid 'xgb.DMatrix' object. Must be constructed again.\")\n }\n if (inherits(data, \"xgb.QuantileDMatrix\")) {\n stop(\"'xgb.QuantileDMatrix' is not supported as input to 'xgb.cv'.\")\n }\n\n params <- check.booster.params(params)\n # TODO: should we deprecate the redundant 'metrics' parameter?\n for (m in metrics)\n params <- c(params, list(\"eval_metric\" = m))\n\n tmp <- check.custom.obj(params, objective)\n params <- tmp$params\n objective <- tmp$objective\n tmp <- check.custom.eval(params, custom_metric, maximize, early_stopping_rounds, callbacks)\n params <- tmp$params\n custom_metric <- tmp$custom_metric\n\n if (stratified == \"auto\") {\n if (is.character(params$objective)) {\n stratified <- (\n (params$objective %in% .CLASSIFICATION_OBJECTIVES())\n && !(params$objective %in% .RANKING_OBJECTIVES())\n )\n } else {\n stratified <- FALSE\n }\n }\n\n # Check the labels and groups\n cv_label <- getinfo(data, \"label\")\n cv_group <- getinfo(data, \"group\")\n if (!is.null(train_folds) && NROW(cv_group)) {\n stop(\"'train_folds' is not supported for DMatrix object with 'group' field.\")\n }\n\n # CV folds\n if (!is.null(folds)) {\n if (!is.list(folds) || length(folds) < 2)\n stop(\"'folds' must be a list with 2 or more elements that are vectors of indices for each CV-fold\")\n nfold <- length(folds)\n } else {\n if (nfold <= 1)\n stop(\"'nfold' must be > 1\")\n folds <- generate.cv.folds(nfold, nrow(data), stratified, cv_label, cv_group, params)\n }\n\n # Callbacks\n tmp <- .process.callbacks(callbacks, is_cv = TRUE)\n callbacks <- tmp$callbacks\n cb_names <- tmp$cb_names\n rm(tmp)\n\n # Early stopping callback\n if (!is.null(early_stopping_rounds) && !(\"early_stop\" %in% cb_names)) {\n callbacks <- add.callback(\n callbacks,\n xgb.cb.early.stop(\n early_stopping_rounds,\n maximize = maximize,\n verbose = verbose,\n save_best = FALSE\n ),\n as_first_elt = TRUE\n )\n }\n # verbosity & evaluation printing callback:\n params <- c(params, list(silent = 1))\n print_every_n <- max(as.integer(print_every_n), 1L)\n if (verbose && !(\"print_evaluation\" %in% cb_names)) {\n callbacks <- add.callback(callbacks, xgb.cb.print.evaluation(print_every_n, showsd = showsd))\n }\n # evaluation log callback: always is on in CV\n if (!(\"evaluation_log\" %in% cb_names)) {\n callbacks <- add.callback(callbacks, xgb.cb.evaluation.log())\n }\n # CV-predictions callback\n if (prediction && !(\"cv_predict\" %in% cb_names)) {\n callbacks <- add.callback(callbacks, xgb.cb.cv.predict(save_models = FALSE))\n }\n\n # create the booster-folds\n # train_folds\n dall <- data\n bst_folds <- lapply(seq_along(folds), function(k) {\n dtest <- xgb.slice.DMatrix(dall, folds[[k]], allow_groups = TRUE)\n # code originally contributed by @RolandASc on stackoverflow\n if (is.null(train_folds))\n dtrain <- xgb.slice.DMatrix(dall, unlist(folds[-k]), allow_groups = TRUE)\n else\n dtrain <- xgb.slice.DMatrix(dall, train_folds[[k]], allow_groups = TRUE)\n if (!is.null(attributes(folds[[k]])$group_test)) {\n setinfo(dtest, \"group\", attributes(folds[[k]])$group_test)\n setinfo(dtrain, \"group\", attributes(folds[[k]])$group_train)\n }\n bst <- xgb.Booster(\n params = params,\n cachelist = list(dtrain, dtest),\n modelfile = NULL\n )\n bst <- bst$bst\n list(dtrain = dtrain, bst = bst, evals = list(train = dtrain, test = dtest), index = folds[[k]])\n })\n\n # extract parameters that can affect the relationship b/w #trees and #iterations\n num_class <- max(as.numeric(NVL(params[['num_class']], 1)), 1) # nolint\n\n # those are fixed for CV (no training continuation)\n begin_iteration <- 1\n end_iteration <- nrounds\n\n .execute.cb.before.training(\n callbacks,\n bst_folds,\n dall,\n NULL,\n begin_iteration,\n end_iteration\n )\n\n # synchronous CV boosting: run CV folds' models within each iteration\n for (iteration in begin_iteration:end_iteration) {\n\n .execute.cb.before.iter(\n callbacks,\n bst_folds,\n dall,\n NULL,\n iteration\n )\n\n msg <- lapply(bst_folds, function(fd) {\n xgb.iter.update(\n bst = fd$bst,\n dtrain = fd$dtrain,\n iter = iteration - 1,\n objective = objective\n )\n xgb.iter.eval(\n bst = fd$bst,\n evals = fd$evals,\n iter = iteration - 1,\n custom_metric = custom_metric\n )\n })\n msg <- simplify2array(msg)\n\n should_stop <- .execute.cb.after.iter(\n callbacks,\n bst_folds,\n dall,\n NULL,\n iteration,\n msg\n )\n\n if (should_stop) break\n }\n\n cb_outputs <- .execute.cb.after.training(\n callbacks,\n bst_folds,\n dall,\n NULL,\n iteration,\n msg\n )\n\n # Just in case if the model is referenced in callbacks.\n lapply(bst_folds, function(fd) {\n xgb.reset.Booster(fd$bst)\n })\n\n # the CV result\n ret <- list(\n call = match.call(),\n params = params,\n niter = iteration,\n nfeatures = ncol(dall),\n folds = folds\n )\n ret <- c(ret, cb_outputs)\n\n class(ret) <- 'xgb.cv.synchronous'\n return(invisible(ret))\n}\n\n\n\n#' Print xgb.cv result\n#'\n#' Prints formatted results of [xgb.cv()].\n#'\n#' @param x An `xgb.cv.synchronous` object.\n#' @param verbose Whether to print detailed data.\n#' @param ... Passed to `data.table.print()`.\n#'\n#' @details\n#' When not verbose, it would only print the evaluation results,\n#' including the best iteration (when available).\n#'\n#' @examples\n#' data(agaricus.train, package = \"xgboost\")\n#'\n#' train <- agaricus.train\n#' cv <- xgb.cv(\n#' data = xgb.DMatrix(train$data, label = train$label, nthread = 1),\n#' nfold = 5,\n#' nrounds = 2,\n#' params = xgb.params(\n#' max_depth = 2,\n#' nthread = 2,\n#' objective = \"binary:logistic\"\n#' )\n#' )\n#' print(cv)\n#' print(cv, verbose = TRUE)\n#'\n#' @rdname print.xgb.cv\n#' @method print xgb.cv.synchronous\n#' @export\nprint.xgb.cv.synchronous <- function(x, verbose = FALSE, ...) {\n cat('##### xgb.cv ', length(x$folds), '-folds\\n', sep = '')\n\n if (verbose) {\n if (!is.null(x$call)) {\n cat('call:\\n ')\n print(x$call)\n }\n if (!is.null(x$params)) {\n cat('params (as set within xgb.cv):\\n')\n cat(' ',\n paste(names(x$params),\n paste0('\"', unlist(x$params), '\"'),\n sep = ' = ', collapse = ', '), '\\n', sep = '')\n }\n\n for (n in c('niter', 'best_iteration')) {\n if (is.null(x$early_stop[[n]]))\n next\n cat(n, ': ', x$early_stop[[n]], '\\n', sep = '')\n }\n\n if (!is.null(x$cv_predict$pred)) {\n cat('pred:\\n')\n str(x$cv_predict$pred)\n }\n }\n\n if (verbose)\n cat('evaluation_log:\\n')\n print(x$evaluation_log, row.names = FALSE, ...)\n\n if (!is.null(x$early_stop$best_iteration)) {\n cat('Best iteration:\\n')\n print(x$evaluation_log[x$early_stop$best_iteration], row.names = FALSE, ...)\n }\n invisible(x)\n}\n" }, { "path": "R-package/R/xgb.dump.R", "content": "#' Dump an XGBoost model in text format.\n#'\n#' Dump an XGBoost model in text format.\n#'\n#' @param model The model object.\n#' @param fname The name of the text file where to save the model text dump.\n#' If not provided or set to `NULL`, the model is returned as a character vector.\n#' @param fmap Feature map file representing feature types. See demo/ for a walkthrough\n#' example in R, and \\url{https://github.com/dmlc/xgboost/blob/master/demo/data/featmap.txt}\n#' to see an example of the value.\n#' @param with_stats Whether to dump some additional statistics about the splits.\n#' When this option is on, the model dump contains two additional values:\n#' gain is the approximate loss function gain we get in each split;\n#' cover is the sum of second order gradient in each node.\n#' @param dump_format Either 'text', 'json', or 'dot' (graphviz) format could be specified.\n#'\n#' Format 'dot' for a single tree can be passed directly to packages that consume this format\n#' for graph visualization, such as function `DiagrammeR::grViz()`\n#' @inheritParams xgb.train\n#' @return\n#' If fname is not provided or set to `NULL` the function will return the model\n#' as a character vector. Otherwise it will return `TRUE`.\n#'\n#' @examples\n#' \\dontshow{RhpcBLASctl::omp_set_num_threads(1)}\n#' data(agaricus.train, package = \"xgboost\")\n#' data(agaricus.test, package = \"xgboost\")\n#'\n#' train <- agaricus.train\n#' test <- agaricus.test\n#'\n#' bst <- xgb.train(\n#' data = xgb.DMatrix(train$data, label = train$label, nthread = 1),\n#' nrounds = 2,\n#' params = xgb.params(\n#' max_depth = 2,\n#' nthread = 2,\n#' objective = \"binary:logistic\"\n#' )\n#' )\n#'\n#' # save the model in file 'xgb.model.dump'\n#' dump_path = file.path(tempdir(), 'model.dump')\n#' xgb.dump(bst, dump_path, with_stats = TRUE)\n#'\n#' # print the model without saving it to a file\n#' print(xgb.dump(bst, with_stats = TRUE))\n#'\n#' # print in JSON format:\n#' cat(xgb.dump(bst, with_stats = TRUE, dump_format = \"json\"))\n#'\n#' # plot first tree leveraging the 'dot' format\n#' if (requireNamespace('DiagrammeR', quietly = TRUE)) {\n#' DiagrammeR::grViz(xgb.dump(bst, dump_format = \"dot\")[[1L]])\n#' }\n#' @export\nxgb.dump <- function(model, fname = NULL, fmap = \"\", with_stats = FALSE,\n dump_format = c(\"text\", \"json\", \"dot\"), ...) {\n check.deprecation(deprecated_dump_params, match.call(), ...)\n dump_format <- match.arg(dump_format)\n if (!inherits(model, \"xgb.Booster\"))\n stop(\"model: argument must be of type xgb.Booster\")\n if (!(is.null(fname) || is.character(fname)))\n stop(\"fname: argument must be a character string (when provided)\")\n if (!(is.null(fmap) || is.character(fmap)))\n stop(\"fmap: argument must be a character string (when provided)\")\n\n model_dump <- .Call(\n XGBoosterDumpModel_R,\n xgb.get.handle(model),\n NVL(fmap, \"\")[1],\n as.integer(with_stats),\n as.character(dump_format)\n )\n if (dump_format == \"dot\") {\n return(sapply(model_dump, function(x) gsub(\"^booster\\\\[\\\\d+\\\\]\\\\n\", \"\\\\1\", x)))\n }\n\n if (is.null(fname))\n model_dump <- gsub('\\t', '', model_dump, fixed = TRUE)\n\n if (dump_format == \"text\")\n model_dump <- unlist(strsplit(model_dump, '\\n', fixed = TRUE))\n\n model_dump <- grep('^\\\\s*$', model_dump, invert = TRUE, value = TRUE)\n\n if (is.null(fname)) {\n return(model_dump)\n } else {\n fname <- path.expand(fname)\n writeLines(model_dump, fname[1])\n return(TRUE)\n }\n}\n" }, { "path": "R-package/R/xgb.ggplot.R", "content": "# ggplot backend for the xgboost plotting facilities\n\n#' @rdname xgb.plot.importance\n#' @export\nxgb.ggplot.importance <- function(importance_matrix = NULL, top_n = NULL, measure = NULL,\n rel_to_first = FALSE, n_clusters = seq_len(10), ...) {\n\n importance_matrix <- xgb.plot.importance(importance_matrix, top_n = top_n, measure = measure,\n rel_to_first = rel_to_first, plot = FALSE, ...)\n if (!requireNamespace(\"ggplot2\", quietly = TRUE)) {\n stop(\"ggplot2 package is required\", call. = FALSE)\n }\n if (!requireNamespace(\"Ckmeans.1d.dp\", quietly = TRUE)) {\n stop(\"Ckmeans.1d.dp package is required\", call. = FALSE)\n }\n\n clusters <- suppressWarnings(\n Ckmeans.1d.dp::Ckmeans.1d.dp(importance_matrix$Importance, n_clusters)\n )\n importance_matrix[, Cluster := as.character(clusters$cluster)]\n\n plot <-\n ggplot2::ggplot(importance_matrix,\n ggplot2::aes(x = factor(Feature, levels = rev(Feature)), y = Importance, width = 0.5),\n environment = environment()) +\n ggplot2::geom_bar(ggplot2::aes(fill = Cluster), stat = \"identity\", position = \"identity\") +\n ggplot2::coord_flip() +\n ggplot2::xlab(\"Features\") +\n ggplot2::ggtitle(\"Feature importance\") +\n ggplot2::theme(plot.title = ggplot2::element_text(lineheight = .9, face = \"bold\"),\n panel.grid.major.y = ggplot2::element_blank())\n return(plot)\n}\n\n\n#' @rdname xgb.plot.deepness\n#' @export\nxgb.ggplot.deepness <- function(model = NULL, which = c(\"2x1\", \"max.depth\", \"med.depth\", \"med.weight\")) {\n\n if (!requireNamespace(\"ggplot2\", quietly = TRUE))\n stop(\"ggplot2 package is required for plotting the graph deepness.\", call. = FALSE)\n\n which <- match.arg(which)\n\n dt_depths <- xgb.plot.deepness(model = model, plot = FALSE)\n dt_summaries <- dt_depths[, .(.N, Cover = mean(Cover)), Depth]\n setkey(dt_summaries, 'Depth')\n\n if (which == \"2x1\") {\n p1 <-\n ggplot2::ggplot(dt_summaries) +\n ggplot2::geom_bar(ggplot2::aes(x = Depth, y = N), stat = \"Identity\") +\n ggplot2::xlab(\"\") +\n ggplot2::ylab(\"Number of leafs\") +\n ggplot2::ggtitle(\"Model complexity\") +\n ggplot2::theme(\n plot.title = ggplot2::element_text(lineheight = 0.9, face = \"bold\"),\n panel.grid.major.y = ggplot2::element_blank(),\n axis.ticks = ggplot2::element_blank(),\n axis.text.x = ggplot2::element_blank()\n )\n\n p2 <-\n ggplot2::ggplot(dt_summaries) +\n ggplot2::geom_bar(ggplot2::aes(x = Depth, y = Cover), stat = \"Identity\") +\n ggplot2::xlab(\"Leaf depth\") +\n ggplot2::ylab(\"Weighted cover\")\n\n multiplot(p1, p2, cols = 1)\n return(invisible(list(p1, p2)))\n\n } else if (which == \"max.depth\") {\n p <-\n ggplot2::ggplot(dt_depths[, max(Depth), Tree]) +\n ggplot2::geom_jitter(ggplot2::aes(x = Tree, y = V1),\n height = 0.15, alpha = 0.4, size = 3, stroke = 0) +\n ggplot2::xlab(\"tree #\") +\n ggplot2::ylab(\"Max tree leaf depth\")\n return(p)\n\n } else if (which == \"med.depth\") {\n p <-\n ggplot2::ggplot(dt_depths[, median(as.numeric(Depth)), Tree]) +\n ggplot2::geom_jitter(ggplot2::aes(x = Tree, y = V1),\n height = 0.15, alpha = 0.4, size = 3, stroke = 0) +\n ggplot2::xlab(\"tree #\") +\n ggplot2::ylab(\"Median tree leaf depth\")\n return(p)\n\n } else if (which == \"med.weight\") {\n p <-\n ggplot2::ggplot(dt_depths[, median(abs(Weight)), Tree]) +\n ggplot2::geom_point(ggplot2::aes(x = Tree, y = V1),\n alpha = 0.4, size = 3, stroke = 0) +\n ggplot2::xlab(\"tree #\") +\n ggplot2::ylab(\"Median absolute leaf weight\")\n return(p)\n }\n}\n\n#' @rdname xgb.plot.shap.summary\n#' @export\nxgb.ggplot.shap.summary <- function(data, shap_contrib = NULL, features = NULL, top_n = 10, model = NULL,\n trees = NULL, target_class = NULL, approxcontrib = FALSE, subsample = NULL) {\n if (inherits(data, \"xgb.DMatrix\")) {\n stop(\n \"'xgb.ggplot.shap.summary' is not compatible with 'xgb.DMatrix' objects. Try passing a matrix or data.frame.\"\n )\n }\n cols_categ <- NULL\n if (!is.null(model)) {\n ftypes <- getinfo(model, \"feature_type\")\n if (NROW(ftypes)) {\n if (length(ftypes) != ncol(data)) {\n stop(sprintf(\"'data' has incorrect number of columns (expected: %d, got: %d).\", length(ftypes), ncol(data)))\n }\n cols_categ <- colnames(data)[ftypes == \"c\"]\n }\n } else if (inherits(data, \"data.frame\")) {\n cols_categ <- names(data)[sapply(data, function(x) is.factor(x) || is.character(x))]\n }\n if (NROW(cols_categ)) {\n warning(\"Categorical features are ignored in 'xgb.ggplot.shap.summary'.\")\n }\n\n data_list <- xgb.shap.data(\n data = data,\n shap_contrib = shap_contrib,\n features = features,\n top_n = top_n,\n model = model,\n trees = trees,\n target_class = target_class,\n approxcontrib = approxcontrib,\n subsample = subsample,\n max_observations = 10000 # 10,000 samples per feature.\n )\n if (NROW(cols_categ)) {\n data_list <- lapply(data_list, function(x) x[, !(colnames(x) %in% cols_categ), drop = FALSE])\n }\n\n p_data <- prepare.ggplot.shap.data(data_list, normalize = TRUE)\n # Reverse factor levels so that the first level is at the top of the plot\n p_data[, \"feature\" := factor(feature, rev(levels(feature)))]\n p <- ggplot2::ggplot(p_data, ggplot2::aes(x = feature, y = p_data$shap_value, colour = p_data$feature_value)) +\n ggplot2::geom_jitter(alpha = 0.5, width = 0.1) +\n ggplot2::scale_colour_viridis_c(limits = c(-3, 3), option = \"plasma\", direction = -1) +\n ggplot2::geom_abline(slope = 0, intercept = 0, colour = \"darkgrey\") +\n ggplot2::coord_flip()\n\n p\n}\n\n#' Combine feature values and SHAP values\n#'\n#' Internal function used to combine and melt feature values and SHAP contributions\n#' as required for ggplot functions related to SHAP.\n#'\n#' @param data_list The result of `xgb.shap.data()`.\n#' @param normalize Whether to standardize feature values to mean 0 and\n#' standard deviation 1. This is useful for comparing multiple features on the same\n#' plot. Default is `FALSE`. Note that it cannot be used when the data contains\n#' categorical features.\n#' @return A `data.table` containing the observation ID, the feature name, the\n#' feature value (normalized if specified), and the SHAP contribution value.\n#' @noRd\n#' @keywords internal\nprepare.ggplot.shap.data <- function(data_list, normalize = FALSE) {\n data <- data_list[[\"data\"]]\n shap_contrib <- data_list[[\"shap_contrib\"]]\n\n data <- data.table::as.data.table(as.matrix(data))\n if (normalize) {\n data[, (names(data)) := lapply(.SD, normalize)]\n }\n data[, \"id\" := seq_len(nrow(data))]\n data_m <- data.table::melt.data.table(data, id.vars = \"id\", variable.name = \"feature\", value.name = \"feature_value\")\n\n shap_contrib <- data.table::as.data.table(as.matrix(shap_contrib))\n shap_contrib[, \"id\" := seq_len(nrow(shap_contrib))]\n shap_contrib_m <- data.table::melt.data.table(shap_contrib, id.vars = \"id\", variable.name = \"feature\", value.name = \"shap_value\")\n\n p_data <- data.table::merge.data.table(data_m, shap_contrib_m, by = c(\"id\", \"feature\"))\n\n p_data\n}\n\n#' Scale feature values\n#'\n#' Internal function that scales feature values to mean 0 and standard deviation 1.\n#' Useful to compare multiple features on the same plot.\n#'\n#' @param x Numeric vector.\n#' @return Numeric vector with mean 0 and standard deviation 1.\n#' @noRd\n#' @keywords internal\nnormalize <- function(x) {\n loc <- mean(x, na.rm = TRUE)\n scale <- stats::sd(x, na.rm = TRUE)\n\n (x - loc) / scale\n}\n\n# Plot multiple ggplot graph aligned by rows and columns.\n# ... the plots\n# cols number of columns\n# internal utility function\nmultiplot <- function(..., cols) {\n plots <- list(...)\n num_plots <- length(plots)\n\n layout <- matrix(seq(1, cols * ceiling(num_plots / cols)),\n ncol = cols, nrow = ceiling(num_plots / cols))\n\n if (num_plots == 1) {\n print(plots[[1]])\n } else {\n grid::grid.newpage()\n grid::pushViewport(grid::viewport(layout = grid::grid.layout(nrow(layout), ncol(layout))))\n for (i in 1:num_plots) {\n # Get the i,j matrix positions of the regions that contain this subplot\n matchidx <- as.data.table(which(layout == i, arr.ind = TRUE))\n\n print(\n plots[[i]], vp = grid::viewport(\n layout.pos.row = matchidx$row,\n layout.pos.col = matchidx$col\n )\n )\n }\n }\n}\n\nglobalVariables(c(\n \"Cluster\", \"ggplot\", \"aes\", \"geom_bar\", \"coord_flip\", \"xlab\", \"ylab\", \"ggtitle\", \"theme\",\n \"element_blank\", \"element_text\", \"V1\", \"Weight\", \"feature\"\n))\n" }, { "path": "R-package/R/xgb.importance.R", "content": "#' Feature importance\n#'\n#' Creates a `data.table` of feature importances.\n#'\n#' @details\n#' This function works for both linear and tree models.\n#'\n#' For linear models, the importance is the absolute magnitude of linear coefficients.\n#' To obtain a meaningful ranking by importance for linear models, the features need to\n#' be on the same scale (which is also recommended when using L1 or L2 regularization).\n#'\n#' @param feature_names Character vector used to overwrite the feature names\n#' of the model. The default is `NULL` (use original feature names).\n#' @param model Object of class `xgb.Booster`.\n#' @param trees An integer vector of (base-1) tree indices that should be included\n#' into the importance calculation (only for the \"gbtree\" booster).\n#' The default (`NULL`) parses all trees.\n#' It could be useful, e.g., in multiclass classification to get feature importances\n#' for each class separately.\n#' @return A `data.table` with the following columns:\n#'\n#' For a tree model:\n#' - `Features`: Names of the features used in the model.\n#' - `Gain`: Fractional contribution of each feature to the model based on\n#' the total gain of this feature's splits. Higher percentage means higher importance.\n#' - `Cover`: Metric of the number of observation related to this feature.\n#' - `Frequency`: Percentage of times a feature has been used in trees.\n#'\n#' For a linear model:\n#' - `Features`: Names of the features used in the model.\n#' - `Weight`: Linear coefficient of this feature.\n#' - `Class`: Class label (only for multiclass models). For objects of class `xgboost` (as\n#' produced by [xgboost()]), it will be a `factor`, while for objects of class `xgb.Booster`\n#' (as produced by [xgb.train()]), it will be a zero-based integer vector.\n#'\n#' If `feature_names` is not provided and `model` doesn't have `feature_names`,\n#' the index of the features will be used instead. Because the index is extracted from the model dump\n#' (based on C++ code), it starts at 0 (as in C/C++ or Python) instead of 1 (usual in R).\n#'\n#' @examples\n#' # binary classification using \"gbtree\":\n#' data(\"ToothGrowth\")\n#' x <- ToothGrowth[, c(\"len\", \"dose\")]\n#' y <- ToothGrowth$supp\n#' model_tree_binary <- xgboost(\n#' x, y,\n#' nrounds = 5L,\n#' nthreads = 1L,\n#' booster = \"gbtree\",\n#' max_depth = 2L\n#' )\n#' xgb.importance(model_tree_binary)\n#'\n#' # binary classification using \"gblinear\":\n#' model_tree_linear <- xgboost(\n#' x, y,\n#' nrounds = 5L,\n#' nthreads = 1L,\n#' booster = \"gblinear\",\n#' learning_rate = 0.3\n#' )\n#' xgb.importance(model_tree_linear)\n#'\n#' # multi-class classification using \"gbtree\":\n#' data(\"iris\")\n#' x <- iris[, c(\"Sepal.Length\", \"Sepal.Width\", \"Petal.Length\", \"Petal.Width\")]\n#' y <- iris$Species\n#' model_tree_multi <- xgboost(\n#' x, y,\n#' nrounds = 5L,\n#' nthreads = 1L,\n#' booster = \"gbtree\",\n#' max_depth = 3\n#' )\n#' # all classes clumped together:\n#' xgb.importance(model_tree_multi)\n#' # inspect importances separately for each class:\n#' num_classes <- 3L\n#' nrounds <- 5L\n#' xgb.importance(\n#' model_tree_multi, trees = seq(from = 1, by = num_classes, length.out = nrounds)\n#' )\n#' xgb.importance(\n#' model_tree_multi, trees = seq(from = 2, by = num_classes, length.out = nrounds)\n#' )\n#' xgb.importance(\n#' model_tree_multi, trees = seq(from = 3, by = num_classes, length.out = nrounds)\n#' )\n#'\n#' # multi-class classification using \"gblinear\":\n#' model_linear_multi <- xgboost(\n#' x, y,\n#' nrounds = 5L,\n#' nthreads = 1L,\n#' booster = \"gblinear\",\n#' learning_rate = 0.2\n#' )\n#' xgb.importance(model_linear_multi)\n#' @export\nxgb.importance <- function(model = NULL, feature_names = getinfo(model, \"feature_name\"), trees = NULL) {\n\n if (!(is.null(feature_names) || is.character(feature_names)))\n stop(\"feature_names: Has to be a character vector\")\n\n if (!is.null(trees)) {\n if (!is.vector(trees)) {\n stop(\"'trees' must be a vector of tree indices.\")\n }\n trees <- trees - 1L\n if (anyNA(trees)) {\n stop(\"Passed invalid tree indices.\")\n }\n }\n\n handle <- xgb.get.handle(model)\n if (xgb.booster_type(model) == \"gblinear\") {\n args <- list(importance_type = \"weight\", feature_names = feature_names)\n results <- .Call(\n XGBoosterFeatureScore_R, handle, jsonlite::toJSON(args, auto_unbox = TRUE, null = \"null\")\n )\n names(results) <- c(\"features\", \"shape\", \"weight\")\n if (length(results$shape) == 2) {\n n_classes <- results$shape[2]\n } else {\n n_classes <- 0\n }\n importance <- if (n_classes == 0) {\n return(data.table(Feature = results$features, Weight = results$weight)[order(-abs(Weight))])\n } else {\n out <- data.table(\n Feature = rep(results$features, each = n_classes), Weight = results$weight, Class = seq_len(n_classes) - 1\n )[order(Class, -abs(Weight))]\n if (inherits(model, \"xgboost\") && NROW(attributes(model)$metadata$y_levels)) {\n class_vec <- out$Class\n class_vec <- as.integer(class_vec) + 1L\n attributes(class_vec)$levels <- attributes(model)$metadata$y_levels\n attributes(class_vec)$class <- \"factor\"\n out[, Class := class_vec]\n }\n return(out[])\n }\n } else {\n concatenated <- list()\n output_names <- vector()\n for (importance_type in c(\"weight\", \"total_gain\", \"total_cover\")) {\n args <- list(importance_type = importance_type, feature_names = feature_names, tree_idx = trees)\n results <- .Call(\n XGBoosterFeatureScore_R, handle, jsonlite::toJSON(args, auto_unbox = TRUE, null = \"null\")\n )\n names(results) <- c(\"features\", \"shape\", importance_type)\n concatenated[\n switch(importance_type, \"weight\" = \"Frequency\", \"total_gain\" = \"Gain\", \"total_cover\" = \"Cover\")\n ] <- results[importance_type]\n output_names <- results$features\n }\n importance <- data.table(\n Feature = output_names,\n Gain = concatenated$Gain / sum(concatenated$Gain),\n Cover = concatenated$Cover / sum(concatenated$Cover),\n Frequency = concatenated$Frequency / sum(concatenated$Frequency)\n )[order(Gain, decreasing = TRUE)]\n }\n importance\n}\n\n# Avoid error messages during CRAN check.\n# The reason is that these variables are never declared\n# They are mainly column names inferred by Data.table...\nglobalVariables(c(\".\", \".N\", \"Gain\", \"Cover\", \"Frequency\", \"Feature\", \"Class\"))\n" }, { "path": "R-package/R/xgb.load.R", "content": "#' Load XGBoost model from binary file\n#'\n#' Load XGBoost model from binary model file.\n#'\n#' @param modelfile The name of the binary input file.\n#'\n#' @details\n#' The input file is expected to contain a model saved in an XGBoost model format\n#' using either [xgb.save()] in R, or using some\n#' appropriate methods from other XGBoost interfaces. E.g., a model trained in Python and\n#' saved from there in XGBoost format, could be loaded from R.\n#'\n#' Note: a model saved as an R object has to be loaded using corresponding R-methods,\n#' not by [xgb.load()].\n#'\n#' @return\n#' An object of `xgb.Booster` class.\n#'\n#' @seealso [xgb.save()]\n#'\n#' @examples\n#' \\dontshow{RhpcBLASctl::omp_set_num_threads(1)}\n#' data(agaricus.train, package = \"xgboost\")\n#' data(agaricus.test, package = \"xgboost\")\n#'\n#' ## Keep the number of threads to 1 for examples\n#' nthread <- 1\n#' data.table::setDTthreads(nthread)\n#'\n#' train <- agaricus.train\n#' test <- agaricus.test\n#'\n#' bst <- xgb.train(\n#' data = xgb.DMatrix(train$data, label = train$label, nthread = 1),\n#' nrounds = 2,\n#' params = xgb.params(\n#' max_depth = 2,\n#' nthread = nthread,\n#' objective = \"binary:logistic\"\n#' )\n#' )\n#'\n#' fname <- file.path(tempdir(), \"xgb.ubj\")\n#' xgb.save(bst, fname)\n#' bst <- xgb.load(fname)\n#' @export\nxgb.load <- function(modelfile) {\n if (is.null(modelfile))\n stop(\"xgb.load: modelfile cannot be NULL\")\n\n bst <- xgb.Booster(\n params = list(),\n cachelist = list(),\n modelfile = modelfile\n )\n bst <- bst$bst\n # re-use modelfile if it is raw so we do not need to serialize\n if (typeof(modelfile) == \"raw\") {\n warning(\n paste(\n \"The support for loading raw booster with `xgb.load` will be \",\n \"discontinued in upcoming release. Use `xgb.load.raw` instead. \"\n )\n )\n }\n return(bst)\n}\n" }, { "path": "R-package/R/xgb.load.raw.R", "content": "#' Load serialised XGBoost model from R's raw vector\n#'\n#' User can generate raw memory buffer by calling [xgb.save.raw()].\n#'\n#' @param buffer The buffer returned by [xgb.save.raw()].\n#' @export\nxgb.load.raw <- function(buffer) {\n cachelist <- list()\n bst <- .Call(XGBoosterCreate_R, cachelist)\n .Call(XGBoosterLoadModelFromRaw_R, xgb.get.handle(bst), buffer)\n return(bst)\n}\n" }, { "path": "R-package/R/xgb.model.dt.tree.R", "content": "#' Parse model text dump\n#'\n#' Parse a boosted tree model text dump into a `data.table` structure.\n#'\n#' Note that this function does not work with models that were fitted to\n#' categorical data, and is only applicable to tree-based boosters (not `gblinear`).\n#' @param model Object of class `xgb.Booster`. If it contains feature names (they can\n#' be set through [setinfo()]), they will be used in the output from this function.\n#'\n#' If the model contains categorical features, an error will be thrown.\n#' @param trees An integer vector of (base-1) tree indices that should be used. The default\n#' (`NULL`) uses all trees. Useful, e.g., in multiclass classification to get only\n#' the trees of one class.\n#' @param use_int_id A logical flag indicating whether nodes in columns \"Yes\", \"No\", and\n#' \"Missing\" should be represented as integers (when `TRUE`) or as \"Tree-Node\"\n#' character strings (when `FALSE`, default).\n#' @inheritParams xgb.train\n#' @return\n#' A `data.table` with detailed information about tree nodes. It has the following columns:\n#' - `Tree`: integer ID of a tree in a model (zero-based index).\n#' - `Node`: integer ID of a node in a tree (zero-based index).\n#' - `ID`: character identifier of a node in a model (only when `use_int_id = FALSE`).\n#' - `Feature`: for a branch node, a feature ID or name (when available);\n#' for a leaf node, it simply labels it as `\"Leaf\"`.\n#' - `Split`: location of the split for a branch node (split condition is always \"less than\").\n#' - `Yes`: ID of the next node when the split condition is met.\n#' - `No`: ID of the next node when the split condition is not met.\n#' - `Missing`: ID of the next node when the branch value is missing.\n#' - `Gain`: either the split gain (change in loss) or the leaf value.\n#' - `Cover`: metric related to the number of observations either seen by a split\n#' or collected by a leaf during training.\n#'\n#' When `use_int_id = FALSE`, columns \"Yes\", \"No\", and \"Missing\" point to model-wide node identifiers\n#' in the \"ID\" column. When `use_int_id = TRUE`, those columns point to node identifiers from\n#' the corresponding trees in the \"Node\" column.\n#'\n#' @examples\n#' # Basic use:\n#'\n#' data(agaricus.train, package = \"xgboost\")\n#' ## Keep the number of threads to 1 for examples\n#' nthread <- 1\n#' data.table::setDTthreads(nthread)\n#'\n#' bst <- xgb.train(\n#' data = xgb.DMatrix(agaricus.train$data, label = agaricus.train$label, nthread = 1),\n#' nrounds = 2,\n#' params = xgb.params(\n#' max_depth = 2,\n#' nthread = nthread,\n#' objective = \"binary:logistic\"\n#' )\n#' )\n#'\n#' # This bst model already has feature_names stored with it, so those would be used when\n#' # feature_names is not set:\n#' dt <- xgb.model.dt.tree(bst)\n#'\n#' # How to match feature names of splits that are following a current 'Yes' branch:\n#' merge(\n#' dt,\n#' dt[, .(ID, Y.Feature = Feature)], by.x = \"Yes\", by.y = \"ID\", all.x = TRUE\n#' )[\n#' order(Tree, Node)\n#' ]\n#'\n#' @export\nxgb.model.dt.tree <- function(model, trees = NULL, use_int_id = FALSE, ...) {\n check.deprecation(deprecated_dttree_params, match.call(), ...)\n\n if (!inherits(model, \"xgb.Booster\")) {\n stop(\"Either 'model' must be an object of class xgb.Booster\")\n }\n\n if (xgb.has_categ_features(model)) {\n stop(\"Cannot produce tables for models having categorical features.\")\n }\n\n if (!is.null(trees)) {\n if (!is.vector(trees) || (!is.numeric(trees) && !is.integer(trees))) {\n stop(\"trees: must be a vector of integers.\")\n }\n trees <- trees - 1L\n if (anyNA(trees) || min(trees) < 0) {\n stop(\"Passed invalid tree indices.\")\n }\n }\n\n feature_names <- NULL\n if (inherits(model, \"xgb.Booster\")) {\n feature_names <- xgb.feature_names(model)\n }\n\n text <- xgb.dump(model = model, with_stats = TRUE)\n\n if (length(text) < 2 || !any(grepl('leaf=(-?\\\\d+)', text))) {\n stop(\"Non-tree model detected! This function can only be used with tree models.\")\n }\n\n position <- which(grepl(\"booster\", text, fixed = TRUE))\n\n add.tree.id <- function(node, tree) if (use_int_id) node else paste(tree, node, sep = \"-\")\n\n anynumber_regex <- \"[-+]?[0-9]*\\\\.?[0-9]+([eE][-+]?[0-9]+)?\"\n\n td <- data.table(t = text)\n td[position, Tree := 1L]\n td[, Tree := cumsum(ifelse(is.na(Tree), 0L, Tree)) - 1L]\n\n if (is.null(trees)) {\n trees <- 0:max(td$Tree)\n } else {\n trees <- trees[trees >= 0 & trees <= max(td$Tree)]\n }\n td <- td[Tree %in% trees & !is.na(t) & !startsWith(t, 'booster')]\n\n td[, Node := as.integer(sub(\"^([0-9]+):.*\", \"\\\\1\", t))]\n if (!use_int_id) td[, ID := add.tree.id(Node, Tree)]\n td[, isLeaf := grepl(\"leaf\", t, fixed = TRUE)]\n\n # parse branch lines\n branch_rx_nonames <- paste0(\"f(\\\\d+)<(\", anynumber_regex, \")\\\\] yes=(\\\\d+),no=(\\\\d+),missing=(\\\\d+),\",\n \"gain=(\", anynumber_regex, \"),cover=(\", anynumber_regex, \")\")\n branch_rx_w_names <- paste0(\"\\\\d+:\\\\[(.+)<(\", anynumber_regex, \")\\\\] yes=(\\\\d+),no=(\\\\d+),missing=(\\\\d+),\",\n \"gain=(\", anynumber_regex, \"),cover=(\", anynumber_regex, \")\")\n text_has_feature_names <- FALSE\n if (NROW(feature_names)) {\n branch_rx <- branch_rx_w_names\n text_has_feature_names <- TRUE\n } else {\n branch_rx <- branch_rx_nonames\n }\n branch_cols <- c(\"Feature\", \"Split\", \"Yes\", \"No\", \"Missing\", \"Gain\", \"Cover\")\n td[\n isLeaf == FALSE,\n (branch_cols) := {\n matches <- regmatches(t, regexec(branch_rx, t))\n # skip some indices with spurious capture groups from anynumber_regex\n xtr <- do.call(rbind, matches)[, c(2, 3, 5, 6, 7, 8, 10), drop = FALSE]\n xtr[, 3:5] <- add.tree.id(xtr[, 3:5], Tree)\n if (length(xtr) == 0) {\n as.data.table(\n list(Feature = \"NA\", Split = \"NA\", Yes = \"NA\", No = \"NA\", Missing = \"NA\", Gain = \"NA\", Cover = \"NA\")\n )\n } else {\n as.data.table(xtr)\n }\n }\n ]\n\n # assign feature_names when available\n is_stump <- function() {\n return(length(td$Feature) == 1 && is.na(td$Feature))\n }\n if (!text_has_feature_names) {\n if (!is.null(feature_names) && !is_stump()) {\n if (length(feature_names) <= max(as.numeric(td$Feature), na.rm = TRUE))\n stop(\"feature_names has less elements than there are features used in the model\")\n td[isLeaf == FALSE, Feature := feature_names[as.numeric(Feature) + 1]]\n }\n }\n\n # parse leaf lines\n leaf_rx <- paste0(\"leaf=(\", anynumber_regex, \"),cover=(\", anynumber_regex, \")\")\n leaf_cols <- c(\"Feature\", \"Gain\", \"Cover\")\n td[\n isLeaf == TRUE,\n (leaf_cols) := {\n matches <- regmatches(t, regexec(leaf_rx, t))\n xtr <- do.call(rbind, matches)[, c(2, 4)]\n if (length(xtr) == 2) {\n c(\"Leaf\", as.data.table(xtr[1]), as.data.table(xtr[2]))\n } else {\n c(\"Leaf\", as.data.table(xtr))\n }\n }\n ]\n\n # convert some columns to numeric\n numeric_cols <- c(\"Split\", \"Gain\", \"Cover\")\n td[, (numeric_cols) := lapply(.SD, as.numeric), .SDcols = numeric_cols]\n if (use_int_id) {\n int_cols <- c(\"Yes\", \"No\", \"Missing\")\n td[, (int_cols) := lapply(.SD, as.integer), .SDcols = int_cols]\n }\n\n td[, t := NULL]\n td[, isLeaf := NULL]\n\n td[order(Tree, Node)]\n}\n\n# Avoid notes during CRAN check.\n# The reason is that these variables are never declared\n# They are mainly column names inferred by Data.table...\nglobalVariables(c(\"Tree\", \"Node\", \"ID\", \"Feature\", \"t\", \"isLeaf\", \".SD\", \".SDcols\"))\n" }, { "path": "R-package/R/xgb.plot.deepness.R", "content": "#' Plot model tree depth\n#'\n#' Visualizes distributions related to the depth of tree leaves.\n#' - `xgb.plot.deepness()` uses base R graphics, while\n#' - `xgb.ggplot.deepness()` uses \"ggplot2\".\n#'\n#' @param model Either an `xgb.Booster` model, or the \"data.table\" returned\n#' by [xgb.model.dt.tree()].\n#' @param which Which distribution to plot (see details).\n#' @param plot Should the plot be shown? Default is `TRUE`.\n#' @param ... Other parameters passed to [graphics::barplot()] or [graphics::plot()].\n#'\n#' @details\n#'\n#' When `which = \"2x1\"`, two distributions with respect to the leaf depth\n#' are plotted on top of each other:\n#' 1. The distribution of the number of leaves in a tree model at a certain depth.\n#' 2. The distribution of the average weighted number of observations (\"cover\")\n#' ending up in leaves at a certain depth.\n#'\n#' Those could be helpful in determining sensible ranges of the `max_depth`\n#' and `min_child_weight` parameters.\n#'\n#' When `which = \"max.depth\"` or `which = \"med.depth\"`, plots of either maximum or\n#' median depth per tree with respect to the tree number are created.\n#'\n#' Finally, `which = \"med.weight\"` allows to see how\n#' a tree's median absolute leaf weight changes through the iterations.\n#'\n#' These functions have been inspired by the blog post\n#' .\n#'\n#' @return\n#' The return value of the two functions is as follows:\n#' - `xgb.plot.deepness()`: A \"data.table\" (invisibly).\n#' Each row corresponds to a terminal leaf in the model. It contains its information\n#' about depth, cover, and weight (used in calculating predictions).\n#' If `plot = TRUE`, also a plot is shown.\n#' - `xgb.ggplot.deepness()`: When `which = \"2x1\"`, a list of two \"ggplot\" objects,\n#' and a single \"ggplot\" object otherwise.\n#'\n#' @seealso [xgb.train()] and [xgb.model.dt.tree()].\n#'\n#' @examples\n#'\n#' data(agaricus.train, package = \"xgboost\")\n#' ## Keep the number of threads to 2 for examples\n#' nthread <- 2\n#' data.table::setDTthreads(nthread)\n#'\n#' ## Change max_depth to a higher number to get a more significant result\n#' model <- xgboost(\n#' agaricus.train$data, factor(agaricus.train$label),\n#' nrounds = 50,\n#' max_depth = 6,\n#' nthreads = nthread,\n#' subsample = 0.5,\n#' min_child_weight = 2\n#' )\n#'\n#' xgb.plot.deepness(model)\n#' xgb.ggplot.deepness(model)\n#'\n#' xgb.plot.deepness(\n#' model, which = \"max.depth\", pch = 16, col = rgb(0, 0, 1, 0.3), cex = 2\n#' )\n#'\n#' xgb.plot.deepness(\n#' model, which = \"med.weight\", pch = 16, col = rgb(0, 0, 1, 0.3), cex = 2\n#' )\n#'\n#' @rdname xgb.plot.deepness\n#' @export\nxgb.plot.deepness <- function(model = NULL, which = c(\"2x1\", \"max.depth\", \"med.depth\", \"med.weight\"),\n plot = TRUE, ...) {\n\n if (!(inherits(model, \"xgb.Booster\") || is.data.table(model)))\n stop(\"model: Has to be either an xgb.Booster model generaged by the xgb.train function\\n\",\n \"or a data.table result of the xgb.importance function\")\n\n if (!requireNamespace(\"igraph\", quietly = TRUE))\n stop(\"igraph package is required for plotting the graph deepness.\", call. = FALSE)\n\n which <- match.arg(which)\n\n dt_tree <- model\n if (inherits(model, \"xgb.Booster\"))\n dt_tree <- xgb.model.dt.tree(model = model)\n\n if (!all(c(\"Feature\", \"Tree\", \"ID\", \"Yes\", \"No\", \"Cover\") %in% colnames(dt_tree)))\n stop(\"Model tree columns are not as expected!\\n\",\n \" Note that this function works only for tree models.\")\n\n dt_depths <- merge(get.leaf.depth(dt_tree), dt_tree[, .(ID, Cover, Weight = Gain)], by = \"ID\")\n setkeyv(dt_depths, c(\"Tree\", \"ID\"))\n # count by depth levels, and also calculate average cover at a depth\n dt_summaries <- dt_depths[, .(.N, Cover = mean(Cover)), Depth]\n setkey(dt_summaries, \"Depth\")\n\n if (plot) {\n if (which == \"2x1\") {\n op <- par(no.readonly = TRUE)\n par(mfrow = c(2, 1),\n oma = c(3, 1, 3, 1) + 0.1,\n mar = c(1, 4, 1, 0) + 0.1)\n\n dt_summaries[, barplot(N, border = NA, ylab = 'Number of leafs', ...)]\n\n dt_summaries[, barplot(Cover, border = NA, ylab = \"Weighted cover\", names.arg = Depth, ...)]\n\n title(\"Model complexity\", xlab = \"Leaf depth\", outer = TRUE, line = 1)\n par(op)\n } else if (which == \"max.depth\") {\n dt_depths[, max(Depth), Tree][\n , plot(jitter(V1, amount = 0.1) ~ Tree, ylab = 'Max tree leaf depth', xlab = \"tree #\", ...)]\n } else if (which == \"med.depth\") {\n dt_depths[, median(as.numeric(Depth)), Tree][\n , plot(jitter(V1, amount = 0.1) ~ Tree, ylab = 'Median tree leaf depth', xlab = \"tree #\", ...)]\n } else if (which == \"med.weight\") {\n dt_depths[, median(abs(Weight)), Tree][\n , plot(V1 ~ Tree, ylab = 'Median absolute leaf weight', xlab = \"tree #\", ...)]\n }\n }\n invisible(dt_depths)\n}\n\n# Extract path depths from root to leaf\n# from data.table containing the nodes and edges of the trees.\n# internal utility function\nget.leaf.depth <- function(dt_tree) {\n # extract tree graph's edges\n dt_edges <- rbindlist(list(\n dt_tree[Feature != \"Leaf\", .(ID, To = Yes, Tree)],\n dt_tree[Feature != \"Leaf\", .(ID, To = No, Tree)]\n ))\n # whether \"To\" is a leaf:\n dt_edges <-\n merge(dt_edges,\n dt_tree[Feature == \"Leaf\", .(ID, Leaf = TRUE)],\n all.x = TRUE, by.x = \"To\", by.y = \"ID\")\n dt_edges[is.na(Leaf), Leaf := FALSE]\n\n dt_edges[, {\n graph <- igraph::graph_from_data_frame(.SD[, .(ID, To)])\n # min(ID) in a tree is a root node\n paths_tmp <- igraph::shortest_paths(graph, from = min(ID), to = To[Leaf == TRUE])\n # list of paths to each leaf in a tree\n paths <- lapply(paths_tmp$vpath, names)\n # combine into a resulting path lengths table for a tree\n data.table(Depth = lengths(paths), ID = To[Leaf == TRUE])\n }, by = Tree]\n}\n\n# Avoid error messages during CRAN check.\n# The reason is that these variables are never declared\n# They are mainly column names inferred by Data.table...\nglobalVariables(\n c(\n \".N\", \"N\", \"Depth\", \"Gain\", \"Cover\", \"Tree\", \"ID\", \"Yes\", \"No\", \"Feature\", \"Leaf\", \"Weight\"\n )\n)\n" }, { "path": "R-package/R/xgb.plot.importance.R", "content": "#' Plot feature importance\n#'\n#' Represents previously calculated feature importance as a bar graph.\n#' - `xgb.plot.importance()` uses base R graphics, while\n#' - `xgb.ggplot.importance()` uses \"ggplot\".\n#'\n#' @details\n#' The graph represents each feature as a horizontal bar of length proportional to the\n#' importance of a feature. Features are sorted by decreasing importance.\n#' It works for both \"gblinear\" and \"gbtree\" models.\n#'\n#' When `rel_to_first = FALSE`, the values would be plotted as in `importance_matrix`.\n#' For a \"gbtree\" model, that would mean being normalized to the total of 1\n#' (\"what is feature's importance contribution relative to the whole model?\").\n#' For linear models, `rel_to_first = FALSE` would show actual values of the coefficients.\n#' Setting `rel_to_first = TRUE` allows to see the picture from the perspective of\n#' \"what is feature's importance contribution relative to the most important feature?\"\n#'\n#' The \"ggplot\" backend performs 1-D clustering of the importance values,\n#' with bar colors corresponding to different clusters having similar importance values.\n#'\n#' @param importance_matrix A `data.table` as returned by [xgb.importance()].\n#' @param top_n Maximal number of top features to include into the plot.\n#' @param measure The name of importance measure to plot.\n#' When `NULL`, 'Gain' would be used for trees and 'Weight' would be used for gblinear.\n#' @param rel_to_first Whether importance values should be represented as relative to\n#' the highest ranked feature, see Details.\n#' @param left_margin Adjust the left margin size to fit feature names.\n#' When `NULL`, the existing `par(\"mar\")` is used.\n#' @param cex Passed as `cex.names` parameter to [graphics::barplot()].\n#' @param plot Should the barplot be shown? Default is `TRUE`.\n#' @param n_clusters A numeric vector containing the min and the max range\n#' of the possible number of clusters of bars.\n#' @param ... Other parameters passed to [graphics::barplot()]\n#' (except `horiz`, `border`, `cex.names`, `names.arg`, and `las`).\n#' Only used in `xgb.plot.importance()`.\n#' @return\n#' The return value depends on the function:\n#' - `xgb.plot.importance()`: Invisibly, a \"data.table\" with `n_top` features sorted\n#' by importance. If `plot = TRUE`, the values are also plotted as barplot.\n#' - `xgb.ggplot.importance()`: A customizable \"ggplot\" object.\n#' E.g., to change the title, set `+ ggtitle(\"A GRAPH NAME\")`.\n#'\n#' @seealso [graphics::barplot()]\n#'\n#' @examples\n#' data(agaricus.train)\n#'\n#' ## Keep the number of threads to 2 for examples\n#' nthread <- 2\n#' data.table::setDTthreads(nthread)\n#'\n#' model <- xgboost(\n#' agaricus.train$data, factor(agaricus.train$label),\n#' nrounds = 2,\n#' max_depth = 3,\n#' nthreads = nthread\n#' )\n#'\n#' importance_matrix <- xgb.importance(model)\n#' xgb.plot.importance(\n#' importance_matrix, rel_to_first = TRUE, xlab = \"Relative importance\"\n#' )\n#'\n#' gg <- xgb.ggplot.importance(\n#' importance_matrix, measure = \"Frequency\", rel_to_first = TRUE\n#' )\n#' gg\n#' gg + ggplot2::ylab(\"Frequency\")\n#'\n#' @rdname xgb.plot.importance\n#' @export\nxgb.plot.importance <- function(importance_matrix = NULL, top_n = NULL, measure = NULL,\n rel_to_first = FALSE, left_margin = 10, cex = NULL, plot = TRUE, ...) {\n check.deprecation(deprecated_plotimp_params, match.call(), ..., allow_unrecognized = TRUE)\n if (!is.data.table(importance_matrix)) {\n stop(\"importance_matrix: must be a data.table\")\n }\n\n imp_names <- colnames(importance_matrix)\n if (is.null(measure)) {\n if (all(c(\"Feature\", \"Gain\") %in% imp_names)) {\n measure <- \"Gain\"\n } else if (all(c(\"Feature\", \"Weight\") %in% imp_names)) {\n measure <- \"Weight\"\n } else {\n stop(\"Importance matrix column names are not as expected!\")\n }\n } else {\n if (!measure %in% imp_names)\n stop(\"Invalid `measure`\")\n if (!\"Feature\" %in% imp_names)\n stop(\"Importance matrix column names are not as expected!\")\n }\n\n # also aggregate, just in case when the values were not yet summed up by feature\n importance_matrix <- importance_matrix[\n , lapply(.SD, sum)\n , .SDcols = setdiff(names(importance_matrix), \"Feature\")\n , by = Feature\n ][\n , Importance := get(measure)\n ]\n\n # make sure it's ordered\n importance_matrix <- importance_matrix[order(-abs(Importance))]\n\n if (!is.null(top_n)) {\n top_n <- min(top_n, nrow(importance_matrix))\n importance_matrix <- head(importance_matrix, top_n)\n }\n if (rel_to_first) {\n importance_matrix[, Importance := Importance / max(abs(Importance))]\n }\n if (is.null(cex)) {\n cex <- 2.5 / log2(1 + nrow(importance_matrix))\n }\n\n if (plot) {\n original_mar <- par()$mar\n\n # reset margins so this function doesn't have side effects\n on.exit({\n par(mar = original_mar)\n })\n\n mar <- original_mar\n if (!is.null(left_margin))\n mar[2] <- left_margin\n par(mar = mar)\n\n # reverse the order of rows to have the highest ranked at the top\n importance_matrix[rev(seq_len(nrow(importance_matrix))),\n barplot(Importance, horiz = TRUE, border = NA, cex.names = cex,\n names.arg = Feature, las = 1, ...)]\n }\n\n invisible(importance_matrix)\n}\n\n# Avoid error messages during CRAN check.\n# The reason is that these variables are never declared\n# They are mainly column names inferred by Data.table...\nglobalVariables(c(\"Feature\", \"Importance\"))\n" }, { "path": "R-package/R/xgb.plot.multi.trees.R", "content": "#' Project all trees on one tree\n#'\n#' Visualization of the ensemble of trees as a single collective unit.\n#'\n#' Note that this function does not work with models that were fitted to\n#' categorical data.\n#' @details\n#' This function tries to capture the complexity of a gradient boosted tree model\n#' in a cohesive way by compressing an ensemble of trees into a single tree-graph representation.\n#' The goal is to improve the interpretability of a model generally seen as black box.\n#'\n#' Note: this function is applicable to tree booster-based models only.\n#'\n#' It takes advantage of the fact that the shape of a binary tree is only defined by\n#' its depth (therefore, in a boosting model, all trees have similar shape).\n#'\n#' Moreover, the trees tend to reuse the same features.\n#'\n#' The function projects each tree onto one, and keeps for each position the\n#' `features_keep` first features (based on the Gain per feature measure).\n#'\n#' This function is inspired by this blog post:\n#' \n#'\n#' @inheritParams xgb.plot.tree\n#' @param features_keep Number of features to keep in each position of the multi trees,\n#' by default 5.\n#' @param render Should the graph be rendered or not? The default is `TRUE`.\n#' @inherit xgb.plot.tree return\n#'\n#' @examples\n#'\n#' data(agaricus.train, package = \"xgboost\")\n#'\n#' ## Keep the number of threads to 2 for examples\n#' nthread <- 2\n#' data.table::setDTthreads(nthread)\n#'\n#' model <- xgboost(\n#' agaricus.train$data, factor(agaricus.train$label),\n#' nrounds = 30,\n#' verbosity = 0L,\n#' nthreads = nthread,\n#' max_depth = 15,\n#' learning_rate = 1,\n#' min_child_weight = 50\n#' )\n#'\n#' p <- xgb.plot.multi.trees(model, features_keep = 3)\n#' print(p)\n#'\n#' # Below is an example of how to save this plot to a file.\n#' if (require(\"DiagrammeR\") && require(\"DiagrammeRsvg\") && require(\"rsvg\")) {\n#' fname <- file.path(tempdir(), \"tree.pdf\")\n#' gr <- xgb.plot.multi.trees(model, features_keep = 3, render = FALSE)\n#' export_graph(gr, fname, width = 1500, height = 600)\n#' }\n#' @export\nxgb.plot.multi.trees <- function(model, features_keep = 5, plot_width = NULL, plot_height = NULL,\n render = TRUE, ...) {\n check.deprecation(deprecated_multitrees_params, match.call(), ...)\n if (!requireNamespace(\"DiagrammeR\", quietly = TRUE)) {\n stop(\"DiagrammeR is required for xgb.plot.multi.trees\")\n }\n if (xgb.has_categ_features(model)) {\n stop(\n \"Cannot use 'xgb.plot.multi.trees' for models with categorical features.\",\n \" Try 'xgb.plot.tree' instead.\"\n )\n }\n tree.matrix <- xgb.model.dt.tree(model = model)\n\n # first number of the path represents the tree, then the following numbers are related to the path to follow\n # root init\n root.nodes <- tree.matrix[Node == 0, ID]\n tree.matrix[ID %in% root.nodes, abs.node.position := root.nodes]\n\n precedent.nodes <- root.nodes\n\n while (tree.matrix[, sum(is.na(abs.node.position))] > 0) {\n yes.row.nodes <- tree.matrix[abs.node.position %in% precedent.nodes & !is.na(Yes)]\n no.row.nodes <- tree.matrix[abs.node.position %in% precedent.nodes & !is.na(No)]\n yes.nodes.abs.pos <- paste0(yes.row.nodes[, abs.node.position], \"_0\")\n no.nodes.abs.pos <- paste0(no.row.nodes[, abs.node.position], \"_1\")\n\n tree.matrix[ID %in% yes.row.nodes[, Yes], abs.node.position := yes.nodes.abs.pos]\n tree.matrix[ID %in% no.row.nodes[, No], abs.node.position := no.nodes.abs.pos]\n precedent.nodes <- c(yes.nodes.abs.pos, no.nodes.abs.pos)\n }\n\n tree.matrix[!is.na(Yes), Yes := paste0(abs.node.position, \"_0\")]\n tree.matrix[!is.na(No), No := paste0(abs.node.position, \"_1\")]\n\n for (nm in c(\"abs.node.position\", \"Yes\", \"No\"))\n data.table::set(tree.matrix, j = nm, value = sub(\"^\\\\d+-\", \"\", tree.matrix[[nm]]))\n\n nodes.dt <- tree.matrix[\n , .(Gain = sum(Gain))\n , by = .(abs.node.position, Feature)\n ][, .(Text = paste0(\n paste0(\n Feature[seq_len(min(length(Feature), features_keep))],\n \" (\",\n format(Gain[seq_len(min(length(Gain), features_keep))], digits = 5),\n \")\"\n ),\n collapse = \"\\n\"\n )\n )\n , by = abs.node.position\n ]\n\n edges.dt <- data.table::rbindlist(\n l = list(\n tree.matrix[Feature != \"Leaf\", .(From = abs.node.position, To = Yes)],\n tree.matrix[Feature != \"Leaf\", .(From = abs.node.position, To = No)]\n )\n )\n edges.dt <- edges.dt[, .N, .(From, To)]\n edges.dt[, N := NULL]\n\n nodes <- DiagrammeR::create_node_df(\n n = nrow(nodes.dt),\n label = nodes.dt[, Text]\n )\n\n edges <- DiagrammeR::create_edge_df(\n from = match(edges.dt[, From], nodes.dt[, abs.node.position]),\n to = match(edges.dt[, To], nodes.dt[, abs.node.position]),\n rel = \"leading_to\")\n\n graph <- DiagrammeR::create_graph(\n nodes_df = nodes,\n edges_df = edges,\n attr_theme = NULL\n )\n graph <- DiagrammeR::add_global_graph_attrs(\n graph = graph,\n attr_type = \"graph\",\n attr = c(\"layout\", \"rankdir\"),\n value = c(\"dot\", \"LR\")\n )\n graph <- DiagrammeR::add_global_graph_attrs(\n graph = graph,\n attr_type = \"node\",\n attr = c(\"color\", \"fillcolor\", \"style\", \"shape\", \"fontname\"),\n value = c(\"DimGray\", \"beige\", \"filled\", \"rectangle\", \"Helvetica\")\n )\n graph <- DiagrammeR::add_global_graph_attrs(\n graph = graph,\n attr_type = \"edge\",\n attr = c(\"color\", \"arrowsize\", \"arrowhead\", \"fontname\"),\n value = c(\"DimGray\", \"1.5\", \"vee\", \"Helvetica\")\n )\n\n if (!render) return(invisible(graph))\n\n DiagrammeR::render_graph(graph, width = plot_width, height = plot_height)\n}\n\nglobalVariables(c(\".N\", \"N\", \"From\", \"To\", \"Text\", \"Feature\", \"no.nodes.abs.pos\",\n \"ID\", \"Yes\", \"No\", \"Tree\", \"yes.nodes.abs.pos\", \"abs.node.position\"))\n" }, { "path": "R-package/R/xgb.plot.shap.R", "content": "#' SHAP dependence plots\n#'\n#' Visualizes SHAP values against feature values to gain an impression of feature effects.\n#'\n#' @param data The data to explain as a `matrix`, `dgCMatrix`, or `data.frame`.\n#' @param shap_contrib Matrix of SHAP contributions of `data`.\n#' The default (`NULL`) computes it from `model` and `data`.\n#' @param features Vector of column indices or feature names to plot. When `NULL`\n#' (default), the `top_n` most important features are selected by [xgb.importance()].\n#' @param top_n How many of the most important features (<= 100) should be selected?\n#' By default 1 for SHAP dependence and 10 for SHAP summary.\n#' Only used when `features = NULL`.\n#' @param model An `xgb.Booster` model. Only required when `shap_contrib = NULL` or\n#' `features = NULL`.\n#' @param trees Passed to [xgb.importance()] when `features = NULL`.\n#' @param target_class Only relevant for multiclass models. The default (`NULL`)\n#' averages the SHAP values over all classes. Pass a (0-based) class index\n#' to show only SHAP values of that class.\n#' @param approxcontrib Passed to [predict.xgb.Booster()] when `shap_contrib = NULL`.\n#' @param subsample Fraction of data points randomly picked for plotting.\n#' The default (`NULL`) will use up to 100k data points.\n#' @param n_col Number of columns in a grid of plots.\n#' @param col Color of the scatterplot markers.\n#' @param pch Scatterplot marker.\n#' @param discrete_n_uniq Maximal number of unique feature values to consider the\n#' feature as discrete.\n#' @param discrete_jitter Jitter amount added to the values of discrete features.\n#' @param ylab The y-axis label in 1D plots.\n#' @param plot_NA Should contributions of cases with missing values be plotted?\n#' Default is `TRUE`.\n#' @param col_NA Color of marker for missing value contributions.\n#' @param pch_NA Marker type for `NA` values.\n#' @param pos_NA Relative position of the x-location where `NA` values are shown:\n#' `min(x) + (max(x) - min(x)) * pos_NA`.\n#' @param plot_loess Should loess-smoothed curves be plotted? (Default is `TRUE`).\n#' The smoothing is only done for features with more than 5 distinct values.\n#' @param col_loess Color of loess curves.\n#' @param span_loess The `span` parameter of [stats::loess()].\n#' @param which Whether to do univariate or bivariate plotting. Currently, only \"1d\" is implemented.\n#' @param plot Should the plot be drawn? (Default is `TRUE`).\n#' If `FALSE`, only a list of matrices is returned.\n#' @param ... Other parameters passed to [graphics::plot()].\n#'\n#' @details\n#'\n#' These scatterplots represent how SHAP feature contributions depend of feature values.\n#' The similarity to partial dependence plots is that they also give an idea for how feature values\n#' affect predictions. However, in partial dependence plots, we see marginal dependencies\n#' of model prediction on feature value, while SHAP dependence plots display the estimated\n#' contributions of a feature to the prediction for each individual case.\n#'\n#' When `plot_loess = TRUE`, feature values are rounded to three significant digits and\n#' weighted LOESS is computed and plotted, where the weights are the numbers of data points\n#' at each rounded value.\n#'\n#' Note: SHAP contributions are on the scale of the model margin.\n#' E.g., for a logistic binomial objective, the margin is on log-odds scale.\n#' Also, since SHAP stands for \"SHapley Additive exPlanation\" (model prediction = sum of SHAP\n#' contributions for all features + bias), depending on the objective used, transforming SHAP\n#' contributions for a feature from the marginal to the prediction space is not necessarily\n#' a meaningful thing to do.\n#'\n#' @return\n#' In addition to producing plots (when `plot = TRUE`), it silently returns a list of two matrices:\n#' - `data`: Feature value matrix.\n#' - `shap_contrib`: Corresponding SHAP value matrix.\n#'\n#' @references\n#' 1. Scott M. Lundberg, Su-In Lee, \"A Unified Approach to Interpreting Model Predictions\",\n#' NIPS Proceedings 2017, \n#' 2. Scott M. Lundberg, Su-In Lee, \"Consistent feature attribution for tree ensembles\",\n#' \n#'\n#' @examples\n#'\n#' data(agaricus.train, package = \"xgboost\")\n#' data(agaricus.test, package = \"xgboost\")\n#'\n#' ## Keep the number of threads to 1 for examples\n#' nthread <- 1\n#' data.table::setDTthreads(nthread)\n#' nrounds <- 20\n#'\n#' model_binary <- xgboost(\n#' agaricus.train$data, factor(agaricus.train$label),\n#' nrounds = nrounds,\n#' verbosity = 0L,\n#' learning_rate = 0.1,\n#' max_depth = 3L,\n#' subsample = 0.5,\n#' nthreads = nthread\n#' )\n#'\n#' xgb.plot.shap(agaricus.test$data, model = model_binary, features = \"odor=none\")\n#'\n#' contr <- predict(model_binary, agaricus.test$data, type = \"contrib\")\n#' xgb.plot.shap(agaricus.test$data, contr, model = model_binary, top_n = 12, n_col = 3)\n#'\n#' # Summary plot\n#' xgb.ggplot.shap.summary(agaricus.test$data, contr, model = model_binary, top_n = 12)\n#'\n#' # Multiclass example - plots for each class separately:\n#' x <- as.matrix(iris[, -5])\n#' set.seed(123)\n#' is.na(x[sample(nrow(x) * 4, 30)]) <- TRUE # introduce some missing values\n#'\n#' model_multiclass <- xgboost(\n#' x, iris$Species,\n#' nrounds = nrounds,\n#' verbosity = 0,\n#' max_depth = 2,\n#' subsample = 0.5,\n#' nthreads = nthread\n#' )\n#' nclass <- 3\n#' trees0 <- seq(from = 1, by = nclass, length.out = nrounds)\n#' col <- rgb(0, 0, 1, 0.5)\n#'\n#' xgb.plot.shap(\n#' x,\n#' model = model_multiclass,\n#' trees = trees0,\n#' target_class = 0,\n#' top_n = 4,\n#' n_col = 2,\n#' col = col,\n#' pch = 16,\n#' pch_NA = 17\n#' )\n#'\n#' xgb.plot.shap(\n#' x,\n#' model = model_multiclass,\n#' trees = trees0 + 1,\n#' target_class = 1,\n#' top_n = 4,\n#' n_col = 2,\n#' col = col,\n#' pch = 16,\n#' pch_NA = 17\n#' )\n#'\n#' xgb.plot.shap(\n#' x,\n#' model = model_multiclass,\n#' trees = trees0 + 2,\n#' target_class = 2,\n#' top_n = 4,\n#' n_col = 2,\n#' col = col,\n#' pch = 16,\n#' pch_NA = 17\n#' )\n#'\n#' # Summary plot\n#' xgb.ggplot.shap.summary(x, model = model_multiclass, target_class = 0, top_n = 4)\n#'\n#' @rdname xgb.plot.shap\n#' @export\nxgb.plot.shap <- function(data, shap_contrib = NULL, features = NULL, top_n = 1, model = NULL,\n trees = NULL, target_class = NULL, approxcontrib = FALSE,\n subsample = NULL, n_col = 1, col = rgb(0, 0, 1, 0.2), pch = '.',\n discrete_n_uniq = 5, discrete_jitter = 0.01, ylab = \"SHAP\",\n plot_NA = TRUE, col_NA = rgb(0.7, 0, 1, 0.6), pch_NA = '.', pos_NA = 1.07,\n plot_loess = TRUE, col_loess = 2, span_loess = 0.5,\n which = c(\"1d\", \"2d\"), plot = TRUE, ...) {\n data_list <- xgb.shap.data(\n data = data,\n shap_contrib = shap_contrib,\n features = features,\n top_n = top_n,\n model = model,\n trees = trees,\n target_class = target_class,\n approxcontrib = approxcontrib,\n subsample = subsample,\n max_observations = 100000\n )\n data <- data_list[[\"data\"]]\n shap_contrib <- data_list[[\"shap_contrib\"]]\n features <- colnames(data)\n\n which <- match.arg(which)\n if (which == \"2d\")\n stop(\"2D plots are not implemented yet\")\n\n if (n_col > length(features)) n_col <- length(features)\n if (plot && which == \"1d\") {\n op <- par(mfrow = c(ceiling(length(features) / n_col), n_col),\n oma = c(0, 0, 0, 0) + 0.2,\n mar = c(3.5, 3.5, 0, 0) + 0.1,\n mgp = c(1.7, 0.6, 0))\n for (f in features) {\n ord <- order(data[, f])\n x <- data[, f][ord]\n y <- shap_contrib[, f][ord]\n x_lim <- range(x, na.rm = TRUE)\n y_lim <- range(y, na.rm = TRUE)\n do_na <- plot_NA && anyNA(x)\n if (do_na) {\n x_range <- diff(x_lim)\n loc_na <- min(x, na.rm = TRUE) + x_range * pos_NA\n x_lim <- range(c(x_lim, loc_na))\n }\n x_uniq <- unique(x)\n x2plot <- x\n # add small jitter for discrete features with <= 5 distinct values\n if (length(x_uniq) <= discrete_n_uniq)\n x2plot <- jitter(x, amount = discrete_jitter * min(diff(x_uniq), na.rm = TRUE))\n plot(x2plot, y, pch = pch, xlab = f, col = col, xlim = x_lim, ylim = y_lim, ylab = ylab, ...)\n grid()\n if (plot_loess) {\n # compress x to 3 digits, and mean-aggregate y\n zz <- data.table(x = signif(x, 3), y)[, .(.N, y = mean(y)), x]\n if (nrow(zz) <= 5) {\n lines(zz$x, zz$y, col = col_loess)\n } else {\n lo <- stats::loess(y ~ x, data = zz, weights = zz$N, span = span_loess)\n zz$y_lo <- predict(lo, zz, type = \"link\")\n lines(zz$x, zz$y_lo, col = col_loess)\n }\n }\n if (do_na) {\n i_na <- which(is.na(x))\n x_na <- rep(loc_na, length(i_na))\n x_na <- jitter(x_na, amount = x_range * 0.01)\n points(x_na, y[i_na], pch = pch_NA, col = col_NA)\n }\n }\n par(op)\n }\n if (plot && which == \"2d\") {\n # TODO\n warning(\"Bivariate plotting is currently not available.\")\n }\n invisible(list(data = data, shap_contrib = shap_contrib))\n}\n\n#' SHAP summary plot\n#'\n#' Visualizes SHAP contributions of different features.\n#'\n#' A point plot (each point representing one observation from `data`) is\n#' produced for each feature, with the points plotted on the SHAP value axis.\n#' Each point (observation) is coloured based on its feature value.\n#'\n#' The plot allows to see which features have a negative / positive contribution\n#' on the model prediction, and whether the contribution is different for larger\n#' or smaller values of the feature. Inspired by the summary plot of\n#' .\n#'\n#' @inheritParams xgb.plot.shap\n#'\n#' @return A `ggplot2` object.\n#' @export\n#'\n#' @examples\n#' # See examples in xgb.plot.shap()\n#'\n#' @seealso [xgb.plot.shap()], [xgb.ggplot.shap.summary()],\n#' and the Python library .\nxgb.plot.shap.summary <- function(data, shap_contrib = NULL, features = NULL, top_n = 10, model = NULL,\n trees = NULL, target_class = NULL, approxcontrib = FALSE, subsample = NULL) {\n # Only ggplot implementation is available.\n xgb.ggplot.shap.summary(data, shap_contrib, features, top_n, model, trees, target_class, approxcontrib, subsample)\n}\n\n#' Prepare data for SHAP plots\n#'\n#' Internal function used in [xgb.plot.shap()], [xgb.plot.shap.summary()], etc.\n#'\n#' @inheritParams xgb.plot.shap\n#' @param max_observations Maximum number of observations to consider.\n#' @keywords internal\n#' @noRd\n#'\n#' @return\n#' A list containing:\n#' - `data`: The matrix of feature values.\n#' - `shap_contrib`: The matrix with corresponding SHAP values.\nxgb.shap.data <- function(data, shap_contrib = NULL, features = NULL, top_n = 1, model = NULL,\n trees = NULL, target_class = NULL, approxcontrib = FALSE,\n subsample = NULL, max_observations = 100000) {\n if (!inherits(data, c(\"matrix\", \"dsparseMatrix\", \"data.frame\")))\n stop(\"data: must be matrix, sparse matrix, or data.frame.\")\n if (inherits(data, \"data.frame\") && length(class(data)) > 1L) {\n data <- as.data.frame(data)\n }\n\n if (is.null(shap_contrib) && (is.null(model) || !inherits(model, \"xgb.Booster\")))\n stop(\"when shap_contrib is not provided, one must provide an xgb.Booster model\")\n\n if (is.null(features) && (is.null(model) || !inherits(model, \"xgb.Booster\")))\n stop(\"when features are not provided, one must provide an xgb.Booster model to rank the features\")\n\n last_dim <- function(v) dim(v)[length(dim(v))]\n\n if (!is.null(shap_contrib) &&\n (!is.array(shap_contrib) || nrow(shap_contrib) != nrow(data) || last_dim(shap_contrib) != ncol(data) + 1))\n stop(\"shap_contrib is not compatible with the provided data\")\n\n if (is.character(features) && is.null(colnames(data)))\n stop(\"either provide `data` with column names or provide `features` as column indices\")\n\n model_feature_names <- NULL\n if (is.null(features) && !is.null(model)) {\n model_feature_names <- xgb.feature_names(model)\n }\n if (is.null(model_feature_names) && xgb.num_feature(model) != ncol(data))\n stop(\"if model has no feature_names, columns in `data` must match features in model\")\n\n if (!is.null(subsample)) {\n if (subsample <= 0 || subsample >= 1) {\n stop(\"'subsample' must be a number between zero and one (non-inclusive).\")\n }\n sample_size <- as.integer(subsample * nrow(data))\n if (sample_size < 2) {\n stop(\"Sampling fraction involves less than 2 rows.\")\n }\n idx <- sample(x = seq_len(nrow(data)), size = sample_size, replace = FALSE)\n } else {\n idx <- seq_len(min(nrow(data), max_observations))\n }\n data <- data[idx, ]\n if (is.null(colnames(data))) {\n colnames(data) <- paste0(\"X\", seq_len(ncol(data)))\n }\n\n reshape_3d_shap_contrib <- function(shap_contrib, target_class) {\n # multiclass: either choose a class or merge\n if (is.list(shap_contrib)) {\n if (!is.null(target_class)) {\n shap_contrib <- shap_contrib[[target_class + 1]]\n } else {\n shap_contrib <- Reduce(\"+\", lapply(shap_contrib, abs))\n }\n } else if (length(dim(shap_contrib)) > 2) {\n if (!is.null(target_class)) {\n orig_shape <- dim(shap_contrib)\n shap_contrib <- shap_contrib[, target_class + 1, , drop = TRUE]\n if (!is.matrix(shap_contrib)) {\n shap_contrib <- matrix(shap_contrib, orig_shape[c(1L, 3L)])\n }\n } else {\n shap_contrib <- apply(abs(shap_contrib), c(1L, 3L), sum)\n }\n }\n return(shap_contrib)\n }\n\n if (is.null(shap_contrib)) {\n shap_contrib <- predict.xgb.Booster(\n model,\n newdata = data,\n predcontrib = TRUE,\n approxcontrib = approxcontrib\n )\n }\n shap_contrib <- reshape_3d_shap_contrib(shap_contrib, target_class)\n if (is.null(colnames(shap_contrib))) {\n colnames(shap_contrib) <- paste0(\"X\", seq_len(ncol(data)))\n }\n\n if (is.null(features)) {\n if (!is.null(model_feature_names)) {\n imp <- xgb.importance(model = model, trees = trees)\n } else {\n imp <- xgb.importance(model = model, trees = trees, feature_names = colnames(data))\n }\n top_n <- top_n[1]\n if (top_n < 1 || top_n > 100) stop(\"top_n: must be an integer within [1, 100]\")\n features <- imp$Feature[seq_len(min(top_n, NROW(imp)))]\n }\n if (is.character(features)) {\n features <- match(features, colnames(data))\n }\n\n shap_contrib <- shap_contrib[, features, drop = FALSE]\n data <- data[, features, drop = FALSE]\n\n list(\n data = data,\n shap_contrib = shap_contrib\n )\n}\n" }, { "path": "R-package/R/xgb.plot.tree.R", "content": "#' Plot boosted trees\n#'\n#' Read a tree model text dump and plot the model.\n#'\n#' @details\n#' The content of each node is visualized as follows:\n#' - For non-terminal nodes, it will display the split condition (number or name\n#' if available, and the condition that would decide to which node to go\n#' next).\n#' - Those nodes will be connected to their children by arrows that indicate\n#' whether the branch corresponds to the condition being met or not being met.\n#' - Terminal (leaf) nodes contain the margin to add when ending there.\n#'\n#' The \"Yes\" branches are marked by the \"< split_value\" label.\n#' The branches also used for missing values are marked as bold\n#' (as in \"carrying extra capacity\").\n#'\n#' This function uses [GraphViz](https://www.graphviz.org/) as DiagrammeR\n#' backend.\n#'\n#' @param model Object of class `xgb.Booster`. If it contains feature names\n#' (they can be set through [setinfo()], they will be used in the\n#' output from this function.\n#' @param tree_idx An integer of the tree index that should be used. This\n#' is an 1-based index.\n#' @param plot_width,plot_height Width and height of the graph in pixels.\n#' The values are passed to `DiagrammeR::render_graph()`.\n#' @param with_stats Whether to dump some additional statistics about the\n#' splits. When this option is on, the model dump contains two additional\n#' values: gain is the approximate loss function gain we get in each split;\n#' cover is the sum of second order gradient in each node.\n#' @inheritParams xgb.train\n#' @return\n#'\n#' Rendered graph object which is an htmlwidget of ' class `grViz`. Similar to\n#' \"ggplot\" objects, it needs to be printed when not running from the command\n#' line.\n#'\n#' @examples\n#' data(\"ToothGrowth\")\n#' x <- ToothGrowth[, c(\"len\", \"dose\")]\n#' y <- ToothGrowth$supp\n#' model <- xgboost(\n#' x, y,\n#' nthreads = 1L,\n#' nrounds = 3L,\n#' max_depth = 3L\n#' )\n#'\n#' # plot the first tree\n#' xgb.plot.tree(model, tree_idx = 1)\n#'\n#' # Below is an example of how to save this plot to a file.\n#' if (require(\"DiagrammeR\") && require(\"htmlwidgets\")) {\n#' fname <- file.path(tempdir(), \"plot.html'\")\n#' gr <- xgb.plot.tree(model, tree_idx = 1)\n#' htmlwidgets::saveWidget(gr, fname)\n#' }\n#' @export\nxgb.plot.tree <- function(model,\n tree_idx = 1,\n plot_width = NULL,\n plot_height = NULL,\n with_stats = FALSE, ...) {\n check.deprecation(deprecated_plottree_params, match.call(), ...)\n if (!inherits(model, \"xgb.Booster\")) {\n stop(\"model has to be an object of the class xgb.Booster\")\n }\n if (!requireNamespace(\"DiagrammeR\", quietly = TRUE)) {\n stop(\"The DiagrammeR package is required for xgb.plot.tree\", call. = FALSE)\n }\n\n txt <- xgb.dump(model, dump_format = \"dot\", with_stats = with_stats)\n DiagrammeR::grViz(\n txt[[tree_idx]], width = plot_width, height = plot_height\n )\n}\n" }, { "path": "R-package/R/xgb.save.R", "content": "#' Save XGBoost model to binary file\n#'\n#' Save XGBoost model to a file in binary or JSON format.\n#'\n#' @param model Model object of `xgb.Booster` class.\n#' @param fname Name of the file to write. Its extension determines the serialization format:\n#' - \".ubj\": Use the universal binary JSON format (recommended).\n#' This format uses binary types for e.g. floating point numbers, thereby preventing any loss\n#' of precision when converting to a human-readable JSON text or similar.\n#' - \".json\": Use plain JSON, which is a human-readable format.\n#' - \".deprecated\": Use **deprecated** binary format. This format will\n#' not be able to save attributes introduced after v1 of XGBoost, such as the \"best_iteration\"\n#' attribute that boosters might keep, nor feature names or user-specifiec attributes.\n#' - If the format is not specified by passing one of the file extensions above, will\n#' default to UBJ.\n#'\n#' @details\n#'\n#' This methods allows to save a model in an XGBoost-internal binary or text format which is universal\n#' among the various xgboost interfaces. In R, the saved model file could be read later\n#' using either the [xgb.load()] function or the `xgb_model` parameter of [xgb.train()].\n#'\n#' Note: a model can also be saved as an R object (e.g., by using [readRDS()]\n#' or [save()]). However, it would then only be compatible with R, and\n#' corresponding R methods would need to be used to load it. Moreover, persisting the model with\n#' [readRDS()] or [save()] might cause compatibility problems in\n#' future versions of XGBoost. Consult [a-compatibility-note-for-saveRDS-save] to learn\n#' how to persist models in a future-proof way, i.e., to make the model accessible in future\n#' releases of XGBoost.\n#'\n#' @seealso [xgb.load()]\n#'\n#' @examples\n#' \\dontshow{RhpcBLASctl::omp_set_num_threads(1)}\n#' data(agaricus.train, package = \"xgboost\")\n#' data(agaricus.test, package = \"xgboost\")\n#'\n#' ## Keep the number of threads to 1 for examples\n#' nthread <- 1\n#' data.table::setDTthreads(nthread)\n#'\n#' train <- agaricus.train\n#' test <- agaricus.test\n#'\n#' bst <- xgb.train(\n#' data = xgb.DMatrix(train$data, label = train$label, nthread = 1),\n#' nrounds = 2,\n#' params = xgb.params(\n#' max_depth = 2,\n#' nthread = nthread,\n#' objective = \"binary:logistic\"\n#' )\n#' )\n#'\n#' fname <- file.path(tempdir(), \"xgb.ubj\")\n#' xgb.save(bst, fname)\n#' bst <- xgb.load(fname)\n#' @export\nxgb.save <- function(model, fname) {\n if (typeof(fname) != \"character\")\n stop(\"fname must be character\")\n if (!inherits(model, \"xgb.Booster\")) {\n stop(\"model must be xgb.Booster.\",\n if (inherits(model, \"xgb.DMatrix\")) \" Use xgb.DMatrix.save to save an xgb.DMatrix object.\" else \"\")\n }\n fname <- path.expand(fname)\n .Call(XGBoosterSaveModel_R, xgb.get.handle(model), enc2utf8(fname[1]))\n return(TRUE)\n}\n" }, { "path": "R-package/R/xgb.save.raw.R", "content": "#' Save XGBoost model to R's raw vector\n#'\n#' Save XGBoost model from [xgboost()] or [xgb.train()].\n#' Call [xgb.load.raw()] to load the model back from raw vector.\n#'\n#' @param model The model object.\n#' @param raw_format The format for encoding the booster:\n#' - \"json\": Encode the booster into JSON text document.\n#' - \"ubj\": Encode the booster into Universal Binary JSON.\n#' - \"deprecated\": Encode the booster into old customized binary format.\n#'\n#' @examples\n#' \\dontshow{RhpcBLASctl::omp_set_num_threads(1)}\n#' data(agaricus.train, package = \"xgboost\")\n#' data(agaricus.test, package = \"xgboost\")\n#'\n#' ## Keep the number of threads to 1 for examples\n#' nthread <- 1\n#' data.table::setDTthreads(nthread)\n#'\n#' train <- agaricus.train\n#' test <- agaricus.test\n#'\n#' bst <- xgb.train(\n#' data = xgb.DMatrix(train$data, label = train$label, nthread = 1),\n#' nrounds = 2,\n#' params = xgb.params(\n#' max_depth = 2,\n#' nthread = nthread,\n#' objective = \"binary:logistic\"\n#' )\n#' )\n#'\n#' raw <- xgb.save.raw(bst)\n#' bst <- xgb.load.raw(raw)\n#'\n#' @export\nxgb.save.raw <- function(model, raw_format = \"ubj\") {\n handle <- xgb.get.handle(model)\n args <- list(format = raw_format)\n .Call(XGBoosterSaveModelToRaw_R, handle, jsonlite::toJSON(args, auto_unbox = TRUE))\n}\n" }, { "path": "R-package/R/xgb.train.R", "content": "#' @title Fit XGBoost Model\n#' @description Fits an XGBoost model to given data in DMatrix format (e.g. as produced by [xgb.DMatrix()]).\n#' See the tutorial [Introduction to Boosted Trees](https://xgboost.readthedocs.io/en/stable/tutorials/model.html)\n#' for a longer explanation of what XGBoost does, and the rest of the\n#' [XGBoost Tutorials](https://xgboost.readthedocs.io/en/latest/tutorials/index.html) for further\n#' explanations XGBoost's features and usage.\n#'\n#' Compared to function [xgboost()] which is a user-friendly function targeted towards interactive\n#' usage, ``xgb.train`` is a lower-level interface which allows finer-grained control and exposes\n#' further functionalities offered by the core library (such as learning-to-rank objectives), but\n#' which works exclusively with XGBoost's own data format (\"DMatrices\") instead of with regular R\n#' objects.\n#'\n#' The syntax of this function closely mimics the same function from the Python package for XGBoost,\n#' and is recommended to use for package developers over `xgboost()` as it will provide a more\n#' stable interface (with fewer breaking changes) and lower overhead from data validations.\n#'\n#' See also the [migration guide](https://xgboost.readthedocs.io/en/latest/R-package/migration_guide.html)\n#' if coming from a previous version of XGBoost in the 1.x series.\n#' @param params List of XGBoost parameters which control the model building process.\n#' See the [online documentation](https://xgboost.readthedocs.io/en/latest/parameter.html)\n#' and the documentation for [xgb.params()] for details.\n#'\n#' Should be passed as list with named entries. Parameters that are not specified in this\n#' list will use their default values.\n#'\n#' A list of named parameters can be created through the function [xgb.params()], which\n#' accepts all valid parameters as function arguments.\n#' @param data Training dataset. `xgb.train()` accepts only an `xgb.DMatrix` as the input.\n#'\n#' Note that there is a function [xgboost()] which is meant to accept R data objects\n#' as inputs, such as data frames and matrices.\n#' @param nrounds Max number of boosting iterations.\n#' @param evals Named list of `xgb.DMatrix` datasets to use for evaluating model performance.\n#' Metrics specified in either `eval_metric` (under params) or `custom_metric` (function\n#' argument here) will be computed for each of these datasets during each boosting iteration,\n#' and stored in the end as a field named `evaluation_log` in the resulting object.\n#'\n#' When either `verbose>=1` or [xgb.cb.print.evaluation()] callback is engaged, the performance\n#' results are continuously printed out during the training.\n#'\n#' E.g., specifying `evals=list(validation1=mat1, validation2=mat2)` allows to track\n#' the performance of each round's model on `mat1` and `mat2`.\n#' @param objective Customized objective function. Should take two arguments: the first one will be the\n#' current predictions (either a numeric vector or matrix depending on the number of targets / classes),\n#' and the second one will be the `data` DMatrix object that is used for training.\n#'\n#' It should return a list with two elements `grad` and `hess` (in that order), as either\n#' numeric vectors or numeric matrices depending on the number of targets / classes (same\n#' dimension as the predictions that are passed as first argument).\n#' @param custom_metric Customized evaluation function. Just like `objective`, should take two arguments,\n#' with the first one being the predictions and the second one the `data` DMatrix.\n#'\n#' Should return a list with two elements `metric` (name that will be displayed for this metric,\n#' should be a string / character), and `value` (the number that the function calculates, should\n#' be a numeric scalar).\n#'\n#' Note that even if passing `custom_metric`, objectives also have an associated default metric that\n#' will be evaluated in addition to it. In order to disable the built-in metric, one can pass\n#' parameter `disable_default_eval_metric = TRUE`.\n#' @param verbose If 0, xgboost will stay silent. If 1, it will print information about performance.\n#' If 2, some additional information will be printed out.\n#' Note that setting `verbose > 0` automatically engages the\n#' `xgb.cb.print.evaluation(period=1)` callback function.\n#' @param print_every_n When passing `verbose>0`, evaluation logs (metrics calculated on the\n#' data passed under `evals`) will be printed every nth iteration according to the value passed\n#' here. The first and last iteration are always included regardless of this 'n'.\n#'\n#' Only has an effect when passing data under `evals` and when passing `verbose>0`. The parameter\n#' is passed to the [xgb.cb.print.evaluation()] callback.\n#' @param early_stopping_rounds Number of boosting rounds after which training will be stopped\n#' if there is no improvement in performance (as measured by the evaluatiation metric that is\n#' supplied or selected by default for the objective) on the evaluation data passed under\n#' `evals`.\n#'\n#' Must pass `evals` in order to use this functionality. Setting this parameter adds the\n#' [xgb.cb.early.stop()] callback.\n#'\n#' If `NULL`, early stopping will not be used.\n#' @param maximize If `feval` and `early_stopping_rounds` are set, then this parameter must be set as well.\n#' When it is `TRUE`, it means the larger the evaluation score the better.\n#' This parameter is passed to the [xgb.cb.early.stop()] callback.\n#' @param save_period When not `NULL`, model is saved to disk after every `save_period` rounds.\n#' 0 means save at the end. The saving is handled by the [xgb.cb.save.model()] callback.\n#' @param save_name the name or path for periodically saved model file.\n#' @param xgb_model A previously built model to continue the training from.\n#' Could be either an object of class `xgb.Booster`, or its raw data, or the name of a\n#' file with a previously saved model.\n#' @param callbacks A list of callback functions to perform various task during boosting.\n#' See [xgb.Callback()]. Some of the callbacks are automatically created depending on the\n#' parameters' values. User can provide either existing or their own callback methods in order\n#' to customize the training process.\n#'\n#' Note that some callbacks might try to leave attributes in the resulting model object,\n#' such as an evaluation log (a `data.table` object) - be aware that these objects are kept\n#' as R attributes, and thus do not get saved when using XGBoost's own serializaters like\n#' [xgb.save()] (but are kept when using R serializers like [saveRDS()]).\n#' @param ... Not used.\n#'\n#' Some arguments that were part of this function in previous XGBoost versions are currently\n#' deprecated or have been renamed. If a deprecated or renamed argument is passed, will throw\n#' a warning (by default) and use its current equivalent instead. This warning will become an\n#' error if using the \\link[=xgboost-options]{'strict mode' option}.\n#'\n#' If some additional argument is passed that is neither a current function argument nor\n#' a deprecated or renamed argument, a warning or error will be thrown depending on the\n#' 'strict mode' option.\n#'\n#' \\bold{Important:} `...` will be removed in a future version, and all the current\n#' deprecation warnings will become errors. Please use only arguments that form part of\n#' the function signature.\n#' @return An object of class `xgb.Booster`.\n#' @details\n#' Compared to [xgboost()], the `xgb.train()` interface supports advanced features such as\n#' `evals`, customized objective and evaluation metric functions, among others, with the\n#' difference these work `xgb.DMatrix` objects and do not follow typical R idioms.\n#'\n#' Parallelization is automatically enabled if OpenMP is present.\n#' Number of threads can also be manually specified via the `nthread` parameter.\n#'\n#' While in XGBoost language bindings, the default random seed defaults to zero, in R, if a parameter `seed`\n#' is not manually supplied, it will generate a random seed through R's own random number generator,\n#' whose seed in turn is controllable through `set.seed`. If `seed` is passed, it will override the\n#' RNG from R.\n#'\n#' The following callbacks are automatically created when certain parameters are set:\n#' - [xgb.cb.print.evaluation()] is turned on when `verbose > 0` and the `print_every_n`\n#' parameter is passed to it.\n#' - [xgb.cb.evaluation.log()] is on when `evals` is present.\n#' - [xgb.cb.early.stop()]: When `early_stopping_rounds` is set.\n#' - [xgb.cb.save.model()]: When `save_period > 0` is set.\n#'\n#' Note that objects of type `xgb.Booster` as returned by this function behave a bit differently\n#' from typical R objects (it's an 'altrep' list class), and it makes a separation between\n#' internal booster attributes (restricted to jsonifyable data), accessed through [xgb.attr()]\n#' and shared between interfaces through serialization functions like [xgb.save()]; and\n#' R-specific attributes (typically the result from a callback), accessed through [attributes()]\n#' and [attr()], which are otherwise\n#' only used in the R interface, only kept when using R's serializers like [saveRDS()], and\n#' not anyhow used by functions like `predict.xgb.Booster()`.\n#'\n#' Be aware that one such R attribute that is automatically added is `params` - this attribute\n#' is assigned from the `params` argument to this function, and is only meant to serve as a\n#' reference for what went into the booster, but is not used in other methods that take a booster\n#' object - so for example, changing the booster's configuration requires calling `xgb.config<-`\n#' or `xgb.model.parameters<-`, while simply modifying `attributes(model)$params$<...>` will have no\n#' effect elsewhere.\n#'\n#' @seealso [xgb.Callback()], [predict.xgb.Booster()], [xgb.cv()]\n#'\n#' @references\n#' Tianqi Chen and Carlos Guestrin, \"XGBoost: A Scalable Tree Boosting System\",\n#' 22nd SIGKDD Conference on Knowledge Discovery and Data Mining, 2016, \\url{https://arxiv.org/abs/1603.02754}\n#'\n#' @examples\n#' data(agaricus.train, package = \"xgboost\")\n#' data(agaricus.test, package = \"xgboost\")\n#'\n#' ## Keep the number of threads to 1 for examples\n#' nthread <- 1\n#' data.table::setDTthreads(nthread)\n#'\n#' dtrain <- with(\n#' agaricus.train, xgb.DMatrix(data, label = label, nthread = nthread)\n#' )\n#' dtest <- with(\n#' agaricus.test, xgb.DMatrix(data, label = label, nthread = nthread)\n#' )\n#' evals <- list(train = dtrain, eval = dtest)\n#'\n#' ## A simple xgb.train example:\n#' param <- xgb.params(\n#' max_depth = 2,\n#' nthread = nthread,\n#' objective = \"binary:logistic\",\n#' eval_metric = \"auc\"\n#' )\n#' bst <- xgb.train(param, dtrain, nrounds = 2, evals = evals, verbose = 0)\n#'\n#' ## An xgb.train example where custom objective and evaluation metric are\n#' ## used:\n#' logregobj <- function(preds, dtrain) {\n#' labels <- getinfo(dtrain, \"label\")\n#' preds <- 1/(1 + exp(-preds))\n#' grad <- preds - labels\n#' hess <- preds * (1 - preds)\n#' return(list(grad = grad, hess = hess))\n#' }\n#' evalerror <- function(preds, dtrain) {\n#' labels <- getinfo(dtrain, \"label\")\n#' err <- as.numeric(sum(labels != (preds > 0)))/length(labels)\n#' return(list(metric = \"error\", value = err))\n#' }\n#'\n#' # These functions could be used by passing them as 'objective' and\n#' # 'eval_metric' parameters in the params list:\n#' param <- xgb.params(\n#' max_depth = 2,\n#' nthread = nthread,\n#' objective = logregobj,\n#' eval_metric = evalerror\n#' )\n#' bst <- xgb.train(param, dtrain, nrounds = 2, evals = evals, verbose = 0)\n#'\n#' # ... or as dedicated 'objective' and 'custom_metric' parameters of xgb.train:\n#' bst <- xgb.train(\n#' within(param, rm(\"objective\", \"eval_metric\")),\n#' dtrain, nrounds = 2, evals = evals,\n#' objective = logregobj, custom_metric = evalerror\n#' )\n#'\n#'\n#' ## An xgb.train example of using variable learning rates at each iteration:\n#' param <- xgb.params(\n#' max_depth = 2,\n#' learning_rate = 1,\n#' nthread = nthread,\n#' objective = \"binary:logistic\",\n#' eval_metric = \"auc\"\n#' )\n#' my_learning_rates <- list(learning_rate = c(0.5, 0.1))\n#'\n#' bst <- xgb.train(\n#' param,\n#' dtrain,\n#' nrounds = 2,\n#' evals = evals,\n#' verbose = 0,\n#' callbacks = list(xgb.cb.reset.parameters(my_learning_rates))\n#' )\n#'\n#' ## Early stopping:\n#' bst <- xgb.train(\n#' param, dtrain, nrounds = 25, evals = evals, early_stopping_rounds = 3\n#' )\n#' @export\nxgb.train <- function(params = xgb.params(), data, nrounds, evals = list(),\n objective = NULL, custom_metric = NULL, verbose = 1, print_every_n = 1L,\n early_stopping_rounds = NULL, maximize = NULL,\n save_period = NULL, save_name = \"xgboost.model\",\n xgb_model = NULL, callbacks = list(), ...) {\n check.deprecation(deprecated_train_params, match.call(), ...)\n\n params <- check.booster.params(params)\n tmp <- check.custom.obj(params, objective)\n params <- tmp$params\n objective <- tmp$objective\n tmp <- check.custom.eval(params, custom_metric, maximize, early_stopping_rounds, callbacks)\n params <- tmp$params\n custom_metric <- tmp$custom_metric\n\n # data & evals checks\n dtrain <- data\n if (!inherits(dtrain, \"xgb.DMatrix\"))\n stop(\"second argument dtrain must be xgb.DMatrix\")\n if (length(evals) > 0) {\n if (typeof(evals) != \"list\" ||\n !all(vapply(evals, inherits, logical(1), what = 'xgb.DMatrix')))\n stop(\"'evals' must be a list of xgb.DMatrix elements\")\n evnames <- names(evals)\n if (is.null(evnames) || any(evnames == \"\"))\n stop(\"each element of 'evals' must have a name tag\")\n }\n # Handle multiple evaluation metrics given as a list\n for (m in params$eval_metric) {\n params <- c(params, list(eval_metric = m))\n }\n\n params <- c(params)\n params['validate_parameters'] <- TRUE\n if (!(\"seed\" %in% names(params))) {\n params[[\"seed\"]] <- sample(.Machine$integer.max, size = 1)\n }\n\n # callbacks\n tmp <- .process.callbacks(callbacks, is_cv = FALSE)\n callbacks <- tmp$callbacks\n cb_names <- tmp$cb_names\n rm(tmp)\n\n # Early stopping callback (should always come first)\n if (!is.null(early_stopping_rounds) && !(\"early_stop\" %in% cb_names)) {\n callbacks <- add.callback(\n callbacks,\n xgb.cb.early.stop(\n early_stopping_rounds,\n maximize = maximize,\n verbose = verbose\n ),\n as_first_elt = TRUE\n )\n }\n # evaluation printing callback\n print_every_n <- max(as.integer(print_every_n), 1L)\n if (verbose && !(\"print_evaluation\" %in% cb_names)) {\n callbacks <- add.callback(callbacks, xgb.cb.print.evaluation(print_every_n))\n }\n # evaluation log callback: it is automatically enabled when 'evals' is provided\n if (length(evals) && !(\"evaluation_log\" %in% cb_names)) {\n callbacks <- add.callback(callbacks, xgb.cb.evaluation.log())\n }\n # Model saving callback\n if (!is.null(save_period) && !(\"save_model\" %in% cb_names)) {\n callbacks <- add.callback(callbacks, xgb.cb.save.model(save_period, save_name))\n }\n\n # The tree updating process would need slightly different handling\n is_update <- NVL(params[['process_type']], '.') == 'update'\n\n # Construct a booster (either a new one or load from xgb_model)\n bst <- xgb.Booster(\n params = params,\n cachelist = append(evals, dtrain),\n modelfile = xgb_model\n )\n niter_init <- bst$niter\n bst <- bst$bst\n .Call(\n XGBoosterCopyInfoFromDMatrix_R,\n xgb.get.handle(bst),\n dtrain\n )\n\n if (is_update && nrounds > niter_init)\n stop(\"nrounds cannot be larger than \", niter_init, \" (nrounds of xgb_model)\")\n\n niter_skip <- ifelse(is_update, 0, niter_init)\n begin_iteration <- niter_skip + 1\n end_iteration <- niter_skip + nrounds\n\n .execute.cb.before.training(\n callbacks,\n bst,\n dtrain,\n evals,\n begin_iteration,\n end_iteration\n )\n\n # the main loop for boosting iterations\n # FIX: Handle nrounds=0 to prevent 1:0 sequence and ensure 'iteration' is defined\n if (nrounds == 0) iteration <- end_iteration\n for (iteration in seq(from = begin_iteration, length.out = nrounds)) {\n\n .execute.cb.before.iter(\n callbacks,\n bst,\n dtrain,\n evals,\n iteration\n )\n\n xgb.iter.update(\n bst = bst,\n dtrain = dtrain,\n iter = iteration - 1,\n objective = objective\n )\n\n bst_evaluation <- NULL\n if (length(evals) > 0) {\n bst_evaluation <- xgb.iter.eval(\n bst = bst,\n evals = evals,\n iter = iteration - 1,\n custom_metric = custom_metric\n )\n }\n\n should_stop <- .execute.cb.after.iter(\n callbacks,\n bst,\n dtrain,\n evals,\n iteration,\n bst_evaluation\n )\n\n if (should_stop) break\n }\n\n cb_outputs <- .execute.cb.after.training(\n callbacks,\n bst,\n dtrain,\n evals,\n iteration,\n bst_evaluation\n )\n\n extra_attrs <- list(\n call = match.call(),\n params = params\n )\n bst <- xgb.reset.Booster(bst)\n curr_attrs <- attributes(bst)\n if (NROW(curr_attrs)) {\n curr_attrs <- curr_attrs[\n setdiff(\n names(curr_attrs),\n c(names(extra_attrs), names(cb_outputs))\n )\n ]\n }\n curr_attrs <- c(extra_attrs, curr_attrs)\n if (NROW(cb_outputs)) {\n curr_attrs <- c(curr_attrs, cb_outputs)\n }\n attributes(bst) <- curr_attrs\n\n return(bst)\n}\n\n# nolint start: line_length_linter.\n#' @title XGBoost Parameters\n#' @description Convenience function to generate a list of named XGBoost parameters, which\n#' can be passed as argument `params` to [xgb.train()]. See the [online documentation](\n#' https://xgboost.readthedocs.io/en/stable/parameter.html) for more details.\n#'\n#' The purpose of this function is to enable IDE autocompletions and to provide in-package\n#' documentation for all the possible parameters that XGBoost accepts. The output from this\n#' function is just a regular R list containing the parameters that were set to non-default\n#' values. Note that this function will not perform any validation on the supplied arguments.\n#'\n#' If passing `NULL` for a given parameter (the default for all of them), then the default\n#' value for that parameter will be used. Default values are automatically determined by the\n#' XGBoost core library upon calls to [xgb.train()] or [xgb.cv()], and are subject to change\n#' over XGBoost library versions. Some of them might differ according to the\n#' booster type (e.g. defaults for regularization are different for linear and tree-based boosters).\n#' @return A list with the entries that were passed non-NULL values. It is intended to\n#' be passed as argument `params` to [xgb.train()] or [xgb.cv()].\n#' @export\n#' @param objective (default=`\"reg:squarederror\"`)\n#' Specify the learning task and the corresponding learning objective or a custom objective function to be used.\n#'\n#' For custom objective, see [Custom Objective and Evaluation Metric](https://xgboost.readthedocs.io/en/latest/tutorials/custom_metric_obj.html)\n#' and [Custom objective and metric](https://xgboost.readthedocs.io/en/latest/tutorials/saving_model.html#custom-obj-metric) for more information,\n#' along with the end note for function signatures.\n#'\n#' Supported values are:\n#' - `\"reg:squarederror\"`: regression with squared loss.\n#' - `\"reg:squaredlogerror\"`: regression with squared log loss \\eqn{\\frac{1}{2}[log(pred + 1) - log(label + 1)]^2}. All input labels are required to be greater than -1. Also, see metric `rmsle` for possible issue with this objective.\n#' - `\"reg:logistic\"`: logistic regression, output probability\n#' - `\"reg:pseudohubererror\"`: regression with Pseudo Huber loss, a twice differentiable alternative to absolute loss.\n#' - `\"reg:absoluteerror\"`: Regression with L1 error. When tree model is used, leaf value is refreshed after tree construction. If used in distributed training, the leaf value is calculated as the mean value from all workers, which is not guaranteed to be optimal.\n#'\n#' Version added: 1.7.0\n#' - `\"reg:quantileerror\"`: Quantile loss, also known as \"pinball loss\". See later sections for its parameter and [Quantile Regression](https://xgboost.readthedocs.io/en/latest/python/examples/quantile_regression.html#sphx-glr-python-examples-quantile-regression-py) for a worked example.\n#'\n#' Version added: 2.0.0\n#' - `\"reg:expectileerror\"`: Expectile loss. See later sections for its parameter.\n#' - `\"binary:logistic\"`: logistic regression for binary classification, output probability\n#' - `\"binary:logitraw\"`: logistic regression for binary classification, output score before logistic transformation\n#' - `\"binary:hinge\"`: hinge loss for binary classification. This makes predictions of 0 or 1, rather than producing probabilities.\n#' - `\"count:poisson\"`: Poisson regression for count data, output mean of Poisson distribution.\n#' `\"max_delta_step\"` is set to 0.7 by default in Poisson regression (used to safeguard optimization)\n#' - `\"survival:cox\"`: Cox regression for right censored survival time data (negative values are considered right censored).\n#'\n#' Note that predictions are returned on the hazard ratio scale (i.e., as HR = exp(marginal_prediction) in the proportional hazard function `h(t) = h0(t) * HR`).\n#' - `\"survival:aft\"`: Accelerated failure time model for censored survival time data.\n#' See [Survival Analysis with Accelerated Failure Time](https://xgboost.readthedocs.io/en/latest/tutorials/aft_survival_analysis.html) for details.\n#' - `\"multi:softmax\"`: set XGBoost to do multiclass classification using the softmax objective, you also need to set num_class(number of classes)\n#' - `\"multi:softprob\"`: same as softmax, but output a vector of `ndata * nclass`, which can be further reshaped to `ndata * nclass` matrix. The result contains predicted probability of each data point belonging to each class.\n#' - `\"rank:ndcg\"`: Use LambdaMART to perform pair-wise ranking where the normalized discounted cumulative gain (NDCG) is maximized. This objective supports position debiasing for click data.\n#' - `\"rank:map\"`: Use LambdaMART to perform pair-wise ranking where the mean average precision (MAP) is maximized\n#' - `\"rank:pairwise\"`: Use LambdaRank to perform pair-wise ranking using the `ranknet` objective.\n#' - `\"reg:gamma\"`: gamma regression with log-link. Output is a mean of gamma distribution. It might be useful, e.g., for modeling insurance claims severity, or for any outcome that might be gamma-distributed.\n#' - `\"reg:tweedie\"`: Tweedie regression with log-link. It might be useful, e.g., for modeling total loss in insurance, or for any outcome that might be tweedie-distributed.\n#' @param verbosity (default=1)\n#' Verbosity of printing messages. Valid values are 0 (silent), 1 (warning), 2 (info), 3\n#' (debug). Sometimes XGBoost tries to change configurations based on heuristics, which\n#' is displayed as warning message. If there's unexpected behaviour, please try to\n#' increase value of verbosity.\n#' @param nthread (default to maximum number of threads available if not set)\n#' Number of parallel threads used to run XGBoost. When choosing it, please keep thread\n#' contention and hyperthreading in mind.\n#' @param seed Random number seed. If not specified, will take a random seed through R's own RNG engine.\n#' @param booster (default= `\"gbtree\"`)\n#' Which booster to use. Can be `\"gbtree\"`, `\"gblinear\"` or `\"dart\"`; `\"gbtree\"` and `\"dart\"` use tree based models while `\"gblinear\"` uses linear functions.\n#' @param eta,learning_rate (two aliases for the same parameter)\n#' Step size shrinkage used in update to prevent overfitting. After each boosting step, we can directly get the weights of new features, and `eta` shrinks the feature weights to make the boosting process more conservative.\n#' - range: \\eqn{[0,1]}\n#' - default value: 0.3 for tree-based boosters, 0.5 for linear booster.\n#'\n#' Note: should only pass one of `eta` or `learning_rate`. Both refer to the same parameter and there's thus no difference between one or the other.\n#' @param gamma,min_split_loss (two aliases for the same parameter) (for Tree Booster) (default=0, alias: `gamma`)\n#' Minimum loss reduction required to make a further partition on a leaf node of the tree. The larger `min_split_loss` is, the more conservative the algorithm will be. Note that a tree where no splits were made might still contain a single terminal node with a non-zero score.\n#'\n#' range: \\eqn{[0, \\infty)}\n#'\n#' Note: should only pass one of `gamma` or `min_split_loss`. Both refer to the same parameter and there's thus no difference between one or the other.\n#' @param max_depth (for Tree Booster) (default=6, type=int32)\n#' Maximum depth of a tree. Increasing this value will make the model more complex and more likely to overfit. 0 indicates no limit on depth. Beware that XGBoost aggressively consumes memory when training a deep tree. `\"exact\"` tree method requires non-zero value.\n#'\n#' range: \\eqn{[0, \\infty)}\n#' @param min_child_weight (for Tree Booster) (default=1)\n#' Minimum sum of instance weight (hessian) needed in a child. If the tree partition step results in a leaf node with the sum of instance weight less than `min_child_weight`, then the building process will give up further partitioning. In linear regression task, this simply corresponds to minimum number of instances needed to be in each node. The larger `min_child_weight` is, the more conservative the algorithm will be.\n#'\n#' range: \\eqn{[0, \\infty)}\n#' @param max_delta_step (for Tree Booster) (default=0)\n#' Maximum delta step we allow each leaf output to be. If the value is set to 0, it means there is no constraint. If it is set to a positive value, it can help making the update step more conservative. Usually this parameter is not needed, but it might help in logistic regression when class is extremely imbalanced. Set it to value of 1-10 might help control the update.\n#'\n#' range: \\eqn{[0, \\infty)}\n#' @param subsample (for Tree Booster) (default=1)\n#' Subsample ratio of the training instances. Setting it to 0.5 means that XGBoost would randomly sample half of the training data prior to growing trees. and this will prevent overfitting. Subsampling will occur once in every boosting iteration.\n#'\n#' range: \\eqn{(0,1]}\n#' @param sampling_method (for Tree Booster) (default= `\"uniform\"`)\n#' The method to use to sample the training instances.\n#' - `\"uniform\"`: each training instance has an equal probability of being selected. Typically set\n#' `\"subsample\"` >= 0.5 for good results.\n#' - `\"gradient_based\"`: the selection probability for each training instance is proportional to the\n#' \\bold{regularized absolute value} of gradients (more specifically, \\eqn{\\sqrt{g^2+\\lambda h^2}}).\n#' `\"subsample\"` may be set to as low as 0.1 without loss of model accuracy. Note that this\n#' sampling method is only supported when `\"tree_method\"` is set to `\"hist\"`; other tree\n#' methods only support `\"uniform\"` sampling.\n#' @param colsample_bytree,colsample_bylevel,colsample_bynode (for Tree Booster) (default=1)\n#' This is a family of parameters for subsampling of columns.\n#' - All `\"colsample_by*\"` parameters have a range of \\eqn{(0, 1]}, the default value of 1, and specify the fraction of columns to be subsampled.\n#' - `\"colsample_bytree\"` is the subsample ratio of columns when constructing each tree. Subsampling occurs once for every tree constructed.\n#' - `\"colsample_bylevel\"` is the subsample ratio of columns for each level. Subsampling occurs once for every new depth level reached in a tree. Columns are subsampled from the set of columns chosen for the current tree.\n#' - `\"colsample_bynode\"` is the subsample ratio of columns for each node (split). Subsampling occurs once every time a new split is evaluated. Columns are subsampled from the set of columns chosen for the current level. This is not supported by the exact tree method.\n#' - `\"colsample_by*\"` parameters work cumulatively. For instance,\n#' the combination `{'colsample_bytree'=0.5, 'colsample_bylevel'=0.5, 'colsample_bynode'=0.5}` with 64 features will leave 8 features to choose from at\n#' each split.\n#'\n#' One can set the `\"feature_weights\"` for DMatrix to\n#' define the probability of each feature being selected when using column sampling.\n#' @param lambda,reg_lambda (two aliases for the same parameter)\n#'\n#' - For tree-based boosters:\n#' - L2 regularization term on weights. Increasing this value will make model more conservative.\n#' - default: 1\n#' - range: \\eqn{[0, \\infty]}\n#' - For linear booster:\n#' - L2 regularization term on weights. Increasing this value will make model more conservative. Normalised to number of training examples.\n#' - default: 0\n#' - range: \\eqn{[0, \\infty)}\n#'\n#' Note: should only pass one of `lambda` or `reg_lambda`. Both refer to the same parameter and there's thus no difference between one or the other.\n#' @param alpha,reg_alpha (two aliases for the same parameter)\n#' - L1 regularization term on weights. Increasing this value will make model more conservative.\n#' - For the linear booster, it's normalised to number of training examples.\n#' - default: 0\n#' - range: \\eqn{[0, \\infty)}\n#'\n#' Note: should only pass one of `alpha` or `reg_alpha`. Both refer to the same parameter and there's thus no difference between one or the other.\n#' @param tree_method (for Tree Booster) (default= `\"auto\"`)\n#' The tree construction algorithm used in XGBoost. See description in the [reference paper](https://arxiv.org/abs/1603.02754) and [Tree Methods](https://xgboost.readthedocs.io/en/latest/treemethod.html).\n#'\n#' Choices: `\"auto\"`, `\"exact\"`, `\"approx\"`, `\"hist\"`, this is a combination of commonly\n#' used updaters. For other updaters like `\"refresh\"`, set the parameter `updater`\n#' directly.\n#' - `\"auto\"`: Same as the `\"hist\"` tree method.\n#' - `\"exact\"`: Exact greedy algorithm. Enumerates all split candidates.\n#' - `\"approx\"`: Approximate greedy algorithm using quantile sketch and gradient histogram.\n#' - `\"hist\"`: Faster histogram optimized approximate greedy algorithm.\n#' @param scale_pos_weight (for Tree Booster) (default=1)\n#' Control the balance of positive and negative weights, useful for unbalanced classes. A typical value to consider: `sum(negative instances) / sum(positive instances)`. See [Parameters Tuning](https://xgboost.readthedocs.io/en/latest/tutorials/param_tuning.html) for more discussion. Also, see Higgs Kaggle competition demo for examples: [R](https://github.com/dmlc/xgboost/blob/master/demo/kaggle-higgs/higgs-train.R), [py1](https://github.com/dmlc/xgboost/blob/master/demo/kaggle-higgs/higgs-numpy.py), [py2](https://github.com/dmlc/xgboost/blob/master/demo/kaggle-higgs/higgs-cv.py), [py3](https://github.com/dmlc/xgboost/blob/master/demo/guide-python/cross_validation.py).\n#' @param updater Has different meanings depending on the type of booster.\n#'\n#' - For tree-based boosters:\n#' A comma separated string defining the sequence of tree updaters to run, providing a modular way to construct and to modify the trees. This is an advanced parameter that is usually set automatically, depending on some other parameters. However, it could be also set explicitly by a user. The following updaters exist:\n#' - `\"grow_colmaker\"`: non-distributed column-based construction of trees.\n#' - `\"grow_histmaker\"`: distributed tree construction with row-based data splitting based on global proposal of histogram counting.\n#' - `\"grow_quantile_histmaker\"`: Grow tree using quantized histogram.\n#' - `\"grow_gpu_hist\"`: Enabled when `tree_method` is set to `\"hist\"` along with `device=\"cuda\"`.\n#' - `\"grow_gpu_approx\"`: Enabled when `tree_method` is set to `\"approx\"` along with `device=\"cuda\"`.\n#' - `\"sync\"`: synchronizes trees in all distributed nodes.\n#' - `\"refresh\"`: refreshes tree's statistics and/or leaf values based on the current data. Note that no random subsampling of data rows is performed.\n#' - `\"prune\"`: prunes the splits where loss < `min_split_loss` (or `gamma`) and nodes that have depth greater than `max_depth`.\n#'\n#' - For `booster=\"gblinear\"`:\n#' (default= `\"shotgun\"`) Choice of algorithm to fit linear model\n#' - `\"shotgun\"`: Parallel coordinate descent algorithm based on shotgun algorithm. Uses 'hogwild' parallelism and therefore produces a nondeterministic solution on each run.\n#' - `\"coord_descent\"`: Ordinary coordinate descent algorithm. Also multithreaded but still produces a deterministic solution. When the `device` parameter is set to `\"cuda\"` or `\"gpu\"`, a GPU variant would be used.\n#' @param refresh_leaf (for Tree Booster) (default=1)\n#' This is a parameter of the `\"refresh\"` updater. When this flag is 1, tree leafs as well as tree nodes' stats are updated. When it is 0, only node stats are updated.\n#' @param grow_policy (for Tree Booster) (default= `\"depthwise\"`)\n#' - Controls a way new nodes are added to the tree.\n#' - Currently supported only if `tree_method` is set to `\"hist\"` or `\"approx\"`.\n#' - Choices: `\"depthwise\"`, `\"lossguide\"`\n#' - `\"depthwise\"`: split at nodes closest to the root.\n#' - `\"lossguide\"`: split at nodes with highest loss change.\n#' @param max_leaves (for Tree Booster) (default=0, type=int32)\n#' Maximum number of nodes to be added. Not used by `\"exact\"` tree method.\n#' @param max_bin (for Tree Booster) (default=256, type=int32)\n#' - Only used if `tree_method` is set to `\"hist\"` or `\"approx\"`.\n#' - Maximum number of discrete bins to bucket continuous features.\n#' - Increasing this number improves the optimality of splits at the cost of higher computation time.\n#' @param num_parallel_tree (for Tree Booster) (default=1)\n#' Number of parallel trees constructed during each iteration. This option is used to support boosted random forest.\n#' @param monotone_constraints (for Tree Booster)\n#' Constraint of variable monotonicity. See [Monotonic Constraints](https://xgboost.readthedocs.io/en/latest/tutorials/monotonic.html) for more information.\n#' @param interaction_constraints (for Tree Booster)\n#' Constraints for interaction representing permitted interactions. The constraints must\n#' be specified in the form of a nest list, e.g. `list(c(0, 1), c(2, 3, 4))`, where each inner\n#' list is a group of indices of features (base-0 numeration) that are allowed to interact with each other.\n#' See [Feature Interaction Constraints](https://xgboost.readthedocs.io/en/latest/tutorials/feature_interaction_constraint.html) for more information.\n#' @param multi_strategy (for Tree Booster) (default = `\"one_output_per_tree\"`)\n#' The strategy used for training multi-target models, including multi-target regression\n#' and multi-class classification. See [Multiple Outputs](https://xgboost.readthedocs.io/en/latest/tutorials/multioutput.html) for more information.\n#' - `\"one_output_per_tree\"`: One model for each target.\n#' - `\"multi_output_tree\"`: Use multi-target trees.\n#'\n#' Version added: 2.0.0\n#'\n#' Note: This parameter is working-in-progress.\n#' @param base_score\n#' - The initial prediction score of all instances, global bias\n#' - The parameter is automatically estimated for selected objectives before training. To\n#' disable the estimation, specify a real number argument.\n#' - If `base_margin` is supplied, `base_score` will not be added.\n#' - For sufficient number of iterations, changing this value will not have too much effect.\n#' @param eval_metric (default according to objective)\n#' - Evaluation metrics for validation data, a default metric will be assigned according to objective (rmse for regression, and logloss for classification, `mean average precision` for ``rank:map``, etc.)\n#' - User can add multiple evaluation metrics.\n#' - The choices are listed below:\n#' - `\"rmse\"`: root mean square error\n#' - `\"rmsle\"`: root mean square log error: \\eqn{\\sqrt{\\frac{1}{N}[log(pred + 1) - log(label + 1)]^2}}. Default metric of `\"reg:squaredlogerror\"` objective. This metric reduces errors generated by outliers in dataset. But because `log` function is employed, `\"rmsle\"` might output `nan` when prediction value is less than -1. See `\"reg:squaredlogerror\"` for other requirements.\n#' - `\"mae\"`: mean absolute error.\n#' - `\"mape\"`: mean absolute percentage error.\n#' - `\"mphe\"`: mean Pseudo Huber error. Default metric of `\"reg:pseudohubererror\"` objective.\n#' - `\"logloss\"`: negative log-likelihood.\n#' - `\"error\"`: Binary classification error rate. It is calculated as `#(wrong cases)/#(all cases)`. For the predictions, the evaluation will regard the instances with prediction value larger than 0.5 as positive instances, and the others as negative instances.\n#' - `\"error@t\"`: a different than 0.5 binary classification threshold value could be specified by providing a numerical value through 't'.\n#' - `\"merror\"`: Multiclass classification error rate. It is calculated as `#(wrong cases)/#(all cases)`.\n#' - `\"mlogloss\"`: [Multiclass logloss](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.log_loss.html).\n#' - `\"auc\"`: area under the receiver-operating characteristic curve.\n#' Available for classification and learning-to-rank tasks.\n#' - When used with binary classification, the objective should be `\"binary:logistic\"` or similar functions that work on probability.\n#' - When used with multi-class classification, objective should be `\"multi:softprob\"` instead of `\"multi:softmax\"`, as the latter doesn't output probability. Also the AUC is calculated by 1-vs-rest with reference class weighted by class prevalence.\n#' - When used with LTR task, the AUC is computed by comparing pairs of documents to count correctly sorted pairs. This corresponds to pairwise learning to rank. The implementation has some issues with average AUC around groups and distributed workers not being well-defined.\n#' - On a single machine the AUC calculation is exact. In a distributed environment the AUC is a weighted average over the AUC of training rows on each node - therefore, distributed AUC is an approximation sensitive to the distribution of data across workers. Use another metric in distributed environments if precision and reproducibility are important.\n#' - When input dataset contains only negative or positive samples, the output is `NaN`. The behavior is implementation defined, for instance, `scikit-learn` returns \\eqn{0.5} instead.\n#' - `\"aucpr\"`: area under the PR curve\n#' Available for classification and learning-to-rank tasks.\n#'\n#' After XGBoost 1.6, both of the requirements and restrictions for using `\"aucpr\"` in classification problem are similar to `\"auc\"`. For ranking task, only binary relevance label \\eqn{y \\in [0, 1]} is supported. Different from `\"map\"` (mean average precision), `\"aucpr\"` calculates the *interpolated* area under precision recall curve using continuous interpolation.\n#'\n#' - `\"pre\"`: Precision at \\eqn{k}. Supports only learning to rank task.\n#' - `\"ndcg\"`: normalized discounted cumulative gain\n#' - `\"map\"`: mean average precision\n#'\n#' The `average precision` is defined as:\n#'\n#' \\eqn{AP@l = \\frac{1}{min{(l, N)}}\\sum^l_{k=1}P@k \\cdot I_{(k)}}\n#'\n#' where \\eqn{I_{(k)}} is an indicator function that equals to \\eqn{1} when the document at \\eqn{k} is relevant and \\eqn{0} otherwise. The \\eqn{P@k} is the precision at \\eqn{k}, and \\eqn{N} is the total number of relevant documents. Lastly, the `mean average precision` is defined as the weighted average across all queries.\n#'\n#' - `\"ndcg@n\"`, `\"map@n\"`, `\"pre@n\"`: \\eqn{n} can be assigned as an integer to cut off the top positions in the lists for evaluation.\n#' - `\"ndcg-\"`, `\"map-\"`, `\"ndcg@n-\"`, `\"map@n-\"`: In XGBoost, the NDCG and MAP evaluate the score of a list without any positive samples as \\eqn{1}. By appending \"-\" to the evaluation metric name, we can ask XGBoost to evaluate these scores as \\eqn{0} to be consistent under some conditions.\n#' - `\"poisson-nloglik\"`: negative log-likelihood for Poisson regression\n#' - `\"gamma-nloglik\"`: negative log-likelihood for gamma regression\n#' - `\"cox-nloglik\"`: negative partial log-likelihood for Cox proportional hazards regression\n#' - `\"gamma-deviance\"`: residual deviance for gamma regression\n#' - `\"tweedie-nloglik\"`: negative log-likelihood for Tweedie regression (at a specified value of the `tweedie_variance_power` parameter)\n#' - `\"aft-nloglik\"`: Negative log likelihood of Accelerated Failure Time model.\n#' See [Survival Analysis with Accelerated Failure Time](https://xgboost.readthedocs.io/en/latest/tutorials/aft_survival_analysis.html) for details.\n#' - `\"interval-regression-accuracy\"`: Fraction of data points whose predicted labels fall in the interval-censored labels.\n#' Only applicable for interval-censored data. See [Survival Analysis with Accelerated Failure Time](https://xgboost.readthedocs.io/en/latest/tutorials/aft_survival_analysis.html) for details.\n#' @param seed_per_iteration (default= `FALSE`)\n#' Seed PRNG determnisticly via iterator number.\n#' @param device (default= `\"cpu\"`)\n#' Device for XGBoost to run. User can set it to one of the following values:\n#' - `\"cpu\"`: Use CPU.\n#' - `\"cuda\"`: Use a GPU (CUDA device).\n#' - `\"cuda:\"`: `` is an integer that specifies the ordinal of the GPU (which GPU do you want to use if you have more than one devices).\n#' - `\"gpu\"`: Default GPU device selection from the list of available and supported devices. Only `\"cuda\"` devices are supported currently.\n#' - `\"gpu:\"`: Default GPU device selection from the list of available and supported devices. Only `\"cuda\"` devices are supported currently.\n#'\n#' For more information about GPU acceleration, see [XGBoost GPU Support](https://xgboost.readthedocs.io/en/latest/gpu/index.html). In distributed environments, ordinal selection is handled by distributed frameworks instead of XGBoost. As a result, using `\"cuda:\"` will result in an error. Use `\"cuda\"` instead.\n#'\n#' Version added: 2.0.0\n#'\n#' Note: if XGBoost was installed from CRAN, it won't have GPU support enabled, thus only `\"cpu\"` will be available.\n#' To get GPU support, the R package for XGBoost must be installed from source or from the GitHub releases - see\n#' [instructions](https://xgboost.readthedocs.io/en/latest/install.html#r).\n#' @param disable_default_eval_metric (default= `FALSE`)\n#' Flag to disable default metric. Set to 1 or `TRUE` to disable.\n#' @param use_rmm Whether to use RAPIDS Memory Manager (RMM) to allocate cache GPU\n#' memory. The primary memory is always allocated on the RMM pool when XGBoost is built\n#' (compiled) with the RMM plugin enabled. Valid values are `TRUE` and `FALSE`. See\n#' [Using XGBoost with RAPIDS Memory Manager (RMM) plugin](https://xgboost.readthedocs.io/en/latest/python/rmm-examples/index.html) for details.\n#' @param max_cached_hist_node (for Non-Exact Tree Methods) (default = 65536)\n#' Maximum number of cached nodes for histogram. This can be used with the `\"hist\"` and the\n#' `\"approx\"` tree methods.\n#'\n#' Version added: 2.0.0\n#'\n#' - For most of the cases this parameter should not be set except for growing deep\n#' trees. After 3.0, this parameter affects GPU algorithms as well.\n#' @param max_cat_to_onehot (for Non-Exact Tree Methods)\n#' A threshold for deciding whether XGBoost should use one-hot encoding based split for\n#' categorical data. When number of categories is lesser than the threshold then one-hot\n#' encoding is chosen, otherwise the categories will be partitioned into children nodes.\n#'\n#' Version added: 1.6.0\n#' @param max_cat_threshold (for Non-Exact Tree Methods)\n#' Maximum number of categories considered for each split. Used only by partition-based\n#' splits for preventing over-fitting.\n#'\n#' Version added: 1.7.0\n#' @param sample_type (for Dart Booster) (default= `\"uniform\"`)\n#' Type of sampling algorithm.\n#' - `\"uniform\"`: dropped trees are selected uniformly.\n#' - `\"weighted\"`: dropped trees are selected in proportion to weight.\n#' @param normalize_type (for Dart Booster) (default= `\"tree\"`)\n#' Type of normalization algorithm.\n#' - `\"tree\"`: new trees have the same weight of each of dropped trees.\n#' - Weight of new trees are `1 / (k + learning_rate)`.\n#' - Dropped trees are scaled by a factor of `k / (k + learning_rate)`.\n#' - `\"forest\"`: new trees have the same weight of sum of dropped trees (forest).\n#' - Weight of new trees are `1 / (1 + learning_rate)`.\n#' - Dropped trees are scaled by a factor of `1 / (1 + learning_rate)`.\n#' @param rate_drop (for Dart Booster) (default=0.0)\n#' Dropout rate (a fraction of previous trees to drop during the dropout).\n#'\n#' range: \\eqn{[0.0, 1.0]}\n#' @param one_drop (for Dart Booster) (default=0)\n#' When this flag is enabled, at least one tree is always dropped during the dropout (allows Binomial-plus-one or epsilon-dropout from the original DART paper).\n#' @param skip_drop (for Dart Booster) (default=0.0)\n#' Probability of skipping the dropout procedure during a boosting iteration.\n#' - If a dropout is skipped, new trees are added in the same manner as `\"gbtree\"`.\n#' - Note that non-zero `skip_drop` has higher priority than `rate_drop` or `one_drop`.\n#'\n#' range: \\eqn{[0.0, 1.0]}\n#' @param feature_selector (for Linear Booster) (default= `\"cyclic\"`)\n#' Feature selection and ordering method\n#' - `\"cyclic\"`: Deterministic selection by cycling through features one at a time.\n#' - `\"shuffle\"`: Similar to `\"cyclic\"` but with random feature shuffling prior to each update.\n#' - `\"random\"`: A random (with replacement) coordinate selector.\n#' - `\"greedy\"`: Select coordinate with the greatest gradient magnitude. It has `O(num_feature^2)` complexity. It is fully deterministic. It allows restricting the selection to `top_k` features per group with the largest magnitude of univariate weight change, by setting the `top_k` parameter. Doing so would reduce the complexity to `O(num_feature*top_k)`.\n#' - `\"thrifty\"`: Thrifty, approximately-greedy feature selector. Prior to cyclic updates, reorders features in descending magnitude of their univariate weight changes. This operation is multithreaded and is a linear complexity approximation of the quadratic greedy selection. It allows restricting the selection to `top_k` features per group with the largest magnitude of univariate weight change, by setting the `top_k` parameter.\n#' @param top_k (for Linear Booster) (default=0)\n#' The number of top features to select in `greedy` and `thrifty` feature selector. The value of 0 means using all the features.\n#' @param num_class Number of classes when using multi-class classification objectives (e.g. `objective=\"multi:softprob\"`)\n#' @param tweedie_variance_power (for Tweedie Regression (`\"objective=reg:tweedie\"`)) (default=1.5)\n#' - Parameter that controls the variance of the Tweedie distribution `var(y) ~ E(y)^tweedie_variance_power`\n#' - range: \\eqn{(1,2)}\n#' - Set closer to 2 to shift towards a gamma distribution\n#' - Set closer to 1 to shift towards a Poisson distribution.\n#' @param huber_slope (for using Pseudo-Huber (`\"reg:pseudohubererror`\")) (default = 1.0)\n#' A parameter used for Pseudo-Huber loss to define the \\eqn{\\delta} term.\n#' @param quantile_alpha (for using Quantile Loss (`\"reg:quantileerror\"`))\n#' A scalar or a list of targeted quantiles (passed as a numeric vector).\n#'\n#' Version added: 2.0.0\n#' @param expectile_alpha (for using Expectile Loss (`\"reg:expectileerror\"`))\n#' A scalar or a list of targeted expectiles (passed as a numeric vector).\n#' @param aft_loss_distribution (when using AFT Survival Loss (`\"survival:aft\"`) and Negative Log Likelihood of AFT metric (`\"aft-nloglik\"`))\n#' Probability Density Function, `\"normal\"`, `\"logistic\"`, or `\"extreme\"`.\n#' @param aft_loss_distribution_scale (when using AFT Survival Loss (`\"survival:aft\"`) and Negative Log Likelihood of AFT metric (`\"aft-nloglik\"`))\n#' Scaling factor for the AFT distribution. Range: \\eqn{(0, \\infty)}.\n#' @param lambdarank_pair_method (for learning to rank (`\"rank:ndcg\"`, `\"rank:map\"`, `\"rank:pairwise\"`)) (default = `\"topk\"`)\n#' How to construct pairs for pair-wise learning.\n#' - `\"mean\"`: Sample `lambdarank_num_pair_per_sample` pairs for each document in the query list.\n#' - `\"topk\"`: Focus on top-`lambdarank_num_pair_per_sample` documents. Construct \\eqn{|query|} pairs for each document at the top-`lambdarank_num_pair_per_sample` ranked by the model.\n#' @param lambdarank_num_pair_per_sample (for learning to rank (`\"rank:ndcg\"`, `\"rank:map\"`, `\"rank:pairwise\"`))\n#' It specifies the number of pairs sampled for each document when pair method is `\"mean\"`, or the truncation level for queries when the pair method is `\"topk\"`. For example, to train with `ndcg@6`, set `\"lambdarank_num_pair_per_sample\"` to \\eqn{6} and `lambdarank_pair_method` to `\"topk\"`.\n#'\n#' range = \\eqn{[1, \\infty)}\n#' @param lambdarank_normalization (for learning to rank (`\"rank:ndcg\"`, `\"rank:map\"`, `\"rank:pairwise\"`)) (default = `TRUE`)\n#' Whether to normalize the leaf value by lambda gradient. This can sometimes stagnate the training progress.\n#'\n#' Version added: 2.1.0\n#'\n#' @param lambdarank_score_normalization\n#'\n#' Whether to normalize the delta metric by the difference of prediction scores. This can\n#' sometimes stagnate the training progress. With pairwise ranking, we can normalize the\n#' gradient using the difference between two samples in each pair to reduce influence from\n#' the pairs that have large difference in ranking scores. This can help us regularize the\n#' model to reduce bias and prevent overfitting. Similar to other regularization\n#' techniques, this might prevent training from converging.\n#'\n#' There was no normalization before 2.0. In 2.0 and later versions this is used by\n#' default. In 3.0, we made this an option that users can disable.\n#'\n#' Version added: 3.0.0\n#'\n#' @param lambdarank_unbiased (for learning to rank (`\"rank:ndcg\"`, `\"rank:map\"`, `\"rank:pairwise\"`)) (default = `FALSE`)\n#' Specify whether do we need to debias input click data.\n#' @param lambdarank_bias_norm (for learning to rank (`\"rank:ndcg\"`, `\"rank:map\"`, `\"rank:pairwise\"`)) (default = 2.0)\n#' \\eqn{L_p} normalization for position debiasing, default is \\eqn{L_2}. Only relevant when `lambdarank_unbiased` is set to `TRUE`.\n#' @param ndcg_exp_gain (for learning to rank (`\"rank:ndcg\"`, `\"rank:map\"`, `\"rank:pairwise\"`)) (default = `TRUE`)\n#' Whether we should use exponential gain function for `NDCG`. There are two forms of gain function for `NDCG`, one is using relevance value directly while the other is using\\eqn{2^{rel} - 1} to emphasize on retrieving relevant documents. When `ndcg_exp_gain` is `TRUE` (the default), relevance degree cannot be greater than 31.\nxgb.params <- function(\n objective = NULL,\n verbosity = NULL,\n nthread = NULL,\n seed = NULL,\n booster = NULL,\n eta = NULL,\n learning_rate = NULL,\n gamma = NULL,\n min_split_loss = NULL,\n max_depth = NULL,\n min_child_weight = NULL,\n max_delta_step = NULL,\n subsample = NULL,\n sampling_method = NULL,\n colsample_bytree = NULL,\n colsample_bylevel = NULL,\n colsample_bynode = NULL,\n lambda = NULL,\n reg_lambda = NULL,\n alpha = NULL,\n reg_alpha = NULL,\n tree_method = NULL,\n scale_pos_weight = NULL,\n updater = NULL,\n refresh_leaf = NULL,\n grow_policy = NULL,\n max_leaves = NULL,\n max_bin = NULL,\n num_parallel_tree = NULL,\n monotone_constraints = NULL,\n interaction_constraints = NULL,\n multi_strategy = NULL,\n base_score = NULL,\n eval_metric = NULL,\n seed_per_iteration = NULL,\n device = NULL,\n disable_default_eval_metric = NULL,\n use_rmm = NULL,\n max_cached_hist_node = NULL,\n max_cat_to_onehot = NULL,\n max_cat_threshold = NULL,\n sample_type = NULL,\n normalize_type = NULL,\n rate_drop = NULL,\n one_drop = NULL,\n skip_drop = NULL,\n feature_selector = NULL,\n top_k = NULL,\n num_class = NULL,\n tweedie_variance_power = NULL,\n huber_slope = NULL,\n quantile_alpha = NULL,\n expectile_alpha = NULL,\n aft_loss_distribution = NULL,\n aft_loss_distribution_scale = NULL,\n lambdarank_pair_method = NULL,\n lambdarank_num_pair_per_sample = NULL,\n lambdarank_normalization = NULL,\n lambdarank_score_normalization = NULL,\n lambdarank_unbiased = NULL,\n lambdarank_bias_norm = NULL,\n ndcg_exp_gain = NULL\n) {\n# nolint end\n out <- as.list(environment())\n out <- out[!sapply(out, is.null)]\n return(out)\n}\n" }, { "path": "R-package/R/xgboost.R", "content": "prescreen.objective <- function(objective) {\n if (!is.null(objective)) {\n if (!is.character(objective) || length(objective) != 1L || is.na(objective)) {\n stop(\"'objective' must be a single character/string variable.\")\n }\n\n if (objective %in% .OBJECTIVES_NON_DEFAULT_MODE()) {\n stop(\n \"Objectives with non-default prediction mode (\",\n paste(.OBJECTIVES_NON_DEFAULT_MODE(), collapse = \", \"),\n \") are not supported in 'xgboost()'. Try 'xgb.train()'.\"\n )\n }\n\n if (objective %in% .RANKING_OBJECTIVES()) {\n stop(\"Ranking objectives are not supported in 'xgboost()'. Try 'xgb.train()'.\")\n }\n }\n}\n\nprocess.base.margin <- function(base_margin, nrows, ncols) {\n if (!NROW(base_margin)) {\n return(NULL)\n }\n if (is.array(base_margin) && length(dim(base_margin)) > 2) {\n stop(\n \"'base_margin' should not have more than 2 dimensions for any objective (got: \",\n length(dim(base_margin)),\n \" dimensions).\"\n )\n }\n if (inherits(base_margin, c(\"sparseMatrix\", \"sparseVector\"))) {\n warning(\n \"Got a sparse matrix type (class: \",\n paste(class(base_margin), collapse = \", \"),\n \") for 'base_margin'. Will convert to dense matrix.\"\n )\n base_margin <- as.matrix(base_margin)\n }\n if (NROW(base_margin) != nrows) {\n stop(\n \"'base_margin' has incorrect number of rows. Expected: \",\n nrows,\n \". Got: \",\n NROW(base_margin)\n )\n }\n\n if (ncols == 1L) {\n if (inherits(base_margin, c(\"matrix\", \"data.frame\"))) {\n if (ncol(base_margin) != 1L) {\n stop(\"'base_margin' should be a 1-d vector for the given objective and data.\")\n }\n if (is.data.frame(base_margin)) {\n base_margin <- base_margin[[1L]]\n } else {\n base_margin <- base_margin[, 1L]\n }\n }\n if (!is.numeric(base_margin)) {\n base_margin <- as.numeric(base_margin)\n }\n } else {\n supported_multicol <- c(\"matrix\", \"data.frame\")\n if (!inherits(base_margin, supported_multicol)) {\n stop(\n \"'base_margin' should be a matrix with \",\n ncols,\n \" columns for the given objective and data. Got class: \",\n paste(class(base_margin), collapse = \", \")\n )\n }\n if (ncol(base_margin) != ncols) {\n stop(\n \"'base_margin' has incorrect number of columns. Expected: \",\n ncols,\n \". Got: \",\n ncol(base_margin)\n )\n }\n if (!is.matrix(base_margin)) {\n base_margin <- as.matrix(base_margin)\n }\n }\n\n return(base_margin)\n}\n\nprocess.y.margin.and.objective <- function(\n y,\n base_margin,\n objective,\n params\n) {\n\n if (!NROW(y)) {\n stop(\"Passed empty 'y'.\")\n }\n\n if (is.array(y) && length(dim(y)) > 2) {\n stop(\n \"'y' should not have more than 2 dimensions for any objective (got: \",\n length(dim(y)),\n \").\"\n )\n }\n\n if (inherits(y, c(\"sparseMatrix\", \"sparseVector\"))) {\n warning(\n \"Got a sparse matrix type (class: \",\n paste(class(y), collapse = \", \"),\n \") for 'y'. Will convert to dense matrix.\"\n )\n y <- as.matrix(y)\n }\n\n if (is.character(y)) {\n if (!is.vector(y)) {\n if (NCOL(y) > 1L) {\n stop(\"Multi-column categorical 'y' is not supported.\")\n }\n y <- as.vector(y)\n }\n y <- factor(y)\n }\n\n if (is.logical(y)) {\n if (!is.vector(y)) {\n if (NCOL(y) > 1L) {\n stop(\"Multi-column logical/boolean 'y' is not supported.\")\n }\n y <- as.vector(y)\n }\n y <- factor(y, c(FALSE, TRUE))\n }\n\n if (is.factor(y)) {\n\n y_levels <- levels(y)\n if (length(y_levels) < 2) {\n stop(\"Factor 'y' has less than 2 levels.\")\n }\n if (length(y_levels) == 2) {\n if (is.null(objective)) {\n objective <- \"binary:logistic\"\n } else {\n if (!(objective %in% .BINARY_CLASSIF_OBJECTIVES())) {\n stop(\n \"Got binary 'y' - supported objectives for this data are: \",\n paste(.BINARY_CLASSIF_OBJECTIVES(), collapse = \", \"),\n \". Was passed: \",\n objective\n )\n }\n }\n\n if (!is.null(base_margin)) {\n base_margin <- process.base.margin(base_margin, length(y), 1)\n }\n\n out <- list(\n params = list(\n objective = objective\n ),\n metadata = list(\n y_levels = y_levels,\n n_targets = 1\n )\n )\n } else { # length(levels) > 2\n if (is.null(objective)) {\n objective <- \"multi:softprob\"\n } else {\n if (!(objective %in% .MULTICLASS_CLASSIF_OBJECTIVES())) {\n stop(\n \"Got non-binary factor 'y' - supported objectives for this data are: \",\n paste(.MULTICLASS_CLASSIF_OBJECTIVES(), collapse = \", \"),\n \". Was passed: \",\n objective\n )\n }\n }\n\n if (!is.null(base_margin)) {\n base_margin <- process.base.margin(base_margin, length(y), length(y_levels))\n }\n\n out <- list(\n params = list(\n objective = objective,\n num_class = length(y_levels)\n ),\n metadata = list(\n y_levels = y_levels,\n n_targets = length(y_levels)\n )\n )\n }\n\n out$dmatrix_args <- list(\n label = as.numeric(y) - 1,\n base_margin = base_margin\n )\n\n } else if (inherits(y, \"Surv\")) {\n\n y_attr <- attributes(y)\n supported_surv_types <- c(\"left\", \"right\", \"interval\")\n if (!(y_attr$type %in% supported_surv_types)) {\n stop(\n \"Survival objectives are only supported for types: \",\n paste(supported_surv_types, collapse = \", \"),\n \". Was passed: \",\n y_attr$type\n )\n }\n\n if (is.null(objective)) {\n objective <- \"survival:aft\"\n } else {\n if (y_attr$type == \"right\") {\n if (!(objective %in% .SURVIVAL_RIGHT_CENSORING_OBJECTIVES())) {\n stop(\n \"Got right-censored 'y' variable - supported objectives for this data are: \",\n paste(.SURVIVAL_RIGHT_CENSORING_OBJECTIVES(), collapse = \", \"),\n \". Was passed: \",\n objective\n )\n }\n } else {\n if (!(objective %in% .SURVIVAL_ALL_CENSORING_OBJECTIVES())) {\n stop(\n \"Got \", y_attr$type, \"-censored 'y' variable - supported objectives for this data are:\",\n paste(.SURVIVAL_ALL_CENSORING_OBJECTIVES(), collapse = \", \"),\n \". Was passed: \",\n objective\n )\n }\n }\n }\n\n if (!is.null(base_margin)) {\n base_margin <- process.base.margin(base_margin, nrow(y), 1)\n }\n\n out <- list(\n params = list(\n objective = objective\n ),\n metadata = list(\n n_targets = 1\n )\n )\n\n # Note: the 'Surv' object class that is passed as 'y' might have either 2 or 3 columns\n # depending on the type of censoring, and the last column in both cases is the one that\n # indicates the observation type (e.g. censored / uncensored).\n # In the case of interval censoring, the second column will not always have values with\n # infinites filled in. For more information, see the code behind the 'print.Surv' method.\n\n if (objective == \"survival:cox\") {\n # Can only get here when using right censoring\n if (y_attr$type != \"right\") {\n stop(\"Internal error.\")\n }\n\n out$dmatrix_args <- list(\n label = y[, 1L] * (2 * (y[, 2L] - 0.5))\n )\n\n } else {\n if (y_attr$type == \"left\") {\n lb <- ifelse(\n y[, 2L] == 0,\n 0,\n y[, 1L]\n )\n ub <- y[, 1L]\n out$dmatrix_args <- list(\n label_lower_bound = lb,\n label_upper_bound = ub\n )\n } else if (y_attr$type == \"right\") {\n lb <- y[, 1L]\n ub <- ifelse(\n y[, 2L] == 0,\n Inf,\n y[, 1L]\n )\n out$dmatrix_args <- list(\n label_lower_bound = lb,\n label_upper_bound = ub\n )\n } else if (y_attr$type == \"interval\") {\n out$dmatrix_args <- list(\n label_lower_bound = ifelse(y[, 3L] == 2, 0, y[, 1L]),\n label_upper_bound = ifelse(\n y[, 3L] == 0, Inf,\n ifelse(y[, 3L] == 3, y[, 2L], y[, 1L])\n )\n )\n }\n\n if (min(out$dmatrix_args$label_lower_bound) < 0) {\n stop(\"Survival objectives are only defined for non-negative 'y'.\")\n }\n }\n\n out$dmatrix_args$base_margin <- base_margin\n\n } else if (is.vector(y)) {\n\n if (is.null(objective)) {\n objective <- \"reg:squarederror\"\n } else if (!(objective %in% .REGRESSION_OBJECTIVES())) {\n stop(\n \"Got numeric 'y' - supported objectives for this data are: \",\n paste(.REGRESSION_OBJECTIVES(), collapse = \", \"),\n \". Was passed: \",\n objective\n )\n }\n\n n_targets <- 1L\n if (objective == \"reg:quantileerror\" && NROW(params$quantile_alpha) > 1) {\n n_targets <- NROW(params$quantile_alpha)\n } else if (objective == \"reg:expectileerror\" && NROW(params$expectile_alpha) > 1) {\n n_targets <- NROW(params$expectile_alpha)\n }\n\n if (!is.null(base_margin)) {\n base_margin <- process.base.margin(base_margin, length(y), n_targets)\n }\n\n out <- list(\n params = list(\n objective = objective\n ),\n metadata = list(\n n_targets = n_targets\n ),\n dmatrix_args = list(\n label = as.numeric(y),\n base_margin = base_margin\n )\n )\n\n } else if (is.data.frame(y)) {\n if (ncol(y) == 1L) {\n return(process.y.margin.and.objective(y[[1L]], base_margin, objective, params))\n }\n\n if (is.null(objective)) {\n objective <- \"reg:squarederror\"\n } else if (!(objective %in% .MULTI_TARGET_OBJECTIVES())) {\n stop(\n \"Got multi-column 'y' - supported objectives for this data are: \",\n paste(.MULTI_TARGET_OBJECTIVES(), collapse = \", \"),\n \". Was passed: \",\n objective\n )\n }\n\n y_names <- names(y)\n y <- lapply(y, function(x) {\n if (!inherits(x, c(\"numeric\", \"integer\"))) {\n stop(\n \"Multi-target 'y' only supports 'numeric' and 'integer' types. Got: \",\n paste(class(x), collapse = \", \")\n )\n }\n return(as.numeric(x))\n })\n y <- as.data.frame(y) |> as.matrix()\n\n if (!is.null(base_margin)) {\n base_margin <- process.base.margin(base_margin, length(y), ncol(y))\n }\n\n out <- list(\n params = list(\n objective = objective\n ),\n dmatrix_args = list(\n label = y,\n base_margin = base_margin\n ),\n metadata = list(\n y_names = y_names,\n n_targets = ncol(y)\n )\n )\n\n } else if (is.matrix(y)) {\n if (ncol(y) == 1L) {\n return(process.y.margin.and.objective(as.vector(y), base_margin, objective, params))\n }\n\n if (!is.null(objective) && !(objective %in% .MULTI_TARGET_OBJECTIVES())) {\n stop(\n \"Got multi-column 'y' - supported objectives for this data are: \",\n paste(.MULTI_TARGET_OBJECTIVES(), collapse = \", \"),\n \". Was passed: \",\n objective\n )\n }\n if (is.null(objective)) {\n objective <- \"reg:squarederror\"\n }\n\n y_names <- colnames(y)\n if (storage.mode(y) != \"double\") {\n storage.mode(y) <- \"double\"\n }\n\n if (!is.null(base_margin)) {\n base_margin <- process.base.margin(base_margin, nrow(y), ncol(y))\n }\n\n out <- list(\n params = list(\n objective = objective\n ),\n dmatrix_args = list(\n label = y,\n base_margin = base_margin\n ),\n metadata = list(\n n_targets = ncol(y)\n )\n )\n\n if (NROW(y_names) == ncol(y)) {\n out$metadata$y_names <- y_names\n }\n\n } else {\n stop(\"Passed 'y' object with unsupported class: \", paste(class(y), collapse = \", \"))\n }\n\n return(out)\n}\n\nprocess.row.weights <- function(w, lst_args) {\n if (!is.null(w)) {\n if (\"label\" %in% names(lst_args$dmatrix_args)) {\n nrow_y <- NROW(lst_args$dmatrix_args$label)\n } else if (\"label_lower_bound\" %in% names(lst_args$dmatrix_args)) {\n nrow_y <- length(lst_args$dmatrix_args$label_lower_bound)\n } else {\n stop(\"Internal error.\")\n }\n if (!is.numeric(w)) {\n w <- as.numeric(w)\n }\n if (length(w) != nrow_y) {\n stop(\n \"'weights' must be a 1-d vector with the same length as 'y' (\",\n length(w), \" vs. \", nrow_y, \").\"\n )\n }\n lst_args$dmatrix_args$weight <- w\n }\n return(lst_args)\n}\n\ncheck.nthreads <- function(nthreads) {\n if (is.null(nthreads)) {\n return(1L)\n }\n if (!inherits(nthreads, c(\"numeric\", \"integer\")) || !NROW(nthreads)) {\n stop(\"'nthreads' must be a positive scalar value.\")\n }\n if (length(nthreads) > 1L) {\n nthreads <- utils::head(nthreads, 1L)\n }\n if (is.na(nthreads) || nthreads < 0) {\n stop(\"Passed invalid 'nthreads': \", nthreads)\n }\n if (is.numeric(nthreads)) {\n if (floor(nthreads) != nthreads) {\n stop(\"'nthreads' must be an integer.\")\n }\n }\n return(as.integer(nthreads))\n}\n\ncheck.can.use.qdm <- function(x, params, eval_set) {\n if (\"booster\" %in% names(params)) {\n if (params$booster == \"gblinear\") {\n return(FALSE)\n }\n }\n if (\"tree_method\" %in% names(params)) {\n if (params$tree_method %in% c(\"exact\", \"approx\")) {\n return(FALSE)\n }\n }\n if (NROW(eval_set)) {\n return(FALSE)\n }\n return(TRUE)\n}\n\nprocess.x.and.col.args <- function(\n x,\n monotone_constraints,\n interaction_constraints,\n feature_weights,\n lst_args,\n use_qdm\n) {\n if (is.null(x)) {\n stop(\"'x' cannot be NULL.\")\n }\n if (inherits(x, \"xgb.DMatrix\")) {\n stop(\"Cannot pass 'xgb.DMatrix' as 'x' to 'xgboost()'. Try 'xgb.train()' instead.\")\n }\n supported_x_types <- c(\"data.frame\", \"matrix\", \"dgTMatrix\", \"dgCMatrix\", \"dgRMatrix\")\n if (!inherits(x, supported_x_types)) {\n stop(\n \"'x' must be one of the following classes: \",\n paste(supported_x_types, collapse = \", \"),\n \". Got: \",\n paste(class(x), collapse = \", \")\n )\n }\n if (use_qdm && inherits(x, \"sparseMatrix\") && !inherits(x, \"dgRMatrix\")) {\n x <- methods::as(x, \"RsparseMatrix\")\n if (!inherits(x, \"RsparseMatrix\")) {\n stop(\"Internal error: casting sparse matrix did not yield 'dgRMatrix'.\")\n }\n }\n\n if (NROW(feature_weights)) {\n if (is.list(feature_weights)) {\n feature_weights <- unlist(feature_weights)\n }\n if (!inherits(feature_weights, c(\"numeric\", \"integer\"))) {\n stop(\"'feature_weights' must be a numeric vector or named list matching to columns of 'x'.\")\n }\n if (NROW(names(feature_weights)) && NROW(colnames(x))) {\n matched <- match(colnames(x), names(feature_weights))\n matched <- matched[!is.na(matched)]\n matched <- matched[!duplicated(matched)]\n if (length(matched) > 0 && length(matched) < length(feature_weights)) {\n stop(\n \"'feature_weights' names do not contain all columns of 'x'. Missing: \",\n utils::head(setdiff(colnames(x), names(feature_weights)))\n )\n }\n if (length(matched)) {\n feature_weights <- feature_weights[matched]\n } else {\n warning(\"Names of 'feature_weights' do not match with 'x'. Names will be ignored.\")\n }\n }\n\n lst_args$dmatrix_args$feature_weights <- unname(feature_weights)\n }\n\n if (NROW(monotone_constraints)) {\n\n if (NROW(monotone_constraints) > ncol(x)) {\n stop(\n \"'monotone_constraints' contains more entries than there are columns in 'x' (\",\n NROW(monotone_constraints), \" vs. \", ncol(x), \").\"\n )\n }\n\n if (is.list(monotone_constraints)) {\n\n if (!NROW(names(monotone_constraints))) {\n stop(\n \"If passing 'monotone_constraints' as a named list,\",\n \" must have names matching to columns of 'x'.\"\n )\n }\n if (!NROW(colnames(x))) {\n stop(\"If passing 'monotone_constraints' as a named list, 'x' must have column names.\")\n }\n if (anyDuplicated(names(monotone_constraints))) {\n stop(\n \"'monotone_constraints' contains duplicated names: \",\n paste(\n names(monotone_constraints)[duplicated(names(monotone_constraints))] |> utils::head(),\n collapse = \", \"\n )\n )\n }\n if (NROW(setdiff(names(monotone_constraints), colnames(x)))) {\n stop(\n \"'monotone_constraints' contains column names not present in 'x': \",\n paste(utils::head(names(monotone_constraints)), collapse = \", \")\n )\n }\n\n vec_monotone_constr <- rep(0, ncol(x))\n matched <- match(names(monotone_constraints), colnames(x))\n vec_monotone_constr[matched] <- unlist(monotone_constraints)\n lst_args$params$monotone_constraints <- unname(vec_monotone_constr)\n\n } else if (inherits(monotone_constraints, c(\"numeric\", \"integer\"))) {\n\n if (NROW(names(monotone_constraints)) && NROW(colnames(x))) {\n if (length(monotone_constraints) < ncol(x)) {\n return(\n process.x.and.col.args(\n x,\n as.list(monotone_constraints),\n interaction_constraints,\n feature_weights,\n lst_args,\n use_qdm\n )\n )\n } else {\n matched <- match(names(monotone_constraints), colnames(x))\n matched <- matched[!is.na(matched)]\n matched <- matched[!duplicated(matched)]\n if (length(matched)) {\n monotone_constraints <- monotone_constraints[matched]\n } else {\n warning(\"Names of 'monotone_constraints' do not match with 'x'. Names will be ignored.\")\n }\n }\n } else {\n if (length(monotone_constraints) != ncol(x)) {\n stop(\n \"If passing 'monotone_constraints' as unnamed vector or not using column names,\",\n \" must have length matching to number of columns in 'x'. Got: \",\n length(monotone_constraints), \" (vs. \", ncol(x), \")\"\n )\n }\n }\n\n lst_args$params$monotone_constraints <- unname(monotone_constraints)\n\n } else if (is.character(monotone_constraints)) {\n lst_args$params$monotone_constraints <- monotone_constraints\n } else {\n stop(\n \"Passed unsupported type for 'monotone_constraints': \",\n paste(class(monotone_constraints), collapse = \", \")\n )\n }\n }\n\n if (NROW(interaction_constraints)) {\n if (!is.list(interaction_constraints)) {\n stop(\"'interaction_constraints' must be a list of vectors.\")\n }\n cnames <- colnames(x)\n lst_args$params$interaction_constraints <- lapply(interaction_constraints, function(idx) {\n if (!NROW(idx)) {\n stop(\"Elements in 'interaction_constraints' cannot be empty.\")\n }\n\n if (is.character(idx)) {\n if (!NROW(cnames)) {\n stop(\n \"Passed a character vector for 'interaction_constraints', but 'x' \",\n \"has no column names to match them against.\"\n )\n }\n out <- match(idx, cnames) - 1L\n if (anyNA(out)) {\n stop(\n \"'interaction_constraints' contains column names not present in 'x': \",\n paste(utils::head(idx[which(is.na(out))]), collapse = \", \")\n )\n }\n return(out)\n } else if (inherits(idx, c(\"numeric\", \"integer\"))) {\n if (anyNA(idx)) {\n stop(\"'interaction_constraints' cannot contain NA values.\")\n }\n if (min(idx) < 1) {\n stop(\"Column indices for 'interaction_constraints' must follow base-1 indexing.\")\n }\n if (max(idx) > ncol(x)) {\n stop(\"'interaction_constraints' contains invalid column indices.\")\n }\n if (is.numeric(idx)) {\n if (any(idx != floor(idx))) {\n stop(\n \"'interaction_constraints' must contain only integer indices. Got non-integer: \",\n paste(utils::head(idx[which(idx != floor(idx))]), collapse = \", \")\n )\n }\n }\n return(idx - 1L)\n } else {\n stop(\n \"Elements in 'interaction_constraints' must be vectors of types \",\n \"'integer', 'numeric', or 'character'. Got: \",\n paste(class(idx), collapse = \", \")\n )\n }\n })\n }\n\n lst_args$dmatrix_args$data <- x\n return(lst_args)\n}\n\nprocess.eval.set <- function(eval_set, lst_args) {\n if (!NROW(eval_set)) {\n return(NULL)\n }\n nrows <- nrow(lst_args$dmatrix_args$data)\n is_classif <- hasName(lst_args$metadata, \"y_levels\")\n processed_y <- lst_args$dmatrix_args$label\n eval_set <- as.vector(eval_set)\n if (length(eval_set) == 1L) {\n\n eval_set <- as.numeric(eval_set)\n if (is.na(eval_set) || eval_set < 0 || eval_set >= 1) {\n stop(\"'eval_set' as a fraction must be a number between zero and one (non-inclusive).\")\n }\n if (eval_set == 0) {\n return(NULL)\n }\n nrow_eval <- as.integer(round(nrows * eval_set, 0))\n if (nrow_eval < 1) {\n warning(\n \"Desired 'eval_set' fraction amounts to zero observations.\",\n \" Will not create evaluation set.\"\n )\n return(NULL)\n }\n nrow_train <- nrows - nrow_eval\n if (nrow_train < 2L) {\n stop(\"Desired 'eval_set' fraction would leave less than 2 observations for training data.\")\n }\n if (is_classif && nrow_train < length(lst_args$metadata$y_levels)) {\n stop(\"Desired 'eval_set' fraction would not leave enough samples for each class of 'y'.\")\n }\n\n seed <- lst_args$params$seed\n if (!is.null(seed)) {\n set.seed(seed)\n }\n\n idx_shuffled <- sample(nrows, nrows, replace = FALSE)\n idx_eval <- idx_shuffled[seq(1L, nrow_eval)]\n idx_train <- idx_shuffled[seq(nrow_eval + 1L, nrows)]\n # Here we want the training set to include all of the classes of 'y' for classification\n # objectives. If that condition doesn't hold with the random sample, then it forcibly\n # makes a new random selection in such a way that the condition would always hold, by\n # first sampling one random example of 'y' for training and then choosing the evaluation\n # set from the remaining rows. The procedure here is quite inefficient, but there aren't\n # enough random-related functions in base R to be able to construct an efficient version.\n if (is_classif && length(unique(processed_y[idx_train])) < length(lst_args$metadata$y_levels)) {\n # These are defined in order to avoid NOTEs from CRAN checks\n # when using non-standard data.table evaluation with column names.\n idx <- NULL\n y <- NULL\n ranked_idx <- NULL\n chosen <- NULL\n\n dt <- data.table::data.table(y = processed_y, idx = seq(1L, nrows))[\n , .(\n ranked_idx = seq(1L, .N),\n chosen = rep(sample(.N, 1L), .N),\n idx\n )\n , by = y\n ]\n min_idx_train <- dt[ranked_idx == chosen, idx]\n rem_idx <- dt[ranked_idx != chosen, idx]\n if (length(rem_idx) == nrow_eval) {\n idx_train <- min_idx_train\n idx_eval <- rem_idx\n } else {\n rem_idx <- rem_idx[sample(length(rem_idx), length(rem_idx), replace = FALSE)]\n idx_eval <- rem_idx[seq(1L, nrow_eval)]\n idx_train <- c(min_idx_train, rem_idx[seq(nrow_eval + 1L, length(rem_idx))])\n }\n }\n\n } else {\n\n if (any(eval_set != floor(eval_set))) {\n stop(\"'eval_set' as indices must contain only integers.\")\n }\n eval_set <- as.integer(eval_set)\n idx_min <- min(eval_set)\n if (is.na(idx_min) || idx_min < 1L) {\n stop(\"'eval_set' contains invalid indices.\")\n }\n idx_max <- max(eval_set)\n if (is.na(idx_max) || idx_max > nrows) {\n stop(\"'eval_set' contains row indices beyond the size of the input data.\")\n }\n idx_train <- seq(1L, nrows)[-eval_set]\n if (is_classif && length(unique(processed_y[idx_train])) < length(lst_args$metadata$y_levels)) {\n warning(\"'eval_set' indices will leave some classes of 'y' outside of the training data.\")\n }\n idx_eval <- eval_set\n\n }\n\n # Note: slicing is done in the constructed DMatrix object instead of in the\n # original input, because objects from 'Matrix' might change class after\n # being sliced (e.g. 'dgRMatrix' turns into 'dgCMatrix').\n return(list(idx_train = idx_train, idx_eval = idx_eval))\n}\n\ncheck.early.stopping.rounds <- function(early_stopping_rounds, eval_set) {\n if (is.null(early_stopping_rounds)) {\n return(NULL)\n }\n if (is.null(eval_set)) {\n stop(\"'early_stopping_rounds' requires passing 'eval_set'.\")\n }\n if (NROW(early_stopping_rounds) != 1L) {\n stop(\"'early_stopping_rounds' must be NULL or an integer greater than zero.\")\n }\n early_stopping_rounds <- as.integer(early_stopping_rounds)\n if (is.na(early_stopping_rounds) || early_stopping_rounds <= 0L) {\n stop(\n \"'early_stopping_rounds' must be NULL or an integer greater than zero. Got: \",\n early_stopping_rounds\n )\n }\n return(early_stopping_rounds)\n}\n\n# nolint start: line_length_linter.\n#' Fit XGBoost Model\n#'\n#' @export\n#' @description\n#' Fits an XGBoost model (boosted decision tree ensemble) to given x/y data.\n#'\n#' See the tutorial [Introduction to Boosted Trees](https://xgboost.readthedocs.io/en/stable/tutorials/model.html)\n#' for a longer explanation of what XGBoost does, and the rest of the\n#' [XGBoost Tutorials](https://xgboost.readthedocs.io/en/latest/tutorials/index.html) for further\n#' explanations XGBoost's features and usage.\n#'\n#' This function is intended to provide a user-friendly interface for XGBoost that follows\n#' R's conventions for model fitting and predictions, but which doesn't expose all of the\n#' possible functionalities of the core XGBoost library.\n#'\n#' See [xgb.train()] for a more flexible low-level alternative which is similar across different\n#' language bindings of XGBoost and which exposes additional functionalities such as training on\n#' external memory data and learning-to-rank objectives.\n#'\n#' See also the [migration guide](https://xgboost.readthedocs.io/en/latest/R-package/migration_guide.html)\n#' if coming from a previous version of XGBoost in the 1.x series.\n#'\n#' By default, most of the parameters here have a value of `NULL`, which signals XGBoost to use its\n#' default value. Default values are automatically determined by the XGBoost core library, and are\n#' subject to change over XGBoost library versions. Some of them might differ according to the\n#' booster type (e.g. defaults for regularization are different for linear and tree-based boosters).\n#' See [xgb.params()] and the [online documentation](https://xgboost.readthedocs.io/en/latest/parameter.html)\n#' for more details about parameters - but note that some of the parameters are not supported in\n#' the `xgboost()` interface.\n#' @details\n#' For package authors using 'xgboost' as a dependency, it is highly recommended to use\n#' [xgb.train()] in package code instead of [xgboost()], since it has a more stable interface\n#' and performs fewer data conversions and copies along the way.\n#' @references\n#' - Chen, Tianqi, and Carlos Guestrin. \"Xgboost: A scalable tree boosting system.\"\n#' Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and\n#' data mining. 2016.\n#' - \\url{https://xgboost.readthedocs.io/en/stable/}\n#' @param x The features / covariates. Can be passed as:\n#' - A numeric or integer `matrix`.\n#' - A `data.frame`, in which all columns are one of the following types:\n#' - `numeric`\n#' - `integer`\n#' - `logical`\n#' - `factor`\n#'\n#' Columns of `factor` type will be assumed to be categorical, while other column types will\n#' be assumed to be numeric.\n#' - A sparse matrix from the `Matrix` package, either as `dgCMatrix` or `dgRMatrix` class.\n#'\n#' Note that categorical features are only supported for `data.frame` inputs, and are automatically\n#' determined based on their types. See [xgb.train()] with [xgb.DMatrix()] for more flexible\n#' variants that would allow something like categorical features on sparse matrices.\n#' @param y The response variable. Allowed values are:\n#' - A numeric or integer vector (for regression tasks).\n#' - A factor or character vector (for binary and multi-class classification tasks).\n#' - A logical (boolean) vector (for binary classification tasks).\n#' - A numeric or integer matrix or `data.frame` with numeric/integer columns\n#' (for multi-task regression tasks).\n#' - A `Surv` object from the 'survival' package (for survival tasks).\n#'\n#' If `objective` is `NULL`, the right task will be determined automatically based on\n#' the class of `y`.\n#'\n#' If `objective` is not `NULL`, it must match with the type of `y` - e.g. `factor` types of `y`\n#' can only be used with classification objectives and vice-versa.\n#'\n#' For binary classification, the last factor level of `y` will be used as the \"positive\"\n#' class - that is, the numbers from `predict` will reflect the probabilities of belonging to this\n#' class instead of to the first factor level. If `y` is a `logical` vector, then `TRUE` will be\n#' set as the last level.\n#' @param objective Optimization objective to minimize based on the supplied data, to be passed\n#' by name as a string / character (e.g. `reg:absoluteerror`). See the\n#' [Learning Task Parameters](https://xgboost.readthedocs.io/en/stable/parameter.html#learning-task-parameters)\n#' page and the [xgb.params()] documentation for more detailed information on allowed values.\n#'\n#' If `NULL` (the default), will be automatically determined from `y` according to the following\n#' logic:\n#' - If `y` is a factor with 2 levels, will use `binary:logistic`.\n#' - If `y` is a factor with more than 2 levels, will use `multi:softprob` (number of classes\n#' will be determined automatically, should not be passed under `params`).\n#' - If `y` is a `Surv` object from the `survival` package, will use `survival:aft` (note that\n#' the only types supported are left / right / interval censored).\n#' - Otherwise, will use `reg:squarederror`.\n#'\n#' If `objective` is not `NULL`, it must match with the type of `y` - e.g. `factor` types of `y`\n#' can only be used with classification objectives and vice-versa.\n#'\n#' Note that not all possible `objective` values supported by the core XGBoost library are allowed\n#' here - for example, objectives which are a variation of another but with a different default\n#' prediction type (e.g. `multi:softmax` vs. `multi:softprob`) are not allowed, and neither are\n#' ranking objectives, nor custom objectives at the moment.\n#'\n#' Supported values are:\n#' - `\"reg:squarederror\"`: regression with squared loss.\n#' - `\"reg:squaredlogerror\"`: regression with squared log loss \\eqn{\\frac{1}{2}[log(pred + 1) - log(label + 1)]^2}. All input labels are required to be greater than -1. Also, see metric `rmsle` for possible issue with this objective.\n#' - `\"reg:pseudohubererror\"`: regression with Pseudo Huber loss, a twice differentiable alternative to absolute loss.\n#' - `\"reg:absoluteerror\"`: Regression with L1 error. When tree model is used, leaf value is refreshed after tree construction. If used in distributed training, the leaf value is calculated as the mean value from all workers, which is not guaranteed to be optimal.\n#' - `\"reg:quantileerror\"`: Quantile loss, also known as \"pinball loss\". See later sections for its parameter and [Quantile Regression](https://xgboost.readthedocs.io/en/latest/python/examples/quantile_regression.html#sphx-glr-python-examples-quantile-regression-py) for a worked example.\n#' - `\"reg:expectileerror\"`: Expectile loss. See later sections for its parameter.\n#' - `\"binary:logistic\"`: logistic regression for binary classification, output probability\n#' - `\"binary:hinge\"`: hinge loss for binary classification. This makes predictions of 0 or 1, rather than producing probabilities.\n#' - `\"count:poisson\"`: Poisson regression for count data, output mean of Poisson distribution.\n#' `\"max_delta_step\"` is set to 0.7 by default in Poisson regression (used to safeguard optimization)\n#' - `\"survival:cox\"`: Cox regression for right censored survival time data (negative values are considered right censored).\n#'\n#' Note that predictions are returned on the hazard ratio scale (i.e., as HR = exp(marginal_prediction) in the proportional hazard function `h(t) = h0(t) * HR`).\n#' - `\"survival:aft\"`: Accelerated failure time model for censored survival time data.\n#' See [Survival Analysis with Accelerated Failure Time](https://xgboost.readthedocs.io/en/latest/tutorials/aft_survival_analysis.html) for details.\n#' - `\"multi:softprob\"`: multi-class classification throgh multinomial logistic likelihood.\n#' - `\"reg:gamma\"`: gamma regression with log-link. Output is a mean of gamma distribution. It might be useful, e.g., for modeling insurance claims severity, or for any outcome that might be [gamma-distributed](https://en.wikipedia.org/wiki/Gamma_distribution#Occurrence_and_applications).\n#' - `\"reg:tweedie\"`: Tweedie regression with log-link. It might be useful, e.g., for modeling total loss in insurance, or for any outcome that might be [Tweedie-distributed](https://en.wikipedia.org/wiki/Tweedie_distribution#Occurrence_and_applications).\n#'\n#' The following values are \\bold{NOT} supported by `xgboost`, but are supported by [xgb.train()]\n#' (see [xgb.params()] for details):\n#' - `\"reg:logistic\"`\n#' - `\"binary:logitraw\"`\n#' - `\"multi:softmax\"`\n#' - `\"rank:ndcg\"`\n#' - `\"rank:map\"`\n#' - `\"rank:pairwise\"`\n#' @param nrounds Number of boosting iterations / rounds.\n#'\n#' Note that the number of default boosting rounds here is not automatically tuned, and different\n#' problems will have vastly different optimal numbers of boosting rounds.\n#' @param weights Sample weights for each row in `x` and `y`. If `NULL` (the default), each row\n#' will have the same weight.\n#'\n#' If not `NULL`, should be passed as a numeric vector with length matching to the number of rows in `x`.\n#' @param verbosity Verbosity of printing messages. Valid values of 0 (silent), 1 (warning),\n#' 2 (info), and 3 (debug).\n#' @param monitor_training Whether to monitor objective optimization progress on the input data.\n#' Note that same 'x' and 'y' data are used for both model fitting and evaluation.\n#' @param eval_set Subset of the data to use as evaluation set. Can be passed as:\n#' - A vector of row indices (base-1 numeration) indicating the observations that are to be designed\n#' as evaluation data.\n#' - A number between zero and one indicating a random fraction of the input data to use as\n#' evaluation data. Note that the selection will be done uniformly at random, regardless of\n#' argument `weights`.\n#'\n#' If passed, this subset of the data will be excluded from the training procedure, and the\n#' evaluation metric(s) supplied under `eval_metric` will be calculated on this dataset after each\n#' boosting iteration (pass `verbosity>0` to have these metrics printed during training). If\n#' `eval_metric` is not passed, a default metric will be selected according to `objective`.\n#'\n#' If passing a fraction, in classification problems, the evaluation set will be chosen in such a\n#' way that at least one observation of each class will be kept in the training data.\n#'\n#' For more elaborate evaluation variants (e.g. custom metrics, multiple evaluation sets, etc.),\n#' one might want to use [xgb.train()] instead.\n#' @param early_stopping_rounds Number of boosting rounds after which training will be stopped\n#' if there is no improvement in performance (as measured by the last metric passed under\n#' `eval_metric`, or by the default metric for the objective if `eval_metric` is not passed) on the\n#' evaluation data from `eval_set`. Must pass `eval_set` in order to use this functionality.\n#'\n#' If `NULL`, early stopping will not be used.\n#' @param print_every_n When passing `verbosity>0` and either `monitor_training=TRUE` or `eval_set`,\n#' evaluation logs (metrics calculated on the training and/or evaluation data) will be printed every\n#' nth iteration according to the value passed here. The first and last iteration are always\n#' included regardless of this 'n'.\n#'\n#' Only has an effect when passing `verbosity>0`.\n#' @param nthreads Number of parallel threads to use. If passing zero, will use all CPU threads.\n#' @param seed Seed to use for random number generation. If passing `NULL`, will draw a random\n#' number using R's PRNG system to use as seed.\n#' @param monotone_constraints Optional monotonicity constraints for features.\n#'\n#' Can be passed either as a named list (when `x` has column names), or as a vector. If passed\n#' as a vector and `x` has column names, will try to match the elements by name.\n#'\n#' A value of `+1` for a given feature makes the model predictions / scores constrained to be\n#' a monotonically increasing function of that feature (that is, as the value of the feature\n#' increases, the model prediction cannot decrease), while a value of `-1` makes it a monotonically\n#' decreasing function. A value of zero imposes no constraint.\n#'\n#' The input for `monotone_constraints` can be a subset of the columns of `x` if named, in which\n#' case the columns that are not referred to in `monotone_constraints` will be assumed to have\n#' a value of zero (no constraint imposed on the model for those features).\n#'\n#' See the tutorial [Monotonic Constraints](https://xgboost.readthedocs.io/en/stable/tutorials/monotonic.html)\n#' for a more detailed explanation.\n#' @param interaction_constraints Constraints for interaction representing permitted interactions.\n#' The constraints must be specified in the form of a list of vectors referencing columns in the\n#' data, e.g. `list(c(1, 2), c(3, 4, 5))` (with these numbers being column indices, numeration\n#' starting at 1 - i.e. the first sublist references the first and second columns) or\n#' `list(c(\"Sepal.Length\", \"Sepal.Width\"), c(\"Petal.Length\", \"Petal.Width\"))` (references\n#' columns by names), where each vector is a group of indices of features that are allowed to\n#' interact with each other.\n#'\n#' See the tutorial [Feature Interaction Constraints](https://xgboost.readthedocs.io/en/stable/tutorials/feature_interaction_constraint.html)\n#' for more information.\n#' @param feature_weights Feature weights for column sampling.\n#'\n#' Can be passed either as a vector with length matching to columns of `x`, or as a named\n#' list (only if `x` has column names) with names matching to columns of 'x'. If it is a\n#' named vector, will try to match the entries to column names of `x` by name.\n#'\n#' If `NULL` (the default), all columns will have the same weight.\n#' @param base_margin Base margin used for boosting from existing model.\n#'\n#' If passing it, will start the gradient boosting procedure from the scores that are provided\n#' here - for example, one can pass the raw scores from a previous model, or some per-observation\n#' offset, or similar.\n#'\n#' Should be either a numeric vector or numeric matrix (for multi-class and multi-target objectives)\n#' with the same number of rows as `x` and number of columns corresponding to number of optimization\n#' targets, and should be in the untransformed scale (for example, for objective `binary:logistic`,\n#' it should have log-odds, not probabilities; and for objective `multi:softprob`, should have\n#' number of columns matching to number of classes in the data).\n#'\n#' Note that, if it contains more than one column, then columns will not be matched by name to\n#' the corresponding `y` - `base_margin` should have the same column order that the model will use\n#' (for example, for objective `multi:softprob`, columns of `base_margin` will be matched against\n#' `levels(y)` by their position, regardless of what `colnames(base_margin)` returns).\n#'\n#' If `NULL`, will start from zero, but note that for most objectives, an intercept is usually\n#' added (controllable through parameter `base_score` instead) when `base_margin` is not passed.\n#' @param min_split_loss (for Tree Booster) (default=0, alias: `gamma`)\n#' Minimum loss reduction required to make a further partition on a leaf node of the tree. The larger `min_split_loss` is, the more conservative the algorithm will be. Note that a tree where no splits were made might still contain a single terminal node with a non-zero score.\n#'\n#' range: \\eqn{[0, \\infty)}\n#' @param learning_rate (alias: `eta`)\n#' Step size shrinkage used in update to prevent overfitting. After each boosting step, we can directly get the weights of new features, and `learning_rate` shrinks the feature weights to make the boosting process more conservative.\n#' - range: \\eqn{[0,1]}\n#' - default value: 0.3 for tree-based boosters, 0.5 for linear booster.\n#' @param reg_lambda (alias: `lambda`)\n#' - For tree-based boosters:\n#' - L2 regularization term on weights. Increasing this value will make model more conservative.\n#' - default: 1\n#' - range: \\eqn{[0, \\infty]}\n#' - For linear booster:\n#' - L2 regularization term on weights. Increasing this value will make model more conservative. Normalised to number of training examples.\n#' - default: 0\n#' - range: \\eqn{[0, \\infty)}\n#' @param reg_alpha (alias: `reg_alpha`)\n#' - L1 regularization term on weights. Increasing this value will make model more conservative.\n#' - For the linear booster, it's normalised to number of training examples.\n#' - default: 0\n#' - range: \\eqn{[0, \\infty)}\n#' @param updater (for Linear Booster) (default= `\"shotgun\"`)\n#' Choice of algorithm to fit linear model\n#' - `\"shotgun\"`: Parallel coordinate descent algorithm based on shotgun algorithm. Uses 'hogwild' parallelism and therefore produces a nondeterministic solution on each run.\n#' - `\"coord_descent\"`: Ordinary coordinate descent algorithm. Also multithreaded but still produces a deterministic solution. When the `device` parameter is set to `\"cuda\"` or `\"gpu\"`, a GPU variant would be used.\n#' @inheritParams xgb.params\n#' @inheritParams xgb.train\n#' @return A model object, inheriting from both `xgboost` and `xgb.Booster`. Compared to the regular\n#' `xgb.Booster` model class produced by [xgb.train()], this `xgboost` class will have an\n#' additional attribute `metadata` containing information which is used for formatting prediction\n#' outputs, such as class names for classification problems.\n#'\n#' @examples\n#' data(mtcars)\n#'\n#' # Fit a small regression model on the mtcars data\n#' model_regression <- xgboost(mtcars[, -1], mtcars$mpg, nthreads = 1, nrounds = 3)\n#' predict(model_regression, mtcars, validate_features = TRUE)\n#'\n#' # Task objective is determined automatically according to the type of 'y'\n#' data(iris)\n#' model_classif <- xgboost(iris[, -5], iris$Species, nthreads = 1, nrounds = 5)\n#' predict(model_classif, iris[1:10,])\n#' predict(model_classif, iris[1:10,], type = \"class\")\n#'\n#' # Can nevertheless choose a non-default objective if needed\n#' model_poisson <- xgboost(\n#' mtcars[, -1], mtcars$mpg,\n#' objective = \"count:poisson\",\n#' nthreads = 1,\n#' nrounds = 3\n#' )\n#'\n#' # Can calculate evaluation metrics during boosting rounds\n#' data(ToothGrowth)\n#' xgboost(\n#' ToothGrowth[, c(\"len\", \"dose\")],\n#' ToothGrowth$supp,\n#' eval_metric = c(\"auc\", \"logloss\"),\n#' eval_set = 0.2,\n#' monitor_training = TRUE,\n#' verbosity = 1,\n#' nthreads = 1,\n#' nrounds = 3\n#' )\nxgboost <- function(\n x,\n y,\n objective = NULL,\n nrounds = 100L,\n max_depth = NULL,\n learning_rate = NULL,\n min_child_weight = NULL,\n min_split_loss = NULL,\n reg_lambda = NULL,\n weights = NULL,\n verbosity = if (is.null(eval_set)) 0L else 1L,\n monitor_training = verbosity > 0,\n eval_set = NULL,\n early_stopping_rounds = NULL,\n print_every_n = 1L,\n eval_metric = NULL,\n nthreads = parallel::detectCores(),\n seed = 0L,\n base_margin = NULL,\n monotone_constraints = NULL,\n interaction_constraints = NULL,\n reg_alpha = NULL,\n max_bin = NULL,\n max_leaves = NULL,\n booster = NULL,\n subsample = NULL,\n sampling_method = NULL,\n feature_weights = NULL,\n colsample_bytree = NULL,\n colsample_bylevel = NULL,\n colsample_bynode = NULL,\n tree_method = NULL,\n max_delta_step = NULL,\n scale_pos_weight = NULL,\n updater = NULL,\n grow_policy = NULL,\n num_parallel_tree = NULL,\n multi_strategy = NULL,\n base_score = NULL,\n seed_per_iteration = NULL,\n device = NULL,\n disable_default_eval_metric = NULL,\n use_rmm = NULL,\n max_cached_hist_node = NULL,\n max_cat_to_onehot = NULL,\n max_cat_threshold = NULL,\n sample_type = NULL,\n normalize_type = NULL,\n rate_drop = NULL,\n one_drop = NULL,\n skip_drop = NULL,\n feature_selector = NULL,\n top_k = NULL,\n tweedie_variance_power = NULL,\n huber_slope = NULL,\n quantile_alpha = NULL,\n expectile_alpha = NULL,\n aft_loss_distribution = NULL,\n ...\n) {\n# nolint end\n check.deprecation(deprecated_xgboost_params, match.call(), ...)\n params <- as.list(environment())\n params <- params[\n (names(params) %in% formalArgs(xgb.params))\n & !sapply(params, is.null)\n & !(names(params) %in% c( # these undergo additional processing here\n \"objective\", \"base_margin\", \"monotone_constraints\", \"interaction_constraints\"\n ))\n ]\n\n prescreen.objective(objective)\n use_qdm <- check.can.use.qdm(x, params, eval_set)\n lst_args <- process.y.margin.and.objective(y, base_margin, objective, params)\n lst_args <- process.row.weights(weights, lst_args)\n lst_args <- process.x.and.col.args(\n x,\n monotone_constraints,\n interaction_constraints,\n feature_weights,\n lst_args,\n use_qdm\n )\n eval_set <- process.eval.set(eval_set, lst_args)\n\n if (use_qdm && hasName(params, \"max_bin\")) {\n lst_args$dmatrix_args$max_bin <- params$max_bin\n }\n\n nthreads <- check.nthreads(nthreads)\n lst_args$dmatrix_args$nthread <- nthreads\n lst_args$params$nthread <- nthreads\n\n params <- c(lst_args$params, params)\n params$verbosity <- verbosity\n\n fn_dm <- if (use_qdm) xgb.QuantileDMatrix else xgb.DMatrix\n dm <- do.call(fn_dm, lst_args$dmatrix_args)\n if (!is.null(eval_set)) {\n dm_eval <- xgb.slice.DMatrix(dm, eval_set$idx_eval)\n dm <- xgb.slice.DMatrix(dm, eval_set$idx_train)\n }\n evals <- list()\n if (monitor_training) {\n evals <- list(train = dm)\n }\n if (!is.null(eval_set)) {\n evals <- c(evals, list(eval = dm_eval))\n }\n model <- xgb.train(\n params = params,\n data = dm,\n nrounds = nrounds,\n verbose = verbosity,\n print_every_n = print_every_n,\n evals = evals\n )\n attributes(model)$metadata <- lst_args$metadata\n attributes(model)$call <- match.call()\n class(model) <- c(\"xgboost\", class(model))\n return(model)\n}\n\n#' @title Compute predictions from XGBoost model on new data\n#' @description Predict values on data based on XGBoost model.\n#' @param object An XGBoost model object of class `xgboost`, as produced by function [xgboost()].\n#'\n#' Note that there is also a lower-level [predict.xgb.Booster()] method for models of class\n#' `xgb.Booster` as produced by [xgb.train()], which can also be used for `xgboost` class models as\n#' an alternative that performs fewer validations and post-processings.\n#' @param newdata Data on which to compute predictions from the model passed in `object`. Supported\n#' input classes are:\n#' - Data Frames (class `data.frame` from base R and subclasses like `data.table`).\n#' - Matrices (class `matrix` from base R).\n#' - Sparse matrices from package `Matrix`, either as class `dgRMatrix` (CSR) or `dgCMatrix` (CSC).\n#' - Sparse vectors from package `Matrix`, which will be interpreted as containing a single\n#' observation.\n#'\n#' In the case of data frames, if there are any categorical features, they should be of class\n#' `factor` and should have the same levels as the `factor` columns of the data from which the model\n#' was constructed. Any columns with type other than `factor` will be interpreted as numeric.\n#'\n#' If there are named columns and the model was fitted to data with named columns, they will be\n#' matched by name by default (see `validate_features`).\n#' @param type Type of prediction to make. Supported options are:\n#' - `\"response\"`: will output model predictions on the scale of the response variable (e.g.\n#' probabilities of belonging to the last class in the case of binary classification). Result will\n#' be either a numeric vector with length matching to rows in `newdata`, or a numeric matrix with\n#' shape `[nrows(newdata), nscores]` (for objectives that produce more than one score per\n#' observation such as multi-class classification or multi-quantile regression).\n#' - `\"raw\"`: will output the unprocessed boosting scores (e.g. log-odds in the case of objective\n#' `binary:logistic`). Same output shape and type as for `\"response\"`.\n#' - `\"class\"`: will output the class with the highest predicted probability, returned as a `factor`\n#' (only applicable to classification objectives) with length matching to rows in `newdata`.\n#' - `\"leaf\"`: will output the terminal node indices of each observation across each tree, as an\n#' integer matrix of shape `[nrows(newdata), ntrees]`, or as an integer array with an extra one or\n#' two dimensions, up to `[nrows(newdata), ntrees, nscores, n_parallel_trees]` for models that\n#' produce more than one score per tree and/or which have more than one parallel tree (e.g.\n#' random forests).\n#'\n#' Only applicable to tree-based boosters (not `gblinear`).\n#' - `\"contrib\"`: will produce per-feature contribution estimates towards the model score for a\n#' given observation, based on SHAP values. The contribution values are on the scale of\n#' untransformed margin (e.g., for binary classification, the values are log-odds deviations from\n#' the baseline).\n#'\n#' Output will be a numeric matrix with shape `[nrows, nfeatures+1]`, with the intercept being the\n#' last feature, or a numeric array with shape `[nrows, nscores, nfeatures+1]` if the model\n#' produces more than one score per observation.\n#' - `\"interaction\"`: similar to `\"contrib\"`, but computing SHAP values of contributions of\n#' interaction of each pair of features. Note that this operation might be rather expensive in\n#' terms of compute and memory.\n#'\n#' Since it quadratically depends on the number of features, it is recommended to perform\n#' selection of the most important features first.\n#'\n#' Output will be a numeric array of shape `[nrows, nfeatures+1, nfeatures+1]`, or shape\n#' `[nrows, nscores, nfeatures+1, nfeatures+1]` (for objectives that produce more than one score\n#' per observation).\n#' @param base_margin Base margin used for boosting from existing model (raw score that gets added to\n#' all observations independently of the trees in the model).\n#'\n#' If supplied, should be either a vector with length equal to the number of rows in `newdata`\n#' (for objectives which produces a single score per observation), or a matrix with number of\n#' rows matching to the number rows in `newdata` and number of columns matching to the number\n#' of scores estimated by the model (e.g. number of classes for multi-class classification).\n#' @param iteration_range Sequence of rounds/iterations from the model to use for prediction, specified by passing\n#' a two-dimensional vector with the start and end numbers in the sequence (same format as R's `seq` - i.e.\n#' base-1 indexing, and inclusive of both ends).\n#'\n#' For example, passing `c(1,20)` will predict using the first twenty iterations, while passing `c(1,1)` will\n#' predict using only the first one.\n#'\n#' If passing `NULL`, will either stop at the best iteration if the model used early stopping, or use all\n#' of the iterations (rounds) otherwise.\n#'\n#' If passing \"all\", will use all of the rounds regardless of whether the model had early stopping or not.\n#'\n#' Not applicable to `gblinear` booster.\n#' @param validate_features Validate that the feature names in the data match to the feature names\n#' in the column, and reorder them in the data otherwise.\n#'\n#' If passing `FALSE`, it is assumed that the feature names and types are the same,\n#' and come in the same order as in the training data.\n#'\n#' Be aware that this only applies to column names and not to factor levels in categorical columns.\n#'\n#' Note that this check might add some sizable latency to the predictions, so it's\n#' recommended to disable it for performance-sensitive applications.\n#' @param ... Not used.\n#' @return Either a numeric vector (for 1D outputs), numeric matrix (for 2D outputs), numeric array\n#' (for 3D and higher), or `factor` (for class predictions). See documentation for parameter `type`\n#' for details about what the output type and shape will be.\n#' @method predict xgboost\n#' @export\n#' @examples\n#' data(\"ToothGrowth\")\n#' y <- ToothGrowth$supp\n#' x <- ToothGrowth[, -2L]\n#' model <- xgboost(x, y, nthreads = 1L, nrounds = 3L, max_depth = 2L)\n#' pred_prob <- predict(model, x[1:5, ], type = \"response\")\n#' pred_raw <- predict(model, x[1:5, ], type = \"raw\")\n#' pred_class <- predict(model, x[1:5, ], type = \"class\")\n#'\n#' # Relationships between these\n#' manual_probs <- 1 / (1 + exp(-pred_raw))\n#' manual_class <- ifelse(manual_probs < 0.5, levels(y)[1], levels(y)[2])\n#'\n#' # They should match up to numerical precision\n#' round(pred_prob, 6) == round(manual_probs, 6)\n#' pred_class == manual_class\npredict.xgboost <- function(\n object,\n newdata,\n type = \"response\",\n base_margin = NULL,\n iteration_range = NULL,\n validate_features = TRUE,\n ...\n) {\n if (inherits(newdata, \"xgb.DMatrix\")) {\n stop(\n \"Predictions on 'xgb.DMatrix' objects are not supported with 'xgboost' class.\",\n \" Try 'xgb.train' or 'predict.xgb.Booster'.\"\n )\n }\n\n outputmargin <- FALSE\n predleaf <- FALSE\n predcontrib <- FALSE\n predinteraction <- FALSE\n pred_class <- FALSE\n strict_shape <- FALSE\n allowed_types <- c(\n \"response\",\n \"raw\",\n \"class\",\n \"leaf\",\n \"contrib\",\n \"interaction\"\n )\n type <- head(type, 1L)\n if (!is.character(type) || !(type %in% allowed_types)) {\n stop(\"'type' must be one of: \", paste(allowed_types, collapse = \", \"))\n }\n\n if (type != \"response\") {\n switch(\n type,\n \"raw\" = {\n outputmargin <- TRUE\n }, \"class\" = {\n if (is.null(attributes(object)$metadata$y_levels)) {\n stop(\"Prediction type 'class' is only for classification objectives.\")\n }\n pred_class <- TRUE\n outputmargin <- TRUE\n }, \"leaf\" = {\n predleaf <- TRUE\n strict_shape <- TRUE # required for 3D and 4D outputs\n }, \"contrib\" = {\n predcontrib <- TRUE\n }, \"interaction\" = {\n predinteraction <- TRUE\n }\n )\n }\n out <- predict.xgb.Booster(\n object,\n newdata,\n outputmargin = outputmargin,\n predleaf = predleaf,\n predcontrib = predcontrib,\n predinteraction = predinteraction,\n iterationrange = iteration_range,\n strict_shape = strict_shape,\n validate_features = validate_features,\n base_margin = base_margin\n )\n\n if (strict_shape) {\n # Should only end up here for leaf predictions\n out_dims <- dim(out)\n dims_remove <- integer()\n if (out_dims[3L] == 1L) {\n dims_remove <- c(dims_remove, -3L)\n }\n if (length(out_dims) >= 4L && out_dims[4L] == 1L) {\n dims_remove <- c(dims_remove, -4L)\n }\n if (length(dims_remove)) {\n new_dimnames <- dimnames(out)[dims_remove]\n dim(out) <- out_dims[dims_remove]\n dimnames(out) <- new_dimnames\n }\n }\n\n if (pred_class) {\n\n if (is.null(dim(out))) {\n out <- as.integer(out >= 0) + 1L\n } else {\n out <- max.col(out, ties.method = \"first\")\n }\n attr_out <- attributes(out)\n attr_out$class <- \"factor\"\n attr_out$levels <- attributes(object)$metadata$y_levels\n attributes(out) <- attr_out\n\n } else if (NCOL(out) > 1L || (strict_shape && length(dim(out)) >= 3L)) {\n\n names_use <- NULL\n if (NROW(attributes(object)$metadata$y_levels) > 2L) {\n names_use <- attributes(object)$metadata$y_levels\n } else if (NROW(attributes(object)$metadata$y_names)) {\n names_use <- attributes(object)$metadata$y_names\n } else if (NROW(attributes(object)$params$quantile_alpha) > 1L) {\n names_use <- paste0(\"q\", attributes(object)$params$quantile_alpha)\n if (anyDuplicated(names_use)) {\n warning(\"Cannot add quantile names to output due to clashes in their character conversions\")\n names_use <- NULL\n }\n } else if (NROW(attributes(object)$params$expectile_alpha) > 1L) {\n names_use <- paste0(\"e\", attributes(object)$params$expectile_alpha)\n if (anyDuplicated(names_use)) {\n warning(\"Cannot add expectile names to output due to clashes in their character conversions\")\n names_use <- NULL\n }\n }\n if (NROW(names_use)) {\n dimnames_out <- dimnames(out)\n dim_with_names <- if (type == \"leaf\") 3L else 2L\n dimnames_out[[dim_with_names]] <- names_use\n .Call(XGSetArrayDimNamesInplace_R, out, dimnames_out)\n }\n\n }\n\n return(out)\n}\n\n#' @title Print info from XGBoost model\n#' @description Prints basic properties of an XGBoost model object.\n#' @param x An XGBoost model object of class `xgboost`, as produced by function [xgboost()].\n#' @param ... Not used.\n#' @return Same object `x`, after printing its info.\n#' @method print xgboost\n#' @export\nprint.xgboost <- function(x, ...) {\n cat(\"XGBoost model object\\n\")\n cat(\"Call:\\n \")\n print(attributes(x)$call)\n cat(\"Objective: \", attributes(x)$params$objective, \"\\n\", sep = \"\")\n cat(\"Number of iterations: \", xgb.get.num.boosted.rounds(x), \"\\n\", sep = \"\")\n cat(\"Number of features: \", xgb.num_feature(x), \"\\n\", sep = \"\")\n\n printable_head <- function(v) {\n v_sub <- utils::head(v, 5L)\n return(\n sprintf(\n \"%s%s\",\n paste(v_sub, collapse = \", \"),\n ifelse(length(v_sub) < length(v), \", ...\", \"\")\n )\n )\n }\n\n if (NROW(attributes(x)$metadata$y_levels)) {\n cat(\n \"Classes: \",\n printable_head(attributes(x)$metadata$y_levels),\n \"\\n\",\n sep = \"\"\n )\n } else if (NROW(attributes(x)$params$quantile_alpha)) {\n cat(\n \"Prediction quantile\",\n ifelse(length(attributes(x)$params$quantile_alpha) > 1L, \"s\", \"\"),\n \": \",\n printable_head(attributes(x)$params$quantile_alpha),\n \"\\n\",\n sep = \"\"\n )\n } else if (NROW(attributes(x)$params$expectile_alpha)) {\n cat(\n \"Prediction expectile\",\n ifelse(length(attributes(x)$params$expectile_alpha) > 1L, \"s\", \"\"),\n \": \",\n printable_head(attributes(x)$params$expectile_alpha),\n \"\\n\",\n sep = \"\"\n )\n } else if (NROW(attributes(x)$metadata$y_names)) {\n cat(\n \"Prediction targets: \",\n printable_head(attributes(x)$metadata$y_names),\n \"\\n\",\n sep = \"\"\n )\n } else if (attributes(x)$metadata$n_targets > 1L) {\n cat(\n \"Number of predition targets: \",\n attributes(x)$metadata$n_targets,\n \"\\n\",\n sep = \"\"\n )\n }\n\n return(x)\n}\n\n\n#' Training part from Mushroom Data Set\n#'\n#' This data set is originally from the Mushroom data set,\n#' UCI Machine Learning Repository.\n#'\n#' It includes the following fields:\n#' - `label`: The label for each record.\n#' - `data`: A sparse Matrix of 'dgCMatrix' class with 126 columns.\n#'\n#' @references\n#' \n#'\n#' Bache, K. & Lichman, M. (2013). UCI Machine Learning Repository\n#' . Irvine, CA: University of California,\n#' School of Information and Computer Science.\n#'\n#' @docType data\n#' @keywords datasets\n#' @name agaricus.train\n#' @usage data(agaricus.train)\n#' @format A list containing a label vector, and a dgCMatrix object with 6513\n#' rows and 127 variables\nNULL\n\n#' Test part from Mushroom Data Set\n#'\n#' This data set is originally from the Mushroom data set,\n#' UCI Machine Learning Repository.\n#'\n#' It includes the following fields:\n#' - `label`: The label for each record.\n#' - `data`: A sparse Matrix of 'dgCMatrix' class with 126 columns.\n#'\n#' @references\n#' \n#'\n#' Bache, K. & Lichman, M. (2013). UCI Machine Learning Repository\n#' . Irvine, CA: University of California,\n#' School of Information and Computer Science.\n#'\n#' @docType data\n#' @keywords datasets\n#' @name agaricus.test\n#' @usage data(agaricus.test)\n#' @format A list containing a label vector, and a dgCMatrix object with 1611\n#' rows and 126 variables\nNULL\n\n# Various imports\n#' @importClassesFrom Matrix dgCMatrix dgRMatrix CsparseMatrix\n#' @importFrom Matrix sparse.model.matrix\n#' @importFrom data.table data.table\n#' @importFrom data.table is.data.table\n#' @importFrom data.table as.data.table\n#' @importFrom data.table :=\n#' @importFrom data.table rbindlist\n#' @importFrom data.table setkey\n#' @importFrom data.table setkeyv\n#' @importFrom data.table setnames\n#' @importFrom jsonlite fromJSON\n#' @importFrom jsonlite toJSON\n#' @importFrom methods new\n#' @importFrom utils object.size str tail\n#' @importFrom stats coef\n#' @importFrom stats predict\n#' @importFrom stats median\n#' @importFrom stats sd\n#' @importFrom stats variable.names\n#' @importFrom utils head\n#' @importFrom utils hasName\n#' @importFrom graphics barplot\n#' @importFrom graphics lines\n#' @importFrom graphics points\n#' @importFrom graphics grid\n#' @importFrom graphics par\n#' @importFrom graphics title\n#' @importFrom grDevices rgb\n#'\n#' @import methods\n#' @useDynLib xgboost, .registration = TRUE\nNULL\n" }, { "path": "R-package/README.md", "content": "XGBoost R Package\n=================\n\n[![CRAN Status Badge](http://www.r-pkg.org/badges/version/xgboost)](https://cran.r-project.org/web/packages/xgboost)\n[![CRAN Downloads](http://cranlogs.r-pkg.org/badges/xgboost)](https://cran.rstudio.com/web/packages/xgboost/index.html)\n[![Documentation Status](https://readthedocs.org/projects/xgboost/badge/?version=latest)](https://xgboost.readthedocs.org/en/latest/R-package/index.html)\n\nResources\n---------\n* [XGBoost R Package Online Documentation](https://xgboost.readthedocs.org/en/stable/R-package/index.html)\n - Check this out for detailed documents, examples and tutorials.\n\nInstallation\n------------\n\nWe are [on CRAN](https://cran.r-project.org/web/packages/xgboost/index.html) now. For stable/pre-compiled(for Windows and OS X) version, please install from CRAN:\n\n```r\ninstall.packages('xgboost')\n```\n\nFor more detailed installation instructions, please see [here](https://xgboost.readthedocs.io/en/stable/install.html).\n\nDevelopment\n-----------\n\n* See the [R Package section](https://xgboost.readthedocs.io/en/latest/contrib/coding_guide.html#r-coding-guideline) of the contributors guide.\n" }, { "path": "R-package/bootstrap.R", "content": "## Script used to bootstrap R-universe build.\n\n## Execute git commands to initialize git submodules\nsystem(\"git submodule init\")\nsystem(\"git submodule update\")\n\n## core\nfile.copy(\"../src\", \"./src/\", recursive = TRUE)\nfile.copy(\"../include\", \"./src/\", recursive = TRUE)\nfile.copy(\"../amalgamation\", \"./src/\", recursive = TRUE)\n\n## dmlc-core\ndir.create(\"./src/dmlc-core\")\nfile.copy(\"../dmlc-core/include\", \"./src/dmlc-core/\", recursive = TRUE)\nfile.copy(\"../dmlc-core/src\", \"./src/dmlc-core/\", recursive = TRUE)\n\npkgroot <- function(path) {\n ## read the file from path, replace the PKGROOT=../../ with PKGROOT=.\n lines <- readLines(path)\n lines <- gsub(\"PKGROOT=../../\", \"PKGROOT=.\", lines, fixed = TRUE)\n writeLines(lines, path)\n}\n\n## makefile and license\nfile.copy(\"../LICENSE\", \"./LICENSE\")\npkgroot(\"./src/Makevars.in\")\npkgroot(\"./src/Makevars.win.in\")\n\n## misc\npath <- file.path(\"remove_warning_suppression_pragma.sh\")\nfile.remove(path)\npath <- file.path(\"CMakeLists.txt\")\nfile.remove(path)\n\n## remove the directory recursively ./tests/helper_scripts\nunlink(\"tests/helper_scripts\", recursive = TRUE)\n" }, { "path": "R-package/cleanup", "content": "#!/bin/sh\n\nrm -f src/Makevars\n" }, { "path": "R-package/configure", "content": "#! /bin/sh\n# Guess values for system-dependent variables and create Makefiles.\n# Generated by GNU Autoconf 2.71 for xgboost 3.3.0.\n#\n#\n# Copyright (C) 1992-1996, 1998-2017, 2020-2021 Free Software Foundation,\n# Inc.\n#\n#\n# This configure script is free software; the Free Software Foundation\n# gives unlimited permission to copy, distribute and modify it.\n## -------------------- ##\n## M4sh Initialization. ##\n## -------------------- ##\n\n# Be more Bourne compatible\nDUALCASE=1; export DUALCASE # for MKS sh\nas_nop=:\nif test ${ZSH_VERSION+y} && (emulate sh) >/dev/null 2>&1\nthen :\n emulate sh\n NULLCMD=:\n # Pre-4.2 versions of Zsh do word splitting on ${1+\"$@\"}, which\n # is contrary to our usage. Disable this feature.\n alias -g '${1+\"$@\"}'='\"$@\"'\n setopt NO_GLOB_SUBST\nelse $as_nop\n case `(set -o) 2>/dev/null` in #(\n *posix*) :\n set -o posix ;; #(\n *) :\n ;;\nesac\nfi\n\n\n\n# Reset variables that may have inherited troublesome values from\n# the environment.\n\n# IFS needs to be set, to space, tab, and newline, in precisely that order.\n# (If _AS_PATH_WALK were called with IFS unset, it would have the\n# side effect of setting IFS to empty, thus disabling word splitting.)\n# Quoting is to prevent editors from complaining about space-tab.\nas_nl='\n'\nexport as_nl\nIFS=\" \"\"\t$as_nl\"\n\nPS1='$ '\nPS2='> '\nPS4='+ '\n\n# Ensure predictable behavior from utilities with locale-dependent output.\nLC_ALL=C\nexport LC_ALL\nLANGUAGE=C\nexport LANGUAGE\n\n# We cannot yet rely on \"unset\" to work, but we need these variables\n# to be unset--not just set to an empty or harmless value--now, to\n# avoid bugs in old shells (e.g. pre-3.0 UWIN ksh). This construct\n# also avoids known problems related to \"unset\" and subshell syntax\n# in other old shells (e.g. bash 2.01 and pdksh 5.2.14).\nfor as_var in BASH_ENV ENV MAIL MAILPATH CDPATH\ndo eval test \\${$as_var+y} \\\n && ( (unset $as_var) || exit 1) >/dev/null 2>&1 && unset $as_var || :\ndone\n\n# Ensure that fds 0, 1, and 2 are open.\nif (exec 3>&0) 2>/dev/null; then :; else exec 0&1) 2>/dev/null; then :; else exec 1>/dev/null; fi\nif (exec 3>&2) ; then :; else exec 2>/dev/null; fi\n\n# The user is always right.\nif ${PATH_SEPARATOR+false} :; then\n PATH_SEPARATOR=:\n (PATH='/bin;/bin'; FPATH=$PATH; sh -c :) >/dev/null 2>&1 && {\n (PATH='/bin:/bin'; FPATH=$PATH; sh -c :) >/dev/null 2>&1 ||\n PATH_SEPARATOR=';'\n }\nfi\n\n\n# Find who we are. Look in the path if we contain no directory separator.\nas_myself=\ncase $0 in #((\n *[\\\\/]* ) as_myself=$0 ;;\n *) as_save_IFS=$IFS; IFS=$PATH_SEPARATOR\nfor as_dir in $PATH\ndo\n IFS=$as_save_IFS\n case $as_dir in #(((\n '') as_dir=./ ;;\n */) ;;\n *) as_dir=$as_dir/ ;;\n esac\n test -r \"$as_dir$0\" && as_myself=$as_dir$0 && break\n done\nIFS=$as_save_IFS\n\n ;;\nesac\n# We did not find ourselves, most probably we were run as `sh COMMAND'\n# in which case we are not to be found in the path.\nif test \"x$as_myself\" = x; then\n as_myself=$0\nfi\nif test ! -f \"$as_myself\"; then\n printf \"%s\\n\" \"$as_myself: error: cannot find myself; rerun with an absolute file name\" >&2\n exit 1\nfi\n\n\n# Use a proper internal environment variable to ensure we don't fall\n # into an infinite loop, continuously re-executing ourselves.\n if test x\"${_as_can_reexec}\" != xno && test \"x$CONFIG_SHELL\" != x; then\n _as_can_reexec=no; export _as_can_reexec;\n # We cannot yet assume a decent shell, so we have to provide a\n# neutralization value for shells without unset; and this also\n# works around shells that cannot unset nonexistent variables.\n# Preserve -v and -x to the replacement shell.\nBASH_ENV=/dev/null\nENV=/dev/null\n(unset BASH_ENV) >/dev/null 2>&1 && unset BASH_ENV ENV\ncase $- in # ((((\n *v*x* | *x*v* ) as_opts=-vx ;;\n *v* ) as_opts=-v ;;\n *x* ) as_opts=-x ;;\n * ) as_opts= ;;\nesac\nexec $CONFIG_SHELL $as_opts \"$as_myself\" ${1+\"$@\"}\n# Admittedly, this is quite paranoid, since all the known shells bail\n# out after a failed `exec'.\nprintf \"%s\\n\" \"$0: could not re-execute with $CONFIG_SHELL\" >&2\nexit 255\n fi\n # We don't want this to propagate to other subprocesses.\n { _as_can_reexec=; unset _as_can_reexec;}\nif test \"x$CONFIG_SHELL\" = x; then\n as_bourne_compatible=\"as_nop=:\nif test \\${ZSH_VERSION+y} && (emulate sh) >/dev/null 2>&1\nthen :\n emulate sh\n NULLCMD=:\n # Pre-4.2 versions of Zsh do word splitting on \\${1+\\\"\\$@\\\"}, which\n # is contrary to our usage. Disable this feature.\n alias -g '\\${1+\\\"\\$@\\\"}'='\\\"\\$@\\\"'\n setopt NO_GLOB_SUBST\nelse \\$as_nop\n case \\`(set -o) 2>/dev/null\\` in #(\n *posix*) :\n set -o posix ;; #(\n *) :\n ;;\nesac\nfi\n\"\n as_required=\"as_fn_return () { (exit \\$1); }\nas_fn_success () { as_fn_return 0; }\nas_fn_failure () { as_fn_return 1; }\nas_fn_ret_success () { return 0; }\nas_fn_ret_failure () { return 1; }\n\nexitcode=0\nas_fn_success || { exitcode=1; echo as_fn_success failed.; }\nas_fn_failure && { exitcode=1; echo as_fn_failure succeeded.; }\nas_fn_ret_success || { exitcode=1; echo as_fn_ret_success failed.; }\nas_fn_ret_failure && { exitcode=1; echo as_fn_ret_failure succeeded.; }\nif ( set x; as_fn_ret_success y && test x = \\\"\\$1\\\" )\nthen :\n\nelse \\$as_nop\n exitcode=1; echo positional parameters were not saved.\nfi\ntest x\\$exitcode = x0 || exit 1\nblah=\\$(echo \\$(echo blah))\ntest x\\\"\\$blah\\\" = xblah || exit 1\ntest -x / || exit 1\"\n as_suggested=\" as_lineno_1=\";as_suggested=$as_suggested$LINENO;as_suggested=$as_suggested\" as_lineno_1a=\\$LINENO\n as_lineno_2=\";as_suggested=$as_suggested$LINENO;as_suggested=$as_suggested\" as_lineno_2a=\\$LINENO\n eval 'test \\\"x\\$as_lineno_1'\\$as_run'\\\" != \\\"x\\$as_lineno_2'\\$as_run'\\\" &&\n test \\\"x\\`expr \\$as_lineno_1'\\$as_run' + 1\\`\\\" = \\\"x\\$as_lineno_2'\\$as_run'\\\"' || exit 1\"\n if (eval \"$as_required\") 2>/dev/null\nthen :\n as_have_required=yes\nelse $as_nop\n as_have_required=no\nfi\n if test x$as_have_required = xyes && (eval \"$as_suggested\") 2>/dev/null\nthen :\n\nelse $as_nop\n as_save_IFS=$IFS; IFS=$PATH_SEPARATOR\nas_found=false\nfor as_dir in /bin$PATH_SEPARATOR/usr/bin$PATH_SEPARATOR$PATH\ndo\n IFS=$as_save_IFS\n case $as_dir in #(((\n '') as_dir=./ ;;\n */) ;;\n *) as_dir=$as_dir/ ;;\n esac\n as_found=:\n case $as_dir in #(\n\t /*)\n\t for as_base in sh bash ksh sh5; do\n\t # Try only shells that exist, to save several forks.\n\t as_shell=$as_dir$as_base\n\t if { test -f \"$as_shell\" || test -f \"$as_shell.exe\"; } &&\n\t\t as_run=a \"$as_shell\" -c \"$as_bourne_compatible\"\"$as_required\" 2>/dev/null\nthen :\n CONFIG_SHELL=$as_shell as_have_required=yes\n\t\t if as_run=a \"$as_shell\" -c \"$as_bourne_compatible\"\"$as_suggested\" 2>/dev/null\nthen :\n break 2\nfi\nfi\n\t done;;\n esac\n as_found=false\ndone\nIFS=$as_save_IFS\nif $as_found\nthen :\n\nelse $as_nop\n if { test -f \"$SHELL\" || test -f \"$SHELL.exe\"; } &&\n\t as_run=a \"$SHELL\" -c \"$as_bourne_compatible\"\"$as_required\" 2>/dev/null\nthen :\n CONFIG_SHELL=$SHELL as_have_required=yes\nfi\nfi\n\n\n if test \"x$CONFIG_SHELL\" != x\nthen :\n export CONFIG_SHELL\n # We cannot yet assume a decent shell, so we have to provide a\n# neutralization value for shells without unset; and this also\n# works around shells that cannot unset nonexistent variables.\n# Preserve -v and -x to the replacement shell.\nBASH_ENV=/dev/null\nENV=/dev/null\n(unset BASH_ENV) >/dev/null 2>&1 && unset BASH_ENV ENV\ncase $- in # ((((\n *v*x* | *x*v* ) as_opts=-vx ;;\n *v* ) as_opts=-v ;;\n *x* ) as_opts=-x ;;\n * ) as_opts= ;;\nesac\nexec $CONFIG_SHELL $as_opts \"$as_myself\" ${1+\"$@\"}\n# Admittedly, this is quite paranoid, since all the known shells bail\n# out after a failed `exec'.\nprintf \"%s\\n\" \"$0: could not re-execute with $CONFIG_SHELL\" >&2\nexit 255\nfi\n\n if test x$as_have_required = xno\nthen :\n printf \"%s\\n\" \"$0: This script requires a shell more modern than all\"\n printf \"%s\\n\" \"$0: the shells that I found on your system.\"\n if test ${ZSH_VERSION+y} ; then\n printf \"%s\\n\" \"$0: In particular, zsh $ZSH_VERSION has bugs and should\"\n printf \"%s\\n\" \"$0: be upgraded to zsh 4.3.4 or later.\"\n else\n printf \"%s\\n\" \"$0: Please tell bug-autoconf@gnu.org about your system,\n$0: including any error possibly output before this\n$0: message. Then install a modern shell, or manually run\n$0: the script under such a shell if you do have one.\"\n fi\n exit 1\nfi\nfi\nfi\nSHELL=${CONFIG_SHELL-/bin/sh}\nexport SHELL\n# Unset more variables known to interfere with behavior of common tools.\nCLICOLOR_FORCE= GREP_OPTIONS=\nunset CLICOLOR_FORCE GREP_OPTIONS\n\n## --------------------- ##\n## M4sh Shell Functions. ##\n## --------------------- ##\n# as_fn_unset VAR\n# ---------------\n# Portably unset VAR.\nas_fn_unset ()\n{\n { eval $1=; unset $1;}\n}\nas_unset=as_fn_unset\n\n\n# as_fn_set_status STATUS\n# -----------------------\n# Set $? to STATUS, without forking.\nas_fn_set_status ()\n{\n return $1\n} # as_fn_set_status\n\n# as_fn_exit STATUS\n# -----------------\n# Exit the shell with STATUS, even in a \"trap 0\" or \"set -e\" context.\nas_fn_exit ()\n{\n set +e\n as_fn_set_status $1\n exit $1\n} # as_fn_exit\n# as_fn_nop\n# ---------\n# Do nothing but, unlike \":\", preserve the value of $?.\nas_fn_nop ()\n{\n return $?\n}\nas_nop=as_fn_nop\n\n# as_fn_mkdir_p\n# -------------\n# Create \"$as_dir\" as a directory, including parents if necessary.\nas_fn_mkdir_p ()\n{\n\n case $as_dir in #(\n -*) as_dir=./$as_dir;;\n esac\n test -d \"$as_dir\" || eval $as_mkdir_p || {\n as_dirs=\n while :; do\n case $as_dir in #(\n *\\'*) as_qdir=`printf \"%s\\n\" \"$as_dir\" | sed \"s/'/'\\\\\\\\\\\\\\\\''/g\"`;; #'(\n *) as_qdir=$as_dir;;\n esac\n as_dirs=\"'$as_qdir' $as_dirs\"\n as_dir=`$as_dirname -- \"$as_dir\" ||\n$as_expr X\"$as_dir\" : 'X\\(.*[^/]\\)//*[^/][^/]*/*$' \\| \\\n\t X\"$as_dir\" : 'X\\(//\\)[^/]' \\| \\\n\t X\"$as_dir\" : 'X\\(//\\)$' \\| \\\n\t X\"$as_dir\" : 'X\\(/\\)' \\| . 2>/dev/null ||\nprintf \"%s\\n\" X\"$as_dir\" |\n sed '/^X\\(.*[^/]\\)\\/\\/*[^/][^/]*\\/*$/{\n\t s//\\1/\n\t q\n\t }\n\t /^X\\(\\/\\/\\)[^/].*/{\n\t s//\\1/\n\t q\n\t }\n\t /^X\\(\\/\\/\\)$/{\n\t s//\\1/\n\t q\n\t }\n\t /^X\\(\\/\\).*/{\n\t s//\\1/\n\t q\n\t }\n\t s/.*/./; q'`\n test -d \"$as_dir\" && break\n done\n test -z \"$as_dirs\" || eval \"mkdir $as_dirs\"\n } || test -d \"$as_dir\" || as_fn_error $? \"cannot create directory $as_dir\"\n\n\n} # as_fn_mkdir_p\n\n# as_fn_executable_p FILE\n# -----------------------\n# Test if FILE is an executable regular file.\nas_fn_executable_p ()\n{\n test -f \"$1\" && test -x \"$1\"\n} # as_fn_executable_p\n# as_fn_append VAR VALUE\n# ----------------------\n# Append the text in VALUE to the end of the definition contained in VAR. Take\n# advantage of any shell optimizations that allow amortized linear growth over\n# repeated appends, instead of the typical quadratic growth present in naive\n# implementations.\nif (eval \"as_var=1; as_var+=2; test x\\$as_var = x12\") 2>/dev/null\nthen :\n eval 'as_fn_append ()\n {\n eval $1+=\\$2\n }'\nelse $as_nop\n as_fn_append ()\n {\n eval $1=\\$$1\\$2\n }\nfi # as_fn_append\n\n# as_fn_arith ARG...\n# ------------------\n# Perform arithmetic evaluation on the ARGs, and store the result in the\n# global $as_val. Take advantage of shells that can avoid forks. The arguments\n# must be portable across $(()) and expr.\nif (eval \"test \\$(( 1 + 1 )) = 2\") 2>/dev/null\nthen :\n eval 'as_fn_arith ()\n {\n as_val=$(( $* ))\n }'\nelse $as_nop\n as_fn_arith ()\n {\n as_val=`expr \"$@\" || test $? -eq 1`\n }\nfi # as_fn_arith\n\n# as_fn_nop\n# ---------\n# Do nothing but, unlike \":\", preserve the value of $?.\nas_fn_nop ()\n{\n return $?\n}\nas_nop=as_fn_nop\n\n# as_fn_error STATUS ERROR [LINENO LOG_FD]\n# ----------------------------------------\n# Output \"`basename $0`: error: ERROR\" to stderr. If LINENO and LOG_FD are\n# provided, also output the error to LOG_FD, referencing LINENO. Then exit the\n# script with STATUS, using 1 if that was 0.\nas_fn_error ()\n{\n as_status=$1; test $as_status -eq 0 && as_status=1\n if test \"$4\"; then\n as_lineno=${as_lineno-\"$3\"} as_lineno_stack=as_lineno_stack=$as_lineno_stack\n printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: error: $2\" >&$4\n fi\n printf \"%s\\n\" \"$as_me: error: $2\" >&2\n as_fn_exit $as_status\n} # as_fn_error\n\nif expr a : '\\(a\\)' >/dev/null 2>&1 &&\n test \"X`expr 00001 : '.*\\(...\\)'`\" = X001; then\n as_expr=expr\nelse\n as_expr=false\nfi\n\nif (basename -- /) >/dev/null 2>&1 && test \"X`basename -- / 2>&1`\" = \"X/\"; then\n as_basename=basename\nelse\n as_basename=false\nfi\n\nif (as_dir=`dirname -- /` && test \"X$as_dir\" = X/) >/dev/null 2>&1; then\n as_dirname=dirname\nelse\n as_dirname=false\nfi\n\nas_me=`$as_basename -- \"$0\" ||\n$as_expr X/\"$0\" : '.*/\\([^/][^/]*\\)/*$' \\| \\\n\t X\"$0\" : 'X\\(//\\)$' \\| \\\n\t X\"$0\" : 'X\\(/\\)' \\| . 2>/dev/null ||\nprintf \"%s\\n\" X/\"$0\" |\n sed '/^.*\\/\\([^/][^/]*\\)\\/*$/{\n\t s//\\1/\n\t q\n\t }\n\t /^X\\/\\(\\/\\/\\)$/{\n\t s//\\1/\n\t q\n\t }\n\t /^X\\/\\(\\/\\).*/{\n\t s//\\1/\n\t q\n\t }\n\t s/.*/./; q'`\n\n# Avoid depending upon Character Ranges.\nas_cr_letters='abcdefghijklmnopqrstuvwxyz'\nas_cr_LETTERS='ABCDEFGHIJKLMNOPQRSTUVWXYZ'\nas_cr_Letters=$as_cr_letters$as_cr_LETTERS\nas_cr_digits='0123456789'\nas_cr_alnum=$as_cr_Letters$as_cr_digits\n\n\n as_lineno_1=$LINENO as_lineno_1a=$LINENO\n as_lineno_2=$LINENO as_lineno_2a=$LINENO\n eval 'test \"x$as_lineno_1'$as_run'\" != \"x$as_lineno_2'$as_run'\" &&\n test \"x`expr $as_lineno_1'$as_run' + 1`\" = \"x$as_lineno_2'$as_run'\"' || {\n # Blame Lee E. McMahon (1931-1989) for sed's syntax. :-)\n sed -n '\n p\n /[$]LINENO/=\n ' <$as_myself |\n sed '\n s/[$]LINENO.*/&-/\n t lineno\n b\n :lineno\n N\n :loop\n s/[$]LINENO\\([^'$as_cr_alnum'_].*\\n\\)\\(.*\\)/\\2\\1\\2/\n t loop\n s/-\\n.*//\n ' >$as_me.lineno &&\n chmod +x \"$as_me.lineno\" ||\n { printf \"%s\\n\" \"$as_me: error: cannot create $as_me.lineno; rerun with a POSIX shell\" >&2; as_fn_exit 1; }\n\n # If we had to re-execute with $CONFIG_SHELL, we're ensured to have\n # already done that, so ensure we don't try to do so again and fall\n # in an infinite loop. This has already happened in practice.\n _as_can_reexec=no; export _as_can_reexec\n # Don't try to exec as it changes $[0], causing all sort of problems\n # (the dirname of $[0] is not the place where we might find the\n # original and so on. Autoconf is especially sensitive to this).\n . \"./$as_me.lineno\"\n # Exit status is that of the last command.\n exit\n}\n\n\n# Determine whether it's possible to make 'echo' print without a newline.\n# These variables are no longer used directly by Autoconf, but are AC_SUBSTed\n# for compatibility with existing Makefiles.\nECHO_C= ECHO_N= ECHO_T=\ncase `echo -n x` in #(((((\n-n*)\n case `echo 'xy\\c'` in\n *c*) ECHO_T='\t';;\t# ECHO_T is single tab character.\n xy) ECHO_C='\\c';;\n *) echo `echo ksh88 bug on AIX 6.1` > /dev/null\n ECHO_T='\t';;\n esac;;\n*)\n ECHO_N='-n';;\nesac\n\n# For backward compatibility with old third-party macros, we provide\n# the shell variables $as_echo and $as_echo_n. New code should use\n# AS_ECHO([\"message\"]) and AS_ECHO_N([\"message\"]), respectively.\nas_echo='printf %s\\n'\nas_echo_n='printf %s'\n\n\nrm -f conf$$ conf$$.exe conf$$.file\nif test -d conf$$.dir; then\n rm -f conf$$.dir/conf$$.file\nelse\n rm -f conf$$.dir\n mkdir conf$$.dir 2>/dev/null\nfi\nif (echo >conf$$.file) 2>/dev/null; then\n if ln -s conf$$.file conf$$ 2>/dev/null; then\n as_ln_s='ln -s'\n # ... but there are two gotchas:\n # 1) On MSYS, both `ln -s file dir' and `ln file dir' fail.\n # 2) DJGPP < 2.04 has no symlinks; `ln -s' creates a wrapper executable.\n # In both cases, we have to default to `cp -pR'.\n ln -s conf$$.file conf$$.dir 2>/dev/null && test ! -f conf$$.exe ||\n as_ln_s='cp -pR'\n elif ln conf$$.file conf$$ 2>/dev/null; then\n as_ln_s=ln\n else\n as_ln_s='cp -pR'\n fi\nelse\n as_ln_s='cp -pR'\nfi\nrm -f conf$$ conf$$.exe conf$$.dir/conf$$.file conf$$.file\nrmdir conf$$.dir 2>/dev/null\n\nif mkdir -p . 2>/dev/null; then\n as_mkdir_p='mkdir -p \"$as_dir\"'\nelse\n test -d ./-p && rmdir ./-p\n as_mkdir_p=false\nfi\n\nas_test_x='test -x'\nas_executable_p=as_fn_executable_p\n\n# Sed expression to map a string onto a valid CPP name.\nas_tr_cpp=\"eval sed 'y%*$as_cr_letters%P$as_cr_LETTERS%;s%[^_$as_cr_alnum]%_%g'\"\n\n# Sed expression to map a string onto a valid variable name.\nas_tr_sh=\"eval sed 'y%*+%pp%;s%[^_$as_cr_alnum]%_%g'\"\n\n\ntest -n \"$DJDIR\" || exec 7<&0 &1\n\n# Name of the host.\n# hostname on some systems (SVR3.2, old GNU/Linux) returns a bogus exit status,\n# so uname gets run too.\nac_hostname=`(hostname || uname -n) 2>/dev/null | sed 1q`\n\n#\n# Initializations.\n#\nac_default_prefix=/usr/local\nac_clean_files=\nac_config_libobj_dir=.\nLIBOBJS=\ncross_compiling=no\nsubdirs=\nMFLAGS=\nMAKEFLAGS=\n\n# Identity of this package.\nPACKAGE_NAME='xgboost'\nPACKAGE_TARNAME='xgboost'\nPACKAGE_VERSION='3.3.0'\nPACKAGE_STRING='xgboost 3.3.0'\nPACKAGE_BUGREPORT=''\nPACKAGE_URL=''\n\n# Factoring default headers for most tests.\nac_includes_default=\"\\\n#include \n#ifdef HAVE_STDIO_H\n# include \n#endif\n#ifdef HAVE_STDLIB_H\n# include \n#endif\n#ifdef HAVE_STRING_H\n# include \n#endif\n#ifdef HAVE_INTTYPES_H\n# include \n#endif\n#ifdef HAVE_STDINT_H\n# include \n#endif\n#ifdef HAVE_STRINGS_H\n# include \n#endif\n#ifdef HAVE_SYS_TYPES_H\n# include \n#endif\n#ifdef HAVE_SYS_STAT_H\n# include \n#endif\n#ifdef HAVE_UNISTD_H\n# include \n#endif\"\n\nac_header_cxx_list=\nac_subst_vars='LTLIBOBJS\nLIBOBJS\nXGBOOST_MM_PREFETCH_PRESENT\nXGBOOST_BUILTIN_PREFETCH_PRESENT\nBACKTRACE_LIB\nDMLC_DEFS\nENDIAN_FLAG\nOPENMP_LIB\nOPENMP_CXXFLAGS\nUSE_LITTLE_ENDIAN\nOBJEXT\nEXEEXT\nac_ct_CXX\nCPPFLAGS\nLDFLAGS\nCXXFLAGS\nCXX\ntarget_alias\nhost_alias\nbuild_alias\nLIBS\nECHO_T\nECHO_N\nECHO_C\nDEFS\nmandir\nlocaledir\nlibdir\npsdir\npdfdir\ndvidir\nhtmldir\ninfodir\ndocdir\noldincludedir\nincludedir\nrunstatedir\nlocalstatedir\nsharedstatedir\nsysconfdir\ndatadir\ndatarootdir\nlibexecdir\nsbindir\nbindir\nprogram_transform_name\nprefix\nexec_prefix\nPACKAGE_URL\nPACKAGE_BUGREPORT\nPACKAGE_STRING\nPACKAGE_VERSION\nPACKAGE_TARNAME\nPACKAGE_NAME\nPATH_SEPARATOR\nSHELL'\nac_subst_files=''\nac_user_opts='\nenable_option_checking\n'\n ac_precious_vars='build_alias\nhost_alias\ntarget_alias\nCXX\nCXXFLAGS\nLDFLAGS\nLIBS\nCPPFLAGS\nCCC\nUSE_LITTLE_ENDIAN'\n\n\n# Initialize some variables set by options.\nac_init_help=\nac_init_version=false\nac_unrecognized_opts=\nac_unrecognized_sep=\n# The variables have the same names as the options, with\n# dashes changed to underlines.\ncache_file=/dev/null\nexec_prefix=NONE\nno_create=\nno_recursion=\nprefix=NONE\nprogram_prefix=NONE\nprogram_suffix=NONE\nprogram_transform_name=s,x,x,\nsilent=\nsite=\nsrcdir=\nverbose=\nx_includes=NONE\nx_libraries=NONE\n\n# Installation directory options.\n# These are left unexpanded so users can \"make install exec_prefix=/foo\"\n# and all the variables that are supposed to be based on exec_prefix\n# by default will actually change.\n# Use braces instead of parens because sh, perl, etc. also accept them.\n# (The list follows the same order as the GNU Coding Standards.)\nbindir='${exec_prefix}/bin'\nsbindir='${exec_prefix}/sbin'\nlibexecdir='${exec_prefix}/libexec'\ndatarootdir='${prefix}/share'\ndatadir='${datarootdir}'\nsysconfdir='${prefix}/etc'\nsharedstatedir='${prefix}/com'\nlocalstatedir='${prefix}/var'\nrunstatedir='${localstatedir}/run'\nincludedir='${prefix}/include'\noldincludedir='/usr/include'\ndocdir='${datarootdir}/doc/${PACKAGE_TARNAME}'\ninfodir='${datarootdir}/info'\nhtmldir='${docdir}'\ndvidir='${docdir}'\npdfdir='${docdir}'\npsdir='${docdir}'\nlibdir='${exec_prefix}/lib'\nlocaledir='${datarootdir}/locale'\nmandir='${datarootdir}/man'\n\nac_prev=\nac_dashdash=\nfor ac_option\ndo\n # If the previous option needs an argument, assign it.\n if test -n \"$ac_prev\"; then\n eval $ac_prev=\\$ac_option\n ac_prev=\n continue\n fi\n\n case $ac_option in\n *=?*) ac_optarg=`expr \"X$ac_option\" : '[^=]*=\\(.*\\)'` ;;\n *=) ac_optarg= ;;\n *) ac_optarg=yes ;;\n esac\n\n case $ac_dashdash$ac_option in\n --)\n ac_dashdash=yes ;;\n\n -bindir | --bindir | --bindi | --bind | --bin | --bi)\n ac_prev=bindir ;;\n -bindir=* | --bindir=* | --bindi=* | --bind=* | --bin=* | --bi=*)\n bindir=$ac_optarg ;;\n\n -build | --build | --buil | --bui | --bu)\n ac_prev=build_alias ;;\n -build=* | --build=* | --buil=* | --bui=* | --bu=*)\n build_alias=$ac_optarg ;;\n\n -cache-file | --cache-file | --cache-fil | --cache-fi \\\n | --cache-f | --cache- | --cache | --cach | --cac | --ca | --c)\n ac_prev=cache_file ;;\n -cache-file=* | --cache-file=* | --cache-fil=* | --cache-fi=* \\\n | --cache-f=* | --cache-=* | --cache=* | --cach=* | --cac=* | --ca=* | --c=*)\n cache_file=$ac_optarg ;;\n\n --config-cache | -C)\n cache_file=config.cache ;;\n\n -datadir | --datadir | --datadi | --datad)\n ac_prev=datadir ;;\n -datadir=* | --datadir=* | --datadi=* | --datad=*)\n datadir=$ac_optarg ;;\n\n -datarootdir | --datarootdir | --datarootdi | --datarootd | --dataroot \\\n | --dataroo | --dataro | --datar)\n ac_prev=datarootdir ;;\n -datarootdir=* | --datarootdir=* | --datarootdi=* | --datarootd=* \\\n | --dataroot=* | --dataroo=* | --dataro=* | --datar=*)\n datarootdir=$ac_optarg ;;\n\n -disable-* | --disable-*)\n ac_useropt=`expr \"x$ac_option\" : 'x-*disable-\\(.*\\)'`\n # Reject names that are not valid shell variable names.\n expr \"x$ac_useropt\" : \".*[^-+._$as_cr_alnum]\" >/dev/null &&\n as_fn_error $? \"invalid feature name: \\`$ac_useropt'\"\n ac_useropt_orig=$ac_useropt\n ac_useropt=`printf \"%s\\n\" \"$ac_useropt\" | sed 's/[-+.]/_/g'`\n case $ac_user_opts in\n *\"\n\"enable_$ac_useropt\"\n\"*) ;;\n *) ac_unrecognized_opts=\"$ac_unrecognized_opts$ac_unrecognized_sep--disable-$ac_useropt_orig\"\n\t ac_unrecognized_sep=', ';;\n esac\n eval enable_$ac_useropt=no ;;\n\n -docdir | --docdir | --docdi | --doc | --do)\n ac_prev=docdir ;;\n -docdir=* | --docdir=* | --docdi=* | --doc=* | --do=*)\n docdir=$ac_optarg ;;\n\n -dvidir | --dvidir | --dvidi | --dvid | --dvi | --dv)\n ac_prev=dvidir ;;\n -dvidir=* | --dvidir=* | --dvidi=* | --dvid=* | --dvi=* | --dv=*)\n dvidir=$ac_optarg ;;\n\n -enable-* | --enable-*)\n ac_useropt=`expr \"x$ac_option\" : 'x-*enable-\\([^=]*\\)'`\n # Reject names that are not valid shell variable names.\n expr \"x$ac_useropt\" : \".*[^-+._$as_cr_alnum]\" >/dev/null &&\n as_fn_error $? \"invalid feature name: \\`$ac_useropt'\"\n ac_useropt_orig=$ac_useropt\n ac_useropt=`printf \"%s\\n\" \"$ac_useropt\" | sed 's/[-+.]/_/g'`\n case $ac_user_opts in\n *\"\n\"enable_$ac_useropt\"\n\"*) ;;\n *) ac_unrecognized_opts=\"$ac_unrecognized_opts$ac_unrecognized_sep--enable-$ac_useropt_orig\"\n\t ac_unrecognized_sep=', ';;\n esac\n eval enable_$ac_useropt=\\$ac_optarg ;;\n\n -exec-prefix | --exec_prefix | --exec-prefix | --exec-prefi \\\n | --exec-pref | --exec-pre | --exec-pr | --exec-p | --exec- \\\n | --exec | --exe | --ex)\n ac_prev=exec_prefix ;;\n -exec-prefix=* | --exec_prefix=* | --exec-prefix=* | --exec-prefi=* \\\n | --exec-pref=* | --exec-pre=* | --exec-pr=* | --exec-p=* | --exec-=* \\\n | --exec=* | --exe=* | --ex=*)\n exec_prefix=$ac_optarg ;;\n\n -gas | --gas | --ga | --g)\n # Obsolete; use --with-gas.\n with_gas=yes ;;\n\n -help | --help | --hel | --he | -h)\n ac_init_help=long ;;\n -help=r* | --help=r* | --hel=r* | --he=r* | -hr*)\n ac_init_help=recursive ;;\n -help=s* | --help=s* | --hel=s* | --he=s* | -hs*)\n ac_init_help=short ;;\n\n -host | --host | --hos | --ho)\n ac_prev=host_alias ;;\n -host=* | --host=* | --hos=* | --ho=*)\n host_alias=$ac_optarg ;;\n\n -htmldir | --htmldir | --htmldi | --htmld | --html | --htm | --ht)\n ac_prev=htmldir ;;\n -htmldir=* | --htmldir=* | --htmldi=* | --htmld=* | --html=* | --htm=* \\\n | --ht=*)\n htmldir=$ac_optarg ;;\n\n -includedir | --includedir | --includedi | --included | --include \\\n | --includ | --inclu | --incl | --inc)\n ac_prev=includedir ;;\n -includedir=* | --includedir=* | --includedi=* | --included=* | --include=* \\\n | --includ=* | --inclu=* | --incl=* | --inc=*)\n includedir=$ac_optarg ;;\n\n -infodir | --infodir | --infodi | --infod | --info | --inf)\n ac_prev=infodir ;;\n -infodir=* | --infodir=* | --infodi=* | --infod=* | --info=* | --inf=*)\n infodir=$ac_optarg ;;\n\n -libdir | --libdir | --libdi | --libd)\n ac_prev=libdir ;;\n -libdir=* | --libdir=* | --libdi=* | --libd=*)\n libdir=$ac_optarg ;;\n\n -libexecdir | --libexecdir | --libexecdi | --libexecd | --libexec \\\n | --libexe | --libex | --libe)\n ac_prev=libexecdir ;;\n -libexecdir=* | --libexecdir=* | --libexecdi=* | --libexecd=* | --libexec=* \\\n | --libexe=* | --libex=* | --libe=*)\n libexecdir=$ac_optarg ;;\n\n -localedir | --localedir | --localedi | --localed | --locale)\n ac_prev=localedir ;;\n -localedir=* | --localedir=* | --localedi=* | --localed=* | --locale=*)\n localedir=$ac_optarg ;;\n\n -localstatedir | --localstatedir | --localstatedi | --localstated \\\n | --localstate | --localstat | --localsta | --localst | --locals)\n ac_prev=localstatedir ;;\n -localstatedir=* | --localstatedir=* | --localstatedi=* | --localstated=* \\\n | --localstate=* | --localstat=* | --localsta=* | --localst=* | --locals=*)\n localstatedir=$ac_optarg ;;\n\n -mandir | --mandir | --mandi | --mand | --man | --ma | --m)\n ac_prev=mandir ;;\n -mandir=* | --mandir=* | --mandi=* | --mand=* | --man=* | --ma=* | --m=*)\n mandir=$ac_optarg ;;\n\n -nfp | --nfp | --nf)\n # Obsolete; use --without-fp.\n with_fp=no ;;\n\n -no-create | --no-create | --no-creat | --no-crea | --no-cre \\\n | --no-cr | --no-c | -n)\n no_create=yes ;;\n\n -no-recursion | --no-recursion | --no-recursio | --no-recursi \\\n | --no-recurs | --no-recur | --no-recu | --no-rec | --no-re | --no-r)\n no_recursion=yes ;;\n\n -oldincludedir | --oldincludedir | --oldincludedi | --oldincluded \\\n | --oldinclude | --oldinclud | --oldinclu | --oldincl | --oldinc \\\n | --oldin | --oldi | --old | --ol | --o)\n ac_prev=oldincludedir ;;\n -oldincludedir=* | --oldincludedir=* | --oldincludedi=* | --oldincluded=* \\\n | --oldinclude=* | --oldinclud=* | --oldinclu=* | --oldincl=* | --oldinc=* \\\n | --oldin=* | --oldi=* | --old=* | --ol=* | --o=*)\n oldincludedir=$ac_optarg ;;\n\n -prefix | --prefix | --prefi | --pref | --pre | --pr | --p)\n ac_prev=prefix ;;\n -prefix=* | --prefix=* | --prefi=* | --pref=* | --pre=* | --pr=* | --p=*)\n prefix=$ac_optarg ;;\n\n -program-prefix | --program-prefix | --program-prefi | --program-pref \\\n | --program-pre | --program-pr | --program-p)\n ac_prev=program_prefix ;;\n -program-prefix=* | --program-prefix=* | --program-prefi=* \\\n | --program-pref=* | --program-pre=* | --program-pr=* | --program-p=*)\n program_prefix=$ac_optarg ;;\n\n -program-suffix | --program-suffix | --program-suffi | --program-suff \\\n | --program-suf | --program-su | --program-s)\n ac_prev=program_suffix ;;\n -program-suffix=* | --program-suffix=* | --program-suffi=* \\\n | --program-suff=* | --program-suf=* | --program-su=* | --program-s=*)\n program_suffix=$ac_optarg ;;\n\n -program-transform-name | --program-transform-name \\\n | --program-transform-nam | --program-transform-na \\\n | --program-transform-n | --program-transform- \\\n | --program-transform | --program-transfor \\\n | --program-transfo | --program-transf \\\n | --program-trans | --program-tran \\\n | --progr-tra | --program-tr | --program-t)\n ac_prev=program_transform_name ;;\n -program-transform-name=* | --program-transform-name=* \\\n | --program-transform-nam=* | --program-transform-na=* \\\n | --program-transform-n=* | --program-transform-=* \\\n | --program-transform=* | --program-transfor=* \\\n | --program-transfo=* | --program-transf=* \\\n | --program-trans=* | --program-tran=* \\\n | --progr-tra=* | --program-tr=* | --program-t=*)\n program_transform_name=$ac_optarg ;;\n\n -pdfdir | --pdfdir | --pdfdi | --pdfd | --pdf | --pd)\n ac_prev=pdfdir ;;\n -pdfdir=* | --pdfdir=* | --pdfdi=* | --pdfd=* | --pdf=* | --pd=*)\n pdfdir=$ac_optarg ;;\n\n -psdir | --psdir | --psdi | --psd | --ps)\n ac_prev=psdir ;;\n -psdir=* | --psdir=* | --psdi=* | --psd=* | --ps=*)\n psdir=$ac_optarg ;;\n\n -q | -quiet | --quiet | --quie | --qui | --qu | --q \\\n | -silent | --silent | --silen | --sile | --sil)\n silent=yes ;;\n\n -runstatedir | --runstatedir | --runstatedi | --runstated \\\n | --runstate | --runstat | --runsta | --runst | --runs \\\n | --run | --ru | --r)\n ac_prev=runstatedir ;;\n -runstatedir=* | --runstatedir=* | --runstatedi=* | --runstated=* \\\n | --runstate=* | --runstat=* | --runsta=* | --runst=* | --runs=* \\\n | --run=* | --ru=* | --r=*)\n runstatedir=$ac_optarg ;;\n\n -sbindir | --sbindir | --sbindi | --sbind | --sbin | --sbi | --sb)\n ac_prev=sbindir ;;\n -sbindir=* | --sbindir=* | --sbindi=* | --sbind=* | --sbin=* \\\n | --sbi=* | --sb=*)\n sbindir=$ac_optarg ;;\n\n -sharedstatedir | --sharedstatedir | --sharedstatedi \\\n | --sharedstated | --sharedstate | --sharedstat | --sharedsta \\\n | --sharedst | --shareds | --shared | --share | --shar \\\n | --sha | --sh)\n ac_prev=sharedstatedir ;;\n -sharedstatedir=* | --sharedstatedir=* | --sharedstatedi=* \\\n | --sharedstated=* | --sharedstate=* | --sharedstat=* | --sharedsta=* \\\n | --sharedst=* | --shareds=* | --shared=* | --share=* | --shar=* \\\n | --sha=* | --sh=*)\n sharedstatedir=$ac_optarg ;;\n\n -site | --site | --sit)\n ac_prev=site ;;\n -site=* | --site=* | --sit=*)\n site=$ac_optarg ;;\n\n -srcdir | --srcdir | --srcdi | --srcd | --src | --sr)\n ac_prev=srcdir ;;\n -srcdir=* | --srcdir=* | --srcdi=* | --srcd=* | --src=* | --sr=*)\n srcdir=$ac_optarg ;;\n\n -sysconfdir | --sysconfdir | --sysconfdi | --sysconfd | --sysconf \\\n | --syscon | --sysco | --sysc | --sys | --sy)\n ac_prev=sysconfdir ;;\n -sysconfdir=* | --sysconfdir=* | --sysconfdi=* | --sysconfd=* | --sysconf=* \\\n | --syscon=* | --sysco=* | --sysc=* | --sys=* | --sy=*)\n sysconfdir=$ac_optarg ;;\n\n -target | --target | --targe | --targ | --tar | --ta | --t)\n ac_prev=target_alias ;;\n -target=* | --target=* | --targe=* | --targ=* | --tar=* | --ta=* | --t=*)\n target_alias=$ac_optarg ;;\n\n -v | -verbose | --verbose | --verbos | --verbo | --verb)\n verbose=yes ;;\n\n -version | --version | --versio | --versi | --vers | -V)\n ac_init_version=: ;;\n\n -with-* | --with-*)\n ac_useropt=`expr \"x$ac_option\" : 'x-*with-\\([^=]*\\)'`\n # Reject names that are not valid shell variable names.\n expr \"x$ac_useropt\" : \".*[^-+._$as_cr_alnum]\" >/dev/null &&\n as_fn_error $? \"invalid package name: \\`$ac_useropt'\"\n ac_useropt_orig=$ac_useropt\n ac_useropt=`printf \"%s\\n\" \"$ac_useropt\" | sed 's/[-+.]/_/g'`\n case $ac_user_opts in\n *\"\n\"with_$ac_useropt\"\n\"*) ;;\n *) ac_unrecognized_opts=\"$ac_unrecognized_opts$ac_unrecognized_sep--with-$ac_useropt_orig\"\n\t ac_unrecognized_sep=', ';;\n esac\n eval with_$ac_useropt=\\$ac_optarg ;;\n\n -without-* | --without-*)\n ac_useropt=`expr \"x$ac_option\" : 'x-*without-\\(.*\\)'`\n # Reject names that are not valid shell variable names.\n expr \"x$ac_useropt\" : \".*[^-+._$as_cr_alnum]\" >/dev/null &&\n as_fn_error $? \"invalid package name: \\`$ac_useropt'\"\n ac_useropt_orig=$ac_useropt\n ac_useropt=`printf \"%s\\n\" \"$ac_useropt\" | sed 's/[-+.]/_/g'`\n case $ac_user_opts in\n *\"\n\"with_$ac_useropt\"\n\"*) ;;\n *) ac_unrecognized_opts=\"$ac_unrecognized_opts$ac_unrecognized_sep--without-$ac_useropt_orig\"\n\t ac_unrecognized_sep=', ';;\n esac\n eval with_$ac_useropt=no ;;\n\n --x)\n # Obsolete; use --with-x.\n with_x=yes ;;\n\n -x-includes | --x-includes | --x-include | --x-includ | --x-inclu \\\n | --x-incl | --x-inc | --x-in | --x-i)\n ac_prev=x_includes ;;\n -x-includes=* | --x-includes=* | --x-include=* | --x-includ=* | --x-inclu=* \\\n | --x-incl=* | --x-inc=* | --x-in=* | --x-i=*)\n x_includes=$ac_optarg ;;\n\n -x-libraries | --x-libraries | --x-librarie | --x-librari \\\n | --x-librar | --x-libra | --x-libr | --x-lib | --x-li | --x-l)\n ac_prev=x_libraries ;;\n -x-libraries=* | --x-libraries=* | --x-librarie=* | --x-librari=* \\\n | --x-librar=* | --x-libra=* | --x-libr=* | --x-lib=* | --x-li=* | --x-l=*)\n x_libraries=$ac_optarg ;;\n\n -*) as_fn_error $? \"unrecognized option: \\`$ac_option'\nTry \\`$0 --help' for more information\"\n ;;\n\n *=*)\n ac_envvar=`expr \"x$ac_option\" : 'x\\([^=]*\\)='`\n # Reject names that are not valid shell variable names.\n case $ac_envvar in #(\n '' | [0-9]* | *[!_$as_cr_alnum]* )\n as_fn_error $? \"invalid variable name: \\`$ac_envvar'\" ;;\n esac\n eval $ac_envvar=\\$ac_optarg\n export $ac_envvar ;;\n\n *)\n # FIXME: should be removed in autoconf 3.0.\n printf \"%s\\n\" \"$as_me: WARNING: you should use --build, --host, --target\" >&2\n expr \"x$ac_option\" : \".*[^-._$as_cr_alnum]\" >/dev/null &&\n printf \"%s\\n\" \"$as_me: WARNING: invalid host type: $ac_option\" >&2\n : \"${build_alias=$ac_option} ${host_alias=$ac_option} ${target_alias=$ac_option}\"\n ;;\n\n esac\ndone\n\nif test -n \"$ac_prev\"; then\n ac_option=--`echo $ac_prev | sed 's/_/-/g'`\n as_fn_error $? \"missing argument to $ac_option\"\nfi\n\nif test -n \"$ac_unrecognized_opts\"; then\n case $enable_option_checking in\n no) ;;\n fatal) as_fn_error $? \"unrecognized options: $ac_unrecognized_opts\" ;;\n *) printf \"%s\\n\" \"$as_me: WARNING: unrecognized options: $ac_unrecognized_opts\" >&2 ;;\n esac\nfi\n\n# Check all directory arguments for consistency.\nfor ac_var in\texec_prefix prefix bindir sbindir libexecdir datarootdir \\\n\t\tdatadir sysconfdir sharedstatedir localstatedir includedir \\\n\t\toldincludedir docdir infodir htmldir dvidir pdfdir psdir \\\n\t\tlibdir localedir mandir runstatedir\ndo\n eval ac_val=\\$$ac_var\n # Remove trailing slashes.\n case $ac_val in\n */ )\n ac_val=`expr \"X$ac_val\" : 'X\\(.*[^/]\\)' \\| \"X$ac_val\" : 'X\\(.*\\)'`\n eval $ac_var=\\$ac_val;;\n esac\n # Be sure to have absolute directory names.\n case $ac_val in\n [\\\\/$]* | ?:[\\\\/]* ) continue;;\n NONE | '' ) case $ac_var in *prefix ) continue;; esac;;\n esac\n as_fn_error $? \"expected an absolute directory name for --$ac_var: $ac_val\"\ndone\n\n# There might be people who depend on the old broken behavior: `$host'\n# used to hold the argument of --host etc.\n# FIXME: To remove some day.\nbuild=$build_alias\nhost=$host_alias\ntarget=$target_alias\n\n# FIXME: To remove some day.\nif test \"x$host_alias\" != x; then\n if test \"x$build_alias\" = x; then\n cross_compiling=maybe\n elif test \"x$build_alias\" != \"x$host_alias\"; then\n cross_compiling=yes\n fi\nfi\n\nac_tool_prefix=\ntest -n \"$host_alias\" && ac_tool_prefix=$host_alias-\n\ntest \"$silent\" = yes && exec 6>/dev/null\n\n\nac_pwd=`pwd` && test -n \"$ac_pwd\" &&\nac_ls_di=`ls -di .` &&\nac_pwd_ls_di=`cd \"$ac_pwd\" && ls -di .` ||\n as_fn_error $? \"working directory cannot be determined\"\ntest \"X$ac_ls_di\" = \"X$ac_pwd_ls_di\" ||\n as_fn_error $? \"pwd does not report name of working directory\"\n\n\n# Find the source files, if location was not specified.\nif test -z \"$srcdir\"; then\n ac_srcdir_defaulted=yes\n # Try the directory containing this script, then the parent directory.\n ac_confdir=`$as_dirname -- \"$as_myself\" ||\n$as_expr X\"$as_myself\" : 'X\\(.*[^/]\\)//*[^/][^/]*/*$' \\| \\\n\t X\"$as_myself\" : 'X\\(//\\)[^/]' \\| \\\n\t X\"$as_myself\" : 'X\\(//\\)$' \\| \\\n\t X\"$as_myself\" : 'X\\(/\\)' \\| . 2>/dev/null ||\nprintf \"%s\\n\" X\"$as_myself\" |\n sed '/^X\\(.*[^/]\\)\\/\\/*[^/][^/]*\\/*$/{\n\t s//\\1/\n\t q\n\t }\n\t /^X\\(\\/\\/\\)[^/].*/{\n\t s//\\1/\n\t q\n\t }\n\t /^X\\(\\/\\/\\)$/{\n\t s//\\1/\n\t q\n\t }\n\t /^X\\(\\/\\).*/{\n\t s//\\1/\n\t q\n\t }\n\t s/.*/./; q'`\n srcdir=$ac_confdir\n if test ! -r \"$srcdir/$ac_unique_file\"; then\n srcdir=..\n fi\nelse\n ac_srcdir_defaulted=no\nfi\nif test ! -r \"$srcdir/$ac_unique_file\"; then\n test \"$ac_srcdir_defaulted\" = yes && srcdir=\"$ac_confdir or ..\"\n as_fn_error $? \"cannot find sources ($ac_unique_file) in $srcdir\"\nfi\nac_msg=\"sources are in $srcdir, but \\`cd $srcdir' does not work\"\nac_abs_confdir=`(\n\tcd \"$srcdir\" && test -r \"./$ac_unique_file\" || as_fn_error $? \"$ac_msg\"\n\tpwd)`\n# When building in place, set srcdir=.\nif test \"$ac_abs_confdir\" = \"$ac_pwd\"; then\n srcdir=.\nfi\n# Remove unnecessary trailing slashes from srcdir.\n# Double slashes in file names in object file debugging info\n# mess up M-x gdb in Emacs.\ncase $srcdir in\n*/) srcdir=`expr \"X$srcdir\" : 'X\\(.*[^/]\\)' \\| \"X$srcdir\" : 'X\\(.*\\)'`;;\nesac\nfor ac_var in $ac_precious_vars; do\n eval ac_env_${ac_var}_set=\\${${ac_var}+set}\n eval ac_env_${ac_var}_value=\\$${ac_var}\n eval ac_cv_env_${ac_var}_set=\\${${ac_var}+set}\n eval ac_cv_env_${ac_var}_value=\\$${ac_var}\ndone\n\n#\n# Report the --help message.\n#\nif test \"$ac_init_help\" = \"long\"; then\n # Omit some internal or obsolete options to make the list less imposing.\n # This message is too long to be a string in the A/UX 3.1 sh.\n cat <<_ACEOF\n\\`configure' configures xgboost 3.3.0 to adapt to many kinds of systems.\n\nUsage: $0 [OPTION]... [VAR=VALUE]...\n\nTo assign environment variables (e.g., CC, CFLAGS...), specify them as\nVAR=VALUE. See below for descriptions of some of the useful variables.\n\nDefaults for the options are specified in brackets.\n\nConfiguration:\n -h, --help display this help and exit\n --help=short display options specific to this package\n --help=recursive display the short help of all the included packages\n -V, --version display version information and exit\n -q, --quiet, --silent do not print \\`checking ...' messages\n --cache-file=FILE cache test results in FILE [disabled]\n -C, --config-cache alias for \\`--cache-file=config.cache'\n -n, --no-create do not create output files\n --srcdir=DIR find the sources in DIR [configure dir or \\`..']\n\nInstallation directories:\n --prefix=PREFIX install architecture-independent files in PREFIX\n [$ac_default_prefix]\n --exec-prefix=EPREFIX install architecture-dependent files in EPREFIX\n [PREFIX]\n\nBy default, \\`make install' will install all the files in\n\\`$ac_default_prefix/bin', \\`$ac_default_prefix/lib' etc. You can specify\nan installation prefix other than \\`$ac_default_prefix' using \\`--prefix',\nfor instance \\`--prefix=\\$HOME'.\n\nFor better control, use the options below.\n\nFine tuning of the installation directories:\n --bindir=DIR user executables [EPREFIX/bin]\n --sbindir=DIR system admin executables [EPREFIX/sbin]\n --libexecdir=DIR program executables [EPREFIX/libexec]\n --sysconfdir=DIR read-only single-machine data [PREFIX/etc]\n --sharedstatedir=DIR modifiable architecture-independent data [PREFIX/com]\n --localstatedir=DIR modifiable single-machine data [PREFIX/var]\n --runstatedir=DIR modifiable per-process data [LOCALSTATEDIR/run]\n --libdir=DIR object code libraries [EPREFIX/lib]\n --includedir=DIR C header files [PREFIX/include]\n --oldincludedir=DIR C header files for non-gcc [/usr/include]\n --datarootdir=DIR read-only arch.-independent data root [PREFIX/share]\n --datadir=DIR read-only architecture-independent data [DATAROOTDIR]\n --infodir=DIR info documentation [DATAROOTDIR/info]\n --localedir=DIR locale-dependent data [DATAROOTDIR/locale]\n --mandir=DIR man documentation [DATAROOTDIR/man]\n --docdir=DIR documentation root [DATAROOTDIR/doc/xgboost]\n --htmldir=DIR html documentation [DOCDIR]\n --dvidir=DIR dvi documentation [DOCDIR]\n --pdfdir=DIR pdf documentation [DOCDIR]\n --psdir=DIR ps documentation [DOCDIR]\n_ACEOF\n\n cat <<\\_ACEOF\n_ACEOF\nfi\n\nif test -n \"$ac_init_help\"; then\n case $ac_init_help in\n short | recursive ) echo \"Configuration of xgboost 3.3.0:\";;\n esac\n cat <<\\_ACEOF\n\nSome influential environment variables:\n CXX C++ compiler command\n CXXFLAGS C++ compiler flags\n LDFLAGS linker flags, e.g. -L if you have libraries in a\n nonstandard directory \n LIBS libraries to pass to the linker, e.g. -l\n CPPFLAGS (Objective) C/C++ preprocessor flags, e.g. -I if\n you have headers in a nonstandard directory \n USE_LITTLE_ENDIAN\n \"Whether to build with little endian (checks at compile time if\n unset)\"\n\nUse these variables to override the choices made by `configure' or to help\nit to find libraries and programs with nonstandard names/locations.\n\nReport bugs to the package provider.\n_ACEOF\nac_status=$?\nfi\n\nif test \"$ac_init_help\" = \"recursive\"; then\n # If there are subdirs, report their specific --help.\n for ac_dir in : $ac_subdirs_all; do test \"x$ac_dir\" = x: && continue\n test -d \"$ac_dir\" ||\n { cd \"$srcdir\" && ac_pwd=`pwd` && srcdir=. && test -d \"$ac_dir\"; } ||\n continue\n ac_builddir=.\n\ncase \"$ac_dir\" in\n.) ac_dir_suffix= ac_top_builddir_sub=. ac_top_build_prefix= ;;\n*)\n ac_dir_suffix=/`printf \"%s\\n\" \"$ac_dir\" | sed 's|^\\.[\\\\/]||'`\n # A \"..\" for each directory in $ac_dir_suffix.\n ac_top_builddir_sub=`printf \"%s\\n\" \"$ac_dir_suffix\" | sed 's|/[^\\\\/]*|/..|g;s|/||'`\n case $ac_top_builddir_sub in\n \"\") ac_top_builddir_sub=. ac_top_build_prefix= ;;\n *) ac_top_build_prefix=$ac_top_builddir_sub/ ;;\n esac ;;\nesac\nac_abs_top_builddir=$ac_pwd\nac_abs_builddir=$ac_pwd$ac_dir_suffix\n# for backward compatibility:\nac_top_builddir=$ac_top_build_prefix\n\ncase $srcdir in\n .) # We are building in place.\n ac_srcdir=.\n ac_top_srcdir=$ac_top_builddir_sub\n ac_abs_top_srcdir=$ac_pwd ;;\n [\\\\/]* | ?:[\\\\/]* ) # Absolute name.\n ac_srcdir=$srcdir$ac_dir_suffix;\n ac_top_srcdir=$srcdir\n ac_abs_top_srcdir=$srcdir ;;\n *) # Relative name.\n ac_srcdir=$ac_top_build_prefix$srcdir$ac_dir_suffix\n ac_top_srcdir=$ac_top_build_prefix$srcdir\n ac_abs_top_srcdir=$ac_pwd/$srcdir ;;\nesac\nac_abs_srcdir=$ac_abs_top_srcdir$ac_dir_suffix\n\n cd \"$ac_dir\" || { ac_status=$?; continue; }\n # Check for configure.gnu first; this name is used for a wrapper for\n # Metaconfig's \"Configure\" on case-insensitive file systems.\n if test -f \"$ac_srcdir/configure.gnu\"; then\n echo &&\n $SHELL \"$ac_srcdir/configure.gnu\" --help=recursive\n elif test -f \"$ac_srcdir/configure\"; then\n echo &&\n $SHELL \"$ac_srcdir/configure\" --help=recursive\n else\n printf \"%s\\n\" \"$as_me: WARNING: no configuration information is in $ac_dir\" >&2\n fi || ac_status=$?\n cd \"$ac_pwd\" || { ac_status=$?; break; }\n done\nfi\n\ntest -n \"$ac_init_help\" && exit $ac_status\nif $ac_init_version; then\n cat <<\\_ACEOF\nxgboost configure 3.3.0\ngenerated by GNU Autoconf 2.71\n\nCopyright (C) 2021 Free Software Foundation, Inc.\nThis configure script is free software; the Free Software Foundation\ngives unlimited permission to copy, distribute and modify it.\n_ACEOF\n exit\nfi\n\n## ------------------------ ##\n## Autoconf initialization. ##\n## ------------------------ ##\n\n# ac_fn_cxx_try_compile LINENO\n# ----------------------------\n# Try to compile conftest.$ac_ext, and return whether this succeeded.\nac_fn_cxx_try_compile ()\n{\n as_lineno=${as_lineno-\"$1\"} as_lineno_stack=as_lineno_stack=$as_lineno_stack\n rm -f conftest.$ac_objext conftest.beam\n if { { ac_try=\"$ac_compile\"\ncase \"(($ac_try\" in\n *\\\"* | *\\`* | *\\\\*) ac_try_echo=\\$ac_try;;\n *) ac_try_echo=$ac_try;;\nesac\neval ac_try_echo=\"\\\"\\$as_me:${as_lineno-$LINENO}: $ac_try_echo\\\"\"\nprintf \"%s\\n\" \"$ac_try_echo\"; } >&5\n (eval \"$ac_compile\") 2>conftest.err\n ac_status=$?\n if test -s conftest.err; then\n grep -v '^ *+' conftest.err >conftest.er1\n cat conftest.er1 >&5\n mv -f conftest.er1 conftest.err\n fi\n printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: \\$? = $ac_status\" >&5\n test $ac_status = 0; } && {\n\t test -z \"$ac_cxx_werror_flag\" ||\n\t test ! -s conftest.err\n } && test -s conftest.$ac_objext\nthen :\n ac_retval=0\nelse $as_nop\n printf \"%s\\n\" \"$as_me: failed program was:\" >&5\nsed 's/^/| /' conftest.$ac_ext >&5\n\n\tac_retval=1\nfi\n eval $as_lineno_stack; ${as_lineno_stack:+:} unset as_lineno\n as_fn_set_status $ac_retval\n\n} # ac_fn_cxx_try_compile\n\n# ac_fn_cxx_try_link LINENO\n# -------------------------\n# Try to link conftest.$ac_ext, and return whether this succeeded.\nac_fn_cxx_try_link ()\n{\n as_lineno=${as_lineno-\"$1\"} as_lineno_stack=as_lineno_stack=$as_lineno_stack\n rm -f conftest.$ac_objext conftest.beam conftest$ac_exeext\n if { { ac_try=\"$ac_link\"\ncase \"(($ac_try\" in\n *\\\"* | *\\`* | *\\\\*) ac_try_echo=\\$ac_try;;\n *) ac_try_echo=$ac_try;;\nesac\neval ac_try_echo=\"\\\"\\$as_me:${as_lineno-$LINENO}: $ac_try_echo\\\"\"\nprintf \"%s\\n\" \"$ac_try_echo\"; } >&5\n (eval \"$ac_link\") 2>conftest.err\n ac_status=$?\n if test -s conftest.err; then\n grep -v '^ *+' conftest.err >conftest.er1\n cat conftest.er1 >&5\n mv -f conftest.er1 conftest.err\n fi\n printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: \\$? = $ac_status\" >&5\n test $ac_status = 0; } && {\n\t test -z \"$ac_cxx_werror_flag\" ||\n\t test ! -s conftest.err\n } && test -s conftest$ac_exeext && {\n\t test \"$cross_compiling\" = yes ||\n\t test -x conftest$ac_exeext\n }\nthen :\n ac_retval=0\nelse $as_nop\n printf \"%s\\n\" \"$as_me: failed program was:\" >&5\nsed 's/^/| /' conftest.$ac_ext >&5\n\n\tac_retval=1\nfi\n # Delete the IPA/IPO (Inter Procedural Analysis/Optimization) information\n # created by the PGI compiler (conftest_ipa8_conftest.oo), as it would\n # interfere with the next link command; also delete a directory that is\n # left behind by Apple's compiler. We do this before executing the actions.\n rm -rf conftest.dSYM conftest_ipa8_conftest.oo\n eval $as_lineno_stack; ${as_lineno_stack:+:} unset as_lineno\n as_fn_set_status $ac_retval\n\n} # ac_fn_cxx_try_link\n\n# ac_fn_cxx_check_func LINENO FUNC VAR\n# ------------------------------------\n# Tests whether FUNC exists, setting the cache variable VAR accordingly\nac_fn_cxx_check_func ()\n{\n as_lineno=${as_lineno-\"$1\"} as_lineno_stack=as_lineno_stack=$as_lineno_stack\n { printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: checking for $2\" >&5\nprintf %s \"checking for $2... \" >&6; }\nif eval test \\${$3+y}\nthen :\n printf %s \"(cached) \" >&6\nelse $as_nop\n cat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h. */\n/* Define $2 to an innocuous variant, in case declares $2.\n For example, HP-UX 11i declares gettimeofday. */\n#define $2 innocuous_$2\n\n/* System header to define __stub macros and hopefully few prototypes,\n which can conflict with char $2 (); below. */\n\n#include \n#undef $2\n\n/* Override any GCC internal prototype to avoid an error.\n Use char because int might match the return type of a GCC\n builtin and then its argument prototype would still apply. */\n#ifdef __cplusplus\nextern \"C\"\n#endif\nchar $2 ();\n/* The GNU C library defines this for functions which it implements\n to always fail with ENOSYS. Some functions are actually named\n something starting with __ and the normal name is an alias. */\n#if defined __stub_$2 || defined __stub___$2\nchoke me\n#endif\n\nint\nmain (void)\n{\nreturn $2 ();\n ;\n return 0;\n}\n_ACEOF\nif ac_fn_cxx_try_link \"$LINENO\"\nthen :\n eval \"$3=yes\"\nelse $as_nop\n eval \"$3=no\"\nfi\nrm -f core conftest.err conftest.$ac_objext conftest.beam \\\n conftest$ac_exeext conftest.$ac_ext\nfi\neval ac_res=\\$$3\n\t { printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: result: $ac_res\" >&5\nprintf \"%s\\n\" \"$ac_res\" >&6; }\n eval $as_lineno_stack; ${as_lineno_stack:+:} unset as_lineno\n\n} # ac_fn_cxx_check_func\n\n# ac_fn_cxx_try_run LINENO\n# ------------------------\n# Try to run conftest.$ac_ext, and return whether this succeeded. Assumes that\n# executables *can* be run.\nac_fn_cxx_try_run ()\n{\n as_lineno=${as_lineno-\"$1\"} as_lineno_stack=as_lineno_stack=$as_lineno_stack\n if { { ac_try=\"$ac_link\"\ncase \"(($ac_try\" in\n *\\\"* | *\\`* | *\\\\*) ac_try_echo=\\$ac_try;;\n *) ac_try_echo=$ac_try;;\nesac\neval ac_try_echo=\"\\\"\\$as_me:${as_lineno-$LINENO}: $ac_try_echo\\\"\"\nprintf \"%s\\n\" \"$ac_try_echo\"; } >&5\n (eval \"$ac_link\") 2>&5\n ac_status=$?\n printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: \\$? = $ac_status\" >&5\n test $ac_status = 0; } && { ac_try='./conftest$ac_exeext'\n { { case \"(($ac_try\" in\n *\\\"* | *\\`* | *\\\\*) ac_try_echo=\\$ac_try;;\n *) ac_try_echo=$ac_try;;\nesac\neval ac_try_echo=\"\\\"\\$as_me:${as_lineno-$LINENO}: $ac_try_echo\\\"\"\nprintf \"%s\\n\" \"$ac_try_echo\"; } >&5\n (eval \"$ac_try\") 2>&5\n ac_status=$?\n printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: \\$? = $ac_status\" >&5\n test $ac_status = 0; }; }\nthen :\n ac_retval=0\nelse $as_nop\n printf \"%s\\n\" \"$as_me: program exited with status $ac_status\" >&5\n printf \"%s\\n\" \"$as_me: failed program was:\" >&5\nsed 's/^/| /' conftest.$ac_ext >&5\n\n ac_retval=$ac_status\nfi\n rm -rf conftest.dSYM conftest_ipa8_conftest.oo\n eval $as_lineno_stack; ${as_lineno_stack:+:} unset as_lineno\n as_fn_set_status $ac_retval\n\n} # ac_fn_cxx_try_run\n\n# ac_fn_cxx_check_header_compile LINENO HEADER VAR INCLUDES\n# ---------------------------------------------------------\n# Tests whether HEADER exists and can be compiled using the include files in\n# INCLUDES, setting the cache variable VAR accordingly.\nac_fn_cxx_check_header_compile ()\n{\n as_lineno=${as_lineno-\"$1\"} as_lineno_stack=as_lineno_stack=$as_lineno_stack\n { printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: checking for $2\" >&5\nprintf %s \"checking for $2... \" >&6; }\nif eval test \\${$3+y}\nthen :\n printf %s \"(cached) \" >&6\nelse $as_nop\n cat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h. */\n$4\n#include <$2>\n_ACEOF\nif ac_fn_cxx_try_compile \"$LINENO\"\nthen :\n eval \"$3=yes\"\nelse $as_nop\n eval \"$3=no\"\nfi\nrm -f core conftest.err conftest.$ac_objext conftest.beam conftest.$ac_ext\nfi\neval ac_res=\\$$3\n\t { printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: result: $ac_res\" >&5\nprintf \"%s\\n\" \"$ac_res\" >&6; }\n eval $as_lineno_stack; ${as_lineno_stack:+:} unset as_lineno\n\n} # ac_fn_cxx_check_header_compile\nac_configure_args_raw=\nfor ac_arg\ndo\n case $ac_arg in\n *\\'*)\n ac_arg=`printf \"%s\\n\" \"$ac_arg\" | sed \"s/'/'\\\\\\\\\\\\\\\\''/g\"` ;;\n esac\n as_fn_append ac_configure_args_raw \" '$ac_arg'\"\ndone\n\ncase $ac_configure_args_raw in\n *$as_nl*)\n ac_safe_unquote= ;;\n *)\n ac_unsafe_z='|&;<>()$`\\\\\"*?[ ''\t' # This string ends in space, tab.\n ac_unsafe_a=\"$ac_unsafe_z#~\"\n ac_safe_unquote=\"s/ '\\\\([^$ac_unsafe_a][^$ac_unsafe_z]*\\\\)'/ \\\\1/g\"\n ac_configure_args_raw=` printf \"%s\\n\" \"$ac_configure_args_raw\" | sed \"$ac_safe_unquote\"`;;\nesac\n\ncat >config.log <<_ACEOF\nThis file contains any messages produced by compilers while\nrunning configure, to aid debugging if configure makes a mistake.\n\nIt was created by xgboost $as_me 3.3.0, which was\ngenerated by GNU Autoconf 2.71. Invocation command line was\n\n $ $0$ac_configure_args_raw\n\n_ACEOF\nexec 5>>config.log\n{\ncat <<_ASUNAME\n## --------- ##\n## Platform. ##\n## --------- ##\n\nhostname = `(hostname || uname -n) 2>/dev/null | sed 1q`\nuname -m = `(uname -m) 2>/dev/null || echo unknown`\nuname -r = `(uname -r) 2>/dev/null || echo unknown`\nuname -s = `(uname -s) 2>/dev/null || echo unknown`\nuname -v = `(uname -v) 2>/dev/null || echo unknown`\n\n/usr/bin/uname -p = `(/usr/bin/uname -p) 2>/dev/null || echo unknown`\n/bin/uname -X = `(/bin/uname -X) 2>/dev/null || echo unknown`\n\n/bin/arch = `(/bin/arch) 2>/dev/null || echo unknown`\n/usr/bin/arch -k = `(/usr/bin/arch -k) 2>/dev/null || echo unknown`\n/usr/convex/getsysinfo = `(/usr/convex/getsysinfo) 2>/dev/null || echo unknown`\n/usr/bin/hostinfo = `(/usr/bin/hostinfo) 2>/dev/null || echo unknown`\n/bin/machine = `(/bin/machine) 2>/dev/null || echo unknown`\n/usr/bin/oslevel = `(/usr/bin/oslevel) 2>/dev/null || echo unknown`\n/bin/universe = `(/bin/universe) 2>/dev/null || echo unknown`\n\n_ASUNAME\n\nas_save_IFS=$IFS; IFS=$PATH_SEPARATOR\nfor as_dir in $PATH\ndo\n IFS=$as_save_IFS\n case $as_dir in #(((\n '') as_dir=./ ;;\n */) ;;\n *) as_dir=$as_dir/ ;;\n esac\n printf \"%s\\n\" \"PATH: $as_dir\"\n done\nIFS=$as_save_IFS\n\n} >&5\n\ncat >&5 <<_ACEOF\n\n\n## ----------- ##\n## Core tests. ##\n## ----------- ##\n\n_ACEOF\n\n\n# Keep a trace of the command line.\n# Strip out --no-create and --no-recursion so they do not pile up.\n# Strip out --silent because we don't want to record it for future runs.\n# Also quote any args containing shell meta-characters.\n# Make two passes to allow for proper duplicate-argument suppression.\nac_configure_args=\nac_configure_args0=\nac_configure_args1=\nac_must_keep_next=false\nfor ac_pass in 1 2\ndo\n for ac_arg\n do\n case $ac_arg in\n -no-create | --no-c* | -n | -no-recursion | --no-r*) continue ;;\n -q | -quiet | --quiet | --quie | --qui | --qu | --q \\\n | -silent | --silent | --silen | --sile | --sil)\n continue ;;\n *\\'*)\n ac_arg=`printf \"%s\\n\" \"$ac_arg\" | sed \"s/'/'\\\\\\\\\\\\\\\\''/g\"` ;;\n esac\n case $ac_pass in\n 1) as_fn_append ac_configure_args0 \" '$ac_arg'\" ;;\n 2)\n as_fn_append ac_configure_args1 \" '$ac_arg'\"\n if test $ac_must_keep_next = true; then\n\tac_must_keep_next=false # Got value, back to normal.\n else\n\tcase $ac_arg in\n\t *=* | --config-cache | -C | -disable-* | --disable-* \\\n\t | -enable-* | --enable-* | -gas | --g* | -nfp | --nf* \\\n\t | -q | -quiet | --q* | -silent | --sil* | -v | -verb* \\\n\t | -with-* | --with-* | -without-* | --without-* | --x)\n\t case \"$ac_configure_args0 \" in\n\t \"$ac_configure_args1\"*\" '$ac_arg' \"* ) continue ;;\n\t esac\n\t ;;\n\t -* ) ac_must_keep_next=true ;;\n\tesac\n fi\n as_fn_append ac_configure_args \" '$ac_arg'\"\n ;;\n esac\n done\ndone\n{ ac_configure_args0=; unset ac_configure_args0;}\n{ ac_configure_args1=; unset ac_configure_args1;}\n\n# When interrupted or exit'd, cleanup temporary files, and complete\n# config.log. We remove comments because anyway the quotes in there\n# would cause problems or look ugly.\n# WARNING: Use '\\'' to represent an apostrophe within the trap.\n# WARNING: Do not start the trap code with a newline, due to a FreeBSD 4.0 bug.\ntrap 'exit_status=$?\n # Sanitize IFS.\n IFS=\" \"\"\t$as_nl\"\n # Save into config.log some information that might help in debugging.\n {\n echo\n\n printf \"%s\\n\" \"## ---------------- ##\n## Cache variables. ##\n## ---------------- ##\"\n echo\n # The following way of writing the cache mishandles newlines in values,\n(\n for ac_var in `(set) 2>&1 | sed -n '\\''s/^\\([a-zA-Z_][a-zA-Z0-9_]*\\)=.*/\\1/p'\\''`; do\n eval ac_val=\\$$ac_var\n case $ac_val in #(\n *${as_nl}*)\n case $ac_var in #(\n *_cv_*) { printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: WARNING: cache variable $ac_var contains a newline\" >&5\nprintf \"%s\\n\" \"$as_me: WARNING: cache variable $ac_var contains a newline\" >&2;} ;;\n esac\n case $ac_var in #(\n _ | IFS | as_nl) ;; #(\n BASH_ARGV | BASH_SOURCE) eval $ac_var= ;; #(\n *) { eval $ac_var=; unset $ac_var;} ;;\n esac ;;\n esac\n done\n (set) 2>&1 |\n case $as_nl`(ac_space='\\'' '\\''; set) 2>&1` in #(\n *${as_nl}ac_space=\\ *)\n sed -n \\\n\t\"s/'\\''/'\\''\\\\\\\\'\\'''\\''/g;\n\t s/^\\\\([_$as_cr_alnum]*_cv_[_$as_cr_alnum]*\\\\)=\\\\(.*\\\\)/\\\\1='\\''\\\\2'\\''/p\"\n ;; #(\n *)\n sed -n \"/^[_$as_cr_alnum]*_cv_[_$as_cr_alnum]*=/p\"\n ;;\n esac |\n sort\n)\n echo\n\n printf \"%s\\n\" \"## ----------------- ##\n## Output variables. ##\n## ----------------- ##\"\n echo\n for ac_var in $ac_subst_vars\n do\n eval ac_val=\\$$ac_var\n case $ac_val in\n *\\'\\''*) ac_val=`printf \"%s\\n\" \"$ac_val\" | sed \"s/'\\''/'\\''\\\\\\\\\\\\\\\\'\\'''\\''/g\"`;;\n esac\n printf \"%s\\n\" \"$ac_var='\\''$ac_val'\\''\"\n done | sort\n echo\n\n if test -n \"$ac_subst_files\"; then\n printf \"%s\\n\" \"## ------------------- ##\n## File substitutions. ##\n## ------------------- ##\"\n echo\n for ac_var in $ac_subst_files\n do\n\teval ac_val=\\$$ac_var\n\tcase $ac_val in\n\t*\\'\\''*) ac_val=`printf \"%s\\n\" \"$ac_val\" | sed \"s/'\\''/'\\''\\\\\\\\\\\\\\\\'\\'''\\''/g\"`;;\n\tesac\n\tprintf \"%s\\n\" \"$ac_var='\\''$ac_val'\\''\"\n done | sort\n echo\n fi\n\n if test -s confdefs.h; then\n printf \"%s\\n\" \"## ----------- ##\n## confdefs.h. ##\n## ----------- ##\"\n echo\n cat confdefs.h\n echo\n fi\n test \"$ac_signal\" != 0 &&\n printf \"%s\\n\" \"$as_me: caught signal $ac_signal\"\n printf \"%s\\n\" \"$as_me: exit $exit_status\"\n } >&5\n rm -f core *.core core.conftest.* &&\n rm -f -r conftest* confdefs* conf$$* $ac_clean_files &&\n exit $exit_status\n' 0\nfor ac_signal in 1 2 13 15; do\n trap 'ac_signal='$ac_signal'; as_fn_exit 1' $ac_signal\ndone\nac_signal=0\n\n# confdefs.h avoids OS command line length limits that DEFS can exceed.\nrm -f -r conftest* confdefs.h\n\nprintf \"%s\\n\" \"/* confdefs.h */\" > confdefs.h\n\n# Predefined preprocessor variables.\n\nprintf \"%s\\n\" \"#define PACKAGE_NAME \\\"$PACKAGE_NAME\\\"\" >>confdefs.h\n\nprintf \"%s\\n\" \"#define PACKAGE_TARNAME \\\"$PACKAGE_TARNAME\\\"\" >>confdefs.h\n\nprintf \"%s\\n\" \"#define PACKAGE_VERSION \\\"$PACKAGE_VERSION\\\"\" >>confdefs.h\n\nprintf \"%s\\n\" \"#define PACKAGE_STRING \\\"$PACKAGE_STRING\\\"\" >>confdefs.h\n\nprintf \"%s\\n\" \"#define PACKAGE_BUGREPORT \\\"$PACKAGE_BUGREPORT\\\"\" >>confdefs.h\n\nprintf \"%s\\n\" \"#define PACKAGE_URL \\\"$PACKAGE_URL\\\"\" >>confdefs.h\n\n\n# Let the site file select an alternate cache file if it wants to.\n# Prefer an explicitly selected file to automatically selected ones.\nif test -n \"$CONFIG_SITE\"; then\n ac_site_files=\"$CONFIG_SITE\"\nelif test \"x$prefix\" != xNONE; then\n ac_site_files=\"$prefix/share/config.site $prefix/etc/config.site\"\nelse\n ac_site_files=\"$ac_default_prefix/share/config.site $ac_default_prefix/etc/config.site\"\nfi\n\nfor ac_site_file in $ac_site_files\ndo\n case $ac_site_file in #(\n */*) :\n ;; #(\n *) :\n ac_site_file=./$ac_site_file ;;\nesac\n if test -f \"$ac_site_file\" && test -r \"$ac_site_file\"; then\n { printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: loading site script $ac_site_file\" >&5\nprintf \"%s\\n\" \"$as_me: loading site script $ac_site_file\" >&6;}\n sed 's/^/| /' \"$ac_site_file\" >&5\n . \"$ac_site_file\" \\\n || { { printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\nprintf \"%s\\n\" \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"failed to load site script $ac_site_file\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\n fi\ndone\n\nif test -r \"$cache_file\"; then\n # Some versions of bash will fail to source /dev/null (special files\n # actually), so we avoid doing that. DJGPP emulates it as a regular file.\n if test /dev/null != \"$cache_file\" && test -f \"$cache_file\"; then\n { printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: loading cache $cache_file\" >&5\nprintf \"%s\\n\" \"$as_me: loading cache $cache_file\" >&6;}\n case $cache_file in\n [\\\\/]* | ?:[\\\\/]* ) . \"$cache_file\";;\n *) . \"./$cache_file\";;\n esac\n fi\nelse\n { printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: creating cache $cache_file\" >&5\nprintf \"%s\\n\" \"$as_me: creating cache $cache_file\" >&6;}\n >$cache_file\nfi\n\n# Test code for whether the C++ compiler supports C++98 (global declarations)\nac_cxx_conftest_cxx98_globals='\n// Does the compiler advertise C++98 conformance?\n#if !defined __cplusplus || __cplusplus < 199711L\n# error \"Compiler does not advertise C++98 conformance\"\n#endif\n\n// These inclusions are to reject old compilers that\n// lack the unsuffixed header files.\n#include \n#include \n\n// and are *not* freestanding headers in C++98.\nextern void assert (int);\nnamespace std {\n extern int strcmp (const char *, const char *);\n}\n\n// Namespaces, exceptions, and templates were all added after \"C++ 2.0\".\nusing std::exception;\nusing std::strcmp;\n\nnamespace {\n\nvoid test_exception_syntax()\n{\n try {\n throw \"test\";\n } catch (const char *s) {\n // Extra parentheses suppress a warning when building autoconf itself,\n // due to lint rules shared with more typical C programs.\n assert (!(strcmp) (s, \"test\"));\n }\n}\n\ntemplate struct test_template\n{\n T const val;\n explicit test_template(T t) : val(t) {}\n template T add(U u) { return static_cast(u) + val; }\n};\n\n} // anonymous namespace\n'\n\n# Test code for whether the C++ compiler supports C++98 (body of main)\nac_cxx_conftest_cxx98_main='\n assert (argc);\n assert (! argv[0]);\n{\n test_exception_syntax ();\n test_template tt (2.0);\n assert (tt.add (4) == 6.0);\n assert (true && !false);\n}\n'\n\n# Test code for whether the C++ compiler supports C++11 (global declarations)\nac_cxx_conftest_cxx11_globals='\n// Does the compiler advertise C++ 2011 conformance?\n#if !defined __cplusplus || __cplusplus < 201103L\n# error \"Compiler does not advertise C++11 conformance\"\n#endif\n\nnamespace cxx11test\n{\n constexpr int get_val() { return 20; }\n\n struct testinit\n {\n int i;\n double d;\n };\n\n class delegate\n {\n public:\n delegate(int n) : n(n) {}\n delegate(): delegate(2354) {}\n\n virtual int getval() { return this->n; };\n protected:\n int n;\n };\n\n class overridden : public delegate\n {\n public:\n overridden(int n): delegate(n) {}\n virtual int getval() override final { return this->n * 2; }\n };\n\n class nocopy\n {\n public:\n nocopy(int i): i(i) {}\n nocopy() = default;\n nocopy(const nocopy&) = delete;\n nocopy & operator=(const nocopy&) = delete;\n private:\n int i;\n };\n\n // for testing lambda expressions\n template Ret eval(Fn f, Ret v)\n {\n return f(v);\n }\n\n // for testing variadic templates and trailing return types\n template auto sum(V first) -> V\n {\n return first;\n }\n template auto sum(V first, Args... rest) -> V\n {\n return first + sum(rest...);\n }\n}\n'\n\n# Test code for whether the C++ compiler supports C++11 (body of main)\nac_cxx_conftest_cxx11_main='\n{\n // Test auto and decltype\n auto a1 = 6538;\n auto a2 = 48573953.4;\n auto a3 = \"String literal\";\n\n int total = 0;\n for (auto i = a3; *i; ++i) { total += *i; }\n\n decltype(a2) a4 = 34895.034;\n}\n{\n // Test constexpr\n short sa[cxx11test::get_val()] = { 0 };\n}\n{\n // Test initializer lists\n cxx11test::testinit il = { 4323, 435234.23544 };\n}\n{\n // Test range-based for\n int array[] = {9, 7, 13, 15, 4, 18, 12, 10, 5, 3,\n 14, 19, 17, 8, 6, 20, 16, 2, 11, 1};\n for (auto &x : array) { x += 23; }\n}\n{\n // Test lambda expressions\n using cxx11test::eval;\n assert (eval ([](int x) { return x*2; }, 21) == 42);\n double d = 2.0;\n assert (eval ([&](double x) { return d += x; }, 3.0) == 5.0);\n assert (d == 5.0);\n assert (eval ([=](double x) mutable { return d += x; }, 4.0) == 9.0);\n assert (d == 5.0);\n}\n{\n // Test use of variadic templates\n using cxx11test::sum;\n auto a = sum(1);\n auto b = sum(1, 2);\n auto c = sum(1.0, 2.0, 3.0);\n}\n{\n // Test constructor delegation\n cxx11test::delegate d1;\n cxx11test::delegate d2();\n cxx11test::delegate d3(45);\n}\n{\n // Test override and final\n cxx11test::overridden o1(55464);\n}\n{\n // Test nullptr\n char *c = nullptr;\n}\n{\n // Test template brackets\n test_template<::test_template> v(test_template(12));\n}\n{\n // Unicode literals\n char const *utf8 = u8\"UTF-8 string \\u2500\";\n char16_t const *utf16 = u\"UTF-8 string \\u2500\";\n char32_t const *utf32 = U\"UTF-32 string \\u2500\";\n}\n'\n\n# Test code for whether the C compiler supports C++11 (complete).\nac_cxx_conftest_cxx11_program=\"${ac_cxx_conftest_cxx98_globals}\n${ac_cxx_conftest_cxx11_globals}\n\nint\nmain (int argc, char **argv)\n{\n int ok = 0;\n ${ac_cxx_conftest_cxx98_main}\n ${ac_cxx_conftest_cxx11_main}\n return ok;\n}\n\"\n\n# Test code for whether the C compiler supports C++98 (complete).\nac_cxx_conftest_cxx98_program=\"${ac_cxx_conftest_cxx98_globals}\nint\nmain (int argc, char **argv)\n{\n int ok = 0;\n ${ac_cxx_conftest_cxx98_main}\n return ok;\n}\n\"\n\nas_fn_append ac_header_cxx_list \" stdio.h stdio_h HAVE_STDIO_H\"\nas_fn_append ac_header_cxx_list \" stdlib.h stdlib_h HAVE_STDLIB_H\"\nas_fn_append ac_header_cxx_list \" string.h string_h HAVE_STRING_H\"\nas_fn_append ac_header_cxx_list \" inttypes.h inttypes_h HAVE_INTTYPES_H\"\nas_fn_append ac_header_cxx_list \" stdint.h stdint_h HAVE_STDINT_H\"\nas_fn_append ac_header_cxx_list \" strings.h strings_h HAVE_STRINGS_H\"\nas_fn_append ac_header_cxx_list \" sys/stat.h sys_stat_h HAVE_SYS_STAT_H\"\nas_fn_append ac_header_cxx_list \" sys/types.h sys_types_h HAVE_SYS_TYPES_H\"\nas_fn_append ac_header_cxx_list \" unistd.h unistd_h HAVE_UNISTD_H\"\n# Check that the precious variables saved in the cache have kept the same\n# value.\nac_cache_corrupted=false\nfor ac_var in $ac_precious_vars; do\n eval ac_old_set=\\$ac_cv_env_${ac_var}_set\n eval ac_new_set=\\$ac_env_${ac_var}_set\n eval ac_old_val=\\$ac_cv_env_${ac_var}_value\n eval ac_new_val=\\$ac_env_${ac_var}_value\n case $ac_old_set,$ac_new_set in\n set,)\n { printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: error: \\`$ac_var' was set to \\`$ac_old_val' in the previous run\" >&5\nprintf \"%s\\n\" \"$as_me: error: \\`$ac_var' was set to \\`$ac_old_val' in the previous run\" >&2;}\n ac_cache_corrupted=: ;;\n ,set)\n { printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: error: \\`$ac_var' was not set in the previous run\" >&5\nprintf \"%s\\n\" \"$as_me: error: \\`$ac_var' was not set in the previous run\" >&2;}\n ac_cache_corrupted=: ;;\n ,);;\n *)\n if test \"x$ac_old_val\" != \"x$ac_new_val\"; then\n\t# differences in whitespace do not lead to failure.\n\tac_old_val_w=`echo x $ac_old_val`\n\tac_new_val_w=`echo x $ac_new_val`\n\tif test \"$ac_old_val_w\" != \"$ac_new_val_w\"; then\n\t { printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: error: \\`$ac_var' has changed since the previous run:\" >&5\nprintf \"%s\\n\" \"$as_me: error: \\`$ac_var' has changed since the previous run:\" >&2;}\n\t ac_cache_corrupted=:\n\telse\n\t { printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: warning: ignoring whitespace changes in \\`$ac_var' since the previous run:\" >&5\nprintf \"%s\\n\" \"$as_me: warning: ignoring whitespace changes in \\`$ac_var' since the previous run:\" >&2;}\n\t eval $ac_var=\\$ac_old_val\n\tfi\n\t{ printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: former value: \\`$ac_old_val'\" >&5\nprintf \"%s\\n\" \"$as_me: former value: \\`$ac_old_val'\" >&2;}\n\t{ printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: current value: \\`$ac_new_val'\" >&5\nprintf \"%s\\n\" \"$as_me: current value: \\`$ac_new_val'\" >&2;}\n fi;;\n esac\n # Pass precious variables to config.status.\n if test \"$ac_new_set\" = set; then\n case $ac_new_val in\n *\\'*) ac_arg=$ac_var=`printf \"%s\\n\" \"$ac_new_val\" | sed \"s/'/'\\\\\\\\\\\\\\\\''/g\"` ;;\n *) ac_arg=$ac_var=$ac_new_val ;;\n esac\n case \" $ac_configure_args \" in\n *\" '$ac_arg' \"*) ;; # Avoid dups. Use of quotes ensures accuracy.\n *) as_fn_append ac_configure_args \" '$ac_arg'\" ;;\n esac\n fi\ndone\nif $ac_cache_corrupted; then\n { printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\nprintf \"%s\\n\" \"$as_me: error: in \\`$ac_pwd':\" >&2;}\n { printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: error: changes in the environment can compromise the build\" >&5\nprintf \"%s\\n\" \"$as_me: error: changes in the environment can compromise the build\" >&2;}\n as_fn_error $? \"run \\`${MAKE-make} distclean' and/or \\`rm $cache_file'\n\t and start over\" \"$LINENO\" 5\nfi\n## -------------------- ##\n## Main body of script. ##\n## -------------------- ##\n\nac_ext=c\nac_cpp='$CPP $CPPFLAGS'\nac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5'\nac_link='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5'\nac_compiler_gnu=$ac_cv_c_compiler_gnu\n\n\n\n: ${R_HOME=`R RHOME`}\nif test -z \"${R_HOME}\"; then\n echo \"could not determine R_HOME\"\n exit 1\nfi\n\nCXX17=`\"${R_HOME}/bin/R\" CMD config CXX17`\nCXX17STD=`\"${R_HOME}/bin/R\" CMD config CXX17STD`\nCXX=\"${CXX17} ${CXX17STD}\"\nCXXFLAGS=`\"${R_HOME}/bin/R\" CMD config CXXFLAGS`\n\nCC=`\"${R_HOME}/bin/R\" CMD config CC`\nCFLAGS=`\"${R_HOME}/bin/R\" CMD config CFLAGS`\nCPPFLAGS=`\"${R_HOME}/bin/R\" CMD config CPPFLAGS`\n\nLDFLAGS=`\"${R_HOME}/bin/R\" CMD config LDFLAGS`\nac_ext=cpp\nac_cpp='$CXXCPP $CPPFLAGS'\nac_compile='$CXX -c $CXXFLAGS $CPPFLAGS conftest.$ac_ext >&5'\nac_link='$CXX -o conftest$ac_exeext $CXXFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5'\nac_compiler_gnu=$ac_cv_cxx_compiler_gnu\n\n\nDMLC_DEFS=\"\"\n\n{ printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: Checking if/where backtrace is available\" >&5\nprintf \"%s\\n\" \"$as_me: Checking if/where backtrace is available\" >&6;}\n\n\n\n\n\n\nac_ext=cpp\nac_cpp='$CXXCPP $CPPFLAGS'\nac_compile='$CXX -c $CXXFLAGS $CPPFLAGS conftest.$ac_ext >&5'\nac_link='$CXX -o conftest$ac_exeext $CXXFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5'\nac_compiler_gnu=$ac_cv_cxx_compiler_gnu\nif test -z \"$CXX\"; then\n if test -n \"$CCC\"; then\n CXX=$CCC\n else\n if test -n \"$ac_tool_prefix\"; then\n for ac_prog in g++ c++ gpp aCC CC cxx cc++ cl.exe FCC KCC RCC xlC_r xlC clang++\n do\n # Extract the first word of \"$ac_tool_prefix$ac_prog\", so it can be a program name with args.\nset dummy $ac_tool_prefix$ac_prog; ac_word=$2\n{ printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: checking for $ac_word\" >&5\nprintf %s \"checking for $ac_word... \" >&6; }\nif test ${ac_cv_prog_CXX+y}\nthen :\n printf %s \"(cached) \" >&6\nelse $as_nop\n if test -n \"$CXX\"; then\n ac_cv_prog_CXX=\"$CXX\" # Let the user override the test.\nelse\nas_save_IFS=$IFS; IFS=$PATH_SEPARATOR\nfor as_dir in $PATH\ndo\n IFS=$as_save_IFS\n case $as_dir in #(((\n '') as_dir=./ ;;\n */) ;;\n *) as_dir=$as_dir/ ;;\n esac\n for ac_exec_ext in '' $ac_executable_extensions; do\n if as_fn_executable_p \"$as_dir$ac_word$ac_exec_ext\"; then\n ac_cv_prog_CXX=\"$ac_tool_prefix$ac_prog\"\n printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: found $as_dir$ac_word$ac_exec_ext\" >&5\n break 2\n fi\ndone\n done\nIFS=$as_save_IFS\n\nfi\nfi\nCXX=$ac_cv_prog_CXX\nif test -n \"$CXX\"; then\n { printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: result: $CXX\" >&5\nprintf \"%s\\n\" \"$CXX\" >&6; }\nelse\n { printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: result: no\" >&5\nprintf \"%s\\n\" \"no\" >&6; }\nfi\n\n\n test -n \"$CXX\" && break\n done\nfi\nif test -z \"$CXX\"; then\n ac_ct_CXX=$CXX\n for ac_prog in g++ c++ gpp aCC CC cxx cc++ cl.exe FCC KCC RCC xlC_r xlC clang++\ndo\n # Extract the first word of \"$ac_prog\", so it can be a program name with args.\nset dummy $ac_prog; ac_word=$2\n{ printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: checking for $ac_word\" >&5\nprintf %s \"checking for $ac_word... \" >&6; }\nif test ${ac_cv_prog_ac_ct_CXX+y}\nthen :\n printf %s \"(cached) \" >&6\nelse $as_nop\n if test -n \"$ac_ct_CXX\"; then\n ac_cv_prog_ac_ct_CXX=\"$ac_ct_CXX\" # Let the user override the test.\nelse\nas_save_IFS=$IFS; IFS=$PATH_SEPARATOR\nfor as_dir in $PATH\ndo\n IFS=$as_save_IFS\n case $as_dir in #(((\n '') as_dir=./ ;;\n */) ;;\n *) as_dir=$as_dir/ ;;\n esac\n for ac_exec_ext in '' $ac_executable_extensions; do\n if as_fn_executable_p \"$as_dir$ac_word$ac_exec_ext\"; then\n ac_cv_prog_ac_ct_CXX=\"$ac_prog\"\n printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: found $as_dir$ac_word$ac_exec_ext\" >&5\n break 2\n fi\ndone\n done\nIFS=$as_save_IFS\n\nfi\nfi\nac_ct_CXX=$ac_cv_prog_ac_ct_CXX\nif test -n \"$ac_ct_CXX\"; then\n { printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: result: $ac_ct_CXX\" >&5\nprintf \"%s\\n\" \"$ac_ct_CXX\" >&6; }\nelse\n { printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: result: no\" >&5\nprintf \"%s\\n\" \"no\" >&6; }\nfi\n\n\n test -n \"$ac_ct_CXX\" && break\ndone\n\n if test \"x$ac_ct_CXX\" = x; then\n CXX=\"g++\"\n else\n case $cross_compiling:$ac_tool_warned in\nyes:)\n{ printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet\" >&5\nprintf \"%s\\n\" \"$as_me: WARNING: using cross tools not prefixed with host triplet\" >&2;}\nac_tool_warned=yes ;;\nesac\n CXX=$ac_ct_CXX\n fi\nfi\n\n fi\nfi\n# Provide some information about the compiler.\nprintf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: checking for C++ compiler version\" >&5\nset X $ac_compile\nac_compiler=$2\nfor ac_option in --version -v -V -qversion; do\n { { ac_try=\"$ac_compiler $ac_option >&5\"\ncase \"(($ac_try\" in\n *\\\"* | *\\`* | *\\\\*) ac_try_echo=\\$ac_try;;\n *) ac_try_echo=$ac_try;;\nesac\neval ac_try_echo=\"\\\"\\$as_me:${as_lineno-$LINENO}: $ac_try_echo\\\"\"\nprintf \"%s\\n\" \"$ac_try_echo\"; } >&5\n (eval \"$ac_compiler $ac_option >&5\") 2>conftest.err\n ac_status=$?\n if test -s conftest.err; then\n sed '10a\\\n... rest of stderr output deleted ...\n 10q' conftest.err >conftest.er1\n cat conftest.er1 >&5\n fi\n rm -f conftest.er1 conftest.err\n printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: \\$? = $ac_status\" >&5\n test $ac_status = 0; }\ndone\n\ncat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h. */\n\nint\nmain (void)\n{\n\n ;\n return 0;\n}\n_ACEOF\nac_clean_files_save=$ac_clean_files\nac_clean_files=\"$ac_clean_files a.out a.out.dSYM a.exe b.out\"\n# Try to create an executable without -o first, disregard a.out.\n# It will help us diagnose broken compilers, and finding out an intuition\n# of exeext.\n{ printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: checking whether the C++ compiler works\" >&5\nprintf %s \"checking whether the C++ compiler works... \" >&6; }\nac_link_default=`printf \"%s\\n\" \"$ac_link\" | sed 's/ -o *conftest[^ ]*//'`\n\n# The possible output files:\nac_files=\"a.out conftest.exe conftest a.exe a_out.exe b.out conftest.*\"\n\nac_rmfiles=\nfor ac_file in $ac_files\ndo\n case $ac_file in\n *.$ac_ext | *.xcoff | *.tds | *.d | *.pdb | *.xSYM | *.bb | *.bbg | *.map | *.inf | *.dSYM | *.o | *.obj ) ;;\n * ) ac_rmfiles=\"$ac_rmfiles $ac_file\";;\n esac\ndone\nrm -f $ac_rmfiles\n\nif { { ac_try=\"$ac_link_default\"\ncase \"(($ac_try\" in\n *\\\"* | *\\`* | *\\\\*) ac_try_echo=\\$ac_try;;\n *) ac_try_echo=$ac_try;;\nesac\neval ac_try_echo=\"\\\"\\$as_me:${as_lineno-$LINENO}: $ac_try_echo\\\"\"\nprintf \"%s\\n\" \"$ac_try_echo\"; } >&5\n (eval \"$ac_link_default\") 2>&5\n ac_status=$?\n printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: \\$? = $ac_status\" >&5\n test $ac_status = 0; }\nthen :\n # Autoconf-2.13 could set the ac_cv_exeext variable to `no'.\n# So ignore a value of `no', otherwise this would lead to `EXEEXT = no'\n# in a Makefile. We should not override ac_cv_exeext if it was cached,\n# so that the user can short-circuit this test for compilers unknown to\n# Autoconf.\nfor ac_file in $ac_files ''\ndo\n test -f \"$ac_file\" || continue\n case $ac_file in\n *.$ac_ext | *.xcoff | *.tds | *.d | *.pdb | *.xSYM | *.bb | *.bbg | *.map | *.inf | *.dSYM | *.o | *.obj )\n\t;;\n [ab].out )\n\t# We found the default executable, but exeext='' is most\n\t# certainly right.\n\tbreak;;\n *.* )\n\tif test ${ac_cv_exeext+y} && test \"$ac_cv_exeext\" != no;\n\tthen :; else\n\t ac_cv_exeext=`expr \"$ac_file\" : '[^.]*\\(\\..*\\)'`\n\tfi\n\t# We set ac_cv_exeext here because the later test for it is not\n\t# safe: cross compilers may not add the suffix if given an `-o'\n\t# argument, so we may need to know it at that point already.\n\t# Even if this section looks crufty: it has the advantage of\n\t# actually working.\n\tbreak;;\n * )\n\tbreak;;\n esac\ndone\ntest \"$ac_cv_exeext\" = no && ac_cv_exeext=\n\nelse $as_nop\n ac_file=''\nfi\nif test -z \"$ac_file\"\nthen :\n { printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: result: no\" >&5\nprintf \"%s\\n\" \"no\" >&6; }\nprintf \"%s\\n\" \"$as_me: failed program was:\" >&5\nsed 's/^/| /' conftest.$ac_ext >&5\n\n{ { printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\nprintf \"%s\\n\" \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error 77 \"C++ compiler cannot create executables\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\nelse $as_nop\n { printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: result: yes\" >&5\nprintf \"%s\\n\" \"yes\" >&6; }\nfi\n{ printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: checking for C++ compiler default output file name\" >&5\nprintf %s \"checking for C++ compiler default output file name... \" >&6; }\n{ printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: result: $ac_file\" >&5\nprintf \"%s\\n\" \"$ac_file\" >&6; }\nac_exeext=$ac_cv_exeext\n\nrm -f -r a.out a.out.dSYM a.exe conftest$ac_cv_exeext b.out\nac_clean_files=$ac_clean_files_save\n{ printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: checking for suffix of executables\" >&5\nprintf %s \"checking for suffix of executables... \" >&6; }\nif { { ac_try=\"$ac_link\"\ncase \"(($ac_try\" in\n *\\\"* | *\\`* | *\\\\*) ac_try_echo=\\$ac_try;;\n *) ac_try_echo=$ac_try;;\nesac\neval ac_try_echo=\"\\\"\\$as_me:${as_lineno-$LINENO}: $ac_try_echo\\\"\"\nprintf \"%s\\n\" \"$ac_try_echo\"; } >&5\n (eval \"$ac_link\") 2>&5\n ac_status=$?\n printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: \\$? = $ac_status\" >&5\n test $ac_status = 0; }\nthen :\n # If both `conftest.exe' and `conftest' are `present' (well, observable)\n# catch `conftest.exe'. For instance with Cygwin, `ls conftest' will\n# work properly (i.e., refer to `conftest.exe'), while it won't with\n# `rm'.\nfor ac_file in conftest.exe conftest conftest.*; do\n test -f \"$ac_file\" || continue\n case $ac_file in\n *.$ac_ext | *.xcoff | *.tds | *.d | *.pdb | *.xSYM | *.bb | *.bbg | *.map | *.inf | *.dSYM | *.o | *.obj ) ;;\n *.* ) ac_cv_exeext=`expr \"$ac_file\" : '[^.]*\\(\\..*\\)'`\n\t break;;\n * ) break;;\n esac\ndone\nelse $as_nop\n { { printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\nprintf \"%s\\n\" \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"cannot compute suffix of executables: cannot compile and link\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\nfi\nrm -f conftest conftest$ac_cv_exeext\n{ printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_exeext\" >&5\nprintf \"%s\\n\" \"$ac_cv_exeext\" >&6; }\n\nrm -f conftest.$ac_ext\nEXEEXT=$ac_cv_exeext\nac_exeext=$EXEEXT\ncat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h. */\n#include \nint\nmain (void)\n{\nFILE *f = fopen (\"conftest.out\", \"w\");\n return ferror (f) || fclose (f) != 0;\n\n ;\n return 0;\n}\n_ACEOF\nac_clean_files=\"$ac_clean_files conftest.out\"\n# Check that the compiler produces executables we can run. If not, either\n# the compiler is broken, or we cross compile.\n{ printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: checking whether we are cross compiling\" >&5\nprintf %s \"checking whether we are cross compiling... \" >&6; }\nif test \"$cross_compiling\" != yes; then\n { { ac_try=\"$ac_link\"\ncase \"(($ac_try\" in\n *\\\"* | *\\`* | *\\\\*) ac_try_echo=\\$ac_try;;\n *) ac_try_echo=$ac_try;;\nesac\neval ac_try_echo=\"\\\"\\$as_me:${as_lineno-$LINENO}: $ac_try_echo\\\"\"\nprintf \"%s\\n\" \"$ac_try_echo\"; } >&5\n (eval \"$ac_link\") 2>&5\n ac_status=$?\n printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: \\$? = $ac_status\" >&5\n test $ac_status = 0; }\n if { ac_try='./conftest$ac_cv_exeext'\n { { case \"(($ac_try\" in\n *\\\"* | *\\`* | *\\\\*) ac_try_echo=\\$ac_try;;\n *) ac_try_echo=$ac_try;;\nesac\neval ac_try_echo=\"\\\"\\$as_me:${as_lineno-$LINENO}: $ac_try_echo\\\"\"\nprintf \"%s\\n\" \"$ac_try_echo\"; } >&5\n (eval \"$ac_try\") 2>&5\n ac_status=$?\n printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: \\$? = $ac_status\" >&5\n test $ac_status = 0; }; }; then\n cross_compiling=no\n else\n if test \"$cross_compiling\" = maybe; then\n\tcross_compiling=yes\n else\n\t{ { printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\nprintf \"%s\\n\" \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error 77 \"cannot run C++ compiled programs.\nIf you meant to cross compile, use \\`--host'.\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\n fi\n fi\nfi\n{ printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: result: $cross_compiling\" >&5\nprintf \"%s\\n\" \"$cross_compiling\" >&6; }\n\nrm -f conftest.$ac_ext conftest$ac_cv_exeext conftest.out\nac_clean_files=$ac_clean_files_save\n{ printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: checking for suffix of object files\" >&5\nprintf %s \"checking for suffix of object files... \" >&6; }\nif test ${ac_cv_objext+y}\nthen :\n printf %s \"(cached) \" >&6\nelse $as_nop\n cat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h. */\n\nint\nmain (void)\n{\n\n ;\n return 0;\n}\n_ACEOF\nrm -f conftest.o conftest.obj\nif { { ac_try=\"$ac_compile\"\ncase \"(($ac_try\" in\n *\\\"* | *\\`* | *\\\\*) ac_try_echo=\\$ac_try;;\n *) ac_try_echo=$ac_try;;\nesac\neval ac_try_echo=\"\\\"\\$as_me:${as_lineno-$LINENO}: $ac_try_echo\\\"\"\nprintf \"%s\\n\" \"$ac_try_echo\"; } >&5\n (eval \"$ac_compile\") 2>&5\n ac_status=$?\n printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: \\$? = $ac_status\" >&5\n test $ac_status = 0; }\nthen :\n for ac_file in conftest.o conftest.obj conftest.*; do\n test -f \"$ac_file\" || continue;\n case $ac_file in\n *.$ac_ext | *.xcoff | *.tds | *.d | *.pdb | *.xSYM | *.bb | *.bbg | *.map | *.inf | *.dSYM ) ;;\n *) ac_cv_objext=`expr \"$ac_file\" : '.*\\.\\(.*\\)'`\n break;;\n esac\ndone\nelse $as_nop\n printf \"%s\\n\" \"$as_me: failed program was:\" >&5\nsed 's/^/| /' conftest.$ac_ext >&5\n\n{ { printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: error: in \\`$ac_pwd':\" >&5\nprintf \"%s\\n\" \"$as_me: error: in \\`$ac_pwd':\" >&2;}\nas_fn_error $? \"cannot compute suffix of object files: cannot compile\nSee \\`config.log' for more details\" \"$LINENO\" 5; }\nfi\nrm -f conftest.$ac_cv_objext conftest.$ac_ext\nfi\n{ printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_objext\" >&5\nprintf \"%s\\n\" \"$ac_cv_objext\" >&6; }\nOBJEXT=$ac_cv_objext\nac_objext=$OBJEXT\n{ printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: checking whether the compiler supports GNU C++\" >&5\nprintf %s \"checking whether the compiler supports GNU C++... \" >&6; }\nif test ${ac_cv_cxx_compiler_gnu+y}\nthen :\n printf %s \"(cached) \" >&6\nelse $as_nop\n cat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h. */\n\nint\nmain (void)\n{\n#ifndef __GNUC__\n choke me\n#endif\n\n ;\n return 0;\n}\n_ACEOF\nif ac_fn_cxx_try_compile \"$LINENO\"\nthen :\n ac_compiler_gnu=yes\nelse $as_nop\n ac_compiler_gnu=no\nfi\nrm -f core conftest.err conftest.$ac_objext conftest.beam conftest.$ac_ext\nac_cv_cxx_compiler_gnu=$ac_compiler_gnu\n\nfi\n{ printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_cxx_compiler_gnu\" >&5\nprintf \"%s\\n\" \"$ac_cv_cxx_compiler_gnu\" >&6; }\nac_compiler_gnu=$ac_cv_cxx_compiler_gnu\n\nif test $ac_compiler_gnu = yes; then\n GXX=yes\nelse\n GXX=\nfi\nac_test_CXXFLAGS=${CXXFLAGS+y}\nac_save_CXXFLAGS=$CXXFLAGS\n{ printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: checking whether $CXX accepts -g\" >&5\nprintf %s \"checking whether $CXX accepts -g... \" >&6; }\nif test ${ac_cv_prog_cxx_g+y}\nthen :\n printf %s \"(cached) \" >&6\nelse $as_nop\n ac_save_cxx_werror_flag=$ac_cxx_werror_flag\n ac_cxx_werror_flag=yes\n ac_cv_prog_cxx_g=no\n CXXFLAGS=\"-g\"\n cat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h. */\n\nint\nmain (void)\n{\n\n ;\n return 0;\n}\n_ACEOF\nif ac_fn_cxx_try_compile \"$LINENO\"\nthen :\n ac_cv_prog_cxx_g=yes\nelse $as_nop\n CXXFLAGS=\"\"\n cat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h. */\n\nint\nmain (void)\n{\n\n ;\n return 0;\n}\n_ACEOF\nif ac_fn_cxx_try_compile \"$LINENO\"\nthen :\n\nelse $as_nop\n ac_cxx_werror_flag=$ac_save_cxx_werror_flag\n\t CXXFLAGS=\"-g\"\n\t cat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h. */\n\nint\nmain (void)\n{\n\n ;\n return 0;\n}\n_ACEOF\nif ac_fn_cxx_try_compile \"$LINENO\"\nthen :\n ac_cv_prog_cxx_g=yes\nfi\nrm -f core conftest.err conftest.$ac_objext conftest.beam conftest.$ac_ext\nfi\nrm -f core conftest.err conftest.$ac_objext conftest.beam conftest.$ac_ext\nfi\nrm -f core conftest.err conftest.$ac_objext conftest.beam conftest.$ac_ext\n ac_cxx_werror_flag=$ac_save_cxx_werror_flag\nfi\n{ printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_prog_cxx_g\" >&5\nprintf \"%s\\n\" \"$ac_cv_prog_cxx_g\" >&6; }\nif test $ac_test_CXXFLAGS; then\n CXXFLAGS=$ac_save_CXXFLAGS\nelif test $ac_cv_prog_cxx_g = yes; then\n if test \"$GXX\" = yes; then\n CXXFLAGS=\"-g -O2\"\n else\n CXXFLAGS=\"-g\"\n fi\nelse\n if test \"$GXX\" = yes; then\n CXXFLAGS=\"-O2\"\n else\n CXXFLAGS=\n fi\nfi\nac_prog_cxx_stdcxx=no\nif test x$ac_prog_cxx_stdcxx = xno\nthen :\n { printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: checking for $CXX option to enable C++11 features\" >&5\nprintf %s \"checking for $CXX option to enable C++11 features... \" >&6; }\nif test ${ac_cv_prog_cxx_cxx11+y}\nthen :\n printf %s \"(cached) \" >&6\nelse $as_nop\n ac_cv_prog_cxx_cxx11=no\nac_save_CXX=$CXX\ncat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h. */\n$ac_cxx_conftest_cxx11_program\n_ACEOF\nfor ac_arg in '' -std=gnu++11 -std=gnu++0x -std=c++11 -std=c++0x -qlanglvl=extended0x -AA\ndo\n CXX=\"$ac_save_CXX $ac_arg\"\n if ac_fn_cxx_try_compile \"$LINENO\"\nthen :\n ac_cv_prog_cxx_cxx11=$ac_arg\nfi\nrm -f core conftest.err conftest.$ac_objext conftest.beam\n test \"x$ac_cv_prog_cxx_cxx11\" != \"xno\" && break\ndone\nrm -f conftest.$ac_ext\nCXX=$ac_save_CXX\nfi\n\nif test \"x$ac_cv_prog_cxx_cxx11\" = xno\nthen :\n { printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: result: unsupported\" >&5\nprintf \"%s\\n\" \"unsupported\" >&6; }\nelse $as_nop\n if test \"x$ac_cv_prog_cxx_cxx11\" = x\nthen :\n { printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: result: none needed\" >&5\nprintf \"%s\\n\" \"none needed\" >&6; }\nelse $as_nop\n { printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_prog_cxx_cxx11\" >&5\nprintf \"%s\\n\" \"$ac_cv_prog_cxx_cxx11\" >&6; }\n CXX=\"$CXX $ac_cv_prog_cxx_cxx11\"\nfi\n ac_cv_prog_cxx_stdcxx=$ac_cv_prog_cxx_cxx11\n ac_prog_cxx_stdcxx=cxx11\nfi\nfi\nif test x$ac_prog_cxx_stdcxx = xno\nthen :\n { printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: checking for $CXX option to enable C++98 features\" >&5\nprintf %s \"checking for $CXX option to enable C++98 features... \" >&6; }\nif test ${ac_cv_prog_cxx_cxx98+y}\nthen :\n printf %s \"(cached) \" >&6\nelse $as_nop\n ac_cv_prog_cxx_cxx98=no\nac_save_CXX=$CXX\ncat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h. */\n$ac_cxx_conftest_cxx98_program\n_ACEOF\nfor ac_arg in '' -std=gnu++98 -std=c++98 -qlanglvl=extended -AA\ndo\n CXX=\"$ac_save_CXX $ac_arg\"\n if ac_fn_cxx_try_compile \"$LINENO\"\nthen :\n ac_cv_prog_cxx_cxx98=$ac_arg\nfi\nrm -f core conftest.err conftest.$ac_objext conftest.beam\n test \"x$ac_cv_prog_cxx_cxx98\" != \"xno\" && break\ndone\nrm -f conftest.$ac_ext\nCXX=$ac_save_CXX\nfi\n\nif test \"x$ac_cv_prog_cxx_cxx98\" = xno\nthen :\n { printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: result: unsupported\" >&5\nprintf \"%s\\n\" \"unsupported\" >&6; }\nelse $as_nop\n if test \"x$ac_cv_prog_cxx_cxx98\" = x\nthen :\n { printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: result: none needed\" >&5\nprintf \"%s\\n\" \"none needed\" >&6; }\nelse $as_nop\n { printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_prog_cxx_cxx98\" >&5\nprintf \"%s\\n\" \"$ac_cv_prog_cxx_cxx98\" >&6; }\n CXX=\"$CXX $ac_cv_prog_cxx_cxx98\"\nfi\n ac_cv_prog_cxx_stdcxx=$ac_cv_prog_cxx_cxx98\n ac_prog_cxx_stdcxx=cxx98\nfi\nfi\n\nac_ext=cpp\nac_cpp='$CXXCPP $CPPFLAGS'\nac_compile='$CXX -c $CXXFLAGS $CPPFLAGS conftest.$ac_ext >&5'\nac_link='$CXX -o conftest$ac_exeext $CXXFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5'\nac_compiler_gnu=$ac_cv_cxx_compiler_gnu\n\n\n{ printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: checking for backtrace in -lexecinfo\" >&5\nprintf %s \"checking for backtrace in -lexecinfo... \" >&6; }\nif test ${ac_cv_lib_execinfo_backtrace+y}\nthen :\n printf %s \"(cached) \" >&6\nelse $as_nop\n ac_check_lib_save_LIBS=$LIBS\nLIBS=\"-lexecinfo $LIBS\"\ncat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h. */\n\nnamespace conftest {\n extern \"C\" int backtrace ();\n}\nint\nmain (void)\n{\nreturn conftest::backtrace ();\n ;\n return 0;\n}\n_ACEOF\nif ac_fn_cxx_try_link \"$LINENO\"\nthen :\n ac_cv_lib_execinfo_backtrace=yes\nelse $as_nop\n ac_cv_lib_execinfo_backtrace=no\nfi\nrm -f core conftest.err conftest.$ac_objext conftest.beam \\\n conftest$ac_exeext conftest.$ac_ext\nLIBS=$ac_check_lib_save_LIBS\nfi\n{ printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_execinfo_backtrace\" >&5\nprintf \"%s\\n\" \"$ac_cv_lib_execinfo_backtrace\" >&6; }\nif test \"x$ac_cv_lib_execinfo_backtrace\" = xyes\nthen :\n BACKTRACE_LIB=-lexecinfo\nelse $as_nop\n BACKTRACE_LIB=''\nfi\n\nif test -z \"${BACKTRACE_LIB}\"\nthen :\n ac_fn_cxx_check_func \"$LINENO\" \"backtrace\" \"ac_cv_func_backtrace\"\nif test \"x$ac_cv_func_backtrace\" = xyes\nthen :\n\nelse $as_nop\n DMLC_DEFS=\"$DMLC_DEFS -DDMLC_LOG_STACK_TRACE=0\"\nfi\n\nfi\n\n{ printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: Checking whether fopen64 is available\" >&5\nprintf \"%s\\n\" \"$as_me: Checking whether fopen64 is available\" >&6;}\nac_fn_cxx_check_func \"$LINENO\" \"fopen64\" \"ac_cv_func_fopen64\"\nif test \"x$ac_cv_func_fopen64\" = xyes\nthen :\n\nelse $as_nop\n DMLC_DEFS=\"$DMLC_DEFS -DDMLC_USE_FOPEN64=0\"\nfi\n\n\n{ printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: Endian detection\" >&5\nprintf \"%s\\n\" \"$as_me: Endian detection\" >&6;}\n\nac_header= ac_cache=\nfor ac_item in $ac_header_cxx_list\ndo\n if test $ac_cache; then\n ac_fn_cxx_check_header_compile \"$LINENO\" $ac_header ac_cv_header_$ac_cache \"$ac_includes_default\"\n if eval test \\\"x\\$ac_cv_header_$ac_cache\\\" = xyes; then\n printf \"%s\\n\" \"#define $ac_item 1\" >> confdefs.h\n fi\n ac_header= ac_cache=\n elif test $ac_header; then\n ac_cache=$ac_item\n else\n ac_header=$ac_item\n fi\ndone\n\n\n\n\n\n\n\n\nif test $ac_cv_header_stdlib_h = yes && test $ac_cv_header_string_h = yes\nthen :\n\nprintf \"%s\\n\" \"#define STDC_HEADERS 1\" >>confdefs.h\n\nfi\nif test -z \"${USE_LITTLE_ENDIAN+x}\"\nthen :\n\n { printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: Checking system endianness as USE_LITTLE_ENDIAN is unset\" >&5\nprintf \"%s\\n\" \"$as_me: Checking system endianness as USE_LITTLE_ENDIAN is unset\" >&6;}\n { printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: checking whether byte ordering is bigendian\" >&5\nprintf %s \"checking whether byte ordering is bigendian... \" >&6; }\nif test ${ac_cv_c_bigendian+y}\nthen :\n printf %s \"(cached) \" >&6\nelse $as_nop\n ac_cv_c_bigendian=unknown\n # See if we're dealing with a universal compiler.\n cat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h. */\n#ifndef __APPLE_CC__\n\t not a universal capable compiler\n\t #endif\n\t typedef int dummy;\n\n_ACEOF\nif ac_fn_cxx_try_compile \"$LINENO\"\nthen :\n\n\t# Check for potential -arch flags. It is not universal unless\n\t# there are at least two -arch flags with different values.\n\tac_arch=\n\tac_prev=\n\tfor ac_word in $CC $CFLAGS $CPPFLAGS $LDFLAGS; do\n\t if test -n \"$ac_prev\"; then\n\t case $ac_word in\n\t i?86 | x86_64 | ppc | ppc64)\n\t if test -z \"$ac_arch\" || test \"$ac_arch\" = \"$ac_word\"; then\n\t\t ac_arch=$ac_word\n\t else\n\t\t ac_cv_c_bigendian=universal\n\t\t break\n\t fi\n\t ;;\n\t esac\n\t ac_prev=\n\t elif test \"x$ac_word\" = \"x-arch\"; then\n\t ac_prev=arch\n\t fi\n done\nfi\nrm -f core conftest.err conftest.$ac_objext conftest.beam conftest.$ac_ext\n if test $ac_cv_c_bigendian = unknown; then\n # See if sys/param.h defines the BYTE_ORDER macro.\n cat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h. */\n#include \n\t #include \n\nint\nmain (void)\n{\n#if ! (defined BYTE_ORDER && defined BIG_ENDIAN \\\n\t\t && defined LITTLE_ENDIAN && BYTE_ORDER && BIG_ENDIAN \\\n\t\t && LITTLE_ENDIAN)\n\t bogus endian macros\n\t #endif\n\n ;\n return 0;\n}\n_ACEOF\nif ac_fn_cxx_try_compile \"$LINENO\"\nthen :\n # It does; now see whether it defined to BIG_ENDIAN or not.\n\t cat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h. */\n#include \n\t\t#include \n\nint\nmain (void)\n{\n#if BYTE_ORDER != BIG_ENDIAN\n\t\t not big endian\n\t\t#endif\n\n ;\n return 0;\n}\n_ACEOF\nif ac_fn_cxx_try_compile \"$LINENO\"\nthen :\n ac_cv_c_bigendian=yes\nelse $as_nop\n ac_cv_c_bigendian=no\nfi\nrm -f core conftest.err conftest.$ac_objext conftest.beam conftest.$ac_ext\nfi\nrm -f core conftest.err conftest.$ac_objext conftest.beam conftest.$ac_ext\n fi\n if test $ac_cv_c_bigendian = unknown; then\n # See if defines _LITTLE_ENDIAN or _BIG_ENDIAN (e.g., Solaris).\n cat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h. */\n#include \n\nint\nmain (void)\n{\n#if ! (defined _LITTLE_ENDIAN || defined _BIG_ENDIAN)\n\t bogus endian macros\n\t #endif\n\n ;\n return 0;\n}\n_ACEOF\nif ac_fn_cxx_try_compile \"$LINENO\"\nthen :\n # It does; now see whether it defined to _BIG_ENDIAN or not.\n\t cat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h. */\n#include \n\nint\nmain (void)\n{\n#ifndef _BIG_ENDIAN\n\t\t not big endian\n\t\t#endif\n\n ;\n return 0;\n}\n_ACEOF\nif ac_fn_cxx_try_compile \"$LINENO\"\nthen :\n ac_cv_c_bigendian=yes\nelse $as_nop\n ac_cv_c_bigendian=no\nfi\nrm -f core conftest.err conftest.$ac_objext conftest.beam conftest.$ac_ext\nfi\nrm -f core conftest.err conftest.$ac_objext conftest.beam conftest.$ac_ext\n fi\n if test $ac_cv_c_bigendian = unknown; then\n # Compile a test program.\n if test \"$cross_compiling\" = yes\nthen :\n # Try to guess by grepping values from an object file.\n\t cat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h. */\nunsigned short int ascii_mm[] =\n\t\t { 0x4249, 0x4765, 0x6E44, 0x6961, 0x6E53, 0x7953, 0 };\n\t\tunsigned short int ascii_ii[] =\n\t\t { 0x694C, 0x5454, 0x656C, 0x6E45, 0x6944, 0x6E61, 0 };\n\t\tint use_ascii (int i) {\n\t\t return ascii_mm[i] + ascii_ii[i];\n\t\t}\n\t\tunsigned short int ebcdic_ii[] =\n\t\t { 0x89D3, 0xE3E3, 0x8593, 0x95C5, 0x89C4, 0x9581, 0 };\n\t\tunsigned short int ebcdic_mm[] =\n\t\t { 0xC2C9, 0xC785, 0x95C4, 0x8981, 0x95E2, 0xA8E2, 0 };\n\t\tint use_ebcdic (int i) {\n\t\t return ebcdic_mm[i] + ebcdic_ii[i];\n\t\t}\n\t\textern int foo;\n\nint\nmain (void)\n{\nreturn use_ascii (foo) == use_ebcdic (foo);\n ;\n return 0;\n}\n_ACEOF\nif ac_fn_cxx_try_compile \"$LINENO\"\nthen :\n if grep BIGenDianSyS conftest.$ac_objext >/dev/null; then\n\t ac_cv_c_bigendian=yes\n\t fi\n\t if grep LiTTleEnDian conftest.$ac_objext >/dev/null ; then\n\t if test \"$ac_cv_c_bigendian\" = unknown; then\n\t\tac_cv_c_bigendian=no\n\t else\n\t\t# finding both strings is unlikely to happen, but who knows?\n\t\tac_cv_c_bigendian=unknown\n\t fi\n\t fi\nfi\nrm -f core conftest.err conftest.$ac_objext conftest.beam conftest.$ac_ext\nelse $as_nop\n cat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h. */\n$ac_includes_default\nint\nmain (void)\n{\n\n\t /* Are we little or big endian? From Harbison&Steele. */\n\t union\n\t {\n\t long int l;\n\t char c[sizeof (long int)];\n\t } u;\n\t u.l = 1;\n\t return u.c[sizeof (long int) - 1] == 1;\n\n ;\n return 0;\n}\n_ACEOF\nif ac_fn_cxx_try_run \"$LINENO\"\nthen :\n ac_cv_c_bigendian=no\nelse $as_nop\n ac_cv_c_bigendian=yes\nfi\nrm -f core *.core core.conftest.* gmon.out bb.out conftest$ac_exeext \\\n conftest.$ac_objext conftest.beam conftest.$ac_ext\nfi\n\n fi\nfi\n{ printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: result: $ac_cv_c_bigendian\" >&5\nprintf \"%s\\n\" \"$ac_cv_c_bigendian\" >&6; }\n case $ac_cv_c_bigendian in #(\n yes)\n { printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: result: using big endian\" >&5\nprintf \"%s\\n\" \"using big endian\" >&6; }\n ENDIAN_FLAG=\"-DDMLC_CMAKE_LITTLE_ENDIAN=0\";; #(\n no)\n { printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: result: using little endian\" >&5\nprintf \"%s\\n\" \"using little endian\" >&6; }\n ENDIAN_FLAG=\"-DDMLC_CMAKE_LITTLE_ENDIAN=1\" ;; #(\n universal)\n\nprintf \"%s\\n\" \"#define AC_APPLE_UNIVERSAL_BUILD 1\" >>confdefs.h\n\n ;; #(\n *)\n { printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: result: unknown\" >&5\nprintf \"%s\\n\" \"unknown\" >&6; }\n as_fn_error $? \"Could not determine endianness. Please set USE_LITTLE_ENDIAN\" \"$LINENO\" 5\n ;;\n esac\n\n\nelse $as_nop\n\n { printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: Forcing endianness to: ${USE_LITTLE_ENDIAN}\" >&5\nprintf \"%s\\n\" \"$as_me: Forcing endianness to: ${USE_LITTLE_ENDIAN}\" >&6;}\n ENDIAN_FLAG=\"-DDMLC_CMAKE_LITTLE_ENDIAN=${USE_LITTLE_ENDIAN}\"\n\nfi\n\n{ printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: Checking for prefetch builtin\" >&5\nprintf \"%s\\n\" \"$as_me: Checking for prefetch builtin\" >&6;}\ncat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h. */\n\nint\nmain (void)\n{\n__builtin_prefetch\n\n ;\n return 0;\n}\n_ACEOF\nif ac_fn_cxx_try_link \"$LINENO\"\nthen :\n XGBOOST_BUILTIN_PREFETCH_PRESENT=\"-DXGBOOST_BUILTIN_PREFETCH_PRESENT=1\"\nelse $as_nop\n XGBOOST_BUILTIN_PREFETCH_PRESENT=\"\"\n\nfi\nrm -f core conftest.err conftest.$ac_objext conftest.beam \\\n conftest$ac_exeext conftest.$ac_ext\nif [ \"$XGBOOST_BUILTIN_PREFETCH_PRESENT\" = \"\" ]; then\n echo \"Has __builtin_prefetch\"\nelse\n echo \"Doesn't have __builtin_prefetch\"\nfi\n\n{ printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: Checking for mm_prefetch\" >&5\nprintf \"%s\\n\" \"$as_me: Checking for mm_prefetch\" >&6;}\ncat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h. */\n#include \nint\nmain (void)\n{\n_mm_prefetch\n\n ;\n return 0;\n}\n_ACEOF\nif ac_fn_cxx_try_link \"$LINENO\"\nthen :\n XGBOOST_MM_PREFETCH_PRESENT=\"-DXGBOOST_MM_PREFETCH_PRESENT=1\"\nelse $as_nop\n XGBOOST_MM_PREFETCH_PRESENT=\"\"\n\nfi\nrm -f core conftest.err conftest.$ac_objext conftest.beam \\\n conftest$ac_exeext conftest.$ac_ext\nif [ \"$XGBOOST_MM_PREFETCH_PRESENT\" = \"\" ]; then\n echo \"Has _mm_prefetch\"\nelse\n echo \"Doesn't have _mm_prefetch\"\nfi\n\nOPENMP_CXXFLAGS=\"\"\n\nif test `uname -s` = \"Linux\"\nthen\n OPENMP_CXXFLAGS=\"\\$(SHLIB_OPENMP_CXXFLAGS)\"\nfi\n\nif test `uname -s` = \"Darwin\"\nthen\n if command -v brew &> /dev/null\n then\n HOMEBREW_LIBOMP_PREFIX=`brew --prefix libomp`\n else\n # Homebrew not found\n HOMEBREW_LIBOMP_PREFIX=''\n fi\n OPENMP_CXXFLAGS=\"-Xpreprocessor -fopenmp -I${HOMEBREW_LIBOMP_PREFIX}/include\"\n OPENMP_LIB=\"-lomp -L${HOMEBREW_LIBOMP_PREFIX}/lib\"\n ac_pkg_openmp=no\n { printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: checking whether OpenMP will work in a package\" >&5\nprintf %s \"checking whether OpenMP will work in a package... \" >&6; }\n cat confdefs.h - <<_ACEOF >conftest.$ac_ext\n/* end confdefs.h. */\n#include \nint\nmain (void)\n{\n return (omp_get_max_threads() <= 1);\n ;\n return 0;\n}\n_ACEOF\n ${CXX} -o conftest conftest.cpp ${CPPFLAGS} ${LDFLAGS} ${OPENMP_LIB} ${OPENMP_CXXFLAGS} 2>/dev/null && ./conftest && ac_pkg_openmp=yes\n { printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: result: ${ac_pkg_openmp}\" >&5\nprintf \"%s\\n\" \"${ac_pkg_openmp}\" >&6; }\n if test \"${ac_pkg_openmp}\" = no; then\n OPENMP_CXXFLAGS=''\n OPENMP_LIB=''\n echo '*****************************************************************************************'\n echo ' OpenMP is unavailable on this Mac OSX system. Training speed may be suboptimal.'\n echo ' To use all CPU cores for training jobs, you should install OpenMP by running'\n echo ' brew install libomp'\n echo '*****************************************************************************************'\n fi\nfi\n\n\n\n\n\n\n\n\nac_config_files=\"$ac_config_files src/Makevars\"\n\nac_config_headers=\"$ac_config_headers src/config.h\"\n\ncat >confcache <<\\_ACEOF\n# This file is a shell script that caches the results of configure\n# tests run on this system so they can be shared between configure\n# scripts and configure runs, see configure's option --config-cache.\n# It is not useful on other systems. If it contains results you don't\n# want to keep, you may remove or edit it.\n#\n# config.status only pays attention to the cache file if you give it\n# the --recheck option to rerun configure.\n#\n# `ac_cv_env_foo' variables (set or unset) will be overridden when\n# loading this file, other *unset* `ac_cv_foo' will be assigned the\n# following values.\n\n_ACEOF\n\n# The following way of writing the cache mishandles newlines in values,\n# but we know of no workaround that is simple, portable, and efficient.\n# So, we kill variables containing newlines.\n# Ultrix sh set writes to stderr and can't be redirected directly,\n# and sets the high bit in the cache file unless we assign to the vars.\n(\n for ac_var in `(set) 2>&1 | sed -n 's/^\\([a-zA-Z_][a-zA-Z0-9_]*\\)=.*/\\1/p'`; do\n eval ac_val=\\$$ac_var\n case $ac_val in #(\n *${as_nl}*)\n case $ac_var in #(\n *_cv_*) { printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: WARNING: cache variable $ac_var contains a newline\" >&5\nprintf \"%s\\n\" \"$as_me: WARNING: cache variable $ac_var contains a newline\" >&2;} ;;\n esac\n case $ac_var in #(\n _ | IFS | as_nl) ;; #(\n BASH_ARGV | BASH_SOURCE) eval $ac_var= ;; #(\n *) { eval $ac_var=; unset $ac_var;} ;;\n esac ;;\n esac\n done\n\n (set) 2>&1 |\n case $as_nl`(ac_space=' '; set) 2>&1` in #(\n *${as_nl}ac_space=\\ *)\n # `set' does not quote correctly, so add quotes: double-quote\n # substitution turns \\\\\\\\ into \\\\, and sed turns \\\\ into \\.\n sed -n \\\n\t\"s/'/'\\\\\\\\''/g;\n\t s/^\\\\([_$as_cr_alnum]*_cv_[_$as_cr_alnum]*\\\\)=\\\\(.*\\\\)/\\\\1='\\\\2'/p\"\n ;; #(\n *)\n # `set' quotes correctly as required by POSIX, so do not add quotes.\n sed -n \"/^[_$as_cr_alnum]*_cv_[_$as_cr_alnum]*=/p\"\n ;;\n esac |\n sort\n) |\n sed '\n /^ac_cv_env_/b end\n t clear\n :clear\n s/^\\([^=]*\\)=\\(.*[{}].*\\)$/test ${\\1+y} || &/\n t end\n s/^\\([^=]*\\)=\\(.*\\)$/\\1=${\\1=\\2}/\n :end' >>confcache\nif diff \"$cache_file\" confcache >/dev/null 2>&1; then :; else\n if test -w \"$cache_file\"; then\n if test \"x$cache_file\" != \"x/dev/null\"; then\n { printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: updating cache $cache_file\" >&5\nprintf \"%s\\n\" \"$as_me: updating cache $cache_file\" >&6;}\n if test ! -f \"$cache_file\" || test -h \"$cache_file\"; then\n\tcat confcache >\"$cache_file\"\n else\n case $cache_file in #(\n */* | ?:*)\n\t mv -f confcache \"$cache_file\"$$ &&\n\t mv -f \"$cache_file\"$$ \"$cache_file\" ;; #(\n *)\n\t mv -f confcache \"$cache_file\" ;;\n\tesac\n fi\n fi\n else\n { printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: not updating unwritable cache $cache_file\" >&5\nprintf \"%s\\n\" \"$as_me: not updating unwritable cache $cache_file\" >&6;}\n fi\nfi\nrm -f confcache\n\ntest \"x$prefix\" = xNONE && prefix=$ac_default_prefix\n# Let make expand exec_prefix.\ntest \"x$exec_prefix\" = xNONE && exec_prefix='${prefix}'\n\nDEFS=-DHAVE_CONFIG_H\n\nac_libobjs=\nac_ltlibobjs=\nU=\nfor ac_i in : $LIBOBJS; do test \"x$ac_i\" = x: && continue\n # 1. Remove the extension, and $U if already installed.\n ac_script='s/\\$U\\././;s/\\.o$//;s/\\.obj$//'\n ac_i=`printf \"%s\\n\" \"$ac_i\" | sed \"$ac_script\"`\n # 2. Prepend LIBOBJDIR. When used with automake>=1.10 LIBOBJDIR\n # will be set to the directory where LIBOBJS objects are built.\n as_fn_append ac_libobjs \" \\${LIBOBJDIR}$ac_i\\$U.$ac_objext\"\n as_fn_append ac_ltlibobjs \" \\${LIBOBJDIR}$ac_i\"'$U.lo'\ndone\nLIBOBJS=$ac_libobjs\n\nLTLIBOBJS=$ac_ltlibobjs\n\n\n\n\n: \"${CONFIG_STATUS=./config.status}\"\nac_write_fail=0\nac_clean_files_save=$ac_clean_files\nac_clean_files=\"$ac_clean_files $CONFIG_STATUS\"\n{ printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: creating $CONFIG_STATUS\" >&5\nprintf \"%s\\n\" \"$as_me: creating $CONFIG_STATUS\" >&6;}\nas_write_fail=0\ncat >$CONFIG_STATUS <<_ASEOF || as_write_fail=1\n#! $SHELL\n# Generated by $as_me.\n# Run this file to recreate the current configuration.\n# Compiler output produced by configure, useful for debugging\n# configure, is in config.log if it exists.\n\ndebug=false\nac_cs_recheck=false\nac_cs_silent=false\n\nSHELL=\\${CONFIG_SHELL-$SHELL}\nexport SHELL\n_ASEOF\ncat >>$CONFIG_STATUS <<\\_ASEOF || as_write_fail=1\n## -------------------- ##\n## M4sh Initialization. ##\n## -------------------- ##\n\n# Be more Bourne compatible\nDUALCASE=1; export DUALCASE # for MKS sh\nas_nop=:\nif test ${ZSH_VERSION+y} && (emulate sh) >/dev/null 2>&1\nthen :\n emulate sh\n NULLCMD=:\n # Pre-4.2 versions of Zsh do word splitting on ${1+\"$@\"}, which\n # is contrary to our usage. Disable this feature.\n alias -g '${1+\"$@\"}'='\"$@\"'\n setopt NO_GLOB_SUBST\nelse $as_nop\n case `(set -o) 2>/dev/null` in #(\n *posix*) :\n set -o posix ;; #(\n *) :\n ;;\nesac\nfi\n\n\n\n# Reset variables that may have inherited troublesome values from\n# the environment.\n\n# IFS needs to be set, to space, tab, and newline, in precisely that order.\n# (If _AS_PATH_WALK were called with IFS unset, it would have the\n# side effect of setting IFS to empty, thus disabling word splitting.)\n# Quoting is to prevent editors from complaining about space-tab.\nas_nl='\n'\nexport as_nl\nIFS=\" \"\"\t$as_nl\"\n\nPS1='$ '\nPS2='> '\nPS4='+ '\n\n# Ensure predictable behavior from utilities with locale-dependent output.\nLC_ALL=C\nexport LC_ALL\nLANGUAGE=C\nexport LANGUAGE\n\n# We cannot yet rely on \"unset\" to work, but we need these variables\n# to be unset--not just set to an empty or harmless value--now, to\n# avoid bugs in old shells (e.g. pre-3.0 UWIN ksh). This construct\n# also avoids known problems related to \"unset\" and subshell syntax\n# in other old shells (e.g. bash 2.01 and pdksh 5.2.14).\nfor as_var in BASH_ENV ENV MAIL MAILPATH CDPATH\ndo eval test \\${$as_var+y} \\\n && ( (unset $as_var) || exit 1) >/dev/null 2>&1 && unset $as_var || :\ndone\n\n# Ensure that fds 0, 1, and 2 are open.\nif (exec 3>&0) 2>/dev/null; then :; else exec 0&1) 2>/dev/null; then :; else exec 1>/dev/null; fi\nif (exec 3>&2) ; then :; else exec 2>/dev/null; fi\n\n# The user is always right.\nif ${PATH_SEPARATOR+false} :; then\n PATH_SEPARATOR=:\n (PATH='/bin;/bin'; FPATH=$PATH; sh -c :) >/dev/null 2>&1 && {\n (PATH='/bin:/bin'; FPATH=$PATH; sh -c :) >/dev/null 2>&1 ||\n PATH_SEPARATOR=';'\n }\nfi\n\n\n# Find who we are. Look in the path if we contain no directory separator.\nas_myself=\ncase $0 in #((\n *[\\\\/]* ) as_myself=$0 ;;\n *) as_save_IFS=$IFS; IFS=$PATH_SEPARATOR\nfor as_dir in $PATH\ndo\n IFS=$as_save_IFS\n case $as_dir in #(((\n '') as_dir=./ ;;\n */) ;;\n *) as_dir=$as_dir/ ;;\n esac\n test -r \"$as_dir$0\" && as_myself=$as_dir$0 && break\n done\nIFS=$as_save_IFS\n\n ;;\nesac\n# We did not find ourselves, most probably we were run as `sh COMMAND'\n# in which case we are not to be found in the path.\nif test \"x$as_myself\" = x; then\n as_myself=$0\nfi\nif test ! -f \"$as_myself\"; then\n printf \"%s\\n\" \"$as_myself: error: cannot find myself; rerun with an absolute file name\" >&2\n exit 1\nfi\n\n\n\n# as_fn_error STATUS ERROR [LINENO LOG_FD]\n# ----------------------------------------\n# Output \"`basename $0`: error: ERROR\" to stderr. If LINENO and LOG_FD are\n# provided, also output the error to LOG_FD, referencing LINENO. Then exit the\n# script with STATUS, using 1 if that was 0.\nas_fn_error ()\n{\n as_status=$1; test $as_status -eq 0 && as_status=1\n if test \"$4\"; then\n as_lineno=${as_lineno-\"$3\"} as_lineno_stack=as_lineno_stack=$as_lineno_stack\n printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: error: $2\" >&$4\n fi\n printf \"%s\\n\" \"$as_me: error: $2\" >&2\n as_fn_exit $as_status\n} # as_fn_error\n\n\n\n# as_fn_set_status STATUS\n# -----------------------\n# Set $? to STATUS, without forking.\nas_fn_set_status ()\n{\n return $1\n} # as_fn_set_status\n\n# as_fn_exit STATUS\n# -----------------\n# Exit the shell with STATUS, even in a \"trap 0\" or \"set -e\" context.\nas_fn_exit ()\n{\n set +e\n as_fn_set_status $1\n exit $1\n} # as_fn_exit\n\n# as_fn_unset VAR\n# ---------------\n# Portably unset VAR.\nas_fn_unset ()\n{\n { eval $1=; unset $1;}\n}\nas_unset=as_fn_unset\n\n# as_fn_append VAR VALUE\n# ----------------------\n# Append the text in VALUE to the end of the definition contained in VAR. Take\n# advantage of any shell optimizations that allow amortized linear growth over\n# repeated appends, instead of the typical quadratic growth present in naive\n# implementations.\nif (eval \"as_var=1; as_var+=2; test x\\$as_var = x12\") 2>/dev/null\nthen :\n eval 'as_fn_append ()\n {\n eval $1+=\\$2\n }'\nelse $as_nop\n as_fn_append ()\n {\n eval $1=\\$$1\\$2\n }\nfi # as_fn_append\n\n# as_fn_arith ARG...\n# ------------------\n# Perform arithmetic evaluation on the ARGs, and store the result in the\n# global $as_val. Take advantage of shells that can avoid forks. The arguments\n# must be portable across $(()) and expr.\nif (eval \"test \\$(( 1 + 1 )) = 2\") 2>/dev/null\nthen :\n eval 'as_fn_arith ()\n {\n as_val=$(( $* ))\n }'\nelse $as_nop\n as_fn_arith ()\n {\n as_val=`expr \"$@\" || test $? -eq 1`\n }\nfi # as_fn_arith\n\n\nif expr a : '\\(a\\)' >/dev/null 2>&1 &&\n test \"X`expr 00001 : '.*\\(...\\)'`\" = X001; then\n as_expr=expr\nelse\n as_expr=false\nfi\n\nif (basename -- /) >/dev/null 2>&1 && test \"X`basename -- / 2>&1`\" = \"X/\"; then\n as_basename=basename\nelse\n as_basename=false\nfi\n\nif (as_dir=`dirname -- /` && test \"X$as_dir\" = X/) >/dev/null 2>&1; then\n as_dirname=dirname\nelse\n as_dirname=false\nfi\n\nas_me=`$as_basename -- \"$0\" ||\n$as_expr X/\"$0\" : '.*/\\([^/][^/]*\\)/*$' \\| \\\n\t X\"$0\" : 'X\\(//\\)$' \\| \\\n\t X\"$0\" : 'X\\(/\\)' \\| . 2>/dev/null ||\nprintf \"%s\\n\" X/\"$0\" |\n sed '/^.*\\/\\([^/][^/]*\\)\\/*$/{\n\t s//\\1/\n\t q\n\t }\n\t /^X\\/\\(\\/\\/\\)$/{\n\t s//\\1/\n\t q\n\t }\n\t /^X\\/\\(\\/\\).*/{\n\t s//\\1/\n\t q\n\t }\n\t s/.*/./; q'`\n\n# Avoid depending upon Character Ranges.\nas_cr_letters='abcdefghijklmnopqrstuvwxyz'\nas_cr_LETTERS='ABCDEFGHIJKLMNOPQRSTUVWXYZ'\nas_cr_Letters=$as_cr_letters$as_cr_LETTERS\nas_cr_digits='0123456789'\nas_cr_alnum=$as_cr_Letters$as_cr_digits\n\n\n# Determine whether it's possible to make 'echo' print without a newline.\n# These variables are no longer used directly by Autoconf, but are AC_SUBSTed\n# for compatibility with existing Makefiles.\nECHO_C= ECHO_N= ECHO_T=\ncase `echo -n x` in #(((((\n-n*)\n case `echo 'xy\\c'` in\n *c*) ECHO_T='\t';;\t# ECHO_T is single tab character.\n xy) ECHO_C='\\c';;\n *) echo `echo ksh88 bug on AIX 6.1` > /dev/null\n ECHO_T='\t';;\n esac;;\n*)\n ECHO_N='-n';;\nesac\n\n# For backward compatibility with old third-party macros, we provide\n# the shell variables $as_echo and $as_echo_n. New code should use\n# AS_ECHO([\"message\"]) and AS_ECHO_N([\"message\"]), respectively.\nas_echo='printf %s\\n'\nas_echo_n='printf %s'\n\nrm -f conf$$ conf$$.exe conf$$.file\nif test -d conf$$.dir; then\n rm -f conf$$.dir/conf$$.file\nelse\n rm -f conf$$.dir\n mkdir conf$$.dir 2>/dev/null\nfi\nif (echo >conf$$.file) 2>/dev/null; then\n if ln -s conf$$.file conf$$ 2>/dev/null; then\n as_ln_s='ln -s'\n # ... but there are two gotchas:\n # 1) On MSYS, both `ln -s file dir' and `ln file dir' fail.\n # 2) DJGPP < 2.04 has no symlinks; `ln -s' creates a wrapper executable.\n # In both cases, we have to default to `cp -pR'.\n ln -s conf$$.file conf$$.dir 2>/dev/null && test ! -f conf$$.exe ||\n as_ln_s='cp -pR'\n elif ln conf$$.file conf$$ 2>/dev/null; then\n as_ln_s=ln\n else\n as_ln_s='cp -pR'\n fi\nelse\n as_ln_s='cp -pR'\nfi\nrm -f conf$$ conf$$.exe conf$$.dir/conf$$.file conf$$.file\nrmdir conf$$.dir 2>/dev/null\n\n\n# as_fn_mkdir_p\n# -------------\n# Create \"$as_dir\" as a directory, including parents if necessary.\nas_fn_mkdir_p ()\n{\n\n case $as_dir in #(\n -*) as_dir=./$as_dir;;\n esac\n test -d \"$as_dir\" || eval $as_mkdir_p || {\n as_dirs=\n while :; do\n case $as_dir in #(\n *\\'*) as_qdir=`printf \"%s\\n\" \"$as_dir\" | sed \"s/'/'\\\\\\\\\\\\\\\\''/g\"`;; #'(\n *) as_qdir=$as_dir;;\n esac\n as_dirs=\"'$as_qdir' $as_dirs\"\n as_dir=`$as_dirname -- \"$as_dir\" ||\n$as_expr X\"$as_dir\" : 'X\\(.*[^/]\\)//*[^/][^/]*/*$' \\| \\\n\t X\"$as_dir\" : 'X\\(//\\)[^/]' \\| \\\n\t X\"$as_dir\" : 'X\\(//\\)$' \\| \\\n\t X\"$as_dir\" : 'X\\(/\\)' \\| . 2>/dev/null ||\nprintf \"%s\\n\" X\"$as_dir\" |\n sed '/^X\\(.*[^/]\\)\\/\\/*[^/][^/]*\\/*$/{\n\t s//\\1/\n\t q\n\t }\n\t /^X\\(\\/\\/\\)[^/].*/{\n\t s//\\1/\n\t q\n\t }\n\t /^X\\(\\/\\/\\)$/{\n\t s//\\1/\n\t q\n\t }\n\t /^X\\(\\/\\).*/{\n\t s//\\1/\n\t q\n\t }\n\t s/.*/./; q'`\n test -d \"$as_dir\" && break\n done\n test -z \"$as_dirs\" || eval \"mkdir $as_dirs\"\n } || test -d \"$as_dir\" || as_fn_error $? \"cannot create directory $as_dir\"\n\n\n} # as_fn_mkdir_p\nif mkdir -p . 2>/dev/null; then\n as_mkdir_p='mkdir -p \"$as_dir\"'\nelse\n test -d ./-p && rmdir ./-p\n as_mkdir_p=false\nfi\n\n\n# as_fn_executable_p FILE\n# -----------------------\n# Test if FILE is an executable regular file.\nas_fn_executable_p ()\n{\n test -f \"$1\" && test -x \"$1\"\n} # as_fn_executable_p\nas_test_x='test -x'\nas_executable_p=as_fn_executable_p\n\n# Sed expression to map a string onto a valid CPP name.\nas_tr_cpp=\"eval sed 'y%*$as_cr_letters%P$as_cr_LETTERS%;s%[^_$as_cr_alnum]%_%g'\"\n\n# Sed expression to map a string onto a valid variable name.\nas_tr_sh=\"eval sed 'y%*+%pp%;s%[^_$as_cr_alnum]%_%g'\"\n\n\nexec 6>&1\n## ----------------------------------- ##\n## Main body of $CONFIG_STATUS script. ##\n## ----------------------------------- ##\n_ASEOF\ntest $as_write_fail = 0 && chmod +x $CONFIG_STATUS || ac_write_fail=1\n\ncat >>$CONFIG_STATUS <<\\_ACEOF || ac_write_fail=1\n# Save the log message, to keep $0 and so on meaningful, and to\n# report actual input values of CONFIG_FILES etc. instead of their\n# values after options handling.\nac_log=\"\nThis file was extended by xgboost $as_me 3.3.0, which was\ngenerated by GNU Autoconf 2.71. Invocation command line was\n\n CONFIG_FILES = $CONFIG_FILES\n CONFIG_HEADERS = $CONFIG_HEADERS\n CONFIG_LINKS = $CONFIG_LINKS\n CONFIG_COMMANDS = $CONFIG_COMMANDS\n $ $0 $@\n\non `(hostname || uname -n) 2>/dev/null | sed 1q`\n\"\n\n_ACEOF\n\ncase $ac_config_files in *\"\n\"*) set x $ac_config_files; shift; ac_config_files=$*;;\nesac\n\ncase $ac_config_headers in *\"\n\"*) set x $ac_config_headers; shift; ac_config_headers=$*;;\nesac\n\n\ncat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1\n# Files that config.status was made for.\nconfig_files=\"$ac_config_files\"\nconfig_headers=\"$ac_config_headers\"\n\n_ACEOF\n\ncat >>$CONFIG_STATUS <<\\_ACEOF || ac_write_fail=1\nac_cs_usage=\"\\\n\\`$as_me' instantiates files and other configuration actions\nfrom templates according to the current configuration. Unless the files\nand actions are specified as TAGs, all are instantiated by default.\n\nUsage: $0 [OPTION]... [TAG]...\n\n -h, --help print this help, then exit\n -V, --version print version number and configuration settings, then exit\n --config print configuration, then exit\n -q, --quiet, --silent\n do not print progress messages\n -d, --debug don't remove temporary files\n --recheck update $as_me by reconfiguring in the same conditions\n --file=FILE[:TEMPLATE]\n instantiate the configuration file FILE\n --header=FILE[:TEMPLATE]\n instantiate the configuration header FILE\n\nConfiguration files:\n$config_files\n\nConfiguration headers:\n$config_headers\n\nReport bugs to the package provider.\"\n\n_ACEOF\nac_cs_config=`printf \"%s\\n\" \"$ac_configure_args\" | sed \"$ac_safe_unquote\"`\nac_cs_config_escaped=`printf \"%s\\n\" \"$ac_cs_config\" | sed \"s/^ //; s/'/'\\\\\\\\\\\\\\\\''/g\"`\ncat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1\nac_cs_config='$ac_cs_config_escaped'\nac_cs_version=\"\\\\\nxgboost config.status 3.3.0\nconfigured by $0, generated by GNU Autoconf 2.71,\n with options \\\\\"\\$ac_cs_config\\\\\"\n\nCopyright (C) 2021 Free Software Foundation, Inc.\nThis config.status script is free software; the Free Software Foundation\ngives unlimited permission to copy, distribute and modify it.\"\n\nac_pwd='$ac_pwd'\nsrcdir='$srcdir'\ntest -n \"\\$AWK\" || AWK=awk\n_ACEOF\n\ncat >>$CONFIG_STATUS <<\\_ACEOF || ac_write_fail=1\n# The default lists apply if the user does not specify any file.\nac_need_defaults=:\nwhile test $# != 0\ndo\n case $1 in\n --*=?*)\n ac_option=`expr \"X$1\" : 'X\\([^=]*\\)='`\n ac_optarg=`expr \"X$1\" : 'X[^=]*=\\(.*\\)'`\n ac_shift=:\n ;;\n --*=)\n ac_option=`expr \"X$1\" : 'X\\([^=]*\\)='`\n ac_optarg=\n ac_shift=:\n ;;\n *)\n ac_option=$1\n ac_optarg=$2\n ac_shift=shift\n ;;\n esac\n\n case $ac_option in\n # Handling of the options.\n -recheck | --recheck | --rechec | --reche | --rech | --rec | --re | --r)\n ac_cs_recheck=: ;;\n --version | --versio | --versi | --vers | --ver | --ve | --v | -V )\n printf \"%s\\n\" \"$ac_cs_version\"; exit ;;\n --config | --confi | --conf | --con | --co | --c )\n printf \"%s\\n\" \"$ac_cs_config\"; exit ;;\n --debug | --debu | --deb | --de | --d | -d )\n debug=: ;;\n --file | --fil | --fi | --f )\n $ac_shift\n case $ac_optarg in\n *\\'*) ac_optarg=`printf \"%s\\n\" \"$ac_optarg\" | sed \"s/'/'\\\\\\\\\\\\\\\\''/g\"` ;;\n '') as_fn_error $? \"missing file argument\" ;;\n esac\n as_fn_append CONFIG_FILES \" '$ac_optarg'\"\n ac_need_defaults=false;;\n --header | --heade | --head | --hea )\n $ac_shift\n case $ac_optarg in\n *\\'*) ac_optarg=`printf \"%s\\n\" \"$ac_optarg\" | sed \"s/'/'\\\\\\\\\\\\\\\\''/g\"` ;;\n esac\n as_fn_append CONFIG_HEADERS \" '$ac_optarg'\"\n ac_need_defaults=false;;\n --he | --h)\n # Conflict between --help and --header\n as_fn_error $? \"ambiguous option: \\`$1'\nTry \\`$0 --help' for more information.\";;\n --help | --hel | -h )\n printf \"%s\\n\" \"$ac_cs_usage\"; exit ;;\n -q | -quiet | --quiet | --quie | --qui | --qu | --q \\\n | -silent | --silent | --silen | --sile | --sil | --si | --s)\n ac_cs_silent=: ;;\n\n # This is an error.\n -*) as_fn_error $? \"unrecognized option: \\`$1'\nTry \\`$0 --help' for more information.\" ;;\n\n *) as_fn_append ac_config_targets \" $1\"\n ac_need_defaults=false ;;\n\n esac\n shift\ndone\n\nac_configure_extra_args=\n\nif $ac_cs_silent; then\n exec 6>/dev/null\n ac_configure_extra_args=\"$ac_configure_extra_args --silent\"\nfi\n\n_ACEOF\ncat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1\nif \\$ac_cs_recheck; then\n set X $SHELL '$0' $ac_configure_args \\$ac_configure_extra_args --no-create --no-recursion\n shift\n \\printf \"%s\\n\" \"running CONFIG_SHELL=$SHELL \\$*\" >&6\n CONFIG_SHELL='$SHELL'\n export CONFIG_SHELL\n exec \"\\$@\"\nfi\n\n_ACEOF\ncat >>$CONFIG_STATUS <<\\_ACEOF || ac_write_fail=1\nexec 5>>config.log\n{\n echo\n sed 'h;s/./-/g;s/^.../## /;s/...$/ ##/;p;x;p;x' <<_ASBOX\n## Running $as_me. ##\n_ASBOX\n printf \"%s\\n\" \"$ac_log\"\n} >&5\n\n_ACEOF\ncat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1\n_ACEOF\n\ncat >>$CONFIG_STATUS <<\\_ACEOF || ac_write_fail=1\n\n# Handling of arguments.\nfor ac_config_target in $ac_config_targets\ndo\n case $ac_config_target in\n \"src/Makevars\") CONFIG_FILES=\"$CONFIG_FILES src/Makevars\" ;;\n \"src/config.h\") CONFIG_HEADERS=\"$CONFIG_HEADERS src/config.h\" ;;\n\n *) as_fn_error $? \"invalid argument: \\`$ac_config_target'\" \"$LINENO\" 5;;\n esac\ndone\n\n\n# If the user did not use the arguments to specify the items to instantiate,\n# then the envvar interface is used. Set only those that are not.\n# We use the long form for the default assignment because of an extremely\n# bizarre bug on SunOS 4.1.3.\nif $ac_need_defaults; then\n test ${CONFIG_FILES+y} || CONFIG_FILES=$config_files\n test ${CONFIG_HEADERS+y} || CONFIG_HEADERS=$config_headers\nfi\n\n# Have a temporary directory for convenience. Make it in the build tree\n# simply because there is no reason against having it here, and in addition,\n# creating and moving files from /tmp can sometimes cause problems.\n# Hook for its removal unless debugging.\n# Note that there is a small window in which the directory will not be cleaned:\n# after its creation but before its name has been assigned to `$tmp'.\n$debug ||\n{\n tmp= ac_tmp=\n trap 'exit_status=$?\n : \"${ac_tmp:=$tmp}\"\n { test ! -d \"$ac_tmp\" || rm -fr \"$ac_tmp\"; } && exit $exit_status\n' 0\n trap 'as_fn_exit 1' 1 2 13 15\n}\n# Create a (secure) tmp directory for tmp files.\n\n{\n tmp=`(umask 077 && mktemp -d \"./confXXXXXX\") 2>/dev/null` &&\n test -d \"$tmp\"\n} ||\n{\n tmp=./conf$$-$RANDOM\n (umask 077 && mkdir \"$tmp\")\n} || as_fn_error $? \"cannot create a temporary directory in .\" \"$LINENO\" 5\nac_tmp=$tmp\n\n# Set up the scripts for CONFIG_FILES section.\n# No need to generate them if there are no CONFIG_FILES.\n# This happens for instance with `./config.status config.h'.\nif test -n \"$CONFIG_FILES\"; then\n\n\nac_cr=`echo X | tr X '\\015'`\n# On cygwin, bash can eat \\r inside `` if the user requested igncr.\n# But we know of no other shell where ac_cr would be empty at this\n# point, so we can use a bashism as a fallback.\nif test \"x$ac_cr\" = x; then\n eval ac_cr=\\$\\'\\\\r\\'\nfi\nac_cs_awk_cr=`$AWK 'BEGIN { print \"a\\rb\" }' /dev/null`\nif test \"$ac_cs_awk_cr\" = \"a${ac_cr}b\"; then\n ac_cs_awk_cr='\\\\r'\nelse\n ac_cs_awk_cr=$ac_cr\nfi\n\necho 'BEGIN {' >\"$ac_tmp/subs1.awk\" &&\n_ACEOF\n\n\n{\n echo \"cat >conf$$subs.awk <<_ACEOF\" &&\n echo \"$ac_subst_vars\" | sed 's/.*/&!$&$ac_delim/' &&\n echo \"_ACEOF\"\n} >conf$$subs.sh ||\n as_fn_error $? \"could not make $CONFIG_STATUS\" \"$LINENO\" 5\nac_delim_num=`echo \"$ac_subst_vars\" | grep -c '^'`\nac_delim='%!_!# '\nfor ac_last_try in false false false false false :; do\n . ./conf$$subs.sh ||\n as_fn_error $? \"could not make $CONFIG_STATUS\" \"$LINENO\" 5\n\n ac_delim_n=`sed -n \"s/.*$ac_delim\\$/X/p\" conf$$subs.awk | grep -c X`\n if test $ac_delim_n = $ac_delim_num; then\n break\n elif $ac_last_try; then\n as_fn_error $? \"could not make $CONFIG_STATUS\" \"$LINENO\" 5\n else\n ac_delim=\"$ac_delim!$ac_delim _$ac_delim!! \"\n fi\ndone\nrm -f conf$$subs.sh\n\ncat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1\ncat >>\"\\$ac_tmp/subs1.awk\" <<\\\\_ACAWK &&\n_ACEOF\nsed -n '\nh\ns/^/S[\"/; s/!.*/\"]=/\np\ng\ns/^[^!]*!//\n:repl\nt repl\ns/'\"$ac_delim\"'$//\nt delim\n:nl\nh\ns/\\(.\\{148\\}\\)..*/\\1/\nt more1\ns/[\"\\\\]/\\\\&/g; s/^/\"/; s/$/\\\\n\"\\\\/\np\nn\nb repl\n:more1\ns/[\"\\\\]/\\\\&/g; s/^/\"/; s/$/\"\\\\/\np\ng\ns/.\\{148\\}//\nt nl\n:delim\nh\ns/\\(.\\{148\\}\\)..*/\\1/\nt more2\ns/[\"\\\\]/\\\\&/g; s/^/\"/; s/$/\"/\np\nb\n:more2\ns/[\"\\\\]/\\\\&/g; s/^/\"/; s/$/\"\\\\/\np\ng\ns/.\\{148\\}//\nt delim\n' >$CONFIG_STATUS || ac_write_fail=1\nrm -f conf$$subs.awk\ncat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1\n_ACAWK\ncat >>\"\\$ac_tmp/subs1.awk\" <<_ACAWK &&\n for (key in S) S_is_set[key] = 1\n FS = \"\u0007\"\n\n}\n{\n line = $ 0\n nfields = split(line, field, \"@\")\n substed = 0\n len = length(field[1])\n for (i = 2; i < nfields; i++) {\n key = field[i]\n keylen = length(key)\n if (S_is_set[key]) {\n value = S[key]\n line = substr(line, 1, len) \"\" value \"\" substr(line, len + keylen + 3)\n len += length(value) + length(field[++i])\n substed = 1\n } else\n len += 1 + keylen\n }\n\n print line\n}\n\n_ACAWK\n_ACEOF\ncat >>$CONFIG_STATUS <<\\_ACEOF || ac_write_fail=1\nif sed \"s/$ac_cr//\" < /dev/null > /dev/null 2>&1; then\n sed \"s/$ac_cr\\$//; s/$ac_cr/$ac_cs_awk_cr/g\"\nelse\n cat\nfi < \"$ac_tmp/subs1.awk\" > \"$ac_tmp/subs.awk\" \\\n || as_fn_error $? \"could not setup config files machinery\" \"$LINENO\" 5\n_ACEOF\n\n# VPATH may cause trouble with some makes, so we remove sole $(srcdir),\n# ${srcdir} and @srcdir@ entries from VPATH if srcdir is \".\", strip leading and\n# trailing colons and then remove the whole line if VPATH becomes empty\n# (actually we leave an empty line to preserve line numbers).\nif test \"x$srcdir\" = x.; then\n ac_vpsub='/^[\t ]*VPATH[\t ]*=[\t ]*/{\nh\ns///\ns/^/:/\ns/[\t ]*$/:/\ns/:\\$(srcdir):/:/g\ns/:\\${srcdir}:/:/g\ns/:@srcdir@:/:/g\ns/^:*//\ns/:*$//\nx\ns/\\(=[\t ]*\\).*/\\1/\nG\ns/\\n//\ns/^[^=]*=[\t ]*$//\n}'\nfi\n\ncat >>$CONFIG_STATUS <<\\_ACEOF || ac_write_fail=1\nfi # test -n \"$CONFIG_FILES\"\n\n# Set up the scripts for CONFIG_HEADERS section.\n# No need to generate them if there are no CONFIG_HEADERS.\n# This happens for instance with `./config.status Makefile'.\nif test -n \"$CONFIG_HEADERS\"; then\ncat >\"$ac_tmp/defines.awk\" <<\\_ACAWK ||\nBEGIN {\n_ACEOF\n\n# Transform confdefs.h into an awk script `defines.awk', embedded as\n# here-document in config.status, that substitutes the proper values into\n# config.h.in to produce config.h.\n\n# Create a delimiter string that does not exist in confdefs.h, to ease\n# handling of long lines.\nac_delim='%!_!# '\nfor ac_last_try in false false :; do\n ac_tt=`sed -n \"/$ac_delim/p\" confdefs.h`\n if test -z \"$ac_tt\"; then\n break\n elif $ac_last_try; then\n as_fn_error $? \"could not make $CONFIG_HEADERS\" \"$LINENO\" 5\n else\n ac_delim=\"$ac_delim!$ac_delim _$ac_delim!! \"\n fi\ndone\n\n# For the awk script, D is an array of macro values keyed by name,\n# likewise P contains macro parameters if any. Preserve backslash\n# newline sequences.\n\nac_word_re=[_$as_cr_Letters][_$as_cr_alnum]*\nsed -n '\ns/.\\{148\\}/&'\"$ac_delim\"'/g\nt rset\n:rset\ns/^[\t ]*#[\t ]*define[\t ][\t ]*/ /\nt def\nd\n:def\ns/\\\\$//\nt bsnl\ns/[\"\\\\]/\\\\&/g\ns/^ \\('\"$ac_word_re\"'\\)\\(([^()]*)\\)[\t ]*\\(.*\\)/P[\"\\1\"]=\"\\2\"\\\nD[\"\\1\"]=\" \\3\"/p\ns/^ \\('\"$ac_word_re\"'\\)[\t ]*\\(.*\\)/D[\"\\1\"]=\" \\2\"/p\nd\n:bsnl\ns/[\"\\\\]/\\\\&/g\ns/^ \\('\"$ac_word_re\"'\\)\\(([^()]*)\\)[\t ]*\\(.*\\)/P[\"\\1\"]=\"\\2\"\\\nD[\"\\1\"]=\" \\3\\\\\\\\\\\\n\"\\\\/p\nt cont\ns/^ \\('\"$ac_word_re\"'\\)[\t ]*\\(.*\\)/D[\"\\1\"]=\" \\2\\\\\\\\\\\\n\"\\\\/p\nt cont\nd\n:cont\nn\ns/.\\{148\\}/&'\"$ac_delim\"'/g\nt clear\n:clear\ns/\\\\$//\nt bsnlc\ns/[\"\\\\]/\\\\&/g; s/^/\"/; s/$/\"/p\nd\n:bsnlc\ns/[\"\\\\]/\\\\&/g; s/^/\"/; s/$/\\\\\\\\\\\\n\"\\\\/p\nb cont\n' >$CONFIG_STATUS || ac_write_fail=1\n\ncat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1\n for (key in D) D_is_set[key] = 1\n FS = \"\u0007\"\n}\n/^[\\t ]*#[\\t ]*(define|undef)[\\t ]+$ac_word_re([\\t (]|\\$)/ {\n line = \\$ 0\n split(line, arg, \" \")\n if (arg[1] == \"#\") {\n defundef = arg[2]\n mac1 = arg[3]\n } else {\n defundef = substr(arg[1], 2)\n mac1 = arg[2]\n }\n split(mac1, mac2, \"(\") #)\n macro = mac2[1]\n prefix = substr(line, 1, index(line, defundef) - 1)\n if (D_is_set[macro]) {\n # Preserve the white space surrounding the \"#\".\n print prefix \"define\", macro P[macro] D[macro]\n next\n } else {\n # Replace #undef with comments. This is necessary, for example,\n # in the case of _POSIX_SOURCE, which is predefined and required\n # on some systems where configure will not decide to define it.\n if (defundef == \"undef\") {\n print \"/*\", prefix defundef, macro, \"*/\"\n next\n }\n }\n}\n{ print }\n_ACAWK\n_ACEOF\ncat >>$CONFIG_STATUS <<\\_ACEOF || ac_write_fail=1\n as_fn_error $? \"could not setup config headers machinery\" \"$LINENO\" 5\nfi # test -n \"$CONFIG_HEADERS\"\n\n\neval set X \" :F $CONFIG_FILES :H $CONFIG_HEADERS \"\nshift\nfor ac_tag\ndo\n case $ac_tag in\n :[FHLC]) ac_mode=$ac_tag; continue;;\n esac\n case $ac_mode$ac_tag in\n :[FHL]*:*);;\n :L* | :C*:*) as_fn_error $? \"invalid tag \\`$ac_tag'\" \"$LINENO\" 5;;\n :[FH]-) ac_tag=-:-;;\n :[FH]*) ac_tag=$ac_tag:$ac_tag.in;;\n esac\n ac_save_IFS=$IFS\n IFS=:\n set x $ac_tag\n IFS=$ac_save_IFS\n shift\n ac_file=$1\n shift\n\n case $ac_mode in\n :L) ac_source=$1;;\n :[FH])\n ac_file_inputs=\n for ac_f\n do\n case $ac_f in\n -) ac_f=\"$ac_tmp/stdin\";;\n *) # Look for the file first in the build tree, then in the source tree\n\t # (if the path is not absolute). The absolute path cannot be DOS-style,\n\t # because $ac_f cannot contain `:'.\n\t test -f \"$ac_f\" ||\n\t case $ac_f in\n\t [\\\\/$]*) false;;\n\t *) test -f \"$srcdir/$ac_f\" && ac_f=\"$srcdir/$ac_f\";;\n\t esac ||\n\t as_fn_error 1 \"cannot find input file: \\`$ac_f'\" \"$LINENO\" 5;;\n esac\n case $ac_f in *\\'*) ac_f=`printf \"%s\\n\" \"$ac_f\" | sed \"s/'/'\\\\\\\\\\\\\\\\''/g\"`;; esac\n as_fn_append ac_file_inputs \" '$ac_f'\"\n done\n\n # Let's still pretend it is `configure' which instantiates (i.e., don't\n # use $as_me), people would be surprised to read:\n # /* config.h. Generated by config.status. */\n configure_input='Generated from '`\n\t printf \"%s\\n\" \"$*\" | sed 's|^[^:]*/||;s|:[^:]*/|, |g'\n\t`' by configure.'\n if test x\"$ac_file\" != x-; then\n configure_input=\"$ac_file. $configure_input\"\n { printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: creating $ac_file\" >&5\nprintf \"%s\\n\" \"$as_me: creating $ac_file\" >&6;}\n fi\n # Neutralize special characters interpreted by sed in replacement strings.\n case $configure_input in #(\n *\\&* | *\\|* | *\\\\* )\n ac_sed_conf_input=`printf \"%s\\n\" \"$configure_input\" |\n sed 's/[\\\\\\\\&|]/\\\\\\\\&/g'`;; #(\n *) ac_sed_conf_input=$configure_input;;\n esac\n\n case $ac_tag in\n *:-:* | *:-) cat >\"$ac_tmp/stdin\" \\\n || as_fn_error $? \"could not create $ac_file\" \"$LINENO\" 5 ;;\n esac\n ;;\n esac\n\n ac_dir=`$as_dirname -- \"$ac_file\" ||\n$as_expr X\"$ac_file\" : 'X\\(.*[^/]\\)//*[^/][^/]*/*$' \\| \\\n\t X\"$ac_file\" : 'X\\(//\\)[^/]' \\| \\\n\t X\"$ac_file\" : 'X\\(//\\)$' \\| \\\n\t X\"$ac_file\" : 'X\\(/\\)' \\| . 2>/dev/null ||\nprintf \"%s\\n\" X\"$ac_file\" |\n sed '/^X\\(.*[^/]\\)\\/\\/*[^/][^/]*\\/*$/{\n\t s//\\1/\n\t q\n\t }\n\t /^X\\(\\/\\/\\)[^/].*/{\n\t s//\\1/\n\t q\n\t }\n\t /^X\\(\\/\\/\\)$/{\n\t s//\\1/\n\t q\n\t }\n\t /^X\\(\\/\\).*/{\n\t s//\\1/\n\t q\n\t }\n\t s/.*/./; q'`\n as_dir=\"$ac_dir\"; as_fn_mkdir_p\n ac_builddir=.\n\ncase \"$ac_dir\" in\n.) ac_dir_suffix= ac_top_builddir_sub=. ac_top_build_prefix= ;;\n*)\n ac_dir_suffix=/`printf \"%s\\n\" \"$ac_dir\" | sed 's|^\\.[\\\\/]||'`\n # A \"..\" for each directory in $ac_dir_suffix.\n ac_top_builddir_sub=`printf \"%s\\n\" \"$ac_dir_suffix\" | sed 's|/[^\\\\/]*|/..|g;s|/||'`\n case $ac_top_builddir_sub in\n \"\") ac_top_builddir_sub=. ac_top_build_prefix= ;;\n *) ac_top_build_prefix=$ac_top_builddir_sub/ ;;\n esac ;;\nesac\nac_abs_top_builddir=$ac_pwd\nac_abs_builddir=$ac_pwd$ac_dir_suffix\n# for backward compatibility:\nac_top_builddir=$ac_top_build_prefix\n\ncase $srcdir in\n .) # We are building in place.\n ac_srcdir=.\n ac_top_srcdir=$ac_top_builddir_sub\n ac_abs_top_srcdir=$ac_pwd ;;\n [\\\\/]* | ?:[\\\\/]* ) # Absolute name.\n ac_srcdir=$srcdir$ac_dir_suffix;\n ac_top_srcdir=$srcdir\n ac_abs_top_srcdir=$srcdir ;;\n *) # Relative name.\n ac_srcdir=$ac_top_build_prefix$srcdir$ac_dir_suffix\n ac_top_srcdir=$ac_top_build_prefix$srcdir\n ac_abs_top_srcdir=$ac_pwd/$srcdir ;;\nesac\nac_abs_srcdir=$ac_abs_top_srcdir$ac_dir_suffix\n\n\n case $ac_mode in\n :F)\n #\n # CONFIG_FILE\n #\n\n_ACEOF\n\ncat >>$CONFIG_STATUS <<\\_ACEOF || ac_write_fail=1\n# If the template does not know about datarootdir, expand it.\n# FIXME: This hack should be removed a few years after 2.60.\nac_datarootdir_hack=; ac_datarootdir_seen=\nac_sed_dataroot='\n/datarootdir/ {\n p\n q\n}\n/@datadir@/p\n/@docdir@/p\n/@infodir@/p\n/@localedir@/p\n/@mandir@/p'\ncase `eval \"sed -n \\\"\\$ac_sed_dataroot\\\" $ac_file_inputs\"` in\n*datarootdir*) ac_datarootdir_seen=yes;;\n*@datadir@*|*@docdir@*|*@infodir@*|*@localedir@*|*@mandir@*)\n { printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: WARNING: $ac_file_inputs seems to ignore the --datarootdir setting\" >&5\nprintf \"%s\\n\" \"$as_me: WARNING: $ac_file_inputs seems to ignore the --datarootdir setting\" >&2;}\n_ACEOF\ncat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1\n ac_datarootdir_hack='\n s&@datadir@&$datadir&g\n s&@docdir@&$docdir&g\n s&@infodir@&$infodir&g\n s&@localedir@&$localedir&g\n s&@mandir@&$mandir&g\n s&\\\\\\${datarootdir}&$datarootdir&g' ;;\nesac\n_ACEOF\n\n# Neutralize VPATH when `$srcdir' = `.'.\n# Shell code in configure.ac might set extrasub.\n# FIXME: do we really want to maintain this feature?\ncat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1\nac_sed_extra=\"$ac_vpsub\n$extrasub\n_ACEOF\ncat >>$CONFIG_STATUS <<\\_ACEOF || ac_write_fail=1\n:t\n/@[a-zA-Z_][a-zA-Z_0-9]*@/!b\ns|@configure_input@|$ac_sed_conf_input|;t t\ns&@top_builddir@&$ac_top_builddir_sub&;t t\ns&@top_build_prefix@&$ac_top_build_prefix&;t t\ns&@srcdir@&$ac_srcdir&;t t\ns&@abs_srcdir@&$ac_abs_srcdir&;t t\ns&@top_srcdir@&$ac_top_srcdir&;t t\ns&@abs_top_srcdir@&$ac_abs_top_srcdir&;t t\ns&@builddir@&$ac_builddir&;t t\ns&@abs_builddir@&$ac_abs_builddir&;t t\ns&@abs_top_builddir@&$ac_abs_top_builddir&;t t\n$ac_datarootdir_hack\n\"\neval sed \\\"\\$ac_sed_extra\\\" \"$ac_file_inputs\" | $AWK -f \"$ac_tmp/subs.awk\" \\\n >$ac_tmp/out || as_fn_error $? \"could not create $ac_file\" \"$LINENO\" 5\n\ntest -z \"$ac_datarootdir_hack$ac_datarootdir_seen\" &&\n { ac_out=`sed -n '/\\${datarootdir}/p' \"$ac_tmp/out\"`; test -n \"$ac_out\"; } &&\n { ac_out=`sed -n '/^[\t ]*datarootdir[\t ]*:*=/p' \\\n \"$ac_tmp/out\"`; test -z \"$ac_out\"; } &&\n { printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: WARNING: $ac_file contains a reference to the variable \\`datarootdir'\nwhich seems to be undefined. Please make sure it is defined\" >&5\nprintf \"%s\\n\" \"$as_me: WARNING: $ac_file contains a reference to the variable \\`datarootdir'\nwhich seems to be undefined. Please make sure it is defined\" >&2;}\n\n rm -f \"$ac_tmp/stdin\"\n case $ac_file in\n -) cat \"$ac_tmp/out\" && rm -f \"$ac_tmp/out\";;\n *) rm -f \"$ac_file\" && mv \"$ac_tmp/out\" \"$ac_file\";;\n esac \\\n || as_fn_error $? \"could not create $ac_file\" \"$LINENO\" 5\n ;;\n :H)\n #\n # CONFIG_HEADER\n #\n if test x\"$ac_file\" != x-; then\n {\n printf \"%s\\n\" \"/* $configure_input */\" >&1 \\\n && eval '$AWK -f \"$ac_tmp/defines.awk\"' \"$ac_file_inputs\"\n } >\"$ac_tmp/config.h\" \\\n || as_fn_error $? \"could not create $ac_file\" \"$LINENO\" 5\n if diff \"$ac_file\" \"$ac_tmp/config.h\" >/dev/null 2>&1; then\n { printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: $ac_file is unchanged\" >&5\nprintf \"%s\\n\" \"$as_me: $ac_file is unchanged\" >&6;}\n else\n rm -f \"$ac_file\"\n mv \"$ac_tmp/config.h\" \"$ac_file\" \\\n\t|| as_fn_error $? \"could not create $ac_file\" \"$LINENO\" 5\n fi\n else\n printf \"%s\\n\" \"/* $configure_input */\" >&1 \\\n && eval '$AWK -f \"$ac_tmp/defines.awk\"' \"$ac_file_inputs\" \\\n || as_fn_error $? \"could not create -\" \"$LINENO\" 5\n fi\n ;;\n\n\n esac\n\ndone # for ac_tag\n\n\nas_fn_exit 0\n_ACEOF\nac_clean_files=$ac_clean_files_save\n\ntest $ac_write_fail = 0 ||\n as_fn_error $? \"write failure creating $CONFIG_STATUS\" \"$LINENO\" 5\n\n\n# configure is writing to config.log, and then calls config.status.\n# config.status does its own redirection, appending to config.log.\n# Unfortunately, on DOS this fails, as config.log is still kept open\n# by configure, so config.status won't be able to write to it; its\n# output is simply discarded. So we exec the FD to /dev/null,\n# effectively closing config.log, so it can be properly (re)opened and\n# appended to by config.status. When coming back to configure, we\n# need to make the FD available again.\nif test \"$no_create\" != yes; then\n ac_cs_success=:\n ac_config_status_args=\n test \"$silent\" = yes &&\n ac_config_status_args=\"$ac_config_status_args --quiet\"\n exec 5>/dev/null\n $SHELL $CONFIG_STATUS $ac_config_status_args || ac_cs_success=false\n exec 5>>config.log\n # Use ||, not &&, to avoid exiting from the if with $? = 1, which\n # would make configure fail if this is the last instruction.\n $ac_cs_success || as_fn_exit 1\nfi\nif test -n \"$ac_unrecognized_opts\" && test \"$enable_option_checking\" != no; then\n { printf \"%s\\n\" \"$as_me:${as_lineno-$LINENO}: WARNING: unrecognized options: $ac_unrecognized_opts\" >&5\nprintf \"%s\\n\" \"$as_me: WARNING: unrecognized options: $ac_unrecognized_opts\" >&2;}\nfi\n" }, { "path": "R-package/configure.ac", "content": "### configure.ac\t\t\t\t\t-*- Autoconf -*-\n\nAC_PREREQ(2.69)\n\nAC_INIT([xgboost],[3.3.0],[],[xgboost],[])\n\n: ${R_HOME=`R RHOME`}\nif test -z \"${R_HOME}\"; then\n echo \"could not determine R_HOME\"\n exit 1\nfi\n\nCXX17=`\"${R_HOME}/bin/R\" CMD config CXX17`\nCXX17STD=`\"${R_HOME}/bin/R\" CMD config CXX17STD`\nCXX=\"${CXX17} ${CXX17STD}\"\nCXXFLAGS=`\"${R_HOME}/bin/R\" CMD config CXXFLAGS`\n\nCC=`\"${R_HOME}/bin/R\" CMD config CC`\nCFLAGS=`\"${R_HOME}/bin/R\" CMD config CFLAGS`\nCPPFLAGS=`\"${R_HOME}/bin/R\" CMD config CPPFLAGS`\n\nLDFLAGS=`\"${R_HOME}/bin/R\" CMD config LDFLAGS`\nAC_LANG(C++)\n\nDMLC_DEFS=\"\"\n\nAC_MSG_NOTICE([Checking if/where backtrace is available])\nAC_CHECK_LIB([execinfo], [backtrace], [BACKTRACE_LIB=-lexecinfo], [BACKTRACE_LIB=''])\nAS_IF([test -z \"${BACKTRACE_LIB}\"],\n dnl backtrace() might be unavailable (e.g., in musl libc)\n [AC_CHECK_FUNC(backtrace, [], [DMLC_DEFS=\"$DMLC_DEFS -DDMLC_LOG_STACK_TRACE=0\"])])\n\nAC_MSG_NOTICE([Checking whether fopen64 is available])\nAC_CHECK_FUNC(fopen64, [], [DMLC_DEFS=\"$DMLC_DEFS -DDMLC_USE_FOPEN64=0\"])\n\nAC_MSG_NOTICE([Endian detection])\nAC_ARG_VAR(USE_LITTLE_ENDIAN, \"Whether to build with little endian (checks at compile time if unset)\")\nAS_IF([test -z \"${USE_LITTLE_ENDIAN+x}\"], [\n AC_MSG_NOTICE([Checking system endianness as USE_LITTLE_ENDIAN is unset])\n AC_C_BIGENDIAN(\n [AC_MSG_RESULT([using big endian])\n ENDIAN_FLAG=\"-DDMLC_CMAKE_LITTLE_ENDIAN=0\"],\n [AC_MSG_RESULT([using little endian])\n ENDIAN_FLAG=\"-DDMLC_CMAKE_LITTLE_ENDIAN=1\"],\n [AC_MSG_RESULT([unknown])\n AC_MSG_ERROR([Could not determine endianness. Please set USE_LITTLE_ENDIAN])]\n )\n], [\n AC_MSG_NOTICE([Forcing endianness to: ${USE_LITTLE_ENDIAN}])\n ENDIAN_FLAG=\"-DDMLC_CMAKE_LITTLE_ENDIAN=${USE_LITTLE_ENDIAN}\"\n])\n\nAC_MSG_NOTICE([Checking for prefetch builtin])\nAC_LINK_IFELSE(\n [AC_LANG_PROGRAM(\n [],\n [__builtin_prefetch]\n )],\n [XGBOOST_BUILTIN_PREFETCH_PRESENT=\"-DXGBOOST_BUILTIN_PREFETCH_PRESENT=1\"],\n [XGBOOST_BUILTIN_PREFETCH_PRESENT=\"\"]\n)\nif [[ \"$XGBOOST_BUILTIN_PREFETCH_PRESENT\" = \"\" ]]; then\n echo \"Has __builtin_prefetch\"\nelse\n echo \"Doesn't have __builtin_prefetch\"\nfi\n\nAC_MSG_NOTICE([Checking for mm_prefetch])\nAC_LINK_IFELSE(\n [AC_LANG_PROGRAM(\n [#include ],\n [_mm_prefetch]\n )],\n [XGBOOST_MM_PREFETCH_PRESENT=\"-DXGBOOST_MM_PREFETCH_PRESENT=1\"],\n [XGBOOST_MM_PREFETCH_PRESENT=\"\"]\n)\nif [[ \"$XGBOOST_MM_PREFETCH_PRESENT\" = \"\" ]]; then\n echo \"Has _mm_prefetch\"\nelse\n echo \"Doesn't have _mm_prefetch\"\nfi\n\nOPENMP_CXXFLAGS=\"\"\n\nif test `uname -s` = \"Linux\"\nthen\n OPENMP_CXXFLAGS=\"\\$(SHLIB_OPENMP_CXXFLAGS)\"\nfi\n\nif test `uname -s` = \"Darwin\"\nthen\n if command -v brew &> /dev/null\n then\n HOMEBREW_LIBOMP_PREFIX=`brew --prefix libomp`\n else\n # Homebrew not found\n HOMEBREW_LIBOMP_PREFIX=''\n fi\n OPENMP_CXXFLAGS=\"-Xpreprocessor -fopenmp -I${HOMEBREW_LIBOMP_PREFIX}/include\"\n OPENMP_LIB=\"-lomp -L${HOMEBREW_LIBOMP_PREFIX}/lib\"\n ac_pkg_openmp=no\n AC_MSG_CHECKING([whether OpenMP will work in a package])\n AC_LANG_CONFTEST([AC_LANG_PROGRAM([[#include ]], [[ return (omp_get_max_threads() <= 1); ]])])\n ${CXX} -o conftest conftest.cpp ${CPPFLAGS} ${LDFLAGS} ${OPENMP_LIB} ${OPENMP_CXXFLAGS} 2>/dev/null && ./conftest && ac_pkg_openmp=yes\n AC_MSG_RESULT([${ac_pkg_openmp}])\n if test \"${ac_pkg_openmp}\" = no; then\n OPENMP_CXXFLAGS=''\n OPENMP_LIB=''\n echo '*****************************************************************************************'\n echo ' OpenMP is unavailable on this Mac OSX system. Training speed may be suboptimal.'\n echo ' To use all CPU cores for training jobs, you should install OpenMP by running'\n echo ' brew install libomp'\n echo '*****************************************************************************************'\n fi\nfi\n\nAC_SUBST(OPENMP_CXXFLAGS)\nAC_SUBST(OPENMP_LIB)\nAC_SUBST(ENDIAN_FLAG)\nAC_SUBST(DMLC_DEFS)\nAC_SUBST(BACKTRACE_LIB)\nAC_SUBST(XGBOOST_BUILTIN_PREFETCH_PRESENT)\nAC_SUBST(XGBOOST_MM_PREFETCH_PRESENT)\nAC_CONFIG_FILES([src/Makevars])\nAC_CONFIG_HEADERS([src/config.h])\nAC_OUTPUT\n" }, { "path": "R-package/configure.win", "content": "R_EXE=\"${R_HOME}/bin${R_ARCH_BIN}/R.exe\"\nCXX=`\"${R_EXE}\" CMD config CXX`\n\ncat > test.cpp <\nint main() {\n char data = 0;\n const char* address = &data;\n _mm_prefetch(address, _MM_HINT_NTA);\n return 0;\n}\nEOL\n\nXGBOOST_MM_PREFETCH_PRESENT=\"\"\n${CXX} -o test test.cpp 2>/dev/null && ./test && XGBOOST_MM_PREFETCH_PRESENT=\"-DXGBOOST_MM_PREFETCH_PRESENT=1\"\nrm -f ./test\nrm -f ./test.cpp\n\nsed \\\n -e \"s/@XGBOOST_MM_PREFETCH_PRESENT@/$XGBOOST_MM_PREFETCH_PRESENT/\" \\\n < src/Makevars.win.in > src/Makevars.win\n" }, { "path": "R-package/inst/make-r-def.R", "content": "# [description]\n# Create a definition file (.def) from a .dll file, using objdump. This\n# is used by FindLibR.cmake when building the R package with MSVC.\n#\n# [usage]\n#\n# Rscript make-r-def.R something.dll something.def\n#\n# [references]\n# * https://www.cs.colorado.edu/~main/cs1300/doc/mingwfaq.html\n\nargs <- commandArgs(trailingOnly = TRUE)\n\nIN_DLL_FILE <- args[[1L]]\nOUT_DEF_FILE <- args[[2L]]\nDLL_BASE_NAME <- basename(IN_DLL_FILE)\n\nmessage(sprintf(\"Creating '%s' from '%s'\", OUT_DEF_FILE, IN_DLL_FILE))\n\n# system() will not raise an R exception if the process called\n# fails. Wrapping it here to get that behavior.\n#\n# system() introduces a lot of overhead, at least on Windows,\n# so trying processx if it is available\n.pipe_shell_command_to_stdout <- function(command, args, out_file) {\n has_processx <- suppressMessages({\n suppressWarnings({\n require(\"processx\") # nolint\n })\n })\n if (has_processx) {\n p <- processx::process$new(\n command = command\n , args = args\n , stdout = out_file\n , windows_verbatim_args = FALSE\n )\n invisible(p$wait())\n } else {\n message(paste0(\n \"Using system2() to run shell commands. Installing \"\n , \"'processx' with install.packages('processx') might \"\n , \"make this faster.\"\n ))\n exit_code <- system2(\n command = command\n , args = shQuote(args)\n , stdout = out_file\n )\n if (exit_code != 0L) {\n stop(paste0(\"Command failed with exit code: \", exit_code))\n }\n }\n return(invisible(NULL))\n}\n\n# use objdump to dump all the symbols\nOBJDUMP_FILE <- file.path(tempdir(), \"objdump-out.txt\")\n.pipe_shell_command_to_stdout(\n command = \"objdump\"\n , args = c(\"-p\", IN_DLL_FILE)\n , out_file = OBJDUMP_FILE\n)\n\nobjdump_results <- readLines(OBJDUMP_FILE)\nresult <- file.remove(OBJDUMP_FILE)\n\n# Only one table in the objdump results matters for our purposes,\n# see https://www.cs.colorado.edu/~main/cs1300/doc/mingwfaq.html\nstart_index <- which(\n grepl(\n pattern = \"[Ordinal/Name Pointer] Table\"\n , x = objdump_results\n , fixed = TRUE\n )\n)\nempty_lines <- which(objdump_results == \"\")\nend_of_table <- empty_lines[empty_lines > start_index][1L]\n\n# Read the contents of the table\nexported_symbols <- objdump_results[(start_index + 1L):end_of_table]\nexported_symbols <- gsub(\"\\t\", \"\", exported_symbols, fixed = TRUE)\nexported_symbols <- gsub(\".*\\\\] \", \"\", exported_symbols)\nexported_symbols <- gsub(\" \", \"\", exported_symbols, fixed = TRUE)\n\n# Write R.def file\nwriteLines(\n text = c(\n paste0(\"LIBRARY \\\"\", DLL_BASE_NAME, \"\\\"\")\n , \"EXPORTS\"\n , exported_symbols\n )\n , con = OUT_DEF_FILE\n , sep = \"\\n\"\n)\nmessage(sprintf(\"Successfully created '%s'\", OUT_DEF_FILE))\n" }, { "path": "R-package/man/a-compatibility-note-for-saveRDS-save.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/utils.R\n\\name{a-compatibility-note-for-saveRDS-save}\n\\alias{a-compatibility-note-for-saveRDS-save}\n\\title{Model Serialization and Compatibility}\n\\description{\nWhen it comes to serializing XGBoost models, it's possible to use R serializers such as\n\\code{\\link[=save]{save()}} or \\code{\\link[=saveRDS]{saveRDS()}} to serialize an XGBoost model object, but XGBoost also provides\nits own serializers with better compatibility guarantees, which allow loading\nsaid models in other language bindings of XGBoost.\n\nNote that an \\code{xgb.Booster} object (\\strong{as produced by \\code{\\link[=xgb.train]{xgb.train()}}}, see rest of the doc\nfor objects produced by \\code{\\link[=xgboost]{xgboost()}}), outside of its core components, might also keep:\n\\itemize{\n\\item Additional model configuration (accessible through \\code{\\link[=xgb.config]{xgb.config()}}), which includes\nmodel fitting parameters like \\code{max_depth} and runtime parameters like \\code{nthread}.\nThese are not necessarily useful for prediction/importance/plotting.\n\\item Additional R specific attributes - e.g. results of callbacks, such as evaluation logs,\nwhich are kept as a \\code{data.table} object, accessible through\n\\code{attributes(model)$evaluation_log} if present.\n}\n\nThe first one (configurations) does not have the same compatibility guarantees as\nthe model itself, including attributes that are set and accessed through\n\\code{\\link[=xgb.attributes]{xgb.attributes()}} - that is, such configuration might be lost after loading the\nbooster in a different XGBoost version, regardless of the serializer that was used.\nThese are saved when using \\code{\\link[=saveRDS]{saveRDS()}}, but will be discarded if loaded into an\nincompatible XGBoost version. They are not saved when using XGBoost's\nserializers from its public interface including \\code{\\link[=xgb.save]{xgb.save()}} and \\code{\\link[=xgb.save.raw]{xgb.save.raw()}}.\n\nThe second ones (R attributes) are not part of the standard XGBoost model structure,\nand thus are not saved when using XGBoost's own serializers. These attributes are\nonly used for informational purposes, such as keeping track of evaluation metrics as\nthe model was fit, or saving the R call that produced the model, but are otherwise\nnot used for prediction / importance / plotting / etc.\nThese R attributes are only preserved when using R's serializers.\n\nIn addition to the regular \\code{xgb.Booster} objects produced by \\code{\\link[=xgb.train]{xgb.train()}}, the\nfunction \\code{\\link[=xgboost]{xgboost()}} produces objects with a different subclass \\code{xgboost} (which\ninherits from \\code{xgb.Booster}), which keeps other additional metadata as R attributes\nsuch as class names in classification problems, and which has a dedicated \\code{predict}\nmethod that uses different defaults and takes different argument names. XGBoost's\nown serializers can work with this \\code{xgboost} class, but as they do not keep R\nattributes, the resulting object, when deserialized, is downcasted to the regular\n\\code{xgb.Booster} class (i.e. it loses the metadata, and the resulting object will use\n\\code{\\link[=predict.xgb.Booster]{predict.xgb.Booster()}} instead of \\code{\\link[=predict.xgboost]{predict.xgboost()}}) - for these \\code{xgboost} objects,\n\\code{saveRDS} might thus be a better option if the extra functionalities are needed.\n\nNote that XGBoost models in R starting from version \\verb{2.1.0} and onwards, and\nXGBoost models before version \\verb{2.1.0}; have a very different R object structure and\nare incompatible with each other. Hence, models that were saved with R serializers\nlike \\code{\\link[=saveRDS]{saveRDS()}} or \\code{\\link[=save]{save()}} before version \\verb{2.1.0} will not work with latter\n\\code{xgboost} versions and vice versa. Be aware that the structure of R model objects\ncould in theory change again in the future, so XGBoost's serializers should be\npreferred for long-term storage.\n\nFurthermore, note that model objects from XGBoost might not be serializable with third-party\nR packages like \\code{qs} or \\code{qs2}.\n}\n\\details{\nUse \\code{\\link[=xgb.save]{xgb.save()}} to save the XGBoost model as a stand-alone file. You may opt into\nthe JSON format by specifying the JSON extension. To read the model back, use\n\\code{\\link[=xgb.load]{xgb.load()}}.\n\nUse \\code{\\link[=xgb.save.raw]{xgb.save.raw()}} to save the XGBoost model as a sequence (vector) of raw bytes\nin a future-proof manner. Future releases of XGBoost will be able to read the raw bytes and\nre-construct the corresponding model. To read the model back, use \\code{\\link[=xgb.load.raw]{xgb.load.raw()}}.\nThe \\code{\\link[=xgb.save.raw]{xgb.save.raw()}} function is useful if you would like to persist the XGBoost model\nas part of another R object.\n\nUse \\code{\\link[=saveRDS]{saveRDS()}} if you require the R-specific attributes that a booster might have, such\nas evaluation logs or the model class \\code{xgboost} instead of \\code{xgb.Booster}, but note that\nfuture compatibility of such objects is outside XGBoost's control as it relies on R's\nserialization format (see e.g. the details section in \\link{serialize} and \\code{\\link[=save]{save()}} from base R).\n\nFor more details and explanation about model persistence and archival, consult the page\n\\url{https://xgboost.readthedocs.io/en/latest/tutorials/saving_model.html}.\n}\n\\examples{\n\\dontshow{RhpcBLASctl::omp_set_num_threads(1)}\ndata(agaricus.train, package = \"xgboost\")\n\nbst <- xgb.train(\n data = xgb.DMatrix(agaricus.train$data, label = agaricus.train$label, nthread = 1),\n nrounds = 2,\n params = xgb.params(\n max_depth = 2,\n nthread = 2,\n objective = \"binary:logistic\"\n )\n)\n\n# Save as a stand-alone file; load it with xgb.load()\nfname <- file.path(tempdir(), \"xgb_model.ubj\")\nxgb.save(bst, fname)\nbst2 <- xgb.load(fname)\n\n# Save as a stand-alone file (JSON); load it with xgb.load()\nfname <- file.path(tempdir(), \"xgb_model.json\")\nxgb.save(bst, fname)\nbst2 <- xgb.load(fname)\n\n# Save as a raw byte vector; load it with xgb.load.raw()\nxgb_bytes <- xgb.save.raw(bst)\nbst2 <- xgb.load.raw(xgb_bytes)\n\n# Persist XGBoost model as part of another R object\nobj <- list(xgb_model_bytes = xgb.save.raw(bst), description = \"My first XGBoost model\")\n# Persist the R object. Here, saveRDS() is okay, since it doesn't persist\n# xgb.Booster directly. What's being persisted is the future-proof byte representation\n# as given by xgb.save.raw().\nfname <- file.path(tempdir(), \"my_object.Rds\")\nsaveRDS(obj, fname)\n# Read back the R object\nobj2 <- readRDS(fname)\n# Re-construct xgb.Booster object from the bytes\nbst2 <- xgb.load.raw(obj2$xgb_model_bytes)\n\n}\n" }, { "path": "R-package/man/agaricus.test.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/xgboost.R\n\\docType{data}\n\\name{agaricus.test}\n\\alias{agaricus.test}\n\\title{Test part from Mushroom Data Set}\n\\format{\nA list containing a label vector, and a dgCMatrix object with 1611\nrows and 126 variables\n}\n\\usage{\ndata(agaricus.test)\n}\n\\description{\nThis data set is originally from the Mushroom data set,\nUCI Machine Learning Repository.\n}\n\\details{\nIt includes the following fields:\n\\itemize{\n\\item \\code{label}: The label for each record.\n\\item \\code{data}: A sparse Matrix of 'dgCMatrix' class with 126 columns.\n}\n}\n\\references{\n\\url{https://archive.ics.uci.edu/ml/datasets/Mushroom}\n\nBache, K. & Lichman, M. (2013). UCI Machine Learning Repository\n\\url{http://archive.ics.uci.edu/ml}. Irvine, CA: University of California,\nSchool of Information and Computer Science.\n}\n\\keyword{datasets}\n" }, { "path": "R-package/man/agaricus.train.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/xgboost.R\n\\docType{data}\n\\name{agaricus.train}\n\\alias{agaricus.train}\n\\title{Training part from Mushroom Data Set}\n\\format{\nA list containing a label vector, and a dgCMatrix object with 6513\nrows and 127 variables\n}\n\\usage{\ndata(agaricus.train)\n}\n\\description{\nThis data set is originally from the Mushroom data set,\nUCI Machine Learning Repository.\n}\n\\details{\nIt includes the following fields:\n\\itemize{\n\\item \\code{label}: The label for each record.\n\\item \\code{data}: A sparse Matrix of 'dgCMatrix' class with 126 columns.\n}\n}\n\\references{\n\\url{https://archive.ics.uci.edu/ml/datasets/Mushroom}\n\nBache, K. & Lichman, M. (2013). UCI Machine Learning Repository\n\\url{http://archive.ics.uci.edu/ml}. Irvine, CA: University of California,\nSchool of Information and Computer Science.\n}\n\\keyword{datasets}\n" }, { "path": "R-package/man/coef.xgb.Booster.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/xgb.Booster.R\n\\name{coef.xgb.Booster}\n\\alias{coef.xgb.Booster}\n\\title{Extract coefficients from linear booster}\n\\usage{\n\\method{coef}{xgb.Booster}(object, ...)\n}\n\\arguments{\n\\item{object}{A fitted booster of 'gblinear' type.}\n\n\\item{...}{Not used.}\n}\n\\value{\nThe extracted coefficients:\n\\itemize{\n\\item If there is only one coefficient per column in the data, will be returned as a\nvector, potentially containing the feature names if available, with the intercept\nas first column.\n\\item If there is more than one coefficient per column in the data (e.g. when using\n\\code{objective=\"multi:softmax\"}), will be returned as a matrix with dimensions equal\nto \\verb{[num_features, num_cols]}, with the intercepts as first row. Note that the column\n(classes in multi-class classification) dimension will not be named.\n}\n\nThe intercept returned here will include the 'base_score' parameter (unlike the 'bias'\nor the last coefficient in the model dump, which doesn't have 'base_score' added to it),\nhence one should get the same values from calling \\code{predict(..., outputmargin = TRUE)} and\nfrom performing a matrix multiplication with \\code{model.matrix(~., ...)}.\n\nBe aware that the coefficients are obtained by first converting them to strings and\nback, so there will always be some very small lose of precision compared to the actual\ncoefficients as used by \\link{predict.xgb.Booster}.\n}\n\\description{\nExtracts the coefficients from a 'gblinear' booster object,\nas produced by \\code{\\link[=xgb.train]{xgb.train()}} when using parameter \\code{booster=\"gblinear\"}.\n\nNote: this function will error out if passing a booster model\nwhich is not of \"gblinear\" type.\n}\n\\examples{\nlibrary(xgboost)\n\ndata(mtcars)\n\ny <- mtcars[, 1]\nx <- as.matrix(mtcars[, -1])\n\ndm <- xgb.DMatrix(data = x, label = y, nthread = 1)\nparams <- xgb.params(booster = \"gblinear\", nthread = 1)\nmodel <- xgb.train(data = dm, params = params, nrounds = 2)\ncoef(model)\n}\n" }, { "path": "R-package/man/dim.xgb.DMatrix.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/xgb.DMatrix.R\n\\name{dim.xgb.DMatrix}\n\\alias{dim.xgb.DMatrix}\n\\title{Dimensions of xgb.DMatrix}\n\\usage{\n\\method{dim}{xgb.DMatrix}(x)\n}\n\\arguments{\n\\item{x}{Object of class \\code{xgb.DMatrix}}\n}\n\\description{\nReturns a vector of numbers of rows and of columns in an \\code{xgb.DMatrix}.\n}\n\\details{\nNote: since \\code{\\link[=nrow]{nrow()}} and \\code{\\link[=ncol]{ncol()}} internally use \\code{\\link[=dim]{dim()}}, they can also\nbe directly used with an \\code{xgb.DMatrix} object.\n}\n\\examples{\ndata(agaricus.train, package = \"xgboost\")\n\ntrain <- agaricus.train\ndtrain <- xgb.DMatrix(train$data, label = train$label, nthread = 2)\n\nstopifnot(nrow(dtrain) == nrow(train$data))\nstopifnot(ncol(dtrain) == ncol(train$data))\nstopifnot(all(dim(dtrain) == dim(train$data)))\n\n}\n" }, { "path": "R-package/man/dimnames.xgb.DMatrix.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/xgb.DMatrix.R\n\\name{dimnames.xgb.DMatrix}\n\\alias{dimnames.xgb.DMatrix}\n\\alias{dimnames<-.xgb.DMatrix}\n\\title{Handling of column names of \\code{xgb.DMatrix}}\n\\usage{\n\\method{dimnames}{xgb.DMatrix}(x)\n\n\\method{dimnames}{xgb.DMatrix}(x) <- value\n}\n\\arguments{\n\\item{x}{Object of class \\code{xgb.DMatrix}.}\n\n\\item{value}{A list of two elements: the first one is ignored\nand the second one is column names}\n}\n\\description{\nOnly column names are supported for \\code{xgb.DMatrix}, thus setting of\nrow names would have no effect and returned row names would be \\code{NULL}.\n}\n\\details{\nGeneric \\code{\\link[=dimnames]{dimnames()}} methods are used by \\code{\\link[=colnames]{colnames()}}.\nSince row names are irrelevant, it is recommended to use \\code{\\link[=colnames]{colnames()}} directly.\n}\n\\examples{\ndata(agaricus.train, package = \"xgboost\")\n\ntrain <- agaricus.train\ndtrain <- xgb.DMatrix(train$data, label = train$label, nthread = 2)\ndimnames(dtrain)\ncolnames(dtrain)\ncolnames(dtrain) <- make.names(1:ncol(train$data))\nprint(dtrain, verbose = TRUE)\n\n}\n" }, { "path": "R-package/man/getinfo.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/xgb.Booster.R, R/xgb.DMatrix.R\n\\name{getinfo.xgb.Booster}\n\\alias{getinfo.xgb.Booster}\n\\alias{setinfo.xgb.Booster}\n\\alias{getinfo}\n\\alias{getinfo.xgb.DMatrix}\n\\alias{setinfo}\n\\alias{setinfo.xgb.DMatrix}\n\\title{Get or set information of xgb.DMatrix and xgb.Booster objects}\n\\usage{\n\\method{getinfo}{xgb.Booster}(object, name)\n\n\\method{setinfo}{xgb.Booster}(object, name, info)\n\ngetinfo(object, name)\n\n\\method{getinfo}{xgb.DMatrix}(object, name)\n\nsetinfo(object, name, info)\n\n\\method{setinfo}{xgb.DMatrix}(object, name, info)\n}\n\\arguments{\n\\item{object}{Object of class \\code{xgb.DMatrix} or \\code{xgb.Booster}.}\n\n\\item{name}{The name of the information field to get (see details).}\n\n\\item{info}{The specific field of information to set.}\n}\n\\value{\nFor \\code{getinfo()}, will return the requested field. For \\code{setinfo()},\nwill always return value \\code{TRUE} if it succeeds.\n}\n\\description{\nGet or set information of xgb.DMatrix and xgb.Booster objects\n}\n\\details{\nThe \\code{name} field can be one of the following for \\code{xgb.DMatrix}:\n\\itemize{\n\\item label\n\\item weight\n\\item base_margin\n\\item label_lower_bound\n\\item label_upper_bound\n\\item group\n\\item feature_type\n\\item feature_name\n\\item nrow\n}\n\nSee the documentation for \\code{\\link[=xgb.DMatrix]{xgb.DMatrix()}} for more information about these fields.\n\nFor \\code{xgb.Booster}, can be one of the following:\n\\itemize{\n\\item \\code{feature_type}\n\\item \\code{feature_name}\n}\n\nNote that, while 'qid' cannot be retrieved, it is possible to get the equivalent 'group'\nfor a DMatrix that had 'qid' assigned.\n\n\\strong{Important}: when calling \\code{\\link[=setinfo]{setinfo()}}, the objects are modified in-place. See\n\\code{\\link[=xgb.copy.Booster]{xgb.copy.Booster()}} for an idea of this in-place assignment works.\n\nSee the documentation for \\code{\\link[=xgb.DMatrix]{xgb.DMatrix()}} for possible fields that can be set\n(which correspond to arguments in that function).\n\nNote that the following fields are allowed in the construction of an \\code{xgb.DMatrix}\nbut \\strong{are not} allowed here:\n\\itemize{\n\\item data\n\\item missing\n\\item silent\n\\item nthread\n}\n}\n\\examples{\ndata(agaricus.train, package = \"xgboost\")\n\ndtrain <- with(agaricus.train, xgb.DMatrix(data, label = label, nthread = 2))\n\nlabels <- getinfo(dtrain, \"label\")\nsetinfo(dtrain, \"label\", 1 - labels)\n\nlabels2 <- getinfo(dtrain, \"label\")\nstopifnot(all(labels2 == 1 - labels))\ndata(agaricus.train, package = \"xgboost\")\n\ndtrain <- with(agaricus.train, xgb.DMatrix(data, label = label, nthread = 2))\n\nlabels <- getinfo(dtrain, \"label\")\nsetinfo(dtrain, \"label\", 1 - labels)\n\nlabels2 <- getinfo(dtrain, \"label\")\nstopifnot(all.equal(labels2, 1 - labels))\n}\n" }, { "path": "R-package/man/predict.xgb.Booster.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/xgb.Booster.R\n\\name{predict.xgb.Booster}\n\\alias{predict.xgb.Booster}\n\\title{Predict method for XGBoost model}\n\\usage{\n\\method{predict}{xgb.Booster}(\n object,\n newdata,\n missing = NA,\n outputmargin = FALSE,\n predleaf = FALSE,\n predcontrib = FALSE,\n approxcontrib = FALSE,\n predinteraction = FALSE,\n training = FALSE,\n iterationrange = NULL,\n strict_shape = FALSE,\n avoid_transpose = FALSE,\n validate_features = FALSE,\n base_margin = NULL,\n ...\n)\n}\n\\arguments{\n\\item{object}{Object of class \\code{xgb.Booster}.}\n\n\\item{newdata}{Takes \\code{data.frame}, \\code{matrix}, \\code{dgCMatrix}, \\code{dgRMatrix}, \\code{dsparseVector},\nlocal data file, or \\code{xgb.DMatrix}.\n\nFor single-row predictions on sparse data, it is recommended to use CSR format. If passing\na sparse vector, it will take it as a row vector.\n\nNote that, for repeated predictions on the same data, one might want to create a DMatrix to\npass here instead of passing R types like matrices or data frames, as predictions will be\nfaster on DMatrix.\n\nIf \\code{newdata} is a \\code{data.frame}, be aware that:\n\\itemize{\n\\item Columns will be converted to numeric if they aren't already, which could potentially make\nthe operation slower than in an equivalent \\code{matrix} object.\n\\item The order of the columns must match with that of the data from which the model was fitted\n(i.e. columns will not be referenced by their names, just by their order in the data),\nunless passing \\code{validate_features = TRUE} (which is not the default).\n\\item If the model was fitted to data with categorical columns, these columns must be of\n\\code{factor} type here, and must use the same encoding (i.e. have the same levels).\n\\item If \\code{newdata} contains any \\code{factor} columns, they will be converted to base-0\nencoding (same as during DMatrix creation) - hence, one should not pass a \\code{factor}\nunder a column which during training had a different type.\n\\item Any columns with type other than \\code{factor} will be interpreted as numeric.\n}}\n\n\\item{missing}{Float value that represents missing values in data\n(e.g., 0 or some other extreme value).\n\nThis parameter is not used when \\code{newdata} is an \\code{xgb.DMatrix} - in such cases,\nshould pass this as an argument to the DMatrix constructor instead.}\n\n\\item{outputmargin}{Whether the prediction should be returned in the form of\noriginal untransformed sum of predictions from boosting iterations' results.\nE.g., setting \\code{outputmargin = TRUE} for logistic regression would return log-odds\ninstead of probabilities.}\n\n\\item{predleaf}{Whether to predict per-tree leaf indices.}\n\n\\item{predcontrib}{Whether to return feature contributions to individual predictions (see Details).}\n\n\\item{approxcontrib}{Whether to use a fast approximation for feature contributions (see Details).}\n\n\\item{predinteraction}{Whether to return contributions of feature interactions to individual predictions (see Details).}\n\n\\item{training}{Whether the prediction result is used for training. When enabled,\nXGBoost uses the training prediction path instead of inplace prediction.}\n\n\\item{iterationrange}{Sequence of rounds/iterations from the model to use for prediction, specified by passing\na two-dimensional vector with the start and end numbers in the sequence (same format as R's \\code{seq} - i.e.\nbase-1 indexing, and inclusive of both ends).\n\nFor example, passing \\code{c(1,20)} will predict using the first twenty iterations, while passing \\code{c(1,1)} will\npredict using only the first one.\n\nIf passing \\code{NULL}, will either stop at the best iteration if the model used early stopping, or use all\nof the iterations (rounds) otherwise.\n\nIf passing \"all\", will use all of the rounds regardless of whether the model had early stopping or not.\n\nNot applicable to \\code{gblinear} booster.}\n\n\\item{strict_shape}{Whether to always return an array with the same dimensions for the given prediction mode\nregardless of the model type - meaning that, for example, both a multi-class and a binary classification\nmodel would generate output arrays with the same number of dimensions, with the 'class' dimension having\nsize equal to '1' for the binary model.\n\nIf passing \\code{FALSE} (the default), dimensions will be simplified according to the model type, so that a\nbinary classification model for example would not have a redundant dimension for 'class'.\n\nSee documentation for the return type for the exact shape of the output arrays for each prediction mode.}\n\n\\item{avoid_transpose}{Whether to output the resulting predictions in the same memory layout in which they\nare generated by the core XGBoost library, without transposing them to match the expected output shape.\n\nInternally, XGBoost uses row-major order for the predictions it generates, while R arrays use column-major\norder, hence the result needs to be transposed in order to have the expected shape when represented as\nan R array or matrix, which might be a slow operation.\n\nIf passing \\code{TRUE}, then the result will have dimensions in reverse order - for example, rows\nwill be the last dimensions instead of the first dimension.}\n\n\\item{validate_features}{When \\code{TRUE}, validate that the Booster's and newdata's\nfeature_names match (only applicable when both \\code{object} and \\code{newdata} have feature names).\n\nIf the column names differ and \\code{newdata} is not an \\code{xgb.DMatrix}, will try to reorder\nthe columns in \\code{newdata} to match with the booster's.\n\nIf the booster has feature types and \\code{newdata} is either an \\code{xgb.DMatrix} or\n\\code{data.frame}, will additionally verify that categorical columns are of the\ncorrect type in \\code{newdata}, throwing an error if they do not match.\n\nIf passing \\code{FALSE}, it is assumed that the feature names and types are the same,\nand come in the same order as in the training data.\n\nNote that this check might add some sizable latency to the predictions, so it's\nrecommended to disable it for performance-sensitive applications.}\n\n\\item{base_margin}{Base margin used for boosting from existing model (raw score that gets added to\nall observations independently of the trees in the model).\n\nIf supplied, should be either a vector with length equal to the number of rows in \\code{newdata}\n(for objectives which produces a single score per observation), or a matrix with number of\nrows matching to the number rows in \\code{newdata} and number of columns matching to the number\nof scores estimated by the model (e.g. number of classes for multi-class classification).\n\nNote that, if \\code{newdata} is an \\code{xgb.DMatrix} object, this argument will\nbe ignored as it needs to be added to the DMatrix instead (e.g. by passing it as\nan argument in its constructor, or by calling \\code{\\link[=setinfo.xgb.DMatrix]{setinfo.xgb.DMatrix()}}.}\n\n\\item{...}{Not used.}\n}\n\\value{\nA numeric vector or array, with corresponding dimensions depending on the prediction mode and on\nparameter \\code{strict_shape} as follows:\n\nIf passing \\code{strict_shape=FALSE}:\\itemize{\n\\item For regression or binary classification: a vector of length \\code{nrows}.\n\\item For multi-class and multi-target objectives: a matrix of dimensions \\verb{[nrows, ngroups]}.\n\nNote that objective variant \\code{multi:softmax} defaults towards predicting most likely class (a vector\n\\code{nrows}) instead of per-class probabilities.\n\\item For \\code{predleaf}: a matrix with one column per tree.\n\nFor multi-class / multi-target, they will be arranged so that columns in the output will have\nthe leafs from one group followed by leafs of the other group (e.g. order will be \\code{group1:feat1},\n\\code{group1:feat2}, ..., \\code{group2:feat1}, \\code{group2:feat2}, ...).\n\nIf there is more than one parallel tree (e.g. random forests), the parallel trees will be the\nlast grouping in the resulting order, which will still be 2D.\n\\item For \\code{predcontrib}: when not multi-class / multi-target, a matrix with dimensions\n\\verb{[nrows, nfeats+1]}. The last \"+ 1\" column corresponds to the baseline value.\n\nFor multi-class and multi-target objectives, will be an array with dimensions \\verb{[nrows, ngroups, nfeats+1]}.\n\nThe contribution values are on the scale of untransformed margin (e.g., for binary classification,\nthe values are log-odds deviations from the baseline).\n\\item For \\code{predinteraction}: when not multi-class / multi-target, the output is a 3D array of\ndimensions \\verb{[nrows, nfeats+1, nfeats+1]}. The off-diagonal (in the last two dimensions)\nelements represent different feature interaction contributions. The array is symmetric w.r.t. the last\ntwo dimensions. The \"+ 1\" columns corresponds to the baselines. Summing this array along the last\ndimension should produce practically the same result as \\code{predcontrib = TRUE}.\n\nFor multi-class and multi-target, will be a 4D array with dimensions \\verb{[nrows, ngroups, nfeats+1, nfeats+1]}\n}\n\nIf passing \\code{strict_shape=TRUE}, the result is always a matrix (if 2D) or array (if 3D or higher):\n\\itemize{\n\\item For normal predictions, the dimension is \\verb{[nrows, ngroups]}.\n\\item For \\code{predcontrib=TRUE}, the dimension is \\verb{[nrows, ngroups, nfeats+1]}.\n\\item For \\code{predinteraction=TRUE}, the dimension is \\verb{[nrows, ngroups, nfeats+1, nfeats+1]}.\n\\item For \\code{predleaf=TRUE}, the dimension is \\verb{[nrows, niter, ngroups, num_parallel_tree]}.\n}\n\nIf passing \\code{avoid_transpose=TRUE}, then the dimensions in all cases will be in reverse order - for\nexample, for \\code{predinteraction}, they will be \\verb{[nfeats+1, nfeats+1, ngroups, nrows]}\ninstead of \\verb{[nrows, ngroups, nfeats+1, nfeats+1]}.\n}\n\\description{\nPredict values on data based on XGBoost model.\n}\n\\details{\nNote that \\code{iterationrange} would currently do nothing for predictions from \"gblinear\",\nsince \"gblinear\" doesn't keep its boosting history.\n\nOne possible practical applications of the \\code{predleaf} option is to use the model\nas a generator of new features which capture non-linearity and interactions,\ne.g., as implemented in \\code{\\link[=xgb.create.features]{xgb.create.features()}}.\n\nSetting \\code{predcontrib = TRUE} allows to calculate contributions of each feature to\nindividual predictions. For \"gblinear\" booster, feature contributions are simply linear terms\n(feature_beta * feature_value). For \"gbtree\" booster, feature contributions are SHAP\nvalues (Lundberg 2017) that sum to the difference between the expected output\nof the model and the current prediction (where the hessian weights are used to compute the expectations).\nSetting \\code{approxcontrib = TRUE} approximates these values following the idea explained\nin \\url{http://blog.datadive.net/interpreting-random-forests/}.\n\nWith \\code{predinteraction = TRUE}, SHAP values of contributions of interaction of each pair of features\nare computed. Note that this operation might be rather expensive in terms of compute and memory.\nSince it quadratically depends on the number of features, it is recommended to perform selection\nof the most important features first. See below about the format of the returned results.\n\nThe \\code{predict()} method uses as many threads as defined in \\code{xgb.Booster} object (all by default).\nIf you want to change their number, assign a new number to \\code{nthread} using \\code{\\link[=xgb.model.parameters<-]{xgb.model.parameters<-()}}.\nNote that converting a matrix to \\code{\\link[=xgb.DMatrix]{xgb.DMatrix()}} uses multiple threads too.\n}\n\\examples{\n\\dontshow{RhpcBLASctl::omp_set_num_threads(1)}\n## binary classification:\n\ndata(agaricus.train, package = \"xgboost\")\ndata(agaricus.test, package = \"xgboost\")\n\n## Keep the number of threads to 2 for examples\nnthread <- 2\ndata.table::setDTthreads(nthread)\n\ntrain <- agaricus.train\ntest <- agaricus.test\n\nbst <- xgb.train(\n data = xgb.DMatrix(train$data, label = train$label, nthread = 1),\n nrounds = 5,\n params = xgb.params(\n max_depth = 2,\n nthread = nthread,\n objective = \"binary:logistic\"\n )\n)\n\n# use all trees by default\npred <- predict(bst, test$data)\n# use only the 1st tree\npred1 <- predict(bst, test$data, iterationrange = c(1, 1))\n\n# Predicting tree leafs:\n# the result is an nsamples X ntrees matrix\npred_leaf <- predict(bst, test$data, predleaf = TRUE)\nstr(pred_leaf)\n\n# Predicting feature contributions to predictions:\n# the result is an nsamples X (nfeatures + 1) matrix\npred_contr <- predict(bst, test$data, predcontrib = TRUE)\nstr(pred_contr)\n# verify that contributions' sums are equal to log-odds of predictions (up to float precision):\nsummary(rowSums(pred_contr) - qlogis(pred))\n# for the 1st record, let's inspect its features that had non-zero contribution to prediction:\ncontr1 <- pred_contr[1,]\ncontr1 <- contr1[-length(contr1)] # drop intercept\ncontr1 <- contr1[contr1 != 0] # drop non-contributing features\ncontr1 <- contr1[order(abs(contr1))] # order by contribution magnitude\nold_mar <- par(\"mar\")\npar(mar = old_mar + c(0,7,0,0))\nbarplot(contr1, horiz = TRUE, las = 2, xlab = \"contribution to prediction in log-odds\")\npar(mar = old_mar)\n\n\n## multiclass classification in iris dataset:\n\nlb <- as.numeric(iris$Species) - 1\nnum_class <- 3\n\nset.seed(11)\n\nbst <- xgb.train(\n data = xgb.DMatrix(as.matrix(iris[, -5], nthread = 1), label = lb),\n nrounds = 10,\n params = xgb.params(\n max_depth = 4,\n nthread = 2,\n subsample = 0.5,\n objective = \"multi:softprob\",\n num_class = num_class\n )\n)\n\n# predict for softmax returns num_class probability numbers per case:\npred <- predict(bst, as.matrix(iris[, -5]))\nstr(pred)\n# convert the probabilities to softmax labels\npred_labels <- max.col(pred) - 1\n# the following should result in the same error as seen in the last iteration\nsum(pred_labels != lb) / length(lb)\n\n# compare with predictions from softmax:\nset.seed(11)\n\nbst <- xgb.train(\n data = xgb.DMatrix(as.matrix(iris[, -5], nthread = 1), label = lb),\n nrounds = 10,\n params = xgb.params(\n max_depth = 4,\n nthread = 2,\n subsample = 0.5,\n objective = \"multi:softmax\",\n num_class = num_class\n )\n)\n\npred <- predict(bst, as.matrix(iris[, -5]))\nstr(pred)\nall.equal(pred, pred_labels)\n# prediction from using only 5 iterations should result\n# in the same error as seen in iteration 5:\npred5 <- predict(bst, as.matrix(iris[, -5]), iterationrange = c(1, 5))\nsum(pred5 != lb) / length(lb)\n\n}\n\\references{\n\\enumerate{\n\\item Scott M. Lundberg, Su-In Lee, \"A Unified Approach to Interpreting Model Predictions\",\nNIPS Proceedings 2017, \\url{https://arxiv.org/abs/1705.07874}\n\\item Scott M. Lundberg, Su-In Lee, \"Consistent feature attribution for tree ensembles\",\n\\url{https://arxiv.org/abs/1706.06060}\n}\n}\n\\seealso{\n\\code{\\link[=xgb.train]{xgb.train()}}\n}\n" }, { "path": "R-package/man/predict.xgboost.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/xgboost.R\n\\name{predict.xgboost}\n\\alias{predict.xgboost}\n\\title{Compute predictions from XGBoost model on new data}\n\\usage{\n\\method{predict}{xgboost}(\n object,\n newdata,\n type = \"response\",\n base_margin = NULL,\n iteration_range = NULL,\n validate_features = TRUE,\n ...\n)\n}\n\\arguments{\n\\item{object}{An XGBoost model object of class \\code{xgboost}, as produced by function \\code{\\link[=xgboost]{xgboost()}}.\n\nNote that there is also a lower-level \\code{\\link[=predict.xgb.Booster]{predict.xgb.Booster()}} method for models of class\n\\code{xgb.Booster} as produced by \\code{\\link[=xgb.train]{xgb.train()}}, which can also be used for \\code{xgboost} class models as\nan alternative that performs fewer validations and post-processings.}\n\n\\item{newdata}{Data on which to compute predictions from the model passed in \\code{object}. Supported\ninput classes are:\n\\itemize{\n\\item Data Frames (class \\code{data.frame} from base R and subclasses like \\code{data.table}).\n\\item Matrices (class \\code{matrix} from base R).\n\\item Sparse matrices from package \\code{Matrix}, either as class \\code{dgRMatrix} (CSR) or \\code{dgCMatrix} (CSC).\n\\item Sparse vectors from package \\code{Matrix}, which will be interpreted as containing a single\nobservation.\n}\n\nIn the case of data frames, if there are any categorical features, they should be of class\n\\code{factor} and should have the same levels as the \\code{factor} columns of the data from which the model\nwas constructed. Any columns with type other than \\code{factor} will be interpreted as numeric.\n\nIf there are named columns and the model was fitted to data with named columns, they will be\nmatched by name by default (see \\code{validate_features}).}\n\n\\item{type}{Type of prediction to make. Supported options are:\n\\itemize{\n\\item \\code{\"response\"}: will output model predictions on the scale of the response variable (e.g.\nprobabilities of belonging to the last class in the case of binary classification). Result will\nbe either a numeric vector with length matching to rows in \\code{newdata}, or a numeric matrix with\nshape \\verb{[nrows(newdata), nscores]} (for objectives that produce more than one score per\nobservation such as multi-class classification or multi-quantile regression).\n\\item \\code{\"raw\"}: will output the unprocessed boosting scores (e.g. log-odds in the case of objective\n\\code{binary:logistic}). Same output shape and type as for \\code{\"response\"}.\n\\item \\code{\"class\"}: will output the class with the highest predicted probability, returned as a \\code{factor}\n(only applicable to classification objectives) with length matching to rows in \\code{newdata}.\n\\item \\code{\"leaf\"}: will output the terminal node indices of each observation across each tree, as an\ninteger matrix of shape \\verb{[nrows(newdata), ntrees]}, or as an integer array with an extra one or\ntwo dimensions, up to \\verb{[nrows(newdata), ntrees, nscores, n_parallel_trees]} for models that\nproduce more than one score per tree and/or which have more than one parallel tree (e.g.\nrandom forests).\n\nOnly applicable to tree-based boosters (not \\code{gblinear}).\n\\item \\code{\"contrib\"}: will produce per-feature contribution estimates towards the model score for a\ngiven observation, based on SHAP values. The contribution values are on the scale of\nuntransformed margin (e.g., for binary classification, the values are log-odds deviations from\nthe baseline).\n\nOutput will be a numeric matrix with shape \\verb{[nrows, nfeatures+1]}, with the intercept being the\nlast feature, or a numeric array with shape \\verb{[nrows, nscores, nfeatures+1]} if the model\nproduces more than one score per observation.\n\\item \\code{\"interaction\"}: similar to \\code{\"contrib\"}, but computing SHAP values of contributions of\ninteraction of each pair of features. Note that this operation might be rather expensive in\nterms of compute and memory.\n\nSince it quadratically depends on the number of features, it is recommended to perform\nselection of the most important features first.\n\nOutput will be a numeric array of shape \\verb{[nrows, nfeatures+1, nfeatures+1]}, or shape\n\\verb{[nrows, nscores, nfeatures+1, nfeatures+1]} (for objectives that produce more than one score\nper observation).\n}}\n\n\\item{base_margin}{Base margin used for boosting from existing model (raw score that gets added to\nall observations independently of the trees in the model).\n\nIf supplied, should be either a vector with length equal to the number of rows in \\code{newdata}\n(for objectives which produces a single score per observation), or a matrix with number of\nrows matching to the number rows in \\code{newdata} and number of columns matching to the number\nof scores estimated by the model (e.g. number of classes for multi-class classification).}\n\n\\item{iteration_range}{Sequence of rounds/iterations from the model to use for prediction, specified by passing\na two-dimensional vector with the start and end numbers in the sequence (same format as R's \\code{seq} - i.e.\nbase-1 indexing, and inclusive of both ends).\n\nFor example, passing \\code{c(1,20)} will predict using the first twenty iterations, while passing \\code{c(1,1)} will\npredict using only the first one.\n\nIf passing \\code{NULL}, will either stop at the best iteration if the model used early stopping, or use all\nof the iterations (rounds) otherwise.\n\nIf passing \"all\", will use all of the rounds regardless of whether the model had early stopping or not.\n\nNot applicable to \\code{gblinear} booster.}\n\n\\item{validate_features}{Validate that the feature names in the data match to the feature names\nin the column, and reorder them in the data otherwise.\n\nIf passing \\code{FALSE}, it is assumed that the feature names and types are the same,\nand come in the same order as in the training data.\n\nBe aware that this only applies to column names and not to factor levels in categorical columns.\n\nNote that this check might add some sizable latency to the predictions, so it's\nrecommended to disable it for performance-sensitive applications.}\n\n\\item{...}{Not used.}\n}\n\\value{\nEither a numeric vector (for 1D outputs), numeric matrix (for 2D outputs), numeric array\n(for 3D and higher), or \\code{factor} (for class predictions). See documentation for parameter \\code{type}\nfor details about what the output type and shape will be.\n}\n\\description{\nPredict values on data based on XGBoost model.\n}\n\\examples{\ndata(\"ToothGrowth\")\ny <- ToothGrowth$supp\nx <- ToothGrowth[, -2L]\nmodel <- xgboost(x, y, nthreads = 1L, nrounds = 3L, max_depth = 2L)\npred_prob <- predict(model, x[1:5, ], type = \"response\")\npred_raw <- predict(model, x[1:5, ], type = \"raw\")\npred_class <- predict(model, x[1:5, ], type = \"class\")\n\n# Relationships between these\nmanual_probs <- 1 / (1 + exp(-pred_raw))\nmanual_class <- ifelse(manual_probs < 0.5, levels(y)[1], levels(y)[2])\n\n# They should match up to numerical precision\nround(pred_prob, 6) == round(manual_probs, 6)\npred_class == manual_class\n}\n" }, { "path": "R-package/man/print.xgb.Booster.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/xgb.Booster.R\n\\name{print.xgb.Booster}\n\\alias{print.xgb.Booster}\n\\title{Print xgb.Booster}\n\\usage{\n\\method{print}{xgb.Booster}(x, ...)\n}\n\\arguments{\n\\item{x}{An \\code{xgb.Booster} object.}\n\n\\item{...}{Not used.}\n}\n\\value{\nThe same \\code{x} object, returned invisibly\n}\n\\description{\nPrint information about \\code{xgb.Booster}.\n}\n\\examples{\ndata(agaricus.train, package = \"xgboost\")\ntrain <- agaricus.train\n\nbst <- xgb.train(\n data = xgb.DMatrix(train$data, label = train$label, nthread = 1),\n nrounds = 2,\n params = xgb.params(\n max_depth = 2,\n nthread = 2,\n objective = \"binary:logistic\"\n )\n)\n\nattr(bst, \"myattr\") <- \"memo\"\n\nprint(bst)\n}\n" }, { "path": "R-package/man/print.xgb.DMatrix.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/xgb.DMatrix.R\n\\name{print.xgb.DMatrix}\n\\alias{print.xgb.DMatrix}\n\\title{Print xgb.DMatrix}\n\\usage{\n\\method{print}{xgb.DMatrix}(x, verbose = FALSE, ...)\n}\n\\arguments{\n\\item{x}{An xgb.DMatrix object.}\n\n\\item{verbose}{Whether to print colnames (when present).}\n\n\\item{...}{Not currently used.}\n}\n\\description{\nPrint information about xgb.DMatrix.\nCurrently it displays dimensions and presence of info-fields and colnames.\n}\n\\examples{\ndata(agaricus.train, package = \"xgboost\")\n\ndtrain <- with(agaricus.train, xgb.DMatrix(data, label = label, nthread = 2))\ndtrain\n\nprint(dtrain, verbose = TRUE)\n\n}\n" }, { "path": "R-package/man/print.xgb.cv.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/xgb.cv.R\n\\name{print.xgb.cv.synchronous}\n\\alias{print.xgb.cv.synchronous}\n\\title{Print xgb.cv result}\n\\usage{\n\\method{print}{xgb.cv.synchronous}(x, verbose = FALSE, ...)\n}\n\\arguments{\n\\item{x}{An \\code{xgb.cv.synchronous} object.}\n\n\\item{verbose}{Whether to print detailed data.}\n\n\\item{...}{Passed to \\code{data.table.print()}.}\n}\n\\description{\nPrints formatted results of \\code{\\link[=xgb.cv]{xgb.cv()}}.\n}\n\\details{\nWhen not verbose, it would only print the evaluation results,\nincluding the best iteration (when available).\n}\n\\examples{\ndata(agaricus.train, package = \"xgboost\")\n\ntrain <- agaricus.train\ncv <- xgb.cv(\n data = xgb.DMatrix(train$data, label = train$label, nthread = 1),\n nfold = 5,\n nrounds = 2,\n params = xgb.params(\n max_depth = 2,\n nthread = 2,\n objective = \"binary:logistic\"\n )\n)\nprint(cv)\nprint(cv, verbose = TRUE)\n\n}\n" }, { "path": "R-package/man/print.xgboost.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/xgboost.R\n\\name{print.xgboost}\n\\alias{print.xgboost}\n\\title{Print info from XGBoost model}\n\\usage{\n\\method{print}{xgboost}(x, ...)\n}\n\\arguments{\n\\item{x}{An XGBoost model object of class \\code{xgboost}, as produced by function \\code{\\link[=xgboost]{xgboost()}}.}\n\n\\item{...}{Not used.}\n}\n\\value{\nSame object \\code{x}, after printing its info.\n}\n\\description{\nPrints basic properties of an XGBoost model object.\n}\n" }, { "path": "R-package/man/variable.names.xgb.Booster.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/xgb.Booster.R\n\\name{variable.names.xgb.Booster}\n\\alias{variable.names.xgb.Booster}\n\\title{Get Features Names from Booster}\n\\usage{\n\\method{variable.names}{xgb.Booster}(object, ...)\n}\n\\arguments{\n\\item{object}{An \\code{xgb.Booster} object.}\n\n\\item{...}{Not used.}\n}\n\\description{\nReturns the feature / variable / column names from a fitted\nbooster object, which are set automatically during the call to \\code{\\link[=xgb.train]{xgb.train()}}\nfrom the DMatrix names, or which can be set manually through \\code{\\link[=setinfo]{setinfo()}}.\n\nIf the object doesn't have feature names, will return \\code{NULL}.\n\nIt is equivalent to calling \\code{getinfo(object, \"feature_name\")}.\n}\n" }, { "path": "R-package/man/xgb.Callback.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/callbacks.R\n\\name{xgb.Callback}\n\\alias{xgb.Callback}\n\\title{XGBoost Callback Constructor}\n\\usage{\nxgb.Callback(\n cb_name = \"custom_callback\",\n env = new.env(),\n f_before_training = function(env, model, data, evals, begin_iteration, end_iteration)\n NULL,\n f_before_iter = function(env, model, data, evals, iteration) NULL,\n f_after_iter = function(env, model, data, evals, iteration, iter_feval) NULL,\n f_after_training = function(env, model, data, evals, iteration, final_feval,\n prev_cb_res) NULL\n)\n}\n\\arguments{\n\\item{cb_name}{Name for the callback.\n\nIf the callback produces some non-NULL result (from executing the function passed under\n\\code{f_after_training}), that result will be added as an R attribute to the resulting booster\n(or as a named element in the result of CV), with the attribute name specified here.\n\nNames of callbacks must be unique - i.e. there cannot be two callbacks with the same name.}\n\n\\item{env}{An environment object that will be passed to the different functions in the callback.\nNote that this environment will not be shared with other callbacks.}\n\n\\item{f_before_training}{A function that will be executed before the training has started.\n\nIf passing \\code{NULL} for this or for the other function inputs, then no function will be executed.\n\nIf passing a function, it will be called with parameters supplied as non-named arguments\nmatching the function signatures that are shown in the default value for each function argument.}\n\n\\item{f_before_iter}{A function that will be executed before each boosting round.\n\nThis function can signal whether the training should be finalized or not, by outputting\na value that evaluates to \\code{TRUE} - i.e. if the output from the function provided here at\na given round is \\code{TRUE}, then training will be stopped before the current iteration happens.\n\nReturn values of \\code{NULL} will be interpreted as \\code{FALSE}.}\n\n\\item{f_after_iter}{A function that will be executed after each boosting round.\n\nThis function can signal whether the training should be finalized or not, by outputting\na value that evaluates to \\code{TRUE} - i.e. if the output from the function provided here at\na given round is \\code{TRUE}, then training will be stopped at that round.\n\nReturn values of \\code{NULL} will be interpreted as \\code{FALSE}.}\n\n\\item{f_after_training}{A function that will be executed after training is finished.\n\nThis function can optionally output something non-NULL, which will become part of the R\nattributes of the booster (assuming one passes \\code{keep_extra_attributes=TRUE} to \\code{\\link[=xgb.train]{xgb.train()}})\nunder the name supplied for parameter \\code{cb_name} imn the case of \\code{\\link[=xgb.train]{xgb.train()}}; or a part\nof the named elements in the result of \\code{\\link[=xgb.cv]{xgb.cv()}}.}\n}\n\\value{\nAn \\code{xgb.Callback} object, which can be passed to \\code{\\link[=xgb.train]{xgb.train()}} or \\code{\\link[=xgb.cv]{xgb.cv()}}.\n}\n\\description{\nConstructor for defining the structure of callback functions that can be executed\nat different stages of model training (before / after training, before / after each boosting\niteration).\n}\n\\details{\nArguments that will be passed to the supplied functions are as follows:\n\\itemize{\n\\item env The same environment that is passed under argument \\code{env}.\n\nIt may be modified by the functions in order to e.g. keep tracking of what happens\nacross iterations or similar.\n\nThis environment is only used by the functions supplied to the callback, and will\nnot be kept after the model fitting function terminates (see parameter \\code{f_after_training}).\n\\item model The booster object when using \\code{\\link[=xgb.train]{xgb.train()}}, or the folds when using \\code{\\link[=xgb.cv]{xgb.cv()}}.\n\nFor \\code{\\link[=xgb.cv]{xgb.cv()}}, folds are a list with a structure as follows:\n\\itemize{\n\\item \\code{dtrain}: The training data for the fold (as an \\code{xgb.DMatrix} object).\n\\item \\code{bst}: Rhe \\code{xgb.Booster} object for the fold.\n\\item \\code{evals}: A list containing two DMatrices, with names \\code{train} and \\code{test}\n(\\code{test} is the held-out data for the fold).\n\\item \\code{index}: The indices of the hold-out data for that fold (base-1 indexing),\nfrom which the \\code{test} entry in \\code{evals} was obtained.\n}\n\nThis object should \\strong{not} be in-place modified in ways that conflict with the\ntraining (e.g. resetting the parameters for a training update in a way that resets\nthe number of rounds to zero in order to overwrite rounds).\n\nNote that any R attributes that are assigned to the booster during the callback functions,\nwill not be kept thereafter as the booster object variable is not re-assigned during\ntraining. It is however possible to set C-level attributes of the booster through\n\\code{\\link[=xgb.attr]{xgb.attr()}} or \\code{\\link[=xgb.attributes]{xgb.attributes()}}, which should remain available for the rest\nof the iterations and after the training is done.\n\nFor keeping variables across iterations, it's recommended to use \\code{env} instead.\n\\item data The data to which the model is being fit, as an \\code{xgb.DMatrix} object.\n\nNote that, for \\code{\\link[=xgb.cv]{xgb.cv()}}, this will be the full data, while data for the specific\nfolds can be found in the \\code{model} object.\n\\item evals The evaluation data, as passed under argument \\code{evals} to \\code{\\link[=xgb.train]{xgb.train()}}.\n\nFor \\code{\\link[=xgb.cv]{xgb.cv()}}, this will always be \\code{NULL}.\n\\item begin_iteration Index of the first boosting iteration that will be executed (base-1 indexing).\n\nThis will typically be '1', but when using training continuation, depending on the\nparameters for updates, boosting rounds will be continued from where the previous\nmodel ended, in which case this will be larger than 1.\n\\item end_iteration Index of the last boostign iteration that will be executed\n(base-1 indexing, inclusive of this end).\n\nIt should match with argument \\code{nrounds} passed to \\code{\\link[=xgb.train]{xgb.train()}} or \\code{\\link[=xgb.cv]{xgb.cv()}}.\n\nNote that boosting might be interrupted before reaching this last iteration, for\nexample by using the early stopping callback \\code{\\link[=xgb.cb.early.stop]{xgb.cb.early.stop()}}.\n\\item iteration Index of the iteration number that is being executed (first iteration\nwill be the same as parameter \\code{begin_iteration}, then next one will add +1, and so on).\n\\item iter_feval Evaluation metrics for \\code{evals} that were supplied, either\ndetermined by the objective, or by parameter \\code{custom_metric}.\n\nFor \\code{\\link[=xgb.train]{xgb.train()}}, this will be a named vector with one entry per element in\n\\code{evals}, where the names are determined as 'evals name' + '-' + 'metric name' - for\nexample, if \\code{evals} contains an entry named \"tr\" and the metric is \"rmse\",\nthis will be a one-element vector with name \"tr-rmse\".\n\nFor \\code{\\link[=xgb.cv]{xgb.cv()}}, this will be a 2d matrix with dimensions \\verb{[length(evals), nfolds]},\nwhere the row names will follow the same naming logic as the one-dimensional vector\nthat is passed in \\code{\\link[=xgb.train]{xgb.train()}}.\n\nNote that, internally, the built-in callbacks such as \\link{xgb.cb.print.evaluation} summarize\nthis table by calculating the row-wise means and standard deviations.\n\\item final_feval The evaluation results after the last boosting round is executed\n(same format as \\code{iter_feval}, and will be the exact same input as passed under\n\\code{iter_feval} to the last round that is executed during model fitting).\n\\item prev_cb_res Result from a previous run of a callback sharing the same name\n(as given by parameter \\code{cb_name}) when conducting training continuation, if there\nwas any in the booster R attributes.\n\nSometimes, one might want to append the new results to the previous one, and this will\nbe done automatically by the built-in callbacks such as \\link{xgb.cb.evaluation.log},\nwhich will append the new rows to the previous table.\n\nIf no such previous callback result is available (which it never will when fitting\na model from start instead of updating an existing model), this will be \\code{NULL}.\n\nFor \\code{\\link[=xgb.cv]{xgb.cv()}}, which doesn't support training continuation, this will always be \\code{NULL}.\n}\n\nThe following names (\\code{cb_name} values) are reserved for internal callbacks:\n\\itemize{\n\\item print_evaluation\n\\item evaluation_log\n\\item reset_parameters\n\\item early_stop\n\\item save_model\n\\item cv_predict\n\\item gblinear_history\n}\n\nThe following names are reserved for other non-callback attributes:\n\\itemize{\n\\item names\n\\item class\n\\item call\n\\item params\n\\item niter\n\\item nfeatures\n\\item folds\n}\n\nWhen using the built-in early stopping callback (\\link{xgb.cb.early.stop}), said callback\nwill always be executed before the others, as it sets some booster C-level attributes\nthat other callbacks might also use. Otherwise, the order of execution will match with\nthe order in which the callbacks are passed to the model fitting function.\n}\n\\examples{\n# Example constructing a custom callback that calculates\n# squared error on the training data (no separate test set),\n# and outputs the per-iteration results.\nssq_callback <- xgb.Callback(\n cb_name = \"ssq\",\n f_before_training = function(env, model, data, evals,\n begin_iteration, end_iteration) {\n # A vector to keep track of a number at each iteration\n env$logs <- rep(NA_real_, end_iteration - begin_iteration + 1)\n },\n f_after_iter = function(env, model, data, evals, iteration, iter_feval) {\n # This calculates the sum of squared errors on the training data.\n # Note that this can be better done by passing an 'evals' entry,\n # but this demonstrates a way in which callbacks can be structured.\n pred <- predict(model, data)\n err <- pred - getinfo(data, \"label\")\n sq_err <- sum(err^2)\n env$logs[iteration] <- sq_err\n cat(\n sprintf(\n \"Squared error at iteration \\%d: \\%.2f\\n\",\n iteration, sq_err\n )\n )\n\n # A return value of 'TRUE' here would signal to finalize the training\n return(FALSE)\n },\n f_after_training = function(env, model, data, evals, iteration,\n final_feval, prev_cb_res) {\n return(env$logs)\n }\n)\n\ndata(mtcars)\n\ny <- mtcars$mpg\nx <- as.matrix(mtcars[, -1])\n\ndm <- xgb.DMatrix(x, label = y, nthread = 1)\nmodel <- xgb.train(\n data = dm,\n params = xgb.params(objective = \"reg:squarederror\", nthread = 1),\n nrounds = 5,\n callbacks = list(ssq_callback)\n)\n\n# Result from 'f_after_iter' will be available as an attribute\nattributes(model)$ssq\n}\n\\seealso{\nBuilt-in callbacks:\n\\itemize{\n\\item \\link{xgb.cb.print.evaluation}\n\\item \\link{xgb.cb.evaluation.log}\n\\item \\link{xgb.cb.reset.parameters}\n\\item \\link{xgb.cb.early.stop}\n\\item \\link{xgb.cb.save.model}\n\\item \\link{xgb.cb.cv.predict}\n\\item \\link{xgb.cb.gblinear.history}\n}\n}\n" }, { "path": "R-package/man/xgb.DMatrix.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/xgb.DMatrix.R\n\\name{xgb.DMatrix}\n\\alias{xgb.DMatrix}\n\\alias{xgb.QuantileDMatrix}\n\\title{Construct xgb.DMatrix object}\n\\usage{\nxgb.DMatrix(\n data,\n label = NULL,\n weight = NULL,\n base_margin = NULL,\n missing = NA,\n silent = FALSE,\n feature_names = colnames(data),\n feature_types = NULL,\n nthread = NULL,\n group = NULL,\n qid = NULL,\n label_lower_bound = NULL,\n label_upper_bound = NULL,\n feature_weights = NULL,\n data_split_mode = \"row\",\n ...\n)\n\nxgb.QuantileDMatrix(\n data,\n label = NULL,\n weight = NULL,\n base_margin = NULL,\n missing = NA,\n feature_names = colnames(data),\n feature_types = NULL,\n nthread = NULL,\n group = NULL,\n qid = NULL,\n label_lower_bound = NULL,\n label_upper_bound = NULL,\n feature_weights = NULL,\n ref = NULL,\n max_bin = NULL\n)\n}\n\\arguments{\n\\item{data}{Data from which to create a DMatrix, which can then be used for fitting models or\nfor getting predictions out of a fitted model.\n\nSupported input types are as follows:\n\\itemize{\n\\item \\code{matrix} objects, with types \\code{numeric}, \\code{integer}, or \\code{logical}.\n\\item \\code{data.frame} objects, with columns of types \\code{numeric}, \\code{integer}, \\code{logical}, or \\code{factor}\n}\n\nNote that xgboost uses base-0 encoding for categorical types, hence \\code{factor} types (which use base-1\nencoding') will be converted inside the function call. Be aware that the encoding used for \\code{factor}\ntypes is not kept as part of the model, so in subsequent calls to \\code{predict}, it is the user's\nresponsibility to ensure that factor columns have the same levels as the ones from which the DMatrix\nwas constructed.\n\nOther column types are not supported.\n\\itemize{\n\\item CSR matrices, as class \\code{dgRMatrix} from package \\code{Matrix}.\n\\item CSC matrices, as class \\code{dgCMatrix} from package \\code{Matrix}.\n}\n\nThese are \\strong{not} supported by \\code{xgb.QuantileDMatrix}.\n\\itemize{\n\\item XGBoost's own binary format for DMatrices, as produced by \\code{\\link[=xgb.DMatrix.save]{xgb.DMatrix.save()}}.\n\\item Single-row CSR matrices, as class \\code{dsparseVector} from package \\code{Matrix}, which is interpreted\nas a single row (only when making predictions from a fitted model).\n}}\n\n\\item{label}{Label of the training data. For classification problems, should be passed encoded as\nintegers with numeration starting at zero.}\n\n\\item{weight}{Weight for each instance.\n\nNote that, for ranking task, weights are per-group. In ranking task, one weight\nis assigned to each group (not each data point). This is because we\nonly care about the relative ordering of data points within each group,\nso it doesn't make sense to assign weights to individual data points.}\n\n\\item{base_margin}{Base margin used for boosting from existing model.\n\nIn the case of multi-output models, one can also pass multi-dimensional base_margin.}\n\n\\item{missing}{A float value to represents missing values in data (not used when creating DMatrix\nfrom text files). It is useful to change when a zero, infinite, or some other\nextreme value represents missing values in data.}\n\n\\item{silent}{whether to suppress printing an informational message after loading from a file.}\n\n\\item{feature_names}{Set names for features. Overrides column names in data frame and matrix.\n\nNote: columns are not referenced by name when calling \\code{predict}, so the column order there\nmust be the same as in the DMatrix construction, regardless of the column names.}\n\n\\item{feature_types}{Set types for features.\n\nIf \\code{data} is a \\code{data.frame} and passing \\code{feature_types} is not supplied,\nfeature types will be deduced automatically from the column types.\n\nOtherwise, one can pass a character vector with the same length as number of columns in \\code{data},\nwith the following possible values:\n\\itemize{\n\\item \"c\", which represents categorical columns.\n\\item \"q\", which represents numeric columns.\n\\item \"int\", which represents integer columns.\n\\item \"i\", which represents logical (boolean) columns.\n}\n\nNote that, while categorical types are treated differently from the rest for model fitting\npurposes, the other types do not influence the generated model, but have effects in other\nfunctionalities such as feature importances.\n\n\\strong{Important}: Categorical features, if specified manually through \\code{feature_types}, must\nbe encoded as integers with numeration starting at zero, and the same encoding needs to be\napplied when passing data to \\code{\\link[=predict]{predict()}}. Even if passing \\code{factor} types, the encoding will\nnot be saved, so make sure that \\code{factor} columns passed to \\code{predict} have the same \\code{levels}.}\n\n\\item{nthread}{Number of threads used for creating DMatrix.}\n\n\\item{group}{Group size for all ranking group.}\n\n\\item{qid}{Query ID for data samples, used for ranking.}\n\n\\item{label_lower_bound}{Lower bound for survival training.}\n\n\\item{label_upper_bound}{Upper bound for survival training.}\n\n\\item{feature_weights}{Set feature weights for column sampling.}\n\n\\item{data_split_mode}{Not used yet. This parameter is for distributed training, which is not yet available for the R package.}\n\n\\item{...}{Not used.\n\nSome arguments that were part of this function in previous XGBoost versions are currently\ndeprecated or have been renamed. If a deprecated or renamed argument is passed, will throw\na warning (by default) and use its current equivalent instead. This warning will become an\nerror if using the \\link[=xgboost-options]{'strict mode' option}.\n\nIf some additional argument is passed that is neither a current function argument nor\na deprecated or renamed argument, a warning or error will be thrown depending on the\n'strict mode' option.\n\n\\bold{Important:} \\code{...} will be removed in a future version, and all the current\ndeprecation warnings will become errors. Please use only arguments that form part of\nthe function signature.}\n\n\\item{ref}{The training dataset that provides quantile information, needed when creating\nvalidation/test dataset with \\code{\\link[=xgb.QuantileDMatrix]{xgb.QuantileDMatrix()}}. Supplying the training DMatrix\nas a reference means that the same quantisation applied to the training data is\napplied to the validation/test data}\n\n\\item{max_bin}{The number of histogram bin, should be consistent with the training parameter\n\\code{max_bin}.\n\nThis is only supported when constructing a QuantileDMatrix.}\n}\n\\value{\nAn 'xgb.DMatrix' object. If calling \\code{xgb.QuantileDMatrix}, it will have additional\nsubclass \\code{xgb.QuantileDMatrix}.\n}\n\\description{\nConstruct an 'xgb.DMatrix' object from a given data source, which can then be passed to functions\nsuch as \\code{\\link[=xgb.train]{xgb.train()}} or \\code{\\link[=predict]{predict()}}.\n}\n\\details{\nFunction \\code{xgb.QuantileDMatrix()} will construct a DMatrix with quantization for the histogram\nmethod already applied to it, which can be used to reduce memory usage (compared to using a\na regular DMatrix first and then creating a quantization out of it) when using the histogram\nmethod (\\code{tree_method = \"hist\"}, which is the default algorithm), but is not usable for the\nsorted-indices method (\\code{tree_method = \"exact\"}), nor for the approximate method\n(\\code{tree_method = \"approx\"}).\n\nNote that DMatrix objects are not serializable through R functions such as \\code{\\link[=saveRDS]{saveRDS()}} or \\code{\\link[=save]{save()}}.\nIf a DMatrix gets serialized and then de-serialized (for example, when saving data in an R session or caching\nchunks in an Rmd file), the resulting object will not be usable anymore and will need to be reconstructed\nfrom the original source of data.\n}\n\\examples{\ndata(agaricus.train, package = \"xgboost\")\n\n## Keep the number of threads to 1 for examples\nnthread <- 1\ndata.table::setDTthreads(nthread)\ndtrain <- with(\n agaricus.train, xgb.DMatrix(data, label = label, nthread = nthread)\n)\nfname <- file.path(tempdir(), \"xgb.DMatrix.data\")\nxgb.DMatrix.save(dtrain, fname)\ndtrain <- xgb.DMatrix(fname, nthread = 1)\n}\n" }, { "path": "R-package/man/xgb.DMatrix.hasinfo.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/xgb.DMatrix.R\n\\name{xgb.DMatrix.hasinfo}\n\\alias{xgb.DMatrix.hasinfo}\n\\title{Check whether DMatrix object has a field}\n\\usage{\nxgb.DMatrix.hasinfo(object, info)\n}\n\\arguments{\n\\item{object}{The DMatrix object to check for the given \\code{info} field.}\n\n\\item{info}{The field to check for presence or absence in \\code{object}.}\n}\n\\description{\nChecks whether an xgb.DMatrix object has a given field assigned to\nit, such as weights, labels, etc.\n}\n\\examples{\nx <- matrix(1:10, nrow = 5)\ndm <- xgb.DMatrix(x, nthread = 1)\n\n# 'dm' so far does not have any fields set\nxgb.DMatrix.hasinfo(dm, \"label\")\n\n# Fields can be added after construction\nsetinfo(dm, \"label\", 1:5)\nxgb.DMatrix.hasinfo(dm, \"label\")\n}\n\\seealso{\n\\code{\\link[=xgb.DMatrix]{xgb.DMatrix()}}, \\code{\\link[=getinfo.xgb.DMatrix]{getinfo.xgb.DMatrix()}}, \\code{\\link[=setinfo.xgb.DMatrix]{setinfo.xgb.DMatrix()}}\n}\n" }, { "path": "R-package/man/xgb.DMatrix.save.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/xgb.DMatrix.save.R\n\\name{xgb.DMatrix.save}\n\\alias{xgb.DMatrix.save}\n\\title{Save xgb.DMatrix object to binary file}\n\\usage{\nxgb.DMatrix.save(dmatrix, fname)\n}\n\\arguments{\n\\item{dmatrix}{the \\code{xgb.DMatrix} object}\n\n\\item{fname}{the name of the file to write.}\n}\n\\description{\nSave xgb.DMatrix object to binary file\n}\n\\examples{\n\\dontshow{RhpcBLASctl::omp_set_num_threads(1)}\ndata(agaricus.train, package = \"xgboost\")\n\ndtrain <- with(agaricus.train, xgb.DMatrix(data, label = label, nthread = 2))\nfname <- file.path(tempdir(), \"xgb.DMatrix.data\")\nxgb.DMatrix.save(dtrain, fname)\ndtrain <- xgb.DMatrix(fname, nthread = 1)\n}\n" }, { "path": "R-package/man/xgb.DataBatch.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/xgb.DMatrix.R\n\\name{xgb.DataBatch}\n\\alias{xgb.DataBatch}\n\\title{Structure for Data Batches}\n\\usage{\nxgb.DataBatch(\n data,\n label = NULL,\n weight = NULL,\n base_margin = NULL,\n feature_names = colnames(data),\n feature_types = NULL,\n group = NULL,\n qid = NULL,\n label_lower_bound = NULL,\n label_upper_bound = NULL,\n feature_weights = NULL\n)\n}\n\\arguments{\n\\item{data}{Batch of data belonging to this batch.\n\nNote that not all of the input types supported by \\code{\\link[=xgb.DMatrix]{xgb.DMatrix()}} are possible\nto pass here. Supported types are:\n\\itemize{\n\\item \\code{matrix}, with types \\code{numeric}, \\code{integer}, and \\code{logical}. Note that for types\n\\code{integer} and \\code{logical}, missing values might not be automatically recognized as\nas such - see the documentation for parameter \\code{missing} in \\code{\\link[=xgb.ExtMemDMatrix]{xgb.ExtMemDMatrix()}}\nfor details on this.\n\\item \\code{data.frame}, with the same types as supported by 'xgb.DMatrix' and same\nconversions applied to it. See the documentation for parameter \\code{data} in\n\\code{\\link[=xgb.DMatrix]{xgb.DMatrix()}} for details on it.\n\\item CSR matrices, as class \\code{dgRMatrix} from package \"Matrix\".\n}}\n\n\\item{label}{Label of the training data. For classification problems, should be passed encoded as\nintegers with numeration starting at zero.}\n\n\\item{weight}{Weight for each instance.\n\nNote that, for ranking task, weights are per-group. In ranking task, one weight\nis assigned to each group (not each data point). This is because we\nonly care about the relative ordering of data points within each group,\nso it doesn't make sense to assign weights to individual data points.}\n\n\\item{base_margin}{Base margin used for boosting from existing model.\n\nIn the case of multi-output models, one can also pass multi-dimensional base_margin.}\n\n\\item{feature_names}{Set names for features. Overrides column names in data frame and matrix.\n\nNote: columns are not referenced by name when calling \\code{predict}, so the column order there\nmust be the same as in the DMatrix construction, regardless of the column names.}\n\n\\item{feature_types}{Set types for features.\n\nIf \\code{data} is a \\code{data.frame} and passing \\code{feature_types} is not supplied,\nfeature types will be deduced automatically from the column types.\n\nOtherwise, one can pass a character vector with the same length as number of columns in \\code{data},\nwith the following possible values:\n\\itemize{\n\\item \"c\", which represents categorical columns.\n\\item \"q\", which represents numeric columns.\n\\item \"int\", which represents integer columns.\n\\item \"i\", which represents logical (boolean) columns.\n}\n\nNote that, while categorical types are treated differently from the rest for model fitting\npurposes, the other types do not influence the generated model, but have effects in other\nfunctionalities such as feature importances.\n\n\\strong{Important}: Categorical features, if specified manually through \\code{feature_types}, must\nbe encoded as integers with numeration starting at zero, and the same encoding needs to be\napplied when passing data to \\code{\\link[=predict]{predict()}}. Even if passing \\code{factor} types, the encoding will\nnot be saved, so make sure that \\code{factor} columns passed to \\code{predict} have the same \\code{levels}.}\n\n\\item{group}{Group size for all ranking group.}\n\n\\item{qid}{Query ID for data samples, used for ranking.}\n\n\\item{label_lower_bound}{Lower bound for survival training.}\n\n\\item{label_upper_bound}{Upper bound for survival training.}\n\n\\item{feature_weights}{Set feature weights for column sampling.}\n}\n\\value{\nAn object of class \\code{xgb.DataBatch}, which is just a list containing the\ndata and parameters passed here. It does \\strong{not} inherit from \\code{xgb.DMatrix}.\n}\n\\description{\nHelper function to supply data in batches of a data iterator when\nconstructing a DMatrix from external memory through \\code{\\link[=xgb.ExtMemDMatrix]{xgb.ExtMemDMatrix()}}\nor through \\code{\\link[=xgb.QuantileDMatrix.from_iterator]{xgb.QuantileDMatrix.from_iterator()}}.\n\nThis function is \\strong{only} meant to be called inside of a callback function (which\nis passed as argument to function \\code{\\link[=xgb.DataIter]{xgb.DataIter()}} to construct a data iterator)\nwhen constructing a DMatrix through external memory - otherwise, one should call\n\\code{\\link[=xgb.DMatrix]{xgb.DMatrix()}} or \\code{\\link[=xgb.QuantileDMatrix]{xgb.QuantileDMatrix()}}.\n\nThe object that results from calling this function directly is \\strong{not} like\nan \\code{xgb.DMatrix} - i.e. cannot be used to train a model, nor to get predictions - only\npossible usage is to supply data to an iterator, from which a DMatrix is then constructed.\n\nFor more information and for example usage, see the documentation for \\code{\\link[=xgb.ExtMemDMatrix]{xgb.ExtMemDMatrix()}}.\n}\n\\seealso{\n\\code{\\link[=xgb.DataIter]{xgb.DataIter()}}, \\code{\\link[=xgb.ExtMemDMatrix]{xgb.ExtMemDMatrix()}}.\n}\n" }, { "path": "R-package/man/xgb.DataIter.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/xgb.DMatrix.R\n\\name{xgb.DataIter}\n\\alias{xgb.DataIter}\n\\title{XGBoost Data Iterator}\n\\usage{\nxgb.DataIter(env = new.env(), f_next, f_reset)\n}\n\\arguments{\n\\item{env}{An R environment to pass to the callback functions supplied here, which can be\nused to keep track of variables to determine how to handle the batches.\n\nFor example, one might want to keep track of an iteration number in this environment in order\nto know which part of the data to pass next.}\n\n\\item{f_next}{\\verb{function(env)} which is responsible for:\n\\itemize{\n\\item Accessing or retrieving the next batch of data in the iterator.\n\\item Supplying this data by calling function \\code{\\link[=xgb.DataBatch]{xgb.DataBatch()}} on it and returning the result.\n\\item Keeping track of where in the iterator batch it is or will go next, which can for example\nbe done by modifiying variables in the \\code{env} variable that is passed here.\n\\item Signaling whether there are more batches to be consumed or not, by returning \\code{NULL}\nwhen the stream of data ends (all batches in the iterator have been consumed), or the result from\ncalling \\code{\\link[=xgb.DataBatch]{xgb.DataBatch()}} when there are more batches in the line to be consumed.\n}}\n\n\\item{f_reset}{\\verb{function(env)} which is responsible for reseting the data iterator\n(i.e. taking it back to the first batch, called before and after the sequence of batches\nhas been consumed).\n\nNote that, after resetting the iterator, the batches will be accessed again, so the same data\n(and in the same order) must be passed in subsequent iterations.}\n}\n\\value{\nAn \\code{xgb.DataIter} object, containing the same inputs supplied here, which can then\nbe passed to \\code{\\link[=xgb.ExtMemDMatrix]{xgb.ExtMemDMatrix()}}.\n}\n\\description{\nInterface to create a custom data iterator in order to construct a DMatrix\nfrom external memory.\n\nThis function is responsible for generating an R object structure containing callback\nfunctions and an environment shared with them.\n\nThe output structure from this function is then meant to be passed to \\code{\\link[=xgb.ExtMemDMatrix]{xgb.ExtMemDMatrix()}},\nwhich will consume the data and create a DMatrix from it by executing the callback functions.\n\nFor more information, and for a usage example, see the documentation for \\code{\\link[=xgb.ExtMemDMatrix]{xgb.ExtMemDMatrix()}}.\n}\n\\seealso{\n\\code{\\link[=xgb.ExtMemDMatrix]{xgb.ExtMemDMatrix()}}, \\code{\\link[=xgb.DataBatch]{xgb.DataBatch()}}.\n}\n" }, { "path": "R-package/man/xgb.ExtMemDMatrix.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/xgb.DMatrix.R\n\\name{xgb.ExtMemDMatrix}\n\\alias{xgb.ExtMemDMatrix}\n\\title{DMatrix from External Data}\n\\usage{\nxgb.ExtMemDMatrix(\n data_iterator,\n cache_prefix = tempdir(),\n missing = NA,\n nthread = NULL\n)\n}\n\\arguments{\n\\item{data_iterator}{A data iterator structure as returned by \\code{\\link[=xgb.DataIter]{xgb.DataIter()}},\nwhich includes an environment shared between function calls, and functions to access\nthe data in batches on-demand.}\n\n\\item{cache_prefix}{The path of cache file, caller must initialize all the directories in this path.}\n\n\\item{missing}{A float value to represents missing values in data.\n\nNote that, while functions like \\code{\\link[=xgb.DMatrix]{xgb.DMatrix()}} can take a generic \\code{NA} and interpret it\ncorrectly for different types like \\code{numeric} and \\code{integer}, if an \\code{NA} value is passed here,\nit will not be adapted for different input types.\n\nFor example, in R \\code{integer} types, missing values are represented by integer number \\code{-2147483648}\n(since machine 'integer' types do not have an inherent 'NA' value) - hence, if one passes \\code{NA},\nwhich is interpreted as a floating-point NaN by \\code{\\link[=xgb.ExtMemDMatrix]{xgb.ExtMemDMatrix()}} and by\n\\code{\\link[=xgb.QuantileDMatrix.from_iterator]{xgb.QuantileDMatrix.from_iterator()}}, these integer missing values will not be treated as missing.\nThis should not pose any problem for \\code{numeric} types, since they do have an inheret NaN value.}\n\n\\item{nthread}{Number of threads used for creating DMatrix.}\n}\n\\value{\nAn 'xgb.DMatrix' object, with subclass 'xgb.ExtMemDMatrix', in which the data is not\nheld internally but accessed through the iterator when needed.\n}\n\\description{\nCreate a special type of XGBoost 'DMatrix' object from external data\nsupplied by an \\code{\\link[=xgb.DataIter]{xgb.DataIter()}} object, potentially passed in batches from a\nbigger set that might not fit entirely in memory.\n\nThe data supplied by the iterator is accessed on-demand as needed, multiple times,\nwithout being concatenated, but note that fields like 'label' \\strong{will} be\nconcatenated from multiple calls to the data iterator.\n\nFor more information, see the guide 'Using XGBoost External Memory Version':\n\\url{https://xgboost.readthedocs.io/en/stable/tutorials/external_memory.html}\n}\n\\details{\nBe aware that construction of external data DMatrices \\bold{will cache data on disk}\nin a compressed format, under the path supplied in \\code{cache_prefix}.\n\nExternal data is not supported for the exact tree method.\n}\n\\examples{\ndata(mtcars)\n\n# This custom environment will be passed to the iterator\n# functions at each call. It is up to the user to keep\n# track of the iteration number in this environment.\niterator_env <- as.environment(\n list(\n iter = 0,\n x = mtcars[, -1],\n y = mtcars[, 1]\n )\n)\n\n# Data is passed in two batches.\n# In this example, batches are obtained by subsetting the 'x' variable.\n# This is not advantageous to do, since the data is already loaded in memory\n# and can be passed in full in one go, but there can be situations in which\n# only a subset of the data will fit in the computer's memory, and it can\n# be loaded in batches that are accessed one-at-a-time only.\niterator_next <- function(iterator_env) {\n curr_iter <- iterator_env[[\"iter\"]]\n if (curr_iter >= 2) {\n # there are only two batches, so this signals end of the stream\n return(NULL)\n }\n\n if (curr_iter == 0) {\n x_batch <- iterator_env[[\"x\"]][1:16, ]\n y_batch <- iterator_env[[\"y\"]][1:16]\n } else {\n x_batch <- iterator_env[[\"x\"]][17:32, ]\n y_batch <- iterator_env[[\"y\"]][17:32]\n }\n on.exit({\n iterator_env[[\"iter\"]] <- curr_iter + 1\n })\n\n # Function 'xgb.DataBatch' must be called manually\n # at each batch with all the appropriate attributes,\n # such as feature names and feature types.\n return(xgb.DataBatch(data = x_batch, label = y_batch))\n}\n\n# This moves the iterator back to its beginning\niterator_reset <- function(iterator_env) {\n iterator_env[[\"iter\"]] <- 0\n}\n\ndata_iterator <- xgb.DataIter(\n env = iterator_env,\n f_next = iterator_next,\n f_reset = iterator_reset\n)\ncache_prefix <- tempdir()\n\n# DMatrix will be constructed from the iterator's batches\ndm <- xgb.ExtMemDMatrix(data_iterator, cache_prefix, nthread = 1)\n\n# After construction, can be used as a regular DMatrix\nparams <- xgb.params(nthread = 1, objective = \"reg:squarederror\")\nmodel <- xgb.train(data = dm, nrounds = 2, params = params)\n\n# Predictions can also be called on it, and should be the same\n# as if the data were passed differently.\npred_dm <- predict(model, dm)\npred_mat <- predict(model, as.matrix(mtcars[, -1]))\n}\n\\seealso{\n\\code{\\link[=xgb.DataIter]{xgb.DataIter()}}, \\code{\\link[=xgb.DataBatch]{xgb.DataBatch()}}, \\code{\\link[=xgb.QuantileDMatrix.from_iterator]{xgb.QuantileDMatrix.from_iterator()}}\n}\n" }, { "path": "R-package/man/xgb.QuantileDMatrix.from_iterator.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/xgb.DMatrix.R\n\\name{xgb.QuantileDMatrix.from_iterator}\n\\alias{xgb.QuantileDMatrix.from_iterator}\n\\title{QuantileDMatrix from External Data}\n\\usage{\nxgb.QuantileDMatrix.from_iterator(\n data_iterator,\n missing = NA,\n nthread = NULL,\n ref = NULL,\n max_bin = NULL\n)\n}\n\\arguments{\n\\item{data_iterator}{A data iterator structure as returned by \\code{\\link[=xgb.DataIter]{xgb.DataIter()}},\nwhich includes an environment shared between function calls, and functions to access\nthe data in batches on-demand.}\n\n\\item{missing}{A float value to represents missing values in data.\n\nNote that, while functions like \\code{\\link[=xgb.DMatrix]{xgb.DMatrix()}} can take a generic \\code{NA} and interpret it\ncorrectly for different types like \\code{numeric} and \\code{integer}, if an \\code{NA} value is passed here,\nit will not be adapted for different input types.\n\nFor example, in R \\code{integer} types, missing values are represented by integer number \\code{-2147483648}\n(since machine 'integer' types do not have an inherent 'NA' value) - hence, if one passes \\code{NA},\nwhich is interpreted as a floating-point NaN by \\code{\\link[=xgb.ExtMemDMatrix]{xgb.ExtMemDMatrix()}} and by\n\\code{\\link[=xgb.QuantileDMatrix.from_iterator]{xgb.QuantileDMatrix.from_iterator()}}, these integer missing values will not be treated as missing.\nThis should not pose any problem for \\code{numeric} types, since they do have an inheret NaN value.}\n\n\\item{nthread}{Number of threads used for creating DMatrix.}\n\n\\item{ref}{The training dataset that provides quantile information, needed when creating\nvalidation/test dataset with \\code{\\link[=xgb.QuantileDMatrix]{xgb.QuantileDMatrix()}}. Supplying the training DMatrix\nas a reference means that the same quantisation applied to the training data is\napplied to the validation/test data}\n\n\\item{max_bin}{The number of histogram bin, should be consistent with the training parameter\n\\code{max_bin}.\n\nThis is only supported when constructing a QuantileDMatrix.}\n}\n\\value{\nAn 'xgb.DMatrix' object, with subclass 'xgb.QuantileDMatrix'.\n}\n\\description{\nCreate an \\code{xgb.QuantileDMatrix} object (exact same class as would be returned by\ncalling function \\code{\\link[=xgb.QuantileDMatrix]{xgb.QuantileDMatrix()}}, with the same advantages and limitations) from\nexternal data supplied by \\code{\\link[=xgb.DataIter]{xgb.DataIter()}}, potentially passed in batches from\na bigger set that might not fit entirely in memory, same way as \\code{\\link[=xgb.ExtMemDMatrix]{xgb.ExtMemDMatrix()}}.\n\nNote that, while external data will only be loaded through the iterator (thus the full data\nmight not be held entirely in-memory), the quantized representation of the data will get\ncreated in-memory, being concatenated from multiple calls to the data iterator. The quantized\nversion is typically lighter than the original data, so there might be cases in which this\nrepresentation could potentially fit in memory even if the full data does not.\n\nFor more information, see the guide 'Using XGBoost External Memory Version':\n\\url{https://xgboost.readthedocs.io/en/stable/tutorials/external_memory.html}\n}\n\\seealso{\n\\code{\\link[=xgb.DataIter]{xgb.DataIter()}}, \\code{\\link[=xgb.DataBatch]{xgb.DataBatch()}}, \\code{\\link[=xgb.ExtMemDMatrix]{xgb.ExtMemDMatrix()}},\n\\code{\\link[=xgb.QuantileDMatrix]{xgb.QuantileDMatrix()}}\n}\n" }, { "path": "R-package/man/xgb.attr.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/xgb.Booster.R\n\\name{xgb.attr}\n\\alias{xgb.attr}\n\\alias{xgb.attr<-}\n\\alias{xgb.attributes}\n\\alias{xgb.attributes<-}\n\\title{Accessors for serializable attributes of a model}\n\\usage{\nxgb.attr(object, name)\n\nxgb.attr(object, name) <- value\n\nxgb.attributes(object)\n\nxgb.attributes(object) <- value\n}\n\\arguments{\n\\item{object}{Object of class \\code{xgb.Booster}. \\strong{Will be modified in-place} when assigning to it.}\n\n\\item{name}{A non-empty character string specifying which attribute is to be accessed.}\n\n\\item{value}{For \\verb{xgb.attr<-}, a value of an attribute; for \\verb{xgb.attributes<-},\nit is a list (or an object coercible to a list) with the names of attributes to set\nand the elements corresponding to attribute values.\nNon-character values are converted to character.\nWhen an attribute value is not a scalar, only the first index is used.\nUse \\code{NULL} to remove an attribute.}\n}\n\\value{\n\\itemize{\n\\item \\code{xgb.attr()} returns either a string value of an attribute\nor \\code{NULL} if an attribute wasn't stored in a model.\n\\item \\code{xgb.attributes()} returns a list of all attributes stored in a model\nor \\code{NULL} if a model has no stored attributes.\n}\n}\n\\description{\nThese methods allow to manipulate the key-value attribute strings of an XGBoost model.\n}\n\\details{\nThe primary purpose of XGBoost model attributes is to store some meta data about the model.\nNote that they are a separate concept from the object attributes in R.\nSpecifically, they refer to key-value strings that can be attached to an XGBoost model,\nstored together with the model's binary representation, and accessed later\n(from R or any other interface).\nIn contrast, any R attribute assigned to an R object of \\code{xgb.Booster} class\nwould not be saved by \\code{\\link[=xgb.save]{xgb.save()}} because an XGBoost model is an external memory object\nand its serialization is handled externally.\nAlso, setting an attribute that has the same name as one of XGBoost's parameters wouldn't\nchange the value of that parameter for a model.\nUse \\code{\\link[=xgb.model.parameters<-]{xgb.model.parameters<-()}} to set or change model parameters.\n\nThe \\verb{xgb.attributes<-} setter either updates the existing or adds one or several attributes,\nbut it doesn't delete the other existing attributes.\n\nImportant: since this modifies the booster's C object, semantics for assignment here\nwill differ from R's, as any object reference to the same booster will be modified\ntoo, while assignment of R attributes through \\verb{attributes(model)$ <- }\nwill follow the usual copy-on-write R semantics (see \\code{\\link[=xgb.copy.Booster]{xgb.copy.Booster()}} for an\nexample of these behaviors).\n}\n\\examples{\ndata(agaricus.train, package = \"xgboost\")\ntrain <- agaricus.train\n\nbst <- xgb.train(\n data = xgb.DMatrix(train$data, label = train$label, nthread = 1),\n nrounds = 2,\n params = xgb.params(\n max_depth = 2,\n nthread = 2,\n objective = \"binary:logistic\"\n )\n)\n\nxgb.attr(bst, \"my_attribute\") <- \"my attribute value\"\nprint(xgb.attr(bst, \"my_attribute\"))\nxgb.attributes(bst) <- list(a = 123, b = \"abc\")\n\nfname <- file.path(tempdir(), \"xgb.ubj\")\nxgb.save(bst, fname)\nbst1 <- xgb.load(fname)\nprint(xgb.attr(bst1, \"my_attribute\"))\nprint(xgb.attributes(bst1))\n\n# deletion:\nxgb.attr(bst1, \"my_attribute\") <- NULL\nprint(xgb.attributes(bst1))\nxgb.attributes(bst1) <- list(a = NULL, b = NULL)\nprint(xgb.attributes(bst1))\n\n}\n" }, { "path": "R-package/man/xgb.cb.cv.predict.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/callbacks.R\n\\name{xgb.cb.cv.predict}\n\\alias{xgb.cb.cv.predict}\n\\title{Callback for returning cross-validation based predictions}\n\\usage{\nxgb.cb.cv.predict(save_models = FALSE, outputmargin = FALSE)\n}\n\\arguments{\n\\item{save_models}{A flag for whether to save the folds' models.}\n\n\\item{outputmargin}{Whether to save margin predictions (same effect as passing this\nparameter to \\link{predict.xgb.Booster}).}\n}\n\\value{\nAn \\code{xgb.Callback} object, which can be passed to \\code{\\link[=xgb.cv]{xgb.cv()}},\nbut \\strong{not} to \\code{\\link[=xgb.train]{xgb.train()}}.\n}\n\\description{\nThis callback function saves predictions for all of the test folds,\nand also allows to save the folds' models.\n}\n\\details{\nPredictions are saved inside of the \\code{pred} element, which is either a vector or a matrix,\ndepending on the number of prediction outputs per data row. The order of predictions corresponds\nto the order of rows in the original dataset. Note that when a custom \\code{folds} list is\nprovided in \\code{\\link[=xgb.cv]{xgb.cv()}}, the predictions would only be returned properly when this list is a\nnon-overlapping list of k sets of indices, as in a standard k-fold CV. The predictions would not be\nmeaningful when user-provided folds have overlapping indices as in, e.g., random sampling splits.\nWhen some of the indices in the training dataset are not included into user-provided \\code{folds},\ntheir prediction value would be \\code{NA}.\n}\n" }, { "path": "R-package/man/xgb.cb.early.stop.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/callbacks.R\n\\name{xgb.cb.early.stop}\n\\alias{xgb.cb.early.stop}\n\\title{Callback to activate early stopping}\n\\usage{\nxgb.cb.early.stop(\n stopping_rounds,\n maximize = FALSE,\n metric_name = NULL,\n verbose = TRUE,\n save_best = FALSE\n)\n}\n\\arguments{\n\\item{stopping_rounds}{The number of rounds with no improvement in\nthe evaluation metric in order to stop the training.}\n\n\\item{maximize}{Whether to maximize the evaluation metric.}\n\n\\item{metric_name}{The name of an evaluation column to use as a criteria for early\nstopping. If not set, the last column would be used.\nLet's say the test data in \\code{evals} was labelled as \\code{dtest},\nand one wants to use the AUC in test data for early stopping regardless of where\nit is in the \\code{evals}, then one of the following would need to be set:\n\\code{metric_name = 'dtest-auc'} or \\code{metric_name = 'dtest_auc'}.\nAll dash '-' characters in metric names are considered equivalent to '_'.}\n\n\\item{verbose}{Whether to print the early stopping information.}\n\n\\item{save_best}{Whether training should return the best model or the last model. If\nset to \\code{TRUE}, it will only keep the boosting rounds up to the detected best\niteration, discarding the ones that come after. This parameter is not supported by\nthe \\code{xgb.cv} function and the \\code{gblinear} booster yet.}\n}\n\\value{\nAn \\code{xgb.Callback} object, which can be passed to \\code{\\link[=xgb.train]{xgb.train()}} or \\code{\\link[=xgb.cv]{xgb.cv()}}.\n}\n\\description{\nThis callback function determines the condition for early stopping.\n\nThe following attributes are assigned to the booster's object:\n\\itemize{\n\\item \\code{best_score} the evaluation score at the best iteration\n\\item \\code{best_iteration} at which boosting iteration the best score has occurred\n(0-based index for interoperability of binary models)\n}\n\nThe same values are also stored as R attributes as a result of the callback, plus an additional\nattribute \\code{stopped_by_max_rounds} which indicates whether an early stopping by the \\code{stopping_rounds}\ncondition occurred. Note that the \\code{best_iteration} that is stored under R attributes will follow\nbase-1 indexing, so it will be larger by '1' than the C-level 'best_iteration' that is accessed\nthrough \\code{\\link[=xgb.attr]{xgb.attr()}} or \\code{\\link[=xgb.attributes]{xgb.attributes()}}.\n\nAt least one dataset is required in \\code{evals} for early stopping to work.\n}\n" }, { "path": "R-package/man/xgb.cb.evaluation.log.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/callbacks.R\n\\name{xgb.cb.evaluation.log}\n\\alias{xgb.cb.evaluation.log}\n\\title{Callback for logging the evaluation history}\n\\usage{\nxgb.cb.evaluation.log()\n}\n\\value{\nAn \\code{xgb.Callback} object, which can be passed to \\code{\\link[=xgb.train]{xgb.train()}} or \\code{\\link[=xgb.cv]{xgb.cv()}}.\n}\n\\description{\nCallback for logging the evaluation history\n}\n\\details{\nThis callback creates a table with per-iteration evaluation metrics (see parameters\n\\code{evals} and \\code{custom_metric} in \\code{\\link[=xgb.train]{xgb.train()}}).\n\nNote: in the column names of the final data.table, the dash '-' character is replaced with\nthe underscore '_' in order to make the column names more like regular R identifiers.\n}\n\\seealso{\n\\link{xgb.cb.print.evaluation}\n}\n" }, { "path": "R-package/man/xgb.cb.gblinear.history.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/callbacks.R\n\\name{xgb.cb.gblinear.history}\n\\alias{xgb.cb.gblinear.history}\n\\title{Callback for collecting coefficients history of a gblinear booster}\n\\usage{\nxgb.cb.gblinear.history(sparse = FALSE)\n}\n\\arguments{\n\\item{sparse}{When set to \\code{FALSE}/\\code{TRUE}, a dense/sparse matrix is used to store the result.\nSparse format is useful when one expects only a subset of coefficients to be non-zero,\nwhen using the \"thrifty\" feature selector with fairly small number of top features\nselected per iteration.}\n}\n\\value{\nAn \\code{xgb.Callback} object, which can be passed to \\code{\\link[=xgb.train]{xgb.train()}} or \\code{\\link[=xgb.cv]{xgb.cv()}}.\n}\n\\description{\nCallback for collecting coefficients history of a gblinear booster\n}\n\\details{\nTo keep things fast and simple, gblinear booster does not internally store the history of linear\nmodel coefficients at each boosting iteration. This callback provides a workaround for storing\nthe coefficients' path, by extracting them after each training iteration.\n\nThis callback will construct a matrix where rows are boosting iterations and columns are\nfeature coefficients (same order as when calling \\link{coef.xgb.Booster}, with the intercept\ncorresponding to the first column).\n\nWhen there is more than one coefficient per feature (e.g. multi-class classification),\nthe result will be reshaped into a vector where coefficients are arranged first by features and\nthen by class (e.g. first 1 through N coefficients will be for the first class, then\ncoefficients N+1 through 2N for the second class, and so on).\n\nIf the result has only one coefficient per feature in the data, then the resulting matrix\nwill have column names matching with the feature names, otherwise (when there's more than\none coefficient per feature) the names will be composed as 'column name' + ':' + 'class index'\n(so e.g. column 'c1' for class '0' will be named 'c1:0').\n\nWith \\code{\\link[=xgb.train]{xgb.train()}}, the output is either a dense or a sparse matrix.\nWith with \\code{\\link[=xgb.cv]{xgb.cv()}}, it is a list (one element per each fold) of such matrices.\n\nFunction \\link{xgb.gblinear.history} provides an easy way to retrieve the\noutputs from this callback.\n}\n\\examples{\n#### Binary classification:\n\n## Keep the number of threads to 1 for examples\nnthread <- 1\ndata.table::setDTthreads(nthread)\n\n# In the iris dataset, it is hard to linearly separate Versicolor class from the rest\n# without considering the 2nd order interactions:\nx <- model.matrix(Species ~ .^2, iris)[, -1]\ncolnames(x)\ndtrain <- xgb.DMatrix(\n scale(x),\n label = 1 * (iris$Species == \"versicolor\"),\n nthread = nthread\n)\nparam <- xgb.params(\n booster = \"gblinear\",\n objective = \"reg:logistic\",\n eval_metric = \"auc\",\n reg_lambda = 0.0003,\n reg_alpha = 0.0003,\n nthread = nthread\n)\n\n# For 'shotgun', which is a default linear updater, using high learning_rate values may result in\n# unstable behaviour in some datasets. With this simple dataset, however, the high learning\n# rate does not break the convergence, but allows us to illustrate the typical pattern of\n# \"stochastic explosion\" behaviour of this lock-free algorithm at early boosting iterations.\nbst <- xgb.train(\n c(param, list(learning_rate = 1.)),\n dtrain,\n evals = list(tr = dtrain),\n nrounds = 200,\n callbacks = list(xgb.cb.gblinear.history())\n)\n\n# Extract the coefficients' path and plot them vs boosting iteration number:\ncoef_path <- xgb.gblinear.history(bst)\nmatplot(coef_path, type = \"l\")\n\n# With the deterministic coordinate descent updater, it is safer to use higher learning rates.\n# Will try the classical componentwise boosting which selects a single best feature per round:\nbst <- xgb.train(\n c(\n param,\n xgb.params(\n learning_rate = 0.8,\n updater = \"coord_descent\",\n feature_selector = \"thrifty\",\n top_k = 1\n )\n ),\n dtrain,\n evals = list(tr = dtrain),\n nrounds = 200,\n callbacks = list(xgb.cb.gblinear.history())\n)\nmatplot(xgb.gblinear.history(bst), type = \"l\")\n# Componentwise boosting is known to have similar effect to Lasso regularization.\n# Try experimenting with various values of top_k, learning_rate, nrounds,\n# as well as different feature_selectors.\n\n# For xgb.cv:\nbst <- xgb.cv(\n c(\n param,\n xgb.params(\n learning_rate = 0.8,\n updater = \"coord_descent\",\n feature_selector = \"thrifty\",\n top_k = 1\n )\n ),\n dtrain,\n nfold = 5,\n nrounds = 100,\n callbacks = list(xgb.cb.gblinear.history())\n)\n# coefficients in the CV fold #3\nmatplot(xgb.gblinear.history(bst)[[3]], type = \"l\")\n\n\n#### Multiclass classification:\ndtrain <- xgb.DMatrix(scale(x), label = as.numeric(iris$Species) - 1, nthread = nthread)\n\nparam <- xgb.params(\n booster = \"gblinear\",\n objective = \"multi:softprob\",\n num_class = 3,\n reg_lambda = 0.0003,\n reg_alpha = 0.0003,\n nthread = nthread\n)\n\n# For the default linear updater 'shotgun' it sometimes is helpful\n# to use smaller learning_rate to reduce instability\nbst <- xgb.train(\n c(param, list(learning_rate = 0.5)),\n dtrain,\n evals = list(tr = dtrain),\n nrounds = 50,\n callbacks = list(xgb.cb.gblinear.history())\n)\n\n# Will plot the coefficient paths separately for each class:\nmatplot(xgb.gblinear.history(bst, class_index = 0), type = \"l\")\nmatplot(xgb.gblinear.history(bst, class_index = 1), type = \"l\")\nmatplot(xgb.gblinear.history(bst, class_index = 2), type = \"l\")\n\n# CV:\nbst <- xgb.cv(\n c(param, list(learning_rate = 0.5)),\n dtrain,\n nfold = 5,\n nrounds = 70,\n callbacks = list(xgb.cb.gblinear.history(FALSE))\n)\n# 1st fold of 1st class\nmatplot(xgb.gblinear.history(bst, class_index = 0)[[1]], type = \"l\")\n\n}\n\\seealso{\n\\link{xgb.gblinear.history}, \\link{coef.xgb.Booster}.\n}\n" }, { "path": "R-package/man/xgb.cb.print.evaluation.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/callbacks.R\n\\name{xgb.cb.print.evaluation}\n\\alias{xgb.cb.print.evaluation}\n\\title{Callback for printing the result of evaluation}\n\\usage{\nxgb.cb.print.evaluation(period = 1, showsd = TRUE)\n}\n\\arguments{\n\\item{period}{Results would be printed every number of periods.}\n\n\\item{showsd}{Whether standard deviations should be printed (when available).}\n}\n\\value{\nAn \\code{xgb.Callback} object, which can be passed to \\code{\\link[=xgb.train]{xgb.train()}} or \\code{\\link[=xgb.cv]{xgb.cv()}}.\n}\n\\description{\nThe callback function prints the result of evaluation at every \\code{period} iterations.\nThe initial and the last iteration's evaluations are always printed.\n\nDoes not leave any attribute in the booster (see \\link{xgb.cb.evaluation.log} for that).\n}\n\\seealso{\n\\link{xgb.Callback}\n}\n" }, { "path": "R-package/man/xgb.cb.reset.parameters.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/callbacks.R\n\\name{xgb.cb.reset.parameters}\n\\alias{xgb.cb.reset.parameters}\n\\title{Callback for resetting booster parameters at each iteration}\n\\usage{\nxgb.cb.reset.parameters(new_params)\n}\n\\arguments{\n\\item{new_params}{List of parameters needed to be reset.\nEach element's value must be either a vector of values of length \\code{nrounds}\nto be set at each iteration,\nor a function of two parameters \\code{learning_rates(iteration, nrounds)}\nwhich returns a new parameter value by using the current iteration number\nand the total number of boosting rounds.}\n}\n\\value{\nAn \\code{xgb.Callback} object, which can be passed to \\code{\\link[=xgb.train]{xgb.train()}} or \\code{\\link[=xgb.cv]{xgb.cv()}}.\n}\n\\description{\nCallback for resetting booster parameters at each iteration\n}\n\\details{\nNote that when training is resumed from some previous model, and a function is used to\nreset a parameter value, the \\code{nrounds} argument in this function would be the\nthe number of boosting rounds in the current training.\n\nDoes not leave any attribute in the booster.\n}\n" }, { "path": "R-package/man/xgb.cb.save.model.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/callbacks.R\n\\name{xgb.cb.save.model}\n\\alias{xgb.cb.save.model}\n\\title{Callback for saving a model file}\n\\usage{\nxgb.cb.save.model(save_period = 0, save_name = \"xgboost.ubj\")\n}\n\\arguments{\n\\item{save_period}{Save the model to disk after every \\code{save_period} iterations;\n0 means save the model at the end.}\n\n\\item{save_name}{The name or path for the saved model file.\nIt can contain a \\code{\\link[=sprintf]{sprintf()}} formatting specifier to include the integer\niteration number in the file name. E.g., with \\code{save_name = 'xgboost_\\%04d.model'},\nthe file saved at iteration 50 would be named \"xgboost_0050.model\".}\n}\n\\value{\nAn \\code{xgb.Callback} object, which can be passed to \\code{\\link[=xgb.train]{xgb.train()}},\nbut \\strong{not} to \\code{\\link[=xgb.cv]{xgb.cv()}}.\n}\n\\description{\nThis callback function allows to save an xgb-model file, either periodically\nafter each \\code{save_period}'s or at the end.\n\nDoes not leave any attribute in the booster.\n}\n" }, { "path": "R-package/man/xgb.config.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/xgb.Booster.R\n\\name{xgb.config}\n\\alias{xgb.config}\n\\alias{xgb.config<-}\n\\title{Accessors for model parameters as JSON string}\n\\usage{\nxgb.config(object)\n\nxgb.config(object) <- value\n}\n\\arguments{\n\\item{object}{Object of class \\code{xgb.Booster}.\\strong{Will be modified in-place} when assigning to it.}\n\n\\item{value}{A list.}\n}\n\\value{\nParameters as a list.\n}\n\\description{\nAccessors for model parameters as JSON string\n}\n\\details{\nNote that assignment is performed in-place on the booster C object, which unlike assignment\nof R attributes, doesn't follow typical copy-on-write semantics for assignment - i.e. all references\nto the same booster will also get updated.\n\nSee \\code{\\link[=xgb.copy.Booster]{xgb.copy.Booster()}} for an example of this behavior.\n}\n\\examples{\ndata(agaricus.train, package = \"xgboost\")\n\n## Keep the number of threads to 1 for examples\nnthread <- 1\ndata.table::setDTthreads(nthread)\ntrain <- agaricus.train\n\nbst <- xgb.train(\n data = xgb.DMatrix(train$data, label = train$label, nthread = 1),\n nrounds = 2,\n params = xgb.params(\n max_depth = 2,\n nthread = nthread,\n objective = \"binary:logistic\"\n )\n)\n\nconfig <- xgb.config(bst)\n\n}\n" }, { "path": "R-package/man/xgb.copy.Booster.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/xgb.Booster.R\n\\name{xgb.copy.Booster}\n\\alias{xgb.copy.Booster}\n\\title{Deep-copies a Booster Object}\n\\usage{\nxgb.copy.Booster(model)\n}\n\\arguments{\n\\item{model}{An 'xgb.Booster' object.}\n}\n\\value{\nA deep copy of \\code{model} - it will be identical in every way, but C-level\nfunctions called on that copy will not affect the \\code{model} variable.\n}\n\\description{\nCreates a deep copy of an 'xgb.Booster' object, such that the\nC object pointer contained will be a different object, and hence functions\nlike \\code{\\link[=xgb.attr]{xgb.attr()}} will not affect the object from which it was copied.\n}\n\\examples{\nlibrary(xgboost)\n\ndata(mtcars)\n\ny <- mtcars$mpg\nx <- mtcars[, -1]\n\ndm <- xgb.DMatrix(x, label = y, nthread = 1)\n\nmodel <- xgb.train(\n data = dm,\n params = xgb.params(nthread = 1),\n nrounds = 3\n)\n\n# Set an arbitrary attribute kept at the C level\nxgb.attr(model, \"my_attr\") <- 100\nprint(xgb.attr(model, \"my_attr\"))\n\n# Just assigning to a new variable will not create\n# a deep copy - C object pointer is shared, and in-place\n# modifications will affect both objects\nmodel_shallow_copy <- model\nxgb.attr(model_shallow_copy, \"my_attr\") <- 333\n# 'model' was also affected by this change:\nprint(xgb.attr(model, \"my_attr\"))\n\nmodel_deep_copy <- xgb.copy.Booster(model)\nxgb.attr(model_deep_copy, \"my_attr\") <- 444\n# 'model' was NOT affected by this change\n# (keeps previous value that was assigned before)\nprint(xgb.attr(model, \"my_attr\"))\n\n# Verify that the new object was actually modified\nprint(xgb.attr(model_deep_copy, \"my_attr\"))\n}\n" }, { "path": "R-package/man/xgb.create.features.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/xgb.create.features.R\n\\name{xgb.create.features}\n\\alias{xgb.create.features}\n\\title{Create new features from a previously learned model}\n\\usage{\nxgb.create.features(model, data)\n}\n\\arguments{\n\\item{model}{Decision tree boosting model learned on the original data.}\n\n\\item{data}{Original data (usually provided as a \\code{dgCMatrix} matrix).}\n}\n\\value{\nA \\code{dgCMatrix} matrix including both the original data and the new features.\n}\n\\description{\nMay improve the learning by adding new features to the training data based on the\ndecision trees from a previously learned model.\n}\n\\details{\nThis is the function inspired from the paragraph 3.1 of the paper:\n\n\\strong{Practical Lessons from Predicting Clicks on Ads at Facebook}\n\n\\emph{(Xinran He, Junfeng Pan, Ou Jin, Tianbing Xu, Bo Liu, Tao Xu, Yan, xin Shi, Antoine Atallah, Ralf Herbrich, Stuart Bowers,\nJoaquin Quinonero Candela)}\n\nInternational Workshop on Data Mining for Online Advertising (ADKDD) - August 24, 2014\n\n\\url{https://research.facebook.com/publications/practical-lessons-from-predicting-clicks-on-ads-at-facebook/}.\n\nExtract explaining the method:\n\n\"We found that boosted decision trees are a powerful and very\nconvenient way to implement non-linear and tuple transformations\nof the kind we just described. We treat each individual\ntree as a categorical feature that takes as value the\nindex of the leaf an instance ends up falling in. We use\n1-of-K coding of this type of features.\n\nFor example, consider the boosted tree model in Figure 1 with 2 subtrees,\nwhere the first subtree has 3 leafs and the second 2 leafs. If an\ninstance ends up in leaf 2 in the first subtree and leaf 1 in\nsecond subtree, the overall input to the linear classifier will\nbe the binary vector \\verb{[0, 1, 0, 1, 0]}, where the first 3 entries\ncorrespond to the leaves of the first subtree and last 2 to\nthose of the second subtree.\n\n...\n\nWe can understand boosted decision tree\nbased transformation as a supervised feature encoding that\nconverts a real-valued vector into a compact binary-valued\nvector. A traversal from root node to a leaf node represents\na rule on certain features.\"\n}\n\\examples{\ndata(agaricus.train, package = \"xgboost\")\ndata(agaricus.test, package = \"xgboost\")\n\ndtrain <- with(agaricus.train, xgb.DMatrix(data, label = label, nthread = 2))\ndtest <- with(agaricus.test, xgb.DMatrix(data, label = label, nthread = 2))\n\nparam <- list(max_depth = 2, learning_rate = 1, objective = 'binary:logistic', nthread = 1)\nnrounds = 4\n\nbst <- xgb.train(params = param, data = dtrain, nrounds = nrounds)\n\n# Model accuracy without new features\naccuracy.before <- sum((predict(bst, agaricus.test$data) >= 0.5) == agaricus.test$label) /\n length(agaricus.test$label)\n\n# Convert previous features to one hot encoding\nnew.features.train <- xgb.create.features(model = bst, agaricus.train$data)\nnew.features.test <- xgb.create.features(model = bst, agaricus.test$data)\n\n# learning with new features\nnew.dtrain <- xgb.DMatrix(\n data = new.features.train, label = agaricus.train$label, nthread = 1\n)\nnew.dtest <- xgb.DMatrix(\n data = new.features.test, label = agaricus.test$label, nthread = 1\n)\nbst <- xgb.train(params = param, data = new.dtrain, nrounds = nrounds)\n\n# Model accuracy with new features\naccuracy.after <- sum((predict(bst, new.dtest) >= 0.5) == agaricus.test$label) /\n length(agaricus.test$label)\n\n# Here the accuracy was already good and is now perfect.\ncat(paste(\"The accuracy was\", accuracy.before, \"before adding leaf features and it is now\",\n accuracy.after, \"!\\n\"))\n\n}\n" }, { "path": "R-package/man/xgb.cv.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/xgb.cv.R\n\\name{xgb.cv}\n\\alias{xgb.cv}\n\\title{Cross Validation}\n\\usage{\nxgb.cv(\n params = xgb.params(),\n data,\n nrounds,\n nfold,\n prediction = FALSE,\n showsd = TRUE,\n metrics = list(),\n objective = NULL,\n custom_metric = NULL,\n stratified = \"auto\",\n folds = NULL,\n train_folds = NULL,\n verbose = TRUE,\n print_every_n = 1L,\n early_stopping_rounds = NULL,\n maximize = NULL,\n callbacks = list(),\n ...\n)\n}\n\\arguments{\n\\item{params}{List of XGBoost parameters which control the model building process.\nSee the \\href{https://xgboost.readthedocs.io/en/latest/parameter.html}{online documentation}\nand the documentation for \\code{\\link[=xgb.params]{xgb.params()}} for details.\n\nShould be passed as list with named entries. Parameters that are not specified in this\nlist will use their default values.\n\nA list of named parameters can be created through the function \\code{\\link[=xgb.params]{xgb.params()}}, which\naccepts all valid parameters as function arguments.}\n\n\\item{data}{An \\code{xgb.DMatrix} object, with corresponding fields like \\code{label} or bounds as required\nfor model training by the objective.\n\nNote that only the basic \\code{xgb.DMatrix} class is supported - variants such as \\code{xgb.QuantileDMatrix}\nor \\code{xgb.ExtMemDMatrix} are not supported here.}\n\n\\item{nrounds}{Max number of boosting iterations.}\n\n\\item{nfold}{The original dataset is randomly partitioned into \\code{nfold} equal size subsamples.}\n\n\\item{prediction}{A logical value indicating whether to return the test fold predictions\nfrom each CV model. This parameter engages the \\code{\\link[=xgb.cb.cv.predict]{xgb.cb.cv.predict()}} callback.}\n\n\\item{showsd}{Logical value whether to show standard deviation of cross validation.}\n\n\\item{metrics}{List of evaluation metrics to be used in cross validation,\nwhen it is not specified, the evaluation metric is chosen according to objective function.\nPossible options are:\n\\itemize{\n\\item \\code{error}: Binary classification error rate\n\\item \\code{rmse}: Root mean square error\n\\item \\code{logloss}: Negative log-likelihood function\n\\item \\code{mae}: Mean absolute error\n\\item \\code{mape}: Mean absolute percentage error\n\\item \\code{auc}: Area under curve\n\\item \\code{aucpr}: Area under PR curve\n\\item \\code{merror}: Exact matching error used to evaluate multi-class classification\n}}\n\n\\item{objective}{Customized objective function. Should take two arguments: the first one will be the\ncurrent predictions (either a numeric vector or matrix depending on the number of targets / classes),\nand the second one will be the \\code{data} DMatrix object that is used for training.\n\nIt should return a list with two elements \\code{grad} and \\code{hess} (in that order), as either\nnumeric vectors or numeric matrices depending on the number of targets / classes (same\ndimension as the predictions that are passed as first argument).}\n\n\\item{custom_metric}{Customized evaluation function. Just like \\code{objective}, should take two arguments,\nwith the first one being the predictions and the second one the \\code{data} DMatrix.\n\nShould return a list with two elements \\code{metric} (name that will be displayed for this metric,\nshould be a string / character), and \\code{value} (the number that the function calculates, should\nbe a numeric scalar).\n\nNote that even if passing \\code{custom_metric}, objectives also have an associated default metric that\nwill be evaluated in addition to it. In order to disable the built-in metric, one can pass\nparameter \\code{disable_default_eval_metric = TRUE}.}\n\n\\item{stratified}{Logical flag indicating whether sampling of folds should be stratified\nby the values of outcome labels. For real-valued labels in regression objectives,\nstratification will be done by discretizing the labels into up to 5 buckets beforehand.\n\nIf passing \"auto\", will be set to \\code{TRUE} if the objective in \\code{params} is a classification\nobjective (from XGBoost's built-in objectives, doesn't apply to custom ones), and to\n\\code{FALSE} otherwise.\n\nThis parameter is ignored when \\code{data} has a \\code{group} field - in such case, the splitting\nwill be based on whole groups (note that this might make the folds have different sizes).\n\nValue \\code{TRUE} here is \\strong{not} supported for custom objectives.}\n\n\\item{folds}{List with pre-defined CV folds (each element must be a vector of test fold's indices).\nWhen folds are supplied, the \\code{nfold} and \\code{stratified} parameters are ignored.\n\nIf \\code{data} has a \\code{group} field and the objective requires this field, each fold (list element)\nmust additionally have two attributes (retrievable through \\code{attributes}) named \\code{group_test}\nand \\code{group_train}, which should hold the \\code{group} to assign through \\code{\\link[=setinfo.xgb.DMatrix]{setinfo.xgb.DMatrix()}} to\nthe resulting DMatrices.}\n\n\\item{train_folds}{List specifying which indices to use for training. If \\code{NULL}\n(the default) all indices not specified in \\code{folds} will be used for training.\n\nThis is not supported when \\code{data} has \\code{group} field.}\n\n\\item{verbose}{If 0, xgboost will stay silent. If 1, it will print information about performance.\nIf 2, some additional information will be printed out.\nNote that setting \\code{verbose > 0} automatically engages the\n\\code{xgb.cb.print.evaluation(period=1)} callback function.}\n\n\\item{print_every_n}{When passing \\code{verbose>0}, evaluation logs (metrics calculated on the\ndata passed under \\code{evals}) will be printed every nth iteration according to the value passed\nhere. The first and last iteration are always included regardless of this 'n'.\n\nOnly has an effect when passing data under \\code{evals} and when passing \\code{verbose>0}. The parameter\nis passed to the \\code{\\link[=xgb.cb.print.evaluation]{xgb.cb.print.evaluation()}} callback.}\n\n\\item{early_stopping_rounds}{Number of boosting rounds after which training will be stopped\nif there is no improvement in performance (as measured by the evaluatiation metric that is\nsupplied or selected by default for the objective) on the evaluation data passed under\n\\code{evals}.\n\nMust pass \\code{evals} in order to use this functionality. Setting this parameter adds the\n\\code{\\link[=xgb.cb.early.stop]{xgb.cb.early.stop()}} callback.\n\nIf \\code{NULL}, early stopping will not be used.}\n\n\\item{maximize}{If \\code{feval} and \\code{early_stopping_rounds} are set, then this parameter must be set as well.\nWhen it is \\code{TRUE}, it means the larger the evaluation score the better.\nThis parameter is passed to the \\code{\\link[=xgb.cb.early.stop]{xgb.cb.early.stop()}} callback.}\n\n\\item{callbacks}{A list of callback functions to perform various task during boosting.\nSee \\code{\\link[=xgb.Callback]{xgb.Callback()}}. Some of the callbacks are automatically created depending on the\nparameters' values. User can provide either existing or their own callback methods in order\nto customize the training process.}\n\n\\item{...}{Not used.\n\nSome arguments that were part of this function in previous XGBoost versions are currently\ndeprecated or have been renamed. If a deprecated or renamed argument is passed, will throw\na warning (by default) and use its current equivalent instead. This warning will become an\nerror if using the \\link[=xgboost-options]{'strict mode' option}.\n\nIf some additional argument is passed that is neither a current function argument nor\na deprecated or renamed argument, a warning or error will be thrown depending on the\n'strict mode' option.\n\n\\bold{Important:} \\code{...} will be removed in a future version, and all the current\ndeprecation warnings will become errors. Please use only arguments that form part of\nthe function signature.}\n}\n\\value{\nAn object of class 'xgb.cv.synchronous' with the following elements:\n\\itemize{\n\\item \\code{call}: Function call.\n\\item \\code{params}: Parameters that were passed to the xgboost library. Note that it does not\ncapture parameters changed by the \\code{\\link[=xgb.cb.reset.parameters]{xgb.cb.reset.parameters()}} callback.\n\\item \\code{evaluation_log}: Evaluation history stored as a \\code{data.table} with the\nfirst column corresponding to iteration number and the rest corresponding to the\nCV-based evaluation means and standard deviations for the training and test CV-sets.\nIt is created by the \\code{\\link[=xgb.cb.evaluation.log]{xgb.cb.evaluation.log()}} callback.\n\\item \\code{niter}: Number of boosting iterations.\n\\item \\code{nfeatures}: Number of features in training data.\n\\item \\code{folds}: The list of CV folds' indices - either those passed through the \\code{folds}\nparameter or randomly generated.\n}\n\nPlus other potential elements that are the result of callbacks, such as a list \\code{cv_predict} with\na sub-element \\code{pred} when passing \\code{prediction = TRUE}, which is added by the \\code{\\link[=xgb.cb.cv.predict]{xgb.cb.cv.predict()}}\ncallback (note that one can also pass it manually under \\code{callbacks} with different settings,\nsuch as saving also the models created during cross validation); or a list \\code{early_stop} which\nwill contain elements such as \\code{best_iteration} when using the early stopping callback (\\code{\\link[=xgb.cb.early.stop]{xgb.cb.early.stop()}}).\n}\n\\description{\nThe cross validation function of xgboost.\n}\n\\details{\nThe original sample is randomly partitioned into \\code{nfold} equal size subsamples.\n\nOf the \\code{nfold} subsamples, a single subsample is retained as the validation data for testing the model,\nand the remaining \\code{nfold - 1} subsamples are used as training data.\n\nThe cross-validation process is then repeated \\code{nrounds} times, with each of the\n\\code{nfold} subsamples used exactly once as the validation data.\n\nAll observations are used for both training and validation.\n\nAdapted from \\url{https://en.wikipedia.org/wiki/Cross-validation_\\%28statistics\\%29}\n}\n\\examples{\ndata(agaricus.train, package = \"xgboost\")\n\ndtrain <- with(agaricus.train, xgb.DMatrix(data, label = label, nthread = 2))\n\ncv <- xgb.cv(\n data = dtrain,\n nrounds = 20,\n early_stopping_rounds = 1,\n params = xgb.params(\n nthread = 2,\n max_depth = 3,\n objective = \"binary:logistic\"\n ),\n nfold = 5,\n metrics = list(\"rmse\",\"auc\"),\n prediction = TRUE\n)\nprint(cv)\nprint(cv, verbose = TRUE)\n\n# Callbacks might add additional attributes, separated by the name of the callback\ncv$early_stop$best_iteration\nhead(cv$cv_predict$pred)\n}\n" }, { "path": "R-package/man/xgb.dump.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/xgb.dump.R\n\\name{xgb.dump}\n\\alias{xgb.dump}\n\\title{Dump an XGBoost model in text format.}\n\\usage{\nxgb.dump(\n model,\n fname = NULL,\n fmap = \"\",\n with_stats = FALSE,\n dump_format = c(\"text\", \"json\", \"dot\"),\n ...\n)\n}\n\\arguments{\n\\item{model}{The model object.}\n\n\\item{fname}{The name of the text file where to save the model text dump.\nIf not provided or set to \\code{NULL}, the model is returned as a character vector.}\n\n\\item{fmap}{Feature map file representing feature types. See demo/ for a walkthrough\nexample in R, and \\url{https://github.com/dmlc/xgboost/blob/master/demo/data/featmap.txt}\nto see an example of the value.}\n\n\\item{with_stats}{Whether to dump some additional statistics about the splits.\nWhen this option is on, the model dump contains two additional values:\ngain is the approximate loss function gain we get in each split;\ncover is the sum of second order gradient in each node.}\n\n\\item{dump_format}{Either 'text', 'json', or 'dot' (graphviz) format could be specified.\n\nFormat 'dot' for a single tree can be passed directly to packages that consume this format\nfor graph visualization, such as function \\code{DiagrammeR::grViz()}}\n\n\\item{...}{Not used.\n\nSome arguments that were part of this function in previous XGBoost versions are currently\ndeprecated or have been renamed. If a deprecated or renamed argument is passed, will throw\na warning (by default) and use its current equivalent instead. This warning will become an\nerror if using the \\link[=xgboost-options]{'strict mode' option}.\n\nIf some additional argument is passed that is neither a current function argument nor\na deprecated or renamed argument, a warning or error will be thrown depending on the\n'strict mode' option.\n\n\\bold{Important:} \\code{...} will be removed in a future version, and all the current\ndeprecation warnings will become errors. Please use only arguments that form part of\nthe function signature.}\n}\n\\value{\nIf fname is not provided or set to \\code{NULL} the function will return the model\nas a character vector. Otherwise it will return \\code{TRUE}.\n}\n\\description{\nDump an XGBoost model in text format.\n}\n\\examples{\n\\dontshow{RhpcBLASctl::omp_set_num_threads(1)}\ndata(agaricus.train, package = \"xgboost\")\ndata(agaricus.test, package = \"xgboost\")\n\ntrain <- agaricus.train\ntest <- agaricus.test\n\nbst <- xgb.train(\n data = xgb.DMatrix(train$data, label = train$label, nthread = 1),\n nrounds = 2,\n params = xgb.params(\n max_depth = 2,\n nthread = 2,\n objective = \"binary:logistic\"\n )\n)\n\n# save the model in file 'xgb.model.dump'\ndump_path = file.path(tempdir(), 'model.dump')\nxgb.dump(bst, dump_path, with_stats = TRUE)\n\n# print the model without saving it to a file\nprint(xgb.dump(bst, with_stats = TRUE))\n\n# print in JSON format:\ncat(xgb.dump(bst, with_stats = TRUE, dump_format = \"json\"))\n\n# plot first tree leveraging the 'dot' format\nif (requireNamespace('DiagrammeR', quietly = TRUE)) {\n DiagrammeR::grViz(xgb.dump(bst, dump_format = \"dot\")[[1L]])\n}\n}\n" }, { "path": "R-package/man/xgb.gblinear.history.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/callbacks.R\n\\name{xgb.gblinear.history}\n\\alias{xgb.gblinear.history}\n\\title{Extract gblinear coefficients history}\n\\usage{\nxgb.gblinear.history(model, class_index = NULL)\n}\n\\arguments{\n\\item{model}{Either an \\code{xgb.Booster} or a result of \\code{\\link[=xgb.cv]{xgb.cv()}}, trained\nusing the \\link{xgb.cb.gblinear.history} callback, but \\strong{not} a booster\nloaded from \\code{\\link[=xgb.load]{xgb.load()}} or \\code{\\link[=xgb.load.raw]{xgb.load.raw()}}.}\n\n\\item{class_index}{zero-based class index to extract the coefficients for only that\nspecific class in a multinomial multiclass model. When it is \\code{NULL}, all the\ncoefficients are returned. Has no effect in non-multiclass models.}\n}\n\\value{\nFor an \\code{\\link[=xgb.train]{xgb.train()}} result, a matrix (either dense or sparse) with the columns\ncorresponding to iteration's coefficients and the rows corresponding to boosting iterations.\n\nFor an \\code{\\link[=xgb.cv]{xgb.cv()}} result, a list of such matrices is returned with the elements\ncorresponding to CV folds.\n\nWhen there is more than one coefficient per feature (e.g. multi-class classification)\nand \\code{class_index} is not provided,\nthe result will be reshaped into a vector where coefficients are arranged first by features and\nthen by class (e.g. first 1 through N coefficients will be for the first class, then\ncoefficients N+1 through 2N for the second class, and so on).\n}\n\\description{\nA helper function to extract the matrix of linear coefficients' history\nfrom a gblinear model created while using the \\link{xgb.cb.gblinear.history}\ncallback (which must be added manually as by default it is not used).\n}\n\\details{\nNote that this is an R-specific function that relies on R attributes that\nare not saved when using XGBoost's own serialization functions like \\code{\\link[=xgb.load]{xgb.load()}}\nor \\code{\\link[=xgb.load.raw]{xgb.load.raw()}}.\n\nIn order for a serialized model to be accepted by this function, one must use R\nserializers such as \\code{\\link[=saveRDS]{saveRDS()}}.\n}\n\\seealso{\n\\link{xgb.cb.gblinear.history}, \\link{coef.xgb.Booster}.\n}\n" }, { "path": "R-package/man/xgb.get.DMatrix.data.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/xgb.DMatrix.R\n\\name{xgb.get.DMatrix.data}\n\\alias{xgb.get.DMatrix.data}\n\\title{Get DMatrix Data}\n\\usage{\nxgb.get.DMatrix.data(dmat)\n}\n\\arguments{\n\\item{dmat}{An \\code{xgb.DMatrix} object, as returned by \\code{\\link[=xgb.DMatrix]{xgb.DMatrix()}}.}\n}\n\\value{\nThe data held in the DMatrix, as a sparse CSR matrix (class \\code{dgRMatrix}\nfrom package \\code{Matrix}). If it had feature names, these will be added as column names\nin the output.\n}\n\\description{\nGet DMatrix Data\n}\n" }, { "path": "R-package/man/xgb.get.DMatrix.num.non.missing.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/xgb.DMatrix.R\n\\name{xgb.get.DMatrix.num.non.missing}\n\\alias{xgb.get.DMatrix.num.non.missing}\n\\title{Get Number of Non-Missing Entries in DMatrix}\n\\usage{\nxgb.get.DMatrix.num.non.missing(dmat)\n}\n\\arguments{\n\\item{dmat}{An \\code{xgb.DMatrix} object, as returned by \\code{\\link[=xgb.DMatrix]{xgb.DMatrix()}}.}\n}\n\\value{\nThe number of non-missing entries in the DMatrix.\n}\n\\description{\nGet Number of Non-Missing Entries in DMatrix\n}\n" }, { "path": "R-package/man/xgb.get.DMatrix.qcut.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/xgb.DMatrix.R\n\\name{xgb.get.DMatrix.qcut}\n\\alias{xgb.get.DMatrix.qcut}\n\\title{Get Quantile Cuts from DMatrix}\n\\usage{\nxgb.get.DMatrix.qcut(dmat, output = c(\"list\", \"arrays\"))\n}\n\\arguments{\n\\item{dmat}{An \\code{xgb.DMatrix} object, as returned by \\code{\\link[=xgb.DMatrix]{xgb.DMatrix()}}.}\n\n\\item{output}{Output format for the quantile cuts. Possible options are:\n\\itemize{\n\\item \"list\"\\verb{will return the output as a list with one entry per column, where each column will have a numeric vector with the cuts. The list will be named if}dmat` has column names assigned to it.\n\\item \\code{\"arrays\"} will return a list with entries \\code{indptr} (base-0 indexing) and\n\\code{data}. Here, the cuts for column 'i' are obtained by slicing 'data' from entries\n\\code{ indptr[i]+1} to \\code{indptr[i+1]}.\n}}\n}\n\\value{\nThe quantile cuts, in the format specified by parameter \\code{output}.\n}\n\\description{\nGet the quantile cuts (a.k.a. borders) from an \\code{xgb.DMatrix}\nthat has been quantized for the histogram method (\\code{tree_method = \"hist\"}).\n\nThese cuts are used in order to assign observations to bins - i.e. these are ordered\nboundaries which are used to determine assignment condition \\verb{border_low < x < border_high}.\nAs such, the first and last bin will be outside of the range of the data, so as to include\nall of the observations there.\n\nIf a given column has 'n' bins, then there will be 'n+1' cuts / borders for that column,\nwhich will be output in sorted order from lowest to highest.\n\nDifferent columns can have different numbers of bins according to their range.\n}\n\\examples{\ndata(mtcars)\n\ny <- mtcars$mpg\nx <- as.matrix(mtcars[, -1])\ndm <- xgb.DMatrix(x, label = y, nthread = 1)\n\n# DMatrix is not quantized right away, but will be once a hist model is generated\nmodel <- xgb.train(\n data = dm,\n params = xgb.params(tree_method = \"hist\", max_bin = 8, nthread = 1),\n nrounds = 3\n)\n\n# Now can get the quantile cuts\nxgb.get.DMatrix.qcut(dm)\n}\n" }, { "path": "R-package/man/xgb.get.num.boosted.rounds.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/xgb.Booster.R\n\\name{xgb.get.num.boosted.rounds}\n\\alias{xgb.get.num.boosted.rounds}\n\\alias{length.xgb.Booster}\n\\title{Get number of boosting in a fitted booster}\n\\usage{\nxgb.get.num.boosted.rounds(model)\n\n\\method{length}{xgb.Booster}(x)\n}\n\\arguments{\n\\item{model, x}{A fitted \\code{xgb.Booster} model.}\n}\n\\value{\nThe number of rounds saved in the model as an integer.\n}\n\\description{\nGet number of boosting in a fitted booster\n}\n\\details{\nNote that setting booster parameters related to training\ncontinuation / updates through \\code{\\link[=xgb.model.parameters<-]{xgb.model.parameters<-()}} will reset the\nnumber of rounds to zero.\n}\n" }, { "path": "R-package/man/xgb.importance.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/xgb.importance.R\n\\name{xgb.importance}\n\\alias{xgb.importance}\n\\title{Feature importance}\n\\usage{\nxgb.importance(\n model = NULL,\n feature_names = getinfo(model, \"feature_name\"),\n trees = NULL\n)\n}\n\\arguments{\n\\item{model}{Object of class \\code{xgb.Booster}.}\n\n\\item{feature_names}{Character vector used to overwrite the feature names\nof the model. The default is \\code{NULL} (use original feature names).}\n\n\\item{trees}{An integer vector of (base-1) tree indices that should be included\ninto the importance calculation (only for the \"gbtree\" booster).\nThe default (\\code{NULL}) parses all trees.\nIt could be useful, e.g., in multiclass classification to get feature importances\nfor each class separately.}\n}\n\\value{\nA \\code{data.table} with the following columns:\n\nFor a tree model:\n\\itemize{\n\\item \\code{Features}: Names of the features used in the model.\n\\item \\code{Gain}: Fractional contribution of each feature to the model based on\nthe total gain of this feature's splits. Higher percentage means higher importance.\n\\item \\code{Cover}: Metric of the number of observation related to this feature.\n\\item \\code{Frequency}: Percentage of times a feature has been used in trees.\n}\n\nFor a linear model:\n\\itemize{\n\\item \\code{Features}: Names of the features used in the model.\n\\item \\code{Weight}: Linear coefficient of this feature.\n\\item \\code{Class}: Class label (only for multiclass models). For objects of class \\code{xgboost} (as\nproduced by \\code{\\link[=xgboost]{xgboost()}}), it will be a \\code{factor}, while for objects of class \\code{xgb.Booster}\n(as produced by \\code{\\link[=xgb.train]{xgb.train()}}), it will be a zero-based integer vector.\n}\n\nIf \\code{feature_names} is not provided and \\code{model} doesn't have \\code{feature_names},\nthe index of the features will be used instead. Because the index is extracted from the model dump\n(based on C++ code), it starts at 0 (as in C/C++ or Python) instead of 1 (usual in R).\n}\n\\description{\nCreates a \\code{data.table} of feature importances.\n}\n\\details{\nThis function works for both linear and tree models.\n\nFor linear models, the importance is the absolute magnitude of linear coefficients.\nTo obtain a meaningful ranking by importance for linear models, the features need to\nbe on the same scale (which is also recommended when using L1 or L2 regularization).\n}\n\\examples{\n# binary classification using \"gbtree\":\ndata(\"ToothGrowth\")\nx <- ToothGrowth[, c(\"len\", \"dose\")]\ny <- ToothGrowth$supp\nmodel_tree_binary <- xgboost(\n x, y,\n nrounds = 5L,\n nthreads = 1L,\n booster = \"gbtree\",\n max_depth = 2L\n)\nxgb.importance(model_tree_binary)\n\n# binary classification using \"gblinear\":\nmodel_tree_linear <- xgboost(\n x, y,\n nrounds = 5L,\n nthreads = 1L,\n booster = \"gblinear\",\n learning_rate = 0.3\n)\nxgb.importance(model_tree_linear)\n\n# multi-class classification using \"gbtree\":\ndata(\"iris\")\nx <- iris[, c(\"Sepal.Length\", \"Sepal.Width\", \"Petal.Length\", \"Petal.Width\")]\ny <- iris$Species\nmodel_tree_multi <- xgboost(\n x, y,\n nrounds = 5L,\n nthreads = 1L,\n booster = \"gbtree\",\n max_depth = 3\n)\n# all classes clumped together:\nxgb.importance(model_tree_multi)\n# inspect importances separately for each class:\nnum_classes <- 3L\nnrounds <- 5L\nxgb.importance(\n model_tree_multi, trees = seq(from = 1, by = num_classes, length.out = nrounds)\n)\nxgb.importance(\n model_tree_multi, trees = seq(from = 2, by = num_classes, length.out = nrounds)\n)\nxgb.importance(\n model_tree_multi, trees = seq(from = 3, by = num_classes, length.out = nrounds)\n)\n\n# multi-class classification using \"gblinear\":\nmodel_linear_multi <- xgboost(\n x, y,\n nrounds = 5L,\n nthreads = 1L,\n booster = \"gblinear\",\n learning_rate = 0.2\n)\nxgb.importance(model_linear_multi)\n}\n" }, { "path": "R-package/man/xgb.is.same.Booster.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/xgb.Booster.R\n\\name{xgb.is.same.Booster}\n\\alias{xgb.is.same.Booster}\n\\title{Check if two boosters share the same C object}\n\\usage{\nxgb.is.same.Booster(obj1, obj2)\n}\n\\arguments{\n\\item{obj1}{Booster model to compare with \\code{obj2}.}\n\n\\item{obj2}{Booster model to compare with \\code{obj1}.}\n}\n\\value{\nEither \\code{TRUE} or \\code{FALSE} according to whether the two boosters share the\nunderlying C object.\n}\n\\description{\nChecks whether two booster objects refer to the same underlying C object.\n}\n\\details{\nAs booster objects (as returned by e.g. \\code{\\link[=xgb.train]{xgb.train()}}) contain an R 'externalptr'\nobject, they don't follow typical copy-on-write semantics of other R objects - that is, if\none assigns a booster to a different variable and modifies that new variable through in-place\nmethods like \\code{\\link[=xgb.attr<-]{xgb.attr<-()}}, the modification will be applied to both the old and the new\nvariable, unlike typical R assignments which would only modify the latter.\n\nThis function allows checking whether two booster objects share the same 'externalptr',\nregardless of the R attributes that they might have.\n\nIn order to duplicate a booster in such a way that the copy wouldn't share the same\n'externalptr', one can use function \\code{\\link[=xgb.copy.Booster]{xgb.copy.Booster()}}.\n}\n\\examples{\nlibrary(xgboost)\n\ndata(mtcars)\n\ny <- mtcars$mpg\nx <- as.matrix(mtcars[, -1])\n\nmodel <- xgb.train(\n params = xgb.params(nthread = 1),\n data = xgb.DMatrix(x, label = y, nthread = 1),\n nrounds = 3\n)\n\nmodel_shallow_copy <- model\nxgb.is.same.Booster(model, model_shallow_copy) # same C object\n\nmodel_deep_copy <- xgb.copy.Booster(model)\nxgb.is.same.Booster(model, model_deep_copy) # different C objects\n\n# In-place assignments modify all references,\n# but not full/deep copies of the booster\nxgb.attr(model_shallow_copy, \"my_attr\") <- 111\nxgb.attr(model, \"my_attr\") # gets modified\nxgb.attr(model_deep_copy, \"my_attr\") # doesn't get modified\n}\n\\seealso{\n\\code{\\link[=xgb.copy.Booster]{xgb.copy.Booster()}}\n}\n" }, { "path": "R-package/man/xgb.load.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/xgb.load.R\n\\name{xgb.load}\n\\alias{xgb.load}\n\\title{Load XGBoost model from binary file}\n\\usage{\nxgb.load(modelfile)\n}\n\\arguments{\n\\item{modelfile}{The name of the binary input file.}\n}\n\\value{\nAn object of \\code{xgb.Booster} class.\n}\n\\description{\nLoad XGBoost model from binary model file.\n}\n\\details{\nThe input file is expected to contain a model saved in an XGBoost model format\nusing either \\code{\\link[=xgb.save]{xgb.save()}} in R, or using some\nappropriate methods from other XGBoost interfaces. E.g., a model trained in Python and\nsaved from there in XGBoost format, could be loaded from R.\n\nNote: a model saved as an R object has to be loaded using corresponding R-methods,\nnot by \\code{\\link[=xgb.load]{xgb.load()}}.\n}\n\\examples{\n\\dontshow{RhpcBLASctl::omp_set_num_threads(1)}\ndata(agaricus.train, package = \"xgboost\")\ndata(agaricus.test, package = \"xgboost\")\n\n## Keep the number of threads to 1 for examples\nnthread <- 1\ndata.table::setDTthreads(nthread)\n\ntrain <- agaricus.train\ntest <- agaricus.test\n\nbst <- xgb.train(\n data = xgb.DMatrix(train$data, label = train$label, nthread = 1),\n nrounds = 2,\n params = xgb.params(\n max_depth = 2,\n nthread = nthread,\n objective = \"binary:logistic\"\n )\n)\n\nfname <- file.path(tempdir(), \"xgb.ubj\")\nxgb.save(bst, fname)\nbst <- xgb.load(fname)\n}\n\\seealso{\n\\code{\\link[=xgb.save]{xgb.save()}}\n}\n" }, { "path": "R-package/man/xgb.load.raw.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/xgb.load.raw.R\n\\name{xgb.load.raw}\n\\alias{xgb.load.raw}\n\\title{Load serialised XGBoost model from R's raw vector}\n\\usage{\nxgb.load.raw(buffer)\n}\n\\arguments{\n\\item{buffer}{The buffer returned by \\code{\\link[=xgb.save.raw]{xgb.save.raw()}}.}\n}\n\\description{\nUser can generate raw memory buffer by calling \\code{\\link[=xgb.save.raw]{xgb.save.raw()}}.\n}\n" }, { "path": "R-package/man/xgb.model.dt.tree.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/xgb.model.dt.tree.R\n\\name{xgb.model.dt.tree}\n\\alias{xgb.model.dt.tree}\n\\title{Parse model text dump}\n\\usage{\nxgb.model.dt.tree(model, trees = NULL, use_int_id = FALSE, ...)\n}\n\\arguments{\n\\item{model}{Object of class \\code{xgb.Booster}. If it contains feature names (they can\nbe set through \\code{\\link[=setinfo]{setinfo()}}), they will be used in the output from this function.\n\nIf the model contains categorical features, an error will be thrown.}\n\n\\item{trees}{An integer vector of (base-1) tree indices that should be used. The default\n(\\code{NULL}) uses all trees. Useful, e.g., in multiclass classification to get only\nthe trees of one class.}\n\n\\item{use_int_id}{A logical flag indicating whether nodes in columns \"Yes\", \"No\", and\n\"Missing\" should be represented as integers (when \\code{TRUE}) or as \"Tree-Node\"\ncharacter strings (when \\code{FALSE}, default).}\n\n\\item{...}{Not used.\n\nSome arguments that were part of this function in previous XGBoost versions are currently\ndeprecated or have been renamed. If a deprecated or renamed argument is passed, will throw\na warning (by default) and use its current equivalent instead. This warning will become an\nerror if using the \\link[=xgboost-options]{'strict mode' option}.\n\nIf some additional argument is passed that is neither a current function argument nor\na deprecated or renamed argument, a warning or error will be thrown depending on the\n'strict mode' option.\n\n\\bold{Important:} \\code{...} will be removed in a future version, and all the current\ndeprecation warnings will become errors. Please use only arguments that form part of\nthe function signature.}\n}\n\\value{\nA \\code{data.table} with detailed information about tree nodes. It has the following columns:\n\\itemize{\n\\item \\code{Tree}: integer ID of a tree in a model (zero-based index).\n\\item \\code{Node}: integer ID of a node in a tree (zero-based index).\n\\item \\code{ID}: character identifier of a node in a model (only when \\code{use_int_id = FALSE}).\n\\item \\code{Feature}: for a branch node, a feature ID or name (when available);\nfor a leaf node, it simply labels it as \\code{\"Leaf\"}.\n\\item \\code{Split}: location of the split for a branch node (split condition is always \"less than\").\n\\item \\code{Yes}: ID of the next node when the split condition is met.\n\\item \\code{No}: ID of the next node when the split condition is not met.\n\\item \\code{Missing}: ID of the next node when the branch value is missing.\n\\item \\code{Gain}: either the split gain (change in loss) or the leaf value.\n\\item \\code{Cover}: metric related to the number of observations either seen by a split\nor collected by a leaf during training.\n}\n\nWhen \\code{use_int_id = FALSE}, columns \"Yes\", \"No\", and \"Missing\" point to model-wide node identifiers\nin the \"ID\" column. When \\code{use_int_id = TRUE}, those columns point to node identifiers from\nthe corresponding trees in the \"Node\" column.\n}\n\\description{\nParse a boosted tree model text dump into a \\code{data.table} structure.\n}\n\\details{\nNote that this function does not work with models that were fitted to\ncategorical data, and is only applicable to tree-based boosters (not \\code{gblinear}).\n}\n\\examples{\n# Basic use:\n\ndata(agaricus.train, package = \"xgboost\")\n## Keep the number of threads to 1 for examples\nnthread <- 1\ndata.table::setDTthreads(nthread)\n\nbst <- xgb.train(\n data = xgb.DMatrix(agaricus.train$data, label = agaricus.train$label, nthread = 1),\n nrounds = 2,\n params = xgb.params(\n max_depth = 2,\n nthread = nthread,\n objective = \"binary:logistic\"\n )\n)\n\n# This bst model already has feature_names stored with it, so those would be used when\n# feature_names is not set:\ndt <- xgb.model.dt.tree(bst)\n\n# How to match feature names of splits that are following a current 'Yes' branch:\nmerge(\n dt,\n dt[, .(ID, Y.Feature = Feature)], by.x = \"Yes\", by.y = \"ID\", all.x = TRUE\n)[\n order(Tree, Node)\n]\n\n}\n" }, { "path": "R-package/man/xgb.model.parameters.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/xgb.Booster.R\n\\name{xgb.model.parameters<-}\n\\alias{xgb.model.parameters<-}\n\\title{Accessors for model parameters}\n\\usage{\nxgb.model.parameters(object) <- value\n}\n\\arguments{\n\\item{object}{Object of class \\code{xgb.Booster}. \\strong{Will be modified in-place}.}\n\n\\item{value}{A list (or an object coercible to a list) with the names of parameters to set\nand the elements corresponding to parameter values.}\n}\n\\value{\nThe same booster \\code{object}, which gets modified in-place.\n}\n\\description{\nOnly the setter for XGBoost parameters is currently implemented.\n}\n\\details{\nJust like \\code{\\link[=xgb.attr]{xgb.attr()}}, this function will make in-place modifications\non the booster object which do not follow typical R assignment semantics - that is,\nall references to the same booster will also be updated, unlike assingment of R\nattributes which follow copy-on-write semantics.\n\nSee \\code{\\link[=xgb.copy.Booster]{xgb.copy.Booster()}} for an example of this behavior.\n\nBe aware that setting parameters of a fitted booster related to training continuation / updates\nwill reset its number of rounds indicator to zero.\n}\n\\examples{\ndata(agaricus.train, package = \"xgboost\")\n\ntrain <- agaricus.train\n\nbst <- xgb.train(\n data = xgb.DMatrix(train$data, label = train$label, nthread = 1),\n nrounds = 2,\n params = xgb.params(\n max_depth = 2,\n learning_rate = 1,\n nthread = 2,\n objective = \"binary:logistic\"\n )\n)\n\nxgb.model.parameters(bst) <- list(learning_rate = 0.1)\n\n}\n" }, { "path": "R-package/man/xgb.params.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/xgb.train.R\n\\name{xgb.params}\n\\alias{xgb.params}\n\\title{XGBoost Parameters}\n\\usage{\nxgb.params(\n objective = NULL,\n verbosity = NULL,\n nthread = NULL,\n seed = NULL,\n booster = NULL,\n eta = NULL,\n learning_rate = NULL,\n gamma = NULL,\n min_split_loss = NULL,\n max_depth = NULL,\n min_child_weight = NULL,\n max_delta_step = NULL,\n subsample = NULL,\n sampling_method = NULL,\n colsample_bytree = NULL,\n colsample_bylevel = NULL,\n colsample_bynode = NULL,\n lambda = NULL,\n reg_lambda = NULL,\n alpha = NULL,\n reg_alpha = NULL,\n tree_method = NULL,\n scale_pos_weight = NULL,\n updater = NULL,\n refresh_leaf = NULL,\n grow_policy = NULL,\n max_leaves = NULL,\n max_bin = NULL,\n num_parallel_tree = NULL,\n monotone_constraints = NULL,\n interaction_constraints = NULL,\n multi_strategy = NULL,\n base_score = NULL,\n eval_metric = NULL,\n seed_per_iteration = NULL,\n device = NULL,\n disable_default_eval_metric = NULL,\n use_rmm = NULL,\n max_cached_hist_node = NULL,\n max_cat_to_onehot = NULL,\n max_cat_threshold = NULL,\n sample_type = NULL,\n normalize_type = NULL,\n rate_drop = NULL,\n one_drop = NULL,\n skip_drop = NULL,\n feature_selector = NULL,\n top_k = NULL,\n num_class = NULL,\n tweedie_variance_power = NULL,\n huber_slope = NULL,\n quantile_alpha = NULL,\n expectile_alpha = NULL,\n aft_loss_distribution = NULL,\n aft_loss_distribution_scale = NULL,\n lambdarank_pair_method = NULL,\n lambdarank_num_pair_per_sample = NULL,\n lambdarank_normalization = NULL,\n lambdarank_score_normalization = NULL,\n lambdarank_unbiased = NULL,\n lambdarank_bias_norm = NULL,\n ndcg_exp_gain = NULL\n)\n}\n\\arguments{\n\\item{objective}{(default=\\code{\"reg:squarederror\"})\nSpecify the learning task and the corresponding learning objective or a custom objective function to be used.\n\nFor custom objective, see \\href{https://xgboost.readthedocs.io/en/latest/tutorials/custom_metric_obj.html}{Custom Objective and Evaluation Metric}\nand \\href{https://xgboost.readthedocs.io/en/latest/tutorials/saving_model.html#custom-obj-metric}{Custom objective and metric} for more information,\nalong with the end note for function signatures.\n\nSupported values are:\n\\itemize{\n\\item \\code{\"reg:squarederror\"}: regression with squared loss.\n\\item \\code{\"reg:squaredlogerror\"}: regression with squared log loss \\eqn{\\frac{1}{2}[log(pred + 1) - log(label + 1)]^2}. All input labels are required to be greater than -1. Also, see metric \\code{rmsle} for possible issue with this objective.\n\\item \\code{\"reg:logistic\"}: logistic regression, output probability\n\\item \\code{\"reg:pseudohubererror\"}: regression with Pseudo Huber loss, a twice differentiable alternative to absolute loss.\n\\item \\code{\"reg:absoluteerror\"}: Regression with L1 error. When tree model is used, leaf value is refreshed after tree construction. If used in distributed training, the leaf value is calculated as the mean value from all workers, which is not guaranteed to be optimal.\n\nVersion added: 1.7.0\n\\item \\code{\"reg:quantileerror\"}: Quantile loss, also known as \"pinball loss\". See later sections for its parameter and \\href{https://xgboost.readthedocs.io/en/latest/python/examples/quantile_regression.html#sphx-glr-python-examples-quantile-regression-py}{Quantile Regression} for a worked example.\n\nVersion added: 2.0.0\n\\item \\code{\"reg:expectileerror\"}: Expectile loss. See later sections for its parameter.\n\\item \\code{\"binary:logistic\"}: logistic regression for binary classification, output probability\n\\item \\code{\"binary:logitraw\"}: logistic regression for binary classification, output score before logistic transformation\n\\item \\code{\"binary:hinge\"}: hinge loss for binary classification. This makes predictions of 0 or 1, rather than producing probabilities.\n\\item \\code{\"count:poisson\"}: Poisson regression for count data, output mean of Poisson distribution.\n\\code{\"max_delta_step\"} is set to 0.7 by default in Poisson regression (used to safeguard optimization)\n\\item \\code{\"survival:cox\"}: Cox regression for right censored survival time data (negative values are considered right censored).\n\nNote that predictions are returned on the hazard ratio scale (i.e., as HR = exp(marginal_prediction) in the proportional hazard function \\code{h(t) = h0(t) * HR}).\n\\item \\code{\"survival:aft\"}: Accelerated failure time model for censored survival time data.\nSee \\href{https://xgboost.readthedocs.io/en/latest/tutorials/aft_survival_analysis.html}{Survival Analysis with Accelerated Failure Time} for details.\n\\item \\code{\"multi:softmax\"}: set XGBoost to do multiclass classification using the softmax objective, you also need to set num_class(number of classes)\n\\item \\code{\"multi:softprob\"}: same as softmax, but output a vector of \\code{ndata * nclass}, which can be further reshaped to \\code{ndata * nclass} matrix. The result contains predicted probability of each data point belonging to each class.\n\\item \\code{\"rank:ndcg\"}: Use LambdaMART to perform pair-wise ranking where the normalized discounted cumulative gain (NDCG) is maximized. This objective supports position debiasing for click data.\n\\item \\code{\"rank:map\"}: Use LambdaMART to perform pair-wise ranking where the mean average precision (MAP) is maximized\n\\item \\code{\"rank:pairwise\"}: Use LambdaRank to perform pair-wise ranking using the \\code{ranknet} objective.\n\\item \\code{\"reg:gamma\"}: gamma regression with log-link. Output is a mean of gamma distribution. It might be useful, e.g., for modeling insurance claims severity, or for any outcome that might be gamma-distributed.\n\\item \\code{\"reg:tweedie\"}: Tweedie regression with log-link. It might be useful, e.g., for modeling total loss in insurance, or for any outcome that might be tweedie-distributed.\n}}\n\n\\item{verbosity}{(default=1)\nVerbosity of printing messages. Valid values are 0 (silent), 1 (warning), 2 (info), 3\n(debug). Sometimes XGBoost tries to change configurations based on heuristics, which\nis displayed as warning message. If there's unexpected behaviour, please try to\nincrease value of verbosity.}\n\n\\item{nthread}{(default to maximum number of threads available if not set)\nNumber of parallel threads used to run XGBoost. When choosing it, please keep thread\ncontention and hyperthreading in mind.}\n\n\\item{seed}{Random number seed. If not specified, will take a random seed through R's own RNG engine.}\n\n\\item{booster}{(default= \\code{\"gbtree\"})\nWhich booster to use. Can be \\code{\"gbtree\"}, \\code{\"gblinear\"} or \\code{\"dart\"}; \\code{\"gbtree\"} and \\code{\"dart\"} use tree based models while \\code{\"gblinear\"} uses linear functions.}\n\n\\item{eta, learning_rate}{(two aliases for the same parameter)\nStep size shrinkage used in update to prevent overfitting. After each boosting step, we can directly get the weights of new features, and \\code{eta} shrinks the feature weights to make the boosting process more conservative.\n\\itemize{\n\\item range: \\eqn{[0,1]}\n\\item default value: 0.3 for tree-based boosters, 0.5 for linear booster.\n}\n\nNote: should only pass one of \\code{eta} or \\code{learning_rate}. Both refer to the same parameter and there's thus no difference between one or the other.}\n\n\\item{gamma, min_split_loss}{(two aliases for the same parameter) (for Tree Booster) (default=0, alias: \\code{gamma})\nMinimum loss reduction required to make a further partition on a leaf node of the tree. The larger \\code{min_split_loss} is, the more conservative the algorithm will be. Note that a tree where no splits were made might still contain a single terminal node with a non-zero score.\n\nrange: \\eqn{[0, \\infty)}\n\nNote: should only pass one of \\code{gamma} or \\code{min_split_loss}. Both refer to the same parameter and there's thus no difference between one or the other.}\n\n\\item{max_depth}{(for Tree Booster) (default=6, type=int32)\nMaximum depth of a tree. Increasing this value will make the model more complex and more likely to overfit. 0 indicates no limit on depth. Beware that XGBoost aggressively consumes memory when training a deep tree. \\code{\"exact\"} tree method requires non-zero value.\n\nrange: \\eqn{[0, \\infty)}}\n\n\\item{min_child_weight}{(for Tree Booster) (default=1)\nMinimum sum of instance weight (hessian) needed in a child. If the tree partition step results in a leaf node with the sum of instance weight less than \\code{min_child_weight}, then the building process will give up further partitioning. In linear regression task, this simply corresponds to minimum number of instances needed to be in each node. The larger \\code{min_child_weight} is, the more conservative the algorithm will be.\n\nrange: \\eqn{[0, \\infty)}}\n\n\\item{max_delta_step}{(for Tree Booster) (default=0)\nMaximum delta step we allow each leaf output to be. If the value is set to 0, it means there is no constraint. If it is set to a positive value, it can help making the update step more conservative. Usually this parameter is not needed, but it might help in logistic regression when class is extremely imbalanced. Set it to value of 1-10 might help control the update.\n\nrange: \\eqn{[0, \\infty)}}\n\n\\item{subsample}{(for Tree Booster) (default=1)\nSubsample ratio of the training instances. Setting it to 0.5 means that XGBoost would randomly sample half of the training data prior to growing trees. and this will prevent overfitting. Subsampling will occur once in every boosting iteration.\n\nrange: \\eqn{(0,1]}}\n\n\\item{sampling_method}{(for Tree Booster) (default= \\code{\"uniform\"})\nThe method to use to sample the training instances.\n\\itemize{\n\\item \\code{\"uniform\"}: each training instance has an equal probability of being selected. Typically set\n\\code{\"subsample\"} >= 0.5 for good results.\n\\item \\code{\"gradient_based\"}: the selection probability for each training instance is proportional to the\n\\bold{regularized absolute value} of gradients (more specifically, \\eqn{\\sqrt{g^2+\\lambda h^2}}).\n\\code{\"subsample\"} may be set to as low as 0.1 without loss of model accuracy. Note that this\nsampling method is only supported when \\code{\"tree_method\"} is set to \\code{\"hist\"}; other tree\nmethods only support \\code{\"uniform\"} sampling.\n}}\n\n\\item{colsample_bytree, colsample_bylevel, colsample_bynode}{(for Tree Booster) (default=1)\nThis is a family of parameters for subsampling of columns.\n\\itemize{\n\\item All \\code{\"colsample_by*\"} parameters have a range of \\eqn{(0, 1]}, the default value of 1, and specify the fraction of columns to be subsampled.\n\\item \\code{\"colsample_bytree\"} is the subsample ratio of columns when constructing each tree. Subsampling occurs once for every tree constructed.\n\\item \\code{\"colsample_bylevel\"} is the subsample ratio of columns for each level. Subsampling occurs once for every new depth level reached in a tree. Columns are subsampled from the set of columns chosen for the current tree.\n\\item \\code{\"colsample_bynode\"} is the subsample ratio of columns for each node (split). Subsampling occurs once every time a new split is evaluated. Columns are subsampled from the set of columns chosen for the current level. This is not supported by the exact tree method.\n\\item \\code{\"colsample_by*\"} parameters work cumulatively. For instance,\nthe combination \\verb{\\{'colsample_bytree'=0.5, 'colsample_bylevel'=0.5, 'colsample_bynode'=0.5\\}} with 64 features will leave 8 features to choose from at\neach split.\n}\n\nOne can set the \\code{\"feature_weights\"} for DMatrix to\ndefine the probability of each feature being selected when using column sampling.}\n\n\\item{lambda, reg_lambda}{(two aliases for the same parameter)\n\\itemize{\n\\item For tree-based boosters:\n\\itemize{\n\\item L2 regularization term on weights. Increasing this value will make model more conservative.\n\\item default: 1\n\\item range: \\eqn{[0, \\infty]}\n}\n\\item For linear booster:\n\\itemize{\n\\item L2 regularization term on weights. Increasing this value will make model more conservative. Normalised to number of training examples.\n\\item default: 0\n\\item range: \\eqn{[0, \\infty)}\n}\n}\n\nNote: should only pass one of \\code{lambda} or \\code{reg_lambda}. Both refer to the same parameter and there's thus no difference between one or the other.}\n\n\\item{alpha, reg_alpha}{(two aliases for the same parameter)\n\\itemize{\n\\item L1 regularization term on weights. Increasing this value will make model more conservative.\n\\item For the linear booster, it's normalised to number of training examples.\n\\item default: 0\n\\item range: \\eqn{[0, \\infty)}\n}\n\nNote: should only pass one of \\code{alpha} or \\code{reg_alpha}. Both refer to the same parameter and there's thus no difference between one or the other.}\n\n\\item{tree_method}{(for Tree Booster) (default= \\code{\"auto\"})\nThe tree construction algorithm used in XGBoost. See description in the \\href{https://arxiv.org/abs/1603.02754}{reference paper} and \\href{https://xgboost.readthedocs.io/en/latest/treemethod.html}{Tree Methods}.\n\nChoices: \\code{\"auto\"}, \\code{\"exact\"}, \\code{\"approx\"}, \\code{\"hist\"}, this is a combination of commonly\nused updaters. For other updaters like \\code{\"refresh\"}, set the parameter \\code{updater}\ndirectly.\n\\itemize{\n\\item \\code{\"auto\"}: Same as the \\code{\"hist\"} tree method.\n\\item \\code{\"exact\"}: Exact greedy algorithm. Enumerates all split candidates.\n\\item \\code{\"approx\"}: Approximate greedy algorithm using quantile sketch and gradient histogram.\n\\item \\code{\"hist\"}: Faster histogram optimized approximate greedy algorithm.\n}}\n\n\\item{scale_pos_weight}{(for Tree Booster) (default=1)\nControl the balance of positive and negative weights, useful for unbalanced classes. A typical value to consider: \\verb{sum(negative instances) / sum(positive instances)}. See \\href{https://xgboost.readthedocs.io/en/latest/tutorials/param_tuning.html}{Parameters Tuning} for more discussion. Also, see Higgs Kaggle competition demo for examples: \\href{https://github.com/dmlc/xgboost/blob/master/demo/kaggle-higgs/higgs-train.R}{R}, \\href{https://github.com/dmlc/xgboost/blob/master/demo/kaggle-higgs/higgs-numpy.py}{py1}, \\href{https://github.com/dmlc/xgboost/blob/master/demo/kaggle-higgs/higgs-cv.py}{py2}, \\href{https://github.com/dmlc/xgboost/blob/master/demo/guide-python/cross_validation.py}{py3}.}\n\n\\item{updater}{Has different meanings depending on the type of booster.\n\\itemize{\n\\item For tree-based boosters:\nA comma separated string defining the sequence of tree updaters to run, providing a modular way to construct and to modify the trees. This is an advanced parameter that is usually set automatically, depending on some other parameters. However, it could be also set explicitly by a user. The following updaters exist:\n\\itemize{\n\\item \\code{\"grow_colmaker\"}: non-distributed column-based construction of trees.\n\\item \\code{\"grow_histmaker\"}: distributed tree construction with row-based data splitting based on global proposal of histogram counting.\n\\item \\code{\"grow_quantile_histmaker\"}: Grow tree using quantized histogram.\n\\item \\code{\"grow_gpu_hist\"}: Enabled when \\code{tree_method} is set to \\code{\"hist\"} along with \\code{device=\"cuda\"}.\n\\item \\code{\"grow_gpu_approx\"}: Enabled when \\code{tree_method} is set to \\code{\"approx\"} along with \\code{device=\"cuda\"}.\n\\item \\code{\"sync\"}: synchronizes trees in all distributed nodes.\n\\item \\code{\"refresh\"}: refreshes tree's statistics and/or leaf values based on the current data. Note that no random subsampling of data rows is performed.\n\\item \\code{\"prune\"}: prunes the splits where loss < \\code{min_split_loss} (or \\code{gamma}) and nodes that have depth greater than \\code{max_depth}.\n}\n\\item For \\code{booster=\"gblinear\"}:\n(default= \\code{\"shotgun\"}) Choice of algorithm to fit linear model\n\\itemize{\n\\item \\code{\"shotgun\"}: Parallel coordinate descent algorithm based on shotgun algorithm. Uses 'hogwild' parallelism and therefore produces a nondeterministic solution on each run.\n\\item \\code{\"coord_descent\"}: Ordinary coordinate descent algorithm. Also multithreaded but still produces a deterministic solution. When the \\code{device} parameter is set to \\code{\"cuda\"} or \\code{\"gpu\"}, a GPU variant would be used.\n}\n}}\n\n\\item{refresh_leaf}{(for Tree Booster) (default=1)\nThis is a parameter of the \\code{\"refresh\"} updater. When this flag is 1, tree leafs as well as tree nodes' stats are updated. When it is 0, only node stats are updated.}\n\n\\item{grow_policy}{(for Tree Booster) (default= \\code{\"depthwise\"})\n\\itemize{\n\\item Controls a way new nodes are added to the tree.\n\\item Currently supported only if \\code{tree_method} is set to \\code{\"hist\"} or \\code{\"approx\"}.\n\\item Choices: \\code{\"depthwise\"}, \\code{\"lossguide\"}\n\\itemize{\n\\item \\code{\"depthwise\"}: split at nodes closest to the root.\n\\item \\code{\"lossguide\"}: split at nodes with highest loss change.\n}\n}}\n\n\\item{max_leaves}{(for Tree Booster) (default=0, type=int32)\nMaximum number of nodes to be added. Not used by \\code{\"exact\"} tree method.}\n\n\\item{max_bin}{(for Tree Booster) (default=256, type=int32)\n\\itemize{\n\\item Only used if \\code{tree_method} is set to \\code{\"hist\"} or \\code{\"approx\"}.\n\\item Maximum number of discrete bins to bucket continuous features.\n\\item Increasing this number improves the optimality of splits at the cost of higher computation time.\n}}\n\n\\item{num_parallel_tree}{(for Tree Booster) (default=1)\nNumber of parallel trees constructed during each iteration. This option is used to support boosted random forest.}\n\n\\item{monotone_constraints}{(for Tree Booster)\nConstraint of variable monotonicity. See \\href{https://xgboost.readthedocs.io/en/latest/tutorials/monotonic.html}{Monotonic Constraints} for more information.}\n\n\\item{interaction_constraints}{(for Tree Booster)\nConstraints for interaction representing permitted interactions. The constraints must\nbe specified in the form of a nest list, e.g. \\code{list(c(0, 1), c(2, 3, 4))}, where each inner\nlist is a group of indices of features (base-0 numeration) that are allowed to interact with each other.\nSee \\href{https://xgboost.readthedocs.io/en/latest/tutorials/feature_interaction_constraint.html}{Feature Interaction Constraints} for more information.}\n\n\\item{multi_strategy}{(for Tree Booster) (default = \\code{\"one_output_per_tree\"})\nThe strategy used for training multi-target models, including multi-target regression\nand multi-class classification. See \\href{https://xgboost.readthedocs.io/en/latest/tutorials/multioutput.html}{Multiple Outputs} for more information.\n\\itemize{\n\\item \\code{\"one_output_per_tree\"}: One model for each target.\n\\item \\code{\"multi_output_tree\"}: Use multi-target trees.\n}\n\nVersion added: 2.0.0\n\nNote: This parameter is working-in-progress.}\n\n\\item{base_score}{\\itemize{\n\\item The initial prediction score of all instances, global bias\n\\item The parameter is automatically estimated for selected objectives before training. To\ndisable the estimation, specify a real number argument.\n\\item If \\code{base_margin} is supplied, \\code{base_score} will not be added.\n\\item For sufficient number of iterations, changing this value will not have too much effect.\n}}\n\n\\item{eval_metric}{(default according to objective)\n\\itemize{\n\\item Evaluation metrics for validation data, a default metric will be assigned according to objective (rmse for regression, and logloss for classification, \\verb{mean average precision} for \\code{rank:map}, etc.)\n\\item User can add multiple evaluation metrics.\n\\item The choices are listed below:\n\\itemize{\n\\item \\code{\"rmse\"}: root mean square error\n\\item \\code{\"rmsle\"}: root mean square log error: \\eqn{\\sqrt{\\frac{1}{N}[log(pred + 1) - log(label + 1)]^2}}. Default metric of \\code{\"reg:squaredlogerror\"} objective. This metric reduces errors generated by outliers in dataset. But because \\code{log} function is employed, \\code{\"rmsle\"} might output \\code{nan} when prediction value is less than -1. See \\code{\"reg:squaredlogerror\"} for other requirements.\n\\item \\code{\"mae\"}: mean absolute error.\n\\item \\code{\"mape\"}: mean absolute percentage error.\n\\item \\code{\"mphe\"}: mean Pseudo Huber error. Default metric of \\code{\"reg:pseudohubererror\"} objective.\n\\item \\code{\"logloss\"}: negative log-likelihood.\n\\item \\code{\"error\"}: Binary classification error rate. It is calculated as \\verb{#(wrong cases)/#(all cases)}. For the predictions, the evaluation will regard the instances with prediction value larger than 0.5 as positive instances, and the others as negative instances.\n\\item \\code{\"error@t\"}: a different than 0.5 binary classification threshold value could be specified by providing a numerical value through 't'.\n\\item \\code{\"merror\"}: Multiclass classification error rate. It is calculated as \\verb{#(wrong cases)/#(all cases)}.\n\\item \\code{\"mlogloss\"}: \\href{https://scikit-learn.org/stable/modules/generated/sklearn.metrics.log_loss.html}{Multiclass logloss}.\n\\item \\code{\"auc\"}: area under the receiver-operating characteristic curve.\nAvailable for classification and learning-to-rank tasks.\n\\itemize{\n\\item When used with binary classification, the objective should be \\code{\"binary:logistic\"} or similar functions that work on probability.\n\\item When used with multi-class classification, objective should be \\code{\"multi:softprob\"} instead of \\code{\"multi:softmax\"}, as the latter doesn't output probability. Also the AUC is calculated by 1-vs-rest with reference class weighted by class prevalence.\n\\item When used with LTR task, the AUC is computed by comparing pairs of documents to count correctly sorted pairs. This corresponds to pairwise learning to rank. The implementation has some issues with average AUC around groups and distributed workers not being well-defined.\n\\item On a single machine the AUC calculation is exact. In a distributed environment the AUC is a weighted average over the AUC of training rows on each node - therefore, distributed AUC is an approximation sensitive to the distribution of data across workers. Use another metric in distributed environments if precision and reproducibility are important.\n\\item When input dataset contains only negative or positive samples, the output is \\code{NaN}. The behavior is implementation defined, for instance, \\code{scikit-learn} returns \\eqn{0.5} instead.\n}\n\\item \\code{\"aucpr\"}: area under the PR curve\nAvailable for classification and learning-to-rank tasks.\n\nAfter XGBoost 1.6, both of the requirements and restrictions for using \\code{\"aucpr\"} in classification problem are similar to \\code{\"auc\"}. For ranking task, only binary relevance label \\eqn{y \\in [0, 1]} is supported. Different from \\code{\"map\"} (mean average precision), \\code{\"aucpr\"} calculates the \\emph{interpolated} area under precision recall curve using continuous interpolation.\n\\item \\code{\"pre\"}: Precision at \\eqn{k}. Supports only learning to rank task.\n\\item \\code{\"ndcg\"}: normalized discounted cumulative gain\n\\item \\code{\"map\"}: mean average precision\n\nThe \\verb{average precision} is defined as:\n\n\\eqn{AP@l = \\frac{1}{min{(l, N)}}\\sum^l_{k=1}P@k \\cdot I_{(k)}}\n\nwhere \\eqn{I_{(k)}} is an indicator function that equals to \\eqn{1} when the document at \\eqn{k} is relevant and \\eqn{0} otherwise. The \\eqn{P@k} is the precision at \\eqn{k}, and \\eqn{N} is the total number of relevant documents. Lastly, the \\verb{mean average precision} is defined as the weighted average across all queries.\n\\item \\code{\"ndcg@n\"}, \\code{\"map@n\"}, \\code{\"pre@n\"}: \\eqn{n} can be assigned as an integer to cut off the top positions in the lists for evaluation.\n\\item \\code{\"ndcg-\"}, \\code{\"map-\"}, \\code{\"ndcg@n-\"}, \\code{\"map@n-\"}: In XGBoost, the NDCG and MAP evaluate the score of a list without any positive samples as \\eqn{1}. By appending \"-\" to the evaluation metric name, we can ask XGBoost to evaluate these scores as \\eqn{0} to be consistent under some conditions.\n\\item \\code{\"poisson-nloglik\"}: negative log-likelihood for Poisson regression\n\\item \\code{\"gamma-nloglik\"}: negative log-likelihood for gamma regression\n\\item \\code{\"cox-nloglik\"}: negative partial log-likelihood for Cox proportional hazards regression\n\\item \\code{\"gamma-deviance\"}: residual deviance for gamma regression\n\\item \\code{\"tweedie-nloglik\"}: negative log-likelihood for Tweedie regression (at a specified value of the \\code{tweedie_variance_power} parameter)\n\\item \\code{\"aft-nloglik\"}: Negative log likelihood of Accelerated Failure Time model.\nSee \\href{https://xgboost.readthedocs.io/en/latest/tutorials/aft_survival_analysis.html}{Survival Analysis with Accelerated Failure Time} for details.\n\\item \\code{\"interval-regression-accuracy\"}: Fraction of data points whose predicted labels fall in the interval-censored labels.\nOnly applicable for interval-censored data. See \\href{https://xgboost.readthedocs.io/en/latest/tutorials/aft_survival_analysis.html}{Survival Analysis with Accelerated Failure Time} for details.\n}\n}}\n\n\\item{seed_per_iteration}{(default= \\code{FALSE})\nSeed PRNG determnisticly via iterator number.}\n\n\\item{device}{(default= \\code{\"cpu\"})\nDevice for XGBoost to run. User can set it to one of the following values:\n\\itemize{\n\\item \\code{\"cpu\"}: Use CPU.\n\\item \\code{\"cuda\"}: Use a GPU (CUDA device).\n\\item \\code{\"cuda:\"}: \\verb{} is an integer that specifies the ordinal of the GPU (which GPU do you want to use if you have more than one devices).\n\\item \\code{\"gpu\"}: Default GPU device selection from the list of available and supported devices. Only \\code{\"cuda\"} devices are supported currently.\n\\item \\code{\"gpu:\"}: Default GPU device selection from the list of available and supported devices. Only \\code{\"cuda\"} devices are supported currently.\n}\n\nFor more information about GPU acceleration, see \\href{https://xgboost.readthedocs.io/en/latest/gpu/index.html}{XGBoost GPU Support}. In distributed environments, ordinal selection is handled by distributed frameworks instead of XGBoost. As a result, using \\code{\"cuda:\"} will result in an error. Use \\code{\"cuda\"} instead.\n\nVersion added: 2.0.0\n\nNote: if XGBoost was installed from CRAN, it won't have GPU support enabled, thus only \\code{\"cpu\"} will be available.\nTo get GPU support, the R package for XGBoost must be installed from source or from the GitHub releases - see\n\\href{https://xgboost.readthedocs.io/en/latest/install.html#r}{instructions}.}\n\n\\item{disable_default_eval_metric}{(default= \\code{FALSE})\nFlag to disable default metric. Set to 1 or \\code{TRUE} to disable.}\n\n\\item{use_rmm}{Whether to use RAPIDS Memory Manager (RMM) to allocate cache GPU\nmemory. The primary memory is always allocated on the RMM pool when XGBoost is built\n(compiled) with the RMM plugin enabled. Valid values are \\code{TRUE} and \\code{FALSE}. See\n\\href{https://xgboost.readthedocs.io/en/latest/python/rmm-examples/index.html}{Using XGBoost with RAPIDS Memory Manager (RMM) plugin} for details.}\n\n\\item{max_cached_hist_node}{(for Non-Exact Tree Methods) (default = 65536)\nMaximum number of cached nodes for histogram. This can be used with the \\code{\"hist\"} and the\n\\code{\"approx\"} tree methods.\n\nVersion added: 2.0.0\n\\itemize{\n\\item For most of the cases this parameter should not be set except for growing deep\ntrees. After 3.0, this parameter affects GPU algorithms as well.\n}}\n\n\\item{max_cat_to_onehot}{(for Non-Exact Tree Methods)\nA threshold for deciding whether XGBoost should use one-hot encoding based split for\ncategorical data. When number of categories is lesser than the threshold then one-hot\nencoding is chosen, otherwise the categories will be partitioned into children nodes.\n\nVersion added: 1.6.0}\n\n\\item{max_cat_threshold}{(for Non-Exact Tree Methods)\nMaximum number of categories considered for each split. Used only by partition-based\nsplits for preventing over-fitting.\n\nVersion added: 1.7.0}\n\n\\item{sample_type}{(for Dart Booster) (default= \\code{\"uniform\"})\nType of sampling algorithm.\n\\itemize{\n\\item \\code{\"uniform\"}: dropped trees are selected uniformly.\n\\item \\code{\"weighted\"}: dropped trees are selected in proportion to weight.\n}}\n\n\\item{normalize_type}{(for Dart Booster) (default= \\code{\"tree\"})\nType of normalization algorithm.\n\\itemize{\n\\item \\code{\"tree\"}: new trees have the same weight of each of dropped trees.\n\\itemize{\n\\item Weight of new trees are \\code{1 / (k + learning_rate)}.\n\\item Dropped trees are scaled by a factor of \\code{k / (k + learning_rate)}.\n}\n\\item \\code{\"forest\"}: new trees have the same weight of sum of dropped trees (forest).\n\\itemize{\n\\item Weight of new trees are \\code{1 / (1 + learning_rate)}.\n\\item Dropped trees are scaled by a factor of \\code{1 / (1 + learning_rate)}.\n}\n}}\n\n\\item{rate_drop}{(for Dart Booster) (default=0.0)\nDropout rate (a fraction of previous trees to drop during the dropout).\n\nrange: \\eqn{[0.0, 1.0]}}\n\n\\item{one_drop}{(for Dart Booster) (default=0)\nWhen this flag is enabled, at least one tree is always dropped during the dropout (allows Binomial-plus-one or epsilon-dropout from the original DART paper).}\n\n\\item{skip_drop}{(for Dart Booster) (default=0.0)\nProbability of skipping the dropout procedure during a boosting iteration.\n\\itemize{\n\\item If a dropout is skipped, new trees are added in the same manner as \\code{\"gbtree\"}.\n\\item Note that non-zero \\code{skip_drop} has higher priority than \\code{rate_drop} or \\code{one_drop}.\n}\n\nrange: \\eqn{[0.0, 1.0]}}\n\n\\item{feature_selector}{(for Linear Booster) (default= \\code{\"cyclic\"})\nFeature selection and ordering method\n\\itemize{\n\\item \\code{\"cyclic\"}: Deterministic selection by cycling through features one at a time.\n\\item \\code{\"shuffle\"}: Similar to \\code{\"cyclic\"} but with random feature shuffling prior to each update.\n\\item \\code{\"random\"}: A random (with replacement) coordinate selector.\n\\item \\code{\"greedy\"}: Select coordinate with the greatest gradient magnitude. It has \\code{O(num_feature^2)} complexity. It is fully deterministic. It allows restricting the selection to \\code{top_k} features per group with the largest magnitude of univariate weight change, by setting the \\code{top_k} parameter. Doing so would reduce the complexity to \\code{O(num_feature*top_k)}.\n\\item \\code{\"thrifty\"}: Thrifty, approximately-greedy feature selector. Prior to cyclic updates, reorders features in descending magnitude of their univariate weight changes. This operation is multithreaded and is a linear complexity approximation of the quadratic greedy selection. It allows restricting the selection to \\code{top_k} features per group with the largest magnitude of univariate weight change, by setting the \\code{top_k} parameter.\n}}\n\n\\item{top_k}{(for Linear Booster) (default=0)\nThe number of top features to select in \\code{greedy} and \\code{thrifty} feature selector. The value of 0 means using all the features.}\n\n\\item{num_class}{Number of classes when using multi-class classification objectives (e.g. \\code{objective=\"multi:softprob\"})}\n\n\\item{tweedie_variance_power}{(for Tweedie Regression (\\code{\"objective=reg:tweedie\"})) (default=1.5)\n\\itemize{\n\\item Parameter that controls the variance of the Tweedie distribution \\code{var(y) ~ E(y)^tweedie_variance_power}\n\\item range: \\eqn{(1,2)}\n\\item Set closer to 2 to shift towards a gamma distribution\n\\item Set closer to 1 to shift towards a Poisson distribution.\n}}\n\n\\item{huber_slope}{(for using Pseudo-Huber (\\verb{\"reg:pseudohubererror}\")) (default = 1.0)\nA parameter used for Pseudo-Huber loss to define the \\eqn{\\delta} term.}\n\n\\item{quantile_alpha}{(for using Quantile Loss (\\code{\"reg:quantileerror\"}))\nA scalar or a list of targeted quantiles (passed as a numeric vector).\n\nVersion added: 2.0.0}\n\n\\item{expectile_alpha}{(for using Expectile Loss (\\code{\"reg:expectileerror\"}))\nA scalar or a list of targeted expectiles (passed as a numeric vector).}\n\n\\item{aft_loss_distribution}{(when using AFT Survival Loss (\\code{\"survival:aft\"}) and Negative Log Likelihood of AFT metric (\\code{\"aft-nloglik\"}))\nProbability Density Function, \\code{\"normal\"}, \\code{\"logistic\"}, or \\code{\"extreme\"}.}\n\n\\item{aft_loss_distribution_scale}{(when using AFT Survival Loss (\\code{\"survival:aft\"}) and Negative Log Likelihood of AFT metric (\\code{\"aft-nloglik\"}))\nScaling factor for the AFT distribution. Range: \\eqn{(0, \\infty)}.}\n\n\\item{lambdarank_pair_method}{(for learning to rank (\\code{\"rank:ndcg\"}, \\code{\"rank:map\"}, \\code{\"rank:pairwise\"})) (default = \\code{\"topk\"})\nHow to construct pairs for pair-wise learning.\n\\itemize{\n\\item \\code{\"mean\"}: Sample \\code{lambdarank_num_pair_per_sample} pairs for each document in the query list.\n\\item \\code{\"topk\"}: Focus on top-\\code{lambdarank_num_pair_per_sample} documents. Construct \\eqn{|query|} pairs for each document at the top-\\code{lambdarank_num_pair_per_sample} ranked by the model.\n}}\n\n\\item{lambdarank_num_pair_per_sample}{(for learning to rank (\\code{\"rank:ndcg\"}, \\code{\"rank:map\"}, \\code{\"rank:pairwise\"}))\nIt specifies the number of pairs sampled for each document when pair method is \\code{\"mean\"}, or the truncation level for queries when the pair method is \\code{\"topk\"}. For example, to train with \\verb{ndcg@6}, set \\code{\"lambdarank_num_pair_per_sample\"} to \\eqn{6} and \\code{lambdarank_pair_method} to \\code{\"topk\"}.\n\nrange = \\eqn{[1, \\infty)}}\n\n\\item{lambdarank_normalization}{(for learning to rank (\\code{\"rank:ndcg\"}, \\code{\"rank:map\"}, \\code{\"rank:pairwise\"})) (default = \\code{TRUE})\nWhether to normalize the leaf value by lambda gradient. This can sometimes stagnate the training progress.\n\nVersion added: 2.1.0}\n\n\\item{lambdarank_score_normalization}{Whether to normalize the delta metric by the difference of prediction scores. This can\nsometimes stagnate the training progress. With pairwise ranking, we can normalize the\ngradient using the difference between two samples in each pair to reduce influence from\nthe pairs that have large difference in ranking scores. This can help us regularize the\nmodel to reduce bias and prevent overfitting. Similar to other regularization\ntechniques, this might prevent training from converging.\n\nThere was no normalization before 2.0. In 2.0 and later versions this is used by\ndefault. In 3.0, we made this an option that users can disable.\n\nVersion added: 3.0.0}\n\n\\item{lambdarank_unbiased}{(for learning to rank (\\code{\"rank:ndcg\"}, \\code{\"rank:map\"}, \\code{\"rank:pairwise\"})) (default = \\code{FALSE})\nSpecify whether do we need to debias input click data.}\n\n\\item{lambdarank_bias_norm}{(for learning to rank (\\code{\"rank:ndcg\"}, \\code{\"rank:map\"}, \\code{\"rank:pairwise\"})) (default = 2.0)\n\\eqn{L_p} normalization for position debiasing, default is \\eqn{L_2}. Only relevant when \\code{lambdarank_unbiased} is set to \\code{TRUE}.}\n\n\\item{ndcg_exp_gain}{(for learning to rank (\\code{\"rank:ndcg\"}, \\code{\"rank:map\"}, \\code{\"rank:pairwise\"})) (default = \\code{TRUE})\nWhether we should use exponential gain function for \\code{NDCG}. There are two forms of gain function for \\code{NDCG}, one is using relevance value directly while the other is using\\eqn{2^{rel} - 1} to emphasize on retrieving relevant documents. When \\code{ndcg_exp_gain} is \\code{TRUE} (the default), relevance degree cannot be greater than 31.}\n}\n\\value{\nA list with the entries that were passed non-NULL values. It is intended to\nbe passed as argument \\code{params} to \\code{\\link[=xgb.train]{xgb.train()}} or \\code{\\link[=xgb.cv]{xgb.cv()}}.\n}\n\\description{\nConvenience function to generate a list of named XGBoost parameters, which\ncan be passed as argument \\code{params} to \\code{\\link[=xgb.train]{xgb.train()}}. See the \\href{https://xgboost.readthedocs.io/en/stable/parameter.html}{online documentation} for more details.\n\nThe purpose of this function is to enable IDE autocompletions and to provide in-package\ndocumentation for all the possible parameters that XGBoost accepts. The output from this\nfunction is just a regular R list containing the parameters that were set to non-default\nvalues. Note that this function will not perform any validation on the supplied arguments.\n\nIf passing \\code{NULL} for a given parameter (the default for all of them), then the default\nvalue for that parameter will be used. Default values are automatically determined by the\nXGBoost core library upon calls to \\code{\\link[=xgb.train]{xgb.train()}} or \\code{\\link[=xgb.cv]{xgb.cv()}}, and are subject to change\nover XGBoost library versions. Some of them might differ according to the\nbooster type (e.g. defaults for regularization are different for linear and tree-based boosters).\n}\n" }, { "path": "R-package/man/xgb.plot.deepness.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/xgb.ggplot.R, R/xgb.plot.deepness.R\n\\name{xgb.ggplot.deepness}\n\\alias{xgb.ggplot.deepness}\n\\alias{xgb.plot.deepness}\n\\title{Plot model tree depth}\n\\usage{\nxgb.ggplot.deepness(\n model = NULL,\n which = c(\"2x1\", \"max.depth\", \"med.depth\", \"med.weight\")\n)\n\nxgb.plot.deepness(\n model = NULL,\n which = c(\"2x1\", \"max.depth\", \"med.depth\", \"med.weight\"),\n plot = TRUE,\n ...\n)\n}\n\\arguments{\n\\item{model}{Either an \\code{xgb.Booster} model, or the \"data.table\" returned\nby \\code{\\link[=xgb.model.dt.tree]{xgb.model.dt.tree()}}.}\n\n\\item{which}{Which distribution to plot (see details).}\n\n\\item{plot}{Should the plot be shown? Default is \\code{TRUE}.}\n\n\\item{...}{Other parameters passed to \\code{\\link[graphics:barplot]{graphics::barplot()}} or \\code{\\link[graphics:plot.default]{graphics::plot()}}.}\n}\n\\value{\nThe return value of the two functions is as follows:\n\\itemize{\n\\item \\code{xgb.plot.deepness()}: A \"data.table\" (invisibly).\nEach row corresponds to a terminal leaf in the model. It contains its information\nabout depth, cover, and weight (used in calculating predictions).\nIf \\code{plot = TRUE}, also a plot is shown.\n\\item \\code{xgb.ggplot.deepness()}: When \\code{which = \"2x1\"}, a list of two \"ggplot\" objects,\nand a single \"ggplot\" object otherwise.\n}\n}\n\\description{\nVisualizes distributions related to the depth of tree leaves.\n\\itemize{\n\\item \\code{xgb.plot.deepness()} uses base R graphics, while\n\\item \\code{xgb.ggplot.deepness()} uses \"ggplot2\".\n}\n}\n\\details{\nWhen \\code{which = \"2x1\"}, two distributions with respect to the leaf depth\nare plotted on top of each other:\n\\enumerate{\n\\item The distribution of the number of leaves in a tree model at a certain depth.\n\\item The distribution of the average weighted number of observations (\"cover\")\nending up in leaves at a certain depth.\n}\n\nThose could be helpful in determining sensible ranges of the \\code{max_depth}\nand \\code{min_child_weight} parameters.\n\nWhen \\code{which = \"max.depth\"} or \\code{which = \"med.depth\"}, plots of either maximum or\nmedian depth per tree with respect to the tree number are created.\n\nFinally, \\code{which = \"med.weight\"} allows to see how\na tree's median absolute leaf weight changes through the iterations.\n\nThese functions have been inspired by the blog post\n\\url{https://github.com/aysent/random-forest-leaf-visualization}.\n}\n\\examples{\n\ndata(agaricus.train, package = \"xgboost\")\n## Keep the number of threads to 2 for examples\nnthread <- 2\ndata.table::setDTthreads(nthread)\n\n## Change max_depth to a higher number to get a more significant result\nmodel <- xgboost(\n agaricus.train$data, factor(agaricus.train$label),\n nrounds = 50,\n max_depth = 6,\n nthreads = nthread,\n subsample = 0.5,\n min_child_weight = 2\n)\n\nxgb.plot.deepness(model)\nxgb.ggplot.deepness(model)\n\nxgb.plot.deepness(\n model, which = \"max.depth\", pch = 16, col = rgb(0, 0, 1, 0.3), cex = 2\n)\n\nxgb.plot.deepness(\n model, which = \"med.weight\", pch = 16, col = rgb(0, 0, 1, 0.3), cex = 2\n)\n\n}\n\\seealso{\n\\code{\\link[=xgb.train]{xgb.train()}} and \\code{\\link[=xgb.model.dt.tree]{xgb.model.dt.tree()}}.\n}\n" }, { "path": "R-package/man/xgb.plot.importance.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/xgb.ggplot.R, R/xgb.plot.importance.R\n\\name{xgb.ggplot.importance}\n\\alias{xgb.ggplot.importance}\n\\alias{xgb.plot.importance}\n\\title{Plot feature importance}\n\\usage{\nxgb.ggplot.importance(\n importance_matrix = NULL,\n top_n = NULL,\n measure = NULL,\n rel_to_first = FALSE,\n n_clusters = seq_len(10),\n ...\n)\n\nxgb.plot.importance(\n importance_matrix = NULL,\n top_n = NULL,\n measure = NULL,\n rel_to_first = FALSE,\n left_margin = 10,\n cex = NULL,\n plot = TRUE,\n ...\n)\n}\n\\arguments{\n\\item{importance_matrix}{A \\code{data.table} as returned by \\code{\\link[=xgb.importance]{xgb.importance()}}.}\n\n\\item{top_n}{Maximal number of top features to include into the plot.}\n\n\\item{measure}{The name of importance measure to plot.\nWhen \\code{NULL}, 'Gain' would be used for trees and 'Weight' would be used for gblinear.}\n\n\\item{rel_to_first}{Whether importance values should be represented as relative to\nthe highest ranked feature, see Details.}\n\n\\item{n_clusters}{A numeric vector containing the min and the max range\nof the possible number of clusters of bars.}\n\n\\item{...}{Other parameters passed to \\code{\\link[graphics:barplot]{graphics::barplot()}}\n(except \\code{horiz}, \\code{border}, \\code{cex.names}, \\code{names.arg}, and \\code{las}).\nOnly used in \\code{xgb.plot.importance()}.}\n\n\\item{left_margin}{Adjust the left margin size to fit feature names.\nWhen \\code{NULL}, the existing \\code{par(\"mar\")} is used.}\n\n\\item{cex}{Passed as \\code{cex.names} parameter to \\code{\\link[graphics:barplot]{graphics::barplot()}}.}\n\n\\item{plot}{Should the barplot be shown? Default is \\code{TRUE}.}\n}\n\\value{\nThe return value depends on the function:\n\\itemize{\n\\item \\code{xgb.plot.importance()}: Invisibly, a \"data.table\" with \\code{n_top} features sorted\nby importance. If \\code{plot = TRUE}, the values are also plotted as barplot.\n\\item \\code{xgb.ggplot.importance()}: A customizable \"ggplot\" object.\nE.g., to change the title, set \\code{+ ggtitle(\"A GRAPH NAME\")}.\n}\n}\n\\description{\nRepresents previously calculated feature importance as a bar graph.\n\\itemize{\n\\item \\code{xgb.plot.importance()} uses base R graphics, while\n\\item \\code{xgb.ggplot.importance()} uses \"ggplot\".\n}\n}\n\\details{\nThe graph represents each feature as a horizontal bar of length proportional to the\nimportance of a feature. Features are sorted by decreasing importance.\nIt works for both \"gblinear\" and \"gbtree\" models.\n\nWhen \\code{rel_to_first = FALSE}, the values would be plotted as in \\code{importance_matrix}.\nFor a \"gbtree\" model, that would mean being normalized to the total of 1\n(\"what is feature's importance contribution relative to the whole model?\").\nFor linear models, \\code{rel_to_first = FALSE} would show actual values of the coefficients.\nSetting \\code{rel_to_first = TRUE} allows to see the picture from the perspective of\n\"what is feature's importance contribution relative to the most important feature?\"\n\nThe \"ggplot\" backend performs 1-D clustering of the importance values,\nwith bar colors corresponding to different clusters having similar importance values.\n}\n\\examples{\ndata(agaricus.train)\n\n## Keep the number of threads to 2 for examples\nnthread <- 2\ndata.table::setDTthreads(nthread)\n\nmodel <- xgboost(\n agaricus.train$data, factor(agaricus.train$label),\n nrounds = 2,\n max_depth = 3,\n nthreads = nthread\n)\n\nimportance_matrix <- xgb.importance(model)\nxgb.plot.importance(\n importance_matrix, rel_to_first = TRUE, xlab = \"Relative importance\"\n)\n\ngg <- xgb.ggplot.importance(\n importance_matrix, measure = \"Frequency\", rel_to_first = TRUE\n)\ngg\ngg + ggplot2::ylab(\"Frequency\")\n\n}\n\\seealso{\n\\code{\\link[graphics:barplot]{graphics::barplot()}}\n}\n" }, { "path": "R-package/man/xgb.plot.multi.trees.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/xgb.plot.multi.trees.R\n\\name{xgb.plot.multi.trees}\n\\alias{xgb.plot.multi.trees}\n\\title{Project all trees on one tree}\n\\usage{\nxgb.plot.multi.trees(\n model,\n features_keep = 5,\n plot_width = NULL,\n plot_height = NULL,\n render = TRUE,\n ...\n)\n}\n\\arguments{\n\\item{model}{Object of class \\code{xgb.Booster}. If it contains feature names\n(they can be set through \\code{\\link[=setinfo]{setinfo()}}, they will be used in the\noutput from this function.}\n\n\\item{features_keep}{Number of features to keep in each position of the multi trees,\nby default 5.}\n\n\\item{plot_width, plot_height}{Width and height of the graph in pixels.\nThe values are passed to \\code{DiagrammeR::render_graph()}.}\n\n\\item{render}{Should the graph be rendered or not? The default is \\code{TRUE}.}\n\n\\item{...}{Not used.\n\nSome arguments that were part of this function in previous XGBoost versions are currently\ndeprecated or have been renamed. If a deprecated or renamed argument is passed, will throw\na warning (by default) and use its current equivalent instead. This warning will become an\nerror if using the \\link[=xgboost-options]{'strict mode' option}.\n\nIf some additional argument is passed that is neither a current function argument nor\na deprecated or renamed argument, a warning or error will be thrown depending on the\n'strict mode' option.\n\n\\bold{Important:} \\code{...} will be removed in a future version, and all the current\ndeprecation warnings will become errors. Please use only arguments that form part of\nthe function signature.}\n}\n\\value{\nRendered graph object which is an htmlwidget of ' class \\code{grViz}. Similar to\n\"ggplot\" objects, it needs to be printed when not running from the command\nline.\n}\n\\description{\nVisualization of the ensemble of trees as a single collective unit.\n}\n\\details{\nNote that this function does not work with models that were fitted to\ncategorical data.\n\nThis function tries to capture the complexity of a gradient boosted tree model\nin a cohesive way by compressing an ensemble of trees into a single tree-graph representation.\nThe goal is to improve the interpretability of a model generally seen as black box.\n\nNote: this function is applicable to tree booster-based models only.\n\nIt takes advantage of the fact that the shape of a binary tree is only defined by\nits depth (therefore, in a boosting model, all trees have similar shape).\n\nMoreover, the trees tend to reuse the same features.\n\nThe function projects each tree onto one, and keeps for each position the\n\\code{features_keep} first features (based on the Gain per feature measure).\n\nThis function is inspired by this blog post:\n\\url{https://wellecks.wordpress.com/2015/02/21/peering-into-the-black-box-visualizing-lambdamart/}\n}\n\\examples{\n\ndata(agaricus.train, package = \"xgboost\")\n\n## Keep the number of threads to 2 for examples\nnthread <- 2\ndata.table::setDTthreads(nthread)\n\nmodel <- xgboost(\n agaricus.train$data, factor(agaricus.train$label),\n nrounds = 30,\n verbosity = 0L,\n nthreads = nthread,\n max_depth = 15,\n learning_rate = 1,\n min_child_weight = 50\n)\n\np <- xgb.plot.multi.trees(model, features_keep = 3)\nprint(p)\n\n# Below is an example of how to save this plot to a file.\nif (require(\"DiagrammeR\") && require(\"DiagrammeRsvg\") && require(\"rsvg\")) {\n fname <- file.path(tempdir(), \"tree.pdf\")\n gr <- xgb.plot.multi.trees(model, features_keep = 3, render = FALSE)\n export_graph(gr, fname, width = 1500, height = 600)\n}\n}\n" }, { "path": "R-package/man/xgb.plot.shap.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/xgb.plot.shap.R\n\\name{xgb.plot.shap}\n\\alias{xgb.plot.shap}\n\\title{SHAP dependence plots}\n\\usage{\nxgb.plot.shap(\n data,\n shap_contrib = NULL,\n features = NULL,\n top_n = 1,\n model = NULL,\n trees = NULL,\n target_class = NULL,\n approxcontrib = FALSE,\n subsample = NULL,\n n_col = 1,\n col = rgb(0, 0, 1, 0.2),\n pch = \".\",\n discrete_n_uniq = 5,\n discrete_jitter = 0.01,\n ylab = \"SHAP\",\n plot_NA = TRUE,\n col_NA = rgb(0.7, 0, 1, 0.6),\n pch_NA = \".\",\n pos_NA = 1.07,\n plot_loess = TRUE,\n col_loess = 2,\n span_loess = 0.5,\n which = c(\"1d\", \"2d\"),\n plot = TRUE,\n ...\n)\n}\n\\arguments{\n\\item{data}{The data to explain as a \\code{matrix}, \\code{dgCMatrix}, or \\code{data.frame}.}\n\n\\item{shap_contrib}{Matrix of SHAP contributions of \\code{data}.\nThe default (\\code{NULL}) computes it from \\code{model} and \\code{data}.}\n\n\\item{features}{Vector of column indices or feature names to plot. When \\code{NULL}\n(default), the \\code{top_n} most important features are selected by \\code{\\link[=xgb.importance]{xgb.importance()}}.}\n\n\\item{top_n}{How many of the most important features (<= 100) should be selected?\nBy default 1 for SHAP dependence and 10 for SHAP summary.\nOnly used when \\code{features = NULL}.}\n\n\\item{model}{An \\code{xgb.Booster} model. Only required when \\code{shap_contrib = NULL} or\n\\code{features = NULL}.}\n\n\\item{trees}{Passed to \\code{\\link[=xgb.importance]{xgb.importance()}} when \\code{features = NULL}.}\n\n\\item{target_class}{Only relevant for multiclass models. The default (\\code{NULL})\naverages the SHAP values over all classes. Pass a (0-based) class index\nto show only SHAP values of that class.}\n\n\\item{approxcontrib}{Passed to \\code{\\link[=predict.xgb.Booster]{predict.xgb.Booster()}} when \\code{shap_contrib = NULL}.}\n\n\\item{subsample}{Fraction of data points randomly picked for plotting.\nThe default (\\code{NULL}) will use up to 100k data points.}\n\n\\item{n_col}{Number of columns in a grid of plots.}\n\n\\item{col}{Color of the scatterplot markers.}\n\n\\item{pch}{Scatterplot marker.}\n\n\\item{discrete_n_uniq}{Maximal number of unique feature values to consider the\nfeature as discrete.}\n\n\\item{discrete_jitter}{Jitter amount added to the values of discrete features.}\n\n\\item{ylab}{The y-axis label in 1D plots.}\n\n\\item{plot_NA}{Should contributions of cases with missing values be plotted?\nDefault is \\code{TRUE}.}\n\n\\item{col_NA}{Color of marker for missing value contributions.}\n\n\\item{pch_NA}{Marker type for \\code{NA} values.}\n\n\\item{pos_NA}{Relative position of the x-location where \\code{NA} values are shown:\n\\code{min(x) + (max(x) - min(x)) * pos_NA}.}\n\n\\item{plot_loess}{Should loess-smoothed curves be plotted? (Default is \\code{TRUE}).\nThe smoothing is only done for features with more than 5 distinct values.}\n\n\\item{col_loess}{Color of loess curves.}\n\n\\item{span_loess}{The \\code{span} parameter of \\code{\\link[stats:loess]{stats::loess()}}.}\n\n\\item{which}{Whether to do univariate or bivariate plotting. Currently, only \"1d\" is implemented.}\n\n\\item{plot}{Should the plot be drawn? (Default is \\code{TRUE}).\nIf \\code{FALSE}, only a list of matrices is returned.}\n\n\\item{...}{Other parameters passed to \\code{\\link[graphics:plot.default]{graphics::plot()}}.}\n}\n\\value{\nIn addition to producing plots (when \\code{plot = TRUE}), it silently returns a list of two matrices:\n\\itemize{\n\\item \\code{data}: Feature value matrix.\n\\item \\code{shap_contrib}: Corresponding SHAP value matrix.\n}\n}\n\\description{\nVisualizes SHAP values against feature values to gain an impression of feature effects.\n}\n\\details{\nThese scatterplots represent how SHAP feature contributions depend of feature values.\nThe similarity to partial dependence plots is that they also give an idea for how feature values\naffect predictions. However, in partial dependence plots, we see marginal dependencies\nof model prediction on feature value, while SHAP dependence plots display the estimated\ncontributions of a feature to the prediction for each individual case.\n\nWhen \\code{plot_loess = TRUE}, feature values are rounded to three significant digits and\nweighted LOESS is computed and plotted, where the weights are the numbers of data points\nat each rounded value.\n\nNote: SHAP contributions are on the scale of the model margin.\nE.g., for a logistic binomial objective, the margin is on log-odds scale.\nAlso, since SHAP stands for \"SHapley Additive exPlanation\" (model prediction = sum of SHAP\ncontributions for all features + bias), depending on the objective used, transforming SHAP\ncontributions for a feature from the marginal to the prediction space is not necessarily\na meaningful thing to do.\n}\n\\examples{\n\ndata(agaricus.train, package = \"xgboost\")\ndata(agaricus.test, package = \"xgboost\")\n\n## Keep the number of threads to 1 for examples\nnthread <- 1\ndata.table::setDTthreads(nthread)\nnrounds <- 20\n\nmodel_binary <- xgboost(\n agaricus.train$data, factor(agaricus.train$label),\n nrounds = nrounds,\n verbosity = 0L,\n learning_rate = 0.1,\n max_depth = 3L,\n subsample = 0.5,\n nthreads = nthread\n)\n\nxgb.plot.shap(agaricus.test$data, model = model_binary, features = \"odor=none\")\n\ncontr <- predict(model_binary, agaricus.test$data, type = \"contrib\")\nxgb.plot.shap(agaricus.test$data, contr, model = model_binary, top_n = 12, n_col = 3)\n\n# Summary plot\nxgb.ggplot.shap.summary(agaricus.test$data, contr, model = model_binary, top_n = 12)\n\n# Multiclass example - plots for each class separately:\nx <- as.matrix(iris[, -5])\nset.seed(123)\nis.na(x[sample(nrow(x) * 4, 30)]) <- TRUE # introduce some missing values\n\nmodel_multiclass <- xgboost(\n x, iris$Species,\n nrounds = nrounds,\n verbosity = 0,\n max_depth = 2,\n subsample = 0.5,\n nthreads = nthread\n)\nnclass <- 3\ntrees0 <- seq(from = 1, by = nclass, length.out = nrounds)\ncol <- rgb(0, 0, 1, 0.5)\n\nxgb.plot.shap(\n x,\n model = model_multiclass,\n trees = trees0,\n target_class = 0,\n top_n = 4,\n n_col = 2,\n col = col,\n pch = 16,\n pch_NA = 17\n)\n\nxgb.plot.shap(\n x,\n model = model_multiclass,\n trees = trees0 + 1,\n target_class = 1,\n top_n = 4,\n n_col = 2,\n col = col,\n pch = 16,\n pch_NA = 17\n)\n\nxgb.plot.shap(\n x,\n model = model_multiclass,\n trees = trees0 + 2,\n target_class = 2,\n top_n = 4,\n n_col = 2,\n col = col,\n pch = 16,\n pch_NA = 17\n)\n\n# Summary plot\nxgb.ggplot.shap.summary(x, model = model_multiclass, target_class = 0, top_n = 4)\n\n}\n\\references{\n\\enumerate{\n\\item Scott M. Lundberg, Su-In Lee, \"A Unified Approach to Interpreting Model Predictions\",\nNIPS Proceedings 2017, \\url{https://arxiv.org/abs/1705.07874}\n\\item Scott M. Lundberg, Su-In Lee, \"Consistent feature attribution for tree ensembles\",\n\\url{https://arxiv.org/abs/1706.06060}\n}\n}\n" }, { "path": "R-package/man/xgb.plot.shap.summary.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/xgb.ggplot.R, R/xgb.plot.shap.R\n\\name{xgb.ggplot.shap.summary}\n\\alias{xgb.ggplot.shap.summary}\n\\alias{xgb.plot.shap.summary}\n\\title{SHAP summary plot}\n\\usage{\nxgb.ggplot.shap.summary(\n data,\n shap_contrib = NULL,\n features = NULL,\n top_n = 10,\n model = NULL,\n trees = NULL,\n target_class = NULL,\n approxcontrib = FALSE,\n subsample = NULL\n)\n\nxgb.plot.shap.summary(\n data,\n shap_contrib = NULL,\n features = NULL,\n top_n = 10,\n model = NULL,\n trees = NULL,\n target_class = NULL,\n approxcontrib = FALSE,\n subsample = NULL\n)\n}\n\\arguments{\n\\item{data}{The data to explain as a \\code{matrix}, \\code{dgCMatrix}, or \\code{data.frame}.}\n\n\\item{shap_contrib}{Matrix of SHAP contributions of \\code{data}.\nThe default (\\code{NULL}) computes it from \\code{model} and \\code{data}.}\n\n\\item{features}{Vector of column indices or feature names to plot. When \\code{NULL}\n(default), the \\code{top_n} most important features are selected by \\code{\\link[=xgb.importance]{xgb.importance()}}.}\n\n\\item{top_n}{How many of the most important features (<= 100) should be selected?\nBy default 1 for SHAP dependence and 10 for SHAP summary.\nOnly used when \\code{features = NULL}.}\n\n\\item{model}{An \\code{xgb.Booster} model. Only required when \\code{shap_contrib = NULL} or\n\\code{features = NULL}.}\n\n\\item{trees}{Passed to \\code{\\link[=xgb.importance]{xgb.importance()}} when \\code{features = NULL}.}\n\n\\item{target_class}{Only relevant for multiclass models. The default (\\code{NULL})\naverages the SHAP values over all classes. Pass a (0-based) class index\nto show only SHAP values of that class.}\n\n\\item{approxcontrib}{Passed to \\code{\\link[=predict.xgb.Booster]{predict.xgb.Booster()}} when \\code{shap_contrib = NULL}.}\n\n\\item{subsample}{Fraction of data points randomly picked for plotting.\nThe default (\\code{NULL}) will use up to 100k data points.}\n}\n\\value{\nA \\code{ggplot2} object.\n}\n\\description{\nVisualizes SHAP contributions of different features.\n}\n\\details{\nA point plot (each point representing one observation from \\code{data}) is\nproduced for each feature, with the points plotted on the SHAP value axis.\nEach point (observation) is coloured based on its feature value.\n\nThe plot allows to see which features have a negative / positive contribution\non the model prediction, and whether the contribution is different for larger\nor smaller values of the feature. Inspired by the summary plot of\n\\url{https://github.com/shap/shap}.\n}\n\\examples{\n# See examples in xgb.plot.shap()\n\n}\n\\seealso{\n\\code{\\link[=xgb.plot.shap]{xgb.plot.shap()}}, \\code{\\link[=xgb.ggplot.shap.summary]{xgb.ggplot.shap.summary()}},\nand the Python library \\url{https://github.com/shap/shap}.\n}\n" }, { "path": "R-package/man/xgb.plot.tree.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/xgb.plot.tree.R\n\\name{xgb.plot.tree}\n\\alias{xgb.plot.tree}\n\\title{Plot boosted trees}\n\\usage{\nxgb.plot.tree(\n model,\n tree_idx = 1,\n plot_width = NULL,\n plot_height = NULL,\n with_stats = FALSE,\n ...\n)\n}\n\\arguments{\n\\item{model}{Object of class \\code{xgb.Booster}. If it contains feature names\n(they can be set through \\code{\\link[=setinfo]{setinfo()}}, they will be used in the\noutput from this function.}\n\n\\item{tree_idx}{An integer of the tree index that should be used. This\nis an 1-based index.}\n\n\\item{plot_width, plot_height}{Width and height of the graph in pixels.\nThe values are passed to \\code{DiagrammeR::render_graph()}.}\n\n\\item{with_stats}{Whether to dump some additional statistics about the\nsplits. When this option is on, the model dump contains two additional\nvalues: gain is the approximate loss function gain we get in each split;\ncover is the sum of second order gradient in each node.}\n\n\\item{...}{Not used.\n\nSome arguments that were part of this function in previous XGBoost versions are currently\ndeprecated or have been renamed. If a deprecated or renamed argument is passed, will throw\na warning (by default) and use its current equivalent instead. This warning will become an\nerror if using the \\link[=xgboost-options]{'strict mode' option}.\n\nIf some additional argument is passed that is neither a current function argument nor\na deprecated or renamed argument, a warning or error will be thrown depending on the\n'strict mode' option.\n\n\\bold{Important:} \\code{...} will be removed in a future version, and all the current\ndeprecation warnings will become errors. Please use only arguments that form part of\nthe function signature.}\n}\n\\value{\nRendered graph object which is an htmlwidget of ' class \\code{grViz}. Similar to\n\"ggplot\" objects, it needs to be printed when not running from the command\nline.\n}\n\\description{\nRead a tree model text dump and plot the model.\n}\n\\details{\nThe content of each node is visualized as follows:\n\\itemize{\n\\item For non-terminal nodes, it will display the split condition (number or name\nif available, and the condition that would decide to which node to go\nnext).\n\\item Those nodes will be connected to their children by arrows that indicate\nwhether the branch corresponds to the condition being met or not being met.\n\\item Terminal (leaf) nodes contain the margin to add when ending there.\n}\n\nThe \"Yes\" branches are marked by the \"< split_value\" label.\nThe branches also used for missing values are marked as bold\n(as in \"carrying extra capacity\").\n\nThis function uses \\href{https://www.graphviz.org/}{GraphViz} as DiagrammeR\nbackend.\n}\n\\examples{\ndata(\"ToothGrowth\")\nx <- ToothGrowth[, c(\"len\", \"dose\")]\ny <- ToothGrowth$supp\nmodel <- xgboost(\n x, y,\n nthreads = 1L,\n nrounds = 3L,\n max_depth = 3L\n)\n\n# plot the first tree\nxgb.plot.tree(model, tree_idx = 1)\n\n# Below is an example of how to save this plot to a file.\nif (require(\"DiagrammeR\") && require(\"htmlwidgets\")) {\n fname <- file.path(tempdir(), \"plot.html'\")\n gr <- xgb.plot.tree(model, tree_idx = 1)\n htmlwidgets::saveWidget(gr, fname)\n}\n}\n" }, { "path": "R-package/man/xgb.save.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/xgb.save.R\n\\name{xgb.save}\n\\alias{xgb.save}\n\\title{Save XGBoost model to binary file}\n\\usage{\nxgb.save(model, fname)\n}\n\\arguments{\n\\item{model}{Model object of \\code{xgb.Booster} class.}\n\n\\item{fname}{Name of the file to write. Its extension determines the serialization format:\n\\itemize{\n\\item \".ubj\": Use the universal binary JSON format (recommended).\nThis format uses binary types for e.g. floating point numbers, thereby preventing any loss\nof precision when converting to a human-readable JSON text or similar.\n\\item \".json\": Use plain JSON, which is a human-readable format.\n\\item \".deprecated\": Use \\strong{deprecated} binary format. This format will\nnot be able to save attributes introduced after v1 of XGBoost, such as the \"best_iteration\"\nattribute that boosters might keep, nor feature names or user-specifiec attributes.\n\\item If the format is not specified by passing one of the file extensions above, will\ndefault to UBJ.\n}}\n}\n\\description{\nSave XGBoost model to a file in binary or JSON format.\n}\n\\details{\nThis methods allows to save a model in an XGBoost-internal binary or text format which is universal\namong the various xgboost interfaces. In R, the saved model file could be read later\nusing either the \\code{\\link[=xgb.load]{xgb.load()}} function or the \\code{xgb_model} parameter of \\code{\\link[=xgb.train]{xgb.train()}}.\n\nNote: a model can also be saved as an R object (e.g., by using \\code{\\link[=readRDS]{readRDS()}}\nor \\code{\\link[=save]{save()}}). However, it would then only be compatible with R, and\ncorresponding R methods would need to be used to load it. Moreover, persisting the model with\n\\code{\\link[=readRDS]{readRDS()}} or \\code{\\link[=save]{save()}} might cause compatibility problems in\nfuture versions of XGBoost. Consult \\link{a-compatibility-note-for-saveRDS-save} to learn\nhow to persist models in a future-proof way, i.e., to make the model accessible in future\nreleases of XGBoost.\n}\n\\examples{\n\\dontshow{RhpcBLASctl::omp_set_num_threads(1)}\ndata(agaricus.train, package = \"xgboost\")\ndata(agaricus.test, package = \"xgboost\")\n\n## Keep the number of threads to 1 for examples\nnthread <- 1\ndata.table::setDTthreads(nthread)\n\ntrain <- agaricus.train\ntest <- agaricus.test\n\nbst <- xgb.train(\n data = xgb.DMatrix(train$data, label = train$label, nthread = 1),\n nrounds = 2,\n params = xgb.params(\n max_depth = 2,\n nthread = nthread,\n objective = \"binary:logistic\"\n )\n)\n\nfname <- file.path(tempdir(), \"xgb.ubj\")\nxgb.save(bst, fname)\nbst <- xgb.load(fname)\n}\n\\seealso{\n\\code{\\link[=xgb.load]{xgb.load()}}\n}\n" }, { "path": "R-package/man/xgb.save.raw.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/xgb.save.raw.R\n\\name{xgb.save.raw}\n\\alias{xgb.save.raw}\n\\title{Save XGBoost model to R's raw vector}\n\\usage{\nxgb.save.raw(model, raw_format = \"ubj\")\n}\n\\arguments{\n\\item{model}{The model object.}\n\n\\item{raw_format}{The format for encoding the booster:\n\\itemize{\n\\item \"json\": Encode the booster into JSON text document.\n\\item \"ubj\": Encode the booster into Universal Binary JSON.\n\\item \"deprecated\": Encode the booster into old customized binary format.\n}}\n}\n\\description{\nSave XGBoost model from \\code{\\link[=xgboost]{xgboost()}} or \\code{\\link[=xgb.train]{xgb.train()}}.\nCall \\code{\\link[=xgb.load.raw]{xgb.load.raw()}} to load the model back from raw vector.\n}\n\\examples{\n\\dontshow{RhpcBLASctl::omp_set_num_threads(1)}\ndata(agaricus.train, package = \"xgboost\")\ndata(agaricus.test, package = \"xgboost\")\n\n## Keep the number of threads to 1 for examples\nnthread <- 1\ndata.table::setDTthreads(nthread)\n\ntrain <- agaricus.train\ntest <- agaricus.test\n\nbst <- xgb.train(\n data = xgb.DMatrix(train$data, label = train$label, nthread = 1),\n nrounds = 2,\n params = xgb.params(\n max_depth = 2,\n nthread = nthread,\n objective = \"binary:logistic\"\n )\n)\n\nraw <- xgb.save.raw(bst)\nbst <- xgb.load.raw(raw)\n\n}\n" }, { "path": "R-package/man/xgb.slice.Booster.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/xgb.Booster.R\n\\name{xgb.slice.Booster}\n\\alias{xgb.slice.Booster}\n\\alias{[.xgb.Booster}\n\\title{Slice Booster by Rounds}\n\\usage{\nxgb.slice.Booster(\n model,\n start,\n end = xgb.get.num.boosted.rounds(model),\n step = 1L\n)\n\n\\method{[}{xgb.Booster}(x, i)\n}\n\\arguments{\n\\item{model, x}{A fitted \\code{xgb.Booster} object, which is to be sliced by taking only a subset\nof its rounds / iterations.}\n\n\\item{start}{Start of the slice (base-1 and inclusive, like R's \\code{\\link[=seq]{seq()}}).}\n\n\\item{end}{End of the slice (base-1 and inclusive, like R's \\code{\\link[=seq]{seq()}}).\nPassing a value of zero here is equivalent to passing the full number of rounds in the\nbooster object.}\n\n\\item{step}{Step size of the slice. Passing '1' will take every round in the sequence defined by\n\\verb{(start, end)}, while passing '2' will take every second value, and so on.}\n\n\\item{i}{The indices - must be an increasing sequence as generated by e.g. \\code{seq(...)}.}\n}\n\\value{\nA sliced booster object containing only the requested rounds.\n}\n\\description{\nCreates a new booster including only a selected range of rounds / iterations\nfrom an existing booster, as given by the sequence \\code{seq(start, end, step)}.\n}\n\\details{\nNote that any R attributes that the booster might have, will not be copied into\nthe resulting object.\n}\n\\examples{\ndata(mtcars)\n\ny <- mtcars$mpg\nx <- as.matrix(mtcars[, -1])\n\ndm <- xgb.DMatrix(x, label = y, nthread = 1)\nmodel <- xgb.train(data = dm, params = xgb.params(nthread = 1), nrounds = 5)\nmodel_slice <- xgb.slice.Booster(model, 1, 3)\n# Prediction for first three rounds\npredict(model, x, predleaf = TRUE)[, 1:3]\n\n# The new model has only those rounds, so\n# a full prediction from it is equivalent\npredict(model_slice, x, predleaf = TRUE)\n}\n" }, { "path": "R-package/man/xgb.slice.DMatrix.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/xgb.DMatrix.R\n\\name{xgb.slice.DMatrix}\n\\alias{xgb.slice.DMatrix}\n\\alias{[.xgb.DMatrix}\n\\title{Slice DMatrix}\n\\usage{\nxgb.slice.DMatrix(object, idxset, allow_groups = FALSE)\n\n\\method{[}{xgb.DMatrix}(object, idxset, colset = NULL)\n}\n\\arguments{\n\\item{object}{Object of class \\code{xgb.DMatrix}.}\n\n\\item{idxset}{An integer vector of indices of rows needed (base-1 indexing).}\n\n\\item{allow_groups}{Whether to allow slicing an \\code{xgb.DMatrix} with \\code{group} (or\nequivalently \\code{qid}) field. Note that in such case, the result will not have\nthe groups anymore - they need to be set manually through \\code{\\link[=setinfo]{setinfo()}}.}\n\n\\item{colset}{Currently not used (columns subsetting is not available).}\n}\n\\description{\nGet a new DMatrix containing the specified rows of original xgb.DMatrix object.\n}\n\\examples{\ndata(agaricus.train, package = \"xgboost\")\n\ndtrain <- with(agaricus.train, xgb.DMatrix(data, label = label, nthread = 2))\n\ndsub <- xgb.slice.DMatrix(dtrain, 1:42)\nlabels1 <- getinfo(dsub, \"label\")\n\ndsub <- dtrain[1:42, ]\nlabels2 <- getinfo(dsub, \"label\")\nall.equal(labels1, labels2)\n\n}\n" }, { "path": "R-package/man/xgb.train.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/xgb.train.R\n\\name{xgb.train}\n\\alias{xgb.train}\n\\title{Fit XGBoost Model}\n\\usage{\nxgb.train(\n params = xgb.params(),\n data,\n nrounds,\n evals = list(),\n objective = NULL,\n custom_metric = NULL,\n verbose = 1,\n print_every_n = 1L,\n early_stopping_rounds = NULL,\n maximize = NULL,\n save_period = NULL,\n save_name = \"xgboost.model\",\n xgb_model = NULL,\n callbacks = list(),\n ...\n)\n}\n\\arguments{\n\\item{params}{List of XGBoost parameters which control the model building process.\nSee the \\href{https://xgboost.readthedocs.io/en/latest/parameter.html}{online documentation}\nand the documentation for \\code{\\link[=xgb.params]{xgb.params()}} for details.\n\nShould be passed as list with named entries. Parameters that are not specified in this\nlist will use their default values.\n\nA list of named parameters can be created through the function \\code{\\link[=xgb.params]{xgb.params()}}, which\naccepts all valid parameters as function arguments.}\n\n\\item{data}{Training dataset. \\code{xgb.train()} accepts only an \\code{xgb.DMatrix} as the input.\n\nNote that there is a function \\code{\\link[=xgboost]{xgboost()}} which is meant to accept R data objects\nas inputs, such as data frames and matrices.}\n\n\\item{nrounds}{Max number of boosting iterations.}\n\n\\item{evals}{Named list of \\code{xgb.DMatrix} datasets to use for evaluating model performance.\nMetrics specified in either \\code{eval_metric} (under params) or \\code{custom_metric} (function\nargument here) will be computed for each of these datasets during each boosting iteration,\nand stored in the end as a field named \\code{evaluation_log} in the resulting object.\n\nWhen either \\code{verbose>=1} or \\code{\\link[=xgb.cb.print.evaluation]{xgb.cb.print.evaluation()}} callback is engaged, the performance\nresults are continuously printed out during the training.\n\nE.g., specifying \\code{evals=list(validation1=mat1, validation2=mat2)} allows to track\nthe performance of each round's model on \\code{mat1} and \\code{mat2}.}\n\n\\item{objective}{Customized objective function. Should take two arguments: the first one will be the\ncurrent predictions (either a numeric vector or matrix depending on the number of targets / classes),\nand the second one will be the \\code{data} DMatrix object that is used for training.\n\nIt should return a list with two elements \\code{grad} and \\code{hess} (in that order), as either\nnumeric vectors or numeric matrices depending on the number of targets / classes (same\ndimension as the predictions that are passed as first argument).}\n\n\\item{custom_metric}{Customized evaluation function. Just like \\code{objective}, should take two arguments,\nwith the first one being the predictions and the second one the \\code{data} DMatrix.\n\nShould return a list with two elements \\code{metric} (name that will be displayed for this metric,\nshould be a string / character), and \\code{value} (the number that the function calculates, should\nbe a numeric scalar).\n\nNote that even if passing \\code{custom_metric}, objectives also have an associated default metric that\nwill be evaluated in addition to it. In order to disable the built-in metric, one can pass\nparameter \\code{disable_default_eval_metric = TRUE}.}\n\n\\item{verbose}{If 0, xgboost will stay silent. If 1, it will print information about performance.\nIf 2, some additional information will be printed out.\nNote that setting \\code{verbose > 0} automatically engages the\n\\code{xgb.cb.print.evaluation(period=1)} callback function.}\n\n\\item{print_every_n}{When passing \\code{verbose>0}, evaluation logs (metrics calculated on the\ndata passed under \\code{evals}) will be printed every nth iteration according to the value passed\nhere. The first and last iteration are always included regardless of this 'n'.\n\nOnly has an effect when passing data under \\code{evals} and when passing \\code{verbose>0}. The parameter\nis passed to the \\code{\\link[=xgb.cb.print.evaluation]{xgb.cb.print.evaluation()}} callback.}\n\n\\item{early_stopping_rounds}{Number of boosting rounds after which training will be stopped\nif there is no improvement in performance (as measured by the evaluatiation metric that is\nsupplied or selected by default for the objective) on the evaluation data passed under\n\\code{evals}.\n\nMust pass \\code{evals} in order to use this functionality. Setting this parameter adds the\n\\code{\\link[=xgb.cb.early.stop]{xgb.cb.early.stop()}} callback.\n\nIf \\code{NULL}, early stopping will not be used.}\n\n\\item{maximize}{If \\code{feval} and \\code{early_stopping_rounds} are set, then this parameter must be set as well.\nWhen it is \\code{TRUE}, it means the larger the evaluation score the better.\nThis parameter is passed to the \\code{\\link[=xgb.cb.early.stop]{xgb.cb.early.stop()}} callback.}\n\n\\item{save_period}{When not \\code{NULL}, model is saved to disk after every \\code{save_period} rounds.\n0 means save at the end. The saving is handled by the \\code{\\link[=xgb.cb.save.model]{xgb.cb.save.model()}} callback.}\n\n\\item{save_name}{the name or path for periodically saved model file.}\n\n\\item{xgb_model}{A previously built model to continue the training from.\nCould be either an object of class \\code{xgb.Booster}, or its raw data, or the name of a\nfile with a previously saved model.}\n\n\\item{callbacks}{A list of callback functions to perform various task during boosting.\nSee \\code{\\link[=xgb.Callback]{xgb.Callback()}}. Some of the callbacks are automatically created depending on the\nparameters' values. User can provide either existing or their own callback methods in order\nto customize the training process.\n\nNote that some callbacks might try to leave attributes in the resulting model object,\nsuch as an evaluation log (a \\code{data.table} object) - be aware that these objects are kept\nas R attributes, and thus do not get saved when using XGBoost's own serializaters like\n\\code{\\link[=xgb.save]{xgb.save()}} (but are kept when using R serializers like \\code{\\link[=saveRDS]{saveRDS()}}).}\n\n\\item{...}{Not used.\n\nSome arguments that were part of this function in previous XGBoost versions are currently\ndeprecated or have been renamed. If a deprecated or renamed argument is passed, will throw\na warning (by default) and use its current equivalent instead. This warning will become an\nerror if using the \\link[=xgboost-options]{'strict mode' option}.\n\nIf some additional argument is passed that is neither a current function argument nor\na deprecated or renamed argument, a warning or error will be thrown depending on the\n'strict mode' option.\n\n\\bold{Important:} \\code{...} will be removed in a future version, and all the current\ndeprecation warnings will become errors. Please use only arguments that form part of\nthe function signature.}\n}\n\\value{\nAn object of class \\code{xgb.Booster}.\n}\n\\description{\nFits an XGBoost model to given data in DMatrix format (e.g. as produced by \\code{\\link[=xgb.DMatrix]{xgb.DMatrix()}}).\nSee the tutorial \\href{https://xgboost.readthedocs.io/en/stable/tutorials/model.html}{Introduction to Boosted Trees}\nfor a longer explanation of what XGBoost does, and the rest of the\n\\href{https://xgboost.readthedocs.io/en/latest/tutorials/index.html}{XGBoost Tutorials} for further\nexplanations XGBoost's features and usage.\n\nCompared to function \\code{\\link[=xgboost]{xgboost()}} which is a user-friendly function targeted towards interactive\nusage, \\code{xgb.train} is a lower-level interface which allows finer-grained control and exposes\nfurther functionalities offered by the core library (such as learning-to-rank objectives), but\nwhich works exclusively with XGBoost's own data format (\"DMatrices\") instead of with regular R\nobjects.\n\nThe syntax of this function closely mimics the same function from the Python package for XGBoost,\nand is recommended to use for package developers over \\code{xgboost()} as it will provide a more\nstable interface (with fewer breaking changes) and lower overhead from data validations.\n\nSee also the \\href{https://xgboost.readthedocs.io/en/latest/R-package/migration_guide.html}{migration guide}\nif coming from a previous version of XGBoost in the 1.x series.\n}\n\\details{\nCompared to \\code{\\link[=xgboost]{xgboost()}}, the \\code{xgb.train()} interface supports advanced features such as\n\\code{evals}, customized objective and evaluation metric functions, among others, with the\ndifference these work \\code{xgb.DMatrix} objects and do not follow typical R idioms.\n\nParallelization is automatically enabled if OpenMP is present.\nNumber of threads can also be manually specified via the \\code{nthread} parameter.\n\nWhile in XGBoost language bindings, the default random seed defaults to zero, in R, if a parameter \\code{seed}\nis not manually supplied, it will generate a random seed through R's own random number generator,\nwhose seed in turn is controllable through \\code{set.seed}. If \\code{seed} is passed, it will override the\nRNG from R.\n\nThe following callbacks are automatically created when certain parameters are set:\n\\itemize{\n\\item \\code{\\link[=xgb.cb.print.evaluation]{xgb.cb.print.evaluation()}} is turned on when \\code{verbose > 0} and the \\code{print_every_n}\nparameter is passed to it.\n\\item \\code{\\link[=xgb.cb.evaluation.log]{xgb.cb.evaluation.log()}} is on when \\code{evals} is present.\n\\item \\code{\\link[=xgb.cb.early.stop]{xgb.cb.early.stop()}}: When \\code{early_stopping_rounds} is set.\n\\item \\code{\\link[=xgb.cb.save.model]{xgb.cb.save.model()}}: When \\code{save_period > 0} is set.\n}\n\nNote that objects of type \\code{xgb.Booster} as returned by this function behave a bit differently\nfrom typical R objects (it's an 'altrep' list class), and it makes a separation between\ninternal booster attributes (restricted to jsonifyable data), accessed through \\code{\\link[=xgb.attr]{xgb.attr()}}\nand shared between interfaces through serialization functions like \\code{\\link[=xgb.save]{xgb.save()}}; and\nR-specific attributes (typically the result from a callback), accessed through \\code{\\link[=attributes]{attributes()}}\nand \\code{\\link[=attr]{attr()}}, which are otherwise\nonly used in the R interface, only kept when using R's serializers like \\code{\\link[=saveRDS]{saveRDS()}}, and\nnot anyhow used by functions like \\code{predict.xgb.Booster()}.\n\nBe aware that one such R attribute that is automatically added is \\code{params} - this attribute\nis assigned from the \\code{params} argument to this function, and is only meant to serve as a\nreference for what went into the booster, but is not used in other methods that take a booster\nobject - so for example, changing the booster's configuration requires calling \\verb{xgb.config<-}\nor \\verb{xgb.model.parameters<-}, while simply modifying \\verb{attributes(model)$params$<...>} will have no\neffect elsewhere.\n}\n\\examples{\ndata(agaricus.train, package = \"xgboost\")\ndata(agaricus.test, package = \"xgboost\")\n\n## Keep the number of threads to 1 for examples\nnthread <- 1\ndata.table::setDTthreads(nthread)\n\ndtrain <- with(\n agaricus.train, xgb.DMatrix(data, label = label, nthread = nthread)\n)\ndtest <- with(\n agaricus.test, xgb.DMatrix(data, label = label, nthread = nthread)\n)\nevals <- list(train = dtrain, eval = dtest)\n\n## A simple xgb.train example:\nparam <- xgb.params(\n max_depth = 2,\n nthread = nthread,\n objective = \"binary:logistic\",\n eval_metric = \"auc\"\n)\nbst <- xgb.train(param, dtrain, nrounds = 2, evals = evals, verbose = 0)\n\n## An xgb.train example where custom objective and evaluation metric are\n## used:\nlogregobj <- function(preds, dtrain) {\n labels <- getinfo(dtrain, \"label\")\n preds <- 1/(1 + exp(-preds))\n grad <- preds - labels\n hess <- preds * (1 - preds)\n return(list(grad = grad, hess = hess))\n}\nevalerror <- function(preds, dtrain) {\n labels <- getinfo(dtrain, \"label\")\n err <- as.numeric(sum(labels != (preds > 0)))/length(labels)\n return(list(metric = \"error\", value = err))\n}\n\n# These functions could be used by passing them as 'objective' and\n# 'eval_metric' parameters in the params list:\nparam <- xgb.params(\n max_depth = 2,\n nthread = nthread,\n objective = logregobj,\n eval_metric = evalerror\n)\nbst <- xgb.train(param, dtrain, nrounds = 2, evals = evals, verbose = 0)\n\n# ... or as dedicated 'objective' and 'custom_metric' parameters of xgb.train:\nbst <- xgb.train(\n within(param, rm(\"objective\", \"eval_metric\")),\n dtrain, nrounds = 2, evals = evals,\n objective = logregobj, custom_metric = evalerror\n)\n\n\n## An xgb.train example of using variable learning rates at each iteration:\nparam <- xgb.params(\n max_depth = 2,\n learning_rate = 1,\n nthread = nthread,\n objective = \"binary:logistic\",\n eval_metric = \"auc\"\n)\nmy_learning_rates <- list(learning_rate = c(0.5, 0.1))\n\nbst <- xgb.train(\n param,\n dtrain,\n nrounds = 2,\n evals = evals,\n verbose = 0,\n callbacks = list(xgb.cb.reset.parameters(my_learning_rates))\n)\n\n## Early stopping:\nbst <- xgb.train(\n param, dtrain, nrounds = 25, evals = evals, early_stopping_rounds = 3\n)\n}\n\\references{\nTianqi Chen and Carlos Guestrin, \"XGBoost: A Scalable Tree Boosting System\",\n22nd SIGKDD Conference on Knowledge Discovery and Data Mining, 2016, \\url{https://arxiv.org/abs/1603.02754}\n}\n\\seealso{\n\\code{\\link[=xgb.Callback]{xgb.Callback()}}, \\code{\\link[=predict.xgb.Booster]{predict.xgb.Booster()}}, \\code{\\link[=xgb.cv]{xgb.cv()}}\n}\n" }, { "path": "R-package/man/xgbConfig.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/xgb.config.R\n\\name{xgb.set.config, xgb.get.config}\n\\alias{xgb.set.config, xgb.get.config}\n\\alias{xgb.set.config}\n\\alias{xgb.get.config}\n\\title{Set and get global configuration}\n\\usage{\nxgb.set.config(...)\n\nxgb.get.config()\n}\n\\arguments{\n\\item{...}{List of parameters to be set, as keyword arguments}\n}\n\\value{\n\\code{xgb.set.config()} returns \\code{TRUE} to signal success. \\code{xgb.get.config()} returns\na list containing all global-scope parameters and their values.\n}\n\\description{\nGlobal configuration consists of a collection of parameters that can be applied in the global\nscope. See \\url{https://xgboost.readthedocs.io/en/stable/parameter.html} for the full list of\nparameters supported in the global configuration. Use \\code{xgb.set.config()} to update the\nvalues of one or more global-scope parameters. Use \\code{xgb.get.config()} to fetch the current\nvalues of all global-scope parameters (listed in\n\\url{https://xgboost.readthedocs.io/en/stable/parameter.html}).\n}\n\\details{\nNote that serialization-related functions might use a globally-configured number of threads,\nwhich is managed by the system's OpenMP (OMP) configuration instead. Typically, XGBoost methods\naccept an \\code{nthreads} parameter, but some methods like \\code{\\link[=readRDS]{readRDS()}} might get executed before such\nparameter can be supplied.\n\nThe number of OMP threads can in turn be configured for example through an environment variable\n\\code{OMP_NUM_THREADS} (needs to be set before R is started), or through \\code{RhpcBLASctl::omp_set_num_threads}.\n}\n\\examples{\n# Set verbosity level to silent (0)\nxgb.set.config(verbosity = 0)\n# Now global verbosity level is 0\nconfig <- xgb.get.config()\nprint(config$verbosity)\n# Set verbosity level to warning (1)\nxgb.set.config(verbosity = 1)\n# Now global verbosity level is 1\nconfig <- xgb.get.config()\nprint(config$verbosity)\n}\n" }, { "path": "R-package/man/xgboost-options.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/utils.R\n\\name{xgboost-options}\n\\alias{xgboost-options}\n\\title{XGBoost Options}\n\\description{\nXGBoost offers an \\link[base:options]{option setting} for controlling the behavior\nof deprecated and removed function arguments.\n\nSome of the arguments in functions like \\code{\\link[=xgb.train]{xgb.train()}} or \\code{\\link[=predict.xgb.Booster]{predict.xgb.Booster()}} been renamed\nfrom how they were in previous versions, or have been removed.\n\nIn order to make the transition to newer XGBoost versions easier, some of these parameters are\nstill accepted but issue a warning when using them. \\bold{Note that these warnings will become\nerrors in the future!!} - this is just a temporary workaround to make the transition easier.\n\nOne can optionally use 'strict mode' to turn these warnings into errors, in order to ensure\nthat code calling xgboost will still work once those are removed in future releases.\n\nCurrently, the only supported option is \\code{xgboost.strict_mode}, which can be set to \\code{TRUE} or\n\\code{FALSE} (default).\n\nIn addition to an R option, it can also be enabled through by setting environment variable\n\\code{XGB_STRICT_MODE=1}. If set, this environment variable will take precedence over the option.\n}\n\\examples{\noptions(\"xgboost.strict_mode\" = FALSE)\noptions(\"xgboost.strict_mode\" = TRUE)\nSys.setenv(\"XGB_STRICT_MODE\" = \"1\")\nSys.setenv(\"XGB_STRICT_MODE\" = \"0\")\n}\n" }, { "path": "R-package/man/xgboost.Rd", "content": "% Generated by roxygen2: do not edit by hand\n% Please edit documentation in R/xgboost.R\n\\name{xgboost}\n\\alias{xgboost}\n\\title{Fit XGBoost Model}\n\\usage{\nxgboost(\n x,\n y,\n objective = NULL,\n nrounds = 100L,\n max_depth = NULL,\n learning_rate = NULL,\n min_child_weight = NULL,\n min_split_loss = NULL,\n reg_lambda = NULL,\n weights = NULL,\n verbosity = if (is.null(eval_set)) 0L else 1L,\n monitor_training = verbosity > 0,\n eval_set = NULL,\n early_stopping_rounds = NULL,\n print_every_n = 1L,\n eval_metric = NULL,\n nthreads = parallel::detectCores(),\n seed = 0L,\n base_margin = NULL,\n monotone_constraints = NULL,\n interaction_constraints = NULL,\n reg_alpha = NULL,\n max_bin = NULL,\n max_leaves = NULL,\n booster = NULL,\n subsample = NULL,\n sampling_method = NULL,\n feature_weights = NULL,\n colsample_bytree = NULL,\n colsample_bylevel = NULL,\n colsample_bynode = NULL,\n tree_method = NULL,\n max_delta_step = NULL,\n scale_pos_weight = NULL,\n updater = NULL,\n grow_policy = NULL,\n num_parallel_tree = NULL,\n multi_strategy = NULL,\n base_score = NULL,\n seed_per_iteration = NULL,\n device = NULL,\n disable_default_eval_metric = NULL,\n use_rmm = NULL,\n max_cached_hist_node = NULL,\n max_cat_to_onehot = NULL,\n max_cat_threshold = NULL,\n sample_type = NULL,\n normalize_type = NULL,\n rate_drop = NULL,\n one_drop = NULL,\n skip_drop = NULL,\n feature_selector = NULL,\n top_k = NULL,\n tweedie_variance_power = NULL,\n huber_slope = NULL,\n quantile_alpha = NULL,\n expectile_alpha = NULL,\n aft_loss_distribution = NULL,\n ...\n)\n}\n\\arguments{\n\\item{x}{The features / covariates. Can be passed as:\n\\itemize{\n\\item A numeric or integer \\code{matrix}.\n\\item A \\code{data.frame}, in which all columns are one of the following types:\n\\itemize{\n\\item \\code{numeric}\n\\item \\code{integer}\n\\item \\code{logical}\n\\item \\code{factor}\n}\n\nColumns of \\code{factor} type will be assumed to be categorical, while other column types will\nbe assumed to be numeric.\n\\item A sparse matrix from the \\code{Matrix} package, either as \\code{dgCMatrix} or \\code{dgRMatrix} class.\n}\n\nNote that categorical features are only supported for \\code{data.frame} inputs, and are automatically\ndetermined based on their types. See \\code{\\link[=xgb.train]{xgb.train()}} with \\code{\\link[=xgb.DMatrix]{xgb.DMatrix()}} for more flexible\nvariants that would allow something like categorical features on sparse matrices.}\n\n\\item{y}{The response variable. Allowed values are:\n\\itemize{\n\\item A numeric or integer vector (for regression tasks).\n\\item A factor or character vector (for binary and multi-class classification tasks).\n\\item A logical (boolean) vector (for binary classification tasks).\n\\item A numeric or integer matrix or \\code{data.frame} with numeric/integer columns\n(for multi-task regression tasks).\n\\item A \\code{Surv} object from the 'survival' package (for survival tasks).\n}\n\nIf \\code{objective} is \\code{NULL}, the right task will be determined automatically based on\nthe class of \\code{y}.\n\nIf \\code{objective} is not \\code{NULL}, it must match with the type of \\code{y} - e.g. \\code{factor} types of \\code{y}\ncan only be used with classification objectives and vice-versa.\n\nFor binary classification, the last factor level of \\code{y} will be used as the \"positive\"\nclass - that is, the numbers from \\code{predict} will reflect the probabilities of belonging to this\nclass instead of to the first factor level. If \\code{y} is a \\code{logical} vector, then \\code{TRUE} will be\nset as the last level.}\n\n\\item{objective}{Optimization objective to minimize based on the supplied data, to be passed\nby name as a string / character (e.g. \\code{reg:absoluteerror}). See the\n\\href{https://xgboost.readthedocs.io/en/stable/parameter.html#learning-task-parameters}{Learning Task Parameters}\npage and the \\code{\\link[=xgb.params]{xgb.params()}} documentation for more detailed information on allowed values.\n\nIf \\code{NULL} (the default), will be automatically determined from \\code{y} according to the following\nlogic:\n\\itemize{\n\\item If \\code{y} is a factor with 2 levels, will use \\code{binary:logistic}.\n\\item If \\code{y} is a factor with more than 2 levels, will use \\code{multi:softprob} (number of classes\nwill be determined automatically, should not be passed under \\code{params}).\n\\item If \\code{y} is a \\code{Surv} object from the \\code{survival} package, will use \\code{survival:aft} (note that\nthe only types supported are left / right / interval censored).\n\\item Otherwise, will use \\code{reg:squarederror}.\n}\n\nIf \\code{objective} is not \\code{NULL}, it must match with the type of \\code{y} - e.g. \\code{factor} types of \\code{y}\ncan only be used with classification objectives and vice-versa.\n\nNote that not all possible \\code{objective} values supported by the core XGBoost library are allowed\nhere - for example, objectives which are a variation of another but with a different default\nprediction type (e.g. \\code{multi:softmax} vs. \\code{multi:softprob}) are not allowed, and neither are\nranking objectives, nor custom objectives at the moment.\n\nSupported values are:\n\\itemize{\n\\item \\code{\"reg:squarederror\"}: regression with squared loss.\n\\item \\code{\"reg:squaredlogerror\"}: regression with squared log loss \\eqn{\\frac{1}{2}[log(pred + 1) - log(label + 1)]^2}. All input labels are required to be greater than -1. Also, see metric \\code{rmsle} for possible issue with this objective.\n\\item \\code{\"reg:pseudohubererror\"}: regression with Pseudo Huber loss, a twice differentiable alternative to absolute loss.\n\\item \\code{\"reg:absoluteerror\"}: Regression with L1 error. When tree model is used, leaf value is refreshed after tree construction. If used in distributed training, the leaf value is calculated as the mean value from all workers, which is not guaranteed to be optimal.\n\\item \\code{\"reg:quantileerror\"}: Quantile loss, also known as \"pinball loss\". See later sections for its parameter and \\href{https://xgboost.readthedocs.io/en/latest/python/examples/quantile_regression.html#sphx-glr-python-examples-quantile-regression-py}{Quantile Regression} for a worked example.\n\\item \\code{\"reg:expectileerror\"}: Expectile loss. See later sections for its parameter.\n\\item \\code{\"binary:logistic\"}: logistic regression for binary classification, output probability\n\\item \\code{\"binary:hinge\"}: hinge loss for binary classification. This makes predictions of 0 or 1, rather than producing probabilities.\n\\item \\code{\"count:poisson\"}: Poisson regression for count data, output mean of Poisson distribution.\n\\code{\"max_delta_step\"} is set to 0.7 by default in Poisson regression (used to safeguard optimization)\n\\item \\code{\"survival:cox\"}: Cox regression for right censored survival time data (negative values are considered right censored).\n\nNote that predictions are returned on the hazard ratio scale (i.e., as HR = exp(marginal_prediction) in the proportional hazard function \\code{h(t) = h0(t) * HR}).\n\\item \\code{\"survival:aft\"}: Accelerated failure time model for censored survival time data.\nSee \\href{https://xgboost.readthedocs.io/en/latest/tutorials/aft_survival_analysis.html}{Survival Analysis with Accelerated Failure Time} for details.\n\\item \\code{\"multi:softprob\"}: multi-class classification throgh multinomial logistic likelihood.\n\\item \\code{\"reg:gamma\"}: gamma regression with log-link. Output is a mean of gamma distribution. It might be useful, e.g., for modeling insurance claims severity, or for any outcome that might be \\href{https://en.wikipedia.org/wiki/Gamma_distribution#Occurrence_and_applications}{gamma-distributed}.\n\\item \\code{\"reg:tweedie\"}: Tweedie regression with log-link. It might be useful, e.g., for modeling total loss in insurance, or for any outcome that might be \\href{https://en.wikipedia.org/wiki/Tweedie_distribution#Occurrence_and_applications}{Tweedie-distributed}.\n}\n\nThe following values are \\bold{NOT} supported by \\code{xgboost}, but are supported by \\code{\\link[=xgb.train]{xgb.train()}}\n(see \\code{\\link[=xgb.params]{xgb.params()}} for details):\n\\itemize{\n\\item \\code{\"reg:logistic\"}\n\\item \\code{\"binary:logitraw\"}\n\\item \\code{\"multi:softmax\"}\n\\item \\code{\"rank:ndcg\"}\n\\item \\code{\"rank:map\"}\n\\item \\code{\"rank:pairwise\"}\n}}\n\n\\item{nrounds}{Number of boosting iterations / rounds.\n\nNote that the number of default boosting rounds here is not automatically tuned, and different\nproblems will have vastly different optimal numbers of boosting rounds.}\n\n\\item{max_depth}{(for Tree Booster) (default=6, type=int32)\nMaximum depth of a tree. Increasing this value will make the model more complex and more likely to overfit. 0 indicates no limit on depth. Beware that XGBoost aggressively consumes memory when training a deep tree. \\code{\"exact\"} tree method requires non-zero value.\n\nrange: \\eqn{[0, \\infty)}}\n\n\\item{learning_rate}{(alias: \\code{eta})\nStep size shrinkage used in update to prevent overfitting. After each boosting step, we can directly get the weights of new features, and \\code{learning_rate} shrinks the feature weights to make the boosting process more conservative.\n\\itemize{\n\\item range: \\eqn{[0,1]}\n\\item default value: 0.3 for tree-based boosters, 0.5 for linear booster.\n}}\n\n\\item{min_child_weight}{(for Tree Booster) (default=1)\nMinimum sum of instance weight (hessian) needed in a child. If the tree partition step results in a leaf node with the sum of instance weight less than \\code{min_child_weight}, then the building process will give up further partitioning. In linear regression task, this simply corresponds to minimum number of instances needed to be in each node. The larger \\code{min_child_weight} is, the more conservative the algorithm will be.\n\nrange: \\eqn{[0, \\infty)}}\n\n\\item{min_split_loss}{(for Tree Booster) (default=0, alias: \\code{gamma})\nMinimum loss reduction required to make a further partition on a leaf node of the tree. The larger \\code{min_split_loss} is, the more conservative the algorithm will be. Note that a tree where no splits were made might still contain a single terminal node with a non-zero score.\n\nrange: \\eqn{[0, \\infty)}}\n\n\\item{reg_lambda}{(alias: \\code{lambda})\n\\itemize{\n\\item For tree-based boosters:\n\\itemize{\n\\item L2 regularization term on weights. Increasing this value will make model more conservative.\n\\item default: 1\n\\item range: \\eqn{[0, \\infty]}\n}\n\\item For linear booster:\n\\itemize{\n\\item L2 regularization term on weights. Increasing this value will make model more conservative. Normalised to number of training examples.\n\\item default: 0\n\\item range: \\eqn{[0, \\infty)}\n}\n}}\n\n\\item{weights}{Sample weights for each row in \\code{x} and \\code{y}. If \\code{NULL} (the default), each row\nwill have the same weight.\n\nIf not \\code{NULL}, should be passed as a numeric vector with length matching to the number of rows in \\code{x}.}\n\n\\item{verbosity}{Verbosity of printing messages. Valid values of 0 (silent), 1 (warning),\n2 (info), and 3 (debug).}\n\n\\item{monitor_training}{Whether to monitor objective optimization progress on the input data.\nNote that same 'x' and 'y' data are used for both model fitting and evaluation.}\n\n\\item{eval_set}{Subset of the data to use as evaluation set. Can be passed as:\n\\itemize{\n\\item A vector of row indices (base-1 numeration) indicating the observations that are to be designed\nas evaluation data.\n\\item A number between zero and one indicating a random fraction of the input data to use as\nevaluation data. Note that the selection will be done uniformly at random, regardless of\nargument \\code{weights}.\n}\n\nIf passed, this subset of the data will be excluded from the training procedure, and the\nevaluation metric(s) supplied under \\code{eval_metric} will be calculated on this dataset after each\nboosting iteration (pass \\code{verbosity>0} to have these metrics printed during training). If\n\\code{eval_metric} is not passed, a default metric will be selected according to \\code{objective}.\n\nIf passing a fraction, in classification problems, the evaluation set will be chosen in such a\nway that at least one observation of each class will be kept in the training data.\n\nFor more elaborate evaluation variants (e.g. custom metrics, multiple evaluation sets, etc.),\none might want to use \\code{\\link[=xgb.train]{xgb.train()}} instead.}\n\n\\item{early_stopping_rounds}{Number of boosting rounds after which training will be stopped\nif there is no improvement in performance (as measured by the last metric passed under\n\\code{eval_metric}, or by the default metric for the objective if \\code{eval_metric} is not passed) on the\nevaluation data from \\code{eval_set}. Must pass \\code{eval_set} in order to use this functionality.\n\nIf \\code{NULL}, early stopping will not be used.}\n\n\\item{print_every_n}{When passing \\code{verbosity>0} and either \\code{monitor_training=TRUE} or \\code{eval_set},\nevaluation logs (metrics calculated on the training and/or evaluation data) will be printed every\nnth iteration according to the value passed here. The first and last iteration are always\nincluded regardless of this 'n'.\n\nOnly has an effect when passing \\code{verbosity>0}.}\n\n\\item{eval_metric}{(default according to objective)\n\\itemize{\n\\item Evaluation metrics for validation data, a default metric will be assigned according to objective (rmse for regression, and logloss for classification, \\verb{mean average precision} for \\code{rank:map}, etc.)\n\\item User can add multiple evaluation metrics.\n\\item The choices are listed below:\n\\itemize{\n\\item \\code{\"rmse\"}: root mean square error\n\\item \\code{\"rmsle\"}: root mean square log error: \\eqn{\\sqrt{\\frac{1}{N}[log(pred + 1) - log(label + 1)]^2}}. Default metric of \\code{\"reg:squaredlogerror\"} objective. This metric reduces errors generated by outliers in dataset. But because \\code{log} function is employed, \\code{\"rmsle\"} might output \\code{nan} when prediction value is less than -1. See \\code{\"reg:squaredlogerror\"} for other requirements.\n\\item \\code{\"mae\"}: mean absolute error.\n\\item \\code{\"mape\"}: mean absolute percentage error.\n\\item \\code{\"mphe\"}: mean Pseudo Huber error. Default metric of \\code{\"reg:pseudohubererror\"} objective.\n\\item \\code{\"logloss\"}: negative log-likelihood.\n\\item \\code{\"error\"}: Binary classification error rate. It is calculated as \\verb{#(wrong cases)/#(all cases)}. For the predictions, the evaluation will regard the instances with prediction value larger than 0.5 as positive instances, and the others as negative instances.\n\\item \\code{\"error@t\"}: a different than 0.5 binary classification threshold value could be specified by providing a numerical value through 't'.\n\\item \\code{\"merror\"}: Multiclass classification error rate. It is calculated as \\verb{#(wrong cases)/#(all cases)}.\n\\item \\code{\"mlogloss\"}: \\href{https://scikit-learn.org/stable/modules/generated/sklearn.metrics.log_loss.html}{Multiclass logloss}.\n\\item \\code{\"auc\"}: area under the receiver-operating characteristic curve.\nAvailable for classification and learning-to-rank tasks.\n\\itemize{\n\\item When used with binary classification, the objective should be \\code{\"binary:logistic\"} or similar functions that work on probability.\n\\item When used with multi-class classification, objective should be \\code{\"multi:softprob\"} instead of \\code{\"multi:softmax\"}, as the latter doesn't output probability. Also the AUC is calculated by 1-vs-rest with reference class weighted by class prevalence.\n\\item When used with LTR task, the AUC is computed by comparing pairs of documents to count correctly sorted pairs. This corresponds to pairwise learning to rank. The implementation has some issues with average AUC around groups and distributed workers not being well-defined.\n\\item On a single machine the AUC calculation is exact. In a distributed environment the AUC is a weighted average over the AUC of training rows on each node - therefore, distributed AUC is an approximation sensitive to the distribution of data across workers. Use another metric in distributed environments if precision and reproducibility are important.\n\\item When input dataset contains only negative or positive samples, the output is \\code{NaN}. The behavior is implementation defined, for instance, \\code{scikit-learn} returns \\eqn{0.5} instead.\n}\n\\item \\code{\"aucpr\"}: area under the PR curve\nAvailable for classification and learning-to-rank tasks.\n\nAfter XGBoost 1.6, both of the requirements and restrictions for using \\code{\"aucpr\"} in classification problem are similar to \\code{\"auc\"}. For ranking task, only binary relevance label \\eqn{y \\in [0, 1]} is supported. Different from \\code{\"map\"} (mean average precision), \\code{\"aucpr\"} calculates the \\emph{interpolated} area under precision recall curve using continuous interpolation.\n\\item \\code{\"pre\"}: Precision at \\eqn{k}. Supports only learning to rank task.\n\\item \\code{\"ndcg\"}: normalized discounted cumulative gain\n\\item \\code{\"map\"}: mean average precision\n\nThe \\verb{average precision} is defined as:\n\n\\eqn{AP@l = \\frac{1}{min{(l, N)}}\\sum^l_{k=1}P@k \\cdot I_{(k)}}\n\nwhere \\eqn{I_{(k)}} is an indicator function that equals to \\eqn{1} when the document at \\eqn{k} is relevant and \\eqn{0} otherwise. The \\eqn{P@k} is the precision at \\eqn{k}, and \\eqn{N} is the total number of relevant documents. Lastly, the \\verb{mean average precision} is defined as the weighted average across all queries.\n\\item \\code{\"ndcg@n\"}, \\code{\"map@n\"}, \\code{\"pre@n\"}: \\eqn{n} can be assigned as an integer to cut off the top positions in the lists for evaluation.\n\\item \\code{\"ndcg-\"}, \\code{\"map-\"}, \\code{\"ndcg@n-\"}, \\code{\"map@n-\"}: In XGBoost, the NDCG and MAP evaluate the score of a list without any positive samples as \\eqn{1}. By appending \"-\" to the evaluation metric name, we can ask XGBoost to evaluate these scores as \\eqn{0} to be consistent under some conditions.\n\\item \\code{\"poisson-nloglik\"}: negative log-likelihood for Poisson regression\n\\item \\code{\"gamma-nloglik\"}: negative log-likelihood for gamma regression\n\\item \\code{\"cox-nloglik\"}: negative partial log-likelihood for Cox proportional hazards regression\n\\item \\code{\"gamma-deviance\"}: residual deviance for gamma regression\n\\item \\code{\"tweedie-nloglik\"}: negative log-likelihood for Tweedie regression (at a specified value of the \\code{tweedie_variance_power} parameter)\n\\item \\code{\"aft-nloglik\"}: Negative log likelihood of Accelerated Failure Time model.\nSee \\href{https://xgboost.readthedocs.io/en/latest/tutorials/aft_survival_analysis.html}{Survival Analysis with Accelerated Failure Time} for details.\n\\item \\code{\"interval-regression-accuracy\"}: Fraction of data points whose predicted labels fall in the interval-censored labels.\nOnly applicable for interval-censored data. See \\href{https://xgboost.readthedocs.io/en/latest/tutorials/aft_survival_analysis.html}{Survival Analysis with Accelerated Failure Time} for details.\n}\n}}\n\n\\item{nthreads}{Number of parallel threads to use. If passing zero, will use all CPU threads.}\n\n\\item{seed}{Seed to use for random number generation. If passing \\code{NULL}, will draw a random\nnumber using R's PRNG system to use as seed.}\n\n\\item{base_margin}{Base margin used for boosting from existing model.\n\nIf passing it, will start the gradient boosting procedure from the scores that are provided\nhere - for example, one can pass the raw scores from a previous model, or some per-observation\noffset, or similar.\n\nShould be either a numeric vector or numeric matrix (for multi-class and multi-target objectives)\nwith the same number of rows as \\code{x} and number of columns corresponding to number of optimization\ntargets, and should be in the untransformed scale (for example, for objective \\code{binary:logistic},\nit should have log-odds, not probabilities; and for objective \\code{multi:softprob}, should have\nnumber of columns matching to number of classes in the data).\n\nNote that, if it contains more than one column, then columns will not be matched by name to\nthe corresponding \\code{y} - \\code{base_margin} should have the same column order that the model will use\n(for example, for objective \\code{multi:softprob}, columns of \\code{base_margin} will be matched against\n\\code{levels(y)} by their position, regardless of what \\code{colnames(base_margin)} returns).\n\nIf \\code{NULL}, will start from zero, but note that for most objectives, an intercept is usually\nadded (controllable through parameter \\code{base_score} instead) when \\code{base_margin} is not passed.}\n\n\\item{monotone_constraints}{Optional monotonicity constraints for features.\n\nCan be passed either as a named list (when \\code{x} has column names), or as a vector. If passed\nas a vector and \\code{x} has column names, will try to match the elements by name.\n\nA value of \\code{+1} for a given feature makes the model predictions / scores constrained to be\na monotonically increasing function of that feature (that is, as the value of the feature\nincreases, the model prediction cannot decrease), while a value of \\code{-1} makes it a monotonically\ndecreasing function. A value of zero imposes no constraint.\n\nThe input for \\code{monotone_constraints} can be a subset of the columns of \\code{x} if named, in which\ncase the columns that are not referred to in \\code{monotone_constraints} will be assumed to have\na value of zero (no constraint imposed on the model for those features).\n\nSee the tutorial \\href{https://xgboost.readthedocs.io/en/stable/tutorials/monotonic.html}{Monotonic Constraints}\nfor a more detailed explanation.}\n\n\\item{interaction_constraints}{Constraints for interaction representing permitted interactions.\nThe constraints must be specified in the form of a list of vectors referencing columns in the\ndata, e.g. \\code{list(c(1, 2), c(3, 4, 5))} (with these numbers being column indices, numeration\nstarting at 1 - i.e. the first sublist references the first and second columns) or\n\\code{list(c(\"Sepal.Length\", \"Sepal.Width\"), c(\"Petal.Length\", \"Petal.Width\"))} (references\ncolumns by names), where each vector is a group of indices of features that are allowed to\ninteract with each other.\n\nSee the tutorial \\href{https://xgboost.readthedocs.io/en/stable/tutorials/feature_interaction_constraint.html}{Feature Interaction Constraints}\nfor more information.}\n\n\\item{reg_alpha}{(alias: \\code{reg_alpha})\n\\itemize{\n\\item L1 regularization term on weights. Increasing this value will make model more conservative.\n\\item For the linear booster, it's normalised to number of training examples.\n\\item default: 0\n\\item range: \\eqn{[0, \\infty)}\n}}\n\n\\item{max_bin}{(for Tree Booster) (default=256, type=int32)\n\\itemize{\n\\item Only used if \\code{tree_method} is set to \\code{\"hist\"} or \\code{\"approx\"}.\n\\item Maximum number of discrete bins to bucket continuous features.\n\\item Increasing this number improves the optimality of splits at the cost of higher computation time.\n}}\n\n\\item{max_leaves}{(for Tree Booster) (default=0, type=int32)\nMaximum number of nodes to be added. Not used by \\code{\"exact\"} tree method.}\n\n\\item{booster}{(default= \\code{\"gbtree\"})\nWhich booster to use. Can be \\code{\"gbtree\"}, \\code{\"gblinear\"} or \\code{\"dart\"}; \\code{\"gbtree\"} and \\code{\"dart\"} use tree based models while \\code{\"gblinear\"} uses linear functions.}\n\n\\item{subsample}{(for Tree Booster) (default=1)\nSubsample ratio of the training instances. Setting it to 0.5 means that XGBoost would randomly sample half of the training data prior to growing trees. and this will prevent overfitting. Subsampling will occur once in every boosting iteration.\n\nrange: \\eqn{(0,1]}}\n\n\\item{sampling_method}{(for Tree Booster) (default= \\code{\"uniform\"})\nThe method to use to sample the training instances.\n\\itemize{\n\\item \\code{\"uniform\"}: each training instance has an equal probability of being selected. Typically set\n\\code{\"subsample\"} >= 0.5 for good results.\n\\item \\code{\"gradient_based\"}: the selection probability for each training instance is proportional to the\n\\bold{regularized absolute value} of gradients (more specifically, \\eqn{\\sqrt{g^2+\\lambda h^2}}).\n\\code{\"subsample\"} may be set to as low as 0.1 without loss of model accuracy. Note that this\nsampling method is only supported when \\code{\"tree_method\"} is set to \\code{\"hist\"}; other tree\nmethods only support \\code{\"uniform\"} sampling.\n}}\n\n\\item{feature_weights}{Feature weights for column sampling.\n\nCan be passed either as a vector with length matching to columns of \\code{x}, or as a named\nlist (only if \\code{x} has column names) with names matching to columns of 'x'. If it is a\nnamed vector, will try to match the entries to column names of \\code{x} by name.\n\nIf \\code{NULL} (the default), all columns will have the same weight.}\n\n\\item{colsample_bytree, colsample_bylevel, colsample_bynode}{(for Tree Booster) (default=1)\nThis is a family of parameters for subsampling of columns.\n\\itemize{\n\\item All \\code{\"colsample_by*\"} parameters have a range of \\eqn{(0, 1]}, the default value of 1, and specify the fraction of columns to be subsampled.\n\\item \\code{\"colsample_bytree\"} is the subsample ratio of columns when constructing each tree. Subsampling occurs once for every tree constructed.\n\\item \\code{\"colsample_bylevel\"} is the subsample ratio of columns for each level. Subsampling occurs once for every new depth level reached in a tree. Columns are subsampled from the set of columns chosen for the current tree.\n\\item \\code{\"colsample_bynode\"} is the subsample ratio of columns for each node (split). Subsampling occurs once every time a new split is evaluated. Columns are subsampled from the set of columns chosen for the current level. This is not supported by the exact tree method.\n\\item \\code{\"colsample_by*\"} parameters work cumulatively. For instance,\nthe combination \\verb{\\{'colsample_bytree'=0.5, 'colsample_bylevel'=0.5, 'colsample_bynode'=0.5\\}} with 64 features will leave 8 features to choose from at\neach split.\n}\n\nOne can set the \\code{\"feature_weights\"} for DMatrix to\ndefine the probability of each feature being selected when using column sampling.}\n\n\\item{tree_method}{(for Tree Booster) (default= \\code{\"auto\"})\nThe tree construction algorithm used in XGBoost. See description in the \\href{https://arxiv.org/abs/1603.02754}{reference paper} and \\href{https://xgboost.readthedocs.io/en/latest/treemethod.html}{Tree Methods}.\n\nChoices: \\code{\"auto\"}, \\code{\"exact\"}, \\code{\"approx\"}, \\code{\"hist\"}, this is a combination of commonly\nused updaters. For other updaters like \\code{\"refresh\"}, set the parameter \\code{updater}\ndirectly.\n\\itemize{\n\\item \\code{\"auto\"}: Same as the \\code{\"hist\"} tree method.\n\\item \\code{\"exact\"}: Exact greedy algorithm. Enumerates all split candidates.\n\\item \\code{\"approx\"}: Approximate greedy algorithm using quantile sketch and gradient histogram.\n\\item \\code{\"hist\"}: Faster histogram optimized approximate greedy algorithm.\n}}\n\n\\item{max_delta_step}{(for Tree Booster) (default=0)\nMaximum delta step we allow each leaf output to be. If the value is set to 0, it means there is no constraint. If it is set to a positive value, it can help making the update step more conservative. Usually this parameter is not needed, but it might help in logistic regression when class is extremely imbalanced. Set it to value of 1-10 might help control the update.\n\nrange: \\eqn{[0, \\infty)}}\n\n\\item{scale_pos_weight}{(for Tree Booster) (default=1)\nControl the balance of positive and negative weights, useful for unbalanced classes. A typical value to consider: \\verb{sum(negative instances) / sum(positive instances)}. See \\href{https://xgboost.readthedocs.io/en/latest/tutorials/param_tuning.html}{Parameters Tuning} for more discussion. Also, see Higgs Kaggle competition demo for examples: \\href{https://github.com/dmlc/xgboost/blob/master/demo/kaggle-higgs/higgs-train.R}{R}, \\href{https://github.com/dmlc/xgboost/blob/master/demo/kaggle-higgs/higgs-numpy.py}{py1}, \\href{https://github.com/dmlc/xgboost/blob/master/demo/kaggle-higgs/higgs-cv.py}{py2}, \\href{https://github.com/dmlc/xgboost/blob/master/demo/guide-python/cross_validation.py}{py3}.}\n\n\\item{updater}{(for Linear Booster) (default= \\code{\"shotgun\"})\nChoice of algorithm to fit linear model\n\\itemize{\n\\item \\code{\"shotgun\"}: Parallel coordinate descent algorithm based on shotgun algorithm. Uses 'hogwild' parallelism and therefore produces a nondeterministic solution on each run.\n\\item \\code{\"coord_descent\"}: Ordinary coordinate descent algorithm. Also multithreaded but still produces a deterministic solution. When the \\code{device} parameter is set to \\code{\"cuda\"} or \\code{\"gpu\"}, a GPU variant would be used.\n}}\n\n\\item{grow_policy}{(for Tree Booster) (default= \\code{\"depthwise\"})\n\\itemize{\n\\item Controls a way new nodes are added to the tree.\n\\item Currently supported only if \\code{tree_method} is set to \\code{\"hist\"} or \\code{\"approx\"}.\n\\item Choices: \\code{\"depthwise\"}, \\code{\"lossguide\"}\n\\itemize{\n\\item \\code{\"depthwise\"}: split at nodes closest to the root.\n\\item \\code{\"lossguide\"}: split at nodes with highest loss change.\n}\n}}\n\n\\item{num_parallel_tree}{(for Tree Booster) (default=1)\nNumber of parallel trees constructed during each iteration. This option is used to support boosted random forest.}\n\n\\item{multi_strategy}{(for Tree Booster) (default = \\code{\"one_output_per_tree\"})\nThe strategy used for training multi-target models, including multi-target regression\nand multi-class classification. See \\href{https://xgboost.readthedocs.io/en/latest/tutorials/multioutput.html}{Multiple Outputs} for more information.\n\\itemize{\n\\item \\code{\"one_output_per_tree\"}: One model for each target.\n\\item \\code{\"multi_output_tree\"}: Use multi-target trees.\n}\n\nVersion added: 2.0.0\n\nNote: This parameter is working-in-progress.}\n\n\\item{base_score}{\\itemize{\n\\item The initial prediction score of all instances, global bias\n\\item The parameter is automatically estimated for selected objectives before training. To\ndisable the estimation, specify a real number argument.\n\\item If \\code{base_margin} is supplied, \\code{base_score} will not be added.\n\\item For sufficient number of iterations, changing this value will not have too much effect.\n}}\n\n\\item{seed_per_iteration}{(default= \\code{FALSE})\nSeed PRNG determnisticly via iterator number.}\n\n\\item{device}{(default= \\code{\"cpu\"})\nDevice for XGBoost to run. User can set it to one of the following values:\n\\itemize{\n\\item \\code{\"cpu\"}: Use CPU.\n\\item \\code{\"cuda\"}: Use a GPU (CUDA device).\n\\item \\code{\"cuda:\"}: \\verb{} is an integer that specifies the ordinal of the GPU (which GPU do you want to use if you have more than one devices).\n\\item \\code{\"gpu\"}: Default GPU device selection from the list of available and supported devices. Only \\code{\"cuda\"} devices are supported currently.\n\\item \\code{\"gpu:\"}: Default GPU device selection from the list of available and supported devices. Only \\code{\"cuda\"} devices are supported currently.\n}\n\nFor more information about GPU acceleration, see \\href{https://xgboost.readthedocs.io/en/latest/gpu/index.html}{XGBoost GPU Support}. In distributed environments, ordinal selection is handled by distributed frameworks instead of XGBoost. As a result, using \\code{\"cuda:\"} will result in an error. Use \\code{\"cuda\"} instead.\n\nVersion added: 2.0.0\n\nNote: if XGBoost was installed from CRAN, it won't have GPU support enabled, thus only \\code{\"cpu\"} will be available.\nTo get GPU support, the R package for XGBoost must be installed from source or from the GitHub releases - see\n\\href{https://xgboost.readthedocs.io/en/latest/install.html#r}{instructions}.}\n\n\\item{disable_default_eval_metric}{(default= \\code{FALSE})\nFlag to disable default metric. Set to 1 or \\code{TRUE} to disable.}\n\n\\item{use_rmm}{Whether to use RAPIDS Memory Manager (RMM) to allocate cache GPU\nmemory. The primary memory is always allocated on the RMM pool when XGBoost is built\n(compiled) with the RMM plugin enabled. Valid values are \\code{TRUE} and \\code{FALSE}. See\n\\href{https://xgboost.readthedocs.io/en/latest/python/rmm-examples/index.html}{Using XGBoost with RAPIDS Memory Manager (RMM) plugin} for details.}\n\n\\item{max_cached_hist_node}{(for Non-Exact Tree Methods) (default = 65536)\nMaximum number of cached nodes for histogram. This can be used with the \\code{\"hist\"} and the\n\\code{\"approx\"} tree methods.\n\nVersion added: 2.0.0\n\\itemize{\n\\item For most of the cases this parameter should not be set except for growing deep\ntrees. After 3.0, this parameter affects GPU algorithms as well.\n}}\n\n\\item{max_cat_to_onehot}{(for Non-Exact Tree Methods)\nA threshold for deciding whether XGBoost should use one-hot encoding based split for\ncategorical data. When number of categories is lesser than the threshold then one-hot\nencoding is chosen, otherwise the categories will be partitioned into children nodes.\n\nVersion added: 1.6.0}\n\n\\item{max_cat_threshold}{(for Non-Exact Tree Methods)\nMaximum number of categories considered for each split. Used only by partition-based\nsplits for preventing over-fitting.\n\nVersion added: 1.7.0}\n\n\\item{sample_type}{(for Dart Booster) (default= \\code{\"uniform\"})\nType of sampling algorithm.\n\\itemize{\n\\item \\code{\"uniform\"}: dropped trees are selected uniformly.\n\\item \\code{\"weighted\"}: dropped trees are selected in proportion to weight.\n}}\n\n\\item{normalize_type}{(for Dart Booster) (default= \\code{\"tree\"})\nType of normalization algorithm.\n\\itemize{\n\\item \\code{\"tree\"}: new trees have the same weight of each of dropped trees.\n\\itemize{\n\\item Weight of new trees are \\code{1 / (k + learning_rate)}.\n\\item Dropped trees are scaled by a factor of \\code{k / (k + learning_rate)}.\n}\n\\item \\code{\"forest\"}: new trees have the same weight of sum of dropped trees (forest).\n\\itemize{\n\\item Weight of new trees are \\code{1 / (1 + learning_rate)}.\n\\item Dropped trees are scaled by a factor of \\code{1 / (1 + learning_rate)}.\n}\n}}\n\n\\item{rate_drop}{(for Dart Booster) (default=0.0)\nDropout rate (a fraction of previous trees to drop during the dropout).\n\nrange: \\eqn{[0.0, 1.0]}}\n\n\\item{one_drop}{(for Dart Booster) (default=0)\nWhen this flag is enabled, at least one tree is always dropped during the dropout (allows Binomial-plus-one or epsilon-dropout from the original DART paper).}\n\n\\item{skip_drop}{(for Dart Booster) (default=0.0)\nProbability of skipping the dropout procedure during a boosting iteration.\n\\itemize{\n\\item If a dropout is skipped, new trees are added in the same manner as \\code{\"gbtree\"}.\n\\item Note that non-zero \\code{skip_drop} has higher priority than \\code{rate_drop} or \\code{one_drop}.\n}\n\nrange: \\eqn{[0.0, 1.0]}}\n\n\\item{feature_selector}{(for Linear Booster) (default= \\code{\"cyclic\"})\nFeature selection and ordering method\n\\itemize{\n\\item \\code{\"cyclic\"}: Deterministic selection by cycling through features one at a time.\n\\item \\code{\"shuffle\"}: Similar to \\code{\"cyclic\"} but with random feature shuffling prior to each update.\n\\item \\code{\"random\"}: A random (with replacement) coordinate selector.\n\\item \\code{\"greedy\"}: Select coordinate with the greatest gradient magnitude. It has \\code{O(num_feature^2)} complexity. It is fully deterministic. It allows restricting the selection to \\code{top_k} features per group with the largest magnitude of univariate weight change, by setting the \\code{top_k} parameter. Doing so would reduce the complexity to \\code{O(num_feature*top_k)}.\n\\item \\code{\"thrifty\"}: Thrifty, approximately-greedy feature selector. Prior to cyclic updates, reorders features in descending magnitude of their univariate weight changes. This operation is multithreaded and is a linear complexity approximation of the quadratic greedy selection. It allows restricting the selection to \\code{top_k} features per group with the largest magnitude of univariate weight change, by setting the \\code{top_k} parameter.\n}}\n\n\\item{top_k}{(for Linear Booster) (default=0)\nThe number of top features to select in \\code{greedy} and \\code{thrifty} feature selector. The value of 0 means using all the features.}\n\n\\item{tweedie_variance_power}{(for Tweedie Regression (\\code{\"objective=reg:tweedie\"})) (default=1.5)\n\\itemize{\n\\item Parameter that controls the variance of the Tweedie distribution \\code{var(y) ~ E(y)^tweedie_variance_power}\n\\item range: \\eqn{(1,2)}\n\\item Set closer to 2 to shift towards a gamma distribution\n\\item Set closer to 1 to shift towards a Poisson distribution.\n}}\n\n\\item{huber_slope}{(for using Pseudo-Huber (\\verb{\"reg:pseudohubererror}\")) (default = 1.0)\nA parameter used for Pseudo-Huber loss to define the \\eqn{\\delta} term.}\n\n\\item{quantile_alpha}{(for using Quantile Loss (\\code{\"reg:quantileerror\"}))\nA scalar or a list of targeted quantiles (passed as a numeric vector).\n\nVersion added: 2.0.0}\n\n\\item{expectile_alpha}{(for using Expectile Loss (\\code{\"reg:expectileerror\"}))\nA scalar or a list of targeted expectiles (passed as a numeric vector).}\n\n\\item{aft_loss_distribution}{(when using AFT Survival Loss (\\code{\"survival:aft\"}) and Negative Log Likelihood of AFT metric (\\code{\"aft-nloglik\"}))\nProbability Density Function, \\code{\"normal\"}, \\code{\"logistic\"}, or \\code{\"extreme\"}.}\n\n\\item{...}{Not used.\n\nSome arguments that were part of this function in previous XGBoost versions are currently\ndeprecated or have been renamed. If a deprecated or renamed argument is passed, will throw\na warning (by default) and use its current equivalent instead. This warning will become an\nerror if using the \\link[=xgboost-options]{'strict mode' option}.\n\nIf some additional argument is passed that is neither a current function argument nor\na deprecated or renamed argument, a warning or error will be thrown depending on the\n'strict mode' option.\n\n\\bold{Important:} \\code{...} will be removed in a future version, and all the current\ndeprecation warnings will become errors. Please use only arguments that form part of\nthe function signature.}\n}\n\\value{\nA model object, inheriting from both \\code{xgboost} and \\code{xgb.Booster}. Compared to the regular\n\\code{xgb.Booster} model class produced by \\code{\\link[=xgb.train]{xgb.train()}}, this \\code{xgboost} class will have an\nadditional attribute \\code{metadata} containing information which is used for formatting prediction\noutputs, such as class names for classification problems.\n}\n\\description{\nFits an XGBoost model (boosted decision tree ensemble) to given x/y data.\n\nSee the tutorial \\href{https://xgboost.readthedocs.io/en/stable/tutorials/model.html}{Introduction to Boosted Trees}\nfor a longer explanation of what XGBoost does, and the rest of the\n\\href{https://xgboost.readthedocs.io/en/latest/tutorials/index.html}{XGBoost Tutorials} for further\nexplanations XGBoost's features and usage.\n\nThis function is intended to provide a user-friendly interface for XGBoost that follows\nR's conventions for model fitting and predictions, but which doesn't expose all of the\npossible functionalities of the core XGBoost library.\n\nSee \\code{\\link[=xgb.train]{xgb.train()}} for a more flexible low-level alternative which is similar across different\nlanguage bindings of XGBoost and which exposes additional functionalities such as training on\nexternal memory data and learning-to-rank objectives.\n\nSee also the \\href{https://xgboost.readthedocs.io/en/latest/R-package/migration_guide.html}{migration guide}\nif coming from a previous version of XGBoost in the 1.x series.\n\nBy default, most of the parameters here have a value of \\code{NULL}, which signals XGBoost to use its\ndefault value. Default values are automatically determined by the XGBoost core library, and are\nsubject to change over XGBoost library versions. Some of them might differ according to the\nbooster type (e.g. defaults for regularization are different for linear and tree-based boosters).\nSee \\code{\\link[=xgb.params]{xgb.params()}} and the \\href{https://xgboost.readthedocs.io/en/latest/parameter.html}{online documentation}\nfor more details about parameters - but note that some of the parameters are not supported in\nthe \\code{xgboost()} interface.\n}\n\\details{\nFor package authors using 'xgboost' as a dependency, it is highly recommended to use\n\\code{\\link[=xgb.train]{xgb.train()}} in package code instead of \\code{\\link[=xgboost]{xgboost()}}, since it has a more stable interface\nand performs fewer data conversions and copies along the way.\n}\n\\examples{\ndata(mtcars)\n\n# Fit a small regression model on the mtcars data\nmodel_regression <- xgboost(mtcars[, -1], mtcars$mpg, nthreads = 1, nrounds = 3)\npredict(model_regression, mtcars, validate_features = TRUE)\n\n# Task objective is determined automatically according to the type of 'y'\ndata(iris)\nmodel_classif <- xgboost(iris[, -5], iris$Species, nthreads = 1, nrounds = 5)\npredict(model_classif, iris[1:10,])\npredict(model_classif, iris[1:10,], type = \"class\")\n\n# Can nevertheless choose a non-default objective if needed\nmodel_poisson <- xgboost(\n mtcars[, -1], mtcars$mpg,\n objective = \"count:poisson\",\n nthreads = 1,\n nrounds = 3\n)\n\n# Can calculate evaluation metrics during boosting rounds\ndata(ToothGrowth)\nxgboost(\n ToothGrowth[, c(\"len\", \"dose\")],\n ToothGrowth$supp,\n eval_metric = c(\"auc\", \"logloss\"),\n eval_set = 0.2,\n monitor_training = TRUE,\n verbosity = 1,\n nthreads = 1,\n nrounds = 3\n)\n}\n\\references{\n\\itemize{\n\\item Chen, Tianqi, and Carlos Guestrin. \"Xgboost: A scalable tree boosting system.\"\nProceedings of the 22nd acm sigkdd international conference on knowledge discovery and\ndata mining. 2016.\n\\item \\url{https://xgboost.readthedocs.io/en/stable/}\n}\n}\n" }, { "path": "R-package/pkgdown/_pkgdown.yml", "content": "url: https://github.com/dmlc/xgboost\n\ntemplate:\n bootstrap: 5\n math-rendering: mathjax\n\nreference:\n - title: High Level Interface\n desc: High level XGBoost interface\n contents:\n - \"xgboost\"\n - \"predict.xgboost\"\n - \"print.xgboost\"\n - title: Datasets\n desc: Test datasets bundled with the R package.\n contents:\n - \"agaricus.train\"\n - \"agaricus.test\"\n - title: Global Configuration\n desc: Global configuration for the XGBoost library.\n contents:\n - \"xgb.config\"\n - \"xgb.set.config\"\n - \"xgb.get.config\"\n - title: DMatrix\n desc: Low-level data storage.\n contents:\n - \"xgb.DMatrix\"\n - \"xgb.DMatrix.hasinfo\"\n - \"xgb.DMatrix.save\"\n - \"dim.xgb.DMatrix\"\n - \"dimnames.xgb.DMatrix\"\n - \"print.xgb.DMatrix\"\n - \"xgb.DataBatch\"\n - \"xgb.DataIter\"\n - \"xgb.get.DMatrix.data\"\n - \"xgb.get.DMatrix.num.non.missing\"\n - \"xgb.ExtMemDMatrix\"\n - \"xgb.QuantileDMatrix.from_iterator\"\n - \"xgb.get.DMatrix.qcut\"\n - \"xgb.slice.DMatrix\"\n - title: Booster\n desc: The model for XGBoost.\n contents:\n - \"a-compatibility-note-for-saveRDS-save\"\n - \"coef.xgb.Booster\"\n - \"getinfo.xgb.Booster\"\n - \"predict.xgb.Booster\"\n - \"print.xgb.Booster\"\n - \"xgb.load\"\n - \"xgb.load.raw\"\n - \"xgb.save\"\n - \"xgb.save.raw\"\n - \"xgb.copy.Booster\"\n - \"xgb.slice.Booster\"\n - \"xgb.get.num.boosted.rounds\"\n - \"xgb.is.same.Booster\"\n - \"xgb.importance\"\n - \"xgb.attr\"\n - \"xgb.create.features\"\n - \"xgb.model.dt.tree\"\n - \"xgb.model.parameters<-\"\n - \"xgb.ggplot.deepness\"\n - \"xgb.dump\"\n - \"variable.names.xgb.Booster\"\n - \"xgb.ggplot.importance\"\n - \"xgb.plot.multi.trees\"\n - \"xgb.plot.shap\"\n - \"xgb.ggplot.shap.summary\"\n - \"xgb.plot.tree\"\n - \"xgb.gblinear.history\"\n - title: Training Callbacks\n desc: Callback functions used for training.\n contents:\n - \"xgb.Callback\"\n - \"xgb.cb.cv.predict\"\n - \"xgb.cb.early.stop\"\n - \"xgb.cb.evaluation.log\"\n - \"xgb.cb.gblinear.history\"\n - \"xgb.cb.print.evaluation\"\n - \"xgb.cb.reset.parameters\"\n - \"xgb.cb.save.model\"\n - title: Low-level Training Functions\n desc: Low-level Training Functions with DMatrix and Booster\n contents:\n - \"xgb.params\"\n - \"xgb.train\"\n - \"xgb.cv\"\n - \"print.xgb.cv.synchronous\"\n - title: Deprecation Settings\n contents:\n - \"xgboost-options\"\n" }, { "path": "R-package/remove_warning_suppression_pragma.sh", "content": "#!/bin/bash\n# remove all #pragma's that suppress compiler warnings\nset -e\nset -x\nfor file in xgboost/src/dmlc-core/include/dmlc/*.h\ndo\n sed -i.bak -e 's/^.*#pragma GCC diagnostic.*$//' -e 's/^.*#pragma clang diagnostic.*$//' -e 's/^.*#pragma warning.*$//' \"${file}\"\ndone\nfor file in xgboost/src/dmlc-core/include/dmlc/*.h.bak\ndo\n rm \"${file}\"\ndone\nset +x\nset +e\n" }, { "path": "R-package/src/Makevars.in", "content": "# package root\nPKGROOT=../../\nENABLE_STD_THREAD=1\n# _*_ mode: Makefile; _*_\n\nCXX_STD = CXX17\n\nXGB_RFLAGS = \\\n @DMLC_DEFS@ \\\n @XGBOOST_BUILTIN_PREFETCH_PRESENT@ \\\n @XGBOOST_MM_PREFETCH_PRESENT@ \\\n -DXGBOOST_STRICT_R_MODE=1 \\\n -DDMLC_LOG_BEFORE_THROW=0 \\\n -DDMLC_ENABLE_STD_THREAD=$(ENABLE_STD_THREAD) \\\n -DDMLC_DISABLE_STDIN=1 \\\n -DDMLC_LOG_CUSTOMIZE=1\n\n# disable the use of thread_local for 32 bit windows:\nifeq ($(R_OSTYPE)$(WIN),windows)\n XGB_RFLAGS += -DDMLC_CXX11_THREAD_LOCAL=0\nendif\n$(foreach v, $(XGB_RFLAGS), $(warning $(v)))\n\nPKG_CPPFLAGS = \\\n -I$(PKGROOT)/include \\\n -I$(PKGROOT)/dmlc-core/include \\\n -I$(PKGROOT) \\\n $(XGB_RFLAGS)\n\nPKG_CXXFLAGS = \\\n @OPENMP_CXXFLAGS@ \\\n @ENDIAN_FLAG@ \\\n -pthread \\\n $(CXX_VISIBILITY)\n\nPKG_LIBS = \\\n @OPENMP_CXXFLAGS@ \\\n @OPENMP_LIB@ \\\n @ENDIAN_FLAG@ \\\n @BACKTRACE_LIB@ \\\n -pthread\n\nOBJECTS= \\\n ./xgboost_R.o \\\n ./xgboost_custom.o \\\n ./init.o \\\n $(PKGROOT)/src/metric/metric.o \\\n $(PKGROOT)/src/metric/elementwise_metric.o \\\n $(PKGROOT)/src/metric/multiclass_metric.o \\\n $(PKGROOT)/src/metric/rank_metric.o \\\n $(PKGROOT)/src/metric/auc.o \\\n $(PKGROOT)/src/metric/survival_metric.o \\\n $(PKGROOT)/src/objective/objective.o \\\n $(PKGROOT)/src/objective/regression_obj.o \\\n $(PKGROOT)/src/objective/multiclass_obj.o \\\n $(PKGROOT)/src/objective/lambdarank_obj.o \\\n $(PKGROOT)/src/objective/hinge.o \\\n $(PKGROOT)/src/objective/aft_obj.o \\\n $(PKGROOT)/src/objective/adaptive.o \\\n $(PKGROOT)/src/objective/init_estimation.o \\\n $(PKGROOT)/src/objective/quantile_obj.o \\\n $(PKGROOT)/src/gbm/gbm.o \\\n $(PKGROOT)/src/gbm/gbtree.o \\\n $(PKGROOT)/src/gbm/gbtree_model.o \\\n $(PKGROOT)/src/gbm/gblinear.o \\\n $(PKGROOT)/src/gbm/gblinear_model.o \\\n $(PKGROOT)/src/data/adapter.o \\\n $(PKGROOT)/src/data/array_interface.o \\\n $(PKGROOT)/src/data/cat_container.o \\\n $(PKGROOT)/src/data/simple_dmatrix.o \\\n $(PKGROOT)/src/data/data.o \\\n $(PKGROOT)/src/data/sparse_page_raw_format.o \\\n $(PKGROOT)/src/data/ellpack_page.o \\\n $(PKGROOT)/src/data/file_iterator.o \\\n $(PKGROOT)/src/data/gradient_index.o \\\n $(PKGROOT)/src/data/gradient_index_page_source.o \\\n $(PKGROOT)/src/data/gradient_index_format.o \\\n $(PKGROOT)/src/data/metainfo.o \\\n $(PKGROOT)/src/data/sparse_page_dmatrix.o \\\n $(PKGROOT)/src/data/sparse_page_source.o \\\n $(PKGROOT)/src/data/extmem_quantile_dmatrix.o \\\n $(PKGROOT)/src/data/quantile_dmatrix.o \\\n $(PKGROOT)/src/data/batch_utils.o \\\n $(PKGROOT)/src/data/proxy_dmatrix.o \\\n $(PKGROOT)/src/data/iterative_dmatrix.o \\\n $(PKGROOT)/src/predictor/predictor.o \\\n $(PKGROOT)/src/predictor/cpu_predictor.o \\\n $(PKGROOT)/src/predictor/interpretability/shap.o \\\n $(PKGROOT)/src/predictor/treeshap.o \\\n $(PKGROOT)/src/tree/constraints.o \\\n $(PKGROOT)/src/tree/param.o \\\n $(PKGROOT)/src/tree/fit_stump.o \\\n $(PKGROOT)/src/tree/tree_model.o \\\n $(PKGROOT)/src/tree/tree_view.o \\\n $(PKGROOT)/src/tree/tree_updater.o \\\n $(PKGROOT)/src/tree/multi_target_tree_model.o \\\n $(PKGROOT)/src/tree/updater_approx.o \\\n $(PKGROOT)/src/tree/updater_colmaker.o \\\n $(PKGROOT)/src/tree/updater_prune.o \\\n $(PKGROOT)/src/tree/updater_quantile_hist.o \\\n $(PKGROOT)/src/tree/updater_refresh.o \\\n $(PKGROOT)/src/tree/updater_sync.o \\\n $(PKGROOT)/src/tree/hist/hist_param.o \\\n $(PKGROOT)/src/tree/hist/histogram.o \\\n $(PKGROOT)/src/tree/hist/sampler.o \\\n $(PKGROOT)/src/linear/linear_updater.o \\\n $(PKGROOT)/src/linear/updater_coordinate.o \\\n $(PKGROOT)/src/linear/updater_shotgun.o \\\n $(PKGROOT)/src/learner.o \\\n $(PKGROOT)/src/context.o \\\n $(PKGROOT)/src/logging.o \\\n $(PKGROOT)/src/global_config.o \\\n $(PKGROOT)/src/collective/result.o \\\n $(PKGROOT)/src/collective/allgather.o \\\n $(PKGROOT)/src/collective/allreduce.o \\\n $(PKGROOT)/src/collective/broadcast.o \\\n $(PKGROOT)/src/collective/comm.o \\\n $(PKGROOT)/src/collective/comm_group.o \\\n $(PKGROOT)/src/collective/coll.o \\\n $(PKGROOT)/src/collective/tracker.o \\\n $(PKGROOT)/src/collective/in_memory_handler.o \\\n $(PKGROOT)/src/collective/loop.o \\\n $(PKGROOT)/src/collective/socket.o \\\n $(PKGROOT)/src/common/cache_manager.o \\\n $(PKGROOT)/src/common/charconv.o \\\n $(PKGROOT)/src/common/column_matrix.o \\\n $(PKGROOT)/src/common/common.o \\\n $(PKGROOT)/src/common/cuda_rt_utils.o \\\n $(PKGROOT)/src/common/error_msg.o \\\n $(PKGROOT)/src/common/hist_util.o \\\n $(PKGROOT)/src/common/host_device_vector.o \\\n $(PKGROOT)/src/common/io.o \\\n $(PKGROOT)/src/common/json.o \\\n $(PKGROOT)/src/common/linalg_op.o \\\n $(PKGROOT)/src/common/numeric.o \\\n $(PKGROOT)/src/common/optional_weight.o \\\n $(PKGROOT)/src/common/pseudo_huber.o \\\n $(PKGROOT)/src/common/quantile.o \\\n $(PKGROOT)/src/common/random.o \\\n $(PKGROOT)/src/common/stats.o \\\n $(PKGROOT)/src/common/survival_util.o \\\n $(PKGROOT)/src/common/threading_utils.o \\\n $(PKGROOT)/src/common/ranking_utils.o \\\n $(PKGROOT)/src/common/param_array.o \\\n $(PKGROOT)/src/common/expectile_loss_utils.o \\\n $(PKGROOT)/src/common/quantile_loss_utils.o \\\n $(PKGROOT)/src/common/timer.o \\\n $(PKGROOT)/src/common/version.o \\\n $(PKGROOT)/src/c_api/c_api.o \\\n $(PKGROOT)/src/c_api/c_api_error.o \\\n $(PKGROOT)/amalgamation/dmlc-minimum0.o\n" }, { "path": "R-package/src/Makevars.win.in", "content": "# package root\nPKGROOT=../../\nENABLE_STD_THREAD=0\n# _*_ mode: Makefile; _*_\n\nCXX_STD = CXX17\n\nXGB_RFLAGS = \\\n -DXGBOOST_STRICT_R_MODE=1 \\\n -DDMLC_LOG_BEFORE_THROW=0 \\\n -DDMLC_ENABLE_STD_THREAD=$(ENABLE_STD_THREAD) \\\n -DDMLC_DISABLE_STDIN=1 \\\n -DDMLC_LOG_CUSTOMIZE=1\n\n# disable the use of thread_local for 32 bit windows:\nifeq ($(R_OSTYPE)$(WIN),windows)\n XGB_RFLAGS += -DDMLC_CXX11_THREAD_LOCAL=0\nendif\n$(foreach v, $(XGB_RFLAGS), $(warning $(v)))\n\nPKG_CPPFLAGS = \\\n -I$(PKGROOT)/include \\\n -I$(PKGROOT)/dmlc-core/include \\\n -I$(PKGROOT) \\\n -DXGBOOST_BUILTIN_PREFETCH_PRESENT=1 \\\n @XGBOOST_MM_PREFETCH_PRESENT@ \\\n $(XGB_RFLAGS)\n\nPKG_CXXFLAGS = \\\n $(SHLIB_OPENMP_CXXFLAGS) \\\n -DDMLC_CMAKE_LITTLE_ENDIAN=1 \\\n $(SHLIB_PTHREAD_FLAGS) \\\n $(CXX_VISIBILITY)\n\nPKG_LIBS = \\\n $(SHLIB_OPENMP_CXXFLAGS) \\\n -DDMLC_CMAKE_LITTLE_ENDIAN=1 \\\n $(SHLIB_PTHREAD_FLAGS) \\\n -lwsock32 \\\n -lws2_32\n\nOBJECTS= \\\n ./xgboost_R.o \\\n ./xgboost_custom.o \\\n ./init.o \\\n $(PKGROOT)/src/metric/metric.o \\\n $(PKGROOT)/src/metric/elementwise_metric.o \\\n $(PKGROOT)/src/metric/multiclass_metric.o \\\n $(PKGROOT)/src/metric/rank_metric.o \\\n $(PKGROOT)/src/metric/auc.o \\\n $(PKGROOT)/src/metric/survival_metric.o \\\n $(PKGROOT)/src/objective/objective.o \\\n $(PKGROOT)/src/objective/regression_obj.o \\\n $(PKGROOT)/src/objective/multiclass_obj.o \\\n $(PKGROOT)/src/objective/lambdarank_obj.o \\\n $(PKGROOT)/src/objective/hinge.o \\\n $(PKGROOT)/src/objective/aft_obj.o \\\n $(PKGROOT)/src/objective/adaptive.o \\\n $(PKGROOT)/src/objective/init_estimation.o \\\n $(PKGROOT)/src/objective/quantile_obj.o \\\n $(PKGROOT)/src/gbm/gbm.o \\\n $(PKGROOT)/src/gbm/gbtree.o \\\n $(PKGROOT)/src/gbm/gbtree_model.o \\\n $(PKGROOT)/src/gbm/gblinear.o \\\n $(PKGROOT)/src/gbm/gblinear_model.o \\\n $(PKGROOT)/src/data/adapter.o \\\n $(PKGROOT)/src/data/array_interface.o \\\n $(PKGROOT)/src/data/cat_container.o \\\n $(PKGROOT)/src/data/simple_dmatrix.o \\\n $(PKGROOT)/src/data/data.o \\\n $(PKGROOT)/src/data/sparse_page_raw_format.o \\\n $(PKGROOT)/src/data/ellpack_page.o \\\n $(PKGROOT)/src/data/file_iterator.o \\\n $(PKGROOT)/src/data/gradient_index.o \\\n $(PKGROOT)/src/data/gradient_index_page_source.o \\\n $(PKGROOT)/src/data/gradient_index_format.o \\\n $(PKGROOT)/src/data/metainfo.o \\\n $(PKGROOT)/src/data/sparse_page_dmatrix.o \\\n $(PKGROOT)/src/data/sparse_page_source.o \\\n $(PKGROOT)/src/data/extmem_quantile_dmatrix.o \\\n $(PKGROOT)/src/data/quantile_dmatrix.o \\\n $(PKGROOT)/src/data/batch_utils.o \\\n $(PKGROOT)/src/data/proxy_dmatrix.o \\\n $(PKGROOT)/src/data/iterative_dmatrix.o \\\n $(PKGROOT)/src/predictor/predictor.o \\\n $(PKGROOT)/src/predictor/cpu_predictor.o \\\n $(PKGROOT)/src/predictor/interpretability/shap.o \\\n $(PKGROOT)/src/predictor/treeshap.o \\\n $(PKGROOT)/src/tree/constraints.o \\\n $(PKGROOT)/src/tree/param.o \\\n $(PKGROOT)/src/tree/fit_stump.o \\\n $(PKGROOT)/src/tree/tree_model.o \\\n $(PKGROOT)/src/tree/tree_view.o \\\n $(PKGROOT)/src/tree/multi_target_tree_model.o \\\n $(PKGROOT)/src/tree/tree_updater.o \\\n $(PKGROOT)/src/tree/updater_approx.o \\\n $(PKGROOT)/src/tree/updater_colmaker.o \\\n $(PKGROOT)/src/tree/updater_prune.o \\\n $(PKGROOT)/src/tree/updater_quantile_hist.o \\\n $(PKGROOT)/src/tree/updater_refresh.o \\\n $(PKGROOT)/src/tree/updater_sync.o \\\n $(PKGROOT)/src/tree/hist/hist_param.o \\\n $(PKGROOT)/src/tree/hist/histogram.o \\\n $(PKGROOT)/src/tree/hist/sampler.o \\\n $(PKGROOT)/src/linear/linear_updater.o \\\n $(PKGROOT)/src/linear/updater_coordinate.o \\\n $(PKGROOT)/src/linear/updater_shotgun.o \\\n $(PKGROOT)/src/learner.o \\\n $(PKGROOT)/src/context.o \\\n $(PKGROOT)/src/logging.o \\\n $(PKGROOT)/src/global_config.o \\\n $(PKGROOT)/src/collective/result.o \\\n $(PKGROOT)/src/collective/allgather.o \\\n $(PKGROOT)/src/collective/allreduce.o \\\n $(PKGROOT)/src/collective/broadcast.o \\\n $(PKGROOT)/src/collective/comm.o \\\n $(PKGROOT)/src/collective/comm_group.o \\\n $(PKGROOT)/src/collective/coll.o \\\n $(PKGROOT)/src/collective/tracker.o \\\n $(PKGROOT)/src/collective/in_memory_handler.o \\\n $(PKGROOT)/src/collective/loop.o \\\n $(PKGROOT)/src/collective/socket.o \\\n $(PKGROOT)/src/common/cache_manager.o \\\n $(PKGROOT)/src/common/charconv.o \\\n $(PKGROOT)/src/common/column_matrix.o \\\n $(PKGROOT)/src/common/common.o \\\n $(PKGROOT)/src/common/cuda_rt_utils.o \\\n $(PKGROOT)/src/common/error_msg.o \\\n $(PKGROOT)/src/common/hist_util.o \\\n $(PKGROOT)/src/common/host_device_vector.o \\\n $(PKGROOT)/src/common/io.o \\\n $(PKGROOT)/src/common/json.o \\\n $(PKGROOT)/src/common/linalg_op.o \\\n $(PKGROOT)/src/common/numeric.o \\\n $(PKGROOT)/src/common/optional_weight.o \\\n $(PKGROOT)/src/common/pseudo_huber.o \\\n $(PKGROOT)/src/common/quantile.o \\\n $(PKGROOT)/src/common/random.o \\\n $(PKGROOT)/src/common/stats.o \\\n $(PKGROOT)/src/common/survival_util.o \\\n $(PKGROOT)/src/common/threading_utils.o \\\n $(PKGROOT)/src/common/ranking_utils.o \\\n $(PKGROOT)/src/common/param_array.o \\\n $(PKGROOT)/src/common/expectile_loss_utils.o \\\n $(PKGROOT)/src/common/quantile_loss_utils.o \\\n $(PKGROOT)/src/common/timer.o \\\n $(PKGROOT)/src/common/version.o \\\n $(PKGROOT)/src/c_api/c_api.o \\\n $(PKGROOT)/src/c_api/c_api_error.o \\\n $(PKGROOT)/amalgamation/dmlc-minimum0.o\n" }, { "path": "R-package/src/config.h.in", "content": "/* src/config.h.in. Generated from configure.ac by autoheader. */\n\n/* Define if building universal (internal helper macro) */\n#undef AC_APPLE_UNIVERSAL_BUILD\n\n/* Define to 1 if you have the header file. */\n#undef HAVE_INTTYPES_H\n\n/* Define to 1 if you have the header file. */\n#undef HAVE_STDINT_H\n\n/* Define to 1 if you have the header file. */\n#undef HAVE_STDIO_H\n\n/* Define to 1 if you have the header file. */\n#undef HAVE_STDLIB_H\n\n/* Define to 1 if you have the header file. */\n#undef HAVE_STRINGS_H\n\n/* Define to 1 if you have the header file. */\n#undef HAVE_STRING_H\n\n/* Define to 1 if you have the header file. */\n#undef HAVE_SYS_STAT_H\n\n/* Define to 1 if you have the header file. */\n#undef HAVE_SYS_TYPES_H\n\n/* Define to 1 if you have the header file. */\n#undef HAVE_UNISTD_H\n\n/* Define to the address where bug reports for this package should be sent. */\n#undef PACKAGE_BUGREPORT\n\n/* Define to the full name of this package. */\n#undef PACKAGE_NAME\n\n/* Define to the full name and version of this package. */\n#undef PACKAGE_STRING\n\n/* Define to the one symbol short name of this package. */\n#undef PACKAGE_TARNAME\n\n/* Define to the home page for this package. */\n#undef PACKAGE_URL\n\n/* Define to the version of this package. */\n#undef PACKAGE_VERSION\n\n/* Define to 1 if all of the C90 standard headers exist (not just the ones\n required in a freestanding environment). This macro is provided for\n backward compatibility; new code need not use it. */\n#undef STDC_HEADERS\n\n/* Define WORDS_BIGENDIAN to 1 if your processor stores words with the most\n significant byte first (like Motorola and SPARC, unlike Intel). */\n#if defined AC_APPLE_UNIVERSAL_BUILD\n# if defined __BIG_ENDIAN__\n# define WORDS_BIGENDIAN 1\n# endif\n#else\n# ifndef WORDS_BIGENDIAN\n# undef WORDS_BIGENDIAN\n# endif\n#endif\n" }, { "path": "R-package/src/init.c", "content": "/* Copyright (c) 2015 by Contributors\n *\n * This file was initially generated using the following R command:\n * tools::package_native_routine_registration_skeleton('.', con = 'src/init.c', character_only = F)\n * and edited to conform to xgboost C linter requirements. For details, see\n * https://cran.r-project.org/doc/manuals/r-release/R-exts.html#Registering-native-routines\n */\n#include \n#include \n#include \n#include \n\n/* FIXME:\nCheck these declarations against the C/Fortran source code.\n*/\n\n/* .Call calls */\nextern void XGBInitializeAltrepClass_R(DllInfo *info);\nextern SEXP XGDuplicate_R(SEXP);\nextern SEXP XGPointerEqComparison_R(SEXP, SEXP);\nextern SEXP XGBoosterTrainOneIter_R(SEXP, SEXP, SEXP, SEXP, SEXP);\nextern SEXP XGBoosterCreate_R(SEXP);\nextern SEXP XGBoosterReset_R(SEXP);\nextern SEXP XGBoosterCopyInfoFromDMatrix_R(SEXP, SEXP);\nextern SEXP XGBoosterSetStrFeatureInfo_R(SEXP, SEXP, SEXP);\nextern SEXP XGBoosterGetStrFeatureInfo_R(SEXP, SEXP);\nextern SEXP XGBoosterBoostedRounds_R(SEXP);\nextern SEXP XGBoosterGetNumFeature_R(SEXP);\nextern SEXP XGBoosterDumpModel_R(SEXP, SEXP, SEXP, SEXP);\nextern SEXP XGBoosterEvalOneIter_R(SEXP, SEXP, SEXP, SEXP);\nextern SEXP XGBoosterGetAttrNames_R(SEXP);\nextern SEXP XGBoosterGetAttr_R(SEXP, SEXP);\nextern SEXP XGBoosterLoadModelFromRaw_R(SEXP, SEXP);\nextern SEXP XGBoosterSaveModelToRaw_R(SEXP handle, SEXP config);\nextern SEXP XGBoosterLoadModel_R(SEXP, SEXP);\nextern SEXP XGBoosterSaveJsonConfig_R(SEXP handle);\nextern SEXP XGBoosterLoadJsonConfig_R(SEXP handle, SEXP value);\nextern SEXP XGBoosterSerializeToBuffer_R(SEXP handle);\nextern SEXP XGBoosterUnserializeFromBuffer_R(SEXP handle, SEXP raw);\nextern SEXP XGBoosterPredictFromDMatrix_R(SEXP, SEXP, SEXP);\nextern SEXP XGBoosterPredictFromDense_R(SEXP, SEXP, SEXP, SEXP, SEXP);\nextern SEXP XGBoosterPredictFromCSR_R(SEXP, SEXP, SEXP, SEXP, SEXP);\nextern SEXP XGBoosterPredictFromColumnar_R(SEXP, SEXP, SEXP, SEXP, SEXP);\nextern SEXP XGBoosterSaveModel_R(SEXP, SEXP);\nextern SEXP XGBoosterSetAttr_R(SEXP, SEXP, SEXP);\nextern SEXP XGBoosterSetParam_R(SEXP, SEXP, SEXP);\nextern SEXP XGBoosterUpdateOneIter_R(SEXP, SEXP, SEXP);\nextern SEXP XGCheckNullPtr_R(SEXP);\nextern SEXP XGSetArrayDimNamesInplace_R(SEXP, SEXP);\nextern SEXP XGSetVectorNamesInplace_R(SEXP, SEXP);\nextern SEXP XGDMatrixCreateFromCSC_R(SEXP, SEXP, SEXP, SEXP, SEXP, SEXP);\nextern SEXP XGDMatrixCreateFromCSR_R(SEXP, SEXP, SEXP, SEXP, SEXP, SEXP);\nextern SEXP XGDMatrixCreateFromURI_R(SEXP, SEXP, SEXP);\nextern SEXP XGDMatrixCreateFromMat_R(SEXP, SEXP, SEXP);\nextern SEXP XGDMatrixGetFloatInfo_R(SEXP, SEXP);\nextern SEXP XGDMatrixGetUIntInfo_R(SEXP, SEXP);\nextern SEXP XGDMatrixCreateFromDF_R(SEXP, SEXP, SEXP);\nextern SEXP XGDMatrixGetStrFeatureInfo_R(SEXP, SEXP);\nextern SEXP XGDMatrixNumCol_R(SEXP);\nextern SEXP XGDMatrixNumRow_R(SEXP);\nextern SEXP XGProxyDMatrixCreate_R(void);\nextern SEXP XGProxyDMatrixSetDataDense_R(SEXP, SEXP);\nextern SEXP XGProxyDMatrixSetDataCSR_R(SEXP, SEXP);\nextern SEXP XGProxyDMatrixSetDataColumnar_R(SEXP, SEXP);\nextern SEXP XGDMatrixCreateFromCallback_R(SEXP, SEXP, SEXP, SEXP, SEXP, SEXP, SEXP);\nextern SEXP XGQuantileDMatrixCreateFromCallback_R(SEXP, SEXP, SEXP, SEXP, SEXP, SEXP, SEXP, SEXP);\nextern SEXP XGDMatrixFree_R(SEXP);\nextern SEXP XGGetRNAIntAsDouble(void);\nextern SEXP XGDMatrixGetQuantileCut_R(SEXP);\nextern SEXP XGDMatrixNumNonMissing_R(SEXP);\nextern SEXP XGDMatrixGetDataAsCSR_R(SEXP);\nextern SEXP XGDMatrixSaveBinary_R(SEXP, SEXP, SEXP);\nextern SEXP XGDMatrixSetInfo_R(SEXP, SEXP, SEXP);\nextern SEXP XGDMatrixSetStrFeatureInfo_R(SEXP, SEXP, SEXP);\nextern SEXP XGDMatrixSliceDMatrix_R(SEXP, SEXP, SEXP);\nextern SEXP XGBSetGlobalConfig_R(SEXP);\nextern SEXP XGBGetGlobalConfig_R(void);\nextern SEXP XGBoosterFeatureScore_R(SEXP, SEXP);\nextern SEXP XGBoosterSlice_R(SEXP, SEXP, SEXP, SEXP);\nextern SEXP XGBoosterSliceAndReplace_R(SEXP, SEXP, SEXP, SEXP);\n\nstatic const R_CallMethodDef CallEntries[] = {\n {\"XGDuplicate_R\", (DL_FUNC) &XGDuplicate_R, 1},\n {\"XGPointerEqComparison_R\", (DL_FUNC) &XGPointerEqComparison_R, 2},\n {\"XGBoosterTrainOneIter_R\", (DL_FUNC) &XGBoosterTrainOneIter_R, 5},\n {\"XGBoosterCreate_R\", (DL_FUNC) &XGBoosterCreate_R, 1},\n {\"XGBoosterReset_R\", (DL_FUNC) &XGBoosterReset_R, 1},\n {\"XGBoosterCopyInfoFromDMatrix_R\", (DL_FUNC) &XGBoosterCopyInfoFromDMatrix_R, 2},\n {\"XGBoosterSetStrFeatureInfo_R\",(DL_FUNC) &XGBoosterSetStrFeatureInfo_R,3}, // NOLINT\n {\"XGBoosterGetStrFeatureInfo_R\",(DL_FUNC) &XGBoosterGetStrFeatureInfo_R,2}, // NOLINT\n {\"XGBoosterBoostedRounds_R\", (DL_FUNC) &XGBoosterBoostedRounds_R, 1},\n {\"XGBoosterGetNumFeature_R\", (DL_FUNC) &XGBoosterGetNumFeature_R, 1},\n {\"XGBoosterDumpModel_R\", (DL_FUNC) &XGBoosterDumpModel_R, 4},\n {\"XGBoosterEvalOneIter_R\", (DL_FUNC) &XGBoosterEvalOneIter_R, 4},\n {\"XGBoosterGetAttrNames_R\", (DL_FUNC) &XGBoosterGetAttrNames_R, 1},\n {\"XGBoosterGetAttr_R\", (DL_FUNC) &XGBoosterGetAttr_R, 2},\n {\"XGBoosterLoadModelFromRaw_R\", (DL_FUNC) &XGBoosterLoadModelFromRaw_R, 2},\n {\"XGBoosterSaveModelToRaw_R\", (DL_FUNC) &XGBoosterSaveModelToRaw_R, 2},\n {\"XGBoosterLoadModel_R\", (DL_FUNC) &XGBoosterLoadModel_R, 2},\n {\"XGBoosterSaveJsonConfig_R\", (DL_FUNC) &XGBoosterSaveJsonConfig_R, 1},\n {\"XGBoosterLoadJsonConfig_R\", (DL_FUNC) &XGBoosterLoadJsonConfig_R, 2},\n {\"XGBoosterSerializeToBuffer_R\", (DL_FUNC) &XGBoosterSerializeToBuffer_R, 1},\n {\"XGBoosterUnserializeFromBuffer_R\", (DL_FUNC) &XGBoosterUnserializeFromBuffer_R, 2},\n {\"XGBoosterPredictFromDMatrix_R\", (DL_FUNC) &XGBoosterPredictFromDMatrix_R, 3},\n {\"XGBoosterPredictFromDense_R\", (DL_FUNC) &XGBoosterPredictFromDense_R, 5},\n {\"XGBoosterPredictFromCSR_R\", (DL_FUNC) &XGBoosterPredictFromCSR_R, 5},\n {\"XGBoosterPredictFromColumnar_R\", (DL_FUNC) &XGBoosterPredictFromColumnar_R, 5},\n {\"XGBoosterSaveModel_R\", (DL_FUNC) &XGBoosterSaveModel_R, 2},\n {\"XGBoosterSetAttr_R\", (DL_FUNC) &XGBoosterSetAttr_R, 3},\n {\"XGBoosterSetParam_R\", (DL_FUNC) &XGBoosterSetParam_R, 3},\n {\"XGBoosterUpdateOneIter_R\", (DL_FUNC) &XGBoosterUpdateOneIter_R, 3},\n {\"XGCheckNullPtr_R\", (DL_FUNC) &XGCheckNullPtr_R, 1},\n {\"XGSetArrayDimNamesInplace_R\", (DL_FUNC) &XGSetArrayDimNamesInplace_R, 2},\n {\"XGSetVectorNamesInplace_R\", (DL_FUNC) &XGSetVectorNamesInplace_R, 2},\n {\"XGDMatrixCreateFromCSC_R\", (DL_FUNC) &XGDMatrixCreateFromCSC_R, 6},\n {\"XGDMatrixCreateFromCSR_R\", (DL_FUNC) &XGDMatrixCreateFromCSR_R, 6},\n {\"XGDMatrixCreateFromURI_R\", (DL_FUNC) &XGDMatrixCreateFromURI_R, 3},\n {\"XGDMatrixCreateFromMat_R\", (DL_FUNC) &XGDMatrixCreateFromMat_R, 3},\n {\"XGDMatrixGetFloatInfo_R\", (DL_FUNC) &XGDMatrixGetFloatInfo_R, 2},\n {\"XGDMatrixGetUIntInfo_R\", (DL_FUNC) &XGDMatrixGetUIntInfo_R, 2},\n {\"XGDMatrixCreateFromDF_R\", (DL_FUNC) &XGDMatrixCreateFromDF_R, 3},\n {\"XGDMatrixGetStrFeatureInfo_R\", (DL_FUNC) &XGDMatrixGetStrFeatureInfo_R, 2},\n {\"XGDMatrixNumCol_R\", (DL_FUNC) &XGDMatrixNumCol_R, 1},\n {\"XGDMatrixNumRow_R\", (DL_FUNC) &XGDMatrixNumRow_R, 1},\n {\"XGProxyDMatrixCreate_R\", (DL_FUNC) &XGProxyDMatrixCreate_R, 0},\n {\"XGProxyDMatrixSetDataDense_R\", (DL_FUNC) &XGProxyDMatrixSetDataDense_R, 2},\n {\"XGProxyDMatrixSetDataCSR_R\", (DL_FUNC) &XGProxyDMatrixSetDataCSR_R, 2},\n {\"XGProxyDMatrixSetDataColumnar_R\", (DL_FUNC) &XGProxyDMatrixSetDataColumnar_R, 2},\n {\"XGDMatrixCreateFromCallback_R\", (DL_FUNC) &XGDMatrixCreateFromCallback_R, 7},\n {\"XGQuantileDMatrixCreateFromCallback_R\", (DL_FUNC) &XGQuantileDMatrixCreateFromCallback_R, 8},\n {\"XGDMatrixFree_R\", (DL_FUNC) &XGDMatrixFree_R, 1},\n {\"XGGetRNAIntAsDouble\", (DL_FUNC) &XGGetRNAIntAsDouble, 0},\n {\"XGDMatrixGetQuantileCut_R\", (DL_FUNC) &XGDMatrixGetQuantileCut_R, 1},\n {\"XGDMatrixNumNonMissing_R\", (DL_FUNC) &XGDMatrixNumNonMissing_R, 1},\n {\"XGDMatrixGetDataAsCSR_R\", (DL_FUNC) &XGDMatrixGetDataAsCSR_R, 1},\n {\"XGDMatrixSaveBinary_R\", (DL_FUNC) &XGDMatrixSaveBinary_R, 3},\n {\"XGDMatrixSetInfo_R\", (DL_FUNC) &XGDMatrixSetInfo_R, 3},\n {\"XGDMatrixSetStrFeatureInfo_R\", (DL_FUNC) &XGDMatrixSetStrFeatureInfo_R, 3},\n {\"XGDMatrixSliceDMatrix_R\", (DL_FUNC) &XGDMatrixSliceDMatrix_R, 3},\n {\"XGBSetGlobalConfig_R\", (DL_FUNC) &XGBSetGlobalConfig_R, 1},\n {\"XGBGetGlobalConfig_R\", (DL_FUNC) &XGBGetGlobalConfig_R, 0},\n {\"XGBoosterFeatureScore_R\", (DL_FUNC) &XGBoosterFeatureScore_R, 2},\n {\"XGBoosterSlice_R\", (DL_FUNC) &XGBoosterSlice_R, 4},\n {\"XGBoosterSliceAndReplace_R\", (DL_FUNC) &XGBoosterSliceAndReplace_R, 4},\n {NULL, NULL, 0}\n};\n\n#if defined(_WIN32)\n__declspec(dllexport)\n#endif // defined(_WIN32)\nvoid attribute_visible R_init_xgboost(DllInfo *dll) {\n R_registerRoutines(dll, NULL, CallEntries, NULL, NULL);\n R_useDynamicSymbols(dll, FALSE);\n XGBInitializeAltrepClass_R(dll);\n}\n" }, { "path": "R-package/src/xgboost-win.def", "content": "LIBRARY xgboost.dll\nEXPORTS\n R_init_xgboost\n" }, { "path": "R-package/src/xgboost_R.cc", "content": "/**\n * Copyright 2014-2024, XGBoost Contributors\n */\n#include \n#include \n#include \n#include \n#include \n#include \n\n#include \n#include \n#include \n#include \n#include \n#include \n#include \n#include \n#include \n#include \n#include \n\n#include \"../../src/c_api/c_api_error.h\"\n#include \"../../src/c_api/c_api_utils.h\" // MakeSparseFromPtr\n#include \"../../src/common/threading_utils.h\"\n#include \"../../src/data/array_interface.h\" // for ArrayInterface\n\n#include \"./xgboost_R.h\" // Must follow other includes.\n\n#ifdef _MSC_VER\n#error \"Compilation of R package with MSVC is not supported due to issues handling R headers\"\n#endif\n\nnamespace {\n\n/* Note: this class is used as a throwable exception.\nSome xgboost C functions that use callbacks will catch exceptions\nthat happen inside of the callback execution, hence it purposefully\ndoesn't inherit from 'std::exception' even if used as such. */\nstruct ErrorWithUnwind {};\n\nvoid ThrowExceptionFromRError(void *, Rboolean jump) {\n if (jump) {\n throw ErrorWithUnwind();\n }\n}\n\nstruct PtrToConstChar {\n const char *ptr;\n};\n\nSEXP WrappedMkChar(void *void_ptr) {\n return Rf_mkChar(static_cast(void_ptr)->ptr);\n}\n\nSEXP SafeMkChar(const char *c_str, SEXP continuation_token) {\n PtrToConstChar ptr_struct{c_str};\n return R_UnwindProtect(\n WrappedMkChar, static_cast(&ptr_struct),\n ThrowExceptionFromRError, nullptr,\n continuation_token);\n}\n\nstruct RFunAndEnv {\n SEXP R_fun;\n SEXP R_calling_env;\n};\n\nSEXP WrappedExecFun(void *void_ptr) {\n RFunAndEnv *r_fun_and_env = static_cast(void_ptr);\n SEXP f_expr = Rf_protect(Rf_lang1(r_fun_and_env->R_fun));\n SEXP out = Rf_protect(Rf_eval(f_expr, r_fun_and_env->R_calling_env));\n Rf_unprotect(2);\n return out;\n}\n\nSEXP SafeExecFun(SEXP R_fun, SEXP R_calling_env, SEXP continuation_token) {\n RFunAndEnv r_fun_and_env{R_fun, R_calling_env};\n return R_UnwindProtect(\n WrappedExecFun, static_cast(&r_fun_and_env),\n ThrowExceptionFromRError, nullptr,\n continuation_token);\n}\n\nSEXP WrappedAllocReal(void *void_ptr) {\n size_t *size = static_cast(void_ptr);\n return Rf_allocVector(REALSXP, *size);\n}\n\nSEXP SafeAllocReal(size_t size, SEXP continuation_token) {\n return R_UnwindProtect(\n WrappedAllocReal, static_cast(&size),\n ThrowExceptionFromRError, nullptr,\n continuation_token);\n}\n\nSEXP WrappedAllocInteger(void *void_ptr) {\n size_t *size = static_cast(void_ptr);\n return Rf_allocVector(INTSXP, *size);\n}\n\nSEXP SafeAllocInteger(size_t size, SEXP continuation_token) {\n return R_UnwindProtect(\n WrappedAllocInteger, static_cast(&size),\n ThrowExceptionFromRError, nullptr,\n continuation_token);\n}\n\n[[nodiscard]] std::string MakeArrayInterfaceFromRMat(SEXP R_mat) {\n SEXP mat_dims = Rf_getAttrib(R_mat, R_DimSymbol);\n if (Rf_xlength(mat_dims) > 2) {\n LOG(FATAL) << \"Passed input array with more than two dimensions, which is not supported.\";\n }\n const int *ptr_mat_dims = INTEGER(mat_dims);\n\n // Lambda for type dispatch.\n auto make_matrix = [=](auto const *ptr) {\n using namespace xgboost; // NOLINT\n using T = std::remove_pointer_t;\n\n auto m = linalg::MatrixView{\n common::Span{ptr,\n static_cast(ptr_mat_dims[0]) * static_cast(ptr_mat_dims[1])},\n {ptr_mat_dims[0], ptr_mat_dims[1]}, // Shape\n DeviceOrd::CPU(),\n linalg::Order::kF // R uses column-major\n };\n CHECK(m.FContiguous());\n return linalg::ArrayInterfaceStr(m);\n };\n\n const SEXPTYPE arr_type = TYPEOF(R_mat);\n switch (arr_type) {\n case REALSXP:\n return make_matrix(REAL(R_mat));\n case INTSXP:\n return make_matrix(INTEGER(R_mat));\n case LGLSXP:\n return make_matrix(LOGICAL(R_mat));\n default:\n LOG(FATAL) << \"Array or matrix has unsupported type.\";\n }\n\n LOG(FATAL) << \"Not reachable\";\n return \"\";\n}\n\n[[nodiscard]] std::string MakeArrayInterfaceFromRVector(SEXP R_vec) {\n const size_t vec_len = Rf_xlength(R_vec);\n\n // Lambda for type dispatch.\n auto make_vec = [=](auto const *ptr) {\n using namespace xgboost; // NOLINT\n auto v = linalg::MakeVec(ptr, vec_len);\n return linalg::ArrayInterfaceStr(v);\n };\n\n const SEXPTYPE arr_type = TYPEOF(R_vec);\n switch (arr_type) {\n case REALSXP:\n return make_vec(REAL(R_vec));\n case INTSXP:\n return make_vec(INTEGER(R_vec));\n case LGLSXP:\n return make_vec(LOGICAL(R_vec));\n default:\n LOG(FATAL) << \"Array or matrix has unsupported type.\";\n }\n\n LOG(FATAL) << \"Not reachable\";\n return \"\";\n}\n\n[[nodiscard]] std::string MakeArrayInterfaceFromRDataFrame(SEXP R_df) {\n auto make_vec = [&](auto const *ptr, std::size_t len) {\n auto v = xgboost::linalg::MakeVec(ptr, len);\n return xgboost::linalg::ArrayInterface(v);\n };\n\n R_xlen_t n_features = Rf_xlength(R_df);\n std::vector array(n_features);\n CHECK_GT(n_features, 0);\n std::size_t len = Rf_xlength(VECTOR_ELT(R_df, 0));\n\n // The `data.frame` in R actually converts all data into numeric. The other type\n // handlers here are not used. At the moment they are kept as a reference for when we\n // can avoid making data copies during transformation.\n for (R_xlen_t i = 0; i < n_features; ++i) {\n switch (TYPEOF(VECTOR_ELT(R_df, i))) {\n case INTSXP: {\n auto const *ptr = INTEGER(VECTOR_ELT(R_df, i));\n array[i] = make_vec(ptr, len);\n break;\n }\n case REALSXP: {\n auto const *ptr = REAL(VECTOR_ELT(R_df, i));\n array[i] = make_vec(ptr, len);\n break;\n }\n case LGLSXP: {\n auto const *ptr = LOGICAL(VECTOR_ELT(R_df, i));\n array[i] = make_vec(ptr, len);\n break;\n }\n default: {\n LOG(FATAL) << \"data.frame has unsupported type.\";\n }\n }\n }\n\n xgboost::Json jinterface{std::move(array)};\n return xgboost::Json::Dump(jinterface);\n}\n\nvoid AddMissingToJson(xgboost::Json *jconfig, SEXP missing, SEXPTYPE arr_type) {\n if (Rf_isNull(missing) || ISNAN(Rf_asReal(missing))) {\n // missing is not specified\n if (arr_type == REALSXP) {\n (*jconfig)[\"missing\"] = std::numeric_limits::quiet_NaN();\n } else {\n (*jconfig)[\"missing\"] = R_NaInt;\n }\n } else {\n // missing specified\n (*jconfig)[\"missing\"] = Rf_asReal(missing);\n }\n}\n\n[[nodiscard]] std::string MakeJsonConfigForArray(SEXP missing, SEXP n_threads, SEXPTYPE arr_type) {\n using namespace ::xgboost; // NOLINT\n Json jconfig{Object{}};\n AddMissingToJson(&jconfig, missing, arr_type);\n jconfig[\"nthread\"] = Rf_asInteger(n_threads);\n return Json::Dump(jconfig);\n}\n\n// Allocate a R vector and copy an array interface encoded object to it.\n[[nodiscard]] SEXP CopyArrayToR(const char *array_str, SEXP ctoken) {\n xgboost::ArrayInterface<1> array{xgboost::StringView{array_str}};\n // R supports only int and double.\n bool is_int_type =\n xgboost::DispatchDType(array.type, [](auto t) { return std::is_integral_v; });\n bool is_float = xgboost::DispatchDType(\n array.type, [](auto v) { return std::is_floating_point_v; });\n CHECK(is_int_type || is_float) << \"Internal error: Invalid DType.\";\n CHECK(array.is_contiguous) << \"Internal error: Return by XGBoost should be contiguous\";\n\n // Note: the only case in which this will receive an integer type is\n // for the 'indptr' part of the quantile cut outputs, which comes\n // in sorted order, so the last element contains the maximum value.\n bool fits_into_C_int = xgboost::DispatchDType(array.type, [&](auto t) {\n using T = decltype(t);\n if (!std::is_integral_v) {\n return false;\n }\n auto ptr = static_cast(array.data);\n T last_elt = ptr[array.n - 1];\n if (last_elt < 0) {\n last_elt = -last_elt; // no std::abs overload for all possible types\n }\n return last_elt <= std::numeric_limits::max();\n });\n bool use_int = is_int_type && fits_into_C_int;\n\n // Allocate memory in R\n SEXP out =\n Rf_protect(use_int ? SafeAllocInteger(array.n, ctoken) : SafeAllocReal(array.n, ctoken));\n\n xgboost::DispatchDType(array.type, [&](auto t) {\n using T = decltype(t);\n auto in_ptr = static_cast(array.data);\n if (use_int) {\n auto out_ptr = INTEGER(out);\n std::copy_n(in_ptr, array.n, out_ptr);\n } else {\n auto out_ptr = REAL(out);\n std::copy_n(in_ptr, array.n, out_ptr);\n }\n });\n\n Rf_unprotect(1);\n return out;\n}\n} // namespace\n\n/*!\n * \\brief macro to annotate begin of api\n */\n#define R_API_BEGIN() \\\n try { \\\n/* Note: an R error triggers a long jump, hence all C++ objects that\nallocated memory through non-R allocators, including the exception\nobject, need to be destructed before triggering the R error.\nIn order to preserve the error message, it gets copied to a temporary\nbuffer, and the R error section is reached through a 'goto' statement\nthat bypasses usual function control flow. */\nnamespace {\nconstexpr std::size_t MsgSize = 512;\nchar cpp_ex_msg[MsgSize];\n} // anonymous namespace\n\n/*!\n * \\brief macro to annotate end of api\n */\n#define R_API_END() \\\n } \\\n catch (std::exception & e) { \\\n cpp_ex_msg[MsgSize - 1] = 0; \\\n std::strncpy(cpp_ex_msg, e.what(), MsgSize - 1); \\\n goto throw_cpp_ex_as_R_err; \\\n } \\\n if (false) { \\\n throw_cpp_ex_as_R_err: \\\n Rf_error(\"%s\", cpp_ex_msg); \\\n }\n\n/**\n * @brief Macro for checking XGBoost return code.\n */\n#define CHECK_CALL(__rc) \\\n if ((__rc) != 0) { \\\n Rf_error(\"%s\", XGBGetLastError()); \\\n }\n\nusing dmlc::BeginPtr;\n\nXGB_DLL SEXP XGCheckNullPtr_R(SEXP handle) {\n return Rf_ScalarLogical(R_ExternalPtrAddr(handle) == nullptr);\n}\n\nXGB_DLL SEXP XGSetArrayDimNamesInplace_R(SEXP arr, SEXP dim_names) {\n Rf_setAttrib(arr, R_DimNamesSymbol, dim_names);\n return R_NilValue;\n}\n\nXGB_DLL SEXP XGSetVectorNamesInplace_R(SEXP arr, SEXP names) {\n Rf_setAttrib(arr, R_NamesSymbol, names);\n return R_NilValue;\n}\n\nnamespace {\nvoid _DMatrixFinalizer(SEXP ext) {\n R_API_BEGIN();\n if (R_ExternalPtrAddr(ext) == NULL) return;\n CHECK_CALL(XGDMatrixFree(R_ExternalPtrAddr(ext)));\n R_ClearExternalPtr(ext);\n R_API_END();\n}\n} /* namespace */\n\nXGB_DLL SEXP XGBSetGlobalConfig_R(SEXP json_str) {\n R_API_BEGIN();\n CHECK_CALL(XGBSetGlobalConfig(CHAR(Rf_asChar(json_str))));\n R_API_END();\n return R_NilValue;\n}\n\nXGB_DLL SEXP XGBGetGlobalConfig_R() {\n const char* json_str;\n R_API_BEGIN();\n CHECK_CALL(XGBGetGlobalConfig(&json_str));\n R_API_END();\n return Rf_mkString(json_str);\n}\n\nXGB_DLL SEXP XGDMatrixCreateFromURI_R(SEXP uri, SEXP silent, SEXP data_split_mode) {\n SEXP ret = Rf_protect(R_MakeExternalPtr(nullptr, R_NilValue, R_NilValue));\n SEXP uri_char = Rf_protect(Rf_asChar(uri));\n const char *uri_ptr = CHAR(uri_char);\n R_API_BEGIN();\n xgboost::Json jconfig{xgboost::Object{}};\n jconfig[\"uri\"] = std::string(uri_ptr);\n jconfig[\"silent\"] = Rf_asLogical(silent);\n jconfig[\"data_split_mode\"] = Rf_asInteger(data_split_mode);\n const std::string sconfig = xgboost::Json::Dump(jconfig);\n DMatrixHandle handle;\n CHECK_CALL(XGDMatrixCreateFromURI(sconfig.c_str(), &handle));\n R_SetExternalPtrAddr(ret, handle);\n R_RegisterCFinalizerEx(ret, _DMatrixFinalizer, TRUE);\n R_API_END();\n Rf_unprotect(2);\n return ret;\n}\n\nXGB_DLL SEXP XGDMatrixCreateFromMat_R(SEXP mat, SEXP missing, SEXP n_threads) {\n SEXP ret = Rf_protect(R_MakeExternalPtr(nullptr, R_NilValue, R_NilValue));\n R_API_BEGIN();\n\n DMatrixHandle handle;\n int res_code;\n {\n auto array_str = MakeArrayInterfaceFromRMat(mat);\n auto config_str = MakeJsonConfigForArray(missing, n_threads, TYPEOF(mat));\n\n res_code = XGDMatrixCreateFromDense(array_str.c_str(), config_str.c_str(), &handle);\n }\n CHECK_CALL(res_code);\n R_SetExternalPtrAddr(ret, handle);\n R_RegisterCFinalizerEx(ret, _DMatrixFinalizer, TRUE);\n R_API_END();\n Rf_unprotect(1);\n return ret;\n}\n\nXGB_DLL SEXP XGDMatrixCreateFromDF_R(SEXP df, SEXP missing, SEXP n_threads) {\n SEXP ret = Rf_protect(R_MakeExternalPtr(nullptr, R_NilValue, R_NilValue));\n R_API_BEGIN();\n\n DMatrixHandle handle;\n std::int32_t rc{0};\n {\n const std::string sinterface = MakeArrayInterfaceFromRDataFrame(df);\n xgboost::Json jconfig{xgboost::Object{}};\n jconfig[\"missing\"] = Rf_asReal(missing);\n jconfig[\"nthread\"] = Rf_asInteger(n_threads);\n std::string sconfig = xgboost::Json::Dump(jconfig);\n\n rc = XGDMatrixCreateFromColumnar(sinterface.c_str(), sconfig.c_str(), &handle);\n }\n\n CHECK_CALL(rc);\n R_SetExternalPtrAddr(ret, handle);\n R_RegisterCFinalizerEx(ret, _DMatrixFinalizer, TRUE);\n R_API_END();\n Rf_unprotect(1);\n\n return ret;\n}\n\nnamespace {\nvoid CreateFromSparse(SEXP indptr, SEXP indices, SEXP data, std::string *indptr_str,\n std::string *indices_str, std::string *data_str) {\n const int *p_indptr = INTEGER(indptr);\n const int *p_indices = INTEGER(indices);\n const double *p_data = REAL(data);\n\n auto nindptr = static_cast(Rf_xlength(indptr));\n auto ndata = static_cast(Rf_xlength(data));\n CHECK_EQ(ndata, p_indptr[nindptr - 1]);\n xgboost::detail::MakeSparseFromPtr(p_indptr, p_indices, p_data, nindptr, indptr_str, indices_str,\n data_str);\n}\n} // namespace\n\nXGB_DLL SEXP XGDMatrixCreateFromCSC_R(SEXP indptr, SEXP indices, SEXP data, SEXP num_row,\n SEXP missing, SEXP n_threads) {\n SEXP ret = Rf_protect(R_MakeExternalPtr(nullptr, R_NilValue, R_NilValue));\n R_API_BEGIN();\n std::int32_t threads = Rf_asInteger(n_threads);\n DMatrixHandle handle;\n\n int res_code;\n {\n using xgboost::Integer;\n using xgboost::Json;\n using xgboost::Object;\n std::string sindptr, sindices, sdata;\n CreateFromSparse(indptr, indices, data, &sindptr, &sindices, &sdata);\n auto nrow = static_cast(INTEGER(num_row)[0]);\n\n Json jconfig{Object{}};\n // Construct configuration\n jconfig[\"nthread\"] = Integer{threads};\n AddMissingToJson(&jconfig, missing, TYPEOF(data));\n std::string config;\n Json::Dump(jconfig, &config);\n res_code = XGDMatrixCreateFromCSC(sindptr.c_str(), sindices.c_str(), sdata.c_str(), nrow,\n config.c_str(), &handle);\n }\n CHECK_CALL(res_code);\n\n R_SetExternalPtrAddr(ret, handle);\n R_RegisterCFinalizerEx(ret, _DMatrixFinalizer, TRUE);\n R_API_END();\n Rf_unprotect(1);\n return ret;\n}\n\nXGB_DLL SEXP XGDMatrixCreateFromCSR_R(SEXP indptr, SEXP indices, SEXP data, SEXP num_col,\n SEXP missing, SEXP n_threads) {\n SEXP ret = Rf_protect(R_MakeExternalPtr(nullptr, R_NilValue, R_NilValue));\n R_API_BEGIN();\n std::int32_t threads = Rf_asInteger(n_threads);\n DMatrixHandle handle;\n\n int res_code;\n {\n using xgboost::Integer;\n using xgboost::Json;\n using xgboost::Object;\n\n std::string sindptr, sindices, sdata;\n CreateFromSparse(indptr, indices, data, &sindptr, &sindices, &sdata);\n auto ncol = static_cast(INTEGER(num_col)[0]);\n\n Json jconfig{Object{}};\n // Construct configuration\n jconfig[\"nthread\"] = Integer{threads};\n AddMissingToJson(&jconfig, missing, TYPEOF(data));\n std::string config;\n Json::Dump(jconfig, &config);\n res_code = XGDMatrixCreateFromCSR(sindptr.c_str(), sindices.c_str(), sdata.c_str(), ncol,\n config.c_str(), &handle);\n }\n CHECK_CALL(res_code);\n R_SetExternalPtrAddr(ret, handle);\n R_RegisterCFinalizerEx(ret, _DMatrixFinalizer, TRUE);\n R_API_END();\n Rf_unprotect(1);\n return ret;\n}\n\nXGB_DLL SEXP XGDMatrixSliceDMatrix_R(SEXP handle, SEXP idxset, SEXP allow_groups) {\n SEXP ret = Rf_protect(R_MakeExternalPtr(nullptr, R_NilValue, R_NilValue));\n R_API_BEGIN();\n R_xlen_t len = Rf_xlength(idxset);\n const int *idxset_ = INTEGER(idxset);\n DMatrixHandle res;\n\n int res_code;\n {\n std::vector idxvec(len);\n #ifndef _MSC_VER\n #pragma omp simd\n #endif\n for (R_xlen_t i = 0; i < len; ++i) {\n idxvec[i] = idxset_[i] - 1;\n }\n res_code = XGDMatrixSliceDMatrixEx(R_ExternalPtrAddr(handle),\n BeginPtr(idxvec), len,\n &res,\n Rf_asLogical(allow_groups));\n }\n CHECK_CALL(res_code);\n R_SetExternalPtrAddr(ret, res);\n R_RegisterCFinalizerEx(ret, _DMatrixFinalizer, TRUE);\n R_API_END();\n Rf_unprotect(1);\n return ret;\n}\n\nXGB_DLL SEXP XGDMatrixSaveBinary_R(SEXP handle, SEXP fname, SEXP silent) {\n R_API_BEGIN();\n CHECK_CALL(XGDMatrixSaveBinary(R_ExternalPtrAddr(handle),\n CHAR(Rf_asChar(fname)),\n Rf_asInteger(silent)));\n R_API_END();\n return R_NilValue;\n}\n\nXGB_DLL SEXP XGDMatrixSetInfo_R(SEXP handle, SEXP field, SEXP array) {\n R_API_BEGIN();\n SEXP field_ = Rf_protect(Rf_asChar(field));\n SEXP arr_dim = Rf_getAttrib(array, R_DimSymbol);\n int res_code;\n {\n const std::string array_str = Rf_isNull(arr_dim)?\n MakeArrayInterfaceFromRVector(array) : MakeArrayInterfaceFromRMat(array);\n res_code = XGDMatrixSetInfoFromInterface(\n R_ExternalPtrAddr(handle), CHAR(field_), array_str.c_str());\n }\n CHECK_CALL(res_code);\n Rf_unprotect(1);\n R_API_END();\n return R_NilValue;\n}\n\nXGB_DLL SEXP XGDMatrixSetStrFeatureInfo_R(SEXP handle, SEXP field, SEXP array) {\n R_API_BEGIN();\n size_t len{0};\n if (!Rf_isNull(array)) {\n len = Rf_xlength(array);\n }\n\n SEXP str_info_holder = Rf_protect(Rf_allocVector(VECSXP, len));\n if (TYPEOF(array) == STRSXP) {\n for (size_t i = 0; i < len; ++i) {\n SET_VECTOR_ELT(str_info_holder, i, STRING_ELT(array, i));\n }\n } else {\n for (size_t i = 0; i < len; ++i) {\n SET_VECTOR_ELT(str_info_holder, i, Rf_asChar(VECTOR_ELT(array, i)));\n }\n }\n\n SEXP field_ = Rf_protect(Rf_asChar(field));\n const char *name = CHAR(field_);\n int res_code;\n {\n std::vector str_info;\n str_info.reserve(len);\n for (size_t i = 0; i < len; ++i) {\n str_info.emplace_back(CHAR(VECTOR_ELT(str_info_holder, i)));\n }\n std::vector vec(len);\n std::transform(str_info.cbegin(), str_info.cend(), vec.begin(),\n [](std::string const &str) { return str.c_str(); });\n res_code = XGDMatrixSetStrFeatureInfo(R_ExternalPtrAddr(handle), name, vec.data(), len);\n }\n CHECK_CALL(res_code);\n Rf_unprotect(2);\n R_API_END();\n return R_NilValue;\n}\n\nXGB_DLL SEXP XGDMatrixGetStrFeatureInfo_R(SEXP handle, SEXP field) {\n SEXP ret;\n R_API_BEGIN();\n char const **out_features{nullptr};\n bst_ulong len{0};\n const char *name = CHAR(Rf_asChar(field));\n XGDMatrixGetStrFeatureInfo(R_ExternalPtrAddr(handle), name, &len, &out_features);\n\n if (len > 0) {\n ret = Rf_protect(Rf_allocVector(STRSXP, len));\n for (size_t i = 0; i < len; ++i) {\n SET_STRING_ELT(ret, i, Rf_mkChar(out_features[i]));\n }\n } else {\n ret = Rf_protect(R_NilValue);\n }\n R_API_END();\n Rf_unprotect(1);\n return ret;\n}\n\nXGB_DLL SEXP XGDMatrixGetFloatInfo_R(SEXP handle, SEXP field) {\n SEXP ret;\n R_API_BEGIN();\n bst_ulong olen;\n const float *res;\n CHECK_CALL(XGDMatrixGetFloatInfo(R_ExternalPtrAddr(handle), CHAR(Rf_asChar(field)), &olen, &res));\n ret = Rf_protect(Rf_allocVector(REALSXP, olen));\n std::copy(res, res + olen, REAL(ret));\n R_API_END();\n Rf_unprotect(1);\n return ret;\n}\n\nXGB_DLL SEXP XGDMatrixGetUIntInfo_R(SEXP handle, SEXP field) {\n SEXP ret;\n R_API_BEGIN();\n bst_ulong olen;\n const unsigned *res;\n CHECK_CALL(XGDMatrixGetUIntInfo(R_ExternalPtrAddr(handle), CHAR(Rf_asChar(field)), &olen, &res));\n ret = Rf_protect(Rf_allocVector(INTSXP, olen));\n std::copy(res, res + olen, INTEGER(ret));\n R_API_END();\n Rf_unprotect(1);\n return ret;\n}\n\nXGB_DLL SEXP XGDMatrixNumRow_R(SEXP handle) {\n bst_ulong nrow;\n R_API_BEGIN();\n CHECK_CALL(XGDMatrixNumRow(R_ExternalPtrAddr(handle), &nrow));\n R_API_END();\n return Rf_ScalarInteger(static_cast(nrow));\n}\n\nXGB_DLL SEXP XGDMatrixNumCol_R(SEXP handle) {\n bst_ulong ncol;\n R_API_BEGIN();\n CHECK_CALL(XGDMatrixNumCol(R_ExternalPtrAddr(handle), &ncol));\n R_API_END();\n return Rf_ScalarInteger(static_cast(ncol));\n}\n\nXGB_DLL SEXP XGProxyDMatrixCreate_R() {\n SEXP out = Rf_protect(R_MakeExternalPtr(nullptr, R_NilValue, R_NilValue));\n R_API_BEGIN();\n DMatrixHandle proxy_dmat_handle;\n CHECK_CALL(XGProxyDMatrixCreate(&proxy_dmat_handle));\n R_SetExternalPtrAddr(out, proxy_dmat_handle);\n R_RegisterCFinalizerEx(out, _DMatrixFinalizer, TRUE);\n R_API_END();\n Rf_unprotect(1);\n return out;\n}\n\nXGB_DLL SEXP XGProxyDMatrixSetDataDense_R(SEXP handle, SEXP R_mat) {\n R_API_BEGIN();\n DMatrixHandle proxy_dmat = R_ExternalPtrAddr(handle);\n int res_code;\n {\n std::string array_str = MakeArrayInterfaceFromRMat(R_mat);\n res_code = XGProxyDMatrixSetDataDense(proxy_dmat, array_str.c_str());\n }\n CHECK_CALL(res_code);\n R_API_END();\n return R_NilValue;\n}\n\nXGB_DLL SEXP XGProxyDMatrixSetDataCSR_R(SEXP handle, SEXP lst) {\n R_API_BEGIN();\n DMatrixHandle proxy_dmat = R_ExternalPtrAddr(handle);\n int res_code;\n {\n std::string array_str_indptr = MakeArrayInterfaceFromRVector(VECTOR_ELT(lst, 0));\n std::string array_str_indices = MakeArrayInterfaceFromRVector(VECTOR_ELT(lst, 1));\n std::string array_str_data = MakeArrayInterfaceFromRVector(VECTOR_ELT(lst, 2));\n const int ncol = Rf_asInteger(VECTOR_ELT(lst, 3));\n res_code = XGProxyDMatrixSetDataCSR(proxy_dmat,\n array_str_indptr.c_str(),\n array_str_indices.c_str(),\n array_str_data.c_str(),\n ncol);\n }\n CHECK_CALL(res_code);\n R_API_END();\n return R_NilValue;\n}\n\nXGB_DLL SEXP XGProxyDMatrixSetDataColumnar_R(SEXP handle, SEXP lst) {\n R_API_BEGIN();\n DMatrixHandle proxy_dmat = R_ExternalPtrAddr(handle);\n int res_code;\n {\n std::string sinterface = MakeArrayInterfaceFromRDataFrame(lst);\n res_code = XGProxyDMatrixSetDataColumnar(proxy_dmat, sinterface.c_str());\n }\n CHECK_CALL(res_code);\n R_API_END();\n return R_NilValue;\n}\n\nnamespace {\n\nstruct _RDataIterator {\n SEXP f_next;\n SEXP f_reset;\n SEXP calling_env;\n SEXP continuation_token;\n\n _RDataIterator(\n SEXP f_next, SEXP f_reset, SEXP calling_env, SEXP continuation_token) :\n f_next(f_next), f_reset(f_reset), calling_env(calling_env),\n continuation_token(continuation_token) {}\n\n void reset() {\n SafeExecFun(this->f_reset, this->calling_env, this->continuation_token);\n }\n\n int next() {\n SEXP R_res = Rf_protect(\n SafeExecFun(this->f_next, this->calling_env, this->continuation_token));\n int res = Rf_asInteger(R_res);\n Rf_unprotect(1);\n return res;\n }\n};\n\nvoid _reset_RDataIterator(DataIterHandle iter) {\n static_cast<_RDataIterator*>(iter)->reset();\n}\n\nint _next_RDataIterator(DataIterHandle iter) {\n return static_cast<_RDataIterator*>(iter)->next();\n}\n\nSEXP XGDMatrixCreateFromCallbackGeneric_R(\n SEXP f_next, SEXP f_reset, SEXP calling_env, SEXP proxy_dmat,\n SEXP n_threads, SEXP missing, SEXP max_bin, SEXP ref_dmat,\n SEXP cache_prefix, bool as_quantile_dmatrix) {\n SEXP continuation_token = Rf_protect(R_MakeUnwindCont());\n SEXP out = Rf_protect(R_MakeExternalPtr(nullptr, R_NilValue, R_NilValue));\n R_API_BEGIN();\n DMatrixHandle out_dmat;\n\n int res_code;\n try {\n _RDataIterator data_iterator(f_next, f_reset, calling_env, continuation_token);\n\n std::string str_cache_prefix;\n xgboost::Json jconfig{xgboost::Object{}};\n jconfig[\"missing\"] = Rf_asReal(missing);\n if (!Rf_isNull(n_threads)) {\n jconfig[\"nthread\"] = Rf_asInteger(n_threads);\n }\n if (as_quantile_dmatrix) {\n if (!Rf_isNull(max_bin)) {\n jconfig[\"max_bin\"] = Rf_asInteger(max_bin);\n }\n } else {\n str_cache_prefix = std::string(CHAR(Rf_asChar(cache_prefix)));\n jconfig[\"cache_prefix\"] = str_cache_prefix;\n }\n std::string json_str = xgboost::Json::Dump(jconfig);\n\n DMatrixHandle ref_dmat_handle = nullptr;\n if (as_quantile_dmatrix && !Rf_isNull(ref_dmat)) {\n ref_dmat_handle = R_ExternalPtrAddr(ref_dmat);\n }\n\n if (as_quantile_dmatrix) {\n res_code = XGQuantileDMatrixCreateFromCallback(\n &data_iterator,\n R_ExternalPtrAddr(proxy_dmat),\n ref_dmat_handle,\n _reset_RDataIterator,\n _next_RDataIterator,\n json_str.c_str(),\n &out_dmat);\n } else {\n res_code = XGDMatrixCreateFromCallback(\n &data_iterator,\n R_ExternalPtrAddr(proxy_dmat),\n _reset_RDataIterator,\n _next_RDataIterator,\n json_str.c_str(),\n &out_dmat);\n }\n } catch (ErrorWithUnwind &e) {\n R_ContinueUnwind(continuation_token);\n }\n CHECK_CALL(res_code);\n\n R_SetExternalPtrAddr(out, out_dmat);\n R_RegisterCFinalizerEx(out, _DMatrixFinalizer, TRUE);\n R_API_END();\n Rf_unprotect(2);\n return out;\n}\n\n} /* namespace */\n\nXGB_DLL SEXP XGQuantileDMatrixCreateFromCallback_R(\n SEXP f_next, SEXP f_reset, SEXP calling_env, SEXP proxy_dmat,\n SEXP n_threads, SEXP missing, SEXP max_bin, SEXP ref_dmat) {\n return XGDMatrixCreateFromCallbackGeneric_R(\n f_next, f_reset, calling_env, proxy_dmat,\n n_threads, missing, max_bin, ref_dmat,\n R_NilValue, true);\n}\n\nXGB_DLL SEXP XGDMatrixCreateFromCallback_R(\n SEXP f_next, SEXP f_reset, SEXP calling_env, SEXP proxy_dmat,\n SEXP n_threads, SEXP missing, SEXP cache_prefix) {\n return XGDMatrixCreateFromCallbackGeneric_R(\n f_next, f_reset, calling_env, proxy_dmat,\n n_threads, missing, R_NilValue, R_NilValue,\n cache_prefix, false);\n}\n\nXGB_DLL SEXP XGDMatrixFree_R(SEXP proxy_dmat) {\n _DMatrixFinalizer(proxy_dmat);\n return R_NilValue;\n}\n\nXGB_DLL SEXP XGGetRNAIntAsDouble() {\n double sentinel_as_double = static_cast(R_NaInt);\n return Rf_ScalarReal(sentinel_as_double);\n}\n\nXGB_DLL SEXP XGDuplicate_R(SEXP obj) {\n return Rf_duplicate(obj);\n}\n\nXGB_DLL SEXP XGPointerEqComparison_R(SEXP obj1, SEXP obj2) {\n return Rf_ScalarLogical(R_ExternalPtrAddr(obj1) == R_ExternalPtrAddr(obj2));\n}\n\nXGB_DLL SEXP XGDMatrixGetQuantileCut_R(SEXP handle) {\n const char *out_names[] = {\"indptr\", \"data\", \"\"};\n SEXP continuation_token = Rf_protect(R_MakeUnwindCont());\n SEXP out = Rf_protect(Rf_mkNamed(VECSXP, out_names));\n R_API_BEGIN();\n const char *out_indptr;\n const char *out_data;\n CHECK_CALL(XGDMatrixGetQuantileCut(R_ExternalPtrAddr(handle), \"{}\", &out_indptr, &out_data));\n try {\n SET_VECTOR_ELT(out, 0, CopyArrayToR(out_indptr, continuation_token));\n SET_VECTOR_ELT(out, 1, CopyArrayToR(out_data, continuation_token));\n } catch (ErrorWithUnwind &e) {\n R_ContinueUnwind(continuation_token);\n }\n R_API_END();\n Rf_unprotect(2);\n return out;\n}\n\nXGB_DLL SEXP XGDMatrixNumNonMissing_R(SEXP handle) {\n SEXP out = Rf_protect(Rf_allocVector(REALSXP, 1));\n R_API_BEGIN();\n bst_ulong out_;\n CHECK_CALL(XGDMatrixNumNonMissing(R_ExternalPtrAddr(handle), &out_));\n REAL(out)[0] = static_cast(out_);\n R_API_END();\n Rf_unprotect(1);\n return out;\n}\n\nXGB_DLL SEXP XGDMatrixGetDataAsCSR_R(SEXP handle) {\n const char *out_names[] = {\"indptr\", \"indices\", \"data\", \"ncols\", \"\"};\n SEXP out = Rf_protect(Rf_mkNamed(VECSXP, out_names));\n R_API_BEGIN();\n\n bst_ulong nrows, ncols, nnz;\n CHECK_CALL(XGDMatrixNumRow(R_ExternalPtrAddr(handle), &nrows));\n CHECK_CALL(XGDMatrixNumCol(R_ExternalPtrAddr(handle), &ncols));\n CHECK_CALL(XGDMatrixNumNonMissing(R_ExternalPtrAddr(handle), &nnz));\n if (std::max(nrows, ncols) > std::numeric_limits::max()) {\n Rf_error(\"%s\", \"Error: resulting DMatrix data does not fit into R 'dgRMatrix'.\");\n }\n\n SET_VECTOR_ELT(out, 0, Rf_allocVector(INTSXP, nrows + 1));\n SET_VECTOR_ELT(out, 1, Rf_allocVector(INTSXP, nnz));\n SET_VECTOR_ELT(out, 2, Rf_allocVector(REALSXP, nnz));\n SET_VECTOR_ELT(out, 3, Rf_ScalarInteger(ncols));\n\n std::unique_ptr indptr(new bst_ulong[nrows + 1]);\n std::unique_ptr indices(new unsigned[nnz]);\n std::unique_ptr data(new float[nnz]);\n\n CHECK_CALL(XGDMatrixGetDataAsCSR(R_ExternalPtrAddr(handle),\n \"{}\",\n indptr.get(),\n indices.get(),\n data.get()));\n\n std::copy(indptr.get(), indptr.get() + nrows + 1, INTEGER(VECTOR_ELT(out, 0)));\n std::copy(indices.get(), indices.get() + nnz, INTEGER(VECTOR_ELT(out, 1)));\n std::copy(data.get(), data.get() + nnz, REAL(VECTOR_ELT(out, 2)));\n\n R_API_END();\n Rf_unprotect(1);\n return out;\n}\n\n// functions related to booster\nnamespace {\nvoid _BoosterFinalizer(SEXP R_ptr) {\n if (R_ExternalPtrAddr(R_ptr) == NULL) return;\n CHECK_CALL(XGBoosterFree(R_ExternalPtrAddr(R_ptr)));\n R_ClearExternalPtr(R_ptr);\n}\n\n/* Booster is represented as an altrep list with one element which\ncorresponds to an 'externalptr' holding the C object, forbidding\nmodification by not implementing setters, and adding custom serialization. */\nR_altrep_class_t XGBAltrepPointerClass;\n\nR_xlen_t XGBAltrepPointerLength_R(SEXP R_altrepped_obj) {\n return 1;\n}\n\nSEXP XGBAltrepPointerGetElt_R(SEXP R_altrepped_obj, R_xlen_t idx) {\n return R_altrep_data1(R_altrepped_obj);\n}\n\nSEXP XGBMakeEmptyAltrep() {\n SEXP class_name = Rf_protect(Rf_mkString(\"xgb.Booster\"));\n SEXP elt_names = Rf_protect(Rf_mkString(\"ptr\"));\n SEXP R_ptr = Rf_protect(R_MakeExternalPtr(nullptr, R_NilValue, R_NilValue));\n SEXP R_altrepped_obj = Rf_protect(R_new_altrep(XGBAltrepPointerClass, R_ptr, R_NilValue));\n Rf_setAttrib(R_altrepped_obj, R_NamesSymbol, elt_names);\n Rf_setAttrib(R_altrepped_obj, R_ClassSymbol, class_name);\n Rf_unprotect(4);\n return R_altrepped_obj;\n}\n\n/* Note: the idea for separating this function from the one above is to be\nable to trigger all R allocations first before doing non-R allocations. */\nvoid XGBAltrepSetPointer(SEXP R_altrepped_obj, BoosterHandle handle) {\n SEXP R_ptr = R_altrep_data1(R_altrepped_obj);\n R_SetExternalPtrAddr(R_ptr, handle);\n R_RegisterCFinalizerEx(R_ptr, _BoosterFinalizer, TRUE);\n}\n\nSEXP XGBAltrepSerializer_R(SEXP R_altrepped_obj) {\n R_API_BEGIN();\n BoosterHandle handle = R_ExternalPtrAddr(R_altrep_data1(R_altrepped_obj));\n char const *serialized_bytes;\n bst_ulong serialized_length;\n CHECK_CALL(XGBoosterSerializeToBuffer(\n handle, &serialized_length, &serialized_bytes));\n SEXP R_state = Rf_protect(Rf_allocVector(RAWSXP, serialized_length));\n if (serialized_length != 0) {\n std::memcpy(RAW(R_state), serialized_bytes, serialized_length);\n }\n Rf_unprotect(1);\n return R_state;\n R_API_END();\n return R_NilValue; /* <- should not be reached */\n}\n\nSEXP XGBAltrepDeserializer_R(SEXP /*unused*/, SEXP R_state) {\n SEXP R_altrepped_obj = Rf_protect(XGBMakeEmptyAltrep());\n R_API_BEGIN();\n BoosterHandle handle = nullptr;\n CHECK_CALL(XGBoosterCreate(nullptr, 0, &handle));\n int res_code = XGBoosterUnserializeFromBuffer(handle,\n RAW(R_state),\n Rf_xlength(R_state));\n if (res_code != 0) {\n XGBoosterFree(handle);\n }\n CHECK_CALL(res_code);\n XGBAltrepSetPointer(R_altrepped_obj, handle);\n R_API_END();\n Rf_unprotect(1);\n return R_altrepped_obj;\n}\n\n// https://purrple.cat/blog/2018/10/14/altrep-and-cpp/\nRboolean XGBAltrepInspector_R(\n SEXP x, int pre, int deep, int pvec,\n void (*inspect_subtree)(SEXP, int, int, int)) {\n Rprintf(\"Altrepped external pointer [address:%p]\\n\",\n R_ExternalPtrAddr(R_altrep_data1(x)));\n return TRUE;\n}\n\nSEXP XGBAltrepDuplicate_R(SEXP R_altrepped_obj, Rboolean deep) {\n R_API_BEGIN();\n if (!deep) {\n SEXP out = Rf_protect(XGBMakeEmptyAltrep());\n R_set_altrep_data1(out, R_altrep_data1(R_altrepped_obj));\n Rf_unprotect(1);\n return out;\n } else {\n SEXP out = Rf_protect(XGBMakeEmptyAltrep());\n char const *serialized_bytes;\n bst_ulong serialized_length;\n CHECK_CALL(XGBoosterSerializeToBuffer(\n R_ExternalPtrAddr(R_altrep_data1(R_altrepped_obj)),\n &serialized_length, &serialized_bytes));\n BoosterHandle new_handle = nullptr;\n CHECK_CALL(XGBoosterCreate(nullptr, 0, &new_handle));\n int res_code = XGBoosterUnserializeFromBuffer(new_handle,\n serialized_bytes,\n serialized_length);\n if (res_code != 0) {\n XGBoosterFree(new_handle);\n }\n CHECK_CALL(res_code);\n XGBAltrepSetPointer(out, new_handle);\n Rf_unprotect(1);\n return out;\n }\n R_API_END();\n return R_NilValue; /* <- should not be reached */\n}\n\n} /* namespace */\n\nXGB_DLL void XGBInitializeAltrepClass_R(DllInfo *dll) {\n XGBAltrepPointerClass = R_make_altlist_class(\"XGBAltrepPointerClass\", \"xgboost\", dll);\n R_set_altrep_Length_method(XGBAltrepPointerClass, XGBAltrepPointerLength_R);\n R_set_altlist_Elt_method(XGBAltrepPointerClass, XGBAltrepPointerGetElt_R);\n R_set_altrep_Inspect_method(XGBAltrepPointerClass, XGBAltrepInspector_R);\n R_set_altrep_Serialized_state_method(XGBAltrepPointerClass, XGBAltrepSerializer_R);\n R_set_altrep_Unserialize_method(XGBAltrepPointerClass, XGBAltrepDeserializer_R);\n R_set_altrep_Duplicate_method(XGBAltrepPointerClass, XGBAltrepDuplicate_R);\n}\n\nXGB_DLL SEXP XGBoosterCreate_R(SEXP dmats) {\n SEXP out = Rf_protect(XGBMakeEmptyAltrep());\n R_API_BEGIN();\n R_xlen_t len = Rf_xlength(dmats);\n BoosterHandle handle;\n\n int res_code;\n {\n std::vector dvec(len);\n for (R_xlen_t i = 0; i < len; ++i) {\n dvec[i] = R_ExternalPtrAddr(VECTOR_ELT(dmats, i));\n }\n res_code = XGBoosterCreate(BeginPtr(dvec), dvec.size(), &handle);\n }\n CHECK_CALL(res_code);\n XGBAltrepSetPointer(out, handle);\n R_API_END();\n Rf_unprotect(1);\n return out;\n}\n\nXGB_DLL SEXP XGBoosterReset_R(SEXP handle) {\n R_API_BEGIN();\n CHECK_CALL(XGBoosterReset(R_ExternalPtrAddr(handle)));\n R_API_END();\n return R_NilValue;\n}\n\nXGB_DLL SEXP XGBoosterCopyInfoFromDMatrix_R(SEXP booster, SEXP dmat) {\n R_API_BEGIN();\n char const **feature_names;\n bst_ulong len_feature_names = 0;\n CHECK_CALL(XGDMatrixGetStrFeatureInfo(R_ExternalPtrAddr(dmat),\n \"feature_name\",\n &len_feature_names,\n &feature_names));\n if (len_feature_names) {\n CHECK_CALL(XGBoosterSetStrFeatureInfo(R_ExternalPtrAddr(booster),\n \"feature_name\",\n feature_names,\n len_feature_names));\n }\n\n char const **feature_types;\n bst_ulong len_feature_types = 0;\n CHECK_CALL(XGDMatrixGetStrFeatureInfo(R_ExternalPtrAddr(dmat),\n \"feature_type\",\n &len_feature_types,\n &feature_types));\n if (len_feature_types) {\n CHECK_CALL(XGBoosterSetStrFeatureInfo(R_ExternalPtrAddr(booster),\n \"feature_type\",\n feature_types,\n len_feature_types));\n }\n R_API_END();\n return R_NilValue;\n}\n\nXGB_DLL SEXP XGBoosterSetStrFeatureInfo_R(SEXP handle, SEXP field, SEXP features) {\n R_API_BEGIN();\n SEXP field_char = Rf_protect(Rf_asChar(field));\n bst_ulong len_features = Rf_xlength(features);\n\n int res_code;\n {\n std::vector str_arr(len_features);\n for (bst_ulong idx = 0; idx < len_features; idx++) {\n str_arr[idx] = CHAR(STRING_ELT(features, idx));\n }\n res_code = XGBoosterSetStrFeatureInfo(R_ExternalPtrAddr(handle),\n CHAR(field_char),\n str_arr.data(),\n len_features);\n }\n CHECK_CALL(res_code);\n Rf_unprotect(1);\n R_API_END();\n return R_NilValue;\n}\n\nXGB_DLL SEXP XGBoosterGetStrFeatureInfo_R(SEXP handle, SEXP field) {\n R_API_BEGIN();\n bst_ulong len;\n const char **out_features;\n SEXP field_char = Rf_protect(Rf_asChar(field));\n CHECK_CALL(XGBoosterGetStrFeatureInfo(R_ExternalPtrAddr(handle),\n CHAR(field_char), &len, &out_features));\n SEXP out = Rf_protect(Rf_allocVector(STRSXP, len));\n for (bst_ulong idx = 0; idx < len; idx++) {\n SET_STRING_ELT(out, idx, Rf_mkChar(out_features[idx]));\n }\n Rf_unprotect(2);\n return out;\n R_API_END();\n return R_NilValue; /* <- should not be reached */\n}\n\nXGB_DLL SEXP XGBoosterBoostedRounds_R(SEXP handle) {\n SEXP out = Rf_protect(Rf_allocVector(INTSXP, 1));\n R_API_BEGIN();\n CHECK_CALL(XGBoosterBoostedRounds(R_ExternalPtrAddr(handle), INTEGER(out)));\n R_API_END();\n Rf_unprotect(1);\n return out;\n}\n\n/* Note: R's integer class is 32-bit-and-signed only, while xgboost\nsupports more, so it returns it as a floating point instead */\nXGB_DLL SEXP XGBoosterGetNumFeature_R(SEXP handle) {\n SEXP out = Rf_protect(Rf_allocVector(REALSXP, 1));\n R_API_BEGIN();\n bst_ulong res;\n CHECK_CALL(XGBoosterGetNumFeature(R_ExternalPtrAddr(handle), &res));\n REAL(out)[0] = static_cast(res);\n R_API_END();\n Rf_unprotect(1);\n return out;\n}\n\nXGB_DLL SEXP XGBoosterSetParam_R(SEXP handle, SEXP name, SEXP val) {\n R_API_BEGIN();\n SEXP name_ = Rf_protect(Rf_asChar(name));\n SEXP val_ = Rf_protect(Rf_asChar(val));\n CHECK_CALL(XGBoosterSetParam(R_ExternalPtrAddr(handle),\n CHAR(name_),\n CHAR(val_)));\n Rf_unprotect(2);\n R_API_END();\n return R_NilValue;\n}\n\nXGB_DLL SEXP XGBoosterUpdateOneIter_R(SEXP handle, SEXP iter, SEXP dtrain) {\n R_API_BEGIN();\n CHECK_CALL(XGBoosterUpdateOneIter(R_ExternalPtrAddr(handle),\n Rf_asInteger(iter),\n R_ExternalPtrAddr(dtrain)));\n R_API_END();\n return R_NilValue;\n}\n\nXGB_DLL SEXP XGBoosterTrainOneIter_R(SEXP handle, SEXP dtrain, SEXP iter, SEXP grad, SEXP hess) {\n R_API_BEGIN();\n CHECK_EQ(Rf_xlength(grad), Rf_xlength(hess)) << \"gradient and hess must have same length.\";\n SEXP gdim = Rf_protect(Rf_getAttrib(grad, R_DimSymbol));\n SEXP hdim = Rf_protect(Rf_getAttrib(hess, R_DimSymbol));\n\n int res_code;\n {\n const std::string s_grad = Rf_isNull(gdim)?\n MakeArrayInterfaceFromRVector(grad) : MakeArrayInterfaceFromRMat(grad);\n const std::string s_hess = Rf_isNull(hdim)?\n MakeArrayInterfaceFromRVector(hess) : MakeArrayInterfaceFromRMat(hess);\n res_code = XGBoosterTrainOneIter(R_ExternalPtrAddr(handle), R_ExternalPtrAddr(dtrain),\n Rf_asInteger(iter), s_grad.c_str(), s_hess.c_str());\n }\n CHECK_CALL(res_code);\n\n Rf_unprotect(2);\n R_API_END();\n return R_NilValue;\n}\n\nXGB_DLL SEXP XGBoosterEvalOneIter_R(SEXP handle, SEXP iter, SEXP dmats, SEXP evnames) {\n const char *ret;\n R_API_BEGIN();\n CHECK_EQ(Rf_xlength(dmats), Rf_xlength(evnames))\n << \"dmats and evnams must have same length\";\n R_xlen_t len = Rf_xlength(dmats);\n SEXP evnames_lst = Rf_protect(Rf_allocVector(VECSXP, len));\n for (R_xlen_t i = 0; i < len; i++) {\n SET_VECTOR_ELT(evnames_lst, i, Rf_asChar(VECTOR_ELT(evnames, i)));\n }\n\n int res_code;\n {\n std::vector vec_dmats(len);\n std::vector vec_names;\n vec_names.reserve(len);\n std::vector vec_sptr(len);\n for (R_xlen_t i = 0; i < len; ++i) {\n vec_dmats[i] = R_ExternalPtrAddr(VECTOR_ELT(dmats, i));\n vec_names.emplace_back(CHAR(VECTOR_ELT(evnames_lst, i)));\n }\n for (R_xlen_t i = 0; i < len; ++i) {\n vec_sptr[i] = vec_names[i].c_str();\n }\n res_code = XGBoosterEvalOneIter(R_ExternalPtrAddr(handle),\n Rf_asInteger(iter),\n BeginPtr(vec_dmats),\n BeginPtr(vec_sptr),\n len, &ret);\n }\n CHECK_CALL(res_code);\n Rf_unprotect(1);\n R_API_END();\n return Rf_mkString(ret);\n}\n\nnamespace {\n\nstruct ProxyDmatrixError : public std::exception {};\n\nstruct ProxyDmatrixWrapper {\n DMatrixHandle proxy_dmat_handle;\n\n ProxyDmatrixWrapper() {\n int res_code = XGProxyDMatrixCreate(&this->proxy_dmat_handle);\n if (res_code != 0) {\n throw ProxyDmatrixError();\n }\n }\n\n ~ProxyDmatrixWrapper() {\n if (this->proxy_dmat_handle) {\n XGDMatrixFree(this->proxy_dmat_handle);\n this->proxy_dmat_handle = nullptr;\n }\n }\n\n DMatrixHandle get_handle() {\n return this->proxy_dmat_handle;\n }\n};\n\nstd::unique_ptr GetProxyDMatrixWithBaseMargin(SEXP base_margin) {\n if (Rf_isNull(base_margin)) {\n return std::unique_ptr(nullptr);\n }\n\n SEXP base_margin_dim = Rf_getAttrib(base_margin, R_DimSymbol);\n int res_code;\n try {\n const std::string array_str = Rf_isNull(base_margin_dim)?\n MakeArrayInterfaceFromRVector(base_margin) : MakeArrayInterfaceFromRMat(base_margin);\n std::unique_ptr proxy_dmat(new ProxyDmatrixWrapper());\n res_code = XGDMatrixSetInfoFromInterface(proxy_dmat->get_handle(),\n \"base_margin\",\n array_str.c_str());\n if (res_code != 0) {\n throw ProxyDmatrixError();\n }\n return proxy_dmat;\n } catch(ProxyDmatrixError &err) {\n Rf_error(\"%s\", XGBGetLastError());\n }\n}\n\nenum class PredictionInputType {DMatrix, DenseMatrix, CSRMatrix, DataFrame};\n\nSEXP XGBoosterPredictGeneric(SEXP handle, SEXP input_data, SEXP json_config,\n PredictionInputType input_type, SEXP missing,\n SEXP base_margin) {\n SEXP r_out_result = R_NilValue;\n R_API_BEGIN();\n SEXP json_config_ = Rf_protect(Rf_asChar(json_config));\n char const *c_json_config = CHAR(json_config_);\n\n bst_ulong out_dim;\n bst_ulong const *out_shape;\n float const *out_result;\n\n int res_code;\n {\n switch (input_type) {\n case PredictionInputType::DMatrix: {\n res_code = XGBoosterPredictFromDMatrix(R_ExternalPtrAddr(handle),\n R_ExternalPtrAddr(input_data), c_json_config,\n &out_shape, &out_dim, &out_result);\n break;\n }\n\n case PredictionInputType::CSRMatrix: {\n std::unique_ptr proxy_dmat = GetProxyDMatrixWithBaseMargin(\n base_margin);\n DMatrixHandle proxy_dmat_handle = proxy_dmat.get()? proxy_dmat->get_handle() : nullptr;\n\n SEXP indptr = VECTOR_ELT(input_data, 0);\n SEXP indices = VECTOR_ELT(input_data, 1);\n SEXP data = VECTOR_ELT(input_data, 2);\n const int ncol_csr = Rf_asInteger(VECTOR_ELT(input_data, 3));\n const SEXPTYPE type_data = TYPEOF(data);\n CHECK_EQ(type_data, REALSXP);\n std::string sindptr, sindices, sdata;\n CreateFromSparse(indptr, indices, data, &sindptr, &sindices, &sdata);\n\n xgboost::StringView json_str(c_json_config);\n xgboost::Json new_json = xgboost::Json::Load(json_str);\n AddMissingToJson(&new_json, missing, type_data);\n const std::string new_c_json = xgboost::Json::Dump(new_json);\n\n res_code = XGBoosterPredictFromCSR(\n R_ExternalPtrAddr(handle), sindptr.c_str(), sindices.c_str(), sdata.c_str(),\n ncol_csr, new_c_json.c_str(), proxy_dmat_handle, &out_shape, &out_dim, &out_result);\n break;\n }\n\n case PredictionInputType::DenseMatrix: {\n std::unique_ptr proxy_dmat = GetProxyDMatrixWithBaseMargin(\n base_margin);\n DMatrixHandle proxy_dmat_handle = proxy_dmat.get()? proxy_dmat->get_handle() : nullptr;\n const std::string array_str = MakeArrayInterfaceFromRMat(input_data);\n\n xgboost::StringView json_str(c_json_config);\n xgboost::Json new_json = xgboost::Json::Load(json_str);\n AddMissingToJson(&new_json, missing, TYPEOF(input_data));\n const std::string new_c_json = xgboost::Json::Dump(new_json);\n\n res_code = XGBoosterPredictFromDense(\n R_ExternalPtrAddr(handle), array_str.c_str(), new_c_json.c_str(),\n proxy_dmat_handle, &out_shape, &out_dim, &out_result);\n break;\n }\n\n case PredictionInputType::DataFrame: {\n std::unique_ptr proxy_dmat = GetProxyDMatrixWithBaseMargin(\n base_margin);\n DMatrixHandle proxy_dmat_handle = proxy_dmat.get()? proxy_dmat->get_handle() : nullptr;\n\n const std::string df_str = MakeArrayInterfaceFromRDataFrame(input_data);\n\n xgboost::StringView json_str(c_json_config);\n xgboost::Json new_json = xgboost::Json::Load(json_str);\n AddMissingToJson(&new_json, missing, REALSXP);\n const std::string new_c_json = xgboost::Json::Dump(new_json);\n\n res_code = XGBoosterPredictFromColumnar(\n R_ExternalPtrAddr(handle), df_str.c_str(), new_c_json.c_str(),\n proxy_dmat_handle, &out_shape, &out_dim, &out_result);\n break;\n }\n }\n }\n CHECK_CALL(res_code);\n\n SEXP r_out_shape = Rf_protect(Rf_allocVector(INTSXP, out_dim));\n size_t len = 1;\n int *r_out_shape_ = INTEGER(r_out_shape);\n for (size_t i = 0; i < out_dim; ++i) {\n r_out_shape_[out_dim - i - 1] = out_shape[i];\n len *= out_shape[i];\n }\n r_out_result = Rf_protect(Rf_allocVector(REALSXP, len));\n std::copy(out_result, out_result + len, REAL(r_out_result));\n\n if (out_dim > 1) {\n Rf_setAttrib(r_out_result, R_DimSymbol, r_out_shape);\n }\n\n R_API_END();\n Rf_unprotect(3);\n\n return r_out_result;\n}\n\n} // namespace\n\nXGB_DLL SEXP XGBoosterPredictFromDMatrix_R(SEXP handle, SEXP dmat, SEXP json_config) {\n return XGBoosterPredictGeneric(handle, dmat, json_config,\n PredictionInputType::DMatrix, R_NilValue, R_NilValue);\n}\n\nXGB_DLL SEXP XGBoosterPredictFromDense_R(SEXP handle, SEXP R_mat, SEXP missing,\n SEXP json_config, SEXP base_margin) {\n return XGBoosterPredictGeneric(handle, R_mat, json_config,\n PredictionInputType::DenseMatrix, missing, base_margin);\n}\n\nXGB_DLL SEXP XGBoosterPredictFromCSR_R(SEXP handle, SEXP lst, SEXP missing,\n SEXP json_config, SEXP base_margin) {\n return XGBoosterPredictGeneric(handle, lst, json_config,\n PredictionInputType::CSRMatrix, missing, base_margin);\n}\n\nXGB_DLL SEXP XGBoosterPredictFromColumnar_R(SEXP handle, SEXP R_df, SEXP missing,\n SEXP json_config, SEXP base_margin) {\n return XGBoosterPredictGeneric(handle, R_df, json_config,\n PredictionInputType::DataFrame, missing, base_margin);\n}\n\nXGB_DLL SEXP XGBoosterLoadModel_R(SEXP handle, SEXP fname) {\n R_API_BEGIN();\n CHECK_CALL(XGBoosterLoadModel(R_ExternalPtrAddr(handle), CHAR(Rf_asChar(fname))));\n R_API_END();\n return R_NilValue;\n}\n\nXGB_DLL SEXP XGBoosterSaveModel_R(SEXP handle, SEXP fname) {\n R_API_BEGIN();\n CHECK_CALL(XGBoosterSaveModel(R_ExternalPtrAddr(handle), CHAR(Rf_asChar(fname))));\n R_API_END();\n return R_NilValue;\n}\n\nXGB_DLL SEXP XGBoosterLoadModelFromRaw_R(SEXP handle, SEXP raw) {\n R_API_BEGIN();\n CHECK_CALL(XGBoosterLoadModelFromBuffer(R_ExternalPtrAddr(handle),\n RAW(raw),\n Rf_xlength(raw)));\n R_API_END();\n return R_NilValue;\n}\n\nXGB_DLL SEXP XGBoosterSaveModelToRaw_R(SEXP handle, SEXP json_config) {\n SEXP ret;\n R_API_BEGIN();\n bst_ulong olen;\n char const *c_json_config = CHAR(Rf_asChar(json_config));\n char const *raw;\n CHECK_CALL(XGBoosterSaveModelToBuffer(R_ExternalPtrAddr(handle), c_json_config, &olen, &raw))\n ret = Rf_protect(Rf_allocVector(RAWSXP, olen));\n if (olen != 0) {\n std::memcpy(RAW(ret), raw, olen);\n }\n R_API_END();\n Rf_unprotect(1);\n return ret;\n}\n\nXGB_DLL SEXP XGBoosterSaveJsonConfig_R(SEXP handle) {\n const char* ret;\n R_API_BEGIN();\n bst_ulong len {0};\n CHECK_CALL(XGBoosterSaveJsonConfig(R_ExternalPtrAddr(handle),\n &len,\n &ret));\n R_API_END();\n return Rf_mkString(ret);\n}\n\nXGB_DLL SEXP XGBoosterLoadJsonConfig_R(SEXP handle, SEXP value) {\n R_API_BEGIN();\n CHECK_CALL(XGBoosterLoadJsonConfig(R_ExternalPtrAddr(handle), CHAR(Rf_asChar(value))));\n R_API_END();\n return R_NilValue;\n}\n\nXGB_DLL SEXP XGBoosterSerializeToBuffer_R(SEXP handle) {\n SEXP ret;\n R_API_BEGIN();\n bst_ulong out_len;\n const char *raw;\n CHECK_CALL(XGBoosterSerializeToBuffer(R_ExternalPtrAddr(handle), &out_len, &raw));\n ret = Rf_protect(Rf_allocVector(RAWSXP, out_len));\n if (out_len != 0) {\n memcpy(RAW(ret), raw, out_len);\n }\n R_API_END();\n Rf_unprotect(1);\n return ret;\n}\n\nXGB_DLL SEXP XGBoosterUnserializeFromBuffer_R(SEXP handle, SEXP raw) {\n R_API_BEGIN();\n CHECK_CALL(XGBoosterUnserializeFromBuffer(R_ExternalPtrAddr(handle),\n RAW(raw),\n Rf_xlength(raw)));\n R_API_END();\n return R_NilValue;\n}\n\nXGB_DLL SEXP XGBoosterDumpModel_R(SEXP handle, SEXP fmap, SEXP with_stats, SEXP dump_format) {\n SEXP out;\n SEXP continuation_token = Rf_protect(R_MakeUnwindCont());\n SEXP dump_format_ = Rf_protect(Rf_asChar(dump_format));\n SEXP fmap_ = Rf_protect(Rf_asChar(fmap));\n R_API_BEGIN();\n bst_ulong olen;\n const char **res;\n const char *fmt = CHAR(dump_format_);\n CHECK_CALL(XGBoosterDumpModelEx(R_ExternalPtrAddr(handle),\n CHAR(fmap_),\n Rf_asInteger(with_stats),\n fmt,\n &olen, &res));\n out = Rf_protect(Rf_allocVector(STRSXP, olen));\n try {\n if (!strcmp(\"json\", fmt)) {\n std::stringstream stream;\n stream << \"[\\n\";\n for (size_t i = 0; i < olen; ++i) {\n stream << res[i];\n if (i < olen - 1) {\n stream << \",\\n\";\n } else {\n stream << \"\\n\";\n }\n }\n stream << \"]\";\n const std::string temp_str = stream.str();\n SET_STRING_ELT(out, 0, SafeMkChar(temp_str.c_str(), continuation_token));\n } else {\n for (size_t i = 0; i < olen; ++i) {\n std::stringstream stream;\n stream << \"booster[\" << i <<\"]\\n\" << res[i];\n const std::string temp_str = stream.str();\n SET_STRING_ELT(out, i, SafeMkChar(temp_str.c_str(), continuation_token));\n }\n }\n } catch (ErrorWithUnwind &e) {\n R_ContinueUnwind(continuation_token);\n }\n R_API_END();\n Rf_unprotect(4);\n return out;\n}\n\nXGB_DLL SEXP XGBoosterGetAttr_R(SEXP handle, SEXP name) {\n SEXP out;\n R_API_BEGIN();\n int success;\n const char *val;\n CHECK_CALL(XGBoosterGetAttr(R_ExternalPtrAddr(handle),\n CHAR(Rf_asChar(name)),\n &val,\n &success));\n if (success) {\n out = Rf_protect(Rf_allocVector(STRSXP, 1));\n SET_STRING_ELT(out, 0, Rf_mkChar(val));\n } else {\n out = Rf_protect(R_NilValue);\n }\n R_API_END();\n Rf_unprotect(1);\n return out;\n}\n\nXGB_DLL SEXP XGBoosterSetAttr_R(SEXP handle, SEXP name, SEXP val) {\n R_API_BEGIN();\n const char *v = nullptr;\n SEXP name_ = Rf_protect(Rf_asChar(name));\n SEXP val_;\n int n_protected = 1;\n if (!Rf_isNull(val)) {\n val_ = Rf_protect(Rf_asChar(val));\n n_protected++;\n v = CHAR(val_);\n }\n\n CHECK_CALL(XGBoosterSetAttr(R_ExternalPtrAddr(handle),\n CHAR(name_), v));\n Rf_unprotect(n_protected);\n R_API_END();\n return R_NilValue;\n}\n\nXGB_DLL SEXP XGBoosterGetAttrNames_R(SEXP handle) {\n SEXP out;\n R_API_BEGIN();\n bst_ulong len;\n const char **res;\n CHECK_CALL(XGBoosterGetAttrNames(R_ExternalPtrAddr(handle),\n &len, &res));\n if (len > 0) {\n out = Rf_protect(Rf_allocVector(STRSXP, len));\n for (size_t i = 0; i < len; ++i) {\n SET_STRING_ELT(out, i, Rf_mkChar(res[i]));\n }\n } else {\n out = Rf_protect(R_NilValue);\n }\n R_API_END();\n Rf_unprotect(1);\n return out;\n}\n\nXGB_DLL SEXP XGBoosterFeatureScore_R(SEXP handle, SEXP json_config) {\n SEXP out_features_sexp;\n SEXP out_scores_sexp;\n SEXP out_shape_sexp;\n SEXP r_out = Rf_protect(Rf_allocVector(VECSXP, 3));\n\n R_API_BEGIN();\n char const *c_json_config = CHAR(Rf_asChar(json_config));\n bst_ulong out_n_features;\n char const **out_features;\n\n bst_ulong out_dim;\n bst_ulong const *out_shape;\n float const *out_scores;\n\n CHECK_CALL(XGBoosterFeatureScore(R_ExternalPtrAddr(handle), c_json_config,\n &out_n_features, &out_features,\n &out_dim, &out_shape, &out_scores));\n out_shape_sexp = Rf_protect(Rf_allocVector(INTSXP, out_dim));\n size_t len = 1;\n int *out_shape_sexp_ = INTEGER(out_shape_sexp);\n for (size_t i = 0; i < out_dim; ++i) {\n out_shape_sexp_[i] = out_shape[i];\n len *= out_shape[i];\n }\n\n out_features_sexp = Rf_protect(Rf_allocVector(STRSXP, out_n_features));\n for (size_t i = 0; i < out_n_features; ++i) {\n SET_STRING_ELT(out_features_sexp, i, Rf_mkChar(out_features[i]));\n }\n\n out_scores_sexp = Rf_protect(Rf_allocVector(REALSXP, len));\n std::copy(out_scores, out_scores + len, REAL(out_scores_sexp));\n\n SET_VECTOR_ELT(r_out, 0, out_features_sexp);\n SET_VECTOR_ELT(r_out, 1, out_shape_sexp);\n SET_VECTOR_ELT(r_out, 2, out_scores_sexp);\n\n R_API_END();\n Rf_unprotect(4);\n\n return r_out;\n}\n\nXGB_DLL SEXP XGBoosterSlice_R(SEXP handle, SEXP begin_layer, SEXP end_layer, SEXP step) {\n SEXP out = Rf_protect(XGBMakeEmptyAltrep());\n R_API_BEGIN();\n BoosterHandle handle_out = nullptr;\n CHECK_CALL(XGBoosterSlice(R_ExternalPtrAddr(handle),\n Rf_asInteger(begin_layer),\n Rf_asInteger(end_layer),\n Rf_asInteger(step),\n &handle_out));\n XGBAltrepSetPointer(out, handle_out);\n R_API_END();\n Rf_unprotect(1);\n return out;\n}\n\nXGB_DLL SEXP XGBoosterSliceAndReplace_R(SEXP handle, SEXP begin_layer, SEXP end_layer, SEXP step) {\n R_API_BEGIN();\n BoosterHandle old_handle = R_ExternalPtrAddr(handle);\n BoosterHandle new_handle = nullptr;\n CHECK_CALL(XGBoosterSlice(old_handle,\n Rf_asInteger(begin_layer),\n Rf_asInteger(end_layer),\n Rf_asInteger(step),\n &new_handle));\n R_SetExternalPtrAddr(handle, new_handle);\n CHECK_CALL(XGBoosterFree(old_handle));\n R_API_END();\n return R_NilValue;\n}\n" }, { "path": "R-package/src/xgboost_R.h", "content": "/*!\n * Copyright 2014-2022 by XGBoost Contributors\n * \\file xgboost_R.h\n * \\author Tianqi Chen\n * \\brief R wrapper of xgboost\n */\n#ifndef XGBOOST_R_H_ // NOLINT(*)\n#define XGBOOST_R_H_ // NOLINT(*)\n\n\n#ifndef R_NO_REMAP\n# define R_NO_REMAP\n#endif\n#include \n#include \n#include \n#include \n\n#include \n\n/*!\n * \\brief check whether a handle is NULL\n * \\param handle\n * \\return whether it is null ptr\n */\nXGB_DLL SEXP XGCheckNullPtr_R(SEXP handle);\n\n/*!\n * \\brief set the names of the dimensions of an array in-place\n * \\param arr\n * \\param dim_names names for the dimensions to set\n * \\return NULL value\n */\nXGB_DLL SEXP XGSetArrayDimNamesInplace_R(SEXP arr, SEXP dim_names);\n\n/*!\n * \\brief set the names of a vector in-place\n * \\param arr\n * \\param names names for the dimensions to set\n * \\return NULL value\n */\nXGB_DLL SEXP XGSetVectorNamesInplace_R(SEXP arr, SEXP names);\n\n/*!\n * \\brief Set global configuration\n * \\param json_str a JSON string representing the list of key-value pairs\n * \\return R_NilValue\n */\nXGB_DLL SEXP XGBSetGlobalConfig_R(SEXP json_str);\n\n/*!\n * \\brief Get global configuration\n * \\return JSON string\n */\nXGB_DLL SEXP XGBGetGlobalConfig_R();\n\n/*!\n * \\brief load a data matrix from URI\n * \\param uri URI to the source file to read data from\n * \\param silent whether print messages\n * \\param Data split mode (0=rows, 1=columns)\n * \\return a loaded data matrix\n */\nXGB_DLL SEXP XGDMatrixCreateFromURI_R(SEXP uri, SEXP silent, SEXP data_split_mode);\n\n/*!\n * \\brief create matrix content from dense matrix\n * This assumes the matrix is stored in column major format\n * \\param data R Matrix object\n * \\param missing which value to represent missing value\n * \\param n_threads Number of threads used to construct DMatrix from dense matrix.\n * \\return created dmatrix\n */\nXGB_DLL SEXP XGDMatrixCreateFromMat_R(SEXP mat,\n SEXP missing,\n SEXP n_threads);\n\n/**\n * @brief Create matrix content from a data frame.\n * @param data R data.frame object\n * @param missing which value to represent missing value\n * @param n_threads Number of threads used to construct DMatrix from dense matrix.\n * @return created dmatrix\n */\nXGB_DLL SEXP XGDMatrixCreateFromDF_R(SEXP df, SEXP missing, SEXP n_threads);\n\n/*!\n * \\brief create a matrix content from CSC format\n * \\param indptr pointer to column headers\n * \\param indices row indices\n * \\param data content of the data\n * \\param num_row numer of rows (when it's set to 0, then guess from data)\n * \\param missing which value to represent missing value\n * \\param n_threads Number of threads used to construct DMatrix from csc matrix.\n * \\return created dmatrix\n */\nXGB_DLL SEXP XGDMatrixCreateFromCSC_R(SEXP indptr, SEXP indices, SEXP data, SEXP num_row,\n SEXP missing, SEXP n_threads);\n\n/*!\n * \\brief create a matrix content from CSR format\n * \\param indptr pointer to row headers\n * \\param indices column indices\n * \\param data content of the data\n * \\param num_col numer of columns (when it's set to 0, then guess from data)\n * \\param missing which value to represent missing value\n * \\param n_threads Number of threads used to construct DMatrix from csr matrix.\n * \\return created dmatrix\n */\nXGB_DLL SEXP XGDMatrixCreateFromCSR_R(SEXP indptr, SEXP indices, SEXP data, SEXP num_col,\n SEXP missing, SEXP n_threads);\n\n/*!\n * \\brief create a new dmatrix from sliced content of existing matrix\n * \\param handle instance of data matrix to be sliced\n * \\param idxset index set\n * \\param allow_groups Whether to allow slicing the DMatrix if it has a 'group' field\n * \\return a sliced new matrix\n */\nXGB_DLL SEXP XGDMatrixSliceDMatrix_R(SEXP handle, SEXP idxset, SEXP allow_groups);\n\n/*!\n * \\brief load a data matrix into binary file\n * \\param handle a instance of data matrix\n * \\param fname file name\n * \\param silent print statistics when saving\n * \\return R_NilValue\n */\nXGB_DLL SEXP XGDMatrixSaveBinary_R(SEXP handle, SEXP fname, SEXP silent);\n\n/*!\n * \\brief set information to dmatrix\n * \\param handle a instance of data matrix\n * \\param field field name, can be label, weight\n * \\param array pointer to float vector\n * \\return R_NilValue\n */\nXGB_DLL SEXP XGDMatrixSetInfo_R(SEXP handle, SEXP field, SEXP array);\n\n/*!\n * \\brief get info vector (float type) from matrix\n * \\param handle a instance of data matrix\n * \\param field field name\n * \\return info vector\n */\nXGB_DLL SEXP XGDMatrixGetFloatInfo_R(SEXP handle, SEXP field);\n\n/*!\n * \\brief get info vector (uint type) from matrix\n * \\param handle a instance of data matrix\n * \\param field field name\n * \\return info vector\n */\nXGB_DLL SEXP XGDMatrixGetUIntInfo_R(SEXP handle, SEXP field);\n\n/*!\n * \\brief return number of rows\n * \\param handle an instance of data matrix\n */\nXGB_DLL SEXP XGDMatrixNumRow_R(SEXP handle);\n\n/*!\n * \\brief return number of columns\n * \\param handle an instance of data matrix\n */\nXGB_DLL SEXP XGDMatrixNumCol_R(SEXP handle);\n\n/*!\n<<<<<<< HEAD\n * \\brief create a ProxyDMatrix and get an R externalptr object for it\n */\nXGB_DLL SEXP XGProxyDMatrixCreate_R();\n\n/*!\n * \\brief Set dense matrix data on a proxy dmatrix\n * \\param handle R externalptr pointing to a ProxyDMatrix\n * \\param R_mat R matrix to set in the proxy dmatrix\n */\nXGB_DLL SEXP XGProxyDMatrixSetDataDense_R(SEXP handle, SEXP R_mat);\n\n/*!\n * \\brief Set dense matrix data on a proxy dmatrix\n * \\param handle R externalptr pointing to a ProxyDMatrix\n * \\param lst R list containing, in this order:\n * 1. 'p' or 'indptr' vector of the CSR matrix.\n * 2. 'j' or 'indices' vector of the CSR matrix.\n * 3. 'x' or 'data' vector of the CSR matrix.\n * 4. Number of columns in the CSR matrix.\n */\nXGB_DLL SEXP XGProxyDMatrixSetDataCSR_R(SEXP handle, SEXP lst);\n\n/*!\n * \\brief Set dense matrix data on a proxy dmatrix\n * \\param handle R externalptr pointing to a ProxyDMatrix\n * \\param lst R list or data.frame object containing its columns as numeric vectors\n */\nXGB_DLL SEXP XGProxyDMatrixSetDataColumnar_R(SEXP handle, SEXP lst);\n\n/*!\n * \\brief Create a DMatrix from a DataIter with callbacks\n * \\param expr_f_next expression for function(env, proxy_dmat) that sets the data on the proxy\n * dmatrix and returns either zero (end of batch) or one (batch continues).\n * \\param expr_f_reset expression for function(env) that resets the data iterator to\n * the beginning (first batch).\n * \\param calling_env R environment where to evaluate the expressions above\n * \\param proxy_dmat R externalptr holding a ProxyDMatrix.\n * \\param n_threads number of parallel threads to use for constructing the DMatrix.\n * \\param missing which value to represent missing value.\n * \\param cache_prefix path of cache file\n * \\return handle R externalptr holding the resulting DMatrix.\n */\nXGB_DLL SEXP XGDMatrixCreateFromCallback_R(\n SEXP expr_f_next, SEXP expr_f_reset, SEXP calling_env, SEXP proxy_dmat,\n SEXP n_threads, SEXP missing, SEXP cache_prefix);\n\n/*!\n * \\brief Create a QuantileDMatrix from a DataIter with callbacks\n * \\param expr_f_next expression for function(env, proxy_dmat) that sets the data on the proxy\n * dmatrix and returns either zero (end of batch) or one (batch continues).\n * \\param expr_f_reset expression for function(env) that resets the data iterator to\n * the beginning (first batch).\n * \\param calling_env R environment where to evaluate the expressions above\n * \\param proxy_dmat R externalptr holding a ProxyDMatrix.\n * \\param n_threads number of parallel threads to use for constructing the QuantileDMatrix.\n * \\param missing which value to represent missing value.\n * \\param max_bin maximum number of bins to have in the resulting QuantileDMatrix.\n * \\param ref_dmat an optional reference DMatrix from which to get the bin boundaries.\n * \\return handle R externalptr holding the resulting QuantileDMatrix.\n */\nXGB_DLL SEXP XGQuantileDMatrixCreateFromCallback_R(\n SEXP expr_f_next, SEXP expr_f_reset, SEXP calling_env, SEXP proxy_dmat,\n SEXP n_threads, SEXP missing, SEXP max_bin, SEXP ref_dmat);\n\n/*!\n * \\brief Frees a ProxyDMatrix and empties out the R externalptr object that holds it\n * \\param proxy_dmat R externalptr containing a ProxyDMatrix\n * \\return NULL\n */\nXGB_DLL SEXP XGDMatrixFree_R(SEXP proxy_dmat);\n\n/*!\n * \\brief Get the value that represents missingness in R integers as a numeric non-missing value.\n */\nXGB_DLL SEXP XGGetRNAIntAsDouble();\n\n/*!\n * \\brief Call R C-level function 'duplicate'\n * \\param obj Object to duplicate\n */\nXGB_DLL SEXP XGDuplicate_R(SEXP obj);\n\n/*!\n * \\brief Equality comparison for two pointers\n * \\param obj1 R 'externalptr'\n * \\param obj2 R 'externalptr'\n */\nXGB_DLL SEXP XGPointerEqComparison_R(SEXP obj1, SEXP obj2);\n\n/*!\n * \\brief Register the Altrep class used for the booster\n * \\param dll DLL info as provided by R_init\n */\nXGB_DLL void XGBInitializeAltrepClass_R(DllInfo *dll);\n\n/*!\n * \\brief return the quantile cuts used for the histogram method\n * \\param handle an instance of data matrix\n * \\return A list with entries 'indptr' and 'data'\n */\nXGB_DLL SEXP XGDMatrixGetQuantileCut_R(SEXP handle);\n\n/*!\n * \\brief get the number of non-missing entries in a dmatrix\n * \\param handle an instance of data matrix\n * \\return the number of non-missing entries\n */\nXGB_DLL SEXP XGDMatrixNumNonMissing_R(SEXP handle);\n\n/*!\n * \\brief get the data in a dmatrix in CSR format\n * \\param handle an instance of data matrix\n * \\return R list with the following entries in this order:\n * - 'indptr\n * - 'indices\n * - 'data'\n * - 'ncol'\n */\nXGB_DLL SEXP XGDMatrixGetDataAsCSR_R(SEXP handle);\n\n/*!\n * \\brief create xgboost learner\n * \\param dmats a list of dmatrix handles that will be cached\n */\nXGB_DLL SEXP XGBoosterCreate_R(SEXP dmats);\n\n/*!\n * \\brief copy information about features from a DMatrix into a Booster\n * \\param booster R 'externalptr' pointing to a booster object\n * \\param dmat R 'externalptr' pointing to a DMatrix object\n */\nXGB_DLL SEXP XGBoosterCopyInfoFromDMatrix_R(SEXP booster, SEXP dmat);\n\n/*!\n * \\brief handle R 'externalptr' holding the booster object\n * \\param field field name\n * \\param features features to set for the field\n */\nXGB_DLL SEXP XGBoosterSetStrFeatureInfo_R(SEXP handle, SEXP field, SEXP features);\n\n/*!\n * \\brief handle R 'externalptr' holding the booster object\n * \\param field field name\n */\nXGB_DLL SEXP XGBoosterGetStrFeatureInfo_R(SEXP handle, SEXP field);\n\n/*!\n * \\brief Get the number of boosted rounds from a model\n * \\param handle R 'externalptr' holding the booster object\n */\nXGB_DLL SEXP XGBoosterBoostedRounds_R(SEXP handle);\n\n/*!\n * \\brief Get the number of features to which the model was fitted\n * \\param handle R 'externalptr' holding the booster object\n */\nXGB_DLL SEXP XGBoosterGetNumFeature_R(SEXP handle);\n\n/*!\n * \\brief set parameters\n * \\param handle handle\n * \\param name parameter name\n * \\param val value of parameter\n * \\return R_NilValue\n */\nXGB_DLL SEXP XGBoosterSetParam_R(SEXP handle, SEXP name, SEXP val);\n\n/*!\n * \\brief update the model in one round using dtrain\n * \\param handle handle\n * \\param iter current iteration rounds\n * \\param dtrain training data\n * \\return R_NilValue\n */\nXGB_DLL SEXP XGBoosterUpdateOneIter_R(SEXP ext, SEXP iter, SEXP dtrain);\n\n/*!\n * \\brief update the model, by directly specify gradient and second order gradient,\n * this can be used to replace UpdateOneIter, to support customized loss function\n * \\param handle handle\n * \\param iter The current training iteration.\n * \\param dtrain training data\n * \\param grad gradient statistics\n * \\param hess second order gradient statistics\n * \\return R_NilValue\n */\nXGB_DLL SEXP XGBoosterTrainOneIter_R(SEXP handle, SEXP dtrain, SEXP iter, SEXP grad, SEXP hess);\n\n/*!\n * \\brief get evaluation statistics for xgboost\n * \\param handle handle\n * \\param iter current iteration rounds\n * \\param dmats list of handles to dmatrices\n * \\param evname name of evaluation\n * \\return the string containing evaluation stats\n */\nXGB_DLL SEXP XGBoosterEvalOneIter_R(SEXP handle, SEXP iter, SEXP dmats, SEXP evnames);\n\n/*!\n * \\brief Run prediction on DMatrix, replacing `XGBoosterPredict_R`\n * \\param handle handle\n * \\param dmat data matrix\n * \\param json_config See `XGBoosterPredictFromDMatrix` in xgboost c_api.h\n *\n * \\return A list containing 2 vectors, first one for shape while second one for prediction result.\n */\nXGB_DLL SEXP XGBoosterPredictFromDMatrix_R(SEXP handle, SEXP dmat, SEXP json_config);\n\n/*!\n * \\brief Run prediction on R dense matrix\n * \\param handle handle\n * \\param R_mat R matrix\n * \\param missing missing value\n * \\param json_config See `XGBoosterPredictFromDense` in xgboost c_api.h. Doesn't include 'missing'\n * \\param base_margin base margin for the prediction\n *\n * \\return A list containing 2 vectors, first one for shape while second one for prediction result.\n */\nXGB_DLL SEXP XGBoosterPredictFromDense_R(SEXP handle, SEXP R_mat, SEXP missing,\n SEXP json_config, SEXP base_margin);\n\n/*!\n * \\brief Run prediction on R CSR matrix\n * \\param handle handle\n * \\param lst An R list, containing, in this order:\n * (a) 'p' array (a.k.a. indptr)\n * (b) 'j' array (a.k.a. indices)\n * (c) 'x' array (a.k.a. data / values)\n * (d) number of columns\n * \\param missing missing value\n * \\param json_config See `XGBoosterPredictFromCSR` in xgboost c_api.h. Doesn't include 'missing'\n * \\param base_margin base margin for the prediction\n *\n * \\return A list containing 2 vectors, first one for shape while second one for prediction result.\n */\nXGB_DLL SEXP XGBoosterPredictFromCSR_R(SEXP handle, SEXP lst, SEXP missing,\n SEXP json_config, SEXP base_margin);\n\n/*!\n * \\brief Run prediction on R data.frame\n * \\param handle handle\n * \\param R_df R data.frame\n * \\param missing missing value\n * \\param json_config See `XGBoosterPredictFromDense` in xgboost c_api.h. Doesn't include 'missing'\n * \\param base_margin base margin for the prediction\n *\n * \\return A list containing 2 vectors, first one for shape while second one for prediction result.\n */\nXGB_DLL SEXP XGBoosterPredictFromColumnar_R(SEXP handle, SEXP R_df, SEXP missing,\n SEXP json_config, SEXP base_margin);\n\n/*!\n * \\brief load model from existing file\n * \\param handle handle\n * \\param fname file name\n * \\return R_NilValue\n */\nXGB_DLL SEXP XGBoosterLoadModel_R(SEXP handle, SEXP fname);\n\n/*!\n * \\brief save model into existing file\n * \\param handle handle\n * \\param fname file name\n * \\return R_NilValue\n */\nXGB_DLL SEXP XGBoosterSaveModel_R(SEXP handle, SEXP fname);\n\n/*!\n * \\brief load model from raw array\n * \\param handle handle\n * \\return R_NilValue\n */\nXGB_DLL SEXP XGBoosterLoadModelFromRaw_R(SEXP handle, SEXP raw);\n\n/*!\n * \\brief Save model into R's raw array\n *\n * \\param handle handle\n * \\param json_config JSON encoded string storing parameters for the function. Following\n * keys are expected in the JSON document:\n *\n * \"format\": str\n * - json: Output booster will be encoded as JSON.\n * - ubj: Output booster will be encoded as Univeral binary JSON.\n * - deprecated: Output booster will be encoded as old custom binary format. Do now use\n * this format except for compatibility reasons.\n *\n * \\return Raw array\n */\nXGB_DLL SEXP XGBoosterSaveModelToRaw_R(SEXP handle, SEXP json_config);\n\n/*!\n * \\brief Save internal parameters as a JSON string\n * \\param handle handle\n * \\return JSON string\n */\n\nXGB_DLL SEXP XGBoosterSaveJsonConfig_R(SEXP handle);\n/*!\n * \\brief Load the JSON string returnd by XGBoosterSaveJsonConfig_R\n * \\param handle handle\n * \\param value JSON string\n * \\return R_NilValue\n */\nXGB_DLL SEXP XGBoosterLoadJsonConfig_R(SEXP handle, SEXP value);\n\n/*!\n * \\brief Memory snapshot based serialization method. Saves everything states\n * into buffer.\n * \\param handle handle to booster\n */\nXGB_DLL SEXP XGBoosterSerializeToBuffer_R(SEXP handle);\n\n/*!\n * \\brief Memory snapshot based serialization method. Loads the buffer returned\n * from `XGBoosterSerializeToBuffer'.\n * \\param handle handle to booster\n * \\return raw byte array\n */\nXGB_DLL SEXP XGBoosterUnserializeFromBuffer_R(SEXP handle, SEXP raw);\n\n/*!\n * \\brief dump model into a string\n * \\param handle handle\n * \\param fmap name to fmap can be empty string\n * \\param with_stats whether dump statistics of splits\n * \\param dump_format the format to dump the model in\n */\nXGB_DLL SEXP XGBoosterDumpModel_R(SEXP handle, SEXP fmap, SEXP with_stats, SEXP dump_format);\n\n/*!\n * \\brief get learner attribute value\n * \\param handle handle\n * \\param name attribute name\n * \\return character containing attribute value\n */\nXGB_DLL SEXP XGBoosterGetAttr_R(SEXP handle, SEXP name);\n\n/*!\n * \\brief set learner attribute value\n * \\param handle handle\n * \\param name attribute name\n * \\param val attribute value; NULL value would delete an attribute\n * \\return R_NilValue\n */\nXGB_DLL SEXP XGBoosterSetAttr_R(SEXP handle, SEXP name, SEXP val);\n\n/*!\n * \\brief get the names of learner attributes\n * \\return string vector containing attribute names\n */\nXGB_DLL SEXP XGBoosterGetAttrNames_R(SEXP handle);\n\n/*!\n * \\brief Get feature scores from the model.\n * \\param json_config See `XGBoosterFeatureScore` in xgboost c_api.h\n * \\return A vector with the first element as feature names, second element as shape of\n * feature scores and thrid element as feature scores.\n */\nXGB_DLL SEXP XGBoosterFeatureScore_R(SEXP handle, SEXP json_config);\n\n/*!\n * \\brief Slice a fitted booster model (by rounds)\n * \\param handle handle to the fitted booster\n * \\param begin_layer start of the slice\n * \\param end_later end of the slice; end_layer=0 is equivalent to end_layer=num_boost_round\n * \\param step step size of the slice\n * \\return The sliced booster with the requested rounds only\n */\nXGB_DLL SEXP XGBoosterSlice_R(SEXP handle, SEXP begin_layer, SEXP end_layer, SEXP step);\n\n/*!\n * \\brief Slice a fitted booster model (by rounds), and replace its handle with the result\n * \\param handle handle to the fitted booster\n * \\param begin_layer start of the slice\n * \\param end_later end of the slice; end_layer=0 is equivalent to end_layer=num_boost_round\n * \\param step step size of the slice\n * \\return NULL\n */\nXGB_DLL SEXP XGBoosterSliceAndReplace_R(SEXP handle, SEXP begin_layer, SEXP end_layer, SEXP step);\n\n#endif // XGBOOST_WRAPPER_R_H_ // NOLINT(*)\n" }, { "path": "R-package/src/xgboost_custom.cc", "content": "// Copyright (c) 2015 by Contributors\n// This file contains the customization implementations of R module\n// to change behavior of libxgboost\n\n#include \n#include \"../../src/common/random.h\"\n#include \"./xgboost_R.h\"\n\n// redirect the messages to R's console.\nnamespace dmlc {\nvoid CustomLogMessage::Log(const std::string& msg) {\n Rprintf(\"%s\\n\", msg.c_str());\n}\n} // namespace dmlc\n\nnamespace xgboost {\nConsoleLogger::~ConsoleLogger() {\n if (cur_verbosity_ == LogVerbosity::kIgnore ||\n cur_verbosity_ <= GlobalVerbosity()) {\n if (cur_verbosity_ == LogVerbosity::kWarning) {\n REprintf(\"%s\\n\", log_stream_.str().c_str());\n } else {\n dmlc::CustomLogMessage::Log(log_stream_.str());\n }\n }\n}\nTrackerLogger::~TrackerLogger() {\n dmlc::CustomLogMessage::Log(log_stream_.str());\n}\n} // namespace xgboost\n\nnamespace xgboost {\nnamespace common {\n\n// redirect the nath functions.\nbool CheckNAN(double v) {\n return ISNAN(v);\n}\n#if !defined(XGBOOST_USE_CUDA)\ndouble LogGamma(double v) {\n return lgammafn(v);\n}\n#endif // !defined(XGBOOST_USE_CUDA)\n\n} // namespace common\n} // namespace xgboost\n" }, { "path": "R-package/tests/helper_scripts/generate_models.R", "content": "# Script to generate reference models. The reference models are used to test backward compatibility\n# of saved model files from XGBoost version 0.90 and 1.0.x.\nlibrary(xgboost)\nlibrary(Matrix)\n\nset.seed(0)\nmetadata <- list(\n kRounds = 2,\n kRows = 1000,\n kCols = 4,\n kForests = 2,\n kMaxDepth = 2,\n kClasses = 3\n)\nX <- Matrix(data = rnorm(metadata$kRows * metadata$kCols), nrow = metadata$kRows,\n ncol = metadata$kCols, sparse = TRUE)\nw <- runif(metadata$kRows)\n\nversion <- packageVersion('xgboost')\ntarget_dir <- 'models'\n\nsave_booster <- function(booster, model_name) {\n booster_bin <- function(model_name) {\n return(file.path(target_dir, paste('xgboost-', version, '.', model_name, '.bin', sep = '')))\n }\n booster_json <- function(model_name) {\n return(file.path(target_dir, paste('xgboost-', version, '.', model_name, '.json', sep = '')))\n }\n booster_rds <- function(model_name) {\n return(file.path(target_dir, paste('xgboost-', version, '.', model_name, '.rds', sep = '')))\n }\n xgb.save(booster, booster_bin(model_name))\n saveRDS(booster, booster_rds(model_name))\n if (version >= '1.0.0') {\n xgb.save(booster, booster_json(model_name))\n }\n}\n\ngenerate_regression_model <- function() {\n print('Regression')\n y <- rnorm(metadata$kRows)\n\n data <- xgb.DMatrix(X, label = y, nthread = 1)\n params <- list(tree_method = 'hist', num_parallel_tree = metadata$kForests,\n max_depth = metadata$kMaxDepth)\n booster <- xgb.train(params, data, nrounds = metadata$kRounds)\n save_booster(booster, 'reg')\n}\n\ngenerate_logistic_model <- function() {\n print('Binary classification with logistic loss')\n y <- sample(0:1, size = metadata$kRows, replace = TRUE)\n stopifnot(max(y) == 1, min(y) == 0)\n\n objective <- c('binary:logistic', 'binary:logitraw')\n name <- c('logit', 'logitraw')\n\n for (i in seq_along(objective)) {\n data <- xgb.DMatrix(X, label = y, weight = w, nthread = 1)\n params <- list(tree_method = 'hist', num_parallel_tree = metadata$kForests,\n max_depth = metadata$kMaxDepth, objective = objective[i])\n booster <- xgb.train(params, data, nrounds = metadata$kRounds)\n save_booster(booster, name[i])\n }\n}\n\ngenerate_classification_model <- function() {\n print('Multi-class classification')\n y <- sample(0:(metadata$kClasses - 1), size = metadata$kRows, replace = TRUE)\n stopifnot(max(y) == metadata$kClasses - 1, min(y) == 0)\n\n data <- xgb.DMatrix(X, label = y, weight = w, nthread = 1)\n params <- list(num_class = metadata$kClasses, tree_method = 'hist',\n num_parallel_tree = metadata$kForests, max_depth = metadata$kMaxDepth,\n objective = 'multi:softmax')\n booster <- xgb.train(params, data, nrounds = metadata$kRounds)\n save_booster(booster, 'cls')\n}\n\ngenerate_ranking_model <- function() {\n print('Learning to rank')\n y <- sample(0:4, size = metadata$kRows, replace = TRUE)\n stopifnot(max(y) == 4, min(y) == 0)\n kGroups <- 20\n w <- runif(kGroups)\n g <- rep(50, times = kGroups)\n\n data <- xgb.DMatrix(X, label = y, group = g, nthread = 1)\n # setinfo(data, 'weight', w)\n # ^^^ does not work in version <= 1.1.0; see https://github.com/dmlc/xgboost/issues/5942\n # So call low-level function XGDMatrixSetInfo_R directly. Since this function is not an exported\n # symbol, use the triple-colon operator.\n .Call(xgboost:::XGDMatrixSetInfo_R, data, 'weight', as.numeric(w))\n params <- list(objective = 'rank:ndcg', num_parallel_tree = metadata$kForests,\n tree_method = 'hist', max_depth = metadata$kMaxDepth)\n booster <- xgb.train(params, data, nrounds = metadata$kRounds)\n save_booster(booster, 'ltr')\n}\n\ndir.create(target_dir)\n\ninvisible(generate_regression_model())\ninvisible(generate_logistic_model())\ninvisible(generate_classification_model())\ninvisible(generate_ranking_model())\n" }, { "path": "R-package/tests/helper_scripts/install_deps.R", "content": "## Install dependencies of R package for testing. The list might not be\n## up-to-date, check DESCRIPTION for the latest list and update this one if\n## inconsistent is found.\npkgs <- c(\n ## CI\n \"pkgbuild\",\n \"roxygen2\",\n \"XML\",\n \"cplm\",\n \"e1071\",\n ## suggests\n \"knitr\",\n \"rmarkdown\",\n \"ggplot2\",\n \"DiagrammeR\",\n \"DiagrammeRsvg\",\n \"rsvg\",\n \"htmlwidgets\",\n \"Ckmeans.1d.dp\",\n \"vcd\",\n \"lintr\",\n \"testthat\",\n \"igraph\",\n \"float\",\n \"titanic\",\n \"RhpcBLASctl\",\n ## imports\n \"Matrix\",\n \"data.table\",\n \"jsonlite\"\n)\n\nncpus <- parallel::detectCores()\nprint(paste0(\"Using \", ncpus, \" cores to install dependencies.\"))\n\nif (.Platform$OS.type == \"unix\") {\n print(\"Installing source packages on unix.\")\n install.packages(\n pkgs,\n repo = \"https://cloud.r-project.org\",\n dependencies = c(\"Depends\", \"Imports\", \"LinkingTo\"),\n Ncpus = parallel::detectCores()\n )\n} else {\n print(\"Installing binary packages on Windows.\")\n install.packages(\n pkgs,\n repo = \"https://cloud.r-project.org\",\n dependencies = c(\"Depends\", \"Imports\", \"LinkingTo\"),\n Ncpus = parallel::detectCores(),\n type = \"binary\"\n )\n}\n" }, { "path": "R-package/tests/helper_scripts/run-examples.R", "content": "## Helper script for running individual examples.\nlibrary(pkgload)\nlibrary(xgboost)\n\nfiles <- list.files(\"./man\")\n\n\nrun_example_timeit <- function(f) {\n path <- paste(\"./man/\", f, sep = \"\")\n print(paste(\"Test\", f))\n flush.console()\n t0 <- proc.time()\n run_example(path)\n t1 <- proc.time()\n list(file = f, time = t1 - t0)\n}\n\ntimings <- lapply(files, run_example_timeit)\n\nfor (t in timings) {\n ratio <- t$time[1] / t$time[3]\n if (!is.na(ratio) && !is.infinite(ratio) && ratio >= 2.5) {\n print(paste(\"Offending example:\", t$file, ratio))\n }\n}\n" }, { "path": "R-package/tests/testthat/helper_model.R", "content": "## A special file sourced by testthat.\n\nget_basescore <- function(model) {\n as.numeric(\n jsonlite::fromJSON(model$learner$learner_model_param$base_score)\n )\n}\n" }, { "path": "R-package/tests/testthat/test_basic.R", "content": "context(\"basic functions\")\n\ndata(agaricus.train, package = \"xgboost\")\ndata(agaricus.test, package = \"xgboost\")\ntrain <- agaricus.train\ntest <- agaricus.test\nset.seed(1994)\n\n# disable some tests for Win32\nwindows_flag <- .Platform$OS.type == \"windows\" &&\n .Machine$sizeof.pointer != 8\nsolaris_flag <- (Sys.info()[\"sysname\"] == \"SunOS\")\nn_threads <- 1\n\n\ntest_that(\"train and predict binary classification\", {\n nrounds <- 2\n expect_output(\n bst <- xgb.train(\n data = xgb.DMatrix(train$data, label = train$label, nthread = 1),\n nrounds = nrounds,\n params = xgb.params(\n max_depth = 2,\n learning_rate = 1,\n nthread = n_threads,\n objective = \"binary:logistic\",\n eval_metric = \"error\"\n ),\n evals = list(train = xgb.DMatrix(train$data, label = train$label, nthread = 1))\n ),\n \"train-error\"\n )\n expect_equal(class(bst), \"xgb.Booster\")\n expect_equal(xgb.get.num.boosted.rounds(bst), nrounds)\n expect_false(is.null(attributes(bst)$evaluation_log))\n expect_equal(nrow(attributes(bst)$evaluation_log), nrounds)\n expect_lt(attributes(bst)$evaluation_log[, min(train_error)], 0.03)\n\n pred <- predict(bst, test$data)\n expect_length(pred, 1611)\n\n pred1 <- predict(bst, train$data, iterationrange = c(1, 1))\n expect_length(pred1, 6513)\n err_pred1 <- sum((pred1 > 0.5) != train$label) / length(train$label)\n err_log <- attributes(bst)$evaluation_log[1, train_error]\n expect_lt(abs(err_pred1 - err_log), 10e-6)\n})\n\ntest_that(\"parameter validation works\", {\n p <- list(foo = \"bar\")\n nrounds <- 1\n set.seed(1994)\n\n d <- cbind(\n x1 = rnorm(10),\n x2 = rnorm(10),\n x3 = rnorm(10)\n )\n y <- d[, \"x1\"] + d[, \"x2\"]^2 +\n ifelse(d[, \"x3\"] > .5, d[, \"x3\"]^2, 2^d[, \"x3\"]) +\n rnorm(10)\n dtrain <- xgb.DMatrix(data = d, label = y, nthread = n_threads)\n\n correct <- function() {\n params <- list(\n max_depth = 2,\n booster = \"dart\",\n rate_drop = 0.5,\n one_drop = TRUE,\n nthread = n_threads,\n objective = \"reg:squarederror\"\n )\n xgb.train(params = params, data = dtrain, nrounds = nrounds)\n }\n expect_silent(correct())\n incorrect <- function() {\n params <- list(\n max_depth = 2,\n booster = \"dart\",\n rate_drop = 0.5,\n one_drop = TRUE,\n objective = \"reg:squarederror\",\n nthread = n_threads,\n foo = \"bar\",\n bar = \"foo\"\n )\n output <- capture.output(\n xgb.train(params = params, data = dtrain, nrounds = nrounds),\n type = \"message\"\n )\n print(output)\n }\n expect_output(incorrect(), '\\\\\\\\\"bar\\\\\\\\\", \\\\\\\\\"foo\\\\\\\\\"')\n})\n\n\ntest_that(\"dart prediction works\", {\n nrounds <- 32\n set.seed(1994)\n\n d <- cbind(\n x1 = rnorm(100),\n x2 = rnorm(100),\n x3 = rnorm(100)\n )\n y <- d[, \"x1\"] + d[, \"x2\"]^2 +\n ifelse(d[, \"x3\"] > .5, d[, \"x3\"]^2, 2^d[, \"x3\"]) +\n rnorm(100)\n\n set.seed(1994)\n booster_by_xgboost <- xgb.train(\n data = xgb.DMatrix(d, label = y, nthread = 1),\n nrounds = nrounds,\n params = xgb.params(\n max_depth = 2,\n booster = \"dart\",\n rate_drop = 0.5,\n one_drop = TRUE,\n learning_rate = 1,\n nthread = n_threads,\n objective = \"reg:squarederror\"\n )\n )\n pred_by_xgboost_0 <- predict(booster_by_xgboost, newdata = d, iterationrange = NULL)\n pred_by_xgboost_1 <- predict(booster_by_xgboost, newdata = d, iterationrange = c(1, nrounds))\n expect_true(all(matrix(pred_by_xgboost_0, byrow = TRUE) == matrix(pred_by_xgboost_1, byrow = TRUE)))\n\n pred_by_xgboost_2 <- predict(booster_by_xgboost, newdata = d, training = TRUE)\n expect_false(all(matrix(pred_by_xgboost_0, byrow = TRUE) == matrix(pred_by_xgboost_2, byrow = TRUE)))\n\n set.seed(1994)\n dtrain <- xgb.DMatrix(data = d, label = y, nthread = n_threads)\n booster_by_train <- xgb.train(\n params = xgb.params(\n booster = \"dart\",\n max_depth = 2,\n learning_rate = 1,\n rate_drop = 0.5,\n one_drop = TRUE,\n nthread = n_threads,\n objective = \"reg:squarederror\"\n ),\n data = dtrain,\n nrounds = nrounds\n )\n pred_by_train_0 <- predict(booster_by_train, newdata = dtrain, iterationrange = NULL)\n pred_by_train_1 <- predict(booster_by_train, newdata = dtrain, iterationrange = c(1, nrounds))\n pred_by_train_2 <- predict(booster_by_train, newdata = dtrain, training = TRUE)\n\n expect_equal(pred_by_train_0, pred_by_xgboost_0, tolerance = 1e-6)\n expect_equal(pred_by_train_1, pred_by_xgboost_1, tolerance = 1e-6)\n expect_true(all(matrix(pred_by_train_2, byrow = TRUE) == matrix(pred_by_xgboost_2, byrow = TRUE)))\n})\n\ntest_that(\"train and predict softprob\", {\n lb <- as.numeric(iris$Species) - 1\n set.seed(11)\n expect_output(\n bst <- xgb.train(\n data = xgb.DMatrix(as.matrix(iris[, -5]), label = lb, nthread = 1),\n nrounds = 5,\n params = xgb.params(\n max_depth = 3, learning_rate = 0.5, nthread = n_threads,\n objective = \"multi:softprob\", num_class = 3, eval_metric = \"merror\"\n ),\n evals = list(train = xgb.DMatrix(as.matrix(iris[, -5]), label = lb, nthread = 1))\n ),\n \"train-merror\"\n )\n expect_false(is.null(attributes(bst)$evaluation_log))\n expect_lt(attributes(bst)$evaluation_log[, min(train_merror)], 0.025)\n expect_equal(xgb.get.num.boosted.rounds(bst), 5)\n pred <- predict(bst, as.matrix(iris[, -5]))\n expect_length(pred, nrow(iris) * 3)\n # row sums add up to total probability of 1:\n expect_equal(rowSums(pred), rep(1, nrow(iris)), tolerance = 1e-7)\n # manually calculate error at the last iteration:\n mpred <- predict(bst, as.matrix(iris[, -5]))\n expect_equal(mpred, pred)\n pred_labels <- max.col(mpred) - 1\n err <- sum(pred_labels != lb) / length(lb)\n expect_equal(attributes(bst)$evaluation_log[5, train_merror], err, tolerance = 5e-6)\n # manually calculate error at the 1st iteration:\n mpred <- predict(bst, as.matrix(iris[, -5]), iterationrange = c(1, 1))\n pred_labels <- max.col(mpred) - 1\n err <- sum(pred_labels != lb) / length(lb)\n expect_equal(attributes(bst)$evaluation_log[1, train_merror], err, tolerance = 5e-6)\n\n mpred1 <- predict(bst, as.matrix(iris[, -5]), iterationrange = c(1, 1))\n expect_equal(mpred, mpred1)\n\n d <- cbind(\n x1 = rnorm(100),\n x2 = rnorm(100),\n x3 = rnorm(100)\n )\n y <- sample.int(10, 100, replace = TRUE) - 1\n dtrain <- xgb.DMatrix(data = d, label = y, nthread = n_threads)\n booster <- xgb.train(\n params = xgb.params(\n objective = \"multi:softprob\",\n num_class = 10,\n tree_method = \"hist\",\n nthread = n_threads\n ),\n data = dtrain,\n nrounds = 4\n )\n predt <- predict(booster, as.matrix(d), strict_shape = FALSE)\n expect_equal(ncol(predt), 10)\n expect_equal(rowSums(predt), rep(1, 100), tolerance = 1e-7)\n})\n\ntest_that(\"train and predict softmax\", {\n lb <- as.numeric(iris$Species) - 1\n set.seed(11)\n expect_output(\n bst <- xgb.train(\n data = xgb.DMatrix(as.matrix(iris[, -5]), label = lb, nthread = 1),\n nrounds = 5,\n params = xgb.params(\n max_depth = 3, learning_rate = 0.5, nthread = n_threads,\n objective = \"multi:softmax\", num_class = 3, eval_metric = \"merror\"\n ),\n evals = list(train = xgb.DMatrix(as.matrix(iris[, -5]), label = lb, nthread = 1))\n ),\n \"train-merror\"\n )\n expect_false(is.null(attributes(bst)$evaluation_log))\n expect_lt(attributes(bst)$evaluation_log[, min(train_merror)], 0.025)\n expect_equal(xgb.get.num.boosted.rounds(bst), 5)\n\n pred <- predict(bst, as.matrix(iris[, -5]))\n expect_length(pred, nrow(iris))\n err <- sum(pred != lb) / length(lb)\n expect_equal(attributes(bst)$evaluation_log[5, train_merror], err, tolerance = 5e-6)\n})\n\ntest_that(\"train and predict RF\", {\n set.seed(11)\n lb <- train$label\n # single iteration\n bst <- xgb.train(\n data = xgb.DMatrix(train$data, label = lb, nthread = 1),\n nrounds = 1,\n params = xgb.params(\n max_depth = 5,\n nthread = n_threads,\n objective = \"binary:logistic\", eval_metric = \"error\",\n num_parallel_tree = 20, subsample = 0.6, colsample_bytree = 0.1\n ),\n evals = list(train = xgb.DMatrix(train$data, label = lb, nthread = 1)),\n verbose = 0\n )\n expect_equal(xgb.get.num.boosted.rounds(bst), 1)\n\n pred <- predict(bst, train$data)\n pred_err <- sum((pred > 0.5) != lb) / length(lb)\n expect_lt(abs(attributes(bst)$evaluation_log[1, train_error] - pred_err), 10e-6)\n # expect_lt(pred_err, 0.03)\n\n pred <- predict(bst, train$data, iterationrange = c(1, 1))\n pred_err_20 <- sum((pred > 0.5) != lb) / length(lb)\n expect_equal(pred_err_20, pred_err)\n})\n\ntest_that(\"train and predict RF with softprob\", {\n lb <- as.numeric(iris$Species) - 1\n nrounds <- 15\n set.seed(11)\n bst <- xgb.train(\n data = xgb.DMatrix(as.matrix(iris[, -5]), label = lb, nthread = 1),\n nrounds = nrounds,\n verbose = 0,\n params = xgb.params(\n max_depth = 3,\n learning_rate = 0.9,\n nthread = n_threads,\n objective = \"multi:softprob\",\n eval_metric = \"merror\",\n num_class = 3,\n num_parallel_tree = 4,\n subsample = 0.5,\n colsample_bytree = 0.5\n ),\n evals = list(train = xgb.DMatrix(as.matrix(iris[, -5]), label = lb, nthread = 1))\n )\n expect_equal(xgb.get.num.boosted.rounds(bst), 15)\n # predict for all iterations:\n pred <- predict(bst, as.matrix(iris[, -5]))\n expect_equal(dim(pred), c(nrow(iris), 3))\n pred_labels <- max.col(pred) - 1\n err <- sum(pred_labels != lb) / length(lb)\n expect_equal(attributes(bst)$evaluation_log[nrounds, train_merror], err, tolerance = 5e-6)\n # predict for 7 iterations and adjust for 4 parallel trees per iteration\n pred <- predict(bst, as.matrix(iris[, -5]), iterationrange = c(1, 7))\n err <- sum((max.col(pred) - 1) != lb) / length(lb)\n expect_equal(attributes(bst)$evaluation_log[7, train_merror], err, tolerance = 5e-6)\n})\n\ntest_that(\"use of multiple eval metrics works\", {\n expect_output(\n bst <- xgb.train(\n data = xgb.DMatrix(train$data, label = train$label, nthread = 1),\n nrounds = 2,\n params = list(\n max_depth = 2,\n learning_rate = 1, nthread = n_threads, objective = \"binary:logistic\",\n eval_metric = \"error\", eval_metric = \"auc\", eval_metric = \"logloss\"\n ),\n evals = list(train = xgb.DMatrix(train$data, label = train$label, nthread = 1))\n ),\n \"train-error.*train-auc.*train-logloss\"\n )\n expect_false(is.null(attributes(bst)$evaluation_log))\n expect_equal(dim(attributes(bst)$evaluation_log), c(2, 4))\n expect_equal(colnames(attributes(bst)$evaluation_log), c(\"iter\", \"train_error\", \"train_auc\", \"train_logloss\"))\n expect_output(\n bst2 <- xgb.train(\n data = xgb.DMatrix(train$data, label = train$label, nthread = 1),\n nrounds = 2,\n params = xgb.params(\n max_depth = 2,\n learning_rate = 1,\n nthread = n_threads,\n objective = \"binary:logistic\",\n eval_metric = list(\"error\", \"auc\", \"logloss\")\n ),\n evals = list(train = xgb.DMatrix(train$data, label = train$label, nthread = 1))\n ),\n \"train-error.*train-auc.*train-logloss\"\n )\n expect_false(is.null(attributes(bst2)$evaluation_log))\n expect_equal(dim(attributes(bst2)$evaluation_log), c(2, 4))\n expect_equal(colnames(attributes(bst2)$evaluation_log), c(\"iter\", \"train_error\", \"train_auc\", \"train_logloss\"))\n})\n\n\ntest_that(\"training continuation works\", {\n dtrain <- xgb.DMatrix(train$data, label = train$label, nthread = n_threads)\n evals <- list(train = dtrain)\n params <- xgb.params(\n objective = \"binary:logistic\", max_depth = 2, learning_rate = 1, nthread = n_threads\n )\n\n # for the reference, use 4 iterations at once:\n set.seed(11)\n bst <- xgb.train(params, dtrain, nrounds = 4, evals = evals, verbose = 0)\n # first two iterations:\n set.seed(11)\n bst1 <- xgb.train(params, dtrain, nrounds = 2, evals = evals, verbose = 0)\n # continue for two more:\n bst2 <- xgb.train(params, dtrain, nrounds = 2, evals = evals, verbose = 0, xgb_model = bst1)\n if (!windows_flag && !solaris_flag) {\n expect_equal(xgb.save.raw(bst), xgb.save.raw(bst2))\n }\n expect_false(is.null(attributes(bst2)$evaluation_log))\n expect_equal(dim(attributes(bst2)$evaluation_log), c(4, 2))\n expect_equal(attributes(bst2)$evaluation_log, attributes(bst)$evaluation_log)\n # test continuing from raw model data\n bst2 <- xgb.train(params, dtrain, nrounds = 2, evals = evals, verbose = 0, xgb_model = xgb.save.raw(bst1))\n if (!windows_flag && !solaris_flag) {\n expect_equal(xgb.save.raw(bst), xgb.save.raw(bst2))\n }\n expect_equal(dim(attributes(bst2)$evaluation_log), c(2, 2))\n # test continuing from a model in file\n fname <- file.path(tempdir(), \"xgboost.json\")\n xgb.save(bst1, fname)\n bst2 <- xgb.train(params, dtrain, nrounds = 2, evals = evals, verbose = 0, xgb_model = fname)\n if (!windows_flag && !solaris_flag) {\n expect_equal(xgb.save.raw(bst), xgb.save.raw(bst2))\n }\n expect_equal(dim(attributes(bst2)$evaluation_log), c(2, 2))\n})\n\ntest_that(\"xgb.cv works\", {\n set.seed(11)\n expect_output(\n cv <- xgb.cv(\n data = xgb.DMatrix(train$data, label = train$label, nthread = 1),\n nfold = 5,\n nrounds = 2,\n params = xgb.params(\n max_depth = 2,\n learning_rate = 1.,\n nthread = n_threads,\n objective = \"binary:logistic\",\n eval_metric = \"error\"\n ),\n verbose = TRUE\n ),\n \"train-error:\"\n )\n expect_is(cv, \"xgb.cv.synchronous\")\n expect_false(is.null(cv$evaluation_log))\n expect_lt(cv$evaluation_log[, min(test_error_mean)], 0.03)\n expect_lt(cv$evaluation_log[, min(test_error_std)], 0.0085)\n expect_equal(cv$niter, 2)\n expect_false(is.null(cv$folds) && is.list(cv$folds))\n expect_length(cv$folds, 5)\n expect_false(is.null(cv$params) && is.list(cv$params))\n expect_false(is.null(cv$call))\n})\n\ntest_that(\"xgb.cv invalid inputs\", {\n data(\"mtcars\")\n y <- mtcars$mpg\n x_df <- mtcars[, -1]\n\n expect_error(\n cv <- xgb.cv(\n data = xgb.QuantileDMatrix(x_df, label = y),\n nfold = 5,\n nrounds = 2,\n params = xgb.params(\n max_depth = 2,\n nthread = n_threads\n )\n ),\n regexp = \".*QuantileDMatrix.*\"\n )\n expect_error(\n cv <- xgb.cv(\n data = xgb.DMatrix(x_df, label = y),\n nfold = 5,\n nrounds = 2,\n params = xgb.params(\n max_depth = 2,\n nthread = n_threads,\n ),\n callbacks = list(\n xgb.cb.early.stop(stopping_rounds = 3, save_best = TRUE)\n )\n ),\n regexp = \".*save_best.*\"\n )\n})\n\ntest_that(\"xgb.cv works with stratified folds\", {\n dtrain <- xgb.DMatrix(train$data, label = train$label, nthread = n_threads)\n set.seed(314159)\n cv <- xgb.cv(\n data = dtrain,\n nrounds = 2,\n nfold = 5,\n params = xgb.params(\n max_depth = 2,\n nthread = n_threads,\n objective = \"binary:logistic\"\n ),\n verbose = FALSE, stratified = FALSE\n )\n set.seed(314159)\n cv2 <- xgb.cv(\n data = dtrain,\n nfold = 5,\n nrounds = 2,\n params = xgb.params(\n max_depth = 2,\n nthread = n_threads,\n objective = \"binary:logistic\"\n ),\n verbose = FALSE, stratified = TRUE\n )\n # Stratified folds should result in a different evaluation logs\n expect_true(all(cv$evaluation_log[, test_logloss_mean] != cv2$evaluation_log[, test_logloss_mean]))\n})\n\ntest_that(\"train and predict with non-strict classes\", {\n # standard dense matrix input\n train_dense <- as.matrix(train$data)\n bst <- xgb.train(\n data = xgb.DMatrix(train_dense, label = train$label, nthread = 1),\n nrounds = 2,\n params = xgb.params(\n max_depth = 2,\n nthread = n_threads,\n objective = \"binary:logistic\"\n ),\n verbose = 0\n )\n pr0 <- predict(bst, train_dense)\n\n # dense matrix-like input of non-matrix class\n class(train_dense) <- \"shmatrix\"\n expect_true(is.matrix(train_dense))\n expect_error(\n bst <- xgb.train(\n data = xgb.DMatrix(train_dense, label = train$label, nthread = 1),\n nrounds = 2,\n params = xgb.params(\n max_depth = 2,\n nthread = n_threads,\n objective = \"binary:logistic\"\n ),\n verbose = 0\n ),\n regexp = NA\n )\n expect_error(pr <- predict(bst, train_dense), regexp = NA)\n expect_equal(pr0, pr)\n\n # dense matrix-like input of non-matrix class with some inheritance\n class(train_dense) <- c(\"pphmatrix\", \"shmatrix\")\n expect_true(is.matrix(train_dense))\n expect_error(\n bst <- xgb.train(\n data = xgb.DMatrix(train_dense, label = train$label, nthread = 1),\n nrounds = 2,\n params = xgb.params(\n max_depth = 2,\n nthread = n_threads,\n objective = \"binary:logistic\"\n ),\n verbose = 0\n ),\n regexp = NA\n )\n expect_error(pr <- predict(bst, train_dense), regexp = NA)\n expect_equal(pr0, pr)\n\n # when someone inherits from xgb.Booster, it should still be possible to use it as xgb.Booster\n class(bst) <- c(\"super.Booster\", \"xgb.Booster\")\n expect_error(pr <- predict(bst, train_dense), regexp = NA)\n expect_equal(pr0, pr)\n})\n\ntest_that(\"max_delta_step works\", {\n dtrain <- xgb.DMatrix(\n agaricus.train$data, label = agaricus.train$label, nthread = n_threads\n )\n evals <- list(train = dtrain)\n params <- xgb.params(\n objective = \"binary:logistic\", eval_metric = \"logloss\", max_depth = 2,\n nthread = n_threads,\n learning_rate = 0.5\n )\n nrounds <- 5\n # model with no restriction on max_delta_step\n bst1 <- xgb.train(params, dtrain, nrounds, evals = evals, verbose = 0)\n # model with restricted max_delta_step\n bst2 <- xgb.train(c(params, list(max_delta_step = 1)), dtrain, nrounds, evals = evals, verbose = 0)\n # the no-restriction model is expected to have consistently lower loss during the initial iterations\n expect_true(all(attributes(bst1)$evaluation_log$train_logloss < attributes(bst2)$evaluation_log$train_logloss))\n expect_lt(mean(attributes(bst1)$evaluation_log$train_logloss) / mean(attributes(bst2)$evaluation_log$train_logloss), 0.8)\n})\n\ntest_that(\"colsample_bytree works\", {\n # Randomly generate data matrix by sampling from uniform distribution [-1, 1]\n set.seed(1)\n train_x <- matrix(runif(1000, min = -1, max = 1), ncol = 100)\n train_y <- as.numeric(rowSums(train_x) > 0)\n test_x <- matrix(runif(1000, min = -1, max = 1), ncol = 100)\n test_y <- as.numeric(rowSums(test_x) > 0)\n colnames(train_x) <- paste0(\"Feature_\", sprintf(\"%03d\", 1:100))\n colnames(test_x) <- paste0(\"Feature_\", sprintf(\"%03d\", 1:100))\n dtrain <- xgb.DMatrix(train_x, label = train_y, nthread = n_threads)\n dtest <- xgb.DMatrix(test_x, label = test_y, nthread = n_threads)\n evals <- list(train = dtrain, eval = dtest)\n ## Use colsample_bytree = 0.01, so that roughly one out of 100 features is chosen for\n ## each tree\n params <- xgb.params(\n max_depth = 2, learning_rate = 0, nthread = n_threads,\n colsample_bytree = 0.01, objective = \"binary:logistic\",\n eval_metric = \"auc\"\n )\n set.seed(2)\n bst <- xgb.train(params, dtrain, nrounds = 100, evals = evals, verbose = 0)\n xgb.importance(model = bst)\n # If colsample_bytree works properly, a variety of features should be used\n # in the 100 trees\n expect_gte(nrow(xgb.importance(model = bst)), 28)\n})\n\ntest_that(\"Configuration works\", {\n bst <- xgb.train(\n data = xgb.DMatrix(train$data, label = train$label, nthread = 1),\n nrounds = 2,\n params = xgb.params(\n max_depth = 2,\n nthread = n_threads,\n objective = \"binary:logistic\"\n )\n )\n config <- xgb.config(bst)\n xgb.config(bst) <- config\n reloaded_config <- xgb.config(bst)\n expect_equal(config, reloaded_config)\n})\n\ntest_that(\"strict_shape works\", {\n n_rounds <- 2\n\n test_strict_shape <- function(bst, X, n_groups) {\n predt <- predict(bst, X, strict_shape = TRUE)\n margin <- predict(bst, X, outputmargin = TRUE, strict_shape = TRUE)\n contri <- predict(bst, X, predcontrib = TRUE, strict_shape = TRUE)\n interact <- predict(bst, X, predinteraction = TRUE, strict_shape = TRUE)\n leaf <- predict(bst, X, predleaf = TRUE, strict_shape = TRUE)\n\n n_rows <- nrow(X)\n n_cols <- ncol(X)\n\n expect_equal(dim(predt), c(n_rows, n_groups))\n expect_equal(dim(margin), c(n_rows, n_groups))\n expect_equal(dim(contri), c(n_rows, n_groups, n_cols + 1))\n expect_equal(dim(interact), c(n_rows, n_groups, n_cols + 1, n_cols + 1))\n expect_equal(dim(leaf), c(n_rows, n_rounds, n_groups, 1))\n\n if (n_groups != 1) {\n for (g in seq_len(n_groups)) {\n expect_lt(max(abs(rowSums(contri[, g, ]) - margin[, g])), 1e-5)\n }\n\n leaf_no_strict <- predict(bst, X, strict_shape = FALSE, predleaf = TRUE)\n for (g in seq_len(n_groups)) {\n g_mask <- rep(FALSE, n_groups)\n g_mask[g] <- TRUE\n expect_equal(\n leaf[, , g, 1L],\n leaf_no_strict[, g_mask]\n )\n }\n }\n }\n\n test_iris <- function() {\n y <- as.numeric(iris$Species) - 1\n X <- as.matrix(iris[, -5])\n\n bst <- xgb.train(\n data = xgb.DMatrix(X, label = y, nthread = 1),\n nrounds = n_rounds,\n params = xgb.params(\n max_depth = 2, nthread = n_threads,\n objective = \"multi:softprob\", num_class = 3\n )\n )\n\n test_strict_shape(bst, X, 3)\n }\n\n\n test_agaricus <- function() {\n data(agaricus.train, package = \"xgboost\")\n X <- agaricus.train$data\n y <- agaricus.train$label\n\n bst <- xgb.train(\n data = xgb.DMatrix(X, label = y, nthread = 1),\n nrounds = n_rounds,\n params = xgb.params(\n max_depth = 2, nthread = n_threads,\n objective = \"binary:logistic\"\n )\n )\n\n test_strict_shape(bst, X, 1)\n }\n\n test_iris()\n test_agaricus()\n})\n\ntest_that(\"'predict' accepts CSR data\", {\n X <- agaricus.train$data\n y <- agaricus.train$label\n x_csc <- as(X[1L, , drop = FALSE], \"CsparseMatrix\")\n x_csr <- as(x_csc, \"RsparseMatrix\")\n x_spv <- as(x_csc, \"sparseVector\")\n bst <- xgb.train(\n data = xgb.DMatrix(X, label = y, nthread = 1),\n nrounds = 5L, verbose = FALSE,\n params = xgb.params(\n objective = \"binary:logistic\",\n nthread = n_threads\n )\n )\n p_csc <- predict(bst, x_csc)\n p_csr <- predict(bst, x_csr)\n p_spv <- predict(bst, x_spv)\n expect_equal(p_csc, p_csr)\n expect_equal(p_csc, p_spv)\n})\n\ntest_that(\"Quantile regression accepts multiple quantiles\", {\n data(mtcars)\n y <- mtcars[, 1]\n x <- as.matrix(mtcars[, -1])\n dm <- xgb.DMatrix(data = x, label = y, nthread = 1)\n model <- xgb.train(\n data = dm,\n params = xgb.params(\n objective = \"reg:quantileerror\",\n tree_method = \"exact\",\n quantile_alpha = c(0.05, 0.5, 0.95),\n nthread = n_threads\n ),\n nrounds = 15\n )\n pred <- predict(model, x)\n\n expect_equal(dim(pred)[1], nrow(x))\n expect_equal(dim(pred)[2], 3)\n expect_true(all(pred[, 1] <= pred[, 3]))\n\n cors <- cor(y, pred)\n expect_true(cors[2] > cors[1])\n expect_true(cors[2] > cors[3])\n expect_true(cors[2] > 0.85)\n})\n\ntest_that(\"Can use multi-output labels with built-in objectives\", {\n data(\"mtcars\")\n y <- mtcars$mpg\n x <- as.matrix(mtcars[, -1])\n y_mirrored <- cbind(y, -y)\n dm <- xgb.DMatrix(x, label = y_mirrored, nthread = n_threads)\n model <- xgb.train(\n params = xgb.params(\n tree_method = \"hist\",\n multi_strategy = \"multi_output_tree\",\n objective = \"reg:squarederror\",\n nthread = n_threads\n ),\n data = dm,\n nrounds = 5\n )\n pred <- predict(model, x)\n expect_equal(pred[, 1], -pred[, 2])\n expect_true(cor(y, pred[, 1]) > 0.9)\n expect_true(cor(y, pred[, 2]) < -0.9)\n})\n\ntest_that(\"Can use multi-output labels with custom objectives\", {\n data(\"mtcars\")\n y <- mtcars$mpg\n x <- as.matrix(mtcars[, -1])\n y_mirrored <- cbind(y, -y)\n dm <- xgb.DMatrix(x, label = y_mirrored, nthread = n_threads)\n model <- xgb.train(\n params = xgb.params(\n tree_method = \"hist\",\n multi_strategy = \"multi_output_tree\",\n base_score = 0,\n objective = function(pred, dtrain) {\n y <- getinfo(dtrain, \"label\")\n grad <- pred - y\n hess <- rep(1, nrow(grad) * ncol(grad))\n hess <- matrix(hess, nrow = nrow(grad))\n return(list(grad = grad, hess = hess))\n },\n nthread = n_threads\n ),\n data = dm,\n nrounds = 5\n )\n pred <- predict(model, x)\n expect_equal(pred[, 1], -pred[, 2])\n expect_true(cor(y, pred[, 1]) > 0.9)\n expect_true(cor(y, pred[, 2]) < -0.9)\n})\n\ntest_that(\"Can use ranking objectives with either 'qid' or 'group'\", {\n set.seed(123)\n x <- matrix(rnorm(100 * 10), nrow = 100)\n y <- sample(2, size = 100, replace = TRUE) - 1\n qid <- c(rep(1, 20), rep(2, 20), rep(3, 60))\n gr <- c(20, 20, 60)\n\n dmat_qid <- xgb.DMatrix(x, label = y, qid = qid, nthread = 1)\n dmat_gr <- xgb.DMatrix(x, label = y, group = gr, nthread = 1)\n\n params <- xgb.params(\n tree_method = \"hist\",\n lambdarank_num_pair_per_sample = 8,\n objective = \"rank:ndcg\",\n lambdarank_pair_method = \"topk\",\n nthread = n_threads\n )\n set.seed(123)\n model_qid <- xgb.train(params, dmat_qid, nrounds = 5)\n set.seed(123)\n model_gr <- xgb.train(params, dmat_gr, nrounds = 5)\n\n pred_qid <- predict(model_qid, x)\n pred_gr <- predict(model_gr, x)\n expect_equal(pred_qid, pred_gr)\n})\n\ntest_that(\"Can predict on data.frame objects\", {\n data(\"mtcars\")\n y <- mtcars$mpg\n x_df <- mtcars[, -1]\n x_mat <- as.matrix(x_df)\n dm <- xgb.DMatrix(x_mat, label = y, nthread = n_threads)\n model <- xgb.train(\n params = xgb.params(\n tree_method = \"hist\",\n objective = \"reg:squarederror\",\n nthread = n_threads\n ),\n data = dm,\n nrounds = 5\n )\n\n pred_mat <- predict(model, xgb.DMatrix(x_mat, nthread = 1))\n pred_df <- predict(model, x_df)\n expect_equal(pred_mat, unname(pred_df))\n})\n\ntest_that(\"'base_margin' gives the same result in DMatrix as in inplace_predict\", {\n data(\"mtcars\")\n y <- mtcars$mpg\n x <- as.matrix(mtcars[, -1])\n dm <- xgb.DMatrix(x, label = y, nthread = n_threads)\n model <- xgb.train(\n params = xgb.params(\n tree_method = \"hist\",\n objective = \"reg:squarederror\",\n nthread = n_threads\n ),\n data = dm,\n nrounds = 5\n )\n\n set.seed(123)\n base_margin <- rnorm(nrow(x))\n dm_w_base <- xgb.DMatrix(data = x, base_margin = base_margin, nthread = 1)\n pred_from_dm <- predict(model, dm_w_base)\n pred_from_mat <- predict(model, x, base_margin = base_margin)\n\n expect_equal(pred_from_dm, unname(pred_from_mat))\n})\n\ntest_that(\"Coefficients from gblinear have the expected shape and names\", {\n # Single-column coefficients\n data(mtcars)\n y <- mtcars$mpg\n x <- as.matrix(mtcars[, -1])\n mm <- model.matrix(~., data = mtcars[, -1])\n dm <- xgb.DMatrix(x, label = y, nthread = 1)\n model <- xgb.train(\n data = dm,\n params = xgb.params(\n booster = \"gblinear\",\n nthread = 1\n ),\n nrounds = 3\n )\n coefs <- coef(model)\n expect_equal(length(coefs), ncol(x) + 1)\n expect_equal(names(coefs), c(\"(Intercept)\", colnames(x)))\n pred_auto <- predict(model, x)\n pred_manual <- as.numeric(mm %*% coefs)\n expect_equal(pred_manual, unname(pred_auto), tolerance = 1e-5)\n\n # Multi-column coefficients\n data(iris)\n y <- as.numeric(iris$Species) - 1\n x <- as.matrix(iris[, -5])\n dm <- xgb.DMatrix(x, label = y, nthread = 1)\n mm <- model.matrix(~., data = iris[, -5])\n model <- xgb.train(\n data = dm,\n params = xgb.params(\n booster = \"gblinear\",\n objective = \"multi:softprob\",\n num_class = 3,\n nthread = 1\n ),\n nrounds = 3\n )\n coefs <- coef(model)\n expect_equal(nrow(coefs), ncol(x) + 1)\n expect_equal(ncol(coefs), 3)\n expect_equal(row.names(coefs), c(\"(Intercept)\", colnames(x)))\n pred_auto <- predict(model, x, outputmargin = TRUE)\n pred_manual <- unname(mm %*% coefs)\n expect_equal(pred_manual, pred_auto, tolerance = 1e-7)\n\n # xgboost() with additional metadata\n model <- xgboost(\n iris[, -5],\n iris$Species,\n booster = \"gblinear\",\n objective = \"multi:softprob\",\n nrounds = 3,\n nthread = 1\n )\n coefs <- coef(model)\n expect_equal(row.names(coefs), c(\"(Intercept)\", colnames(x)))\n expect_equal(colnames(coefs), levels(iris$Species))\n})\n\ntest_that(\"Deep copies work as expected\", {\n data(mtcars)\n y <- mtcars$mpg\n x <- mtcars[, -1]\n dm <- xgb.DMatrix(x, label = y, nthread = 1)\n model <- xgb.train(\n data = dm,\n params = xgb.params(nthread = 1),\n nrounds = 3\n )\n\n xgb.attr(model, \"my_attr\") <- 100\n model_shallow_copy <- model\n xgb.attr(model_shallow_copy, \"my_attr\") <- 333\n attr_orig <- xgb.attr(model, \"my_attr\")\n attr_shallow <- xgb.attr(model_shallow_copy, \"my_attr\")\n expect_equal(attr_orig, attr_shallow)\n\n model_deep_copy <- xgb.copy.Booster(model)\n xgb.attr(model_deep_copy, \"my_attr\") <- 444\n attr_orig <- xgb.attr(model, \"my_attr\")\n attr_deep <- xgb.attr(model_deep_copy, \"my_attr\")\n expect_false(attr_orig == attr_deep)\n})\n\ntest_that(\"Pointer comparison works as expected\", {\n library(xgboost)\n y <- mtcars$mpg\n x <- as.matrix(mtcars[, -1])\n model <- xgb.train(\n params = xgb.params(nthread = 1),\n data = xgb.DMatrix(x, label = y, nthread = 1),\n nrounds = 3\n )\n\n model_shallow_copy <- model\n expect_true(xgb.is.same.Booster(model, model_shallow_copy))\n\n model_deep_copy <- xgb.copy.Booster(model)\n expect_false(xgb.is.same.Booster(model, model_deep_copy))\n\n xgb.attr(model_shallow_copy, \"my_attr\") <- 111\n expect_equal(xgb.attr(model, \"my_attr\"), \"111\")\n expect_null(xgb.attr(model_deep_copy, \"my_attr\"))\n})\n\ntest_that(\"DMatrix field are set to booster when training\", {\n set.seed(123)\n y <- rnorm(100)\n x <- matrix(rnorm(100 * 3), nrow = 100)\n x[, 2] <- abs(as.integer(x[, 2]))\n\n dm_unnamed <- xgb.DMatrix(x, label = y, nthread = 1)\n dm_feature_names <- xgb.DMatrix(x, label = y, feature_names = c(\"a\", \"b\", \"c\"), nthread = 1)\n dm_feature_types <- xgb.DMatrix(x, label = y, nthread = 1)\n setinfo(dm_feature_types, \"feature_type\", c(\"q\", \"c\", \"q\"))\n dm_both <- xgb.DMatrix(x, label = y, feature_names = c(\"a\", \"b\", \"c\"), nthread = 1)\n setinfo(dm_both, \"feature_type\", c(\"q\", \"c\", \"q\"))\n\n params <- xgb.params(nthread = 1)\n model_unnamed <- xgb.train(data = dm_unnamed, params = params, nrounds = 3)\n model_feature_names <- xgb.train(data = dm_feature_names, params = params, nrounds = 3)\n model_feature_types <- xgb.train(data = dm_feature_types, params = params, nrounds = 3)\n model_both <- xgb.train(data = dm_both, params = params, nrounds = 3)\n\n expect_null(getinfo(model_unnamed, \"feature_name\"))\n expect_equal(getinfo(model_feature_names, \"feature_name\"), c(\"a\", \"b\", \"c\"))\n expect_null(getinfo(model_feature_types, \"feature_name\"))\n expect_equal(getinfo(model_both, \"feature_name\"), c(\"a\", \"b\", \"c\"))\n\n expect_null(variable.names(model_unnamed))\n expect_equal(variable.names(model_feature_names), c(\"a\", \"b\", \"c\"))\n expect_null(variable.names(model_feature_types))\n expect_equal(variable.names(model_both), c(\"a\", \"b\", \"c\"))\n\n expect_null(getinfo(model_unnamed, \"feature_type\"))\n expect_null(getinfo(model_feature_names, \"feature_type\"))\n expect_equal(getinfo(model_feature_types, \"feature_type\"), c(\"q\", \"c\", \"q\"))\n expect_equal(getinfo(model_both, \"feature_type\"), c(\"q\", \"c\", \"q\"))\n})\n\ntest_that(\"Seed in params override PRNG from R\", {\n set.seed(123)\n model1 <- xgb.train(\n data = xgb.DMatrix(\n agaricus.train$data,\n label = agaricus.train$label, nthread = 1L\n ),\n params = xgb.params(\n objective = \"binary:logistic\",\n max_depth = 3L,\n subsample = 0.1,\n colsample_bytree = 0.1,\n seed = 111L\n ),\n nrounds = 3L\n )\n\n set.seed(456)\n model2 <- xgb.train(\n data = xgb.DMatrix(\n agaricus.train$data,\n label = agaricus.train$label, nthread = 1L\n ),\n params = xgb.params(\n objective = \"binary:logistic\",\n max_depth = 3L,\n subsample = 0.1,\n colsample_bytree = 0.1,\n seed = 111L\n ),\n nrounds = 3L\n )\n\n expect_equal(\n xgb.save.raw(model1, raw_format = \"json\"),\n xgb.save.raw(model2, raw_format = \"json\")\n )\n\n set.seed(123)\n model3 <- xgb.train(\n data = xgb.DMatrix(\n agaricus.train$data,\n label = agaricus.train$label, nthread = 1L\n ),\n params = xgb.params(\n objective = \"binary:logistic\",\n max_depth = 3L,\n subsample = 0.1,\n colsample_bytree = 0.1,\n seed = 222L\n ),\n nrounds = 3L\n )\n expect_false(\n isTRUE(\n all.equal(\n xgb.save.raw(model1, raw_format = \"json\"),\n xgb.save.raw(model3, raw_format = \"json\")\n )\n )\n )\n})\n\ntest_that(\"xgb.cv works for AFT\", {\n X <- matrix(c(1, -1, -1, 1, 0, 1, 1, 0), nrow = 4, byrow = TRUE) # 4x2 matrix\n dtrain <- xgb.DMatrix(X, nthread = n_threads)\n\n params <- xgb.params(objective = 'survival:aft', learning_rate = 0.2, max_depth = 2L, nthread = n_threads)\n\n # data must have bounds\n expect_error(\n xgb.cv(\n params = params,\n data = dtrain,\n nround = 5L,\n nfold = 4L\n )\n )\n\n setinfo(dtrain, 'label_lower_bound', c(2, 3, 0, 4))\n setinfo(dtrain, 'label_upper_bound', c(2, Inf, 4, 5))\n\n # automatic stratified splitting is turned off\n expect_warning(\n xgb.cv(\n params = params, data = dtrain, nround = 5L, nfold = 4L,\n stratified = TRUE, verbose = FALSE\n )\n )\n\n # this works without any issue\n expect_no_warning(\n xgb.cv(params = params, data = dtrain, nround = 5L, nfold = 4L, verbose = FALSE)\n )\n})\n\ntest_that(\"xgb.cv works for ranking\", {\n data(iris)\n x <- iris[, -(4:5)]\n y <- as.integer(iris$Petal.Width)\n group <- rep(50, 3)\n dm <- xgb.DMatrix(x, label = y, group = group, nthread = 1)\n res <- xgb.cv(\n data = dm,\n params = xgb.params(\n objective = \"rank:pairwise\",\n max_depth = 3,\n nthread = 1L\n ),\n nrounds = 3,\n nfold = 2,\n verbose = FALSE,\n stratified = FALSE\n )\n expect_equal(length(res$folds), 2L)\n})\n\ntest_that(\"Row names are preserved in outputs\", {\n data(iris)\n x <- iris[, -5]\n y <- as.numeric(iris$Species) - 1\n dm <- xgb.DMatrix(x, label = y, nthread = 1)\n model <- xgb.train(\n data = dm,\n params = xgb.params(\n objective = \"multi:softprob\",\n num_class = 3,\n max_depth = 2,\n nthread = 1\n ),\n nrounds = 3\n )\n row.names(x) <- paste0(\"r\", seq(1, nrow(x)))\n pred <- predict(model, x)\n expect_equal(row.names(pred), row.names(x))\n pred <- predict(model, x, avoid_transpose = TRUE)\n expect_equal(colnames(pred), row.names(x))\n\n data(mtcars)\n y <- mtcars[, 1]\n x <- as.matrix(mtcars[, -1])\n dm <- xgb.DMatrix(data = x, label = y, nthread = 1)\n model <- xgb.train(\n data = dm,\n params = xgb.params(\n max_depth = 2,\n nthread = 1\n ),\n nrounds = 3\n )\n row.names(x) <- paste0(\"r\", seq(1, nrow(x)))\n pred <- predict(model, x)\n expect_equal(names(pred), row.names(x))\n pred <- predict(model, x, avoid_transpose = TRUE)\n expect_equal(names(pred), row.names(x))\n pred <- predict(model, x, predleaf = TRUE)\n expect_equal(row.names(pred), row.names(x))\n pred <- predict(model, x, predleaf = TRUE, avoid_transpose = TRUE)\n expect_equal(colnames(pred), row.names(x))\n})\n\ntest_that(\"xgb.train works with nrounds=0 (serialization, continuation, callbacks)\", {\n # Reuse global data variable 'train' defined at the top of test_basic.R\n dtrain <- xgb.DMatrix(train$data, label = train$label)\n watchlist <- list(train = dtrain)\n\n # --- Case 1: Basic check & Serialization symmetry ---\n bst_0 <- xgb.train(\n params = list(objective = \"binary:logistic\", nthread = 1),\n data = dtrain,\n nrounds = 0,\n verbose = 0\n )\n\n # Check niter is 0 (handling NULL case common for empty boosters)\n iter_0 <- bst_0$niter\n if (is.null(iter_0)) {\n iter_0 <- 0\n }\n expect_equal(iter_0, 0)\n\n # Check that 0-round model provides a valid \"base score\" prediction\n preds_0 <- predict(bst_0, dtrain)\n expect_true(all((preds_0 >= 0) & (preds_0 <= 1)))\n\n # Serialize via RAM (Raw) instead of disk (tempfile) for cleaner tests\n raw <- xgb.save.raw(bst_0)\n bst_loaded <- xgb.load.raw(raw)\n\n # Verify predictions match before/after serialization\n preds_loaded <- predict(bst_loaded, dtrain)\n expect_equal(preds_0, preds_loaded, tolerance = 1e-6)\n\n # --- Case 2: Training Continuation Numeric Consistency ---\n # Initialize empty model with fixed seed & single thread\n bst_init <- xgb.train(\n params = list(objective = \"binary:logistic\", seed = 123, nthread = 1),\n data = dtrain,\n nrounds = 0,\n verbose = 0\n )\n\n # Continue training for 10 rounds from empty booster\n bst_cont <- xgb.train(\n params = list(objective = \"binary:logistic\", seed = 123, nthread = 1),\n data = dtrain,\n nrounds = 10,\n xgb_model = bst_init,\n verbose = 0\n )\n\n # Reference training from scratch\n bst_ref <- xgb.train(\n params = list(objective = \"binary:logistic\", seed = 123, nthread = 1),\n data = dtrain,\n nrounds = 10,\n verbose = 0\n )\n\n # Predictions must be numerically identical within 1e-6\n p_cont <- predict(bst_cont, dtrain)\n p_ref <- predict(bst_ref, dtrain)\n expect_equal(p_cont, p_ref, tolerance = 1e-6)\n\n # --- Case 3: Callback Robustness ---\n # Verify early stopping and evals work with nrounds=0\n bst_cb <- xgb.train(\n params = list(objective = \"binary:logistic\", seed = 456, nthread = 1),\n data = dtrain,\n nrounds = 0,\n evals = watchlist,\n early_stopping_rounds = 3,\n verbose = 0\n )\n\n # Verify that continuation works\n bst_cb_cont <- xgb.train(\n params = list(objective = \"binary:logistic\", seed = 456, nthread = 1),\n data = dtrain,\n nrounds = 5,\n evals = watchlist,\n early_stopping_rounds = 3,\n xgb_model = bst_cb,\n verbose = 0\n )\n\n # Handle NULL niter for continued model with early stopping\n iter_cb <- bst_cb_cont$niter\n if (is.null(iter_cb)) {\n # Verify the continued model works and learned signal\n preds_cb <- predict(bst_cb_cont, dtrain)\n expect_true(stats::sd(preds_cb) > 0)\n expect_equal(length(preds_cb), nrow(train$data))\n } else {\n expect_equal(iter_cb, 5)\n }\n})\n" }, { "path": "R-package/tests/testthat/test_booster_slicing.R", "content": "context(\"testing xgb.Booster slicing\")\n\ndata(agaricus.train, package = \"xgboost\")\ndm <- xgb.DMatrix(agaricus.train$data, label = agaricus.train$label, nthread = 1)\n# Note: here need large step sizes in order for the predictions\n# to have substantially different leaf assignments on each tree\nmodel <- xgb.train(\n params = xgb.params(objective = \"binary:logistic\", nthread = 1, max_depth = 4),\n data = dm,\n nrounds = 20\n)\npred <- predict(model, dm, predleaf = TRUE)\n\ntest_that(\"Slicing full model\", {\n new_model <- xgb.slice.Booster(model, 1, 0)\n expect_equal(xgb.save.raw(new_model), xgb.save.raw(model))\n\n new_model <- model[]\n expect_equal(xgb.save.raw(new_model), xgb.save.raw(model))\n\n new_model <- model[1:length(model)] # nolint\n expect_equal(xgb.save.raw(new_model), xgb.save.raw(model))\n})\n\ntest_that(\"Slicing sequence from start\", {\n new_model <- xgb.slice.Booster(model, 1, 10)\n new_pred <- predict(new_model, dm, predleaf = TRUE)\n expect_equal(new_pred, pred[, seq(1, 10)])\n\n new_model <- model[1:10]\n new_pred <- predict(new_model, dm, predleaf = TRUE)\n expect_equal(new_pred, pred[, seq(1, 10)])\n})\n\ntest_that(\"Slicing sequence from middle\", {\n new_model <- xgb.slice.Booster(model, 5, 10)\n new_pred <- predict(new_model, dm, predleaf = TRUE)\n expect_equal(new_pred, pred[, seq(5, 10)])\n\n new_model <- model[5:10]\n new_pred <- predict(new_model, dm, predleaf = TRUE)\n expect_equal(new_pred, pred[, seq(5, 10)])\n})\n\ntest_that(\"Slicing with non-unit step\", {\n for (s in 2:5) {\n new_model <- xgb.slice.Booster(model, 1, 17, s)\n new_pred <- predict(new_model, dm, predleaf = TRUE)\n expect_equal(new_pred, pred[, seq(1, 17, s)])\n\n new_model <- model[seq(1, 17, s)]\n new_pred <- predict(new_model, dm, predleaf = TRUE)\n expect_equal(new_pred, pred[, seq(1, 17, s)])\n }\n})\n\ntest_that(\"Slicing with non-unit step from middle\", {\n for (s in 2:5) {\n new_model <- xgb.slice.Booster(model, 4, 17, s)\n new_pred <- predict(new_model, dm, predleaf = TRUE)\n expect_equal(new_pred, pred[, seq(4, 17, s)])\n\n new_model <- model[seq(4, 17, s)]\n new_pred <- predict(new_model, dm, predleaf = TRUE)\n expect_equal(new_pred, pred[, seq(4, 17, s)])\n }\n})\n" }, { "path": "R-package/tests/testthat/test_callbacks.R", "content": "# More specific testing of callbacks\ncontext(\"callbacks\")\n\ndata(agaricus.train, package = 'xgboost')\ndata(agaricus.test, package = 'xgboost')\ntrain <- agaricus.train\ntest <- agaricus.test\n\nn_threads <- 2\n\n# add some label noise for early stopping tests\nadd.noise <- function(label, frac) {\n inoise <- sample(length(label), length(label) * frac)\n label[inoise] <- !label[inoise]\n label\n}\nset.seed(11)\nltrain <- add.noise(train$label, 0.2)\nltest <- add.noise(test$label, 0.2)\ndtrain <- xgb.DMatrix(train$data, label = ltrain, nthread = n_threads)\ndtest <- xgb.DMatrix(test$data, label = ltest, nthread = n_threads)\nevals <- list(train = dtrain, test = dtest)\n\n\nerr <- function(label, pr) sum((pr > 0.5) != label) / length(label)\n\nparams <- xgb.params(\n objective = \"binary:logistic\", eval_metric = \"error\",\n max_depth = 2, nthread = n_threads\n)\n\n\ntest_that(\"xgb.cb.print.evaluation works as expected for xgb.train\", {\n logs1 <- capture.output({\n model <- xgb.train(\n data = dtrain,\n params = xgb.params(\n objective = \"binary:logistic\",\n eval_metric = \"auc\",\n max_depth = 2,\n nthread = n_threads\n ),\n nrounds = 10,\n evals = list(train = dtrain, test = dtest),\n callbacks = list(xgb.cb.print.evaluation(period = 1))\n )\n })\n expect_equal(length(logs1), 10)\n expect_true(all(grepl(\"^\\\\[\\\\d{1,2}\\\\]\\ttrain-auc:0\\\\.\\\\d+\\ttest-auc:0\\\\.\\\\d+\\\\s*$\", logs1)))\n lapply(seq(1, 10), function(x) expect_true(grepl(paste0(\"^\\\\[\", x), logs1[x])))\n\n logs2 <- capture.output({\n model <- xgb.train(\n data = dtrain,\n params = xgb.params(\n objective = \"binary:logistic\",\n eval_metric = \"auc\",\n max_depth = 2,\n nthread = n_threads\n ),\n nrounds = 10,\n evals = list(train = dtrain, test = dtest),\n callbacks = list(xgb.cb.print.evaluation(period = 2))\n )\n })\n expect_equal(length(logs2), 6)\n expect_true(all(grepl(\"^\\\\[\\\\d{1,2}\\\\]\\ttrain-auc:0\\\\.\\\\d+\\ttest-auc:0\\\\.\\\\d+\\\\s*$\", logs2)))\n seq_matches <- c(seq(1, 10, 2), 10)\n lapply(seq_along(seq_matches), function(x) expect_true(grepl(paste0(\"^\\\\[\", seq_matches[x]), logs2[x])))\n})\n\ntest_that(\"xgb.cb.print.evaluation works as expected for xgb.cv\", {\n logs1 <- capture.output({\n model <- xgb.cv(\n data = dtrain,\n params = xgb.params(\n objective = \"binary:logistic\",\n eval_metric = \"auc\",\n max_depth = 2,\n nthread = n_threads\n ),\n nrounds = 10,\n nfold = 3,\n callbacks = list(xgb.cb.print.evaluation(period = 1, showsd = TRUE))\n )\n })\n expect_equal(length(logs1), 10)\n expect_true(all(grepl(\"^\\\\[\\\\d{1,2}\\\\]\\ttrain-auc:0\\\\.\\\\d+±0\\\\.\\\\d+\\ttest-auc:0\\\\.\\\\d+±0\\\\.\\\\d+\\\\s*$\", logs1)))\n lapply(seq(1, 10), function(x) expect_true(grepl(paste0(\"^\\\\[\", x), logs1[x])))\n\n logs2 <- capture.output({\n model <- xgb.cv(\n data = dtrain,\n params = xgb.params(\n objective = \"binary:logistic\",\n eval_metric = \"auc\",\n max_depth = 2,\n nthread = n_threads\n ),\n nrounds = 10,\n nfold = 3,\n callbacks = list(xgb.cb.print.evaluation(period = 2, showsd = TRUE))\n )\n })\n expect_equal(length(logs2), 6)\n expect_true(all(grepl(\"^\\\\[\\\\d{1,2}\\\\]\\ttrain-auc:0\\\\.\\\\d+±0\\\\.\\\\d+\\ttest-auc:0\\\\.\\\\d+±0\\\\.\\\\d+\\\\s*$\", logs2)))\n seq_matches <- c(seq(1, 10, 2), 10)\n lapply(seq_along(seq_matches), function(x) expect_true(grepl(paste0(\"^\\\\[\", seq_matches[x]), logs2[x])))\n})\n\ntest_that(\"xgb.cb.evaluation.log works as expected for xgb.train\", {\n model <- xgb.train(\n data = dtrain,\n params = xgb.params(\n objective = \"binary:logistic\",\n eval_metric = \"auc\",\n max_depth = 2,\n nthread = n_threads\n ),\n nrounds = 10,\n verbose = FALSE,\n evals = list(train = dtrain, test = dtest),\n callbacks = list(xgb.cb.evaluation.log())\n )\n logs <- attributes(model)$evaluation_log\n\n expect_equal(nrow(logs), 10)\n expect_equal(colnames(logs), c(\"iter\", \"train_auc\", \"test_auc\"))\n})\n\ntest_that(\"xgb.cb.evaluation.log works as expected for xgb.cv\", {\n model <- xgb.cv(\n data = dtrain,\n params = xgb.params(\n objective = \"binary:logistic\",\n eval_metric = \"auc\",\n max_depth = 2,\n nthread = n_threads\n ),\n nrounds = 10,\n verbose = FALSE,\n nfold = 3,\n callbacks = list(xgb.cb.evaluation.log())\n )\n logs <- model$evaluation_log\n\n expect_equal(nrow(logs), 10)\n expect_equal(\n colnames(logs),\n c(\"iter\", \"train_auc_mean\", \"train_auc_std\", \"test_auc_mean\", \"test_auc_std\")\n )\n})\n\n\nparams <- xgb.params(\n objective = \"binary:logistic\", eval_metric = \"error\",\n max_depth = 4, nthread = n_threads\n)\n\ntest_that(\"can store evaluation_log without printing\", {\n expect_silent(\n bst <- xgb.train(params, dtrain, nrounds = 10, evals = evals, verbose = 0)\n )\n expect_false(is.null(attributes(bst)$evaluation_log))\n expect_false(is.null(attributes(bst)$evaluation_log$train_error))\n expect_lt(attributes(bst)$evaluation_log[, min(train_error)], 0.2)\n})\n\ntest_that(\"xgb.cb.reset.parameters works as expected\", {\n\n # fixed learning_rate\n params <- c(params, list(learning_rate = 0.9))\n set.seed(111)\n bst0 <- xgb.train(params, dtrain, nrounds = 2, evals = evals, verbose = 0)\n expect_false(is.null(attributes(bst0)$evaluation_log))\n expect_false(is.null(attributes(bst0)$evaluation_log$train_error))\n\n # same learning_rate but re-set as a vector parameter in the callback\n set.seed(111)\n my_par <- list(learning_rate = c(0.9, 0.9))\n bst1 <- xgb.train(params, dtrain, nrounds = 2, evals = evals, verbose = 0,\n callbacks = list(xgb.cb.reset.parameters(my_par)))\n expect_false(is.null(attributes(bst1)$evaluation_log$train_error))\n expect_equal(attributes(bst0)$evaluation_log$train_error,\n attributes(bst1)$evaluation_log$train_error)\n\n # same learning_rate but re-set via a function in the callback\n set.seed(111)\n my_par <- list(learning_rate = function(itr, itr_end) 0.9)\n bst2 <- xgb.train(params, dtrain, nrounds = 2, evals = evals, verbose = 0,\n callbacks = list(xgb.cb.reset.parameters(my_par)))\n expect_false(is.null(attributes(bst2)$evaluation_log$train_error))\n expect_equal(attributes(bst0)$evaluation_log$train_error,\n attributes(bst2)$evaluation_log$train_error)\n\n # different learning_rate re-set as a vector parameter in the callback\n set.seed(111)\n my_par <- list(learning_rate = c(0.6, 0.5))\n bst3 <- xgb.train(params, dtrain, nrounds = 2, evals = evals, verbose = 0,\n callbacks = list(xgb.cb.reset.parameters(my_par)))\n expect_false(is.null(attributes(bst3)$evaluation_log$train_error))\n expect_false(all(attributes(bst0)$evaluation_log$train_error == attributes(bst3)$evaluation_log$train_error))\n\n # resetting multiple parameters at the same time runs with no error\n my_par <- list(learning_rate = c(1., 0.5), min_split_loss = c(1, 2), max_depth = c(4, 8))\n expect_error(\n bst4 <- xgb.train(params, dtrain, nrounds = 2, evals = evals, verbose = 0,\n callbacks = list(xgb.cb.reset.parameters(my_par)))\n , NA) # NA = no error\n # CV works as well\n expect_error(\n bst4 <- xgb.cv(params, dtrain, nfold = 2, nrounds = 2, verbose = 0,\n callbacks = list(xgb.cb.reset.parameters(my_par)))\n , NA) # NA = no error\n\n # expect no learning with 0 learning rate\n my_par <- list(learning_rate = c(0., 0.))\n bstX <- xgb.train(params, dtrain, nrounds = 2, evals = evals, verbose = 0,\n callbacks = list(xgb.cb.reset.parameters(my_par)))\n expect_false(is.null(attributes(bstX)$evaluation_log$train_error))\n er <- unique(attributes(bstX)$evaluation_log$train_error)\n expect_length(er, 1)\n expect_gt(er, 0.4)\n})\n\ntest_that(\"xgb.cb.save.model works as expected\", {\n files <- c('xgboost_01.json', 'xgboost_02.json', 'xgboost.json')\n files <- unname(sapply(files, function(f) file.path(tempdir(), f)))\n for (f in files) if (file.exists(f)) file.remove(f)\n\n bst <- xgb.train(params, dtrain, nrounds = 2, evals = evals, verbose = 0,\n save_period = 1, save_name = file.path(tempdir(), \"xgboost_%02d.json\"))\n expect_true(file.exists(files[1]))\n expect_true(file.exists(files[2]))\n b1 <- xgb.load(files[1])\n xgb.model.parameters(b1) <- list(nthread = 2)\n expect_equal(xgb.get.num.boosted.rounds(b1), 1)\n b2 <- xgb.load(files[2])\n xgb.model.parameters(b2) <- list(nthread = 2)\n expect_equal(xgb.get.num.boosted.rounds(b2), 2)\n\n xgb.config(b2) <- xgb.config(bst)\n expect_equal(xgb.config(bst), xgb.config(b2))\n expect_equal(xgb.save.raw(bst), xgb.save.raw(b2))\n\n # save_period = 0 saves the last iteration's model\n bst <- xgb.train(params, dtrain, nrounds = 2, evals = evals, verbose = 0,\n save_period = 0, save_name = file.path(tempdir(), 'xgboost.json'))\n expect_true(file.exists(files[3]))\n b2 <- xgb.load(files[3])\n xgb.config(b2) <- xgb.config(bst)\n expect_equal(xgb.save.raw(bst), xgb.save.raw(b2))\n\n for (f in files) if (file.exists(f)) file.remove(f)\n})\n\ntest_that(\"early stopping xgb.train works\", {\n params <- c(params, list(learning_rate = 0.3))\n set.seed(11)\n expect_output(\n bst <- xgb.train(params, dtrain, nrounds = 20, evals = evals,\n early_stopping_rounds = 3, maximize = FALSE)\n , \"Stopping. Best iteration\")\n expect_false(is.null(xgb.attr(bst, \"best_iteration\")))\n expect_lt(xgb.attr(bst, \"best_iteration\"), 19)\n\n pred <- predict(bst, dtest)\n expect_equal(length(pred), 1611)\n err_pred <- err(ltest, pred)\n err_log <- attributes(bst)$evaluation_log[xgb.attr(bst, \"best_iteration\") + 1, test_error]\n expect_equal(err_log, err_pred, tolerance = 5e-6)\n\n set.seed(11)\n expect_silent(\n bst0 <- xgb.train(params, dtrain, nrounds = 20, evals = evals,\n early_stopping_rounds = 3, maximize = FALSE, verbose = 0)\n )\n expect_equal(attributes(bst)$evaluation_log, attributes(bst0)$evaluation_log)\n\n fname <- file.path(tempdir(), \"model.ubj\")\n xgb.save(bst, fname)\n loaded <- xgb.load(fname)\n\n expect_false(is.null(xgb.attr(loaded, \"best_iteration\")))\n expect_equal(xgb.attr(loaded, \"best_iteration\"), xgb.attr(bst, \"best_iteration\"))\n})\n\ntest_that(\"early stopping using a specific metric works\", {\n set.seed(11)\n expect_output(\n bst <- xgb.train(\n c(\n within(params, rm(\"eval_metric\")),\n list(\n learning_rate = 0.6,\n eval_metric = \"logloss\",\n eval_metric = \"auc\"\n )\n ),\n dtrain,\n nrounds = 20,\n evals = evals,\n callbacks = list(\n xgb.cb.early.stop(stopping_rounds = 3, maximize = FALSE, metric_name = 'test_logloss')\n )\n )\n , \"Stopping. Best iteration\")\n expect_false(is.null(xgb.attr(bst, \"best_iteration\")))\n expect_lt(xgb.attr(bst, \"best_iteration\"), 19)\n\n pred <- predict(bst, dtest, iterationrange = c(1, xgb.attr(bst, \"best_iteration\") + 1))\n expect_equal(length(pred), 1611)\n logloss_pred <- sum(-ltest * log(pred) - (1 - ltest) * log(1 - pred)) / length(ltest)\n logloss_log <- attributes(bst)$evaluation_log[xgb.attr(bst, \"best_iteration\") + 1, test_logloss]\n expect_equal(logloss_log, logloss_pred, tolerance = 1e-5)\n})\n\ntest_that(\"early stopping works with titanic\", {\n if (!requireNamespace(\"titanic\")) {\n testthat::skip(\"Optional testing dependency 'titanic' not found.\")\n }\n # This test was inspired by https://github.com/dmlc/xgboost/issues/5935\n # It catches possible issues on noLD R\n titanic <- titanic::titanic_train\n titanic$Pclass <- as.factor(titanic$Pclass)\n dtx <- model.matrix(~ 0 + ., data = titanic[, c(\"Pclass\", \"Sex\")])\n dty <- titanic$Survived\n\n xgb.train(\n data = xgb.DMatrix(dtx, label = dty, nthread = 1),\n params = xgb.params(\n objective = \"binary:logistic\",\n eval_metric = \"auc\",\n nthread = n_threads\n ),\n nrounds = 100,\n early_stopping_rounds = 3,\n verbose = 0,\n evals = list(train = xgb.DMatrix(dtx, label = dty, nthread = 1))\n )\n\n expect_true(TRUE) # should not crash\n})\n\ntest_that(\"early stopping xgb.cv works\", {\n set.seed(11)\n expect_output(\n {\n cv <- xgb.cv(\n c(params, list(learning_rate = 0.3)),\n dtrain,\n nfold = 5,\n nrounds = 20,\n early_stopping_rounds = 3,\n maximize = FALSE\n )\n },\n \"Stopping. Best iteration\"\n )\n expect_false(is.null(cv$early_stop$best_iteration))\n expect_lt(cv$early_stop$best_iteration, 19)\n # the best error is min error:\n expect_true(cv$evaluation_log[, test_error_mean[cv$early_stop$best_iteration] == min(test_error_mean)])\n})\n\ntest_that(\"prediction in xgb.cv works\", {\n params <- c(params, list(learning_rate = 0.5))\n set.seed(11)\n nrounds <- 4\n cv <- xgb.cv(params, dtrain, nfold = 5, nrounds = nrounds, prediction = TRUE, verbose = 0)\n expect_false(is.null(cv$evaluation_log))\n expect_false(is.null(cv$cv_predict$pred))\n expect_length(cv$cv_predict$pred, nrow(train$data))\n err_pred <- mean(sapply(cv$folds, function(f) mean(err(ltrain[f], cv$cv_predict$pred[f]))))\n err_log <- cv$evaluation_log[nrounds, test_error_mean]\n expect_equal(err_pred, err_log, tolerance = 1e-6)\n\n # save CV models\n set.seed(11)\n cvx <- xgb.cv(params, dtrain, nfold = 5, nrounds = nrounds, prediction = TRUE, verbose = 0,\n callbacks = list(xgb.cb.cv.predict(save_models = TRUE)))\n expect_equal(cv$evaluation_log, cvx$evaluation_log)\n expect_length(cvx$cv_predict$models, 5)\n expect_true(all(sapply(cvx$cv_predict$models, class) == 'xgb.Booster'))\n})\n\ntest_that(\"prediction in xgb.cv works for gblinear too\", {\n set.seed(11)\n p <- xgb.params(\n booster = 'gblinear',\n objective = \"reg:logistic\",\n learning_rate = 0.5,\n nthread = n_threads\n )\n cv <- xgb.cv(p, dtrain, nfold = 5, nrounds = 2, prediction = TRUE, verbose = 0)\n expect_false(is.null(cv$evaluation_log))\n expect_false(is.null(cv$cv_predict$pred))\n expect_length(cv$cv_predict$pred, nrow(train$data))\n})\n\ntest_that(\"prediction in early-stopping xgb.cv works\", {\n params <- c(params, list(learning_rate = 0.1, base_score = 0.5))\n set.seed(11)\n expect_output(\n cv <- xgb.cv(params, dtrain, nfold = 5, nrounds = 20,\n early_stopping_rounds = 5, maximize = FALSE, stratified = FALSE,\n prediction = TRUE, verbose = TRUE)\n , \"Stopping. Best iteration\")\n\n expect_false(is.null(cv$early_stop$best_iteration))\n expect_lt(cv$early_stop$best_iteration, 19)\n expect_false(is.null(cv$evaluation_log))\n expect_false(is.null(cv$cv_predict$pred))\n expect_length(cv$cv_predict$pred, nrow(train$data))\n\n err_pred <- mean(sapply(cv$folds, function(f) mean(err(ltrain[f], cv$cv_predict$pred[f]))))\n err_log <- cv$evaluation_log[cv$early_stop$best_iteration, test_error_mean]\n expect_equal(err_pred, err_log, tolerance = 1e-6)\n err_log_last <- cv$evaluation_log[cv$niter, test_error_mean]\n expect_gt(abs(err_pred - err_log_last), 1e-4)\n})\n\ntest_that(\"prediction in xgb.cv for softprob works\", {\n lb <- as.numeric(iris$Species) - 1\n set.seed(11)\n expect_warning(\n {\n cv <- xgb.cv(\n data = xgb.DMatrix(as.matrix(iris[, -5]), label = lb, nthread = 1),\n nfold = 4,\n nrounds = 5,\n params = xgb.params(\n objective = \"multi:softprob\",\n num_class = 3,\n learning_rate = 0.5,\n max_depth = 3,\n nthread = n_threads,\n subsample = 0.8,\n min_split_loss = 2\n ),\n verbose = 0,\n prediction = TRUE\n )\n },\n NA\n )\n expect_false(is.null(cv$cv_predict$pred))\n expect_equal(dim(cv$cv_predict$pred), c(nrow(iris), 3))\n expect_lt(diff(range(rowSums(cv$cv_predict$pred))), 1e-6)\n})\n\ntest_that(\"prediction in xgb.cv works for multi-quantile\", {\n data(mtcars)\n y <- mtcars$mpg\n x <- as.matrix(mtcars[, -1])\n dm <- xgb.DMatrix(x, label = y, nthread = 1)\n cv <- xgb.cv(\n data = dm,\n params = xgb.params(\n objective = \"reg:quantileerror\",\n quantile_alpha = c(0.1, 0.2, 0.5, 0.8, 0.9),\n nthread = 1\n ),\n nrounds = 5,\n nfold = 3,\n prediction = TRUE,\n verbose = 0\n )\n expect_equal(dim(cv$cv_predict$pred), c(nrow(x), 5))\n})\n\ntest_that(\"prediction in xgb.cv works for multi-output\", {\n data(mtcars)\n y <- mtcars$mpg\n x <- as.matrix(mtcars[, -1])\n dm <- xgb.DMatrix(x, label = cbind(y, -y), nthread = 1)\n cv <- xgb.cv(\n data = dm,\n params = xgb.params(\n tree_method = \"hist\",\n multi_strategy = \"multi_output_tree\",\n objective = \"reg:squarederror\",\n nthread = n_threads\n ),\n nrounds = 5,\n nfold = 3,\n prediction = TRUE,\n verbose = 0\n )\n expect_equal(dim(cv$cv_predict$pred), c(nrow(x), 2))\n})\n\ntest_that(\"prediction in xgb.cv works for multi-quantile\", {\n data(mtcars)\n y <- mtcars$mpg\n x <- as.matrix(mtcars[, -1])\n dm <- xgb.DMatrix(x, label = y, nthread = 1)\n cv <- xgb.cv(\n data = dm,\n params = xgb.params(\n objective = \"reg:quantileerror\",\n quantile_alpha = c(0.1, 0.2, 0.5, 0.8, 0.9),\n nthread = 1\n ),\n nrounds = 5,\n nfold = 3,\n prediction = TRUE,\n verbose = 0\n )\n expect_equal(dim(cv$cv_predict$pred), c(nrow(x), 5))\n})\n\ntest_that(\"prediction in xgb.cv works for multi-output\", {\n data(mtcars)\n y <- mtcars$mpg\n x <- as.matrix(mtcars[, -1])\n dm <- xgb.DMatrix(x, label = cbind(y, -y), nthread = 1)\n cv <- xgb.cv(\n data = dm,\n params = xgb.params(\n tree_method = \"hist\",\n multi_strategy = \"multi_output_tree\",\n objective = \"reg:squarederror\",\n nthread = n_threads\n ),\n nrounds = 5,\n nfold = 3,\n prediction = TRUE,\n verbose = 0\n )\n expect_equal(dim(cv$cv_predict$pred), c(nrow(x), 2))\n})\n" }, { "path": "R-package/tests/testthat/test_config.R", "content": "context('Test global configuration')\n\ntest_that('Global configuration works with verbosity', {\n old_verbosity <- xgb.get.config()$verbosity\n for (v in c(0, 1, 2, 3)) {\n xgb.set.config(verbosity = v)\n expect_equal(xgb.get.config()$verbosity, v)\n }\n xgb.set.config(verbosity = old_verbosity)\n expect_equal(xgb.get.config()$verbosity, old_verbosity)\n})\n\ntest_that('Global configuration works with use_rmm flag', {\n old_use_rmm_flag <- xgb.get.config()$use_rmm\n for (v in c(TRUE, FALSE)) {\n xgb.set.config(use_rmm = v)\n expect_equal(xgb.get.config()$use_rmm, v)\n }\n xgb.set.config(use_rmm = old_use_rmm_flag)\n expect_equal(xgb.get.config()$use_rmm, old_use_rmm_flag)\n})\n" }, { "path": "R-package/tests/testthat/test_custom_objective.R", "content": "context('Test models with custom objective')\n\nset.seed(1994)\n\nn_threads <- 2\n\ndata(agaricus.train, package = 'xgboost')\ndata(agaricus.test, package = 'xgboost')\ndtrain <- xgb.DMatrix(\n agaricus.train$data, label = agaricus.train$label, nthread = n_threads\n)\ndtest <- xgb.DMatrix(\n agaricus.test$data, label = agaricus.test$label, nthread = n_threads\n)\nevals <- list(eval = dtest, train = dtrain)\n\nlogregobj <- function(preds, dtrain) {\n labels <- getinfo(dtrain, \"label\")\n preds <- 1 / (1 + exp(-preds))\n grad <- preds - labels\n hess <- preds * (1 - preds)\n return(list(grad = grad, hess = hess))\n}\n\nevalerror <- function(preds, dtrain) {\n labels <- getinfo(dtrain, \"label\")\n err <- as.numeric(sum(labels != (preds > 0.5))) / length(labels)\n return(list(metric = \"error\", value = err))\n}\n\nparam <- list(max_depth = 2, learning_rate = 1, nthread = n_threads,\n objective = logregobj, eval_metric = evalerror)\nnum_round <- 2\n\ntest_that(\"custom objective works\", {\n bst <- xgb.train(param, dtrain, num_round, evals, verbose = 0)\n expect_equal(class(bst), \"xgb.Booster\")\n expect_false(is.null(attributes(bst)$evaluation_log))\n expect_false(is.null(attributes(bst)$evaluation_log$eval_error))\n expect_lt(attributes(bst)$evaluation_log[num_round, eval_error], 0.03)\n})\n\ntest_that(\"custom objective in CV works\", {\n cv <- xgb.cv(param, dtrain, num_round, nfold = 10, verbose = FALSE, stratified = FALSE)\n expect_false(is.null(cv$evaluation_log))\n expect_equal(dim(cv$evaluation_log), c(2, 5))\n expect_lt(cv$evaluation_log[num_round, test_error_mean], 0.03)\n})\n\ntest_that(\"custom objective with early stop works\", {\n bst <- xgb.train(param, dtrain, 10, evals, verbose = 0)\n expect_equal(class(bst), \"xgb.Booster\")\n train_log <- attributes(bst)$evaluation_log$train_error\n expect_true(all(diff(train_log) <= 0))\n})\n\ntest_that(\"custom objective using DMatrix attr works\", {\n\n attr(dtrain, 'label') <- getinfo(dtrain, 'label')\n\n logregobjattr <- function(preds, dtrain) {\n labels <- attr(dtrain, 'label')\n preds <- 1 / (1 + exp(-preds))\n grad <- preds - labels\n hess <- preds * (1 - preds)\n return(list(grad = grad, hess = hess))\n }\n param$objective <- logregobjattr\n bst <- xgb.train(param, dtrain, num_round, evals, verbose = 0)\n expect_equal(class(bst), \"xgb.Booster\")\n})\n\ntest_that(\"custom objective with multi-class shape\", {\n data <- as.matrix(iris[, -5])\n label <- as.numeric(iris$Species) - 1\n dtrain <- xgb.DMatrix(data = data, label = label, nthread = n_threads)\n n_classes <- 3\n\n fake_softprob <- function(preds, dtrain) {\n mpreds <- as.matrix(preds)\n expect_equal(\n mpreds,\n matrix(0.5, nrow = nrow(mpreds), ncol = ncol(mpreds)),\n tolerance = 1e-4\n )\n ## use numeric vector here to test compatibility with XGBoost < 2.1\n grad <- rnorm(length(as.matrix(preds)))\n expect_equal(dim(data)[1] * n_classes, dim(as.matrix(preds))[1] * n_classes)\n hess <- rnorm(length(as.matrix(preds)))\n list(grad = grad, hess = hess)\n }\n fake_merror <- function(preds, dtrain) {\n expect_equal(dim(data)[1] * n_classes, dim(as.matrix(preds))[1])\n }\n param$objective <- fake_softprob\n param$eval_metric <- fake_merror\n expect_warning({\n bst <- xgb.train(c(param, list(num_class = n_classes)), dtrain, nrounds = 1)\n })\n})\n\nsoftmax <- function(values) {\n values <- as.numeric(values)\n exps <- exp(values)\n den <- sum(exps)\n return(exps / den)\n}\n\nsoftprob <- function(predt, dtrain) {\n y <- getinfo(dtrain, \"label\")\n\n n_samples <- dim(predt)[1]\n n_classes <- dim(predt)[2]\n\n grad <- matrix(nrow = n_samples, ncol = n_classes)\n hess <- matrix(nrow = n_samples, ncol = n_classes)\n\n for (i in seq_len(n_samples)) {\n t <- y[i]\n p <- softmax(predt[i, ])\n for (c in seq_len(n_classes)) {\n g <- if (c - 1 == t) {\n p[c] - 1.0\n } else {\n p[c]\n }\n h <- max((2.0 * p[c] * (1.0 - p[c])), 1e-6)\n grad[i, c] <- g\n hess[i, c] <- h\n }\n }\n\n return(list(grad = grad, hess = hess))\n}\n\n\ntest_that(\"custom objective with multi-class works\", {\n data <- as.matrix(iris[, -5])\n label <- as.numeric(iris$Species) - 1\n\n dtrain <- xgb.DMatrix(data = data, label = label, nthread = 1)\n\n param$num_class <- 3\n param$objective <- softprob\n param$eval_metric <- \"merror\"\n param$base_score <- 0.5\n\n custom_bst <- xgb.train(param, dtrain, 2)\n custom_predt <- predict(custom_bst, dtrain)\n\n param$objective <- \"multi:softmax\"\n builtin_bst <- xgb.train(param, dtrain, 2)\n builtin_predt <- predict(builtin_bst, dtrain)\n\n expect_equal(custom_predt, builtin_predt)\n})\n\ntest_that(\"custom metric with multi-target passes reshaped data to feval\", {\n x <- as.matrix(iris[, -5])\n y <- as.numeric(iris$Species) - 1\n dtrain <- xgb.DMatrix(data = x, label = y, nthread = 1)\n\n multinomial.ll <- function(predt, dtrain) {\n expect_equal(dim(predt), c(nrow(iris), 3L))\n y <- getinfo(dtrain, \"label\")\n probs <- apply(predt, 1, softmax) |> t()\n probs.y <- probs[cbind(seq(1L, nrow(predt)), y + 1L)]\n ll <- sum(log(probs.y))\n return(list(metric = \"multinomial-ll\", value = -ll))\n }\n\n model <- xgb.train(\n params = list(\n objective = \"multi:softmax\",\n num_class = 3L,\n base_score = 0,\n disable_default_eval_metric = TRUE,\n eval_metric = multinomial.ll,\n max_depth = 123,\n seed = 123\n ),\n data = dtrain,\n nrounds = 2L,\n evals = list(Train = dtrain),\n verbose = 0\n )\n\n model <- xgb.train(\n params = list(\n objective = \"multi:softmax\",\n num_class = 3L,\n base_score = 0,\n disable_default_eval_metric = TRUE,\n max_depth = 123,\n seed = 123\n ),\n data = dtrain,\n nrounds = 2L,\n evals = list(Train = dtrain),\n custom_metric = multinomial.ll,\n verbose = 0\n )\n})\n" }, { "path": "R-package/tests/testthat/test_dmatrix.R", "content": "library(Matrix)\ncontext(\"testing xgb.DMatrix functionality\")\n\ndata(agaricus.test, package = \"xgboost\")\ntest_data <- agaricus.test$data[1:100, ]\ntest_label <- agaricus.test$label[1:100]\n\nn_threads <- 2\n\ntest_that(\"xgb.DMatrix: basic construction\", {\n # from sparse matrix\n dtest1 <- xgb.DMatrix(test_data, label = test_label, nthread = n_threads)\n\n # from dense matrix\n dtest2 <- xgb.DMatrix(as.matrix(test_data), label = test_label, nthread = n_threads)\n expect_equal(getinfo(dtest1, \"label\"), getinfo(dtest2, \"label\"))\n expect_equal(dim(dtest1), dim(dtest2))\n\n # from dense integer matrix\n int_data <- as.matrix(test_data)\n storage.mode(int_data) <- \"integer\"\n dtest3 <- xgb.DMatrix(int_data, label = test_label, nthread = n_threads)\n expect_equal(dim(dtest1), dim(dtest3))\n\n n_samples <- 100\n X <- cbind(\n x1 = sample(x = 4, size = n_samples, replace = TRUE),\n x2 = sample(x = 4, size = n_samples, replace = TRUE),\n x3 = sample(x = 4, size = n_samples, replace = TRUE)\n )\n X <- matrix(X, nrow = n_samples)\n y <- rbinom(n = n_samples, size = 1, prob = 1 / 2)\n\n fd <- xgb.DMatrix(X, label = y, missing = 1, nthread = n_threads)\n\n dgc <- as(X, \"dgCMatrix\")\n fdgc <- xgb.DMatrix(dgc, label = y, missing = 1.0, nthread = n_threads)\n\n dgr <- as(X, \"dgRMatrix\")\n fdgr <- xgb.DMatrix(dgr, label = y, missing = 1, nthread = n_threads)\n\n params <- list(tree_method = \"hist\", nthread = n_threads)\n bst_fd <- xgb.train(\n params, nrounds = 8, fd, evals = list(train = fd), verbose = 0\n )\n bst_dgr <- xgb.train(\n params, nrounds = 8, fdgr, evals = list(train = fdgr), verbose = 0\n )\n bst_dgc <- xgb.train(\n params, nrounds = 8, fdgc, evals = list(train = fdgc), verbose = 0\n )\n\n raw_fd <- xgb.save.raw(bst_fd, raw_format = \"ubj\")\n raw_dgr <- xgb.save.raw(bst_dgr, raw_format = \"ubj\")\n raw_dgc <- xgb.save.raw(bst_dgc, raw_format = \"ubj\")\n\n expect_equal(raw_fd, raw_dgr)\n expect_equal(raw_fd, raw_dgc)\n})\n\ntest_that(\"xgb.DMatrix: NA\", {\n n_samples <- 3\n x <- cbind(\n x1 = sample(x = 4, size = n_samples, replace = TRUE),\n x2 = sample(x = 4, size = n_samples, replace = TRUE)\n )\n x[1, \"x1\"] <- NA\n\n m <- xgb.DMatrix(x, nthread = n_threads)\n fname_int <- file.path(tempdir(), \"int.dmatrix\")\n xgb.DMatrix.save(m, fname_int)\n\n x <- matrix(as.numeric(x), nrow = n_samples, ncol = 2)\n colnames(x) <- c(\"x1\", \"x2\")\n m <- xgb.DMatrix(x, nthread = n_threads)\n\n fname_float <- file.path(tempdir(), \"float.dmatrix\")\n xgb.DMatrix.save(m, fname_float)\n\n iconn <- file(fname_int, \"rb\")\n fconn <- file(fname_float, \"rb\")\n\n expect_equal(file.size(fname_int), file.size(fname_float))\n\n bytes <- file.size(fname_int)\n idmatrix <- readBin(iconn, \"raw\", n = bytes)\n fdmatrix <- readBin(fconn, \"raw\", n = bytes)\n\n expect_equal(length(idmatrix), length(fdmatrix))\n expect_equal(idmatrix, fdmatrix)\n\n close(iconn)\n close(fconn)\n\n file.remove(fname_int)\n file.remove(fname_float)\n})\n\ntest_that(\"xgb.DMatrix: saving, loading\", {\n # save to a local file\n dtest1 <- xgb.DMatrix(test_data, label = test_label, nthread = n_threads)\n tmp_file <- tempfile('xgb.DMatrix_')\n on.exit(unlink(tmp_file))\n expect_true(xgb.DMatrix.save(dtest1, tmp_file))\n # read from a local file\n xgb.set.config(verbosity = 2)\n expect_output(dtest3 <- xgb.DMatrix(tmp_file, nthread = 1), \"entries loaded from\")\n xgb.set.config(verbosity = 1)\n expect_output(dtest3 <- xgb.DMatrix(tmp_file, nthread = 1), NA)\n unlink(tmp_file)\n expect_equal(getinfo(dtest1, 'label'), getinfo(dtest3, 'label'))\n\n # from a libsvm text file\n tmp <- c(\"0 1:1 2:1\", \"1 3:1\", \"0 1:1\")\n tmp_file <- tempfile(fileext = \".libsvm\")\n writeLines(tmp, tmp_file)\n expect_true(file.exists(tmp_file))\n dtest4 <- xgb.DMatrix(\n paste(tmp_file, \"?format=libsvm\", sep = \"\"), silent = TRUE, nthread = n_threads\n )\n expect_equal(dim(dtest4), c(3, 4))\n expect_equal(getinfo(dtest4, 'label'), c(0, 1, 0))\n\n # check that feature info is saved\n data(agaricus.train, package = 'xgboost')\n dtrain <- xgb.DMatrix(\n data = agaricus.train$data, label = agaricus.train$label, nthread = n_threads\n )\n cnames <- colnames(dtrain)\n expect_equal(length(cnames), 126)\n tmp_file <- tempfile('xgb.DMatrix_')\n xgb.DMatrix.save(dtrain, tmp_file)\n xgb.set.config(verbosity = 0)\n dtrain <- xgb.DMatrix(tmp_file, nthread = 1)\n expect_equal(colnames(dtrain), cnames)\n\n ft <- rep(c(\"c\", \"q\"), each = length(cnames) / 2)\n setinfo(dtrain, \"feature_type\", ft)\n expect_equal(ft, getinfo(dtrain, \"feature_type\"))\n})\n\ntest_that(\"xgb.DMatrix: getinfo & setinfo\", {\n dtest <- xgb.DMatrix(test_data, nthread = n_threads)\n expect_true(setinfo(dtest, 'label', test_label))\n labels <- getinfo(dtest, 'label')\n expect_equal(test_label, getinfo(dtest, 'label'))\n\n expect_true(setinfo(dtest, 'label_lower_bound', test_label))\n expect_equal(test_label, getinfo(dtest, 'label_lower_bound'))\n\n expect_true(setinfo(dtest, 'label_upper_bound', test_label))\n expect_equal(test_label, getinfo(dtest, 'label_upper_bound'))\n\n expect_true(length(getinfo(dtest, 'weight')) == 0)\n expect_true(length(getinfo(dtest, 'base_margin')) == 0)\n\n expect_true(setinfo(dtest, 'weight', test_label))\n expect_true(setinfo(dtest, 'base_margin', test_label))\n expect_true(setinfo(dtest, 'group', c(50, 50)))\n expect_error(setinfo(dtest, 'group', test_label))\n\n # providing character values will give an error\n expect_error(setinfo(dtest, 'weight', rep('a', nrow(test_data))))\n\n # any other label should error\n expect_error(setinfo(dtest, 'asdf', test_label))\n})\n\ntest_that(\"xgb.DMatrix: slice, dim\", {\n dtest <- xgb.DMatrix(test_data, label = test_label, nthread = n_threads)\n expect_equal(dim(dtest), dim(test_data))\n dsub1 <- xgb.slice.DMatrix(dtest, 1:42)\n expect_equal(nrow(dsub1), 42)\n expect_equal(ncol(dsub1), ncol(test_data))\n\n dsub2 <- dtest[1:42, ]\n expect_equal(dim(dtest), dim(test_data))\n expect_equal(getinfo(dsub1, 'label'), getinfo(dsub2, 'label'))\n})\n\ntest_that(\"xgb.DMatrix: slice, trailing empty rows\", {\n data(agaricus.train, package = 'xgboost')\n train_data <- agaricus.train$data\n train_label <- agaricus.train$label\n dtrain <- xgb.DMatrix(\n data = train_data, label = train_label, nthread = n_threads\n )\n xgb.slice.DMatrix(dtrain, 6513L)\n train_data[6513, ] <- 0\n dtrain <- xgb.DMatrix(\n data = train_data, label = train_label, nthread = n_threads\n )\n xgb.slice.DMatrix(dtrain, 6513L)\n expect_equal(nrow(dtrain), 6513)\n})\n\ntest_that(\"xgb.DMatrix: colnames\", {\n dtest <- xgb.DMatrix(test_data, label = test_label, nthread = n_threads)\n expect_equal(colnames(dtest), colnames(test_data))\n expect_error(colnames(dtest) <- 'asdf')\n new_names <- make.names(seq_len(ncol(test_data)))\n expect_silent(colnames(dtest) <- new_names)\n expect_equal(colnames(dtest), new_names)\n expect_silent(colnames(dtest) <- NULL)\n expect_null(colnames(dtest))\n})\n\ntest_that(\"xgb.DMatrix: nrow is correct for a very sparse matrix\", {\n set.seed(123)\n nr <- 1000\n x <- Matrix::rsparsematrix(nr, 100, density = 0.0005)\n # we want it very sparse, so that last rows are empty\n expect_lt(max(x@i), nr)\n dtest <- xgb.DMatrix(x, nthread = n_threads)\n expect_equal(dim(dtest), dim(x))\n})\n\ntest_that(\"xgb.DMatrix: print\", {\n data(agaricus.train, package = 'xgboost')\n\n # core DMatrix with just data and labels\n dtrain <- xgb.DMatrix(\n data = agaricus.train$data, label = agaricus.train$label,\n nthread = n_threads\n )\n txt <- capture.output({\n print(dtrain)\n })\n expect_equal(txt, \"xgb.DMatrix dim: 6513 x 126 info: label colnames: yes\")\n\n # verbose=TRUE prints feature names\n txt <- capture.output({\n print(dtrain, verbose = TRUE)\n })\n expect_equal(txt[[1L]], \"xgb.DMatrix dim: 6513 x 126 info: label colnames:\")\n expect_equal(txt[[2L]], sprintf(\"'%s'\", paste(colnames(dtrain), collapse = \"','\")))\n\n # DMatrix with weights and base_margin\n dtrain <- xgb.DMatrix(\n data = agaricus.train$data,\n label = agaricus.train$label,\n weight = seq_along(agaricus.train$label),\n base_margin = agaricus.train$label,\n nthread = n_threads\n )\n txt <- capture.output({\n print(dtrain)\n })\n expect_equal(txt, \"xgb.DMatrix dim: 6513 x 126 info: base_margin, label, weight colnames: yes\")\n\n # DMatrix with just features\n dtrain <- xgb.DMatrix(\n data = agaricus.train$data,\n nthread = n_threads\n )\n txt <- capture.output({\n print(dtrain)\n })\n expect_equal(txt, \"xgb.DMatrix dim: 6513 x 126 info: NA colnames: yes\")\n\n # DMatrix with no column names\n data_no_colnames <- agaricus.train$data\n colnames(data_no_colnames) <- NULL\n dtrain <- xgb.DMatrix(\n data = data_no_colnames,\n nthread = n_threads\n )\n txt <- capture.output({\n print(dtrain)\n })\n expect_equal(txt, \"xgb.DMatrix dim: 6513 x 126 info: NA colnames: no\")\n})\n\ntest_that(\"xgb.DMatrix: Inf as missing\", {\n x_inf <- matrix(as.numeric(1:10), nrow = 5)\n x_inf[2, 1] <- Inf\n\n x_nan <- x_inf\n x_nan[2, 1] <- NA_real_\n\n m_inf <- xgb.DMatrix(x_inf, nthread = n_threads, missing = Inf)\n fname_inf <- file.path(tempdir(), \"inf.dmatrix\")\n xgb.DMatrix.save(m_inf, fname_inf)\n\n m_nan <- xgb.DMatrix(x_nan, nthread = n_threads, missing = NA_real_)\n fname_nan <- file.path(tempdir(), \"nan.dmatrix\")\n xgb.DMatrix.save(m_nan, fname_nan)\n\n infconn <- file(fname_inf, \"rb\")\n nanconn <- file(fname_nan, \"rb\")\n\n expect_equal(file.size(fname_inf), file.size(fname_nan))\n\n bytes <- file.size(fname_inf)\n infdmatrix <- readBin(infconn, \"raw\", n = bytes)\n nandmatrix <- readBin(nanconn, \"raw\", n = bytes)\n\n expect_equal(length(infdmatrix), length(nandmatrix))\n expect_equal(infdmatrix, nandmatrix)\n\n close(infconn)\n close(nanconn)\n\n file.remove(fname_inf)\n file.remove(fname_nan)\n})\n\ntest_that(\"xgb.DMatrix: missing in CSR\", {\n x_dense <- matrix(as.numeric(1:10), nrow = 5)\n x_dense[2, 1] <- NA_real_\n\n x_csr <- as(x_dense, \"RsparseMatrix\")\n\n m_dense <- xgb.DMatrix(x_dense, nthread = n_threads, missing = NA_real_)\n xgb.DMatrix.save(m_dense, \"dense.dmatrix\")\n\n m_csr <- xgb.DMatrix(x_csr, nthread = n_threads, missing = NA)\n xgb.DMatrix.save(m_csr, \"csr.dmatrix\")\n\n denseconn <- file(\"dense.dmatrix\", \"rb\")\n csrconn <- file(\"csr.dmatrix\", \"rb\")\n\n expect_equal(file.size(\"dense.dmatrix\"), file.size(\"csr.dmatrix\"))\n\n bytes <- file.size(\"dense.dmatrix\")\n densedmatrix <- readBin(denseconn, \"raw\", n = bytes)\n csrmatrix <- readBin(csrconn, \"raw\", n = bytes)\n\n expect_equal(length(densedmatrix), length(csrmatrix))\n expect_equal(densedmatrix, csrmatrix)\n\n close(denseconn)\n close(csrconn)\n\n file.remove(\"dense.dmatrix\")\n file.remove(\"csr.dmatrix\")\n})\n\ntest_that(\"xgb.DMatrix: error on three-dimensional array\", {\n set.seed(123)\n x <- matrix(rnorm(500), nrow = 50)\n y <- rnorm(400)\n dim(y) <- c(50, 4, 2)\n expect_error(xgb.DMatrix(data = x, label = y))\n})\n\ntest_that(\"xgb.DMatrix: can get group for both 'qid' and 'group' constructors\", {\n set.seed(123)\n x <- matrix(rnorm(1000), nrow = 100)\n group <- c(20, 20, 60)\n qid <- c(rep(1, 20), rep(2, 20), rep(3, 60))\n\n gr_mat <- xgb.DMatrix(x, group = group, nthread = 1)\n qid_mat <- xgb.DMatrix(x, qid = qid, nthread = 1)\n\n info_gr <- getinfo(gr_mat, \"group\")\n info_qid <- getinfo(qid_mat, \"group\")\n expect_equal(info_gr, info_qid)\n\n expected_gr <- c(0, 20, 40, 100)\n expect_equal(info_gr, expected_gr)\n})\n\ntest_that(\"xgb.DMatrix: data.frame\", {\n df <- data.frame(\n a = (1:4) / 10,\n num = c(1, NA, 3, 4),\n as.int = as.integer(c(1, 2, 3, 4)),\n lo = c(TRUE, FALSE, NA, TRUE),\n str.fac = c(\"a\", \"b\", \"d\", \"c\"),\n as.fac = as.factor(c(3, 5, 8, 11)),\n stringsAsFactors = TRUE\n )\n\n m <- xgb.DMatrix(df, nthread = 1)\n expect_equal(colnames(m), colnames(df))\n expect_equal(\n getinfo(m, \"feature_type\"), c(\"float\", \"float\", \"int\", \"i\", \"c\", \"c\")\n )\n\n df <- data.frame(\n missing = c(\"a\", \"b\", \"d\", NA),\n valid = c(\"a\", \"b\", \"d\", \"c\"),\n stringsAsFactors = TRUE\n )\n m <- xgb.DMatrix(df, nthread = 1)\n expect_equal(getinfo(m, \"feature_type\"), c(\"c\", \"c\"))\n})\n\ntest_that(\"xgb.DMatrix: can take multi-dimensional 'base_margin'\", {\n set.seed(123)\n x <- matrix(rnorm(100 * 10), nrow = 100)\n y <- matrix(rnorm(100 * 2), nrow = 100)\n b <- matrix(rnorm(100 * 2), nrow = 100)\n model <- xgb.train(\n data = xgb.DMatrix(data = x, label = y, nthread = n_threads),\n params = list(\n objective = \"reg:squarederror\",\n tree_method = \"hist\",\n multi_strategy = \"multi_output_tree\",\n base_score = 0,\n nthread = n_threads\n ),\n nround = 1\n )\n pred_only_x <- predict(model, x)\n pred_w_base <- predict(\n model,\n xgb.DMatrix(data = x, base_margin = b, nthread = 1)\n )\n expect_equal(pred_only_x, pred_w_base - b, tolerance = 1e-5)\n})\n\ntest_that(\"xgb.DMatrix: QuantileDMatrix produces same result as DMatrix\", {\n data(mtcars)\n y <- mtcars[, 1]\n x <- mtcars[, -1]\n\n cast_matrix <- function(x) as.matrix(x)\n cast_df <- function(x) as.data.frame(x)\n cast_csr <- function(x) as(as.matrix(x), \"RsparseMatrix\")\n casting_funs <- list(cast_matrix, cast_df, cast_csr)\n\n for (casting_fun in casting_funs) {\n\n qdm <- xgb.QuantileDMatrix(\n data = casting_fun(x),\n label = y,\n nthread = n_threads,\n max_bin = 5\n )\n params <- list(\n tree_method = \"hist\",\n objective = \"reg:squarederror\",\n nthread = n_threads,\n max_bin = 5\n )\n model_qdm <- xgb.train(\n params = params,\n data = qdm,\n nrounds = 2\n )\n pred_qdm <- predict(model_qdm, x)\n\n dm <- xgb.DMatrix(\n data = x,\n label = y,\n nthread = n_threads\n )\n model_dm <- xgb.train(\n params = params,\n data = dm,\n nrounds = 2\n )\n pred_dm <- predict(model_dm, x)\n\n expect_equal(pred_qdm, pred_dm)\n }\n})\n\ntest_that(\"xgb.DMatrix: QuantileDMatrix is not accepted by exact method\", {\n data(mtcars)\n y <- mtcars[, 1]\n x <- as.matrix(mtcars[, -1])\n qdm <- xgb.QuantileDMatrix(\n data = x,\n label = y,\n nthread = n_threads\n )\n params <- list(\n tree_method = \"exact\",\n objective = \"reg:squarederror\",\n nthread = n_threads\n )\n expect_error({\n xgb.train(\n params = params,\n data = qdm,\n nrounds = 2\n )\n })\n})\n\ntest_that(\"xgb.DMatrix: ExtMemDMatrix produces the same results as regular DMatrix\", {\n data(mtcars)\n y <- mtcars[, 1]\n x <- as.matrix(mtcars[, -1])\n set.seed(123)\n params <- list(\n objective = \"reg:squarederror\",\n nthread = n_threads\n )\n model <- xgb.train(\n data = xgb.DMatrix(x, label = y, nthread = 1),\n params = params,\n nrounds = 5\n )\n pred <- predict(model, x)\n pred <- unname(pred)\n\n iterator_env <- as.environment(\n list(\n iter = 0,\n x = mtcars[, -1],\n y = mtcars[, 1]\n )\n )\n iterator_next <- function(iterator_env) {\n curr_iter <- iterator_env[[\"iter\"]]\n if (curr_iter >= 2) {\n return(NULL)\n }\n if (curr_iter == 0) {\n x_batch <- iterator_env[[\"x\"]][1:16, ]\n y_batch <- iterator_env[[\"y\"]][1:16]\n } else {\n x_batch <- iterator_env[[\"x\"]][17:32, ]\n y_batch <- iterator_env[[\"y\"]][17:32]\n }\n on.exit({\n iterator_env[[\"iter\"]] <- curr_iter + 1\n })\n return(xgb.DataBatch(data = x_batch, label = y_batch))\n }\n iterator_reset <- function(iterator_env) {\n iterator_env[[\"iter\"]] <- 0\n }\n data_iterator <- xgb.DataIter(\n env = iterator_env,\n f_next = iterator_next,\n f_reset = iterator_reset\n )\n cache_prefix <- tempdir()\n edm <- xgb.ExtMemDMatrix(data_iterator, cache_prefix, nthread = 1)\n expect_true(inherits(edm, \"xgb.ExtMemDMatrix\"))\n expect_true(inherits(edm, \"xgb.DMatrix\"))\n set.seed(123)\n model_ext <- xgb.train(\n data = edm,\n params = params,\n nrounds = 5\n )\n\n pred_model1_edm <- predict(model, edm)\n pred_model2_mat <- predict(model_ext, x) |> unname()\n pred_model2_edm <- predict(model_ext, edm)\n\n expect_equal(pred_model1_edm, pred)\n expect_equal(pred_model2_mat, pred)\n expect_equal(pred_model2_edm, pred)\n})\n\ntest_that(\"xgb.DMatrix: External QDM produces same results as regular QDM\", {\n data(mtcars)\n y <- mtcars[, 1]\n x <- as.matrix(mtcars[, -1])\n set.seed(123)\n params <- list(\n objective = \"reg:squarederror\",\n nthread = n_threads,\n max_bin = 3\n )\n model <- xgb.train(\n data = xgb.QuantileDMatrix(\n x,\n label = y,\n nthread = 1,\n max_bin = 3\n ),\n params = params,\n nrounds = 5\n )\n pred <- predict(model, x)\n pred <- unname(pred)\n\n iterator_env <- as.environment(\n list(\n iter = 0,\n x = mtcars[, -1],\n y = mtcars[, 1]\n )\n )\n iterator_next <- function(iterator_env) {\n curr_iter <- iterator_env[[\"iter\"]]\n if (curr_iter >= 2) {\n return(NULL)\n }\n if (curr_iter == 0) {\n x_batch <- iterator_env[[\"x\"]][1:16, ]\n y_batch <- iterator_env[[\"y\"]][1:16]\n } else {\n x_batch <- iterator_env[[\"x\"]][17:32, ]\n y_batch <- iterator_env[[\"y\"]][17:32]\n }\n on.exit({\n iterator_env[[\"iter\"]] <- curr_iter + 1\n })\n return(xgb.DataBatch(data = x_batch, label = y_batch))\n }\n iterator_reset <- function(iterator_env) {\n iterator_env[[\"iter\"]] <- 0\n }\n data_iterator <- xgb.DataIter(\n env = iterator_env,\n f_next = iterator_next,\n f_reset = iterator_reset\n )\n cache_prefix <- tempdir()\n qdm <- xgb.QuantileDMatrix.from_iterator(\n data_iterator,\n max_bin = 3,\n nthread = 1\n )\n expect_true(inherits(qdm, \"xgb.QuantileDMatrix\"))\n expect_true(inherits(qdm, \"xgb.DMatrix\"))\n set.seed(123)\n model_ext <- xgb.train(\n data = qdm,\n params = params,\n nrounds = 5\n )\n\n pred_model1_qdm <- predict(model, qdm)\n pred_model2_mat <- predict(model_ext, x) |> unname()\n pred_model2_qdm <- predict(model_ext, qdm)\n\n expect_equal(pred_model1_qdm, pred)\n expect_equal(pred_model2_mat, pred)\n expect_equal(pred_model2_qdm, pred)\n})\n\ntest_that(\"xgb.DMatrix: R errors thrown on DataIterator are thrown back to the user\", {\n data(mtcars)\n iterator_env <- as.environment(\n list(\n iter = 0,\n x = mtcars[, -1],\n y = mtcars[, 1]\n )\n )\n iterator_next <- function(iterator_env) {\n curr_iter <- iterator_env[[\"iter\"]]\n if (curr_iter >= 2) {\n return(0)\n }\n if (curr_iter == 0) {\n x_batch <- iterator_env[[\"x\"]][1:16, ]\n y_batch <- iterator_env[[\"y\"]][1:16]\n } else {\n stop(\"custom error\")\n }\n on.exit({\n iterator_env[[\"iter\"]] <- curr_iter + 1\n })\n return(xgb.DataBatch(data = x_batch, label = y_batch))\n }\n iterator_reset <- function(iterator_env) {\n iterator_env[[\"iter\"]] <- 0\n }\n data_iterator <- xgb.DataIter(\n env = iterator_env,\n f_next = iterator_next,\n f_reset = iterator_reset\n )\n expect_error(\n {xgb.ExtMemDMatrix(data_iterator, nthread = 1)},\n \"custom error\"\n )\n})\n\ntest_that(\"xgb.DMatrix: number of non-missing matches data\", {\n x <- matrix(1:10, nrow = 5)\n dm1 <- xgb.DMatrix(x, nthread = 1)\n expect_equal(xgb.get.DMatrix.num.non.missing(dm1), 10)\n\n x[2, 2] <- NA\n x[4, 1] <- NA\n dm2 <- xgb.DMatrix(x, nthread = 1)\n expect_equal(xgb.get.DMatrix.num.non.missing(dm2), 8)\n})\n\ntest_that(\"xgb.DMatrix: retrieving data as CSR\", {\n data(mtcars)\n dm <- xgb.DMatrix(as.matrix(mtcars), nthread = 1)\n csr <- xgb.get.DMatrix.data(dm)\n expect_equal(dim(csr), dim(mtcars))\n expect_equal(colnames(csr), colnames(mtcars))\n expect_equal(unname(as.matrix(csr)), unname(as.matrix(mtcars)), tolerance = 1e-6)\n})\n\ntest_that(\"xgb.DMatrix: quantile cuts look correct\", {\n data(mtcars)\n y <- mtcars$mpg\n x <- as.matrix(mtcars[, -1])\n dm <- xgb.DMatrix(x, label = y, nthread = 1)\n model <- xgb.train(\n data = dm,\n params = list(\n tree_method = \"hist\",\n max_bin = 8,\n nthread = 1\n ),\n nrounds = 3\n )\n qcut_list <- xgb.get.DMatrix.qcut(dm, \"list\")\n qcut_arrays <- xgb.get.DMatrix.qcut(dm, \"arrays\")\n\n expect_equal(length(qcut_arrays), 2)\n expect_equal(names(qcut_arrays), c(\"indptr\", \"data\"))\n expect_equal(length(qcut_arrays$indptr), ncol(x) + 1)\n expect_true(min(diff(qcut_arrays$indptr)) > 0)\n\n col_min <- apply(x, 2, min)\n col_max <- apply(x, 2, max)\n\n expect_equal(length(qcut_list), ncol(x))\n expect_equal(names(qcut_list), colnames(x))\n lapply(\n seq(1, ncol(x)),\n function(col) {\n cuts <- qcut_list[[col]]\n expect_true(min(diff(cuts)) > 0)\n expect_true(col_min[col] > cuts[1])\n expect_true(col_max[col] < cuts[length(cuts)])\n expect_true(length(cuts) <= 9)\n }\n )\n})\n\ntest_that(\"xgb.DMatrix: slicing keeps field indicators\", {\n data(mtcars)\n x <- as.matrix(mtcars[, -1])\n y <- mtcars[, 1]\n dm <- xgb.DMatrix(\n data = x,\n label_lower_bound = -y,\n label_upper_bound = y,\n nthread = 1\n )\n idx_take <- seq(1, 5)\n dm_slice <- xgb.slice.DMatrix(dm, idx_take)\n\n expect_true(xgb.DMatrix.hasinfo(dm_slice, \"label_lower_bound\"))\n expect_true(xgb.DMatrix.hasinfo(dm_slice, \"label_upper_bound\"))\n expect_false(xgb.DMatrix.hasinfo(dm_slice, \"label\"))\n\n expect_equal(getinfo(dm_slice, \"label_lower_bound\"), -y[idx_take], tolerance = 1e-6)\n expect_equal(getinfo(dm_slice, \"label_upper_bound\"), y[idx_take], tolerance = 1e-6)\n})\n\ntest_that(\"xgb.DMatrix: can slice with groups\", {\n data(iris)\n x <- as.matrix(iris[, -5])\n set.seed(123)\n y <- sample(3, size = nrow(x), replace = TRUE)\n group <- c(50, 50, 50)\n dm <- xgb.DMatrix(x, label = y, group = group, nthread = 1)\n idx_take <- seq(1, 50)\n dm_slice <- xgb.slice.DMatrix(dm, idx_take, allow_groups = TRUE)\n\n expect_true(xgb.DMatrix.hasinfo(dm_slice, \"label\"))\n expect_false(xgb.DMatrix.hasinfo(dm_slice, \"group\"))\n expect_false(xgb.DMatrix.hasinfo(dm_slice, \"qid\"))\n expect_null(getinfo(dm_slice, \"group\"))\n expect_equal(getinfo(dm_slice, \"label\"), y[idx_take], tolerance = 1e-6)\n})\n\ntest_that(\"xgb.DMatrix: can read CSV\", {\n txt <- paste(\n \"1,2,3\",\n \"-1,3,2\",\n sep = \"\\n\"\n )\n fname <- file.path(tempdir(), \"data.csv\")\n writeChar(txt, fname)\n uri <- paste0(fname, \"?format=csv&label_column=0\")\n dm <- xgb.DMatrix(uri, silent = TRUE, nthread = 1)\n expect_equal(getinfo(dm, \"label\"), c(1, -1))\n expect_equal(\n as.matrix(xgb.get.DMatrix.data(dm)),\n matrix(c(2, 3, 3, 2), nrow = 2, byrow = TRUE)\n )\n})\n" }, { "path": "R-package/tests/testthat/test_feature_weights.R", "content": "context(\"feature weights\")\n\nn_threads <- 2\n\ntest_that(\"training with feature weights works\", {\n nrows <- 1000\n ncols <- 9\n set.seed(2022)\n x <- matrix(rnorm(nrows * ncols), nrow = nrows)\n y <- rowSums(x)\n weights <- seq(from = 1, to = ncols)\n\n test <- function(tm) {\n names <- paste0(\"f\", 1:ncols)\n xy <- xgb.DMatrix(\n data = x, label = y, feature_weights = weights, nthread = n_threads\n )\n params <- list(\n colsample_bynode = 0.4, tree_method = tm, nthread = n_threads\n )\n model <- xgb.train(params = params, data = xy, nrounds = 32)\n importance <- xgb.importance(model = model, feature_names = names)\n expect_equal(dim(importance), c(ncols, 4))\n importance <- importance[order(importance$Feature)]\n expect_lt(importance[1, Frequency], importance[9, Frequency])\n }\n\n for (tm in c(\"hist\", \"approx\")) {\n test(tm)\n }\n})\n" }, { "path": "R-package/tests/testthat/test_glm.R", "content": "context('Test generalized linear models')\n\nn_threads <- 2\n\ntest_that(\"gblinear works\", {\n data(agaricus.train, package = 'xgboost')\n data(agaricus.test, package = 'xgboost')\n dtrain <- xgb.DMatrix(\n agaricus.train$data, label = agaricus.train$label, nthread = n_threads\n )\n dtest <- xgb.DMatrix(\n agaricus.test$data, label = agaricus.test$label, nthread = n_threads\n )\n\n param <- list(objective = \"binary:logistic\", eval_metric = \"error\", booster = \"gblinear\",\n nthread = n_threads, learning_rate = 0.8, reg_alpha = 0.0001, reg_lambda = 0.0001)\n evals <- list(eval = dtest, train = dtrain)\n\n n <- 5 # iterations\n ERR_UL <- 0.005 # upper limit for the test set error\n VERB <- 0 # chatterbox switch\n\n param$updater <- 'shotgun'\n bst <- xgb.train(c(param, list(feature_selector = 'shuffle')), dtrain, n, evals, verbose = VERB)\n ypred <- predict(bst, dtest)\n expect_equal(length(getinfo(dtest, 'label')), 1611)\n expect_lt(attributes(bst)$evaluation_log$eval_error[n], ERR_UL)\n\n bst <- xgb.train(c(param, list(feature_selector = 'cyclic')), dtrain, n, evals, verbose = VERB,\n callbacks = list(xgb.cb.gblinear.history()))\n expect_lt(attributes(bst)$evaluation_log$eval_error[n], ERR_UL)\n h <- xgb.gblinear.history(bst)\n expect_equal(dim(h), c(n, ncol(dtrain) + 1))\n expect_is(h, \"matrix\")\n\n param$updater <- 'coord_descent'\n bst <- xgb.train(c(param, list(feature_selector = 'cyclic')), dtrain, n, evals, verbose = VERB)\n expect_lt(attributes(bst)$evaluation_log$eval_error[n], ERR_UL)\n\n bst <- xgb.train(c(param, list(feature_selector = 'shuffle')), dtrain, n, evals, verbose = VERB)\n expect_lt(attributes(bst)$evaluation_log$eval_error[n], ERR_UL)\n\n bst <- xgb.train(c(param, list(feature_selector = 'greedy')), dtrain, 2, evals, verbose = VERB)\n expect_lt(attributes(bst)$evaluation_log$eval_error[2], ERR_UL)\n\n bst <- xgb.train(c(param, list(feature_selector = 'thrifty', top_k = 50)), dtrain, n, evals, verbose = VERB,\n callbacks = list(xgb.cb.gblinear.history(sparse = TRUE)))\n expect_lt(attributes(bst)$evaluation_log$eval_error[n], ERR_UL)\n h <- xgb.gblinear.history(bst)\n expect_equal(dim(h), c(n, ncol(dtrain) + 1))\n expect_s4_class(h, \"dgCMatrix\")\n})\n\ntest_that(\"gblinear early stopping works\", {\n data(agaricus.train, package = 'xgboost')\n data(agaricus.test, package = 'xgboost')\n dtrain <- xgb.DMatrix(\n agaricus.train$data, label = agaricus.train$label, nthread = n_threads\n )\n dtest <- xgb.DMatrix(\n agaricus.test$data, label = agaricus.test$label, nthread = n_threads\n )\n\n param <- xgb.params(\n objective = \"binary:logistic\", eval_metric = \"error\", booster = \"gblinear\",\n nthread = n_threads, learning_rate = 0.8, reg_alpha = 0.0001, reg_lambda = 0.0001,\n updater = \"coord_descent\"\n )\n\n es_round <- 1\n n <- 10\n booster <- xgb.train(\n param, dtrain, nrounds = n, evals = list(eval = dtest, train = dtrain),\n early_stopping_rounds = es_round, verbose = 0\n )\n expect_equal(xgb.attr(booster, \"best_iteration\"), 4)\n predt_es <- predict(booster, dtrain)\n\n n <- xgb.attr(booster, \"best_iteration\") + es_round + 1\n booster <- xgb.train(\n param, dtrain, nrounds = n, evals = list(eval = dtest, train = dtrain),\n early_stopping_rounds = es_round, verbose = 0\n )\n predt <- predict(booster, dtrain)\n expect_equal(predt_es, predt)\n})\n" }, { "path": "R-package/tests/testthat/test_helpers.R", "content": "context('Test helper functions')\n\nVCD_AVAILABLE <- requireNamespace(\"vcd\", quietly = TRUE)\n.skip_if_vcd_not_available <- function() {\n if (!VCD_AVAILABLE) {\n testthat::skip(\"Optional testing dependency 'vcd' not found.\")\n }\n}\n\nfloat_tolerance <- 5e-6\n\n# disable some tests for 32-bit environment\nflag_32bit <- .Machine$sizeof.pointer != 8\n\nset.seed(1982)\n\nnrounds <- 12\nif (isTRUE(VCD_AVAILABLE)) {\n data(Arthritis, package = \"vcd\")\n df <- data.table::data.table(Arthritis, keep.rownames = FALSE)\n df[, AgeDiscret := as.factor(round(Age / 10, 0))]\n df[, AgeCat := as.factor(ifelse(Age > 30, \"Old\", \"Young\"))]\n df[, ID := NULL]\n sparse_matrix <- Matrix::sparse.model.matrix(Improved~.-1, data = df) # nolint\n label <- df[, ifelse(Improved == \"Marked\", 1, 0)]\n\n # binary\n bst.Tree <- xgb.train(\n data = xgb.DMatrix(sparse_matrix, label = label, nthread = 1),\n nrounds = nrounds, verbose = 0,\n params = xgb.params(\n max_depth = 9,\n learning_rate = 1,\n nthread = 2,\n objective = \"binary:logistic\",\n booster = \"gbtree\",\n base_score = 0.5\n )\n )\n\n bst.GLM <- xgb.train(\n data = xgb.DMatrix(sparse_matrix, label = label, nthread = 1),\n nrounds = nrounds, verbose = 0,\n params = xgb.params(\n learning_rate = 1,\n nthread = 1,\n objective = \"binary:logistic\",\n booster = \"gblinear\",\n base_score = 0.5\n )\n )\n\n feature.names <- colnames(sparse_matrix)\n\n # without feature names\n bst.Tree.unnamed <- xgb.copy.Booster(bst.Tree)\n setinfo(bst.Tree.unnamed, \"feature_name\", NULL)\n}\n\n# multiclass\nmlabel <- as.numeric(iris$Species) - 1\nnclass <- 3\nmbst.Tree <- xgb.train(\n data = xgb.DMatrix(as.matrix(iris[, -5]), label = mlabel, nthread = 1),\n verbose = 0,\n nrounds = nrounds,\n params = xgb.params(\n max_depth = 3, learning_rate = 0.5, nthread = 2,\n objective = \"multi:softprob\", num_class = nclass, base_score = 0\n )\n)\n\nmbst.GLM <- xgb.train(\n data = xgb.DMatrix(as.matrix(iris[, -5]), label = mlabel, nthread = 1),\n verbose = 0,\n nrounds = nrounds,\n params = xgb.params(\n booster = \"gblinear\", learning_rate = 0.1, nthread = 1,\n objective = \"multi:softprob\", num_class = nclass, base_score = 0\n )\n)\n\ntest_that(\"xgb.dump works\", {\n .skip_if_vcd_not_available()\n if (!flag_32bit)\n expect_length(xgb.dump(bst.Tree), 200)\n dump_file <- file.path(tempdir(), 'xgb.model.dump')\n expect_true(xgb.dump(bst.Tree, dump_file, with_stats = TRUE))\n expect_true(file.exists(dump_file))\n expect_gt(file.size(dump_file), 8000)\n\n # JSON format\n dmp <- xgb.dump(bst.Tree, dump_format = \"json\")\n expect_length(dmp, 1)\n if (!flag_32bit)\n expect_length(grep('nodeid', strsplit(dmp, '\\n', fixed = TRUE)[[1]], fixed = TRUE), 188)\n})\n\ntest_that(\"xgb.dump works for gblinear\", {\n .skip_if_vcd_not_available()\n expect_length(xgb.dump(bst.GLM), 14)\n # also make sure that it works properly for a sparse model where some coefficients\n # are 0 from setting large L1 regularization:\n bst.GLM.sp <- xgb.train(\n data = xgb.DMatrix(sparse_matrix, label = label, nthread = 1),\n nrounds = 1,\n params = xgb.params(\n learning_rate = 1,\n nthread = 2,\n reg_alpha = 2,\n objective = \"binary:logistic\",\n booster = \"gblinear\"\n )\n )\n d.sp <- xgb.dump(bst.GLM.sp)\n expect_length(d.sp, 14)\n expect_gt(sum(d.sp == \"0\"), 0)\n\n # JSON format\n dmp <- xgb.dump(bst.GLM.sp, dump_format = \"json\")\n expect_length(dmp, 1)\n expect_length(grep('\\\\d', strsplit(dmp, '\\n', fixed = TRUE)[[1]]), 11)\n})\n\ntest_that(\"predict leafs works\", {\n .skip_if_vcd_not_available()\n # no error for gbtree\n expect_error(pred_leaf <- predict(bst.Tree, sparse_matrix, predleaf = TRUE), regexp = NA)\n expect_equal(dim(pred_leaf), c(nrow(sparse_matrix), nrounds))\n # error for gblinear\n expect_error(predict(bst.GLM, sparse_matrix, predleaf = TRUE))\n})\n\ntest_that(\"predict feature contributions works\", {\n .skip_if_vcd_not_available()\n # gbtree binary classifier\n expect_error(pred_contr <- predict(bst.Tree, sparse_matrix, predcontrib = TRUE), regexp = NA)\n expect_equal(dim(pred_contr), c(nrow(sparse_matrix), ncol(sparse_matrix) + 1))\n expect_equal(colnames(pred_contr), c(colnames(sparse_matrix), \"(Intercept)\"))\n pred <- predict(bst.Tree, sparse_matrix, outputmargin = TRUE)\n expect_lt(max(abs(rowSums(pred_contr) - pred)), 1e-5)\n # must work with data that has no column names\n X <- sparse_matrix\n colnames(X) <- NULL\n expect_error(pred_contr_ <- predict(bst.Tree, X, predcontrib = TRUE), regexp = NA)\n expect_equal(pred_contr, pred_contr_, check.attributes = FALSE,\n tolerance = float_tolerance)\n\n # gbtree binary classifier (approximate method)\n expect_error(pred_contr <- predict(bst.Tree, sparse_matrix, predcontrib = TRUE, approxcontrib = TRUE), regexp = NA)\n expect_equal(dim(pred_contr), c(nrow(sparse_matrix), ncol(sparse_matrix) + 1))\n expect_equal(colnames(pred_contr), c(colnames(sparse_matrix), \"(Intercept)\"))\n pred <- predict(bst.Tree, sparse_matrix, outputmargin = TRUE)\n expect_lt(max(abs(rowSums(pred_contr) - pred)), 1e-5)\n\n # gblinear binary classifier\n expect_error(pred_contr <- predict(bst.GLM, sparse_matrix, predcontrib = TRUE), regexp = NA)\n expect_equal(dim(pred_contr), c(nrow(sparse_matrix), ncol(sparse_matrix) + 1))\n expect_equal(colnames(pred_contr), c(colnames(sparse_matrix), \"(Intercept)\"))\n pred <- predict(bst.GLM, sparse_matrix, outputmargin = TRUE)\n expect_lt(max(abs(rowSums(pred_contr) - pred)), 1e-5)\n # manual calculation of linear terms\n coefs <- as.numeric(xgb.dump(bst.GLM)[-c(1, 2, 4)])\n coefs <- c(coefs[-1], coefs[1]) # intercept must be the last\n pred_contr_manual <- sweep(cbind(sparse_matrix, 1), 2, coefs, FUN = \"*\")\n expect_equal(as.numeric(pred_contr), as.numeric(pred_contr_manual),\n tolerance = float_tolerance)\n\n # gbtree multiclass\n pred <- predict(mbst.Tree, as.matrix(iris[, -5]), outputmargin = TRUE)\n pred_contr <- predict(mbst.Tree, as.matrix(iris[, -5]), predcontrib = TRUE)\n expect_is(pred_contr, \"array\")\n expect_length(dim(pred_contr), 3)\n for (g in seq_len(dim(pred_contr)[2])) {\n expect_equal(colnames(pred_contr[, g, ]), c(colnames(iris[, -5]), \"(Intercept)\"))\n expect_lt(max(abs(rowSums(pred_contr[, g, ]) - pred[, g])), 1e-5)\n }\n\n # gblinear multiclass (set base_score = 0, which is base margin in multiclass)\n pred <- predict(mbst.GLM, as.matrix(iris[, -5]), outputmargin = TRUE)\n pred_contr <- predict(mbst.GLM, as.matrix(iris[, -5]), predcontrib = TRUE)\n expect_length(dim(pred_contr), 3)\n coefs_all <- matrix(\n data = as.numeric(xgb.dump(mbst.GLM)[-c(1, 2, 6)]),\n ncol = 3,\n byrow = TRUE\n )\n for (g in seq_along(dim(pred_contr)[2])) {\n expect_equal(colnames(pred_contr[, g, ]), c(colnames(iris[, -5]), \"(Intercept)\"))\n expect_lt(max(abs(rowSums(pred_contr[, g, ]) - pred[, g])), float_tolerance)\n # manual calculation of linear terms\n coefs <- c(coefs_all[-1, g], coefs_all[1, g]) # intercept needs to be the last\n pred_contr_manual <- sweep(as.matrix(cbind(iris[, -5], 1)), 2, coefs, FUN = \"*\")\n expect_equal(as.numeric(pred_contr[, g, ]), as.numeric(pred_contr_manual),\n tolerance = float_tolerance)\n }\n})\n\ntest_that(\"SHAPs sum to predictions, with or without DART\", {\n d <- cbind(\n x1 = rnorm(100),\n x2 = rnorm(100),\n x3 = rnorm(100))\n y <- d[, \"x1\"] + d[, \"x2\"]^2 +\n ifelse(d[, \"x3\"] > .5, d[, \"x3\"]^2, 2^d[, \"x3\"]) +\n rnorm(100)\n nrounds <- 30\n\n for (booster in list(\"gbtree\", \"dart\")) {\n fit <- xgb.train(\n params = c(\n list(\n nthread = 2,\n booster = booster,\n objective = \"reg:squarederror\",\n eval_metric = \"rmse\"),\n if (booster == \"dart\")\n list(rate_drop = .01, one_drop = TRUE)),\n data = xgb.DMatrix(d, label = y, nthread = 1),\n nrounds = nrounds)\n\n pr <- function(...) {\n predict(fit, newdata = d, ...)\n }\n pred <- pr()\n shap <- pr(predcontrib = TRUE)\n shapi <- pr(predinteraction = TRUE)\n tol <- 1e-5\n\n expect_equal(rowSums(shap), pred, tol = tol)\n expect_equal(rowSums(shapi), pred, tol = tol)\n for (i in seq_len(nrow(d)))\n for (f in list(rowSums, colSums))\n expect_equal(f(shapi[i, , ]), shap[i, ], tol = tol)\n }\n})\n\ntest_that(\"xgb-attribute functionality\", {\n .skip_if_vcd_not_available()\n val <- \"my attribute value\"\n list.val <- list(my_attr = val, a = 123, b = 'ok')\n list.ch <- list.val[order(names(list.val))]\n list.ch <- lapply(list.ch, as.character)\n # note: iter is 0-index in xgb attributes\n list.default <- list()\n list.ch <- c(list.ch, list.default)\n # proper input:\n expect_error(xgb.attr(bst.Tree, NULL))\n expect_error(xgb.attr(val, val))\n # set & get:\n expect_null(xgb.attr(bst.Tree, \"asdf\"))\n expect_equal(xgb.attributes(bst.Tree), list.default)\n bst.Tree.copy <- xgb.copy.Booster(bst.Tree)\n xgb.attr(bst.Tree.copy, \"my_attr\") <- val\n expect_equal(xgb.attr(bst.Tree.copy, \"my_attr\"), val)\n xgb.attributes(bst.Tree.copy) <- list.val\n expect_equal(xgb.attributes(bst.Tree.copy), list.ch)\n # serializing:\n fname <- file.path(tempdir(), \"xgb.ubj\")\n xgb.save(bst.Tree.copy, fname)\n bst <- xgb.load(fname)\n expect_equal(xgb.attr(bst, \"my_attr\"), val)\n expect_equal(xgb.attributes(bst), list.ch)\n # deletion:\n xgb.attr(bst, \"my_attr\") <- NULL\n expect_null(xgb.attr(bst, \"my_attr\"))\n expect_equal(xgb.attributes(bst), list.ch[c(\"a\", \"b\")])\n xgb.attributes(bst) <- list(a = NULL, b = NULL)\n expect_equal(xgb.attributes(bst), list.default)\n xgb.attributes(bst) <- list(niter = NULL)\n expect_equal(xgb.attributes(bst), list())\n})\n\nif (grepl('Windows', Sys.info()[['sysname']], fixed = TRUE) ||\n grepl('Linux', Sys.info()[['sysname']], fixed = TRUE) ||\n grepl('Darwin', Sys.info()[['sysname']], fixed = TRUE)) {\n test_that(\"xgb-attribute numeric precision\", {\n .skip_if_vcd_not_available()\n # check that lossless conversion works with 17 digits\n # numeric -> character -> numeric\n X <- 10^runif(100, -20, 20)\n if (capabilities('long.double')) {\n X2X <- as.numeric(format(X, digits = 17))\n expect_equal(X, X2X, tolerance = float_tolerance)\n }\n # retrieved attributes to be the same as written\n for (x in X) {\n xgb.attr(bst.Tree, \"x\") <- x\n expect_equal(as.numeric(xgb.attr(bst.Tree, \"x\")), x, tolerance = float_tolerance)\n xgb.attributes(bst.Tree) <- list(a = \"A\", b = x)\n expect_equal(as.numeric(xgb.attr(bst.Tree, \"b\")), x, tolerance = float_tolerance)\n }\n })\n}\n\ntest_that(\"xgb.Booster serializing as R object works\", {\n .skip_if_vcd_not_available()\n fname_rds <- file.path(tempdir(), \"xgb.model.rds\")\n saveRDS(bst.Tree, fname_rds)\n bst <- readRDS(fname_rds)\n dtrain <- xgb.DMatrix(sparse_matrix, label = label, nthread = 2)\n expect_equal(predict(bst.Tree, dtrain), predict(bst, dtrain), tolerance = float_tolerance)\n expect_equal(xgb.dump(bst.Tree), xgb.dump(bst))\n\n fname_bin <- file.path(tempdir(), \"xgb.model\")\n xgb.save(bst, fname_bin)\n bst <- readRDS(fname_rds)\n expect_equal(predict(bst.Tree, dtrain), predict(bst, dtrain), tolerance = float_tolerance)\n})\n\ntest_that(\"xgb.model.dt.tree works with and without feature names\", {\n .skip_if_vcd_not_available()\n names.dt.trees <- c(\"Tree\", \"Node\", \"ID\", \"Feature\", \"Split\", \"Yes\", \"No\", \"Missing\", \"Gain\", \"Cover\")\n dt.tree <- xgb.model.dt.tree(model = bst.Tree)\n expect_equal(names.dt.trees, names(dt.tree))\n if (!flag_32bit)\n expect_equal(dim(dt.tree), c(188, 10))\n expect_output(str(dt.tree), 'Feature.*\\\\\"Age\\\\\"')\n\n # when model contains no feature names:\n dt.tree.x <- xgb.model.dt.tree(model = bst.Tree.unnamed)\n expect_output(str(dt.tree.x), 'Feature.*\\\\\"3\\\\\"')\n expect_equal(dt.tree[, -4, with = FALSE], dt.tree.x[, -4, with = FALSE])\n\n # using integer node ID instead of character\n dt.tree.int <- xgb.model.dt.tree(model = bst.Tree, use_int_id = TRUE)\n expect_equal(as.integer(data.table::tstrsplit(dt.tree$Yes, '-', fixed = TRUE)[[2]]), dt.tree.int$Yes)\n expect_equal(as.integer(data.table::tstrsplit(dt.tree$No, '-', fixed = TRUE)[[2]]), dt.tree.int$No)\n expect_equal(as.integer(data.table::tstrsplit(dt.tree$Missing, '-', fixed = TRUE)[[2]]), dt.tree.int$Missing)\n})\n\ntest_that(\"xgb.model.dt.tree throws error for gblinear\", {\n .skip_if_vcd_not_available()\n expect_error(xgb.model.dt.tree(model = bst.GLM))\n})\n\ntest_that(\"xgb.importance works with and without feature names\", {\n .skip_if_vcd_not_available()\n importance.Tree <- xgb.importance(feature_names = feature.names, model = bst.Tree.unnamed)\n if (!flag_32bit)\n expect_equal(dim(importance.Tree), c(7, 4))\n expect_equal(colnames(importance.Tree), c(\"Feature\", \"Gain\", \"Cover\", \"Frequency\"))\n expect_output(str(importance.Tree), 'Feature.*\\\\\"Age\\\\\"')\n\n importance.Tree.0 <- xgb.importance(model = bst.Tree)\n expect_equal(importance.Tree, importance.Tree.0, tolerance = float_tolerance)\n\n # when model contains no feature names:\n importance.Tree.x <- xgb.importance(model = bst.Tree.unnamed)\n expect_equal(importance.Tree[, -1, with = FALSE], importance.Tree.x[, -1, with = FALSE],\n tolerance = float_tolerance)\n\n imp2plot <- xgb.plot.importance(importance_matrix = importance.Tree)\n expect_equal(colnames(imp2plot), c(\"Feature\", \"Gain\", \"Cover\", \"Frequency\", \"Importance\"))\n xgb.ggplot.importance(importance_matrix = importance.Tree)\n\n # for multiclass\n imp.Tree <- xgb.importance(model = mbst.Tree)\n expect_equal(dim(imp.Tree), c(4, 4))\n\n trees <- seq(from = 1, by = 2, length.out = 2)\n importance <- xgb.importance(feature_names = feature.names, model = bst.Tree, trees = trees)\n\n importance_from_dump <- function() {\n imp <- xgb.model.dt.tree(\n model = bst.Tree,\n trees = trees\n )[\n Feature != \"Leaf\", .(\n Gain = sum(Gain),\n Cover = sum(Cover),\n Frequency = .N\n ),\n by = Feature\n ][\n , `:=`(\n Gain = Gain / sum(Gain),\n Cover = Cover / sum(Cover),\n Frequency = Frequency / sum(Frequency)\n )\n ][\n order(Gain, decreasing = TRUE)\n ]\n imp\n }\n expect_equal(importance_from_dump(), importance, tolerance = 1e-6)\n\n ## decision stump\n m <- xgb.train(\n data = xgb.DMatrix(as.matrix(data.frame(x = c(0, 1))), label = c(1, 2), nthread = 1),\n nrounds = 1,\n params = xgb.params(\n base_score = 0.5,\n nthread = 2\n )\n )\n df <- xgb.model.dt.tree(model = m)\n expect_equal(df$Feature, \"Leaf\")\n expect_equal(df$Cover, 2)\n})\n\ntest_that(\"xgb.importance works with GLM model\", {\n .skip_if_vcd_not_available()\n importance.GLM <- xgb.importance(feature_names = feature.names, model = bst.GLM)\n expect_equal(dim(importance.GLM), c(10, 2))\n expect_equal(colnames(importance.GLM), c(\"Feature\", \"Weight\"))\n xgb.importance(model = bst.GLM)\n imp2plot <- xgb.plot.importance(importance.GLM)\n expect_equal(colnames(imp2plot), c(\"Feature\", \"Weight\", \"Importance\"))\n xgb.ggplot.importance(importance.GLM)\n\n # check that the input is not modified in-place\n expect_false(\"Importance\" %in% names(importance.GLM))\n\n # for multiclass\n imp.GLM <- xgb.importance(model = mbst.GLM)\n expect_equal(dim(imp.GLM), c(12, 3))\n expect_equal(imp.GLM$Class, rep(0:2, each = 4))\n})\n\ntest_that(\"xgb.model.dt.tree and xgb.importance work with a single split model\", {\n .skip_if_vcd_not_available()\n bst1 <- xgb.train(\n data = xgb.DMatrix(sparse_matrix, label = label, nthread = 1),\n nrounds = 1, verbose = 0,\n params = xgb.params(\n max_depth = 1,\n learning_rate = 1,\n nthread = 2,\n objective = \"binary:logistic\"\n )\n )\n expect_error(dt <- xgb.model.dt.tree(model = bst1), regexp = NA) # no error\n expect_equal(nrow(dt), 3)\n expect_error(imp <- xgb.importance(model = bst1), regexp = NA) # no error\n expect_equal(nrow(imp), 1)\n expect_equal(imp$Gain, 1)\n})\n\ntest_that(\"xgb.plot.importance de-duplicates features\", {\n importances <- data.table(\n Feature = c(\"col1\", \"col2\", \"col2\"),\n Gain = c(0.4, 0.3, 0.3)\n )\n imp2plot <- xgb.plot.importance(importances)\n expect_equal(nrow(imp2plot), 2L)\n expect_equal(imp2plot$Feature, c(\"col2\", \"col1\"))\n})\n\ntest_that(\"xgb.plot.tree works with and without feature names\", {\n .skip_if_vcd_not_available()\n expect_silent(xgb.plot.tree(model = bst.Tree.unnamed))\n expect_silent(xgb.plot.tree(model = bst.Tree))\n\n ## Categorical\n y <- rnorm(100)\n x <- sample(3, size = 100 * 3, replace = TRUE) |> matrix(nrow = 100)\n x <- x - 1\n dm <- xgb.DMatrix(data = x, label = y, nthread = 1)\n setinfo(dm, \"feature_type\", c(\"c\", \"c\", \"c\"))\n model <- xgb.train(\n data = dm,\n params = list(tree_method = \"hist\"),\n nrounds = 2\n )\n expect_silent(xgb.plot.tree(model = model))\n})\n\ntest_that(\"xgb.plot.multi.trees works with and without feature names\", {\n .skip_if_vcd_not_available()\n xgb.plot.multi.trees(model = bst.Tree.unnamed, features_keep = 3)\n xgb.plot.multi.trees(model = bst.Tree, features_keep = 3)\n expect_true(TRUE)\n})\n\ntest_that(\"xgb.plot.deepness works\", {\n .skip_if_vcd_not_available()\n d2p <- xgb.plot.deepness(model = bst.Tree)\n expect_equal(colnames(d2p), c(\"ID\", \"Tree\", \"Depth\", \"Cover\", \"Weight\"))\n xgb.plot.deepness(model = bst.Tree, which = \"med.depth\")\n xgb.ggplot.deepness(model = bst.Tree)\n})\n\ntest_that(\"xgb.shap.data works when top_n is provided\", {\n .skip_if_vcd_not_available()\n data_list <- xgb.shap.data(data = sparse_matrix, model = bst.Tree, top_n = 2)\n expect_equal(names(data_list), c(\"data\", \"shap_contrib\"))\n expect_equal(NCOL(data_list$data), 2)\n expect_equal(NCOL(data_list$shap_contrib), 2)\n expect_equal(NROW(data_list$data), NROW(data_list$shap_contrib))\n expect_gt(length(colnames(data_list$data)), 0)\n expect_gt(length(colnames(data_list$shap_contrib)), 0)\n\n # for multiclass without target class provided\n data_list <- xgb.shap.data(data = as.matrix(iris[, -5]), model = mbst.Tree, top_n = 2)\n expect_equal(dim(data_list$shap_contrib), c(nrow(iris), 2))\n # for multiclass with target class provided\n data_list <- xgb.shap.data(data = as.matrix(iris[, -5]), model = mbst.Tree, top_n = 2, target_class = 0)\n expect_equal(dim(data_list$shap_contrib), c(nrow(iris), 2))\n})\n\ntest_that(\"xgb.shap.data works with subsampling\", {\n .skip_if_vcd_not_available()\n data_list <- xgb.shap.data(data = sparse_matrix, model = bst.Tree, top_n = 2, subsample = 0.8)\n expect_equal(NROW(data_list$data), as.integer(0.8 * nrow(sparse_matrix)))\n expect_equal(NROW(data_list$data), NROW(data_list$shap_contrib))\n})\n\ntest_that(\"xgb.shap.data works with data frames\", {\n data(mtcars)\n df <- mtcars\n df$cyl <- factor(df$cyl)\n x <- df[, -1]\n y <- df$mpg\n dm <- xgb.DMatrix(x, label = y, nthread = 1L)\n model <- xgb.train(\n data = dm,\n params = list(\n max_depth = 2,\n nthread = 1\n ),\n nrounds = 2\n )\n data_list <- xgb.shap.data(data = df[, -1], model = model, top_n = 2, subsample = 0.8)\n expect_equal(NROW(data_list$data), as.integer(0.8 * nrow(df)))\n expect_equal(NROW(data_list$data), NROW(data_list$shap_contrib))\n})\n\ntest_that(\"prepare.ggplot.shap.data works\", {\n .skip_if_vcd_not_available()\n data_list <- xgb.shap.data(data = sparse_matrix, model = bst.Tree, top_n = 2)\n plot_data <- prepare.ggplot.shap.data(data_list, normalize = TRUE)\n expect_s3_class(plot_data, \"data.frame\")\n expect_equal(names(plot_data), c(\"id\", \"feature\", \"feature_value\", \"shap_value\"))\n expect_s3_class(plot_data$feature, \"factor\")\n # Each observation should have 1 row for each feature\n expect_equal(nrow(plot_data), nrow(sparse_matrix) * 2)\n})\n\ntest_that(\"xgb.plot.shap works\", {\n .skip_if_vcd_not_available()\n sh <- xgb.plot.shap(data = sparse_matrix, model = bst.Tree, top_n = 2, col = 4)\n expect_equal(names(sh), c(\"data\", \"shap_contrib\"))\n})\n\ntest_that(\"xgb.plot.shap.summary works\", {\n .skip_if_vcd_not_available()\n expect_silent(xgb.plot.shap.summary(data = sparse_matrix, model = bst.Tree, top_n = 2))\n expect_silent(xgb.ggplot.shap.summary(data = sparse_matrix, model = bst.Tree, top_n = 2))\n})\n\ntest_that(\"xgb.plot.shap.summary ignores categorical features\", {\n .skip_if_vcd_not_available()\n data(mtcars)\n df <- mtcars\n df$cyl <- factor(df$cyl)\n levels(df$cyl) <- c(\"a\", \"b\", \"c\")\n x <- df[, -1]\n y <- df$mpg\n dm <- xgb.DMatrix(x, label = y, nthread = 1L)\n model <- xgb.train(\n data = dm,\n params = list(\n max_depth = 2,\n nthread = 1\n ),\n nrounds = 2\n )\n expect_warning({\n xgb.ggplot.shap.summary(data = x, model = model, top_n = 2)\n })\n\n x_num <- mtcars[, -1]\n x_num$gear <- as.numeric(x_num$gear) - 1\n x_num <- as.matrix(x_num)\n dm <- xgb.DMatrix(x_num, label = y, feature_types = c(rep(\"q\", 8), \"c\", \"q\"), nthread = 1L)\n model <- xgb.train(\n data = dm,\n params = list(\n max_depth = 2,\n nthread = 1\n ),\n nrounds = 2\n )\n expect_warning({\n xgb.ggplot.shap.summary(data = x_num, model = model, top_n = 2)\n })\n})\n\ntest_that(\"check.deprecation works\", {\n data(mtcars)\n dm <- xgb.DMatrix(mtcars[, -1L], label = mtcars$mpg, nthread = 1)\n params <- xgb.params(nthread = 1, max_depth = 2, eval_metric = \"rmse\")\n args_train <- list(\n data = dm,\n params = params,\n nrounds = 10,\n verbose = 0\n )\n\n # with exact name\n options(\"xgboost.strict_mode\" = TRUE)\n expect_error({\n model <- xgb.train(\n data = dm,\n params = params,\n nrounds = 10,\n watchlist = list(tr = dm),\n verbose = 0\n )\n }, regexp = \"watchlist\")\n options(\"xgboost.strict_mode\" = FALSE)\n expect_warning({\n model <- xgb.train(\n data = dm,\n params = params,\n nrounds = 10,\n watchlist = list(tr = dm),\n verbose = 0\n )\n }, regexp = \"watchlist\")\n expect_true(hasName(attributes(model), \"evaluation_log\"))\n expect_equal(names(attributes(model)$evaluation_log), c(\"iter\", \"tr_rmse\"))\n\n # with partial name match\n expect_warning({\n model <- xgb.train(\n data = dm,\n params = params,\n nrounds = 10,\n watchlis = list(train = dm),\n verbose = 0\n )\n }, regexp = \"watchlist\")\n expect_true(hasName(attributes(model), \"evaluation_log\"))\n expect_equal(names(attributes(model)$evaluation_log), c(\"iter\", \"train_rmse\"))\n\n # error/warning is thrown if argument cannot be matched\n options(\"xgboost.strict_mode\" = TRUE)\n expect_error({\n model <- xgb.train(\n data = dm,\n params = params,\n nrounds = 10,\n watchlistt = list(train = dm),\n verbose = 0\n )\n }, regexp = \"unrecognized\")\n options(\"xgboost.strict_mode\" = FALSE)\n expect_warning({\n model <- xgb.train(\n data = dm,\n params = params,\n nrounds = 10,\n watchlistt = list(train = dm),\n verbose = 0\n )\n }, regexp = \"unrecognized\")\n\n # error should suggest to put under 'params' if it goes there\n options(\"xgboost.strict_mode\" = TRUE)\n expect_error({\n model <- xgb.train(\n data = dm,\n nthread = 1, max_depth = 2, eval_metric = \"rmse\",\n nrounds = 10,\n evals = list(train = dm),\n verbose = 0\n )\n }, regexp = \"should be passed as a list to argument 'params'\")\n options(\"xgboost.strict_mode\" = FALSE)\n expect_warning({\n model <- xgb.train(\n data = dm,\n nthread = 1, max_depth = 2, eval_metric = \"mae\",\n nrounds = 10,\n evals = list(train = dm),\n verbose = 0\n )\n }, regexp = \"should be passed as a list to argument 'params'\")\n expect_true(hasName(attributes(model), \"evaluation_log\"))\n expect_equal(names(attributes(model)$evaluation_log), c(\"iter\", \"train_mae\"))\n\n # can take more than one deprecated parameter\n expect_warning({\n model <- xgb.train(\n training.data = dm,\n params = params,\n nrounds = 10,\n watchlis = list(tr = dm),\n verbose = 0\n )\n }, regexp = \"training.data\")\n expect_true(hasName(attributes(model), \"evaluation_log\"))\n expect_equal(names(attributes(model)$evaluation_log), c(\"iter\", \"tr_rmse\"))\n})\n\ntest_that('convert.labels works', {\n y <- c(0, 1, 0, 0, 1)\n for (objective in c('binary:logistic', 'binary:logitraw', 'binary:hinge')) {\n res <- xgboost:::convert.labels(y, objective_name = objective)\n expect_s3_class(res, 'factor')\n expect_equal(res, factor(res))\n }\n y <- c(0, 1, 3, 2, 1, 4)\n for (objective in c('multi:softmax', 'multi:softprob', 'rank:pairwise', 'rank:ndcg',\n 'rank:map')) {\n res <- xgboost:::convert.labels(y, objective_name = objective)\n expect_s3_class(res, 'factor')\n expect_equal(res, factor(res))\n }\n y <- c(1.2, 3.0, -1.0, 10.0)\n for (objective in c('reg:squarederror', 'reg:squaredlogerror', 'reg:logistic',\n 'reg:pseudohubererror', 'count:poisson', 'survival:cox', 'survival:aft',\n 'reg:gamma', 'reg:tweedie')) {\n res <- xgboost:::convert.labels(y, objective_name = objective)\n expect_equal(class(res), 'numeric')\n }\n})\n\ntest_that(\"validate.features works as expected\", {\n data(mtcars)\n y <- mtcars$mpg\n x <- as.matrix(mtcars[, -1])\n dm <- xgb.DMatrix(x, label = y, nthread = 1)\n model <- xgb.train(\n params = list(nthread = 1),\n data = dm,\n nrounds = 3\n )\n\n # result is output as-is when needed\n res <- validate.features(model, x)\n expect_equal(res, x)\n res <- validate.features(model, dm)\n expect_identical(res, dm)\n res <- validate.features(model, as(x[1, ], \"dsparseVector\"))\n expect_equal(as.numeric(res), unname(x[1, ]))\n res <- validate.features(model, \"file.txt\")\n expect_equal(res, \"file.txt\")\n\n # columns are reordered\n res <- validate.features(model, mtcars[, rev(names(mtcars))])\n expect_equal(names(res), colnames(x))\n expect_equal(as.matrix(res), x)\n res <- validate.features(model, as.matrix(mtcars[, rev(names(mtcars))]))\n expect_equal(colnames(res), colnames(x))\n expect_equal(res, x)\n res <- validate.features(model, mtcars[1, rev(names(mtcars)), drop = FALSE])\n expect_equal(names(res), colnames(x))\n expect_equal(unname(as.matrix(res)), unname(x[1, , drop = FALSE]))\n res <- validate.features(model, as.data.table(mtcars[, rev(names(mtcars))]))\n expect_equal(names(res), colnames(x))\n expect_equal(unname(as.matrix(res)), unname(x))\n\n # error when columns are missing\n expect_error({\n validate.features(model, mtcars[, 1:3])\n })\n expect_error({\n validate.features(model, as.matrix(mtcars[, 1:ncol(x)])) # nolint\n })\n expect_error({\n validate.features(model, xgb.DMatrix(mtcars[, 1:3], nthread = 1))\n })\n expect_error({\n validate.features(model, as(x[, 1:3], \"CsparseMatrix\"))\n })\n\n # error when it cannot reorder or subset\n expect_error({\n validate.features(model, xgb.DMatrix(mtcars, nthread = 1))\n }, \"Feature names\")\n expect_error({\n validate.features(model, xgb.DMatrix(x[, rev(colnames(x))], nthread = 1))\n }, \"Feature names\")\n\n # no error about types if the booster doesn't have types\n expect_error({\n validate.features(model, xgb.DMatrix(x, feature_types = c(rep(\"q\", 5), rep(\"c\", 5)), nthread = 1))\n }, NA)\n tmp <- mtcars\n tmp[[\"vs\"]] <- factor(tmp[[\"vs\"]])\n expect_error({\n validate.features(model, tmp)\n }, NA)\n\n # error when types do not match\n setinfo(model, \"feature_type\", rep(\"q\", 10))\n expect_error({\n validate.features(model, xgb.DMatrix(x, feature_types = c(rep(\"q\", 5), rep(\"c\", 5)), nthread = 1))\n }, \"Feature types\")\n tmp <- mtcars\n tmp[[\"vs\"]] <- factor(tmp[[\"vs\"]])\n expect_error({\n validate.features(model, tmp)\n }, \"Feature types\")\n})\n\ntest_that(\"Parameters constructor works as expected\", {\n empty_list <- list()\n names(empty_list) <- character()\n\n params <- xgb.params()\n expect_equal(params, empty_list)\n\n params <- xgb.params(max_depth = 2)\n expect_equal(params, list(max_depth = 2))\n\n params <- xgb.params(max_depth = NULL)\n expect_equal(params, empty_list)\n\n max_depth <- 3\n params <- xgb.params(max_depth = max_depth)\n expect_equal(params, list(max_depth = 3))\n\n four <- 4L\n params <- xgb.params(max_depth = four)\n expect_equal(params, list(max_depth = 4L))\n\n params <- xgb.params(objective = \"binary:logistic\", nthread = 10)\n expect_equal(params, list(objective = \"binary:logistic\", nthread = 10))\n\n expect_error({\n xgb.params(max_xgboost = 10)\n })\n expect_error({\n xgb.params(max_depth = 2, max_depth = 3)\n })\n})\n" }, { "path": "R-package/tests/testthat/test_interaction_constraints.R", "content": "require(xgboost)\n\ncontext(\"interaction constraints\")\n\nn_threads <- 2\n\nset.seed(1024)\nx1 <- rnorm(1000, 1)\nx2 <- rnorm(1000, 1)\nx3 <- sample(c(1, 2, 3), size = 1000, replace = TRUE)\ny <- x1 + x2 + x3 + x1 * x2 * x3 + rnorm(1000, 0.001) + 3 * sin(x1)\ntrain <- matrix(c(x1, x2, x3), ncol = 3)\n\ntest_that(\"interaction constraints for regression\", {\n # Fit a model that only allows interaction between x1 and x2\n bst <- xgb.train(\n data = xgb.DMatrix(train, label = y, nthread = 1),\n nrounds = 100, verbose = 0,\n params = xgb.params(\n max_depth = 3,\n learning_rate = 0.1,\n nthread = 2,\n interaction_constraints = list(c(0, 1))\n )\n )\n\n # Set all observations to have the same x3 values then increment\n # by the same amount\n preds <- lapply(c(1, 2, 3), function(x) {\n tmat <- matrix(c(x1, x2, rep(x, 1000)), ncol = 3)\n return(predict(bst, tmat))\n })\n\n # Check incrementing x3 has the same effect on all observations\n # since x3 is constrained to be independent of x1 and x2\n # and all observations start off from the same x3 value\n diff1 <- preds[[2]] - preds[[1]]\n test1 <- all(abs(diff1 - diff1[1]) < 1e-4)\n\n diff2 <- preds[[3]] - preds[[2]]\n test2 <- all(abs(diff2 - diff2[1]) < 1e-4)\n\n expect_true({\n test1 & test2\n }, \"Interaction Contraint Satisfied\")\n})\n\ntest_that(\"interaction constraints scientific representation\", {\n rows <- 10\n ## When number exceeds 1e5, R paste function uses scientific representation.\n ## See: https://github.com/dmlc/xgboost/issues/5179\n cols <- 1e5 + 10\n\n d <- matrix(rexp(rows, rate = .1), nrow = rows, ncol = cols)\n y <- rnorm(rows)\n\n dtrain <- xgb.DMatrix(data = d, label = y, nthread = n_threads)\n inc <- list(c(seq.int(from = 0, to = cols, by = 1)))\n\n with_inc <- xgb.train(\n data = dtrain,\n nrounds = 10,\n params = xgb.params(\n tree_method = 'hist',\n interaction_constraints = inc,\n nthread = n_threads\n )\n )\n without_inc <- xgb.train(\n data = dtrain,\n nrounds = 10,\n params = xgb.params(\n tree_method = 'hist',\n nthread = n_threads\n )\n )\n expect_equal(xgb.save.raw(with_inc), xgb.save.raw(without_inc))\n})\n" }, { "path": "R-package/tests/testthat/test_interactions.R", "content": "context('Test prediction of feature interactions')\n\nset.seed(123)\nn_threads <- 2\n\ntest_that(\"predict feature interactions works\", {\n # simulate some binary data and a linear outcome with an interaction term\n N <- 1000\n P <- 5\n X <- matrix(rbinom(N * P, 1, 0.5), ncol = P, dimnames = list(NULL, letters[1:P]))\n # center the data (as contributions are computed WRT feature means)\n X <- scale(X, scale = FALSE)\n\n # outcome without any interactions, without any noise:\n f <- function(x) 2 * x[, 1] - 3 * x[, 2]\n # outcome with interactions, without noise:\n f_int <- function(x) f(x) + 2 * x[, 2] * x[, 3]\n # outcome with interactions, with noise:\n #f_int_noise <- function(x) f_int(x) + rnorm(N, 0, 0.3)\n\n y <- f_int(X)\n\n dm <- xgb.DMatrix(X, label = y, nthread = n_threads)\n param <- xgb.params(\n learning_rate = 0.1,\n max_depth = 4,\n base_score = mean(y),\n reg_lambda = 0,\n nthread = n_threads\n )\n b <- xgb.train(param, dm, 100)\n\n pred <- predict(b, dm, outputmargin = TRUE)\n\n # SHAP contributions:\n cont <- predict(b, dm, predcontrib = TRUE)\n expect_equal(dim(cont), c(N, P + 1))\n # make sure for each row they add up to marginal predictions\n expect_lt(max(abs(rowSums(cont) - pred)), 0.001)\n # Hand-construct the 'ground truth' feature contributions:\n gt_cont <- cbind(\n 2. * X[, 1],\n -3. * X[, 2] + 1. * X[, 2] * X[, 3], # attribute a HALF of the interaction term to feature #2\n 1. * X[, 2] * X[, 3] # and another HALF of the interaction term to feature #3\n )\n gt_cont <- cbind(gt_cont, matrix(0, nrow = N, ncol = P + 1 - 3))\n # These should be relatively close:\n expect_lt(max(abs(cont - gt_cont)), 0.05)\n\n\n # SHAP interaction contributions:\n intr <- predict(b, dm, predinteraction = TRUE)\n expect_equal(dim(intr), c(N, P + 1, P + 1))\n # check assigned colnames\n cn <- c(letters[1:P], \"(Intercept)\")\n expect_equal(dimnames(intr), list(NULL, cn, cn))\n\n # check the symmetry\n expect_lt(max(abs(aperm(intr, c(1, 3, 2)) - intr)), 0.00001)\n\n # sums WRT columns must be close to feature contributions\n expect_lt(max(abs(apply(intr, c(1, 2), sum) - cont)), 0.00001)\n\n # diagonal terms for features 3,4,5 must be close to zero\n expect_lt(Reduce(max, sapply(3:P, function(i) max(abs(intr[, i, i])))), 0.05)\n\n # Intercept must have no interactions\n expect_lt(max(abs(intr[, 1:P, P + 1])), 0.00001)\n\n # interactions other than 2 x 3 must be close to zero\n intr23 <- intr\n intr23[, 2, 3] <- 0\n expect_lt(\n Reduce(max, sapply(1:P, function(i) max(abs(intr23[, i, (i + 1):(P + 1)])))),\n 0.05\n )\n\n # Construct the 'ground truth' contributions of interactions directly from the linear terms:\n gt_intr <- array(0, c(N, P + 1, P + 1))\n gt_intr[, 2, 3] <- 1. * X[, 2] * X[, 3] # attribute a HALF of the interaction term to each symmetric element\n gt_intr[, 3, 2] <- gt_intr[, 2, 3]\n # merge-in the diagonal based on 'ground truth' feature contributions\n intr_diag <- gt_cont - apply(gt_intr, c(1, 2), sum)\n for (j in seq_len(P)) {\n gt_intr[, j, j] <- intr_diag[, j]\n }\n # These should be relatively close:\n expect_lt(max(abs(intr - gt_intr)), 0.1)\n})\n\ntest_that(\"SHAP contribution values are not NAN\", {\n d <- data.frame(\n x1 = c(-2.3, 1.4, 5.9, 2, 2.5, 0.3, -3.6, -0.2, 0.5, -2.8, -4.6, 3.3, -1.2,\n -1.1, -2.3, 0.4, -1.5, -0.2, -1, 3.7),\n x2 = c(291.179171, 269.198331, 289.942097, 283.191669, 269.673332,\n 294.158346, 287.255835, 291.530838, 285.899586, 269.290833,\n 268.649586, 291.530841, 280.074593, 269.484168, 293.94042,\n 294.327506, 296.20709, 295.441669, 283.16792, 270.227085),\n y = c(9, 15, 5.7, 9.2, 22.4, 5, 9, 3.2, 7.2, 13.1, 7.8, 16.9, 6.5, 22.1,\n 5.3, 10.4, 11.1, 13.9, 11, 20.5),\n fold = c(2, 2, 2, 1, 2, 2, 1, 2, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2))\n\n ivs <- c(\"x1\", \"x2\")\n\n fit <- xgb.train(\n verbose = 0,\n params = list(\n objective = \"reg:squarederror\",\n eval_metric = \"rmse\",\n nthread = n_threads\n ),\n data = xgb.DMatrix(as.matrix(subset(d, fold == 2)[, ivs]), label = subset(d, fold == 2)$y, nthread = 1),\n nrounds = 3\n )\n\n shaps <- as.data.frame(predict(fit,\n newdata = as.matrix(subset(d, fold == 1)[, ivs]),\n predcontrib = TRUE))\n result <- cbind(shaps, sum = rowSums(shaps), pred = predict(fit,\n newdata = as.matrix(subset(d, fold == 1)[, ivs])))\n\n expect_true(identical(TRUE, all.equal(result$sum, result$pred, tol = 1e-6)))\n})\n\n\ntest_that(\"multiclass feature interactions work\", {\n dm <- xgb.DMatrix(\n as.matrix(iris[, -5]), label = as.numeric(iris$Species) - 1, nthread = n_threads\n )\n param <- xgb.params(\n learning_rate = 0.1,\n max_depth = 4,\n objective = 'multi:softprob',\n num_class = 3,\n nthread = n_threads\n )\n b <- xgb.train(param, dm, 40)\n pred <- predict(b, dm, outputmargin = TRUE)\n\n # SHAP contributions:\n cont <- predict(b, dm, predcontrib = TRUE)\n expect_length(dim(cont), 3)\n\n # make sure for each row they add up to marginal predictions\n expect_lt(max(abs(apply(cont, c(1, 2), sum) - pred)), 0.001)\n\n # SHAP interaction contributions:\n intr <- predict(b, dm, predinteraction = TRUE)\n expect_length(dim(intr), 4)\n\n # check the symmetry\n expect_lt(max(abs(aperm(intr, c(1, 2, 4, 3)) - intr)), 0.00001)\n # sums WRT columns must be close to feature contributions\n expect_lt(max(abs(apply(intr, c(1, 2, 3), sum) - cont)), 0.00001)\n})\n\n\ntest_that(\"SHAP single sample works\", {\n train <- agaricus.train\n test <- agaricus.test\n booster <- xgb.train(\n data = xgb.DMatrix(train$data, label = train$label, nthread = 1),\n nrounds = 4,\n params = xgb.params(\n max_depth = 2,\n objective = \"binary:logistic\",\n nthread = n_threads\n )\n )\n\n predt <- predict(\n booster,\n newdata = train$data[1, , drop = FALSE], predcontrib = TRUE\n )\n expect_equal(dim(predt), c(1, dim(train$data)[2] + 1))\n\n predt <- predict(\n booster,\n newdata = train$data[1, , drop = FALSE], predinteraction = TRUE\n )\n expect_equal(dim(predt), c(1, dim(train$data)[2] + 1, dim(train$data)[2] + 1))\n})\n" }, { "path": "R-package/tests/testthat/test_io.R", "content": "context(\"Test model IO.\")\n\ndata(agaricus.train, package = \"xgboost\")\ndata(agaricus.test, package = \"xgboost\")\ntrain <- agaricus.train\ntest <- agaricus.test\n\ntest_that(\"load/save raw works\", {\n nrounds <- 8\n booster <- xgb.train(\n data = xgb.DMatrix(train$data, label = train$label, nthread = 1),\n nrounds = nrounds,\n params = xgb.params(\n objective = \"binary:logistic\",\n nthread = 2\n )\n )\n\n json_bytes <- xgb.save.raw(booster, raw_format = \"json\")\n ubj_bytes <- xgb.save.raw(booster, raw_format = \"ubj\")\n\n from_json <- xgb.load.raw(json_bytes)\n from_ubj <- xgb.load.raw(ubj_bytes)\n\n json2ubj <- xgb.save.raw(from_json, raw_format = \"ubj\")\n ubj2ubj <- xgb.save.raw(from_ubj, raw_format = \"ubj\")\n\n expect_equal(json2ubj, ubj2ubj)\n})\n\ntest_that(\"saveRDS preserves C and R attributes\", {\n data(mtcars)\n y <- mtcars$mpg\n x <- as.matrix(mtcars[, -1])\n dm <- xgb.DMatrix(x, label = y, nthread = 1)\n model <- xgb.train(\n data = dm,\n params = xgb.params(nthread = 1, max_depth = 2),\n nrounds = 5\n )\n attributes(model)$my_attr <- \"qwerty\"\n xgb.attr(model, \"c_attr\") <- \"asdf\"\n\n fname <- file.path(tempdir(), \"xgb_model.Rds\")\n saveRDS(model, fname)\n model_new <- readRDS(fname)\n\n expect_equal(attributes(model_new)$my_attr, attributes(model)$my_attr)\n expect_equal(xgb.attr(model, \"c_attr\"), xgb.attr(model_new, \"c_attr\"))\n})\n\ntest_that(\"R serializers keep C config\", {\n data(mtcars)\n y <- mtcars$mpg\n x <- as.matrix(mtcars[, -1])\n dm <- xgb.DMatrix(x, label = y, nthread = 1)\n model <- xgb.train(\n data = dm,\n params = list(\n tree_method = \"approx\",\n nthread = 1,\n max_depth = 2\n ),\n nrounds = 3\n )\n model_new <- unserialize(serialize(model, NULL))\n expect_equal(\n xgb.config(model)$learner$gradient_booster$gbtree_train_param$tree_method,\n xgb.config(model_new)$learner$gradient_booster$gbtree_train_param$tree_method\n )\n expect_equal(variable.names(model), variable.names(model_new))\n})\n" }, { "path": "R-package/tests/testthat/test_model_compatibility.R", "content": "context(\"Models from previous versions of XGBoost can be loaded\")\n\nmetadata <- list(\n kRounds = 4,\n kRows = 1000,\n kCols = 4,\n kForests = 2,\n kMaxDepth = 2,\n kClasses = 3\n)\n\nrun_model_param_check <- function(config) {\n testthat::expect_equal(config$learner$learner_model_param$num_feature, \"4\")\n testthat::expect_equal(config$learner$learner_train_param$booster, \"gbtree\")\n}\n\nget_n_rounds <- function(model_file) {\n is_10 <- grepl(\"1.0.0rc1\", model_file, fixed = TRUE)\n if (is_10) {\n 2\n } else {\n metadata$kRounds\n }\n}\n\nget_num_tree <- function(booster) {\n dump <- xgb.dump(booster)\n m <- regexec(\"booster\\\\[[0-9]+\\\\]\", dump, perl = TRUE)\n m <- regmatches(dump, m)\n num_tree <- Reduce(\"+\", lapply(m, length))\n num_tree\n}\n\nrun_booster_check <- function(booster, model_file) {\n config <- xgb.config(booster)\n run_model_param_check(config)\n is_model <- function(typ) {\n grepl(typ, model_file, fixed = TRUE)\n }\n n_rounds <- get_n_rounds(model_file = model_file)\n if (is_model(\"cls\")) {\n testthat::expect_equal(\n get_num_tree(booster), metadata$kForests * n_rounds * metadata$kClasses\n )\n testthat::expect_equal(get_basescore(config), c(0.5, 0.5, 0.5)) # nolint\n testthat::expect_equal(\n config$learner$learner_train_param$objective, \"multi:softmax\"\n )\n testthat::expect_equal(\n as.numeric(config$learner$learner_model_param$num_class),\n metadata$kClasses\n )\n } else if (is_model(\"logitraw\")) {\n testthat::expect_equal(get_num_tree(booster), metadata$kForests * n_rounds)\n testthat::expect_equal(\n as.numeric(config$learner$learner_model_param$num_class), 0\n )\n testthat::expect_equal(\n config$learner$learner_train_param$objective, \"binary:logitraw\"\n )\n } else if (is_model(\"logit\")) {\n testthat::expect_equal(get_num_tree(booster), metadata$kForests * n_rounds)\n testthat::expect_equal(\n as.numeric(config$learner$learner_model_param$num_class), 0\n )\n testthat::expect_equal(\n config$learner$learner_train_param$objective, \"binary:logistic\"\n )\n } else if (is_model(\"ltr\")) {\n testthat::expect_equal(get_num_tree(booster), metadata$kForests * n_rounds)\n testthat::expect_equal(\n config$learner$learner_train_param$objective, \"rank:ndcg\"\n )\n } else if (is_model(\"aft\")) {\n testthat::expect_equal(get_num_tree(booster), metadata$kForests * n_rounds)\n testthat::expect_equal(\n config$learner$learner_train_param$objective, \"survival:aft\"\n )\n } else {\n testthat::expect_true(is_model(\"reg\"))\n testthat::expect_equal(get_num_tree(booster), metadata$kForests * n_rounds)\n testthat::expect_equal(get_basescore(config), 0.5) # nolint\n testthat::expect_equal(\n config$learner$learner_train_param$objective, \"reg:squarederror\"\n )\n }\n}\n\ntest_that(\"Models from previous versions of XGBoost can be loaded\", {\n bucket <- \"xgboost-ci-jenkins-artifacts\"\n region <- \"us-west-2\"\n file_name <- \"xgboost_model_compatibility_tests-3.0.2.zip\"\n zipfile <- tempfile(fileext = \".zip\")\n extract_dir <- tempdir()\n result <- tryCatch(\n {\n download.file(\n paste(\n \"https://\", bucket, \".s3-\", region, \".amazonaws.com/\", file_name,\n sep = \"\"\n ),\n destfile = zipfile, mode = \"wb\", quiet = TRUE\n )\n zipfile\n },\n error = function(e) {\n print(e)\n NA_character_\n }\n )\n if (is.na(result)) {\n print(\"Failed to download old models.\")\n return()\n }\n\n unzip(zipfile, exdir = extract_dir, overwrite = TRUE)\n model_dir <- file.path(extract_dir, \"models\")\n\n pred_data <- xgb.DMatrix(\n matrix(c(0, 0, 0, 0), nrow = 1, ncol = 4),\n nthread = 2\n )\n\n lapply(list.files(model_dir), function(x) {\n model_file <- file.path(model_dir, x)\n is_skl <- grepl(\"scikit\", model_file, fixed = TRUE)\n if (is_skl) {\n return()\n }\n booster <- xgb.load(model_file)\n xgb.model.parameters(booster) <- list(nthread = 2)\n predict(booster, newdata = pred_data)\n run_booster_check(booster, model_file)\n })\n})\n" }, { "path": "R-package/tests/testthat/test_monotone.R", "content": "context(\"monotone constraints\")\n\nset.seed(1024)\nx <- rnorm(1000, 10)\ny <- -1 * x + rnorm(1000, 0.001) + 3 * sin(x)\ntrain <- matrix(x, ncol = 1)\n\n\ntest_that(\"monotone constraints for regression\", {\n bst <- xgb.train(\n data = xgb.DMatrix(train, label = y, nthread = 1),\n nrounds = 100, verbose = 0,\n params = xgb.params(\n max_depth = 2,\n learning_rate = 0.1,\n nthread = 2,\n monotone_constraints = -1\n )\n )\n\n pred <- predict(bst, train)\n\n ind <- order(train[, 1])\n pred.ord <- pred[ind]\n expect_true({\n !any(diff(pred.ord) > 0)\n }, \"Monotone constraint satisfied\")\n})\n" }, { "path": "R-package/tests/testthat/test_parameter_exposure.R", "content": "context('Test model params and call are exposed to R')\n\ndata(agaricus.train, package = 'xgboost')\ndata(agaricus.test, package = 'xgboost')\n\ndtrain <- xgb.DMatrix(\n agaricus.train$data, label = agaricus.train$label, nthread = 2\n)\ndtest <- xgb.DMatrix(\n agaricus.test$data, label = agaricus.test$label, nthread = 2\n)\n\nbst <- xgb.train(\n data = dtrain,\n verbose = 0,\n nrounds = 10,\n params = xgb.params(\n max_depth = 2,\n learning_rate = 1,\n nthread = 1,\n objective = \"binary:logistic\"\n )\n)\n\ntest_that(\"call is exposed to R\", {\n expect_false(is.null(attributes(bst)$call))\n expect_is(attributes(bst)$call, \"call\")\n})\n\ntest_that(\"params is exposed to R\", {\n model_params <- attributes(bst)$params\n expect_is(model_params, \"list\")\n expect_equal(model_params$learning_rate, 1)\n expect_equal(model_params$max_depth, 2)\n expect_equal(model_params$objective, \"binary:logistic\")\n})\n" }, { "path": "R-package/tests/testthat/test_poisson_regression.R", "content": "context('Test Poisson regression model')\n\nset.seed(1994)\n\ntest_that(\"Poisson regression works\", {\n data(mtcars)\n bst <- xgb.train(\n data = xgb.DMatrix(as.matrix(mtcars[, -11]), label = mtcars[, 11], nthread = 1),\n nrounds = 10, verbose = 0,\n params = xgb.params(objective = 'count:poisson', nthread = 2)\n )\n expect_equal(class(bst), \"xgb.Booster\")\n pred <- predict(bst, as.matrix(mtcars[, -11]))\n expect_equal(length(pred), 32)\n expect_lt(sqrt(mean((pred - mtcars[, 11])^2)), 1.2)\n})\n\ntest_that(\"Poisson regression is centered around mean\", {\n m <- 50L\n n <- 10L\n y <- rpois(m, n)\n x <- matrix(rnorm(m * n), nrow = m)\n model <- xgb.train(\n data = xgb.DMatrix(x, label = y, nthread = 1),\n params = xgb.params(objective = \"count:poisson\", min_split_loss = 1e4),\n nrounds = 1\n )\n model_json <- xgb.save.raw(model, \"json\") |> rawToChar() |> jsonlite::fromJSON()\n expect_equal(\n get_basescore(model_json),\n mean(y),\n tolerance = 1e-4\n )\n\n pred <- predict(model, x)\n expect_equal(\n pred,\n rep(mean(y), m),\n tolerance = 1e-4\n )\n\n w <- y + 1\n model_weighted <- xgb.train(\n data = xgb.DMatrix(x, label = y, weight = w, nthread = 1),\n params = xgb.params(objective = \"count:poisson\", min_split_loss = 1e4),\n nrounds = 1\n )\n model_json <- xgb.save.raw(model_weighted, \"json\") |> rawToChar() |> jsonlite::fromJSON()\n expect_equal(\n get_basescore(model_json),\n weighted.mean(y, w),\n tolerance = 1e-4\n )\n})\n" }, { "path": "R-package/tests/testthat/test_ranking.R", "content": "context('Learning to rank')\n\nn_threads <- 2\n\ntest_that('Test ranking with unweighted data', {\n X <- Matrix::sparseMatrix(\n i = c(2, 3, 7, 9, 12, 15, 17, 18)\n , j = c(1, 1, 2, 2, 3, 3, 4, 4)\n , x = rep(1.0, 8)\n , dims = c(20, 4)\n )\n y <- c(0, 1, 1, 0, 0, 0, 1, 0, 1, 0, 0, 1, 0, 0, 1, 0, 1, 1, 0, 0)\n group <- c(5, 5, 5, 5)\n dtrain <- xgb.DMatrix(X, label = y, group = group, nthread = n_threads)\n\n params <- xgb.params(\n learning_rate = 1,\n tree_method = 'exact',\n objective = 'rank:pairwise',\n max_depth = 1,\n eval_metric = c('auc', 'aucpr'),\n nthread = n_threads\n )\n bst <- xgb.train(params, dtrain, nrounds = 10, evals = list(train = dtrain), verbose = 0)\n # Check if the metric is monotone increasing\n expect_true(all(diff(attributes(bst)$evaluation_log$train_auc) >= 0))\n expect_true(all(diff(attributes(bst)$evaluation_log$train_aucpr) >= 0))\n})\n\ntest_that('Test ranking with weighted data', {\n X <- Matrix::sparseMatrix(\n i = c(2, 3, 7, 9, 12, 15, 17, 18)\n , j = c(1, 1, 2, 2, 3, 3, 4, 4)\n , x = rep(1.0, 8)\n , dims = c(20, 4)\n )\n y <- c(0, 1, 1, 0, 0, 0, 1, 0, 1, 0, 0, 1, 0, 0, 1, 0, 1, 1, 0, 0)\n group <- c(5, 5, 5, 5)\n weight <- c(1.0, 2.0, 3.0, 4.0)\n dtrain <- xgb.DMatrix(\n X, label = y, group = group, weight = weight, nthread = n_threads\n )\n\n params <- xgb.params(\n learning_rate = 1,\n tree_method = \"exact\",\n objective = \"rank:pairwise\",\n max_depth = 1,\n eval_metric = c(\"auc\", \"aucpr\"),\n nthread = n_threads\n )\n bst <- xgb.train(params, dtrain, nrounds = 10, evals = list(train = dtrain), verbose = 0)\n # Check if the metric is monotone increasing\n expect_true(all(diff(attributes(bst)$evaluation_log$train_auc) >= 0))\n expect_true(all(diff(attributes(bst)$evaluation_log$train_aucpr) >= 0))\n for (i in 1:10) {\n pred <- predict(bst, newdata = dtrain, iterationrange = c(1, i))\n # is_sorted[i]: is i-th group correctly sorted by the ranking predictor?\n is_sorted <- lapply(seq(1, 20, by = 5),\n function(k) {\n ind <- order(-pred[k:(k + 4)])\n z <- y[ind + (k - 1)]\n all(diff(z) <= 0) # Check if z is monotone decreasing\n })\n # Since we give weights 1, 2, 3, 4 to the four query groups,\n # the ranking predictor will first try to correctly sort the last query group\n # before correctly sorting other groups.\n expect_true(all(diff(as.numeric(is_sorted)) >= 0))\n }\n})\n" }, { "path": "R-package/tests/testthat/test_unicode.R", "content": "context(\"Test Unicode handling\")\n\ndata(agaricus.train, package = 'xgboost')\ndata(agaricus.test, package = 'xgboost')\ntrain <- agaricus.train\ntest <- agaricus.test\nset.seed(1994)\n\ntest_that(\"Can save and load models with Unicode paths\", {\n nrounds <- 2\n bst <- xgb.train(\n data = xgb.DMatrix(train$data, label = train$label, nthread = 1),\n nrounds = nrounds,\n params = xgb.params(\n max_depth = 2,\n nthread = 2,\n objective = \"binary:logistic\"\n )\n )\n tmpdir <- tempdir()\n lapply(c(\"모델.json\", \"がうる・ぐら.json\", \"类继承.ubj\"), function(x) {\n path <- file.path(tmpdir, x)\n xgb.save(bst, path)\n bst2 <- xgb.load(path)\n xgb.model.parameters(bst2) <- list(nthread = 2)\n expect_equal(predict(bst, test$data), predict(bst2, test$data))\n })\n})\n" }, { "path": "R-package/tests/testthat/test_update.R", "content": "context(\"update trees in an existing model\")\n\ndata(agaricus.train, package = 'xgboost')\ndata(agaricus.test, package = 'xgboost')\n\nn_threads <- 1\n\ndtrain <- xgb.DMatrix(\n agaricus.train$data, label = agaricus.train$label, nthread = n_threads\n)\ndtest <- xgb.DMatrix(\n agaricus.test$data, label = agaricus.test$label, nthread = n_threads\n)\n\n# Disable flaky tests for 32-bit Windows.\n# See https://github.com/dmlc/xgboost/issues/3720\nwin32_flag <- .Platform$OS.type == \"windows\" && .Machine$sizeof.pointer != 8\n\ntest_that(\"updating the model works\", {\n evals <- list(train = dtrain, test = dtest)\n\n # no-subsampling\n p1 <- xgb.params(\n objective = \"binary:logistic\",\n max_depth = 2,\n learning_rate = 0.05,\n nthread = n_threads,\n updater = \"grow_colmaker,prune\"\n )\n set.seed(11)\n bst1 <- xgb.train(p1, dtrain, nrounds = 10, evals = evals, verbose = 0)\n tr1 <- xgb.model.dt.tree(model = bst1)\n\n # with subsampling\n p2 <- modifyList(p1, list(subsample = 0.1))\n set.seed(11)\n bst2 <- xgb.train(p2, dtrain, nrounds = 10, evals = evals, verbose = 0)\n tr2 <- xgb.model.dt.tree(model = bst2)\n\n # the same no-subsampling boosting with an extra 'refresh' updater:\n p1r <- modifyList(p1, list(updater = 'grow_colmaker,prune,refresh', refresh_leaf = FALSE))\n set.seed(11)\n bst1r <- xgb.train(p1r, dtrain, nrounds = 10, evals = evals, verbose = 0)\n tr1r <- xgb.model.dt.tree(model = bst1r)\n # all should be the same when no subsampling\n expect_equal(attributes(bst1)$evaluation_log, attributes(bst1r)$evaluation_log)\n expect_equal(\n jsonlite::fromJSON(rawToChar(xgb.save.raw(bst1, raw_format = \"json\"))),\n jsonlite::fromJSON(rawToChar(xgb.save.raw(bst1r, raw_format = \"json\"))),\n tolerance = 1e-6\n )\n if (!win32_flag) {\n expect_equal(tr1, tr1r, tolerance = 0.00001, check.attributes = FALSE)\n }\n\n # the same boosting with subsampling with an extra 'refresh' updater:\n p2r <- modifyList(p2, list(updater = 'grow_colmaker,prune,refresh', refresh_leaf = FALSE))\n set.seed(11)\n bst2r <- xgb.train(p2r, dtrain, nrounds = 10, evals = evals, verbose = 0)\n tr2r <- xgb.model.dt.tree(model = bst2r)\n # should be the same evaluation but different gains and larger cover\n expect_equal(attributes(bst2)$evaluation_log, attributes(bst2r)$evaluation_log)\n if (!win32_flag) {\n expect_equal(tr2[Feature == 'Leaf']$Gain, tr2r[Feature == 'Leaf']$Gain)\n }\n expect_gt(sum(abs(tr2[Feature != 'Leaf']$Gain - tr2r[Feature != 'Leaf']$Gain)), 100)\n expect_gt(sum(tr2r$Cover) / sum(tr2$Cover), 1.5)\n\n # process type 'update' for no-subsampling model, refreshing the tree stats AND leaves from training data:\n set.seed(123)\n p1u <- modifyList(p1, list(process_type = 'update', updater = 'refresh', refresh_leaf = TRUE))\n bst1u <- xgb.train(p1u, dtrain, nrounds = 10, evals = evals, verbose = 0, xgb_model = bst1)\n tr1u <- xgb.model.dt.tree(model = bst1u)\n # all should be the same when no subsampling\n expect_equal(attributes(bst1)$evaluation_log, attributes(bst1u)$evaluation_log)\n expect_equal(\n jsonlite::fromJSON(rawToChar(xgb.save.raw(bst1, raw_format = \"json\"))),\n jsonlite::fromJSON(rawToChar(xgb.save.raw(bst1u, raw_format = \"json\"))),\n tolerance = 1e-6\n )\n expect_equal(tr1, tr1u, tolerance = 0.00001, check.attributes = FALSE)\n\n # same thing but with a serialized model\n set.seed(123)\n bst1u <- xgb.train(p1u, dtrain, nrounds = 10, evals = evals, verbose = 0, xgb_model = xgb.save.raw(bst1))\n tr1u <- xgb.model.dt.tree(model = bst1u)\n # all should be the same when no subsampling\n expect_equal(attributes(bst1)$evaluation_log, attributes(bst1u)$evaluation_log)\n expect_equal(tr1, tr1u, tolerance = 0.00001, check.attributes = FALSE)\n\n # process type 'update' for model with subsampling, refreshing only the tree stats from training data:\n p2u <- modifyList(p2, list(process_type = 'update', updater = 'refresh', refresh_leaf = FALSE))\n bst2u <- xgb.train(p2u, dtrain, nrounds = 10, evals = evals, verbose = 0, xgb_model = bst2)\n tr2u <- xgb.model.dt.tree(model = bst2u)\n # should be the same evaluation but different gains and larger cover\n expect_equal(attributes(bst2)$evaluation_log, attributes(bst2u)$evaluation_log)\n expect_equal(tr2[Feature == 'Leaf']$Gain, tr2u[Feature == 'Leaf']$Gain)\n expect_gt(sum(abs(tr2[Feature != 'Leaf']$Gain - tr2u[Feature != 'Leaf']$Gain)), 100)\n expect_gt(sum(tr2u$Cover) / sum(tr2$Cover), 1.5)\n # the results should be the same as for the model with an extra 'refresh' updater\n expect_equal(attributes(bst2r)$evaluation_log, attributes(bst2u)$evaluation_log)\n if (!win32_flag) {\n expect_equal(tr2r, tr2u, tolerance = 0.00001, check.attributes = FALSE)\n }\n\n # process type 'update' for no-subsampling model, refreshing only the tree stats from TEST data:\n p1ut <- modifyList(p1, list(process_type = 'update', updater = 'refresh', refresh_leaf = FALSE))\n bst1ut <- xgb.train(p1ut, dtest, nrounds = 10, evals = evals, verbose = 0, xgb_model = bst1)\n tr1ut <- xgb.model.dt.tree(model = bst1ut)\n # should be the same evaluations but different gains and smaller cover (test data is smaller)\n expect_equal(attributes(bst1)$evaluation_log, attributes(bst1ut)$evaluation_log)\n expect_equal(tr1[Feature == 'Leaf']$Gain, tr1ut[Feature == 'Leaf']$Gain)\n expect_gt(sum(abs(tr1[Feature != 'Leaf']$Gain - tr1ut[Feature != 'Leaf']$Gain)), 100)\n expect_lt(sum(tr1ut$Cover) / sum(tr1$Cover), 0.5)\n})\n\ntest_that(\"updating works for multiclass & multitree\", {\n dtr <- xgb.DMatrix(\n as.matrix(iris[, -5]), label = as.numeric(iris$Species) - 1, nthread = n_threads\n )\n evals <- list(train = dtr)\n p0 <- xgb.params(\n max_depth = 2,\n learning_rate = 0.5,\n nthread = n_threads,\n subsample = 0.6,\n objective = \"multi:softprob\",\n num_class = 3,\n num_parallel_tree = 2,\n base_score = 0\n )\n set.seed(121)\n bst0 <- xgb.train(p0, dtr, 5, evals = evals, verbose = 0)\n tr0 <- xgb.model.dt.tree(model = bst0)\n\n # run update process for an original model with subsampling\n p0u <- modifyList(p0, list(process_type = 'update', updater = 'refresh', refresh_leaf = FALSE))\n bst0u <- xgb.train(p0u, dtr, nrounds = xgb.get.num.boosted.rounds(bst0),\n evals = evals, xgb_model = bst0, verbose = 0)\n tr0u <- xgb.model.dt.tree(model = bst0u)\n\n # should be the same evaluation but different gains and larger cover\n expect_equal(attributes(bst0)$evaluation_log, attributes(bst0u)$evaluation_log)\n expect_equal(tr0[Feature == 'Leaf']$Gain, tr0u[Feature == 'Leaf']$Gain)\n expect_gt(sum(abs(tr0[Feature != 'Leaf']$Gain - tr0u[Feature != 'Leaf']$Gain)), 100)\n expect_gt(sum(tr0u$Cover) / sum(tr0$Cover), 1.5)\n})\n" }, { "path": "R-package/tests/testthat/test_xgboost.R", "content": "library(survival)\nlibrary(data.table)\ndata(\"iris\")\ndata(\"mtcars\")\ndata(\"ToothGrowth\")\n\ntest_that(\"Auto determine objective\", {\n y_num <- seq(1, 10)\n res_num <- process.y.margin.and.objective(y_num, NULL, NULL, NULL)\n expect_equal(res_num$params$objective, \"reg:squarederror\")\n\n y_bin <- factor(c('a', 'b', 'a', 'b'), c('a', 'b'))\n res_bin <- process.y.margin.and.objective(y_bin, NULL, NULL, NULL)\n expect_equal(res_bin$params$objective, \"binary:logistic\")\n\n y_multi <- factor(c('a', 'b', 'a', 'b', 'c'), c('a', 'b', 'c'))\n res_multi <- process.y.margin.and.objective(y_multi, NULL, NULL, NULL)\n expect_equal(res_multi$params$objective, \"multi:softprob\")\n\n y_surv <- Surv(1:10, rep(c(0, 1), 5), type = \"right\")\n res_surv <- process.y.margin.and.objective(y_surv, NULL, NULL, NULL)\n expect_equal(res_surv$params$objective, \"survival:aft\")\n\n y_multicol <- matrix(seq(1, 20), nrow = 5)\n res_multicol <- process.y.margin.and.objective(y_multicol, NULL, NULL, NULL)\n expect_equal(res_multicol$params$objective, \"reg:squarederror\")\n})\n\ntest_that(\"Process vectors\", {\n y <- seq(1, 10)\n for (y_inp in list(as.integer(y), as.numeric(y))) {\n res <- process.y.margin.and.objective(y_inp, NULL, \"reg:pseudohubererror\", NULL)\n expect_equal(\n res$dmatrix_args$label,\n y\n )\n expect_equal(\n res$params$objective,\n \"reg:pseudohubererror\"\n )\n }\n})\n\ntest_that(\"Process factors\", {\n y_bin <- factor(c('a', 'b', 'a', 'b'), c('a', 'b'))\n expect_error({\n process.y.margin.and.objective(y_bin, NULL, \"multi:softprob\", NULL)\n })\n for (bin_obj in c(\"binary:logistic\", \"binary:hinge\")) {\n for (y_inp in list(y_bin, as.ordered(y_bin))) {\n res_bin <- process.y.margin.and.objective(y_inp, NULL, bin_obj, NULL)\n expect_equal(\n res_bin$dmatrix_args$label,\n c(0, 1, 0, 1)\n )\n expect_equal(\n res_bin$metadata$y_levels,\n c('a', 'b')\n )\n expect_equal(\n res_bin$params$objective,\n bin_obj\n )\n }\n }\n\n y_bin2 <- factor(c(1, 0, 1, 0), c(1, 0))\n res_bin <- process.y.margin.and.objective(y_bin2, NULL, \"binary:logistic\", NULL)\n expect_equal(\n res_bin$dmatrix_args$label,\n c(0, 1, 0, 1)\n )\n expect_equal(\n res_bin$metadata$y_levels,\n c(\"1\", \"0\")\n )\n\n y_bin3 <- c(TRUE, FALSE, TRUE)\n res_bin <- process.y.margin.and.objective(y_bin3, NULL, \"binary:logistic\", NULL)\n expect_equal(\n res_bin$dmatrix_args$label,\n c(1, 0, 1)\n )\n expect_equal(\n res_bin$metadata$y_levels,\n c(\"FALSE\", \"TRUE\")\n )\n\n y_multi <- factor(c('a', 'b', 'c', 'd', 'a', 'b'), c('a', 'b', 'c', 'd'))\n expect_error({\n process.y.margin.and.objective(y_multi, NULL, \"binary:logistic\", NULL)\n })\n expect_error({\n process.y.margin.and.objective(y_multi, NULL, \"binary:logistic\", NULL)\n })\n res_multi <- process.y.margin.and.objective(y_multi, NULL, \"multi:softprob\", NULL)\n expect_equal(\n res_multi$dmatrix_args$label,\n c(0, 1, 2, 3, 0, 1)\n )\n expect_equal(\n res_multi$metadata$y_levels,\n c('a', 'b', 'c', 'd')\n )\n expect_equal(\n res_multi$params$num_class,\n 4\n )\n expect_equal(\n res_multi$params$objective,\n \"multi:softprob\"\n )\n})\n\ntest_that(\"Process survival objects\", {\n data(cancer, package = \"survival\")\n y_right <- Surv(cancer$time, cancer$status - 1, type = \"right\")\n res_cox <- process.y.margin.and.objective(y_right, NULL, \"survival:cox\", NULL)\n expect_equal(\n res_cox$dmatrix_args$label,\n ifelse(cancer$status == 2, cancer$time, -cancer$time)\n )\n expect_equal(\n res_cox$params$objective,\n \"survival:cox\"\n )\n\n res_aft <- process.y.margin.and.objective(y_right, NULL, \"survival:aft\", NULL)\n expect_equal(\n res_aft$dmatrix_args$label_lower_bound,\n cancer$time\n )\n expect_equal(\n res_aft$dmatrix_args$label_upper_bound,\n ifelse(cancer$status == 2, cancer$time, Inf)\n )\n expect_equal(\n res_aft$params$objective,\n \"survival:aft\"\n )\n\n y_left <- Surv(seq(1, 4), c(1, 0, 1, 0), type = \"left\")\n expect_error({\n process.y.margin.and.objective(y_left, NULL, \"survival:cox\", NULL)\n })\n res_aft <- process.y.margin.and.objective(y_left, NULL, \"survival:aft\", NULL)\n expect_equal(\n res_aft$dmatrix_args$label_lower_bound,\n c(1, 0, 3, 0)\n )\n expect_equal(\n res_aft$dmatrix_args$label_upper_bound,\n seq(1, 4)\n )\n expect_equal(\n res_aft$params$objective,\n \"survival:aft\"\n )\n\n y_interval <- Surv(\n time = c(1, 5, 2, 10, 3),\n time2 = c(2, 5, 2.5, 10, 3),\n event = c(3, 1, 3, 0, 2),\n type = \"interval\"\n )\n expect_error({\n process.y.margin.and.objective(y_interval, NULL, \"survival:cox\", NULL)\n })\n res_aft <- process.y.margin.and.objective(y_interval, NULL, \"survival:aft\", NULL)\n expect_equal(\n res_aft$dmatrix_args$label_lower_bound,\n c(1, 5, 2, 10, 0)\n )\n expect_equal(\n res_aft$dmatrix_args$label_upper_bound,\n c(2, 5, 2.5, Inf, 3)\n )\n expect_equal(\n res_aft$params$objective,\n \"survival:aft\"\n )\n\n y_interval_neg <- Surv(\n time = c(1, -5, 2, 10, 3),\n time2 = c(2, -5, 2.5, 10, 3),\n event = c(3, 1, 3, 0, 2),\n type = \"interval\"\n )\n expect_error({\n process.y.margin.and.objective(y_interval_neg, NULL, \"survival:aft\", NULL)\n })\n})\n\ntest_that(\"Process multi-target\", {\n data(mtcars)\n y_multi <- data.frame(\n y1 = mtcars$mpg,\n y2 = mtcars$mpg ^ 2\n )\n for (y_inp in list(y_multi, as.matrix(y_multi), data.table::as.data.table(y_multi))) {\n res_multi <- process.y.margin.and.objective(y_inp, NULL, \"reg:pseudohubererror\", NULL)\n expect_equal(\n res_multi$dmatrix_args$label,\n as.matrix(y_multi)\n )\n expect_equal(\n res_multi$metadata$y_names,\n c(\"y1\", \"y2\")\n )\n expect_equal(\n res_multi$params$objective,\n \"reg:pseudohubererror\"\n )\n }\n\n expect_error({\n process.y.margin.and.objective(y_multi, NULL, \"count:poisson\", NULL)\n })\n\n y_bad <- data.frame(\n c1 = seq(1, 3),\n c2 = rep(as.Date(\"2024-01-01\"), 3)\n )\n expect_error({\n process.y.margin.and.objective(y_bad, NULL, \"reg:squarederror\", NULL)\n })\n\n y_bad <- data.frame(\n c1 = seq(1, 3),\n c2 = factor(c('a', 'b', 'a'), c('a', 'b'))\n )\n expect_error({\n process.y.margin.and.objective(y_bad, NULL, \"reg:squarederror\", NULL)\n })\n\n y_bad <- seq(1, 20)\n dim(y_bad) <- c(5, 2, 2)\n expect_error({\n process.y.margin.and.objective(y_bad, NULL, \"reg:squarederror\", NULL)\n })\n})\n\ntest_that(\"Process base_margin\", {\n y <- seq(101, 110)\n bm_good <- seq(1, 10)\n for (bm in list(bm_good, as.matrix(bm_good), as.data.frame(as.matrix(bm_good)))) {\n res <- process.y.margin.and.objective(y, bm, \"reg:squarederror\", NULL)\n expect_equal(\n res$dmatrix_args$base_margin,\n seq(1, 10)\n )\n }\n expect_error({\n process.y.margin.and.objective(y, 5, \"reg:squarederror\", NULL)\n })\n expect_error({\n process.y.margin.and.objective(y, seq(1, 5), \"reg:squarederror\", NULL)\n })\n expect_error({\n process.y.margin.and.objective(y, matrix(seq(1, 20), ncol = 2), \"reg:squarederror\", NULL)\n })\n expect_error({\n process.y.margin.and.objective(\n y,\n as.data.frame(matrix(seq(1, 20), ncol = 2)),\n \"reg:squarederror\",\n NULL\n )\n })\n\n y <- factor(c('a', 'b', 'c', 'a'))\n bm_good <- matrix(seq(1, 12), ncol = 3)\n for (bm in list(bm_good, as.data.frame(bm_good))) {\n res <- process.y.margin.and.objective(y, bm, \"multi:softprob\", NULL)\n expect_equal(\n res$dmatrix_args$base_margin |> unname(),\n matrix(seq(1, 12), ncol = 3)\n )\n }\n expect_error({\n process.y.margin.and.objective(y, as.numeric(bm_good), \"multi:softprob\", NULL)\n })\n expect_error({\n process.y.margin.and.objective(y, 5, \"multi:softprob\", NULL)\n })\n expect_error({\n process.y.margin.and.objective(y, bm_good[, 1], \"multi:softprob\", NULL)\n })\n expect_error({\n process.y.margin.and.objective(y, bm_good[, c(1, 2)], \"multi:softprob\", NULL)\n })\n expect_error({\n process.y.margin.and.objective(y, bm_good[c(1, 2), ], \"multi:softprob\", NULL)\n })\n\n y <- seq(101, 110)\n bm_good <- matrix(seq(1, 30), ncol = 3)\n params <- list(quantile_alpha = c(0.1, 0.5, 0.9))\n for (bm in list(bm_good, as.data.frame(bm_good))) {\n res <- process.y.margin.and.objective(y, bm, \"reg:quantileerror\", params)\n expect_equal(\n res$dmatrix_args$base_margin |> unname(),\n matrix(seq(1, 30), ncol = 3)\n )\n }\n expect_error({\n process.y.margin.and.objective(y, as.numeric(bm_good), \"reg:quantileerror\", params)\n })\n expect_error({\n process.y.margin.and.objective(y, 5, \"reg:quantileerror\", params)\n })\n expect_error({\n process.y.margin.and.objective(y, bm_good[, 1], \"reg:quantileerror\", params)\n })\n expect_error({\n process.y.margin.and.objective(y, bm_good[, c(1, 2)], \"reg:quantileerror\", params)\n })\n expect_error({\n process.y.margin.and.objective(y, bm_good[c(1, 2, 3), ], \"reg:quantileerror\", params)\n })\n\n params <- list(expectile_alpha = c(0.1, 0.5, 0.9))\n for (bm in list(bm_good, as.data.frame(bm_good))) {\n res <- process.y.margin.and.objective(y, bm, \"reg:expectileerror\", params)\n expect_equal(\n res$dmatrix_args$base_margin |> unname(),\n matrix(seq(1, 30), ncol = 3)\n )\n }\n expect_error({\n process.y.margin.and.objective(y, as.numeric(bm_good), \"reg:expectileerror\", params)\n })\n expect_error({\n process.y.margin.and.objective(y, 5, \"reg:expectileerror\", params)\n })\n expect_error({\n process.y.margin.and.objective(y, bm_good[, 1], \"reg:expectileerror\", params)\n })\n expect_error({\n process.y.margin.and.objective(y, bm_good[, c(1, 2)], \"reg:expectileerror\", params)\n })\n expect_error({\n process.y.margin.and.objective(y, bm_good[c(1, 2, 3), ], \"reg:expectileerror\", params)\n })\n\n y <- matrix(seq(101, 130), ncol = 3)\n for (bm in list(bm_good, as.data.frame(bm_good))) {\n res <- process.y.margin.and.objective(y, bm, \"reg:squarederror\", params)\n expect_equal(\n res$dmatrix_args$base_margin |> unname(),\n matrix(seq(1, 30), ncol = 3)\n )\n }\n expect_error({\n process.y.margin.and.objective(y, as.numeric(bm_good), \"reg:squarederror\", params)\n })\n expect_error({\n process.y.margin.and.objective(y, 5, \"reg:squarederror\", params)\n })\n expect_error({\n process.y.margin.and.objective(y, bm_good[, 1], \"reg:squarederror\", params)\n })\n expect_error({\n process.y.margin.and.objective(y, bm_good[, c(1, 2)], \"reg:squarederror\", params)\n })\n expect_error({\n process.y.margin.and.objective(y, bm_good[c(1, 2, 3), ], \"reg:squarederror\", params)\n })\n})\n\ntest_that(\"Process monotone constraints\", {\n data(iris)\n mc_list <- list(Sepal.Width = 1)\n res <- process.x.and.col.args(\n iris,\n monotone_constraints = mc_list,\n interaction_constraints = NULL,\n feature_weights = NULL,\n lst_args = list(),\n use_qdm = FALSE\n )\n expect_equal(\n res$params$monotone_constraints,\n c(0, 1, 0, 0, 0)\n )\n\n mc_list2 <- list(Sepal.Width = 1, Petal.Width = -1)\n res <- process.x.and.col.args(\n iris,\n monotone_constraints = mc_list2,\n interaction_constraints = NULL,\n feature_weights = NULL,\n lst_args = list(),\n use_qdm = FALSE\n )\n expect_equal(\n res$params$monotone_constraints,\n c(0, 1, 0, -1, 0)\n )\n\n mc_vec <- c(0, 1, -1, 0, 0)\n res <- process.x.and.col.args(\n iris,\n monotone_constraints = mc_vec,\n interaction_constraints = NULL,\n feature_weights = NULL,\n lst_args = list(),\n use_qdm = FALSE\n )\n expect_equal(\n res$params$monotone_constraints,\n c(0, 1, -1, 0, 0)\n )\n\n mc_named_vec <- c(1, 1)\n names(mc_named_vec) <- names(iris)[1:2]\n res <- process.x.and.col.args(\n iris,\n monotone_constraints = mc_named_vec,\n interaction_constraints = NULL,\n feature_weights = NULL,\n lst_args = list(),\n use_qdm = FALSE\n )\n expect_equal(\n res$params$monotone_constraints,\n c(1, 1, 0, 0, 0)\n )\n\n mc_named_all <- c(0, -1, 1, 0, -1)\n names(mc_named_all) <- rev(names(iris))\n res <- process.x.and.col.args(\n iris,\n monotone_constraints = mc_named_all,\n interaction_constraints = NULL,\n feature_weights = NULL,\n lst_args = list(),\n use_qdm = FALSE\n )\n expect_equal(\n res$params$monotone_constraints,\n rev(mc_named_all) |> unname()\n )\n\n expect_error({\n process.x.and.col.args(\n iris,\n monotone_constraints = list(\n Sepal.Width = 1,\n Petal.Width = -1,\n Sepal.Width = -1\n ),\n interaction_constraints = NULL,\n feature_weights = NULL,\n lst_args = list(),\n use_qdm = FALSE\n )\n })\n\n expect_error({\n process.x.and.col.args(\n iris,\n monotone_constraints = rep(0, 6),\n interaction_constraints = NULL,\n feature_weights = NULL,\n lst_args = list(),\n use_qdm = FALSE\n )\n })\n})\n\ntest_that(\"Process interaction_constraints\", {\n data(iris)\n res <- process.x.and.col.args(iris, NULL, list(c(1L, 2L)), NULL, NULL, FALSE)\n expect_equal(\n res$params$interaction_constraints,\n list(c(0, 1))\n )\n res <- process.x.and.col.args(iris, NULL, list(c(1.0, 2.0)), NULL, NULL, FALSE)\n expect_equal(\n res$params$interaction_constraints,\n list(c(0, 1))\n )\n res <- process.x.and.col.args(iris, NULL, list(c(1, 2), c(3, 4)), NULL, NULL, FALSE)\n expect_equal(\n res$params$interaction_constraints,\n list(c(0, 1), c(2, 3))\n )\n res <- process.x.and.col.args(\n iris, NULL, list(c(\"Sepal.Length\", \"Sepal.Width\")), NULL, NULL, FALSE\n )\n expect_equal(\n res$params$interaction_constraints,\n list(c(0, 1))\n )\n res <- process.x.and.col.args(\n as.matrix(iris),\n NULL,\n list(c(\"Sepal.Length\", \"Sepal.Width\")),\n NULL,\n NULL,\n FALSE\n )\n expect_equal(\n res$params$interaction_constraints,\n list(c(0, 1))\n )\n res <- process.x.and.col.args(\n iris,\n NULL,\n list(c(\"Sepal.Width\", \"Petal.Length\"), c(\"Sepal.Length\", \"Petal.Width\", \"Species\")),\n NULL,\n NULL,\n FALSE\n )\n expect_equal(\n res$params$interaction_constraints,\n list(c(1, 2), c(0, 3, 4))\n )\n\n expect_error({\n process.x.and.col.args(iris, NULL, list(c(1L, 20L)), NULL, NULL, FALSE)\n })\n expect_error({\n process.x.and.col.args(iris, NULL, list(c(0L, 2L)), NULL, NULL, FALSE)\n })\n expect_error({\n process.x.and.col.args(iris, NULL, list(c(\"1\", \"2\")), NULL, NULL, FALSE)\n })\n expect_error({\n process.x.and.col.args(iris, NULL, list(c(\"Sepal\", \"Petal\")), NULL, NULL, FALSE)\n })\n expect_error({\n process.x.and.col.args(iris, NULL, c(1L, 2L), NULL, NULL, FALSE)\n })\n expect_error({\n process.x.and.col.args(iris, NULL, matrix(c(1L, 2L)), NULL, NULL, FALSE)\n })\n expect_error({\n process.x.and.col.args(iris, NULL, list(c(1, 2.5)), NULL, NULL, FALSE)\n })\n})\n\ntest_that(\"Sparse matrices are casted to CSR for QDM\", {\n data(agaricus.test, package = \"xgboost\")\n x <- agaricus.test$data\n for (x_in in list(x, methods::as(x, \"TsparseMatrix\"))) {\n res <- process.x.and.col.args(\n x_in,\n NULL,\n NULL,\n NULL,\n NULL,\n TRUE\n )\n expect_s4_class(res$dmatrix_args$data, \"dgRMatrix\")\n }\n})\n\ntest_that(\"Process feature_weights\", {\n data(iris)\n w_vector <- seq(1, 5)\n res <- process.x.and.col.args(\n iris,\n monotone_constraints = NULL,\n interaction_constraints = NULL,\n feature_weights = w_vector,\n lst_args = list(),\n use_qdm = FALSE\n )\n expect_equal(\n res$dmatrix_args$feature_weights,\n seq(1, 5)\n )\n\n w_named_vector <- seq(1, 5)\n names(w_named_vector) <- rev(names(iris))\n res <- process.x.and.col.args(\n iris,\n monotone_constraints = NULL,\n interaction_constraints = NULL,\n feature_weights = w_named_vector,\n lst_args = list(),\n use_qdm = FALSE\n )\n expect_equal(\n res$dmatrix_args$feature_weights,\n rev(seq(1, 5))\n )\n\n w_list <- list(\n Species = 5,\n Sepal.Length = 1,\n Sepal.Width = 2,\n Petal.Length = 3,\n Petal.Width = 4\n )\n res <- process.x.and.col.args(\n iris,\n monotone_constraints = NULL,\n interaction_constraints = NULL,\n feature_weights = w_list,\n lst_args = list(),\n use_qdm = FALSE\n )\n expect_equal(\n res$dmatrix_args$feature_weights,\n seq(1, 5)\n )\n})\n\ntest_that(\"Whole function works\", {\n data(cancer, package = \"survival\")\n y <- Surv(cancer$time, cancer$status - 1, type = \"right\")\n x <- as.data.table(cancer)[, -c(\"time\", \"status\")]\n model <- xgboost(\n x,\n y,\n monotone_constraints = list(age = -1),\n nthreads = 1L,\n nrounds = 5L,\n learning_rate = 3\n )\n expect_equal(\n attributes(model)$params$objective,\n \"survival:aft\"\n )\n expect_equal(\n attributes(model)$metadata$n_targets,\n 1L\n )\n expect_equal(\n attributes(model)$params$monotone_constraints,\n \"(0,-1,0,0,0,0,0,0)\"\n )\n expect_false(\n \"interaction_constraints\" %in% names(attributes(model)$params)\n )\n expect_equal(\n attributes(model)$params$learning_rate,\n 3\n )\n txt <- capture.output({\n print(model)\n })\n expect_true(any(grepl(\"Objective: survival:aft\", txt, fixed = TRUE)))\n expect_true(any(grepl(\"monotone_constraints\", txt, fixed = TRUE)))\n expect_true(any(grepl(\"Number of iterations: 5\", txt, fixed = TRUE)))\n expect_true(any(grepl(\"Number of features: 8\", txt, fixed = TRUE)))\n})\n\ntest_that(\"Print shows expectile metadata\", {\n y <- mtcars$mpg\n x <- mtcars[, -1L]\n model <- xgboost(\n x,\n y,\n nthreads = 1L,\n nrounds = 2L,\n max_depth = 2L,\n objective = \"reg:expectileerror\",\n expectile_alpha = c(0.25, 0.5)\n )\n txt <- capture.output({\n print(model)\n })\n expect_true(any(grepl(\"Prediction expectile\", txt, fixed = TRUE)))\n})\n\ntest_that(\"Can predict probabilities and raw scores\", {\n y <- ToothGrowth$supp\n x <- ToothGrowth[, -2L]\n model <- xgboost(x, y, nthreads = 1L, nrounds = 3L, max_depth = 2L)\n pred_prob <- predict(model, x, type = \"response\")\n pred_raw <- predict(model, x, type = \"raw\")\n expect_true(is.vector(pred_prob))\n expect_equal(length(pred_prob), nrow(x))\n expect_true(min(pred_prob) >= 0)\n expect_true(max(pred_prob) <= 1)\n\n expect_equal(length(pred_raw), nrow(x))\n expect_true(is.vector(pred_raw))\n expect_true(min(pred_raw) < 0)\n expect_true(max(pred_raw) > 0)\n\n expect_equal(\n pred_prob,\n 1 / (1 + exp(-pred_raw)),\n tolerance = 1e-6\n )\n})\n\ntest_that(\"Can predict class\", {\n y <- iris$Species\n x <- iris[, -5L]\n model <- xgboost(x, y, nthreads = 1L, nrounds = 3L, max_depth = 2L)\n pred_class <- predict(model, x, type = \"class\")\n expect_true(is.factor(pred_class))\n expect_equal(levels(pred_class), levels(y))\n\n y <- ToothGrowth$supp\n x <- ToothGrowth[, -2L]\n model <- xgboost(x, y, nthreads = 1L, nrounds = 3L, max_depth = 2L)\n pred_class <- predict(model, x, type = \"class\")\n expect_true(is.factor(pred_class))\n expect_equal(levels(pred_class), levels(y))\n\n probs <- predict(model, x, type = \"response\")\n expect_true(all(pred_class[probs >= 0.5] == levels(y)[[2L]]))\n expect_true(all(pred_class[probs < 0.5] == levels(y)[[1L]]))\n\n # Check that it fails for regression models\n y <- mtcars$mpg\n x <- mtcars[, -1L]\n model <- xgboost(x, y, nthreads = 1L, nrounds = 3L, max_depth = 2L)\n expect_error({\n predict(model, x, type = \"class\")\n })\n})\n\ntest_that(\"Metadata survives serialization\", {\n y <- iris$Species\n x <- iris[, -5L]\n model_fresh <- xgboost(x, y, nthreads = 1L, nrounds = 3L, max_depth = 2L)\n temp_file <- file.path(tempdir(), \"xgb_model.Rds\")\n saveRDS(model_fresh, temp_file)\n model <- readRDS(temp_file)\n pred_class <- predict(model, x, type = \"class\")\n expect_true(is.factor(pred_class))\n expect_equal(levels(pred_class), levels(y))\n})\n\ntest_that(\"Column names aren't added when not appropriate\", {\n pred_types <- c(\n \"response\",\n \"raw\",\n \"leaf\"\n )\n for (pred_type in pred_types) {\n y <- mtcars$mpg\n x <- mtcars[, -1L]\n model <- xgboost(\n x,\n y,\n nthreads = 1L,\n nrounds = 3L,\n max_depth = 2L,\n objective = \"reg:quantileerror\",\n quantile_alpha = 0.5\n )\n pred <- predict(model, x, type = pred_type)\n if (pred_type %in% c(\"raw\", \"response\")) {\n expect_true(is.vector(pred))\n } else {\n expect_true(length(dim(pred)) >= 2L)\n expect_null(colnames(pred))\n }\n\n y <- ToothGrowth$supp\n x <- ToothGrowth[, -2L]\n model <- xgboost(x, y, nthreads = 1L, nrounds = 3L, max_depth = 2L)\n pred <- predict(model, x, type = pred_type)\n if (pred_type %in% c(\"raw\", \"response\")) {\n expect_true(is.vector(pred))\n } else {\n expect_true(length(dim(pred)) >= 2L)\n expect_null(colnames(pred))\n }\n }\n})\n\ntest_that(\"Column names from multiclass are added to non-class predictions\", {\n y <- iris$Species\n x <- iris[, -5L]\n model <- xgboost(x, y, nthreads = 1L, nrounds = 3L, max_depth = 2L)\n\n pred_types_with_colnames <- c(\n \"response\",\n \"raw\",\n \"contrib\",\n \"interaction\"\n )\n\n for (pred_type in pred_types_with_colnames) {\n pred <- predict(model, x, type = pred_type)\n expect_equal(nrow(pred), nrow(x))\n expect_equal(ncol(pred), 3L)\n expect_equal(colnames(pred), levels(y))\n }\n})\n\ntest_that(\"Column names from multitarget are added to predictions\", {\n y <- data.frame(\n ylog = log(mtcars$mpg),\n ysqrt = sqrt(mtcars$mpg)\n )\n x <- mtcars[, -1L]\n model <- xgboost(x, y, nthreads = 1L, nrounds = 3L, max_depth = 2L)\n\n pred_types_with_colnames <- c(\n \"response\",\n \"raw\",\n \"contrib\",\n \"interaction\"\n )\n\n for (pred_type in pred_types_with_colnames) {\n pred <- predict(model, x, type = pred_type)\n expect_equal(nrow(pred), nrow(x))\n expect_equal(ncol(pred), 2L)\n expect_equal(colnames(pred), colnames(y))\n }\n})\n\ntest_that(\"Column names from multiquantile are added to predictions\", {\n y <- mtcars$mpg\n x <- mtcars[, -1L]\n model <- xgboost(\n x,\n y,\n nthreads = 1L,\n nrounds = 3L,\n max_depth = 2L,\n objective = \"reg:quantileerror\",\n quantile_alpha = c(0.25, 0.5, 0.75)\n )\n\n pred_types_with_colnames <- c(\n \"response\",\n \"raw\",\n \"contrib\",\n \"interaction\"\n )\n\n for (pred_type in pred_types_with_colnames) {\n pred <- predict(model, x, type = pred_type)\n expect_equal(nrow(pred), nrow(x))\n expect_equal(ncol(pred), 3L)\n expect_equal(colnames(pred), c(\"q0.25\", \"q0.5\", \"q0.75\"))\n }\n})\n\ntest_that(\"Column names from multiexpectile are added to predictions\", {\n y <- mtcars$mpg\n x <- mtcars[, -1L]\n model <- xgboost(\n x,\n y,\n nthreads = 1L,\n nrounds = 3L,\n max_depth = 2L,\n objective = \"reg:expectileerror\",\n expectile_alpha = c(0.25, 0.5, 0.75)\n )\n\n pred_types_with_colnames <- c(\n \"response\",\n \"raw\",\n \"contrib\",\n \"interaction\"\n )\n\n for (pred_type in pred_types_with_colnames) {\n pred <- predict(model, x, type = pred_type)\n expect_equal(nrow(pred), nrow(x))\n expect_equal(ncol(pred), 3L)\n expect_equal(colnames(pred), c(\"e0.25\", \"e0.5\", \"e0.75\"))\n }\n})\n\ntest_that(\"Leaf predictions have multiple dimensions when needed\", {\n # single score, multiple trees\n y <- mtcars$mpg\n x <- mtcars[, -1L]\n model <- xgboost(\n x,\n y,\n nthreads = 1L,\n nrounds = 4L,\n max_depth = 2L,\n objective = \"reg:quantileerror\",\n quantile_alpha = 0.5\n )\n pred <- predict(model, x, type = \"leaf\")\n expect_equal(dim(pred), c(nrow(x), 4L))\n expect_equal(row.names(pred), row.names(x))\n expect_null(colnames(pred))\n\n # single score, single tree\n model <- xgboost(\n x,\n y,\n nthreads = 1L,\n nrounds = 1L,\n max_depth = 2L,\n objective = \"reg:quantileerror\",\n quantile_alpha = 0.5\n )\n pred <- predict(model, x, type = \"leaf\")\n expect_equal(dim(pred), c(nrow(x), 1L))\n expect_equal(row.names(pred), row.names(x))\n expect_null(colnames(pred))\n\n # multiple score, multiple trees\n model <- xgboost(\n x,\n y,\n nthreads = 1L,\n nrounds = 4L,\n max_depth = 2L,\n objective = \"reg:quantileerror\",\n quantile_alpha = c(0.25, 0.5, 0.75)\n )\n pred <- predict(model, x, type = \"leaf\")\n expect_equal(dim(pred), c(nrow(x), 4L, 3L))\n expect_equal(row.names(pred), row.names(x))\n expect_null(colnames(pred))\n expect_equal(dimnames(pred)[[3L]], c(\"q0.25\", \"q0.5\", \"q0.75\"))\n\n # multiple score, single tree\n model <- xgboost(\n x,\n y,\n nthreads = 1L,\n nrounds = 1L,\n max_depth = 2L,\n objective = \"reg:quantileerror\",\n quantile_alpha = c(0.25, 0.5, 0.75)\n )\n pred <- predict(model, x, type = \"leaf\")\n expect_equal(dim(pred), c(nrow(x), 1L, 3L))\n expect_equal(row.names(pred), row.names(x))\n expect_null(colnames(pred))\n expect_equal(dimnames(pred)[[3L]], c(\"q0.25\", \"q0.5\", \"q0.75\"))\n\n # parallel trees, single tree, single score\n model <- xgboost(\n x,\n y,\n nthreads = 1L,\n nrounds = 1L,\n max_depth = 2L,\n objective = \"count:poisson\",\n num_parallel_tree = 2L\n )\n pred <- predict(model, x, type = \"leaf\")\n expect_equal(dim(pred), c(nrow(x), 1L, 2L))\n expect_equal(row.names(pred), row.names(x))\n expect_null(colnames(pred))\n expect_null(dimnames(pred)[[3L]])\n\n # num_parallel_tree>1 + multiple scores is not supported at the moment so no test for it.\n})\n\ntest_that(\"Column names from multiclass are added to leaf predictions\", {\n y <- iris$Species\n x <- iris[, -5L]\n model <- xgboost(x, y, nthreads = 1L, nrounds = 4L, max_depth = 2L)\n pred <- predict(model, x, type = \"leaf\")\n expect_equal(dim(pred), c(nrow(x), 4L, 3L))\n expect_equal(dimnames(pred)[[3L]], levels(y))\n\n # Check also for a single tree\n model <- xgboost(x, y, nthreads = 1L, nrounds = 1L, max_depth = 2L)\n pred <- predict(model, x, type = \"leaf\")\n expect_equal(dim(pred), c(nrow(x), 1L, 3L))\n expect_equal(dimnames(pred)[[3L]], levels(y))\n})\n\ntest_that(\"Column names from multitarget are added to leaf predictions\", {\n y <- data.frame(\n ylog = log(mtcars$mpg),\n ysqrt = sqrt(mtcars$mpg)\n )\n x <- mtcars[, -1L]\n model <- xgboost(x, y, nthreads = 1L, nrounds = 4L, max_depth = 2L)\n pred <- predict(model, x, type = \"leaf\")\n expect_equal(dim(pred), c(nrow(x), 4L, 2L))\n expect_equal(dimnames(pred)[[3L]], colnames(y))\n\n # Check also for a single tree\n model <- xgboost(x, y, nthreads = 1L, nrounds = 1L, max_depth = 2L)\n pred <- predict(model, x, type = \"leaf\")\n expect_equal(dim(pred), c(nrow(x), 1L, 2L))\n expect_equal(dimnames(pred)[[3L]], colnames(y))\n})\n\ntest_that(\"Column names from multiquantile are added to leaf predictions\", {\n y <- mtcars$mpg\n x <- mtcars[, -1L]\n model <- xgboost(\n x,\n y,\n nthreads = 1L,\n nrounds = 4L,\n max_depth = 2L,\n objective = \"reg:quantileerror\",\n quantile_alpha = c(0.25, 0.5, 0.75)\n )\n pred <- predict(model, x, type = \"leaf\")\n expect_equal(dim(pred), c(nrow(x), 4L, 3L))\n expect_equal(dimnames(pred)[[3L]], c(\"q0.25\", \"q0.5\", \"q0.75\"))\n\n # Check also for a single tree\n model <- xgboost(\n x,\n y,\n nthreads = 1L,\n nrounds = 1L,\n max_depth = 2L,\n objective = \"reg:quantileerror\",\n quantile_alpha = c(0.25, 0.5, 0.75)\n )\n pred <- predict(model, x, type = \"leaf\")\n expect_equal(dim(pred), c(nrow(x), 1L, 3L))\n expect_equal(dimnames(pred)[[3L]], c(\"q0.25\", \"q0.5\", \"q0.75\"))\n})\n\ntest_that(\"Column names from multiexpectile are added to leaf predictions\", {\n y <- mtcars$mpg\n x <- mtcars[, -1L]\n model <- xgboost(\n x,\n y,\n nthreads = 1L,\n nrounds = 4L,\n max_depth = 2L,\n objective = \"reg:expectileerror\",\n expectile_alpha = c(0.25, 0.5, 0.75)\n )\n pred <- predict(model, x, type = \"leaf\")\n expect_equal(dim(pred), c(nrow(x), 4L, 3L))\n expect_equal(dimnames(pred)[[3L]], c(\"e0.25\", \"e0.5\", \"e0.75\"))\n\n # Check also for a single tree\n model <- xgboost(\n x,\n y,\n nthreads = 1L,\n nrounds = 1L,\n max_depth = 2L,\n objective = \"reg:expectileerror\",\n expectile_alpha = c(0.25, 0.5, 0.75)\n )\n pred <- predict(model, x, type = \"leaf\")\n expect_equal(dim(pred), c(nrow(x), 1L, 3L))\n expect_equal(dimnames(pred)[[3L]], c(\"e0.25\", \"e0.5\", \"e0.75\"))\n})\n\ntest_that(\"Evaluation fraction leaves examples of all classes for training\", {\n # With minimal sample leave no remainder\n lst_args <- list(\n dmatrix_args = list(\n data = matrix(seq(1, 4), ncol = 1L),\n label = c(0, 0, 1, 1)\n ),\n metadata = list(\n y_levels = c(\"a\", \"b\")\n ),\n params = list(\n seed = 123\n )\n )\n for (retry in seq_len(10)) {\n lst_args$params$seed <- retry\n res <- process.eval.set(0.5, lst_args)\n expect_equal(length(intersect(res$idx_train, res$idx_eval)), 0)\n expect_equal(length(res$idx_train), 2L)\n expect_equal(length(res$idx_eval), 2L)\n expect_true(length(intersect(c(1L, 2L), res$idx_train)) >= 1L)\n expect_true(length(intersect(c(3L, 4L), res$idx_train)) >= 1L)\n }\n\n # With minimal sample leaving some remainder\n lst_args <- list(\n dmatrix_args = list(\n data = matrix(seq(1, 5), ncol = 1L),\n label = c(0, 0, 1, 1, 1)\n ),\n metadata = list(\n y_levels = c(\"a\", \"b\")\n ),\n params = list(\n seed = 123\n )\n )\n for (retry in seq_len(20)) {\n lst_args$params$seed <- retry\n res <- process.eval.set(0.4, lst_args)\n expect_equal(length(intersect(res$idx_train, res$idx_eval)), 0)\n expect_equal(length(res$idx_train), 3L)\n expect_equal(length(res$idx_eval), 2L)\n expect_true(length(intersect(c(1L, 2L), res$idx_train)) >= 1L)\n expect_true(length(intersect(c(3L, 4L, 5L), res$idx_train)) >= 1L)\n }\n})\n\ntest_that(\"'eval_set' as fraction works\", {\n y <- iris$Species\n x <- iris[, -5L]\n model <- xgboost(\n x,\n y,\n base_margin = matrix(0.1, nrow = nrow(x), ncol = 3L),\n eval_set = 0.2,\n nthreads = 1L,\n nrounds = 4L,\n max_depth = 2L,\n verbosity = 0L\n )\n expect_true(hasName(attributes(model), \"evaluation_log\"))\n evaluation_log <- attributes(model)$evaluation_log\n expect_equal(nrow(evaluation_log), 4L)\n expect_true(hasName(evaluation_log, \"eval_mlogloss\"))\n expect_equal(length(attributes(model)$metadata$y_levels), 3L)\n})\n\ntest_that(\"Linear booster importance uses class names\", {\n y <- iris$Species\n x <- iris[, -5L]\n model <- xgboost(\n x,\n y,\n nthreads = 1L,\n nrounds = 4L,\n verbosity = 0L,\n booster = \"gblinear\",\n learning_rate = 0.2\n )\n imp <- xgb.importance(model)\n expect_true(is.factor(imp$Class))\n expect_equal(levels(imp$Class), levels(y))\n})\n" }, { "path": "R-package/tests/testthat.R", "content": "library(testthat)\nlibrary(xgboost)\nlibrary(Matrix)\n\ntest_check(\"xgboost\", reporter = ProgressReporter)\nRhpcBLASctl::omp_set_num_threads(1)\n" }, { "path": "R-package/vignettes/xgboost_introduction.Rmd", "content": "---\ntitle: \"XGBoost for R introduction\"\nvignette: >\n %\\VignetteIndexEntry{XGBoost for R introduction}\n %\\VignetteEncoding{UTF-8}\n %\\VignetteEngine{knitr::rmarkdown}\noutput:\n html_document:\n theme: \"spacelab\"\n highlight: \"kate\"\n toc: true\n toc_float: true\n---\n\nXGBoost for R introduction\n==========================\n\n## Introduction\n\n**XGBoost** is an optimized distributed gradient boosting library designed to be highly **efficient**, **flexible** and **portable**. It implements machine learning algorithms under the `gradient boosting` framework. XGBoost provides a parallel tree boosting (also known as GBDT, GBM) that solve many data science problems in a fast and accurate way. The same code runs on major distributed environment (Hadoop, SGE, MPI) and can solve problems beyond billions of examples.\n\nFor an introduction to the concept of gradient boosting, see the tutorial [Introduction to Boosted Trees](https://xgboost.readthedocs.io/en/stable/tutorials/model.html) in XGBoost's online docs.\n\nFor more details about XGBoost's features and usage, see the [online documentation](https://xgboost.readthedocs.io/en/stable/) which contains more tutorials, examples, and details.\n\nThis short vignette outlines the basic usage of the R interface for XGBoost, assuming the reader has some familiarity with the underlying concepts behind statistical modeling with gradient-boosted decision trees.\n\n## Building a predictive model\n\nAt its core, XGBoost consists of a C++ library which offers bindings for different programming languages, including R. The R package for XGBoost provides an idiomatic interface similar to those of other statistical modeling packages using and x/y design, as well as a lower-level interface that interacts more directly with the underlying core library and which is similar to those of other language bindings like Python, plus various helpers to interact with its model objects such as by plotting their feature importances or converting them to other formats.\n\nThe main function of interest is `xgboost(x, y, ...)`, which calls the XGBoost model building procedure on observed data of covariates/features/predictors \"x\", and a response variable \"y\" - it should feel familiar to users of packages like `glmnet` or `ncvreg`:\n\n```{r}\nlibrary(xgboost)\ndata(ToothGrowth)\n\ny <- ToothGrowth$supp # the response which we want to model/predict\nx <- ToothGrowth[, c(\"len\", \"dose\")] # the features from which we want to predict it\nmodel <- xgboost(x, y, nthreads = 1, nrounds = 2)\nmodel\n```\n\nIn this case, the \"y\" response variable that was supplied is a \"factor\" type with two classes (\"OJ\" and \"VC\") - hence, XGBoost builds a binary classification model for it based on the features \"x\", by finding a maximum likelihood estimate (similar to the `family=\"binomial\"` model from R's `glm` function) through rule buckets obtained from the sum of two decision trees (from `nrounds=2`), from which we can then predict probabilities, log-odds, class with highest likelihood, among others:\n\n```{r}\npredict(model, x[1:6, ], type = \"response\") # probabilities for y's last level (\"VC\")\npredict(model, x[1:6, ], type = \"raw\") # log-odds\npredict(model, x[1:6, ], type = \"class\") # class with highest probability\n```\n\nCompared to R's `glm` function which follows the concepts of \"families\" and \"links\" from GLM theory to fit models for different kinds of response distributions, XGBoost follows the simpler concept of \"objectives\" which mix both of them into one, and which just like `glm`, allow modeling very different kinds of response distributions (e.g. discrete choices, real-valued numbers, counts, censored measurements, etc.) through a common framework.\n\nXGBoost will automatically determine a suitable objective for the response given its object class (can pass factors for classification, numeric vectors for regression, `Surv` objects from the `survival` package for survival, etc. - see `?xgboost` for more details), but this can be controlled manually through an `objective` parameter based the kind of model that is desired:\n\n```{r}\ndata(mtcars)\n\ny <- mtcars$mpg\nx <- mtcars[, -1]\nmodel_gaussian <- xgboost(x, y, nthreads = 1, nrounds = 2) # default is squared loss (Gaussian)\nmodel_poisson <- xgboost(x, y, objective = \"count:poisson\", nthreads = 1, nrounds = 2)\nmodel_abserr <- xgboost(x, y, objective = \"reg:absoluteerror\", nthreads = 1, nrounds = 2)\n```\n\n_Note: the objective must match with the type of the \"y\" response variable - for example, classification objectives for discrete choices require \"factor\" types, while regression models for real-valued data require \"numeric\" types._\n\n## Model parameters\n\nXGBoost models allow a large degree of control over how they are built. By their nature, gradient-boosted decision tree ensembles are able to capture very complex patterns between features in the data and a response variable, which also means they can suffer from overfitting if not controlled appropirately.\n\nFor best results, one needs to find suitable parameters for the data being modeled. Note that XGBoost does not adjust its default hyperparameters based on the data, and different datasets will require vastly different hyperparameters for optimal predictive performance.\n\nFor example, for a small dataset like \"TootGrowth\" which has only two features and 60 observations, the defaults from XGBoost are an overkill which lead to severe overfitting - for such data, one might want to have smaller trees (i.e. more convervative decision rules, capturing simpler patterns) and fewer of them, for example.\n\nParameters can be controlled by passing additional arguments to `xgboost()`. See `?xgb.params` for details about what parameters are available to control.\n\n```{r}\ny <- ToothGrowth$supp\nx <- ToothGrowth[, c(\"len\", \"dose\")]\nmodel_conservative <- xgboost(\n x, y, nthreads = 1,\n nrounds = 5,\n max_depth = 2,\n reg_lambda = 0.5,\n learning_rate = 0.15\n)\npred_conservative <- predict(\n model_conservative,\n x\n)\npred_conservative[1:6] # probabilities are all closer to 0.5 now\n```\n\nXGBoost also allows the possibility of calculating evaluation metrics for model quality over boosting rounds, with a wide variety of built-in metrics available to use. It's possible to automatically set aside a fraction of the data to use as evaluation set, from which one can then visually monitor progress and overfitting:\n\n```{r}\nxgboost(\n x, y, nthreads = 1,\n eval_set = 0.2,\n monitor_training = TRUE,\n verbosity = 1,\n eval_metric = c(\"auc\", \"logloss\"),\n nrounds = 5,\n max_depth = 2,\n reg_lambda = 0.5,\n learning_rate = 0.15\n)\n```\n\n## Examining model objects\n\nXGBoost model objects for the most part consist of a pointer to a C++ object where most of the information is held and which is interfaced through the utility functions and methods in the package, but also contains some R attributes that can be retrieved (and new ones added) through `attributes()`:\n\n```{r}\nattributes(model)\n```\n\nIn addition to R attributes (which can be arbitrary R objects), it may also keep some standardized C-level attributes that one can access and modify (but which can only be JSON-format):\n\n```{r}\nxgb.attributes(model)\n```\n\n(they are empty for this model)\n\n... but usually, when it comes to getting something out of a model object, one would typically want to do this through the built-in utility functions. Some examples:\n\n```{r}\nxgb.importance(model)\n```\n\n```{r}\nxgb.model.dt.tree(model)\n```\n\n## Other features\n\nXGBoost supports many additional features on top of its traditional gradient-boosting framework, including, among others:\n\n* Building decision tree models with characteristics such as per-feature monotonicity constraints or interaction constraints.\n* Calculating feature contributions in individual predictions.\n* Using custom objectives and custom evaluation metrics.\n* Fitting linear models.\n* Fitting models on GPUs and/or on data that doesn't fit in RAM (\"external memory\").\n\nSee the [online documentation](https://xgboost.readthedocs.io/en/stable/index.html) - particularly the [tutorials section](https://xgboost.readthedocs.io/en/stable/tutorials/index.html) - for a glimpse over further functionalities that XGBoost offers.\n\n## The low-level interface\n\nIn addition to the `xgboost(x, y, ...)` function, XGBoost also provides a lower-level interface for creating model objects through the function `xgb.train()`, which resembles the same `xgb.train` functions in other language bindings of XGBoost.\n\nThis `xgb.train()` interface exposes additional functionalities (such as user-supplied callbacks or external-memory data support) and performs fewer data validations and castings compared to the `xgboost()` function interface.\n\nSome key differences between the two interfaces:\n\n* Unlike `xgboost()` which takes R objects such as `matrix` or `data.frame` as inputs, the function `xgb.train()` uses XGBoost's own data container called \"DMatrix\", which can be created from R objects through the function `xgb.DMatrix()`. Note that there are other \"DMatrix\" constructors too, such as \"xgb.QuantileDMatrix()\", which might be more beneficial for some use-cases.\n* A \"DMatrix\" object may contain a mixture of features/covariates, the response variable, observation weights, base margins, among others; and unlike `xgboost()`, requires its inputs to have already been encoded into the representation that XGBoost uses behind the scenes - for example, while `xgboost()` may take a `factor` object as \"y\", `xgb.DMatrix()` requires instead a binary response variable to be passed as a vector of zeros and ones.\n* Hyperparameters are passed as function arguments in `xgboost()`, while they are passed as a named list to `xgb.train()`.\n* The `xgb.train()` interface keeps less metadata about its inputs - for example, it will not add levels of factors as column names to estimated probabilities when calling `predict`.\n\nExample usage of `xgb.train()`:\n\n```{r}\ndata(\"agaricus.train\")\ndmatrix <- xgb.DMatrix(\n data = agaricus.train$data, # a sparse CSC matrix ('dgCMatrix')\n label = agaricus.train$label, # zeros and ones\n nthread = 1\n)\nbooster <- xgb.train(\n data = dmatrix,\n nrounds = 10,\n params = xgb.params(\n objective = \"binary:logistic\",\n nthread = 1,\n max_depth = 3\n )\n)\n\ndata(\"agaricus.test\")\ndmatrix_test <- xgb.DMatrix(agaricus.test$data, nthread = 1)\npred_prob <- predict(booster, dmatrix_test)\npred_raw <- predict(booster, dmatrix_test, outputmargin = TRUE)\n```\n\nModel objects produced by `xgb.train()` have class `xgb.Booster`, while model objects produced by `xgboost()` have class `xgboost`, which is a subclass of `xgb.Booster`. Their `predict` methods also take different arguments - for example, `predict.xgboost` has a `type` parameter, while `predict.xgb.Booster` controls this through binary arguments - but as `xgboost` is a subclass of `xgb.Booster`, methods for `xgb.Booster` can be called on `xgboost` objects if needed.\n\nUtility functions in the XGBoost R package will work with both model classes - for example:\n\n```{r}\nxgb.importance(model)\nxgb.importance(booster)\n```\n\nWhile `xgboost()` aims to provide a user-friendly interface, there are still many situations where one should prefer the `xgb.train()` interface - for example:\n\n* For latency-sensitive applications (e.g. when serving models in real time), `xgb.train()` will have a speed advantage, as it performs fewer validations, conversions, and post-processings with metadata.\n* If you are developing an R package that depends on XGBoost, `xgb.train()` will provide a more stable interface (less subject to changes) and will have lower time/memory overhead.\n* If you need functionalities that are not exposed by the `xgboost()` interface - for example, if your dataset does not fit into the computer's RAM, it's still possible to construct a DMatrix from it if the data is loaded in batches through `xgb.ExtMemDMatrix()`.\n" }, { "path": "R-package/vignettes/xgboostfromJSON.Rmd", "content": "---\ntitle: \"XGBoost from JSON\"\noutput:\n rmarkdown::html_vignette:\n number_sections: yes\n toc: yes\nauthor: Roland Stevenson\nvignette: >\n %\\VignetteIndexEntry{XGBoost from JSON}\n %\\VignetteEngine{knitr::rmarkdown}\n \\usepackage[utf8]{inputenc}\n---\n\nXGBoost from JSON\n=================\n\n## Introduction\n\nThe purpose of this Vignette is to show you how to correctly load and work with an **XGBoost** model that has been dumped to JSON. **XGBoost** internally converts all data to [32-bit floats](https://en.wikipedia.org/wiki/Single-precision_floating-point_format), and the values dumped to JSON are decimal representations of these values. When working with a model that has been parsed from a JSON file, care must be taken to correctly treat:\n\n- the input data, which should be converted to 32-bit floats\n- any 32-bit floats that were stored in JSON as decimal representations\n- any calculations must be done with 32-bit mathematical operators\n\n## Setup\n\nFor the purpose of this tutorial we will load the xgboost, jsonlite, and float packages. We'll also set `digits=22` in our options in case we want to inspect many digits of our results.\n\n```{r}\nrequire(xgboost)\nrequire(jsonlite)\nrequire(float)\noptions(digits = 22)\n```\n\nWe will create a toy binary logistic model based on the example first provided [here](https://github.com/dmlc/xgboost/issues/3960), so that we can easily understand the structure of the dumped JSON model object. This will allow us to understand where discrepancies can occur and how they should be handled.\n\n```{r}\ndates <- c(20180130, 20180130, 20180130,\n 20180130, 20180130, 20180130,\n 20180131, 20180131, 20180131,\n 20180131, 20180131, 20180131,\n 20180131, 20180131, 20180131,\n 20180134, 20180134, 20180134)\n\nlabels <- c(1, 1, 1,\n 1, 1, 1,\n 0, 0, 0,\n 0, 0, 0,\n 0, 0, 0,\n 0, 0, 0)\n\ndata <- data.frame(dates = dates, labels = labels)\n\nbst <- xgb.train(\n data = xgb.DMatrix(as.matrix(data$dates), label = labels, missing = NA, nthread = 1),\n nrounds = 1,\n params = xgb.params(\n objective = \"binary:logistic\",\n nthread = 2,\n max_depth = 1\n )\n)\n```\n\n## Comparing results\nWe will now dump the model to JSON and attempt to illustrate a variety of issues that can arise, and how to properly deal with them.\n\nFirst let's dump the model to JSON:\n\n```{r}\nbst_json <- xgb.dump(bst, with_stats = FALSE, dump_format = 'json')\nbst_from_json <- fromJSON(bst_json, simplifyDataFrame = FALSE)\nnode <- bst_from_json[[1]]\ncat(bst_json)\n```\n\nThe tree JSON shown by the above code-chunk tells us that if the data is less than 20180132, the tree will output the value in the first leaf. Otherwise it will output the value in the second leaf. Let's try to reproduce this manually with the data we have and confirm that it matches the model predictions we've already calculated.\n\n```{r}\nbst_preds_logodds <- predict(bst, as.matrix(data$dates), outputmargin = TRUE)\n\n# calculate the logodds values using the JSON representation\nbst_from_json_logodds <- ifelse(data$dates < node$split_condition,\n node$children[[1]]$leaf,\n node$children[[2]]$leaf)\n\nbst_preds_logodds\nbst_from_json_logodds\n\n# test that values are equal\nbst_preds_logodds == bst_from_json_logodds\n\n```\nNone are equal. What happened?\n\nAt this stage two things happened:\n\n- input data was not converted to 32-bit floats\n- the JSON variables were not converted to 32-bit floats\n\n### Lesson 1: All data is 32-bit floats\n\n> When working with imported JSON, all data must be converted to 32-bit floats\n\nTo explain this, let's repeat the comparison and round to two decimals:\n\n```{r}\nround(bst_preds_logodds, 2) == round(bst_from_json_logodds, 2)\n```\n\nIf we round to two decimals, we see that only the elements related to data values of `20180131` don't agree. If we convert the data to floats, they agree:\n\n```{r}\n# now convert the dates to floats first\nbst_from_json_logodds <- ifelse(fl(data$dates) < node$split_condition,\n node$children[[1]]$leaf,\n node$children[[2]]$leaf)\n\n# test that values are equal\nround(bst_preds_logodds, 2) == round(bst_from_json_logodds, 2)\n```\n\nWhat's the lesson? If we are going to work with an imported JSON model, any data must be converted to floats first. In this case, since '20180131' cannot be represented as a 32-bit float, it is rounded up to 20180132, as shown here:\n\n```{r}\nfl(20180131)\n```\n\n\n### Lesson 2: JSON parameters are 32-bit floats\n\n> All JSON parameters stored as floats must be converted to floats.\n\nLet's now say we do care about numbers past the first two decimals.\n\n```{r}\n# test that values are equal\nbst_preds_logodds == bst_from_json_logodds\n```\n\nNone are exactly equal. What happened? Although we've converted the data to 32-bit floats, we also need to convert the JSON parameters to 32-bit floats. Let's do this:\n\n```{r}\n# now convert the dates to floats first\nbst_from_json_logodds <- ifelse(fl(data$dates) < fl(node$split_condition),\n as.numeric(fl(node$children[[1]]$leaf)),\n as.numeric(fl(node$children[[2]]$leaf)))\n\n# test that values are equal\nbst_preds_logodds == bst_from_json_logodds\n```\nAll equal. What's the lesson? If we are going to work with an imported JSON model, any JSON parameters that were stored as floats must also be converted to floats first.\n\n### Lesson 3: Use 32-bit math\n\n> Always use 32-bit numbers and operators\n\nWe were able to get the log-odds to agree, so now let's manually calculate the sigmoid of the log-odds. This should agree with the xgboost predictions.\n\n\n```{r}\nbst_preds <- predict(bst, as.matrix(data$dates))\n\n# calculate the predictions casting doubles to floats\nbst_from_json_preds <- ifelse(\n fl(data$dates) < fl(node$split_condition)\n , as.numeric(1 / (1 + exp(-1 * fl(node$children[[1]]$leaf))))\n , as.numeric(1 / (1 + exp(-1 * fl(node$children[[2]]$leaf))))\n)\n\n# test that values are equal\nbst_preds == bst_from_json_preds\n```\n\nNone are exactly equal again. What is going on here? Well, since we are using the value `1` in the calculations, we have introduced a double into the calculation. Because of this, all float values are promoted to 64-bit doubles and the 64-bit version of the exponentiation operator `exp` is also used. On the other hand, xgboost uses the 32-bit version of the exponentiation operator in its [sigmoid function](https://github.com/dmlc/xgboost/blob/54980b8959680a0da06a3fc0ec776e47c8cbb0a1/src/common/math.h#L25-L27).\n\nHow do we fix this? We have to ensure we use the correct data types everywhere and the correct operators. If we use only floats, the float library that we have loaded will ensure the 32-bit float exponentiation operator is applied.\n```{r}\n# calculate the predictions casting doubles to floats\nbst_from_json_preds <- ifelse(\n fl(data$dates) < fl(node$split_condition)\n , as.numeric(fl(1) / (fl(1) + exp(fl(-1) * fl(node$children[[1]]$leaf))))\n , as.numeric(fl(1) / (fl(1) + exp(fl(-1) * fl(node$children[[2]]$leaf))))\n)\n\n# test that values are equal\nbst_preds == bst_from_json_preds\n```\n\nAll equal. What's the lesson? We have to ensure that all calculations are done with 32-bit floating point operators if we want to reproduce the results that we see with xgboost.\n" }, { "path": "README.md", "content": " eXtreme Gradient Boosting\n===========\n\n[![XGBoost-CI](https://github.com/dmlc/xgboost/workflows/XGBoost%20CI/badge.svg?branch=master)](https://github.com/dmlc/xgboost/actions)\n[![Documentation Status](https://readthedocs.org/projects/xgboost/badge/?version=latest)](https://xgboost.readthedocs.org)\n[![GitHub license](https://dmlc.github.io/img/apache2.svg)](./LICENSE)\n[![CRAN Status Badge](https://www.r-pkg.org/badges/version/xgboost)](https://cran.r-project.org/web/packages/xgboost)\n[![PyPI version](https://badge.fury.io/py/xgboost.svg)](https://pypi.python.org/pypi/xgboost/)\n[![Conda version](https://img.shields.io/conda/vn/conda-forge/py-xgboost.svg)](https://anaconda.org/conda-forge/py-xgboost)\n[![Optuna](https://img.shields.io/badge/Optuna-integrated-blue)](https://optuna.org)\n[![Twitter](https://img.shields.io/badge/@XGBoostProject--_.svg?style=social&logo=twitter)](https://twitter.com/XGBoostProject)\n[![OpenSSF Scorecard](https://api.securityscorecards.dev/projects/github.com/dmlc/xgboost/badge)](https://api.securityscorecards.dev/projects/github.com/dmlc/xgboost)\n[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/comet-ml/comet-examples/blob/master/integrations/model-training/xgboost/notebooks/how_to_use_comet_with_xgboost_tutorial.ipynb)\n\n[Community](https://xgboost.ai/community) |\n[Documentation](https://xgboost.readthedocs.org) |\n[Resources](demo/README.md) |\n[Contributors](CONTRIBUTORS.md) |\n[Release Notes](https://xgboost.readthedocs.io/en/latest/changes/index.html)\n\nXGBoost is an optimized distributed gradient boosting library designed to be highly ***efficient***, ***flexible*** and ***portable***.\nIt implements machine learning algorithms under the [Gradient Boosting](https://en.wikipedia.org/wiki/Gradient_boosting) framework.\nXGBoost provides a parallel tree boosting (also known as GBDT, GBM) that solve many data science problems in a fast and accurate way.\nThe same code runs on major distributed environment (Kubernetes, Hadoop, SGE, Dask, Spark, PySpark) and can solve problems beyond billions of examples.\n\nLicense\n-------\n© Contributors, 2021. Licensed under an [Apache-2](https://github.com/dmlc/xgboost/blob/master/LICENSE) license.\n\nContribute to XGBoost\n---------------------\nXGBoost has been developed and used by a group of active community members. Your help is very valuable to make the package better for everyone.\nCheckout the [Community Page](https://xgboost.ai/community).\n\nReference\n---------\n- Tianqi Chen and Carlos Guestrin. [XGBoost: A Scalable Tree Boosting System](https://arxiv.org/abs/1603.02754). In 22nd SIGKDD Conference on Knowledge Discovery and Data Mining, 2016\n- XGBoost originates from research project at University of Washington.\n\nSponsors\n--------\nBecome a sponsor and get a logo here. See details at [Sponsoring the XGBoost Project](https://xgboost.ai/sponsors). The funds are used to defray the cost of continuous integration and testing infrastructure (https://xgboost-ci.net).\n\n## Open Source Collective sponsors\n[![Backers on Open Collective](https://opencollective.com/xgboost/backers/badge.svg)](#backers) [![Sponsors on Open Collective](https://opencollective.com/xgboost/sponsors/badge.svg)](#sponsors)\n\n### Sponsors\n[[Become a sponsor](https://opencollective.com/xgboost#sponsor)]\n\n\"NVIDIA\"\n\n\n\n\n\n### Backers\n[[Become a backer](https://opencollective.com/xgboost#backer)]\n\n\n" }, { "path": "SECURITY.md", "content": "# Security Policy\n\n## Supported Versions\n\n\nSecurity updates are applied only to the most recent release.\n\n## Reporting a Vulnerability\n\n\n\nTo report a security issue, please email\n[security@xgboost-ci.net](mailto:security@xgboost-ci.net)\nwith a description of the issue, the steps you took to create the issue,\naffected versions, and, if known, mitigations for the issue.\n\nAll support will be made on the best effort base, so please indicate the \"urgency level\" of the vulnerability as Critical, High, Medium or Low.\n" }, { "path": "amalgamation/dmlc-minimum0.cc", "content": "/*!\n * Copyright 2015 by Contributors.\n * \\brief Mininum DMLC library Amalgamation, used for easy plugin of dmlc lib.\n * Normally this is not needed.\n */\n#include \"../dmlc-core/src/io/line_split.cc\"\n#include \"../dmlc-core/src/io/recordio_split.cc\"\n#include \"../dmlc-core/src/io/input_split_base.cc\"\n#include \"../dmlc-core/src/io/local_filesys.cc\"\n#include \"../dmlc-core/src/io/filesys.cc\"\n#include \"../dmlc-core/src/io/indexed_recordio_split.cc\"\n#include \"../dmlc-core/src/data.cc\"\n#include \"../dmlc-core/src/io.cc\"\n#include \"../dmlc-core/src/recordio.cc\"\n" }, { "path": "cmake/Doc.cmake", "content": "function(run_doxygen)\n find_package(Doxygen REQUIRED)\n\n if(NOT DOXYGEN_DOT_FOUND)\n message(FATAL_ERROR \"Command `dot` not found. Please install graphviz.\")\n endif()\n\n configure_file(\n ${xgboost_SOURCE_DIR}/doc/Doxyfile.in\n ${CMAKE_CURRENT_BINARY_DIR}/Doxyfile @ONLY)\n add_custom_target(\n doc_doxygen ALL\n COMMAND ${DOXYGEN_EXECUTABLE} ${CMAKE_CURRENT_BINARY_DIR}/Doxyfile\n WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}\n COMMENT \"Generate C APIs documentation.\"\n VERBATIM)\nendfunction()\n" }, { "path": "cmake/FindOpenMPMacOS.cmake", "content": "# Find OpenMP library on MacOS\n# Automatically handle locating libomp from the Homebrew package manager\n\n# lint_cmake: -package/consistency\n\nmacro(find_openmp_macos)\n if(NOT APPLE)\n message(FATAL_ERROR \"${CMAKE_CURRENT_FUNCTION}() must only be used on MacOS\")\n endif()\n find_package(OpenMP)\n if(NOT OpenMP_FOUND)\n # Try again with extra path info. This step is required for libomp 15+ from Homebrew,\n # as libomp 15.0+ from brew is keg-only\n # See https://github.com/Homebrew/homebrew-core/issues/112107#issuecomment-1278042927.\n execute_process(COMMAND brew --prefix libomp\n OUTPUT_VARIABLE HOMEBREW_LIBOMP_PREFIX\n OUTPUT_STRIP_TRAILING_WHITESPACE)\n set(OpenMP_C_FLAGS\n \"-Xpreprocessor -fopenmp -I${HOMEBREW_LIBOMP_PREFIX}/include\")\n set(OpenMP_CXX_FLAGS\n \"-Xpreprocessor -fopenmp -I${HOMEBREW_LIBOMP_PREFIX}/include\")\n set(OpenMP_C_LIB_NAMES omp)\n set(OpenMP_CXX_LIB_NAMES omp)\n set(OpenMP_omp_LIBRARY ${HOMEBREW_LIBOMP_PREFIX}/lib/libomp.dylib)\n find_package(OpenMP REQUIRED)\n endif()\nendmacro()\n\n# Patch libxgboost.dylib so that it depends on @rpath/libomp.dylib instead of\n# /opt/homebrew/opt/libomp/lib/libomp.dylib or other hard-coded paths.\n# Doing so enables XGBoost to interoperate with multiple kinds of OpenMP\n# libraries. See https://github.com/lightgbm-org/LightGBM/pull/6391 for detailed\n# explanation. Adapted from https://github.com/lightgbm-org/LightGBM/pull/6391\n# by James Lamb.\n# MacOS only.\nfunction(patch_openmp_path_macos target target_default_output_name)\n if(NOT APPLE)\n message(FATAL_ERROR \"${CMAKE_CURRENT_FUNCTION}() must only be used on MacOS\")\n endif()\n # Get path to libomp found at build time\n get_target_property(\n __OpenMP_LIBRARY_LOCATION\n OpenMP::OpenMP_CXX\n INTERFACE_LINK_LIBRARIES\n )\n # Get the base name of the OpenMP lib\n # Usually: libomp.dylib, libgomp.dylib, or libiomp.dylib\n get_filename_component(\n __OpenMP_LIBRARY_NAME\n ${__OpenMP_LIBRARY_LOCATION}\n NAME\n )\n # Get the directory containing the OpenMP lib\n get_filename_component(\n __OpenMP_LIBRARY_DIR\n ${__OpenMP_LIBRARY_LOCATION}\n DIRECTORY\n )\n # Get the name of the XGBoost lib, e.g. libxgboost\n get_target_property(\n __LIBXGBOOST_OUTPUT_NAME\n ${target}\n OUTPUT_NAME\n )\n if(NOT __LIBXGBOOST_OUTPUT_NAME)\n set(__LIBXGBOOST_OUTPUT_NAME \"${target_default_output_name}\")\n endif()\n\n # Get the file name of the XGBoost lib, e.g. libxgboost.dylib\n if(CMAKE_SHARED_LIBRARY_SUFFIX_CXX)\n set(\n __LIBXGBOOST_FILENAME_${target} \"${__LIBXGBOOST_OUTPUT_NAME}${CMAKE_SHARED_LIBRARY_SUFFIX_CXX}\"\n CACHE INTERNAL \"Shared library filename ${target}\"\n )\n else()\n set(\n __LIBXGBOOST_FILENAME_${target} \"${__LIBXGBOOST_OUTPUT_NAME}.dylib\"\n CACHE INTERNAL \"Shared library filename ${target}\"\n )\n endif()\n\n message(STATUS \"Creating shared lib for target ${target}: ${__LIBXGBOOST_FILENAME_${target}}\")\n\n # Override the absolute path to OpenMP with a relative one using @rpath.\n #\n # This also ensures that if a libomp.dylib has already been loaded, it'll just use that.\n if(KEEP_BUILD_ARTIFACTS_IN_BINARY_DIR)\n set(__LIB_DIR ${xgboost_BINARY_DIR}/lib)\n else()\n set(__LIB_DIR ${xgboost_SOURCE_DIR}/lib)\n endif()\n add_custom_command(\n TARGET ${target}\n POST_BUILD\n COMMAND\n install_name_tool\n -change\n ${__OpenMP_LIBRARY_LOCATION}\n \"@rpath/${__OpenMP_LIBRARY_NAME}\"\n \"${__LIBXGBOOST_FILENAME_${target}}\"\n WORKING_DIRECTORY ${__LIB_DIR}\n )\n message(STATUS\n \"${__LIBXGBOOST_FILENAME_${target}}: \"\n \"Replacing hard-coded OpenMP install_name with '@rpath/${__OpenMP_LIBRARY_NAME}'...\"\n )\n # Add RPATH entries to ensure the loader looks in the following, in the following order:\n #\n # - /opt/homebrew/opt/libomp/lib (where 'brew install' / 'brew link' puts libomp.dylib)\n # - ${__OpenMP_LIBRARY_DIR} (wherever find_package(OpenMP) found OpenMP at build time)\n #\n # Note: This list will only be used if libomp.dylib isn't already loaded into memory.\n # So Conda users will likely use ${CONDA_PREFIX}/libomp.dylib\n execute_process(COMMAND brew --prefix libomp\n OUTPUT_VARIABLE HOMEBREW_LIBOMP_PREFIX\n OUTPUT_STRIP_TRAILING_WHITESPACE)\n set_target_properties(\n ${target}\n PROPERTIES\n BUILD_WITH_INSTALL_RPATH TRUE\n INSTALL_RPATH \"${HOMEBREW_LIBOMP_PREFIX}/lib;${__OpenMP_LIBRARY_DIR}\"\n INSTALL_RPATH_USE_LINK_PATH FALSE\n )\nendfunction()\n" }, { "path": "cmake/PrefetchIntrinsics.cmake", "content": "function(find_prefetch_intrinsics)\n include(CheckCXXSourceCompiles)\n check_cxx_source_compiles(\"\n #include \n int main() {\n char data = 0;\n const char* address = &data;\n _mm_prefetch(address, _MM_HINT_NTA);\n return 0;\n }\n \" XGBOOST_MM_PREFETCH_PRESENT)\n check_cxx_source_compiles(\"\n int main() {\n char data = 0;\n const char* address = &data;\n __builtin_prefetch(address, 0, 0);\n return 0;\n }\n \" XGBOOST_BUILTIN_PREFETCH_PRESENT)\n set(XGBOOST_MM_PREFETCH_PRESENT ${XGBOOST_MM_PREFETCH_PRESENT} PARENT_SCOPE)\n set(XGBOOST_BUILTIN_PREFETCH_PRESENT ${XGBOOST_BUILTIN_PREFETCH_PRESENT} PARENT_SCOPE)\nendfunction()\n" }, { "path": "cmake/RPackageInstall.cmake.in", "content": "# Commands to install the R package as a CMake install target\n\nfunction(check_call)\n set(cmd COMMAND)\n cmake_parse_arguments(\n PARSE_ARGV 0\n CALL_ARG \"\" \"\" \"${cmd}\"\n )\n string(REPLACE \";\" \" \" commands \"${CALL_ARG_COMMAND}\")\n message(\"Command: ${commands}\")\n execute_process(COMMAND ${CALL_ARG_COMMAND}\n OUTPUT_VARIABLE _out\n\t\t\t\t ERROR_VARIABLE _err\n\t\t\t\t RESULT_VARIABLE _res)\n if(NOT \"${_res}\" EQUAL \"0\")\n message(FATAL_ERROR \"out: ${_out}, err: ${_err}, res: ${_res}\")\n endif()\nendfunction()\n\n# Important paths\nset(build_dir \"@build_dir@\")\nset(LIBR_EXECUTABLE \"@LIBR_EXECUTABLE@\")\n\n# Back up cmake_install.cmake\nfile(WRITE \"${build_dir}/R-package/src/Makevars\" \"all:\")\nfile(WRITE \"${build_dir}/R-package/src/Makevars.win\" \"all:\")\n\n# Install dependencies\nset(XGB_DEPS_SCRIPT\n \"deps = setdiff(c('data.table', 'jsonlite', 'Matrix'), rownames(installed.packages())); if(length(deps)>0) install.packages(deps, repo = 'https://cloud.r-project.org/')\")\ncheck_call(COMMAND \"${LIBR_EXECUTABLE}\" -q -e \"${XGB_DEPS_SCRIPT}\")\n\n# Install the XGBoost R package\ncheck_call(COMMAND \"${LIBR_EXECUTABLE}\" CMD INSTALL --no-multiarch --build \"${build_dir}/R-package\")" }, { "path": "cmake/RPackageInstallTargetSetup.cmake", "content": "# Assembles the R-package files in build_dir;\n# if necessary, installs the main R package dependencies;\n# runs R CMD INSTALL.\nfunction(setup_rpackage_install_target rlib_target build_dir)\n configure_file(${PROJECT_SOURCE_DIR}/cmake/RPackageInstall.cmake.in ${PROJECT_BINARY_DIR}/RPackageInstall.cmake @ONLY)\n install(\n DIRECTORY \"${xgboost_SOURCE_DIR}/R-package\"\n DESTINATION \"${build_dir}\"\n PATTERN \"src/*\" EXCLUDE\n PATTERN \"R-package/configure\" EXCLUDE\n )\n install(TARGETS ${rlib_target}\n LIBRARY DESTINATION \"${build_dir}/R-package/src/\"\n RUNTIME DESTINATION \"${build_dir}/R-package/src/\")\n install(SCRIPT ${PROJECT_BINARY_DIR}/RPackageInstall.cmake)\nendfunction()\n" }, { "path": "cmake/Sanitizer.cmake", "content": "# Set appropriate compiler and linker flags for sanitizers.\n#\n# Usage of this module:\n# enable_sanitizers(\"address;leak\")\n\n# Add flags\nmacro(enable_sanitizer sanitizer)\n if(${sanitizer} MATCHES \"address\")\n find_package(ASan)\n set(SAN_COMPILE_FLAGS \"${SAN_COMPILE_FLAGS} -fsanitize=address\")\n\n elseif(${sanitizer} MATCHES \"thread\")\n find_package(TSan)\n set(SAN_COMPILE_FLAGS \"${SAN_COMPILE_FLAGS} -fsanitize=thread\")\n if(TSan_FOUND)\n link_libraries(${TSan_LIBRARY})\n endif()\n\n elseif(${sanitizer} MATCHES \"leak\")\n find_package(LSan)\n set(SAN_COMPILE_FLAGS \"${SAN_COMPILE_FLAGS} -fsanitize=leak\")\n\n elseif(${sanitizer} MATCHES \"undefined\")\n find_package(UBSan)\n set(SAN_COMPILE_FLAGS \"${SAN_COMPILE_FLAGS} -fsanitize=undefined -fno-sanitize-recover=undefined\")\n\n else()\n message(FATAL_ERROR \"Santizer ${sanitizer} not supported.\")\n endif()\nendmacro()\n\nmacro(enable_sanitizers SANITIZERS)\n # Check sanitizers compatibility.\n # Idealy, we should use if(san IN_LIST SANITIZERS) ... endif()\n # But I haven't figure out how to make it work.\n foreach( _san ${SANITIZERS} )\n string(TOLOWER ${_san} _san)\n if(_san MATCHES \"thread\")\n if(${_use_other_sanitizers})\n message(FATAL_ERROR\n \"thread sanitizer is not compatible with ${_san} sanitizer.\")\n endif()\n set(_use_thread_sanitizer 1)\n else()\n if(${_use_thread_sanitizer})\n message(FATAL_ERROR\n \"${_san} sanitizer is not compatible with thread sanitizer.\")\n endif()\n set(_use_other_sanitizers 1)\n endif()\n endforeach()\n\n message(\"Sanitizers: ${SANITIZERS}\")\n\n foreach( _san ${SANITIZERS} )\n string(TOLOWER ${_san} _san)\n enable_sanitizer(${_san})\n endforeach()\n message(\"Sanitizers compile flags: ${SAN_COMPILE_FLAGS}\")\n set(CMAKE_CXX_FLAGS \"${CMAKE_CXX_FLAGS} ${SAN_COMPILE_FLAGS}\")\n set(CMAKE_C_FLAGS \"${CMAKE_C_FLAGS} ${SAN_COMPILE_FLAGS}\")\nendmacro()\n" }, { "path": "cmake/Utils.cmake", "content": "# Automatically set source group based on folder\nfunction(auto_source_group SOURCES)\n\n foreach(FILE ${SOURCES})\n get_filename_component(PARENT_DIR \"${FILE}\" PATH)\n\n # skip src or include and changes /'s to \\\\'s\n string(REPLACE \"${CMAKE_CURRENT_LIST_DIR}\" \"\" GROUP \"${PARENT_DIR}\")\n string(REPLACE \"/\" \"\\\\\\\\\" GROUP \"${GROUP}\")\n string(REGEX REPLACE \"^\\\\\\\\\" \"\" GROUP \"${GROUP}\")\n\n source_group(\"${GROUP}\" FILES \"${FILE}\")\n endforeach()\nendfunction()\n\n# Set output directory of target, ignoring debug or release\nfunction(set_output_directory target dir)\n set_target_properties(${target} PROPERTIES\n RUNTIME_OUTPUT_DIRECTORY ${dir}\n RUNTIME_OUTPUT_DIRECTORY_DEBUG ${dir}\n RUNTIME_OUTPUT_DIRECTORY_RELEASE ${dir}\n RUNTIME_OUTPUT_DIRECTORY_RELWITHDEBINFO ${dir}\n RUNTIME_OUTPUT_DIRECTORY_MINSIZEREL ${dir}\n LIBRARY_OUTPUT_DIRECTORY ${dir}\n LIBRARY_OUTPUT_DIRECTORY_DEBUG ${dir}\n LIBRARY_OUTPUT_DIRECTORY_RELEASE ${dir}\n LIBRARY_OUTPUT_DIRECTORY_RELWITHDEBINFO ${dir}\n LIBRARY_OUTPUT_DIRECTORY_MINSIZEREL ${dir}\n ARCHIVE_OUTPUT_DIRECTORY ${dir}\n ARCHIVE_OUTPUT_DIRECTORY_DEBUG ${dir}\n ARCHIVE_OUTPUT_DIRECTORY_RELEASE ${dir}\n ARCHIVE_OUTPUT_DIRECTORY_RELWITHDEBINFO ${dir}\n ARCHIVE_OUTPUT_DIRECTORY_MINSIZEREL ${dir})\nendfunction()\n\n# Set a default build type to release if none was specified\nfunction(set_default_configuration_release)\n if(CMAKE_CONFIGURATION_TYPES STREQUAL \"Debug;Release;MinSizeRel;RelWithDebInfo\") # multiconfig generator?\n set(CMAKE_CONFIGURATION_TYPES Release CACHE STRING \"\" FORCE)\n elseif(NOT CMAKE_BUILD_TYPE AND NOT CMAKE_CONFIGURATION_TYPES)\n message(STATUS \"Setting build type to 'Release' as none was specified.\")\n set(CMAKE_BUILD_TYPE Release CACHE STRING \"Choose the type of build.\" FORCE)\n endif()\nendfunction()\n\nif(BUILD_WITH_GIT_HASH)\n execute_process(COMMAND git rev-parse --short HEAD\n WORKING_DIRECTORY ${xgboost_SOURCE_DIR}\n OUTPUT_STRIP_TRAILING_WHITESPACE\n OUTPUT_VARIABLE XGBOOST_GIT_HASH\n ERROR_VARIABLE XGBOOST_GIT_ERROR\n RESULT_VARIABLE GIT_COMMAND_RESULT)\n\n if(NOT GIT_COMMAND_RESULT EQUAL 0)\n message(FATAL_ERROR \"Failed to retrieve the git hash:\\n${XGBOOST_GIT_ERROR}\")\n endif()\n message(STATUS \"Git hash: ${XGBOOST_GIT_HASH}\")\nendif()\n\n# Generate CMAKE_CUDA_ARCHITECTURES form a list of architectures\n# Also generates PTX for the most recent architecture for forwards compatibility\nfunction(compute_cmake_cuda_archs archs)\n if(CMAKE_CUDA_COMPILER_VERSION MATCHES \"^([0-9]+\\\\.[0-9]+)\")\n set(CUDA_VERSION \"${CMAKE_MATCH_1}\")\n endif()\n list(SORT archs)\n unset(CMAKE_CUDA_ARCHITECTURES CACHE)\n set(CMAKE_CUDA_ARCHITECTURES ${archs})\n\n # Set up defaults based on CUDA varsion\n # Remember to update arch-specific tunings when supporting new archs.\n if(NOT CMAKE_CUDA_ARCHITECTURES)\n if(CUDA_VERSION VERSION_GREATER_EQUAL \"13.0\")\n set(CMAKE_CUDA_ARCHITECTURES 75 80 90 100 120)\n elseif(CUDA_VERSION VERSION_GREATER_EQUAL \"12.8\")\n set(CMAKE_CUDA_ARCHITECTURES 50 60 70 80 90 100 120)\n elseif(CUDA_VERSION VERSION_GREATER_EQUAL \"11.8\")\n set(CMAKE_CUDA_ARCHITECTURES 50 60 70 80 90)\n elseif(CUDA_VERSION VERSION_GREATER_EQUAL \"11.0\")\n set(CMAKE_CUDA_ARCHITECTURES 50 60 70 80)\n elseif(CUDA_VERSION VERSION_GREATER_EQUAL \"10.0\")\n set(CMAKE_CUDA_ARCHITECTURES 35 50 60 70)\n elseif(CUDA_VERSION VERSION_GREATER_EQUAL \"9.0\")\n set(CMAKE_CUDA_ARCHITECTURES 35 50 60 70)\n else()\n set(CMAKE_CUDA_ARCHITECTURES 35 50 60)\n endif()\n endif()\n\n list(TRANSFORM CMAKE_CUDA_ARCHITECTURES APPEND \"-real\")\n list(TRANSFORM CMAKE_CUDA_ARCHITECTURES REPLACE \"([0-9]+)-real\" \"\\\\0;\\\\1-virtual\" AT -1)\n set(CMAKE_CUDA_ARCHITECTURES \"${CMAKE_CUDA_ARCHITECTURES}\" PARENT_SCOPE)\n message(STATUS \"CMAKE_CUDA_ARCHITECTURES: ${CMAKE_CUDA_ARCHITECTURES}\")\nendfunction()\n\n# Set CUDA related flags to target. Must be used after code `format_gencode_flags`.\nfunction(xgboost_set_cuda_flags target)\n target_compile_options(${target} PRIVATE\n $<$:--expt-extended-lambda>\n $<$:--expt-relaxed-constexpr>\n $<$:-Xcompiler=${OpenMP_CXX_FLAGS}>\n $<$:-Xfatbin=-compress-all>\n $<$:--default-stream per-thread>\n )\n\n if(FORCE_COLORED_OUTPUT)\n if(FORCE_COLORED_OUTPUT AND (CMAKE_GENERATOR STREQUAL \"Ninja\") AND\n ((CMAKE_CXX_COMPILER_ID STREQUAL \"GNU\") OR\n (CMAKE_CXX_COMPILER_ID STREQUAL \"Clang\")))\n target_compile_options(${target} PRIVATE\n $<$:-Xcompiler=-fdiagnostics-color=always>)\n endif()\n endif()\n\n if(USE_DEVICE_DEBUG)\n target_compile_options(${target} PRIVATE\n $<$,$>:-G;-src-in-ptx>)\n endif()\n\n if(USE_NVTX)\n target_compile_definitions(${target} PRIVATE -DXGBOOST_USE_NVTX=1)\n if(NOT USE_DEVICE_DEBUG)\n target_compile_options(${target} PRIVATE $<$:-lineinfo>)\n endif()\n endif()\n\n # Use CCCL we find before CUDA Toolkit to make sure we get newer headers as intended\n # The CUDA Toolkit includes its own copy of CCCL that often lags the latest releases\n # (and would be picked up otherwise)\n if(BUILD_STATIC_LIB)\n # If the downstream user is statically linking with libxgboost, it needs to\n # explicitly link with CCCL and CUDA runtime.\n target_link_libraries(${target}\n PUBLIC CCCL::CCCL CUDA::cudart_static)\n else()\n # If the downstream user is dynamically linking with libxgboost, it does not\n # need to link with CCCL and CUDA runtime.\n target_link_libraries(${target} PRIVATE CCCL::CCCL CUDA::cudart_static)\n endif()\n target_compile_definitions(${target} PRIVATE -DXGBOOST_USE_CUDA=1)\n target_include_directories(\n ${target} PRIVATE\n ${xgboost_SOURCE_DIR}/gputreeshap)\n\n if(MSVC)\n target_compile_options(${target} PRIVATE\n $<$:-Xcompiler=/utf-8>)\n endif()\n\n set_target_properties(${target} PROPERTIES\n CUDA_STANDARD 17\n CUDA_STANDARD_REQUIRED ON)\n if(USE_CUDA_LTO)\n set_target_properties(${target} PROPERTIES\n INTERPROCEDURAL_OPTIMIZATION ON\n CUDA_SEPARABLE_COMPILATION ON)\n else()\n set_target_properties(${target} PROPERTIES\n CUDA_SEPARABLE_COMPILATION OFF)\n endif()\nendfunction()\n\nfunction(xgboost_link_nccl target)\n set(xgboost_nccl_flags -DXGBOOST_USE_NCCL=1)\n if(USE_DLOPEN_NCCL)\n list(APPEND xgboost_nccl_flags -DXGBOOST_USE_DLOPEN_NCCL=1)\n target_link_libraries(${target} PRIVATE ${CMAKE_DL_LIBS})\n endif()\n\n if(BUILD_STATIC_LIB)\n target_include_directories(${target} PUBLIC ${NCCL_INCLUDE_DIR})\n target_compile_definitions(${target} PUBLIC ${xgboost_nccl_flags})\n target_link_libraries(${target} PUBLIC ${NCCL_LIBRARY})\n else()\n target_include_directories(${target} PRIVATE ${NCCL_INCLUDE_DIR})\n target_compile_definitions(${target} PRIVATE ${xgboost_nccl_flags})\n if(NOT USE_DLOPEN_NCCL)\n target_link_libraries(${target} PRIVATE ${NCCL_LIBRARY})\n endif()\n endif()\nendfunction()\n\n# compile options\nmacro(xgboost_target_properties target)\n set_target_properties(${target} PROPERTIES\n CXX_STANDARD 17\n CXX_STANDARD_REQUIRED ON\n POSITION_INDEPENDENT_CODE ON)\n\n if(HIDE_CXX_SYMBOLS)\n #-- Hide all C++ symbols\n set_target_properties(${target} PROPERTIES\n C_VISIBILITY_PRESET hidden\n CXX_VISIBILITY_PRESET hidden\n CUDA_VISIBILITY_PRESET hidden\n )\n endif()\n\n if(ENABLE_ALL_WARNINGS)\n target_compile_options(${target} PUBLIC\n $,\n -Xcompiler=-Wall -Xcompiler=-Wextra -Xcompiler=-Wno-expansion-to-defined,\n -Wall -Wextra -Wno-expansion-to-defined>\n )\n endif()\n\n target_compile_options(${target}\n PRIVATE\n $<$,$>:/MP>\n $<$>,$>:-funroll-loops>)\n\n if(MSVC)\n target_compile_options(${target} PRIVATE\n $<$>:/utf-8>\n -D_CRT_SECURE_NO_WARNINGS\n -D_CRT_SECURE_NO_DEPRECATE\n )\n endif()\n\n if(WIN32 AND MINGW)\n target_compile_options(${target} PUBLIC -static-libstdc++)\n endif()\n\n if(NOT WIN32 AND ENABLE_ALL_WARNINGS)\n target_compile_options(${target} PRIVATE\n $<$:-Werror=cross-execution-space-call>\n )\n endif()\nendmacro()\n\n# Custom definitions used in xgboost.\nmacro(xgboost_target_defs target)\n if(NOT ${target} STREQUAL \"dmlc\") # skip dmlc core for custom logging.\n target_compile_definitions(${target}\n PRIVATE\n -DDMLC_LOG_CUSTOMIZE=1\n $<$>:_MWAITXINTRIN_H_INCLUDED>)\n endif()\n if(USE_DEBUG_OUTPUT)\n target_compile_definitions(${target} PRIVATE -DXGBOOST_USE_DEBUG_OUTPUT=1)\n endif()\n if(XGBOOST_MM_PREFETCH_PRESENT)\n target_compile_definitions(${target}\n PRIVATE\n -DXGBOOST_MM_PREFETCH_PRESENT=1)\n endif()\n if(XGBOOST_BUILTIN_PREFETCH_PRESENT)\n target_compile_definitions(${target}\n PRIVATE\n -DXGBOOST_BUILTIN_PREFETCH_PRESENT=1)\n endif()\n\n if(PLUGIN_RMM)\n target_compile_definitions(objxgboost PUBLIC -DXGBOOST_USE_RMM=1)\n endif()\n\n if(USE_NVCOMP)\n target_compile_definitions(objxgboost PUBLIC -DXGBOOST_USE_NVCOMP=1)\n endif()\n if(BUILD_WITH_GIT_HASH)\n target_compile_definitions(objxgboost PUBLIC -DXGBOOST_GIT_HASH=\"${XGBOOST_GIT_HASH}\")\n endif()\nendmacro()\n\n# handles dependencies\nmacro(xgboost_target_link_libraries target)\n if(NOT (CMAKE_SYSTEM_NAME STREQUAL \"Emscripten\"))\n if(BUILD_STATIC_LIB)\n target_link_libraries(${target} PUBLIC Threads::Threads ${CMAKE_THREAD_LIBS_INIT})\n else()\n target_link_libraries(${target} PRIVATE Threads::Threads ${CMAKE_THREAD_LIBS_INIT})\n endif()\n endif()\n\n if(USE_OPENMP)\n if(BUILD_STATIC_LIB)\n target_link_libraries(${target} PUBLIC OpenMP::OpenMP_CXX)\n else()\n target_link_libraries(${target} PRIVATE OpenMP::OpenMP_CXX)\n endif()\n endif()\n\n if(USE_CUDA)\n xgboost_set_cuda_flags(${target})\n endif()\n\n if(PLUGIN_RMM)\n target_link_libraries(${target} PRIVATE rmm::rmm)\n endif()\n\n if(USE_NVCOMP)\n target_link_libraries(${target} PRIVATE nvcomp::nvcomp)\n endif()\n\n if(USE_NCCL)\n xgboost_link_nccl(${target})\n endif()\n\n if(USE_NVTX)\n target_link_libraries(${target} PRIVATE CUDA::nvtx3)\n endif()\n\n if(MINGW)\n target_link_libraries(${target} PRIVATE wsock32 ws2_32)\n endif()\nendmacro()\n" }, { "path": "cmake/Version.cmake", "content": "function(write_version)\n message(STATUS \"xgboost VERSION: ${xgboost_VERSION}\")\n configure_file(\n ${xgboost_SOURCE_DIR}/cmake/version_config.h.in\n ${xgboost_SOURCE_DIR}/include/xgboost/version_config.h\n @ONLY\n NEWLINE_STYLE UNIX)\nendfunction()\n" }, { "path": "cmake/modules/FindASan.cmake", "content": "set(ASan_LIB_NAME ASan)\n\nfind_library(ASan_LIBRARY\n NAMES libasan.so libasan.so.6 libasan.so.5 libasan.so.4 libasan.so.3 libasan.so.2 libasan.so.1 libasan.so.0\n PATHS ${SANITIZER_PATH} /usr/lib64 /usr/lib /usr/local/lib64 /usr/local/lib ${CMAKE_PREFIX_PATH}/lib)\n\ninclude(FindPackageHandleStandardArgs)\nfind_package_handle_standard_args(ASan DEFAULT_MSG\n ASan_LIBRARY)\n\nmark_as_advanced(\n ASan_LIBRARY\n ASan_LIB_NAME)\n" }, { "path": "cmake/modules/FindLSan.cmake", "content": "set(LSan_LIB_NAME lsan)\n\nfind_library(LSan_LIBRARY\n NAMES liblsan.so liblsan.so.0 liblsan.so.0.0.0\n PATHS ${SANITIZER_PATH} /usr/lib64 /usr/lib /usr/local/lib64 /usr/local/lib ${CMAKE_PREFIX_PATH}/lib)\n\ninclude(FindPackageHandleStandardArgs)\nfind_package_handle_standard_args(LSan DEFAULT_MSG\n LSan_LIBRARY)\n\nmark_as_advanced(\n LSan_LIBRARY\n LSan_LIB_NAME)\n" }, { "path": "cmake/modules/FindLibR.cmake", "content": "# CMake module for R\n# Borrows ideas from RStudio's FindLibR.cmake\n#\n# Defines the following:\n# LIBR_FOUND\n# LIBR_HOME\n# LIBR_EXECUTABLE\n# LIBR_INCLUDE_DIRS\n# LIBR_LIB_DIR\n# LIBR_CORE_LIBRARY\n# and a cmake function to create R.lib for MSVC\n#\n# The following could be provided by user through cmake's -D options:\n# LIBR_EXECUTABLE (for unix and win)\n# R_VERSION (for win)\n# R_ARCH (for win 64 when want 32 bit build)\n#\n# TODO:\n# - someone to verify OSX detection,\n# - possibly, add OSX detection based on current R in PATH or LIBR_EXECUTABLE\n# - improve registry-based R_HOME detection in Windows (from a set of R_VERSION's)\n\n\n# Windows users might want to change this to their R version:\nif(NOT R_VERSION)\n set(R_VERSION \"4.0.0\")\nendif()\nif(NOT R_ARCH)\n if(\"${CMAKE_SIZEOF_VOID_P}\" STREQUAL \"4\")\n set(R_ARCH \"i386\")\n else()\n set(R_ARCH \"x64\")\n endif()\nendif()\n\n\n# Creates R.lib and R.def in the build directory for linking with MSVC\nfunction(create_rlib_for_msvc)\n # various checks and warnings\n if(NOT WIN32 OR (NOT MSVC AND NOT MINGW))\n message(FATAL_ERROR \"create_rlib_for_msvc() can only be used with MSVC or MINGW\")\n endif()\n if(NOT EXISTS \"${LIBR_LIB_DIR}\")\n message(FATAL_ERROR \"LIBR_LIB_DIR was not set!\")\n endif()\n find_program(DLLTOOL_EXE dlltool)\n if(NOT DLLTOOL_EXE)\n message(FATAL_ERROR \"\\ndlltool.exe not found!\\\n \\nDo you have Rtools installed with its MinGW's bin/ in PATH?\")\n endif()\n\n # extract symbols from R.dll into R.def and R.lib import library\n get_filename_component(\n LIBR_RSCRIPT_EXECUTABLE_DIR\n ${LIBR_EXECUTABLE}\n DIRECTORY\n )\n set(LIBR_RSCRIPT_EXECUTABLE \"${LIBR_RSCRIPT_EXECUTABLE_DIR}/Rscript\")\n\n execute_process(\n COMMAND ${LIBR_RSCRIPT_EXECUTABLE}\n \"${CMAKE_CURRENT_BINARY_DIR}/../../R-package/inst/make-r-def.R\"\n \"${LIBR_LIB_DIR}/R.dll\" \"${CMAKE_CURRENT_BINARY_DIR}/R.def\"\n )\n\n execute_process(COMMAND ${DLLTOOL_EXE}\n \"--input-def\" \"${CMAKE_CURRENT_BINARY_DIR}/R.def\"\n \"--output-lib\" \"${CMAKE_CURRENT_BINARY_DIR}/R.lib\"\n \"--temp-prefix\" \"Rlibtemp\"\n COMMAND_ECHO STDOUT\n COMMAND_ERROR_IS_FATAL ANY)\nendfunction()\n\n\n# detection for OSX\nif(APPLE)\n\n find_library(LIBR_LIBRARIES R)\n\n if(LIBR_LIBRARIES MATCHES \".*\\\\.framework\")\n set(LIBR_HOME \"${LIBR_LIBRARIES}/Resources\" CACHE PATH \"R home directory\")\n set(LIBR_INCLUDE_DIRS \"${LIBR_HOME}/include\" CACHE PATH \"R include directory\")\n set(LIBR_EXECUTABLE \"${LIBR_HOME}/R\" CACHE PATH \"R executable\")\n set(LIBR_LIB_DIR \"${LIBR_HOME}/lib\" CACHE PATH \"R lib directory\")\n else()\n get_filename_component(_LIBR_LIBRARIES \"${LIBR_LIBRARIES}\" REALPATH)\n get_filename_component(_LIBR_LIBRARIES_DIR \"${_LIBR_LIBRARIES}\" DIRECTORY)\n set(LIBR_EXECUTABLE \"${_LIBR_LIBRARIES_DIR}/../bin/R\")\n execute_process(\n COMMAND ${LIBR_EXECUTABLE} \"--slave\" \"--vanilla\" \"-e\" \"cat(R.home())\"\n OUTPUT_VARIABLE LIBR_HOME)\n set(LIBR_HOME ${LIBR_HOME} CACHE PATH \"R home directory\")\n set(LIBR_INCLUDE_DIRS \"${LIBR_HOME}/include\" CACHE PATH \"R include directory\")\n set(LIBR_LIB_DIR \"${LIBR_HOME}/lib\" CACHE PATH \"R lib directory\")\n endif()\n\n# detection for UNIX & Win32\nelse()\n\n # attempt to find R executable\n if(NOT LIBR_EXECUTABLE)\n find_program(LIBR_EXECUTABLE NAMES R R.exe)\n endif()\n\n if(UNIX)\n\n if(NOT LIBR_EXECUTABLE)\n message(FATAL_ERROR \"Unable to locate R executable.\\\n \\nEither add its location to PATH or provide it through the LIBR_EXECUTABLE cmake variable\")\n endif()\n\n # ask R for the home path\n execute_process(\n COMMAND ${LIBR_EXECUTABLE} \"--slave\" \"--vanilla\" \"-e\" \"cat(R.home())\"\n OUTPUT_VARIABLE LIBR_HOME\n )\n # ask R for the include dir\n execute_process(\n COMMAND ${LIBR_EXECUTABLE} \"--slave\" \"--vanilla\" \"-e\" \"cat(R.home('include'))\"\n OUTPUT_VARIABLE LIBR_INCLUDE_DIRS\n )\n # ask R for the lib dir\n execute_process(\n COMMAND ${LIBR_EXECUTABLE} \"--slave\" \"--vanilla\" \"-e\" \"cat(R.home('lib'))\"\n OUTPUT_VARIABLE LIBR_LIB_DIR\n )\n\n # Windows\n else()\n # ask R for R_HOME\n if(LIBR_EXECUTABLE)\n execute_process(\n COMMAND ${LIBR_EXECUTABLE} \"--slave\" \"--no-save\" \"-e\" \"cat(normalizePath(R.home(),winslash='/'))\"\n OUTPUT_VARIABLE LIBR_HOME)\n endif()\n # if R executable not available, query R_HOME path from registry\n if(NOT LIBR_HOME)\n get_filename_component(LIBR_HOME\n \"[HKEY_LOCAL_MACHINE\\\\SOFTWARE\\\\R-core\\\\R\\\\${R_VERSION};InstallPath]\"\n ABSOLUTE)\n if(NOT LIBR_HOME)\n message(FATAL_ERROR \"\\nUnable to locate R executable.\\\n \\nEither add its location to PATH or provide it through the LIBR_EXECUTABLE cmake variable\")\n endif()\n endif()\n # set exe location based on R_ARCH\n if(NOT LIBR_EXECUTABLE)\n set(LIBR_EXECUTABLE \"${LIBR_HOME}/bin/${R_ARCH}/R.exe\")\n endif()\n # set other R paths based on home path\n set(LIBR_INCLUDE_DIRS \"${LIBR_HOME}/include\")\n set(LIBR_LIB_DIR \"${LIBR_HOME}/bin/${R_ARCH}\")\n\nmessage(STATUS \"LIBR_HOME [${LIBR_HOME}]\")\nmessage(STATUS \"LIBR_EXECUTABLE [${LIBR_EXECUTABLE}]\")\nmessage(STATUS \"LIBR_INCLUDE_DIRS [${LIBR_INCLUDE_DIRS}]\")\nmessage(STATUS \"LIBR_LIB_DIR [${LIBR_LIB_DIR}]\")\nmessage(STATUS \"LIBR_CORE_LIBRARY [${LIBR_CORE_LIBRARY}]\")\n\n endif()\n\nendif()\n\nif((WIN32 AND MSVC) OR (WIN32 AND MINGW))\n # create a local R.lib import library for R.dll if it doesn't exist\n if(NOT EXISTS \"${CMAKE_CURRENT_BINARY_DIR}/R.lib\")\n create_rlib_for_msvc()\n endif()\nendif()\n\n# look for the core R library\nfind_library(LIBR_CORE_LIBRARY NAMES R\n HINTS \"${CMAKE_CURRENT_BINARY_DIR}\" \"${LIBR_LIB_DIR}\" \"${LIBR_HOME}/bin\" \"${LIBR_LIBRARIES}\")\nif(LIBR_CORE_LIBRARY-NOTFOUND)\n message(STATUS \"Could not find R core shared library.\")\nendif()\n\nset(LIBR_HOME ${LIBR_HOME} CACHE PATH \"R home directory\")\nset(LIBR_EXECUTABLE ${LIBR_EXECUTABLE} CACHE PATH \"R executable\")\nset(LIBR_INCLUDE_DIRS ${LIBR_INCLUDE_DIRS} CACHE PATH \"R include directory\")\nset(LIBR_LIB_DIR ${LIBR_LIB_DIR} CACHE PATH \"R shared libraries directory\")\nset(LIBR_CORE_LIBRARY ${LIBR_CORE_LIBRARY} CACHE PATH \"R core shared library\")\n\n# define find requirements\ninclude(FindPackageHandleStandardArgs)\nfind_package_handle_standard_args(LibR DEFAULT_MSG\n LIBR_HOME\n LIBR_EXECUTABLE\n LIBR_INCLUDE_DIRS\n LIBR_LIB_DIR\n LIBR_CORE_LIBRARY\n)\n\nif(LIBR_FOUND)\n message(STATUS \"Found R: ${LIBR_EXECUTABLE}\")\nendif()\n" }, { "path": "cmake/modules/FindNVML.cmake", "content": "if(NVML_LIBRARY)\n unset(NVML_LIBRARY CACHE)\nendif()\n\nset(NVML_LIB_NAME nvml)\n\nfind_path(NVML_INCLUDE_DIR\n NAMES nvml.h\n PATHS ${CUDA_HOME}/include ${CUDA_INCLUDE} /usr/local/cuda/include)\n\nfind_library(NVML_LIBRARY\n NAMES nvidia-ml)\n\nmessage(STATUS \"Using nvml library: ${NVML_LIBRARY}\")\n\ninclude(FindPackageHandleStandardArgs)\nfind_package_handle_standard_args(NVML DEFAULT_MSG\n NVML_INCLUDE_DIR NVML_LIBRARY)\n\nmark_as_advanced(\n NVML_INCLUDE_DIR\n NVML_LIBRARY\n)\n" }, { "path": "cmake/modules/FindNccl.cmake", "content": "#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n#\n# Tries to find NCCL headers and libraries.\n#\n# Usage of this module as follows:\n#\n# find_package(NCCL)\n#\n# Variables used by this module, they can change the default behaviour and need\n# to be set before calling find_package:\n#\n# NCCL_ROOT - When set, this path is inspected instead of standard library\n# locations as the root of the NCCL installation.\n# The environment variable NCCL_ROOT overrides this variable.\n#\n# This module defines\n# Nccl_FOUND, whether nccl has been found\n# NCCL_INCLUDE_DIR, directory containing header\n# NCCL_LIBRARY, directory containing nccl library\n# NCCL_LIB_NAME, nccl library name\n# USE_NCCL_LIB_PATH, when set, NCCL_LIBRARY path is also inspected for the\n# location of the nccl library. This would disable\n# switching between static and shared.\n#\n# This module assumes that the user has already called find_package(CUDA)\n\nif(NCCL_LIBRARY)\n if(NOT USE_NCCL_LIB_PATH)\n # Don't cache NCCL_LIBRARY to enable switching between static and shared.\n unset(NCCL_LIBRARY CACHE)\n endif()\nendif()\n\nif(BUILD_WITH_SHARED_NCCL)\n # libnccl.so\n set(NCCL_LIB_NAME nccl)\nelse()\n # libnccl_static.a\n set(NCCL_LIB_NAME nccl_static)\nendif()\n\nfind_path(NCCL_INCLUDE_DIR\n NAMES nccl.h\n HINTS ${NCCL_ROOT}/include $ENV{NCCL_ROOT}/include)\n\nif(USE_DLOPEN_NCCL)\n include(FindPackageHandleStandardArgs)\n find_package_handle_standard_args(Nccl DEFAULT_MSG NCCL_INCLUDE_DIR)\n\n mark_as_advanced(NCCL_INCLUDE_DIR)\nelse()\n find_library(NCCL_LIBRARY\n NAMES ${NCCL_LIB_NAME}\n HINTS ${NCCL_ROOT}/lib $ENV{NCCL_ROOT}/lib/)\n\n message(STATUS \"Using nccl library: ${NCCL_LIBRARY}\")\n\n include(FindPackageHandleStandardArgs)\n find_package_handle_standard_args(Nccl DEFAULT_MSG\n NCCL_INCLUDE_DIR NCCL_LIBRARY)\n\n mark_as_advanced(\n NCCL_INCLUDE_DIR\n NCCL_LIBRARY\n )\nendif()\n" }, { "path": "cmake/modules/FindTSan.cmake", "content": "set(TSan_LIB_NAME tsan)\n\nfind_library(TSan_LIBRARY\n NAMES libtsan.so libtsan.so.0 libtsan.so.0.0.0\n PATHS ${SANITIZER_PATH} /usr/lib64 /usr/lib /usr/local/lib64 /usr/local/lib ${CMAKE_PREFIX_PATH}/lib)\n\ninclude(FindPackageHandleStandardArgs)\nfind_package_handle_standard_args(TSan DEFAULT_MSG\n TSan_LIBRARY)\n\nmark_as_advanced(\n TSan_LIBRARY\n TSan_LIB_NAME)\n" }, { "path": "cmake/modules/FindUBSan.cmake", "content": "set(UBSan_LIB_NAME UBSan)\n\nfind_library(UBSan_LIBRARY\n NAMES libubsan.so libubsan.so.5 libubsan.so.4 libubsan.so.3 libubsan.so.2 libubsan.so.1 libubsan.so.0\n PATHS ${SANITIZER_PATH} /usr/lib64 /usr/lib /usr/local/lib64 /usr/local/lib ${CMAKE_PREFIX_PATH}/lib)\n\ninclude(FindPackageHandleStandardArgs)\nfind_package_handle_standard_args(UBSan DEFAULT_MSG\n UBSan_LIBRARY)\n\nmark_as_advanced(\n UBSan_LIBRARY\n UBSan_LIB_NAME)\n" }, { "path": "cmake/version_config.h.in", "content": "/**\n * Copyright 2019-2026, XGBoost contributors\n */\n#ifndef XGBOOST_VERSION_CONFIG_H_\n#define XGBOOST_VERSION_CONFIG_H_\n\n#define XGBOOST_VER_MAJOR @xgboost_VERSION_MAJOR@ /* NOLINT */\n#define XGBOOST_VER_MINOR @xgboost_VERSION_MINOR@ /* NOLINT */\n#define XGBOOST_VER_PATCH @xgboost_VERSION_PATCH@ /* NOLINT */\n\n#endif // XGBOOST_VERSION_CONFIG_H_\n" }, { "path": "cmake/xgboost-config.cmake.in", "content": "@PACKAGE_INIT@\n\nset(USE_OPENMP @USE_OPENMP@)\nset(USE_CUDA @USE_CUDA@)\nset(USE_NCCL @USE_NCCL@)\nset(XGBOOST_BUILD_STATIC_LIB @BUILD_STATIC_LIB@)\n\ninclude(CMakeFindDependencyMacro)\n\nif (XGBOOST_BUILD_STATIC_LIB)\n find_dependency(Threads)\n if(USE_OPENMP)\n find_dependency(OpenMP)\n endif()\n if(USE_CUDA)\n find_dependency(CUDA)\n endif()\n # nccl should be linked statically if xgboost is built as static library.\nendif (XGBOOST_BUILD_STATIC_LIB)\n\nif(NOT TARGET xgboost::xgboost)\n include(${CMAKE_CURRENT_LIST_DIR}/XGBoostTargets.cmake)\nendif()\n\nmessage(STATUS \"Found XGBoost (found version \\\"${xgboost_VERSION}\\\")\")\n" }, { "path": "cmake/xgboost.pc.in", "content": "prefix=@CMAKE_INSTALL_PREFIX@\nversion=@xgboost_VERSION@\nexec_prefix=${prefix}/bin\nlibdir=${prefix}/lib\nincludedir=${prefix}/include\n\nName: xgboost\nDescription: XGBoost - Scalable and Flexible Gradient Boosting.\nVersion: ${version}\n\nCflags: -I${includedir}\nLibs: -L${libdir} -lxgboost\n" }, { "path": "demo/.gitignore", "content": "*.libsvm\n*.pkl\n" }, { "path": "demo/README.md", "content": "Awesome XGBoost\n===============\nThis page contains a curated list of examples, tutorials, blogs about XGBoost usecases.\nIt is inspired by [awesome-MXNet](https://github.com/dmlc/mxnet/blob/master/example/README.md),\n[awesome-php](https://github.com/ziadoz/awesome-php) and [awesome-machine-learning](https://github.com/josephmisiti/awesome-machine-learning).\n\nPlease send a pull request if you find things that belongs to here.\n\nContents\n--------\n- [Code Examples](#code-examples)\n - [Features Walkthrough](#features-walkthrough)\n - [Benchmarks](#benchmarks)\n- [Machine Learning Challenge Winning Solutions](#machine-learning-challenge-winning-solutions)\n- [Tutorials](#tutorials)\n- [Usecases](#usecases)\n- [Tools using XGBoost](#tools-using-xgboost)\n- [Integrations with 3rd party software](#integrations-with-3rd-party-software)\n- [Awards](#awards)\n- [Windows Binaries](#windows-binaries)\n\nCode Examples\n-------------\n\n### Features Walkthrough\n\n_Note: for the R package, see the in-package examples and vignettes instead_\n\n_Note: For the Python package, see [Feature Walk through](https://xgboost.readthedocs.io/en/stable/python/examples/index.html)._\n\nThis is a list of short codes introducing different functionalities of xgboost packages.\n\n* Basic walkthrough of packages\n [python](guide-python/basic_walkthrough.py)\n [Julia](https://github.com/antinucleon/XGBoost.jl/blob/master/demo/basic_walkthrough.jl)\n* Customize loss function, and evaluation metric\n [python](guide-python/custom_objective.py)\n [Julia](https://github.com/antinucleon/XGBoost.jl/blob/master/demo/custom_objective.jl)\n* Boosting from existing prediction\n [python](guide-python/boost_from_prediction.py)\n [Julia](https://github.com/antinucleon/XGBoost.jl/blob/master/demo/boost_from_prediction.jl)\n* Predicting using first n trees\n [python](guide-python/predict_first_ntree.py)\n [Julia](https://github.com/antinucleon/XGBoost.jl/blob/master/demo/predict_first_ntree.jl)\n* Generalized Linear Model\n [python](guide-python/generalized_linear_model.py)\n [Julia](https://github.com/antinucleon/XGBoost.jl/blob/master/demo/generalized_linear_model.jl)\n* Cross validation\n [python](guide-python/cross_validation.py)\n [Julia](https://github.com/antinucleon/XGBoost.jl/blob/master/demo/cross_validation.jl)\n* Predicting leaf indices\n [python](guide-python/predict_leaf_indices.py)\n\n### Benchmarks\n\n- [Starter script for Kaggle Higgs Boson](kaggle-higgs)\n- [Kaggle Tradeshift winning solution by daxiongshu](https://github.com/daxiongshu/kaggle-tradeshift-winning-solution)\n- [Benchmarking the most commonly used open source tools for binary classification](https://github.com/szilard/benchm-ml#boosting-gradient-boosted-treesgradient-boosting-machines)\n\n\n## Machine Learning Challenge Winning Solutions\n\nXGBoost is extensively used by machine learning practitioners to create state of art data science solutions,\nthis is a list of machine learning winning solutions with XGBoost.\nPlease send pull requests if you find ones that are missing here.\n\n- Gábor Melis, 1st place winner of [Kaggle Higgs competition](https://github.com/ghl3/higgs-kaggle) conducted between May and September 2014. Link to [discussion](http://no2147483647.wordpress.com/2014/09/17/winning-solution-of-kaggle-higgs-competition-what-a-single-model-can-do/), [code](https://github.com/phunterlau/kaggle_higgs) and [news article](https://atlas.cern/updates/news/machine-learning-wins-higgs-challenge)\n- Bishwarup Bhattacharjee, 1st place winner of [Allstate Claims Severity](https://www.kaggle.com/competitions/allstate-claims-severity/overview) conducted on December 2016. Link to [discussion](https://www.kaggle.com/competitions/allstate-claims-severity/discussion/26416)\n- Benedikt Schifferer, Gilberto Titericz, Chris Deotte, Christof Henkel, Kazuki Onodera, Jiwei Liu, Bojan Tunguz, Even Oldridge, Gabriel De Souza Pereira Moreira and Ahmet Erdem, 1st place winner of [Twitter RecSys Challenge 2020](https://recsys-twitter.com/) conducted from June,20-August,20. [GPU Accelerated Feature Engineering and Training for Recommender Systems](https://medium.com/rapids-ai/winning-solution-of-recsys2020-challenge-gpu-accelerated-feature-engineering-and-training-for-cd67c5a87b1f)\n- Eugene Khvedchenya,Jessica Fridrich, Jan Butora, Yassine Yousfi 1st place winner in [ALASKA2 Image Steganalysis](https://www.kaggle.com/c/alaska2-image-steganalysis/overview). Link to [discussion](https://www.kaggle.com/c/alaska2-image-steganalysis/discussion/168546)\n- Dan Ofer, Seffi Cohen, Noa Dagan, Nurit, 1st place in WiDS Datathon 2020. Link to [discussion](https://www.kaggle.com/c/widsdatathon2020/discussion/133189)\n- Chris Deotte, Konstantin Yakovlev 1st place in [IEEE-CIS Fraud Detection](https://www.kaggle.com/c/ieee-fraud-detection/overview). Link to [discussion](https://www.kaggle.com/c/ieee-fraud-detection/discussion/111308)\n- Giba, Lucasz, 1st place winner in [Santander Value Prediction Challenge](https://www.kaggle.com/c/santander-value-prediction-challenge) organized on August,2018. Solution [discussion](https://www.kaggle.com/c/santander-value-prediction-challenge/discussion/65272) and [code](https://www.kaggle.com/titericz/winner-model-giba-single-xgb-lb0-5178/comments)\n- Beluga, 2nd place and Evgeny Nekrasov, 3rd place winner in Statoil/C-CORE Iceberg Classifier Challenge'2018. Link to [discussion](https://www.kaggle.com/c/statoil-iceberg-classifier-challenge/discussion/48294)\n- Radek Osmulski, 1st place of the [iMaterialist Challenge (Fashion) at FGVC5](https://www.kaggle.com/c/imaterialist-challenge-fashion-2018/overview). Link to [the winning solution](https://www.kaggle.com/c/imaterialist-challenge-fashion-2018/discussion/57944).\n- Maksims Volkovs, Guangwei Yu and Tomi Poutanen, 1st place of the [2017 ACM RecSys challenge](http://2017.recsyschallenge.com/). Link to [paper](http://www.cs.toronto.edu/~mvolkovs/recsys2017_challenge.pdf).\n- Vlad Sandulescu, Mihai Chiru, 1st place of the [KDD Cup 2016 competition](https://kddcup2016.azurewebsites.net). Link to [the arxiv paper](http://arxiv.org/abs/1609.02728).\n- Marios Michailidis, Mathias Müller and HJ van Veen, 1st place of the [Dato Truely Native? competition](https://www.kaggle.com/c/dato-native). Link to [the Kaggle interview](http://blog.kaggle.com/2015/12/03/dato-winners-interview-1st-place-mad-professors/).\n- Vlad Mironov, Alexander Guschin, 1st place of the [CERN LHCb experiment Flavour of Physics competition](https://www.kaggle.com/c/flavours-of-physics). Link to [the Kaggle interview](http://blog.kaggle.com/2015/11/30/flavour-of-physics-technical-write-up-1st-place-go-polar-bears/).\n- Josef Slavicek, 3rd place of the [CERN LHCb experiment Flavour of Physics competition](https://www.kaggle.com/c/flavours-of-physics). Link to [the Kaggle interview](http://blog.kaggle.com/2015/11/23/flavour-of-physics-winners-interview-3rd-place-josef-slavicek/).\n- Mario Filho, Josef Feigl, Lucas, Gilberto, 1st place of the [Caterpillar Tube Pricing competition](https://www.kaggle.com/c/caterpillar-tube-pricing). Link to [the Kaggle interview](http://blog.kaggle.com/2015/09/22/caterpillar-winners-interview-1st-place-gilberto-josef-leustagos-mario/).\n- Qingchen Wang, 1st place of the [Liberty Mutual Property Inspection](https://www.kaggle.com/c/liberty-mutual-group-property-inspection-prediction). Link to [the Kaggle interview](http://blog.kaggle.com/2015/09/28/liberty-mutual-property-inspection-winners-interview-qingchen-wang/).\n- Chenglong Chen, 1st place of the [Crowdflower Search Results Relevance](https://www.kaggle.com/c/crowdflower-search-relevance). Link to [the winning solution](https://www.kaggle.com/c/crowdflower-search-relevance/forums/t/15186/1st-place-winner-solution-chenglong-chen/).\n- Alexandre Barachant (“Cat”) and Rafał Cycoń (“Dog”), 1st place of the [Grasp-and-Lift EEG Detection](https://www.kaggle.com/c/grasp-and-lift-eeg-detection). Link to [the Kaggle interview](http://blog.kaggle.com/2015/10/12/grasp-and-lift-eeg-winners-interview-1st-place-cat-dog/).\n- Halla Yang, 2nd place of the [Recruit Coupon Purchase Prediction Challenge](https://www.kaggle.com/c/coupon-purchase-prediction). Link to [the Kaggle interview](http://blog.kaggle.com/2015/10/21/recruit-coupon-purchase-winners-interview-2nd-place-halla-yang/).\n- Owen Zhang, 1st place of the [Avito Context Ad Clicks competition](https://www.kaggle.com/c/avito-context-ad-clicks). Link to [the Kaggle interview](http://blog.kaggle.com/2015/08/26/avito-winners-interview-1st-place-owen-zhang/).\n- Keiichi Kuroyanagi, 2nd place of the [Airbnb New User Bookings](https://www.kaggle.com/c/airbnb-recruiting-new-user-bookings). Link to [the Kaggle interview](http://blog.kaggle.com/2016/03/17/airbnb-new-user-bookings-winners-interview-2nd-place-keiichi-kuroyanagi-keiku/).\n- Marios Michailidis, Mathias Müller and Ning Situ, 1st place [Homesite Quote Conversion](https://www.kaggle.com/c/homesite-quote-conversion). Link to [the Kaggle interview](http://blog.kaggle.com/2016/04/08/homesite-quote-conversion-winners-write-up-1st-place-kazanova-faron-clobber/).\n- Gilberto Titericz, Stanislav Semenov, 1st place in challenge to classify products into the correct category organized by Otto Group in 2015. Link to [challenge](https://www.kaggle.com/c/otto-group-product-classification-challenge). Link to [kaggle winning solution](https://www.kaggle.com/c/otto-group-product-classification-challenge/discussion/14335)\n- Darius Barušauskas, 1st place winner in [Predicting Red Hat Business Value](https://www.kaggle.com/c/predicting-red-hat-business-value). Link to [interview](https://medium.com/kaggle-blog/red-hat-business-value-competition-1st-place-winners-interview-darius-baru%C5%A1auskas-646692a2841b). Link to [discussion](https://www.kaggle.com/c/predicting-red-hat-business-value/discussion/23786)\n- David Austin, Weimin Wang, 1st place winner in [Iceberg-classifier-challenge](https://www.kaggle.com/c/statoil-iceberg-classifier-challenge/leaderboard) Link to [discussion](https://www.kaggle.com/c/statoil-iceberg-classifier-challenge/discussion/48241)\n- Kazuki Onodera, Kazuki Fujikawa, 2nd place winner in [OpenVaccine: COVID-19 mRNA Vaccine Degradation Prediction](https://www.kaggle.com/c/stanford-covid-vaccine/overview) Link to [Discussion](https://www.kaggle.com/c/stanford-covid-vaccine/discussion/189709)\n- Prarthana Bhat, 2nd place winner in [DYD Competition](https://datahack.analyticsvidhya.com/contest/date-your-data/). Link to [Solution](https://github.com/analyticsvidhya/DateYourData/blob/master/Prathna_Bhat_Model.R).\n- Benedikt Schifferer, Chris Deotte, Gilberto Titericz, Bo Liu, 1st place winner of [ACM RecSys Challenge 2021](https://recsys.acm.org/recsys21/challenge/). An ensemble of 5 XGBoost models and 3 neural networks. Link to [writeup](https://medium.com/nvidia-merlin/winning-the-recsys2021-challenge-by-a-diverse-set-of-xgboost-and-neural-network-models-4c5422a642d8) and [paper](https://dl.acm.org/doi/10.1145/3487572.3487605).\n- Chris Deotte, 1st place winner of [Kaggle Playground Series S5E2 - Backpack Prediction Challenge](https://www.kaggle.com/competitions/playground-series-s5e2) (Feb 2025). A single XGBoost model with 500 engineered features. Link to [discussion](https://www.kaggle.com/competitions/playground-series-s5e2/discussion/565539).\n- Chris Deotte, 1st place winner of [Kaggle Playground Series S5E4 - Predict Podcast Listening Time](https://www.kaggle.com/competitions/playground-series-s5e4) (Apr 2025). A three-level ensemble that includes XGBoost/GBDT models. Link to [writeup](https://www.kaggle.com/competitions/playground-series-s5e4/writeups/chris-deotte-1st-place-rapids-cuml-stack-3-levels) and [technical blog](https://developer.nvidia.com/blog/grandmaster-pro-tip-winning-first-place-in-a-kaggle-competition-with-stacking-using-cuml/).\n- Mohammad Odeh, 1st place winner of [March Machine Learning Mania 2025](https://www.kaggle.com/competitions/march-machine-learning-mania-2025) ($50K prize). XGBoost outperformed CatBoost and LightGBM. Link to [writeup](https://www.kaggle.com/competitions/march-machine-learning-mania-2025/writeups/mohammad-odeh-first-place-solution).\n\n## Talks\n- XGBoost: A Scalable Tree Boosting System ([video] (https://www.youtube.com/watch?v=Vly8xGnNiWs) + [slides](https://speakerdeck.com/datasciencela/tianqi-chen-xgboost-overview-and-latest-news-la-meetup-talk)) by Tianqi Chen at the Los Angeles Data Science meetup\n\n## Tutorials\n\n- [XGBoost Training with Dask, using Saturn Cloud](https://www.saturncloud.io/docs/tutorials/xgboost/)\n- [Machine Learning with XGBoost on Qubole Spark Cluster](https://www.qubole.com/blog/machine-learning-xgboost-qubole-spark-cluster/)\n- [XGBoost Official RMarkdown Tutorials](https://xgboost.readthedocs.org/en/latest/R-package/index.html#tutorials)\n- [An Introduction to XGBoost R Package](http://dmlc.ml/rstats/2016/03/10/xgboost.html) by Tong He\n- [Open Source Tools & Data Science Competitions](http://www.slideshare.net/odsc/owen-zhangopen-sourcetoolsanddscompetitions1) by Owen Zhang - XGBoost parameter tuning tips\n* [Feature Importance Analysis with XGBoost in Tax audit](http://fr.slideshare.net/MichaelBENESTY/feature-importance-analysis-with-xgboost-in-tax-audit)\n* [Winning solution of Kaggle Higgs competition: what a single model can do](http://no2147483647.wordpress.com/2014/09/17/winning-solution-of-kaggle-higgs-competition-what-a-single-model-can-do/)\n- [XGBoost - eXtreme Gradient Boosting](http://www.slideshare.net/ShangxuanZhang/xgboost) by Tong He\n- [How to use XGBoost algorithm in R in easy steps](http://www.analyticsvidhya.com/blog/2016/01/xgboost-algorithm-easy-steps/) by TAVISH SRIVASTAVA ([Chinese Translation 中文翻译](https://segmentfault.com/a/1190000004421821) by [HarryZhu](https://segmentfault.com/u/harryprince))\n- [Kaggle Solution: What’s Cooking ? (Text Mining Competition)](http://www.analyticsvidhya.com/blog/2015/12/kaggle-solution-cooking-text-mining-competition/) by MANISH SARASWAT\n- Better Optimization with Repeated Cross Validation and the XGBoost model - Machine Learning with R) by Manuel Amunategui ([Youtube Link](https://www.youtube.com/watch?v=Og7CGAfSr_Y)) ([GitHub Link](https://github.com/amunategui/BetterCrossValidation))\n- [XGBoost Rossman Parameter Tuning](https://www.kaggle.com/khozzy/rossmann-store-sales/xgboost-parameter-tuning-template/run/90168/notebook) by [Norbert Kozlowski](https://www.kaggle.com/khozzy)\n- [Featurizing log data before XGBoost](http://www.slideshare.net/DataRobot/featurizing-log-data-before-xgboost) by Xavier Conort, Owen Zhang etc\n- [West Nile Virus Competition Benchmarks & Tutorials](http://blog.kaggle.com/2015/07/21/west-nile-virus-competition-benchmarks-tutorials/) by [Anna Montoya](http://blog.kaggle.com/author/annamontoya/)\n- [Ensemble Decision Tree with XGBoost](https://www.kaggle.com/binghsu/predict-west-nile-virus/xgboost-starter-code-python-0-69) by [Bing Xu](https://www.kaggle.com/binghsu)\n- [Notes on eXtreme Gradient Boosting](http://startup.ml/blog/xgboost) by ARSHAK NAVRUZYAN ([iPython Notebook](https://github.com/startupml/koan/blob/master/eXtreme%20Gradient%20Boosting.ipynb))\n- [Complete Guide to Parameter Tuning in XGBoost](http://www.analyticsvidhya.com/blog/2016/03/complete-guide-parameter-tuning-xgboost-with-codes-python/) by Aarshay Jain\n- [Practical XGBoost in Python online course](http://education.parrotprediction.teachable.com/courses/practical-xgboost-in-python) by Parrot Prediction\n- [Spark and XGBoost using Scala](http://www.elenacuoco.com/2016/10/10/scala-spark-xgboost-classification/) by Elena Cuoco\n\n## Usecases\nIf you have particular usecase of xgboost that you would like to highlight.\nSend a PR to add a one sentence description:)\n\n- XGBoost is used in [Kaggle Script](https://www.kaggle.com/scripts) to solve data science challenges.\n- Distribute XGBoost as Rest API server from Jupyter notebook with [BentoML](https://github.com/bentoml/bentoml). [Link to notebook](https://github.com/bentoml/BentoML/blob/master/examples/xgboost-predict-titanic-survival/XGBoost-titanic-survival-prediction.ipynb)\n- [Seldon predictive service powered by XGBoost](https://docs.seldon.io/projects/seldon-core/en/latest/servers/xgboost.html)\n- XGBoost Distributed is used in [ODPS Cloud Service by Alibaba](https://yq.aliyun.com/articles/6355) (in Chinese)\n- XGBoost is incoporated as part of [Graphlab Create](https://dato.com/products/create/) for scalable machine learning.\n- [Hanjing Su](https://www.52cs.org) from Tencent data platform team: \"We use distributed XGBoost for click through prediction in wechat shopping and lookalikes. The problems involve hundreds millions of users and thousands of features. XGBoost is cleanly designed and can be easily integrated into our production environment, reducing our cost in developments.\"\n- [CNevd](https://github.com/CNevd) from autohome.com ad platform team: \"Distributed XGBoost is used for click through rate prediction in our display advertising, XGBoost is highly efficient and flexible and can be easily used on our distributed platform, our ctr made a great improvement with hundred millions samples and millions features due to this awesome XGBoost\"\n\n## Tools using XGBoost\n\n- [BayesBoost](https://github.com/mpearmain/BayesBoost) - Bayesian Optimization using xgboost and sklearn API\n- [FLAML](https://github.com/microsoft/FLAML) - An open source AutoML library\ndesigned to automatically produce accurate machine learning models with low computational cost. FLAML includes [XGBoost as one of the default learners](https://github.com/microsoft/FLAML/blob/main/flaml/model.py) and can also be used as a fast hyperparameter tuning tool for XGBoost ([code example](https://microsoft.github.io/FLAML/docs/Examples/AutoML-for-XGBoost)).\n- [gp_xgboost_gridsearch](https://github.com/vatsan/gp_xgboost_gridsearch) - In-database parallel grid-search for XGBoost on [Greenplum](https://github.com/greenplum-db/gpdb) using PL/Python\n- [tpot](https://github.com/rhiever/tpot) - A Python tool that automatically creates and optimizes machine learning pipelines using genetic programming.\n\n## Integrations with 3rd party software\nOpen source integrations with XGBoost:\n* [Neptune.ai](http://neptune.ai/) - Experiment management and collaboration tool for ML/DL/RL specialists. Integration has a form of the [XGBoost callback](https://docs.neptune.ai/integrations/xgboost.html) that automatically logs training and evaluation metrics, as well as saved model (booster), feature importance chart and visualized trees.\n* [Optuna](https://optuna.org/) - An open source hyperparameter optimization framework to automate hyperparameter search. Optuna integrates with XGBoost in the [XGBoostPruningCallback](https://optuna.readthedocs.io/en/stable/reference/integration.html#optuna.integration.XGBoostPruningCallback) that let users easily prune unpromising trials.\n* [dtreeviz](https://github.com/parrt/dtreeviz) - A python library for decision tree visualization and model interpretation. Starting from version 1.0, dtreeviz is able to visualize tree ensembles produced by XGBoost.\n\n## Awards\n- [John Chambers Award](http://stat-computing.org/awards/jmc/winners.html) - 2016 Winner: XGBoost R Package, by Tong He (Simon Fraser University) and Tianqi Chen (University of Washington)\n- [InfoWorld’s 2019 Technology of the Year Award](https://www.infoworld.com/article/3336072/application-development/infoworlds-2019-technology-of-the-year-award-winners.html)\n\n## Windows Binaries\nUnofficial windows binaries and instructions on how to use them are hosted on [Guido Tapia's blog](http://www.picnet.com.au/blogs/guido/post/2016/09/22/xgboost-windows-x64-binaries-for-download/)\n" }, { "path": "demo/aft_survival/README.rst", "content": "Survival Analysis Walkthrough\n=============================\n\nThis is a collection of examples for using the XGBoost Python package for training\nsurvival models. For an introduction, see :doc:`/tutorials/aft_survival_analysis`\n" }, { "path": "demo/aft_survival/aft_survival_demo.py", "content": "\"\"\"\nDemo for survival analysis (regression).\n========================================\n\nDemo for survival analysis (regression). using Accelerated Failure Time (AFT) model.\n\"\"\"\n\nimport os\n\nimport numpy as np\nimport pandas as pd\nfrom sklearn.model_selection import ShuffleSplit\n\nimport xgboost as xgb\n\n# The Veterans' Administration Lung Cancer Trial\n# The Statistical Analysis of Failure Time Data by Kalbfleisch J. and Prentice R (1980)\nCURRENT_DIR = os.path.dirname(__file__)\ndf = pd.read_csv(os.path.join(CURRENT_DIR, '../data/veterans_lung_cancer.csv'))\nprint('Training data:')\nprint(df)\n\n# Split features and labels\ny_lower_bound = df['Survival_label_lower_bound']\ny_upper_bound = df['Survival_label_upper_bound']\nX = df.drop(['Survival_label_lower_bound', 'Survival_label_upper_bound'], axis=1)\n\n# Split data into training and validation sets\nrs = ShuffleSplit(n_splits=2, test_size=.7, random_state=0)\ntrain_index, valid_index = next(rs.split(X))\ndtrain = xgb.DMatrix(X.values[train_index, :])\ndtrain.set_float_info('label_lower_bound', y_lower_bound[train_index])\ndtrain.set_float_info('label_upper_bound', y_upper_bound[train_index])\ndvalid = xgb.DMatrix(X.values[valid_index, :])\ndvalid.set_float_info('label_lower_bound', y_lower_bound[valid_index])\ndvalid.set_float_info('label_upper_bound', y_upper_bound[valid_index])\n\n# Train gradient boosted trees using AFT loss and metric\nparams = {'verbosity': 0,\n 'objective': 'survival:aft',\n 'eval_metric': 'aft-nloglik',\n 'tree_method': 'hist',\n 'learning_rate': 0.05,\n 'aft_loss_distribution': 'normal',\n 'aft_loss_distribution_scale': 1.20,\n 'max_depth': 6,\n 'lambda': 0.01,\n 'alpha': 0.02}\nbst = xgb.train(params, dtrain, num_boost_round=10000,\n evals=[(dtrain, 'train'), (dvalid, 'valid')],\n early_stopping_rounds=50)\n\n# Run prediction on the validation set\ndf = pd.DataFrame({'Label (lower bound)': y_lower_bound[valid_index],\n 'Label (upper bound)': y_upper_bound[valid_index],\n 'Predicted label': bst.predict(dvalid)})\nprint(df)\n# Show only data points with right-censored labels\nprint(df[np.isinf(df['Label (upper bound)'])])\n\n# Save trained model\nbst.save_model('aft_model.json')\n" }, { "path": "demo/aft_survival/aft_survival_demo_with_optuna.py", "content": "\"\"\"\nDemo for survival analysis (regression) with Optuna.\n====================================================\n\nDemo for survival analysis (regression) using Accelerated Failure Time (AFT) model,\nusing Optuna to tune hyperparameters\n\n\"\"\"\nimport numpy as np\nimport optuna\nimport pandas as pd\nfrom sklearn.model_selection import ShuffleSplit\n\nimport xgboost as xgb\n\n# The Veterans' Administration Lung Cancer Trial\n# The Statistical Analysis of Failure Time Data by Kalbfleisch J. and Prentice R (1980)\ndf = pd.read_csv('../data/veterans_lung_cancer.csv')\nprint('Training data:')\nprint(df)\n\n# Split features and labels\ny_lower_bound = df['Survival_label_lower_bound']\ny_upper_bound = df['Survival_label_upper_bound']\nX = df.drop(['Survival_label_lower_bound', 'Survival_label_upper_bound'], axis=1)\n\n# Split data into training and validation sets\nrs = ShuffleSplit(n_splits=2, test_size=.7, random_state=0)\ntrain_index, valid_index = next(rs.split(X))\ndtrain = xgb.DMatrix(X.values[train_index, :])\ndtrain.set_float_info('label_lower_bound', y_lower_bound[train_index])\ndtrain.set_float_info('label_upper_bound', y_upper_bound[train_index])\ndvalid = xgb.DMatrix(X.values[valid_index, :])\ndvalid.set_float_info('label_lower_bound', y_lower_bound[valid_index])\ndvalid.set_float_info('label_upper_bound', y_upper_bound[valid_index])\n\n# Define hyperparameter search space\nbase_params = {'verbosity': 0,\n 'objective': 'survival:aft',\n 'eval_metric': 'aft-nloglik',\n 'tree_method': 'hist'} # Hyperparameters common to all trials\ndef objective(trial):\n params = {'learning_rate': trial.suggest_loguniform('learning_rate', 0.01, 1.0),\n 'aft_loss_distribution': trial.suggest_categorical('aft_loss_distribution',\n ['normal', 'logistic', 'extreme']),\n 'aft_loss_distribution_scale': trial.suggest_loguniform('aft_loss_distribution_scale', 0.1, 10.0),\n 'max_depth': trial.suggest_int('max_depth', 3, 8),\n 'lambda': trial.suggest_loguniform('lambda', 1e-8, 1.0),\n 'alpha': trial.suggest_loguniform('alpha', 1e-8, 1.0)} # Search space\n params.update(base_params)\n pruning_callback = optuna.integration.XGBoostPruningCallback(trial, 'valid-aft-nloglik')\n bst = xgb.train(params, dtrain, num_boost_round=10000,\n evals=[(dtrain, 'train'), (dvalid, 'valid')],\n early_stopping_rounds=50, verbose_eval=False, callbacks=[pruning_callback])\n if bst.best_iteration >= 25:\n return bst.best_score\n else:\n return np.inf # Reject models with < 25 trees\n\n# Run hyperparameter search\nstudy = optuna.create_study(direction='minimize')\nstudy.optimize(objective, n_trials=200)\nprint('Completed hyperparameter tuning with best aft-nloglik = {}.'.format(study.best_trial.value))\nparams = {}\nparams.update(base_params)\nparams.update(study.best_trial.params)\n\n# Re-run training with the best hyperparameter combination\nprint('Re-running the best trial... params = {}'.format(params))\nbst = xgb.train(params, dtrain, num_boost_round=10000,\n evals=[(dtrain, 'train'), (dvalid, 'valid')],\n early_stopping_rounds=50)\n\n# Run prediction on the validation set\ndf = pd.DataFrame({'Label (lower bound)': y_lower_bound[valid_index],\n 'Label (upper bound)': y_upper_bound[valid_index],\n 'Predicted label': bst.predict(dvalid)})\nprint(df)\n# Show only data points with right-censored labels\nprint(df[np.isinf(df['Label (upper bound)'])])\n\n# Save trained model\nbst.save_model('aft_best_model.json')\n" }, { "path": "demo/aft_survival/aft_survival_viz_demo.py", "content": "\"\"\"\nVisual demo for survival analysis (regression) with Accelerated Failure Time (AFT) model.\n=========================================================================================\n\nThis demo uses 1D toy data and visualizes how XGBoost fits a tree ensemble. The ensemble\nmodel starts out as a flat line and evolves into a step function in order to account for\nall ranged labels.\n\"\"\"\n\nimport matplotlib.pyplot as plt\nimport numpy as np\n\nimport xgboost as xgb\n\nplt.rcParams.update({\"font.size\": 13})\n\n\n# Function to visualize censored labels\ndef plot_censored_labels(\n X: np.ndarray, y_lower: np.ndarray, y_upper: np.ndarray\n) -> None:\n def replace_inf(x: np.ndarray, target_value: float) -> np.ndarray:\n x[np.isinf(x)] = target_value\n return x\n\n plt.plot(X, y_lower, \"o\", label=\"y_lower\", color=\"blue\")\n plt.plot(X, y_upper, \"o\", label=\"y_upper\", color=\"fuchsia\")\n plt.vlines(\n X,\n ymin=replace_inf(y_lower, 0.01),\n ymax=replace_inf(y_upper, 1000.0),\n label=\"Range for y\",\n color=\"gray\",\n )\n\n\n# Toy data\nX = np.array([1, 2, 3, 4, 5]).reshape((-1, 1))\nINF = np.inf\ny_lower = np.array([10, 15, -INF, 30, 100])\ny_upper = np.array([INF, INF, 20, 50, INF])\n\n# Visualize toy data\nplt.figure(figsize=(5, 4))\nplot_censored_labels(X, y_lower, y_upper)\nplt.ylim((6, 200))\nplt.legend(loc=\"lower right\")\nplt.title(\"Toy data\")\nplt.xlabel(\"Input feature\")\nplt.ylabel(\"Label\")\nplt.yscale(\"log\")\nplt.tight_layout()\nplt.show(block=True)\n\n# Will be used to visualize XGBoost model\ngrid_pts = np.linspace(0.8, 5.2, 1000).reshape((-1, 1))\n\n# Train AFT model using XGBoost\ndmat = xgb.DMatrix(X)\ndmat.set_float_info(\"label_lower_bound\", y_lower)\ndmat.set_float_info(\"label_upper_bound\", y_upper)\nparams = {\"max_depth\": 3, \"objective\": \"survival:aft\", \"min_child_weight\": 0}\n\naccuracy_history = []\n\n\nclass PlotIntermediateModel(xgb.callback.TrainingCallback):\n \"\"\"Custom callback to plot intermediate models.\"\"\"\n\n def __init__(self) -> None:\n super().__init__()\n\n def after_iteration(\n self,\n model: xgb.Booster,\n epoch: int,\n evals_log: xgb.callback.TrainingCallback.EvalsLog,\n ) -> bool:\n \"\"\"Run after training is finished.\"\"\"\n # Compute y_pred = prediction using the intermediate model, at current boosting\n # iteration\n y_pred = model.predict(dmat)\n # \"Accuracy\" = the number of data points whose ranged label (y_lower, y_upper)\n # includes the corresponding predicted label (y_pred)\n acc = np.sum(\n np.logical_and(y_pred >= y_lower, y_pred <= y_upper) / len(X) * 100\n )\n accuracy_history.append(acc)\n\n # Plot ranged labels as well as predictions by the model\n plt.subplot(5, 3, epoch + 1)\n plot_censored_labels(X, y_lower, y_upper)\n y_pred_grid_pts = model.predict(xgb.DMatrix(grid_pts))\n plt.plot(\n grid_pts, y_pred_grid_pts, \"r-\", label=\"XGBoost AFT model\", linewidth=4\n )\n plt.title(\"Iteration {}\".format(epoch), x=0.5, y=0.8)\n plt.xlim((0.8, 5.2))\n plt.ylim((1 if np.min(y_pred) < 6 else 6, 200))\n plt.yscale(\"log\")\n return False\n\n\nres: xgb.callback.TrainingCallback.EvalsLog = {}\nplt.figure(figsize=(12, 13))\nbst = xgb.train(\n params,\n dmat,\n num_boost_round=15,\n evals=[(dmat, \"train\")],\n evals_result=res,\n callbacks=[PlotIntermediateModel()],\n)\nplt.tight_layout()\nplt.legend(\n loc=\"lower center\",\n ncol=4,\n bbox_to_anchor=(0.5, 0),\n bbox_transform=plt.gcf().transFigure,\n)\nplt.tight_layout()\n\n# Plot negative log likelihood over boosting iterations\nplt.figure(figsize=(8, 3))\nplt.subplot(1, 2, 1)\nplt.plot(res[\"train\"][\"aft-nloglik\"], \"b-o\", label=\"aft-nloglik\")\nplt.xlabel(\"# Boosting Iterations\")\nplt.legend(loc=\"best\")\n\n# Plot \"accuracy\" over boosting iterations\n# \"Accuracy\" = the number of data points whose ranged label (y_lower, y_upper) includes\n# the corresponding predicted label (y_pred)\nplt.subplot(1, 2, 2)\nplt.plot(accuracy_history, \"r-o\", label=\"Accuracy (%)\")\nplt.xlabel(\"# Boosting Iterations\")\nplt.legend(loc=\"best\")\nplt.tight_layout()\n\nplt.show()\n" }, { "path": "demo/c-api/.gitignore", "content": "c-api-demo\n" }, { "path": "demo/c-api/basic/Makefile", "content": "SRC=c-api-demo.c\nTGT=c-api-demo\n\ncc=cc\nCFLAGS ?=-O3\nXGBOOST_ROOT ?=../..\nINCLUDE_DIR=-I$(XGBOOST_ROOT)/include -I$(XGBOOST_ROOT)/dmlc-core/include\nLIB_DIR=-L$(XGBOOST_ROOT)/lib\n\nbuild: $(TGT)\n\n$(TGT): $(SRC) Makefile\n\t$(cc) $(CFLAGS) $(INCLUDE_DIR) $(LIB_DIR) -o $(TGT) $(SRC) -lxgboost\n\nrun: $(TGT)\n\tLD_LIBRARY_PATH=$(XGBOOST_ROOT)/lib ./$(TGT)\n\nclean:\n\trm -f $(TGT)\n" }, { "path": "demo/c-api/basic/README.md", "content": "C-APIs\n===\n\n**XGBoost** implements a C API originally designed for various language\nbindings. For detailed reference, please check xgboost/c_api.h. Here is a\ndemonstration of using the API.\n\n# CMake\nIf you use **CMake** for your project, you can either install **XGBoost**\nsomewhere in your system and tell CMake to find it by calling\n`find_package(xgboost)`, or put **XGBoost** inside your project's source tree\nand call **CMake** command: `add_subdirectory(xgboost)`. To use\n`find_package()`, put the following in your **CMakeLists.txt**:\n\n``` CMake\nfind_package(xgboost REQUIRED)\nadd_executable(api-demo c-api-demo.c)\ntarget_link_libraries(api-demo xgboost::xgboost)\n```\n\nIf you want to put XGBoost inside your project (like git submodule), use this\ninstead:\n``` CMake\nadd_subdirectory(xgboost)\nadd_executable(api-demo c-api-demo.c)\ntarget_link_libraries(api-demo xgboost)\n```\n\n# make\nYou can start by modifying the makefile in this directory to fit your need.\n" }, { "path": "demo/c-api/basic/c-api-demo.c", "content": "/**\n * Copyright 2019-2023 by XGBoost contributors\n *\n * \\file c-api-demo.c\n * \\brief A simple example of using xgboost C API.\n */\n\n#include \n#include \n#include /* uint32_t,uint64_t */\n#include \n#include \n#include \n#include \n\n#define safe_xgboost(call) { \\\nint err = (call); \\\nif (err != 0) { \\\n fprintf(stderr, \"%s:%d: error in %s: %s\\n\", __FILE__, __LINE__, #call, XGBGetLastError()); \\\n exit(1); \\\n} \\\n}\n\n/* Make Json encoded array interface. */\nstatic void MakeArrayInterface(size_t data, size_t n, char const* typestr, size_t length,\n char* out) {\n static char const kTemplate[] =\n \"{\\\"data\\\": [%lu, true], \\\"shape\\\": [%lu, %lu], \\\"typestr\\\": \\\"%s\\\", \\\"version\\\": 3}\";\n memset(out, '\\0', length);\n sprintf(out, kTemplate, data, n, 1ul, typestr);\n}\n/* Make Json encoded DMatrix configuration. */\nstatic void MakeConfig(int n_threads, size_t length, char* out) {\n static char const kTemplate[] = \"{\\\"missing\\\": NaN, \\\"nthread\\\": %d}\";\n memset(out, '\\0', length);\n sprintf(out, kTemplate, n_threads);\n}\n\nint main() {\n int silent = 0;\n int use_gpu = 0; // set to 1 to use the GPU for training\n\n // load the data\n DMatrixHandle dtrain, dtest;\n safe_xgboost(XGDMatrixCreateFromFile(\"../../data/agaricus.txt.train?format=libsvm\", silent, &dtrain));\n safe_xgboost(XGDMatrixCreateFromFile(\"../../data/agaricus.txt.test?format=libsvm\", silent, &dtest));\n\n // create the booster\n BoosterHandle booster;\n DMatrixHandle eval_dmats[2] = {dtrain, dtest};\n safe_xgboost(XGBoosterCreate(eval_dmats, 2, &booster));\n\n // configure the training\n // available parameters are described here:\n // https://xgboost.readthedocs.io/en/latest/parameter.html\n safe_xgboost(XGBoosterSetParam(booster, \"device\", use_gpu ? \"cuda\" : \"cpu\"));\n\n safe_xgboost(XGBoosterSetParam(booster, \"objective\", \"binary:logistic\"));\n safe_xgboost(XGBoosterSetParam(booster, \"min_child_weight\", \"1\"));\n safe_xgboost(XGBoosterSetParam(booster, \"gamma\", \"0.1\"));\n safe_xgboost(XGBoosterSetParam(booster, \"max_depth\", \"3\"));\n safe_xgboost(XGBoosterSetParam(booster, \"verbosity\", silent ? \"0\" : \"1\"));\n\n // train and evaluate for 10 iterations\n int n_trees = 10;\n const char* eval_names[2] = {\"train\", \"test\"};\n const char* eval_result = NULL;\n for (int i = 0; i < n_trees; ++i) {\n safe_xgboost(XGBoosterUpdateOneIter(booster, i, dtrain));\n safe_xgboost(XGBoosterEvalOneIter(booster, i, eval_dmats, eval_names, 2, &eval_result));\n printf(\"%s\\n\", eval_result);\n }\n\n bst_ulong num_feature = 0;\n safe_xgboost(XGBoosterGetNumFeature(booster, &num_feature));\n printf(\"num_feature: %lu\\n\", (unsigned long)(num_feature));\n\n // predict\n bst_ulong out_len = 0;\n int n_print = 10;\n\n /* Run prediction with DMatrix object. */\n char const config[] =\n \"{\\\"training\\\": false, \\\"type\\\": 0, \"\n \"\\\"iteration_begin\\\": 0, \\\"iteration_end\\\": 0, \\\"strict_shape\\\": false}\";\n /* Shape of output prediction */\n uint64_t const* out_shape;\n /* Dimension of output prediction */\n uint64_t out_dim;\n /* Pointer to a thread local contigious array, assigned in prediction function. */\n float const* out_result = NULL;\n safe_xgboost(\n XGBoosterPredictFromDMatrix(booster, dtest, config, &out_shape, &out_dim, &out_result));\n\n printf(\"y_pred: \");\n for (int i = 0; i < n_print; ++i) {\n printf(\"%1.4f \", out_result[i]);\n }\n printf(\"\\n\");\n\n // print true labels\n safe_xgboost(XGDMatrixGetFloatInfo(dtest, \"label\", &out_len, &out_result));\n printf(\"y_test: \");\n for (int i = 0; i < n_print; ++i) {\n printf(\"%1.4f \", out_result[i]);\n }\n printf(\"\\n\");\n\n {\n printf(\"Dense Matrix Example (XGDMatrixCreateFromMat): \");\n\n const float values[] = {0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0,\n 0, 0, 1, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 1, 0, 0,\n 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0,\n 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0,\n 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n 1, 0, 0, 0, 0, 1, 0, 0, 0, 0};\n\n DMatrixHandle dmat;\n safe_xgboost(XGDMatrixCreateFromMat(values, 1, 127, 0.0, &dmat));\n\n const float* out_result = NULL;\n\n safe_xgboost(\n XGBoosterPredictFromDMatrix(booster, dmat, config, &out_shape, &out_dim, &out_result));\n assert(out_dim == 1);\n assert(out_shape[0] == 1);\n\n printf(\"%1.4f \\n\", out_result[0]);\n safe_xgboost(XGDMatrixFree(dmat));\n }\n\n {\n printf(\"Sparse Matrix Example (XGDMatrixCreateFromCSR): \");\n\n const uint64_t indptr[] = {0, 22};\n const uint32_t indices[] = {1, 9, 19, 21, 24, 34, 36, 39, 42, 53, 56,\n 65, 69, 77, 86, 88, 92, 95, 102, 106, 117, 122};\n const float data[] = {1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0,\n 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0};\n\n DMatrixHandle dmat;\n char j_indptr[128];\n MakeArrayInterface((size_t)indptr, 2ul, \"\n#include \n#include \n#include \n\n#define safe_xgboost(err) \\\n if ((err) != 0) { \\\n fprintf(stderr, \"%s:%d: error in %s: %s\\n\", __FILE__, __LINE__, #err, \\\n XGBGetLastError()); \\\n exit(1); \\\n }\n\n#define N_BATCHS 32\n#define BATCH_LEN 512\n\n/* Shorthands. */\ntypedef DMatrixHandle DMatrix;\ntypedef BoosterHandle Booster;\n\ntypedef struct _DataIter {\n /* Data of each batch. */\n float **data;\n /* Labels of each batch */\n float **labels;\n /* Length of each batch. */\n size_t *lengths;\n /* Total number of batches. */\n size_t n;\n /* Current iteration. */\n size_t cur_it;\n\n /* Private fields */\n DMatrix _proxy;\n char _array[128];\n} DataIter;\n\n#define safe_malloc(ptr) \\\n if ((ptr) == NULL) { \\\n fprintf(stderr, \"%s:%d: Failed to allocate memory.\\n\", __FILE__, \\\n __LINE__); \\\n exit(1); \\\n }\n\n/**\n * Initialize with random data for demo. In practice the data should be loaded\n * from external memory. We just demonstrate how to use the iterator in\n * XGBoost.\n *\n * \\param batch_size Number of elements for each batch. The demo here is only using 1\n * column.\n * \\param n_batches Number of batches.\n */\nvoid DataIterator_Init(DataIter *self, size_t batch_size, size_t n_batches) {\n self->n = n_batches;\n\n self->lengths = (size_t *)malloc(self->n * sizeof(size_t));\n safe_malloc(self->lengths);\n for (size_t i = 0; i < self->n; ++i) {\n self->lengths[i] = batch_size;\n }\n\n self->data = (float **)malloc(self->n * sizeof(float *));\n safe_malloc(self->data);\n self->labels = (float **)malloc(self->n * sizeof(float *));\n safe_malloc(self->labels);\n\n /* Generate some random data. */\n for (size_t i = 0; i < self->n; ++i) {\n self->data[i] = (float *)malloc(self->lengths[i] * sizeof(float));\n safe_malloc(self->data[i]);\n for (size_t j = 0; j < self->lengths[i]; ++j) {\n float x = (float)rand() / (float)(RAND_MAX);\n self->data[i][j] = x;\n }\n\n self->labels[i] = (float *)malloc(self->lengths[i] * sizeof(float));\n safe_malloc(self->labels[i]);\n for (size_t j = 0; j < self->lengths[i]; ++j) {\n float y = (float)rand() / (float)(RAND_MAX);\n self->labels[i][j] = y;\n }\n }\n\n self->cur_it = 0;\n safe_xgboost(XGProxyDMatrixCreate(&self->_proxy));\n}\n\nvoid DataIterator_Free(DataIter *self) {\n for (size_t i = 0; i < self->n; ++i) {\n free(self->data[i]);\n free(self->labels[i]);\n }\n free(self->data);\n free(self->lengths);\n free(self->labels);\n safe_xgboost(XGDMatrixFree(self->_proxy));\n};\n\nint DataIterator_Next(DataIterHandle handle) {\n DataIter *self = (DataIter *)(handle);\n if (self->cur_it == self->n) {\n self->cur_it = 0;\n return 0; /* At end */\n }\n\n /* A JSON string encoding array interface (standard from numpy). */\n char array[] = \"{\\\"data\\\": [%lu, false], \\\"shape\\\":[%lu, 1], \\\"typestr\\\": \"\n \"\\\"_array, '\\0', sizeof(self->_array));\n sprintf(self->_array, array, (size_t)self->data[self->cur_it],\n self->lengths[self->cur_it]);\n\n safe_xgboost(XGProxyDMatrixSetDataDense(self->_proxy, self->_array));\n /* The data passed in the iterator must remain valid (not being freed until the next\n * iteration or reset) */\n safe_xgboost(XGDMatrixSetDenseInfo(self->_proxy, \"label\",\n self->labels[self->cur_it],\n self->lengths[self->cur_it], 1));\n self->cur_it++;\n return 1; /* Continue. */\n}\n\nvoid DataIterator_Reset(DataIterHandle handle) {\n DataIter *self = (DataIter *)(handle);\n self->cur_it = 0;\n}\n\n/**\n * Train a regression model and save it into JSON model file.\n */\nvoid TrainModel(DMatrix Xy) {\n /* Create booster for training. */\n Booster booster;\n DMatrix cache[] = {Xy};\n safe_xgboost(XGBoosterCreate(cache, 1, &booster));\n /* Use approx or hist for external memory training. */\n safe_xgboost(XGBoosterSetParam(booster, \"tree_method\", \"hist\"));\n safe_xgboost(XGBoosterSetParam(booster, \"objective\", \"reg:squarederror\"));\n\n /* Start training. */\n const char *validation_names[1] = {\"train\"};\n const char *validation_result = NULL;\n size_t n_rounds = 10;\n for (size_t i = 0; i < n_rounds; ++i) {\n safe_xgboost(XGBoosterUpdateOneIter(booster, i, Xy));\n safe_xgboost(XGBoosterEvalOneIter(booster, i, cache, validation_names, 1,\n &validation_result));\n printf(\"%s\\n\", validation_result);\n }\n\n /* Save the model to a JSON file. */\n safe_xgboost(XGBoosterSaveModel(booster, \"model.json\"));\n\n safe_xgboost(XGBoosterFree(booster));\n}\n\nint main() {\n DataIter iter;\n DataIterator_Init(&iter, BATCH_LEN, N_BATCHS);\n\n /* Create DMatrix from iterator. During training, some cache files with the\n * prefix \"cache-\" will be generated in current directory */\n char config[] = \"{\\\"missing\\\": NaN, \\\"cache_prefix\\\": \\\"cache\\\"}\";\n DMatrix Xy;\n safe_xgboost(XGDMatrixCreateFromCallback(\n &iter, iter._proxy, DataIterator_Reset, DataIterator_Next, config, &Xy));\n\n TrainModel(Xy);\n\n safe_xgboost(XGDMatrixFree(Xy));\n\n DataIterator_Free(&iter);\n return 0;\n}\n" }, { "path": "demo/c-api/inference/inference.c", "content": "/**\n * Copyright 2021-2025, XGBoost contributors\n *\n * @brief A simple example of using prediction functions.\n *\n * See more examples in test_c_api.cc on how to reuse a ProxyDMatrix object for reducing\n * the latency of DMatrix creation.\n */\n#include \n#include \n#include \n#include \n\n#define safe_xgboost(err) \\\n if ((err) != 0) { \\\n fprintf(stderr, \"%s:%d: error in %s: %s\\n\", __FILE__, __LINE__, #err, \\\n XGBGetLastError()); \\\n exit(1); \\\n }\n\n#define safe_malloc(ptr) \\\n if ((ptr) == NULL) { \\\n fprintf(stderr, \"%s:%d: Failed to allocate memory.\\n\", __FILE__, \\\n __LINE__); \\\n exit(1); \\\n }\n\n#define N_SAMPLES 128\n#define N_FEATURES 16\n\ntypedef BoosterHandle Booster;\ntypedef DMatrixHandle DMatrix;\n\n/* Row-major matrix */\nstruct _Matrix {\n float *data;\n size_t shape[2];\n\n /* private members */\n char _array_intrerface[256];\n};\n\n/* A custom data type for demo. */\ntypedef struct _Matrix *Matrix;\n\n/* Initialize matrix, copy data from `data` if it's not NULL. */\nvoid Matrix_Create(Matrix *self, float const *data, size_t n_samples,\n size_t n_features) {\n if (self == NULL) {\n fprintf(stderr, \"Invalid pointer to %s\\n\", __func__);\n exit(-1);\n }\n\n *self = (Matrix)malloc(sizeof(struct _Matrix));\n safe_malloc(*self);\n (*self)->data = (float *)malloc(n_samples * n_features * sizeof(float));\n safe_malloc((*self)->data);\n (*self)->shape[0] = n_samples;\n (*self)->shape[1] = n_features;\n\n if (data != NULL) {\n memcpy((*self)->data, data,\n (*self)->shape[0] * (*self)->shape[1] * sizeof(float));\n }\n}\n\n/* Generate random matrix. */\nvoid Matrix_Random(Matrix *self, size_t n_samples, size_t n_features) {\n Matrix_Create(self, NULL, n_samples, n_features);\n for (size_t i = 0; i < n_samples * n_features; ++i) {\n float x = (float)rand() / (float)(RAND_MAX);\n (*self)->data[i] = x;\n }\n}\n\n/* Array interface specified by numpy. */\nchar const *Matrix_ArrayInterface(Matrix self) {\n char const template[] = \"{\\\"data\\\": [%lu, true], \\\"shape\\\": [%lu, %lu], \"\n \"\\\"typestr\\\": \\\"_array_intrerface, '\\0', sizeof(self->_array_intrerface));\n sprintf(self->_array_intrerface, template, (size_t)self->data, self->shape[0],\n self->shape[1]);\n return self->_array_intrerface;\n}\n\nsize_t Matrix_NSamples(Matrix self) { return self->shape[0]; }\n\nsize_t Matrix_NFeatures(Matrix self) { return self->shape[1]; }\n\nfloat Matrix_At(Matrix self, size_t i, size_t j) {\n return self->data[i * self->shape[1] + j];\n}\n\nvoid Matrix_Print(Matrix self) {\n for (size_t i = 0; i < Matrix_NSamples(self); i++) {\n for (size_t j = 0; j < Matrix_NFeatures(self); ++j) {\n printf(\"%f, \", Matrix_At(self, i, j));\n }\n }\n printf(\"\\n\");\n}\n\nvoid Matrix_Free(Matrix self) {\n if (self != NULL) {\n if (self->data != NULL) {\n self->shape[0] = 0;\n self->shape[1] = 0;\n free(self->data);\n self->data = NULL;\n }\n free(self);\n }\n}\n\nint main() {\n Matrix X;\n Matrix y;\n\n Matrix_Random(&X, N_SAMPLES, N_FEATURES);\n Matrix_Random(&y, N_SAMPLES, 1);\n\n char const *X_interface = Matrix_ArrayInterface(X);\n char config[] = \"{\\\"nthread\\\": 16, \\\"missing\\\": NaN}\";\n DMatrix Xy;\n /* Dense means \"dense matrix\". */\n safe_xgboost(XGDMatrixCreateFromDense(X_interface, config, &Xy));\n /* Label must be in a contigious array. */\n safe_xgboost(XGDMatrixSetDenseInfo(Xy, \"label\", y->data, y->shape[0], 1));\n\n DMatrix cache[] = {Xy};\n Booster booster;\n /* Train a booster for demo. */\n safe_xgboost(XGBoosterCreate(cache, 1, &booster));\n\n size_t n_rounds = 10;\n for (size_t i = 0; i < n_rounds; ++i) {\n safe_xgboost(XGBoosterUpdateOneIter(booster, i, Xy));\n }\n\n /* Save the trained model in JSON format. */\n safe_xgboost(XGBoosterSaveModel(booster, \"model.json\"));\n safe_xgboost(XGBoosterFree(booster));\n\n /* Load it back for inference. The save and load is not required, only shown here for\n * demonstration purpose. */\n safe_xgboost(XGBoosterCreate(NULL, 0, &booster));\n safe_xgboost(XGBoosterLoadModel(booster, \"model.json\"));\n {\n /* Run prediction with DMatrix object. */\n char const config[] =\n \"{\\\"training\\\": false, \\\"type\\\": 0, \"\n \"\\\"iteration_begin\\\": 0, \\\"iteration_end\\\": 0, \\\"strict_shape\\\": true}\";\n /* Shape of output prediction */\n uint64_t const *out_shape;\n /* Dimension of output prediction */\n uint64_t out_dim;\n /* Pointer to a thread local contigious array, assigned in prediction function. */\n float const *out_results;\n\n safe_xgboost(XGBoosterPredictFromDMatrix(booster, Xy, config, &out_shape,\n &out_dim, &out_results));\n if (out_dim != 2 || out_shape[0] != N_SAMPLES || out_shape[1] != 1) {\n fprintf(stderr, \"Regression model should output prediction as vector.\");\n exit(-1);\n }\n\n Matrix predt;\n /* Always copy output from XGBoost before calling next API function. */\n Matrix_Create(&predt, out_results, out_shape[0], out_shape[1]);\n printf(\"Results from prediction\\n\");\n Matrix_Print(predt);\n Matrix_Free(predt);\n }\n\n {\n /* Run inplace prediction, which is faster and more memory efficient, but supports\n * only basic inference types. */\n char const config[] = \"{\\\"type\\\": 0, \\\"iteration_begin\\\": 0, \"\n \"\\\"iteration_end\\\": 0, \\\"strict_shape\\\": true, \"\n \"\\\"cache_id\\\": 0, \\\"missing\\\": NaN}\";\n /* Shape of output prediction */\n uint64_t const *out_shape;\n /* Dimension of output prediction */\n uint64_t out_dim;\n /* Pointer to a thread local contigious array, assigned in prediction function. */\n float const *out_results;\n\n char const *X_interface = Matrix_ArrayInterface(X);\n safe_xgboost(XGBoosterPredictFromDense(booster, X_interface, config, NULL,\n &out_shape, &out_dim, &out_results));\n\n if (out_dim != 2 || out_shape[0] != N_SAMPLES || out_shape[1] != 1) {\n fprintf(stderr,\n \"Regression model should output prediction as vector, %lu, %lu\",\n out_dim, out_shape[0]);\n exit(-1);\n }\n\n Matrix predt;\n /* Always copy output from XGBoost before calling next API function. */\n Matrix_Create(&predt, out_results, out_shape[0], out_shape[1]);\n printf(\"Results from inplace prediction\\n\");\n Matrix_Print(predt);\n Matrix_Free(predt);\n }\n\n XGBoosterFree(booster);\n\n XGDMatrixFree(Xy);\n Matrix_Free(X);\n Matrix_Free(y);\n return 0;\n}\n" }, { "path": "demo/dask/README.rst", "content": ".. _dask-examples:\n\nXGBoost Dask Feature Walkthrough\n================================\n\nThis directory contains some demonstrations for using `dask` with `XGBoost`. For an\noverview, see :doc:`/tutorials/dask`\n" }, { "path": "demo/dask/cpu_survival.py", "content": "\"\"\"\nExample of training survival model with Dask on CPU\n===================================================\n\n\"\"\"\n\nimport os\n\nimport dask.array as da\nimport dask.dataframe as dd\nfrom dask.distributed import Client, LocalCluster\n\nfrom xgboost import dask as dxgb\nfrom xgboost.dask import DaskDMatrix\n\n\ndef main(client: Client) -> da.Array:\n # Load an example survival data from CSV into a Dask data frame.\n # The Veterans' Administration Lung Cancer Trial\n # The Statistical Analysis of Failure Time Data by Kalbfleisch J. and Prentice R (1980)\n CURRENT_DIR = os.path.dirname(__file__)\n df = dd.read_csv(\n os.path.join(CURRENT_DIR, os.pardir, \"data\", \"veterans_lung_cancer.csv\")\n )\n\n # DaskDMatrix acts like normal DMatrix, works as a proxy for local\n # DMatrix scatter around workers.\n # For AFT survival, you'd need to extract the lower and upper bounds for the label\n # and pass them as arguments to DaskDMatrix.\n y_lower_bound = df[\"Survival_label_lower_bound\"]\n y_upper_bound = df[\"Survival_label_upper_bound\"]\n X = df.drop([\"Survival_label_lower_bound\", \"Survival_label_upper_bound\"], axis=1)\n dtrain = DaskDMatrix(\n client, X, label_lower_bound=y_lower_bound, label_upper_bound=y_upper_bound\n )\n\n # Use train method from xgboost.dask instead of xgboost. This\n # distributed version of train returns a dictionary containing the\n # resulting booster and evaluation history obtained from\n # evaluation metrics.\n params = {\n \"verbosity\": 1,\n \"objective\": \"survival:aft\",\n \"eval_metric\": \"aft-nloglik\",\n \"learning_rate\": 0.05,\n \"aft_loss_distribution_scale\": 1.20,\n \"aft_loss_distribution\": \"normal\",\n \"max_depth\": 6,\n \"lambda\": 0.01,\n \"alpha\": 0.02,\n }\n output = dxgb.train(\n client, params, dtrain, num_boost_round=100, evals=[(dtrain, \"train\")]\n )\n bst = output[\"booster\"]\n history = output[\"history\"]\n\n # you can pass output directly into `predict` too.\n prediction = dxgb.predict(client, bst, dtrain)\n print(\"Evaluation history: \", history)\n\n # Uncomment the following line to save the model to the disk\n # bst.save_model('survival_model.json')\n\n return prediction\n\n\nif __name__ == \"__main__\":\n # or use other clusters for scaling\n with LocalCluster(n_workers=7, threads_per_worker=4) as cluster:\n with Client(cluster) as client:\n main(client)\n" }, { "path": "demo/dask/cpu_training.py", "content": "\"\"\"\nExample of training with Dask on CPU\n====================================\n\n\"\"\"\n\nfrom dask import array as da\nfrom dask.distributed import Client, LocalCluster\n\nfrom xgboost import dask as dxgb\nfrom xgboost.dask import DaskDMatrix\n\n\ndef main(client: Client) -> None:\n # generate some random data for demonstration\n m = 100000\n n = 100\n rng = da.random.default_rng(1)\n X = rng.normal(size=(m, n), chunks=(10000, -1))\n y = X.sum(axis=1)\n\n # DaskDMatrix acts like normal DMatrix, works as a proxy for local\n # DMatrix scatter around workers.\n dtrain = DaskDMatrix(client, X, y)\n\n # Use train method from xgboost.dask instead of xgboost. This\n # distributed version of train returns a dictionary containing the\n # resulting booster and evaluation history obtained from\n # evaluation metrics.\n output = dxgb.train(\n client,\n {\"verbosity\": 1, \"tree_method\": \"hist\"},\n dtrain,\n num_boost_round=4,\n evals=[(dtrain, \"train\")],\n )\n bst = output[\"booster\"]\n history = output[\"history\"]\n\n # you can pass output directly into `predict` too.\n prediction = dxgb.predict(client, bst, dtrain)\n print(\"Evaluation history:\", history)\n print(\"Error:\", da.sqrt((prediction - y) ** 2).mean().compute())\n\n\nif __name__ == \"__main__\":\n # or use other clusters for scaling\n with LocalCluster(n_workers=7, threads_per_worker=4) as cluster:\n with Client(cluster) as client:\n main(client)\n" }, { "path": "demo/dask/dask_callbacks.py", "content": "\"\"\"\nExample of using callbacks with Dask\n====================================\n\"\"\"\n\nfrom typing import Any\n\nimport numpy as np\nfrom dask.distributed import Client, LocalCluster\nfrom dask_ml.datasets import make_regression\nfrom dask_ml.model_selection import train_test_split\n\nimport xgboost as xgb\nimport xgboost.dask as dxgb\nfrom xgboost.dask import DaskDMatrix\n\n\ndef probability_for_going_backward(epoch: int) -> float:\n return 0.999 / (1.0 + 0.05 * np.log(1.0 + epoch))\n\n\n# All callback functions must inherit from TrainingCallback\nclass CustomEarlyStopping(xgb.callback.TrainingCallback):\n \"\"\"A custom early stopping class where early stopping is determined stochastically.\n In the beginning, allow the metric to become worse with a probability of 0.999.\n As boosting progresses, the probability should be adjusted downward\"\"\"\n\n def __init__(\n self, *, validation_set: str, target_metric: str, maximize: bool, seed: int\n ) -> None:\n self.validation_set = validation_set\n self.target_metric = target_metric\n self.maximize = maximize\n self.seed = seed\n self.rng = np.random.default_rng(seed=seed)\n if maximize:\n self.better = lambda x, y: x > y\n else:\n self.better = lambda x, y: x < y\n\n def after_iteration(\n self, model: Any, epoch: int, evals_log: xgb.callback.TrainingCallback.EvalsLog\n ) -> bool:\n metric_history = evals_log[self.validation_set][self.target_metric]\n if len(metric_history) < 2 or self.better(\n metric_history[-1], metric_history[-2]\n ):\n return False # continue training\n p = probability_for_going_backward(epoch)\n go_backward = self.rng.choice(2, size=(1,), replace=True, p=[1 - p, p]).astype(\n np.bool_\n )[0]\n print(\n \"The validation metric went into the wrong direction. \"\n + f\"Stopping training with probability {1 - p}...\"\n )\n if go_backward:\n return False # continue training\n else:\n return True # stop training\n\n\ndef main(client: Client) -> None:\n m = 100000\n n = 100\n X, y = make_regression(n_samples=m, n_features=n, chunks=200, random_state=0)\n X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=0)\n\n dtrain = DaskDMatrix(client, X_train, y_train)\n dtest = DaskDMatrix(client, X_test, y_test)\n\n output = dxgb.train(\n client,\n {\n \"verbosity\": 1,\n \"tree_method\": \"hist\",\n \"objective\": \"reg:squarederror\",\n \"eval_metric\": \"rmse\",\n \"max_depth\": 6,\n \"learning_rate\": 1.0,\n },\n dtrain,\n num_boost_round=1000,\n evals=[(dtrain, \"train\"), (dtest, \"test\")],\n callbacks=[\n CustomEarlyStopping(\n validation_set=\"test\", target_metric=\"rmse\", maximize=False, seed=0\n )\n ],\n )\n\n\nif __name__ == \"__main__\":\n # or use other clusters for scaling\n with LocalCluster(n_workers=4, threads_per_worker=1) as cluster:\n with Client(cluster) as client:\n main(client)\n" }, { "path": "demo/dask/dask_learning_to_rank.py", "content": "\"\"\"\nLearning to rank with the Dask Interface\n========================================\n\n .. versionadded:: 3.0.0\n\nThis is a demonstration of using XGBoost for learning to rank tasks using the\nMSLR_10k_letor dataset. For more infomation about the dataset, please visit its\n`description page `_.\n\nSee :ref:`ltr-dist` for a general description for distributed learning to rank and\n:ref:`ltr-dask` for Dask-specific features.\n\n\"\"\"\n\nfrom __future__ import annotations\n\nimport argparse\nimport os\nfrom contextlib import contextmanager\nfrom typing import Generator\n\nimport dask\nimport numpy as np\nfrom dask import dataframe as dd\nfrom distributed import Client, LocalCluster, wait\nfrom sklearn.datasets import load_svmlight_file\n\nfrom xgboost import dask as dxgb\n\n\ndef load_mslr_10k(\n device: str, data_path: str, cache_path: str\n) -> tuple[dd.DataFrame, dd.DataFrame, dd.DataFrame]:\n \"\"\"Load the MSLR10k dataset from data_path and save parquet files in the cache_path.\"\"\"\n root_path = os.path.expanduser(args.data)\n cache_path = os.path.expanduser(args.cache)\n\n # Use only the Fold1 for demo:\n # Train, Valid, Test\n # {S1,S2,S3}, S4, S5\n fold = 1\n\n if not os.path.exists(cache_path):\n os.mkdir(cache_path)\n fold_path = os.path.join(root_path, f\"Fold{fold}\")\n train_path = os.path.join(fold_path, \"train.txt\")\n valid_path = os.path.join(fold_path, \"vali.txt\")\n test_path = os.path.join(fold_path, \"test.txt\")\n\n X_train, y_train, qid_train = load_svmlight_file(\n train_path, query_id=True, dtype=np.float32\n )\n columns = [f\"f{i}\" for i in range(X_train.shape[1])]\n X_train = dd.from_array(X_train.toarray(), columns=columns)\n y_train = y_train.astype(np.int32)\n qid_train = qid_train.astype(np.int32)\n\n X_train[\"y\"] = dd.from_array(y_train)\n X_train[\"qid\"] = dd.from_array(qid_train)\n X_train.to_parquet(os.path.join(cache_path, \"train\"), engine=\"pyarrow\")\n\n X_valid, y_valid, qid_valid = load_svmlight_file(\n valid_path, query_id=True, dtype=np.float32\n )\n X_valid = dd.from_array(X_valid.toarray(), columns=columns)\n y_valid = y_valid.astype(np.int32)\n qid_valid = qid_valid.astype(np.int32)\n\n X_valid[\"y\"] = dd.from_array(y_valid)\n X_valid[\"qid\"] = dd.from_array(qid_valid)\n X_valid.to_parquet(os.path.join(cache_path, \"valid\"), engine=\"pyarrow\")\n\n X_test, y_test, qid_test = load_svmlight_file(\n test_path, query_id=True, dtype=np.float32\n )\n\n X_test = dd.from_array(X_test.toarray(), columns=columns)\n y_test = y_test.astype(np.int32)\n qid_test = qid_test.astype(np.int32)\n\n X_test[\"y\"] = dd.from_array(y_test)\n X_test[\"qid\"] = dd.from_array(qid_test)\n X_test.to_parquet(os.path.join(cache_path, \"test\"), engine=\"pyarrow\")\n\n df_train = dd.read_parquet(\n os.path.join(cache_path, \"train\"), calculate_divisions=True\n )\n df_valid = dd.read_parquet(\n os.path.join(cache_path, \"valid\"), calculate_divisions=True\n )\n df_test = dd.read_parquet(\n os.path.join(cache_path, \"test\"), calculate_divisions=True\n )\n\n return df_train, df_valid, df_test\n\n\ndef ranking_demo(client: Client, args: argparse.Namespace) -> None:\n \"\"\"Learning to rank with data sorted locally.\"\"\"\n df_tr, df_va, _ = load_mslr_10k(args.device, args.data, args.cache)\n\n X_train: dd.DataFrame = df_tr[df_tr.columns.difference([\"y\", \"qid\"])]\n y_train = df_tr[[\"y\", \"qid\"]]\n Xy_train = dxgb.DaskQuantileDMatrix(client, X_train, y_train.y, qid=y_train.qid)\n\n X_valid: dd.DataFrame = df_va[df_va.columns.difference([\"y\", \"qid\"])]\n y_valid = df_va[[\"y\", \"qid\"]]\n Xy_valid = dxgb.DaskQuantileDMatrix(\n client, X_valid, y_valid.y, qid=y_valid.qid, ref=Xy_train\n )\n # Upon training, you will see a performance warning about sorting data based on\n # query groups.\n dxgb.train(\n client,\n {\"objective\": \"rank:ndcg\", \"device\": args.device},\n Xy_train,\n evals=[(Xy_train, \"Train\"), (Xy_valid, \"Valid\")],\n num_boost_round=100,\n )\n\n\ndef ranking_wo_split_demo(client: Client, args: argparse.Namespace) -> None:\n \"\"\"Learning to rank with data partitioned according to query groups.\"\"\"\n df_tr, df_va, df_te = load_mslr_10k(args.device, args.data, args.cache)\n\n X_tr = df_tr[df_tr.columns.difference([\"y\", \"qid\"])]\n X_va = df_va[df_va.columns.difference([\"y\", \"qid\"])]\n\n # `allow_group_split=False` makes sure data is partitioned according to the query\n # groups.\n ltr = dxgb.DaskXGBRanker(allow_group_split=False, device=args.device)\n ltr.client = client\n ltr = ltr.fit(\n X_tr,\n df_tr.y,\n qid=df_tr.qid,\n eval_set=[(X_tr, df_tr.y), (X_va, df_va.y)],\n eval_qid=[df_tr.qid, df_va.qid],\n verbose=True,\n )\n\n df_te = df_te.persist()\n wait([df_te])\n\n X_te = df_te[df_te.columns.difference([\"y\", \"qid\"])]\n predt = ltr.predict(X_te)\n y = client.compute(df_te.y)\n wait([predt, y])\n\n\n@contextmanager\ndef gen_client(device: str) -> Generator[Client, None, None]:\n match device:\n case \"cuda\":\n from dask_cuda import LocalCUDACluster\n\n with LocalCUDACluster() as cluster:\n with Client(cluster) as client:\n with dask.config.set(\n {\n \"array.backend\": \"cupy\",\n \"dataframe.backend\": \"cudf\",\n }\n ):\n yield client\n case \"cpu\":\n with LocalCluster() as cluster:\n with Client(cluster) as client:\n yield client\n\n\nif __name__ == \"__main__\":\n parser = argparse.ArgumentParser(\n description=\"Demonstration of learning to rank using XGBoost.\"\n )\n parser.add_argument(\n \"--data\",\n type=str,\n help=\"Root directory of the MSLR-WEB10K data.\",\n required=True,\n )\n parser.add_argument(\n \"--cache\",\n type=str,\n help=\"Directory for caching processed data.\",\n required=True,\n )\n parser.add_argument(\"--device\", choices=[\"cpu\", \"cuda\"], default=\"cpu\")\n parser.add_argument(\n \"--no-split\",\n action=\"store_true\",\n help=\"Flag to indicate query groups should not be split.\",\n )\n args = parser.parse_args()\n\n with gen_client(args.device) as client:\n if args.no_split:\n ranking_wo_split_demo(client, args)\n else:\n ranking_demo(client, args)\n" }, { "path": "demo/dask/forward_logging.py", "content": "\"\"\"\nExample of forwarding evaluation logs to the client\n===================================================\n\nThe example runs on GPU. Two classes are defined to show how to use Dask builtins to\nforward the logs to the client process.\n\n\"\"\"\n\nimport logging\n\nimport dask\nimport distributed\nfrom dask import array as da\nfrom dask_cuda import LocalCUDACluster\nfrom distributed import Client\n\nfrom xgboost import dask as dxgb\nfrom xgboost.callback import EvaluationMonitor\n\n\ndef _get_logger() -> logging.Logger:\n logger = logging.getLogger(\"[xgboost.dask]\")\n logger.setLevel(logging.INFO)\n if not logger.hasHandlers():\n handler = logging.StreamHandler()\n logger.addHandler(handler)\n return logger\n\n\nclass ForwardLoggingMonitor(EvaluationMonitor):\n def __init__(\n self,\n client: Client,\n rank: int = 0,\n period: int = 1,\n ) -> None:\n \"\"\"Print the evaluation result at each iteration. The default monitor in the\n native interface logs the result to the Dask scheduler process. This class can\n be used to forward the logging to the client process. Important: see the\n `client` parameter for more info.\n\n Parameters\n ----------\n client :\n Distributed client. This must be the top-level client. The class uses\n :py:meth:`distributed.Client.forward_logging` in conjunction with the Python\n :py:mod:`logging` module to forward the evaluation results to the client\n process. It has undefined behaviour if called in a nested task. As a result,\n client-side logging is not enabled by default.\n\n \"\"\"\n client.forward_logging(_get_logger().name)\n\n super().__init__(\n rank=rank,\n period=period,\n logger=lambda msg: _get_logger().info(msg.strip()),\n )\n\n\nclass WorkerEventMonitor(EvaluationMonitor):\n \"\"\"Use :py:meth:`distributed.print` to forward the log. A downside is that not only\n all clients connected to the cluster can see the log, the logs are also printed on\n the worker. If you use a local cluster, the log is duplicated.\n\n \"\"\"\n\n def __init__(self, rank: int = 0, period: int = 1) -> None:\n super().__init__(\n rank=rank, period=period, logger=lambda msg: distributed.print(msg.strip())\n )\n\n\ndef hist_train(\n client: Client, X: da.Array, y: da.Array, monitor: EvaluationMonitor\n) -> da.Array:\n # `DaskQuantileDMatrix` is used instead of `DaskDMatrix`, be careful that it can not\n # be used for anything else other than as a training DMatrix, unless a reference is\n # specified. See the `ref` argument of `DaskQuantileDMatrix`.\n dtrain = dxgb.DaskQuantileDMatrix(client, X, y)\n output = dxgb.train(\n client,\n # Make sure the device is set to CUDA.\n {\"tree_method\": \"hist\", \"device\": \"cuda\"},\n dtrain,\n num_boost_round=4,\n evals=[(dtrain, \"train\")],\n # Use the monitor to forward the log.\n callbacks=[monitor],\n # Disable the internal logging and prefer the client-side `EvaluationMonitor`.\n verbose_eval=False,\n )\n bst = output[\"booster\"]\n history = output[\"history\"]\n\n prediction = dxgb.predict(client, bst, X)\n print(\"Evaluation history:\", history)\n return prediction\n\n\nif __name__ == \"__main__\":\n # `LocalCUDACluster` is used for assigning GPU to XGBoost processes. Here\n # `n_workers` represents the number of GPUs since we use one GPU per worker process.\n with LocalCUDACluster(n_workers=2, threads_per_worker=4) as cluster:\n # Create client from cluster, set the backend to GPU array (cupy).\n with Client(cluster) as client, dask.config.set({\"array.backend\": \"cupy\"}):\n # Generate some random data for demonstration\n rng = da.random.default_rng(1)\n\n m = 2**18\n n = 100\n X = rng.uniform(size=(m, n), chunks=(128**2, -1))\n y = X.sum(axis=1)\n\n # Use forwarding, the client must be the top client.\n monitor: EvaluationMonitor = ForwardLoggingMonitor(client)\n hist_train(client, X, y, monitor).compute()\n\n # Use distributed.print, the logs in this demo are duplicated as the same\n # log is printed in all workers along with the client.\n monitor = WorkerEventMonitor()\n hist_train(client, X, y, monitor).compute()\n" }, { "path": "demo/dask/gpu_training.py", "content": "\"\"\"\nExample of training with Dask on GPU\n====================================\n\"\"\"\n\nimport dask\nimport dask_cudf\nfrom dask import array as da\nfrom dask import dataframe as dd\nfrom dask.distributed import Client\nfrom dask_cuda import LocalCUDACluster\n\nfrom xgboost import dask as dxgb\nfrom xgboost.dask import DaskDMatrix\n\n\ndef using_dask_matrix(client: Client, X: da.Array, y: da.Array) -> da.Array:\n # DaskDMatrix acts like normal DMatrix, works as a proxy for local DMatrix scatter\n # around workers.\n dtrain = DaskDMatrix(client, X, y)\n\n # Use train method from xgboost.dask instead of xgboost. This distributed version\n # of train returns a dictionary containing the resulting booster and evaluation\n # history obtained from evaluation metrics.\n output = dxgb.train(\n client,\n # Make sure the device is set to CUDA.\n {\"tree_method\": \"hist\", \"device\": \"cuda\"},\n dtrain,\n num_boost_round=4,\n evals=[(dtrain, \"train\")],\n )\n bst = output[\"booster\"]\n history = output[\"history\"]\n\n # you can pass output directly into `predict` too.\n prediction = dxgb.predict(client, bst, dtrain)\n print(\"Evaluation history:\", history)\n return prediction\n\n\ndef using_quantile_device_dmatrix(client: Client, X: da.Array, y: da.Array) -> da.Array:\n \"\"\"`DaskQuantileDMatrix` is a data type specialized for `hist` tree methods for\n reducing memory usage.\n\n .. versionadded:: 1.2.0\n\n \"\"\"\n # `DaskQuantileDMatrix` is used instead of `DaskDMatrix`, be careful that it can not\n # be used for anything else other than training unless a reference is specified. See\n # the `ref` argument of `DaskQuantileDMatrix`.\n dtrain = dxgb.DaskQuantileDMatrix(client, X, y)\n output = dxgb.train(\n client,\n # Make sure the device is set to CUDA.\n {\"tree_method\": \"hist\", \"device\": \"cuda\"},\n dtrain,\n num_boost_round=4,\n evals=[(dtrain, \"train\")],\n )\n\n prediction = dxgb.predict(client, output, X)\n return prediction\n\n\nif __name__ == \"__main__\":\n # `LocalCUDACluster` is used for assigning GPU to XGBoost processes. Here\n # `n_workers` represents the number of GPUs since we use one GPU per worker process.\n with LocalCUDACluster(n_workers=2, threads_per_worker=4) as cluster:\n # Create client from cluster, set the backend to GPU array (cupy).\n with Client(cluster) as client, dask.config.set({\"array.backend\": \"cupy\"}):\n # Generate some random data for demonstration\n rng = da.random.default_rng(1)\n\n m = 2**18\n n = 100\n X = rng.uniform(size=(m, n), chunks=(128**2, -1))\n y = X.sum(axis=1)\n\n X = dd.from_dask_array(X)\n y = dd.from_dask_array(y)\n # XGBoost can take arrays. This is to show that DataFrame uses the GPU\n # backend as well.\n assert isinstance(X, dask_cudf.DataFrame)\n assert isinstance(y, dask_cudf.Series)\n\n print(\"Using DaskQuantileDMatrix\")\n from_ddqdm = using_quantile_device_dmatrix(client, X, y).compute()\n print(\"Using DMatrix\")\n from_dmatrix = using_dask_matrix(client, X, y).compute()\n" }, { "path": "demo/dask/sklearn_cpu_training.py", "content": "\"\"\"\nUse scikit-learn regressor interface with CPU histogram tree method\n===================================================================\n\"\"\"\n\nfrom dask import array as da\nfrom dask.distributed import Client, LocalCluster\n\nfrom xgboost import dask as dxgb\n\n\ndef main(client: Client) -> dxgb.Booster:\n # generate some random data for demonstration\n n = 100\n m = 10000\n partition_size = 100\n X = da.random.random((m, n), partition_size)\n y = da.random.random(m, partition_size)\n\n regressor = dxgb.DaskXGBRegressor(verbosity=1, n_estimators=2)\n regressor.set_params(tree_method=\"hist\")\n # assigning client here is optional\n regressor.client = client\n\n regressor.fit(X, y, eval_set=[(X, y)])\n prediction = regressor.predict(X)\n\n bst = regressor.get_booster()\n history = regressor.evals_result()\n\n print(\"Evaluation history:\", history)\n # returned prediction is always a dask array.\n assert isinstance(prediction, da.Array)\n return bst # returning the trained model\n\n\nif __name__ == \"__main__\":\n # or use other clusters for scaling\n with LocalCluster(n_workers=4, threads_per_worker=1) as cluster:\n with Client(cluster) as client:\n main(client)\n" }, { "path": "demo/dask/sklearn_gpu_training.py", "content": "\"\"\"\nUse scikit-learn regressor interface with GPU histogram tree method\n===================================================================\n\"\"\"\n\nimport dask\nfrom dask import array as da\nfrom dask.distributed import Client\n\n# It's recommended to use dask_cuda for GPU assignment\nfrom dask_cuda import LocalCUDACluster\n\nfrom xgboost import dask as dxgb\n\n\ndef main(client: Client) -> dxgb.Booster:\n # Generate some random data for demonstration\n rng = da.random.default_rng(1)\n\n m = 2**18\n n = 100\n X = rng.uniform(size=(m, n), chunks=(128**2, -1))\n y = X.sum(axis=1)\n\n regressor = dxgb.DaskXGBRegressor(verbosity=1)\n # Set the device to CUDA\n regressor.set_params(tree_method=\"hist\", device=\"cuda\")\n # Assigning client here is optional\n regressor.client = client\n\n regressor.fit(X, y, eval_set=[(X, y)])\n prediction = regressor.predict(X)\n\n bst = regressor.get_booster()\n history = regressor.evals_result()\n\n print(\"Evaluation history:\", history)\n # returned prediction is always a dask array.\n assert isinstance(prediction, da.Array)\n return bst # returning the trained model\n\n\nif __name__ == \"__main__\":\n # With dask cuda, one can scale up XGBoost to arbitrary GPU clusters.\n # `LocalCUDACluster` used here is only for demonstration purpose.\n with LocalCUDACluster() as cluster:\n # Create client from cluster, set the backend to GPU array (cupy).\n with Client(cluster) as client, dask.config.set({\"array.backend\": \"cupy\"}):\n main(client)\n" }, { "path": "demo/data/README.md", "content": "This folder contains processed example dataset used by the demos.\nCopyright of the dataset belongs to the original copyright holder\n" }, { "path": "demo/data/gen_autoclaims.R", "content": "site <- 'http://cran.r-project.org'\nif (!require('dummies')) {\n install.packages('dummies', repos = site)\n}\nif (!require('insuranceData')) {\n install.packages('insuranceData', repos = site)\n}\n\nlibrary(dummies)\nlibrary(insuranceData)\n\ndata(AutoClaims)\ndata <- AutoClaims\n\ndata$STATE <- as.factor(data$STATE)\ndata$CLASS <- as.factor(data$CLASS)\ndata$GENDER <- as.factor(data$GENDER)\n\ndata.dummy <- dummy.data.frame(\n data\n , dummy.class = 'factor'\n , omit.constants = TRUE\n)\nwrite.table(\n data.dummy\n , 'autoclaims.csv'\n , sep = ','\n , row.names = FALSE\n , col.names = FALSE\n , quote = FALSE\n)\n" }, { "path": "demo/data/regression/README.md", "content": "Regression\n==========\n\nThe dataset is the [computer hardware dataset from UCI repository](https://archive.ics.uci.edu/ml/datasets/Computer+Hardware).\n" }, { "path": "demo/data/regression/machine.names", "content": "1. Title: Relative CPU Performance Data \n\n2. Source Information\n -- Creators: Phillip Ein-Dor and Jacob Feldmesser\n -- Ein-Dor: Faculty of Management; Tel Aviv University; Ramat-Aviv; \n Tel Aviv, 69978; Israel\n -- Donor: David W. Aha (aha@ics.uci.edu) (714) 856-8779 \n -- Date: October, 1987\n \n3. Past Usage:\n 1. Ein-Dor and Feldmesser (CACM 4/87, pp 308-317)\n -- Results: \n -- linear regression prediction of relative cpu performance\n -- Recorded 34% average deviation from actual values \n 2. Kibler,D. & Aha,D. (1988). Instance-Based Prediction of\n Real-Valued Attributes. In Proceedings of the CSCSI (Canadian\n AI) Conference.\n -- Results:\n -- instance-based prediction of relative cpu performance\n -- similar results; no transformations required\n - Predicted attribute: cpu relative performance (numeric)\n\n4. Relevant Information:\n -- The estimated relative performance values were estimated by the authors\n using a linear regression method. See their article (pp 308-313) for\n more details on how the relative performance values were set.\n\n5. Number of Instances: 209 \n\n6. Number of Attributes: 10 (6 predictive attributes, 2 non-predictive, \n 1 goal field, and the linear regression's guess)\n\n7. Attribute Information:\n 1. vendor name: 30 \n (adviser, amdahl,apollo, basf, bti, burroughs, c.r.d, cambex, cdc, dec, \n dg, formation, four-phase, gould, honeywell, hp, ibm, ipl, magnuson, \n microdata, nas, ncr, nixdorf, perkin-elmer, prime, siemens, sperry, \n sratus, wang)\n 2. Model Name: many unique symbols\n 3. MYCT: machine cycle time in nanoseconds (integer)\n 4. MMIN: minimum main memory in kilobytes (integer)\n 5. MMAX: maximum main memory in kilobytes (integer)\n 6. CACH: cache memory in kilobytes (integer)\n 7. CHMIN: minimum channels in units (integer)\n 8. CHMAX: maximum channels in units (integer)\n 9. PRP: published relative performance (integer)\n 10. ERP: estimated relative performance from the original article (integer)\n\n8. Missing Attribute Values: None\n\n9. Class Distribution: the class value (PRP) is continuously valued.\n PRP Value Range: Number of Instances in Range:\n 0-20 31\n 21-100 121\n 101-200 27\n 201-300 13\n 301-400 7\n 401-500 4\n 501-600 2\n above 600 4\n\nSummary Statistics:\n\t Min Max Mean SD PRP Correlation\n MCYT: 17 1500 203.8 260.3 -0.3071\n MMIN: 64 32000 2868.0 3878.7 0.7949\n MMAX: 64 64000 11796.1 11726.6 0.8630\n CACH: 0 256 25.2 40.6 0.6626\n CHMIN: 0 52 4.7 6.8 0.6089\n CHMAX: 0 176 18.2 26.0 0.6052\n PRP: 6 1150 105.6 160.8 1.0000\n ERP: 15 1238 99.3 154.8 0.9665\n\n" }, { "path": "demo/data/regression/mapfeat.py", "content": "#!/usr/bin/env python3\n\nfo = open('machine.txt', 'w')\ncnt = 6\nfmap = {}\nfor l in open('machine.data'):\n arr = l.split(',')\n fo.write(arr[8])\n for i in range(0, 6):\n fo.write(' %d:%s' % (i, arr[i + 2]))\n\n if arr[0] not in fmap:\n fmap[arr[0]] = cnt\n cnt += 1\n\n fo.write(' %d:1' % fmap[arr[0]])\n fo.write('\\n')\n\nfo.close()\n\n# create feature map for machine data\nfo = open('featmap.txt', 'w')\n# list from machine.names\nnames = [\n 'vendor', 'MYCT', 'MMIN', 'MMAX', 'CACH', 'CHMIN', 'CHMAX', 'PRP', 'ERP'\n]\n\nfor i in range(0, 6):\n fo.write('%d\\t%s\\tint\\n' % (i, names[i + 1]))\n\nfor v, k in sorted(fmap.items(), key=lambda x: x[1]):\n fo.write('%d\\tvendor=%s\\ti\\n' % (k, v))\nfo.close()\n" }, { "path": "demo/data/regression/mknfold.py", "content": "#!/usr/bin/env python3\n\nimport random\nimport sys\n\nif len(sys.argv) < 2:\n print('Usage: [nfold = 5]')\n exit(0)\n\nrandom.seed(10)\n\nk = int(sys.argv[2])\nif len(sys.argv) > 3:\n nfold = int(sys.argv[3])\nelse:\n nfold = 5\n\nfi = open(sys.argv[1], 'r')\nftr = open(sys.argv[1] + '.train', 'w')\nfte = open(sys.argv[1] + '.test', 'w')\nfor l in fi:\n if random.randint(1, nfold) == k:\n fte.write(l)\n else:\n ftr.write(l)\n\nfi.close()\nftr.close()\nfte.close()\n" }, { "path": "demo/guide-python/README.rst", "content": "XGBoost Python Feature Walkthrough\n==================================\n\n\nThis is a collection of examples for using the XGBoost Python package.\n" }, { "path": "demo/guide-python/basic_walkthrough.py", "content": "\"\"\"\nGetting started with XGBoost\n============================\n\nThis is a simple example of using the native XGBoost interface, there are other\ninterfaces in the Python package like scikit-learn interface and Dask interface.\n\n\nSee :doc:`/python/python_intro` and :doc:`/tutorials/index` for other references.\n\n\"\"\"\nimport os\nimport pickle\n\nimport numpy as np\nfrom sklearn.datasets import load_svmlight_file\n\nimport xgboost as xgb\n\n# Make sure the demo knows where to load the data.\nCURRENT_DIR = os.path.dirname(os.path.abspath(__file__))\nXGBOOST_ROOT_DIR = os.path.dirname(os.path.dirname(CURRENT_DIR))\nDEMO_DIR = os.path.join(XGBOOST_ROOT_DIR, \"demo\")\n\n# X is a scipy csr matrix, XGBoost supports many other input types,\nX, y = load_svmlight_file(os.path.join(DEMO_DIR, \"data\", \"agaricus.txt.train\"))\ndtrain = xgb.DMatrix(X, y)\n# validation set\nX_test, y_test = load_svmlight_file(os.path.join(DEMO_DIR, \"data\", \"agaricus.txt.test\"))\ndtest = xgb.DMatrix(X_test, y_test)\n\n# specify parameters via map, definition are same as c++ version\nparam = {\"max_depth\": 2, \"eta\": 1, \"objective\": \"binary:logistic\"}\n\n# specify validations set to watch performance\nwatchlist = [(dtest, \"eval\"), (dtrain, \"train\")]\n# number of boosting rounds\nnum_round = 2\nbst = xgb.train(param, dtrain, num_boost_round=num_round, evals=watchlist)\n\n# run prediction\npreds = bst.predict(dtest)\nlabels = dtest.get_label()\nprint(\n \"error=%f\"\n % (\n sum(1 for i in range(len(preds)) if int(preds[i] > 0.5) != labels[i])\n / float(len(preds))\n )\n)\nbst.save_model(\"model-0.json\")\n# dump model\nbst.dump_model(\"dump.raw.txt\")\n# dump model with feature map\nbst.dump_model(\"dump.nice.txt\", os.path.join(DEMO_DIR, \"data/featmap.txt\"))\n\n# save dmatrix into binary buffer\ndtest.save_binary(\"dtest.dmatrix\")\n# save model\nbst.save_model(\"model-1.json\")\n# load model and data in\nbst2 = xgb.Booster(model_file=\"model-1.json\")\ndtest2 = xgb.DMatrix(\"dtest.dmatrix\")\npreds2 = bst2.predict(dtest2)\n# assert they are the same\nassert np.sum(np.abs(preds2 - preds)) == 0\n\n# alternatively, you can pickle the booster\npks = pickle.dumps(bst2)\n# load model and data in\nbst3 = pickle.loads(pks)\npreds3 = bst3.predict(dtest2)\n# assert they are the same\nassert np.sum(np.abs(preds3 - preds)) == 0\n" }, { "path": "demo/guide-python/boost_from_prediction.py", "content": "\"\"\"\nDemo for boosting from prediction\n=================================\n\"\"\"\nimport os\n\nimport xgboost as xgb\n\nCURRENT_DIR = os.path.dirname(__file__)\ndtrain = xgb.DMatrix(\n os.path.join(CURRENT_DIR, \"../data/agaricus.txt.train?format=libsvm\")\n)\ndtest = xgb.DMatrix(\n os.path.join(CURRENT_DIR, \"../data/agaricus.txt.test?format=libsvm\")\n)\nwatchlist = [(dtest, \"eval\"), (dtrain, \"train\")]\n###\n# advanced: start from a initial base prediction\n#\nprint(\"start running example to start from a initial prediction\")\n# specify parameters via map, definition are same as c++ version\nparam = {\"max_depth\": 2, \"eta\": 1, \"objective\": \"binary:logistic\"}\n# train xgboost for 1 round\nbst = xgb.train(param, dtrain, 1, watchlist)\n# Note: we need the margin value instead of transformed prediction in\n# set_base_margin\n# do predict with output_margin=True, will always give you margin values\n# before logistic transformation\nptrain = bst.predict(dtrain, output_margin=True)\nptest = bst.predict(dtest, output_margin=True)\ndtrain.set_base_margin(ptrain)\ndtest.set_base_margin(ptest)\n\nprint(\"this is result of running from initial prediction\")\nbst = xgb.train(param, dtrain, 1, watchlist)\n" }, { "path": "demo/guide-python/callbacks.py", "content": "\"\"\"\nDemo for using and defining callback functions\n==============================================\n\n .. versionadded:: 1.3.0\n\"\"\"\n\nimport argparse\nimport os\nimport tempfile\nfrom typing import Dict\n\nimport numpy as np\nfrom matplotlib import pyplot as plt\nfrom sklearn.datasets import load_breast_cancer\nfrom sklearn.model_selection import train_test_split\n\nimport xgboost as xgb\n\n\nclass Plotting(xgb.callback.TrainingCallback):\n \"\"\"Plot evaluation result during training. Only for demonstration purpose as it's\n quite slow to draw using matplotlib.\n\n \"\"\"\n\n def __init__(self, rounds: int) -> None:\n self.fig = plt.figure()\n self.ax = self.fig.add_subplot(111)\n self.rounds = rounds\n self.lines: Dict[str, plt.Line2D] = {}\n self.fig.show()\n self.x = np.linspace(0, self.rounds, self.rounds)\n plt.ion()\n\n def _get_key(self, data: str, metric: str) -> str:\n return f\"{data}-{metric}\"\n\n def after_iteration(\n self, model: xgb.Booster, epoch: int, evals_log: Dict[str, dict]\n ) -> bool:\n \"\"\"Update the plot.\"\"\"\n if not self.lines:\n for data, metric in evals_log.items():\n for metric_name, log in metric.items():\n key = self._get_key(data, metric_name)\n expanded = log + [0] * (self.rounds - len(log))\n (self.lines[key],) = self.ax.plot(self.x, expanded, label=key)\n self.ax.legend()\n else:\n # https://pythonspot.com/matplotlib-update-plot/\n for data, metric in evals_log.items():\n for metric_name, log in metric.items():\n key = self._get_key(data, metric_name)\n expanded = log + [0] * (self.rounds - len(log))\n self.lines[key].set_ydata(expanded)\n self.fig.canvas.draw()\n # False to indicate training should not stop.\n return False\n\n\ndef custom_callback() -> None:\n \"\"\"Demo for defining a custom callback function that plots evaluation result during\n training.\"\"\"\n X, y = load_breast_cancer(return_X_y=True)\n X_train, X_valid, y_train, y_valid = train_test_split(X, y, random_state=0)\n\n D_train = xgb.DMatrix(X_train, y_train)\n D_valid = xgb.DMatrix(X_valid, y_valid)\n\n num_boost_round = 100\n plotting = Plotting(num_boost_round)\n\n # Pass it to the `callbacks` parameter as a list.\n xgb.train(\n {\n \"objective\": \"binary:logistic\",\n \"eval_metric\": [\"error\", \"rmse\"],\n \"tree_method\": \"hist\",\n \"device\": \"cuda\",\n },\n D_train,\n evals=[(D_train, \"Train\"), (D_valid, \"Valid\")],\n num_boost_round=num_boost_round,\n callbacks=[plotting],\n )\n\n\ndef check_point_callback() -> None:\n \"\"\"Demo for using the checkpoint callback. Custom logic for handling output is\n usually required and users are encouraged to define their own callback for\n checkpointing operations. The builtin one can be used as a starting point.\n\n \"\"\"\n # Only for demo, set a larger value (like 100) in practice as checkpointing is quite\n # slow.\n rounds = 2\n\n def check(as_pickle: bool) -> None:\n for i in range(0, 10, rounds):\n if i == 0:\n continue\n if as_pickle:\n path = os.path.join(tmpdir, \"model_\" + str(i) + \".pkl\")\n else:\n path = os.path.join(\n tmpdir,\n f\"model_{i}.{xgb.callback.TrainingCheckPoint.default_format}\",\n )\n assert os.path.exists(path)\n\n X, y = load_breast_cancer(return_X_y=True)\n m = xgb.DMatrix(X, y)\n # Check point to a temporary directory for demo\n with tempfile.TemporaryDirectory() as tmpdir:\n # Use callback class from xgboost.callback\n # Feel free to subclass/customize it to suit your need.\n check_point = xgb.callback.TrainingCheckPoint(\n directory=tmpdir, interval=rounds, name=\"model\"\n )\n xgb.train(\n {\"objective\": \"binary:logistic\"},\n m,\n num_boost_round=10,\n verbose_eval=False,\n callbacks=[check_point],\n )\n check(False)\n\n # This version of checkpoint saves everything including parameters and\n # model. See: doc/tutorials/saving_model.rst\n check_point = xgb.callback.TrainingCheckPoint(\n directory=tmpdir, interval=rounds, as_pickle=True, name=\"model\"\n )\n xgb.train(\n {\"objective\": \"binary:logistic\"},\n m,\n num_boost_round=10,\n verbose_eval=False,\n callbacks=[check_point],\n )\n check(True)\n\n\nif __name__ == \"__main__\":\n parser = argparse.ArgumentParser()\n parser.add_argument(\"--plot\", default=1, type=int)\n args = parser.parse_args()\n\n check_point_callback()\n\n if args.plot:\n custom_callback()\n" }, { "path": "demo/guide-python/cat_in_the_dat.py", "content": "\"\"\"\nTrain XGBoost with cat_in_the_dat dataset\n=========================================\n\nA simple demo for categorical data support using dataset from Kaggle categorical data\ntutorial.\n\nThe excellent tutorial is at:\nhttps://www.kaggle.com/shahules/an-overview-of-encoding-techniques\n\nAnd the data can be found at:\nhttps://www.kaggle.com/shahules/an-overview-of-encoding-techniques/data\n\n .. versionadded:: 1.6.0\n\nSee Also\n--------\n- :doc:`Tutorial `\n- :ref:`sphx_glr_python_examples_categorical.py`\n- :ref:`sphx_glr_python_examples_cat_pipeline.py`\n\n\"\"\"\n\nfrom __future__ import annotations\n\nimport os\nfrom tempfile import TemporaryDirectory\nfrom time import time\n\nimport pandas as pd\nfrom sklearn.metrics import roc_auc_score\nfrom sklearn.model_selection import train_test_split\n\nimport xgboost as xgb\n\n\ndef load_cat_in_the_dat() -> tuple[pd.DataFrame, pd.Series]:\n \"\"\"Assuming you have already downloaded the data into `input` directory.\"\"\"\n\n df_train = pd.read_csv(\"./input/cat-in-the-dat/train.csv\")\n\n print(\n \"train data set has got {} rows and {} columns\".format(\n df_train.shape[0], df_train.shape[1]\n )\n )\n X = df_train.drop([\"target\"], axis=1)\n y = df_train[\"target\"]\n\n for i in range(0, 5):\n X[\"bin_\" + str(i)] = X[\"bin_\" + str(i)].astype(\"category\")\n\n for i in range(0, 5):\n X[\"nom_\" + str(i)] = X[\"nom_\" + str(i)].astype(\"category\")\n\n for i in range(5, 10):\n X[\"nom_\" + str(i)] = X[\"nom_\" + str(i)].apply(int, base=16)\n\n for i in range(0, 6):\n X[\"ord_\" + str(i)] = X[\"ord_\" + str(i)].astype(\"category\")\n\n print(\n \"train data set has got {} rows and {} columns\".format(X.shape[0], X.shape[1])\n )\n return X, y\n\n\nparams = {\n \"tree_method\": \"hist\",\n \"device\": \"cuda\",\n \"n_estimators\": 32,\n \"colsample_bylevel\": 0.7,\n}\n\n\ndef categorical_model(X: pd.DataFrame, y: pd.Series, output_dir: str) -> None:\n \"\"\"Train using builtin categorical data support from XGBoost\"\"\"\n X_train, X_test, y_train, y_test = train_test_split(\n X, y, random_state=1994, test_size=0.2\n )\n # Be aware that the encoding for X_train and X_test are the same here. In practice,\n # we should try to use an encoder like (sklearn OrdinalEncoder) to obtain the\n # categorical values.\n\n # Specify `enable_categorical` to True.\n clf = xgb.XGBClassifier(\n **params,\n eval_metric=\"auc\",\n enable_categorical=True,\n max_cat_to_onehot=1, # We use optimal partitioning exclusively\n )\n clf.fit(X_train, y_train, eval_set=[(X_test, y_test), (X_train, y_train)])\n clf.save_model(os.path.join(output_dir, \"categorical.json\"))\n\n y_score = clf.predict_proba(X_test)[:, 1] # proba of positive samples\n auc = roc_auc_score(y_test, y_score)\n print(\"AUC of using builtin categorical data support:\", auc)\n\n\ndef onehot_encoding_model(X: pd.DataFrame, y: pd.Series, output_dir: str) -> None:\n \"\"\"Train using one-hot encoded data.\"\"\"\n X_train, X_test, y_train, y_test = train_test_split(\n X, y, random_state=42, test_size=0.2\n )\n # Specify `enable_categorical` to False as we are using encoded data.\n clf = xgb.XGBClassifier(**params, eval_metric=\"auc\", enable_categorical=False)\n clf.fit(\n X_train,\n y_train,\n eval_set=[(X_test, y_test), (X_train, y_train)],\n )\n clf.save_model(os.path.join(output_dir, \"one-hot.json\"))\n\n y_score = clf.predict_proba(X_test)[:, 1] # proba of positive samples\n auc = roc_auc_score(y_test, y_score)\n print(\"AUC of using onehot encoding:\", auc)\n\n\nif __name__ == \"__main__\":\n X, y = load_cat_in_the_dat()\n\n with TemporaryDirectory() as tmpdir:\n start = time()\n categorical_model(X, y, tmpdir)\n end = time()\n print(\"Duration:categorical\", end - start)\n\n X = pd.get_dummies(X)\n start = time()\n onehot_encoding_model(X, y, tmpdir)\n end = time()\n print(\"Duration:onehot\", end - start)\n" }, { "path": "demo/guide-python/cat_pipeline.py", "content": "\"\"\"\nFeature engineering pipeline for categorical data\n=================================================\n\nThe script showcases how to keep the categorical data encoding consistent across\ntraining and inference. There are many ways to attain the same goal, this script can be\nused as a starting point.\n\n.. versionchanged:: 3.1\n\n Start with 3.1, users don't need this for most of the cases. See :ref:`cat-recode`\n for more info.\n\nSee Also\n--------\n- :doc:`Tutorial `\n- :ref:`sphx_glr_python_examples_categorical.py`\n- :ref:`sphx_glr_python_examples_cat_in_the_dat.py`\n\n\"\"\"\n\nfrom typing import List, Tuple\n\nimport numpy as np\nimport pandas as pd\nfrom sklearn.compose import make_column_selector, make_column_transformer\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.pipeline import make_pipeline\nfrom sklearn.preprocessing import OrdinalEncoder\n\nimport xgboost as xgb\n\n\ndef make_example_data() -> Tuple[pd.DataFrame, pd.Series, List[str]]:\n \"\"\"Generate data for demo.\"\"\"\n n_samples = 2048\n rng = np.random.default_rng(1994)\n\n # We have three categorical features, while the rest are numerical.\n categorical_features = [\"brand_id\", \"retailer_id\", \"category_id\"]\n\n df = pd.DataFrame(\n np.random.randint(32, 96, size=(n_samples, 3)),\n columns=categorical_features,\n )\n\n df[\"price\"] = rng.integers(100, 200, size=(n_samples,))\n df[\"stock_status\"] = rng.choice([True, False], n_samples)\n df[\"on_sale\"] = rng.choice([True, False], n_samples)\n df[\"label\"] = rng.normal(loc=0.0, scale=1.0, size=n_samples)\n\n X = df.drop([\"label\"], axis=1)\n y = df[\"label\"]\n\n return X, y, categorical_features\n\n\ndef native() -> None:\n \"\"\"Using the native XGBoost interface.\"\"\"\n X, y, cat_feats = make_example_data()\n\n X_train, X_test, y_train, y_test = train_test_split(\n X, y, random_state=1994, test_size=0.2\n )\n\n # Create an encoder based on training data.\n enc = OrdinalEncoder(handle_unknown=\"use_encoded_value\", unknown_value=np.nan)\n enc.set_output(transform=\"pandas\")\n enc = enc.fit(X_train[cat_feats])\n\n def enc_transform(X: pd.DataFrame) -> pd.DataFrame:\n # don't make change inplace so that we can have demonstrations for encoding\n X = X.copy()\n cat_cols = enc.transform(X[cat_feats])\n for i, name in enumerate(cat_feats):\n # create pd.Series based on the encoder\n cat_cols[name] = pd.Categorical.from_codes(\n codes=cat_cols[name].astype(np.int32), categories=enc.categories_[i]\n )\n X[cat_feats] = cat_cols\n return X\n\n # Encode the data based on fitted encoder.\n X_train_enc = enc_transform(X_train)\n X_test_enc = enc_transform(X_test)\n # Train XGBoost model using the native interface.\n Xy_train = xgb.QuantileDMatrix(X_train_enc, y_train, enable_categorical=True)\n Xy_test = xgb.QuantileDMatrix(\n X_test_enc, y_test, enable_categorical=True, ref=Xy_train\n )\n booster = xgb.train({}, Xy_train)\n booster.predict(Xy_test)\n\n # Following shows that data are encoded consistently.\n\n # We first obtain result from newly encoded data\n predt0 = booster.inplace_predict(enc_transform(X_train.head(16)))\n # then we obtain result from already encoded data from training.\n predt1 = booster.inplace_predict(X_train_enc.head(16))\n\n np.testing.assert_allclose(predt0, predt1)\n\n\ndef pipeline() -> None:\n \"\"\"Using the sklearn pipeline.\"\"\"\n X, y, cat_feats = make_example_data()\n\n X_train, X_test, y_train, y_test = train_test_split(\n X, y, random_state=3, test_size=0.2\n )\n\n enc = make_column_transformer(\n (\n OrdinalEncoder(handle_unknown=\"use_encoded_value\", unknown_value=np.nan),\n # all categorical feature names end with \"_id\"\n make_column_selector(pattern=\".*_id\"),\n ),\n remainder=\"passthrough\",\n verbose_feature_names_out=False,\n )\n # No need to set pandas output, we use `feature_types` to indicate the type of\n # features.\n\n # enc.set_output(transform=\"pandas\")\n\n feature_types = [\"c\" if fn in cat_feats else \"q\" for fn in X_train.columns]\n reg = xgb.XGBRegressor(\n feature_types=feature_types, enable_categorical=True, n_estimators=10\n )\n p = make_pipeline(enc, reg)\n p.fit(X_train, y_train)\n # check XGBoost is using the feature type correctly.\n model_types = reg.get_booster().feature_types\n assert model_types is not None\n for a, b in zip(model_types, feature_types):\n assert a == b\n\n # Following shows that data are encoded consistently.\n\n # We first create a slice of data that doesn't contain all the categories\n predt0 = p.predict(X_train.iloc[:16, :])\n # Then we use the dataframe that contains all the categories\n predt1 = p.predict(X_train)[:16]\n\n # The resulting encoding is the same\n np.testing.assert_allclose(predt0, predt1)\n\n\nif __name__ == \"__main__\":\n pipeline()\n native()\n" }, { "path": "demo/guide-python/categorical.py", "content": "\"\"\"\nGetting started with categorical data\n=====================================\n\nExperimental support for categorical data.\n\nIn before, users need to run an encoder themselves before passing the data into XGBoost,\nwhich creates a sparse matrix and potentially increase memory usage. This demo\nshowcases the experimental categorical data support, more advanced features are planned.\n\n .. versionadded:: 1.5.0\n\nSee Also\n--------\n- :doc:`Tutorial `\n- :ref:`sphx_glr_python_examples_cat_in_the_dat.py`\n- :ref:`sphx_glr_python_examples_cat_pipeline.py`\n\n\"\"\"\n\nfrom typing import Tuple\n\nimport numpy as np\nimport pandas as pd\n\nimport xgboost as xgb\n\n\ndef make_categorical(\n n_samples: int, n_features: int, n_categories: int, onehot: bool\n) -> Tuple[pd.DataFrame, pd.Series]:\n \"\"\"Make some random data for demo.\"\"\"\n rng = np.random.RandomState(1994)\n\n pd_dict = {}\n for i in range(n_features + 1):\n c = rng.randint(low=0, high=n_categories, size=n_samples)\n pd_dict[str(i)] = pd.Series(c, dtype=np.int64)\n\n df = pd.DataFrame(pd_dict)\n label = df.iloc[:, 0]\n df = df.iloc[:, 1:]\n for i in range(0, n_features):\n label += df.iloc[:, i]\n label += 1\n\n df = df.astype(\"category\")\n categories = np.arange(0, n_categories)\n for col in df.columns:\n df[col] = df[col].cat.set_categories(categories)\n\n if onehot:\n return pd.get_dummies(df), label\n return df, label\n\n\ndef main() -> None:\n # Use builtin categorical data support\n\n # For scikit-learn interface, the input data should be pandas DataFrame or cudf\n # DataFrame with categorical features. If an numpy/cupy array is used instead, the\n # `feature_types` for `XGBRegressor` should be set accordingly.\n X, y = make_categorical(100, 10, 4, False)\n # Specify `enable_categorical` to True, also we use onehot-encoding-based split here\n # for demonstration. For details see the document of `max_cat_to_onehot`.\n reg = xgb.XGBRegressor(\n tree_method=\"hist\", enable_categorical=True, max_cat_to_onehot=5, device=\"cuda\"\n )\n reg.fit(X, y, eval_set=[(X, y)])\n\n # Pass in already encoded data\n X_enc, y_enc = make_categorical(100, 10, 4, True)\n reg_enc = xgb.XGBRegressor(tree_method=\"hist\", device=\"cuda\")\n reg_enc.fit(X_enc, y_enc, eval_set=[(X_enc, y_enc)])\n\n reg_results = np.array(reg.evals_result()[\"validation_0\"][\"rmse\"])\n reg_enc_results = np.array(reg_enc.evals_result()[\"validation_0\"][\"rmse\"])\n\n # Check that they have same results\n np.testing.assert_allclose(reg_results, reg_enc_results)\n\n # Convert to DMatrix for SHAP value\n booster: xgb.Booster = reg.get_booster()\n m = xgb.DMatrix(X, enable_categorical=True) # specify categorical data support.\n SHAP = booster.predict(m, pred_contribs=True)\n margin = booster.predict(m, output_margin=True)\n np.testing.assert_allclose(\n np.sum(SHAP, axis=len(SHAP.shape) - 1), margin, rtol=1e-3\n )\n\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "demo/guide-python/continuation.py", "content": "\"\"\"\nDemo for training continuation\n==============================\n\"\"\"\n\nimport os\nimport pickle\nimport tempfile\n\nfrom sklearn.datasets import load_breast_cancer\n\nimport xgboost\n\n\ndef training_continuation(tmpdir: str, use_pickle: bool) -> None:\n \"\"\"Basic training continuation.\"\"\"\n # Train 128 iterations in 1 session\n X, y = load_breast_cancer(return_X_y=True)\n clf = xgboost.XGBClassifier(n_estimators=128, eval_metric=\"logloss\")\n clf.fit(X, y, eval_set=[(X, y)])\n print(\"Total boosted rounds:\", clf.get_booster().num_boosted_rounds())\n\n # Train 128 iterations in 2 sessions, with the first one runs for 32 iterations and\n # the second one runs for 96 iterations\n clf = xgboost.XGBClassifier(n_estimators=32, eval_metric=\"logloss\")\n clf.fit(X, y, eval_set=[(X, y)])\n assert clf.get_booster().num_boosted_rounds() == 32\n\n # load back the model, this could be a checkpoint\n if use_pickle:\n path = os.path.join(tmpdir, \"model-first-32.pkl\")\n with open(path, \"wb\") as fd:\n pickle.dump(clf, fd)\n with open(path, \"rb\") as fd:\n loaded = pickle.load(fd)\n else:\n path = os.path.join(tmpdir, \"model-first-32.json\")\n clf.save_model(path)\n loaded = xgboost.XGBClassifier()\n loaded.load_model(path)\n\n clf = xgboost.XGBClassifier(n_estimators=128 - 32, eval_metric=\"logloss\")\n clf.fit(X, y, eval_set=[(X, y)], xgb_model=loaded)\n\n print(\"Total boosted rounds:\", clf.get_booster().num_boosted_rounds())\n\n assert clf.get_booster().num_boosted_rounds() == 128\n\n\ndef training_continuation_early_stop(tmpdir: str, use_pickle: bool) -> None:\n \"\"\"Training continuation with early stopping.\"\"\"\n early_stopping_rounds = 5\n early_stop = xgboost.callback.EarlyStopping(\n rounds=early_stopping_rounds, save_best=True\n )\n n_estimators = 512\n\n X, y = load_breast_cancer(return_X_y=True)\n clf = xgboost.XGBClassifier(\n n_estimators=n_estimators, eval_metric=\"logloss\", callbacks=[early_stop]\n )\n clf.fit(X, y, eval_set=[(X, y)])\n print(\"Total boosted rounds:\", clf.get_booster().num_boosted_rounds())\n best = clf.best_iteration\n\n # Train 512 iterations in 2 sessions, with the first one runs for 128 iterations and\n # the second one runs until early stop.\n clf = xgboost.XGBClassifier(\n n_estimators=128, eval_metric=\"logloss\", callbacks=[early_stop]\n )\n # Reinitialize the early stop callback\n early_stop = xgboost.callback.EarlyStopping(\n rounds=early_stopping_rounds, save_best=True\n )\n clf.set_params(callbacks=[early_stop])\n clf.fit(X, y, eval_set=[(X, y)])\n assert clf.get_booster().num_boosted_rounds() == 128\n\n # load back the model, this could be a checkpoint\n if use_pickle:\n path = os.path.join(tmpdir, \"model-first-128.pkl\")\n with open(path, \"wb\") as fd:\n pickle.dump(clf, fd)\n with open(path, \"rb\") as fd:\n loaded = pickle.load(fd)\n else:\n path = os.path.join(tmpdir, \"model-first-128.json\")\n clf.save_model(path)\n loaded = xgboost.XGBClassifier()\n loaded.load_model(path)\n\n early_stop = xgboost.callback.EarlyStopping(\n rounds=early_stopping_rounds, save_best=True\n )\n clf = xgboost.XGBClassifier(\n n_estimators=n_estimators - 128, eval_metric=\"logloss\", callbacks=[early_stop]\n )\n clf.fit(\n X,\n y,\n eval_set=[(X, y)],\n xgb_model=loaded,\n )\n\n print(\"Total boosted rounds:\", clf.get_booster().num_boosted_rounds())\n assert clf.best_iteration == best\n\n\nif __name__ == \"__main__\":\n with tempfile.TemporaryDirectory() as tmpdir:\n training_continuation_early_stop(tmpdir, False)\n training_continuation_early_stop(tmpdir, True)\n\n training_continuation(tmpdir, True)\n training_continuation(tmpdir, False)\n" }, { "path": "demo/guide-python/cover_type.py", "content": "\"\"\"\nUsing xgboost on GPU devices\n============================\n\nShows how to train a model on the `forest cover type\n`_ dataset using GPU\nacceleration. The forest cover type dataset has 581,012 rows and 54 features, making it\ntime consuming to process. We compare the run-time and accuracy of the GPU and CPU\nhistogram algorithms.\n\nIn addition, The demo showcases using GPU with other GPU-related libraries including\ncupy and cuml. These libraries are not strictly required.\n\n\"\"\"\nimport time\n\nimport cupy as cp\nfrom cuml.model_selection import train_test_split\nfrom sklearn.datasets import fetch_covtype\n\nimport xgboost as xgb\n\n# Fetch dataset using sklearn\nX, y = fetch_covtype(return_X_y=True)\nX = cp.array(X)\ny = cp.array(y)\ny -= y.min()\n\n# Create 0.75/0.25 train/test split\nX_train, X_test, y_train, y_test = train_test_split(\n X, y, test_size=0.25, train_size=0.75, random_state=42\n)\n\n# Specify sufficient boosting iterations to reach a minimum\nnum_round = 3000\n\n# Leave most parameters as default\nclf = xgb.XGBClassifier(device=\"cuda\", n_estimators=num_round)\n# Train model\nstart = time.time()\nclf.fit(X_train, y_train, eval_set=[(X_test, y_test)])\ngpu_res = clf.evals_result()\nprint(\"GPU Training Time: %s seconds\" % (str(time.time() - start)))\n\n# Repeat for CPU algorithm\nclf = xgb.XGBClassifier(device=\"cpu\", n_estimators=num_round)\nstart = time.time()\nclf.fit(X_train, y_train, eval_set=[(X_test, y_test)])\ncpu_res = clf.evals_result()\nprint(\"CPU Training Time: %s seconds\" % (str(time.time() - start)))\n" }, { "path": "demo/guide-python/cross_validation.py", "content": "\"\"\"\nDemo for using cross validation\n===============================\n\"\"\"\n\nimport os\nfrom typing import Any, Dict, Tuple\n\nimport numpy as np\n\nimport xgboost as xgb\n\n# load data in do training\nCURRENT_DIR = os.path.dirname(__file__)\ndtrain = xgb.DMatrix(\n os.path.join(CURRENT_DIR, \"../data/agaricus.txt.train?format=libsvm\")\n)\nparam = {\"max_depth\": 2, \"eta\": 1, \"objective\": \"binary:logistic\"}\nnum_round = 2\n\nprint(\"running cross validation\")\n# do cross validation, this will print result out as\n# [iteration] metric_name:mean_value+std_value\n# std_value is standard deviation of the metric\nxgb.cv(\n param,\n dtrain,\n num_round,\n nfold=5,\n metrics={\"error\"},\n seed=0,\n callbacks=[xgb.callback.EvaluationMonitor(show_stdv=True)],\n)\n\nprint(\"running cross validation, disable standard deviation display\")\n# do cross validation, this will print result out as\n# [iteration] metric_name:mean_value\nres = xgb.cv(\n param,\n dtrain,\n num_boost_round=10,\n nfold=5,\n metrics={\"error\"},\n seed=0,\n callbacks=[\n xgb.callback.EvaluationMonitor(show_stdv=False),\n xgb.callback.EarlyStopping(3),\n ],\n)\nprint(res)\nprint(\"running cross validation, with preprocessing function\")\n\n\n# define the preprocessing function\n# used to return the preprocessed training, test data, and parameter\n# we can use this to do weight rescale, etc.\n# as a example, we try to set scale_pos_weight\ndef fpreproc(\n dtrain: xgb.DMatrix, dtest: xgb.DMatrix, param: Any\n) -> Tuple[xgb.DMatrix, xgb.DMatrix, Dict[str, Any]]:\n label = dtrain.get_label()\n ratio = float(np.sum(label == 0)) / np.sum(label == 1)\n param[\"scale_pos_weight\"] = ratio\n return (dtrain, dtest, param)\n\n\n# do cross validation, for each fold\n# the dtrain, dtest, param will be passed into fpreproc\n# then the return value of fpreproc will be used to generate\n# results of that fold\nxgb.cv(param, dtrain, num_round, nfold=5, metrics={\"auc\"}, seed=0, fpreproc=fpreproc)\n\n###\n# you can also do cross validation with customized loss function\n# See custom_objective.py\n##\nprint(\"running cross validation, with customized loss function\")\n\n\ndef logregobj(preds: np.ndarray, dtrain: xgb.DMatrix) -> Tuple[np.ndarray, np.ndarray]:\n labels = dtrain.get_label()\n preds = 1.0 / (1.0 + np.exp(-preds))\n grad = preds - labels\n hess = preds * (1.0 - preds)\n return grad, hess\n\n\ndef evalerror(preds: np.ndarray, dtrain: xgb.DMatrix) -> Tuple[str, float]:\n labels = dtrain.get_label()\n preds = 1.0 / (1.0 + np.exp(-preds))\n return \"error\", float(sum(labels != (preds > 0.0))) / len(labels)\n\n\nparam = {\"max_depth\": 2, \"eta\": 1}\n# train with customized objective\nxgb.cv(\n param, dtrain, num_round, nfold=5, seed=0, obj=logregobj, custom_metric=evalerror\n)\n" }, { "path": "demo/guide-python/custom_rmsle.py", "content": "\"\"\"\nDemo for defining a custom regression objective and metric\n==========================================================\n\nDemo for defining customized metric and objective. Notice that for simplicity reason\nweight is not used in following example. In this script, we implement the Squared Log\nError (SLE) objective and RMSLE metric as customized functions, then compare it with\nnative implementation in XGBoost.\n\nSee :doc:`/tutorials/custom_metric_obj` for a step by step walkthrough, with other\ndetails.\n\nThe `SLE` objective reduces impact of outliers in training dataset, hence here we also\ncompare its performance with standard squared error.\n\n\"\"\"\nimport argparse\nfrom time import time\nfrom typing import Dict, List, Tuple\n\nimport numpy as np\nfrom matplotlib import pyplot as plt\n\nimport xgboost as xgb\n\n# shape of generated data.\nkRows = 4096\nkCols = 16\n\nkOutlier = 10000 # mean of generated outliers\nkNumberOfOutliers = 64\n\nkRatio = 0.7\nkSeed = 1994\n\nkBoostRound = 20\n\nnp.random.seed(seed=kSeed)\n\n\ndef generate_data() -> Tuple[xgb.DMatrix, xgb.DMatrix]:\n '''Generate data containing outliers.'''\n x = np.random.randn(kRows, kCols)\n y = np.random.randn(kRows)\n y += np.abs(np.min(y))\n\n # Create outliers\n for i in range(0, kNumberOfOutliers):\n ind = np.random.randint(0, len(y)-1)\n y[ind] += np.random.randint(0, kOutlier)\n\n train_portion = int(kRows * kRatio)\n\n # rmsle requires all label be greater than -1.\n assert np.all(y > -1.0)\n\n train_x: np.ndarray = x[: train_portion]\n train_y: np.ndarray = y[: train_portion]\n dtrain = xgb.DMatrix(train_x, label=train_y)\n\n test_x = x[train_portion:]\n test_y = y[train_portion:]\n dtest = xgb.DMatrix(test_x, label=test_y)\n return dtrain, dtest\n\n\ndef native_rmse(dtrain: xgb.DMatrix,\n dtest: xgb.DMatrix) -> Dict[str, Dict[str, List[float]]]:\n '''Train using native implementation of Root Mean Squared Loss.'''\n print('Squared Error')\n squared_error = {\n 'objective': 'reg:squarederror',\n 'eval_metric': 'rmse',\n 'tree_method': 'hist',\n 'seed': kSeed\n }\n start = time()\n results: Dict[str, Dict[str, List[float]]] = {}\n xgb.train(squared_error,\n dtrain=dtrain,\n num_boost_round=kBoostRound,\n evals=[(dtrain, 'dtrain'), (dtest, 'dtest')],\n evals_result=results)\n print('Finished Squared Error in:', time() - start, '\\n')\n return results\n\n\ndef native_rmsle(dtrain: xgb.DMatrix,\n dtest: xgb.DMatrix) -> Dict[str, Dict[str, List[float]]]:\n '''Train using native implementation of Squared Log Error.'''\n print('Squared Log Error')\n results: Dict[str, Dict[str, List[float]]] = {}\n squared_log_error = {\n 'objective': 'reg:squaredlogerror',\n 'eval_metric': 'rmsle',\n 'tree_method': 'hist',\n 'seed': kSeed\n }\n start = time()\n xgb.train(squared_log_error,\n dtrain=dtrain,\n num_boost_round=kBoostRound,\n evals=[(dtrain, 'dtrain'), (dtest, 'dtest')],\n evals_result=results)\n print('Finished Squared Log Error in:', time() - start)\n return results\n\n\ndef py_rmsle(dtrain: xgb.DMatrix, dtest: xgb.DMatrix) -> Dict:\n '''Train using Python implementation of Squared Log Error.'''\n def gradient(predt: np.ndarray, dtrain: xgb.DMatrix) -> np.ndarray:\n '''Compute the gradient squared log error.'''\n y = dtrain.get_label()\n return (np.log1p(predt) - np.log1p(y)) / (predt + 1)\n\n def hessian(predt: np.ndarray, dtrain: xgb.DMatrix) -> np.ndarray:\n '''Compute the hessian for squared log error.'''\n y = dtrain.get_label()\n return ((-np.log1p(predt) + np.log1p(y) + 1) /\n np.power(predt + 1, 2))\n\n def squared_log(predt: np.ndarray,\n dtrain: xgb.DMatrix) -> Tuple[np.ndarray, np.ndarray]:\n '''Squared Log Error objective. A simplified version for RMSLE used as\n objective function.\n\n :math:`\\frac{1}{2}[log(pred + 1) - log(label + 1)]^2`\n\n '''\n predt[predt < -1] = -1 + 1e-6\n grad = gradient(predt, dtrain)\n hess = hessian(predt, dtrain)\n return grad, hess\n\n def rmsle(predt: np.ndarray, dtrain: xgb.DMatrix) -> Tuple[str, float]:\n ''' Root mean squared log error metric.\n\n :math:`\\\\sqrt{\\frac{1}{N}[log(pred + 1) - log(label + 1)]^2}`\n '''\n y = dtrain.get_label()\n predt[predt < -1] = -1 + 1e-6\n elements = np.power(np.log1p(y) - np.log1p(predt), 2)\n return 'PyRMSLE', float(np.sqrt(np.sum(elements) / len(y)))\n\n results: Dict[str, Dict[str, List[float]]] = {}\n xgb.train({'tree_method': 'hist', 'seed': kSeed,\n 'disable_default_eval_metric': 1},\n dtrain=dtrain,\n num_boost_round=kBoostRound,\n obj=squared_log,\n custom_metric=rmsle,\n evals=[(dtrain, 'dtrain'), (dtest, 'dtest')],\n evals_result=results)\n\n return results\n\n\ndef plot_history(\n rmse_evals: Dict[str, Dict],\n rmsle_evals: Dict[str, Dict],\n py_rmsle_evals: Dict[str, Dict]\n) -> None:\n fig, axs = plt.subplots(3, 1)\n assert isinstance(axs, np.ndarray)\n ax0 = axs[0]\n ax1 = axs[1]\n ax2 = axs[2]\n\n x = np.arange(0, kBoostRound, 1)\n\n ax0.plot(x, rmse_evals['dtrain']['rmse'], label='train-RMSE')\n ax0.plot(x, rmse_evals['dtest']['rmse'], label='test-RMSE')\n ax0.legend()\n\n ax1.plot(x, rmsle_evals['dtrain']['rmsle'], label='train-native-RMSLE')\n ax1.plot(x, rmsle_evals['dtest']['rmsle'], label='test-native-RMSLE')\n ax1.legend()\n\n ax2.plot(x, py_rmsle_evals['dtrain']['PyRMSLE'], label='train-PyRMSLE')\n ax2.plot(x, py_rmsle_evals['dtest']['PyRMSLE'], label='test-PyRMSLE')\n ax2.legend()\n\n\ndef main(args: argparse.Namespace) -> None:\n dtrain, dtest = generate_data()\n rmse_evals = native_rmse(dtrain, dtest)\n rmsle_evals = native_rmsle(dtrain, dtest)\n py_rmsle_evals = py_rmsle(dtrain, dtest)\n\n if args.plot != 0:\n plot_history(rmse_evals, rmsle_evals, py_rmsle_evals)\n plt.show()\n\n\nif __name__ == \"__main__\":\n parser = argparse.ArgumentParser(\n description='Arguments for custom RMSLE objective function demo.')\n parser.add_argument(\n '--plot',\n type=int,\n default=1,\n help='Set to 0 to disable plotting the evaluation history.')\n args = parser.parse_args()\n main(args)\n" }, { "path": "demo/guide-python/custom_softmax.py", "content": "\"\"\"\nDemo for creating customized multi-class objective function\n===========================================================\n\nThis demo is only applicable after (excluding) XGBoost 1.0.0, as before this version\nXGBoost returns transformed prediction for multi-class objective function. More details\nin comments.\n\nSee :doc:`/tutorials/custom_metric_obj` and :doc:`/tutorials/advanced_custom_obj` for\ndetailed tutorial and notes.\n\n\"\"\"\n\n# pylint: disable=missing-function-docstring,redefined-outer-name,unused-variable\n\nimport argparse\nfrom typing import Dict, Tuple\n\nimport numpy as np\nimport xgboost as xgb\nfrom matplotlib import pyplot as plt\n\nnp.random.seed(1994)\n\nkRows = 100\nkCols = 10\nkClasses = 4 # number of classes\n\nkRounds = 10 # number of boosting rounds.\n\n# Generate some random data for demo.\nX = np.random.randn(kRows, kCols)\ny = np.random.randint(0, 4, size=kRows)\n\nm = xgb.DMatrix(X, y)\n\n\ndef softmax(x: np.ndarray) -> np.ndarray:\n \"\"\"Softmax function with x as input vector.\"\"\"\n e = np.exp(x)\n return e / np.sum(e)\n\n\ndef softprob_obj(predt: np.ndarray, data: xgb.DMatrix) -> Tuple[np.ndarray, np.ndarray]:\n \"\"\"Loss function. Computing the gradient and upper bound on the\n Hessian with a diagonal structure for XGBoost (note that this is\n not the true Hessian).\n Reimplements the `multi:softprob` inside XGBoost.\n\n \"\"\"\n labels = data.get_label()\n if data.get_weight().size == 0:\n # Use 1 as weight if we don't have custom weight.\n weights = np.ones(kRows, dtype=float)\n else:\n weights = data.get_weight()\n\n # The prediction is of shape (rows, classes), each element in a row\n # represents a raw prediction (leaf weight, hasn't gone through softmax\n # yet). In XGBoost 1.0.0, the prediction is transformed by a softmax\n # function, fixed in later versions.\n assert predt.shape == (kRows, kClasses)\n\n grad = np.zeros((kRows, kClasses), dtype=float)\n hess = np.zeros((kRows, kClasses), dtype=float)\n\n eps = 1e-6\n\n # compute the gradient and hessian upper bound, slow iterations in Python, only\n # suitable for demo. Also the one in native XGBoost core is more robust to\n # numeric overflow as we don't do anything to mitigate the `exp` in\n # `softmax` here.\n for r in range(predt.shape[0]):\n target = int(labels[r])\n weight = float(weights[r])\n p = softmax(predt[r, :])\n for c in range(predt.shape[1]):\n assert 0 <= target < kClasses\n pc = float(p[c])\n g = pc - 1.0 if c == target else pc\n g = g * weight\n h = max(2.0 * pc * (1.0 - pc) * weight, eps)\n grad[r, c] = g\n hess[r, c] = h\n\n # After 2.1.0, pass the gradient as it is.\n return grad, hess\n\n\ndef predict(booster: xgb.Booster, X: xgb.DMatrix) -> np.ndarray:\n \"\"\"A customized prediction function that converts raw prediction to\n target class.\n\n \"\"\"\n # Output margin means we want to obtain the raw prediction obtained from\n # tree leaf weight.\n predt = booster.predict(X, output_margin=True)\n out = np.zeros(kRows)\n for r in range(predt.shape[0]):\n # the class with maximum prob (not strictly prob as it haven't gone\n # through softmax yet so it doesn't sum to 1, but result is the same\n # for argmax).\n i = np.argmax(predt[r])\n out[r] = i\n return out\n\n\ndef merror(predt: np.ndarray, dtrain: xgb.DMatrix) -> Tuple[str, np.float64]:\n y = dtrain.get_label()\n # Like custom objective, the predt is untransformed leaf weight when custom\n # objective is provided.\n\n # With the use of `custom_metric` parameter in train function, custom metric\n # receives raw input only when custom objective is also being used. Otherwise\n # custom metric will receive transformed prediction.\n assert predt.shape == (kRows, kClasses)\n out = np.zeros(kRows)\n for r in range(predt.shape[0]):\n i = np.argmax(predt[r])\n out[r] = i\n\n assert y.shape == out.shape\n\n errors = np.zeros(kRows)\n errors[y != out] = 1.0\n return \"PyMError\", np.sum(errors) / kRows\n\n\ndef plot_history(\n custom_results: Dict[str, Dict], native_results: Dict[str, Dict]\n) -> None:\n axs: np.ndarray\n fig, axs = plt.subplots(2, 1) # type: ignore\n ax0 = axs[0]\n ax1 = axs[1]\n\n pymerror = custom_results[\"train\"][\"PyMError\"]\n merror = native_results[\"train\"][\"merror\"]\n\n x = np.arange(0, kRounds, 1)\n ax0.plot(x, pymerror, label=\"Custom objective\")\n ax0.legend()\n ax1.plot(x, merror, label=\"multi:softmax\")\n ax1.legend()\n\n plt.show()\n\n\ndef main(args: argparse.Namespace) -> None:\n # Since 3.1, XGBoost can estimate the base_score automatically for built-in\n # multi-class objectives.\n #\n # We explicitly specify it here to disable the automatic estimation to have a proper\n # comparison between the custom implementation and the built-in implementation.\n intercept = np.full(shape=(kClasses,), fill_value=1 / kClasses)\n\n custom_results: Dict[str, Dict] = {}\n # Use our custom objective function\n booster_custom = xgb.train(\n {\n \"num_class\": kClasses,\n \"base_score\": intercept,\n \"disable_default_eval_metric\": True,\n },\n m,\n num_boost_round=kRounds,\n obj=softprob_obj,\n custom_metric=merror,\n evals_result=custom_results,\n evals=[(m, \"train\")],\n )\n\n predt_custom = predict(booster_custom, m)\n\n native_results: Dict[str, Dict] = {}\n # Use the same objective function defined in XGBoost.\n booster_native = xgb.train(\n {\n \"num_class\": kClasses,\n \"base_score\": intercept,\n \"objective\": \"multi:softmax\",\n \"eval_metric\": \"merror\",\n },\n m,\n num_boost_round=kRounds,\n evals_result=native_results,\n evals=[(m, \"train\")],\n )\n predt_native = booster_native.predict(m)\n\n # We are reimplementing the loss function in XGBoost, so it should\n # be the same for normal cases.\n assert np.all(predt_custom == predt_native)\n np.testing.assert_allclose(\n custom_results[\"train\"][\"PyMError\"], native_results[\"train\"][\"merror\"]\n )\n\n if args.plot != 0:\n plot_history(custom_results, native_results)\n\n\nif __name__ == \"__main__\":\n parser = argparse.ArgumentParser(\n description=\"Arguments for custom softmax objective function demo.\"\n )\n parser.add_argument(\n \"--plot\",\n type=int,\n default=1,\n help=\"Set to 0 to disable plotting the evaluation history.\",\n )\n args = parser.parse_args()\n main(args)\n" }, { "path": "demo/guide-python/distributed_extmem_basic.py", "content": "\"\"\"\nExperimental support for distributed training with external memory\n==================================================================\n\n .. versionadded:: 3.0.0\n\nSee :doc:`the tutorial ` for more details. To run the\nexample, following packages in addition to XGBoost native dependencies are required:\n\n- scikit-learn\n- loky\n\nIf `device` is `cuda`, following are also needed:\n\n- cupy\n- cuda-python\n- pyhwloc\n\n\"\"\"\n\nimport argparse\nimport multiprocessing as mp\nimport os\nimport sys\nimport tempfile\nimport traceback\nfrom functools import partial, update_wrapper, wraps\nfrom typing import TYPE_CHECKING, Callable, List, ParamSpec, Tuple, TypeVar\n\nimport numpy as np\nimport xgboost\nfrom loky import get_reusable_executor\nfrom sklearn.datasets import make_regression\nfrom xgboost import collective as coll\nfrom xgboost.tracker import RabitTracker\n\nif TYPE_CHECKING:\n from cuda.bindings.runtime import cudaError_t\n\n\ndef _checkcu(status: \"cudaError_t\") -> None:\n import cuda.bindings.runtime as cudart\n\n if status != cudart.cudaError_t.cudaSuccess:\n raise RuntimeError(cudart.cudaGetErrorString(status))\n\n\ndef device_mem_total() -> int:\n \"\"\"The total number of bytes of memory this GPU has.\"\"\"\n import cuda.bindings.runtime as cudart\n\n status, _, total = cudart.cudaMemGetInfo()\n _checkcu(status)\n return total\n\n\ndef make_batches(\n n_samples_per_batch: int, n_features: int, n_batches: int, tmpdir: str, rank: int\n) -> List[Tuple[str, str]]:\n \"\"\"Create multiple batches of synthetic data and return their file paths.\"\"\"\n files: List[Tuple[str, str]] = []\n rng = np.random.RandomState(rank)\n for i in range(n_batches):\n X, y = make_regression(n_samples_per_batch, n_features, random_state=rng)\n X_path = os.path.join(tmpdir, f\"X-r{rank}-{i}.npy\")\n y_path = os.path.join(tmpdir, f\"y-r{rank}-{i}.npy\")\n np.save(X_path, X)\n np.save(y_path, y)\n files.append((X_path, y_path))\n return files\n\n\nclass Iterator(xgboost.DataIter):\n \"\"\"A custom iterator for loading files in batches.\"\"\"\n\n def __init__(self, device: str, file_paths: List[Tuple[str, str]]) -> None:\n self.device = device\n\n self._file_paths = file_paths\n self._it = 0\n # XGBoost will generate some cache files under the current directory with the\n # prefix \"cache\"\n super().__init__(cache_prefix=os.path.join(\".\", \"cache\"))\n\n def load_file(self) -> Tuple[np.ndarray, np.ndarray]:\n \"\"\"Load a single batch of data.\"\"\"\n X_path, y_path = self._file_paths[self._it]\n # When the `ExtMemQuantileDMatrix` is used, the device must match. GPU cannot\n # consume CPU input data and vice-versa.\n if self.device == \"cpu\":\n X = np.load(X_path)\n y = np.load(y_path)\n else:\n import cupy as cp\n\n X = cp.load(X_path)\n y = cp.load(y_path)\n\n assert X.shape[0] == y.shape[0]\n return X, y\n\n def next(self, input_data: Callable) -> bool:\n \"\"\"Advance the iterator by 1 step and pass the data to XGBoost. This function\n is called by XGBoost during the construction of ``DMatrix``\n\n \"\"\"\n if self._it == len(self._file_paths):\n # return False to let XGBoost know this is the end of iteration\n return False\n\n # input_data is a keyword-only function passed in by XGBoost and has the similar\n # signature to the ``DMatrix`` constructor.\n X, y = self.load_file()\n input_data(data=X, label=y)\n self._it += 1\n return True\n\n def reset(self) -> None:\n \"\"\"Reset the iterator to its beginning\"\"\"\n self._it = 0\n\n\ndef setup_numa() -> None:\n \"\"\"Set correct NUMA binding for GPU-based external memory training.\"\"\"\n from pyhwloc import from_this_system\n from pyhwloc.cuda_runtime import get_device\n from pyhwloc.topology import MemBindFlags, MemBindPolicy, TypeFilter\n\n devices = os.getenv(\"CUDA_VISIBLE_DEVICES\", None)\n assert devices is not None, \"CUDA_VISIBLE_DEVICES must be set.\"\n\n with from_this_system().set_io_types_filter(TypeFilter.KEEP_ALL) as topo:\n # Get CPU affinity for this GPU. Device ordinal 0 is used because\n # CUDA_VISIBLE_DEVICES has already reordered the devices.\n dev = get_device(topo, device=0)\n cpuset = dev.get_affinity()\n\n # Set CPU binding\n topo.set_cpubind(cpuset)\n # Set memory binding with STRICT policy - ensures all memory allocations come\n # from the local NUMA node. hwloc determines the NUMA nodes from cpuset.\n topo.set_membind(cpuset, MemBindPolicy.BIND, MemBindFlags.STRICT)\n\n\ndef setup_async_pool() -> None:\n \"\"\"Setup CUDA async pool. As an alternative, the RMM plugin can be used as well.\n This is the same as using the `CudaAsyncMemoryResource` from RMM, but without the\n RMM dependency.\n\n .. versionadded:: 3.2.0\n\n \"\"\"\n import cuda.bindings.runtime as cudart\n from cuda.bindings import driver\n from cupy.cuda import MemoryAsyncPool\n\n status, dft_pool = cudart.cudaDeviceGetDefaultMemPool(0)\n _checkcu(status)\n\n total = device_mem_total()\n\n v = driver.cuuint64_t(int(total * 0.9))\n (status,) = cudart.cudaMemPoolSetAttribute(\n dft_pool,\n cudart.cudaMemPoolAttr.cudaMemPoolAttrReleaseThreshold,\n v,\n )\n _checkcu(status)\n # Set the allocator for cupy as well.\n import cupy as cp\n\n cp.cuda.set_allocator(MemoryAsyncPool().malloc)\n\n\nR = TypeVar(\"R\")\nP = ParamSpec(\"P\")\n\n\ndef try_run(fn: Callable[P, R]) -> Callable[P, R]:\n \"\"\"Loky aborts the process without printing out any error message if there's an\n exception.\n\n \"\"\"\n\n @wraps(fn)\n def inner(*args: P.args, **kwargs: P.kwargs) -> R:\n try:\n return fn(*args, **kwargs)\n except Exception as e:\n print(traceback.format_exc(), file=sys.stderr)\n raise RuntimeError(\"Running into exception in worker.\") from e\n\n return inner\n\n\n@try_run\ndef hist_train(\n worker_idx: int,\n tmpdir: str,\n device: str,\n rabit_args: dict,\n) -> None:\n \"\"\"The hist tree method can use a special data structure `ExtMemQuantileDMatrix` for\n faster initialization and lower memory usage.\n\n \"\"\"\n\n # Make sure XGBoost is using the configured memory pool for all allocations.\n with (\n coll.CommunicatorContext(**rabit_args),\n xgboost.config_context(\n use_cuda_async_pool=device == \"cuda\",\n ),\n ):\n print(\"Worker: \", worker_idx)\n # Generate the data for demonstration. The synthetic data is sharded by workers.\n files = make_batches(\n n_samples_per_batch=4096,\n n_features=16,\n n_batches=17,\n tmpdir=tmpdir,\n rank=coll.get_rank(),\n )\n # Since we are running two workers on a single node, we should divide the number\n # of threads between workers.\n n_threads = os.cpu_count()\n assert n_threads is not None\n n_threads = max(n_threads // coll.get_world_size(), 1)\n it = Iterator(device, files)\n Xy = xgboost.ExtMemQuantileDMatrix(\n it, missing=np.nan, enable_categorical=False, nthread=n_threads\n )\n # Check the device is correctly set.\n if device == \"cuda\":\n # Check the first device\n assert (\n int(os.environ[\"CUDA_VISIBLE_DEVICES\"].split(\",\")[0])\n < coll.get_world_size()\n )\n booster = xgboost.train(\n {\n \"tree_method\": \"hist\",\n \"max_depth\": 4,\n \"device\": it.device,\n \"nthread\": n_threads,\n },\n Xy,\n evals=[(Xy, \"Train\")],\n num_boost_round=10,\n )\n booster.predict(Xy)\n\n\ndef launch_workers(tmpdir: str, args: argparse.Namespace) -> None:\n \"\"\"Client function to launch workers.\"\"\"\n n_workers = 2\n\n tracker = RabitTracker(host_ip=\"127.0.0.1\", n_workers=n_workers)\n tracker.start()\n rabit_args = tracker.worker_args()\n\n def initializer(device: str) -> None:\n # Set CUDA device before launching child processes.\n if device == \"cuda\":\n # name: LokyProcess-1\n _, sidx = mp.current_process().name.split(\"-\")\n idx = int(sidx) - 1 # 1-based indexing from loky\n # Assuming two workers for demo.\n devices = \",\".join([str(idx), str((idx + 1) % n_workers)])\n # P0: CUDA_VISIBLE_DEVICES=0,1\n # P1: CUDA_VISIBLE_DEVICES=1,0\n os.environ[\"CUDA_VISIBLE_DEVICES\"] = devices\n setup_numa()\n setup_async_pool()\n\n with get_reusable_executor(\n max_workers=n_workers,\n initargs=(args.device,),\n initializer=initializer,\n ) as pool:\n # Poor man's currying\n fn = update_wrapper(\n partial(\n hist_train,\n tmpdir=tmpdir,\n device=args.device,\n rabit_args=rabit_args,\n ),\n hist_train,\n )\n pool.map(fn, range(n_workers))\n\n\ndef main() -> None:\n \"\"\"Demo for distributed training from scratch.\"\"\"\n parser = argparse.ArgumentParser()\n parser.add_argument(\"--device\", choices=[\"cpu\", \"cuda\"], default=\"cpu\")\n args = parser.parse_args()\n with tempfile.TemporaryDirectory() as tmpdir:\n launch_workers(tmpdir, args)\n\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "demo/guide-python/evals_result.py", "content": "\"\"\"\nThis script demonstrate how to access the eval metrics\n======================================================\n\"\"\"\nimport os\nfrom typing import Any, Dict\n\nimport xgboost as xgb\n\nCURRENT_DIR = os.path.dirname(__file__)\ndtrain = xgb.DMatrix(\n os.path.join(CURRENT_DIR, \"../data/agaricus.txt.train?format=libsvm\")\n)\ndtest = xgb.DMatrix(\n os.path.join(CURRENT_DIR, \"../data/agaricus.txt.test?format=libsvm\")\n)\n\nparam = [\n (\"max_depth\", 2),\n (\"objective\", \"binary:logistic\"),\n (\"eval_metric\", \"logloss\"),\n (\"eval_metric\", \"error\"),\n]\n\nnum_round = 2\nwatchlist = [(dtest, \"eval\"), (dtrain, \"train\")]\n\nevals_result: Dict[str, Any] = {}\nbst = xgb.train(param, dtrain, num_round, watchlist, evals_result=evals_result)\n\nprint(\"Access logloss metric directly from evals_result:\")\nprint(evals_result[\"eval\"][\"logloss\"])\n\nprint(\"\")\nprint(\"Access metrics through a loop:\")\nfor e_name, e_mtrs in evals_result.items():\n print(\"- {}\".format(e_name))\n for e_mtr_name, e_mtr_vals in e_mtrs.items():\n print(\" - {}\".format(e_mtr_name))\n print(\" - {}\".format(e_mtr_vals))\n\nprint(\"\")\nprint(\"Access complete dictionary:\")\nprint(evals_result)\n" }, { "path": "demo/guide-python/external_memory.py", "content": "\"\"\"\nExperimental support for external memory\n========================================\n\nThis is similar to the one in `quantile_data_iterator.py`, but for external memory\ninstead of Quantile DMatrix. The feature is not ready for production use yet.\n\n .. versionadded:: 1.5.0\n\n\nSee :doc:`the tutorial ` for more details.\n\n .. versionchanged:: 3.0.0\n\n Added :py:class:`~xgboost.ExtMemQuantileDMatrix`.\n\nTo run the example, following packages in addition to XGBoost native dependencies are\nrequired:\n\n- scikit-learn\n\nIf `device` is `cuda`, following are also needed:\n\n- cupy\n- rmm\n- cuda-python\n\n.. seealso::\n\n :ref:`sphx_glr_python_examples_distributed_extmem_basic.py`\n\nNot shown in this example, but you should pay attention to NUMA configuration as\ndiscussed in the tutorial.\n\n\"\"\"\n\nimport argparse\nimport os\nimport tempfile\nfrom typing import TYPE_CHECKING, Callable, List, Literal, Tuple\n\nimport numpy as np\nfrom sklearn.datasets import make_regression\n\nimport xgboost\n\nif TYPE_CHECKING:\n from cuda.bindings.runtime import cudaError_t\n\n\ndef _checkcu(status: \"cudaError_t\") -> None:\n import cuda.bindings.runtime as cudart\n\n if status != cudart.cudaError_t.cudaSuccess:\n raise RuntimeError(cudart.cudaGetErrorString(status))\n\n\ndef device_mem_total() -> int:\n \"\"\"The total number of bytes of memory this GPU has.\"\"\"\n import cuda.bindings.runtime as cudart\n\n status, free, total = cudart.cudaMemGetInfo()\n _checkcu(status)\n return total\n\n\ndef make_batches(\n n_samples_per_batch: int,\n n_features: int,\n n_batches: int,\n tmpdir: str,\n) -> List[Tuple[str, str]]:\n files: List[Tuple[str, str]] = []\n rng = np.random.RandomState(1994)\n for i in range(n_batches):\n X, y = make_regression(n_samples_per_batch, n_features, random_state=rng)\n X_path = os.path.join(tmpdir, \"X-\" + str(i) + \".npy\")\n y_path = os.path.join(tmpdir, \"y-\" + str(i) + \".npy\")\n np.save(X_path, X)\n np.save(y_path, y)\n files.append((X_path, y_path))\n return files\n\n\nclass Iterator(xgboost.DataIter):\n \"\"\"A custom iterator for loading files in batches.\"\"\"\n\n def __init__(\n self, device: Literal[\"cpu\", \"cuda\"], file_paths: List[Tuple[str, str]]\n ) -> None:\n self.device = device\n\n self._file_paths = file_paths\n self._it = 0\n # XGBoost will generate some cache files under the current directory with the\n # prefix \"cache\"\n super().__init__(cache_prefix=os.path.join(\".\", \"cache\"))\n\n def load_file(self) -> Tuple[np.ndarray, np.ndarray]:\n \"\"\"Load a single batch of data.\"\"\"\n X_path, y_path = self._file_paths[self._it]\n # When the `ExtMemQuantileDMatrix` is used, the device must match. GPU cannot\n # consume CPU input data and vice-versa.\n if self.device == \"cpu\":\n X = np.load(X_path)\n y = np.load(y_path)\n else:\n X = cp.load(X_path)\n y = cp.load(y_path)\n\n assert X.shape[0] == y.shape[0]\n return X, y\n\n def next(self, input_data: Callable) -> bool:\n \"\"\"Advance the iterator by 1 step and pass the data to XGBoost. This function\n is called by XGBoost during the construction of ``DMatrix``\n\n \"\"\"\n if self._it == len(self._file_paths):\n # return False to let XGBoost know this is the end of iteration\n return False\n\n # input_data is a keyword-only function passed in by XGBoost and has the similar\n # signature to the ``DMatrix`` constructor.\n X, y = self.load_file()\n input_data(data=X, label=y)\n self._it += 1\n return True\n\n def reset(self) -> None:\n \"\"\"Reset the iterator to its beginning\"\"\"\n self._it = 0\n\n\ndef hist_train(it: Iterator) -> None:\n \"\"\"The hist tree method can use a special data structure `ExtMemQuantileDMatrix` for\n faster initialization and lower memory usage (recommended).\n\n .. versionadded:: 3.0.0\n\n \"\"\"\n # For non-data arguments, specify it here once instead of passing them by the `next`\n # method.\n Xy = xgboost.ExtMemQuantileDMatrix(it, missing=np.nan, enable_categorical=False)\n booster = xgboost.train(\n {\"tree_method\": \"hist\", \"max_depth\": 4, \"device\": it.device},\n Xy,\n evals=[(Xy, \"Train\")],\n num_boost_round=10,\n )\n booster.predict(Xy)\n\n\ndef approx_train(it: Iterator) -> None:\n \"\"\"The approx tree method uses the basic `DMatrix` (not recommended).\"\"\"\n\n # For non-data arguments, specify it here once instead of passing them by the `next`\n # method.\n Xy = xgboost.DMatrix(it, missing=np.nan, enable_categorical=False)\n # ``approx`` is also supported, but less efficient due to sketching. It's\n # recommended to use `hist` instead.\n booster = xgboost.train(\n {\"tree_method\": \"approx\", \"max_depth\": 4, \"device\": it.device},\n Xy,\n evals=[(Xy, \"Train\")],\n num_boost_round=10,\n )\n booster.predict(Xy)\n\n\ndef main(tmpdir: str, args: argparse.Namespace) -> None:\n \"\"\"Entry point for training.\"\"\"\n\n # generate some random data for demo\n files = make_batches(\n n_samples_per_batch=1024, n_features=17, n_batches=31, tmpdir=tmpdir\n )\n it = Iterator(args.device, files)\n\n hist_train(it)\n approx_train(it)\n\n\ndef setup_async_pool() -> None:\n \"\"\"Setup CUDA async pool. As an alternative, the RMM plugin can be used as well. See\n the `setup_rmm`. This is the same as using the `CudaAsyncMemoryResource` from RMM,\n but without the RMM dependency.\n\n .. versionadded:: 3.2.0\n\n \"\"\"\n import cuda.bindings.driver as driver\n import cuda.bindings.runtime as cudart\n from cupy.cuda import MemoryAsyncPool\n\n status, dft_pool = cudart.cudaDeviceGetDefaultMemPool(0)\n _checkcu(status)\n\n total = device_mem_total()\n\n v = driver.cuuint64_t(int(total * 0.9))\n (status,) = cudart.cudaMemPoolSetAttribute(\n dft_pool,\n cudart.cudaMemPoolAttr.cudaMemPoolAttrReleaseThreshold,\n v,\n )\n _checkcu(status)\n # Set the allocator for cupy as well.\n cp.cuda.set_allocator(MemoryAsyncPool().malloc)\n\n\ndef setup_rmm() -> None:\n \"\"\"Setup RMM for GPU-based external memory training.\n\n It's important to use RMM with `CudaAsyncMemoryResource` or `ArenaMemoryResource`\n for GPU-based external memory to improve performance. If XGBoost is not built with\n RMM support, a warning is raised when constructing the `DMatrix`.\n\n \"\"\"\n\n import rmm\n from rmm.allocators.cupy import rmm_cupy_allocator\n from rmm.mr import ArenaMemoryResource\n\n if not xgboost.build_info()[\"USE_RMM\"]:\n return\n\n total = device_mem_total()\n\n mr = rmm.mr.CudaMemoryResource()\n mr = ArenaMemoryResource(mr, arena_size=int(total * 0.9))\n\n rmm.mr.set_current_device_resource(mr)\n # Set the allocator for cupy as well.\n cp.cuda.set_allocator(rmm_cupy_allocator)\n\n\nif __name__ == \"__main__\":\n parser = argparse.ArgumentParser()\n parser.add_argument(\"--device\", choices=[\"cpu\", \"cuda\"], default=\"cpu\")\n parser.add_argument(\n \"--memory_pool\",\n choices=[\"rmm\", \"cuda\"],\n default=\"rmm\",\n help=\"Use a memory pool for asynchronous memory allocation in XGBoost.\",\n )\n args = parser.parse_args()\n if args.device == \"cuda\":\n import cupy as cp\n\n if args.memory_pool == \"rmm\":\n setup_rmm()\n elif args.memory_pool == \"cuda\":\n setup_async_pool()\n # Make sure XGBoost is using RMM for all allocations.\n with xgboost.config_context(\n use_rmm=args.memory_pool == \"rmm\",\n use_cuda_async_pool=args.memory_pool == \"cuda\",\n ):\n with tempfile.TemporaryDirectory() as tmpdir:\n main(tmpdir, args)\n else:\n with tempfile.TemporaryDirectory() as tmpdir:\n main(tmpdir, args)\n" }, { "path": "demo/guide-python/feature_weights.py", "content": "\"\"\"\nDemo for using feature weight to change column sampling\n=======================================================\n\n .. versionadded:: 1.3.0\n\"\"\"\n\nimport argparse\n\nimport numpy as np\nfrom matplotlib import pyplot as plt\n\nimport xgboost\n\n\ndef main(args: argparse.Namespace) -> None:\n rng = np.random.RandomState(1994)\n\n kRows = 4196\n kCols = 10\n\n X = rng.randn(kRows, kCols)\n y = rng.randn(kRows)\n fw = np.ones(shape=(kCols,))\n for i in range(kCols):\n fw[i] *= float(i)\n\n dtrain = xgboost.DMatrix(X, y)\n dtrain.set_info(feature_weights=fw)\n\n # Perform column sampling for each node split evaluation, the sampling process is\n # weighted by feature weights.\n bst = xgboost.train(\n {\"tree_method\": \"hist\", \"colsample_bynode\": 0.2},\n dtrain,\n num_boost_round=10,\n evals=[(dtrain, \"d\")],\n )\n feature_map = bst.get_fscore()\n\n # feature zero has 0 weight\n assert feature_map.get(\"f0\", None) is None\n assert max(feature_map.values()) == feature_map.get(\"f9\")\n\n if args.plot:\n xgboost.plot_importance(bst)\n plt.show()\n\n\nif __name__ == \"__main__\":\n parser = argparse.ArgumentParser()\n parser.add_argument(\n \"--plot\",\n type=int,\n default=1,\n help=\"Set to 0 to disable plotting the evaluation history.\",\n )\n args = parser.parse_args()\n main(args)\n" }, { "path": "demo/guide-python/gamma_regression.py", "content": "\"\"\"\nDemo for gamma regression\n=========================\n\"\"\"\nimport numpy as np\n\nimport xgboost as xgb\n\n# this script demonstrates how to fit gamma regression model (with log link function)\n# in xgboost, before running the demo you need to generate the autoclaims dataset\n# by running gen_autoclaims.R located in xgboost/demo/data.\n\ndata = np.genfromtxt('../data/autoclaims.csv', delimiter=',')\ndtrain = xgb.DMatrix(data[0:4741, 0:34], data[0:4741, 34])\ndtest = xgb.DMatrix(data[4741:6773, 0:34], data[4741:6773, 34])\n\n# for gamma regression, we need to set the objective to 'reg:gamma', it also suggests\n# to set the base_score to a value between 1 to 5 if the number of iteration is small\nparam = {'objective':'reg:gamma', 'booster':'gbtree', 'base_score':3}\n\n# the rest of settings are the same\nwatchlist = [(dtest, 'eval'), (dtrain, 'train')]\nnum_round = 30\n\n# training and evaluation\nbst = xgb.train(param, dtrain, num_round, watchlist)\npreds = bst.predict(dtest)\nlabels = dtest.get_label()\nprint('test deviance=%f' % (2 * np.sum((labels - preds) / preds - np.log(labels) + np.log(preds))))\n" }, { "path": "demo/guide-python/generalized_linear_model.py", "content": "\"\"\"\nDemo for GLM\n============\n\"\"\"\nimport os\n\nimport xgboost as xgb\n\n##\n# this script demonstrate how to fit generalized linear model in xgboost\n# basically, we are using linear model, instead of tree for our boosters\n##\nCURRENT_DIR = os.path.dirname(__file__)\ndtrain = xgb.DMatrix(\n os.path.join(CURRENT_DIR, \"../data/agaricus.txt.train?format=libsvm\")\n)\ndtest = xgb.DMatrix(\n os.path.join(CURRENT_DIR, \"../data/agaricus.txt.test?format=libsvm\")\n)\n# change booster to gblinear, so that we are fitting a linear model\n# alpha is the L1 regularizer\n# lambda is the L2 regularizer\n# you can also set lambda_bias which is L2 regularizer on the bias term\nparam = {\n \"objective\": \"binary:logistic\",\n \"booster\": \"gblinear\",\n \"alpha\": 0.0001,\n \"lambda\": 1,\n}\n\n# normally, you do not need to set eta (step_size)\n# XGBoost uses a parallel coordinate descent algorithm (shotgun),\n# there could be affection on convergence with parallelization on certain cases\n# setting eta to be smaller value, e.g 0.5 can make the optimization more stable\n# param['eta'] = 1\n\n##\n# the rest of settings are the same\n##\nwatchlist = [(dtest, \"eval\"), (dtrain, \"train\")]\nnum_round = 4\nbst = xgb.train(param, dtrain, num_round, watchlist)\npreds = bst.predict(dtest)\nlabels = dtest.get_label()\nprint(\n \"error=%f\"\n % (\n sum(1 for i in range(len(preds)) if int(preds[i] > 0.5) != labels[i])\n / float(len(preds))\n )\n)\n" }, { "path": "demo/guide-python/gpu_tree_shap.py", "content": "\"\"\"\nUse GPU to speedup SHAP value computation\n=========================================\n\nDemonstrates using GPU acceleration to compute SHAP values for feature importance.\n\n\"\"\"\nfrom urllib.error import HTTPError\n\nimport shap\nfrom sklearn.datasets import fetch_california_housing, make_regression\n\nimport xgboost as xgb\n\n# Fetch dataset using sklearn\ntry:\n _data = fetch_california_housing(return_X_y=True)\n X = _data.data\n y = _data.target\n feature_names = _data.feature_names\n print(_data.DESCR)\nexcept HTTPError:\n # Use a synthetic dataset instead if we couldn't\n X, y = make_regression(n_samples=20640, n_features=8, random_state=1234)\n feature_names = [f\"f{i}\" for i in range(8)]\n\nnum_round = 500\n\nparam = {\n \"eta\": 0.05,\n \"max_depth\": 10,\n \"tree_method\": \"hist\",\n \"device\": \"cuda\",\n}\n\n# GPU accelerated training\ndtrain = xgb.DMatrix(X, label=y, feature_names=feature_names)\nmodel = xgb.train(param, dtrain, num_round)\n\n# Compute shap values using GPU with xgboost\nmodel.set_param({\"device\": \"cuda\"})\nshap_values = model.predict(dtrain, pred_contribs=True)\n\n# Compute shap interaction values using GPU\nshap_interaction_values = model.predict(dtrain, pred_interactions=True)\n\n\n# shap will call the GPU accelerated version as long as the device parameter is set to\n# \"cuda\"\nexplainer = shap.TreeExplainer(model)\nshap_values = explainer.shap_values(X)\n\n# visualize the first prediction's explanation\nshap.force_plot(\n explainer.expected_value,\n shap_values[0, :],\n X[0, :],\n feature_names=feature_names,\n matplotlib=True,\n)\n\n# Show a summary of feature importance\nshap.summary_plot(shap_values, X, plot_type=\"bar\", feature_names=feature_names)\n" }, { "path": "demo/guide-python/individual_trees.py", "content": "\"\"\"\nDemo for prediction using individual trees and model slices\n===========================================================\n\"\"\"\n\nimport os\n\nimport numpy as np\nfrom scipy.special import logit\nfrom sklearn.datasets import load_svmlight_file\n\nimport xgboost as xgb\n\nCURRENT_DIR = os.path.dirname(__file__)\ntrain = os.path.join(CURRENT_DIR, \"../data/agaricus.txt.train\")\ntest = os.path.join(CURRENT_DIR, \"../data/agaricus.txt.test\")\n\n\ndef individual_tree() -> None:\n \"\"\"Get prediction from each individual tree and combine them together.\"\"\"\n X_train, y_train = load_svmlight_file(train)\n X_test, y_test = load_svmlight_file(test)\n Xy_train = xgb.QuantileDMatrix(X_train, y_train)\n\n n_rounds = 4\n # Specify the base score, otherwise xgboost will estimate one from the training\n # data.\n base_score = 0.5\n params = {\n \"max_depth\": 2,\n \"eta\": 1,\n \"objective\": \"reg:logistic\",\n \"tree_method\": \"hist\",\n \"base_score\": base_score,\n }\n booster = xgb.train(params, Xy_train, num_boost_round=n_rounds)\n\n # Use logit to inverse the base score back to raw leaf value (margin)\n scores = np.full((X_test.shape[0],), logit(base_score))\n for i in range(n_rounds):\n # - Use output_margin to get raw leaf values\n # - Use iteration_range to get prediction for only one tree\n # - Use previous prediction as base marign for the model\n Xy_test = xgb.DMatrix(X_test, base_margin=scores)\n\n if i == n_rounds - 1:\n # last round, get the transformed prediction\n scores = booster.predict(\n Xy_test, iteration_range=(i, i + 1), output_margin=False\n )\n else:\n # get raw leaf value for accumulation\n scores = booster.predict(\n Xy_test, iteration_range=(i, i + 1), output_margin=True\n )\n\n full = booster.predict(xgb.DMatrix(X_test), output_margin=False)\n np.testing.assert_allclose(scores, full)\n\n\ndef model_slices() -> None:\n \"\"\"Inference with each individual tree using model slices.\"\"\"\n X_train, y_train = load_svmlight_file(train)\n X_test, y_test = load_svmlight_file(test)\n Xy_train = xgb.QuantileDMatrix(X_train, y_train)\n\n n_rounds = 4\n # Specify the base score, otherwise xgboost will estimate one from the training\n # data.\n base_score = 0.5\n params = {\n \"max_depth\": 2,\n \"eta\": 1,\n \"objective\": \"reg:logistic\",\n \"tree_method\": \"hist\",\n \"base_score\": base_score,\n }\n booster = xgb.train(params, Xy_train, num_boost_round=n_rounds)\n trees = [booster[t] for t in range(n_rounds)]\n\n # Use logit to inverse the base score back to raw leaf value (margin)\n scores = np.full((X_test.shape[0],), logit(base_score))\n for i, t in enumerate(trees):\n # Feed previous scores into base margin.\n Xy_test = xgb.DMatrix(X_test, base_margin=scores)\n\n if i == n_rounds - 1:\n # last round, get the transformed prediction\n scores = t.predict(Xy_test, output_margin=False)\n else:\n # get raw leaf value for accumulation\n scores = t.predict(Xy_test, output_margin=True)\n\n full = booster.predict(xgb.DMatrix(X_test), output_margin=False)\n np.testing.assert_allclose(scores, full)\n\n\nif __name__ == \"__main__\":\n individual_tree()\n model_slices()\n" }, { "path": "demo/guide-python/learning_to_rank.py", "content": "\"\"\"\nGetting started with learning to rank\n=====================================\n\n .. versionadded:: 2.0.0\n\nThis is a demonstration of using XGBoost for learning to rank tasks using the\nMSLR_10k_letor dataset. For more infomation about the dataset, please visit its\n`description page `_.\n\nThis is a two-part demo, the first one contains a basic example of using XGBoost to\ntrain on relevance degree, and the second part simulates click data and enable the\nposition debiasing training.\n\nFor an overview of learning to rank in XGBoost, please see :doc:`Learning to Rank\n`.\n\"\"\"\n\nfrom __future__ import annotations\n\nimport argparse\nimport json\nimport os\nimport pickle as pkl\n\nimport numpy as np\nimport pandas as pd\nfrom sklearn.datasets import load_svmlight_file\n\nimport xgboost as xgb\nfrom xgboost.testing.data import RelDataCV, simulate_clicks, sort_ltr_samples\n\n\ndef load_mslr_10k(data_path: str, cache_path: str) -> RelDataCV:\n \"\"\"Load the MSLR10k dataset from data_path and cache a pickle object in cache_path.\n\n Returns\n -------\n\n A list of tuples [(X, y, qid), ...].\n\n \"\"\"\n root_path = os.path.expanduser(args.data)\n cacheroot_path = os.path.expanduser(args.cache)\n cache_path = os.path.join(cacheroot_path, \"MSLR_10K_LETOR.pkl\")\n\n # Use only the Fold1 for demo:\n # Train, Valid, Test\n # {S1,S2,S3}, S4, S5\n fold = 1\n\n if not os.path.exists(cache_path):\n fold_path = os.path.join(root_path, f\"Fold{fold}\")\n train_path = os.path.join(fold_path, \"train.txt\")\n valid_path = os.path.join(fold_path, \"vali.txt\")\n test_path = os.path.join(fold_path, \"test.txt\")\n X_train, y_train, qid_train = load_svmlight_file(\n train_path, query_id=True, dtype=np.float32\n )\n y_train = y_train.astype(np.int32)\n qid_train = qid_train.astype(np.int32)\n\n X_valid, y_valid, qid_valid = load_svmlight_file(\n valid_path, query_id=True, dtype=np.float32\n )\n y_valid = y_valid.astype(np.int32)\n qid_valid = qid_valid.astype(np.int32)\n\n X_test, y_test, qid_test = load_svmlight_file(\n test_path, query_id=True, dtype=np.float32\n )\n y_test = y_test.astype(np.int32)\n qid_test = qid_test.astype(np.int32)\n\n data = RelDataCV(\n train=(X_train, y_train, qid_train),\n test=(X_test, y_test, qid_test),\n max_rel=4,\n )\n\n with open(cache_path, \"wb\") as fd:\n pkl.dump(data, fd)\n\n with open(cache_path, \"rb\") as fd:\n data = pkl.load(fd)\n\n return data\n\n\ndef ranking_demo(args: argparse.Namespace) -> None:\n \"\"\"Demonstration for learning to rank with relevance degree.\"\"\"\n data = load_mslr_10k(args.data, args.cache)\n\n # Sort data according to query index\n X_train, y_train, qid_train = data.train\n sorted_idx = np.argsort(qid_train)\n X_train = X_train[sorted_idx]\n y_train = y_train[sorted_idx]\n qid_train = qid_train[sorted_idx]\n\n X_test, y_test, qid_test = data.test\n sorted_idx = np.argsort(qid_test)\n X_test = X_test[sorted_idx]\n y_test = y_test[sorted_idx]\n qid_test = qid_test[sorted_idx]\n\n ranker = xgb.XGBRanker(\n tree_method=\"hist\",\n device=\"cuda\",\n lambdarank_pair_method=\"topk\",\n lambdarank_num_pair_per_sample=13,\n eval_metric=[\"ndcg@1\", \"ndcg@8\"],\n )\n ranker.fit(\n X_train,\n y_train,\n qid=qid_train,\n eval_set=[(X_test, y_test)],\n eval_qid=[qid_test],\n verbose=True,\n )\n\n\ndef click_data_demo(args: argparse.Namespace) -> None:\n \"\"\"Demonstration for learning to rank with click data.\"\"\"\n data = load_mslr_10k(args.data, args.cache)\n train, test = simulate_clicks(data)\n assert test is not None\n\n assert train.X.shape[0] == train.click.size\n assert test.X.shape[0] == test.click.size\n assert test.score.dtype == np.float32\n assert test.click.dtype == np.int32\n\n X_train, clicks_train, y_train, qid_train = sort_ltr_samples(\n train.X,\n train.y,\n train.qid,\n train.click,\n train.pos,\n )\n X_test, clicks_test, y_test, qid_test = sort_ltr_samples(\n test.X,\n test.y,\n test.qid,\n test.click,\n test.pos,\n )\n\n class ShowPosition(xgb.callback.TrainingCallback):\n def after_iteration(\n self,\n model: xgb.Booster,\n epoch: int,\n evals_log: xgb.callback.TrainingCallback.EvalsLog,\n ) -> bool:\n config = json.loads(model.save_config())\n ti_plus = np.array(config[\"learner\"][\"objective\"][\"ti+\"])\n tj_minus = np.array(config[\"learner\"][\"objective\"][\"tj-\"])\n df = pd.DataFrame({\"ti+\": ti_plus, \"tj-\": tj_minus})\n print(df)\n return False\n\n ranker = xgb.XGBRanker(\n n_estimators=512,\n tree_method=\"hist\",\n device=\"cuda\",\n learning_rate=0.01,\n reg_lambda=1.5,\n subsample=0.8,\n sampling_method=\"gradient_based\",\n # LTR specific parameters\n objective=\"rank:ndcg\",\n # - Enable bias estimation\n lambdarank_unbiased=True,\n # - normalization (1 / (norm + 1))\n lambdarank_bias_norm=1,\n # - Focus on the top 12 documents\n lambdarank_num_pair_per_sample=12,\n lambdarank_pair_method=\"topk\",\n ndcg_exp_gain=True,\n eval_metric=[\"ndcg@1\", \"ndcg@3\", \"ndcg@5\", \"ndcg@10\"],\n callbacks=[ShowPosition()],\n )\n ranker.fit(\n X_train,\n clicks_train,\n qid=qid_train,\n eval_set=[(X_test, y_test), (X_test, clicks_test)],\n eval_qid=[qid_test, qid_test],\n verbose=True,\n )\n ranker.predict(X_test)\n\n\nif __name__ == \"__main__\":\n parser = argparse.ArgumentParser(\n description=\"Demonstration of learning to rank using XGBoost.\"\n )\n parser.add_argument(\n \"--data\",\n type=str,\n help=\"Root directory of the MSLR-WEB10K data.\",\n required=True,\n )\n parser.add_argument(\n \"--cache\",\n type=str,\n help=\"Directory for caching processed data.\",\n required=True,\n )\n args = parser.parse_args()\n\n ranking_demo(args)\n click_data_demo(args)\n" }, { "path": "demo/guide-python/model_parser.py", "content": "\"\"\"\nDemonstration for parsing JSON/UBJSON tree model files\n======================================================\n\nSee :doc:`/tutorials/saving_model` for details about the model serialization.\n\n\"\"\"\n\nimport argparse\nimport json\nfrom dataclasses import dataclass\nfrom enum import IntEnum, unique\nfrom typing import Any, Dict, List, Sequence, Union\n\nimport numpy as np\n\ntry:\n import ubjson\nexcept ImportError:\n ubjson = None\n\n\nParamT = Dict[str, str]\n\n\ndef to_integers(data: Union[bytes, List[int]]) -> List[int]:\n \"\"\"Convert a sequence of bytes to a list of Python integer\"\"\"\n return [v for v in data]\n\n\n@unique\nclass SplitType(IntEnum):\n numerical = 0\n categorical = 1\n\n\n@dataclass\nclass Node:\n # properties\n left: int\n right: int\n parent: int\n split_idx: int\n split_cond: float\n default_left: bool\n split_type: SplitType\n categories: List[int]\n # statistic\n base_weight: float\n loss_chg: float\n sum_hess: float\n\n\nclass Tree:\n \"\"\"A tree built by XGBoost.\"\"\"\n\n def __init__(self, tree_id: int, nodes: Sequence[Node]) -> None:\n self.tree_id = tree_id\n self.nodes = nodes\n\n def loss_change(self, node_id: int) -> float:\n \"\"\"Loss gain of a node.\"\"\"\n return self.nodes[node_id].loss_chg\n\n def sum_hessian(self, node_id: int) -> float:\n \"\"\"Sum Hessian of a node.\"\"\"\n return self.nodes[node_id].sum_hess\n\n def base_weight(self, node_id: int) -> float:\n \"\"\"Base weight of a node.\"\"\"\n return self.nodes[node_id].base_weight\n\n def split_index(self, node_id: int) -> int:\n \"\"\"Split feature index of node.\"\"\"\n return self.nodes[node_id].split_idx\n\n def split_condition(self, node_id: int) -> float:\n \"\"\"Split value of a node.\"\"\"\n return self.nodes[node_id].split_cond\n\n def split_categories(self, node_id: int) -> List[int]:\n \"\"\"Categories in a node.\"\"\"\n return self.nodes[node_id].categories\n\n def is_categorical(self, node_id: int) -> bool:\n \"\"\"Whether a node has categorical split.\"\"\"\n return self.nodes[node_id].split_type == SplitType.categorical\n\n def is_numerical(self, node_id: int) -> bool:\n return not self.is_categorical(node_id)\n\n def parent(self, node_id: int) -> int:\n \"\"\"Parent ID of a node.\"\"\"\n return self.nodes[node_id].parent\n\n def left_child(self, node_id: int) -> int:\n \"\"\"Left child ID of a node.\"\"\"\n return self.nodes[node_id].left\n\n def right_child(self, node_id: int) -> int:\n \"\"\"Right child ID of a node.\"\"\"\n return self.nodes[node_id].right\n\n def is_leaf(self, node_id: int) -> bool:\n \"\"\"Whether a node is leaf.\"\"\"\n return self.nodes[node_id].left == -1\n\n def is_deleted(self, node_id: int) -> bool:\n \"\"\"Whether a node is deleted.\"\"\"\n return self.split_index(node_id) == np.iinfo(np.uint32).max\n\n def __str__(self) -> str:\n stack = [0]\n nodes = []\n while stack:\n node: Dict[str, Union[float, int, List[int]]] = {}\n nid = stack.pop()\n\n node[\"node id\"] = nid\n node[\"gain\"] = self.loss_change(nid)\n node[\"cover\"] = self.sum_hessian(nid)\n nodes.append(node)\n\n if not self.is_leaf(nid) and not self.is_deleted(nid):\n left = self.left_child(nid)\n right = self.right_child(nid)\n stack.append(left)\n stack.append(right)\n categories = self.split_categories(nid)\n if categories:\n assert self.is_categorical(nid)\n node[\"categories\"] = categories\n else:\n assert self.is_numerical(nid)\n node[\"condition\"] = self.split_condition(nid)\n if self.is_leaf(nid):\n node[\"weight\"] = self.split_condition(nid)\n\n string = \"\\n\".join(map(lambda x: \" \" + str(x), nodes))\n return string\n\n\nclass Model:\n \"\"\"Gradient boosted tree model.\"\"\"\n\n def __init__(self, model: dict) -> None:\n \"\"\"Construct the Model from a JSON object.\n\n parameters\n ----------\n model : A dictionary loaded by json representing a XGBoost boosted tree model.\n \"\"\"\n # Basic properties of a model\n self.learner_model_shape: ParamT = model[\"learner\"][\"learner_model_param\"]\n self.num_output_group = int(self.learner_model_shape[\"num_class\"])\n self.num_feature = int(self.learner_model_shape[\"num_feature\"])\n self.base_score: List[float] = json.loads(\n self.learner_model_shape[\"base_score\"]\n )\n # A field encoding which output group a tree belongs\n self.tree_info = model[\"learner\"][\"gradient_booster\"][\"model\"][\"tree_info\"]\n\n model_shape: ParamT = model[\"learner\"][\"gradient_booster\"][\"model\"][\n \"gbtree_model_param\"\n ]\n\n # JSON representation of trees\n j_trees = model[\"learner\"][\"gradient_booster\"][\"model\"][\"trees\"]\n\n # Load the trees\n self.num_trees = int(model_shape[\"num_trees\"])\n\n trees: List[Tree] = []\n for i in range(self.num_trees):\n tree: Dict[str, Any] = j_trees[i]\n tree_id = int(tree[\"id\"])\n assert tree_id == i, (tree_id, i)\n # - properties\n left_children: List[int] = tree[\"left_children\"]\n right_children: List[int] = tree[\"right_children\"]\n parents: List[int] = tree[\"parents\"]\n split_conditions: List[float] = tree[\"split_conditions\"]\n split_indices: List[int] = tree[\"split_indices\"]\n # when ubjson is used, this is a byte array with each element as uint8\n default_left = to_integers(tree[\"default_left\"])\n\n # - categorical features\n # when ubjson is used, this is a byte array with each element as uint8\n split_types = to_integers(tree[\"split_type\"])\n # categories for each node is stored in a CSR style storage with segment as\n # the begin ptr and the `categories' as values.\n cat_segments: List[int] = tree[\"categories_segments\"]\n cat_sizes: List[int] = tree[\"categories_sizes\"]\n # node index for categorical nodes\n cat_nodes: List[int] = tree[\"categories_nodes\"]\n assert len(cat_segments) == len(cat_sizes) == len(cat_nodes)\n cats = tree[\"categories\"]\n assert len(left_children) == len(split_types)\n\n # The storage for categories is only defined for categorical nodes to\n # prevent unnecessary overhead for numerical splits, we track the\n # categorical node that are processed using a counter.\n cat_cnt = 0\n if cat_nodes:\n last_cat_node = cat_nodes[cat_cnt]\n else:\n last_cat_node = -1\n node_categories: List[List[int]] = []\n for node_id in range(len(left_children)):\n if node_id == last_cat_node:\n beg = cat_segments[cat_cnt]\n size = cat_sizes[cat_cnt]\n end = beg + size\n node_cats = cats[beg:end]\n # categories are unique for each node\n assert len(set(node_cats)) == len(node_cats)\n cat_cnt += 1\n if cat_cnt == len(cat_nodes):\n last_cat_node = -1 # continue to process the rest of the nodes\n else:\n last_cat_node = cat_nodes[cat_cnt]\n assert node_cats\n node_categories.append(node_cats)\n else:\n # append an empty node, it's either a numerical node or a leaf.\n node_categories.append([])\n\n # - stats\n base_weights: List[float] = tree[\"base_weights\"]\n loss_changes: List[float] = tree[\"loss_changes\"]\n sum_hessian: List[float] = tree[\"sum_hessian\"]\n\n # Construct a list of nodes that have complete information\n nodes: List[Node] = [\n Node(\n left_children[node_id],\n right_children[node_id],\n parents[node_id],\n split_indices[node_id],\n split_conditions[node_id],\n default_left[node_id] == 1, # to boolean\n SplitType(split_types[node_id]),\n node_categories[node_id],\n base_weights[node_id],\n loss_changes[node_id],\n sum_hessian[node_id],\n )\n for node_id in range(len(left_children))\n ]\n\n pytree = Tree(tree_id, nodes)\n trees.append(pytree)\n\n self.trees = trees\n\n def print_model(self) -> None:\n for i, tree in enumerate(self.trees):\n print(\"\\ntree_id:\", i)\n print(tree)\n\n\nif __name__ == \"__main__\":\n parser = argparse.ArgumentParser(\n description=\"Demonstration for loading XGBoost JSON/UBJSON model.\"\n )\n parser.add_argument(\n \"--model\", type=str, required=True, help=\"Path to .json/.ubj model file.\"\n )\n args = parser.parse_args()\n if args.model.endswith(\"json\"):\n # use json format\n with open(args.model, \"r\") as fd:\n model = json.load(fd)\n elif args.model.endswith(\"ubj\"):\n if ubjson is None:\n raise ImportError(\"ubjson is not installed.\")\n # use ubjson format\n with open(args.model, \"rb\") as bfd:\n model = ubjson.load(bfd)\n else:\n raise ValueError(\n \"Unexpected file extension. Supported file extension are json and ubj.\"\n )\n model = Model(model)\n model.print_model()\n" }, { "path": "demo/guide-python/multioutput_reduced_gradient.py", "content": "\"\"\"\nA demo for multi-output regression using reduced gradient\n=========================================================\n\nSee :doc:`/tutorials/multioutput` for more information.\n\n.. versionadded:: 3.2.0\n\n.. note::\n\n The implementation is experimental and many features are missing.\n\n.. seealso:: :ref:`sphx_glr_python_examples_multioutput_regression.py`\n\n\"\"\"\n\nimport argparse\nfrom typing import Tuple\n\nimport numpy as np\nfrom sklearn.base import BaseEstimator\nfrom sklearn.datasets import make_regression\n\nimport xgboost as xgb\nfrom xgboost.objective import TreeObjective\n\n\nclass LsObjMean(TreeObjective):\n \"\"\"Least squared error. Reduce the size of the gradient using mean value.\"\"\"\n\n def __init__(self, device: str) -> None:\n self.device = device\n\n def __call__(\n self, iteration: int, y_pred: np.ndarray, dtrain: xgb.DMatrix\n ) -> Tuple[np.ndarray, np.ndarray]:\n y_true = dtrain.get_label()\n grad = y_pred - y_true\n if self.device == \"cpu\":\n hess = np.ones(grad.shape)\n return grad, hess\n else:\n import cupy as cp\n\n hess = cp.ones(grad.shape)\n\n return cp.array(grad), cp.array(hess)\n\n def split_grad(\n self, iteration: int, grad: np.ndarray, hess: np.ndarray\n ) -> Tuple[np.ndarray, np.ndarray]:\n if self.device == \"cpu\":\n from numpy import mean\n else:\n from cupy import mean # type: ignore[no-redef]\n\n sgrad = mean(grad, axis=1)\n shess = mean(hess, axis=1)\n return sgrad, shess\n\n\ndef svd_class(device: str) -> BaseEstimator:\n \"\"\"One of the methods in the sketch boost paper.\"\"\"\n from sklearn.decomposition import TruncatedSVD\n\n svd_params = {\"algorithm\": \"arpack\", \"n_components\": 2, \"n_iter\": 8}\n svd = TruncatedSVD(**svd_params)\n return svd\n\n\nclass LsObjSvd(LsObjMean):\n \"\"\"Reduce the size of the gradient using SVD.\"\"\"\n\n def __init__(self, device: str) -> None:\n super().__init__(device=device)\n\n def split_grad(\n self, iteration: int, grad: np.ndarray, hess: np.ndarray\n ) -> Tuple[np.ndarray, np.ndarray]:\n svd = svd_class(self.device)\n if self.device == \"cuda\":\n grad = grad.get() # type: ignore\n hess = hess.get() # type: ignore\n\n svd.fit(grad)\n grad = svd.transform(grad)\n hess = svd.transform(hess)\n if self.device == \"cpu\":\n hess = np.clip(hess, 0.01, None)\n else:\n import cupy as cp\n\n hess = cp.clip(hess, 0.01, None)\n return grad, hess\n\n\ndef main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"--device\", choices=[\"cpu\", \"cuda\"], default=\"cpu\")\n args = parser.parse_args()\n\n X, y = make_regression(\n n_samples=8192, n_features=32, n_targets=8, random_state=2026\n )\n Xy = xgb.QuantileDMatrix(X, y)\n\n for obj in (LsObjMean(args.device), LsObjSvd(args.device)):\n xgb.train(\n {\n \"device\": args.device,\n \"multi_strategy\": \"multi_output_tree\",\n },\n Xy,\n evals=[(Xy, \"Train\")],\n obj=obj,\n num_boost_round=16,\n )\n\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "demo/guide-python/multioutput_regression.py", "content": "\"\"\"\nA demo for multi-output regression\n==================================\n\nThe demo is adopted from scikit-learn:\n\nhttps://scikit-learn.org/stable/auto_examples/ensemble/plot_random_forest_regression_multioutput.html#sphx-glr-auto-examples-ensemble-plot-random-forest-regression-multioutput-py\n\nSee :doc:`/tutorials/multioutput` for more information.\n\n.. note::\n\n The feature is experimental. For the `multi_output_tree` strategy, many features are\n missing.\n\n.. seealso:: :ref:`sphx_glr_python_examples_multioutput_reduced_gradient.py`\n\n\"\"\"\n\nimport argparse\nfrom typing import Dict, List, Optional, Tuple\n\nimport matplotlib\nimport numpy as np\nfrom matplotlib import pyplot as plt\n\nimport xgboost as xgb\n\n\ndef plot_predt(\n y: np.ndarray, y_predt: np.ndarray, name: str, ax: matplotlib.axes.Axes\n) -> None:\n s = 25\n ax.scatter(y[:, 0], y[:, 1], c=\"navy\", s=s, edgecolor=\"black\", label=name)\n ax.scatter(y_predt[:, 0], y_predt[:, 1], c=\"cornflowerblue\", s=s, edgecolor=\"black\")\n ax.legend()\n\n\ndef gen_circle() -> Tuple[np.ndarray, np.ndarray]:\n \"Generate a sample dataset that y is a 2 dim circle.\"\n rng = np.random.RandomState(1994)\n X = np.sort(200 * rng.rand(100, 1) - 100, axis=0)\n y = np.array([np.pi * np.sin(X).ravel(), np.pi * np.cos(X).ravel()]).T\n y[::5, :] += 0.5 - rng.rand(20, 2)\n y = y - y.min()\n y = y / y.max()\n return X, y\n\n\ndef rmse_model(strategy: str, ax: Optional[matplotlib.axes.Axes]) -> None:\n \"\"\"Draw a circle with 2-dim coordinate as target variables.\"\"\"\n X, y = gen_circle()\n # Train a regressor on it\n reg = xgb.XGBRegressor(\n tree_method=\"hist\",\n n_estimators=128,\n n_jobs=16,\n max_depth=8,\n multi_strategy=strategy,\n subsample=0.6,\n )\n reg.fit(X, y, eval_set=[(X, y)])\n\n y_predt = reg.predict(X)\n if ax:\n plot_predt(y, y_predt, f\"RMSE-{strategy}\", ax)\n\n\ndef custom_rmse_model(strategy: str, ax: Optional[matplotlib.axes.Axes]) -> None:\n \"\"\"Train using Python implementation of Squared Error.\"\"\"\n\n def gradient(predt: np.ndarray, dtrain: xgb.DMatrix) -> np.ndarray:\n \"\"\"Compute the gradient squared error.\"\"\"\n y = dtrain.get_label().reshape(predt.shape)\n return predt - y\n\n def hessian(predt: np.ndarray, dtrain: xgb.DMatrix) -> np.ndarray:\n \"\"\"Compute the hessian for squared error.\"\"\"\n return np.ones(predt.shape)\n\n def squared_log(\n predt: np.ndarray, dtrain: xgb.DMatrix\n ) -> Tuple[np.ndarray, np.ndarray]:\n grad = gradient(predt, dtrain)\n hess = hessian(predt, dtrain)\n # both numpy.ndarray and cupy.ndarray works.\n return grad, hess\n\n def rmse(predt: np.ndarray, dtrain: xgb.DMatrix) -> Tuple[str, float]:\n y = dtrain.get_label().reshape(predt.shape)\n v = np.sqrt(np.mean(np.power(y - predt, 2)))\n return \"PyRMSE\", v\n\n X, y = gen_circle()\n Xy = xgb.DMatrix(X, y)\n results: Dict[str, Dict[str, List[float]]] = {}\n # Make sure the `num_target` is passed to XGBoost when custom objective is used.\n # When builtin objective is used, XGBoost can figure out the number of targets\n # automatically.\n booster = xgb.train(\n {\n \"tree_method\": \"hist\",\n \"num_target\": y.shape[1],\n \"multi_strategy\": strategy,\n },\n dtrain=Xy,\n num_boost_round=128,\n obj=squared_log,\n evals=[(Xy, \"Train\")],\n evals_result=results,\n custom_metric=rmse,\n )\n\n y_predt = booster.inplace_predict(X)\n if ax:\n plot_predt(y, y_predt, f\"PyRMSE-{strategy}\", ax)\n\n np.testing.assert_allclose(\n results[\"Train\"][\"rmse\"], results[\"Train\"][\"PyRMSE\"], rtol=1e-2\n )\n\n\nif __name__ == \"__main__\":\n parser = argparse.ArgumentParser()\n parser.add_argument(\"--plot\", choices=[0, 1], type=int, default=1)\n args = parser.parse_args()\n if args.plot == 1:\n _, axs = plt.subplots(2, 2)\n else:\n axs = np.full(shape=(2, 2), fill_value=None)\n assert isinstance(axs, np.ndarray)\n\n # Train with builtin RMSE objective\n # - One model per output.\n rmse_model(\"one_output_per_tree\", axs[0, 0])\n # - One model for all outputs, this is still working in progress, many features are\n # missing.\n rmse_model(\"multi_output_tree\", axs[0, 1])\n\n # Train with custom objective.\n # - One model per output.\n custom_rmse_model(\"one_output_per_tree\", axs[1, 0])\n # - One model for all outputs, this is still working in progress, many features are\n # missing.\n custom_rmse_model(\"multi_output_tree\", axs[1, 1])\n if args.plot == 1:\n plt.show()\n" }, { "path": "demo/guide-python/predict_first_ntree.py", "content": "\"\"\"\nDemo for prediction using number of trees\n=========================================\n\"\"\"\nimport os\n\nimport numpy as np\nfrom sklearn.datasets import load_svmlight_file\n\nimport xgboost as xgb\n\nCURRENT_DIR = os.path.dirname(__file__)\ntrain = os.path.join(CURRENT_DIR, \"../data/agaricus.txt.train\")\ntest = os.path.join(CURRENT_DIR, \"../data/agaricus.txt.test\")\n\n\ndef native_interface() -> None:\n # load data in do training\n dtrain = xgb.DMatrix(train + \"?format=libsvm\")\n dtest = xgb.DMatrix(test + \"?format=libsvm\")\n param = {\"max_depth\": 2, \"eta\": 1, \"objective\": \"binary:logistic\"}\n watchlist = [(dtest, \"eval\"), (dtrain, \"train\")]\n num_round = 3\n bst = xgb.train(param, dtrain, num_round, watchlist)\n\n print(\"start testing prediction from first n trees\")\n # predict using first 1 tree\n label = dtest.get_label()\n ypred1 = bst.predict(dtest, iteration_range=(0, 1))\n # by default, we predict using all the trees\n ypred2 = bst.predict(dtest)\n\n print(\"error of ypred1=%f\" % (np.sum((ypred1 > 0.5) != label) / float(len(label))))\n print(\"error of ypred2=%f\" % (np.sum((ypred2 > 0.5) != label) / float(len(label))))\n\n\ndef sklearn_interface() -> None:\n X_train, y_train = load_svmlight_file(train)\n X_test, y_test = load_svmlight_file(test)\n clf = xgb.XGBClassifier(n_estimators=3, max_depth=2, eta=1)\n clf.fit(X_train, y_train, eval_set=[(X_test, y_test)])\n assert clf.n_classes_ == 2\n\n print(\"start testing prediction from first n trees\")\n # predict using first 1 tree\n ypred1 = clf.predict(X_test, iteration_range=(0, 1))\n # by default, we predict using all the trees\n ypred2 = clf.predict(X_test)\n\n print(\n \"error of ypred1=%f\" % (np.sum((ypred1 > 0.5) != y_test) / float(len(y_test)))\n )\n print(\n \"error of ypred2=%f\" % (np.sum((ypred2 > 0.5) != y_test) / float(len(y_test)))\n )\n\n\nif __name__ == \"__main__\":\n native_interface()\n sklearn_interface()\n" }, { "path": "demo/guide-python/predict_leaf_indices.py", "content": "\"\"\"\nDemo for obtaining leaf index\n=============================\n\"\"\"\nimport os\n\nimport xgboost as xgb\n\n# load data in do training\nCURRENT_DIR = os.path.dirname(__file__)\ndtrain = xgb.DMatrix(\n os.path.join(CURRENT_DIR, \"../data/agaricus.txt.train?format=libsvm\")\n)\ndtest = xgb.DMatrix(\n os.path.join(CURRENT_DIR, \"../data/agaricus.txt.test?format=libsvm\")\n)\nparam = {\"max_depth\": 2, \"eta\": 1, \"objective\": \"binary:logistic\"}\nwatchlist = [(dtest, \"eval\"), (dtrain, \"train\")]\nnum_round = 3\nbst = xgb.train(param, dtrain, num_round, watchlist)\n\nprint(\"start testing predict the leaf indices\")\n# predict using first 2 tree\nleafindex = bst.predict(\n dtest, iteration_range=(0, 2), pred_leaf=True, strict_shape=True\n)\nprint(leafindex.shape)\nprint(leafindex)\n# predict all trees\nleafindex = bst.predict(dtest, pred_leaf=True)\nprint(leafindex.shape)\n" }, { "path": "demo/guide-python/quantile_data_iterator.py", "content": "\"\"\"\nDemo for using data iterator with Quantile DMatrix\n==================================================\n\n .. versionadded:: 1.2.0\n\nThe demo that defines a customized iterator for passing batches of data into\n:py:class:`xgboost.QuantileDMatrix` and use this ``QuantileDMatrix`` for training. The\nfeature is primarily designed to reduce the required GPU memory for training on\ndistributed environment.\n\nAftering going through the demo, one might ask why don't we use more native Python\niterator? That's because XGBoost requires a `reset` function, while using\n`itertools.tee` might incur significant memory usage according to:\n\n https://docs.python.org/3/library/itertools.html#itertools.tee.\n\n.. seealso::\n\n :ref:`sphx_glr_python_examples_external_memory.py`\n\n\"\"\"\n\nfrom typing import Callable\n\nimport cupy\nimport numpy\n\nimport xgboost\n\nCOLS = 64\nROWS_PER_BATCH = 1000 # data is splited by rows\nBATCHES = 32\n\n\nclass IterForDMatrixDemo(xgboost.core.DataIter):\n \"\"\"A data iterator for XGBoost DMatrix.\n\n `reset` and `next` are required for any data iterator, other functions here\n are utilites for demonstration's purpose.\n\n \"\"\"\n\n def __init__(self) -> None:\n \"\"\"Generate some random data for demostration.\n\n Actual data can be anything that is currently supported by XGBoost.\n \"\"\"\n self.rows = ROWS_PER_BATCH\n self.cols = COLS\n rng = cupy.random.RandomState(numpy.uint64(1994))\n self._data = [rng.randn(self.rows, self.cols)] * BATCHES\n self._labels = [rng.randn(self.rows)] * BATCHES\n self._weights = [rng.uniform(size=self.rows)] * BATCHES\n\n self.it = 0 # set iterator to 0\n super().__init__()\n\n def as_array(self) -> cupy.ndarray:\n return cupy.concatenate(self._data)\n\n def as_array_labels(self) -> cupy.ndarray:\n return cupy.concatenate(self._labels)\n\n def as_array_weights(self) -> cupy.ndarray:\n return cupy.concatenate(self._weights)\n\n def data(self) -> cupy.ndarray:\n \"\"\"Utility function for obtaining current batch of data.\"\"\"\n return self._data[self.it]\n\n def labels(self) -> cupy.ndarray:\n \"\"\"Utility function for obtaining current batch of label.\"\"\"\n return self._labels[self.it]\n\n def weights(self) -> cupy.ndarray:\n return self._weights[self.it]\n\n def reset(self) -> None:\n \"\"\"Reset the iterator\"\"\"\n self.it = 0\n\n def next(self, input_data: Callable) -> bool:\n \"\"\"Yield the next batch of data.\"\"\"\n if self.it == len(self._data):\n # Return False to let XGBoost know this is the end of iteration\n return False\n\n # input_data is a keyword-only function passed in by XGBoost and has the similar\n # signature to the ``DMatrix`` constructor.\n input_data(data=self.data(), label=self.labels(), weight=self.weights())\n self.it += 1\n return True\n\n\ndef main() -> None:\n rounds = 100\n it = IterForDMatrixDemo()\n\n # Use iterator, must be `QuantileDMatrix`.\n\n # In this demo, the input batches are created using cupy, and the data processing\n # (quantile sketching) will be performed on GPU. If data is loaded with CPU based\n # data structures like numpy or pandas, then the processing step will be performed\n # on CPU instead.\n m_with_it = xgboost.QuantileDMatrix(it)\n\n # Use regular DMatrix.\n m = xgboost.DMatrix(\n it.as_array(), it.as_array_labels(), weight=it.as_array_weights()\n )\n\n assert m_with_it.num_col() == m.num_col()\n assert m_with_it.num_row() == m.num_row()\n # Tree method must be `hist`.\n reg_with_it = xgboost.train(\n {\"tree_method\": \"hist\", \"device\": \"cuda\"},\n m_with_it,\n num_boost_round=rounds,\n evals=[(m_with_it, \"Train\")],\n )\n predict_with_it = reg_with_it.predict(m_with_it)\n\n reg = xgboost.train(\n {\"tree_method\": \"hist\", \"device\": \"cuda\"},\n m,\n num_boost_round=rounds,\n evals=[(m, \"Train\")],\n )\n predict = reg.predict(m)\n\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "demo/guide-python/quantile_regression.py", "content": "\"\"\"\nQuantile Regression\n===================\n\n .. versionadded:: 2.0.0\n\nThe script is inspired by this awesome example in sklearn:\nhttps://scikit-learn.org/stable/auto_examples/ensemble/plot_gradient_boosting_quantile.html\n\n.. note::\n\n The feature is only supported using the Python, R, and C packages. In addition, quantile\n crossing can happen due to limitation in the algorithm.\n\n\"\"\"\n\nimport argparse\nfrom typing import Dict\n\nimport numpy as np\nfrom sklearn.model_selection import train_test_split\n\nimport xgboost as xgb\n\n\ndef f(x: np.ndarray) -> np.ndarray:\n \"\"\"The function to predict.\"\"\"\n return x * np.sin(x)\n\n\ndef quantile_loss(args: argparse.Namespace) -> None:\n \"\"\"Train a quantile regression model.\"\"\"\n rng = np.random.RandomState(1994)\n # Generate a synthetic dataset for demo, the generate process is from the sklearn\n # example.\n X = np.atleast_2d(rng.uniform(0, 10.0, size=1000)).T\n expected_y = f(X).ravel()\n\n sigma = 0.5 + X.ravel() / 10.0\n noise = rng.lognormal(sigma=sigma) - np.exp(sigma**2.0 / 2.0)\n y = expected_y + noise\n\n # Train on 0.05 and 0.95 quantiles. The model is similar to multi-class and\n # multi-target models.\n alpha = np.array([0.05, 0.5, 0.95])\n evals_result: Dict[str, Dict] = {}\n\n X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=rng)\n # We will be using the `hist` tree method, quantile DMatrix can be used to preserve\n # memory (which has nothing to do with quantile regression itself, see its document\n # for details).\n # Do not use the `exact` tree method for quantile regression, otherwise the\n # performance might drop.\n Xy = xgb.QuantileDMatrix(X_train, y_train)\n # use Xy as a reference\n Xy_test = xgb.QuantileDMatrix(X_test, y_test, ref=Xy)\n\n booster = xgb.train(\n {\n # Use the quantile objective function.\n \"objective\": \"reg:quantileerror\",\n \"tree_method\": \"hist\",\n \"quantile_alpha\": alpha,\n # Let's try not to overfit.\n \"learning_rate\": 0.04,\n \"max_depth\": 5,\n \"multi_strategy\": args.multi_strategy,\n \"device\": args.device,\n },\n Xy,\n num_boost_round=32,\n early_stopping_rounds=2,\n # The evaluation result is a weighted average across multiple quantiles.\n evals=[(Xy, \"Train\"), (Xy_test, \"Test\")],\n evals_result=evals_result,\n )\n xx = np.atleast_2d(np.linspace(0, 10, 1000)).T\n scores = booster.inplace_predict(xx)\n # dim 1 is the quantiles\n assert scores.shape[0] == xx.shape[0]\n assert scores.shape[1] == alpha.shape[0]\n\n y_lower = scores[:, 0] # alpha=0.05\n y_med = scores[:, 1] # alpha=0.5, median\n y_upper = scores[:, 2] # alpha=0.95\n\n # Train a mse model for comparison\n booster = xgb.train(\n {\n \"objective\": \"reg:squarederror\",\n \"tree_method\": \"hist\",\n # Let's try not to overfit.\n \"learning_rate\": 0.04,\n \"max_depth\": 5,\n },\n Xy,\n num_boost_round=32,\n early_stopping_rounds=2,\n evals=[(Xy, \"Train\"), (Xy_test, \"Test\")],\n evals_result=evals_result,\n )\n xx = np.atleast_2d(np.linspace(0, 10, 1000)).T\n y_pred = booster.inplace_predict(xx)\n\n if args.plot:\n from matplotlib import pyplot as plt\n\n fig = plt.figure(figsize=(10, 10))\n plt.plot(xx, f(xx), \"g:\", linewidth=3, label=r\"$f(x) = x\\,\\sin(x)$\")\n plt.plot(X_test, y_test, \"b.\", markersize=10, label=\"Test observations\")\n plt.plot(xx, y_med, \"r-\", label=\"Predicted median\")\n plt.plot(xx, y_pred, \"m-\", label=\"Predicted mean\")\n plt.plot(xx, y_upper, \"k-\")\n plt.plot(xx, y_lower, \"k-\")\n plt.fill_between(\n xx.ravel(), y_lower, y_upper, alpha=0.4, label=\"Predicted 90% interval\"\n )\n plt.xlabel(\"$x$\")\n plt.ylabel(\"$f(x)$\")\n plt.ylim(-10, 25)\n plt.legend(loc=\"upper left\")\n plt.show()\n\n\nif __name__ == \"__main__\":\n parser = argparse.ArgumentParser()\n parser.add_argument(\n \"--plot\",\n action=\"store_true\",\n help=\"Specify it to enable plotting the outputs.\",\n )\n parser.add_argument(\n \"--multi_strategy\",\n choices=[\"multi_output_tree\", \"one_output_per_tree\"],\n default=\"one_output_per_tree\",\n help=\"See the parameter `multi_strategy` for more info. (Experimental)\",\n )\n parser.add_argument(\"--device\", choices=[\"cpu\", \"cuda\"], default=\"cpu\")\n args = parser.parse_args()\n quantile_loss(args)\n" }, { "path": "demo/guide-python/sklearn_evals_result.py", "content": "\"\"\"\nDemo for accessing the xgboost eval metrics by using sklearn interface\n======================================================================\n\"\"\"\n\nimport numpy as np\nfrom sklearn.datasets import make_hastie_10_2\n\nimport xgboost as xgb\n\nX, y = make_hastie_10_2(n_samples=2000, random_state=42)\n\n# Map labels from {-1, 1} to {0, 1}\nlabels, y = np.unique(y, return_inverse=True)\n\nX_train, X_test = X[:1600], X[1600:]\ny_train, y_test = y[:1600], y[1600:]\n\nparam_dist = {\"objective\": \"binary:logistic\", \"n_estimators\": 2}\n\nclf = xgb.XGBModel(\n **param_dist,\n eval_metric=\"logloss\",\n)\n# Or you can use: clf = xgb.XGBClassifier(**param_dist)\n\nclf.fit(\n X_train,\n y_train,\n eval_set=[(X_train, y_train), (X_test, y_test)],\n verbose=True,\n)\n\n# Load evals result by calling the evals_result() function\nevals_result = clf.evals_result()\n\nprint(\"Access logloss metric directly from validation_0:\")\nprint(evals_result[\"validation_0\"][\"logloss\"])\n\nprint(\"\")\nprint(\"Access metrics through a loop:\")\nfor e_name, e_mtrs in evals_result.items():\n print(\"- {}\".format(e_name))\n for e_mtr_name, e_mtr_vals in e_mtrs.items():\n print(\" - {}\".format(e_mtr_name))\n print(\" - {}\".format(e_mtr_vals))\n\nprint(\"\")\nprint(\"Access complete dict:\")\nprint(evals_result)\n" }, { "path": "demo/guide-python/sklearn_examples.py", "content": "\"\"\"\nCollection of examples for using sklearn interface\n==================================================\n\nFor an introduction to XGBoost's scikit-learn estimator interface, see\n:doc:`/python/sklearn_estimator`.\n\nCreated on 1 Apr 2015\n\n@author: Jamie Hall\n\"\"\"\n\nimport pickle\nfrom urllib.error import HTTPError\n\nimport numpy as np\nfrom sklearn.datasets import (\n fetch_california_housing,\n load_digits,\n load_iris,\n make_regression,\n)\nfrom sklearn.metrics import confusion_matrix, mean_squared_error\nfrom sklearn.model_selection import GridSearchCV, KFold, train_test_split\n\nimport xgboost as xgb\n\nrng = np.random.RandomState(31337)\n\nprint(\"Zeros and Ones from the Digits dataset: binary classification\")\ndigits = load_digits(n_class=2)\ny = digits[\"target\"]\nX = digits[\"data\"]\nkf = KFold(n_splits=2, shuffle=True, random_state=rng)\nfor train_index, test_index in kf.split(X):\n xgb_model = xgb.XGBClassifier(n_jobs=1).fit(X[train_index], y[train_index])\n predictions = xgb_model.predict(X[test_index])\n actuals = y[test_index]\n print(confusion_matrix(actuals, predictions))\n\nprint(\"Iris: multiclass classification\")\niris = load_iris()\ny = iris[\"target\"]\nX = iris[\"data\"]\nkf = KFold(n_splits=2, shuffle=True, random_state=rng)\nfor train_index, test_index in kf.split(X):\n xgb_model = xgb.XGBClassifier(n_jobs=1).fit(X[train_index], y[train_index])\n predictions = xgb_model.predict(X[test_index])\n actuals = y[test_index]\n print(confusion_matrix(actuals, predictions))\n\nprint(\"California Housing: regression\")\n\ntry:\n X, y = fetch_california_housing(return_X_y=True)\nexcept HTTPError:\n # Use a synthetic dataset instead if we couldn't\n X, y = make_regression(n_samples=20640, n_features=8, random_state=1234)\n\nkf = KFold(n_splits=2, shuffle=True, random_state=rng)\nfor train_index, test_index in kf.split(X):\n xgb_model = xgb.XGBRegressor(n_jobs=1).fit(X[train_index], y[train_index])\n predictions = xgb_model.predict(X[test_index])\n actuals = y[test_index]\n print(mean_squared_error(actuals, predictions))\n\nprint(\"Parameter optimization\")\nxgb_model = xgb.XGBRegressor(n_jobs=1)\nclf = GridSearchCV(\n xgb_model,\n {\"max_depth\": [2, 4], \"n_estimators\": [50, 100]},\n verbose=1,\n n_jobs=1,\n cv=3,\n)\nclf.fit(X, y)\nprint(clf.best_score_)\nprint(clf.best_params_)\n\n# The sklearn API models are picklable\nprint(\"Pickling sklearn API models\")\n# must open in binary format to pickle\npickle.dump(clf, open(\"best_calif.pkl\", \"wb\"))\nclf2 = pickle.load(open(\"best_calif.pkl\", \"rb\"))\nprint(np.allclose(clf.predict(X), clf2.predict(X)))\n\n# Early-stopping\n\nX = digits[\"data\"]\ny = digits[\"target\"]\nX_train, X_test, y_train, y_test = train_test_split(X, y, random_state=0)\nclf = xgb.XGBClassifier(n_jobs=1, early_stopping_rounds=10, eval_metric=\"auc\")\nclf.fit(X_train, y_train, eval_set=[(X_test, y_test)])\n" }, { "path": "demo/guide-python/sklearn_parallel.py", "content": "\"\"\"\nDemo for using xgboost with sklearn\n===================================\n\"\"\"\n\nimport multiprocessing\nfrom urllib.error import HTTPError\n\nfrom sklearn.datasets import fetch_california_housing, make_regression\nfrom sklearn.model_selection import GridSearchCV\n\nimport xgboost as xgb\n\nif __name__ == \"__main__\":\n print(\"Parallel Parameter optimization\")\n try:\n X, y = fetch_california_housing(return_X_y=True)\n except HTTPError:\n # Use a synthetic dataset instead if we couldn't\n X, y = make_regression(n_samples=20640, n_features=8, random_state=1234)\n # Make sure the number of threads is balanced.\n xgb_model = xgb.XGBRegressor(\n n_jobs=multiprocessing.cpu_count() // 2, tree_method=\"hist\"\n )\n clf = GridSearchCV(\n xgb_model,\n {\"max_depth\": [2, 4, 6], \"n_estimators\": [50, 100, 200]},\n verbose=1,\n n_jobs=2,\n )\n clf.fit(X, y)\n print(clf.best_score_)\n print(clf.best_params_)\n" }, { "path": "demo/guide-python/spark_estimator_examples.py", "content": "\"\"\"\nCollection of examples for using xgboost.spark estimator interface\n==================================================================\n\n@author: Weichen Xu\n\"\"\"\n\nimport numpy as np\nimport sklearn.datasets\nfrom pyspark.ml.evaluation import MulticlassClassificationEvaluator, RegressionEvaluator\nfrom pyspark.ml.linalg import Vectors\nfrom pyspark.sql import DataFrame, SparkSession\nfrom pyspark.sql.functions import rand\nfrom sklearn.model_selection import train_test_split\n\nfrom xgboost.spark import SparkXGBClassifier, SparkXGBRegressor\n\nspark = SparkSession.builder.master(\"local[*]\").getOrCreate()\n\n\ndef create_spark_df(X: np.ndarray, y: np.ndarray) -> DataFrame:\n return spark.createDataFrame(\n spark.sparkContext.parallelize(\n [(Vectors.dense(features), float(label)) for features, label in zip(X, y)]\n ),\n [\"features\", \"label\"],\n )\n\n\n# load diabetes dataset (regression dataset)\ndiabetes_X, diabetes_y = sklearn.datasets.load_diabetes(return_X_y=True)\ndiabetes_X_train, diabetes_X_test, diabetes_y_train, diabetes_y_test = train_test_split(\n diabetes_X, diabetes_y, test_size=0.3, shuffle=True\n)\n\ndiabetes_train_spark_df = create_spark_df(diabetes_X_train, diabetes_y_train)\ndiabetes_test_spark_df = create_spark_df(diabetes_X_test, diabetes_y_test)\n\n# train xgboost regressor model\nxgb_regressor = SparkXGBRegressor(max_depth=5)\nxgb_regressor_model = xgb_regressor.fit(diabetes_train_spark_df)\n\ntransformed_diabetes_test_spark_df = xgb_regressor_model.transform(\n diabetes_test_spark_df\n)\nregressor_evaluator = RegressionEvaluator(metricName=\"rmse\")\nprint(\n f\"regressor rmse={regressor_evaluator.evaluate(transformed_diabetes_test_spark_df)}\"\n)\n\ndiabetes_train_spark_df2 = diabetes_train_spark_df.withColumn(\n \"validationIndicatorCol\", rand(1) > 0.7\n)\n\n# train xgboost regressor model with validation dataset\nxgb_regressor2 = SparkXGBRegressor(\n max_depth=5, validation_indicator_col=\"validationIndicatorCol\"\n)\nxgb_regressor_model2 = xgb_regressor2.fit(diabetes_train_spark_df2)\ntransformed_diabetes_test_spark_df2 = xgb_regressor_model2.transform(\n diabetes_test_spark_df\n)\nprint(\n f\"regressor2 rmse={regressor_evaluator.evaluate(transformed_diabetes_test_spark_df2)}\"\n)\n\n\n# load iris dataset (classification dataset)\niris_X, iris_y = sklearn.datasets.load_iris(return_X_y=True)\niris_X_train, iris_X_test, iris_y_train, iris_y_test = train_test_split(\n iris_X, iris_y, test_size=0.3, shuffle=True\n)\n\niris_train_spark_df = create_spark_df(iris_X_train, iris_y_train)\niris_test_spark_df = create_spark_df(iris_X_test, iris_y_test)\n\n# train xgboost classifier model\nxgb_classifier = SparkXGBClassifier(max_depth=5)\nxgb_classifier_model = xgb_classifier.fit(iris_train_spark_df)\n\ntransformed_iris_test_spark_df = xgb_classifier_model.transform(iris_test_spark_df)\nclassifier_evaluator = MulticlassClassificationEvaluator(metricName=\"f1\")\nprint(f\"classifier f1={classifier_evaluator.evaluate(transformed_iris_test_spark_df)}\")\n\niris_train_spark_df2 = iris_train_spark_df.withColumn(\n \"validationIndicatorCol\", rand(1) > 0.7\n)\n\n# train xgboost classifier model with validation dataset\nxgb_classifier2 = SparkXGBClassifier(\n max_depth=5, validation_indicator_col=\"validationIndicatorCol\"\n)\nxgb_classifier_model2 = xgb_classifier2.fit(iris_train_spark_df2)\ntransformed_iris_test_spark_df2 = xgb_classifier_model2.transform(iris_test_spark_df)\nprint(\n f\"classifier2 f1={classifier_evaluator.evaluate(transformed_iris_test_spark_df2)}\"\n)\n\nspark.stop()\n" }, { "path": "demo/guide-python/update_process.py", "content": "\"\"\"\nDemo for using `process_type` with `prune` and `refresh`\n========================================================\n\nModifying existing trees is not a well established use for XGBoost, so feel free to\nexperiment.\n\n\"\"\"\n\nfrom urllib.error import HTTPError\n\nimport numpy as np\nfrom sklearn.datasets import fetch_california_housing, make_regression\n\nimport xgboost as xgb\n\n\ndef main() -> None:\n n_rounds = 32\n\n try:\n X, y = fetch_california_housing(return_X_y=True)\n except HTTPError:\n # Use a synthetic dataset instead if we couldn't\n X, y = make_regression(n_samples=20640, n_features=8, random_state=1234)\n\n # Train a model first\n X_train = X[: X.shape[0] // 2]\n y_train = y[: y.shape[0] // 2]\n Xy = xgb.DMatrix(X_train, y_train)\n evals_result: xgb.callback.EvaluationMonitor.EvalsLog = {}\n booster = xgb.train(\n {\"tree_method\": \"hist\", \"max_depth\": 6, \"device\": \"cuda\"},\n Xy,\n num_boost_round=n_rounds,\n evals=[(Xy, \"Train\")],\n evals_result=evals_result,\n )\n SHAP = booster.predict(Xy, pred_contribs=True)\n\n # Refresh the leaf value and tree statistic\n X_refresh = X[X.shape[0] // 2 :]\n y_refresh = y[y.shape[0] // 2 :]\n Xy_refresh = xgb.DMatrix(X_refresh, y_refresh)\n # The model will adapt to other half of the data by changing leaf value (no change in\n # split condition) with refresh_leaf set to True.\n refresh_result: xgb.callback.EvaluationMonitor.EvalsLog = {}\n refreshed = xgb.train(\n {\"process_type\": \"update\", \"updater\": \"refresh\", \"refresh_leaf\": True},\n Xy_refresh,\n num_boost_round=n_rounds,\n xgb_model=booster,\n evals=[(Xy, \"Original\"), (Xy_refresh, \"Train\")],\n evals_result=refresh_result,\n )\n\n # Refresh the model without changing the leaf value, but tree statistic including\n # cover and weight are refreshed.\n refresh_result = {}\n refreshed = xgb.train(\n {\"process_type\": \"update\", \"updater\": \"refresh\", \"refresh_leaf\": False},\n Xy_refresh,\n num_boost_round=n_rounds,\n xgb_model=booster,\n evals=[(Xy, \"Original\"), (Xy_refresh, \"Train\")],\n evals_result=refresh_result,\n )\n # Without refreshing the leaf value, resulting trees should be the same with original\n # model except for accumulated statistic. The rtol is for floating point error in\n # prediction.\n np.testing.assert_allclose(\n refresh_result[\"Original\"][\"rmse\"], evals_result[\"Train\"][\"rmse\"], rtol=1e-5\n )\n # But SHAP value is changed as cover in tree nodes are changed.\n refreshed_SHAP = refreshed.predict(Xy, pred_contribs=True)\n assert not np.allclose(SHAP, refreshed_SHAP, rtol=1e-3)\n\n # Prune the trees with smaller max_depth\n X_update = X_train\n y_update = y_train\n Xy_update = xgb.DMatrix(X_update, y_update)\n\n prune_result: xgb.callback.EvaluationMonitor.EvalsLog = {}\n pruned = xgb.train(\n {\"process_type\": \"update\", \"updater\": \"prune\", \"max_depth\": 2},\n Xy_update,\n num_boost_round=n_rounds,\n xgb_model=booster,\n evals=[(Xy, \"Original\"), (Xy_update, \"Train\")],\n evals_result=prune_result,\n )\n # Have a smaller model, but similar accuracy.\n np.testing.assert_allclose(\n np.array(prune_result[\"Original\"][\"rmse\"]),\n np.array(prune_result[\"Train\"][\"rmse\"]),\n atol=1e-5,\n )\n\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "demo/kaggle-higgs/README.md", "content": "Highlights\n=====\nHiggs challenge ends recently, xgboost is being used by many users. This list highlights the xgboost solutions of players\n* Blogpost by phunther: [Winning solution of Kaggle Higgs competition: what a single model can do](http://no2147483647.wordpress.com/2014/09/17/winning-solution-of-kaggle-higgs-competition-what-a-single-model-can-do/)\n* The solution by Tianqi Chen and Tong He [Link](https://github.com/hetong007/higgsml)\n\nGuide for Kaggle Higgs Challenge\n=====\n\nThis is the folder giving example of how to use XGBoost Python Module to run Kaggle Higgs competition\n\nThis script will achieve about 3.600 AMS score in public leaderboard. To get start, you need do following step:\n\n1. Compile the XGBoost python lib\n```bash\ncd ../..\nmake\n```\n\n2. Put training.csv test.csv on folder './data' (you can create a symbolic link)\n\n3. Run ./run.sh\n\nSpeed\n=====\nspeedtest.py compares xgboost's speed on this dataset with sklearn.GBM\n\n\nUsing R module\n=====\n* Alternatively, you can run using R, higgs-train.R and higgs-pred.R.\n" }, { "path": "demo/kaggle-higgs/higgs-cv.py", "content": "#!/usr/bin/python\nimport numpy as np\n\nimport xgboost as xgb\n\n### load data in do training\ntrain = np.loadtxt('./data/training.csv', delimiter=',', skiprows=1, converters={32: lambda x:int(x=='s'.encode('utf-8')) } )\nlabel = train[:,32]\ndata = train[:,1:31]\nweight = train[:,31]\ndtrain = xgb.DMatrix( data, label=label, missing = -999.0, weight=weight )\nparam = {'max_depth':6, 'eta':0.1, 'objective':'binary:logitraw', 'nthread':4}\nnum_round = 120\n\nprint ('running cross validation, with preprocessing function')\n# define the preprocessing function\n# used to return the preprocessed training, test data, and parameter\n# we can use this to do weight rescale, etc.\n# as a example, we try to set scale_pos_weight\ndef fpreproc(dtrain, dtest, param):\n label = dtrain.get_label()\n ratio = float(np.sum(label == 0)) / np.sum(label==1)\n param['scale_pos_weight'] = ratio\n wtrain = dtrain.get_weight()\n wtest = dtest.get_weight()\n sum_weight = sum(wtrain) + sum(wtest)\n wtrain *= sum_weight / sum(wtrain)\n wtest *= sum_weight / sum(wtest)\n dtrain.set_weight(wtrain)\n dtest.set_weight(wtest)\n return (dtrain, dtest, param)\n\n# do cross validation, for each fold\n# the dtrain, dtest, param will be passed into fpreproc\n# then the return value of fpreproc will be used to generate\n# results of that fold\nxgb.cv(param, dtrain, num_round, nfold=5,\n metrics={'ams@0.15', 'auc'}, seed = 0, fpreproc = fpreproc)\n" }, { "path": "demo/kaggle-higgs/higgs-numpy.py", "content": "#!/usr/bin/python\n# this is the example script to use xgboost to train\nimport numpy as np\n\nimport xgboost as xgb\n\ntest_size = 550000\n\n# path to where the data lies\ndpath = 'data'\n\n# load in training data, directly use numpy\ndtrain = np.loadtxt( dpath+'/training.csv', delimiter=',', skiprows=1, converters={32: lambda x:int(x=='s'.encode('utf-8')) } )\nprint ('finish loading from csv ')\n\nlabel = dtrain[:,32]\ndata = dtrain[:,1:31]\n# rescale weight to make it same as test set\nweight = dtrain[:,31] * float(test_size) / len(label)\n\nsum_wpos = sum( weight[i] for i in range(len(label)) if label[i] == 1.0 )\nsum_wneg = sum( weight[i] for i in range(len(label)) if label[i] == 0.0 )\n\n# print weight statistics\nprint ('weight statistics: wpos=%g, wneg=%g, ratio=%g' % ( sum_wpos, sum_wneg, sum_wneg/sum_wpos ))\n\n# construct xgboost.DMatrix from numpy array, treat -999.0 as missing value\nxgmat = xgb.DMatrix( data, label=label, missing = -999.0, weight=weight )\n\n# setup parameters for xgboost\nparam = {}\n# use logistic regression loss, use raw prediction before logistic transformation\n# since we only need the rank\nparam['objective'] = 'binary:logitraw'\n# scale weight of positive examples\nparam['scale_pos_weight'] = sum_wneg/sum_wpos\nparam['eta'] = 0.1\nparam['max_depth'] = 6\nparam['eval_metric'] = 'auc'\nparam['nthread'] = 16\n\n# you can directly throw param in, though we want to watch multiple metrics here\nplst = list(param.items())+[('eval_metric', 'ams@0.15')]\n\nwatchlist = [ (xgmat,'train') ]\n# boost 120 trees\nnum_round = 120\nprint ('loading data end, start to boost trees')\nbst = xgb.train( plst, xgmat, num_round, watchlist );\n# save out model\nbst.save_model('higgs.model')\n\nprint ('finish training')\n" }, { "path": "demo/kaggle-higgs/higgs-pred.R", "content": "# install xgboost package, see R-package in root folder\nrequire(xgboost)\nrequire(methods)\n\nmodelfile <- \"higgs.model\"\noutfile <- \"higgs.pred.csv\"\ndtest <- read.csv(\"data/test.csv\", header = TRUE)\ndata <- as.matrix(dtest[2:31])\nidx <- dtest[[1]]\n\nxgmat <- xgb.DMatrix(data, missing = -999.0)\nbst <- xgb.load(modelfile = modelfile)\nypred <- predict(bst, xgmat)\n\nrorder <- rank(ypred, ties.method = \"first\")\n\nthreshold <- 0.15\n# to be completed\nntop <- length(rorder) - as.integer(threshold * length(rorder))\nplabel <- ifelse(rorder > ntop, \"s\", \"b\")\noutdata <- list(\"EventId\" = idx,\n \"RankOrder\" = rorder,\n \"Class\" = plabel)\nwrite.csv(outdata, file = outfile, quote = FALSE, row.names = FALSE)\n" }, { "path": "demo/kaggle-higgs/higgs-pred.py", "content": "#!/usr/bin/python\n# make prediction\nimport numpy as np\n\nimport xgboost as xgb\n\n# path to where the data lies\ndpath = 'data'\n\nmodelfile = 'higgs.model'\noutfile = 'higgs.pred.csv'\n# make top 15% as positive\nthreshold_ratio = 0.15\n\n# load in training data, directly use numpy\ndtest = np.loadtxt( dpath+'/test.csv', delimiter=',', skiprows=1 )\ndata = dtest[:,1:31]\nidx = dtest[:,0]\n\nprint ('finish loading from csv ')\nxgmat = xgb.DMatrix( data, missing = -999.0 )\nbst = xgb.Booster({'nthread':16}, model_file = modelfile)\nypred = bst.predict( xgmat )\n\nres = [ ( int(idx[i]), ypred[i] ) for i in range(len(ypred)) ]\n\nrorder = {}\nfor k, v in sorted( res, key = lambda x:-x[1] ):\n rorder[ k ] = len(rorder) + 1\n\n# write out predictions\nntop = int( threshold_ratio * len(rorder ) )\nfo = open(outfile, 'w')\nnhit = 0\nntot = 0\nfo.write('EventId,RankOrder,Class\\n')\nfor k, v in res:\n if rorder[k] <= ntop:\n lb = 's'\n nhit += 1\n else:\n lb = 'b'\n # change output rank order to follow Kaggle convention\n fo.write('%s,%d,%s\\n' % ( k, len(rorder)+1-rorder[k], lb ) )\n ntot += 1\nfo.close()\n\nprint ('finished writing into prediction file')\n" }, { "path": "demo/kaggle-higgs/higgs-train.R", "content": "# install xgboost package, see R-package in root folder\nrequire(xgboost)\nrequire(methods)\n\ntestsize <- 550000\n\ndtrain <- read.csv(\"data/training.csv\", header = TRUE)\ndtrain[33] <- dtrain[33] == \"s\"\nlabel <- as.numeric(dtrain[[33]])\ndata <- as.matrix(dtrain[2:31])\nweight <- as.numeric(dtrain[[32]]) * testsize / length(label)\n\nsumwpos <- sum(weight * (label == 1.0))\nsumwneg <- sum(weight * (label == 0.0))\nprint(paste(\"weight statistics: wpos=\", sumwpos, \"wneg=\", sumwneg, \"ratio=\", sumwneg / sumwpos))\n\nxgmat <- xgb.DMatrix(data, label = label, weight = weight, missing = -999.0)\nparam <- list(\"objective\" = \"binary:logitraw\",\n \"scale_pos_weight\" = sumwneg / sumwpos,\n \"bst:eta\" = 0.1,\n \"bst:max_depth\" = 6,\n \"eval_metric\" = \"auc\",\n \"eval_metric\" = \"ams@0.15\",\n \"nthread\" = 16)\nwatchlist <- list(\"train\" = xgmat)\nnrounds <- 120\nprint(\"loading data end, start to boost trees\")\nbst <- xgb.train(param, xgmat, nrounds, watchlist)\n# save out model\nxgb.save(bst, \"higgs.model\")\nprint('finish training')\n" }, { "path": "demo/kaggle-higgs/run.sh", "content": "#!/bin/bash\n\npython -u higgs-numpy.py\nret=$?\nif [[ $ret != 0 ]]; then\n echo \"ERROR in higgs-numpy.py\"\n exit $ret\nfi\npython -u higgs-pred.py\nret=$?\nif [[ $ret != 0 ]]; then\n echo \"ERROR in higgs-pred.py\"\n exit $ret\nfi\n" }, { "path": "demo/kaggle-higgs/speedtest.R", "content": "# install xgboost package, see R-package in root folder\nrequire(xgboost)\nrequire(gbm)\nrequire(methods)\n\ntestsize <- 550000\n\ndtrain <- read.csv(\"data/training.csv\", header = TRUE, nrows = 350001)\ndtrain$Label <- as.numeric(dtrain$Label == 's')\n# gbm.time = system.time({\n# gbm.model <- gbm(Label ~ ., data = dtrain[, -c(1,32)], n.trees = 120,\n# interaction.depth = 6, shrinkage = 0.1, bag.fraction = 1,\n# verbose = TRUE)\n# })\n# print(gbm.time)\n# Test result: 761.48 secs\n\n# dtrain[33] <- dtrain[33] == \"s\"\n# label <- as.numeric(dtrain[[33]])\ndata <- as.matrix(dtrain[2:31])\nweight <- as.numeric(dtrain[[32]]) * testsize / length(label)\n\nsumwpos <- sum(weight * (label == 1.0))\nsumwneg <- sum(weight * (label == 0.0))\nprint(paste(\"weight statistics: wpos=\", sumwpos, \"wneg=\", sumwneg, \"ratio=\", sumwneg / sumwpos))\n\nxgboost.time <- list()\nthreads <- c(1, 2, 4, 8, 16)\nfor (i in seq_along(threads)){\n thread <- threads[i]\n xgboost.time[[i]] <- system.time({\n xgmat <- xgb.DMatrix(data, label = label, weight = weight, missing = -999.0)\n param <- list(\"objective\" = \"binary:logitraw\",\n \"scale_pos_weight\" = sumwneg / sumwpos,\n \"bst:eta\" = 0.1,\n \"bst:max_depth\" = 6,\n \"eval_metric\" = \"auc\",\n \"eval_metric\" = \"ams@0.15\",\n \"nthread\" = thread)\n watchlist <- list(\"train\" = xgmat)\n nrounds <- 120\n print(\"loading data end, start to boost trees\")\n bst <- xgb.train(param, xgmat, nrounds, watchlist)\n # save out model\n xgb.save(bst, \"higgs.model\")\n print('finish training')\n })\n}\n\nxgboost.time\n# [[1]]\n# user system elapsed\n# 99.015 0.051 98.982\n#\n# [[2]]\n# user system elapsed\n# 100.268 0.317 55.473\n#\n# [[3]]\n# user system elapsed\n# 111.682 0.777 35.963\n#\n# [[4]]\n# user system elapsed\n# 149.396 1.851 32.661\n#\n# [[5]]\n# user system elapsed\n# 157.390 5.988 40.949\n" }, { "path": "demo/kaggle-higgs/speedtest.py", "content": "#!/usr/bin/python\n# this is the example script to use xgboost to train\nimport time\n\nimport numpy as np\nfrom sklearn.ensemble import GradientBoostingClassifier\n\nimport xgboost as xgb\n\ntest_size = 550000\n\n# path to where the data lies\ndpath = 'data'\n\n# load in training data, directly use numpy\ndtrain = np.loadtxt( dpath+'/training.csv', delimiter=',', skiprows=1, converters={32: lambda x:int(x=='s') } )\nprint ('finish loading from csv ')\n\nlabel = dtrain[:,32]\ndata = dtrain[:,1:31]\n# rescale weight to make it same as test set\nweight = dtrain[:,31] * float(test_size) / len(label)\n\nsum_wpos = sum( weight[i] for i in range(len(label)) if label[i] == 1.0 )\nsum_wneg = sum( weight[i] for i in range(len(label)) if label[i] == 0.0 )\n\n# print weight statistics\nprint ('weight statistics: wpos=%g, wneg=%g, ratio=%g' % ( sum_wpos, sum_wneg, sum_wneg/sum_wpos ))\n\n# construct xgboost.DMatrix from numpy array, treat -999.0 as missing value\nxgmat = xgb.DMatrix( data, label=label, missing = -999.0, weight=weight )\n\n# setup parameters for xgboost\nparam = {}\n# use logistic regression loss\nparam['objective'] = 'binary:logitraw'\n# scale weight of positive examples\nparam['scale_pos_weight'] = sum_wneg/sum_wpos\nparam['bst:eta'] = 0.1\nparam['bst:max_depth'] = 6\nparam['eval_metric'] = 'auc'\nparam['nthread'] = 4\n\nplst = param.items()+[('eval_metric', 'ams@0.15')]\n\nwatchlist = [ (xgmat,'train') ]\n# boost 10 trees\nnum_round = 10\nprint ('loading data end, start to boost trees')\nprint (\"training GBM from sklearn\")\ntmp = time.time()\ngbm = GradientBoostingClassifier(n_estimators=num_round, max_depth=6, verbose=2)\ngbm.fit(data, label)\nprint (\"sklearn.GBM costs: %s seconds\" % str(time.time() - tmp))\n#raw_input()\nprint (\"training xgboost\")\nthreads = [1, 2, 4, 16]\nfor i in threads:\n param['nthread'] = i\n tmp = time.time()\n plst = param.items()+[('eval_metric', 'ams@0.15')]\n bst = xgb.train( plst, xgmat, num_round, watchlist );\n print (\"XGBoost with %d thread costs: %s seconds\" % (i, str(time.time() - tmp)))\n\nprint ('finish training')\n" }, { "path": "demo/kaggle-otto/README.MD", "content": "Benchmark for Otto Group Competition\n=========\n\nThis is a folder containing the benchmark for the [Otto Group Competition on Kaggle](http://www.kaggle.com/c/otto-group-product-classification-challenge).\n\n## Getting started\n\n1. Put `train.csv` and `test.csv` under the `data` folder\n2. Run the script\n3. Submit the `submission.csv`\n\nThe parameter `nthread` controls the number of cores to run on, please set it to suit your machine.\n\n## R-package\n\nTo install the R-package of xgboost, please run\n\n```r\ninstall.packages(\"xgboost\", repos = \"https://cran.r-project.org\")\n```\n\nWindows users may need to install [RTools](http://cran.r-project.org/bin/windows/Rtools/) first.\n" }, { "path": "demo/kaggle-otto/otto_train_pred.R", "content": "require(xgboost)\nrequire(methods)\n\ntrain <- read.csv('data/train.csv', header = TRUE, stringsAsFactors = FALSE)\ntest <- read.csv('data/test.csv', header = TRUE, stringsAsFactors = FALSE)\ntrain <- train[, -1]\ntest <- test[, -1]\n\ny <- train[, ncol(train)]\ny <- gsub('Class_', '', y, fixed = TRUE)\ny <- as.integer(y) - 1 # xgboost take features in [0,numOfClass)\n\nx <- rbind(train[, -ncol(train)], test)\nx <- as.matrix(x)\nx <- matrix(as.numeric(x), nrow(x), ncol(x))\ntrind <- seq_along(y)\nteind <- (nrow(train) + 1):nrow(x)\n\n# Set necessary parameter\nparam <- list(\"objective\" = \"multi:softprob\",\n \"eval_metric\" = \"mlogloss\",\n \"num_class\" = 9,\n \"nthread\" = 8)\n\n# Run Cross Validation\ncv.nrounds <- 50\nbst.cv <- xgb.cv(\n param = param\n , data = x[trind, ]\n , label = y\n , nfold = 3\n , nrounds = cv.nrounds\n)\n\n# Train the model\nnrounds <- 50\nbst <- xgboost(param = param, data = x[trind, ], label = y, nrounds = nrounds)\n\n# Make prediction\npred <- predict(bst, x[teind, ])\npred <- matrix(pred, 9, length(pred) / 9)\npred <- t(pred)\n\n# Output submission\npred <- format(pred, digits = 2, scientific = FALSE) # shrink the size of submission\npred <- data.frame(seq_len(nrow(pred)), pred)\nnames(pred) <- c('id', paste0('Class_', 1:9))\nwrite.csv(pred, file = 'submission.csv', quote = FALSE, row.names = FALSE)\n" }, { "path": "demo/kaggle-otto/understandingXGBoostModel.Rmd", "content": "---\ntitle: \"Understanding XGBoost Model on Otto Dataset\"\nauthor: \"Michaël Benesty\"\noutput:\n rmarkdown::html_vignette:\n css: ../../R-package/vignettes/vignette.css\n number_sections: yes\n toc: yes\n---\n\nIntroduction\n============\n\n**XGBoost** is an implementation of the famous gradient boosting algorithm. This model is often described as a *blackbox*, meaning it works well but it is not trivial to understand how. Indeed, the model is made of hundreds (thousands?) of decision trees. You may wonder how possible a human would be able to have a general view of the model?\n\nWhile XGBoost is known for its fast speed and accurate predictive power, it also comes with various functions to help you understand the model.\nThe purpose of this RMarkdown document is to demonstrate how easily we can leverage the functions already implemented in **XGBoost R** package. Of course, everything showed below can be applied to the dataset you may have to manipulate at work or wherever!\n\nFirst we will prepare the **Otto** dataset and train a model, then we will generate two visualisations to get a clue of what is important to the model, finally, we will see how we can leverage these information.\n\nPreparation of the data\n=======================\n\nThis part is based on the **R** tutorial example by [Tong He](https://github.com/dmlc/xgboost/blob/master/demo/kaggle-otto/otto_train_pred.R)\n\nFirst, let's load the packages and the dataset.\n\n```{r loading}\nrequire(xgboost)\nrequire(methods)\nrequire(data.table)\nrequire(magrittr)\ntrain <- fread('data/train.csv', header = TRUE, stringsAsFactors = FALSE)\ntest <- fread('data/test.csv', header = TRUE, stringsAsFactors = FALSE)\n```\n> `magrittr` and `data.table` are here to make the code cleaner and much more rapid.\n\nLet's explore the dataset.\n\n```{r explore}\n# Train dataset dimensions\ndim(train)\n\n# Training content\ntrain[1:6, 1:5, with = FALSE]\n\n# Test dataset dimensions\ndim(test)\n\n# Test content\ntest[1:6, 1:5, with = FALSE]\n```\n> We only display the 6 first rows and 5 first columns for convenience\n\nEach *column* represents a feature measured by an `integer`. Each *row* is an **Otto** product.\n\nObviously the first column (`ID`) doesn't contain any useful information.\n\nTo let the algorithm focus on real stuff, we will delete it.\n\n```{r clean, results='hide'}\n# Delete ID column in training dataset\ntrain[, id := NULL]\n\n# Delete ID column in testing dataset\ntest[, id := NULL]\n```\n\nAccording to its description, the **Otto** challenge is a multi class classification challenge. We need to extract the labels (here the name of the different classes) from the dataset. We only have two files (test and training), it seems logical that the training file contains the class we are looking for. Usually the labels is in the first or the last column. We already know what is in the first column, let's check the content of the last one.\n\n```{r searchLabel}\n# Check the content of the last column\ntrain[1:6, ncol(train), with = FALSE]\n# Save the name of the last column\nnameLastCol <- names(train)[ncol(train)]\n```\n\nThe classes are provided as character string in the `r ncol(train)`th column called `r nameLastCol`. As you may know, **XGBoost** doesn't support anything else than numbers. So we will convert classes to `integer`. Moreover, according to the documentation, it should start at `0`.\n\nFor that purpose, we will:\n\n* extract the target column\n* remove `Class_` from each class name\n* convert to `integer`\n* remove `1` to the new value\n\n```{r classToIntegers}\n# Convert from classes to numbers\ny <- train[, nameLastCol, with = FALSE][[1]] %>%\n gsub('Class_', '', ., fixed = TRUE) %>%\n as.integer %>%\n subtract(., 1)\n\n# Display the first 5 levels\ny[1:5]\n```\n\nWe remove label column from training dataset, otherwise **XGBoost** would use it to guess the labels!\n\n```{r deleteCols, results='hide'}\ntrain[, nameLastCol := NULL, with = FALSE]\n```\n\n`data.table` is an awesome implementation of data.frame, unfortunately it is not a format supported natively by **XGBoost**. We need to convert both datasets (training and test) in `numeric` Matrix format.\n\n```{r convertToNumericMatrix}\ntrainMatrix <- train[, lapply(.SD, as.numeric)] %>% as.matrix\ntestMatrix <- test[, lapply(.SD, as.numeric)] %>% as.matrix\n```\n\nModel training\n==============\n\nBefore the learning we will use the cross validation to evaluate the our error rate.\n\nBasically **XGBoost** will divide the training data in `nfold` parts, then **XGBoost** will retain the first part to use it as the test data and perform a training. Then it will reintegrate the first part and retain the second part, do a training and so on...\n\nYou can look at the function documentation for more information.\n\n```{r crossValidation}\nnumberOfClasses <- max(y) + 1\n\nparam <- list(\"objective\" = \"multi:softprob\",\n \"eval_metric\" = \"mlogloss\",\n \"num_class\" = numberOfClasses)\n\ncv.nrounds <- 5\ncv.nfold <- 3\n\nbst.cv <- xgb.cv(param = param, data = trainMatrix, label = y,\n nfold = cv.nfold, nrounds = cv.nrounds)\n```\n> As we can see the error rate is low on the test dataset (for a 5mn trained model).\n\nFinally, we are ready to train the real model!!!\n\n```{r modelTraining}\nnrounds <- 50\nbst <- xgboost(param = param, data = trainMatrix, label = y, nrounds = nrounds)\n```\n\nModel understanding\n===================\n\nFeature importance\n------------------\n\nSo far, we have built a model made of **`r nrounds`** trees.\n\nTo build a tree, the dataset is divided recursively several times. At the end of the process, you get groups of observations (here, these observations are properties regarding **Otto** products).\n\nEach division operation is called a *split*.\n\nEach group at each division level is called a branch and the deepest level is called a *leaf*.\n\nIn the final model, these *leafs* are supposed to be as pure as possible for each tree, meaning in our case that each *leaf* should be made of one class of **Otto** product only (of course it is not true, but that's what we try to achieve in a minimum of splits).\n\n**Not all *splits* are equally important**. Basically the first *split* of a tree will have more impact on the purity that, for instance, the deepest *split*. Intuitively, we understand that the first *split* makes most of the work, and the following *splits* focus on smaller parts of the dataset which have been misclassified by the first *tree*.\n\nIn the same way, in Boosting we try to optimize the misclassification at each round (it is called the *loss*). So the first *tree* will do the big work and the following trees will focus on the remaining, on the parts not correctly learned by the previous *trees*.\n\nThe improvement brought by each *split* can be measured, it is the *gain*.\n\nEach *split* is done on one feature only at one value.\n\nLet's see what the model looks like.\n\n```{r modelDump}\nmodel <- xgb.dump(bst, with.stats = TRUE)\nmodel[1:10]\n```\n> For convenience, we are displaying the first 10 lines of the model only.\n\nClearly, it is not easy to understand what it means.\n\nBasically each line represents a *branch*, there is the *tree* ID, the feature ID, the point where it *splits*, and information regarding the next *branches* (left, right, when the row for this feature is N/A).\n\nHopefully, **XGBoost** offers a better representation: **feature importance**.\n\nFeature importance is about averaging the *gain* of each feature for all *split* and all *trees*.\n\nThen we can use the function `xgb.plot.importance`.\n\n```{r importanceFeature, fig.align='center', fig.height=5, fig.width=10}\n# Get the feature real names\nnames <- dimnames(trainMatrix)[[2]]\n\n# Compute feature importance matrix\nimportance_matrix <- xgb.importance(names, model = bst)\n\n# Nice graph\nxgb.plot.importance(importance_matrix[1:10, ])\n```\n\n> To make it understandable we first extract the column names from the `Matrix`.\n\nInterpretation\n--------------\n\nIn the feature importance above, we can see the first 10 most important features.\n\nThis function gives a color to each bar. These colors represent groups of features. Basically a K-means clustering is applied to group each feature by importance.\n\nFrom here you can take several actions. For instance you can remove the less important feature (feature selection process), or go deeper in the interaction between the most important features and labels.\n\nOr you can just reason about why these features are so important (in **Otto** challenge we can't go this way because there is not enough information).\n\nTree graph\n----------\n\nFeature importance gives you feature weight information but not interaction between features.\n\n**XGBoost R** package have another useful function for that.\n\nPlease, scroll on the right to see the tree.\n\n```{r treeGraph, dpi=1500, fig.align='left'}\nxgb.plot.tree(feature_names = names, model = bst, n_first_tree = 2)\n```\n\nWe are just displaying the first two trees here.\n\nOn simple models the first two trees may be enough. Here, it might not be the case. We can see from the size of the trees that the interaction between features is complicated.\nBesides, **XGBoost** generate `k` trees at each round for a `k`-classification problem. Therefore the two trees illustrated here are trying to classify data into different classes.\n\nGoing deeper\n============\n\nThere are 4 documents you may also be interested in:\n\n* [xgboostPresentation.Rmd](https://github.com/dmlc/xgboost/blob/master/R-package/vignettes/xgboostPresentation.Rmd): general presentation\n* [discoverYourData.Rmd](https://github.com/dmlc/xgboost/blob/master/R-package/vignettes/discoverYourData.Rmd): explaining feature analysis\n* [Feature Importance Analysis with XGBoost in Tax audit](http://fr.slideshare.net/MichaelBENESTY/feature-importance-analysis-with-xgboost-in-tax-audit): use case\n* [The Elements of Statistical Learning](http://statweb.stanford.edu/~tibs/ElemStatLearn/): very good book to have a good understanding of the model\n" }, { "path": "demo/multiclass_classification/README.md", "content": "Demonstrating how to use XGBoost accomplish Multi-Class classification task on [UCI Dermatology dataset](https://archive.ics.uci.edu/ml/datasets/Dermatology)\n\nMake sure you make xgboost python module in ../../python\n\n1. Run runexp.sh\n```bash\n./runexp.sh\n```\n\n**R version** please see the `train.R`.\n" }, { "path": "demo/multiclass_classification/runexp.sh", "content": "#!/bin/bash\nif [ -f dermatology.data ]\nthen\n echo \"use existing data to run multi class classification\"\nelse\n echo \"getting data from uci, make sure you are connected to internet\"\n wget https://archive.ics.uci.edu/ml/machine-learning-databases/dermatology/dermatology.data\nfi\npython train.py\n" }, { "path": "demo/multiclass_classification/train.R", "content": "library(data.table)\nlibrary(xgboost)\n\nif (!file.exists(\"./dermatology.data\")) {\n download.file(\n \"https://archive.ics.uci.edu/ml/machine-learning-databases/dermatology/dermatology.data\",\n \"dermatology.data\",\n method = \"curl\"\n )\n}\n\ndf <- fread(\"dermatology.data\", sep = \",\", header = FALSE)\n\ndf[, `:=`(V34 = as.integer(ifelse(V34 == \"?\", 0L, V34)),\n V35 = V35 - 1L)]\n\nidx <- sample(nrow(df), size = round(0.7 * nrow(df)), replace = FALSE)\n\ntrain <- df[idx, ]\ntest <- df[-idx, ]\n\ntrain_x <- train[, 1:34]\ntrain_y <- train[, V35]\n\ntest_x <- test[, 1:34]\ntest_y <- test[, V35]\n\nxg_train <- xgb.DMatrix(data = as.matrix(train_x), label = train_y)\nxg_test <- xgb.DMatrix(as.matrix(test_x), label = test_y)\n\nparams <- list(\n objective = 'multi:softmax',\n num_class = 6,\n max_depth = 6,\n nthread = 4,\n eta = 0.1\n)\n\nwatchlist <- list(train = xg_train, test = xg_test)\n\nbst <- xgb.train(\n params = params,\n data = xg_train,\n watchlist = watchlist,\n nrounds = 5\n)\n\npred <- predict(bst, xg_test)\nerror_rate <- sum(pred != test_y) / length(test_y)\nprint(paste(\"Test error using softmax =\", error_rate))\n\n# do the same thing again, but output probabilities\nparams$objective <- 'multi:softprob'\nbst <- xgb.train(params, xg_train, nrounds = 5, watchlist)\n\npred_prob <- predict(bst, xg_test)\n\npred_mat <- matrix(pred_prob, ncol = 6, byrow = TRUE)\n# validation\n# rowSums(pred_mat)\n\npred_label <- apply(pred_mat, 1, which.max) - 1L\nerror_rate <- sum(pred_label != test_y) / length(test_y)\nprint(paste(\"Test error using softprob =\", error_rate))\n" }, { "path": "demo/multiclass_classification/train.py", "content": "#!/usr/bin/python\n\nfrom __future__ import division\n\nimport numpy as np\n\nimport xgboost as xgb\n\n# label need to be 0 to num_class -1\ndata = np.loadtxt('./dermatology.data', delimiter=',',\n converters={33: lambda x:int(x == '?'), 34: lambda x:int(x) - 1})\nsz = data.shape\n\ntrain = data[:int(sz[0] * 0.7), :]\ntest = data[int(sz[0] * 0.7):, :]\n\ntrain_X = train[:, :33]\ntrain_Y = train[:, 34]\n\ntest_X = test[:, :33]\ntest_Y = test[:, 34]\n\nxg_train = xgb.DMatrix(train_X, label=train_Y)\nxg_test = xgb.DMatrix(test_X, label=test_Y)\n# setup parameters for xgboost\nparam = {}\n# use softmax multi-class classification\nparam['objective'] = 'multi:softmax'\n# scale weight of positive examples\nparam['eta'] = 0.1\nparam['max_depth'] = 6\nparam['nthread'] = 4\nparam['num_class'] = 6\n\nwatchlist = [(xg_train, 'train'), (xg_test, 'test')]\nnum_round = 5\nbst = xgb.train(param, xg_train, num_round, watchlist)\n# get prediction\npred = bst.predict(xg_test)\nerror_rate = np.sum(pred != test_Y) / test_Y.shape[0]\nprint('Test error using softmax = {}'.format(error_rate))\n\n# do the same thing again, but output probabilities\nparam['objective'] = 'multi:softprob'\nbst = xgb.train(param, xg_train, num_round, watchlist)\n# Note: this convention has been changed since xgboost-unity\n# get prediction, this is in 1D array, need reshape to (ndata, nclass)\npred_prob = bst.predict(xg_test).reshape(test_Y.shape[0], 6)\npred_label = np.argmax(pred_prob, axis=1)\nerror_rate = np.sum(pred_label != test_Y) / test_Y.shape[0]\nprint('Test error using softprob = {}'.format(error_rate))\n" }, { "path": "demo/nvflare/.gitignore", "content": "!config\n" }, { "path": "demo/nvflare/README.md", "content": "# Experimental Support of Federated XGBoost using NVFlare\n\nThis directory contains a demo of Federated Learning using\n[NVFlare](https://nvidia.github.io/NVFlare/).\n\n## Horizontal Federated XGBoost\n\nFor horizontal federated learning using XGBoost (data is split row-wise), check out the `horizontal` directory\n(see the [README](horizontal/README.md)).\n\n## Vertical Federated XGBoost\n\nFor vertical federated learning using XGBoost (data is split column-wise), check out the `vertical` directory\n(see the [README](vertical/README.md)).\n" }, { "path": "demo/nvflare/config/config_fed_client.json", "content": "{\n \"format_version\": 2,\n \"executors\": [\n {\n \"tasks\": [\n \"train\"\n ],\n \"executor\": {\n \"path\": \"trainer.XGBoostTrainer\",\n \"args\": {\n \"server_address\": \"localhost:9091\",\n \"world_size\": 2,\n \"server_cert_path\": \"server-cert.pem\",\n \"client_key_path\": \"client-key.pem\",\n \"client_cert_path\": \"client-cert.pem\",\n \"use_gpus\": false\n }\n }\n }\n ],\n \"task_result_filters\": [],\n \"task_data_filters\": []\n}\n" }, { "path": "demo/nvflare/config/config_fed_server.json", "content": "{\n \"format_version\": 2,\n \"server\": {\n \"heart_beat_timeout\": 600\n },\n \"task_data_filters\": [],\n \"task_result_filters\": [],\n \"workflows\": [\n {\n \"id\": \"server_workflow\",\n \"path\": \"controller.XGBoostController\",\n \"args\": {\n \"port\": 9091,\n \"world_size\": 2,\n \"server_key_path\": \"server-key.pem\",\n \"server_cert_path\": \"server-cert.pem\",\n \"client_cert_path\": \"client-cert.pem\"\n }\n }\n ],\n \"components\": []\n}\n" }, { "path": "demo/nvflare/horizontal/README.md", "content": "# Experimental Support of Horizontal Federated XGBoost using NVFlare\n\nThis directory contains a demo of Horizontal Federated Learning using\n[NVFlare](https://nvidia.github.io/NVFlare/).\n\n## Training with CPU only\n\nTo run the demo, first build XGBoost with the federated learning plugin enabled (see the\n[README](../../../plugin/federated/README.md)).\n\nInstall NVFlare:\n```shell\npip install nvflare\n```\n\nPrepare the data:\n```shell\n./prepare_data.sh\n```\n\nStart the NVFlare federated server:\n```shell\n/tmp/nvflare/poc/server/startup/start.sh\n```\n\nIn another terminal, start the first worker:\n```shell\n/tmp/nvflare/poc/site-1/startup/start.sh\n```\n\nAnd the second worker:\n```shell\n/tmp/nvflare/poc/site-2/startup/start.sh\n```\n\nThen start the admin CLI:\n```shell\n/tmp/nvflare/poc/admin/startup/fl_admin.sh\n```\n\nIn the admin CLI, run the following command:\n```shell\nsubmit_job horizontal-xgboost\n```\n\nMake a note of the job id:\n```console\nSubmitted job: 28309e77-a7c5-45e6-b2bc-c2e3655122d8\n```\n\nOn both workers, you should see train and eval losses printed:\n```console\n[10:45:41] [0]\teval-logloss:0.22646\ttrain-logloss:0.23316\n[10:45:41] [1]\teval-logloss:0.13776\ttrain-logloss:0.13654\n[10:45:41] [2]\teval-logloss:0.08036\ttrain-logloss:0.08243\n[10:45:41] [3]\teval-logloss:0.05830\ttrain-logloss:0.05645\n[10:45:41] [4]\teval-logloss:0.03825\ttrain-logloss:0.04148\n[10:45:41] [5]\teval-logloss:0.02660\ttrain-logloss:0.02958\n[10:45:41] [6]\teval-logloss:0.01386\ttrain-logloss:0.01918\n[10:45:41] [7]\teval-logloss:0.01018\ttrain-logloss:0.01331\n[10:45:41] [8]\teval-logloss:0.00847\ttrain-logloss:0.01112\n[10:45:41] [9]\teval-logloss:0.00691\ttrain-logloss:0.00662\n[10:45:41] [10]\teval-logloss:0.00543\ttrain-logloss:0.00503\n[10:45:41] [11]\teval-logloss:0.00445\ttrain-logloss:0.00420\n[10:45:41] [12]\teval-logloss:0.00336\ttrain-logloss:0.00355\n[10:45:41] [13]\teval-logloss:0.00277\ttrain-logloss:0.00280\n[10:45:41] [14]\teval-logloss:0.00252\ttrain-logloss:0.00244\n[10:45:41] [15]\teval-logloss:0.00177\ttrain-logloss:0.00193\n[10:45:41] [16]\teval-logloss:0.00156\ttrain-logloss:0.00161\n[10:45:41] [17]\teval-logloss:0.00135\ttrain-logloss:0.00142\n[10:45:41] [18]\teval-logloss:0.00123\ttrain-logloss:0.00125\n[10:45:41] [19]\teval-logloss:0.00106\ttrain-logloss:0.00107\n```\n\nOnce the training finishes, the model file should be written into\n`/tmp/nvlfare/poc/site-1/${job_id}/test.model.json` and `/tmp/nvflare/poc/site-2/${job_id}/test.model.json`\nrespectively, where `job_id` is the UUID printed out when we ran `submit_job`.\n\nFinally, shutdown everything from the admin CLI, using `admin` as password:\n```shell\nshutdown client\nshutdown server\n```\n\n## Training with GPUs\n\nTo demo with Federated Learning using GPUs, make sure your machine has at least 2 GPUs.\nBuild XGBoost with the federated learning plugin enabled along with CUDA\n(see the [README](../../plugin/federated/README.md)).\n\nModify `../config/config_fed_client.json` and set `use_gpus` to `true`, then repeat the steps\nabove.\n" }, { "path": "demo/nvflare/horizontal/custom/controller.py", "content": "\"\"\"\nExample of training controller with NVFlare\n===========================================\n\"\"\"\nimport multiprocessing\n\nfrom nvflare.apis.client import Client\nfrom nvflare.apis.fl_context import FLContext\nfrom nvflare.apis.impl.controller import Controller, Task\nfrom nvflare.apis.shareable import Shareable\nfrom nvflare.apis.signal import Signal\nfrom trainer import SupportedTasks\n\nimport xgboost.federated\n\n\nclass XGBoostController(Controller):\n def __init__(self, port: int, world_size: int, server_key_path: str,\n server_cert_path: str, client_cert_path: str):\n \"\"\"Controller for federated XGBoost.\n\n Args:\n port: the port for the gRPC server to listen on.\n world_size: the number of sites.\n server_key_path: the path to the server key file.\n server_cert_path: the path to the server certificate file.\n client_cert_path: the path to the client certificate file.\n \"\"\"\n super().__init__()\n self._port = port\n self._world_size = world_size\n self._server_key_path = server_key_path\n self._server_cert_path = server_cert_path\n self._client_cert_path = client_cert_path\n self._server = None\n\n def start_controller(self, fl_ctx: FLContext):\n self._server = multiprocessing.Process(\n target=xgboost.federated.run_federated_server,\n args=(self._port, self._world_size, self._server_key_path,\n self._server_cert_path, self._client_cert_path))\n self._server.start()\n\n def stop_controller(self, fl_ctx: FLContext):\n if self._server:\n self._server.terminate()\n\n def process_result_of_unknown_task(self, client: Client, task_name: str,\n client_task_id: str, result: Shareable,\n fl_ctx: FLContext):\n self.log_warning(fl_ctx, f\"Unknown task: {task_name} from client {client.name}.\")\n\n def control_flow(self, abort_signal: Signal, fl_ctx: FLContext):\n self.log_info(fl_ctx, \"XGBoost training control flow started.\")\n if abort_signal.triggered:\n return\n task = Task(name=SupportedTasks.TRAIN, data=Shareable())\n self.broadcast_and_wait(\n task=task,\n min_responses=self._world_size,\n fl_ctx=fl_ctx,\n wait_time_after_min_received=1,\n abort_signal=abort_signal,\n )\n if abort_signal.triggered:\n return\n\n self.log_info(fl_ctx, \"XGBoost training control flow finished.\")\n" }, { "path": "demo/nvflare/horizontal/custom/trainer.py", "content": "import os\n\nfrom nvflare.apis.executor import Executor\nfrom nvflare.apis.fl_constant import FLContextKey, ReturnCode\nfrom nvflare.apis.fl_context import FLContext\nfrom nvflare.apis.shareable import Shareable, make_reply\nfrom nvflare.apis.signal import Signal\n\nimport xgboost as xgb\nfrom xgboost import callback\n\n\nclass SupportedTasks(object):\n TRAIN = \"train\"\n\n\nclass XGBoostTrainer(Executor):\n def __init__(self, server_address: str, world_size: int, server_cert_path: str,\n client_key_path: str, client_cert_path: str, use_gpus: bool):\n \"\"\"Trainer for federated XGBoost.\n\n Args:\n server_address: address for the gRPC server to connect to.\n world_size: the number of sites.\n server_cert_path: the path to the server certificate file.\n client_key_path: the path to the client key file.\n client_cert_path: the path to the client certificate file.\n \"\"\"\n super().__init__()\n self._server_address = server_address\n self._world_size = world_size\n self._server_cert_path = server_cert_path\n self._client_key_path = client_key_path\n self._client_cert_path = client_cert_path\n self._use_gpus = use_gpus\n\n def execute(self, task_name: str, shareable: Shareable, fl_ctx: FLContext,\n abort_signal: Signal) -> Shareable:\n self.log_info(fl_ctx, f\"Executing {task_name}\")\n try:\n if task_name == SupportedTasks.TRAIN:\n self._do_training(fl_ctx)\n return make_reply(ReturnCode.OK)\n else:\n self.log_error(fl_ctx, f\"{task_name} is not a supported task.\")\n return make_reply(ReturnCode.TASK_UNKNOWN)\n except BaseException as e:\n self.log_exception(fl_ctx,\n f\"Task {task_name} failed. Exception: {e.__str__()}\")\n return make_reply(ReturnCode.EXECUTION_EXCEPTION)\n\n def _do_training(self, fl_ctx: FLContext):\n client_name = fl_ctx.get_prop(FLContextKey.CLIENT_NAME)\n rank = int(client_name.split('-')[1]) - 1\n communicator_env = {\n 'xgboost_communicator': 'federated',\n 'federated_server_address': self._server_address,\n 'federated_world_size': self._world_size,\n 'federated_rank': rank,\n 'federated_server_cert': self._server_cert_path,\n 'federated_client_key': self._client_key_path,\n 'federated_client_cert': self._client_cert_path\n }\n with xgb.collective.CommunicatorContext(**communicator_env):\n # Load file, file will not be sharded in federated mode.\n dtrain = xgb.DMatrix('agaricus.txt.train?format=libsvm')\n dtest = xgb.DMatrix('agaricus.txt.test?format=libsvm')\n\n # Specify parameters via map, definition are same as c++ version\n param = {'tree_method': 'hist', 'max_depth': 2, 'eta': 1, 'objective': 'binary:logistic'}\n if self._use_gpus:\n self.log_info(fl_ctx, f'Training with GPU {rank}')\n param['device'] = f\"cuda:{rank}\"\n\n # Specify validations set to watch performance\n watchlist = [(dtest, 'eval'), (dtrain, 'train')]\n num_round = 20\n\n # Run training, all the features in training API is available.\n bst = xgb.train(param, dtrain, num_round, evals=watchlist,\n early_stopping_rounds=2, verbose_eval=False,\n callbacks=[callback.EvaluationMonitor(rank=rank)])\n\n # Save the model.\n workspace = fl_ctx.get_prop(FLContextKey.WORKSPACE_OBJECT)\n run_number = fl_ctx.get_prop(FLContextKey.CURRENT_RUN)\n run_dir = workspace.get_run_dir(run_number)\n bst.save_model(os.path.join(run_dir, \"test.model.json\"))\n xgb.collective.communicator_print(\"Finished training\\n\")\n" }, { "path": "demo/nvflare/horizontal/prepare_data.sh", "content": "#!/bin/bash\n\nset -e\n\nrm -fr ./agaricus* ./*.pem /tmp/nvflare\n\nworld_size=2\n\n# Generate server and client certificates.\nopenssl req -x509 -newkey rsa:2048 -days 7 -nodes -keyout server-key.pem -out server-cert.pem -subj \"/C=US/CN=localhost\"\nopenssl req -x509 -newkey rsa:2048 -days 7 -nodes -keyout client-key.pem -out client-cert.pem -subj \"/C=US/CN=localhost\"\n\n# Split train and test files manually to simulate a federated environment.\nsplit -n l/${world_size} --numeric-suffixes=1 -a 1 ../../data/agaricus.txt.train agaricus.txt.train-site-\nsplit -n l/${world_size} --numeric-suffixes=1 -a 1 ../../data/agaricus.txt.test agaricus.txt.test-site-\n\nnvflare poc -n 2 --prepare\nmkdir -p /tmp/nvflare/poc/admin/transfer/horizontal-xgboost\ncp -fr ../config custom /tmp/nvflare/poc/admin/transfer/horizontal-xgboost\ncp server-*.pem client-cert.pem /tmp/nvflare/poc/server/\nfor (( site=1; site<=world_size; site++ )); do\n cp server-cert.pem client-*.pem /tmp/nvflare/poc/site-\"$site\"/\n cp agaricus.txt.train-site-\"$site\" /tmp/nvflare/poc/site-\"$site\"/agaricus.txt.train\n cp agaricus.txt.test-site-\"$site\" /tmp/nvflare/poc/site-\"$site\"/agaricus.txt.test\ndone\n" }, { "path": "demo/nvflare/vertical/README.md", "content": "# Experimental Support of Vertical Federated XGBoost using NVFlare\n\nThis directory contains a demo of Vertical Federated Learning using\n[NVFlare](https://nvidia.github.io/NVFlare/).\n\n## Training with CPU only\n\nTo run the demo, first build XGBoost with the federated learning plugin enabled (see the\n[README](../../../plugin/federated/README.md)).\n\nInstall NVFlare:\n```shell\npip install nvflare\n```\n\nPrepare the data (note that this step will download the HIGGS dataset, which is 2.6GB compressed, and 7.5GB\nuncompressed, so make sure you have enough disk space and are on a fast internet connection):\n```shell\n./prepare_data.sh\n```\n\nStart the NVFlare federated server:\n```shell\n/tmp/nvflare/poc/server/startup/start.sh\n```\n\nIn another terminal, start the first worker:\n```shell\n/tmp/nvflare/poc/site-1/startup/start.sh\n```\n\nAnd the second worker:\n```shell\n/tmp/nvflare/poc/site-2/startup/start.sh\n```\n\nThen start the admin CLI:\n```shell\n/tmp/nvflare/poc/admin/startup/fl_admin.sh\n```\n\nIn the admin CLI, run the following command:\n```shell\nsubmit_job vertical-xgboost\n```\n\nOnce the training finishes, the model file should be written into\n`/tmp/nvlfare/poc/site-1/run_1/test.model.json` and `/tmp/nvflare/poc/site-2/run_1/test.model.json`\nrespectively.\n\nFinally, shutdown everything from the admin CLI, using `admin` as password:\n```shell\nshutdown client\nshutdown server\n```\n\n## Training with GPUs\n\nTo demo with Vertical Federated Learning using GPUs, make sure your machine has at least 2 GPUs.\nBuild XGBoost with the federated learning plugin enabled along with CUDA\n(see the [README](../../plugin/federated/README.md)).\n\nModify `../config/config_fed_client.json` and set `use_gpus` to `true`, then repeat the steps\nabove.\n" }, { "path": "demo/nvflare/vertical/custom/controller.py", "content": "\"\"\"\nExample of training controller with NVFlare\n===========================================\n\"\"\"\nimport multiprocessing\n\nfrom nvflare.apis.client import Client\nfrom nvflare.apis.fl_context import FLContext\nfrom nvflare.apis.impl.controller import Controller, Task\nfrom nvflare.apis.shareable import Shareable\nfrom nvflare.apis.signal import Signal\nfrom trainer import SupportedTasks\n\nimport xgboost.federated\n\n\nclass XGBoostController(Controller):\n def __init__(self, port: int, world_size: int, server_key_path: str,\n server_cert_path: str, client_cert_path: str):\n \"\"\"Controller for federated XGBoost.\n\n Args:\n port: the port for the gRPC server to listen on.\n world_size: the number of sites.\n server_key_path: the path to the server key file.\n server_cert_path: the path to the server certificate file.\n client_cert_path: the path to the client certificate file.\n \"\"\"\n super().__init__()\n self._port = port\n self._world_size = world_size\n self._server_key_path = server_key_path\n self._server_cert_path = server_cert_path\n self._client_cert_path = client_cert_path\n self._server = None\n\n def start_controller(self, fl_ctx: FLContext):\n self._server = multiprocessing.Process(\n target=xgboost.federated.run_federated_server,\n args=(self._port, self._world_size, self._server_key_path,\n self._server_cert_path, self._client_cert_path))\n self._server.start()\n\n def stop_controller(self, fl_ctx: FLContext):\n if self._server:\n self._server.terminate()\n\n def process_result_of_unknown_task(self, client: Client, task_name: str,\n client_task_id: str, result: Shareable,\n fl_ctx: FLContext):\n self.log_warning(fl_ctx, f\"Unknown task: {task_name} from client {client.name}.\")\n\n def control_flow(self, abort_signal: Signal, fl_ctx: FLContext):\n self.log_info(fl_ctx, \"XGBoost training control flow started.\")\n if abort_signal.triggered:\n return\n task = Task(name=SupportedTasks.TRAIN, data=Shareable())\n self.broadcast_and_wait(\n task=task,\n min_responses=self._world_size,\n fl_ctx=fl_ctx,\n wait_time_after_min_received=1,\n abort_signal=abort_signal,\n )\n if abort_signal.triggered:\n return\n\n self.log_info(fl_ctx, \"XGBoost training control flow finished.\")\n" }, { "path": "demo/nvflare/vertical/custom/trainer.py", "content": "import os\n\nfrom nvflare.apis.executor import Executor\nfrom nvflare.apis.fl_constant import FLContextKey, ReturnCode\nfrom nvflare.apis.fl_context import FLContext\nfrom nvflare.apis.shareable import Shareable, make_reply\nfrom nvflare.apis.signal import Signal\n\nimport xgboost as xgb\nfrom xgboost import callback\n\n\nclass SupportedTasks(object):\n TRAIN = \"train\"\n\n\nclass XGBoostTrainer(Executor):\n def __init__(self, server_address: str, world_size: int, server_cert_path: str,\n client_key_path: str, client_cert_path: str, use_gpus: bool):\n \"\"\"Trainer for federated XGBoost.\n\n Args:\n server_address: address for the gRPC server to connect to.\n world_size: the number of sites.\n server_cert_path: the path to the server certificate file.\n client_key_path: the path to the client key file.\n client_cert_path: the path to the client certificate file.\n \"\"\"\n super().__init__()\n self._server_address = server_address\n self._world_size = world_size\n self._server_cert_path = server_cert_path\n self._client_key_path = client_key_path\n self._client_cert_path = client_cert_path\n self._use_gpus = use_gpus\n\n def execute(self, task_name: str, shareable: Shareable, fl_ctx: FLContext,\n abort_signal: Signal) -> Shareable:\n self.log_info(fl_ctx, f\"Executing {task_name}\")\n try:\n if task_name == SupportedTasks.TRAIN:\n self._do_training(fl_ctx)\n return make_reply(ReturnCode.OK)\n else:\n self.log_error(fl_ctx, f\"{task_name} is not a supported task.\")\n return make_reply(ReturnCode.TASK_UNKNOWN)\n except BaseException as e:\n self.log_exception(fl_ctx,\n f\"Task {task_name} failed. Exception: {e.__str__()}\")\n return make_reply(ReturnCode.EXECUTION_EXCEPTION)\n\n def _do_training(self, fl_ctx: FLContext):\n client_name = fl_ctx.get_prop(FLContextKey.CLIENT_NAME)\n rank = int(client_name.split('-')[1]) - 1\n communicator_env = {\n 'xgboost_communicator': 'federated',\n 'federated_server_address': self._server_address,\n 'federated_world_size': self._world_size,\n 'federated_rank': rank,\n 'federated_server_cert': self._server_cert_path,\n 'federated_client_key': self._client_key_path,\n 'federated_client_cert': self._client_cert_path\n }\n with xgb.collective.CommunicatorContext(**communicator_env):\n # Load file, file will not be sharded in federated mode.\n if rank == 0:\n label = '&label_column=0'\n else:\n label = ''\n dtrain = xgb.DMatrix(f'higgs.train.csv?format=csv{label}', data_split_mode=1)\n dtest = xgb.DMatrix(f'higgs.test.csv?format=csv{label}', data_split_mode=1)\n\n # specify parameters via map\n param = {\n 'validate_parameters': True,\n 'eta': 0.1,\n 'gamma': 1.0,\n 'max_depth': 8,\n 'min_child_weight': 100,\n 'tree_method': 'hist',\n 'grow_policy': 'depthwise',\n 'objective': 'binary:logistic',\n 'eval_metric': 'auc',\n }\n if self._use_gpus:\n self.log_info(fl_ctx, f'Training with GPU {rank}')\n param['device'] = f\"cuda:{rank}\"\n\n # specify validations set to watch performance\n watchlist = [(dtest, \"eval\"), (dtrain, \"train\")]\n # number of boosting rounds\n num_round = 10\n\n bst = xgb.train(param, dtrain, num_round, evals=watchlist, early_stopping_rounds=2)\n\n # Save the model.\n workspace = fl_ctx.get_prop(FLContextKey.WORKSPACE_OBJECT)\n run_number = fl_ctx.get_prop(FLContextKey.CURRENT_RUN)\n run_dir = workspace.get_run_dir(run_number)\n bst.save_model(os.path.join(run_dir, \"higgs.model.federated.vertical.json\"))\n xgb.collective.communicator_print(\"Finished training\\n\")\n" }, { "path": "demo/nvflare/vertical/prepare_data.sh", "content": "#!/bin/bash\n\nset -e\n\nrm -fr ./*.pem /tmp/nvflare/poc\n\nworld_size=2\n\n# Generate server and client certificates.\nopenssl req -x509 -newkey rsa:2048 -days 7 -nodes -keyout server-key.pem -out server-cert.pem -subj \"/C=US/CN=localhost\"\nopenssl req -x509 -newkey rsa:2048 -days 7 -nodes -keyout client-key.pem -out client-cert.pem -subj \"/C=US/CN=localhost\"\n\n# Download HIGGS dataset.\nif [ -f \"HIGGS.csv\" ]; then\n echo \"HIGGS.csv exists, skipping download.\"\nelse\n echo \"Downloading HIGGS dataset.\"\n wget https://archive.ics.uci.edu/ml/machine-learning-databases/00280/HIGGS.csv.gz\n gunzip HIGGS.csv.gz\nfi\n\n# Split into train/test.\nif [[ -f higgs.train.csv && -f higgs.test.csv ]]; then\n echo \"higgs.train.csv and higgs.test.csv exist, skipping split.\"\nelse\n echo \"Splitting HIGGS dataset into train/test.\"\n head -n 10450000 HIGGS.csv > higgs.train.csv\n tail -n 550000 HIGGS.csv > higgs.test.csv\nfi\n\n# Split train and test files by column to simulate a federated environment.\nsite_files=(higgs.{train,test}.csv-site-*)\nif [ ${#site_files[@]} -eq $((world_size*2)) ]; then\n echo \"Site files exist, skipping split.\"\nelse\n echo \"Splitting train/test into site files.\"\n total_cols=28 # plus label\n cols=$((total_cols/world_size))\n echo \"Columns per site: $cols\"\n for (( site=1; site<=world_size; site++ )); do\n if (( site == 1 )); then\n start=$((cols*(site-1)+1))\n else\n start=$((cols*(site-1)+2))\n fi\n if (( site == world_size )); then\n end=$((total_cols+1))\n else\n end=$((cols*site+1))\n fi\n echo \"Site $site, columns $start-$end\"\n cut -d, -f${start}-${end} higgs.train.csv > higgs.train.csv-site-\"${site}\"\n cut -d, -f${start}-${end} higgs.test.csv > higgs.test.csv-site-\"${site}\"\n done\nfi\n\nnvflare poc -n 2 --prepare\nmkdir -p /tmp/nvflare/poc/admin/transfer/vertical-xgboost\ncp -fr ../config custom /tmp/nvflare/poc/admin/transfer/vertical-xgboost\ncp server-*.pem client-cert.pem /tmp/nvflare/poc/server/\nfor (( site=1; site<=world_size; site++ )); do\n cp server-cert.pem client-*.pem /tmp/nvflare/poc/site-\"${site}\"/\n ln -s \"${PWD}\"/higgs.train.csv-site-\"${site}\" /tmp/nvflare/poc/site-\"${site}\"/higgs.train.csv\n ln -s \"${PWD}\"/higgs.test.csv-site-\"${site}\" /tmp/nvflare/poc/site-\"${site}\"/higgs.test.csv\ndone\n" }, { "path": "demo/rmm_plugin/README.rst", "content": "Using XGBoost with RAPIDS Memory Manager (RMM) plugin\n=====================================================\n\n`RAPIDS Memory Manager (RMM) `__ library provides a\ncollection of efficient memory allocators for NVIDIA GPUs. It is now possible to use\nXGBoost with memory allocators provided by RMM, by enabling the RMM integration plugin.\n\nThe demos in this directory highlights one RMM allocator in particular: **the pool\nsub-allocator**. This allocator addresses the slow speed of ``cudaMalloc()`` by\nallocating a large chunk of memory upfront. Subsequent allocations will draw from the pool\nof already allocated memory and thus avoid the overhead of calling ``cudaMalloc()``\ndirectly. See `this GTC talk slides\n`_ for\nmore details.\n\nBefore running the demos, ensure that XGBoost is compiled with the RMM plugin enabled. To do this,\nrun CMake with option ``-DPLUGIN_RMM=ON`` (``-DUSE_CUDA=ON`` also required):\n\n.. code-block:: sh\n\n cmake -B build -S . -DUSE_CUDA=ON -DUSE_NCCL=ON -DPLUGIN_RMM=ON\n cmake --build build -j$(nproc)\n\nCMake will attempt to locate the RMM library in your build environment. You may choose to build\nRMM from the source, or install it using the Conda package manager. If CMake cannot find RMM, you\nshould specify the location of RMM with the CMake prefix:\n\n.. code-block:: sh\n\n # If using Conda:\n cmake -B build -S . -DUSE_CUDA=ON -DUSE_NCCL=ON -DPLUGIN_RMM=ON -DCMAKE_PREFIX_PATH=$CONDA_PREFIX\n # If using RMM installed with a custom location\n cmake -B build -S . -DUSE_CUDA=ON -DUSE_NCCL=ON -DPLUGIN_RMM=ON -DCMAKE_PREFIX_PATH=/path/to/rmm\n\n********************************\nInforming XGBoost about RMM pool\n********************************\n\nWhen XGBoost is compiled with RMM, most of the large size allocation will go through RMM\nallocators, but some small allocations in performance critical areas are using a different\ncaching allocator so that we can have better control over memory allocation behavior.\nUsers can override this behavior and force the use of rmm for all allocations by setting\nthe global configuration ``use_rmm``:\n\n.. code-block:: python\n\n with xgb.config_context(use_rmm=True):\n clf = xgb.XGBClassifier(tree_method=\"hist\", device=\"cuda\")\n\nDepending on the choice of memory pool size and the type of the allocator, this can add\nmore consistency to memory usage but with slightly degraded performance impact.\n\n*******************************\nNo Device Ordinal for Multi-GPU\n*******************************\n\nSince with RMM the memory pool is pre-allocated on a specific device, changing the CUDA\ndevice ordinal in XGBoost can result in memory error ``cudaErrorIllegalAddress``. Use the\n``CUDA_VISIBLE_DEVICES`` environment variable instead of the ``device=\"cuda:1\"`` parameter\nfor selecting device. For distributed training, the distributed computing frameworks like\n``dask-cuda`` are responsible for device management. For Scala-Spark, see\n:doc:`/jvm/xgboost4j_spark_gpu_tutorial` for more info.\n\n************************\nMemory Over-Subscription\n************************\n\n.. warning::\n\n This feature is still experimental and is under active development.\n\nThe newer NVIDIA platforms like `Grace-Hopper\n`__ use `NVLink-C2C\n`__, which allows the CPU and GPU to\nhave a coherent memory model. Users can use the `SamHeadroomMemoryResource` in the latest\nRMM to utilize system memory for storing data. This can help XGBoost utilize memory from\nthe host for GPU computation, but it may reduce performance due to slower CPU memory speed\nand page migration overhead." }, { "path": "demo/rmm_plugin/rmm_mgpu_with_dask.py", "content": "\"\"\"\nUsing rmm with Dask\n===================\n\"\"\"\n\nimport dask\nfrom dask.distributed import Client\nfrom dask_cuda import LocalCUDACluster\nfrom sklearn.datasets import make_classification\n\nimport xgboost as xgb\n\n\ndef main(client):\n # Optionally force XGBoost to use RMM for all GPU memory allocation, see ./README.md\n # xgb.set_config(use_rmm=True)\n\n X, y = make_classification(n_samples=10000, n_informative=5, n_classes=3)\n # In pratice one should prefer loading the data with dask collections instead of\n # using `from_array`.\n X = dask.array.from_array(X)\n y = dask.array.from_array(y)\n dtrain = xgb.dask.DaskDMatrix(client, X, label=y)\n\n params = {\n \"max_depth\": 8,\n \"eta\": 0.01,\n \"objective\": \"multi:softprob\",\n \"num_class\": 3,\n \"tree_method\": \"hist\",\n \"eval_metric\": \"merror\",\n \"device\": \"cuda\",\n }\n output = xgb.dask.train(\n client, params, dtrain, num_boost_round=100, evals=[(dtrain, \"train\")]\n )\n bst = output[\"booster\"]\n history = output[\"history\"]\n for i, e in enumerate(history[\"train\"][\"merror\"]):\n print(f\"[{i}] train-merror: {e}\")\n\n\nif __name__ == \"__main__\":\n # To use RMM pool allocator with a GPU Dask cluster, just add rmm_pool_size option\n # to LocalCUDACluster constructor.\n with LocalCUDACluster(rmm_pool_size=\"2GB\") as cluster:\n with Client(cluster) as client:\n main(client)\n" }, { "path": "demo/rmm_plugin/rmm_singlegpu.py", "content": "\"\"\"\nUsing rmm on a single node device\n=================================\n\"\"\"\n\nimport rmm\nfrom sklearn.datasets import make_classification\n\nimport xgboost as xgb\n\n# Initialize RMM pool allocator\nrmm.reinitialize(pool_allocator=True)\n# Optionally force XGBoost to use RMM for all GPU memory allocation, see ./README.md\n# xgb.set_config(use_rmm=True)\n\nX, y = make_classification(n_samples=10000, n_informative=5, n_classes=3)\ndtrain = xgb.DMatrix(X, label=y)\n\nparams = {\n \"max_depth\": 8,\n \"eta\": 0.01,\n \"objective\": \"multi:softprob\",\n \"num_class\": 3,\n \"tree_method\": \"hist\",\n \"device\": \"cuda\",\n}\n# XGBoost will automatically use the RMM pool allocator\nbst = xgb.train(params, dtrain, num_boost_round=100, evals=[(dtrain, \"train\")])\n" }, { "path": "dev/prepare_jvm_release.py", "content": "\"\"\"\nHelper script to prepare for releasing XGBoost JVM packages to\nMaven Central.\n\n## Prerequisite\n\n1. You must have the right to upload artifacts to the Maven Central repo.\n If you do not, contact Hyunsu Cho (chohyu01@cs.washington.edu) so that\n he can contact Sonatype on your behalf in order to add you as a\n \"producer\" user for the ml.dmlc namespace. See\n https://central.sonatype.org/pages/support/#status to learn about\n the process for adding or removing users who can publish to the project.\n\n2. Follow instructions in\n https://central.sonatype.org/publish/publish-portal-maven/#credentials\n to set up the authentication token in your machine.\n\n3. Set up GPG for signing artifacts:\n https://central.sonatype.org/publish/requirements/gpg/\n\n## Making the release\nRun this script 4 times:\n\npython3 dev/prepare_jvm_release.py --scala-version 2.12 --variant cpu\npython3 dev/prepare_jvm_release.py --scala-version 2.12 --variant gpu\npython3 dev/prepare_jvm_release.py --scala-version 2.13 --variant cpu\npython3 dev/prepare_jvm_release.py --scala-version 2.13 --variant gpu\n\n\"\"\"\n\nimport argparse\nimport errno\nimport glob\nimport os\nimport re\nimport shutil\nimport subprocess\nimport sys\nimport tempfile\nimport zipfile\nfrom contextlib import contextmanager\nfrom urllib.request import urlretrieve\n\n\ndef normpath(path):\n \"\"\"Normalize UNIX path to a native path.\"\"\"\n normalized = os.path.join(*path.split(\"/\"))\n if os.path.isabs(path):\n return os.path.abspath(\"/\") + normalized\n else:\n return normalized\n\n\ndef cp(source, target):\n source = normpath(source)\n target = normpath(target)\n print(\"cp {0} {1}\".format(source, target))\n shutil.copy(source, target)\n\n\ndef maybe_makedirs(path):\n path = normpath(path)\n print(\"mkdir -p \" + path)\n try:\n os.makedirs(path)\n except OSError as e:\n if e.errno != errno.EEXIST:\n raise\n\n\n@contextmanager\ndef cd(path):\n path = normpath(path)\n cwd = os.getcwd()\n os.chdir(path)\n print(\"cd \" + path)\n try:\n yield path\n finally:\n os.chdir(cwd)\n\n\ndef run(command, **kwargs):\n print(command)\n subprocess.run(command, shell=True, check=True, **kwargs)\n\n\ndef get_current_commit_hash():\n out = subprocess.check_output([\"git\", \"rev-parse\", \"HEAD\"])\n return out.decode().split(\"\\n\")[0]\n\n\ndef get_current_git_branch():\n out = subprocess.check_output([\"git\", \"log\", \"-n\", \"1\", \"--pretty=%d\", \"HEAD\"])\n m = re.search(r\"release_[0-9\\.]+\", out.decode())\n if not m:\n raise ValueError(\"Expected branch name of form release_xxx\")\n return m.group(0)\n\n\ndef retrieve(url, filename=None):\n print(f\"{url} -> {filename}\")\n return urlretrieve(url, filename)\n\n\ndef main():\n parser = argparse.ArgumentParser(\n description=__doc__, formatter_class=argparse.RawTextHelpFormatter\n )\n parser.add_argument(\n \"--release-version\",\n type=str,\n required=True,\n help=\"Version of the release being prepared\",\n )\n parser.add_argument(\n \"--scala-version\",\n type=str,\n required=True,\n help=\"Version of Scala to use in the JVM packages\",\n choices=[\"2.12\", \"2.13\"],\n )\n parser.add_argument(\n \"--variant\",\n type=str,\n required=True,\n choices=[\"cpu\", \"gpu\"],\n help=\"JVM package variant to package and publish\",\n )\n\n args = parser.parse_args()\n version = args.release_version\n scala_version = args.scala_version\n use_cuda = args.variant == \"gpu\"\n\n commit_hash = get_current_commit_hash()\n git_branch = get_current_git_branch()\n print(f\"Using commit {commit_hash} of branch {git_branch}\")\n print(f\"====Update pom.xml to use Scala {scala_version}====\")\n run(\n f\"{sys.executable} ops/script/change_scala_version.py \"\n f\"--scala-version {scala_version} --purge-artifacts\"\n )\n\n with cd(\"jvm-packages/\"):\n print(\"====Copying resources for testing====\")\n with cd(\"../demo/data/regression\"):\n run(f\"{sys.executable} mapfeat.py\")\n run(f\"{sys.executable} mknfold.py machine.txt 1\")\n xgboost4j_spark = \"xgboost4j-spark-gpu\" if use_cuda else \"xgboost4j-spark\"\n maybe_makedirs(\"xgboost4j/src/test/resources\")\n maybe_makedirs(f\"{xgboost4j_spark}/src/test/resources\")\n for file in glob.glob(\"../demo/data/agaricus.*\"):\n cp(file, \"xgboost4j/src/test/resources\")\n cp(file, f\"{xgboost4j_spark}/src/test/resources\")\n for file in glob.glob(\"../demo/data/regression/machine.txt.t*\"):\n cp(file, f\"{xgboost4j_spark}/src/test/resources\")\n\n print(\"====Creating directories to hold native binaries====\")\n if use_cuda:\n # TODO(hcho3): Add GPU build for linux aarch64\n matrix = [(\"linux\", \"x86_64\")]\n else:\n matrix = [\n (\"linux\", \"x86_64\"),\n (\"linux\", \"aarch64\"),\n (\"windows\", \"x86_64\"),\n (\"macos\", \"x86_64\"),\n (\"macos\", \"aarch64\"),\n ]\n for os_ident, arch in matrix:\n output_dir = f\"xgboost4j/src/main/resources/lib/{os_ident}/{arch}\"\n maybe_makedirs(output_dir)\n\n print(\"====Downloading native binaries from CI====\")\n if use_cuda:\n url_prefix = (\n \"https://s3-us-west-2.amazonaws.com/xgboost-maven-repo/release/ml/dmlc\"\n )\n with tempfile.TemporaryDirectory() as tempdir:\n # libxgboost4j.so for Linux x86_64, GPU support\n zip_path = os.path.join(tempdir, \"xgboost4j-spark-gpu_2.12.jar\")\n extract_dir = os.path.join(tempdir, \"xgboost4j-spark-gpu\")\n retrieve(\n url=f\"{url_prefix}/xgboost4j-spark-gpu_2.12/{version}/\"\n f\"xgboost4j-spark-gpu_2.12-{version}.jar\",\n filename=zip_path,\n )\n os.mkdir(extract_dir)\n with zipfile.ZipFile(zip_path, \"r\") as t:\n t.extractall(extract_dir)\n cp(\n os.path.join(\n extract_dir, \"lib\", \"linux\", \"x86_64\", \"libxgboost4j.so\"\n ),\n \"xgboost4j/src/main/resources/lib/linux/x86_64/libxgboost4j.so\",\n )\n run(\n \"mvn --no-transfer-progress install -Pgpu \"\n \"-DskipTests -Dmaven.test.skip=true -Dskip.native.build=true\"\n )\n run(\n \"mvn deploy -Pgpu,release -pl xgboost4j-spark-gpu \"\n \"-DskipTests -Dmaven.test.skip=true -Dskip.native.build=true\"\n )\n else:\n url_prefix = \"https://s3-us-west-2.amazonaws.com/xgboost-nightly-builds\"\n for os_ident, arch, src_libname, dest_libname in [\n (\"linux\", \"x86_64\", \"libxgboost4j_linux_x86_64.so\", \"libxgboost4j.so\"),\n (\n \"linux\",\n \"aarch64\",\n \"libxgboost4j_linux_aarch64.so\",\n \"libxgboost4j.so\",\n ),\n (\"windows\", \"x86_64\", \"xgboost4j.dll\", \"xgboost4j.dll\"),\n (\"macos\", \"x86_64\", \"libxgboost4j_intel.dylib\", \"libxgboost4j.dylib\"),\n (\"macos\", \"aarch64\", \"libxgboost4j_m1.dylib\", \"libxgboost4j.dylib\"),\n ]:\n retrieve(\n url=f\"{url_prefix}/{git_branch}/{commit_hash}/{src_libname}\",\n filename=(\n \"xgboost4j/src/main/resources/lib/\"\n f\"{os_ident}/{arch}/{dest_libname}\"\n ),\n )\n run(\n \"mvn --no-transfer-progress deploy -Pdefault,release \"\n \"-DskipTests -Dmaven.test.skip=true -Dskip.native.build=true\"\n )\n\n print(\"====Next Steps====\")\n print(\n \"Visit https://central.sonatype.com/publishing/deployments to verify the deployment. \"\n \"You can either drop the deployment or publish it. Note: publishing is final.\"\n )\n\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "dev/query_contributors.py", "content": "\"\"\"Query list of all contributors and reviewers in a release\"\"\"\n\nimport json\nimport re\nimport sys\n\nimport requests\nfrom sh.contrib import git\n\nif len(sys.argv) != 5:\n print(f'Usage: {sys.argv[0]} [starting commit/tag] [ending commit/tag] [GitHub username] ' +\n '[GitHub password]')\n sys.exit(1)\n\nfrom_commit = sys.argv[1]\nto_commit = sys.argv[2]\nusername = sys.argv[3]\npassword = sys.argv[4]\n\ncontributors = set()\nreviewers = set()\n\ndef paginate_request(url, callback):\n r = requests.get(url, auth=(username, password))\n assert r.status_code == requests.codes.ok, f'Code: {r.status_code}, Text: {r.text}'\n callback(json.loads(r.text))\n while 'next' in r.links:\n r = requests.get(r.links['next']['url'], auth=(username, password))\n callback(json.loads(r.text))\n\nfor line in git.log(f'{from_commit}..{to_commit}', '--pretty=format:%s', '--reverse', '--first-parent'):\n m = re.search('\\(#([0-9]+)\\)$', line.rstrip())\n if m:\n pr_id = m.group(1)\n print(f'PR #{pr_id}')\n\n def process_commit_list(commit_list):\n try:\n contributors.update([commit['author']['login'] for commit in commit_list])\n except TypeError:\n prompt = (f'Error fetching contributors for PR #{pr_id}. Enter it manually, ' +\n 'as a space-separated list: ')\n contributors.update(str(input(prompt)).split(' '))\n def process_review_list(review_list):\n reviewers.update([x['user']['login'] for x in review_list])\n def process_comment_list(comment_list):\n reviewers.update([x['user']['login'] for x in comment_list])\n\n paginate_request(f'https://api.github.com/repos/dmlc/xgboost/pulls/{pr_id}/commits',\n process_commit_list)\n paginate_request(f'https://api.github.com/repos/dmlc/xgboost/pulls/{pr_id}/reviews',\n process_review_list)\n paginate_request(f'https://api.github.com/repos/dmlc/xgboost/issues/{pr_id}/comments',\n process_comment_list)\n\nprint('Contributors: ', end='')\nfor x in sorted(contributors):\n r = requests.get(f'https://api.github.com/users/{x}', auth=(username, password))\n assert r.status_code == requests.codes.ok, f'Code: {r.status_code}, Text: {r.text}'\n user_info = json.loads(r.text)\n if user_info['name'] is None:\n print(f\"@{x}, \", end='')\n else:\n print(f\"{user_info['name']} (@{x}), \", end='')\n\nprint('\\nReviewers: ', end='')\nfor x in sorted(reviewers):\n r = requests.get(f'https://api.github.com/users/{x}', auth=(username, password))\n assert r.status_code == requests.codes.ok, f'Code: {r.status_code}, Text: {r.text}'\n user_info = json.loads(r.text)\n if user_info['name'] is None:\n print(f\"@{x}, \", end='')\n else:\n print(f\"{user_info['name']} (@{x}), \", end='')\nprint('')\n" }, { "path": "doc/.gitignore", "content": "html\nlatex\n*.sh\n_*\nsg_execution_times.rst\ndoxygen\nparser.py\n*.pyc\nweb-data\n# generated by doxygen\ntmp" }, { "path": "doc/Doxyfile.in", "content": "# Doxyfile 1.9.1\n\n# This file describes the settings to be used by the documentation system\n# doxygen (www.doxygen.org) for a project.\n#\n# All text after a double hash (##) is considered a comment and is placed in\n# front of the TAG it is preceding.\n#\n# All text after a single hash (#) is considered a comment and will be ignored.\n# The format is:\n# TAG = value [value, ...]\n# For lists, items can also be appended using:\n# TAG += value [value, ...]\n# Values that contain spaces should be placed between quotes (\\\" \\\").\n\n#---------------------------------------------------------------------------\n# Project related configuration options\n#---------------------------------------------------------------------------\n\n# This tag specifies the encoding used for all characters in the configuration\n# file that follow. The default is UTF-8 which is also the encoding used for all\n# text before the first occurrence of this tag. Doxygen uses libiconv (or the\n# iconv built into libc) for the transcoding. See\n# https://www.gnu.org/software/libiconv/ for the list of possible encodings.\n# The default value is: UTF-8.\n\nDOXYFILE_ENCODING = UTF-8\n\n# The PROJECT_NAME tag is a single word (or a sequence of words surrounded by\n# double-quotes, unless you are using Doxywizard) that should identify the\n# project for which the documentation is generated. This name is used in the\n# title of most generated pages and in a few other places.\n# The default value is: My Project.\n\nPROJECT_NAME = xgboost\n\n# The PROJECT_NUMBER tag can be used to enter a project or revision number. This\n# could be handy for archiving the generated documentation or if some version\n# control system is used.\n\nPROJECT_NUMBER = @XGBOOST_VERSION@\n\n# Using the PROJECT_BRIEF tag one can provide an optional one line description\n# for a project that appears at the top of each page and should give viewer a\n# quick idea about the purpose of the project. Keep the description short.\n\nPROJECT_BRIEF =\n\n# With the PROJECT_LOGO tag one can specify a logo or an icon that is included\n# in the documentation. The maximum height of the logo should not exceed 55\n# pixels and the maximum width should not exceed 200 pixels. Doxygen will copy\n# the logo to the output directory.\n\nPROJECT_LOGO =\n\n# The OUTPUT_DIRECTORY tag is used to specify the (relative or absolute) path\n# into which the generated documentation will be written. If a relative path is\n# entered, it will be relative to the location where doxygen was started. If\n# left blank the current directory will be used.\n\nOUTPUT_DIRECTORY = @PROJECT_BINARY_DIR@/doc_doxygen\n\n# If the CREATE_SUBDIRS tag is set to YES then doxygen will create 4096 sub-\n# directories (in 2 levels) under the output directory of each output format and\n# will distribute the generated files over these directories. Enabling this\n# option can be useful when feeding doxygen a huge amount of source files, where\n# putting all generated files in the same directory would otherwise causes\n# performance problems for the file system.\n# The default value is: NO.\n\nCREATE_SUBDIRS = NO\n\n# If the ALLOW_UNICODE_NAMES tag is set to YES, doxygen will allow non-ASCII\n# characters to appear in the names of generated files. If set to NO, non-ASCII\n# characters will be escaped, for example _xE3_x81_x84 will be used for Unicode\n# U+3044.\n# The default value is: NO.\n\nALLOW_UNICODE_NAMES = NO\n\n# The OUTPUT_LANGUAGE tag is used to specify the language in which all\n# documentation generated by doxygen is written. Doxygen will use this\n# information to generate all constant output in the proper language.\n# Possible values are: Afrikaans, Arabic, Armenian, Brazilian, Catalan, Chinese,\n# Chinese-Traditional, Croatian, Czech, Danish, Dutch, English (United States),\n# Esperanto, Farsi (Persian), Finnish, French, German, Greek, Hungarian,\n# Indonesian, Italian, Japanese, Japanese-en (Japanese with English messages),\n# Korean, Korean-en (Korean with English messages), Latvian, Lithuanian,\n# Macedonian, Norwegian, Persian (Farsi), Polish, Portuguese, Romanian, Russian,\n# Serbian, Serbian-Cyrillic, Slovak, Slovene, Spanish, Swedish, Turkish,\n# Ukrainian and Vietnamese.\n# The default value is: English.\n\nOUTPUT_LANGUAGE = English\n\n# The OUTPUT_TEXT_DIRECTION tag is used to specify the direction in which all\n# documentation generated by doxygen is written. Doxygen will use this\n# information to generate all generated output in the proper direction.\n# Possible values are: None, LTR, RTL and Context.\n# The default value is: None.\n\nOUTPUT_TEXT_DIRECTION = None\n\n# If the BRIEF_MEMBER_DESC tag is set to YES, doxygen will include brief member\n# descriptions after the members that are listed in the file and class\n# documentation (similar to Javadoc). Set to NO to disable this.\n# The default value is: YES.\n\nBRIEF_MEMBER_DESC = YES\n\n# If the REPEAT_BRIEF tag is set to YES, doxygen will prepend the brief\n# description of a member or function before the detailed description\n#\n# Note: If both HIDE_UNDOC_MEMBERS and BRIEF_MEMBER_DESC are set to NO, the\n# brief descriptions will be completely suppressed.\n# The default value is: YES.\n\nREPEAT_BRIEF = YES\n\n# This tag implements a quasi-intelligent brief description abbreviator that is\n# used to form the text in various listings. Each string in this list, if found\n# as the leading text of the brief description, will be stripped from the text\n# and the result, after processing the whole list, is used as the annotated\n# text. Otherwise, the brief description is used as-is. If left blank, the\n# following values are used ($name is automatically replaced with the name of\n# the entity):The $name class, The $name widget, The $name file, is, provides,\n# specifies, contains, represents, a, an and the.\n\nABBREVIATE_BRIEF =\n\n# If the ALWAYS_DETAILED_SEC and REPEAT_BRIEF tags are both set to YES then\n# doxygen will generate a detailed section even if there is only a brief\n# description.\n# The default value is: NO.\n\nALWAYS_DETAILED_SEC = NO\n\n# If the INLINE_INHERITED_MEMB tag is set to YES, doxygen will show all\n# inherited members of a class in the documentation of that class as if those\n# members were ordinary class members. Constructors, destructors and assignment\n# operators of the base classes will not be shown.\n# The default value is: NO.\n\nINLINE_INHERITED_MEMB = NO\n\n# If the FULL_PATH_NAMES tag is set to YES, doxygen will prepend the full path\n# before files name in the file list and in the header files. If set to NO the\n# shortest path that makes the file name unique will be used\n# The default value is: YES.\n\nFULL_PATH_NAMES = YES\n\n# The STRIP_FROM_PATH tag can be used to strip a user-defined part of the path.\n# Stripping is only done if one of the specified strings matches the left-hand\n# part of the path. The tag can be used to show relative paths in the file list.\n# If left blank the directory from which doxygen is run is used as the path to\n# strip.\n#\n# Note that you can specify absolute paths here, but also relative paths, which\n# will be relative from the directory where doxygen is started.\n# This tag requires that the tag FULL_PATH_NAMES is set to YES.\n\nSTRIP_FROM_PATH =\n\n# The STRIP_FROM_INC_PATH tag can be used to strip a user-defined part of the\n# path mentioned in the documentation of a class, which tells the reader which\n# header file to include in order to use a class. If left blank only the name of\n# the header file containing the class definition is used. Otherwise one should\n# specify the list of include paths that are normally passed to the compiler\n# using the -I flag.\n\nSTRIP_FROM_INC_PATH =\n\n# If the SHORT_NAMES tag is set to YES, doxygen will generate much shorter (but\n# less readable) file names. This can be useful is your file systems doesn't\n# support long names like on DOS, Mac, or CD-ROM.\n# The default value is: NO.\n\nSHORT_NAMES = NO\n\n# If the JAVADOC_AUTOBRIEF tag is set to YES then doxygen will interpret the\n# first line (until the first dot) of a Javadoc-style comment as the brief\n# description. If set to NO, the Javadoc-style will behave just like regular Qt-\n# style comments (thus requiring an explicit @brief command for a brief\n# description.)\n# The default value is: NO.\n\nJAVADOC_AUTOBRIEF = NO\n\n# If the JAVADOC_BANNER tag is set to YES then doxygen will interpret a line\n# such as\n# /***************\n# as being the beginning of a Javadoc-style comment \"banner\". If set to NO, the\n# Javadoc-style will behave just like regular comments and it will not be\n# interpreted by doxygen.\n# The default value is: NO.\n\nJAVADOC_BANNER = NO\n\n# If the QT_AUTOBRIEF tag is set to YES then doxygen will interpret the first\n# line (until the first dot) of a Qt-style comment as the brief description. If\n# set to NO, the Qt-style will behave just like regular Qt-style comments (thus\n# requiring an explicit \\brief command for a brief description.)\n# The default value is: NO.\n\nQT_AUTOBRIEF = NO\n\n# The MULTILINE_CPP_IS_BRIEF tag can be set to YES to make doxygen treat a\n# multi-line C++ special comment block (i.e. a block of //! or /// comments) as\n# a brief description. This used to be the default behavior. The new default is\n# to treat a multi-line C++ comment block as a detailed description. Set this\n# tag to YES if you prefer the old behavior instead.\n#\n# Note that setting this tag to YES also means that rational rose comments are\n# not recognized any more.\n# The default value is: NO.\n\nMULTILINE_CPP_IS_BRIEF = NO\n\n# By default Python docstrings are displayed as preformatted text and doxygen's\n# special commands cannot be used. By setting PYTHON_DOCSTRING to NO the\n# doxygen's special commands can be used and the contents of the docstring\n# documentation blocks is shown as doxygen documentation.\n# The default value is: YES.\n\nPYTHON_DOCSTRING = YES\n\n# If the INHERIT_DOCS tag is set to YES then an undocumented member inherits the\n# documentation from any documented member that it re-implements.\n# The default value is: YES.\n\nINHERIT_DOCS = YES\n\n# If the SEPARATE_MEMBER_PAGES tag is set to YES then doxygen will produce a new\n# page for each member. If set to NO, the documentation of a member will be part\n# of the file/class/namespace that contains it.\n# The default value is: NO.\n\nSEPARATE_MEMBER_PAGES = NO\n\n# The TAB_SIZE tag can be used to set the number of spaces in a tab. Doxygen\n# uses this value to replace tabs by spaces in code fragments.\n# Minimum value: 1, maximum value: 16, default value: 4.\n\nTAB_SIZE = 8\n\n# This tag can be used to specify a number of aliases that act as commands in\n# the documentation. An alias has the form:\n# name=value\n# For example adding\n# \"sideeffect=@par Side Effects:\\n\"\n# will allow you to put the command \\sideeffect (or @sideeffect) in the\n# documentation, which will result in a user-defined paragraph with heading\n# \"Side Effects:\". You can put \\n's in the value part of an alias to insert\n# newlines (in the resulting output). You can put ^^ in the value part of an\n# alias to insert a newline as if a physical newline was in the original file.\n# When you need a literal { or } or , in the value part of an alias you have to\n# escape them by means of a backslash (\\), this can lead to conflicts with the\n# commands \\{ and \\} for these it is advised to use the version @{ and @} or use\n# a double escape (\\\\{ and \\\\})\n\nALIASES =\n\n# Set the OPTIMIZE_OUTPUT_FOR_C tag to YES if your project consists of C sources\n# only. Doxygen will then generate output that is more tailored for C. For\n# instance, some of the names that are used will be different. The list of all\n# members will be omitted, etc.\n# The default value is: NO.\n\nOPTIMIZE_OUTPUT_FOR_C = NO\n\n# Set the OPTIMIZE_OUTPUT_JAVA tag to YES if your project consists of Java or\n# Python sources only. Doxygen will then generate output that is more tailored\n# for that language. For instance, namespaces will be presented as packages,\n# qualified scopes will look different, etc.\n# The default value is: NO.\n\nOPTIMIZE_OUTPUT_JAVA = NO\n\n# Set the OPTIMIZE_FOR_FORTRAN tag to YES if your project consists of Fortran\n# sources. Doxygen will then generate output that is tailored for Fortran.\n# The default value is: NO.\n\nOPTIMIZE_FOR_FORTRAN = NO\n\n# Set the OPTIMIZE_OUTPUT_VHDL tag to YES if your project consists of VHDL\n# sources. Doxygen will then generate output that is tailored for VHDL.\n# The default value is: NO.\n\nOPTIMIZE_OUTPUT_VHDL = NO\n\n# Set the OPTIMIZE_OUTPUT_SLICE tag to YES if your project consists of Slice\n# sources only. Doxygen will then generate output that is more tailored for that\n# language. For instance, namespaces will be presented as modules, types will be\n# separated into more groups, etc.\n# The default value is: NO.\n\nOPTIMIZE_OUTPUT_SLICE = NO\n\n# Doxygen selects the parser to use depending on the extension of the files it\n# parses. With this tag you can assign which parser to use for a given\n# extension. Doxygen has a built-in mapping, but you can override or extend it\n# using this tag. The format is ext=language, where ext is a file extension, and\n# language is one of the parsers supported by doxygen: IDL, Java, JavaScript,\n# Csharp (C#), C, C++, D, PHP, md (Markdown), Objective-C, Python, Slice, VHDL,\n# Fortran (fixed format Fortran: FortranFixed, free formatted Fortran:\n# FortranFree, unknown formatted Fortran: Fortran. In the later case the parser\n# tries to guess whether the code is fixed or free formatted code, this is the\n# default for Fortran type files). For instance to make doxygen treat .inc files\n# as Fortran files (default is PHP), and .f files as C (default is Fortran),\n# use: inc=Fortran f=C.\n#\n# Note: For files without extension you can use no_extension as a placeholder.\n#\n# Note that for custom extensions you also need to set FILE_PATTERNS otherwise\n# the files are not read by doxygen. When specifying no_extension you should add\n# * to the FILE_PATTERNS.\n#\n# Note see also the list of default file extension mappings.\n\nEXTENSION_MAPPING =\n\n# If the MARKDOWN_SUPPORT tag is enabled then doxygen pre-processes all comments\n# according to the Markdown format, which allows for more readable\n# documentation. See https://daringfireball.net/projects/markdown/ for details.\n# The output of markdown processing is further processed by doxygen, so you can\n# mix doxygen, HTML, and XML commands with Markdown formatting. Disable only in\n# case of backward compatibilities issues.\n# The default value is: YES.\n\nMARKDOWN_SUPPORT = YES\n\n# When the TOC_INCLUDE_HEADINGS tag is set to a non-zero value, all headings up\n# to that level are automatically included in the table of contents, even if\n# they do not have an id attribute.\n# Note: This feature currently applies only to Markdown headings.\n# Minimum value: 0, maximum value: 99, default value: 5.\n# This tag requires that the tag MARKDOWN_SUPPORT is set to YES.\n\nTOC_INCLUDE_HEADINGS = 5\n\n# When enabled doxygen tries to link words that correspond to documented\n# classes, or namespaces to their corresponding documentation. Such a link can\n# be prevented in individual cases by putting a % sign in front of the word or\n# globally by setting AUTOLINK_SUPPORT to NO.\n# The default value is: YES.\n\nAUTOLINK_SUPPORT = YES\n\n# If you use STL classes (i.e. std::string, std::vector, etc.) but do not want\n# to include (a tag file for) the STL sources as input, then you should set this\n# tag to YES in order to let doxygen match functions declarations and\n# definitions whose arguments contain STL classes (e.g. func(std::string);\n# versus func(std::string) {}). This also make the inheritance and collaboration\n# diagrams that involve STL classes more complete and accurate.\n# The default value is: NO.\n\nBUILTIN_STL_SUPPORT = NO\n\n# If you use Microsoft's C++/CLI language, you should set this option to YES to\n# enable parsing support.\n# The default value is: NO.\n\nCPP_CLI_SUPPORT = NO\n\n# Set the SIP_SUPPORT tag to YES if your project consists of sip (see:\n# https://www.riverbankcomputing.com/software/sip/intro) sources only. Doxygen\n# will parse them like normal C++ but will assume all classes use public instead\n# of private inheritance when no explicit protection keyword is present.\n# The default value is: NO.\n\nSIP_SUPPORT = NO\n\n# For Microsoft's IDL there are propget and propput attributes to indicate\n# getter and setter methods for a property. Setting this option to YES will make\n# doxygen to replace the get and set methods by a property in the documentation.\n# This will only work if the methods are indeed getting or setting a simple\n# type. If this is not the case, or you want to show the methods anyway, you\n# should set this option to NO.\n# The default value is: YES.\n\nIDL_PROPERTY_SUPPORT = YES\n\n# If member grouping is used in the documentation and the DISTRIBUTE_GROUP_DOC\n# tag is set to YES then doxygen will reuse the documentation of the first\n# member in the group (if any) for the other members of the group. By default\n# all members of a group must be documented explicitly.\n# The default value is: NO.\n\nDISTRIBUTE_GROUP_DOC = NO\n\n# If one adds a struct or class to a group and this option is enabled, then also\n# any nested class or struct is added to the same group. By default this option\n# is disabled and one has to add nested compounds explicitly via \\ingroup.\n# The default value is: NO.\n\nGROUP_NESTED_COMPOUNDS = NO\n\n# Set the SUBGROUPING tag to YES to allow class member groups of the same type\n# (for instance a group of public functions) to be put as a subgroup of that\n# type (e.g. under the Public Functions section). Set it to NO to prevent\n# subgrouping. Alternatively, this can be done per class using the\n# \\nosubgrouping command.\n# The default value is: YES.\n\nSUBGROUPING = YES\n\n# When the INLINE_GROUPED_CLASSES tag is set to YES, classes, structs and unions\n# are shown inside the group in which they are included (e.g. using \\ingroup)\n# instead of on a separate page (for HTML and Man pages) or section (for LaTeX\n# and RTF).\n#\n# Note that this feature does not work in combination with\n# SEPARATE_MEMBER_PAGES.\n# The default value is: NO.\n\nINLINE_GROUPED_CLASSES = NO\n\n# When the INLINE_SIMPLE_STRUCTS tag is set to YES, structs, classes, and unions\n# with only public data fields or simple typedef fields will be shown inline in\n# the documentation of the scope in which they are defined (i.e. file,\n# namespace, or group documentation), provided this scope is documented. If set\n# to NO, structs, classes, and unions are shown on a separate page (for HTML and\n# Man pages) or section (for LaTeX and RTF).\n# The default value is: NO.\n\nINLINE_SIMPLE_STRUCTS = NO\n\n# When TYPEDEF_HIDES_STRUCT tag is enabled, a typedef of a struct, union, or\n# enum is documented as struct, union, or enum with the name of the typedef. So\n# typedef struct TypeS {} TypeT, will appear in the documentation as a struct\n# with name TypeT. When disabled the typedef will appear as a member of a file,\n# namespace, or class. And the struct will be named TypeS. This can typically be\n# useful for C code in case the coding convention dictates that all compound\n# types are typedef'ed and only the typedef is referenced, never the tag name.\n# The default value is: NO.\n\nTYPEDEF_HIDES_STRUCT = NO\n\n# The size of the symbol lookup cache can be set using LOOKUP_CACHE_SIZE. This\n# cache is used to resolve symbols given their name and scope. Since this can be\n# an expensive process and often the same symbol appears multiple times in the\n# code, doxygen keeps a cache of pre-resolved symbols. If the cache is too small\n# doxygen will become slower. If the cache is too large, memory is wasted. The\n# cache size is given by this formula: 2^(16+LOOKUP_CACHE_SIZE). The valid range\n# is 0..9, the default is 0, corresponding to a cache size of 2^16=65536\n# symbols. At the end of a run doxygen will report the cache usage and suggest\n# the optimal cache size from a speed point of view.\n# Minimum value: 0, maximum value: 9, default value: 0.\n\nLOOKUP_CACHE_SIZE = 0\n\n# The NUM_PROC_THREADS specifies the number threads doxygen is allowed to use\n# during processing. When set to 0 doxygen will based this on the number of\n# cores available in the system. You can set it explicitly to a value larger\n# than 0 to get more control over the balance between CPU load and processing\n# speed. At this moment only the input processing can be done using multiple\n# threads. Since this is still an experimental feature the default is set to 1,\n# which efficively disables parallel processing. Please report any issues you\n# encounter. Generating dot graphs in parallel is controlled by the\n# DOT_NUM_THREADS setting.\n# Minimum value: 0, maximum value: 32, default value: 1.\n\nNUM_PROC_THREADS = 1\n\n#---------------------------------------------------------------------------\n# Build related configuration options\n#---------------------------------------------------------------------------\n\n# If the EXTRACT_ALL tag is set to YES, doxygen will assume all entities in\n# documentation are documented, even if no documentation was available. Private\n# class members and static file members will be hidden unless the\n# EXTRACT_PRIVATE respectively EXTRACT_STATIC tags are set to YES.\n# Note: This will also disable the warnings about undocumented members that are\n# normally produced when WARNINGS is set to YES.\n# The default value is: NO.\n\nEXTRACT_ALL = YES\n\n# If the EXTRACT_PRIVATE tag is set to YES, all private members of a class will\n# be included in the documentation.\n# The default value is: NO.\n\nEXTRACT_PRIVATE = NO\n\n# If the EXTRACT_PRIV_VIRTUAL tag is set to YES, documented private virtual\n# methods of a class will be included in the documentation.\n# The default value is: NO.\n\nEXTRACT_PRIV_VIRTUAL = NO\n\n# If the EXTRACT_PACKAGE tag is set to YES, all members with package or internal\n# scope will be included in the documentation.\n# The default value is: NO.\n\nEXTRACT_PACKAGE = NO\n\n# If the EXTRACT_STATIC tag is set to YES, all static members of a file will be\n# included in the documentation.\n# The default value is: NO.\n\nEXTRACT_STATIC = NO\n\n# If the EXTRACT_LOCAL_CLASSES tag is set to YES, classes (and structs) defined\n# locally in source files will be included in the documentation. If set to NO,\n# only classes defined in header files are included. Does not have any effect\n# for Java sources.\n# The default value is: YES.\n\nEXTRACT_LOCAL_CLASSES = YES\n\n# This flag is only useful for Objective-C code. If set to YES, local methods,\n# which are defined in the implementation section but not in the interface are\n# included in the documentation. If set to NO, only methods in the interface are\n# included.\n# The default value is: NO.\n\nEXTRACT_LOCAL_METHODS = NO\n\n# If this flag is set to YES, the members of anonymous namespaces will be\n# extracted and appear in the documentation as a namespace called\n# 'anonymous_namespace{file}', where file will be replaced with the base name of\n# the file that contains the anonymous namespace. By default anonymous namespace\n# are hidden.\n# The default value is: NO.\n\nEXTRACT_ANON_NSPACES = NO\n\n# If this flag is set to YES, the name of an unnamed parameter in a declaration\n# will be determined by the corresponding definition. By default unnamed\n# parameters remain unnamed in the output.\n# The default value is: YES.\n\nRESOLVE_UNNAMED_PARAMS = YES\n\n# If the HIDE_UNDOC_MEMBERS tag is set to YES, doxygen will hide all\n# undocumented members inside documented classes or files. If set to NO these\n# members will be included in the various overviews, but no documentation\n# section is generated. This option has no effect if EXTRACT_ALL is enabled.\n# The default value is: NO.\n\nHIDE_UNDOC_MEMBERS = NO\n\n# If the HIDE_UNDOC_CLASSES tag is set to YES, doxygen will hide all\n# undocumented classes that are normally visible in the class hierarchy. If set\n# to NO, these classes will be included in the various overviews. This option\n# has no effect if EXTRACT_ALL is enabled.\n# The default value is: NO.\n\nHIDE_UNDOC_CLASSES = NO\n\n# If the HIDE_FRIEND_COMPOUNDS tag is set to YES, doxygen will hide all friend\n# declarations. If set to NO, these declarations will be included in the\n# documentation.\n# The default value is: NO.\n\nHIDE_FRIEND_COMPOUNDS = NO\n\n# If the HIDE_IN_BODY_DOCS tag is set to YES, doxygen will hide any\n# documentation blocks found inside the body of a function. If set to NO, these\n# blocks will be appended to the function's detailed documentation block.\n# The default value is: NO.\n\nHIDE_IN_BODY_DOCS = NO\n\n# The INTERNAL_DOCS tag determines if documentation that is typed after a\n# \\internal command is included. If the tag is set to NO then the documentation\n# will be excluded. Set it to YES to include the internal documentation.\n# The default value is: NO.\n\nINTERNAL_DOCS = NO\n\n# With the correct setting of option CASE_SENSE_NAMES doxygen will better be\n# able to match the capabilities of the underlying filesystem. In case the\n# filesystem is case sensitive (i.e. it supports files in the same directory\n# whose names only differ in casing), the option must be set to YES to properly\n# deal with such files in case they appear in the input. For filesystems that\n# are not case sensitive the option should be be set to NO to properly deal with\n# output files written for symbols that only differ in casing, such as for two\n# classes, one named CLASS and the other named Class, and to also support\n# references to files without having to specify the exact matching casing. On\n# Windows (including Cygwin) and MacOS, users should typically set this option\n# to NO, whereas on Linux or other Unix flavors it should typically be set to\n# YES.\n# The default value is: system dependent.\n\nCASE_SENSE_NAMES = YES\n\n# If the HIDE_SCOPE_NAMES tag is set to NO then doxygen will show members with\n# their full class and namespace scopes in the documentation. If set to YES, the\n# scope will be hidden.\n# The default value is: NO.\n\nHIDE_SCOPE_NAMES = NO\n\n# If the HIDE_COMPOUND_REFERENCE tag is set to NO (default) then doxygen will\n# append additional text to a page's title, such as Class Reference. If set to\n# YES the compound reference will be hidden.\n# The default value is: NO.\n\nHIDE_COMPOUND_REFERENCE= NO\n\n# If the SHOW_INCLUDE_FILES tag is set to YES then doxygen will put a list of\n# the files that are included by a file in the documentation of that file.\n# The default value is: YES.\n\nSHOW_INCLUDE_FILES = YES\n\n# If the SHOW_GROUPED_MEMB_INC tag is set to YES then Doxygen will add for each\n# grouped member an include statement to the documentation, telling the reader\n# which file to include in order to use the member.\n# The default value is: NO.\n\nSHOW_GROUPED_MEMB_INC = NO\n\n# If the FORCE_LOCAL_INCLUDES tag is set to YES then doxygen will list include\n# files with double quotes in the documentation rather than with sharp brackets.\n# The default value is: NO.\n\nFORCE_LOCAL_INCLUDES = NO\n\n# If the INLINE_INFO tag is set to YES then a tag [inline] is inserted in the\n# documentation for inline members.\n# The default value is: YES.\n\nINLINE_INFO = YES\n\n# If the SORT_MEMBER_DOCS tag is set to YES then doxygen will sort the\n# (detailed) documentation of file and class members alphabetically by member\n# name. If set to NO, the members will appear in declaration order.\n# The default value is: YES.\n\nSORT_MEMBER_DOCS = YES\n\n# If the SORT_BRIEF_DOCS tag is set to YES then doxygen will sort the brief\n# descriptions of file, namespace and class members alphabetically by member\n# name. If set to NO, the members will appear in declaration order. Note that\n# this will also influence the order of the classes in the class list.\n# The default value is: NO.\n\nSORT_BRIEF_DOCS = NO\n\n# If the SORT_MEMBERS_CTORS_1ST tag is set to YES then doxygen will sort the\n# (brief and detailed) documentation of class members so that constructors and\n# destructors are listed first. If set to NO the constructors will appear in the\n# respective orders defined by SORT_BRIEF_DOCS and SORT_MEMBER_DOCS.\n# Note: If SORT_BRIEF_DOCS is set to NO this option is ignored for sorting brief\n# member documentation.\n# Note: If SORT_MEMBER_DOCS is set to NO this option is ignored for sorting\n# detailed member documentation.\n# The default value is: NO.\n\nSORT_MEMBERS_CTORS_1ST = NO\n\n# If the SORT_GROUP_NAMES tag is set to YES then doxygen will sort the hierarchy\n# of group names into alphabetical order. If set to NO the group names will\n# appear in their defined order.\n# The default value is: NO.\n\nSORT_GROUP_NAMES = NO\n\n# If the SORT_BY_SCOPE_NAME tag is set to YES, the class list will be sorted by\n# fully-qualified names, including namespaces. If set to NO, the class list will\n# be sorted only by class name, not including the namespace part.\n# Note: This option is not very useful if HIDE_SCOPE_NAMES is set to YES.\n# Note: This option applies only to the class list, not to the alphabetical\n# list.\n# The default value is: NO.\n\nSORT_BY_SCOPE_NAME = NO\n\n# If the STRICT_PROTO_MATCHING option is enabled and doxygen fails to do proper\n# type resolution of all parameters of a function it will reject a match between\n# the prototype and the implementation of a member function even if there is\n# only one candidate or it is obvious which candidate to choose by doing a\n# simple string match. By disabling STRICT_PROTO_MATCHING doxygen will still\n# accept a match between prototype and implementation in such cases.\n# The default value is: NO.\n\nSTRICT_PROTO_MATCHING = NO\n\n# The GENERATE_TODOLIST tag can be used to enable (YES) or disable (NO) the todo\n# list. This list is created by putting \\todo commands in the documentation.\n# The default value is: YES.\n\nGENERATE_TODOLIST = YES\n\n# The GENERATE_TESTLIST tag can be used to enable (YES) or disable (NO) the test\n# list. This list is created by putting \\test commands in the documentation.\n# The default value is: YES.\n\nGENERATE_TESTLIST = YES\n\n# The GENERATE_BUGLIST tag can be used to enable (YES) or disable (NO) the bug\n# list. This list is created by putting \\bug commands in the documentation.\n# The default value is: YES.\n\nGENERATE_BUGLIST = YES\n\n# The GENERATE_DEPRECATEDLIST tag can be used to enable (YES) or disable (NO)\n# the deprecated list. This list is created by putting \\deprecated commands in\n# the documentation.\n# The default value is: YES.\n\nGENERATE_DEPRECATEDLIST= YES\n\n# The ENABLED_SECTIONS tag can be used to enable conditional documentation\n# sections, marked by \\if ... \\endif and \\cond \n# ... \\endcond blocks.\n\nENABLED_SECTIONS =\n\n# The MAX_INITIALIZER_LINES tag determines the maximum number of lines that the\n# initial value of a variable or macro / define can have for it to appear in the\n# documentation. If the initializer consists of more lines than specified here\n# it will be hidden. Use a value of 0 to hide initializers completely. The\n# appearance of the value of individual variables and macros / defines can be\n# controlled using \\showinitializer or \\hideinitializer command in the\n# documentation regardless of this setting.\n# Minimum value: 0, maximum value: 10000, default value: 30.\n\nMAX_INITIALIZER_LINES = 30\n\n# Set the SHOW_USED_FILES tag to NO to disable the list of files generated at\n# the bottom of the documentation of classes and structs. If set to YES, the\n# list will mention the files that were used to generate the documentation.\n# The default value is: YES.\n\nSHOW_USED_FILES = YES\n\n# Set the SHOW_FILES tag to NO to disable the generation of the Files page. This\n# will remove the Files entry from the Quick Index and from the Folder Tree View\n# (if specified).\n# The default value is: YES.\n\nSHOW_FILES = YES\n\n# Set the SHOW_NAMESPACES tag to NO to disable the generation of the Namespaces\n# page. This will remove the Namespaces entry from the Quick Index and from the\n# Folder Tree View (if specified).\n# The default value is: YES.\n\nSHOW_NAMESPACES = YES\n\n# The FILE_VERSION_FILTER tag can be used to specify a program or script that\n# doxygen should invoke to get the current version for each file (typically from\n# the version control system). Doxygen will invoke the program by executing (via\n# popen()) the command command input-file, where command is the value of the\n# FILE_VERSION_FILTER tag, and input-file is the name of an input file provided\n# by doxygen. Whatever the program writes to standard output is used as the file\n# version. For an example see the documentation.\n\nFILE_VERSION_FILTER =\n\n# The LAYOUT_FILE tag can be used to specify a layout file which will be parsed\n# by doxygen. The layout file controls the global structure of the generated\n# output files in an output format independent way. To create the layout file\n# that represents doxygen's defaults, run doxygen with the -l option. You can\n# optionally specify a file name after the option, if omitted DoxygenLayout.xml\n# will be used as the name of the layout file.\n#\n# Note that if you run doxygen from a directory containing a file called\n# DoxygenLayout.xml, doxygen will parse it automatically even if the LAYOUT_FILE\n# tag is left empty.\n\nLAYOUT_FILE =\n\n# The CITE_BIB_FILES tag can be used to specify one or more bib files containing\n# the reference definitions. This must be a list of .bib files. The .bib\n# extension is automatically appended if omitted. This requires the bibtex tool\n# to be installed. See also https://en.wikipedia.org/wiki/BibTeX for more info.\n# For LaTeX the style of the bibliography can be controlled using\n# LATEX_BIB_STYLE. To use this feature you need bibtex and perl available in the\n# search path. See also \\cite for info how to create references.\n\nCITE_BIB_FILES =\n\n#---------------------------------------------------------------------------\n# Configuration options related to warning and progress messages\n#---------------------------------------------------------------------------\n\n# The QUIET tag can be used to turn on/off the messages that are generated to\n# standard output by doxygen. If QUIET is set to YES this implies that the\n# messages are off.\n# The default value is: NO.\n\nQUIET = NO\n\n# The WARNINGS tag can be used to turn on/off the warning messages that are\n# generated to standard error (stderr) by doxygen. If WARNINGS is set to YES\n# this implies that the warnings are on.\n#\n# Tip: Turn warnings on while writing the documentation.\n# The default value is: YES.\n\nWARNINGS = YES\n\n# If the WARN_IF_UNDOCUMENTED tag is set to YES then doxygen will generate\n# warnings for undocumented members. If EXTRACT_ALL is set to YES then this flag\n# will automatically be disabled.\n# The default value is: YES.\n\nWARN_IF_UNDOCUMENTED = YES\n\n# If the WARN_IF_DOC_ERROR tag is set to YES, doxygen will generate warnings for\n# potential errors in the documentation, such as not documenting some parameters\n# in a documented function, or documenting parameters that don't exist or using\n# markup commands wrongly.\n# The default value is: YES.\n\nWARN_IF_DOC_ERROR = YES\n\n# This WARN_NO_PARAMDOC option can be enabled to get warnings for functions that\n# are documented, but have no documentation for their parameters or return\n# value. If set to NO, doxygen will only warn about wrong or incomplete\n# parameter documentation, but not about the absence of documentation. If\n# EXTRACT_ALL is set to YES then this flag will automatically be disabled.\n# The default value is: NO.\n\nWARN_NO_PARAMDOC = YES\n\n# If the WARN_AS_ERROR tag is set to YES then doxygen will immediately stop when\n# a warning is encountered. If the WARN_AS_ERROR tag is set to FAIL_ON_WARNINGS\n# then doxygen will continue running as if WARN_AS_ERROR tag is set to NO, but\n# at the end of the doxygen process doxygen will return with a non-zero status.\n# Possible values are: NO, YES and FAIL_ON_WARNINGS.\n# The default value is: NO.\n\nWARN_AS_ERROR = NO\n\n# The WARN_FORMAT tag determines the format of the warning messages that doxygen\n# can produce. The string should contain the $file, $line, and $text tags, which\n# will be replaced by the file and line number from which the warning originated\n# and the warning text. Optionally the format may contain $version, which will\n# be replaced by the version of the file (if it could be obtained via\n# FILE_VERSION_FILTER)\n# The default value is: $file:$line: $text.\n\nWARN_FORMAT = \"$file:$line: $text\"\n\n# The WARN_LOGFILE tag can be used to specify a file to which warning and error\n# messages should be written. If left blank the output is written to standard\n# error (stderr).\n\nWARN_LOGFILE =\n\n#---------------------------------------------------------------------------\n# Configuration options related to the input files\n#---------------------------------------------------------------------------\n\n# The INPUT tag is used to specify the files and/or directories that contain\n# documented source files. You may enter file names like myfile.cpp or\n# directories like /usr/src/myproject. Separate the files or directories with\n# spaces. See also FILE_PATTERNS and EXTENSION_MAPPING\n# Note: If this tag is empty the current directory is searched.\n\nINPUT = @PROJECT_SOURCE_DIR@/include\n\n# This tag can be used to specify the character encoding of the source files\n# that doxygen parses. Internally doxygen uses the UTF-8 encoding. Doxygen uses\n# libiconv (or the iconv built into libc) for the transcoding. See the libiconv\n# documentation (see:\n# https://www.gnu.org/software/libiconv/) for the list of possible encodings.\n# The default value is: UTF-8.\n\nINPUT_ENCODING = UTF-8\n\n# If the value of the INPUT tag contains directories, you can use the\n# FILE_PATTERNS tag to specify one or more wildcard patterns (like *.cpp and\n# *.h) to filter out the source-files in the directories.\n#\n# Note that for custom extensions or not directly supported extensions you also\n# need to set EXTENSION_MAPPING for the extension otherwise the files are not\n# read by doxygen.\n#\n# Note the list of default checked file patterns might differ from the list of\n# default file extension mappings.\n#\n# If left blank the following patterns are tested:*.c, *.cc, *.cxx, *.cpp,\n# *.c++, *.java, *.ii, *.ixx, *.ipp, *.i++, *.inl, *.idl, *.ddl, *.odl, *.h,\n# *.hh, *.hxx, *.hpp, *.h++, *.cs, *.d, *.php, *.php4, *.php5, *.phtml, *.inc,\n# *.m, *.markdown, *.md, *.mm, *.dox (to be provided as doxygen C comment),\n# *.py, *.pyw, *.f90, *.f95, *.f03, *.f08, *.f18, *.f, *.for, *.vhd, *.vhdl,\n# *.ucf, *.qsf and *.ice.\n\nFILE_PATTERNS = *.h\n\n# The RECURSIVE tag can be used to specify whether or not subdirectories should\n# be searched for input files as well.\n# The default value is: NO.\n\nRECURSIVE = YES\n\n# The EXCLUDE tag can be used to specify files and/or directories that should be\n# excluded from the INPUT source files. This way you can easily exclude a\n# subdirectory from a directory tree whose root is specified with the INPUT tag.\n#\n# Note that relative paths are relative to the directory from which doxygen is\n# run.\n\nEXCLUDE =\n\n# The EXCLUDE_SYMLINKS tag can be used to select whether or not files or\n# directories that are symbolic links (a Unix file system feature) are excluded\n# from the input.\n# The default value is: NO.\n\nEXCLUDE_SYMLINKS = NO\n\n# If the value of the INPUT tag contains directories, you can use the\n# EXCLUDE_PATTERNS tag to specify one or more wildcard patterns to exclude\n# certain files from those directories.\n#\n# Note that the wildcards are matched against the file with absolute path, so to\n# exclude all test directories for example use the pattern */test/*\n\nEXCLUDE_PATTERNS = */test/* \\\n logging.h\n\n# The EXCLUDE_SYMBOLS tag can be used to specify one or more symbol names\n# (namespaces, classes, functions, etc.) that should be excluded from the\n# output. The symbol name can be a fully qualified name, a word, or if the\n# wildcard * is used, a substring. Examples: ANamespace, AClass,\n# AClass::ANamespace, ANamespace::*Test\n#\n# Note that the wildcards are matched against the file with absolute path, so to\n# exclude all test directories use the pattern */test/*\n\nEXCLUDE_SYMBOLS =\n\n# The EXAMPLE_PATH tag can be used to specify one or more files or directories\n# that contain example code fragments that are included (see the \\include\n# command).\n\nEXAMPLE_PATH = @PROJECT_SOURCE_DIR@/demo/c-api/\n\n# If the value of the EXAMPLE_PATH tag contains directories, you can use the\n# EXAMPLE_PATTERNS tag to specify one or more wildcard pattern (like *.cpp and\n# *.h) to filter out the source-files in the directories. If left blank all\n# files are included.\n\nEXAMPLE_PATTERNS =\n\n# If the EXAMPLE_RECURSIVE tag is set to YES then subdirectories will be\n# searched for input files to be used with the \\include or \\dontinclude commands\n# irrespective of the value of the RECURSIVE tag.\n# The default value is: NO.\n\nEXAMPLE_RECURSIVE = YES\n\n# The IMAGE_PATH tag can be used to specify one or more files or directories\n# that contain images that are to be included in the documentation (see the\n# \\image command).\n\nIMAGE_PATH =\n\n# The INPUT_FILTER tag can be used to specify a program that doxygen should\n# invoke to filter for each input file. Doxygen will invoke the filter program\n# by executing (via popen()) the command:\n#\n# \n#\n# where is the value of the INPUT_FILTER tag, and is the\n# name of an input file. Doxygen will then use the output that the filter\n# program writes to standard output. If FILTER_PATTERNS is specified, this tag\n# will be ignored.\n#\n# Note that the filter must not add or remove lines; it is applied before the\n# code is scanned, but not when the output code is generated. If lines are added\n# or removed, the anchors will not be placed correctly.\n#\n# Note that for custom extensions or not directly supported extensions you also\n# need to set EXTENSION_MAPPING for the extension otherwise the files are not\n# properly processed by doxygen.\n\nINPUT_FILTER =\n\n# The FILTER_PATTERNS tag can be used to specify filters on a per file pattern\n# basis. Doxygen will compare the file name with each pattern and apply the\n# filter if there is a match. The filters are a list of the form: pattern=filter\n# (like *.cpp=my_cpp_filter). See INPUT_FILTER for further information on how\n# filters are used. If the FILTER_PATTERNS tag is empty or if none of the\n# patterns match the file name, INPUT_FILTER is applied.\n#\n# Note that for custom extensions or not directly supported extensions you also\n# need to set EXTENSION_MAPPING for the extension otherwise the files are not\n# properly processed by doxygen.\n\nFILTER_PATTERNS =\n\n# If the FILTER_SOURCE_FILES tag is set to YES, the input filter (if set using\n# INPUT_FILTER) will also be used to filter the input files that are used for\n# producing the source files to browse (i.e. when SOURCE_BROWSER is set to YES).\n# The default value is: NO.\n\nFILTER_SOURCE_FILES = NO\n\n# The FILTER_SOURCE_PATTERNS tag can be used to specify source filters per file\n# pattern. A pattern will override the setting for FILTER_PATTERN (if any) and\n# it is also possible to disable source filtering for a specific pattern using\n# *.ext= (so without naming a filter).\n# This tag requires that the tag FILTER_SOURCE_FILES is set to YES.\n\nFILTER_SOURCE_PATTERNS =\n\n# If the USE_MDFILE_AS_MAINPAGE tag refers to the name of a markdown file that\n# is part of the input, its contents will be placed on the main page\n# (index.html). This can be useful if you have a project on for instance GitHub\n# and want to reuse the introduction page also for the doxygen output.\n\nUSE_MDFILE_AS_MAINPAGE =\n\n#---------------------------------------------------------------------------\n# Configuration options related to source browsing\n#---------------------------------------------------------------------------\n\n# If the SOURCE_BROWSER tag is set to YES then a list of source files will be\n# generated. Documented entities will be cross-referenced with these sources.\n#\n# Note: To get rid of all source code in the generated output, make sure that\n# also VERBATIM_HEADERS is set to NO.\n# The default value is: NO.\n\nSOURCE_BROWSER = NO\n\n# Setting the INLINE_SOURCES tag to YES will include the body of functions,\n# classes and enums directly into the documentation.\n# The default value is: NO.\n\nINLINE_SOURCES = NO\n\n# Setting the STRIP_CODE_COMMENTS tag to YES will instruct doxygen to hide any\n# special comment blocks from generated source code fragments. Normal C, C++ and\n# Fortran comments will always remain visible.\n# The default value is: YES.\n\nSTRIP_CODE_COMMENTS = YES\n\n# If the REFERENCED_BY_RELATION tag is set to YES then for each documented\n# entity all documented functions referencing it will be listed.\n# The default value is: NO.\n\nREFERENCED_BY_RELATION = NO\n\n# If the REFERENCES_RELATION tag is set to YES then for each documented function\n# all documented entities called/used by that function will be listed.\n# The default value is: NO.\n\nREFERENCES_RELATION = NO\n\n# If the REFERENCES_LINK_SOURCE tag is set to YES and SOURCE_BROWSER tag is set\n# to YES then the hyperlinks from functions in REFERENCES_RELATION and\n# REFERENCED_BY_RELATION lists will link to the source code. Otherwise they will\n# link to the documentation.\n# The default value is: YES.\n\nREFERENCES_LINK_SOURCE = YES\n\n# If SOURCE_TOOLTIPS is enabled (the default) then hovering a hyperlink in the\n# source code will show a tooltip with additional information such as prototype,\n# brief description and links to the definition and documentation. Since this\n# will make the HTML file larger and loading of large files a bit slower, you\n# can opt to disable this feature.\n# The default value is: YES.\n# This tag requires that the tag SOURCE_BROWSER is set to YES.\n\nSOURCE_TOOLTIPS = YES\n\n# If the USE_HTAGS tag is set to YES then the references to source code will\n# point to the HTML generated by the htags(1) tool instead of doxygen built-in\n# source browser. The htags tool is part of GNU's global source tagging system\n# (see https://www.gnu.org/software/global/global.html). You will need version\n# 4.8.6 or higher.\n#\n# To use it do the following:\n# - Install the latest version of global\n# - Enable SOURCE_BROWSER and USE_HTAGS in the configuration file\n# - Make sure the INPUT points to the root of the source tree\n# - Run doxygen as normal\n#\n# Doxygen will invoke htags (and that will in turn invoke gtags), so these\n# tools must be available from the command line (i.e. in the search path).\n#\n# The result: instead of the source browser generated by doxygen, the links to\n# source code will now point to the output of htags.\n# The default value is: NO.\n# This tag requires that the tag SOURCE_BROWSER is set to YES.\n\nUSE_HTAGS = NO\n\n# If the VERBATIM_HEADERS tag is set the YES then doxygen will generate a\n# verbatim copy of the header file for each class for which an include is\n# specified. Set to NO to disable this.\n# See also: Section \\class.\n# The default value is: YES.\n\nVERBATIM_HEADERS = YES\n\n# If the CLANG_ASSISTED_PARSING tag is set to YES then doxygen will use the\n# clang parser (see:\n# http://clang.llvm.org/) for more accurate parsing at the cost of reduced\n# performance. This can be particularly helpful with template rich C++ code for\n# which doxygen's built-in parser lacks the necessary type information.\n# Note: The availability of this option depends on whether or not doxygen was\n# generated with the -Duse_libclang=ON option for CMake.\n# The default value is: NO.\n\nCLANG_ASSISTED_PARSING = NO\n\n# If clang assisted parsing is enabled and the CLANG_ADD_INC_PATHS tag is set to\n# YES then doxygen will add the directory of each input to the include path.\n# The default value is: YES.\n\nCLANG_ADD_INC_PATHS = YES\n\n# If clang assisted parsing is enabled you can provide the compiler with command\n# line options that you would normally use when invoking the compiler. Note that\n# the include paths will already be set by doxygen for the files and directories\n# specified with INPUT and INCLUDE_PATH.\n# This tag requires that the tag CLANG_ASSISTED_PARSING is set to YES.\n\nCLANG_OPTIONS =\n\n# If clang assisted parsing is enabled you can provide the clang parser with the\n# path to the directory containing a file called compile_commands.json. This\n# file is the compilation database (see:\n# http://clang.llvm.org/docs/HowToSetupToolingForLLVM.html) containing the\n# options used when the source files were built. This is equivalent to\n# specifying the -p option to a clang tool, such as clang-check. These options\n# will then be passed to the parser. Any options specified with CLANG_OPTIONS\n# will be added as well.\n# Note: The availability of this option depends on whether or not doxygen was\n# generated with the -Duse_libclang=ON option for CMake.\n\nCLANG_DATABASE_PATH =\n\n#---------------------------------------------------------------------------\n# Configuration options related to the alphabetical class index\n#---------------------------------------------------------------------------\n\n# If the ALPHABETICAL_INDEX tag is set to YES, an alphabetical index of all\n# compounds will be generated. Enable this if the project contains a lot of\n# classes, structs, unions or interfaces.\n# The default value is: YES.\n\nALPHABETICAL_INDEX = YES\n\n# In case all classes in a project start with a common prefix, all classes will\n# be put under the same header in the alphabetical index. The IGNORE_PREFIX tag\n# can be used to specify a prefix (or a list of prefixes) that should be ignored\n# while generating the index headers.\n# This tag requires that the tag ALPHABETICAL_INDEX is set to YES.\n\nIGNORE_PREFIX =\n\n#---------------------------------------------------------------------------\n# Configuration options related to the HTML output\n#---------------------------------------------------------------------------\n\n# If the GENERATE_HTML tag is set to YES, doxygen will generate HTML output\n# The default value is: YES.\n\nGENERATE_HTML = YES\n\n# The HTML_OUTPUT tag is used to specify where the HTML docs will be put. If a\n# relative path is entered the value of OUTPUT_DIRECTORY will be put in front of\n# it.\n# The default directory is: html.\n# This tag requires that the tag GENERATE_HTML is set to YES.\n\nHTML_OUTPUT = html\n\n# The HTML_FILE_EXTENSION tag can be used to specify the file extension for each\n# generated HTML page (for example: .htm, .php, .asp).\n# The default value is: .html.\n# This tag requires that the tag GENERATE_HTML is set to YES.\n\nHTML_FILE_EXTENSION = .html\n\n# The HTML_HEADER tag can be used to specify a user-defined HTML header file for\n# each generated HTML page. If the tag is left blank doxygen will generate a\n# standard header.\n#\n# To get valid HTML the header file that includes any scripts and style sheets\n# that doxygen needs, which is dependent on the configuration options used (e.g.\n# the setting GENERATE_TREEVIEW). It is highly recommended to start with a\n# default header using\n# doxygen -w html new_header.html new_footer.html new_stylesheet.css\n# YourConfigFile\n# and then modify the file new_header.html. See also section \"Doxygen usage\"\n# for information on how to generate the default header that doxygen normally\n# uses.\n# Note: The header is subject to change so you typically have to regenerate the\n# default header when upgrading to a newer version of doxygen. For a description\n# of the possible markers and block names see the documentation.\n# This tag requires that the tag GENERATE_HTML is set to YES.\n\nHTML_HEADER =\n\n# The HTML_FOOTER tag can be used to specify a user-defined HTML footer for each\n# generated HTML page. If the tag is left blank doxygen will generate a standard\n# footer. See HTML_HEADER for more information on how to generate a default\n# footer and what special commands can be used inside the footer. See also\n# section \"Doxygen usage\" for information on how to generate the default footer\n# that doxygen normally uses.\n# This tag requires that the tag GENERATE_HTML is set to YES.\n\nHTML_FOOTER =\n\n# The HTML_STYLESHEET tag can be used to specify a user-defined cascading style\n# sheet that is used by each HTML page. It can be used to fine-tune the look of\n# the HTML output. If left blank doxygen will generate a default style sheet.\n# See also section \"Doxygen usage\" for information on how to generate the style\n# sheet that doxygen normally uses.\n# Note: It is recommended to use HTML_EXTRA_STYLESHEET instead of this tag, as\n# it is more robust and this tag (HTML_STYLESHEET) will in the future become\n# obsolete.\n# This tag requires that the tag GENERATE_HTML is set to YES.\n\nHTML_STYLESHEET =\n\n# The HTML_EXTRA_STYLESHEET tag can be used to specify additional user-defined\n# cascading style sheets that are included after the standard style sheets\n# created by doxygen. Using this option one can overrule certain style aspects.\n# This is preferred over using HTML_STYLESHEET since it does not replace the\n# standard style sheet and is therefore more robust against future updates.\n# Doxygen will copy the style sheet files to the output directory.\n# Note: The order of the extra style sheet files is of importance (e.g. the last\n# style sheet in the list overrules the setting of the previous ones in the\n# list). For an example see the documentation.\n# This tag requires that the tag GENERATE_HTML is set to YES.\n\nHTML_EXTRA_STYLESHEET =\n\n# The HTML_EXTRA_FILES tag can be used to specify one or more extra images or\n# other source files which should be copied to the HTML output directory. Note\n# that these files will be copied to the base HTML output directory. Use the\n# $relpath^ marker in the HTML_HEADER and/or HTML_FOOTER files to load these\n# files. In the HTML_STYLESHEET file, use the file name only. Also note that the\n# files will be copied as-is; there are no commands or markers available.\n# This tag requires that the tag GENERATE_HTML is set to YES.\n\nHTML_EXTRA_FILES =\n\n# The HTML_COLORSTYLE_HUE tag controls the color of the HTML output. Doxygen\n# will adjust the colors in the style sheet and background images according to\n# this color. Hue is specified as an angle on a colorwheel, see\n# https://en.wikipedia.org/wiki/Hue for more information. For instance the value\n# 0 represents red, 60 is yellow, 120 is green, 180 is cyan, 240 is blue, 300\n# purple, and 360 is red again.\n# Minimum value: 0, maximum value: 359, default value: 220.\n# This tag requires that the tag GENERATE_HTML is set to YES.\n\nHTML_COLORSTYLE_HUE = 220\n\n# The HTML_COLORSTYLE_SAT tag controls the purity (or saturation) of the colors\n# in the HTML output. For a value of 0 the output will use grayscales only. A\n# value of 255 will produce the most vivid colors.\n# Minimum value: 0, maximum value: 255, default value: 100.\n# This tag requires that the tag GENERATE_HTML is set to YES.\n\nHTML_COLORSTYLE_SAT = 100\n\n# The HTML_COLORSTYLE_GAMMA tag controls the gamma correction applied to the\n# luminance component of the colors in the HTML output. Values below 100\n# gradually make the output lighter, whereas values above 100 make the output\n# darker. The value divided by 100 is the actual gamma applied, so 80 represents\n# a gamma of 0.8, The value 220 represents a gamma of 2.2, and 100 does not\n# change the gamma.\n# Minimum value: 40, maximum value: 240, default value: 80.\n# This tag requires that the tag GENERATE_HTML is set to YES.\n\nHTML_COLORSTYLE_GAMMA = 80\n\n# If the HTML_TIMESTAMP tag is set to YES then the footer of each generated HTML\n# page will contain the date and time when the page was generated. Setting this\n# to YES can help to show when doxygen was last run and thus if the\n# documentation is up to date.\n# The default value is: NO.\n# This tag requires that the tag GENERATE_HTML is set to YES.\n\nHTML_TIMESTAMP = YES\n\n# If the HTML_DYNAMIC_MENUS tag is set to YES then the generated HTML\n# documentation will contain a main index with vertical navigation menus that\n# are dynamically created via JavaScript. If disabled, the navigation index will\n# consists of multiple levels of tabs that are statically embedded in every HTML\n# page. Disable this option to support browsers that do not have JavaScript,\n# like the Qt help browser.\n# The default value is: YES.\n# This tag requires that the tag GENERATE_HTML is set to YES.\n\nHTML_DYNAMIC_MENUS = YES\n\n# If the HTML_DYNAMIC_SECTIONS tag is set to YES then the generated HTML\n# documentation will contain sections that can be hidden and shown after the\n# page has loaded.\n# The default value is: NO.\n# This tag requires that the tag GENERATE_HTML is set to YES.\n\nHTML_DYNAMIC_SECTIONS = NO\n\n# With HTML_INDEX_NUM_ENTRIES one can control the preferred number of entries\n# shown in the various tree structured indices initially; the user can expand\n# and collapse entries dynamically later on. Doxygen will expand the tree to\n# such a level that at most the specified number of entries are visible (unless\n# a fully collapsed tree already exceeds this amount). So setting the number of\n# entries 1 will produce a full collapsed tree by default. 0 is a special value\n# representing an infinite number of entries and will result in a full expanded\n# tree by default.\n# Minimum value: 0, maximum value: 9999, default value: 100.\n# This tag requires that the tag GENERATE_HTML is set to YES.\n\nHTML_INDEX_NUM_ENTRIES = 100\n\n# If the GENERATE_DOCSET tag is set to YES, additional index files will be\n# generated that can be used as input for Apple's Xcode 3 integrated development\n# environment (see:\n# https://developer.apple.com/xcode/), introduced with OSX 10.5 (Leopard). To\n# create a documentation set, doxygen will generate a Makefile in the HTML\n# output directory. Running make will produce the docset in that directory and\n# running make install will install the docset in\n# ~/Library/Developer/Shared/Documentation/DocSets so that Xcode will find it at\n# startup. See https://developer.apple.com/library/archive/featuredarticles/Doxy\n# genXcode/_index.html for more information.\n# The default value is: NO.\n# This tag requires that the tag GENERATE_HTML is set to YES.\n\nGENERATE_DOCSET = NO\n\n# This tag determines the name of the docset feed. A documentation feed provides\n# an umbrella under which multiple documentation sets from a single provider\n# (such as a company or product suite) can be grouped.\n# The default value is: Doxygen generated docs.\n# This tag requires that the tag GENERATE_DOCSET is set to YES.\n\nDOCSET_FEEDNAME = \"Doxygen generated docs\"\n\n# This tag specifies a string that should uniquely identify the documentation\n# set bundle. This should be a reverse domain-name style string, e.g.\n# com.mycompany.MyDocSet. Doxygen will append .docset to the name.\n# The default value is: org.doxygen.Project.\n# This tag requires that the tag GENERATE_DOCSET is set to YES.\n\nDOCSET_BUNDLE_ID = org.doxygen.Project\n\n# The DOCSET_PUBLISHER_ID tag specifies a string that should uniquely identify\n# the documentation publisher. This should be a reverse domain-name style\n# string, e.g. com.mycompany.MyDocSet.documentation.\n# The default value is: org.doxygen.Publisher.\n# This tag requires that the tag GENERATE_DOCSET is set to YES.\n\nDOCSET_PUBLISHER_ID = org.doxygen.Publisher\n\n# The DOCSET_PUBLISHER_NAME tag identifies the documentation publisher.\n# The default value is: Publisher.\n# This tag requires that the tag GENERATE_DOCSET is set to YES.\n\nDOCSET_PUBLISHER_NAME = Publisher\n\n# If the GENERATE_HTMLHELP tag is set to YES then doxygen generates three\n# additional HTML index files: index.hhp, index.hhc, and index.hhk. The\n# index.hhp is a project file that can be read by Microsoft's HTML Help Workshop\n# (see:\n# https://www.microsoft.com/en-us/download/details.aspx?id=21138) on Windows.\n#\n# The HTML Help Workshop contains a compiler that can convert all HTML output\n# generated by doxygen into a single compiled HTML file (.chm). Compiled HTML\n# files are now used as the Windows 98 help format, and will replace the old\n# Windows help format (.hlp) on all Windows platforms in the future. Compressed\n# HTML files also contain an index, a table of contents, and you can search for\n# words in the documentation. The HTML workshop also contains a viewer for\n# compressed HTML files.\n# The default value is: NO.\n# This tag requires that the tag GENERATE_HTML is set to YES.\n\nGENERATE_HTMLHELP = NO\n\n# The CHM_FILE tag can be used to specify the file name of the resulting .chm\n# file. You can add a path in front of the file if the result should not be\n# written to the html output directory.\n# This tag requires that the tag GENERATE_HTMLHELP is set to YES.\n\nCHM_FILE =\n\n# The HHC_LOCATION tag can be used to specify the location (absolute path\n# including file name) of the HTML help compiler (hhc.exe). If non-empty,\n# doxygen will try to run the HTML help compiler on the generated index.hhp.\n# The file has to be specified with full path.\n# This tag requires that the tag GENERATE_HTMLHELP is set to YES.\n\nHHC_LOCATION =\n\n# The GENERATE_CHI flag controls if a separate .chi index file is generated\n# (YES) or that it should be included in the main .chm file (NO).\n# The default value is: NO.\n# This tag requires that the tag GENERATE_HTMLHELP is set to YES.\n\nGENERATE_CHI = NO\n\n# The CHM_INDEX_ENCODING is used to encode HtmlHelp index (hhk), content (hhc)\n# and project file content.\n# This tag requires that the tag GENERATE_HTMLHELP is set to YES.\n\nCHM_INDEX_ENCODING =\n\n# The BINARY_TOC flag controls whether a binary table of contents is generated\n# (YES) or a normal table of contents (NO) in the .chm file. Furthermore it\n# enables the Previous and Next buttons.\n# The default value is: NO.\n# This tag requires that the tag GENERATE_HTMLHELP is set to YES.\n\nBINARY_TOC = NO\n\n# The TOC_EXPAND flag can be set to YES to add extra items for group members to\n# the table of contents of the HTML help documentation and to the tree view.\n# The default value is: NO.\n# This tag requires that the tag GENERATE_HTMLHELP is set to YES.\n\nTOC_EXPAND = NO\n\n# If the GENERATE_QHP tag is set to YES and both QHP_NAMESPACE and\n# QHP_VIRTUAL_FOLDER are set, an additional index file will be generated that\n# can be used as input for Qt's qhelpgenerator to generate a Qt Compressed Help\n# (.qch) of the generated HTML documentation.\n# The default value is: NO.\n# This tag requires that the tag GENERATE_HTML is set to YES.\n\nGENERATE_QHP = NO\n\n# If the QHG_LOCATION tag is specified, the QCH_FILE tag can be used to specify\n# the file name of the resulting .qch file. The path specified is relative to\n# the HTML output folder.\n# This tag requires that the tag GENERATE_QHP is set to YES.\n\nQCH_FILE =\n\n# The QHP_NAMESPACE tag specifies the namespace to use when generating Qt Help\n# Project output. For more information please see Qt Help Project / Namespace\n# (see:\n# https://doc.qt.io/archives/qt-4.8/qthelpproject.html#namespace).\n# The default value is: org.doxygen.Project.\n# This tag requires that the tag GENERATE_QHP is set to YES.\n\nQHP_NAMESPACE = org.doxygen.Project\n\n# The QHP_VIRTUAL_FOLDER tag specifies the namespace to use when generating Qt\n# Help Project output. For more information please see Qt Help Project / Virtual\n# Folders (see:\n# https://doc.qt.io/archives/qt-4.8/qthelpproject.html#virtual-folders).\n# The default value is: doc.\n# This tag requires that the tag GENERATE_QHP is set to YES.\n\nQHP_VIRTUAL_FOLDER = doc\n\n# If the QHP_CUST_FILTER_NAME tag is set, it specifies the name of a custom\n# filter to add. For more information please see Qt Help Project / Custom\n# Filters (see:\n# https://doc.qt.io/archives/qt-4.8/qthelpproject.html#custom-filters).\n# This tag requires that the tag GENERATE_QHP is set to YES.\n\nQHP_CUST_FILTER_NAME =\n\n# The QHP_CUST_FILTER_ATTRS tag specifies the list of the attributes of the\n# custom filter to add. For more information please see Qt Help Project / Custom\n# Filters (see:\n# https://doc.qt.io/archives/qt-4.8/qthelpproject.html#custom-filters).\n# This tag requires that the tag GENERATE_QHP is set to YES.\n\nQHP_CUST_FILTER_ATTRS =\n\n# The QHP_SECT_FILTER_ATTRS tag specifies the list of the attributes this\n# project's filter section matches. Qt Help Project / Filter Attributes (see:\n# https://doc.qt.io/archives/qt-4.8/qthelpproject.html#filter-attributes).\n# This tag requires that the tag GENERATE_QHP is set to YES.\n\nQHP_SECT_FILTER_ATTRS =\n\n# The QHG_LOCATION tag can be used to specify the location (absolute path\n# including file name) of Qt's qhelpgenerator. If non-empty doxygen will try to\n# run qhelpgenerator on the generated .qhp file.\n# This tag requires that the tag GENERATE_QHP is set to YES.\n\nQHG_LOCATION =\n\n# If the GENERATE_ECLIPSEHELP tag is set to YES, additional index files will be\n# generated, together with the HTML files, they form an Eclipse help plugin. To\n# install this plugin and make it available under the help contents menu in\n# Eclipse, the contents of the directory containing the HTML and XML files needs\n# to be copied into the plugins directory of eclipse. The name of the directory\n# within the plugins directory should be the same as the ECLIPSE_DOC_ID value.\n# After copying Eclipse needs to be restarted before the help appears.\n# The default value is: NO.\n# This tag requires that the tag GENERATE_HTML is set to YES.\n\nGENERATE_ECLIPSEHELP = NO\n\n# A unique identifier for the Eclipse help plugin. When installing the plugin\n# the directory name containing the HTML and XML files should also have this\n# name. Each documentation set should have its own identifier.\n# The default value is: org.doxygen.Project.\n# This tag requires that the tag GENERATE_ECLIPSEHELP is set to YES.\n\nECLIPSE_DOC_ID = org.doxygen.Project\n\n# If you want full control over the layout of the generated HTML pages it might\n# be necessary to disable the index and replace it with your own. The\n# DISABLE_INDEX tag can be used to turn on/off the condensed index (tabs) at top\n# of each HTML page. A value of NO enables the index and the value YES disables\n# it. Since the tabs in the index contain the same information as the navigation\n# tree, you can set this option to YES if you also set GENERATE_TREEVIEW to YES.\n# The default value is: NO.\n# This tag requires that the tag GENERATE_HTML is set to YES.\n\nDISABLE_INDEX = NO\n\n# The GENERATE_TREEVIEW tag is used to specify whether a tree-like index\n# structure should be generated to display hierarchical information. If the tag\n# value is set to YES, a side panel will be generated containing a tree-like\n# index structure (just like the one that is generated for HTML Help). For this\n# to work a browser that supports JavaScript, DHTML, CSS and frames is required\n# (i.e. any modern browser). Windows users are probably better off using the\n# HTML help feature. Via custom style sheets (see HTML_EXTRA_STYLESHEET) one can\n# further fine-tune the look of the index. As an example, the default style\n# sheet generated by doxygen has an example that shows how to put an image at\n# the root of the tree instead of the PROJECT_NAME. Since the tree basically has\n# the same information as the tab index, you could consider setting\n# DISABLE_INDEX to YES when enabling this option.\n# The default value is: NO.\n# This tag requires that the tag GENERATE_HTML is set to YES.\n\nGENERATE_TREEVIEW = NO\n\n# The ENUM_VALUES_PER_LINE tag can be used to set the number of enum values that\n# doxygen will group on one line in the generated HTML documentation.\n#\n# Note that a value of 0 will completely suppress the enum values from appearing\n# in the overview section.\n# Minimum value: 0, maximum value: 20, default value: 4.\n# This tag requires that the tag GENERATE_HTML is set to YES.\n\nENUM_VALUES_PER_LINE = 4\n\n# If the treeview is enabled (see GENERATE_TREEVIEW) then this tag can be used\n# to set the initial width (in pixels) of the frame in which the tree is shown.\n# Minimum value: 0, maximum value: 1500, default value: 250.\n# This tag requires that the tag GENERATE_HTML is set to YES.\n\nTREEVIEW_WIDTH = 250\n\n# If the EXT_LINKS_IN_WINDOW option is set to YES, doxygen will open links to\n# external symbols imported via tag files in a separate window.\n# The default value is: NO.\n# This tag requires that the tag GENERATE_HTML is set to YES.\n\nEXT_LINKS_IN_WINDOW = NO\n\n# If the HTML_FORMULA_FORMAT option is set to svg, doxygen will use the pdf2svg\n# tool (see https://github.com/dawbarton/pdf2svg) or inkscape (see\n# https://inkscape.org) to generate formulas as SVG images instead of PNGs for\n# the HTML output. These images will generally look nicer at scaled resolutions.\n# Possible values are: png (the default) and svg (looks nicer but requires the\n# pdf2svg or inkscape tool).\n# The default value is: png.\n# This tag requires that the tag GENERATE_HTML is set to YES.\n\nHTML_FORMULA_FORMAT = png\n\n# Use this tag to change the font size of LaTeX formulas included as images in\n# the HTML documentation. When you change the font size after a successful\n# doxygen run you need to manually remove any form_*.png images from the HTML\n# output directory to force them to be regenerated.\n# Minimum value: 8, maximum value: 50, default value: 10.\n# This tag requires that the tag GENERATE_HTML is set to YES.\n\nFORMULA_FONTSIZE = 10\n\n# Use the FORMULA_TRANSPARENT tag to determine whether or not the images\n# generated for formulas are transparent PNGs. Transparent PNGs are not\n# supported properly for IE 6.0, but are supported on all modern browsers.\n#\n# Note that when changing this option you need to delete any form_*.png files in\n# the HTML output directory before the changes have effect.\n# The default value is: YES.\n# This tag requires that the tag GENERATE_HTML is set to YES.\n\nFORMULA_TRANSPARENT = YES\n\n# The FORMULA_MACROFILE can contain LaTeX \\newcommand and \\renewcommand commands\n# to create new LaTeX commands to be used in formulas as building blocks. See\n# the section \"Including formulas\" for details.\n\nFORMULA_MACROFILE =\n\n# Enable the USE_MATHJAX option to render LaTeX formulas using MathJax (see\n# https://www.mathjax.org) which uses client side JavaScript for the rendering\n# instead of using pre-rendered bitmaps. Use this if you do not have LaTeX\n# installed or if you want to formulas look prettier in the HTML output. When\n# enabled you may also need to install MathJax separately and configure the path\n# to it using the MATHJAX_RELPATH option.\n# The default value is: NO.\n# This tag requires that the tag GENERATE_HTML is set to YES.\n\nUSE_MATHJAX = NO\n\n# When MathJax is enabled you can set the default output format to be used for\n# the MathJax output. See the MathJax site (see:\n# http://docs.mathjax.org/en/v2.7-latest/output.html) for more details.\n# Possible values are: HTML-CSS (which is slower, but has the best\n# compatibility), NativeMML (i.e. MathML) and SVG.\n# The default value is: HTML-CSS.\n# This tag requires that the tag USE_MATHJAX is set to YES.\n\nMATHJAX_FORMAT = HTML-CSS\n\n# When MathJax is enabled you need to specify the location relative to the HTML\n# output directory using the MATHJAX_RELPATH option. The destination directory\n# should contain the MathJax.js script. For instance, if the mathjax directory\n# is located at the same level as the HTML output directory, then\n# MATHJAX_RELPATH should be ../mathjax. The default value points to the MathJax\n# Content Delivery Network so you can quickly see the result without installing\n# MathJax. However, it is strongly recommended to install a local copy of\n# MathJax from https://www.mathjax.org before deployment.\n# The default value is: https://cdn.jsdelivr.net/npm/mathjax@2.\n# This tag requires that the tag USE_MATHJAX is set to YES.\n\nMATHJAX_RELPATH = http://www.mathjax.org/mathjax\n\n# The MATHJAX_EXTENSIONS tag can be used to specify one or more MathJax\n# extension names that should be enabled during MathJax rendering. For example\n# MATHJAX_EXTENSIONS = TeX/AMSmath TeX/AMSsymbols\n# This tag requires that the tag USE_MATHJAX is set to YES.\n\nMATHJAX_EXTENSIONS =\n\n# The MATHJAX_CODEFILE tag can be used to specify a file with javascript pieces\n# of code that will be used on startup of the MathJax code. See the MathJax site\n# (see:\n# http://docs.mathjax.org/en/v2.7-latest/output.html) for more details. For an\n# example see the documentation.\n# This tag requires that the tag USE_MATHJAX is set to YES.\n\nMATHJAX_CODEFILE =\n\n# When the SEARCHENGINE tag is enabled doxygen will generate a search box for\n# the HTML output. The underlying search engine uses javascript and DHTML and\n# should work on any modern browser. Note that when using HTML help\n# (GENERATE_HTMLHELP), Qt help (GENERATE_QHP), or docsets (GENERATE_DOCSET)\n# there is already a search function so this one should typically be disabled.\n# For large projects the javascript based search engine can be slow, then\n# enabling SERVER_BASED_SEARCH may provide a better solution. It is possible to\n# search using the keyboard; to jump to the search box use + S\n# (what the is depends on the OS and browser, but it is typically\n# , /