Showing preview only (399K chars total). Download the full file or copy to clipboard to get everything.
Repository: projectdiscovery/subfinder
Branch: dev
Commit: 13e5db750080
Files: 111
Total size: 369.5 KB
Directory structure:
gitextract_djozq64v/
├── .github/
│ ├── ISSUE_TEMPLATE/
│ │ ├── bug_report.md
│ │ ├── config.yml
│ │ ├── feature_request.md
│ │ └── issue-report.md
│ ├── PULL_REQUEST_TEMPLATE.md
│ ├── dependabot.yml
│ ├── release.yml
│ └── workflows/
│ ├── build-test.yml
│ ├── codeql-analysis.yml
│ ├── compat-checks.yaml
│ ├── dep-auto-merge.yml
│ ├── dockerhub-push.yml
│ ├── release-binary.yml
│ └── release-test.yml
├── .gitignore
├── .goreleaser.yml
├── DISCLAIMER.md
├── Dockerfile
├── LICENSE.md
├── Makefile
├── README.md
├── THANKS.md
├── cmd/
│ └── subfinder/
│ └── main.go
├── examples/
│ └── main.go
├── go.mod
├── go.sum
└── pkg/
├── passive/
│ ├── doc.go
│ ├── passive.go
│ ├── sources.go
│ ├── sources_test.go
│ ├── sources_w_auth_test.go
│ └── sources_wo_auth_test.go
├── resolve/
│ ├── client.go
│ ├── doc.go
│ └── resolve.go
├── runner/
│ ├── banners.go
│ ├── config.go
│ ├── doc.go
│ ├── enumerate.go
│ ├── enumerate_test.go
│ ├── initialize.go
│ ├── options.go
│ ├── outputter.go
│ ├── runner.go
│ ├── stats.go
│ ├── util.go
│ └── validate.go
├── subscraping/
│ ├── agent.go
│ ├── doc.go
│ ├── extractor.go
│ ├── sources/
│ │ ├── alienvault/
│ │ │ └── alienvault.go
│ │ ├── anubis/
│ │ │ └── anubis.go
│ │ ├── bevigil/
│ │ │ └── bevigil.go
│ │ ├── bufferover/
│ │ │ └── bufferover.go
│ │ ├── builtwith/
│ │ │ └── builtwith.go
│ │ ├── c99/
│ │ │ └── c99.go
│ │ ├── censys/
│ │ │ ├── censys.go
│ │ │ └── censys_test.go
│ │ ├── certspotter/
│ │ │ └── certspotter.go
│ │ ├── chaos/
│ │ │ └── chaos.go
│ │ ├── chinaz/
│ │ │ └── chinaz.go
│ │ ├── commoncrawl/
│ │ │ └── commoncrawl.go
│ │ ├── crtsh/
│ │ │ └── crtsh.go
│ │ ├── digitalyama/
│ │ │ └── digitalyama.go
│ │ ├── digitorus/
│ │ │ └── digitorus.go
│ │ ├── dnsdb/
│ │ │ └── dnsdb.go
│ │ ├── dnsdumpster/
│ │ │ └── dnsdumpster.go
│ │ ├── dnsrepo/
│ │ │ └── dnsrepo.go
│ │ ├── domainsproject/
│ │ │ └── domainsproject.go
│ │ ├── driftnet/
│ │ │ └── driftnet.go
│ │ ├── facebook/
│ │ │ ├── ctlogs.go
│ │ │ ├── ctlogs_test.go
│ │ │ └── types.go
│ │ ├── fofa/
│ │ │ └── fofa.go
│ │ ├── fullhunt/
│ │ │ └── fullhunt.go
│ │ ├── github/
│ │ │ ├── github.go
│ │ │ └── tokenmanager.go
│ │ ├── gitlab/
│ │ │ └── gitlab.go
│ │ ├── hackertarget/
│ │ │ └── hackertarget.go
│ │ ├── hudsonrock/
│ │ │ └── hudsonrock.go
│ │ ├── intelx/
│ │ │ └── intelx.go
│ │ ├── leakix/
│ │ │ └── leakix.go
│ │ ├── merklemap/
│ │ │ └── merklemap.go
│ │ ├── netlas/
│ │ │ └── netlas.go
│ │ ├── onyphe/
│ │ │ └── onyphe.go
│ │ ├── profundis/
│ │ │ └── profundis.go
│ │ ├── pugrecon/
│ │ │ └── pugrecon.go
│ │ ├── quake/
│ │ │ └── quake.go
│ │ ├── rapiddns/
│ │ │ └── rapiddns.go
│ │ ├── reconcloud/
│ │ │ └── reconcloud.go
│ │ ├── reconeer/
│ │ │ └── reconeer.go
│ │ ├── redhuntlabs/
│ │ │ └── redhuntlabs.go
│ │ ├── riddler/
│ │ │ └── riddler.go
│ │ ├── robtex/
│ │ │ └── robtext.go
│ │ ├── rsecloud/
│ │ │ └── rsecloud.go
│ │ ├── securitytrails/
│ │ │ └── securitytrails.go
│ │ ├── shodan/
│ │ │ └── shodan.go
│ │ ├── sitedossier/
│ │ │ └── sitedossier.go
│ │ ├── thc/
│ │ │ └── thc.go
│ │ ├── threatbook/
│ │ │ └── threatbook.go
│ │ ├── threatcrowd/
│ │ │ └── threatcrowd.go
│ │ ├── threatminer/
│ │ │ └── threatminer.go
│ │ ├── urlscan/
│ │ │ └── urlscan.go
│ │ ├── virustotal/
│ │ │ └── virustotal.go
│ │ ├── waybackarchive/
│ │ │ └── waybackarchive.go
│ │ ├── whoisxmlapi/
│ │ │ └── whoisxmlapi.go
│ │ ├── windvane/
│ │ │ └── windvane.go
│ │ └── zoomeyeapi/
│ │ └── zoomeyeapi.go
│ ├── types.go
│ └── utils.go
└── testutils/
└── integration.go
================================================
FILE CONTENTS
================================================
================================================
FILE: .github/ISSUE_TEMPLATE/bug_report.md
================================================
---
name: Bug report
about: Create a report to help us improve
title: "[Issue] "
labels: ''
assignees: ''
---
**Describe the bug**
A clear and concise description of what the bug is.
**Subfinder version**
Include the version of subfinder you are using, `subfinder -version`
**Complete command you used to reproduce this**
**Screenshots**
Add screenshots of the error for a better context.
================================================
FILE: .github/ISSUE_TEMPLATE/config.yml
================================================
blank_issues_enabled: false
contact_links:
- name: Ask an question / advise on using subfinder
url: https://github.com/projectdiscovery/subfinder/discussions/categories/q-a
about: Ask a question or request support for using subfinder
- name: Share idea / feature to discuss for subfinder
url: https://github.com/projectdiscovery/subfinder/discussions/categories/ideas
about: Share idea / feature to discuss for subfinder
- name: Connect with PD Team (Discord)
url: https://discord.gg/projectdiscovery
about: Connect with PD Team for direct communication
================================================
FILE: .github/ISSUE_TEMPLATE/feature_request.md
================================================
---
name: Feature request
about: Request feature to implement in this project
labels: 'Type: Enhancement'
---
<!--
1. Please make sure to provide a detailed description with all the relevant information that might be required to start working on this feature.
2. In case you are not sure about your request or whether the particular feature is already supported or not, please start a discussion instead.
3. GitHub Discussion: https://github.com/projectdiscovery/subfinder/discussions/categories/ideas
4. Join our discord server at https://discord.gg/projectdiscovery to discuss the idea on the #subfinder channel.
-->
### Please describe your feature request:
<!-- A clear and concise description of feature to implement -->
### Describe the use case of this feature:
<!-- A clear and concise description of the feature request's motivation and the use-cases in which it could be useful. -->
================================================
FILE: .github/ISSUE_TEMPLATE/issue-report.md
================================================
---
name: Issue report
about: Create a report to help us to improve the project
labels: 'Type: Bug'
---
<!--
1. Please search to see if an issue already exists for the bug you encountered.
2. For support requests, FAQs or "How to" questions, please use the GitHub Discussions section instead - https://github.com/projectdiscovery/subfinder/discussions or
3. Join our discord server at https://discord.gg/projectdiscovery and post the question on the #subfinder channel.
-->
<!-- ISSUES MISSING IMPORTANT INFORMATION MAY BE CLOSED WITHOUT INVESTIGATION. -->
### Subfinder version:
<!-- You can find current version of subfinder with "subfinder -version" -->
<!-- We only accept issues that are reproducible on the latest version of subfinder. -->
<!-- You can find the latest version of project at https://github.com/projectdiscovery/subfinder/releases/ -->
### Current Behavior:
<!-- A concise description of what you're experiencing. -->
### Expected Behavior:
<!-- A concise description of what you expected to happen. -->
### Steps To Reproduce:
<!--
Example: steps to reproduce the behavior:
1. Run 'subfinder ..'
2. See error...
-->
### Anything else:
<!-- Links? References? Screnshots? Anything that will give us more context about the issue that you are encountering! -->
================================================
FILE: .github/PULL_REQUEST_TEMPLATE.md
================================================
## Proposed changes
<!-- Describe the overall picture of your modifications to help maintainers understand the pull request. PRs are required to be associated to their related issue tickets or feature request. -->
### Proof
<!-- How has this been tested? Please describe the tests that you ran to verify your changes. -->
## Checklist
<!-- Put an "x" in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code. -->
- [ ] Pull request is created against the [dev](https://github.com/projectdiscovery/subfinder/tree/dev) branch
- [ ] All checks passed (lint, unit/integration/regression tests etc.) with my changes
- [ ] I have added tests that prove my fix is effective or that my feature works
- [ ] I have added necessary documentation (if appropriate)
================================================
FILE: .github/dependabot.yml
================================================
# To get started with Dependabot version updates, you'll need to specify which
# package ecosystems to update and where the package manifests are located.
# Please see the documentation for all configuration options:
# https://help.github.com/github/administering-a-repository/configuration-options-for-dependency-updates
version: 2
updates:
# Maintain dependencies for go modules
- package-ecosystem: "gomod"
directory: "/"
schedule:
interval: "weekly"
target-branch: "dev"
commit-message:
prefix: "chore"
include: "scope"
allow:
- dependency-name: "github.com/projectdiscovery/*"
groups:
modules:
patterns: ["github.com/projectdiscovery/*"]
labels:
- "Type: Maintenance"
# # Maintain dependencies for GitHub Actions
# - package-ecosystem: "github-actions"
# directory: "/"
# schedule:
# interval: "weekly"
# target-branch: "dev"
# commit-message:
# prefix: "chore"
# include: "scope"
# labels:
# - "Type: Maintenance"
#
# # Maintain dependencies for docker
# - package-ecosystem: "docker"
# directory: "/"
# schedule:
# interval: "weekly"
# target-branch: "dev"
# commit-message:
# prefix: "chore"
# include: "scope"
# labels:
# - "Type: Maintenance"
================================================
FILE: .github/release.yml
================================================
changelog:
exclude:
authors:
- dependabot
categories:
- title: 🎉 New Features
labels:
- "Type: Enhancement"
- title: 🐞 Bug Fixes
labels:
- "Type: Bug"
- title: 🔨 Maintenance
labels:
- "Type: Maintenance"
- title: Other Changes
labels:
- "*"
================================================
FILE: .github/workflows/build-test.yml
================================================
name: 🔨 Build Test
on:
pull_request:
paths:
- "**.go"
- "**.mod"
workflow_dispatch:
inputs:
short:
description: "Use -short flag for tests"
required: false
type: boolean
default: false
jobs:
lint:
name: Lint Test
if: "${{ !endsWith(github.actor, '[bot]') }}"
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: projectdiscovery/actions/setup/go@v1
with:
go-version-file: go.mod
- name: Run golangci-lint
uses: golangci/golangci-lint-action@v8
with:
version: latest
args: --timeout 5m
build:
name: Test Builds
needs: [lint]
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-latest, windows-latest, macOS-latest]
steps:
- uses: actions/checkout@v4
- uses: projectdiscovery/actions/setup/go@v1
with:
go-version-file: go.mod
- run: go build ./...
- name: Run tests
env:
BEVIGIL_API_KEY: ${{secrets.BEVIGIL_API_KEY}}
BINARYEDGE_API_KEY: ${{secrets.BINARYEDGE_API_KEY}}
BUFFEROVER_API_KEY: ${{secrets.BUFFEROVER_API_KEY}}
C99_API_KEY: ${{secrets.C99_API_KEY}}
CENSYS_API_KEY: ${{secrets.CENSYS_API_KEY}}
CERTSPOTTER_API_KEY: ${{secrets.CERTSPOTTER_API_KEY}}
CHAOS_API_KEY: ${{secrets.CHAOS_API_KEY}}
CHINAZ_API_KEY: ${{secrets.CHINAZ_API_KEY}}
DNSDB_API_KEY: ${{secrets.DNSDB_API_KEY}}
DNSREPO_API_KEY: ${{secrets.DNSREPO_API_KEY}}
FOFA_API_KEY: ${{secrets.FOFA_API_KEY}}
FULLHUNT_API_KEY: ${{secrets.FULLHUNT_API_KEY}}
GITHUB_API_KEY: ${{secrets.GITHUB_API_KEY}}
INTELX_API_KEY: ${{secrets.INTELX_API_KEY}}
LEAKIX_API_KEY: ${{secrets.LEAKIX_API_KEY}}
QUAKE_API_KEY: ${{secrets.QUAKE_API_KEY}}
ROBTEX_API_KEY: ${{secrets.ROBTEX_API_KEY}}
SECURITYTRAILS_API_KEY: ${{secrets.SECURITYTRAILS_API_KEY}}
SHODAN_API_KEY: ${{secrets.SHODAN_API_KEY}}
THREATBOOK_API_KEY: ${{secrets.THREATBOOK_API_KEY}}
URLSCAN_API_KEY: ${{secrets.URLSCAN_API_KEY}}
VIRUSTOTAL_API_KEY: ${{secrets.VIRUSTOTAL_API_KEY}}
WHOISXMLAPI_API_KEY: ${{secrets.WHOISXMLAPI_API_KEY}}
ZOOMEYEAPI_API_KEY: ${{secrets.ZOOMEYEAPI_API_KEY}}
uses: nick-invision/retry@v2
with:
timeout_seconds: 360
max_attempts: 3
command: go test ./... -v ${{ github.event.inputs.short == 'true' && '-short' || '' }}
- name: Race Condition Tests
run: go build -race ./...
- name: Run Example
run: go run .
working-directory: examples
================================================
FILE: .github/workflows/codeql-analysis.yml
================================================
name: 🚨 CodeQL Analysis
on:
workflow_dispatch:
pull_request:
paths:
- '**.go'
- '**.mod'
jobs:
analyze:
name: Analyze
runs-on: ubuntu-latest
permissions:
actions: read
contents: read
security-events: write
strategy:
fail-fast: false
matrix:
language: [ 'go' ]
# CodeQL supports [ 'cpp', 'csharp', 'go', 'java', 'javascript', 'python' ]
steps:
- name: Checkout repository
uses: actions/checkout@v3
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v2
with:
languages: ${{ matrix.language }}
- name: Autobuild
uses: github/codeql-action/autobuild@v2
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v2
================================================
FILE: .github/workflows/compat-checks.yaml
================================================
name: ♾️ Compatibility Checks
on:
pull_request:
types: [opened, synchronize]
branches:
- dev
jobs:
check:
if: github.actor == 'dependabot[bot]'
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- uses: actions/checkout@v4
- uses: projectdiscovery/actions/setup/go/compat-checks@master
with:
go-version-file: 'go.mod'
================================================
FILE: .github/workflows/dep-auto-merge.yml
================================================
name: 🤖 dep auto merge
on:
pull_request:
branches:
- dev
workflow_dispatch:
permissions:
pull-requests: write
issues: write
repository-projects: write
jobs:
automerge:
runs-on: ubuntu-latest
if: github.actor == 'dependabot[bot]'
steps:
- uses: actions/checkout@v3
with:
token: ${{ secrets.DEPENDABOT_PAT }}
- uses: ahmadnassri/action-dependabot-auto-merge@v2
with:
github-token: ${{ secrets.DEPENDABOT_PAT }}
target: all
================================================
FILE: .github/workflows/dockerhub-push.yml
================================================
name: 🌥 Docker Push
on:
workflow_run:
workflows: ["🎉 Release Binary"]
types:
- completed
workflow_dispatch:
jobs:
docker:
runs-on: ubuntu-latest-16-cores
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Get Github tag
id: meta
run: |
curl --silent "https://api.github.com/repos/projectdiscovery/subfinder/releases/latest" | jq -r .tag_name | xargs -I {} echo TAG={} >> $GITHUB_OUTPUT
- name: Set up QEMU
uses: docker/setup-qemu-action@v2
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
- name: Login to DockerHub
uses: docker/login-action@v2
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_TOKEN }}
- name: Build and push
uses: docker/build-push-action@v4
with:
context: .
platforms: linux/amd64,linux/arm64,linux/arm
push: true
tags: projectdiscovery/subfinder:latest,projectdiscovery/subfinder:${{ steps.meta.outputs.TAG }}
================================================
FILE: .github/workflows/release-binary.yml
================================================
name: 🎉 Release Binary
on:
push:
tags:
- v*
workflow_dispatch:
jobs:
release:
runs-on: ubuntu-latest-16-cores
steps:
- name: "Check out code"
uses: actions/checkout@v3
with:
fetch-depth: 0
- name: "Set up Go"
uses: actions/setup-go@v4
with:
go-version: 1.21.x
- name: "Create release on GitHub"
uses: goreleaser/goreleaser-action@v3
with:
args: "release --clean"
version: latest
env:
GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
SLACK_WEBHOOK: "${{ secrets.RELEASE_SLACK_WEBHOOK }}"
DISCORD_WEBHOOK_ID: "${{ secrets.DISCORD_WEBHOOK_ID }}"
DISCORD_WEBHOOK_TOKEN: "${{ secrets.DISCORD_WEBHOOK_TOKEN }}"
================================================
FILE: .github/workflows/release-test.yml
================================================
name: 🔨 Release Test
on:
pull_request:
paths:
- '**.go'
- '**.mod'
workflow_dispatch:
jobs:
release-test:
runs-on: ubuntu-latest-16-cores
steps:
- name: "Check out code"
uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Set up Go
uses: actions/setup-go@v4
with:
go-version: 1.21.x
- name: release test
uses: goreleaser/goreleaser-action@v4
with:
args: "release --clean --snapshot"
version: latest
================================================
FILE: .gitignore
================================================
.DS_Store
cmd/subfinder/subfinder
# subfinder binary when built with `go build`
v2/cmd/subfinder/subfinder
# subfinder binary when built with `make`
v2/subfinder
vendor/
.idea
.devcontainer
.vscode
dist
/subfinder
================================================
FILE: .goreleaser.yml
================================================
version: 2
before:
hooks:
- go mod tidy
builds:
- env:
- CGO_ENABLED=0
goos:
- windows
- linux
- darwin
goarch:
- amd64
- '386'
- arm
- arm64
ignore:
- goos: darwin
goarch: '386'
- goos: windows
goarch: 'arm'
binary: '{{ .ProjectName }}'
main: cmd/subfinder/main.go
archives:
- formats:
- zip
name_template: '{{ .ProjectName }}_{{ .Version }}_{{ if eq .Os "darwin" }}macOS{{ else }}{{ .Os }}{{ end }}_{{ .Arch }}'
checksum:
algorithm: sha256
announce:
slack:
enabled: true
channel: '#release'
username: GoReleaser
message_template: 'New Release: {{ .ProjectName }} {{.Tag}} is published! Check it out at {{ .ReleaseURL }}'
discord:
enabled: true
message_template: '**New Release: {{ .ProjectName }} {{.Tag}}** is published! Check it out at {{ .ReleaseURL }}'
================================================
FILE: DISCLAIMER.md
================================================
## Disclaimer
Subfinder leverages multiple open APIs, it is developed for individuals to help them for research or internal work. If you wish to incorporate this tool into a commercial offering or purposes, you must agree to the Terms of the leveraged services:
- Bufferover: https://tls.bufferover.run
- CommonCrawl: https://commoncrawl.org/terms-of-use/full
- certspotter: https://sslmate.com/terms
- dnsdumpster: https://hackertarget.com/terms
- Google Transparency: https://policies.google.com/terms
- Alienvault: https://www.alienvault.com/terms/website-terms-of-use07may2018
---
You expressly understand and agree that Subfinder (creators and contributors) shall not be liable for any damages or losses resulting from your use of this tool or third-party products that use it.
Creators aren't in charge of any and have/has no responsibility for any kind of:
- Unlawful or illegal use of the tool.
- Legal or Law infringement (acted in any country, state, municipality, place) by third parties and users.
- Act against ethical and / or human moral, ethic, and peoples and cultures of the world.
- Malicious act, capable of causing damage to third parties, promoted or distributed by third parties or the user through this tool.
### Contact
Please contact at contact@projectdiscovery.io for any questions.
================================================
FILE: Dockerfile
================================================
# Build
FROM golang:1.24-alpine AS build-env
RUN apk add build-base
WORKDIR /app
COPY . /app
RUN go mod download
RUN go build ./cmd/subfinder
# Release
FROM alpine:latest
RUN apk upgrade --no-cache \
&& apk add --no-cache bind-tools ca-certificates
COPY --from=build-env /app/subfinder /usr/local/bin/
ENTRYPOINT ["subfinder"]
================================================
FILE: LICENSE.md
================================================
MIT License
Copyright (c) 2021 ProjectDiscovery, Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
================================================
FILE: Makefile
================================================
# Go parameters
GOCMD=go
GOBUILD=$(GOCMD) build
GOMOD=$(GOCMD) mod
GOTEST=$(GOCMD) test
GOFLAGS := -v
LDFLAGS := -s -w
ifneq ($(shell go env GOOS),darwin)
LDFLAGS := -extldflags "-static"
endif
all: build
build:
$(GOBUILD) $(GOFLAGS) -ldflags '$(LDFLAGS)' -o "subfinder" cmd/subfinder/main.go
test:
$(GOTEST) $(GOFLAGS) ./...
tidy:
$(GOMOD) tidy
================================================
FILE: README.md
================================================
<h1 align="center">
<img src="static/subfinder-logo.png" alt="subfinder" width="200px">
<br>
</h1>
<h4 align="center">Fast passive subdomain enumeration tool.</h4>
<p align="center">
<a href="https://goreportcard.com/report/github.com/projectdiscovery/subfinder/v2"><img src="https://goreportcard.com/badge/github.com/projectdiscovery/subfinder"></a>
<a href="https://github.com/projectdiscovery/subfinder/issues"><img src="https://img.shields.io/badge/contributions-welcome-brightgreen.svg?style=flat"></a>
<a href="https://github.com/projectdiscovery/subfinder/releases"><img src="https://img.shields.io/github/release/projectdiscovery/subfinder"></a>
<a href="https://twitter.com/pdiscoveryio"><img src="https://img.shields.io/twitter/follow/pdiscoveryio.svg?logo=twitter"></a>
<a href="https://discord.gg/projectdiscovery"><img src="https://img.shields.io/discord/695645237418131507.svg?logo=discord"></a>
</p>
<p align="center">
<a href="#features">Features</a> •
<a href="#installation">Install</a> •
<a href="#running-subfinder">Usage</a> •
<a href="#post-installation-instructions">API Setup</a> •
<a href="#subfinder-go-library">Library</a> •
<a href="https://discord.gg/projectdiscovery">Join Discord</a>
</p>
---
`subfinder` is a subdomain discovery tool that returns valid subdomains for websites, using passive online sources. It has a simple, modular architecture and is optimized for speed. `subfinder` is built for
doing one thing only - passive subdomain enumeration, and it does that very well.
We have made it to comply with all the used passive source licenses and usage restrictions. The passive model guarantees speed and stealthiness that can be leveraged by both penetration testers and bug bounty
hunters alike.
# Features
<h1 align="left">
<img src="static/subfinder-run.png" alt="subfinder" width="700px"></a>
<br>
</h1>
- Fast and powerful resolution and wildcard elimination modules
- **Curated** passive sources to maximize results
- Multiple output formats supported (JSON, file, stdout)
- Optimized for speed and **lightweight** on resources
- **STDIN/OUT** support enables easy integration into workflows
# Usage
```sh
subfinder -h
```
This will display help for the tool. Here are all the switches it supports.
```yaml
Usage:
./subfinder [flags]
Flags:
INPUT:
-d, -domain string[] domains to find subdomains for
-dL, -list string file containing list of domains for subdomain discovery
SOURCE:
-s, -sources string[] specific sources to use for discovery (-s crtsh,github). Use -ls to display all available sources.
-recursive use only sources that can handle subdomains recursively (e.g. subdomain.domain.tld vs domain.tld)
-all use all sources for enumeration (slow)
-es, -exclude-sources string[] sources to exclude from enumeration (-es alienvault,zoomeyeapi)
FILTER:
-m, -match string[] subdomain or list of subdomain to match (file or comma separated)
-f, -filter string[] subdomain or list of subdomain to filter (file or comma separated)
RATE-LIMIT:
-rl, -rate-limit int maximum number of http requests to send per second
-rls value maximum number of http requests to send per second for providers in key=value format (-rls "hackertarget=10/s,shodan=15/s")
-t int number of concurrent goroutines for resolving (-active only) (default 10)
UPDATE:
-up, -update update subfinder to latest version
-duc, -disable-update-check disable automatic subfinder update check
OUTPUT:
-o, -output string file to write output to
-oJ, -json write output in JSONL(ines) format
-oD, -output-dir string directory to write output (-dL only)
-cs, -collect-sources include all sources in the output (-json only)
-oI, -ip include host IP in output (-active only)
CONFIGURATION:
-config string flag config file (default "$CONFIG/subfinder/config.yaml")
-pc, -provider-config string provider config file (default "$CONFIG/subfinder/provider-config.yaml")
-r string[] comma separated list of resolvers to use
-rL, -rlist string file containing list of resolvers to use
-nW, -active display active subdomains only
-proxy string http proxy to use with subfinder
-ei, -exclude-ip exclude IPs from the list of domains
DEBUG:
-silent show only subdomains in output
-version show version of subfinder
-v show verbose output
-nc, -no-color disable color in output
-ls, -list-sources list all available sources
OPTIMIZATION:
-timeout int seconds to wait before timing out (default 30)
-max-time int minutes to wait for enumeration results (default 10)
```
## Environment Variables
Subfinder supports environment variables to specify custom paths for configuration files:
- `SUBFINDER_CONFIG` - Path to config.yaml file (overrides default `$CONFIG/subfinder/config.yaml`)
- `SUBFINDER_PROVIDER_CONFIG` - Path to provider-config.yaml file (overrides default `$CONFIG/subfinder/provider-config.yaml`)
# Installation
`subfinder` requires **go1.24** to install successfully. Run the following command to install the latest version:
```sh
go install -v github.com/projectdiscovery/subfinder/v2/cmd/subfinder@latest
```
Learn about more ways to install subfinder here: https://docs.projectdiscovery.io/tools/subfinder/install.
## Post Installation Instructions
`subfinder` can be used right after the installation, however many sources required API keys to work. Learn more here: https://docs.projectdiscovery.io/tools/subfinder/install#post-install-configuration.
## Running Subfinder
Learn about how to run Subfinder here: https://docs.projectdiscovery.io/tools/subfinder/running.
## Subfinder Go library
Subfinder can also be used as library and a minimal examples of using subfinder SDK is available [here](examples/main.go)
</td>
</tr>
</table>
### Resources
- [Recon with Me !!!](https://dhiyaneshgeek.github.io/bug/bounty/2020/02/06/recon-with-me/)
# License
`subfinder` is made with 🖤 by the [projectdiscovery](https://projectdiscovery.io) team. Community contributions have made the project what it is. See
the **[THANKS.md](https://github.com/projectdiscovery/subfinder/blob/main/THANKS.md)** file for more details.
Read the usage disclaimer at [DISCLAIMER.md](https://github.com/projectdiscovery/subfinder/blob/main/DISCLAIMER.md) and [contact us](mailto:contact@projectdiscovery.io) for any API removal.
================================================
FILE: THANKS.md
================================================
### Thanks
Many people have contributed to subfinder making it a wonderful tool either by making a pull request fixing some stuff or giving generous donations to support the further development of this tool. Here, we recognize these persons and thank them.
- All the contributors at [CONTRIBUTORS](https://github.com/projectdiscovery/subfinder/graphs/contributors) who made subfinder what it is.
We'd like to thank some additional amazing people, who contributed a lot in subfinder's journey -
- [@vzamanillo](https://github.com/vzamanillo) - For adding multiple features and overall project improvements.
- [@infosec-au](https://github.com/infosec-au) - Donating to the project.
- [@codingo](https://github.com/codingo) - Initial work on the project, managing it, lot of work!
- [@picatz](https://github.com/picatz) - Improving the structure of the project a lot. New ideas!
================================================
FILE: cmd/subfinder/main.go
================================================
package main
import (
"github.com/projectdiscovery/subfinder/v2/pkg/runner"
// Attempts to increase the OS file descriptors - Fail silently
_ "github.com/projectdiscovery/fdmax/autofdmax"
"github.com/projectdiscovery/gologger"
)
func main() {
// Parse the command line flags and read config files
options := runner.ParseOptions()
newRunner, err := runner.NewRunner(options)
if err != nil {
gologger.Fatal().Msgf("Could not create runner: %s\n", err)
}
err = newRunner.RunEnumeration()
if err != nil {
gologger.Fatal().Msgf("Could not run enumeration: %s\n", err)
}
}
================================================
FILE: examples/main.go
================================================
package main
import (
"bytes"
"context"
"io"
"log"
"github.com/projectdiscovery/subfinder/v2/pkg/runner"
)
func main() {
subfinderOpts := &runner.Options{
Threads: 10, // Thread controls the number of threads to use for active enumerations
Timeout: 30, // Timeout is the seconds to wait for sources to respond
MaxEnumerationTime: 10, // MaxEnumerationTime is the maximum amount of time in mins to wait for enumeration
// ResultCallback: func(s *resolve.HostEntry) {
// callback function executed after each unique subdomain is found
// },
// ProviderConfig: "your_provider_config.yaml",
// and other config related options
}
// disable timestamps in logs / configure logger
log.SetFlags(0)
subfinder, err := runner.NewRunner(subfinderOpts)
if err != nil {
log.Fatalf("failed to create subfinder runner: %v", err)
}
output := &bytes.Buffer{}
var sourceMap map[string]map[string]struct{}
// To run subdomain enumeration on a single domain
if sourceMap, err = subfinder.EnumerateSingleDomainWithCtx(context.Background(), "hackerone.com", []io.Writer{output}); err != nil {
log.Fatalf("failed to enumerate single domain: %v", err)
}
// To run subdomain enumeration on a list of domains from file/reader
// file, err := os.Open("domains.txt")
// if err != nil {
// log.Fatalf("failed to open domains file: %v", err)
// }
// defer file.Close()
// if err = subfinder.EnumerateMultipleDomainsWithCtx(context.Background(), file, []io.Writer{output}); err != nil {
// log.Fatalf("failed to enumerate subdomains from file: %v", err)
// }
// print the output
log.Println(output.String())
// Or use sourceMap to access the results in your application
for subdomain, sources := range sourceMap {
sourcesList := make([]string, 0, len(sources))
for source := range sources {
sourcesList = append(sourcesList, source)
}
log.Printf("%s %s (%d)\n", subdomain, sourcesList, len(sources))
}
}
================================================
FILE: go.mod
================================================
module github.com/projectdiscovery/subfinder/v2
go 1.24.0
toolchain go1.24.1
require (
github.com/corpix/uarand v0.2.0
github.com/hako/durafmt v0.0.0-20210316092057-3a2c319c1acd
github.com/json-iterator/go v1.1.12
github.com/lib/pq v1.10.9
github.com/projectdiscovery/chaos-client v0.5.2
github.com/projectdiscovery/dnsx v1.2.3
github.com/projectdiscovery/fdmax v0.0.4
github.com/projectdiscovery/gologger v1.1.62
github.com/projectdiscovery/ratelimit v0.0.82
github.com/projectdiscovery/retryablehttp-go v1.1.0
github.com/projectdiscovery/utils v0.7.3
github.com/rs/xid v1.5.0
github.com/stretchr/testify v1.11.1
github.com/tomnomnom/linkheader v0.0.0-20180905144013-02ca5825eb80
golang.org/x/exp v0.0.0-20250106191152-7588d65b2ba8
gopkg.in/yaml.v3 v3.0.1
)
require (
aead.dev/minisign v0.2.0 // indirect
github.com/Masterminds/semver/v3 v3.2.1 // indirect
github.com/Mzack9999/gcache v0.0.0-20230410081825-519e28eab057 // indirect
github.com/Mzack9999/go-http-digest-auth-client v0.6.1-0.20220414142836-eb8883508809 // indirect
github.com/STARRY-S/zip v0.2.1 // indirect
github.com/VividCortex/ewma v1.2.0 // indirect
github.com/akrylysov/pogreb v0.10.1 // indirect
github.com/alecthomas/chroma/v2 v2.14.0 // indirect
github.com/andybalholm/brotli v1.1.1 // indirect
github.com/asaskevich/govalidator v0.0.0-20230301143203-a9d515a09cc2 // indirect
github.com/aymanbagabas/go-osc52/v2 v2.0.1 // indirect
github.com/aymerick/douceur v0.2.0 // indirect
github.com/bodgit/plumbing v1.3.0 // indirect
github.com/bodgit/sevenzip v1.6.0 // indirect
github.com/bodgit/windows v1.0.1 // indirect
github.com/charmbracelet/glamour v0.8.0 // indirect
github.com/charmbracelet/lipgloss v0.13.0 // indirect
github.com/charmbracelet/x/ansi v0.3.2 // indirect
github.com/cheggaaa/pb/v3 v3.1.4 // indirect
github.com/cloudflare/circl v1.6.1 // indirect
github.com/dimchansky/utfbom v1.1.1 // indirect
github.com/dlclark/regexp2 v1.11.5 // indirect
github.com/docker/go-units v0.5.0 // indirect
github.com/dsnet/compress v0.0.2-0.20230904184137-39efe44ab707 // indirect
github.com/fatih/color v1.15.0 // indirect
github.com/gaissmai/bart v0.26.0 // indirect
github.com/go-ole/go-ole v1.2.6 // indirect
github.com/golang/snappy v0.0.4 // indirect
github.com/google/go-github/v30 v30.1.0 // indirect
github.com/google/go-querystring v1.1.0 // indirect
github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 // indirect
github.com/google/uuid v1.3.1 // indirect
github.com/gorilla/css v1.0.1 // indirect
github.com/hashicorp/errwrap v1.1.0 // indirect
github.com/hashicorp/go-multierror v1.1.1 // indirect
github.com/hashicorp/golang-lru/v2 v2.0.7 // indirect
github.com/klauspost/compress v1.17.11 // indirect
github.com/klauspost/pgzip v1.2.6 // indirect
github.com/kr/pretty v0.3.1 // indirect
github.com/lucasb-eyer/go-colorful v1.2.0 // indirect
github.com/lufia/plan9stats v0.0.0-20211012122336-39d0f177ccd0 // indirect
github.com/mattn/go-colorable v0.1.13 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/mattn/go-runewidth v0.0.16 // indirect
github.com/mholt/archives v0.1.0 // indirect
github.com/microcosm-cc/bluemonday v1.0.27 // indirect
github.com/minio/selfupdate v0.6.1-0.20230907112617-f11e74f84ca7 // indirect
github.com/muesli/reflow v0.3.0 // indirect
github.com/muesli/termenv v0.15.3-0.20240618155329-98d742f6907a // indirect
github.com/nwaples/rardecode/v2 v2.2.0 // indirect
github.com/pierrec/lz4/v4 v4.1.21 // indirect
github.com/power-devops/perfstat v0.0.0-20210106213030-5aafc221ea8c // indirect
github.com/projectdiscovery/blackrock v0.0.1 // indirect
github.com/projectdiscovery/cdncheck v1.2.13 // indirect
github.com/projectdiscovery/fastdialer v0.4.19 // indirect
github.com/projectdiscovery/hmap v0.0.98 // indirect
github.com/projectdiscovery/machineid v0.0.0-20240226150047-2e2c51e35983 // indirect
github.com/projectdiscovery/networkpolicy v0.1.31 // indirect
github.com/refraction-networking/utls v1.7.1 // indirect
github.com/rivo/uniseg v0.4.7 // indirect
github.com/saintfish/chardet v0.0.0-20230101081208-5e3ef4b5456d // indirect
github.com/shirou/gopsutil/v3 v3.23.7 // indirect
github.com/shoenig/go-m1cpu v0.1.6 // indirect
github.com/sorairolake/lzip-go v0.3.5 // indirect
github.com/syndtr/goleveldb v1.0.0 // indirect
github.com/therootcompany/xz v1.0.1 // indirect
github.com/tidwall/btree v1.6.0 // indirect
github.com/tidwall/buntdb v1.3.0 // indirect
github.com/tidwall/gjson v1.18.0 // indirect
github.com/tidwall/grect v0.1.4 // indirect
github.com/tidwall/match v1.1.1 // indirect
github.com/tidwall/pretty v1.2.1 // indirect
github.com/tidwall/rtred v0.1.2 // indirect
github.com/tidwall/tinyqueue v0.1.1 // indirect
github.com/tklauser/go-sysconf v0.3.12 // indirect
github.com/tklauser/numcpus v0.6.1 // indirect
github.com/ulikunitz/xz v0.5.15 // indirect
github.com/weppos/publicsuffix-go v0.40.3-0.20250408071509-6074bbe7fd39 // indirect
github.com/yuin/goldmark v1.7.4 // indirect
github.com/yuin/goldmark-emoji v1.0.3 // indirect
github.com/yusufpapurcu/wmi v1.2.4 // indirect
github.com/zcalusic/sysinfo v1.0.2 // indirect
github.com/zmap/rc2 v0.0.0-20190804163417-abaa70531248 // indirect
github.com/zmap/zcrypto v0.0.0-20230422215203-9a665e1e9968 // indirect
go.etcd.io/bbolt v1.3.7 // indirect
go.uber.org/multierr v1.11.0 // indirect
go4.org v0.0.0-20230225012048-214862532bf5 // indirect
golang.org/x/crypto v0.45.0 // indirect
golang.org/x/mod v0.29.0 // indirect
golang.org/x/oauth2 v0.27.0 // indirect
golang.org/x/sync v0.18.0 // indirect
golang.org/x/term v0.37.0 // indirect
golang.org/x/text v0.31.0 // indirect
golang.org/x/time v0.5.0 // indirect
golang.org/x/tools v0.38.0 // indirect
gopkg.in/djherbis/times.v1 v1.3.0 // indirect
)
require (
github.com/cnf/structhash v0.0.0-20201127153200-e1b16c1ebc08 // indirect
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
github.com/logrusorgru/aurora v2.0.3+incompatible // indirect
github.com/miekg/dns v1.1.62 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
github.com/projectdiscovery/goflags v0.1.74
github.com/projectdiscovery/retryabledns v1.0.111 // indirect
golang.org/x/net v0.47.0 // indirect
golang.org/x/sys v0.38.0 // indirect
)
================================================
FILE: go.sum
================================================
aead.dev/minisign v0.2.0 h1:kAWrq/hBRu4AARY6AlciO83xhNnW9UaC8YipS2uhLPk=
aead.dev/minisign v0.2.0/go.mod h1:zdq6LdSd9TbuSxchxwhpA9zEb9YXcVGoE8JakuiGaIQ=
cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
cloud.google.com/go v0.34.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
cloud.google.com/go v0.38.0/go.mod h1:990N+gfupTy94rShfmMCWGDn0LpTmnzTp2qbd1dvSRU=
cloud.google.com/go v0.44.1/go.mod h1:iSa0KzasP4Uvy3f1mN/7PiObzGgflwredwwASm/v6AU=
cloud.google.com/go v0.44.2/go.mod h1:60680Gw3Yr4ikxnPRS/oxxkBccT6SA1yMk63TGekxKY=
cloud.google.com/go v0.45.1/go.mod h1:RpBamKRgapWJb87xiFSdk4g1CME7QZg3uwTez+TSTjc=
cloud.google.com/go v0.46.3/go.mod h1:a6bKKbmY7er1mI7TEI4lsAkts/mkhTSZK8w33B4RAg0=
cloud.google.com/go v0.50.0/go.mod h1:r9sluTvynVuxRIOHXQEHMFffphuXHOMZMycpNR5e6To=
cloud.google.com/go v0.53.0/go.mod h1:fp/UouUEsRkN6ryDKNW/Upv/JBKnv6WDthjR6+vze6M=
cloud.google.com/go/bigquery v1.0.1/go.mod h1:i/xbL2UlR5RvWAURpBYZTtm/cXjCha9lbfbpx4poX+o=
cloud.google.com/go/bigquery v1.3.0/go.mod h1:PjpwJnslEMmckchkHFfq+HTD2DmtT67aNFKH1/VBDHE=
cloud.google.com/go/compute/metadata v0.2.0/go.mod h1:zFmK7XCadkQkj6TtorcaGlCW1hT1fIilQDwofLpJ20k=
cloud.google.com/go/datastore v1.0.0/go.mod h1:LXYbyblFSglQ5pkeyhO+Qmw7ukd3C+pD7TKLgZqpHYE=
cloud.google.com/go/pubsub v1.0.1/go.mod h1:R0Gpsv3s54REJCy4fxDixWD93lHJMoZTyQ2kNxGRt3I=
cloud.google.com/go/pubsub v1.1.0/go.mod h1:EwwdRX2sKPjnvnqCa270oGRyludottCI76h+R3AArQw=
cloud.google.com/go/storage v1.0.0/go.mod h1:IhtSnM/ZTZV8YYJWCY8RULGVqBDmpoyjwiyrjsg+URw=
cloud.google.com/go/storage v1.5.0/go.mod h1:tpKbwo567HUNpVclU5sGELwQWBDZ8gh0ZeosJ0Rtdos=
dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU=
github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
github.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802/go.mod h1:IVnqGOEym/WlBOVXweHU+Q+/VP0lqqI8lqeDx9IjBqo=
github.com/Masterminds/semver/v3 v3.2.1 h1:RN9w6+7QoMeJVGyfmbcgs28Br8cvmnucEXnY0rYXWg0=
github.com/Masterminds/semver/v3 v3.2.1/go.mod h1:qvl/7zhW3nngYb5+80sSMF+FG2BjYrf8m9wsX0PNOMQ=
github.com/Mzack9999/gcache v0.0.0-20230410081825-519e28eab057 h1:KFac3SiGbId8ub47e7kd2PLZeACxc1LkiiNoDOFRClE=
github.com/Mzack9999/gcache v0.0.0-20230410081825-519e28eab057/go.mod h1:iLB2pivrPICvLOuROKmlqURtFIEsoJZaMidQfCG1+D4=
github.com/Mzack9999/go-http-digest-auth-client v0.6.1-0.20220414142836-eb8883508809 h1:ZbFL+BDfBqegi+/Ssh7im5+aQfBRx6it+kHnC7jaDU8=
github.com/Mzack9999/go-http-digest-auth-client v0.6.1-0.20220414142836-eb8883508809/go.mod h1:upgc3Zs45jBDnBT4tVRgRcgm26ABpaP7MoTSdgysca4=
github.com/STARRY-S/zip v0.2.1 h1:pWBd4tuSGm3wtpoqRZZ2EAwOmcHK6XFf7bU9qcJXyFg=
github.com/STARRY-S/zip v0.2.1/go.mod h1:xNvshLODWtC4EJ702g7cTYn13G53o1+X9BWnPFpcWV4=
github.com/VividCortex/ewma v1.2.0 h1:f58SaIzcDXrSy3kWaHNvuJgJ3Nmz59Zji6XoJR/q1ow=
github.com/VividCortex/ewma v1.2.0/go.mod h1:nz4BbCtbLyFDeC9SUHbtcT5644juEuWfUAUnGx7j5l4=
github.com/akrylysov/pogreb v0.10.1 h1:FqlR8VR7uCbJdfUob916tPM+idpKgeESDXOA1K0DK4w=
github.com/akrylysov/pogreb v0.10.1/go.mod h1:pNs6QmpQ1UlTJKDezuRWmaqkgUE2TuU0YTWyqJZ7+lI=
github.com/alecthomas/assert/v2 v2.7.0 h1:QtqSACNS3tF7oasA8CU6A6sXZSBDqnm7RfpLl9bZqbE=
github.com/alecthomas/assert/v2 v2.7.0/go.mod h1:Bze95FyfUr7x34QZrjL+XP+0qgp/zg8yS+TtBj1WA3k=
github.com/alecthomas/chroma/v2 v2.14.0 h1:R3+wzpnUArGcQz7fCETQBzO5n9IMNi13iIs46aU4V9E=
github.com/alecthomas/chroma/v2 v2.14.0/go.mod h1:QolEbTfmUHIMVpBqxeDnNBj2uoeI4EbYP4i6n68SG4I=
github.com/alecthomas/repr v0.4.0 h1:GhI2A8MACjfegCPVq9f1FLvIBS+DrQ2KQBFZP1iFzXc=
github.com/alecthomas/repr v0.4.0/go.mod h1:Fr0507jx4eOXV7AlPV6AVZLYrLIuIeSOWtW57eE/O/4=
github.com/andybalholm/brotli v1.1.1 h1:PR2pgnyFznKEugtsUo0xLdDop5SKXd5Qf5ysW+7XdTA=
github.com/andybalholm/brotli v1.1.1/go.mod h1:05ib4cKhjx3OQYUY22hTVd34Bc8upXjOLL2rKwwZBoA=
github.com/asaskevich/govalidator v0.0.0-20230301143203-a9d515a09cc2 h1:DklsrG3dyBCFEj5IhUbnKptjxatkF07cF2ak3yi77so=
github.com/asaskevich/govalidator v0.0.0-20230301143203-a9d515a09cc2/go.mod h1:WaHUgvxTVq04UNunO+XhnAqY/wQc+bxr74GqbsZ/Jqw=
github.com/aymanbagabas/go-osc52/v2 v2.0.1 h1:HwpRHbFMcZLEVr42D4p7XBqjyuxQH5SMiErDT4WkJ2k=
github.com/aymanbagabas/go-osc52/v2 v2.0.1/go.mod h1:uYgXzlJ7ZpABp8OJ+exZzJJhRNQ2ASbcXHWsFqH8hp8=
github.com/aymanbagabas/go-udiff v0.2.0 h1:TK0fH4MteXUDspT88n8CKzvK0X9O2xu9yQjWpi6yML8=
github.com/aymanbagabas/go-udiff v0.2.0/go.mod h1:RE4Ex0qsGkTAJoQdQQCA0uG+nAzJO/pI/QwceO5fgrA=
github.com/aymerick/douceur v0.2.0 h1:Mv+mAeH1Q+n9Fr+oyamOlAkUNPWPlA8PPGR0QAaYuPk=
github.com/aymerick/douceur v0.2.0/go.mod h1:wlT5vV2O3h55X9m7iVYN0TBM0NH/MmbLnd30/FjWUq4=
github.com/bits-and-blooms/bitset v1.13.0 h1:bAQ9OPNFYbGHV6Nez0tmNI0RiEu7/hxlYJRUA0wFAVE=
github.com/bits-and-blooms/bitset v1.13.0/go.mod h1:7hO7Gc7Pp1vODcmWvKMRA9BNmbv6a/7QIWpPxHddWR8=
github.com/bits-and-blooms/bloom/v3 v3.5.0 h1:AKDvi1V3xJCmSR6QhcBfHbCN4Vf8FfxeWkMNQfmAGhY=
github.com/bits-and-blooms/bloom/v3 v3.5.0/go.mod h1:Y8vrn7nk1tPIlmLtW2ZPV+W7StdVMor6bC1xgpjMZFs=
github.com/bodgit/plumbing v1.3.0 h1:pf9Itz1JOQgn7vEOE7v7nlEfBykYqvUYioC61TwWCFU=
github.com/bodgit/plumbing v1.3.0/go.mod h1:JOTb4XiRu5xfnmdnDJo6GmSbSbtSyufrsyZFByMtKEs=
github.com/bodgit/sevenzip v1.6.0 h1:a4R0Wu6/P1o1pP/3VV++aEOcyeBxeO/xE2Y9NSTrr6A=
github.com/bodgit/sevenzip v1.6.0/go.mod h1:zOBh9nJUof7tcrlqJFv1koWRrhz3LbDbUNngkuZxLMc=
github.com/bodgit/windows v1.0.1 h1:tF7K6KOluPYygXa3Z2594zxlkbKPAOvqr97etrGNIz4=
github.com/bodgit/windows v1.0.1/go.mod h1:a6JLwrB4KrTR5hBpp8FI9/9W9jJfeQ2h4XDXU74ZCdM=
github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=
github.com/charmbracelet/glamour v0.8.0 h1:tPrjL3aRcQbn++7t18wOpgLyl8wrOHUEDS7IZ68QtZs=
github.com/charmbracelet/glamour v0.8.0/go.mod h1:ViRgmKkf3u5S7uakt2czJ272WSg2ZenlYEZXT2x7Bjw=
github.com/charmbracelet/lipgloss v0.13.0 h1:4X3PPeoWEDCMvzDvGmTajSyYPcZM4+y8sCA/SsA3cjw=
github.com/charmbracelet/lipgloss v0.13.0/go.mod h1:nw4zy0SBX/F/eAO1cWdcvy6qnkDUxr8Lw7dvFrAIbbY=
github.com/charmbracelet/x/ansi v0.3.2 h1:wsEwgAN+C9U06l9dCVMX0/L3x7ptvY1qmjMwyfE6USY=
github.com/charmbracelet/x/ansi v0.3.2/go.mod h1:dk73KoMTT5AX5BsX0KrqhsTqAnhZZoCBjs7dGWp4Ktw=
github.com/charmbracelet/x/exp/golden v0.0.0-20240806155701-69247e0abc2a h1:G99klV19u0QnhiizODirwVksQB91TJKV/UaTnACcG30=
github.com/charmbracelet/x/exp/golden v0.0.0-20240806155701-69247e0abc2a/go.mod h1:wDlXFlCrmJ8J+swcL/MnGUuYnqgQdW9rhSD61oNMb6U=
github.com/cheggaaa/pb/v3 v3.1.4 h1:DN8j4TVVdKu3WxVwcRKu0sG00IIU6FewoABZzXbRQeo=
github.com/cheggaaa/pb/v3 v3.1.4/go.mod h1:6wVjILNBaXMs8c21qRiaUM8BR82erfgau1DQ4iUXmSA=
github.com/chzyer/logex v1.1.10/go.mod h1:+Ywpsq7O8HXn0nuIou7OrIPyXbp3wmkHB+jjWRnGsAI=
github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI=
github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU=
github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
github.com/cloudflare/circl v1.6.1 h1:zqIqSPIndyBh1bjLVVDHMPpVKqp8Su/V+6MeDzzQBQ0=
github.com/cloudflare/circl v1.6.1/go.mod h1:uddAzsPgqdMAYatqJ0lsjX1oECcQLIlRpzZh3pJrofs=
github.com/cnf/structhash v0.0.0-20201127153200-e1b16c1ebc08 h1:ox2F0PSMlrAAiAdknSRMDrAr8mfxPCfSZolH+/qQnyQ=
github.com/cnf/structhash v0.0.0-20201127153200-e1b16c1ebc08/go.mod h1:pCxVEbcm3AMg7ejXyorUXi6HQCzOIBf7zEDVPtw0/U4=
github.com/corpix/uarand v0.2.0 h1:U98xXwud/AVuCpkpgfPF7J5TQgr7R5tqT8VZP5KWbzE=
github.com/corpix/uarand v0.2.0/go.mod h1:/3Z1QIqWkDIhf6XWn/08/uMHoQ8JUoTIKc2iPchBOmM=
github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1VwoXQT9A3Wy9MM3WgvqSxFWenqJduM=
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/dimchansky/utfbom v1.1.1 h1:vV6w1AhK4VMnhBno/TPVCoK9U/LP0PkLCS9tbxHdi/U=
github.com/dimchansky/utfbom v1.1.1/go.mod h1:SxdoEBH5qIqFocHMyGOXVAybYJdr71b1Q/j0mACtrfE=
github.com/dlclark/regexp2 v1.11.5 h1:Q/sSnsKerHeCkc/jSTNq1oCm7KiVgUMZRDUoRu0JQZQ=
github.com/dlclark/regexp2 v1.11.5/go.mod h1:DHkYz0B9wPfa6wondMfaivmHpzrQ3v9q8cnmRbL6yW8=
github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4=
github.com/docker/go-units v0.5.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk=
github.com/dsnet/compress v0.0.2-0.20230904184137-39efe44ab707 h1:2tV76y6Q9BB+NEBasnqvs7e49aEBFI8ejC89PSnWH+4=
github.com/dsnet/compress v0.0.2-0.20230904184137-39efe44ab707/go.mod h1:qssHWj60/X5sZFNxpG4HBPDHVqxNm4DfnCKgrbZOT+s=
github.com/dsnet/golib v0.0.0-20171103203638-1ea166775780/go.mod h1:Lj+Z9rebOhdfkVLjJ8T6VcRQv3SXugXy999NBtR9aFY=
github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c=
github.com/fatih/color v1.15.0 h1:kOqh6YHBtK8aywxGerMG2Eq3H6Qgoqeo13Bk2Mv/nBs=
github.com/fatih/color v1.15.0/go.mod h1:0h5ZqXfHYED7Bhv2ZJamyIOUej9KtShiJESRwBDUSsw=
github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=
github.com/fsnotify/fsnotify v1.6.0 h1:n+5WquG0fcWoWp6xPWfHdbskMCQaFnG6PfBrh1Ky4HY=
github.com/fsnotify/fsnotify v1.6.0/go.mod h1:sl3t1tCWJFWoRz9R8WJCbQihKKwmorjAbSClcnxKAGw=
github.com/gaissmai/bart v0.26.0 h1:xOZ57E9hJLBiQaSyeZa9wgWhGuzfGACgqp4BE77OkO0=
github.com/gaissmai/bart v0.26.0/go.mod h1:GREWQfTLRWz/c5FTOsIw+KkscuFkIV5t8Rp7Nd1Td5c=
github.com/go-gl/glfw v0.0.0-20190409004039-e6da0acd62b1/go.mod h1:vR7hzQXu2zJy9AVAgeJqvqgH9Q5CA+iKCZ2gyEVpxRU=
github.com/go-gl/glfw/v3.3/glfw v0.0.0-20191125211704-12ad95a8df72/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=
github.com/go-ole/go-ole v1.2.6 h1:/Fpf6oFPoeFik9ty7siob0G6Ke8QvQEuVcuChpwXzpY=
github.com/go-ole/go-ole v1.2.6/go.mod h1:pprOEPIfldk/42T2oK7lQ4v4JSDwmV0As9GaiUsvbm0=
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
github.com/golang/groupcache v0.0.0-20190702054246-869f871628b6/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/groupcache v0.0.0-20191227052852-215e87163ea7/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
github.com/golang/mock v1.2.0/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
github.com/golang/mock v1.3.1/go.mod h1:sBzyDLLjw3U8JLTeZvSv8jJB+tU5PVekmnlKIyFUx0Y=
github.com/golang/mock v1.4.0/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=
github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.3/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw=
github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk=
github.com/golang/protobuf v1.5.2/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY=
github.com/golang/snappy v0.0.0-20180518054509-2e65f85255db/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=
github.com/golang/snappy v0.0.4 h1:yAGX7huGHXlcLOEtBnF4w7FQwA26wojNCwOYAEhLjQM=
github.com/golang/snappy v0.0.4/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=
github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
github.com/google/btree v1.0.0/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.2/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.6/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.8/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/go-cmp v0.5.9/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/go-github/v30 v30.1.0 h1:VLDx+UolQICEOKu2m4uAoMti1SxuEBAl7RSEG16L+Oo=
github.com/google/go-github/v30 v30.1.0/go.mod h1:n8jBpHl45a/rlBUtRJMOG4GhNADUQFEufcolZ95JfU8=
github.com/google/go-github/v50 v50.1.0/go.mod h1:Ev4Tre8QoKiolvbpOSG3FIi4Mlon3S2Nt9W5JYqKiwA=
github.com/google/go-querystring v1.0.0/go.mod h1:odCYkC5MyYFN7vkCjXpyrEuKhc/BUO6wN/zVPAxq5ck=
github.com/google/go-querystring v1.1.0 h1:AnCroh3fv4ZBgVIf1Iwtovgjaw/GiKJo8M8yD/fhyJ8=
github.com/google/go-querystring v1.1.0/go.mod h1:Kcdr2DB4koayq7X8pmAG4sNG59So17icRSOU623lUBU=
github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
github.com/google/martian v2.1.0+incompatible/go.mod h1:9I4somxYTbIHy5NJKHRl3wXiIaQGbYVAs8BPL6v8lEs=
github.com/google/pprof v0.0.0-20181206194817-3ea8567a2e57/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc=
github.com/google/pprof v0.0.0-20190515194954-54271f7e092f/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc=
github.com/google/pprof v0.0.0-20200212024743-f11f1df84d12/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
github.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI=
github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 h1:El6M4kTTCOh6aBiKaUGG7oYTSPP8MxqL4YI3kZKwcP4=
github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510/go.mod h1:pupxD2MaaD3pAXIBCelhxNneeOaAeabZDe5s4K6zSpQ=
github.com/google/uuid v1.3.1 h1:KjJaJ9iWZ3jOFZIf1Lqf4laDRCasjl0BCmnEGxkdLb4=
github.com/google/uuid v1.3.1/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/googleapis/gax-go/v2 v2.0.4/go.mod h1:0Wqv26UfaUD9n4G6kQubkQ+KchISgw+vpHVxEJEs9eg=
github.com/googleapis/gax-go/v2 v2.0.5/go.mod h1:DWXyrwAJ9X0FpwwEdw+IPEYBICEFu5mhpdKc/us6bOk=
github.com/gorilla/css v1.0.1 h1:ntNaBIghp6JmvWnxbZKANoLyuXTPZ4cAMlo6RyhlbO8=
github.com/gorilla/css v1.0.1/go.mod h1:BvnYkspnSzMmwRK+b8/xgNPLiIuNZr6vbZBTPQ2A3b0=
github.com/hako/durafmt v0.0.0-20210316092057-3a2c319c1acd h1:FsX+T6wA8spPe4c1K9vi7T0LvNCO1TTqiL8u7Wok2hw=
github.com/hako/durafmt v0.0.0-20210316092057-3a2c319c1acd/go.mod h1:VzxiSdG6j1pi7rwGm/xYI5RbtpBgM8sARDXlvEvxlu0=
github.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4=
github.com/hashicorp/errwrap v1.1.0 h1:OxrOeh75EUXMY8TBjag2fzXGZ40LB6IKw45YeGUDY2I=
github.com/hashicorp/errwrap v1.1.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4=
github.com/hashicorp/go-multierror v1.1.1 h1:H5DkEtf6CXdFp0N0Em5UCwQpXMWke8IA0+lD48awMYo=
github.com/hashicorp/go-multierror v1.1.1/go.mod h1:iw975J/qwKPdAO1clOe2L8331t/9/fmwbPZ6JB6eMoM=
github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
github.com/hashicorp/golang-lru v0.5.1/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k=
github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM=
github.com/hexops/gotextdiff v1.0.3 h1:gitA9+qJrrTCsiCl7+kh75nPqQt1cx4ZkudSTLoUqJM=
github.com/hexops/gotextdiff v1.0.3/go.mod h1:pSWU5MAI3yDq+fZBTazCSJysOMbxWL1BSow5/V2vxeg=
github.com/hpcloud/tail v1.0.0/go.mod h1:ab1qPbhIpdTxEkNHXyeSf5vhxWSCs/tWer42PpOxQnU=
github.com/ianlancetaylor/demangle v0.0.0-20181102032728-5e5cf60278f6/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc=
github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM=
github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo=
github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU=
github.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/XSXhF0NWZEnDohbsk=
github.com/julienschmidt/httprouter v1.3.0 h1:U0609e9tgbseu3rBINet9P48AI/D3oJs4dN7jwJOQ1U=
github.com/julienschmidt/httprouter v1.3.0/go.mod h1:JR6WtHb+2LUe8TCKY3cZOxFyyO8IZAc4RVcycCCAKdM=
github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
github.com/klauspost/compress v1.4.1/go.mod h1:RyIbtBH6LamlWaDj8nUwkbUhJ87Yi3uG0guNDohfE1A=
github.com/klauspost/compress v1.17.11 h1:In6xLpyWOi1+C7tXUUWv2ot1QvBjxevKAaI6IXrJmUc=
github.com/klauspost/compress v1.17.11/go.mod h1:pMDklpSncoRMuLFrf1W9Ss9KT+0rH90U12bZKk7uwG0=
github.com/klauspost/cpuid v1.2.0/go.mod h1:Pj4uuM528wm8OyEC2QMXAi2YiTZ96dNQPGgoMS4s3ek=
github.com/klauspost/pgzip v1.2.6 h1:8RXeL5crjEUFnR2/Sn6GJNWtSQ3Dk8pq4CL3jvdDyjU=
github.com/klauspost/pgzip v1.2.6/go.mod h1:Ch1tH69qFZu15pkjo5kYi6mth2Zzwzt50oCQKQE9RUs=
github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/lib/pq v1.10.9 h1:YXG7RB+JIjhP29X+OtkiDnYaXQwpS4JEWq7dtCCRUEw=
github.com/lib/pq v1.10.9/go.mod h1:AlVN5x4E4T544tWzH6hKfbfQvm3HdbOxrmggDNAPY9o=
github.com/logrusorgru/aurora v2.0.3+incompatible h1:tOpm7WcpBTn4fjmVfgpQq0EfczGlG91VSDkswnjF5A8=
github.com/logrusorgru/aurora v2.0.3+incompatible/go.mod h1:7rIyQOR62GCctdiQpZ/zOJlFyk6y+94wXzv6RNZgaR4=
github.com/lucasb-eyer/go-colorful v1.2.0 h1:1nnpGOrhyZZuNyfu1QjKiUICQ74+3FNCN69Aj6K7nkY=
github.com/lucasb-eyer/go-colorful v1.2.0/go.mod h1:R4dSotOR9KMtayYi1e77YzuveK+i7ruzyGqttikkLy0=
github.com/lufia/plan9stats v0.0.0-20211012122336-39d0f177ccd0 h1:6E+4a0GO5zZEnZ81pIr0yLvtUWk2if982qA3F3QD6H4=
github.com/lufia/plan9stats v0.0.0-20211012122336-39d0f177ccd0/go.mod h1:zJYVVT2jmtg6P3p1VtQj7WsuWi/y4VnjVBn7F8KPB3I=
github.com/mattn/go-colorable v0.1.13 h1:fFA4WZxdEF4tXPZVKMLwD8oUnCTTo08duU7wxecdEvA=
github.com/mattn/go-colorable v0.1.13/go.mod h1:7S9/ev0klgBDR4GtXTXX8a3vIGJpMovkB8vQcUbaXHg=
github.com/mattn/go-isatty v0.0.16/go.mod h1:kYGgaQfpe5nmfYZH+SKPsOc2e4SrIfOl2e/yFXSvRLM=
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/mattn/go-runewidth v0.0.12/go.mod h1:RAqKPSqVFrSLVXbA8x7dzmKdmGzieGRCM46jaSJTDAk=
github.com/mattn/go-runewidth v0.0.16 h1:E5ScNMtiwvlvB5paMFdw9p4kSQzbXFikJ5SQO6TULQc=
github.com/mattn/go-runewidth v0.0.16/go.mod h1:Jdepj2loyihRzMpdS35Xk/zdY8IAYHsh153qUoGf23w=
github.com/mholt/archives v0.1.0 h1:FacgJyrjiuyomTuNA92X5GyRBRZjE43Y/lrzKIlF35Q=
github.com/mholt/archives v0.1.0/go.mod h1:j/Ire/jm42GN7h90F5kzj6hf6ZFzEH66de+hmjEKu+I=
github.com/microcosm-cc/bluemonday v1.0.27 h1:MpEUotklkwCSLeH+Qdx1VJgNqLlpY2KXwXFM08ygZfk=
github.com/microcosm-cc/bluemonday v1.0.27/go.mod h1:jFi9vgW+H7c3V0lb6nR74Ib/DIB5OBs92Dimizgw2cA=
github.com/miekg/dns v1.1.62 h1:cN8OuEF1/x5Rq6Np+h1epln8OiyPWV+lROx9LxcGgIQ=
github.com/miekg/dns v1.1.62/go.mod h1:mvDlcItzm+br7MToIKqkglaGhlFMHJ9DTNNWONWXbNQ=
github.com/minio/selfupdate v0.6.1-0.20230907112617-f11e74f84ca7 h1:yRZGarbxsRytL6EGgbqK2mCY+Lk5MWKQYKJT2gEglhc=
github.com/minio/selfupdate v0.6.1-0.20230907112617-f11e74f84ca7/go.mod h1:bO02GTIPCMQFTEvE5h4DjYB58bCoZ35XLeBf0buTDdM=
github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg=
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
github.com/modern-go/reflect2 v1.0.2 h1:xBagoLtFs94CBntxluKeaWgTMpvLxC4ur3nMaC9Gz0M=
github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk=
github.com/mreiferson/go-httpclient v0.0.0-20160630210159-31f0106b4474/go.mod h1:OQA4XLvDbMgS8P0CevmM4m9Q3Jq4phKUzcocxuGJ5m8=
github.com/mreiferson/go-httpclient v0.0.0-20201222173833-5e475fde3a4d/go.mod h1:OQA4XLvDbMgS8P0CevmM4m9Q3Jq4phKUzcocxuGJ5m8=
github.com/muesli/reflow v0.3.0 h1:IFsN6K9NfGtjeggFP+68I4chLZV2yIKsXJFNZ+eWh6s=
github.com/muesli/reflow v0.3.0/go.mod h1:pbwTDkVPibjO2kyvBQRBxTWEEGDGq0FlB1BIKtnHY/8=
github.com/muesli/termenv v0.15.3-0.20240618155329-98d742f6907a h1:2MaM6YC3mGu54x+RKAA6JiFFHlHDY1UbkxqppT7wYOg=
github.com/muesli/termenv v0.15.3-0.20240618155329-98d742f6907a/go.mod h1:hxSnBBYLK21Vtq/PHd0S2FYCxBXzBua8ov5s1RobyRQ=
github.com/nwaples/rardecode/v2 v2.2.0 h1:4ufPGHiNe1rYJxYfehALLjup4Ls3ck42CWwjKiOqu0A=
github.com/nwaples/rardecode/v2 v2.2.0/go.mod h1:7uz379lSxPe6j9nvzxUZ+n7mnJNgjsRNb6IbvGVHRmw=
github.com/nxadm/tail v1.4.11 h1:8feyoE3OzPrcshW5/MJ4sGESc5cqmGkGCWlco4l0bqY=
github.com/nxadm/tail v1.4.11/go.mod h1:OTaG3NK980DZzxbRq6lEuzgU+mug70nY11sMd4JXXHc=
github.com/onsi/ginkgo v1.6.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
github.com/onsi/ginkgo v1.7.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
github.com/onsi/ginkgo v1.16.4 h1:29JGrr5oVBm5ulCWet69zQkzWipVXIol6ygQUe/EzNc=
github.com/onsi/ginkgo v1.16.4/go.mod h1:dX+/inL/fNMqNlz0e9LfyB9TswhZpCVdJM/Z6Vvnwo0=
github.com/onsi/gomega v1.4.3/go.mod h1:ex+gbHU/CVuBBDIJjb2X0qEXbFg53c61hWP/1CpauHY=
github.com/onsi/gomega v1.27.6 h1:ENqfyGeS5AX/rlXDd/ETokDz93u0YufY1Pgxuy/PvWE=
github.com/onsi/gomega v1.27.6/go.mod h1:PIQNjfQwkP3aQAH7lf7j87O/5FiNr+ZR8+ipb+qQlhg=
github.com/op/go-logging v0.0.0-20160315200505-970db520ece7/go.mod h1:HzydrMdWErDVzsI23lYNej1Htcns9BCg93Dk0bBINWk=
github.com/pierrec/lz4/v4 v4.1.21 h1:yOVMLb6qSIDP67pl/5F7RepeKYu/VmTyEXvuMI5d9mQ=
github.com/pierrec/lz4/v4 v4.1.21/go.mod h1:gZWDp/Ze/IJXGXf23ltt2EXimqmTUXEy0GFuRQyBid4=
github.com/pkg/diff v0.0.0-20210226163009-20ebb0f2a09e/go.mod h1:pJLUxLENpZxwdsKMEsNbx1VGcRFpLqf3715MtcvvzbA=
github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRIccs7FGNTlIRMkT8wgtp5eCXdBlqhYGL6U=
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/power-devops/perfstat v0.0.0-20210106213030-5aafc221ea8c h1:ncq/mPwQF4JjgDlrVEn3C11VoGHZN7m8qihwgMEtzYw=
github.com/power-devops/perfstat v0.0.0-20210106213030-5aafc221ea8c/go.mod h1:OmDBASR4679mdNQnz2pUhc2G8CO2JrUAVFDRBDP/hJE=
github.com/projectdiscovery/blackrock v0.0.1 h1:lHQqhaaEFjgf5WkuItbpeCZv2DUIE45k0VbGJyft6LQ=
github.com/projectdiscovery/blackrock v0.0.1/go.mod h1:ANUtjDfaVrqB453bzToU+YB4cUbvBRpLvEwoWIwlTss=
github.com/projectdiscovery/cdncheck v1.2.13 h1:6zs4Mn8JV3yKyMoAr857Hf2NLvyOMpOfqCCT2V2OI1Q=
github.com/projectdiscovery/cdncheck v1.2.13/go.mod h1:/OhuZ9T25yXSqU6+oWvmVQ3QFvtew/Tp03u0jM+NJBE=
github.com/projectdiscovery/chaos-client v0.5.2 h1:dN+7GXEypsJAbCD//dBcUxzAEAEH1fjc/7Rf4F/RiNU=
github.com/projectdiscovery/chaos-client v0.5.2/go.mod h1:KnoJ/NJPhll42uaqlDga6oafFfNw5l2XI2ajRijtDuU=
github.com/projectdiscovery/dnsx v1.2.3 h1:S87U9kYuuqqvMFyen8mZQy1FMuR5EGCsXHqfHPQAeuc=
github.com/projectdiscovery/dnsx v1.2.3/go.mod h1:NjAEyJt6+meNqZqnYHL4ZPxXfysuva+et56Eq/e1cVE=
github.com/projectdiscovery/fastdialer v0.4.19 h1:MLHwEGM0x0pyltJaNvAVvwc27bnXdZ5mYr50S/2kMEE=
github.com/projectdiscovery/fastdialer v0.4.19/go.mod h1:HGdVsez+JgJ9/ljXjHRplOqkB7og+nqi0nrNWVNi03o=
github.com/projectdiscovery/fdmax v0.0.4 h1:K9tIl5MUZrEMzjvwn/G4drsHms2aufTn1xUdeVcmhmc=
github.com/projectdiscovery/fdmax v0.0.4/go.mod h1:oZLqbhMuJ5FmcoaalOm31B1P4Vka/CqP50nWjgtSz+I=
github.com/projectdiscovery/goflags v0.1.74 h1:n85uTRj5qMosm0PFBfsvOL24I7TdWRcWq/1GynhXS7c=
github.com/projectdiscovery/goflags v0.1.74/go.mod h1:UMc9/7dFz2oln+10tv6cy+7WZKTHf9UGhaNkF95emh4=
github.com/projectdiscovery/gologger v1.1.62 h1:wzKqvL6HQRzf0/PpBEhInZqqL1q4mKe2gFGJeDG3FqE=
github.com/projectdiscovery/gologger v1.1.62/go.mod h1:YWvMSxlHybU3SkFCcWn+driSJ8yY+3CR3g/textnp+Y=
github.com/projectdiscovery/hmap v0.0.98 h1:XxYIi7yJCNiDAKCJXvuY9IBM5O6OgDgx4XHgKxkR4eg=
github.com/projectdiscovery/hmap v0.0.98/go.mod h1:bgN5fuZPJMj2YnAGEEnCypoifCnALJixHEVQszktQIU=
github.com/projectdiscovery/machineid v0.0.0-20240226150047-2e2c51e35983 h1:ZScLodGSezQVwsQDtBSMFp72WDq0nNN+KE/5DHKY5QE=
github.com/projectdiscovery/machineid v0.0.0-20240226150047-2e2c51e35983/go.mod h1:3G3BRKui7nMuDFAZKR/M2hiOLtaOmyukT20g88qRQjI=
github.com/projectdiscovery/networkpolicy v0.1.31 h1:mE6iJeYOSql8gps/91vwiztE/kEHe5Im8oUO5Mkj9Zg=
github.com/projectdiscovery/networkpolicy v0.1.31/go.mod h1:5x4rGh4XhnoYl9wACnZyrjDGKIB/bQqxw2KrIM5V+XU=
github.com/projectdiscovery/ratelimit v0.0.82 h1:rtO5SQf5uQFu5zTahTaTcO06OxmG8EIF1qhdFPIyTak=
github.com/projectdiscovery/ratelimit v0.0.82/go.mod h1:z076BrLkBb5yS7uhHNoCTf8X/BvFSGRxwQ8EzEL9afM=
github.com/projectdiscovery/retryabledns v1.0.111 h1:iyMdCDgNmaSRJYcGqB+SLlvlw9WijlbJ6Q9OEpRAWsQ=
github.com/projectdiscovery/retryabledns v1.0.111/go.mod h1:6TOPJ3QAE4reBu6bvsGsTcyEb+OypcKYFQH7yVsjyIM=
github.com/projectdiscovery/retryablehttp-go v1.1.0 h1:uYp3EnuhhamTwvG41X6q6TAc/SHEO9pw9CBWbRASIQk=
github.com/projectdiscovery/retryablehttp-go v1.1.0/go.mod h1:9DU57ezv5cfZSWw/m5XFDTMjy1yKeMyn1kj35lPlcfM=
github.com/projectdiscovery/utils v0.7.3 h1:kX+77AA58yK6EZgkTRJEnK9V/7AZYzlXdcu/o/kJhFs=
github.com/projectdiscovery/utils v0.7.3/go.mod h1:uDdQ3/VWomai98l+a3Ye/srDXdJ4xUIar/mSXlQ9gBM=
github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
github.com/refraction-networking/utls v1.7.1 h1:dxg+jla3uocgN8HtX+ccwDr68uCBBO3qLrkZUbqkcw0=
github.com/refraction-networking/utls v1.7.1/go.mod h1:TUhh27RHMGtQvjQq+RyO11P6ZNQNBb3N0v7wsEjKAIQ=
github.com/rivo/uniseg v0.1.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=
github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=
github.com/rivo/uniseg v0.4.7 h1:WUdvkW8uEhrYfLC4ZzdpI2ztxP1I582+49Oc5Mq64VQ=
github.com/rivo/uniseg v0.4.7/go.mod h1:FN3SvrM+Zdj16jyLfmOkMNblXMcoc8DfTHruCPUcx88=
github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=
github.com/rogpeppe/go-internal v1.9.0/go.mod h1:WtVeX8xhTBvf0smdhujwtBcq4Qrzq/fJaraNFVN+nFs=
github.com/rogpeppe/go-internal v1.12.0 h1:exVL4IDcn6na9z1rAb56Vxr+CgyK3nn3O+epU5NdKM8=
github.com/rogpeppe/go-internal v1.12.0/go.mod h1:E+RYuTGaKKdloAfM02xzb0FW3Paa99yedzYV+kq4uf4=
github.com/rs/xid v1.5.0 h1:mKX4bl4iPYJtEIxp6CYiUuLQ/8DYMoz0PUdtGgMFRVc=
github.com/rs/xid v1.5.0/go.mod h1:trrq9SKmegXys3aeAKXMUTdJsYXVwGY3RLcfgqegfbg=
github.com/rwcarlsen/goexif v0.0.0-20190401172101-9e8deecbddbd/go.mod h1:hPqNNc0+uJM6H+SuU8sEs5K5IQeKccPqeSjfgcKGgPk=
github.com/saintfish/chardet v0.0.0-20230101081208-5e3ef4b5456d h1:hrujxIzL1woJ7AwssoOcM/tq5JjjG2yYOc8odClEiXA=
github.com/saintfish/chardet v0.0.0-20230101081208-5e3ef4b5456d/go.mod h1:uugorj2VCxiV1x+LzaIdVa9b4S4qGAcH6cbhh4qVxOU=
github.com/shirou/gopsutil/v3 v3.23.7 h1:C+fHO8hfIppoJ1WdsVm1RoI0RwXoNdfTK7yWXV0wVj4=
github.com/shirou/gopsutil/v3 v3.23.7/go.mod h1:c4gnmoRC0hQuaLqvxnx1//VXQ0Ms/X9UnJF8pddY5z4=
github.com/shoenig/go-m1cpu v0.1.6 h1:nxdKQNcEB6vzgA2E2bvzKIYRuNj7XNJ4S/aRSwKzFtM=
github.com/shoenig/go-m1cpu v0.1.6/go.mod h1:1JJMcUBvfNwpq05QDQVAnx3gUHr9IYF7GNg9SUEw2VQ=
github.com/shoenig/test v0.6.4 h1:kVTaSd7WLz5WZ2IaoM0RSzRsUD+m8wRR+5qvntpn4LU=
github.com/shoenig/test v0.6.4/go.mod h1:byHiCGXqrVaflBLAMq/srcZIHynQPQgeyvkvXnjqq0k=
github.com/sirupsen/logrus v1.3.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo=
github.com/sirupsen/logrus v1.7.0/go.mod h1:yWOB1SBYBC5VeMP7gHvWumXLIWorT60ONWic61uBYv0=
github.com/sirupsen/logrus v1.9.0/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ=
github.com/sorairolake/lzip-go v0.3.5 h1:ms5Xri9o1JBIWvOFAorYtUNik6HI3HgBTkISiqu0Cwg=
github.com/sorairolake/lzip-go v0.3.5/go.mod h1:N0KYq5iWrMXI0ZEXKXaS9hCyOjZUQdBDEIbXfoUwbdk=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
github.com/syndtr/goleveldb v1.0.0 h1:fBdIW9lB4Iz0n9khmH8w27SJ3QEJ7+IgjPEwGSZiFdE=
github.com/syndtr/goleveldb v1.0.0/go.mod h1:ZVVdQEZoIme9iO1Ch2Jdy24qqXrMMOU6lpPAyBWyWuQ=
github.com/therootcompany/xz v1.0.1 h1:CmOtsn1CbtmyYiusbfmhmkpAAETj0wBIH6kCYaX+xzw=
github.com/therootcompany/xz v1.0.1/go.mod h1:3K3UH1yCKgBneZYhuQUvJ9HPD19UEXEI0BWbMn8qNMY=
github.com/tidwall/assert v0.1.0 h1:aWcKyRBUAdLoVebxo95N7+YZVTFF/ASTr7BN4sLP6XI=
github.com/tidwall/assert v0.1.0/go.mod h1:QLYtGyeqse53vuELQheYl9dngGCJQ+mTtlxcktb+Kj8=
github.com/tidwall/btree v1.6.0 h1:LDZfKfQIBHGHWSwckhXI0RPSXzlo+KYdjK7FWSqOzzg=
github.com/tidwall/btree v1.6.0/go.mod h1:twD9XRA5jj9VUQGELzDO4HPQTNJsoWWfYEL+EUQ2cKY=
github.com/tidwall/buntdb v1.3.0 h1:gdhWO+/YwoB2qZMeAU9JcWWsHSYU3OvcieYgFRS0zwA=
github.com/tidwall/buntdb v1.3.0/go.mod h1:lZZrZUWzlyDJKlLQ6DKAy53LnG7m5kHyrEHvvcDmBpU=
github.com/tidwall/gjson v1.12.1/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=
github.com/tidwall/gjson v1.18.0 h1:FIDeeyB800efLX89e5a8Y0BNH+LOngJyGrIWxG2FKQY=
github.com/tidwall/gjson v1.18.0/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=
github.com/tidwall/grect v0.1.4 h1:dA3oIgNgWdSspFzn1kS4S/RDpZFLrIxAZOdJKjYapOg=
github.com/tidwall/grect v0.1.4/go.mod h1:9FBsaYRaR0Tcy4UwefBX/UDcDcDy9V5jUcxHzv2jd5Q=
github.com/tidwall/lotsa v1.0.2 h1:dNVBH5MErdaQ/xd9s769R31/n2dXavsQ0Yf4TMEHHw8=
github.com/tidwall/lotsa v1.0.2/go.mod h1:X6NiU+4yHA3fE3Puvpnn1XMDrFZrE9JO2/w+UMuqgR8=
github.com/tidwall/match v1.1.1 h1:+Ho715JplO36QYgwN9PGYNhgZvoUSc9X2c80KVTi+GA=
github.com/tidwall/match v1.1.1/go.mod h1:eRSPERbgtNPcGhD8UCthc6PmLEQXEWd3PRB5JTxsfmM=
github.com/tidwall/pretty v1.2.0/go.mod h1:ITEVvHYasfjBbM0u2Pg8T2nJnzm8xPwvNhhsoaGGjNU=
github.com/tidwall/pretty v1.2.1 h1:qjsOFOWWQl+N3RsoF5/ssm1pHmJJwhjlSbZ51I6wMl4=
github.com/tidwall/pretty v1.2.1/go.mod h1:ITEVvHYasfjBbM0u2Pg8T2nJnzm8xPwvNhhsoaGGjNU=
github.com/tidwall/rtred v0.1.2 h1:exmoQtOLvDoO8ud++6LwVsAMTu0KPzLTUrMln8u1yu8=
github.com/tidwall/rtred v0.1.2/go.mod h1:hd69WNXQ5RP9vHd7dqekAz+RIdtfBogmglkZSRxCHFQ=
github.com/tidwall/tinyqueue v0.1.1 h1:SpNEvEggbpyN5DIReaJ2/1ndroY8iyEGxPYxoSaymYE=
github.com/tidwall/tinyqueue v0.1.1/go.mod h1:O/QNHwrnjqr6IHItYrzoHAKYhBkLI67Q096fQP5zMYw=
github.com/tklauser/go-sysconf v0.3.11/go.mod h1:GqXfhXY3kiPa0nAXPDIQIWzJbMCB7AmcWpGR8lSZfqI=
github.com/tklauser/go-sysconf v0.3.12 h1:0QaGUFOdQaIVdPgfITYzaTegZvdCjmYO52cSFAEVmqU=
github.com/tklauser/go-sysconf v0.3.12/go.mod h1:Ho14jnntGE1fpdOqQEEaiKRpvIavV0hSfmBq8nJbHYI=
github.com/tklauser/numcpus v0.6.0/go.mod h1:FEZLMke0lhOUG6w2JadTzp0a+Nl8PF/GFkQ5UVIcaL4=
github.com/tklauser/numcpus v0.6.1 h1:ng9scYS7az0Bk4OZLvrNXNSAO2Pxr1XXRAPyjhIx+Fk=
github.com/tklauser/numcpus v0.6.1/go.mod h1:1XfjsgE2zo8GVw7POkMbHENHzVg3GzmoZ9fESEdAacY=
github.com/tomnomnom/linkheader v0.0.0-20180905144013-02ca5825eb80 h1:nrZ3ySNYwJbSpD6ce9duiP+QkD3JuLCcWkdaehUS/3Y=
github.com/tomnomnom/linkheader v0.0.0-20180905144013-02ca5825eb80/go.mod h1:iFyPdL66DjUD96XmzVL3ZntbzcflLnznH0fr99w5VqE=
github.com/ulikunitz/xz v0.5.8/go.mod h1:nbz6k7qbPmH4IRqmfOplQw/tblSgqTqBwxkY0oWt/14=
github.com/ulikunitz/xz v0.5.15 h1:9DNdB5s+SgV3bQ2ApL10xRc35ck0DuIX/isZvIk+ubY=
github.com/ulikunitz/xz v0.5.15/go.mod h1:nbz6k7qbPmH4IRqmfOplQw/tblSgqTqBwxkY0oWt/14=
github.com/weppos/publicsuffix-go v0.13.0/go.mod h1:z3LCPQ38eedDQSwmsSRW4Y7t2L8Ln16JPQ02lHAdn5k=
github.com/weppos/publicsuffix-go v0.30.1-0.20230422193905-8fecedd899db/go.mod h1:aiQaH1XpzIfgrJq3S1iw7w+3EDbRP7mF5fmwUhWyRUs=
github.com/weppos/publicsuffix-go v0.40.3-0.20250408071509-6074bbe7fd39 h1:Bz/zVM/LoGZ9IztGBHrq2zlFQQbEG8dBYnxb4hamIHM=
github.com/weppos/publicsuffix-go v0.40.3-0.20250408071509-6074bbe7fd39/go.mod h1:2oFzEwGYI7lhiqG0YkkcKa6VcpjVinQbWxaPzytDmLA=
github.com/xyproto/randomstring v1.0.5 h1:YtlWPoRdgMu3NZtP45drfy1GKoojuR7hmRcnhZqKjWU=
github.com/xyproto/randomstring v1.0.5/go.mod h1:rgmS5DeNXLivK7YprL0pY+lTuhNQW3iGxZ18UQApw/E=
github.com/yl2chen/cidranger v1.0.2 h1:lbOWZVCG1tCRX4u24kuM1Tb4nHqWkDxwLdoS+SevawU=
github.com/yl2chen/cidranger v1.0.2/go.mod h1:9U1yz7WPYDwf0vpNWFaeRh0bjwz5RVgRy/9UEQfHl0g=
github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
github.com/yuin/goldmark v1.7.1/go.mod h1:uzxRWxtg69N339t3louHJ7+O03ezfj6PlliRlaOzY1E=
github.com/yuin/goldmark v1.7.4 h1:BDXOHExt+A7gwPCJgPIIq7ENvceR7we7rOS9TNoLZeg=
github.com/yuin/goldmark v1.7.4/go.mod h1:uzxRWxtg69N339t3louHJ7+O03ezfj6PlliRlaOzY1E=
github.com/yuin/goldmark-emoji v1.0.3 h1:aLRkLHOuBR2czCY4R8olwMjID+tENfhyFDMCRhbIQY4=
github.com/yuin/goldmark-emoji v1.0.3/go.mod h1:tTkZEbwu5wkPmgTcitqddVxY9osFZiavD+r4AzQrh1U=
github.com/yusufpapurcu/wmi v1.2.3/go.mod h1:SBZ9tNy3G9/m5Oi98Zks0QjeHVDvuK0qfxQmPyzfmi0=
github.com/yusufpapurcu/wmi v1.2.4 h1:zFUKzehAFReQwLys1b/iSMl+JQGSCSjtVqQn9bBrPo0=
github.com/yusufpapurcu/wmi v1.2.4/go.mod h1:SBZ9tNy3G9/m5Oi98Zks0QjeHVDvuK0qfxQmPyzfmi0=
github.com/zcalusic/sysinfo v1.0.2 h1:nwTTo2a+WQ0NXwo0BGRojOJvJ/5XKvQih+2RrtWqfxc=
github.com/zcalusic/sysinfo v1.0.2/go.mod h1:kluzTYflRWo6/tXVMJPdEjShsbPpsFRyy+p1mBQPC30=
github.com/zmap/rc2 v0.0.0-20131011165748-24b9757f5521/go.mod h1:3YZ9o3WnatTIZhuOtot4IcUfzoKVjUHqu6WALIyI0nE=
github.com/zmap/rc2 v0.0.0-20190804163417-abaa70531248 h1:Nzukz5fNOBIHOsnP+6I79kPx3QhLv8nBy2mfFhBRq30=
github.com/zmap/rc2 v0.0.0-20190804163417-abaa70531248/go.mod h1:3YZ9o3WnatTIZhuOtot4IcUfzoKVjUHqu6WALIyI0nE=
github.com/zmap/zcertificate v0.0.0-20180516150559-0e3d58b1bac4/go.mod h1:5iU54tB79AMBcySS0R2XIyZBAVmeHranShAFELYx7is=
github.com/zmap/zcertificate v0.0.1/go.mod h1:q0dlN54Jm4NVSSuzisusQY0hqDWvu92C+TWveAxiVWk=
github.com/zmap/zcrypto v0.0.0-20201128221613-3719af1573cf/go.mod h1:aPM7r+JOkfL+9qSB4KbYjtoEzJqUK50EXkkJabeNJDQ=
github.com/zmap/zcrypto v0.0.0-20201211161100-e54a5822fb7e/go.mod h1:aPM7r+JOkfL+9qSB4KbYjtoEzJqUK50EXkkJabeNJDQ=
github.com/zmap/zcrypto v0.0.0-20230422215203-9a665e1e9968 h1:YOQ1vXEwE4Rnj+uQ/3oCuJk5wgVsvUyW+glsndwYuyA=
github.com/zmap/zcrypto v0.0.0-20230422215203-9a665e1e9968/go.mod h1:xIuOvYCZX21S5Z9bK1BMrertTGX/F8hgAPw7ERJRNS0=
github.com/zmap/zlint/v3 v3.0.0/go.mod h1:paGwFySdHIBEMJ61YjoqT4h7Ge+fdYG4sUQhnTb1lJ8=
go.etcd.io/bbolt v1.3.7 h1:j+zJOnnEjF/kyHlDDgGnVL/AIqIJPq8UoB2GSNfkUfQ=
go.etcd.io/bbolt v1.3.7/go.mod h1:N9Mkw9X8x5fupy0IKsmuqVtoGDyxsaDlbk4Rd05IAQw=
go.opencensus.io v0.21.0/go.mod h1:mSImk1erAIZhrmZN+AvHh14ztQfjbGwt4TtuofqLduU=
go.opencensus.io v0.22.0/go.mod h1:+kGneAE2xo2IficOXnaByMWTGM9T73dGwxeWcUqIpI8=
go.opencensus.io v0.22.2/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
go.opencensus.io v0.22.3/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
go.uber.org/multierr v1.11.0 h1:blXXJkSxSSfBVBlC76pxqeO+LN3aDfLQo+309xJstO0=
go.uber.org/multierr v1.11.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y=
go4.org v0.0.0-20230225012048-214862532bf5 h1:nifaUDeh+rPaBCMPMQHZmvJf+QdpLFnuQPwx+LxVmtc=
go4.org v0.0.0-20230225012048-214862532bf5/go.mod h1:F57wTi5Lrj6WLyswp5EYV1ncrEbFGHD4hhz6S1ZYeaU=
golang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20190605123033-f99c8df09eb5/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20201124201722-c8d3bf9c5392/go.mod h1:jdWPYTVW3xRLrWPugEBEK3UY2ZEsg3UU495nc5E+M+I=
golang.org/x/crypto v0.0.0-20201208171446-5f87f3452ae9/go.mod h1:jdWPYTVW3xRLrWPugEBEK3UY2ZEsg3UU495nc5E+M+I=
golang.org/x/crypto v0.0.0-20210220033148-5ea612d1eb83/go.mod h1:jdWPYTVW3xRLrWPugEBEK3UY2ZEsg3UU495nc5E+M+I=
golang.org/x/crypto v0.0.0-20210817164053-32db794688a5/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/crypto v0.0.0-20211209193657-4570a0811e8b/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
golang.org/x/crypto v0.6.0/go.mod h1:OFC/31mSvZgRz0V1QTNCzfAI1aIRzbiufJtkMIlEp58=
golang.org/x/crypto v0.7.0/go.mod h1:pYwdfH91IfpZVANVyUOhSIPZaFoJGxTFbZhFTx+dXZU=
golang.org/x/crypto v0.45.0 h1:jMBrvKuj23MTlT0bQEOBcAE0mjg8mK9RXFhRH6nyF3Q=
golang.org/x/crypto v0.45.0/go.mod h1:XTGrrkGJve7CYK7J8PEww4aY7gM3qMCElcJQ8n8JdX4=
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8=
golang.org/x/exp v0.0.0-20190829153037-c13cbed26979/go.mod h1:86+5VVa7VpoJ4kLfm080zCjGlMRFzhUhsZKEZO7MGek=
golang.org/x/exp v0.0.0-20191030013958-a1ab85dbe136/go.mod h1:JXzH8nQsPlswgeRAPE3MuO9GYsAcnJvJ4vnMwN/5qkY=
golang.org/x/exp v0.0.0-20191129062945-2f5052295587/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=
golang.org/x/exp v0.0.0-20191227195350-da58074b4299/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=
golang.org/x/exp v0.0.0-20200207192155-f17229e696bd/go.mod h1:J/WKrq2StrnmMY6+EHIKF9dgMWnmCNThgcyBT1FY9mM=
golang.org/x/exp v0.0.0-20250106191152-7588d65b2ba8 h1:yqrTHse8TCMW1M1ZCP+VAR/l0kKxwaAIqN/il7x4voA=
golang.org/x/exp v0.0.0-20250106191152-7588d65b2ba8/go.mod h1:tujkw807nyEEAamNbDrEGzRav+ilXA7PCRAd6xsmwiU=
golang.org/x/image v0.0.0-20190227222117-0694c2d4d067/go.mod h1:kZ7UVZpmo3dzQBMxlp+ypCbDeSB+sBbTgSJuh5dn5js=
golang.org/x/image v0.0.0-20190802002840-cff245a6509b/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0=
golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU=
golang.org/x/lint v0.0.0-20190301231843-5614ed5bae6f/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/lint v0.0.0-20190409202823-959b441ac422/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/lint v0.0.0-20190909230951-414d861bb4ac/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/lint v0.0.0-20190930215403-16217165b5de/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/lint v0.0.0-20191125180803-fdd1cda4f05f/go.mod h1:5qLYkcX4OjUUV8bRuDixDT3tpyyb+LUpUlRWLxfhWrs=
golang.org/x/lint v0.0.0-20200130185559-910be7a94367/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=
golang.org/x/mobile v0.0.0-20190312151609-d3739f865fa6/go.mod h1:z+o9i4GpDbdi3rU15maQ/Ox0txvL9dWGYEHz965HBQE=
golang.org/x/mobile v0.0.0-20190719004257-d2bd2a29d028/go.mod h1:E/iHnbuqvinMTCcRqshq8CkpyQDoeVncDDYHnLhea+o=
golang.org/x/mod v0.0.0-20190513183733-4bf6d317e70e/go.mod h1:mXi4GBBbnImb6dmsKGUJ2LatrhH/nqhxcFungHvyanc=
golang.org/x/mod v0.1.0/go.mod h1:0QHyrYULN0/3qlju5TqG8bIK38QM8yzMo5ekMj3DlcY=
golang.org/x/mod v0.1.1-0.20191105210325-c90efee705ee/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg=
golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
golang.org/x/mod v0.29.0 h1:HV8lRxZC4l2cr3Zq1LvtOsi/ThTgWnUk/y64QSs8GwA=
golang.org/x/mod v0.29.0/go.mod h1:NyhrlYXJ2H4eJiRy/WDBO6HMqZQ6q9nk4JzS3NuCK+w=
golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190108225652-1e06a53dbb7e/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190501004415-9ce7a6920f09/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190503192946-f4e77d36d62c/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20190724013045-ca1201d0de80/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20191209160850-c0dbc17a3553/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200202094626-16171245cfb2/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20201110031124-69a78807bb2b/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.0.0-20201209123823-ac852fbbde11/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20211112202133-69e39bad7dc2/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
golang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
golang.org/x/net v0.7.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
golang.org/x/net v0.8.0/go.mod h1:QVkue5JL9kW//ek3r6jTKnTFis1tRmNAW2P1shuFdJc=
golang.org/x/net v0.47.0 h1:Mx+4dIFzqraBXUugkia1OOvlD6LemFo1ALMHjrXDOhY=
golang.org/x/net v0.47.0/go.mod h1:/jNxtkgq5yWUGYkaZGqo27cfGZ1c5Nen03aYrrKpVRU=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20191202225959-858c2ad4c8b6/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.5.0/go.mod h1:9/XBHVqLaWO3/BRHs5jbpYCnOZVjj5V0ndyaAM7KB4I=
golang.org/x/oauth2 v0.27.0 h1:da9Vo7/tDv5RH/7nZDz1eMGS/q1Vv1N/7FCrBhI9I3M=
golang.org/x/oauth2 v0.27.0/go.mod h1:onh5ek6nERTohokkhCD/y2cV4Do3fxFHFuAejCkRWT8=
golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190227155943-e225da77a7e6/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.18.0 h1:kr88TuHDroi+UVf+0hZnirlk8o8T+4MrK6mr60WkH/I=
golang.org/x/sync v0.18.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI=
golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190312061237-fead79001313/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190502145724-3ef323f4f1fd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190507160741-ecd444e8653b/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190606165138-5da285871e9c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190624142023-c5567b49c5d0/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190726091711-fc99dfbffb4e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190916202348-b4ddaad3f8a3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191204072324-ce4227a45e2e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191228213918-04cbcbbfeed8/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200212091648-12a6c2dcc1e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201126233918-771906719818/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201204225414-ed752295db88/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210228012217-479acdf4ea46/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.2.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.8.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.10.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.11.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.38.0 h1:3yZWxaJjBmCWXqhN1qh02AkOnCQ1poK6oF+a7xWL6Gc=
golang.org/x/sys v0.38.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=
golang.org/x/term v0.0.0-20201117132131-f5c789dd3221/go.mod h1:Nr5EML6q2oocZ2LXRh80K7BxOlk5/8JxuGnuhpl+muw=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k=
golang.org/x/term v0.6.0/go.mod h1:m6U89DPEgQRMq3DNkDClhWw02AUbt2daBVO4cn4Hv9U=
golang.org/x/term v0.37.0 h1:8EGAD0qCmHYZg6J17DvsMy9/wJ7/D/4pV/wfnld5lTU=
golang.org/x/term v0.37.0/go.mod h1:5pB4lxRNYYVZuTLmy8oR2BH8dflOR+IbTYFD8fi3254=
golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.4/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/text v0.8.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8=
golang.org/x/text v0.31.0 h1:aC8ghyu4JhP8VojJ2lEHBnochRno1sgL6nEi9WGFGMM=
golang.org/x/text v0.31.0/go.mod h1:tKRAlv61yKIjGGHX/4tP1LTbc13YSec1pxVEWXzfoeM=
golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.5.0 h1:o7cqy6amK/52YcAKIPlM3a+Fpj35zvRj2TP+e1xFSfk=
golang.org/x/time v0.5.0/go.mod h1:3BpzKBy/shNhVucY/MWOyx10tF3SFh9QdLuxbVysPQM=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY=
golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20190312151545-0bb0c0a6e846/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20190312170243-e65039ee4138/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20190425150028-36563e24a262/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
golang.org/x/tools v0.0.0-20190506145303-2d16b83fe98c/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
golang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
golang.org/x/tools v0.0.0-20190606124116-d0a3d012864b/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=
golang.org/x/tools v0.0.0-20190621195816-6e04913cbbac/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=
golang.org/x/tools v0.0.0-20190628153133-6cdbf07be9d0/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=
golang.org/x/tools v0.0.0-20190816200558-6889da9d5479/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20190911174233-4f2ddba30aff/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191012152004-8de300cfc20a/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191113191852-77e3bb0ad9e7/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191115202509-3a792d9c32b2/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191125144606-a911d9008d1f/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191216173652-a0e659d51361/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20191227053925-7b8e75db28f4/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200130002326-2f3ba24bd6e7/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200207183749-b753a1ba74fa/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200212150539-ea181f53ac56/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU=
golang.org/x/tools v0.38.0 h1:Hx2Xv8hISq8Lm16jvBZ2VQf+RLmbd7wVUsALibYI/IQ=
golang.org/x/tools v0.38.0/go.mod h1:yEsQ/d/YK8cjh0L6rZlY8tgtlKiBNTL14pGDJPJpYQs=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
google.golang.org/api v0.4.0/go.mod h1:8k5glujaEP+g9n7WNsDg8QP6cUVNI86fCNMcbazEtwE=
google.golang.org/api v0.7.0/go.mod h1:WtwebWUNSVBH/HAw79HIFXZNqEvBhG+Ra+ax0hx3E3M=
google.golang.org/api v0.8.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg=
google.golang.org/api v0.9.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg=
google.golang.org/api v0.13.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI=
google.golang.org/api v0.14.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI=
google.golang.org/api v0.15.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI=
google.golang.org/api v0.17.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
google.golang.org/appengine v1.5.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
google.golang.org/appengine v1.6.1/go.mod h1:i06prIuMbXzDqacNJfV5OdTW448YApPu5ww/cMBSeb0=
google.golang.org/appengine v1.6.5/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=
google.golang.org/appengine v1.6.7/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=
google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=
google.golang.org/genproto v0.0.0-20190307195333-5fe7a883aa19/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
google.golang.org/genproto v0.0.0-20190418145605-e7d98fc518a7/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
google.golang.org/genproto v0.0.0-20190425155659-357c62f0e4bb/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
google.golang.org/genproto v0.0.0-20190502173448-54afdca5d873/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
google.golang.org/genproto v0.0.0-20190801165951-fa694d86fc64/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc=
google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc=
google.golang.org/genproto v0.0.0-20190911173649-1774047e7e51/go.mod h1:IbNlFCBrqXvoKpeg0TB2l7cyZUmoaFKYIwrEpbDKLA8=
google.golang.org/genproto v0.0.0-20191108220845-16a3f7862a1a/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
google.golang.org/genproto v0.0.0-20191115194625-c23dd37a84c9/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
google.golang.org/genproto v0.0.0-20191216164720-4f79533eabd1/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
google.golang.org/genproto v0.0.0-20191230161307-f3c370f40bfb/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
google.golang.org/genproto v0.0.0-20200212174721-66ed5ce911ce/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
google.golang.org/grpc v1.20.1/go.mod h1:10oTOabMzJvdu6/UiuZezV6QK5dSlG84ov/aaiqXj38=
google.golang.org/grpc v1.21.1/go.mod h1:oYelfM1adQP15Ek0mdvEgi9Df8B9CZIaU1084ijfRaM=
google.golang.org/grpc v1.23.0/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg=
google.golang.org/grpc v1.26.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
google.golang.org/grpc v1.27.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
google.golang.org/grpc v1.27.1/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw=
google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc=
google.golang.org/protobuf v1.28.0/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I=
google.golang.org/protobuf v1.28.1/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
gopkg.in/djherbis/times.v1 v1.3.0 h1:uxMS4iMtH6Pwsxog094W0FYldiNnfY/xba00vq6C2+o=
gopkg.in/djherbis/times.v1 v1.3.0/go.mod h1:AQlg6unIsrsCEdQYhTzERy542dz6SFdQFZFv6mUY0P8=
gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI=
gopkg.in/fsnotify.v1 v1.4.7/go.mod h1:Tz8NjZHkW78fSQdbUxIjBTcgA1z1m8ZHf0WmKUhAMys=
gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 h1:uRGJdciOHaEIrze2W8Q3AKkepLTh2hOroT7a+7czfdQ=
gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7/go.mod h1:dt/ZhP58zS4L8KSrWDmTeBkI65Dw0HsyUHuEVlX15mw=
gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190106161140-3f1c8253044a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190418001031-e561f6794a2a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.1-2019.2.3/go.mod h1:a3bituU0lyd329TUQxRnasdCoJDkEUEAqEt0JzvZhAg=
rsc.io/binaryregexp v0.2.0/go.mod h1:qTv7/COck+e2FymRvadv62gMdZztPaShugOCi3I+8D8=
rsc.io/quote/v3 v3.1.0/go.mod h1:yEA65RcK8LyAZtP9Kv3t0HmxON59tX3rD+tICJqUlj0=
rsc.io/sampler v1.3.0/go.mod h1:T1hPZKmBbMNahiBKFy5HrXp6adAjACjK9JXDnKaTXpA=
================================================
FILE: pkg/passive/doc.go
================================================
// Package passive provides capability for doing passive subdomain
// enumeration on targets.
package passive
================================================
FILE: pkg/passive/passive.go
================================================
package passive
import (
"context"
"fmt"
"math"
"sort"
"strings"
"sync"
"time"
"github.com/projectdiscovery/ratelimit"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping"
)
type EnumerationOptions struct {
customRateLimiter *subscraping.CustomRateLimit
}
type EnumerateOption func(opts *EnumerationOptions)
func WithCustomRateLimit(crl *subscraping.CustomRateLimit) EnumerateOption {
return func(opts *EnumerationOptions) {
opts.customRateLimiter = crl
}
}
// EnumerateSubdomains wraps EnumerateSubdomainsWithCtx with an empty context
func (a *Agent) EnumerateSubdomains(domain string, proxy string, rateLimit int, timeout int, maxEnumTime time.Duration, options ...EnumerateOption) chan subscraping.Result {
return a.EnumerateSubdomainsWithCtx(context.Background(), domain, proxy, rateLimit, timeout, maxEnumTime, options...)
}
// EnumerateSubdomainsWithCtx enumerates all the subdomains for a given domain
func (a *Agent) EnumerateSubdomainsWithCtx(ctx context.Context, domain string, proxy string, rateLimit int, timeout int, maxEnumTime time.Duration, options ...EnumerateOption) chan subscraping.Result {
results := make(chan subscraping.Result)
go func() {
defer close(results)
var enumerateOptions EnumerationOptions
for _, enumerateOption := range options {
enumerateOption(&enumerateOptions)
}
multiRateLimiter, err := a.buildMultiRateLimiter(ctx, rateLimit, enumerateOptions.customRateLimiter)
if err != nil {
results <- subscraping.Result{
Type: subscraping.Error, Error: fmt.Errorf("could not init multi rate limiter for %s: %s", domain, err),
}
return
}
session, err := subscraping.NewSession(domain, proxy, multiRateLimiter, timeout)
if err != nil {
results <- subscraping.Result{
Type: subscraping.Error, Error: fmt.Errorf("could not init passive session for %s: %s", domain, err),
}
return
}
defer session.Close()
ctx, cancel := context.WithTimeout(ctx, maxEnumTime)
wg := &sync.WaitGroup{}
// Run each source in parallel on the target domain
for _, runner := range a.sources {
wg.Add(1)
go func(source subscraping.Source) {
defer wg.Done()
ctxWithValue := context.WithValue(ctx, subscraping.CtxSourceArg, source.Name())
for resp := range source.Run(ctxWithValue, domain, session) {
select {
case <-ctx.Done():
return
case results <- resp:
}
}
}(runner)
}
wg.Wait()
cancel()
}()
return results
}
func (a *Agent) buildMultiRateLimiter(ctx context.Context, globalRateLimit int, rateLimit *subscraping.CustomRateLimit) (*ratelimit.MultiLimiter, error) {
var multiRateLimiter *ratelimit.MultiLimiter
var err error
for _, source := range a.sources {
var rl uint
if sourceRateLimit, ok := rateLimit.Custom.Get(strings.ToLower(source.Name())); ok {
rl = sourceRateLimitOrDefault(uint(globalRateLimit), sourceRateLimit)
}
if rl > 0 {
multiRateLimiter, err = addRateLimiter(ctx, multiRateLimiter, source.Name(), rl, time.Second)
} else {
multiRateLimiter, err = addRateLimiter(ctx, multiRateLimiter, source.Name(), math.MaxUint32, time.Millisecond)
}
if err != nil {
break
}
}
return multiRateLimiter, err
}
func sourceRateLimitOrDefault(defaultRateLimit uint, sourceRateLimit uint) uint {
if sourceRateLimit > 0 {
return sourceRateLimit
}
return defaultRateLimit
}
func addRateLimiter(ctx context.Context, multiRateLimiter *ratelimit.MultiLimiter, key string, maxCount uint, duration time.Duration) (*ratelimit.MultiLimiter, error) {
if multiRateLimiter == nil {
mrl, err := ratelimit.NewMultiLimiter(ctx, &ratelimit.Options{
Key: key,
IsUnlimited: maxCount == math.MaxUint32,
MaxCount: maxCount,
Duration: duration,
})
return mrl, err
}
err := multiRateLimiter.Add(&ratelimit.Options{
Key: key,
IsUnlimited: maxCount == math.MaxUint32,
MaxCount: maxCount,
Duration: duration,
})
return multiRateLimiter, err
}
func (a *Agent) GetStatistics() map[string]subscraping.Statistics {
stats := make(map[string]subscraping.Statistics)
sort.Slice(a.sources, func(i, j int) bool {
return a.sources[i].Name() > a.sources[j].Name()
})
for _, source := range a.sources {
stats[source.Name()] = source.Statistics()
}
return stats
}
================================================
FILE: pkg/passive/sources.go
================================================
package passive
import (
"fmt"
"os"
"strings"
"golang.org/x/exp/maps"
"github.com/projectdiscovery/gologger"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping/sources/alienvault"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping/sources/anubis"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping/sources/bevigil"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping/sources/bufferover"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping/sources/builtwith"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping/sources/c99"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping/sources/censys"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping/sources/certspotter"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping/sources/chaos"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping/sources/chinaz"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping/sources/commoncrawl"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping/sources/crtsh"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping/sources/digitalyama"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping/sources/digitorus"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping/sources/dnsdb"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping/sources/dnsdumpster"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping/sources/dnsrepo"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping/sources/domainsproject"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping/sources/driftnet"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping/sources/facebook"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping/sources/fofa"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping/sources/fullhunt"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping/sources/github"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping/sources/hackertarget"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping/sources/hudsonrock"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping/sources/intelx"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping/sources/leakix"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping/sources/merklemap"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping/sources/netlas"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping/sources/onyphe"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping/sources/profundis"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping/sources/pugrecon"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping/sources/quake"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping/sources/rapiddns"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping/sources/reconeer"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping/sources/redhuntlabs"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping/sources/robtex"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping/sources/rsecloud"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping/sources/securitytrails"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping/sources/shodan"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping/sources/sitedossier"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping/sources/thc"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping/sources/threatbook"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping/sources/threatcrowd"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping/sources/urlscan"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping/sources/virustotal"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping/sources/waybackarchive"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping/sources/whoisxmlapi"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping/sources/windvane"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping/sources/zoomeyeapi"
mapsutil "github.com/projectdiscovery/utils/maps"
)
var AllSources = [...]subscraping.Source{
&alienvault.Source{},
&anubis.Source{},
&bevigil.Source{},
&bufferover.Source{},
&builtwith.Source{},
&c99.Source{},
&censys.Source{},
&certspotter.Source{},
&chaos.Source{},
&chinaz.Source{},
&commoncrawl.Source{},
&crtsh.Source{},
&digitalyama.Source{},
&digitorus.Source{},
&dnsdb.Source{},
&dnsdumpster.Source{},
&dnsrepo.Source{},
&domainsproject.Source{},
&driftnet.Source{},
&facebook.Source{},
&fofa.Source{},
&fullhunt.Source{},
&github.Source{},
&hackertarget.Source{},
&hudsonrock.Source{},
&intelx.Source{},
&leakix.Source{},
&merklemap.Source{},
&netlas.Source{},
&onyphe.Source{},
&profundis.Source{},
&pugrecon.Source{},
&quake.Source{},
&rapiddns.Source{},
// &reconcloud.Source{}, // failing due to cloudflare bot protection
&reconeer.Source{},
&redhuntlabs.Source{},
// &riddler.Source{}, // failing due to cloudfront protection
&robtex.Source{},
&rsecloud.Source{},
&securitytrails.Source{},
&shodan.Source{},
&sitedossier.Source{},
&thc.Source{},
&threatbook.Source{},
&threatcrowd.Source{},
// &threatminer.Source{}, // failing api
&urlscan.Source{},
&virustotal.Source{},
&waybackarchive.Source{},
&whoisxmlapi.Source{},
&windvane.Source{},
&zoomeyeapi.Source{},
}
var sourceWarnings = mapsutil.NewSyncLockMap[string, string](
mapsutil.WithMap(mapsutil.Map[string, string]{}))
var NameSourceMap = make(map[string]subscraping.Source, len(AllSources))
func init() {
for _, currentSource := range AllSources {
NameSourceMap[strings.ToLower(currentSource.Name())] = currentSource
}
}
// Agent is a struct for running passive subdomain enumeration
// against a given host. It wraps subscraping package and provides
// a layer to build upon.
type Agent struct {
sources []subscraping.Source
}
// New creates a new agent for passive subdomain discovery
func New(sourceNames, excludedSourceNames []string, useAllSources, useSourcesSupportingRecurse bool) *Agent {
sources := make(map[string]subscraping.Source, len(AllSources))
if useAllSources {
maps.Copy(sources, NameSourceMap)
} else {
if len(sourceNames) > 0 {
for _, source := range sourceNames {
if NameSourceMap[source] == nil {
gologger.Warning().Msgf("There is no source with the name: %s", source)
} else {
sources[source] = NameSourceMap[source]
}
}
} else {
for _, currentSource := range AllSources {
if currentSource.IsDefault() {
sources[currentSource.Name()] = currentSource
}
}
}
}
if len(excludedSourceNames) > 0 {
for _, sourceName := range excludedSourceNames {
delete(sources, sourceName)
}
}
if useSourcesSupportingRecurse {
for sourceName, source := range sources {
if !source.HasRecursiveSupport() {
delete(sources, sourceName)
}
}
}
if len(sources) == 0 {
gologger.Fatal().Msg("No sources selected for this search")
}
gologger.Debug().Msgf("Selected source(s) for this search: %s", strings.Join(maps.Keys(sources), ", "))
for _, currentSource := range sources {
if warning, ok := sourceWarnings.Get(strings.ToLower(currentSource.Name())); ok {
gologger.Warning().Msg(warning)
}
}
for _, source := range sources {
keyReq := source.KeyRequirement()
if keyReq == subscraping.RequiredKey || keyReq == subscraping.OptionalKey {
if apiKey := os.Getenv(fmt.Sprintf("%s_API_KEY", strings.ToUpper(source.Name()))); apiKey != "" {
source.AddApiKeys([]string{apiKey})
}
}
}
// Create the agent, insert the sources and remove the excluded sources
agent := &Agent{sources: maps.Values(sources)}
return agent
}
================================================
FILE: pkg/passive/sources_test.go
================================================
package passive
import (
"fmt"
"strconv"
"testing"
"github.com/stretchr/testify/assert"
"golang.org/x/exp/maps"
)
var (
expectedAllSources = []string{
"alienvault",
"anubis",
"bevigil",
"bufferover",
"c99",
"censys",
"certspotter",
"chaos",
"chinaz",
"commoncrawl",
"crtsh",
"digitorus",
"dnsdumpster",
"dnsdb",
"dnsrepo",
"domainsproject",
"driftnet",
"fofa",
"fullhunt",
"github",
"hackertarget",
"intelx",
"netlas",
"onyphe",
"quake",
"pugrecon",
"rapiddns",
"redhuntlabs",
// "riddler", // failing due to cloudfront protection
"robtex",
"rsecloud",
"securitytrails",
"profundis",
"shodan",
"sitedossier",
"threatbook",
"threatcrowd",
"virustotal",
"waybackarchive",
"whoisxmlapi",
"windvane",
"zoomeyeapi",
"leakix",
"facebook",
// "threatminer",
// "reconcloud",
"reconeer",
"builtwith",
"hudsonrock",
"digitalyama",
"merklemap",
"thc",
"urlscan",
}
expectedDefaultSources = []string{
"alienvault",
"anubis",
"bevigil",
"bufferover",
"c99",
"certspotter",
"censys",
"chaos",
"chinaz",
"crtsh",
"digitorus",
"dnsdumpster",
"domainsproject",
"dnsrepo",
"driftnet",
"fofa",
"fullhunt",
"hackertarget",
"intelx",
"onyphe",
"quake",
"redhuntlabs",
"robtex",
// "riddler", // failing due to cloudfront protection
"rsecloud",
"securitytrails",
"profundis",
"shodan",
"windvane",
"virustotal",
"whoisxmlapi",
"leakix",
"facebook",
// "threatminer",
// "reconcloud",
"reconeer",
"builtwith",
"digitalyama",
"thc",
"urlscan",
}
expectedDefaultRecursiveSources = []string{
"alienvault",
"bufferover",
"certspotter",
"crtsh",
"dnsdb",
"digitorus",
"driftnet",
"hackertarget",
"securitytrails",
"virustotal",
"leakix",
"facebook",
"merklemap",
"urlscan",
// "reconcloud",
}
)
func TestSourceCategorization(t *testing.T) {
defaultSources := make([]string, 0, len(AllSources))
recursiveSources := make([]string, 0, len(AllSources))
for _, source := range AllSources {
sourceName := source.Name()
if source.IsDefault() {
defaultSources = append(defaultSources, sourceName)
}
if source.HasRecursiveSupport() {
recursiveSources = append(recursiveSources, sourceName)
}
}
assert.ElementsMatch(t, expectedDefaultSources, defaultSources)
assert.ElementsMatch(t, expectedDefaultRecursiveSources, recursiveSources)
assert.ElementsMatch(t, expectedAllSources, maps.Keys(NameSourceMap))
}
// Review: not sure if this test is necessary/useful
// implementation is straightforward where sources are stored in maps and filtered based on options
// the test is just checking if the filtering works as expected using count of sources
func TestSourceFiltering(t *testing.T) {
someSources := []string{
"alienvault",
"chaos",
"crtsh",
"virustotal",
}
someExclusions := []string{
"alienvault",
"virustotal",
}
tests := []struct {
sources []string
exclusions []string
withAllSources bool
withRecursion bool
expectedLength int
}{
{someSources, someExclusions, false, false, len(someSources) - len(someExclusions)},
{someSources, someExclusions, false, true, 1},
{someSources, someExclusions, true, false, len(AllSources) - len(someExclusions)},
{someSources, []string{}, false, false, len(someSources)},
{someSources, []string{}, true, false, len(AllSources)},
{[]string{}, []string{}, false, false, len(expectedDefaultSources)},
{[]string{}, []string{}, true, false, len(AllSources)},
{[]string{}, []string{}, true, true, len(expectedDefaultRecursiveSources)},
}
for index, test := range tests {
t.Run(strconv.Itoa(index+1), func(t *testing.T) {
agent := New(test.sources, test.exclusions, test.withAllSources, test.withRecursion)
for _, v := range agent.sources {
fmt.Println(v.Name())
}
assert.Equal(t, test.expectedLength, len(agent.sources))
agent = nil
})
}
}
================================================
FILE: pkg/passive/sources_w_auth_test.go
================================================
package passive
import (
"context"
"fmt"
"math"
"os"
"reflect"
"strings"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/projectdiscovery/gologger"
"github.com/projectdiscovery/gologger/levels"
"github.com/projectdiscovery/ratelimit"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping"
)
func TestSourcesWithKeys(t *testing.T) {
if testing.Short() {
t.Skip("skipping test in short mode.")
}
domain := "hackerone.com"
timeout := 60
gologger.DefaultLogger.SetMaxLevel(levels.LevelDebug)
ctxParent := context.Background()
var multiRateLimiter *ratelimit.MultiLimiter
for _, source := range AllSources {
if !source.NeedsKey() {
continue
}
multiRateLimiter, _ = addRateLimiter(ctxParent, multiRateLimiter, source.Name(), math.MaxInt32, time.Millisecond)
}
session, err := subscraping.NewSession(domain, "", multiRateLimiter, timeout)
assert.Nil(t, err)
var expected = subscraping.Result{Type: subscraping.Subdomain, Value: domain, Error: nil}
for _, source := range AllSources {
if !source.NeedsKey() {
continue
}
var apiKey string
if source.Name() == "chaos" {
apiKey = os.Getenv("PDCP_API_KEY")
} else {
apiKey = os.Getenv(fmt.Sprintf("%s_API_KEY", strings.ToUpper(source.Name())))
}
if apiKey == "" {
fmt.Printf("Skipping %s as no API key is provided\n", source.Name())
continue
}
source.AddApiKeys([]string{apiKey})
t.Run(source.Name(), func(t *testing.T) {
var results []subscraping.Result
ctxWithValue := context.WithValue(ctxParent, subscraping.CtxSourceArg, source.Name())
for result := range source.Run(ctxWithValue, domain, session) {
results = append(results, result)
assert.Equal(t, source.Name(), result.Source, "wrong source name")
if result.Type != subscraping.Error {
assert.True(t, strings.HasSuffix(strings.ToLower(result.Value), strings.ToLower(expected.Value)),
fmt.Sprintf("result(%s) is not subdomain of %s", strings.ToLower(result.Value), expected.Value))
} else {
assert.Equal(t, reflect.TypeOf(expected.Error), reflect.TypeOf(result.Error), fmt.Sprintf("%s: %s", result.Source, result.Error))
}
}
assert.GreaterOrEqual(t, len(results), 1, fmt.Sprintf("No result found for %s", source.Name()))
})
}
}
================================================
FILE: pkg/passive/sources_wo_auth_test.go
================================================
package passive
import (
"context"
"fmt"
"math"
"reflect"
"strings"
"testing"
"time"
"github.com/stretchr/testify/assert"
"golang.org/x/exp/slices"
"github.com/projectdiscovery/gologger"
"github.com/projectdiscovery/gologger/levels"
"github.com/projectdiscovery/ratelimit"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping"
)
func TestSourcesWithoutKeys(t *testing.T) {
if testing.Short() {
t.Skip("skipping test in short mode.")
}
ignoredSources := []string{
"commoncrawl", // commoncrawl is under resourced and will likely time-out so step over it for this test https://groups.google.com/u/2/g/common-crawl/c/3QmQjFA_3y4/m/vTbhGqIBBQAJ
"riddler", // failing due to cloudfront protection
"crtsh", // Fails in GH Action (possibly IP-based ban) causing a timeout.
"hackertarget", // Fails in GH Action (possibly IP-based ban) but works locally
"waybackarchive", // Fails randomly
"alienvault", // 503 Service Temporarily Unavailable
"digitorus", // failing with "Failed to retrieve certificate"
"dnsdumpster", // failing with "unexpected status code 403 received"
"anubis", // failing with "too many redirects"
"threatcrowd", // failing with "randomly failing with unmarshal error when hit multiple times"
"leakix", // now requires API key (returns 401)
"reconeer", // now requires API key (returns 401)
"sitedossier", // flaky - returns no results in CI
}
domain := "hackerone.com"
timeout := 60
gologger.DefaultLogger.SetMaxLevel(levels.LevelDebug)
ctxParent := context.Background()
var multiRateLimiter *ratelimit.MultiLimiter
for _, source := range AllSources {
if source.NeedsKey() || slices.Contains(ignoredSources, source.Name()) {
continue
}
multiRateLimiter, _ = addRateLimiter(ctxParent, multiRateLimiter, source.Name(), math.MaxInt32, time.Millisecond)
}
session, err := subscraping.NewSession(domain, "", multiRateLimiter, timeout)
assert.Nil(t, err)
var expected = subscraping.Result{Type: subscraping.Subdomain, Value: domain, Error: nil}
for _, source := range AllSources {
if source.NeedsKey() || slices.Contains(ignoredSources, source.Name()) {
continue
}
t.Run(source.Name(), func(t *testing.T) {
var results []subscraping.Result
ctxWithValue := context.WithValue(ctxParent, subscraping.CtxSourceArg, source.Name())
for result := range source.Run(ctxWithValue, domain, session) {
results = append(results, result)
assert.Equal(t, source.Name(), result.Source, "wrong source name")
if result.Type != subscraping.Error {
assert.True(t, strings.HasSuffix(strings.ToLower(result.Value), strings.ToLower(expected.Value)),
fmt.Sprintf("result(%s) is not subdomain of %s", strings.ToLower(result.Value), expected.Value))
} else {
assert.Equal(t, reflect.TypeOf(expected.Error), reflect.TypeOf(result.Error), fmt.Sprintf("%s: %s", result.Source, result.Error))
}
}
assert.GreaterOrEqual(t, len(results), 1, fmt.Sprintf("No result found for %s", source.Name()))
})
}
}
================================================
FILE: pkg/resolve/client.go
================================================
package resolve
import (
"github.com/projectdiscovery/dnsx/libs/dnsx"
)
// DefaultResolvers contains the default list of resolvers known to be good
var DefaultResolvers = []string{
"1.1.1.1:53", // Cloudflare primary
"1.0.0.1:53", // Cloudflare secondary
"8.8.8.8:53", // Google primary
"8.8.4.4:53", // Google secondary
"9.9.9.9:53", // Quad9 Primary
"9.9.9.10:53", // Quad9 Secondary
"77.88.8.8:53", // Yandex Primary
"77.88.8.1:53", // Yandex Secondary
"208.67.222.222:53", // OpenDNS Primary
"208.67.220.220:53", // OpenDNS Secondary
}
// Resolver is a struct for resolving DNS names
type Resolver struct {
DNSClient *dnsx.DNSX
Resolvers []string
}
// New creates a new resolver struct with the default resolvers
func New() *Resolver {
return &Resolver{
Resolvers: []string{},
}
}
================================================
FILE: pkg/resolve/doc.go
================================================
// Package resolve is used to handle resolving records
// It also handles wildcard subdomains and rotating resolvers.
package resolve
================================================
FILE: pkg/resolve/resolve.go
================================================
package resolve
import (
"fmt"
"sync"
"github.com/rs/xid"
)
const (
maxWildcardChecks = 3
)
// ResolutionPool is a pool of resolvers created for resolving subdomains
// for a given host.
type ResolutionPool struct {
*Resolver
Tasks chan HostEntry
Results chan Result
wg *sync.WaitGroup
removeWildcard bool
wildcardIPs map[string]struct{}
}
// HostEntry defines a host with the source
type HostEntry struct {
Domain string
Host string
Source string
WildcardCertificate bool
}
// Result contains the result for a host resolution
type Result struct {
Type ResultType
Host string
IP string
Error error
Source string
WildcardCertificate bool
}
// ResultType is the type of result found
type ResultType int
// Types of data result can return
const (
Subdomain ResultType = iota
Error
)
// NewResolutionPool creates a pool of resolvers for resolving subdomains of a given domain
func (r *Resolver) NewResolutionPool(workers int, removeWildcard bool) *ResolutionPool {
resolutionPool := &ResolutionPool{
Resolver: r,
Tasks: make(chan HostEntry),
Results: make(chan Result),
wg: &sync.WaitGroup{},
removeWildcard: removeWildcard,
wildcardIPs: make(map[string]struct{}),
}
go func() {
for range workers {
resolutionPool.wg.Add(1)
go resolutionPool.resolveWorker()
}
resolutionPool.wg.Wait()
close(resolutionPool.Results)
}()
return resolutionPool
}
// InitWildcards inits the wildcard ips array
func (r *ResolutionPool) InitWildcards(domain string) error {
for range maxWildcardChecks {
uid := xid.New().String()
hosts, _ := r.DNSClient.Lookup(uid + "." + domain)
if len(hosts) == 0 {
return fmt.Errorf("%s is not a wildcard domain", domain)
}
// Append all wildcard ips found for domains
for _, host := range hosts {
r.wildcardIPs[host] = struct{}{}
}
}
return nil
}
func (r *ResolutionPool) resolveWorker() {
for task := range r.Tasks {
if !r.removeWildcard {
r.Results <- Result{Type: Subdomain, Host: task.Host, IP: "", Source: task.Source, WildcardCertificate: task.WildcardCertificate}
continue
}
hosts, err := r.DNSClient.Lookup(task.Host)
if err != nil {
r.Results <- Result{Type: Error, Host: task.Host, Source: task.Source, Error: err, WildcardCertificate: task.WildcardCertificate}
continue
}
if len(hosts) == 0 {
continue
}
var skip bool
for _, host := range hosts {
// Ignore the host if it exists in wildcard ips map
if _, ok := r.wildcardIPs[host]; ok {
skip = true
break
}
}
if !skip {
r.Results <- Result{Type: Subdomain, Host: task.Host, IP: hosts[0], Source: task.Source, WildcardCertificate: task.WildcardCertificate}
}
}
r.wg.Done()
}
================================================
FILE: pkg/runner/banners.go
================================================
package runner
import (
"github.com/projectdiscovery/gologger"
updateutils "github.com/projectdiscovery/utils/update"
)
const banner = `
__ _____ __
_______ __/ /_ / __(_)___ ____/ /__ _____
/ ___/ / / / __ \/ /_/ / __ \/ __ / _ \/ ___/
(__ ) /_/ / /_/ / __/ / / / / /_/ / __/ /
/____/\__,_/_.___/_/ /_/_/ /_/\__,_/\___/_/
`
// Name
const ToolName = `subfinder`
// Version is the current version of subfinder
const version = `v2.13.0`
// showBanner is used to show the banner to the user
func showBanner() {
gologger.Print().Msgf("%s\n", banner)
gologger.Print().Msgf("\t\tprojectdiscovery.io\n\n")
}
// GetUpdateCallback returns a callback function that updates subfinder
func GetUpdateCallback() func() {
return func() {
showBanner()
updateutils.GetUpdateToolCallback("subfinder", version)()
}
}
================================================
FILE: pkg/runner/config.go
================================================
package runner
import (
"os"
"strings"
"gopkg.in/yaml.v3"
"github.com/projectdiscovery/gologger"
"github.com/projectdiscovery/subfinder/v2/pkg/passive"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping"
fileutil "github.com/projectdiscovery/utils/file"
)
// createProviderConfigYAML marshals the input map to the given location on the disk
func createProviderConfigYAML(configFilePath string) error {
configFile, err := os.Create(configFilePath)
if err != nil {
return err
}
defer func() {
if err := configFile.Close(); err != nil {
gologger.Error().Msgf("Error closing config file: %s", err)
}
}()
sourcesRequiringApiKeysMap := make(map[string][]string)
for _, source := range passive.AllSources {
keyReq := source.KeyRequirement()
if keyReq == subscraping.RequiredKey || keyReq == subscraping.OptionalKey {
sourceName := strings.ToLower(source.Name())
sourcesRequiringApiKeysMap[sourceName] = []string{}
}
}
return yaml.NewEncoder(configFile).Encode(sourcesRequiringApiKeysMap)
}
// UnmarshalFrom writes the marshaled yaml config to disk
func UnmarshalFrom(file string) error {
reader, err := fileutil.SubstituteConfigFromEnvVars(file)
if err != nil {
return err
}
sourceApiKeysMap := map[string][]string{}
err = yaml.NewDecoder(reader).Decode(sourceApiKeysMap)
for _, source := range passive.AllSources {
sourceName := strings.ToLower(source.Name())
apiKeys := sourceApiKeysMap[sourceName]
if len(apiKeys) > 0 {
gologger.Debug().Msgf("API key(s) found for %s.", sourceName)
source.AddApiKeys(apiKeys)
}
}
return err
}
================================================
FILE: pkg/runner/doc.go
================================================
// Package runner implements the mechanism to drive the
// subdomain enumeration process
package runner
================================================
FILE: pkg/runner/enumerate.go
================================================
package runner
import (
"context"
"io"
"strings"
"sync"
"time"
"github.com/hako/durafmt"
"github.com/projectdiscovery/gologger"
"github.com/projectdiscovery/subfinder/v2/pkg/passive"
"github.com/projectdiscovery/subfinder/v2/pkg/resolve"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping"
)
const maxNumCount = 2
var replacer = strings.NewReplacer(
"/", "",
"•.", "",
"•", "",
"*.", "",
"http://", "",
"https://", "",
)
// EnumerateSingleDomain wraps EnumerateSingleDomainWithCtx with an empty context
func (r *Runner) EnumerateSingleDomain(domain string, writers []io.Writer) (map[string]map[string]struct{}, error) {
return r.EnumerateSingleDomainWithCtx(context.Background(), domain, writers)
}
// EnumerateSingleDomainWithCtx performs subdomain enumeration against a single domain
func (r *Runner) EnumerateSingleDomainWithCtx(ctx context.Context, domain string, writers []io.Writer) (map[string]map[string]struct{}, error) {
gologger.Info().Msgf("Enumerating subdomains for %s\n", domain)
// Check if the user has asked to remove wildcards explicitly.
// If yes, create the resolution pool and get the wildcards for the current domain
var resolutionPool *resolve.ResolutionPool
if r.options.RemoveWildcard {
resolutionPool = r.resolverClient.NewResolutionPool(r.options.Threads, r.options.RemoveWildcard)
err := resolutionPool.InitWildcards(domain)
if err != nil {
// Log the error but don't quit.
gologger.Warning().Msgf("Could not get wildcards for domain %s: %s\n", domain, err)
}
}
// Run the passive subdomain enumeration
now := time.Now()
passiveResults := r.passiveAgent.EnumerateSubdomainsWithCtx(ctx, domain, r.options.Proxy, r.options.RateLimit, r.options.Timeout, time.Duration(r.options.MaxEnumerationTime)*time.Minute, passive.WithCustomRateLimit(r.rateLimit))
wg := &sync.WaitGroup{}
wg.Add(1)
// Create a unique map for filtering duplicate subdomains out
uniqueMap := make(map[string]resolve.HostEntry)
// Create a map to track sources for each host
sourceMap := make(map[string]map[string]struct{})
skippedCounts := make(map[string]int)
// Process the results in a separate goroutine
go func() {
for result := range passiveResults {
switch result.Type {
case subscraping.Error:
gologger.Warning().Msgf("Encountered an error with source %s: %s\n", result.Source, result.Error)
case subscraping.Subdomain:
subdomain := replacer.Replace(result.Value)
// check if this subdomain is actually a wildcard subdomain
// that may have furthur subdomains associated with it
isWildcard := strings.Contains(result.Value, "*."+subdomain)
// Validate the subdomain found and remove wildcards from
if !strings.HasSuffix(subdomain, "."+domain) {
skippedCounts[result.Source]++
continue
}
if matchSubdomain := r.filterAndMatchSubdomain(subdomain); matchSubdomain {
if _, ok := uniqueMap[subdomain]; !ok {
sourceMap[subdomain] = make(map[string]struct{})
}
// Log the verbose message about the found subdomain per source
if _, ok := sourceMap[subdomain][result.Source]; !ok {
gologger.Verbose().Label(result.Source).Msg(subdomain)
}
sourceMap[subdomain][result.Source] = struct{}{}
// Check if the subdomain is a duplicate. If not,
// send the subdomain for resolution.
if _, ok := uniqueMap[subdomain]; ok {
skippedCounts[result.Source]++
// even if it is duplicate if it was not marked as wildcard before but this source says it is wildcard
// then we should mark it as wildcard
if !uniqueMap[subdomain].WildcardCertificate && isWildcard {
val := uniqueMap[subdomain]
val.WildcardCertificate = true
uniqueMap[subdomain] = val
}
continue
}
hostEntry := resolve.HostEntry{Domain: domain, Host: subdomain, Source: result.Source, WildcardCertificate: isWildcard}
if r.options.ResultCallback != nil && !r.options.RemoveWildcard {
r.options.ResultCallback(&hostEntry)
}
uniqueMap[subdomain] = hostEntry
// If the user asked to remove wildcard then send on the resolve
// queue. Otherwise, if mode is not verbose print the results on
// the screen as they are discovered.
if r.options.RemoveWildcard {
resolutionPool.Tasks <- hostEntry
}
}
}
}
// Close the task channel only if wildcards are asked to be removed
if r.options.RemoveWildcard {
close(resolutionPool.Tasks)
}
wg.Done()
}()
// If the user asked to remove wildcards, listen from the results
// queue and write to the map. At the end, print the found results to the screen
foundResults := make(map[string]resolve.Result)
if r.options.RemoveWildcard {
// Process the results coming from the resolutions pool
for result := range resolutionPool.Results {
switch result.Type {
case resolve.Error:
gologger.Warning().Msgf("Could not resolve host: %s\n", result.Error)
case resolve.Subdomain:
// Add the found subdomain to a map.
if _, ok := foundResults[result.Host]; !ok {
foundResults[result.Host] = result
if r.options.ResultCallback != nil {
r.options.ResultCallback(&resolve.HostEntry{Domain: domain, Host: result.Host, Source: result.Source, WildcardCertificate: result.WildcardCertificate})
}
}
}
}
// Merge wildcard certificate information from uniqueMap into foundResults
// This handles cases where a later source marked a subdomain as wildcard
// after it was already sent to the resolution pool
for host, result := range foundResults {
if entry, ok := uniqueMap[host]; ok && entry.WildcardCertificate && !result.WildcardCertificate {
result.WildcardCertificate = true
foundResults[host] = result
}
}
}
wg.Wait()
outputWriter := NewOutputWriter(r.options.JSON)
// Now output all results in output writers
var err error
for _, writer := range writers {
if r.options.HostIP {
err = outputWriter.WriteHostIP(domain, foundResults, writer)
} else {
if r.options.RemoveWildcard {
err = outputWriter.WriteHostNoWildcard(domain, foundResults, writer)
} else {
if r.options.CaptureSources {
err = outputWriter.WriteSourceHost(domain, sourceMap, writer)
} else {
err = outputWriter.WriteHost(domain, uniqueMap, writer)
}
}
}
if err != nil {
gologger.Error().Msgf("Could not write results for %s: %s\n", domain, err)
return nil, err
}
}
// Show found subdomain count in any case.
duration := durafmt.Parse(time.Since(now)).LimitFirstN(maxNumCount).String()
var numberOfSubDomains int
if r.options.RemoveWildcard {
numberOfSubDomains = len(foundResults)
} else {
numberOfSubDomains = len(uniqueMap)
}
gologger.Info().Msgf("Found %d subdomains for %s in %s\n", numberOfSubDomains, domain, duration)
if r.options.Statistics {
gologger.Info().Msgf("Printing source statistics for %s", domain)
statistics := r.passiveAgent.GetStatistics()
// This is a hack to remove the skipped count from the statistics
// as we don't want to show it in the statistics.
// TODO: Design a better way to do this.
for source, count := range skippedCounts {
if stat, ok := statistics[source]; ok {
stat.Results -= count
statistics[source] = stat
}
}
printStatistics(statistics)
}
return sourceMap, nil
}
func (r *Runner) filterAndMatchSubdomain(subdomain string) bool {
if r.options.filterRegexes != nil {
for _, filter := range r.options.filterRegexes {
if m := filter.MatchString(subdomain); m {
return false
}
}
}
if r.options.matchRegexes != nil {
for _, match := range r.options.matchRegexes {
if m := match.MatchString(subdomain); m {
return true
}
}
return false
}
return true
}
================================================
FILE: pkg/runner/enumerate_test.go
================================================
package runner
import (
"os"
"testing"
"github.com/stretchr/testify/require"
)
func TestFilterAndMatchSubdomain(t *testing.T) {
options := &Options{}
options.Domain = []string{"example.com"}
options.Threads = 10
options.Timeout = 10
options.Output = os.Stdout
t.Run("Literal Match", func(t *testing.T) {
options.Match = []string{"req.example.com"}
err := options.validateOptions()
if err != nil {
t.Fatalf("Expected nil got %v while validation\n", err)
}
runner, err := NewRunner(options)
if err != nil {
t.Fatalf("Expected nil got %v while creating runner\n", err)
}
match := runner.filterAndMatchSubdomain("req.example.com")
require.True(t, match, "Expecting a boolean True value ")
})
t.Run("Multiple Wildcards Match", func(t *testing.T) {
options.Match = []string{"*.ns.*.com"}
err := options.validateOptions()
if err != nil {
t.Fatalf("Expected nil got %v while validation\n", err)
}
runner, err := NewRunner(options)
if err != nil {
t.Fatalf("Expected nil got %v while creating runner\n", err)
}
subdomain := []string{"a.ns.example.com", "b.ns.hackerone.com"}
for _, sub := range subdomain {
match := runner.filterAndMatchSubdomain(sub)
require.True(t, match, "Expecting a boolean True value ")
}
})
t.Run("Sequential Match", func(t *testing.T) {
options.Match = []string{"*.ns.example.com", "*.hackerone.com"}
err := options.validateOptions()
if err != nil {
t.Fatalf("Expected nil got %v while validation\n", err)
}
runner, err := NewRunner(options)
if err != nil {
t.Fatalf("Expected nil got %v while creating runner\n", err)
}
subdomain := []string{"a.ns.example.com", "b.hackerone.com"}
for _, sub := range subdomain {
match := runner.filterAndMatchSubdomain(sub)
require.True(t, match, "Expecting a boolean True value ")
}
})
t.Run("Literal Filter", func(t *testing.T) {
options.Filter = []string{"req.example.com"}
err := options.validateOptions()
if err != nil {
t.Fatalf("Expected nil got %v while validation\n", err)
}
runner, err := NewRunner(options)
if err != nil {
t.Fatalf("Expected nil got %v while creating runner\n", err)
}
match := runner.filterAndMatchSubdomain("req.example.com")
require.False(t, match, "Expecting a boolean False value ")
})
t.Run("Multiple Wildcards Filter", func(t *testing.T) {
options.Filter = []string{"*.ns.*.com"}
err := options.validateOptions()
if err != nil {
t.Fatalf("Expected nil got %v while validation\n", err)
}
runner, err := NewRunner(options)
if err != nil {
t.Fatalf("Expected nil got %v while creating runner\n", err)
}
subdomain := []string{"a.ns.example.com", "b.ns.hackerone.com"}
for _, sub := range subdomain {
match := runner.filterAndMatchSubdomain(sub)
require.False(t, match, "Expecting a boolean False value ")
}
})
t.Run("Sequential Filter", func(t *testing.T) {
options.Filter = []string{"*.ns.example.com", "*.hackerone.com"}
err := options.validateOptions()
if err != nil {
t.Fatalf("Expected nil got %v while validation\n", err)
}
runner, err := NewRunner(options)
if err != nil {
t.Fatalf("Expected nil got %v while creating runner\n", err)
}
subdomain := []string{"a.ns.example.com", "b.hackerone.com"}
for _, sub := range subdomain {
match := runner.filterAndMatchSubdomain(sub)
require.False(t, match, "Expecting a boolean False value ")
}
})
t.Run("Filter and Match", func(t *testing.T) {
options.Filter = []string{"example.com"}
options.Match = []string{"hackerone.com"}
err := options.validateOptions()
if err != nil {
t.Fatalf("Expected nil got %v while validation\n", err)
}
runner, err := NewRunner(options)
if err != nil {
t.Fatalf("Expected nil got %v while creating runner\n", err)
}
subdomain := []string{"example.com", "example.com"}
for _, sub := range subdomain {
match := runner.filterAndMatchSubdomain(sub)
require.False(t, match, "Expecting a boolean False value ")
}
})
t.Run("Filter and Match - Same Root Domain", func(t *testing.T) {
options.Filter = []string{"example.com"}
options.Match = []string{"www.example.com"}
err := options.validateOptions()
if err != nil {
t.Fatalf("Expected nil got %v while validation\n", err)
}
runner, err := NewRunner(options)
if err != nil {
t.Fatalf("Expected nil got %v while creating runner\n", err)
}
subdomain := map[string]string{"filter": "example.com", "match": "www.example.com"}
for key, sub := range subdomain {
result := runner.filterAndMatchSubdomain(sub)
if key == "filter" {
require.False(t, result, "Expecting a boolean False value ")
} else {
require.True(t, result, "Expecting a boolean True value ")
}
}
})
}
================================================
FILE: pkg/runner/initialize.go
================================================
package runner
import (
"net"
"strings"
"github.com/projectdiscovery/dnsx/libs/dnsx"
"github.com/projectdiscovery/subfinder/v2/pkg/passive"
"github.com/projectdiscovery/subfinder/v2/pkg/resolve"
)
// initializePassiveEngine creates the passive engine and loads sources etc
func (r *Runner) initializePassiveEngine() {
r.passiveAgent = passive.New(r.options.Sources, r.options.ExcludeSources, r.options.All, r.options.OnlyRecursive)
}
// initializeResolver creates the resolver used to resolve the found subdomains
func (r *Runner) initializeResolver() error {
var resolvers []string
// If the file has been provided, read resolvers from the file
if r.options.ResolverList != "" {
var err error
resolvers, err = loadFromFile(r.options.ResolverList)
if err != nil {
return err
}
}
if len(r.options.Resolvers) > 0 {
resolvers = append(resolvers, r.options.Resolvers...)
} else {
resolvers = append(resolvers, resolve.DefaultResolvers...)
}
// Add default 53 UDP port if missing
for i, resolver := range resolvers {
if !strings.Contains(resolver, ":") {
resolvers[i] = net.JoinHostPort(resolver, "53")
}
}
r.resolverClient = resolve.New()
var err error
r.resolverClient.DNSClient, err = dnsx.New(dnsx.Options{BaseResolvers: resolvers, MaxRetries: 5})
if err != nil {
return nil
}
return nil
}
================================================
FILE: pkg/runner/options.go
================================================
package runner
import (
"errors"
"fmt"
"io"
"math"
"os"
"path/filepath"
"regexp"
"strings"
"github.com/projectdiscovery/chaos-client/pkg/chaos"
"github.com/projectdiscovery/goflags"
"github.com/projectdiscovery/gologger"
"github.com/projectdiscovery/subfinder/v2/pkg/passive"
"github.com/projectdiscovery/subfinder/v2/pkg/resolve"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping"
envutil "github.com/projectdiscovery/utils/env"
fileutil "github.com/projectdiscovery/utils/file"
folderutil "github.com/projectdiscovery/utils/folder"
logutil "github.com/projectdiscovery/utils/log"
updateutils "github.com/projectdiscovery/utils/update"
)
var (
configDir = folderutil.AppConfigDirOrDefault(".", "subfinder")
defaultConfigLocation = envutil.GetEnvOrDefault("SUBFINDER_CONFIG", filepath.Join(configDir, "config.yaml"))
defaultProviderConfigLocation = envutil.GetEnvOrDefault("SUBFINDER_PROVIDER_CONFIG", filepath.Join(configDir, "provider-config.yaml"))
)
// Options contains the configuration options for tuning
// the subdomain enumeration process.
type Options struct {
Verbose bool // Verbose flag indicates whether to show verbose output or not
NoColor bool // NoColor disables the colored output
JSON bool // JSON specifies whether to use json for output format or text file
HostIP bool // HostIP specifies whether to write subdomains in host:ip format
Silent bool // Silent suppresses any extra text and only writes subdomains to screen
ListSources bool // ListSources specifies whether to list all available sources
RemoveWildcard bool // RemoveWildcard specifies whether to remove potential wildcard or dead subdomains from the results.
CaptureSources bool // CaptureSources specifies whether to save all sources that returned a specific domains or just the first source
Stdin bool // Stdin specifies whether stdin input was given to the process
Version bool // Version specifies if we should just show version and exit
OnlyRecursive bool // Recursive specifies whether to use only recursive subdomain enumeration sources
All bool // All specifies whether to use all (slow) sources.
Statistics bool // Statistics specifies whether to report source statistics
Threads int // Threads controls the number of threads to use for active enumerations
Timeout int // Timeout is the seconds to wait for sources to respond
MaxEnumerationTime int // MaxEnumerationTime is the maximum amount of time in minutes to wait for enumeration
Domain goflags.StringSlice // Domain is the domain to find subdomains for
DomainsFile string // DomainsFile is the file containing list of domains to find subdomains for
Output io.Writer
OutputFile string // Output is the file to write found subdomains to.
OutputDirectory string // OutputDirectory is the directory to write results to in case list of domains is given
Sources goflags.StringSlice `yaml:"sources,omitempty"` // Sources contains a comma-separated list of sources to use for enumeration
ExcludeSources goflags.StringSlice `yaml:"exclude-sources,omitempty"` // ExcludeSources contains the comma-separated sources to not include in the enumeration process
Resolvers goflags.StringSlice `yaml:"resolvers,omitempty"` // Resolvers is the comma-separated resolvers to use for enumeration
ResolverList string // ResolverList is a text file containing list of resolvers to use for enumeration
Config string // Config contains the location of the config file
ProviderConfig string // ProviderConfig contains the location of the provider config file
Proxy string // HTTP proxy
RateLimit int // Global maximum number of HTTP requests to send per second
RateLimits goflags.RateLimitMap // Maximum number of HTTP requests to send per second
ExcludeIps bool
Match goflags.StringSlice
Filter goflags.StringSlice
matchRegexes []*regexp.Regexp
filterRegexes []*regexp.Regexp
ResultCallback OnResultCallback // OnResult callback
DisableUpdateCheck bool // DisableUpdateCheck disable update checking
}
// OnResultCallback (hostResult)
type OnResultCallback func(result *resolve.HostEntry)
// ParseOptions parses the command line flags provided by a user
func ParseOptions() *Options {
logutil.DisableDefaultLogger()
options := &Options{}
var err error
flagSet := goflags.NewFlagSet()
flagSet.SetDescription(`Subfinder is a subdomain discovery tool that discovers subdomains for websites by using passive online sources.`)
flagSet.CreateGroup("input", "Input",
flagSet.StringSliceVarP(&options.Domain, "domain", "d", nil, "domains to find subdomains for", goflags.NormalizedStringSliceOptions),
flagSet.StringVarP(&options.DomainsFile, "list", "dL", "", "file containing list of domains for subdomain discovery"),
)
flagSet.CreateGroup("source", "Source",
flagSet.StringSliceVarP(&options.Sources, "sources", "s", nil, "specific sources to use for discovery (-s crtsh,github). Use -ls to display all available sources.", goflags.NormalizedStringSliceOptions),
flagSet.BoolVar(&options.OnlyRecursive, "recursive", false, "use only sources that can handle subdomains recursively rather than both recursive and non-recursive sources"),
flagSet.BoolVar(&options.All, "all", false, "use all sources for enumeration (slow)"),
flagSet.StringSliceVarP(&options.ExcludeSources, "exclude-sources", "es", nil, "sources to exclude from enumeration (-es alienvault,zoomeyeapi)", goflags.NormalizedStringSliceOptions),
)
flagSet.CreateGroup("filter", "Filter",
flagSet.StringSliceVarP(&options.Match, "match", "m", nil, "subdomain or list of subdomain to match (file or comma separated)", goflags.FileNormalizedStringSliceOptions),
flagSet.StringSliceVarP(&options.Filter, "filter", "f", nil, " subdomain or list of subdomain to filter (file or comma separated)", goflags.FileNormalizedStringSliceOptions),
)
flagSet.CreateGroup("rate-limit", "Rate-limit",
flagSet.IntVarP(&options.RateLimit, "rate-limit", "rl", 0, "maximum number of http requests to send per second (global)"),
flagSet.RateLimitMapVarP(&options.RateLimits, "rate-limits", "rls", defaultRateLimits, "maximum number of http requests to send per second for providers in key=value format (-rls hackertarget=10/m)", goflags.NormalizedStringSliceOptions),
flagSet.IntVar(&options.Threads, "t", 10, "number of concurrent goroutines for resolving (-active only)"),
)
flagSet.CreateGroup("update", "Update",
flagSet.CallbackVarP(GetUpdateCallback(), "update", "up", "update subfinder to latest version"),
flagSet.BoolVarP(&options.DisableUpdateCheck, "disable-update-check", "duc", false, "disable automatic subfinder update check"),
)
flagSet.CreateGroup("output", "Output",
flagSet.StringVarP(&options.OutputFile, "output", "o", "", "file to write output to"),
flagSet.BoolVarP(&options.JSON, "json", "oJ", false, "write output in JSONL(ines) format"),
flagSet.StringVarP(&options.OutputDirectory, "output-dir", "oD", "", "directory to write output (-dL only)"),
flagSet.BoolVarP(&options.CaptureSources, "collect-sources", "cs", false, "include all sources in the output (-json only)"),
flagSet.BoolVarP(&options.HostIP, "ip", "oI", false, "include host IP in output (-active only)"),
)
flagSet.CreateGroup("configuration", "Configuration",
flagSet.StringVar(&options.Config, "config", defaultConfigLocation, "flag config file"),
flagSet.StringVarP(&options.ProviderConfig, "provider-config", "pc", defaultProviderConfigLocation, "provider config file"),
flagSet.StringSliceVar(&options.Resolvers, "r", nil, "comma separated list of resolvers to use", goflags.NormalizedStringSliceOptions),
flagSet.StringVarP(&options.ResolverList, "rlist", "rL", "", "file containing list of resolvers to use"),
flagSet.BoolVarP(&options.RemoveWildcard, "active", "nW", false, "display active subdomains only"),
flagSet.StringVar(&options.Proxy, "proxy", "", "http proxy to use with subfinder"),
flagSet.BoolVarP(&options.ExcludeIps, "exclude-ip", "ei", false, "exclude IPs from the list of domains"),
)
flagSet.CreateGroup("debug", "Debug",
flagSet.BoolVar(&options.Silent, "silent", false, "show only subdomains in output"),
flagSet.BoolVar(&options.Version, "version", false, "show version of subfinder"),
flagSet.BoolVar(&options.Verbose, "v", false, "show verbose output"),
flagSet.BoolVarP(&options.NoColor, "no-color", "nc", false, "disable color in output"),
flagSet.BoolVarP(&options.ListSources, "list-sources", "ls", false, "list all available sources"),
flagSet.BoolVar(&options.Statistics, "stats", false, "report source statistics"),
)
flagSet.CreateGroup("optimization", "Optimization",
flagSet.IntVar(&options.Timeout, "timeout", 30, "seconds to wait before timing out"),
flagSet.IntVar(&options.MaxEnumerationTime, "max-time", 10, "minutes to wait for enumeration results"),
)
if err := flagSet.Parse(); err != nil {
fmt.Println(err.Error())
os.Exit(1)
}
// set chaos mode
chaos.IsSDK = false
if exists := fileutil.FileExists(defaultProviderConfigLocation); !exists {
if err := createProviderConfigYAML(defaultProviderConfigLocation); err != nil {
gologger.Error().Msgf("Could not create provider config file: %s\n", err)
}
}
if options.Config != defaultConfigLocation {
// An empty source file is not a fatal error
if err := flagSet.MergeConfigFile(options.Config); err != nil && !errors.Is(err, io.EOF) {
gologger.Fatal().Msgf("Could not read config: %s\n", err)
}
}
// Default output is stdout
options.Output = os.Stdout
// Check if stdin pipe was given
options.Stdin = fileutil.HasStdin()
if options.Version {
gologger.Info().Msgf("Current Version: %s\n", version)
gologger.Info().Msgf("Subfinder Config Directory: %s", configDir)
os.Exit(0)
}
options.preProcessDomains()
options.ConfigureOutput()
showBanner()
if !options.DisableUpdateCheck {
latestVersion, err := updateutils.GetToolVersionCallback("subfinder", version)()
if err != nil {
if options.Verbose {
gologger.Error().Msgf("subfinder version check failed: %v", err.Error())
}
} else {
gologger.Info().Msgf("Current subfinder version %v %v", version, updateutils.GetVersionDescription(version, latestVersion))
}
}
if options.ListSources {
listSources(options)
os.Exit(0)
}
// Validate the options passed by the user and if any
// invalid options have been used, exit.
err = options.validateOptions()
if err != nil {
gologger.Fatal().Msgf("Program exiting: %s\n", err)
}
return options
}
// loadProvidersFrom runs the app with source config
func (options *Options) loadProvidersFrom(location string) {
// todo: move elsewhere
if len(options.Resolvers) == 0 {
options.Resolvers = resolve.DefaultResolvers
}
// We skip bailing out if file doesn't exist because we'll create it
// at the end of options parsing from default via goflags.
if err := UnmarshalFrom(location); err != nil && (!strings.Contains(err.Error(), "file doesn't exist") || errors.Is(err, os.ErrNotExist)) {
gologger.Error().Msgf("Could not read providers from %s: %s\n", location, err)
}
}
func listSources(options *Options) {
gologger.Info().Msgf("Current list of available sources. [%d]\n", len(passive.AllSources))
gologger.Info().Msgf("Sources marked with an * require key(s) or token(s) to work.\n")
gologger.Info().Msgf("Sources marked with a ~ optionally support key(s) for better results.\n")
gologger.Info().Msgf("You can modify %s to configure your keys/tokens.\n\n", options.ProviderConfig)
for _, source := range passive.AllSources {
sourceName := source.Name()
switch source.KeyRequirement() {
case subscraping.RequiredKey:
gologger.Silent().Msgf("%s *\n", sourceName)
case subscraping.OptionalKey:
gologger.Silent().Msgf("%s ~\n", sourceName)
default:
gologger.Silent().Msgf("%s\n", sourceName)
}
}
}
func (options *Options) preProcessDomains() {
for i, domain := range options.Domain {
options.Domain[i] = preprocessDomain(domain)
}
}
var defaultRateLimits = []string{
"github=30/m",
"fullhunt=60/m",
"pugrecon=10/s",
fmt.Sprintf("robtex=%d/ms", uint(math.MaxUint)),
"securitytrails=1/s",
"shodan=1/s",
"virustotal=4/m",
"hackertarget=2/s",
// "threatminer=10/m",
"waybackarchive=15/m",
"whoisxmlapi=50/s",
"securitytrails=2/s",
"sitedossier=8/m",
"netlas=1/s",
// "gitlab=2/s",
"github=83/m",
"hudsonrock=5/s",
"urlscan=1/s",
}
================================================
FILE: pkg/runner/outputter.go
================================================
package runner
import (
"bufio"
"errors"
"io"
"os"
"path/filepath"
"strings"
jsoniter "github.com/json-iterator/go"
"github.com/projectdiscovery/subfinder/v2/pkg/resolve"
)
// OutputWriter outputs content to writers.
type OutputWriter struct {
JSON bool
}
type jsonSourceResult struct {
Host string `json:"host"`
Input string `json:"input"`
Source string `json:"source"`
WildcardCertificate bool `json:"wildcard_certificate,omitempty"`
}
type jsonSourceIPResult struct {
Host string `json:"host"`
IP string `json:"ip"`
Input string `json:"input"`
Source string `json:"source"`
WildcardCertificate bool `json:"wildcard_certificate,omitempty"`
}
type jsonSourcesResult struct {
Host string `json:"host"`
Input string `json:"input"`
Sources []string `json:"sources"`
WildcardCertificate bool `json:"wildcard_certificate,omitempty"`
}
// NewOutputWriter creates a new OutputWriter
func NewOutputWriter(json bool) *OutputWriter {
return &OutputWriter{JSON: json}
}
func (o *OutputWriter) createFile(filename string, appendToFile bool) (*os.File, error) {
if filename == "" {
return nil, errors.New("empty filename")
}
dir := filepath.Dir(filename)
if dir != "" {
if _, err := os.Stat(dir); os.IsNotExist(err) {
err := os.MkdirAll(dir, os.ModePerm)
if err != nil {
return nil, err
}
}
}
var file *os.File
var err error
if appendToFile {
file, err = os.OpenFile(filename, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)
} else {
file, err = os.Create(filename)
}
if err != nil {
return nil, err
}
return file, nil
}
// WriteHostIP writes the output list of subdomain to an io.Writer
func (o *OutputWriter) WriteHostIP(input string, results map[string]resolve.Result, writer io.Writer) error {
var err error
if o.JSON {
err = writeJSONHostIP(input, results, writer)
} else {
err = writePlainHostIP(input, results, writer)
}
return err
}
func writePlainHostIP(_ string, results map[string]resolve.Result, writer io.Writer) error {
bufwriter := bufio.NewWriter(writer)
sb := &strings.Builder{}
for _, result := range results {
sb.WriteString(result.Host)
sb.WriteString(",")
sb.WriteString(result.IP)
sb.WriteString(",")
sb.WriteString(result.Source)
sb.WriteString("\n")
_, err := bufwriter.WriteString(sb.String())
if err != nil {
if flushErr := bufwriter.Flush(); flushErr != nil {
return errors.Join(err, flushErr)
}
return err
}
sb.Reset()
}
return bufwriter.Flush()
}
func writeJSONHostIP(input string, results map[string]resolve.Result, writer io.Writer) error {
encoder := jsoniter.NewEncoder(writer)
var data jsonSourceIPResult
for _, result := range results {
data.Host = result.Host
data.IP = result.IP
data.Input = input
data.Source = result.Source
data.WildcardCertificate = result.WildcardCertificate
err := encoder.Encode(&data)
if err != nil {
return err
}
}
return nil
}
// WriteHostNoWildcard writes the output list of subdomain with nW flag to an io.Writer
func (o *OutputWriter) WriteHostNoWildcard(input string, results map[string]resolve.Result, writer io.Writer) error {
hosts := make(map[string]resolve.HostEntry)
for host, result := range results {
hosts[host] = resolve.HostEntry{Domain: host, Host: result.Host, Source: result.Source, WildcardCertificate: result.WildcardCertificate}
}
return o.WriteHost(input, hosts, writer)
}
// WriteHost writes the output list of subdomain to an io.Writer
func (o *OutputWriter) WriteHost(input string, results map[string]resolve.HostEntry, writer io.Writer) error {
var err error
if o.JSON {
err = writeJSONHost(input, results, writer)
} else {
err = writePlainHost(input, results, writer)
}
return err
}
func writePlainHost(_ string, results map[string]resolve.HostEntry, writer io.Writer) error {
bufwriter := bufio.NewWriter(writer)
sb := &strings.Builder{}
for _, result := range results {
sb.WriteString(result.Host)
sb.WriteString("\n")
_, err := bufwriter.WriteString(sb.String())
if err != nil {
if flushErr := bufwriter.Flush(); flushErr != nil {
return errors.Join(err, flushErr)
}
return err
}
sb.Reset()
}
return bufwriter.Flush()
}
func writeJSONHost(input string, results map[string]resolve.HostEntry, writer io.Writer) error {
encoder := jsoniter.NewEncoder(writer)
var data jsonSourceResult
for _, result := range results {
data.Host = result.Host
data.Input = input
data.Source = result.Source
data.WildcardCertificate = result.WildcardCertificate
err := encoder.Encode(data)
if err != nil {
return err
}
}
return nil
}
// WriteSourceHost writes the output list of subdomain to an io.Writer
func (o *OutputWriter) WriteSourceHost(input string, sourceMap map[string]map[string]struct{}, writer io.Writer) error {
var err error
if o.JSON {
err = writeSourceJSONHost(input, sourceMap, writer)
} else {
err = writeSourcePlainHost(input, sourceMap, writer)
}
return err
}
func writeSourceJSONHost(input string, sourceMap map[string]map[string]struct{}, writer io.Writer) error {
encoder := jsoniter.NewEncoder(writer)
var data jsonSourcesResult
for host, sources := range sourceMap {
data.Host = host
data.Input = input
keys := make([]string, 0, len(sources))
for source := range sources {
keys = append(keys, source)
}
data.Sources = keys
err := encoder.Encode(&data)
if err != nil {
return err
}
}
return nil
}
func writeSourcePlainHost(_ string, sourceMap map[string]map[string]struct{}, writer io.Writer) error {
bufwriter := bufio.NewWriter(writer)
sb := &strings.Builder{}
for host, sources := range sourceMap {
sb.WriteString(host)
sb.WriteString(",[")
var sourcesString strings.Builder
for source := range sources {
sourcesString.WriteString(source)
sourcesString.WriteRune(',')
}
sb.WriteString(strings.TrimSuffix(sourcesString.String(), ","))
sb.WriteString("]\n")
_, err := bufwriter.WriteString(sb.String())
if err != nil {
if flushErr := bufwriter.Flush(); flushErr != nil {
return errors.Join(err, flushErr)
}
return err
}
sb.Reset()
}
return bufwriter.Flush()
}
================================================
FILE: pkg/runner/runner.go
================================================
package runner
import (
"bufio"
"context"
"io"
"math"
"os"
"path"
"regexp"
"strconv"
"strings"
"github.com/projectdiscovery/gologger"
contextutil "github.com/projectdiscovery/utils/context"
fileutil "github.com/projectdiscovery/utils/file"
mapsutil "github.com/projectdiscovery/utils/maps"
"github.com/projectdiscovery/subfinder/v2/pkg/passive"
"github.com/projectdiscovery/subfinder/v2/pkg/resolve"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping"
)
// Runner is an instance of the subdomain enumeration
// client used to orchestrate the whole process.
type Runner struct {
options *Options
passiveAgent *passive.Agent
resolverClient *resolve.Resolver
rateLimit *subscraping.CustomRateLimit
}
// NewRunner creates a new runner struct instance by parsing
// the configuration options, configuring sources, reading lists
// and setting up loggers, etc.
func NewRunner(options *Options) (*Runner, error) {
options.ConfigureOutput()
runner := &Runner{options: options}
// Check if the application loading with any provider configuration, then take it
// Otherwise load the default provider config
if fileutil.FileExists(options.ProviderConfig) {
gologger.Info().Msgf("Loading provider config from %s", options.ProviderConfig)
options.loadProvidersFrom(options.ProviderConfig)
} else {
gologger.Info().Msgf("Loading provider config from the default location: %s", defaultProviderConfigLocation)
options.loadProvidersFrom(defaultProviderConfigLocation)
}
// Initialize the passive subdomain enumeration engine
runner.initializePassiveEngine()
// Initialize the subdomain resolver
err := runner.initializeResolver()
if err != nil {
return nil, err
}
// Initialize the custom rate limit
runner.rateLimit = &subscraping.CustomRateLimit{
Custom: mapsutil.SyncLockMap[string, uint]{
Map: make(map[string]uint),
},
}
for source, sourceRateLimit := range options.RateLimits.AsMap() {
if sourceRateLimit.MaxCount > 0 && sourceRateLimit.MaxCount <= math.MaxUint {
_ = runner.rateLimit.Custom.Set(source, sourceRateLimit.MaxCount)
}
}
return runner, nil
}
// RunEnumeration wraps RunEnumerationWithCtx with an empty context
func (r *Runner) RunEnumeration() error {
ctx, _ := contextutil.WithValues(context.Background(), contextutil.ContextArg("All"), contextutil.ContextArg(strconv.FormatBool(r.options.All)))
return r.RunEnumerationWithCtx(ctx)
}
// RunEnumerationWithCtx runs the subdomain enumeration flow on the targets specified
func (r *Runner) RunEnumerationWithCtx(ctx context.Context) error {
outputs := []io.Writer{r.options.Output}
if len(r.options.Domain) > 0 {
domainsReader := strings.NewReader(strings.Join(r.options.Domain, "\n"))
return r.EnumerateMultipleDomainsWithCtx(ctx, domainsReader, outputs)
}
// If we have multiple domains as input,
if r.options.DomainsFile != "" {
f, err := os.Open(r.options.DomainsFile)
if err != nil {
return err
}
err = r.EnumerateMultipleDomainsWithCtx(ctx, f, outputs)
if closeErr := f.Close(); closeErr != nil {
gologger.Error().Msgf("Error closing file %s: %s", r.options.DomainsFile, closeErr)
}
return err
}
// If we have STDIN input, treat it as multiple domains
if r.options.Stdin {
return r.EnumerateMultipleDomainsWithCtx(ctx, os.Stdin, outputs)
}
return nil
}
// EnumerateMultipleDomains wraps EnumerateMultipleDomainsWithCtx with an empty context
func (r *Runner) EnumerateMultipleDomains(reader io.Reader, writers []io.Writer) error {
ctx, _ := contextutil.WithValues(context.Background(), contextutil.ContextArg("All"), contextutil.ContextArg(strconv.FormatBool(r.options.All)))
return r.EnumerateMultipleDomainsWithCtx(ctx, reader, writers)
}
// EnumerateMultipleDomainsWithCtx enumerates subdomains for multiple domains
// We keep enumerating subdomains for a given domain until we reach an error
func (r *Runner) EnumerateMultipleDomainsWithCtx(ctx context.Context, reader io.Reader, writers []io.Writer) error {
var err error
scanner := bufio.NewScanner(reader)
ip, _ := regexp.Compile(`^([0-9\.]+$)`)
for scanner.Scan() {
domain := preprocessDomain(scanner.Text())
domain = replacer.Replace(domain)
if domain == "" || (r.options.ExcludeIps && ip.MatchString(domain)) {
continue
}
var file *os.File
// If the user has specified an output file, use that output file instead
// of creating a new output file for each domain. Else create a new file
// for each domain in the directory.
if r.options.OutputFile != "" {
outputWriter := NewOutputWriter(r.options.JSON)
file, err = outputWriter.createFile(r.options.OutputFile, true)
if err != nil {
gologger.Error().Msgf("Could not create file %s for %s: %s\n", r.options.OutputFile, r.options.Domain, err)
return err
}
_, err = r.EnumerateSingleDomainWithCtx(ctx, domain, append(writers, file))
if closeErr := file.Close(); closeErr != nil {
gologger.Error().Msgf("Error closing file %s: %s", r.options.OutputFile, closeErr)
}
} else if r.options.OutputDirectory != "" {
outputFile := path.Join(r.options.OutputDirectory, domain)
if r.options.JSON {
outputFile += ".json"
} else {
outputFile += ".txt"
}
outputWriter := NewOutputWriter(r.options.JSON)
file, err = outputWriter.createFile(outputFile, false)
if err != nil {
gologger.Error().Msgf("Could not create file %s for %s: %s\n", r.options.OutputFile, r.options.Domain, err)
return err
}
_, err = r.EnumerateSingleDomainWithCtx(ctx, domain, append(writers, file))
if closeErr := file.Close(); closeErr != nil {
gologger.Error().Msgf("Error closing file %s: %s", outputFile, closeErr)
}
} else {
_, err = r.EnumerateSingleDomainWithCtx(ctx, domain, writers)
}
if err != nil {
return err
}
}
return nil
}
================================================
FILE: pkg/runner/stats.go
================================================
package runner
import (
"fmt"
"sort"
"strings"
"time"
"github.com/projectdiscovery/gologger"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping"
"golang.org/x/exp/maps"
)
func printStatistics(stats map[string]subscraping.Statistics) {
sources := maps.Keys(stats)
sort.Strings(sources)
var lines []string
var skipped []string
for _, source := range sources {
sourceStats := stats[source]
if sourceStats.Skipped {
skipped = append(skipped, fmt.Sprintf(" %s", source))
} else {
lines = append(lines, fmt.Sprintf(" %-20s %-10s %10d %10d %10d", source, sourceStats.TimeTaken.Round(time.Millisecond).String(), sourceStats.Results, sourceStats.Requests, sourceStats.Errors))
}
}
if len(lines) > 0 {
gologger.Print().Msgf("\n Source Duration Results Requests Errors\n%s\n", strings.Repeat("─", 68))
gologger.Print().Msg(strings.Join(lines, "\n"))
gologger.Print().Msgf("\n")
}
if len(skipped) > 0 {
gologger.Print().Msgf("\n The following sources were included but skipped...\n\n")
gologger.Print().Msg(strings.Join(skipped, "\n"))
gologger.Print().Msgf("\n\n")
}
}
func (r *Runner) GetStatistics() map[string]subscraping.Statistics {
return r.passiveAgent.GetStatistics()
}
================================================
FILE: pkg/runner/util.go
================================================
package runner
import (
fileutil "github.com/projectdiscovery/utils/file"
stringsutil "github.com/projectdiscovery/utils/strings"
)
func loadFromFile(file string) ([]string, error) {
chanItems, err := fileutil.ReadFile(file)
if err != nil {
return nil, err
}
var items []string
for item := range chanItems {
item = preprocessDomain(item)
if item == "" {
continue
}
items = append(items, item)
}
return items, nil
}
func preprocessDomain(s string) string {
return stringsutil.NormalizeWithOptions(s,
stringsutil.NormalizeOptions{
StripComments: true,
TrimCutset: "\n\t\"'` ",
Lowercase: true,
},
)
}
================================================
FILE: pkg/runner/validate.go
================================================
package runner
import (
"errors"
"fmt"
"regexp"
"strings"
"github.com/projectdiscovery/gologger"
"github.com/projectdiscovery/gologger/formatter"
"github.com/projectdiscovery/gologger/levels"
"github.com/projectdiscovery/subfinder/v2/pkg/passive"
mapsutil "github.com/projectdiscovery/utils/maps"
sliceutil "github.com/projectdiscovery/utils/slice"
)
// validateOptions validates the configuration options passed
func (options *Options) validateOptions() error {
// Check if domain, list of domains, or stdin info was provided.
// If none was provided, then return.
if len(options.Domain) == 0 && options.DomainsFile == "" && !options.Stdin {
return errors.New("no input list provided")
}
// Both verbose and silent flags were used
if options.Verbose && options.Silent {
return errors.New("both verbose and silent mode specified")
}
// Validate threads and options
if options.Threads == 0 {
return errors.New("threads cannot be zero")
}
if options.Timeout == 0 {
return errors.New("timeout cannot be zero")
}
// Always remove wildcard with hostip
if options.HostIP && !options.RemoveWildcard {
return errors.New("hostip flag must be used with RemoveWildcard option")
}
if options.Match != nil {
options.matchRegexes = make([]*regexp.Regexp, len(options.Match))
var err error
for i, re := range options.Match {
if options.matchRegexes[i], err = regexp.Compile(stripRegexString(re)); err != nil {
return errors.New("invalid value for match regex option")
}
}
}
if options.Filter != nil {
options.filterRegexes = make([]*regexp.Regexp, len(options.Filter))
var err error
for i, re := range options.Filter {
if options.filterRegexes[i], err = regexp.Compile(stripRegexString(re)); err != nil {
return errors.New("invalid value for filter regex option")
}
}
}
sources := mapsutil.GetKeys(passive.NameSourceMap)
for source := range options.RateLimits.AsMap() {
if !sliceutil.Contains(sources, source) {
return fmt.Errorf("invalid source %s specified in -rls flag", source)
}
}
return nil
}
func stripRegexString(val string) string {
val = strings.ReplaceAll(val, ".", "\\.")
val = strings.ReplaceAll(val, "*", ".*")
return fmt.Sprint("^", val, "$")
}
// ConfigureOutput configures the output on the screen
func (options *Options) ConfigureOutput() {
// If the user desires verbose output, show verbose output
if options.Verbose {
gologger.DefaultLogger.SetMaxLevel(levels.LevelVerbose)
}
if options.NoColor {
gologger.DefaultLogger.SetFormatter(formatter.NewCLI(true))
}
if options.Silent {
gologger.DefaultLogger.SetMaxLevel(levels.LevelSilent)
}
}
================================================
FILE: pkg/subscraping/agent.go
================================================
package subscraping
import (
"bytes"
"context"
"crypto/tls"
"fmt"
"io"
"net"
"net/http"
"net/url"
"time"
"github.com/corpix/uarand"
"github.com/projectdiscovery/ratelimit"
"github.com/projectdiscovery/gologger"
)
// NewSession creates a new session object for a domain
func NewSession(domain string, proxy string, multiRateLimiter *ratelimit.MultiLimiter, timeout int) (*Session, error) {
Transport := &http.Transport{
MaxIdleConns: 100,
MaxIdleConnsPerHost: 100,
TLSClientConfig: &tls.Config{
InsecureSkipVerify: true,
},
Dial: (&net.Dialer{
Timeout: time.Duration(timeout) * time.Second,
}).Dial,
}
// Add proxy
if proxy != "" {
proxyURL, _ := url.Parse(proxy)
if proxyURL == nil {
// Log warning but continue anyway
gologger.Warning().Msgf("Invalid proxy provided: %s", proxy)
} else {
Transport.Proxy = http.ProxyURL(proxyURL)
}
}
client := &http.Client{
Transport: Transport,
Timeout: time.Duration(timeout) * time.Second,
}
session := &Session{Client: client, Timeout: timeout}
// Initiate rate limit instance
session.MultiRateLimiter = multiRateLimiter
// Create a new extractor object for the current domain
extractor, err := NewSubdomainExtractor(domain)
session.Extractor = extractor
return session, err
}
// Get makes a GET request to a URL with extended parameters
func (s *Session) Get(ctx context.Context, getURL, cookies string, headers map[string]string) (*http.Response, error) {
return s.HTTPRequest(ctx, http.MethodGet, getURL, cookies, headers, nil, BasicAuth{})
}
// SimpleGet makes a simple GET request to a URL
func (s *Session) SimpleGet(ctx context.Context, getURL string) (*http.Response, error) {
return s.HTTPRequest(ctx, http.MethodGet, getURL, "", map[string]string{}, nil, BasicAuth{})
}
// Post makes a POST request to a URL with extended parameters
func (s *Session) Post(ctx context.Context, postURL, cookies string, headers map[string]string, body io.Reader) (*http.Response, error) {
return s.HTTPRequest(ctx, http.MethodPost, postURL, cookies, headers, body, BasicAuth{})
}
// SimplePost makes a simple POST request to a URL
func (s *Session) SimplePost(ctx context.Context, postURL, contentType string, body io.Reader) (*http.Response, error) {
return s.HTTPRequest(ctx, http.MethodPost, postURL, "", map[string]string{"Content-Type": contentType}, body, BasicAuth{})
}
// HTTPRequest makes any HTTP request to a URL with extended parameters
func (s *Session) HTTPRequest(ctx context.Context, method, requestURL, cookies string, headers map[string]string, body io.Reader, basicAuth BasicAuth) (*http.Response, error) {
req, err := http.NewRequestWithContext(ctx, method, requestURL, body)
if err != nil {
return nil, err
}
req.Header.Set("User-Agent", uarand.GetRandom())
req.Header.Set("Accept", "*/*")
req.Header.Set("Accept-Language", "en")
req.Header.Set("Connection", "close")
if basicAuth.Username != "" || basicAuth.Password != "" {
req.SetBasicAuth(basicAuth.Username, basicAuth.Password)
}
if cookies != "" {
req.Header.Set("Cookie", cookies)
}
for key, value := range headers {
req.Header.Set(key, value)
}
sourceName := ctx.Value(CtxSourceArg).(string)
mrlErr := s.MultiRateLimiter.Take(sourceName)
if mrlErr != nil {
return nil, mrlErr
}
return httpRequestWrapper(s.Client, req)
}
// DiscardHTTPResponse discards the response content by demand
func (s *Session) DiscardHTTPResponse(response *http.Response) {
if response != nil {
_, err := io.Copy(io.Discard, response.Body)
if err != nil {
gologger.Warning().Msgf("Could not discard response body: %s\n", err)
return
}
if closeErr := response.Body.Close(); closeErr != nil {
gologger.Warning().Msgf("Could not close response body: %s\n", closeErr)
}
}
}
// Close the session
func (s *Session) Close() {
s.MultiRateLimiter.Stop()
s.Client.CloseIdleConnections()
}
func httpRequestWrapper(client *http.Client, request *http.Request) (*http.Response, error) {
response, err := client.Do(request)
if err != nil {
return nil, err
}
if response.StatusCode != http.StatusOK {
requestURL, _ := url.QueryUnescape(request.URL.String())
gologger.Debug().MsgFunc(func() string {
buffer := new(bytes.Buffer)
_, _ = buffer.ReadFrom(response.Body)
return fmt.Sprintf("Response for failed request against %s:\n%s", requestURL, buffer.String())
})
return response, fmt.Errorf("unexpected status code %d received from %s", response.StatusCode, requestURL)
}
return response, nil
}
================================================
FILE: pkg/subscraping/doc.go
================================================
// Package subscraping contains the logic of scraping agents
package subscraping
================================================
FILE: pkg/subscraping/extractor.go
================================================
package subscraping
import (
"regexp"
"strings"
)
// RegexSubdomainExtractor is a concrete implementation of the SubdomainExtractor interface, using regex for extraction.
type RegexSubdomainExtractor struct {
extractor *regexp.Regexp
}
// NewSubdomainExtractor creates a new regular expression to extract
// subdomains from text based on the given domain.
func NewSubdomainExtractor(domain string) (*RegexSubdomainExtractor, error) {
extractor, err := regexp.Compile(`(?i)[a-zA-Z0-9\*_.-]+\.` + domain)
if err != nil {
return nil, err
}
return &RegexSubdomainExtractor{extractor: extractor}, nil
}
// Extract implements the SubdomainExtractor interface, using the regex to find subdomains in the given text.
func (re *RegexSubdomainExtractor) Extract(text string) []string {
matches := re.extractor.FindAllString(text, -1)
for i, match := range matches {
matches[i] = strings.ToLower(match)
}
return matches
}
================================================
FILE: pkg/subscraping/sources/alienvault/alienvault.go
================================================
// Package alienvault logic
package alienvault
import (
"context"
"encoding/json"
"fmt"
"time"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping"
)
type alienvaultResponse struct {
Detail string `json:"detail"`
Error string `json:"error"`
PassiveDNS []struct {
Hostname string `json:"hostname"`
} `json:"passive_dns"`
}
// Source is the passive scraping agent
type Source struct {
timeTaken time.Duration
results int
errors int
requests int
apiKeys []string
skipped bool
}
// Run function returns all subdomains found with the service
func (s *Source) Run(ctx context.Context, domain string, session *subscraping.Session) <-chan subscraping.Result {
results := make(chan subscraping.Result)
s.errors = 0
s.results = 0
s.requests = 0
go func() {
defer func(startTime time.Time) {
s.timeTaken = time.Since(startTime)
close(results)
}(time.Now())
randomApiKey := subscraping.PickRandom(s.apiKeys, s.Name())
if randomApiKey == "" {
s.skipped = true
return
}
s.requests++
resp, err := session.Get(ctx, fmt.Sprintf("https://otx.alienvault.com/api/v1/indicators/domain/%s/passive_dns", domain), "",
map[string]string{"Authorization": "Bearer " + randomApiKey})
if err != nil && resp == nil {
results <- subscraping.Result{Source: s.Name(), Type: subscraping.Error, Error: err}
s.errors++
session.DiscardHTTPResponse(resp)
return
}
var response alienvaultResponse
// Get the response body and decode
err = json.NewDecoder(resp.Body).Decode(&response)
if err != nil {
results <- subscraping.Result{Source: s.Name(), Type: subscraping.Error, Error: err}
s.errors++
session.DiscardHTTPResponse(resp)
return
}
session.DiscardHTTPResponse(resp)
if response.Error != "" {
results <- subscraping.Result{
Source: s.Name(), Type: subscraping.Error, Error: fmt.Errorf("%s, %s", response.Detail, response.Error),
}
return
}
for _, record := range response.PassiveDNS {
select {
case <-ctx.Done():
return
case results <- subscraping.Result{Source: s.Name(), Type: subscraping.Subdomain, Value: record.Hostname}:
s.results++
}
}
}()
return results
}
// Name returns the name of the source
func (s *Source) Name() string {
return "alienvault"
}
func (s *Source) IsDefault() bool {
return true
}
func (s *Source) HasRecursiveSupport() bool {
return true
}
func (s *Source) KeyRequirement() subscraping.KeyRequirement {
return subscraping.RequiredKey
}
func (s *Source) NeedsKey() bool {
return s.KeyRequirement() == subscraping.RequiredKey
}
func (s *Source) AddApiKeys(keys []string) {
s.apiKeys = keys
}
func (s *Source) Statistics() subscraping.Statistics {
return subscraping.Statistics{
Errors: s.errors,
Results: s.results,
Requests: s.requests,
TimeTaken: s.timeTaken,
Skipped: s.skipped,
}
}
================================================
FILE: pkg/subscraping/sources/anubis/anubis.go
================================================
// Package anubis logic
package anubis
import (
"context"
"fmt"
"net/http"
"time"
jsoniter "github.com/json-iterator/go"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping"
)
// Source is the passive scraping agent
type Source struct {
timeTaken time.Duration
errors int
results int
requests int
}
// Run function returns all subdomains found with the service
func (s *Source) Run(ctx context.Context, domain string, session *subscraping.Session) <-chan subscraping.Result {
results := make(chan subscraping.Result)
s.errors = 0
s.results = 0
s.requests = 0
go func() {
defer func(startTime time.Time) {
s.timeTaken = time.Since(startTime)
close(results)
}(time.Now())
s.requests++
resp, err := session.SimpleGet(ctx, fmt.Sprintf("https://jonlu.ca/anubis/subdomains/%s", domain))
if err != nil {
results <- subscraping.Result{Source: s.Name(), Type: subscraping.Error, Error: err}
s.errors++
session.DiscardHTTPResponse(resp)
return
}
if resp.StatusCode != http.StatusOK {
session.DiscardHTTPResponse(resp)
return
}
var subdomains []string
err = jsoniter.NewDecoder(resp.Body).Decode(&subdomains)
if err != nil {
results <- subscraping.Result{Source: s.Name(), Type: subscraping.Error, Error: err}
s.errors++
session.DiscardHTTPResponse(resp)
return
}
session.DiscardHTTPResponse(resp)
for _, record := range subdomains {
select {
case <-ctx.Done():
return
case results <- subscraping.Result{Source: s.Name(), Type: subscraping.Subdomain, Value: record}:
s.results++
}
}
}()
return results
}
// Name returns the name of the source
func (s *Source) Name() string {
return "anubis"
}
func (s *Source) IsDefault() bool {
return true
}
func (s *Source) HasRecursiveSupport() bool {
return false
}
func (s *Source) KeyRequirement() subscraping.KeyRequirement {
return subscraping.NoKey
}
func (s *Source) NeedsKey() bool {
return s.KeyRequirement() == subscraping.RequiredKey
}
func (s *Source) AddApiKeys(_ []string) {
// no key needed
}
func (s *Source) Statistics() subscraping.Statistics {
return subscraping.Statistics{
Errors: s.errors,
Results: s.results,
Requests: s.requests,
TimeTaken: s.timeTaken,
}
}
================================================
FILE: pkg/subscraping/sources/bevigil/bevigil.go
================================================
// Package bevigil logic
package bevigil
import (
"context"
"fmt"
"time"
jsoniter "github.com/json-iterator/go"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping"
)
type Response struct {
Domain string `json:"domain"`
Subdomains []string `json:"subdomains"`
}
type Source struct {
apiKeys []string
timeTaken time.Duration
errors int
results int
requests int
skipped bool
}
func (s *Source) Run(ctx context.Context, domain string, session *subscraping.Session) <-chan subscraping.Result {
results := make(chan subscraping.Result)
s.errors = 0
s.results = 0
s.requests = 0
go func() {
defer func(startTime time.Time) {
s.timeTaken = time.Since(startTime)
close(results)
}(time.Now())
randomApiKey := subscraping.PickRandom(s.apiKeys, s.Name())
if randomApiKey == "" {
s.skipped = true
return
}
getUrl := fmt.Sprintf("https://osint.bevigil.com/api/%s/subdomains/", domain)
s.requests++
resp, err := session.Get(ctx, getUrl, "", map[string]string{
"X-Access-Token": randomApiKey, "User-Agent": "subfinder",
})
if err != nil {
results <- subscraping.Result{Source: s.Name(), Type: subscraping.Error, Error: err}
s.errors++
session.DiscardHTTPResponse(resp)
return
}
var subdomains []string
var response Response
err = jsoniter.NewDecoder(resp.Body).Decode(&response)
if err != nil {
results <- subscraping.Result{Source: s.Name(), Type: subscraping.Error, Error: err}
s.errors++
session.DiscardHTTPResponse(resp)
return
}
session.DiscardHTTPResponse(resp)
if len(response.Subdomains) > 0 {
subdomains = response.Subdomains
}
for _, subdomain := range subdomains {
select {
case <-ctx.Done():
return
case results <- subscraping.Result{Source: s.Name(), Type: subscraping.Subdomain, Value: subdomain}:
s.results++
}
}
}()
return results
}
func (s *Source) Name() string {
return "bevigil"
}
func (s *Source) IsDefault() bool {
return true
}
func (s *Source) HasRecursiveSupport() bool {
return false
}
func (s *Source) KeyRequirement() subscraping.KeyRequirement {
return subscraping.RequiredKey
}
func (s *Source) NeedsKey() bool {
return s.KeyRequirement() == subscraping.RequiredKey
}
func (s *Source) AddApiKeys(keys []string) {
s.apiKeys = keys
}
func (s *Source) Statistics() subscraping.Statistics {
return subscraping.Statistics{
Errors: s.errors,
Results: s.results,
Requests: s.requests,
TimeTaken: s.timeTaken,
Skipped: s.skipped,
}
}
================================================
FILE: pkg/subscraping/sources/bufferover/bufferover.go
================================================
// Package bufferover is a bufferover Scraping Engine in Golang
package bufferover
import (
"context"
"fmt"
"strings"
"time"
jsoniter "github.com/json-iterator/go"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping"
)
type response struct {
Meta struct {
Errors []string `json:"Errors"`
} `json:"Meta"`
FDNSA []string `json:"FDNS_A"`
RDNS []string `json:"RDNS"`
Results []string `json:"Results"`
}
// Source is the passive scraping agent
type Source struct {
apiKeys []string
timeTaken time.Duration
errors int
results int
requests int
skipped bool
}
// Run function returns all subdomains found with the service
func (s *Source) Run(ctx context.Context, domain string, session *subscraping.Session) <-chan subscraping.Result {
results := make(chan subscraping.Result)
s.errors = 0
s.results = 0
s.requests = 0
go func() {
defer func(startTime time.Time) {
s.timeTaken = time.Since(startTime)
close(results)
}(time.Now())
randomApiKey := subscraping.PickRandom(s.apiKeys, s.Name())
if randomApiKey == "" {
s.skipped = true
return
}
s.getData(ctx, fmt.Sprintf("https://tls.bufferover.run/dns?q=.%s", domain), randomApiKey, session, results)
}()
return results
}
func (s *Source) getData(ctx context.Context, sourceURL string, apiKey string, session *subscraping.Session, results chan subscraping.Result) {
s.requests++
resp, err := session.Get(ctx, sourceURL, "", map[string]string{"x-api-key": apiKey})
if err != nil && resp == nil {
results <- subscraping.Result{Source: s.Name(), Type: subscraping.Error, Error: err}
s.errors++
session.DiscardHTTPResponse(resp)
return
}
var bufforesponse response
err = jsoniter.NewDecoder(resp.Body).Decode(&bufforesponse)
if err != nil {
results <- subscraping.Result{Source: s.Name(), Type: subscraping.Error, Error: err}
s.errors++
session.DiscardHTTPResponse(resp)
return
}
session.DiscardHTTPResponse(resp)
metaErrors := bufforesponse.Meta.Errors
if len(metaErrors) > 0 {
results <- subscraping.Result{
Source: s.Name(), Type: subscraping.Error, Error: fmt.Errorf("%s", strings.Join(metaErrors, ", ")),
}
s.errors++
return
}
var subdomains []string
if len(bufforesponse.FDNSA) > 0 {
subdomains = bufforesponse.FDNSA
subdomains = append(subdomains, bufforesponse.RDNS...)
} else if len(bufforesponse.Results) > 0 {
subdomains = bufforesponse.Results
}
for _, subdomain := range subdomains {
for _, value := range session.Extractor.Extract(subdomain) {
select {
case <-ctx.Done():
return
case results <- subscraping.Result{Source: s.Name(), Type: subscraping.Subdomain, Value: value}:
s.results++
}
}
}
}
// Name returns the name of the source
func (s *Source) Name() string {
return "bufferover"
}
func (s *Source) IsDefault() bool {
return true
}
func (s *Source) HasRecursiveSupport() bool {
return true
}
func (s *Source) KeyRequirement() subscraping.KeyRequirement {
return subscraping.RequiredKey
}
func (s *Source) NeedsKey() bool {
return s.KeyRequirement() == subscraping.RequiredKey
}
func (s *Source) AddApiKeys(keys []string) {
s.apiKeys = keys
}
func (s *Source) Statistics() subscraping.Statistics {
return subscraping.Statistics{
Errors: s.errors,
Results: s.results,
Requests: s.requests,
TimeTaken: s.timeTaken,
Skipped: s.skipped,
}
}
================================================
FILE: pkg/subscraping/sources/builtwith/builtwith.go
================================================
// Package builtwith logic
package builtwith
import (
"context"
"fmt"
"time"
jsoniter "github.com/json-iterator/go"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping"
)
type response struct {
Results []resultItem `json:"Results"`
}
type resultItem struct {
Result result `json:"Result"`
}
type result struct {
Paths []path `json:"Paths"`
}
type path struct {
Domain string `json:"Domain"`
Url string `json:"Url"`
SubDomain string `json:"SubDomain"`
}
// Source is the passive scraping agent
type Source struct {
apiKeys []string
timeTaken time.Duration
errors int
results int
requests int
skipped bool
}
// Run function returns all subdomains found with the service
func (s *Source) Run(ctx context.Context, domain string, session *subscraping.Session) <-chan subscraping.Result {
results := make(chan subscraping.Result)
s.errors = 0
s.results = 0
s.requests = 0
go func() {
defer func(startTime time.Time) {
s.timeTaken = time.Since(startTime)
close(results)
}(time.Now())
randomApiKey := subscraping.PickRandom(s.apiKeys, s.Name())
if randomApiKey == "" {
return
}
s.requests++
resp, err := session.SimpleGet(ctx, fmt.Sprintf("https://api.builtwith.com/v21/api.json?KEY=%s&HIDETEXT=yes&HIDEDL=yes&NOLIVE=yes&NOMETA=yes&NOPII=yes&NOATTR=yes&LOOKUP=%s", randomApiKey, domain))
if err != nil {
results <- subscraping.Result{Source: s.Name(), Type: subscraping.Error, Error: err}
s.errors++
session.DiscardHTTPResponse(resp)
return
}
var data response
err = jsoniter.NewDecoder(resp.Body).Decode(&data)
if err != nil {
results <- subscraping.Result{Source: s.Name(), Type: subscraping.Error, Error: err}
s.errors++
session.DiscardHTTPResponse(resp)
return
}
session.DiscardHTTPResponse(resp)
for _, result := range data.Results {
for _, path := range result.Result.Paths {
select {
case <-ctx.Done():
return
case results <- subscraping.Result{Source: s.Name(), Type: subscraping.Subdomain, Value: fmt.Sprintf("%s.%s", path.SubDomain, path.Domain)}:
s.results++
}
}
}
}()
return results
}
// Name returns the name of the source
func (s *Source) Name() string {
return "builtwith"
}
func (s *Source) IsDefault() bool {
return true
}
func (s *Source) HasRecursiveSupport() bool {
return false
}
func (s *Source) KeyRequirement() subscraping.KeyRequirement {
return subscraping.RequiredKey
}
func (s *Source) NeedsKey() bool {
return s.KeyRequirement() == subscraping.RequiredKey
}
func (s *Source) AddApiKeys(keys []string) {
s.apiKeys = keys
}
func (s *Source) Statistics() subscraping.Statistics {
return subscraping.Statistics{
Errors: s.errors,
Results: s.results,
Requests: s.requests,
TimeTaken: s.timeTaken,
Skipped: s.skipped,
}
}
================================================
FILE: pkg/subscraping/sources/c99/c99.go
================================================
// Package c99 logic
package c99
import (
"context"
"fmt"
"strings"
"time"
jsoniter "github.com/json-iterator/go"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping"
)
// Source is the passive scraping agent
type Source struct {
apiKeys []string
timeTaken time.Duration
errors int
results int
requests int
skipped bool
}
type dnsdbLookupResponse struct {
Success bool `json:"success"`
Subdomains []struct {
Subdomain string `json:"subdomain"`
IP string `json:"ip"`
Cloudflare bool `json:"cloudflare"`
} `json:"subdomains"`
Error string `json:"error"`
}
// Run function returns all subdomains found with the service
func (s *Source) Run(ctx context.Context, domain string, session *subscraping.Session) <-chan subscraping.Result {
results := make(chan subscraping.Result)
s.errors = 0
s.results = 0
s.requests = 0
go func() {
defer func(startTime time.Time) {
s.timeTaken = time.Since(startTime)
close(results)
}(time.Now())
randomApiKey := subscraping.PickRandom(s.apiKeys, s.Name())
if randomApiKey == "" {
s.skipped = true
return
}
searchURL := fmt.Sprintf("https://api.c99.nl/subdomainfinder?key=%s&domain=%s&json", randomApiKey, domain)
s.requests++
resp, err := session.SimpleGet(ctx, searchURL)
if err != nil {
session.DiscardHTTPResponse(resp)
return
}
defer func() {
if err := resp.Body.Close(); err != nil {
results <- subscraping.Result{Source: s.Name(), Type: subscraping.Error, Error: err}
s.errors++
}
}()
var response dnsdbLookupResponse
err = jsoniter.NewDecoder(resp.Body).Decode(&response)
if err != nil {
results <- subscraping.Result{Source: s.Name(), Type: subscraping.Error, Error: err}
s.errors++
return
}
if response.Error != "" {
results <- subscraping.Result{
Source: s.Name(), Type: subscraping.Error, Error: fmt.Errorf("%v", response.Error),
}
s.errors++
return
}
for _, data := range response.Subdomains {
if !strings.HasPrefix(data.Subdomain, ".") {
select {
case <-ctx.Done():
return
case results <- subscraping.Result{Source: s.Name(), Type: subscraping.Subdomain, Value: data.Subdomain}:
s.results++
}
}
}
}()
return results
}
// Name returns the name of the source
func (s *Source) Name() string {
return "c99"
}
func (s *Source) IsDefault() bool {
return true
}
func (s *Source) HasRecursiveSupport() bool {
return false
}
func (s *Source) KeyRequirement() subscraping.KeyRequirement {
return subscraping.RequiredKey
}
func (s *Source) NeedsKey() bool {
return s.KeyRequirement() == subscraping.RequiredKey
}
func (s *Source) AddApiKeys(keys []string) {
s.apiKeys = keys
}
func (s *Source) Statistics() subscraping.Statistics {
return subscraping.Statistics{
Errors: s.errors,
Results: s.results,
Requests: s.requests,
TimeTaken: s.timeTaken,
Skipped: s.skipped,
}
}
================================================
FILE: pkg/subscraping/sources/censys/censys.go
================================================
// Package censys logic
package censys
import (
"bytes"
"context"
"net/http"
"strings"
"time"
jsoniter "github.com/json-iterator/go"
"github.com/projectdiscovery/subfinder/v2/pkg/subscraping"
)
const (
// maxCensysPages is the maximum number of pages to fetch from the API
maxCensysPages = 10
// maxPerPage is the maximum number of results per page
maxPerPage = 100
// baseURL is the Censys Platform API base URL
baseURL = "https://api.platform.censys.io"
// searchEndpoint is the global data search query endpoint
searchEndpoint = "/v3/global/search/query"
// queryPrefix is the Censys query language prefix for certificate name search
queryPrefix = "cert.names: "
// authHeaderPrefix is the Bearer token prefix for Authorization header
authHeaderPrefix = "Bearer "
// contentTypeJSON is the Content-Type header value for JSON
contentTypeJSON = "application/json"
// orgIDHeader is the header name for organization ID
orgIDHeader = "X-Organization-ID"
)
// apiKey holds the Personal Access Token and optional Organization ID
type apiKey struct {
pat string
orgID string
}
// Platform API request body
type searchRequest struct {
Query string `json:"query"`
Fields []string `json:"fields,omitempty"`
PageSize int `json:"page_size,omitempty"`
Cursor string `json:"cursor,omitempty"`
}
// Platform API response structures
type response struct {
Result result `json:"result"`
}
type result struct {
Hits []hit `json:"hits"`
TotalHits int64 `json:"total_hits"`
NextPageToken string `json:"next_page_token"`
}
type hit struct {
CertificateV1 certificateV1 `json:"certificate_v1"`
}
type certificateV1 struct {
Resource resource `json:"resource"`
}
type resource struct {
Names []string `json:"names"`
}
// Source is the passive scraping agent
type Source struct {
apiKeys []apiKey
timeTaken time.Duration
errors int
results int
requests int
skipped bool
}
// Run function returns all subdomains found with the service
func (s *Source) Run(ctx context.Context, domain string, session *subscraping.Session) <-chan subscraping.Result {
results := make(chan subscraping.Result)
s.errors = 0
s.results = 0
s.requests = 0
go func() {
defer func(startTime time.Time) {
s.timeTaken = time.Since(startTime)
close(results)
}(time.Now())
// PickRandom selects a random API key from configured keys.
// This enables load balancing when users configure multiple PATs
// (e.g., CENSYS_API_KEY=pat1:org1,pat2:org2) to distribute requests
// and avoid hitting rate limits on a single key.
randomApiKey := subscraping.PickRandom(s.apiKeys, s.Name())
if randomApiKey.pat == "" {
s.skipped = true
return
}
apiURL := baseURL + searchEndpoint
cursor := ""
currentPage := 1
for {
select {
case <-ctx.Done():
return
default:
}
reqBody := searchRequest{
Query: queryPrefix + domain,
Fields: []string{"cert.names"},
PageSize: maxPerPage,
}
if cursor != "" {
reqBody.Cursor = cursor
}
bodyBytes, err := jsoniter.Marshal(reqBody)
if err != nil {
results <- subscraping.Result{Source: s.Name(), Type: subscraping.Error, Error: err}
s.errors++
return
}
headers := map[string]string{
"Content-Type": contentTypeJSON,
"Authorization": authHeaderPrefix + randomApiKey.pat,
}
// Add Organization ID header if provided
if randomApiKey.orgID != "" {
headers[orgIDHeader] = randomApiKey.orgID
}
s.requests++
resp, err := session.HTTPRequest(
ctx,
http.MethodPost,
apiURL,
"",
headers,
bytes.NewReader(bodyBytes),
subscraping.BasicAuth{},
)
if err != nil {
results <- subscraping.Result{Source: s.Name(), Type: subscraping.Error, Error: err}
s.errors++
session.DiscardHTTPResponse(resp)
return
}
var censysResponse response
err = jsoniter.NewDecoder(resp.Body).Decode(&censysResponse)
_ = resp.Body.Close()
if err != nil {
results <- subscraping.Resu
gitextract_djozq64v/
├── .github/
│ ├── ISSUE_TEMPLATE/
│ │ ├── bug_report.md
│ │ ├── config.yml
│ │ ├── feature_request.md
│ │ └── issue-report.md
│ ├── PULL_REQUEST_TEMPLATE.md
│ ├── dependabot.yml
│ ├── release.yml
│ └── workflows/
│ ├── build-test.yml
│ ├── codeql-analysis.yml
│ ├── compat-checks.yaml
│ ├── dep-auto-merge.yml
│ ├── dockerhub-push.yml
│ ├── release-binary.yml
│ └── release-test.yml
├── .gitignore
├── .goreleaser.yml
├── DISCLAIMER.md
├── Dockerfile
├── LICENSE.md
├── Makefile
├── README.md
├── THANKS.md
├── cmd/
│ └── subfinder/
│ └── main.go
├── examples/
│ └── main.go
├── go.mod
├── go.sum
└── pkg/
├── passive/
│ ├── doc.go
│ ├── passive.go
│ ├── sources.go
│ ├── sources_test.go
│ ├── sources_w_auth_test.go
│ └── sources_wo_auth_test.go
├── resolve/
│ ├── client.go
│ ├── doc.go
│ └── resolve.go
├── runner/
│ ├── banners.go
│ ├── config.go
│ ├── doc.go
│ ├── enumerate.go
│ ├── enumerate_test.go
│ ├── initialize.go
│ ├── options.go
│ ├── outputter.go
│ ├── runner.go
│ ├── stats.go
│ ├── util.go
│ └── validate.go
├── subscraping/
│ ├── agent.go
│ ├── doc.go
│ ├── extractor.go
│ ├── sources/
│ │ ├── alienvault/
│ │ │ └── alienvault.go
│ │ ├── anubis/
│ │ │ └── anubis.go
│ │ ├── bevigil/
│ │ │ └── bevigil.go
│ │ ├── bufferover/
│ │ │ └── bufferover.go
│ │ ├── builtwith/
│ │ │ └── builtwith.go
│ │ ├── c99/
│ │ │ └── c99.go
│ │ ├── censys/
│ │ │ ├── censys.go
│ │ │ └── censys_test.go
│ │ ├── certspotter/
│ │ │ └── certspotter.go
│ │ ├── chaos/
│ │ │ └── chaos.go
│ │ ├── chinaz/
│ │ │ └── chinaz.go
│ │ ├── commoncrawl/
│ │ │ └── commoncrawl.go
│ │ ├── crtsh/
│ │ │ └── crtsh.go
│ │ ├── digitalyama/
│ │ │ └── digitalyama.go
│ │ ├── digitorus/
│ │ │ └── digitorus.go
│ │ ├── dnsdb/
│ │ │ └── dnsdb.go
│ │ ├── dnsdumpster/
│ │ │ └── dnsdumpster.go
│ │ ├── dnsrepo/
│ │ │ └── dnsrepo.go
│ │ ├── domainsproject/
│ │ │ └── domainsproject.go
│ │ ├── driftnet/
│ │ │ └── driftnet.go
│ │ ├── facebook/
│ │ │ ├── ctlogs.go
│ │ │ ├── ctlogs_test.go
│ │ │ └── types.go
│ │ ├── fofa/
│ │ │ └── fofa.go
│ │ ├── fullhunt/
│ │ │ └── fullhunt.go
│ │ ├── github/
│ │ │ ├── github.go
│ │ │ └── tokenmanager.go
│ │ ├── gitlab/
│ │ │ └── gitlab.go
│ │ ├── hackertarget/
│ │ │ └── hackertarget.go
│ │ ├── hudsonrock/
│ │ │ └── hudsonrock.go
│ │ ├── intelx/
│ │ │ └── intelx.go
│ │ ├── leakix/
│ │ │ └── leakix.go
│ │ ├── merklemap/
│ │ │ └── merklemap.go
│ │ ├── netlas/
│ │ │ └── netlas.go
│ │ ├── onyphe/
│ │ │ └── onyphe.go
│ │ ├── profundis/
│ │ │ └── profundis.go
│ │ ├── pugrecon/
│ │ │ └── pugrecon.go
│ │ ├── quake/
│ │ │ └── quake.go
│ │ ├── rapiddns/
│ │ │ └── rapiddns.go
│ │ ├── reconcloud/
│ │ │ └── reconcloud.go
│ │ ├── reconeer/
│ │ │ └── reconeer.go
│ │ ├── redhuntlabs/
│ │ │ └── redhuntlabs.go
│ │ ├── riddler/
│ │ │ └── riddler.go
│ │ ├── robtex/
│ │ │ └── robtext.go
│ │ ├── rsecloud/
│ │ │ └── rsecloud.go
│ │ ├── securitytrails/
│ │ │ └── securitytrails.go
│ │ ├── shodan/
│ │ │ └── shodan.go
│ │ ├── sitedossier/
│ │ │ └── sitedossier.go
│ │ ├── thc/
│ │ │ └── thc.go
│ │ ├── threatbook/
│ │ │ └── threatbook.go
│ │ ├── threatcrowd/
│ │ │ └── threatcrowd.go
│ │ ├── threatminer/
│ │ │ └── threatminer.go
│ │ ├── urlscan/
│ │ │ └── urlscan.go
│ │ ├── virustotal/
│ │ │ └── virustotal.go
│ │ ├── waybackarchive/
│ │ │ └── waybackarchive.go
│ │ ├── whoisxmlapi/
│ │ │ └── whoisxmlapi.go
│ │ ├── windvane/
│ │ │ └── windvane.go
│ │ └── zoomeyeapi/
│ │ └── zoomeyeapi.go
│ ├── types.go
│ └── utils.go
└── testutils/
└── integration.go
SYMBOL INDEX (739 symbols across 83 files)
FILE: cmd/subfinder/main.go
function main (line 10) | func main() {
FILE: examples/main.go
function main (line 12) | func main() {
FILE: pkg/passive/passive.go
type EnumerationOptions (line 16) | type EnumerationOptions struct
type EnumerateOption (line 20) | type EnumerateOption
function WithCustomRateLimit (line 22) | func WithCustomRateLimit(crl *subscraping.CustomRateLimit) EnumerateOpti...
method EnumerateSubdomains (line 29) | func (a *Agent) EnumerateSubdomains(domain string, proxy string, rateLim...
method EnumerateSubdomainsWithCtx (line 34) | func (a *Agent) EnumerateSubdomainsWithCtx(ctx context.Context, domain s...
method buildMultiRateLimiter (line 85) | func (a *Agent) buildMultiRateLimiter(ctx context.Context, globalRateLim...
function sourceRateLimitOrDefault (line 107) | func sourceRateLimitOrDefault(defaultRateLimit uint, sourceRateLimit uin...
function addRateLimiter (line 114) | func addRateLimiter(ctx context.Context, multiRateLimiter *ratelimit.Mul...
method GetStatistics (line 133) | func (a *Agent) GetStatistics() map[string]subscraping.Statistics {
FILE: pkg/passive/sources.go
function init (line 126) | func init() {
type Agent (line 135) | type Agent struct
function New (line 140) | func New(sourceNames, excludedSourceNames []string, useAllSources, useSo...
FILE: pkg/passive/sources_test.go
function TestSourceCategorization (line 131) | func TestSourceCategorization(t *testing.T) {
function TestSourceFiltering (line 153) | func TestSourceFiltering(t *testing.T) {
FILE: pkg/passive/sources_w_auth_test.go
function TestSourcesWithKeys (line 21) | func TestSourcesWithKeys(t *testing.T) {
FILE: pkg/passive/sources_wo_auth_test.go
function TestSourcesWithoutKeys (line 21) | func TestSourcesWithoutKeys(t *testing.T) {
FILE: pkg/resolve/client.go
type Resolver (line 22) | type Resolver struct
function New (line 28) | func New() *Resolver {
FILE: pkg/resolve/resolve.go
constant maxWildcardChecks (line 11) | maxWildcardChecks = 3
type ResolutionPool (line 16) | type ResolutionPool struct
method InitWildcards (line 77) | func (r *ResolutionPool) InitWildcards(domain string) error {
method resolveWorker (line 94) | func (r *ResolutionPool) resolveWorker() {
type HostEntry (line 27) | type HostEntry struct
type Result (line 35) | type Result struct
type ResultType (line 45) | type ResultType
constant Subdomain (line 49) | Subdomain ResultType = iota
constant Error (line 50) | Error
method NewResolutionPool (line 54) | func (r *Resolver) NewResolutionPool(workers int, removeWildcard bool) *...
FILE: pkg/runner/banners.go
constant banner (line 8) | banner = `
constant ToolName (line 17) | ToolName = `subfinder`
constant version (line 20) | version = `v2.13.0`
function showBanner (line 23) | func showBanner() {
function GetUpdateCallback (line 29) | func GetUpdateCallback() func() {
FILE: pkg/runner/config.go
function createProviderConfigYAML (line 16) | func createProviderConfigYAML(configFilePath string) error {
function UnmarshalFrom (line 40) | func UnmarshalFrom(file string) error {
FILE: pkg/runner/enumerate.go
constant maxNumCount (line 19) | maxNumCount = 2
method EnumerateSingleDomain (line 31) | func (r *Runner) EnumerateSingleDomain(domain string, writers []io.Write...
method EnumerateSingleDomainWithCtx (line 36) | func (r *Runner) EnumerateSingleDomainWithCtx(ctx context.Context, domai...
method filterAndMatchSubdomain (line 212) | func (r *Runner) filterAndMatchSubdomain(subdomain string) bool {
FILE: pkg/runner/enumerate_test.go
function TestFilterAndMatchSubdomain (line 10) | func TestFilterAndMatchSubdomain(t *testing.T) {
FILE: pkg/runner/initialize.go
method initializePassiveEngine (line 13) | func (r *Runner) initializePassiveEngine() {
method initializeResolver (line 18) | func (r *Runner) initializeResolver() error {
FILE: pkg/runner/options.go
type Options (line 34) | type Options struct
method loadProvidersFrom (line 212) | func (options *Options) loadProvidersFrom(location string) {
method preProcessDomains (line 244) | func (options *Options) preProcessDomains() {
type OnResultCallback (line 75) | type OnResultCallback
function ParseOptions (line 78) | func ParseOptions() *Options {
function listSources (line 225) | func listSources(options *Options) {
FILE: pkg/runner/outputter.go
type OutputWriter (line 17) | type OutputWriter struct
method createFile (line 48) | func (o *OutputWriter) createFile(filename string, appendToFile bool) ...
method WriteHostIP (line 79) | func (o *OutputWriter) WriteHostIP(input string, results map[string]re...
method WriteHostNoWildcard (line 133) | func (o *OutputWriter) WriteHostNoWildcard(input string, results map[s...
method WriteHost (line 143) | func (o *OutputWriter) WriteHost(input string, results map[string]reso...
method WriteSourceHost (line 191) | func (o *OutputWriter) WriteSourceHost(input string, sourceMap map[str...
type jsonSourceResult (line 21) | type jsonSourceResult struct
type jsonSourceIPResult (line 28) | type jsonSourceIPResult struct
type jsonSourcesResult (line 36) | type jsonSourcesResult struct
function NewOutputWriter (line 44) | func NewOutputWriter(json bool) *OutputWriter {
function writePlainHostIP (line 89) | func writePlainHostIP(_ string, results map[string]resolve.Result, write...
function writeJSONHostIP (line 113) | func writeJSONHostIP(input string, results map[string]resolve.Result, wr...
function writePlainHost (line 153) | func writePlainHost(_ string, results map[string]resolve.HostEntry, writ...
function writeJSONHost (line 173) | func writeJSONHost(input string, results map[string]resolve.HostEntry, w...
function writeSourceJSONHost (line 201) | func writeSourceJSONHost(input string, sourceMap map[string]map[string]s...
function writeSourcePlainHost (line 223) | func writeSourcePlainHost(_ string, sourceMap map[string]map[string]stru...
FILE: pkg/runner/runner.go
type Runner (line 26) | type Runner struct
method RunEnumeration (line 76) | func (r *Runner) RunEnumeration() error {
method RunEnumerationWithCtx (line 82) | func (r *Runner) RunEnumerationWithCtx(ctx context.Context) error {
method EnumerateMultipleDomains (line 111) | func (r *Runner) EnumerateMultipleDomains(reader io.Reader, writers []...
method EnumerateMultipleDomainsWithCtx (line 118) | func (r *Runner) EnumerateMultipleDomainsWithCtx(ctx context.Context, ...
function NewRunner (line 36) | func NewRunner(options *Options) (*Runner, error) {
FILE: pkg/runner/stats.go
function printStatistics (line 14) | func printStatistics(stats map[string]subscraping.Statistics) {
method GetStatistics (line 44) | func (r *Runner) GetStatistics() map[string]subscraping.Statistics {
FILE: pkg/runner/util.go
function loadFromFile (line 8) | func loadFromFile(file string) ([]string, error) {
function preprocessDomain (line 24) | func preprocessDomain(s string) string {
FILE: pkg/runner/validate.go
method validateOptions (line 18) | func (options *Options) validateOptions() error {
function stripRegexString (line 70) | func stripRegexString(val string) string {
method ConfigureOutput (line 77) | func (options *Options) ConfigureOutput() {
FILE: pkg/subscraping/agent.go
function NewSession (line 21) | func NewSession(domain string, proxy string, multiRateLimiter *ratelimit...
method Get (line 62) | func (s *Session) Get(ctx context.Context, getURL, cookies string, heade...
method SimpleGet (line 67) | func (s *Session) SimpleGet(ctx context.Context, getURL string) (*http.R...
method Post (line 72) | func (s *Session) Post(ctx context.Context, postURL, cookies string, hea...
method SimplePost (line 77) | func (s *Session) SimplePost(ctx context.Context, postURL, contentType s...
method HTTPRequest (line 82) | func (s *Session) HTTPRequest(ctx context.Context, method, requestURL, c...
method DiscardHTTPResponse (line 115) | func (s *Session) DiscardHTTPResponse(response *http.Response) {
method Close (line 129) | func (s *Session) Close() {
function httpRequestWrapper (line 134) | func httpRequestWrapper(client *http.Client, request *http.Request) (*ht...
FILE: pkg/subscraping/extractor.go
type RegexSubdomainExtractor (line 9) | type RegexSubdomainExtractor struct
method Extract (line 24) | func (re *RegexSubdomainExtractor) Extract(text string) []string {
function NewSubdomainExtractor (line 15) | func NewSubdomainExtractor(domain string) (*RegexSubdomainExtractor, err...
FILE: pkg/subscraping/sources/alienvault/alienvault.go
type alienvaultResponse (line 13) | type alienvaultResponse struct
type Source (line 22) | type Source struct
method Run (line 32) | func (s *Source) Run(ctx context.Context, domain string, session *subs...
method Name (line 92) | func (s *Source) Name() string {
method IsDefault (line 96) | func (s *Source) IsDefault() bool {
method HasRecursiveSupport (line 100) | func (s *Source) HasRecursiveSupport() bool {
method KeyRequirement (line 104) | func (s *Source) KeyRequirement() subscraping.KeyRequirement {
method NeedsKey (line 108) | func (s *Source) NeedsKey() bool {
method AddApiKeys (line 112) | func (s *Source) AddApiKeys(keys []string) {
method Statistics (line 116) | func (s *Source) Statistics() subscraping.Statistics {
FILE: pkg/subscraping/sources/anubis/anubis.go
type Source (line 16) | type Source struct
method Run (line 24) | func (s *Source) Run(ctx context.Context, domain string, session *subs...
method Name (line 76) | func (s *Source) Name() string {
method IsDefault (line 80) | func (s *Source) IsDefault() bool {
method HasRecursiveSupport (line 84) | func (s *Source) HasRecursiveSupport() bool {
method KeyRequirement (line 88) | func (s *Source) KeyRequirement() subscraping.KeyRequirement {
method NeedsKey (line 92) | func (s *Source) NeedsKey() bool {
method AddApiKeys (line 96) | func (s *Source) AddApiKeys(_ []string) {
method Statistics (line 100) | func (s *Source) Statistics() subscraping.Statistics {
FILE: pkg/subscraping/sources/bevigil/bevigil.go
type Response (line 14) | type Response struct
type Source (line 19) | type Source struct
method Run (line 28) | func (s *Source) Run(ctx context.Context, domain string, session *subs...
method Name (line 88) | func (s *Source) Name() string {
method IsDefault (line 92) | func (s *Source) IsDefault() bool {
method HasRecursiveSupport (line 96) | func (s *Source) HasRecursiveSupport() bool {
method KeyRequirement (line 100) | func (s *Source) KeyRequirement() subscraping.KeyRequirement {
method NeedsKey (line 104) | func (s *Source) NeedsKey() bool {
method AddApiKeys (line 108) | func (s *Source) AddApiKeys(keys []string) {
method Statistics (line 112) | func (s *Source) Statistics() subscraping.Statistics {
FILE: pkg/subscraping/sources/bufferover/bufferover.go
type response (line 15) | type response struct
type Source (line 25) | type Source struct
method Run (line 35) | func (s *Source) Run(ctx context.Context, domain string, session *subs...
method getData (line 59) | func (s *Source) getData(ctx context.Context, sourceURL string, apiKey...
method Name (line 113) | func (s *Source) Name() string {
method IsDefault (line 117) | func (s *Source) IsDefault() bool {
method HasRecursiveSupport (line 121) | func (s *Source) HasRecursiveSupport() bool {
method KeyRequirement (line 125) | func (s *Source) KeyRequirement() subscraping.KeyRequirement {
method NeedsKey (line 129) | func (s *Source) NeedsKey() bool {
method AddApiKeys (line 133) | func (s *Source) AddApiKeys(keys []string) {
method Statistics (line 137) | func (s *Source) Statistics() subscraping.Statistics {
FILE: pkg/subscraping/sources/builtwith/builtwith.go
type response (line 14) | type response struct
type resultItem (line 18) | type resultItem struct
type result (line 22) | type result struct
type path (line 26) | type path struct
type Source (line 33) | type Source struct
method Run (line 43) | func (s *Source) Run(ctx context.Context, domain string, session *subs...
method Name (line 94) | func (s *Source) Name() string {
method IsDefault (line 98) | func (s *Source) IsDefault() bool {
method HasRecursiveSupport (line 102) | func (s *Source) HasRecursiveSupport() bool {
method KeyRequirement (line 106) | func (s *Source) KeyRequirement() subscraping.KeyRequirement {
method NeedsKey (line 110) | func (s *Source) NeedsKey() bool {
method AddApiKeys (line 114) | func (s *Source) AddApiKeys(keys []string) {
method Statistics (line 118) | func (s *Source) Statistics() subscraping.Statistics {
FILE: pkg/subscraping/sources/c99/c99.go
type Source (line 16) | type Source struct
method Run (line 36) | func (s *Source) Run(ctx context.Context, domain string, session *subs...
method Name (line 101) | func (s *Source) Name() string {
method IsDefault (line 105) | func (s *Source) IsDefault() bool {
method HasRecursiveSupport (line 109) | func (s *Source) HasRecursiveSupport() bool {
method KeyRequirement (line 113) | func (s *Source) KeyRequirement() subscraping.KeyRequirement {
method NeedsKey (line 117) | func (s *Source) NeedsKey() bool {
method AddApiKeys (line 121) | func (s *Source) AddApiKeys(keys []string) {
method Statistics (line 125) | func (s *Source) Statistics() subscraping.Statistics {
type dnsdbLookupResponse (line 25) | type dnsdbLookupResponse struct
FILE: pkg/subscraping/sources/censys/censys.go
constant maxCensysPages (line 18) | maxCensysPages = 10
constant maxPerPage (line 20) | maxPerPage = 100
constant baseURL (line 22) | baseURL = "https://api.platform.censys.io"
constant searchEndpoint (line 24) | searchEndpoint = "/v3/global/search/query"
constant queryPrefix (line 26) | queryPrefix = "cert.names: "
constant authHeaderPrefix (line 28) | authHeaderPrefix = "Bearer "
constant contentTypeJSON (line 30) | contentTypeJSON = "application/json"
constant orgIDHeader (line 32) | orgIDHeader = "X-Organization-ID"
type apiKey (line 36) | type apiKey struct
type searchRequest (line 42) | type searchRequest struct
type response (line 50) | type response struct
type result (line 54) | type result struct
type hit (line 60) | type hit struct
type certificateV1 (line 64) | type certificateV1 struct
type resource (line 68) | type resource struct
type Source (line 73) | type Source struct
method Run (line 83) | func (s *Source) Run(ctx context.Context, domain string, session *subs...
method Name (line 191) | func (s *Source) Name() string {
method IsDefault (line 195) | func (s *Source) IsDefault() bool {
method HasRecursiveSupport (line 199) | func (s *Source) HasRecursiveSupport() bool {
method KeyRequirement (line 203) | func (s *Source) KeyRequirement() subscraping.KeyRequirement {
method NeedsKey (line 207) | func (s *Source) NeedsKey() bool {
method AddApiKeys (line 214) | func (s *Source) AddApiKeys(keys []string) {
method Statistics (line 226) | func (s *Source) Statistics() subscraping.Statistics {
FILE: pkg/subscraping/sources/censys/censys_test.go
function createTestMultiRateLimiter (line 17) | func createTestMultiRateLimiter(ctx context.Context) *ratelimit.MultiLim...
function TestCensysSource_NoApiKey (line 27) | func TestCensysSource_NoApiKey(t *testing.T) {
function TestCensysSource_ContextCancellation (line 53) | func TestCensysSource_ContextCancellation(t *testing.T) {
function TestCensysSource_Metadata (line 91) | func TestCensysSource_Metadata(t *testing.T) {
function TestCensysSource_AddApiKeys (line 100) | func TestCensysSource_AddApiKeys(t *testing.T) {
function TestCensysSource_Statistics (line 124) | func TestCensysSource_Statistics(t *testing.T) {
FILE: pkg/subscraping/sources/certspotter/certspotter.go
type certspotterObject (line 14) | type certspotterObject struct
type Source (line 20) | type Source struct
method Run (line 30) | func (s *Source) Run(ctx context.Context, domain string, session *subs...
method Name (line 135) | func (s *Source) Name() string {
method IsDefault (line 139) | func (s *Source) IsDefault() bool {
method HasRecursiveSupport (line 143) | func (s *Source) HasRecursiveSupport() bool {
method KeyRequirement (line 147) | func (s *Source) KeyRequirement() subscraping.KeyRequirement {
method NeedsKey (line 151) | func (s *Source) NeedsKey() bool {
method AddApiKeys (line 155) | func (s *Source) AddApiKeys(keys []string) {
method Statistics (line 159) | func (s *Source) Statistics() subscraping.Statistics {
FILE: pkg/subscraping/sources/chaos/chaos.go
type Source (line 14) | type Source struct
method Run (line 24) | func (s *Source) Run(ctx context.Context, domain string, _ *subscrapin...
method Name (line 68) | func (s *Source) Name() string {
method IsDefault (line 72) | func (s *Source) IsDefault() bool {
method HasRecursiveSupport (line 76) | func (s *Source) HasRecursiveSupport() bool {
method KeyRequirement (line 80) | func (s *Source) KeyRequirement() subscraping.KeyRequirement {
method NeedsKey (line 84) | func (s *Source) NeedsKey() bool {
method AddApiKeys (line 88) | func (s *Source) AddApiKeys(keys []string) {
method Statistics (line 92) | func (s *Source) Statistics() subscraping.Statistics {
FILE: pkg/subscraping/sources/chinaz/chinaz.go
type Source (line 15) | type Source struct
method Run (line 25) | func (s *Source) Run(ctx context.Context, domain string, session *subs...
method Name (line 85) | func (s *Source) Name() string {
method IsDefault (line 89) | func (s *Source) IsDefault() bool {
method HasRecursiveSupport (line 93) | func (s *Source) HasRecursiveSupport() bool {
method KeyRequirement (line 97) | func (s *Source) KeyRequirement() subscraping.KeyRequirement {
method NeedsKey (line 101) | func (s *Source) NeedsKey() bool {
method AddApiKeys (line 105) | func (s *Source) AddApiKeys(keys []string) {
method Statistics (line 109) | func (s *Source) Statistics() subscraping.Statistics {
FILE: pkg/subscraping/sources/commoncrawl/commoncrawl.go
constant indexURL (line 19) | indexURL = "https://index.commoncrawl.org/collinfo.json"
constant maxYearsBack (line 20) | maxYearsBack = 5
type indexResponse (line 25) | type indexResponse struct
type Source (line 31) | type Source struct
method Run (line 39) | func (s *Source) Run(ctx context.Context, domain string, session *subs...
method Name (line 99) | func (s *Source) Name() string {
method IsDefault (line 103) | func (s *Source) IsDefault() bool {
method HasRecursiveSupport (line 107) | func (s *Source) HasRecursiveSupport() bool {
method KeyRequirement (line 111) | func (s *Source) KeyRequirement() subscraping.KeyRequirement {
method NeedsKey (line 115) | func (s *Source) NeedsKey() bool {
method AddApiKeys (line 119) | func (s *Source) AddApiKeys(_ []string) {
method Statistics (line 123) | func (s *Source) Statistics() subscraping.Statistics {
method getSubdomains (line 132) | func (s *Source) getSubdomains(ctx context.Context, searchURL, domain ...
FILE: pkg/subscraping/sources/crtsh/crtsh.go
type subdomain (line 22) | type subdomain struct
type Source (line 28) | type Source struct
method Run (line 36) | func (s *Source) Run(ctx context.Context, domain string, session *subs...
method getSubdomainsFromSQL (line 58) | func (s *Source) getSubdomainsFromSQL(ctx context.Context, domain stri...
method getSubdomainsFromHTTP (line 152) | func (s *Source) getSubdomainsFromHTTP(ctx context.Context, domain str...
method Name (line 197) | func (s *Source) Name() string {
method IsDefault (line 201) | func (s *Source) IsDefault() bool {
method HasRecursiveSupport (line 205) | func (s *Source) HasRecursiveSupport() bool {
method KeyRequirement (line 209) | func (s *Source) KeyRequirement() subscraping.KeyRequirement {
method NeedsKey (line 213) | func (s *Source) NeedsKey() bool {
method AddApiKeys (line 217) | func (s *Source) AddApiKeys(_ []string) {
method Statistics (line 221) | func (s *Source) Statistics() subscraping.Statistics {
FILE: pkg/subscraping/sources/digitalyama/digitalyama.go
type Source (line 14) | type Source struct
method Run (line 34) | func (s *Source) Run(ctx context.Context, domain string, session *subs...
method Name (line 113) | func (s *Source) Name() string {
method IsDefault (line 117) | func (s *Source) IsDefault() bool {
method HasRecursiveSupport (line 121) | func (s *Source) HasRecursiveSupport() bool {
method KeyRequirement (line 125) | func (s *Source) KeyRequirement() subscraping.KeyRequirement {
method NeedsKey (line 129) | func (s *Source) NeedsKey() bool {
method AddApiKeys (line 133) | func (s *Source) AddApiKeys(keys []string) {
method Statistics (line 137) | func (s *Source) Statistics() subscraping.Statistics {
type digitalYamaResponse (line 23) | type digitalYamaResponse struct
FILE: pkg/subscraping/sources/digitorus/digitorus.go
type Source (line 17) | type Source struct
method Run (line 25) | func (s *Source) Run(ctx context.Context, domain string, session *subs...
method Name (line 81) | func (s *Source) Name() string {
method IsDefault (line 85) | func (s *Source) IsDefault() bool {
method HasRecursiveSupport (line 89) | func (s *Source) HasRecursiveSupport() bool {
method KeyRequirement (line 93) | func (s *Source) KeyRequirement() subscraping.KeyRequirement {
method NeedsKey (line 97) | func (s *Source) NeedsKey() bool {
method AddApiKeys (line 101) | func (s *Source) AddApiKeys(_ []string) {
method Statistics (line 105) | func (s *Source) Statistics() subscraping.Statistics {
FILE: pkg/subscraping/sources/dnsdb/dnsdb.go
constant urlBase (line 20) | urlBase string = "https://api.dnsdb.info/dnsdb/v2"
type rateResponse (line 22) | type rateResponse struct
type rate (line 26) | type rate struct
type safResponse (line 30) | type safResponse struct
type dnsdbObj (line 36) | type dnsdbObj struct
type Source (line 41) | type Source struct
method Run (line 51) | func (s *Source) Run(ctx context.Context, domain string, session *subs...
method Name (line 178) | func (s *Source) Name() string {
method IsDefault (line 182) | func (s *Source) IsDefault() bool {
method HasRecursiveSupport (line 186) | func (s *Source) HasRecursiveSupport() bool {
method KeyRequirement (line 190) | func (s *Source) KeyRequirement() subscraping.KeyRequirement {
method NeedsKey (line 194) | func (s *Source) NeedsKey() bool {
method AddApiKeys (line 198) | func (s *Source) AddApiKeys(keys []string) {
method Statistics (line 202) | func (s *Source) Statistics() subscraping.Statistics {
function getMaxOffset (line 212) | func getMaxOffset(ctx context.Context, session *subscraping.Session, hea...
FILE: pkg/subscraping/sources/dnsdumpster/dnsdumpster.go
type response (line 13) | type response struct
type Source (line 23) | type Source struct
method Run (line 33) | func (s *Source) Run(ctx context.Context, domain string, session *subs...
method Name (line 85) | func (s *Source) Name() string {
method IsDefault (line 89) | func (s *Source) IsDefault() bool {
method HasRecursiveSupport (line 93) | func (s *Source) HasRecursiveSupport() bool {
method KeyRequirement (line 97) | func (s *Source) KeyRequirement() subscraping.KeyRequirement {
method NeedsKey (line 101) | func (s *Source) NeedsKey() bool {
method AddApiKeys (line 105) | func (s *Source) AddApiKeys(keys []string) {
method Statistics (line 109) | func (s *Source) Statistics() subscraping.Statistics {
FILE: pkg/subscraping/sources/dnsrepo/dnsrepo.go
type Source (line 15) | type Source struct
method Run (line 28) | func (s *Source) Run(ctx context.Context, domain string, session *subs...
method Name (line 94) | func (s *Source) Name() string {
method IsDefault (line 98) | func (s *Source) IsDefault() bool {
method HasRecursiveSupport (line 102) | func (s *Source) HasRecursiveSupport() bool {
method KeyRequirement (line 106) | func (s *Source) KeyRequirement() subscraping.KeyRequirement {
method NeedsKey (line 110) | func (s *Source) NeedsKey() bool {
method AddApiKeys (line 114) | func (s *Source) AddApiKeys(keys []string) {
method Statistics (line 118) | func (s *Source) Statistics() subscraping.Statistics {
type DnsRepoResponse (line 24) | type DnsRepoResponse
FILE: pkg/subscraping/sources/domainsproject/domainsproject.go
type Source (line 16) | type Source struct
method Run (line 36) | func (s *Source) Run(ctx context.Context, domain string, session *subs...
method Name (line 111) | func (s *Source) Name() string {
method IsDefault (line 115) | func (s *Source) IsDefault() bool {
method HasRecursiveSupport (line 119) | func (s *Source) HasRecursiveSupport() bool {
method KeyRequirement (line 123) | func (s *Source) KeyRequirement() subscraping.KeyRequirement {
method NeedsKey (line 127) | func (s *Source) NeedsKey() bool {
method AddApiKeys (line 131) | func (s *Source) AddApiKeys(keys []string) {
method Statistics (line 137) | func (s *Source) Statistics() subscraping.Statistics {
type apiKey (line 25) | type apiKey struct
type domainsProjectResponse (line 30) | type domainsProjectResponse struct
FILE: pkg/subscraping/sources/driftnet/driftnet.go
constant baseURL (line 19) | baseURL = "https://api.driftnet.io/v1/"
constant summaryLimit (line 22) | summaryLimit = 10000
type Source (line 26) | type Source struct
method Run (line 64) | func (s *Source) Run(ctx context.Context, domain string, session *subs...
method Name (line 95) | func (s *Source) Name() string {
method IsDefault (line 100) | func (s *Source) IsDefault() bool {
method HasRecursiveSupport (line 105) | func (s *Source) HasRecursiveSupport() bool {
method KeyRequirement (line 110) | func (s *Source) KeyRequirement() subscraping.KeyRequirement {
method NeedsKey (line 115) | func (s *Source) NeedsKey() bool {
method AddApiKeys (line 120) | func (s *Source) AddApiKeys(keys []string) {
method Statistics (line 125) | func (s *Source) Statistics() subscraping.Statistics {
method runSubsource (line 136) | func (s *Source) runSubsource(ctx context.Context, domain string, sess...
type endpointConfig (line 36) | type endpointConfig struct
type summaryResponse (line 56) | type summaryResponse struct
FILE: pkg/subscraping/sources/facebook/ctlogs.go
type apiKey (line 27) | type apiKey struct
method FetchAccessToken (line 37) | func (k *apiKey) FetchAccessToken() {
method IsValid (line 70) | func (k *apiKey) IsValid() bool {
type Source (line 75) | type Source struct
method Run (line 85) | func (s *Source) Run(ctx context.Context, domain string, session *subs...
method Name (line 154) | func (s *Source) Name() string {
method IsDefault (line 159) | func (s *Source) IsDefault() bool {
method HasRecursiveSupport (line 165) | func (s *Source) HasRecursiveSupport() bool {
method KeyRequirement (line 170) | func (s *Source) KeyRequirement() subscraping.KeyRequirement {
method NeedsKey (line 175) | func (s *Source) NeedsKey() bool {
method AddApiKeys (line 180) | func (s *Source) AddApiKeys(keys []string) {
method Statistics (line 198) | func (s *Source) Statistics() subscraping.Statistics {
function updateParamInURL (line 208) | func updateParamInURL(url, param, value string) string {
FILE: pkg/subscraping/sources/facebook/ctlogs_test.go
function TestFacebookSource (line 21) | func TestFacebookSource(t *testing.T) {
function updateWithEnv (line 63) | func updateWithEnv(key *string) {
FILE: pkg/subscraping/sources/facebook/types.go
type authResponse (line 3) | type authResponse struct
type response (line 29) | type response struct
FILE: pkg/subscraping/sources/fofa/fofa.go
type fofaResponse (line 17) | type fofaResponse struct
type Source (line 25) | type Source struct
method Run (line 40) | func (s *Source) Run(ctx context.Context, domain string, session *subs...
method Name (line 111) | func (s *Source) Name() string {
method IsDefault (line 115) | func (s *Source) IsDefault() bool {
method HasRecursiveSupport (line 119) | func (s *Source) HasRecursiveSupport() bool {
method KeyRequirement (line 123) | func (s *Source) KeyRequirement() subscraping.KeyRequirement {
method NeedsKey (line 127) | func (s *Source) NeedsKey() bool {
method AddApiKeys (line 131) | func (s *Source) AddApiKeys(keys []string) {
method Statistics (line 137) | func (s *Source) Statistics() subscraping.Statistics {
type apiKey (line 34) | type apiKey struct
FILE: pkg/subscraping/sources/fullhunt/fullhunt.go
type fullHuntResponse (line 14) | type fullHuntResponse struct
type Source (line 21) | type Source struct
method Run (line 30) | func (s *Source) Run(ctx context.Context, domain string, session *subs...
method Name (line 80) | func (s *Source) Name() string {
method IsDefault (line 84) | func (s *Source) IsDefault() bool {
method HasRecursiveSupport (line 88) | func (s *Source) HasRecursiveSupport() bool {
method KeyRequirement (line 92) | func (s *Source) KeyRequirement() subscraping.KeyRequirement {
method NeedsKey (line 96) | func (s *Source) NeedsKey() bool {
method AddApiKeys (line 100) | func (s *Source) AddApiKeys(keys []string) {
method Statistics (line 104) | func (s *Source) Statistics() subscraping.Statistics {
FILE: pkg/subscraping/sources/github/github.go
type textMatch (line 26) | type textMatch struct
type item (line 30) | type item struct
type response (line 36) | type response struct
type Source (line 42) | type Source struct
method Run (line 52) | func (s *Source) Run(ctx context.Context, domain string, session *subs...
method enumerate (line 79) | func (s *Source) enumerate(ctx context.Context, searchURL string, doma...
method proccesItems (line 154) | func (s *Source) proccesItems(ctx context.Context, items []item, domai...
method Name (line 261) | func (s *Source) Name() string {
method IsDefault (line 265) | func (s *Source) IsDefault() bool {
method HasRecursiveSupport (line 269) | func (s *Source) HasRecursiveSupport() bool {
method KeyRequirement (line 273) | func (s *Source) KeyRequirement() subscraping.KeyRequirement {
method NeedsKey (line 277) | func (s *Source) NeedsKey() bool {
method AddApiKeys (line 281) | func (s *Source) AddApiKeys(keys []string) {
method Statistics (line 285) | func (s *Source) Statistics() subscraping.Statistics {
function normalizeContent (line 241) | func normalizeContent(content string) string {
function rawURL (line 249) | func rawURL(htmlURL string) string {
function domainRegexp (line 255) | func domainRegexp(domain string) *regexp.Regexp {
FILE: pkg/subscraping/sources/github/tokenmanager.go
type Token (line 6) | type Token struct
type Tokens (line 14) | type Tokens struct
method setCurrentTokenExceeded (line 33) | func (r *Tokens) setCurrentTokenExceeded(retryAfter int64) {
method Get (line 44) | func (r *Tokens) Get() *Token {
function NewTokenManager (line 20) | func NewTokenManager(keys []string) *Tokens {
function resetExceededTokens (line 57) | func resetExceededTokens(r *Tokens) {
FILE: pkg/subscraping/sources/gitlab/gitlab.go
type Source (line 21) | type Source struct
method Run (line 38) | func (s *Source) Run(ctx context.Context, domain string, session *subs...
method enumerate (line 65) | func (s *Source) enumerate(ctx context.Context, searchURL string, doma...
method Name (line 156) | func (s *Source) Name() string {
method IsDefault (line 160) | func (s *Source) IsDefault() bool {
method HasRecursiveSupport (line 164) | func (s *Source) HasRecursiveSupport() bool {
method KeyRequirement (line 168) | func (s *Source) KeyRequirement() subscraping.KeyRequirement {
method NeedsKey (line 172) | func (s *Source) NeedsKey() bool {
method AddApiKeys (line 176) | func (s *Source) AddApiKeys(keys []string) {
method Statistics (line 181) | func (s *Source) Statistics() subscraping.Statistics {
type item (line 30) | type item struct
function domainRegexp (line 150) | func domainRegexp(domain string) *regexp.Regexp {
FILE: pkg/subscraping/sources/hackertarget/hackertarget.go
type Source (line 14) | type Source struct
method Run (line 24) | func (s *Source) Run(ctx context.Context, domain string, session *subs...
method Name (line 87) | func (s *Source) Name() string {
method IsDefault (line 91) | func (s *Source) IsDefault() bool {
method HasRecursiveSupport (line 95) | func (s *Source) HasRecursiveSupport() bool {
method KeyRequirement (line 99) | func (s *Source) KeyRequirement() subscraping.KeyRequirement {
method NeedsKey (line 103) | func (s *Source) NeedsKey() bool {
method AddApiKeys (line 107) | func (s *Source) AddApiKeys(keys []string) {
method Statistics (line 111) | func (s *Source) Statistics() subscraping.Statistics {
FILE: pkg/subscraping/sources/hudsonrock/hudsonrock.go
type hudsonrockResponse (line 13) | type hudsonrockResponse struct
type Source (line 25) | type Source struct
method Run (line 33) | func (s *Source) Run(ctx context.Context, domain string, session *subs...
method Name (line 81) | func (s *Source) Name() string {
method IsDefault (line 85) | func (s *Source) IsDefault() bool {
method HasRecursiveSupport (line 89) | func (s *Source) HasRecursiveSupport() bool {
method KeyRequirement (line 93) | func (s *Source) KeyRequirement() subscraping.KeyRequirement {
method NeedsKey (line 97) | func (s *Source) NeedsKey() bool {
method AddApiKeys (line 101) | func (s *Source) AddApiKeys(_ []string) {
method Statistics (line 105) | func (s *Source) Statistics() subscraping.Statistics {
FILE: pkg/subscraping/sources/intelx/intelx.go
type searchResponseType (line 17) | type searchResponseType struct
type selectorType (line 22) | type selectorType struct
type searchResultType (line 26) | type searchResultType struct
type requestBody (line 31) | type requestBody struct
type Source (line 41) | type Source struct
method Run (line 56) | func (s *Source) Run(ctx context.Context, domain string, session *subs...
method Name (line 160) | func (s *Source) Name() string {
method IsDefault (line 164) | func (s *Source) IsDefault() bool {
method HasRecursiveSupport (line 168) | func (s *Source) HasRecursiveSupport() bool {
method KeyRequirement (line 172) | func (s *Source) KeyRequirement() subscraping.KeyRequirement {
method NeedsKey (line 176) | func (s *Source) NeedsKey() bool {
method AddApiKeys (line 180) | func (s *Source) AddApiKeys(keys []string) {
method Statistics (line 186) | func (s *Source) Statistics() subscraping.Statistics {
type apiKey (line 50) | type apiKey struct
FILE: pkg/subscraping/sources/leakix/leakix.go
type Source (line 14) | type Source struct
method Run (line 24) | func (s *Source) Run(ctx context.Context, domain string, session *subs...
method Name (line 82) | func (s *Source) Name() string {
method IsDefault (line 86) | func (s *Source) IsDefault() bool {
method HasRecursiveSupport (line 90) | func (s *Source) HasRecursiveSupport() bool {
method KeyRequirement (line 94) | func (s *Source) KeyRequirement() subscraping.KeyRequirement {
method NeedsKey (line 98) | func (s *Source) NeedsKey() bool {
method AddApiKeys (line 102) | func (s *Source) AddApiKeys(keys []string) {
method Statistics (line 106) | func (s *Source) Statistics() subscraping.Statistics {
type subResponse (line 116) | type subResponse struct
FILE: pkg/subscraping/sources/merklemap/merklemap.go
type Source (line 19) | type Source struct
method Run (line 29) | func (s *Source) Run(ctx context.Context, domain string, session *subs...
method fetchAllPages (line 63) | func (s *Source) fetchAllPages(ctx context.Context, domain string, hea...
method fetchPage (line 99) | func (s *Source) fetchPage(ctx context.Context, baseURL string, page i...
method Name (line 132) | func (s *Source) Name() string {
method IsDefault (line 136) | func (s *Source) IsDefault() bool {
method HasRecursiveSupport (line 141) | func (s *Source) HasRecursiveSupport() bool {
method KeyRequirement (line 145) | func (s *Source) KeyRequirement() subscraping.KeyRequirement {
method NeedsKey (line 149) | func (s *Source) NeedsKey() bool {
method AddApiKeys (line 153) | func (s *Source) AddApiKeys(keys []string) {
method Statistics (line 157) | func (s *Source) Statistics() subscraping.Statistics {
type response (line 167) | type response struct
FILE: pkg/subscraping/sources/netlas/netlas.go
type Item (line 18) | type Item struct
type DomainsCountResponse (line 33) | type DomainsCountResponse struct
type Source (line 38) | type Source struct
method Run (line 47) | func (s *Source) Run(ctx context.Context, domain string, session *subs...
method Name (line 179) | func (s *Source) Name() string {
method IsDefault (line 183) | func (s *Source) IsDefault() bool {
method HasRecursiveSupport (line 187) | func (s *Source) HasRecursiveSupport() bool {
method KeyRequirement (line 191) | func (s *Source) KeyRequirement() subscraping.KeyRequirement {
method NeedsKey (line 195) | func (s *Source) NeedsKey() bool {
method AddApiKeys (line 199) | func (s *Source) AddApiKeys(keys []string) {
method Statistics (line 203) | func (s *Source) Statistics() subscraping.Statistics {
FILE: pkg/subscraping/sources/onyphe/onyphe.go
type OnypheResponse (line 16) | type OnypheResponse struct
method UnmarshalJSON (line 184) | func (o *OnypheResponse) UnmarshalJSON(data []byte) error {
type Result (line 25) | type Result struct
method UnmarshalJSON (line 257) | func (r *Result) UnmarshalJSON(data []byte) error {
type Source (line 34) | type Source struct
method Run (line 44) | func (s *Source) Run(ctx context.Context, domain string, session *subs...
method Name (line 141) | func (s *Source) Name() string {
method IsDefault (line 145) | func (s *Source) IsDefault() bool {
method HasRecursiveSupport (line 149) | func (s *Source) HasRecursiveSupport() bool {
method KeyRequirement (line 153) | func (s *Source) KeyRequirement() subscraping.KeyRequirement {
method NeedsKey (line 157) | func (s *Source) NeedsKey() bool {
method AddApiKeys (line 161) | func (s *Source) AddApiKeys(keys []string) {
method Statistics (line 165) | func (s *Source) Statistics() subscraping.Statistics {
type OnypheResponseRaw (line 175) | type OnypheResponseRaw struct
type ResultRaw (line 248) | type ResultRaw struct
FILE: pkg/subscraping/sources/profundis/profundis.go
type Source (line 16) | type Source struct
method Run (line 26) | func (s *Source) Run(ctx context.Context, domain string, session *subs...
method Name (line 103) | func (s *Source) Name() string {
method IsDefault (line 107) | func (s *Source) IsDefault() bool {
method HasRecursiveSupport (line 111) | func (s *Source) HasRecursiveSupport() bool {
method KeyRequirement (line 115) | func (s *Source) KeyRequirement() subscraping.KeyRequirement {
method NeedsKey (line 119) | func (s *Source) NeedsKey() bool {
method AddApiKeys (line 123) | func (s *Source) AddApiKeys(keys []string) {
method Statistics (line 127) | func (s *Source) Statistics() subscraping.Statistics {
FILE: pkg/subscraping/sources/pugrecon/pugrecon.go
type pugreconResult (line 16) | type pugreconResult struct
type pugreconAPIResponse (line 21) | type pugreconAPIResponse struct
type Source (line 30) | type Source struct
method Run (line 40) | func (s *Source) Run(ctx context.Context, domain string, session *subs...
method Name (line 125) | func (s *Source) Name() string {
method IsDefault (line 130) | func (s *Source) IsDefault() bool {
method HasRecursiveSupport (line 135) | func (s *Source) HasRecursiveSupport() bool {
method KeyRequirement (line 140) | func (s *Source) KeyRequirement() subscraping.KeyRequirement {
method NeedsKey (line 145) | func (s *Source) NeedsKey() bool {
method AddApiKeys (line 150) | func (s *Source) AddApiKeys(keys []string) {
method Statistics (line 155) | func (s *Source) Statistics() subscraping.Statistics {
FILE: pkg/subscraping/sources/quake/quake.go
type quakeResults (line 16) | type quakeResults struct
type Source (line 34) | type Source struct
method Run (line 44) | func (s *Source) Run(ctx context.Context, domain string, session *subs...
method Name (line 133) | func (s *Source) Name() string {
method IsDefault (line 137) | func (s *Source) IsDefault() bool {
method HasRecursiveSupport (line 141) | func (s *Source) HasRecursiveSupport() bool {
method KeyRequirement (line 145) | func (s *Source) KeyRequirement() subscraping.KeyRequirement {
method NeedsKey (line 149) | func (s *Source) NeedsKey() bool {
method AddApiKeys (line 153) | func (s *Source) AddApiKeys(keys []string) {
method Statistics (line 157) | func (s *Source) Statistics() subscraping.Statistics {
FILE: pkg/subscraping/sources/rapiddns/rapiddns.go
type Source (line 18) | type Source struct
method Run (line 26) | func (s *Source) Run(ctx context.Context, domain string, session *subs...
method Name (line 96) | func (s *Source) Name() string {
method IsDefault (line 100) | func (s *Source) IsDefault() bool {
method HasRecursiveSupport (line 104) | func (s *Source) HasRecursiveSupport() bool {
method KeyRequirement (line 108) | func (s *Source) KeyRequirement() subscraping.KeyRequirement {
method NeedsKey (line 112) | func (s *Source) NeedsKey() bool {
method AddApiKeys (line 116) | func (s *Source) AddApiKeys(_ []string) {
method Statistics (line 120) | func (s *Source) Statistics() subscraping.Statistics {
FILE: pkg/subscraping/sources/reconcloud/reconcloud.go
type reconCloudResponse (line 13) | type reconCloudResponse struct
type cloudAssetsList (line 21) | type cloudAssetsList struct
type Source (line 28) | type Source struct
method Run (line 36) | func (s *Source) Run(ctx context.Context, domain string, session *subs...
method Name (line 83) | func (s *Source) Name() string {
method IsDefault (line 87) | func (s *Source) IsDefault() bool {
method HasRecursiveSupport (line 91) | func (s *Source) HasRecursiveSupport() bool {
method KeyRequirement (line 95) | func (s *Source) KeyRequirement() subscraping.KeyRequirement {
method NeedsKey (line 99) | func (s *Source) NeedsKey() bool {
method AddApiKeys (line 103) | func (s *Source) AddApiKeys(_ []string) {
method Statistics (line 107) | func (s *Source) Statistics() subscraping.Statistics {
FILE: pkg/subscraping/sources/reconeer/reconeer.go
type response (line 12) | type response struct
type subdomain (line 16) | type subdomain struct
type Source (line 20) | type Source struct
method Run (line 29) | func (s *Source) Run(ctx context.Context, domain string, session *subs...
method Name (line 84) | func (s *Source) Name() string {
method IsDefault (line 88) | func (s *Source) IsDefault() bool {
method HasRecursiveSupport (line 92) | func (s *Source) HasRecursiveSupport() bool {
method KeyRequirement (line 96) | func (s *Source) KeyRequirement() subscraping.KeyRequirement {
method NeedsKey (line 100) | func (s *Source) NeedsKey() bool {
method AddApiKeys (line 104) | func (s *Source) AddApiKeys(keys []string) {
method Statistics (line 108) | func (s *Source) Statistics() subscraping.Statistics {
FILE: pkg/subscraping/sources/redhuntlabs/redhuntlabs.go
type Response (line 15) | type Response struct
type ResponseMetadata (line 20) | type ResponseMetadata struct
type Source (line 26) | type Source struct
method Run (line 35) | func (s *Source) Run(ctx context.Context, domain string, session *subs...
method Name (line 131) | func (s *Source) Name() string {
method IsDefault (line 135) | func (s *Source) IsDefault() bool {
method HasRecursiveSupport (line 139) | func (s *Source) HasRecursiveSupport() bool {
method KeyRequirement (line 143) | func (s *Source) KeyRequirement() subscraping.KeyRequirement {
method NeedsKey (line 147) | func (s *Source) NeedsKey() bool {
method AddApiKeys (line 151) | func (s *Source) AddApiKeys(keys []string) {
method Statistics (line 155) | func (s *Source) Statistics() subscraping.Statistics {
FILE: pkg/subscraping/sources/riddler/riddler.go
type Source (line 14) | type Source struct
method Run (line 22) | func (s *Source) Run(ctx context.Context, domain string, session *subs...
method Name (line 72) | func (s *Source) Name() string {
method IsDefault (line 76) | func (s *Source) IsDefault() bool {
method HasRecursiveSupport (line 80) | func (s *Source) HasRecursiveSupport() bool {
method KeyRequirement (line 84) | func (s *Source) KeyRequirement() subscraping.KeyRequirement {
method NeedsKey (line 88) | func (s *Source) NeedsKey() bool {
method AddApiKeys (line 92) | func (s *Source) AddApiKeys(_ []string) {
method Statistics (line 96) | func (s *Source) Statistics() subscraping.Statistics {
FILE: pkg/subscraping/sources/robtex/robtext.go
constant addrRecord (line 17) | addrRecord = "A"
constant iPv6AddrRecord (line 18) | iPv6AddrRecord = "AAAA"
constant baseURL (line 19) | baseURL = "https://proapi.robtex.com/pdns"
type Source (line 23) | type Source struct
method Run (line 38) | func (s *Source) Run(ctx context.Context, domain string, session *subs...
method Name (line 122) | func (s *Source) Name() string {
method IsDefault (line 126) | func (s *Source) IsDefault() bool {
method HasRecursiveSupport (line 130) | func (s *Source) HasRecursiveSupport() bool {
method KeyRequirement (line 134) | func (s *Source) KeyRequirement() subscraping.KeyRequirement {
method NeedsKey (line 138) | func (s *Source) NeedsKey() bool {
method AddApiKeys (line 142) | func (s *Source) AddApiKeys(keys []string) {
method Statistics (line 146) | func (s *Source) Statistics() subscraping.Statistics {
type result (line 31) | type result struct
function enumerate (line 92) | func enumerate(ctx context.Context, session *subscraping.Session, target...
FILE: pkg/subscraping/sources/rsecloud/rsecloud.go
type response (line 13) | type response struct
type Source (line 22) | type Source struct
method Run (line 32) | func (s *Source) Run(ctx context.Context, domain string, session *subs...
method Name (line 102) | func (s *Source) Name() string {
method IsDefault (line 106) | func (s *Source) IsDefault() bool {
method HasRecursiveSupport (line 110) | func (s *Source) HasRecursiveSupport() bool {
method KeyRequirement (line 114) | func (s *Source) KeyRequirement() subscraping.KeyRequirement {
method NeedsKey (line 118) | func (s *Source) NeedsKey() bool {
method AddApiKeys (line 122) | func (s *Source) AddApiKeys(keys []string) {
method Statistics (line 126) | func (s *Source) Statistics() subscraping.Statistics {
FILE: pkg/subscraping/sources/securitytrails/securitytrails.go
type response (line 18) | type response struct
type Source (line 29) | type Source struct
method Run (line 39) | func (s *Source) Run(ctx context.Context, domain string, session *subs...
method Name (line 138) | func (s *Source) Name() string {
method IsDefault (line 142) | func (s *Source) IsDefault() bool {
method HasRecursiveSupport (line 146) | func (s *Source) HasRecursiveSupport() bool {
method KeyRequirement (line 150) | func (s *Source) KeyRequirement() subscraping.KeyRequirement {
method NeedsKey (line 154) | func (s *Source) NeedsKey() bool {
method AddApiKeys (line 158) | func (s *Source) AddApiKeys(keys []string) {
method Statistics (line 162) | func (s *Source) Statistics() subscraping.Statistics {
FILE: pkg/subscraping/sources/shodan/shodan.go
type Source (line 15) | type Source struct
method Run (line 33) | func (s *Source) Run(ctx context.Context, domain string, session *subs...
method Name (line 109) | func (s *Source) Name() string {
method IsDefault (line 113) | func (s *Source) IsDefault() bool {
method HasRecursiveSupport (line 117) | func (s *Source) HasRecursiveSupport() bool {
method KeyRequirement (line 121) | func (s *Source) KeyRequirement() subscraping.KeyRequirement {
method NeedsKey (line 125) | func (s *Source) NeedsKey() bool {
method AddApiKeys (line 129) | func (s *Source) AddApiKeys(keys []string) {
method Statistics (line 133) | func (s *Source) Statistics() subscraping.Statistics {
type dnsdbLookupResponse (line 24) | type dnsdbLookupResponse struct
FILE: pkg/subscraping/sources/sitedossier/sitedossier.go
constant SleepRandIntn (line 17) | SleepRandIntn = 5
type Source (line 22) | type Source struct
method Run (line 30) | func (s *Source) Run(ctx context.Context, domain string, session *subs...
method enumerate (line 48) | func (s *Source) enumerate(ctx context.Context, session *subscraping.S...
method Name (line 91) | func (s *Source) Name() string {
method IsDefault (line 95) | func (s *Source) IsDefault() bool {
method HasRecursiveSupport (line 99) | func (s *Source) HasRecursiveSupport() bool {
method KeyRequirement (line 103) | func (s *Source) KeyRequirement() subscraping.KeyRequirement {
method NeedsKey (line 107) | func (s *Source) NeedsKey() bool {
method AddApiKeys (line 111) | func (s *Source) AddApiKeys(_ []string) {
method Statistics (line 115) | func (s *Source) Statistics() subscraping.Statistics {
FILE: pkg/subscraping/sources/thc/thc.go
type response (line 15) | type response struct
type Source (line 23) | type Source struct
method Run (line 38) | func (s *Source) Run(ctx context.Context, domain string, session *subs...
method Name (line 103) | func (s *Source) Name() string {
method IsDefault (line 107) | func (s *Source) IsDefault() bool {
method HasRecursiveSupport (line 111) | func (s *Source) HasRecursiveSupport() bool {
method KeyRequirement (line 115) | func (s *Source) KeyRequirement() subscraping.KeyRequirement {
method NeedsKey (line 119) | func (s *Source) NeedsKey() bool {
method AddApiKeys (line 123) | func (s *Source) AddApiKeys(_ []string) {
method Statistics (line 127) | func (s *Source) Statistics() subscraping.Statistics {
type requestBody (line 31) | type requestBody struct
FILE: pkg/subscraping/sources/threatbook/threatbook.go
type threatBookResponse (line 15) | type threatBookResponse struct
type Source (line 28) | type Source struct
method Run (line 38) | func (s *Source) Run(ctx context.Context, domain string, session *subs...
method Name (line 107) | func (s *Source) Name() string {
method IsDefault (line 111) | func (s *Source) IsDefault() bool {
method HasRecursiveSupport (line 115) | func (s *Source) HasRecursiveSupport() bool {
method KeyRequirement (line 119) | func (s *Source) KeyRequirement() subscraping.KeyRequirement {
method NeedsKey (line 123) | func (s *Source) NeedsKey() bool {
method AddApiKeys (line 127) | func (s *Source) AddApiKeys(keys []string) {
method Statistics (line 131) | func (s *Source) Statistics() subscraping.Statistics {
FILE: pkg/subscraping/sources/threatcrowd/threatcrowd.go
type threatCrowdResponse (line 15) | type threatCrowdResponse struct
type Source (line 22) | type Source struct
method Run (line 30) | func (s *Source) Run(ctx context.Context, domain string, session *subs...
method Name (line 100) | func (s *Source) Name() string {
method IsDefault (line 105) | func (s *Source) IsDefault() bool {
method HasRecursiveSupport (line 110) | func (s *Source) HasRecursiveSupport() bool {
method KeyRequirement (line 115) | func (s *Source) KeyRequirement() subscraping.KeyRequirement {
method NeedsKey (line 120) | func (s *Source) NeedsKey() bool {
method AddApiKeys (line 125) | func (s *Source) AddApiKeys(_ []string) {}
method Statistics (line 128) | func (s *Source) Statistics() subscraping.Statistics {
FILE: pkg/subscraping/sources/threatminer/threatminer.go
type response (line 14) | type response struct
type Source (line 21) | type Source struct
method Run (line 29) | func (s *Source) Run(ctx context.Context, domain string, session *subs...
method Name (line 74) | func (s *Source) Name() string {
method IsDefault (line 78) | func (s *Source) IsDefault() bool {
method HasRecursiveSupport (line 82) | func (s *Source) HasRecursiveSupport() bool {
method KeyRequirement (line 86) | func (s *Source) KeyRequirement() subscraping.KeyRequirement {
method NeedsKey (line 90) | func (s *Source) NeedsKey() bool {
method AddApiKeys (line 94) | func (s *Source) AddApiKeys(_ []string) {
method Statistics (line 98) | func (s *Source) Statistics() subscraping.Statistics {
FILE: pkg/subscraping/sources/urlscan/urlscan.go
constant baseURL (line 18) | baseURL = "https://urlscan.io/api/v1/search/"
constant maxPages (line 20) | maxPages = 5
constant maxPerPage (line 22) | maxPerPage = 100
type response (line 26) | type response struct
type Source (line 43) | type Source struct
method Run (line 53) | func (s *Source) Run(ctx context.Context, domain string, session *subs...
method enumerate (line 82) | func (s *Source) enumerate(ctx context.Context, domain string, headers...
method Name (line 183) | func (s *Source) Name() string {
method IsDefault (line 187) | func (s *Source) IsDefault() bool {
method HasRecursiveSupport (line 191) | func (s *Source) HasRecursiveSupport() bool {
method KeyRequirement (line 195) | func (s *Source) KeyRequirement() subscraping.KeyRequirement {
method NeedsKey (line 199) | func (s *Source) NeedsKey() bool {
method AddApiKeys (line 203) | func (s *Source) AddApiKeys(keys []string) {
method Statistics (line 207) | func (s *Source) Statistics() subscraping.Statistics {
FILE: pkg/subscraping/sources/virustotal/virustotal.go
type response (line 14) | type response struct
type Object (line 19) | type Object struct
type Meta (line 23) | type Meta struct
type Source (line 28) | type Source struct
method Run (line 38) | func (s *Source) Run(ctx context.Context, domain string, session *subs...
method Name (line 107) | func (s *Source) Name() string {
method IsDefault (line 111) | func (s *Source) IsDefault() bool {
method HasRecursiveSupport (line 115) | func (s *Source) HasRecursiveSupport() bool {
method KeyRequirement (line 119) | func (s *Source) KeyRequirement() subscraping.KeyRequirement {
method NeedsKey (line 123) | func (s *Source) NeedsKey() bool {
method AddApiKeys (line 127) | func (s *Source) AddApiKeys(keys []string) {
method Statistics (line 131) | func (s *Source) Statistics() subscraping.Statistics {
FILE: pkg/subscraping/sources/waybackarchive/waybackarchive.go
type Source (line 16) | type Source struct
method Run (line 24) | func (s *Source) Run(ctx context.Context, domain string, session *subs...
method Name (line 78) | func (s *Source) Name() string {
method IsDefault (line 82) | func (s *Source) IsDefault() bool {
method HasRecursiveSupport (line 86) | func (s *Source) HasRecursiveSupport() bool {
method KeyRequirement (line 90) | func (s *Source) KeyRequirement() subscraping.KeyRequirement {
method NeedsKey (line 94) | func (s *Source) NeedsKey() bool {
method AddApiKeys (line 98) | func (s *Source) AddApiKeys(_ []string) {
method Statistics (line 102) | func (s *Source) Statistics() subscraping.Statistics {
FILE: pkg/subscraping/sources/whoisxmlapi/whoisxmlapi.go
type response (line 14) | type response struct
type Result (line 19) | type Result struct
type Record (line 24) | type Record struct
type Source (line 31) | type Source struct
method Run (line 41) | func (s *Source) Run(ctx context.Context, domain string, session *subs...
method Name (line 93) | func (s *Source) Name() string {
method IsDefault (line 97) | func (s *Source) IsDefault() bool {
method HasRecursiveSupport (line 101) | func (s *Source) HasRecursiveSupport() bool {
method KeyRequirement (line 105) | func (s *Source) KeyRequirement() subscraping.KeyRequirement {
method NeedsKey (line 109) | func (s *Source) NeedsKey() bool {
method AddApiKeys (line 113) | func (s *Source) AddApiKeys(keys []string) {
method Statistics (line 117) | func (s *Source) Statistics() subscraping.Statistics {
FILE: pkg/subscraping/sources/windvane/windvane.go
type response (line 15) | type response struct
type responseData (line 21) | type responseData struct
type domainEntry (line 26) | type domainEntry struct
type pageInfo (line 30) | type pageInfo struct
type Source (line 36) | type Source struct
method Run (line 45) | func (s *Source) Run(ctx context.Context, domain string, session *subs...
method Name (line 130) | func (s *Source) Name() string {
method IsDefault (line 134) | func (s *Source) IsDefault() bool {
method HasRecursiveSupport (line 138) | func (s *Source) HasRecursiveSupport() bool {
method KeyRequirement (line 142) | func (s *Source) KeyRequirement() subscraping.KeyRequirement {
method NeedsKey (line 146) | func (s *Source) NeedsKey() bool {
method AddApiKeys (line 150) | func (s *Source) AddApiKeys(keys []string) {
method Statistics (line 154) | func (s *Source) Statistics() subscraping.Statistics {
FILE: pkg/subscraping/sources/zoomeyeapi/zoomeyeapi.go
type zoomeyeResults (line 15) | type zoomeyeResults struct
type Source (line 25) | type Source struct
method Run (line 35) | func (s *Source) Run(ctx context.Context, domain string, session *subs...
method Name (line 112) | func (s *Source) Name() string {
method IsDefault (line 116) | func (s *Source) IsDefault() bool {
method HasRecursiveSupport (line 120) | func (s *Source) HasRecursiveSupport() bool {
method KeyRequirement (line 124) | func (s *Source) KeyRequirement() subscraping.KeyRequirement {
method NeedsKey (line 128) | func (s *Source) NeedsKey() bool {
method AddApiKeys (line 132) | func (s *Source) AddApiKeys(keys []string) {
method Statistics (line 136) | func (s *Source) Statistics() subscraping.Statistics {
FILE: pkg/subscraping/types.go
type CtxArg (line 12) | type CtxArg
constant CtxSourceArg (line 15) | CtxSourceArg CtxArg = "source"
type CustomRateLimit (line 18) | type CustomRateLimit struct
type BasicAuth (line 23) | type BasicAuth struct
type Statistics (line 29) | type Statistics struct
type KeyRequirement (line 38) | type KeyRequirement
constant NoKey (line 41) | NoKey KeyRequirement = iota
constant OptionalKey (line 42) | OptionalKey
constant RequiredKey (line 43) | RequiredKey
type Source (line 47) | type Source interface
type SubdomainExtractor (line 79) | type SubdomainExtractor interface
type Session (line 85) | type Session struct
type Result (line 97) | type Result struct
type ResultType (line 105) | type ResultType
constant Subdomain (line 109) | Subdomain ResultType = iota
constant Error (line 110) | Error
FILE: pkg/subscraping/utils.go
constant MultipleKeyPartsLength (line 10) | MultipleKeyPartsLength = 2
function PickRandom (line 12) | func PickRandom[T any](v []T, sourceName string) T {
function CreateApiKeys (line 22) | func CreateApiKeys[T any](keys []string, provider func(k, v string) T) [...
function createMultiPartKey (line 32) | func createMultiPartKey(key string) (keyPartA, keyPartB string, ok bool) {
FILE: pkg/testutils/integration.go
function RunSubfinderAndGetResults (line 10) | func RunSubfinderAndGetResults(debug bool, domain string, extra ...strin...
type TestCase (line 40) | type TestCase interface
Condensed preview — 111 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (418K chars).
[
{
"path": ".github/ISSUE_TEMPLATE/bug_report.md",
"chars": 395,
"preview": "---\nname: Bug report\nabout: Create a report to help us improve\ntitle: \"[Issue] \"\nlabels: ''\nassignees: ''\n\n---\n\n**Descri"
},
{
"path": ".github/ISSUE_TEMPLATE/config.yml",
"chars": 586,
"preview": "blank_issues_enabled: false\n\ncontact_links:\n - name: Ask an question / advise on using subfinder\n url: https://githu"
},
{
"path": ".github/ISSUE_TEMPLATE/feature_request.md",
"chars": 896,
"preview": "---\nname: Feature request\nabout: Request feature to implement in this project\nlabels: 'Type: Enhancement'\n---\n\n<!--\n1. P"
},
{
"path": ".github/ISSUE_TEMPLATE/issue-report.md",
"chars": 1289,
"preview": "---\nname: Issue report\nabout: Create a report to help us to improve the project\nlabels: 'Type: Bug'\n\n---\n\n<!-- \n1. Pleas"
},
{
"path": ".github/PULL_REQUEST_TEMPLATE.md",
"chars": 939,
"preview": "## Proposed changes\n\n<!-- Describe the overall picture of your modifications to help maintainers understand the pull req"
},
{
"path": ".github/dependabot.yml",
"chars": 1306,
"preview": "# To get started with Dependabot version updates, you'll need to specify which\n# package ecosystems to update and where "
},
{
"path": ".github/release.yml",
"chars": 328,
"preview": "changelog:\n exclude:\n authors:\n - dependabot\n categories:\n - title: 🎉 New Features\n labels:\n - "
},
{
"path": ".github/workflows/build-test.yml",
"chars": 2754,
"preview": "name: 🔨 Build Test\n\non:\n pull_request:\n paths:\n - \"**.go\"\n - \"**.mod\"\n workflow_dispatch:\n inputs:\n "
},
{
"path": ".github/workflows/codeql-analysis.yml",
"chars": 823,
"preview": "name: 🚨 CodeQL Analysis\n\non:\n workflow_dispatch:\n pull_request:\n paths:\n - '**.go'\n - '**.mod'\n\njobs:\n a"
},
{
"path": ".github/workflows/compat-checks.yaml",
"chars": 399,
"preview": "name: ♾️ Compatibility Checks\n\non:\n pull_request:\n types: [opened, synchronize]\n branches:\n - dev\n\njobs:\n c"
},
{
"path": ".github/workflows/dep-auto-merge.yml",
"chars": 516,
"preview": "name: 🤖 dep auto merge\n\non:\n pull_request:\n branches:\n - dev\n workflow_dispatch:\n\npermissions:\n pull-requests"
},
{
"path": ".github/workflows/dockerhub-push.yml",
"chars": 1094,
"preview": "name: 🌥 Docker Push\n\non:\n workflow_run:\n workflows: [\"🎉 Release Binary\"]\n types:\n - completed\n workflow_dis"
},
{
"path": ".github/workflows/release-binary.yml",
"chars": 796,
"preview": "name: 🎉 Release Binary\n\non:\n push:\n tags:\n - v*\n workflow_dispatch:\n\njobs:\n release:\n runs-on: ubuntu-late"
},
{
"path": ".github/workflows/release-test.yml",
"chars": 549,
"preview": "name: 🔨 Release Test\n\non:\n pull_request:\n paths:\n - '**.go'\n - '**.mod'\n workflow_dispatch:\n\njobs:\n rele"
},
{
"path": ".gitignore",
"chars": 213,
"preview": ".DS_Store\ncmd/subfinder/subfinder\n# subfinder binary when built with `go build`\nv2/cmd/subfinder/subfinder\n# subfinder b"
},
{
"path": ".goreleaser.yml",
"chars": 869,
"preview": "version: 2\n\nbefore:\n hooks:\n - go mod tidy\n\nbuilds:\n- env:\n - CGO_ENABLED=0\n goos:\n - windows\n - linux\n -"
},
{
"path": "DISCLAIMER.md",
"chars": 1320,
"preview": "## Disclaimer\n\nSubfinder leverages multiple open APIs, it is developed for individuals to help them for research or inte"
},
{
"path": "Dockerfile",
"chars": 333,
"preview": "# Build\nFROM golang:1.24-alpine AS build-env\nRUN apk add build-base\nWORKDIR /app\nCOPY . /app\nRUN go mod download\nRUN go "
},
{
"path": "LICENSE.md",
"chars": 1079,
"preview": "MIT License\n\nCopyright (c) 2021 ProjectDiscovery, Inc.\n\nPermission is hereby granted, free of charge, to any person obta"
},
{
"path": "Makefile",
"chars": 357,
"preview": "# Go parameters\nGOCMD=go\nGOBUILD=$(GOCMD) build\nGOMOD=$(GOCMD) mod\nGOTEST=$(GOCMD) test\nGOFLAGS := -v \nLDFLAGS := -s -w\n"
},
{
"path": "README.md",
"chars": 6637,
"preview": "<h1 align=\"center\">\n <img src=\"static/subfinder-logo.png\" alt=\"subfinder\" width=\"200px\">\n <br>\n</h1>\n\n<h4 align=\"cente"
},
{
"path": "THANKS.md",
"chars": 881,
"preview": "### Thanks\n\nMany people have contributed to subfinder making it a wonderful tool either by making a pull request fixing "
},
{
"path": "cmd/subfinder/main.go",
"chars": 587,
"preview": "package main\n\nimport (\n\t\"github.com/projectdiscovery/subfinder/v2/pkg/runner\"\n\t// Attempts to increase the OS file descr"
},
{
"path": "examples/main.go",
"chars": 1967,
"preview": "package main\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"io\"\n\t\"log\"\n\n\t\"github.com/projectdiscovery/subfinder/v2/pkg/runner\"\n)\n\nfunc "
},
{
"path": "go.mod",
"chars": 6541,
"preview": "module github.com/projectdiscovery/subfinder/v2\n\ngo 1.24.0\n\ntoolchain go1.24.1\n\nrequire (\n\tgithub.com/corpix/uarand v0.2"
},
{
"path": "go.sum",
"chars": 60963,
"preview": "aead.dev/minisign v0.2.0 h1:kAWrq/hBRu4AARY6AlciO83xhNnW9UaC8YipS2uhLPk=\naead.dev/minisign v0.2.0/go.mod h1:zdq6LdSd9Tbu"
},
{
"path": "pkg/passive/doc.go",
"chars": 110,
"preview": "// Package passive provides capability for doing passive subdomain\n// enumeration on targets.\npackage passive\n"
},
{
"path": "pkg/passive/passive.go",
"chars": 4295,
"preview": "package passive\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"math\"\n\t\"sort\"\n\t\"strings\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/projectdiscovery/ra"
},
{
"path": "pkg/passive/sources.go",
"chars": 7715,
"preview": "package passive\n\nimport (\n\t\"fmt\"\n\t\"os\"\n\t\"strings\"\n\n\t\"golang.org/x/exp/maps\"\n\n\t\"github.com/projectdiscovery/gologger\"\n\t\"g"
},
{
"path": "pkg/passive/sources_test.go",
"chars": 3982,
"preview": "package passive\n\nimport (\n\t\"fmt\"\n\t\"strconv\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/assert\"\n\t\"golang.org/x/exp/maps\"\n)"
},
{
"path": "pkg/passive/sources_w_auth_test.go",
"chars": 2287,
"preview": "package passive\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"math\"\n\t\"os\"\n\t\"reflect\"\n\t\"strings\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/stretch"
},
{
"path": "pkg/passive/sources_wo_auth_test.go",
"chars": 3091,
"preview": "package passive\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"math\"\n\t\"reflect\"\n\t\"strings\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/stretchr/test"
},
{
"path": "pkg/resolve/client.go",
"chars": 857,
"preview": "package resolve\n\nimport (\n\t\"github.com/projectdiscovery/dnsx/libs/dnsx\"\n)\n\n// DefaultResolvers contains the default list"
},
{
"path": "pkg/resolve/doc.go",
"chars": 134,
"preview": "// Package resolve is used to handle resolving records\n// It also handles wildcard subdomains and rotating resolvers.\npa"
},
{
"path": "pkg/resolve/resolve.go",
"chars": 2884,
"preview": "package resolve\n\nimport (\n\t\"fmt\"\n\t\"sync\"\n\n\t\"github.com/rs/xid\"\n)\n\nconst (\n\tmaxWildcardChecks = 3\n)\n\n// ResolutionPool is"
},
{
"path": "pkg/runner/banners.go",
"chars": 868,
"preview": "package runner\n\nimport (\n\t\"github.com/projectdiscovery/gologger\"\n\tupdateutils \"github.com/projectdiscovery/utils/update\""
},
{
"path": "pkg/runner/config.go",
"chars": 1598,
"preview": "package runner\n\nimport (\n\t\"os\"\n\t\"strings\"\n\n\t\"gopkg.in/yaml.v3\"\n\n\t\"github.com/projectdiscovery/gologger\"\n\t\"github.com/pro"
},
{
"path": "pkg/runner/doc.go",
"chars": 104,
"preview": "// Package runner implements the mechanism to drive the\n// subdomain enumeration process\npackage runner\n"
},
{
"path": "pkg/runner/enumerate.go",
"chars": 7774,
"preview": "package runner\n\nimport (\n\t\"context\"\n\t\"io\"\n\t\"strings\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/hako/durafmt\"\n\n\t\"github.com/projectdi"
},
{
"path": "pkg/runner/enumerate_test.go",
"chars": 4752,
"preview": "package runner\n\nimport (\n\t\"os\"\n\t\"testing\"\n\n\t\"github.com/stretchr/testify/require\"\n)\n\nfunc TestFilterAndMatchSubdomain(t "
},
{
"path": "pkg/runner/initialize.go",
"chars": 1344,
"preview": "package runner\n\nimport (\n\t\"net\"\n\t\"strings\"\n\n\t\"github.com/projectdiscovery/dnsx/libs/dnsx\"\n\t\"github.com/projectdiscovery/"
},
{
"path": "pkg/runner/options.go",
"chars": 13110,
"preview": "package runner\n\nimport (\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"math\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"regexp\"\n\t\"strings\"\n\n\t\"github.com/projec"
},
{
"path": "pkg/runner/outputter.go",
"chars": 6313,
"preview": "package runner\n\nimport (\n\t\"bufio\"\n\t\"errors\"\n\t\"io\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"strings\"\n\n\tjsoniter \"github.com/json-iterator"
},
{
"path": "pkg/runner/runner.go",
"chars": 5838,
"preview": "package runner\n\nimport (\n\t\"bufio\"\n\t\"context\"\n\t\"io\"\n\t\"math\"\n\t\"os\"\n\t\"path\"\n\t\"regexp\"\n\t\"strconv\"\n\t\"strings\"\n\n\t\"github.com/p"
},
{
"path": "pkg/runner/stats.go",
"chars": 1252,
"preview": "package runner\n\nimport (\n\t\"fmt\"\n\t\"sort\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/projectdiscovery/gologger\"\n\t\"github.com/project"
},
{
"path": "pkg/runner/util.go",
"chars": 646,
"preview": "package runner\n\nimport (\n\tfileutil \"github.com/projectdiscovery/utils/file\"\n\tstringsutil \"github.com/projectdiscovery/ut"
},
{
"path": "pkg/runner/validate.go",
"chars": 2655,
"preview": "package runner\n\nimport (\n\t\"errors\"\n\t\"fmt\"\n\t\"regexp\"\n\t\"strings\"\n\n\t\"github.com/projectdiscovery/gologger\"\n\t\"github.com/pro"
},
{
"path": "pkg/subscraping/agent.go",
"chars": 4544,
"preview": "package subscraping\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"crypto/tls\"\n\t\"fmt\"\n\t\"io\"\n\t\"net\"\n\t\"net/http\"\n\t\"net/url\"\n\t\"time\"\n\n\t\"gi"
},
{
"path": "pkg/subscraping/doc.go",
"chars": 81,
"preview": "// Package subscraping contains the logic of scraping agents\npackage subscraping\n"
},
{
"path": "pkg/subscraping/extractor.go",
"chars": 929,
"preview": "package subscraping\n\nimport (\n\t\"regexp\"\n\t\"strings\"\n)\n\n// RegexSubdomainExtractor is a concrete implementation of the Sub"
},
{
"path": "pkg/subscraping/sources/alienvault/alienvault.go",
"chars": 2883,
"preview": "// Package alienvault logic\npackage alienvault\n\nimport (\n\t\"context\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"time\"\n\n\t\"github.com/projec"
},
{
"path": "pkg/subscraping/sources/anubis/anubis.go",
"chars": 2266,
"preview": "// Package anubis logic\npackage anubis\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"net/http\"\n\t\"time\"\n\n\tjsoniter \"github.com/json-itera"
},
{
"path": "pkg/subscraping/sources/bevigil/bevigil.go",
"chars": 2535,
"preview": "// Package bevigil logic\npackage bevigil\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"time\"\n\n\tjsoniter \"github.com/json-iterator/go\"\n\n\t"
},
{
"path": "pkg/subscraping/sources/bufferover/bufferover.go",
"chars": 3392,
"preview": "// Package bufferover is a bufferover Scraping Engine in Golang\npackage bufferover\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"strings"
},
{
"path": "pkg/subscraping/sources/builtwith/builtwith.go",
"chars": 2830,
"preview": "// Package builtwith logic\npackage builtwith\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"time\"\n\n\tjsoniter \"github.com/json-iterator/go"
},
{
"path": "pkg/subscraping/sources/c99/c99.go",
"chars": 2937,
"preview": "// Package c99 logic\npackage c99\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"strings\"\n\t\"time\"\n\n\tjsoniter \"github.com/json-iterator/go\""
},
{
"path": "pkg/subscraping/sources/censys/censys.go",
"chars": 5729,
"preview": "// Package censys logic\npackage censys\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"net/http\"\n\t\"strings\"\n\t\"time\"\n\n\tjsoniter \"github.c"
},
{
"path": "pkg/subscraping/sources/censys/censys_test.go",
"chars": 3622,
"preview": "package censys\n\nimport (\n\t\"context\"\n\t\"math\"\n\t\"net/http\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/projectdiscovery/ratelimit\"\n\t\"g"
},
{
"path": "pkg/subscraping/sources/certspotter/certspotter.go",
"chars": 3863,
"preview": "// Package certspotter logic\npackage certspotter\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"time\"\n\n\tjsoniter \"github.com/json-iterato"
},
{
"path": "pkg/subscraping/sources/chaos/chaos.go",
"chars": 2105,
"preview": "// Package chaos logic\npackage chaos\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"time\"\n\n\t\"github.com/projectdiscovery/chaos-client/pkg"
},
{
"path": "pkg/subscraping/sources/chinaz/chinaz.go",
"chars": 2645,
"preview": "package chinaz\n\n// chinaz http://my.chinaz.com/ChinazAPI/DataCenter/MyDataApi\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"io\"\n\t\"time\"\n"
},
{
"path": "pkg/subscraping/sources/commoncrawl/commoncrawl.go",
"chars": 4116,
"preview": "// Package commoncrawl logic\npackage commoncrawl\n\nimport (\n\t\"bufio\"\n\t\"context\"\n\t\"fmt\"\n\t\"net/url\"\n\t\"strconv\"\n\t\"strings\"\n\t"
},
{
"path": "pkg/subscraping/sources/crtsh/crtsh.go",
"chars": 5980,
"preview": "// Package crtsh logic\npackage crtsh\n\nimport (\n\t\"context\"\n\t\"database/sql\"\n\t\"fmt\"\n\t\"strconv\"\n\t\"strings\"\n\t\"time\"\n\n\tjsonite"
},
{
"path": "pkg/subscraping/sources/digitalyama/digitalyama.go",
"chars": 3677,
"preview": "package digitalyama\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"time\"\n\n\tjsoniter \"github.com/json-iterator/go\"\n\n\t\"github.com/projectdi"
},
{
"path": "pkg/subscraping/sources/digitorus/digitorus.go",
"chars": 2533,
"preview": "// Package waybackarchive logic\npackage digitorus\n\nimport (\n\t\"bufio\"\n\t\"context\"\n\t\"fmt\"\n\t\"net/http\"\n\t\"strings\"\n\t\"time\"\n\n\t"
},
{
"path": "pkg/subscraping/sources/dnsdb/dnsdb.go",
"chars": 5986,
"preview": "// Package dnsdb logic\npackage dnsdb\n\nimport (\n\t\"bufio\"\n\t\"context\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"io\"\n\t\"net/url\"\n\t\"strconv\"\n\t"
},
{
"path": "pkg/subscraping/sources/dnsdumpster/dnsdumpster.go",
"chars": 2555,
"preview": "// Package dnsdumpster logic\npackage dnsdumpster\n\nimport (\n\t\"context\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"time\"\n\n\t\"github.com/proj"
},
{
"path": "pkg/subscraping/sources/dnsrepo/dnsrepo.go",
"chars": 2788,
"preview": "package dnsrepo\n\nimport (\n\t\"context\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"io\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/projectdiscovery/sub"
},
{
"path": "pkg/subscraping/sources/domainsproject/domainsproject.go",
"chars": 3219,
"preview": "// Package domainsproject logic\npackage domainsproject\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"strings\"\n\t\"time\"\n\n\tjsoniter \"github"
},
{
"path": "pkg/subscraping/sources/driftnet/driftnet.go",
"chars": 5452,
"preview": "package driftnet\n\nimport (\n\t\"context\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"net/http\"\n\t\"net/url\"\n\t\"strings\"\n\t\"sync\"\n\t\"sync/atomic\"\n\t"
},
{
"path": "pkg/subscraping/sources/facebook/ctlogs.go",
"chars": 5757,
"preview": "package facebook\n\nimport (\n\t\"context\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"io\"\n\t\"time\"\n\n\t\"github.com/projectdiscovery/gologger\"\n\t\"g"
},
{
"path": "pkg/subscraping/sources/facebook/ctlogs_test.go",
"chars": 1393,
"preview": "package facebook\n\nimport (\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"io\"\n\t\"os\"\n\t\"strings\"\n\t\"testing\"\n\n\t\"github.com/projectdiscovery/golo"
},
{
"path": "pkg/subscraping/sources/facebook/types.go",
"chars": 732,
"preview": "package facebook\n\ntype authResponse struct {\n\tAccessToken string `json:\"access_token\"`\n}\n\n/*\n{\n \"data\": [\n {\n \""
},
{
"path": "pkg/subscraping/sources/fofa/fofa.go",
"chars": 3559,
"preview": "// Package fofa logic\npackage fofa\n\nimport (\n\t\"context\"\n\t\"encoding/base64\"\n\t\"fmt\"\n\t\"regexp\"\n\t\"strings\"\n\t\"time\"\n\n\tjsonite"
},
{
"path": "pkg/subscraping/sources/fullhunt/fullhunt.go",
"chars": 2488,
"preview": "package fullhunt\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"time\"\n\n\tjsoniter \"github.com/json-iterator/go\"\n\n\t\"github.com/projectdisco"
},
{
"path": "pkg/subscraping/sources/github/github.go",
"chars": 7595,
"preview": "// Package github GitHub search package\n// Based on gwen001's https://github.com/gwen001/github-search github-subdomains"
},
{
"path": "pkg/subscraping/sources/github/tokenmanager.go",
"chars": 1299,
"preview": "package github\n\nimport \"time\"\n\n// Token struct\ntype Token struct {\n\tHash string\n\tRetryAfter int64\n\tExceededTim"
},
{
"path": "pkg/subscraping/sources/gitlab/gitlab.go",
"chars": 4711,
"preview": "package gitlab\n\nimport (\n\t\"bufio\"\n\t\"context\"\n\t\"fmt\"\n\t\"net/http\"\n\t\"net/url\"\n\t\"regexp\"\n\t\"strings\"\n\t\"sync\"\n\t\"sync/atomic\"\n\t"
},
{
"path": "pkg/subscraping/sources/hackertarget/hackertarget.go",
"chars": 2619,
"preview": "// Package hackertarget logic\npackage hackertarget\n\nimport (\n\t\"bufio\"\n\t\"context\"\n\t\"fmt\"\n\t\"time\"\n\n\t\"github.com/projectdis"
},
{
"path": "pkg/subscraping/sources/hudsonrock/hudsonrock.go",
"chars": 2567,
"preview": "// Package hudsonrock logic\npackage hudsonrock\n\nimport (\n\t\"context\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"time\"\n\n\t\"github.com/projec"
},
{
"path": "pkg/subscraping/sources/intelx/intelx.go",
"chars": 4445,
"preview": "// Package intelx logic\npackage intelx\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"io\"\n\t\"time\"\n\n\tjsoniter \"g"
},
{
"path": "pkg/subscraping/sources/leakix/leakix.go",
"chars": 2800,
"preview": "// Package leakix logic\npackage leakix\n\nimport (\n\t\"context\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"time\"\n\n\t\"github.com/projectdiscove"
},
{
"path": "pkg/subscraping/sources/merklemap/merklemap.go",
"chars": 4346,
"preview": "// Package merklemap logic\npackage merklemap\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"io\"\n\t\"math\"\n\t\"net/u"
},
{
"path": "pkg/subscraping/sources/netlas/netlas.go",
"chars": 5701,
"preview": "// Package netlas logic\npackage netlas\n\nimport (\n\t\"context\"\n\t\"io\"\n\t\"strings\"\n\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"net/http\"\n\t\"net/"
},
{
"path": "pkg/subscraping/sources/onyphe/onyphe.go",
"chars": 7640,
"preview": "// Package onyphe logic\npackage onyphe\n\nimport (\n\t\"context\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"net/http\"\n\t\"net/url\"\n\t\"strconv\"\n\t\""
},
{
"path": "pkg/subscraping/sources/profundis/profundis.go",
"chars": 2914,
"preview": "// Package profundis logic\npackage profundis\n\nimport (\n\t\"bufio\"\n\t\"bytes\"\n\t\"context\"\n\t\"encoding/json\"\n\t\"strings\"\n\t\"time\"\n"
},
{
"path": "pkg/subscraping/sources/pugrecon/pugrecon.go",
"chars": 4547,
"preview": "// Package pugrecon logic\npackage pugrecon\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"net/http\"\n\t\"time\"\n\n\t\""
},
{
"path": "pkg/subscraping/sources/quake/quake.go",
"chars": 3694,
"preview": "// Package quake logic\npackage quake\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"fmt\"\n\t\"strings\"\n\t\"time\"\n\n\tjsoniter \"github.com/json"
},
{
"path": "pkg/subscraping/sources/rapiddns/rapiddns.go",
"chars": 2721,
"preview": "// Package rapiddns is a RapidDNS Scraping Engine in Golang\npackage rapiddns\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"io\"\n\t\"regexp\""
},
{
"path": "pkg/subscraping/sources/reconcloud/reconcloud.go",
"chars": 2736,
"preview": "// Package reconcloud logic\npackage reconcloud\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"time\"\n\n\tjsoniter \"github.com/json-iterator/"
},
{
"path": "pkg/subscraping/sources/reconeer/reconeer.go",
"chars": 2583,
"preview": "package reconeer\n\nimport (\n\t\"context\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"time\"\n\n\t\"github.com/projectdiscovery/subfinder/v2/pkg/su"
},
{
"path": "pkg/subscraping/sources/redhuntlabs/redhuntlabs.go",
"chars": 4309,
"preview": "// Package redhuntlabs logic\npackage redhuntlabs\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"strings\"\n\t\"time\"\n\n\tjsoniter \"github.com/j"
},
{
"path": "pkg/subscraping/sources/riddler/riddler.go",
"chars": 2197,
"preview": "// Package riddler logic\npackage riddler\n\nimport (\n\t\"bufio\"\n\t\"context\"\n\t\"fmt\"\n\t\"time\"\n\n\t\"github.com/projectdiscovery/sub"
},
{
"path": "pkg/subscraping/sources/robtex/robtext.go",
"chars": 3407,
"preview": "// Package robtex logic\npackage robtex\n\nimport (\n\t\"bufio\"\n\t\"bytes\"\n\t\"context\"\n\t\"fmt\"\n\t\"time\"\n\n\tjsoniter \"github.com/json"
},
{
"path": "pkg/subscraping/sources/rsecloud/rsecloud.go",
"chars": 2997,
"preview": "package rsecloud\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"time\"\n\n\tjsoniter \"github.com/json-iterator/go\"\n\n\t\"github.com/projectdisco"
},
{
"path": "pkg/subscraping/sources/securitytrails/securitytrails.go",
"chars": 3963,
"preview": "// Package securitytrails logic\npackage securitytrails\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"fmt\"\n\t\"net/http\"\n\t\"strings\"\n\t\"tim"
},
{
"path": "pkg/subscraping/sources/shodan/shodan.go",
"chars": 2939,
"preview": "// Package shodan logic\npackage shodan\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"time\"\n\n\tjsoniter \"github.com/json-iterator/go\"\n\n\t\"g"
},
{
"path": "pkg/subscraping/sources/sitedossier/sitedossier.go",
"chars": 2832,
"preview": "// Package sitedossier logic\npackage sitedossier\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"io\"\n\t\"net/http\"\n\t\"regexp\"\n\t\"time\"\n\n\t\"gith"
},
{
"path": "pkg/subscraping/sources/thc/thc.go",
"chars": 2969,
"preview": "// Package thc logic\npackage thc\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"encoding/json\"\n\t\"time\"\n\n\tjsoniter \"github.com/json-iter"
},
{
"path": "pkg/subscraping/sources/threatbook/threatbook.go",
"chars": 3246,
"preview": "// Package threatbook logic\npackage threatbook\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"strconv\"\n\t\"time\"\n\n\tjsoniter \"github.com/jso"
},
{
"path": "pkg/subscraping/sources/threatcrowd/threatcrowd.go",
"chars": 3550,
"preview": "package threatcrowd\n\nimport (\n\t\"context\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"io\"\n\t\"net/http\"\n\t\"time\"\n\n\t\"github.com/projectdiscover"
},
{
"path": "pkg/subscraping/sources/threatminer/threatminer.go",
"chars": 2315,
"preview": "// Package threatminer logic\npackage threatminer\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"time\"\n\n\tjsoniter \"github.com/json-iterato"
},
{
"path": "pkg/subscraping/sources/urlscan/urlscan.go",
"chars": 5015,
"preview": "// Package urlscan logic\npackage urlscan\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"net/url\"\n\t\"strings\"\n\t\"time\"\n\n\tjsoniter \"github.co"
},
{
"path": "pkg/subscraping/sources/virustotal/virustotal.go",
"chars": 2976,
"preview": "// Package virustotal logic\npackage virustotal\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"time\"\n\n\tjsoniter \"github.com/json-iterator/"
},
{
"path": "pkg/subscraping/sources/waybackarchive/waybackarchive.go",
"chars": 2388,
"preview": "// Package waybackarchive logic\npackage waybackarchive\n\nimport (\n\t\"bufio\"\n\t\"context\"\n\t\"fmt\"\n\t\"net/url\"\n\t\"strings\"\n\t\"time"
},
{
"path": "pkg/subscraping/sources/whoisxmlapi/whoisxmlapi.go",
"chars": 2725,
"preview": "// Package whoisxmlapi logic\npackage whoisxmlapi\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"time\"\n\n\tjsoniter \"github.com/json-iterato"
},
{
"path": "pkg/subscraping/sources/windvane/windvane.go",
"chars": 3501,
"preview": "// Package windvane logic\npackage windvane\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"encoding/json\"\n\t\"net/http\"\n\t\"strconv\"\n\t\"time\""
},
{
"path": "pkg/subscraping/sources/zoomeyeapi/zoomeyeapi.go",
"chars": 3173,
"preview": "package zoomeyeapi\n\nimport (\n\t\"context\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"net/http\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/projectdisc"
},
{
"path": "pkg/subscraping/types.go",
"chars": 2605,
"preview": "package subscraping\n\nimport (\n\t\"context\"\n\t\"net/http\"\n\t\"time\"\n\n\t\"github.com/projectdiscovery/ratelimit\"\n\tmapsutil \"github"
},
{
"path": "pkg/subscraping/utils.go",
"chars": 886,
"preview": "package subscraping\n\nimport (\n\t\"math/rand\"\n\t\"strings\"\n\n\t\"github.com/projectdiscovery/gologger\"\n)\n\nconst MultipleKeyParts"
},
{
"path": "pkg/testutils/integration.go",
"chars": 852,
"preview": "package testutils\n\nimport (\n\t\"fmt\"\n\t\"os\"\n\t\"os/exec\"\n\t\"strings\"\n)\n\nfunc RunSubfinderAndGetResults(debug bool, domain stri"
}
]
About this extraction
This page contains the full source code of the projectdiscovery/subfinder GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 111 files (369.5 KB), approximately 122.5k tokens, and a symbol index with 739 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.