Full Code of Alikhalkhali/programs-watcher for AI

Repository: Alikhalkhali/programs-watcher
Branch: main
Commit: 0651949dd5bf
Files: 16
Total size: 46.5 KB

Directory structure:
gitextract_cqcphwt2/

├── .gitignore
├── Dockerfile
├── LICENSE
├── README.md
├── config.yml
├── crontab
├── docker-compose.yml
├── main.py
├── modules/
│   ├── notifier/
│   │   ├── discord.py
│   │   └── functions.py
│   └── platforms/
│       ├── bugcrowd.py
│       ├── functions.py
│       ├── hackerone.py
│       ├── intigriti.py
│       └── yeswehack.py
└── requirements.txt

================================================
FILE CONTENTS
================================================

================================================
FILE: .gitignore
================================================
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class

# C extensions
*.so

# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# PyInstaller
#  Usually these files are written by a python script from a template
#  before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec

# Installer logs
pip-log.txt
pip-delete-this-directory.txt

# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/

# Translations
*.mo
*.pot

# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal

# Flask stuff:
instance/
.webassets-cache

# Scrapy stuff:
.scrapy

# Sphinx documentation
docs/_build/

# PyBuilder
.pybuilder/
target/

# Jupyter Notebook
.ipynb_checkpoints

# IPython
profile_default/
ipython_config.py

# pyenv
#   For a library or package, you might want to ignore these files since the code is
#   intended to run in multiple environments; otherwise, check them in:
# .python-version

# pipenv
#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
#   However, in case of collaboration, if having platform-specific dependencies or dependencies
#   having no cross-platform support, pipenv may install dependencies that don't work, or not
#   install all needed dependencies.
#Pipfile.lock

# poetry
#   Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
#   This is especially recommended for binary packages to ensure reproducibility, and is more
#   commonly ignored for libraries.
#   https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
#poetry.lock

# pdm
#   Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
#pdm.lock
#   pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
#   in version control.
#   https://pdm.fming.dev/#use-with-ide
.pdm.toml

# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
__pypackages__/

# Celery stuff
celerybeat-schedule
celerybeat.pid

# SageMath parsed files
*.sage.py

# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# Spyder project settings
.spyderproject
.spyproject

# Rope project settings
.ropeproject

# mkdocs documentation
/site

# mypy
.mypy_cache/
.dmypy.json
dmypy.json

# Pyre type checker
.pyre/

# pytype static type analyzer
.pytype/

# Cython debug symbols
cython_debug/

# PyCharm
#  JetBrains specific template is maintained in a separate JetBrains.gitignore that can
#  be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
#  and can be added to the global gitignore or merged into this file.  For a more nuclear
#  option (not recommended) you can uncomment the following to ignore the entire idea folder.
#.idea/


================================================
FILE: Dockerfile
================================================
FROM python:3.9

RUN apt-get update && apt-get -y install cron

WORKDIR /app

COPY . .

RUN pip install -r requirements.txt

COPY crontab /etc/cron.d/crontab

RUN touch /var/log/cron.log

RUN chmod 0644 /etc/cron.d/crontab

RUN crontab /etc/cron.d/crontab

CMD cron && tail -f /var/log/cron.log

================================================
FILE: LICENSE
================================================
MIT License

Copyright (c) 2023 Ali Khalkhali

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.


================================================
FILE: README.md
================================================
<h2 align="center" >
  Table of Contents 
</h2>
<p align="center">
  <a href="#Installation">Installation</a> •
  <a href="#Configuring-Programs-Watcher">Configuring Programs Watcher</a> •
  <a href="#Contributing">Contributing</a> •
  <a href="#License">License</a> •
   <a href="#Contact">Contact</a> •
  </p>
  
# Programs Watcher
Programs Watcher is a Python program that monitors and notifies you of new updates from various bug bounty platforms. It uses MongoDB for data storage and Discord webhooks for notifications.
<br>
<br>
## Youtube Video

[![Video Thumbnail](https://github.com/Alikhalkhali/programs-watcher/blob/main/img/Thumbnail.jpg)](https://www.youtube.com/watch?v=V6d6_YVUSR8)
# Installation

To install Programs Watcher, you have two options depending on your needs.

## Option 1: Docker Installation (Recommended for server environments)

1. Clone the repository to your local machine:

```bash
git clone https://github.com/Alikhalkhali/programs-watcher.git
```
2. Change directory to the project folder:
```bash
cd programs-watcher
```
3. In the `config.yml` file, replace `<YOUR DISCORD WEBHOOK>` with your Discord webhook URL.
4. Run:
 ```bash
docker-compose up -d
```
## Option 2: Manual Installation (Recommended for desktop and laptop usage)

This option is suitable for users who want to run Programs Watcher on their local machines and manually trigger updates and notifications.

1. Install MongoDB:
   - Download and install MongoDB from the official website: [MongoDB Download](https://www.mongodb.com/try/download/community)
   - Follow the installation instructions for your operating system.

2. Clone the repository to your local machine:
‍‍‍
```bash
git clone https://github.com/Alikhalkhali/programs-watcher.git
```

3. Change directory to the project folder:
```bash
cd programs-watcher
```
4. In the `config.yml` file, replace `<YOUR DISCORD WEBHOOK>` with your Discord webhook URL and update the MongoDB connection URL to `mongodb://localhost:27017/`.

5. Install the required dependencies:
```bash
pip3 install -r requirements.txt
```
6. Run the program:
```bash
python3 main.py
```

Remember to manually run the program whenever you want to check for updates and receive notifications. This option is recommended for desktop and laptop installations.

# Configuring Programs Watcher

The Programs Watcher program uses a configuration file named `config.yml` to store information about the bug bounty platforms to monitor and the notification options to use.

## Discord Webhook

To use the Discord webhook, replace `<YOUR DISCORD WEBHOOK>` with the actual URL of your webhook in the following line:

```yaml
discordWebhook:
programs_watcher: <YOUR DISCORD WEBHOOK>
```

## MongoDB

- `uri`: This is the URI of the MongoDB database that the application will use to store data.
- `database`: This is the name of the database that the application will use to store data.

## Platforms

This section contains a list of bug bounty platforms to monitor. For each platform, provide the name, URL, and a set of notification options. The notification options specify which types of changes should trigger notifications.

### Monitor:

Specify the monitoring options for the platform.

- `rdp`: Set to true if you want to monitor RDP programs.
- `vdp`: Set to true if you want to monitor VDP programs.
- `excluded_programs`: List the URLs of the programs you wish to exclude from monitoring. Please note that this applies only to public programs.
- `specific_programs`:  List the specific program URLs you want to monitor. Note that it only monitors public programs. For example:
  - bugcrowd:
    ```yml
      specific_programs: 
        - https://bugcrowd.com/rarible-ogmbb
        - https://bugcrowd.com/smartthings
    ```
  - hackerone:
    ```yml
      specific_programs: 
        - https://hackerone.com/phabricator?type=team
        - https://hackerone.com/yahoo?type=team
    ```
  - intigriti:
    ```yml
      specific_programs: 
        - https://app.intigriti.com/programs/portofantwerp/nxtport
        - https://app.intigriti.com/programs/intergamma/intergamma
    ```
  - yeswehack:
    ```yml
      specific_programs: 
        - https://yeswehack.com/programs/swiss-post-evoting
        - https://yeswehack.com/programs/dana-bug-bounty-programd
    ```

### Notifications

To receive notifications for a specific type of change, set the corresponding notification option to `true`. The available notification options are:

- `new_program`: Notify when a new program is added.
- `removed_program`: Notify when a program is removed.
- `new_inscope`: Notify when a new scope is added.
- `removed_inscope`: Notify when a scope is removed.
- `new_out_of_scope`: Notify when a new out-of-scope item is added.
- `removed_out_of_scope`: Notify when an out-of-scope item is removed.
- `new_scope`: Notify when a scope is added or removed.
- `changed_scope`: Notify when a scope is modified.
- `new_type`: Notify when a new program type is added.
- `new_bounty_table`: Notify when a new bounty table is added.

Fill in the necessary information for each platform that you want to monitor.

# Contributing

If you would like to contribute to this project, please fork the repository and submit a pull request.

# License

This project is licensed under the MIT license. See the LICENSE file for details.

# Contact

If you have any questions or concerns, please feel free to contact me directly on social media:

- Twitter: [ali_khalkhali0](https://twitter.com/ali_khalkhali0)
- Instagram: [ali_khalkhali0](https://instagram.com/ali_khalkhali0)

I am always happy to hear from you and will do my best to respond to your questions as soon as possible.
<br>
#### For Support:

[!["Buy Me A Coffee"](https://www.buymeacoffee.com/assets/img/custom_images/orange_img.png)](https://www.buymeacoffee.com/alikhalkhali)


================================================
FILE: config.yml
================================================
discordWebhook:
  programs_watcher: <discord-webhook>
mongoDB:
  uri: mongodb://db:27017/
  database: programs_watcher
platforms:
  - name: bugcrowd
    url: https://raw.githubusercontent.com/Osb0rn3/bugbounty-targets/main/programs/bugcrowd.json
    monitor:
      rdp: true
      vdp: true
      specific_programs: []
      excluded_programs: []
    notifications:
      new_program: true
      removed_program: true
      new_inscope: true
      removed_inscope: true
      new_out_of_scope: true
      removed_out_of_scope: true
      new_type: true
      new_bounty_table: true
  - name: hackerone
    url: https://raw.githubusercontent.com/Osb0rn3/bugbounty-targets/main/programs/hackerone.json
    monitor:
      rdp: true
      vdp: true
      specific_programs: []
      excluded_programs: 
        - https://hackerone.com/alshaya?type=team
        - https://hackerone.com/gsa_vdp?type=team
        - https://hackerone.com/clarivate?type=team
    notifications:
      new_program: true
      removed_program: true
      new_scope: true
      removed_scope: true
      changed_scope: true
      new_type: true
  - name: intigriti
    url: https://raw.githubusercontent.com/Osb0rn3/bugbounty-targets/main/programs/intigriti.json
    monitor:
      rdp: true
      vdp: true
      specific_programs: []
      excluded_programs: [] 
    notifications:
      new_program: true
      removed_program: true
      new_scope: true
      removed_scope: true
      changed_scope: true
      new_type: true
      new_bounty_table: true
  - name: yeswehack
    url: https://raw.githubusercontent.com/Osb0rn3/bugbounty-targets/main/programs/yeswehack.json
    monitor:
      rdp: true
      vdp: true
      specific_programs: []
      excluded_programs: []
    notifications:
      new_program: true
      removed_program: true
      new_inscope: true
      removed_inscope: true
      new_type: true
      new_bounty_table: true

================================================
FILE: crontab
================================================
*/30 * * * * root cd /app && /usr/local/bin/python main.py >> /var/log/cron.log 2>&1


================================================
FILE: docker-compose.yml
================================================
version: "3"

services:
  app:
    build: .
    volumes:
      - ./config.yml:/app/config.yml
  db:
    image: mongo:4.0-xenial
    volumes:
      - mongo_data:/data/db

volumes:
  mongo_data:


================================================
FILE: main.py
================================================
import yaml
from pymongo import MongoClient
import os
import shutil
from modules.platforms.bugcrowd import check_bugcrowd
from modules.platforms.hackerone import check_hackerone
from modules.platforms.intigriti import check_intigriti
from modules.platforms.yeswehack import check_yeswehack
from modules.notifier.discord import send_startup_message


# load config file
with open("config.yml", "r") as ymlfile:
    cfg = yaml.full_load(ymlfile)
discord_webhook = cfg['discordWebhook']['programs_watcher']
platforms = {}
for platform in cfg['platforms']:
    platforms[platform['name']] = {
        'url': platform['url'],
        'notifications': platform['notifications'],
        'monitor': platform['monitor']
    }

# connect to MongoDB
client = MongoClient(cfg['mongoDB']['uri'])
dbName = cfg['mongoDB']['database']
db = client[dbName]
first_time = False
if dbName not in client.list_database_names():
    first_time = True

# Check ./tmp directory exists
tmp_dir = f"./tmp/"
if os.path.exists(tmp_dir):
    shutil.rmtree(tmp_dir)
    os.mkdir(tmp_dir)
else:
    os.mkdir(tmp_dir)

# checking bugcrowd
check_bugcrowd(tmp_dir, discord_webhook, first_time, db, platforms['bugcrowd'])

# checking hackerone
check_hackerone(tmp_dir, discord_webhook, first_time, db, platforms['hackerone'])

# checking intigriti
check_intigriti(tmp_dir, discord_webhook, first_time, db, platforms['intigriti'])

# checking yeswehack
check_yeswehack(tmp_dir, discord_webhook, first_time, db, platforms['yeswehack'])

# Clean up resources and remove tmp_dir
client.close()
shutil.rmtree(tmp_dir)

if first_time:
    send_startup_message(discord_webhook)

================================================
FILE: modules/notifier/discord.py
================================================
from discord_webhook import DiscordWebhook, DiscordEmbed
from modules.notifier.functions import generate_diff, split_text, get_platform_profile, shorten_string


def add_field(embed, data, message, diff=False):
    texts = split_text(data)
    isFirstTime = 0
    type = ""
    if diff:
        type = "diff\n"
    for text in texts:
        if isFirstTime == 0:
            embed.add_embed_field(
                name=message, value=f"```{type}{text}```", inline=False)
            isFirstTime = 1
        else:
            embed.add_embed_field(
                name=" ", value=f"```{type}{text}```", inline=False)


def changed_program_message(data):
    embed = DiscordEmbed(title=f"{data['programName']}",
                         description=f"** {data['programName']} ** has changed!\n** Program type: ** {data['programType']}\n\n** Program page: ** [Click here]({data['programURL']})", color=data['color'])
    embed.set_thumbnail(url=data['logo'])
    embed.set_footer(text='Powered by Ali Khalkhali',
                     icon_url='https://cdn.discordapp.com/avatars/941457826662985808/488f3bcab0de041de57860b4e05e2e9f.webp')
    if data["platformName"] in ["HackerOne", "Intigriti"]:
        if data["newProgramType"]:
            embed.add_embed_field(name="Program type changed:",
                                  value=f"```changed to {data['newProgramType']}```", inline=False)
        if data["platformName"] == "Intigriti":
            if data['newReward']:
                embed.add_embed_field(
                    name="Bounty Table changed:", value=f"```{data['newReward']['min']} -- {data['newReward']['max']}```", inline=False)
        if data["newScope"]:
            scope = '\n'.join(data['newScope'])
            add_field(embed, scope, "Following scope added:")
        if data["removedScope"]:
            removedScope = '\n'.join(data['removedScope'])
            add_field(embed, removedScope, "Following Scope removed:")
        if data["changedScope"]:
            for scope in data["changedScope"]:
                old = scope["old"]
                new = scope["new"]
                diff = generate_diff(old, new)
                add_field(embed, diff, "Following scope changed:", True)
    if data["platformName"] in ["Bugcrowd", "YesWeHack"]:
        if data['newType']:
            embed.add_embed_field(
                name="Program Type changed", value=f"```New Type: {data['newType']}```", inline=False)
        if data['reward']:
            embed.add_embed_field(name="Bounty Table changed:",
                                  value=f"```{data['reward']['min']} -- {data['reward']['max']}```", inline=False)
        if data['newInScope']:
            inscope = '\n'.join(data['newInScope'])
            add_field(embed, inscope, "Following inScope added:")
        if data['removeInScope']:
            removeInScope = '\n'.join(data['removeInScope'])
            add_field(embed, removeInScope, "Following Inscope removed:")

        if data["platformName"] == "Bugcrowd":
            if data['newOutOfScope']:
                newOutOfScope = '\n'.join(data['newOutOfScope'])
                add_field(embed, newOutOfScope,
                          "Following out of scope added:")
            if data['removeOutOfScope']:
                removeOutOfScope = '\n'.join(data['removeOutOfScope'])
                add_field(embed, removeOutOfScope,
                          "Following out of scope removed:")
    embed.add_embed_field(name=" ", value= "[Click here to Buy me a coffee ☕](https://www.buymeacoffee.com/alikhalkhali)", inline=False)
    return embed


def new_program_message(data):
    embed = DiscordEmbed(title=f"{data['programName']}",
                         description=f"** New program ** named** {data['programName']} ** added to platform!\n** Program type: ** {data['programType']}\n\n** Program page: ** [Click here]({data['programURL']})", color=data['color'])
    embed.set_thumbnail(url=data['logo'])
    embed.set_footer(text='Powered by Ali Khalkhali',
                     icon_url='https://cdn.discordapp.com/avatars/941457826662985808/488f3bcab0de041de57860b4e05e2e9f.webp')
    if data["platformName"] in ["HackerOne", "Intigriti"]:
        if data["platformName"] == "Intigriti":
            if data['newReward']:
                embed.add_embed_field(
                    name="Bounty Table:", value=f"```{data['newReward']['min']} -- {data['newReward']['max']}```", inline=False)
        if data["newScope"]:
            scope = ""
            for newScope in data["newScope"]:
                scope += f"{shorten_string(newScope)}\n"
            add_field(embed, scope, "Scope:")
    if data["platformName"] in ["Bugcrowd", "YesWeHack"]:
        if data['reward']:
            embed.add_embed_field(
                name="Bounty Table:", value=f"```{data['reward']['min']} -- {data['reward']['max']}```", inline=False)
        if data['newInScope']:
            inscope = '\n'.join(data['newInScope'])
            add_field(embed, inscope, "InScope:")
        if data["platformName"] == "Bugcrowd":
            if data['newOutOfScope']:
                newOutOfScope = '\n'.join(data['newOutOfScope'])
                add_field(embed, newOutOfScope, "Out of scope:")

    embed.add_embed_field(name=" ", value= "[Click here to Buy me a coffee ☕](https://www.buymeacoffee.com/alikhalkhali)", inline=False)
    return embed

def removed_program_message(data):
    embed = DiscordEmbed(title=f"{data['programName']}",
                         description=f"Program named ** {data['programName']} ** has removed from platform!\n** Program type: ** {data['programType']}", color=data['color'])
    embed.set_thumbnail(url=data['logo'])
    embed.set_footer(text='Powered by Ali Khalkhali',
                     icon_url='https://cdn.discordapp.com/avatars/941457826662985808/488f3bcab0de041de57860b4e05e2e9f.webp')

    embed.add_embed_field(name=" ", value= "[Click here to Buy me a coffee ☕](https://www.buymeacoffee.com/alikhalkhali)", inline=False)
    return embed

def send_notification(data, webhook_url):

    webhook = DiscordWebhook(
        url=webhook_url, username=data['platformName'], avatar_url=get_platform_profile(data['platformName']))
    if data["isRemoved"]:
        embed = removed_program_message(data)
    elif data["isNewProgram"]:
        embed = new_program_message(data)
    else:
        embed = changed_program_message(data)
    webhook.add_embed(embed)
    response = webhook.execute()
    if response.status_code != 200:
        print(data["programName"])
        print("Error sending message:", response.content)

def send_startup_message(webhook_url):
    webhook = DiscordWebhook(
        url=webhook_url, username="Programs Watcher", avatar_url="https://t3.ftcdn.net/jpg/00/91/64/62/360_F_91646202_b3K6ELfgM2E8QIwuNTlzco7K1r3mOJvp.jpg")
    embed = DiscordEmbed(title=f"🎉 Successful Start of Programs Watcher 🎉",
                         description= "Hi, welcome to ** Programs Watcher **! 🎉\nThe program has started successfully and is now waiting for a change. 🗼✨\n** Github page: ** https://github.com/Alikhalkhali/programs-watcher", color="23ff1f")
    embed.set_footer(text='Powered by Ali Khalkhali',
                     icon_url='https://cdn.discordapp.com/avatars/941457826662985808/488f3bcab0de041de57860b4e05e2e9f.webp')
    embed.add_embed_field(name=" ", value= "[Click here to Buy me a coffee ☕](https://www.buymeacoffee.com/alikhalkhali)", inline=False)

    webhook.add_embed(embed)
    response = webhook.execute()

    if response.status_code != 200:
       print("There was an error sending the Discord message")


================================================
FILE: modules/notifier/functions.py
================================================
import difflib


def get_platform_profile(platform_name):
    if platform_name == "HackerOne":
        profile_url = "https://profile-photos.hackerone-user-content.com/variants/000/000/013/fa942b9b1cbf4faf37482bf68458e1195aab9c02_original.png/e60fe2d979b041d2254f8a36a3d2d7a24d7c4a8ad33ea024d13fc56668c7c4f6"
    elif platform_name == "Bugcrowd":
        profile_url = "https://logos.bugcrowdusercontent.com/logos/ef74/d1fa/62a5b64c/3809e0af42850a579f02c3434743e3ca_bugcrowd__1_.png"
    elif platform_name == "Intigriti":
        profile_url = "https://api.intigriti.com/file/api/file/public_bucket_d23a1f29-c2fe-4d03-8daf-df24d1e076ea-c2449aa2-3a08-4bf5-a430-441a11020851"
    elif platform_name == "YesWeHack":
        profile_url = "https://avatars.githubusercontent.com/u/16663829?s=280&v=4"
    return profile_url


def shorten_string(string):
    if len(string) > 30:
        return string[:30] + "...\n"
    else:
        return string


def split_text(huge_text, max_chunk_size=1000):
    sentences = huge_text.split('\n')
    chunks = []
    current_chunk = ""

    for sentence in sentences:
        if len(current_chunk) + len(sentence) + 1 <= max_chunk_size:
            current_chunk += (sentence + '\n')
        else:
            chunks.append(current_chunk.strip())
            current_chunk = sentence + '\n'

    if current_chunk:
        chunks.append(current_chunk.strip())

    return chunks


def generate_diff(old_str, new_str):
    diff = difflib.ndiff(old_str.splitlines(), new_str.splitlines())
    result = []
    for line in diff:
        if line.startswith('+'):
            result.append(f'+{line[1:]}')
        elif line.startswith('-'):
            result.append(f'-{line[1:]}')
        elif line.startswith(' '):
            result.append(line)
    return '\n'.join(result)


================================================
FILE: modules/platforms/bugcrowd.py
================================================
import json
from modules.platforms.functions import find_program, generate_program_key, get_resource, remove_elements, save_data, check_send_notification
from modules.notifier.discord import send_notification


# checking bugcrowd
def check_bugcrowd(tmp_dir, mUrl, first_time, db, config):
    json_programs_key = []
    notifications = config['notifications']
    monitor = config['monitor']
    get_resource(tmp_dir, config['url'], "bugcrowd")
    bugcrowdFile = open(f"{tmp_dir}bugcrowd.json")
    bugcrowd = json.load(bugcrowdFile)
    bugcrowdFile.close()
    for program in bugcrowd:
        programName = program["name"]
        programURL = "https://bugcrowd.com"+program["briefUrl"]
        logo = program["logoUrl"]
        data = {"programName": programName, "reward": {},"isRemoved": False, "newType": "", "newInScope": [], "removeInScope": [], "newOutOfScope": [], "removeOutOfScope": [], "programURL": programURL,
                "logo": logo, "platformName": "Bugcrowd", "isNewProgram": False, "color": 14584064}
        dataJson = {"programName": programName, "programURL": programURL, "programType": "",
                    "outOfScope": [], "inScope": [], "reward": {}}
        programKey = generate_program_key(programName, programURL)
        json_programs_key.append(programKey)
        watcherData = find_program(db, 'bugcrowd', programKey)
        if watcherData is None:
            data["isNewProgram"] = True
            watcherData = {"programKey": programKey, "programName": programName, "programURL": programURL, "programType": "",
                           "outOfScope": [], "inScope": [], "reward": {}}
        for target in program["target_groups"]:
            if target["in_scope"] == False:
                for item in target["targets"]:
                    dataJson["outOfScope"].append(item["name"])

            else:
                for item in target["targets"]:
                    dataJson["inScope"].append((item["name"]))
            bounty = {
                "min": "",
                "max": ""
            }
            if "rewardSummary" in program and program["rewardSummary"] != None:
                dataJson["programType"] = "rdp"
                data["programType"] = "rdp"
                bounty["max"] = program["rewardSummary"]["maxReward"]
                bounty["min"] = program["rewardSummary"]["minReward"]
            else:
                dataJson["programType"] = "vdp"
                data["programType"] = "vdp"
            dataJson["reward"] = bounty
        newInScope = [i for i in dataJson["inScope"]
                      if i not in watcherData["inScope"]]
        removeInScope = [i for i in watcherData["inScope"]
                         if i not in dataJson["inScope"]]
        removedOutOfScope = [i for i in watcherData["outOfScope"]
                             if i not in dataJson["outOfScope"]]
        newOutOfScope = [i for i in dataJson["outOfScope"]
                         if i not in watcherData["outOfScope"]]
        hasChanged = False
        is_update = False
        if newInScope:
            watcherData["inScope"].extend(newInScope)
            notifi_status = notifications['new_inscope']
            hasChanged = True
            if notifi_status:
                data["newInScope"] = newInScope
                is_update = True
        if removeInScope:
            remove_elements(watcherData["inScope"], removeInScope)
            hasChanged = True
            notifi_status = notifications['removed_inscope']
            if notifi_status:
                data["removeInScope"] = removeInScope
                is_update = True
        if newOutOfScope:
            watcherData["outOfScope"].extend(newOutOfScope)
            hasChanged = True
            notifi_status = notifications['new_out_of_scope']
            if notifi_status:
                data["newOutOfScope"] = newOutOfScope
                is_update = True
        if removedOutOfScope:
            remove_elements(watcherData["outOfScope"], removedOutOfScope)
            hasChanged = True
            notifi_status = notifications['removed_out_of_scope']
            if notifi_status:
                data["removeOutOfScope"] = removedOutOfScope
                is_update = True
        if dataJson["programType"] != watcherData["programType"]:
            watcherData["programType"] = dataJson["programType"]
            hasChanged = True
            notifi_status = notifications['new_type']
            if notifi_status:
                data["newType"] = dataJson["programType"]
                is_update = True
        if dataJson["reward"] != watcherData["reward"]:
            watcherData["reward"] = bounty
            hasChanged = True
            notifi_status = notifications['new_bounty_table']
            if notifi_status:
                data["reward"] = bounty
                is_update = True
        if hasChanged:
            save_data(db, "bugcrowd", programKey, watcherData)
            if check_send_notification(first_time, is_update, data,watcherData, monitor, notifications):
                    send_notification(data, mUrl)
        
    db_programs_key = db['bugcrowd'].distinct("programKey")
    removed_programs_key = set(db_programs_key) - set(json_programs_key)
    for program_key in removed_programs_key:
        program = find_program(db,'bugcrowd', program_key)
        data = {
            "color": 14584064,
            "logo": "https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcTwToiI8YA0eLclDkd-vJ0xXs7bun5LdHfTrgJucvI&s",
            "platformName": "Bugcrowd",
            "isRemoved": True, 
            "programName": program["programName"],
            "programType": program["programType"]
        }
        if notifications['removed_program'] and not first_time:
            send_notification(data,mUrl)
        db['bugcrowd'].delete_many({"programKey": program_key})        

================================================
FILE: modules/platforms/functions.py
================================================
import requests
from hashlib import md5


def get_resource(tmp_dir, url, platformName):
    domainList = requests.get(url, allow_redirects=True)
    open(F"{tmp_dir}{platformName}.json", 'wb').write(domainList.content)


def generate_program_key(programName, programURL):
    program_key = md5(
        f"{programName}|{programURL}".encode('utf-8')).hexdigest()
    return program_key


def find_program(db, platformName, programKey):
    data = db[platformName].find_one({'programKey': programKey})
    return data


def remove_elements(array1, array2):
    for element in array2:
        array1.remove(element)


def save_data(db, platformName, programKey, dataJson):
    db[platformName].update_one({'programKey': programKey}, {
        '$set': dataJson}, upsert=True)

def check_send_notification(first_time,is_update,data,watcherData,monitor,notifications):
    pt_notify_status = False
    notify_status = False
    if data['isNewProgram']:
        if data['programType'] == "rdp" and monitor['rdp']:
            pt_notify_status = True
        elif data['programType'] == "vdp" and monitor["vdp"]: 
            pt_notify_status = True
    else:
        if watcherData['programType'] == "rdp" and monitor['rdp']:
            pt_notify_status = True
        elif watcherData['programType'] == "vdp" and monitor["vdp"]: 
            pt_notify_status = True   
    if watcherData['programURL'] in monitor['specific_programs']:
            pt_notify_status = True
    if pt_notify_status:
        if not first_time and data['isNewProgram'] and notifications['new_program']:
            notify_status = True
        elif not first_time and is_update and not data['isNewProgram']:
            notify_status = True
    if watcherData['programURL'] in monitor['excluded_programs']:
        print(watcherData['programURL'])
        notify_status = False 
    return notify_status  
    

================================================
FILE: modules/platforms/hackerone.py
================================================
import json
from modules.platforms.functions import find_program, generate_program_key, get_resource, save_data, check_send_notification
from modules.notifier.discord import send_notification


# checking hackerone
def check_hackerone(tmp_dir, mUrl, first_time, db, config):
    json_programs_key = []
    notifications = config['notifications']
    monitor = config['monitor']
    get_resource(tmp_dir, config['url'], "hackerone")
    hackeroneFile = open(f"{tmp_dir}hackerone.json")
    hackerone = json.load(hackeroneFile)
    hackeroneFile.close()
    for program in hackerone:
        programName = program["attributes"]["name"]
        logo = program["attributes"]["profile_picture"]
        if logo.startswith("https://hackerone-us-west-2-p"):
            logo = "https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcTBmCle7j7K48bRZz483rDz52Nc6w0au28ASw&usqp=CAU"
        programURL = "https://hackerone.com/" + \
            program["attributes"]["handle"]+"?type=team"
        data = {"programName": programName, "programType": "", "programURL": programURL,
                "logo": logo, "platformName": "HackerOne", "isRemoved": False,"isNewProgram": False, "color": 16777215}
        dataJson = {"programName": programName,
                    "programURL": programURL, "programType": "", "scope": {}}
        programKey = generate_program_key(programName, programURL)
        json_programs_key.append(programKey)
        watcherData = find_program(db, 'hackerone', programKey)

        if watcherData is None:
            data["isNewProgram"] = True
            watcherData = {"programName": programName,
                           "programURL": programURL, "programType": "", "scope": {}}
        if program["attributes"]["offers_bounties"]:
            dataJson["programType"] = "rdp"
            data["programType"] = "rdp"
        else:
            dataJson["programType"] = "vdp"
            data["programType"] = "vdp"
        for target in program["relationships"]["structured_scopes"]["data"]:
            targetType = ""
            if target['attributes']['eligible_for_submission']:
                targetType = "(InScope)"
            else:
                targetType = "(OutOfScope)"
            if target['attributes']['instruction'] is not None:
                dataJson["scope"][target["id"]
                                  ] = f"{target['attributes']['asset_identifier']} {targetType}\n{target['attributes']['instruction']}\n"
            else:
                dataJson["scope"][target["id"]
                                  ] = f"{target['attributes']['asset_identifier']} {targetType}\n"

        hasChanged = False

        scopeId = {prop for prop in dataJson["scope"]}
        dbScopeId = {prop for prop in watcherData["scope"]}

        newScope = scopeId - dbScopeId
        removedScope = dbScopeId - scopeId
        scopeId = scopeId - newScope
        data["newScope"] = []
        data["changedScope"] = []
        data["removedScope"] = []
        data["newProgramType"] = []
        hasChanged = False
        is_update = False
        if newScope:
            notifi_status = notifications['new_scope']
            for i in newScope:
                if notifi_status:
                    data["newScope"].append(dataJson["scope"][i])
                    is_update = True
                watcherData["scope"][i] = dataJson["scope"][i]
            hasChanged = True
        if removedScope:
            notifi_status = notifications['removed_scope']
            for i in removedScope:
                if notifi_status:
                    data["removedScope"].append(watcherData["scope"][i])
                    is_update = True
                del watcherData["scope"][i]
                hasChanged = True
        if dataJson["programType"] != watcherData["programType"]:
            notifi_status = notifications['new_type']
            if notifi_status:
                data["newProgramType"] = dataJson["programType"]
                is_update = True
            watcherData["programType"] = dataJson["programType"]
            hasChanged = True
        notifi_status = notifications['changed_scope']
        for id in scopeId:
            if dataJson["scope"][id] != watcherData["scope"][id]:
                if notifi_status:
                    scope = {
                        "new": dataJson["scope"][id],
                        "old": watcherData["scope"][id],
                    }
                    data["changedScope"].append(scope)
                    is_update = True
                watcherData["scope"][id] = dataJson["scope"][id]
                hasChanged = True

        if hasChanged:
            save_data(db, "hackerone", programKey, watcherData)
            if check_send_notification(first_time, is_update, data,watcherData, monitor, notifications):
                    send_notification(data, mUrl)

    db_programs_key = db['hackerone'].distinct("programKey")
    removed_programs_key = set(db_programs_key) - set(json_programs_key)
    for program_key in removed_programs_key:
        program = find_program(db,'hackerone', program_key)
        data = {
            "color": 16777215,
            "logo": "https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcTwToiI8YA0eLclDkd-vJ0xXs7bun5LdHfTrgJucvI&s",
            "platformName": "HackerOne",
            "isRemoved": True, 
            "programName": program["programName"],
            "programType": program["programType"]
        }
        if notifications['removed_program'] and not first_time:
            send_notification(data,mUrl)
        db['hackerone'].delete_many({"programKey": program_key})        


================================================
FILE: modules/platforms/intigriti.py
================================================
import json
from modules.platforms.functions import find_program, generate_program_key, get_resource, save_data, check_send_notification
from modules.notifier.discord import send_notification


# checking intigriti
def check_intigriti(tmp_dir, mUrl, first_time, db, config):
    json_programs_key = []
    notifications = config['notifications']
    monitor = config['monitor']
    get_resource(tmp_dir, config['url'], "intigriti")
    intigriti = open(f"{tmp_dir}intigriti.json")
    intigriti = json.load(intigriti)
    for program in intigriti:
        programName = program["name"]
        logo = "https://secure.gravatar.com/avatar/8609b9cc1f1333c3632b2afdc8607f4d?s=500&d=identicon&r=g"
        programURL = f"https://app.intigriti.com/{program['webLinks']['detail']}"
        data = {"programName": programName, "programType": "", "programURL": programURL,
                "logo": logo, "platformName": "Intigriti","isRemoved": False, "isNewProgram": False, "color": 10858237}
        dataJson = {"programName": programName,
                    "programURL": programURL, "programType": "", "scope": {}, "reward": {}}
        programKey = generate_program_key(programName, programURL)
        json_programs_key.append(programKey)
        watcherData = find_program(db, 'intigriti', programKey)

        if watcherData is None:
            data["isNewProgram"] = True
            watcherData = {"programName": programName,
                           "programURL": programURL, "programType": "", "scope": {}, "reward": {}}
        if "domains" in program:
            for target in program['domains']:
                if target['description'] is not None:
                    dataJson['scope'][target['id']
                                    ] = f"{target['endpoint']}\n{target['description']}\n"
                else:
                    dataJson['scope'][target['id']] = f"{target['endpoint']}\n"
        if program["maxBounty"]["value"] > 0:
            dataJson["programType"] = "rdp"
            bounty = {
                "min": f"{program['minBounty']['value']} {program['minBounty']['currency']}",
                "max": f"{program['maxBounty']['value']} {program['maxBounty']['currency']}"
            }
            dataJson["reward"] = bounty
            data["programType"] = "rdp"

        else:
            dataJson["programType"] = "vdp"
            data["programType"] = "vdp"
        # Checking for changes
        hasChanged = False

        scopeId = {prop for prop in dataJson["scope"]}
        dbScopeId = {prop for prop in watcherData["scope"]}
        newScope = scopeId - dbScopeId
        removedScope = dbScopeId - scopeId
        scopeId = scopeId - newScope
        data["newScope"] = []
        data["changedScope"] = []
        data["removedScope"] = []
        data["newProgramType"] = []
        data["newReward"] = []
        hasChanged = False
        is_update = False
        if newScope:
            notifi_status = notifications['new_scope']
            for i in newScope:
                if notifi_status:
                    data["newScope"].append(dataJson["scope"][i])
                    is_update = True
                watcherData["scope"][i] = dataJson["scope"][i]
            hasChanged = True
        if removedScope:
            notifi_status = notifications['removed_scope']
            for i in removedScope:
                if notifi_status:
                    data["removedScope"].append(watcherData["scope"][i])
                    is_update = True
                del watcherData["scope"][i]
            hasChanged = True
        if dataJson["programType"] != watcherData["programType"]:
            notifi_status = notifications['new_type']
            if notifi_status:
                data["newProgramType"] = dataJson["programType"]
                is_update = True
            watcherData["programType"] = dataJson["programType"]
            hasChanged = True
        if dataJson["reward"] != watcherData["reward"]:
            notifi_status = notifications['new_bounty_table']
            if notifi_status:
                data["newReward"] = dataJson["reward"]
                is_update = True
            watcherData["reward"] = dataJson["reward"]
            hasChanged = True

        notifi_status = notifications['changed_scope']
        for id in scopeId:
            if dataJson["scope"][id] != watcherData["scope"][id]:
                if notifi_status:
                    scope = {
                        "new": dataJson["scope"][id],
                        "old": watcherData["scope"][id],
                    }
                    data["changedScope"].append(scope)
                    is_update = True
                watcherData["scope"][id] = dataJson["scope"][id]
                hasChanged = True
        if hasChanged:
            save_data(db, "intigriti", programKey, watcherData)
            if check_send_notification(first_time, is_update, data,watcherData, monitor, notifications):
                    send_notification(data, mUrl)
    db_programs_key = db['intigriti'].distinct("programKey")
    removed_programs_key = set(db_programs_key) - set(json_programs_key)
    for program_key in removed_programs_key:
        program = find_program(db,'intigriti', program_key)
        data = {
            "color": 10858237,
            "logo": "https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcTwToiI8YA0eLclDkd-vJ0xXs7bun5LdHfTrgJucvI&s",
            "platformName": "Intigriti",
            "isRemoved": True, 
            "programName": program["programName"],
            "programType": program["programType"]
        }
        if notifications['removed_program'] and not first_time:
            send_notification(data,mUrl)
        db['intigriti'].delete_many({"programKey": program_key})        


================================================
FILE: modules/platforms/yeswehack.py
================================================
import json
from modules.platforms.functions import find_program, generate_program_key, get_resource, remove_elements, save_data, check_send_notification
from modules.notifier.discord import send_notification


# checking yeswehack
def check_yeswehack(tmp_dir, mUrl, first_time, db, config):
    json_programs_key = []
    notifications = config['notifications']
    monitor = config['monitor']
    get_resource(tmp_dir, config['url'], "yeswehack")
    yeswehack = open(f"{tmp_dir}yeswehack.json")
    yeswehack = json.load(yeswehack)
    for program in yeswehack:
        programName = program['title']
        logo = program['thumbnail']['url']
        programURL = f"https://yeswehack.com/programs/{program['slug']}"
        data = {"programName": programName, "programType": "", "programURL": programURL,
                "logo": logo, "platformName": "YesWeHack","isRemoved": False, "isNewProgram": False, "color": 16270147}
        dataJson = {"programName": programName,
                    "programURL": programURL, "programType": "", "inScope": [], "reward": {}}
        programKey = generate_program_key(programName, programURL)
        json_programs_key.append(programKey)
        watcherData = find_program(db, 'yeswehack', programKey)

        if watcherData is None:
            data["isNewProgram"] = True
            watcherData = {"programName": programName,
                           "programURL": programURL, "programType": "", "inScope": [], "reward": {}}
        for target in program["scopes"]:
            dataJson['inScope'].append(target['scope'])
        if program["bounty"]:
            dataJson['programType'] = "rdp"
            data['programType'] = "rdp"
            currency = program['business_unit']['currency']
            bounty = {
                "min": f"{program['bounty_reward_min']} {currency}",
                "max": f"{program['bounty_reward_max']} {currency}"
            }
            dataJson['reward'] = bounty
        else:
            dataJson['programType'] = "vdp"
            data['programType'] = "vdp"

        newInScope = [i for i in dataJson["inScope"]
                      if i not in watcherData["inScope"]]
        removeInScope = [i for i in watcherData["inScope"]
                         if i not in dataJson["inScope"]]

        data["newType"] = []
        data["reward"] = []
        data["removeInScope"] = []
        data["newInScope"] = []
        hasChanged = False
        is_update = False
        if newInScope:
            watcherData["inScope"].extend(newInScope)
            notifi_status = notifications['new_inscope']
            if notifi_status:
                data["newInScope"] = newInScope
                is_update = True
            hasChanged = True
        if removeInScope:
            remove_elements(watcherData["inScope"], removeInScope)
            notifi_status = notifications['removed_inscope']
            if notifi_status:
                data["removeInScope"] = removeInScope
                is_update = True
            hasChanged = True
        if dataJson['reward'] != watcherData['reward']:
            watcherData["reward"] = dataJson['reward']
            notifi_status = notifications['new_bounty_table']
            if notifi_status:
                data["reward"] = dataJson['reward']
                is_update = True
            hasChanged = True
        if dataJson["programType"] != watcherData["programType"]:
            notifi_status = notifications['new_type']
            if notifi_status:
                data["newType"] = dataJson["programType"]
                is_update = True
            watcherData["programType"] = dataJson["programType"]
            hasChanged = True
        if hasChanged:
            save_data(db, "yeswehack", programKey, watcherData)
            if check_send_notification(first_time, is_update, data,watcherData, monitor, notifications):
                    send_notification(data, mUrl)

    db_programs_key = db['yeswehack'].distinct("programKey")
    removed_programs_key = set(db_programs_key) - set(json_programs_key)
    for program_key in removed_programs_key:
        program = find_program(db,'yeswehack', program_key)
        data = {
            "color": 16270147,
            "logo": "https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcTwToiI8YA0eLclDkd-vJ0xXs7bun5LdHfTrgJucvI&s",
            "platformName": "YesWeHack",
            "isRemoved": True, 
            "programName": program["programName"],
            "programType": program["programType"]
        }
        if notifications['removed_program'] and not first_time:
            send_notification(data,mUrl)
        db['yeswehack'].delete_many({"programKey": program_key})        


================================================
FILE: requirements.txt
================================================
certifi==2023.7.22
charset-normalizer==3.2.0
discord-webhook==1.3.0
dnspython==2.4.2
idna==3.4
pymongo==4.5.0
PyYAML==6.0.1
requests==2.31.0
urllib3==2.0.5