Spug
\n\n\n\nSpug是面向中小型企业设计的轻量级无Agent的自动化运维平台，整合了主机管理、主机批量执行、主机在线终端、应用发布部署、在线任务计划、配置中心、监控、报警等一系列功能。\n\n
\n\n- 公司官网：https://www.spug.cc\n- 项目官网：https://ops.spug.cc\n- 使用文档：https://ops.spug.cc/docs/about-spug/\n\n## 演示环境\n\n演示地址：https://demo.spug.cc\n\n## 🔐免费通配符SSL证书\n免费通配符，付费证书价格亲民，性价比超高，低于市场其他平台价格，免费专家一对一配置服务，购买流程简单快速，且支持7天无理由退款和开具发票。提供一键下载和SSL过期通知配置，免费申请：[https://ssl.spug.cc](https://ssl.spug.cc)\n\n\n## 🔥推送助手\n\n推送助手是一个集成了电话、短信、邮件、飞书、钉钉、微信、企业微信等多通道的消息推送平台，可以3分钟实现个人电话短信推送，点击体验：[https://push.spug.cc](https://push.spug.cc)\n\n\n## 特性\n\n- **批量执行**: 主机命令在线批量执行\n- **在线终端**: 主机支持浏览器在线终端登录\n- **文件管理**: 主机文件在线上传下载\n- **任务计划**: 灵活的在线任务计划\n- **发布部署**: 支持自定义发布部署流程\n- **配置中心**: 支持KV、文本、json等格式的配置\n- **监控中心**: 支持站点、端口、进程、自定义等监控\n- **报警中心**: 支持短信、邮件、钉钉、微信等报警方式\n- **优雅美观**: 基于 Ant Design 的UI界面\n- **开源免费**: 前后端代码完全开源\n\n\n## 环境\n\n* Python 3.6+\n* Django 2.2\n* Node 12.14\n* React 16.11\n\n## 安装\n\n[官方文档](https://ops.spug.cc/docs/install-docker)\n\n更多使用帮助请参考： [使用文档](https://ops.spug.cc/docs/host-manage/)\n\n\n## 推荐项目\n[Yearning — MYSQL 开源SQL语句审核平台](https://github.com/cookieY/Yearning)\n\n\n## 预览\n\n### 主机管理\n\n\n#### 主机在线终端\n\n\n#### 文件在线上传下载\n\n\n#### 主机批量执行\n\n\n\n#### 应用发布\n\n\n#### 监控报警\n\n\n#### 角色权限\n\n\n\n## 赞助\n\n \n ![](\"https://cdn.spug.cc/img/ucloud.png\") \n UCloud \n 5 元/月云主机\n \n | \n \n \n ![](\"https://cdn.spug.cc/img/aliyun-logo.png\"){kind=logo} \n 阿里云 \n 2核心2G低至99元/年\n \n | \n \n \n ![](\"https://cdn.spug.cc/img/magedu-logo.jpeg\"){kind=logo} \n 马哥教育 \n IT人高薪职业学院\n \n | \n
|---|
\n ![\"spug-qq\"](\"https://cdn.spug.cc/img/spug-club.jpg\")
\n
\n\n \n## License & Copyright\n[AGPL-3.0](https://opensource.org/licenses/AGPL-3.0)\n"
},
{
"path": "docs/FQA.md",
"content": "### install mysqlclient\n```shell\n# for centos 7\nyum install mariadb-devel python3-devel gcc\npip install mysqlclient\n```\n"
},
{
"path": "docs/docker/Dockerfile",
"content": "FROM centos:7.9.2009\n\nENV TZ=Asia/Shanghai\nRUN yum install -y epel-release https://packages.endpointdev.com/rhel/7/os/x86_64/endpoint-repo.x86_64.rpm && yum install -y --setopt=tsflags=nodocs nginx redis mariadb-devel python36 python36-devel openldap-devel supervisor git gcc wget unzip net-tools sshpass rsync sshfs && yum -y clean all --enablerepo='*'\n\nRUN pip3 install --no-cache-dir --upgrade pip -i https://mirrors.aliyun.com/pypi/simple/\nRUN pip3 install --no-cache-dir -i https://mirrors.aliyun.com/pypi/simple/ \\\n gunicorn \\\n mysqlclient \\\n cryptography==36.0.2 \\\n apscheduler==3.7.0 \\\n asgiref==3.2.10 \\\n Django==2.2.28 \\\n channels==2.3.1 \\\n channels_redis==2.4.1 \\\n paramiko==2.11.0 \\\n django-redis==4.10.0 \\\n requests==2.22.0 \\\n GitPython==3.0.8 \\\n python-ldap==3.4.0 \\\n openpyxl==3.0.3 \\\n user_agents==2.2.0\n\nRUN localedef -c -i en_US -f UTF-8 en_US.UTF-8\nENV LANG=en_US.UTF-8\nENV LC_ALL=en_US.UTF-8\nRUN echo -e '\\n# Source definitions\\n. /etc/profile\\n' >> /root/.bashrc\nRUN mkdir -p /data/repos\nCOPY init_spug /usr/bin/\nCOPY nginx.conf /etc/nginx/\nCOPY ssh_config /etc/ssh/\nCOPY spug.ini /etc/supervisord.d/\nCOPY redis.conf /etc/\nCOPY entrypoint.sh /\n\nVOLUME /data\nEXPOSE 80\nENTRYPOINT [\"/entrypoint.sh\"]\n"
},
{
"path": "docs/docker/docker-compose.yml",
"content": "version: \"3.3\"\nservices:\n db:\n image: mariadb:10.8\n container_name: spug-db\n restart: always\n command: --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci\n volumes:\n - /data/spug/mysql:/var/lib/mysql\n environment:\n - MYSQL_DATABASE=spug\n - MYSQL_USER=spug\n - MYSQL_PASSWORD=spug.cc\n - MYSQL_ROOT_PASSWORD=spug.cc\n spug:\n image: openspug/spug-service\n container_name: spug\n privileged: true\n restart: always\n volumes:\n - /data/spug/service:/data/spug\n - /data/spug/repos:/data/repos\n ports:\n # 如果80端口被占用可替换为其他端口，例如: - \"8000:80\"\n - \"80:80\"\n environment:\n - SPUG_DOCKER_VERSION=v3.2.4\n - MYSQL_DATABASE=spug\n - MYSQL_USER=spug\n - MYSQL_PASSWORD=spug.cc\n - MYSQL_HOST=db\n - MYSQL_PORT=3306\n depends_on:\n - db\n"
},
{
"path": "docs/docker/entrypoint.sh",
"content": "#!/bin/bash\n#\nset -e\n\nif [ -e /root/.bashrc ]; then\n source /root/.bashrc\nfi\n\nif [ ! -d /data/spug/spug_api ]; then\n git clone -b $SPUG_DOCKER_VERSION https://gitee.com/openspug/spug.git /data/spug\n curl -o web.tar.gz https://cdn.spug.cc/spug/web_${SPUG_DOCKER_VERSION}.tar.gz\n tar xf web.tar.gz -C /data/spug/spug_web/\n rm -f web.tar.gz\n SECRET_KEY=$(< /dev/urandom tr -dc '!@#%^.a-zA-Z0-9' | head -c50)\n cat > /data/spug/spug_api/spug/overrides.py << EOF\nimport os\n\n\nDEBUG = False\nALLOWED_HOSTS = ['127.0.0.1']\nSECRET_KEY = '${SECRET_KEY}'\n\nDATABASES = {\n 'default': {\n 'ATOMIC_REQUESTS': True,\n 'ENGINE': 'django.db.backends.mysql',\n 'NAME': os.environ.get('MYSQL_DATABASE'),\n 'USER': os.environ.get('MYSQL_USER'),\n 'PASSWORD': os.environ.get('MYSQL_PASSWORD'),\n 'HOST': os.environ.get('MYSQL_HOST'),\n 'PORT': os.environ.get('MYSQL_PORT'),\n 'OPTIONS': {\n 'charset': 'utf8mb4',\n 'sql_mode': 'STRICT_TRANS_TABLES',\n }\n }\n}\nEOF\nfi\n\nexec supervisord -c /etc/supervisord.conf\n"
},
{
"path": "docs/docker/init_spug",
"content": "#!/bin/bash\n#\nset -e\nset -u\n\npython3 /data/spug/spug_api/manage.py updatedb\npython3 /data/spug/spug_api/manage.py user add -u $1 -p $2 -n 管理员 -s\n"
},
{
"path": "docs/docker/nginx.conf",
"content": "# For more information on configuration, see:\n# * Official English Documentation: http://nginx.org/en/docs/\n# * Official Russian Documentation: http://nginx.org/ru/docs/\n\nuser nginx;\nworker_processes auto;\nerror_log /var/log/nginx/error.log;\npid /run/nginx.pid;\n\n# Load dynamic modules. See /usr/share/doc/nginx/README.dynamic.\ninclude /usr/share/nginx/modules/*.conf;\n\nevents {\n worker_connections 1024;\n}\n\nhttp {\n log_format main '$remote_addr - $remote_user [$time_local] \"$request\" '\n '$status $body_bytes_sent \"$http_referer\" '\n '\"$http_user_agent\" \"$http_x_forwarded_for\"';\n\n access_log /var/log/nginx/access.log main;\n\n sendfile on;\n tcp_nopush on;\n tcp_nodelay on;\n keepalive_timeout 65;\n types_hash_max_size 2048;\n server_tokens off;\n\n include /etc/nginx/mime.types;\n default_type application/octet-stream;\n\n # Load modular configuration files from the /etc/nginx/conf.d directory.\n # See http://nginx.org/en/docs/ngx_core_module.html#include\n # for more information.\n include /etc/nginx/conf.d/*.conf;\n\n server {\n listen 80 default_server;\n listen [::]:80 default_server;\n server_name _;\n root /data/spug/spug_web/build;\n\tclient_max_body_size\t0;\n add_header X-Frame-Options SAMEORIGIN always;\n\n\tgzip on;\n\tgzip_min_length 1k;\n\tgzip_buffers 4 16k;\n\tgzip_http_version 1.1;\n\tgzip_comp_level 7;\n\tgzip_types text/plain text/css text/javascript application/javascript application/json;\n\tgzip_disable \"MSIE [1-6]\\.\";\n\tgzip_vary on;\n\n location ^~ /api/ {\n rewrite ^/api(.*) $1 break;\n proxy_pass http://127.0.0.1:9001;\n\t\tproxy_read_timeout 180s;\n proxy_redirect off;\n proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;\n }\n\n location ^~ /api/ws/ {\n rewrite ^/api(.*) $1 break;\n proxy_pass http://127.0.0.1:9002;\n proxy_http_version 1.1;\n proxy_set_header Upgrade $http_upgrade;\n proxy_set_header Connection \"Upgrade\";\n proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;\n }\n \n location / {\n try_files $uri /index.html;\n }\n }\n}\n"
},
{
"path": "docs/docker/redis.conf",
"content": "# Redis configuration file example.\n#\n# Note that in order to read the configuration file, Redis must be\n# started with the file path as first argument:\n#\n# ./redis-server /path/to/redis.conf\n\n# Note on units: when memory size is needed, it is possible to specify\n# it in the usual form of 1k 5GB 4M and so forth:\n#\n# 1k => 1000 bytes\n# 1kb => 1024 bytes\n# 1m => 1000000 bytes\n# 1mb => 1024*1024 bytes\n# 1g => 1000000000 bytes\n# 1gb => 1024*1024*1024 bytes\n#\n# units are case insensitive so 1GB 1Gb 1gB are all the same.\n\n################################## INCLUDES ###################################\n\n# Include one or more other config files here. This is useful if you\n# have a standard template that goes to all Redis servers but also need\n# to customize a few per-server settings. Include files can include\n# other files, so use this wisely.\n#\n# Notice option \"include\" won't be rewritten by command \"CONFIG REWRITE\"\n# from admin or Redis Sentinel. Since Redis always uses the last processed\n# line as value of a configuration directive, you'd better put includes\n# at the beginning of this file to avoid overwriting config change at runtime.\n#\n# If instead you are interested in using includes to override configuration\n# options, it is better to use include as the last line.\n#\n# include /path/to/local.conf\n# include /path/to/other.conf\n\n################################## NETWORK #####################################\n\n# By default, if no \"bind\" configuration directive is specified, Redis listens\n# for connections from all the network interfaces available on the server.\n# It is possible to listen to just one or multiple selected interfaces using\n# the \"bind\" configuration directive, followed by one or more IP addresses.\n#\n# Examples:\n#\n# bind 192.168.1.100 10.0.0.1\n# bind 127.0.0.1 ::1\n#\n# ~~~ WARNING ~~~ If the computer running Redis is directly exposed to the\n# internet, binding to all the interfaces is dangerous and will expose the\n# instance to everybody on the internet. So by default we uncomment the\n# following bind directive, that will force Redis to listen only into\n# the IPv4 lookback interface address (this means Redis will be able to\n# accept connections only from clients running into the same computer it\n# is running).\n#\n# IF YOU ARE SURE YOU WANT YOUR INSTANCE TO LISTEN TO ALL THE INTERFACES\n# JUST COMMENT THE FOLLOWING LINE.\n# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\nbind 127.0.0.1\n\n# Protected mode is a layer of security protection, in order to avoid that\n# Redis instances left open on the internet are accessed and exploited.\n#\n# When protected mode is on and if:\n#\n# 1) The server is not binding explicitly to a set of addresses using the\n# \"bind\" directive.\n# 2) No password is configured.\n#\n# The server only accepts connections from clients connecting from the\n# IPv4 and IPv6 loopback addresses 127.0.0.1 and ::1, and from Unix domain\n# sockets.\n#\n# By default protected mode is enabled. You should disable it only if\n# you are sure you want clients from other hosts to connect to Redis\n# even if no authentication is configured, nor a specific set of interfaces\n# are explicitly listed using the \"bind\" directive.\nprotected-mode yes\n\n# Accept connections on the specified port, default is 6379 (IANA #815344).\n# If port 0 is specified Redis will not listen on a TCP socket.\nport 6379\n\n# TCP listen() backlog.\n#\n# In high requests-per-second environments you need an high backlog in order\n# to avoid slow clients connections issues. Note that the Linux kernel\n# will silently truncate it to the value of /proc/sys/net/core/somaxconn so\n# make sure to raise both the value of somaxconn and tcp_max_syn_backlog\n# in order to get the desired effect.\ntcp-backlog 511\n\n# Unix socket.\n#\n# Specify the path for the Unix socket that will be used to listen for\n# incoming connections. There is no default, so Redis will not listen\n# on a unix socket when not specified.\n#\n# unixsocket /tmp/redis.sock\n# unixsocketperm 700\n\n# Close the connection after a client is idle for N seconds (0 to disable)\ntimeout 0\n\n# TCP keepalive.\n#\n# If non-zero, use SO_KEEPALIVE to send TCP ACKs to clients in absence\n# of communication. This is useful for two reasons:\n#\n# 1) Detect dead peers.\n# 2) Take the connection alive from the point of view of network\n# equipment in the middle.\n#\n# On Linux, the specified value (in seconds) is the period used to send ACKs.\n# Note that to close the connection the double of the time is needed.\n# On other kernels the period depends on the kernel configuration.\n#\n# A reasonable value for this option is 300 seconds, which is the new\n# Redis default starting with Redis 3.2.1.\ntcp-keepalive 300\n\n################################# GENERAL #####################################\n\n# By default Redis does not run as a daemon. Use 'yes' if you need it.\n# Note that Redis will write a pid file in /var/run/redis.pid when daemonized.\ndaemonize no\n\n# If you run Redis from upstart or systemd, Redis can interact with your\n# supervision tree. Options:\n# supervised no - no supervision interaction\n# supervised upstart - signal upstart by putting Redis into SIGSTOP mode\n# supervised systemd - signal systemd by writing READY=1 to $NOTIFY_SOCKET\n# supervised auto - detect upstart or systemd method based on\n# UPSTART_JOB or NOTIFY_SOCKET environment variables\n# Note: these supervision methods only signal \"process is ready.\"\n# They do not enable continuous liveness pings back to your supervisor.\nsupervised no\n\n# If a pid file is specified, Redis writes it where specified at startup\n# and removes it at exit.\n#\n# When the server runs non daemonized, no pid file is created if none is\n# specified in the configuration. When the server is daemonized, the pid file\n# is used even if not specified, defaulting to \"/var/run/redis.pid\".\n#\n# Creating a pid file is best effort: if Redis is not able to create it\n# nothing bad happens, the server will start and run normally.\npidfile /var/run/redis_6379.pid\n\n# Specify the server verbosity level.\n# This can be one of:\n# debug (a lot of information, useful for development/testing)\n# verbose (many rarely useful info, but not a mess like the debug level)\n# notice (moderately verbose, what you want in production probably)\n# warning (only very important / critical messages are logged)\nloglevel notice\n\n# Specify the log file name. Also the empty string can be used to force\n# Redis to log on the standard output. Note that if you use standard\n# output for logging but daemonize, logs will be sent to /dev/null\nlogfile /var/log/redis/redis.log\n\n# To enable logging to the system logger, just set 'syslog-enabled' to yes,\n# and optionally update the other syslog parameters to suit your needs.\n# syslog-enabled no\n\n# Specify the syslog identity.\n# syslog-ident redis\n\n# Specify the syslog facility. Must be USER or between LOCAL0-LOCAL7.\n# syslog-facility local0\n\n# Set the number of databases. The default database is DB 0, you can select\n# a different one on a per-connection basis using SELECT where\n# dbid is a number between 0 and 'databases'-1\ndatabases 16\n\n################################ SNAPSHOTTING ################################\n#\n# Save the DB on disk:\n#\n# save \n#\n# Will save the DB if both the given number of seconds and the given\n# number of write operations against the DB occurred.\n#\n# In the example below the behaviour will be to save:\n# after 900 sec (15 min) if at least 1 key changed\n# after 300 sec (5 min) if at least 10 keys changed\n# after 60 sec if at least 10000 keys changed\n#\n# Note: you can disable saving completely by commenting out all \"save\" lines.\n#\n# It is also possible to remove all the previously configured save\n# points by adding a save directive with a single empty string argument\n# like in the following example:\n#\n# save \"\"\n\nsave 900 1\nsave 300 10\nsave 60 10000\n\n# By default Redis will stop accepting writes if RDB snapshots are enabled\n# (at least one save point) and the latest background save failed.\n# This will make the user aware (in a hard way) that data is not persisting\n# on disk properly, otherwise chances are that no one will notice and some\n# disaster will happen.\n#\n# If the background saving process will start working again Redis will\n# automatically allow writes again.\n#\n# However if you have setup your proper monitoring of the Redis server\n# and persistence, you may want to disable this feature so that Redis will\n# continue to work as usual even if there are problems with disk,\n# permissions, and so forth.\nstop-writes-on-bgsave-error yes\n\n# Compress string objects using LZF when dump .rdb databases?\n# For default that's set to 'yes' as it's almost always a win.\n# If you want to save some CPU in the saving child set it to 'no' but\n# the dataset will likely be bigger if you have compressible values or keys.\nrdbcompression yes\n\n# Since version 5 of RDB a CRC64 checksum is placed at the end of the file.\n# This makes the format more resistant to corruption but there is a performance\n# hit to pay (around 10%) when saving and loading RDB files, so you can disable it\n# for maximum performances.\n#\n# RDB files created with checksum disabled have a checksum of zero that will\n# tell the loading code to skip the check.\nrdbchecksum yes\n\n# The filename where to dump the DB\ndbfilename dump.rdb\n\n# The working directory.\n#\n# The DB will be written inside this directory, with the filename specified\n# above using the 'dbfilename' configuration directive.\n#\n# The Append Only File will also be created inside this directory.\n#\n# Note that you must specify a directory here, not a file name.\ndir /var/lib/redis\n\n################################# REPLICATION #################################\n\n# Master-Slave replication. Use slaveof to make a Redis instance a copy of\n# another Redis server. A few things to understand ASAP about Redis replication.\n#\n# 1) Redis replication is asynchronous, but you can configure a master to\n# stop accepting writes if it appears to be not connected with at least\n# a given number of slaves.\n# 2) Redis slaves are able to perform a partial resynchronization with the\n# master if the replication link is lost for a relatively small amount of\n# time. You may want to configure the replication backlog size (see the next\n# sections of this file) with a sensible value depending on your needs.\n# 3) Replication is automatic and does not need user intervention. After a\n# network partition slaves automatically try to reconnect to masters\n# and resynchronize with them.\n#\n# slaveof \n\n# If the master is password protected (using the \"requirepass\" configuration\n# directive below) it is possible to tell the slave to authenticate before\n# starting the replication synchronization process, otherwise the master will\n# refuse the slave request.\n#\n# masterauth \n\n# When a slave loses its connection with the master, or when the replication\n# is still in progress, the slave can act in two different ways:\n#\n# 1) if slave-serve-stale-data is set to 'yes' (the default) the slave will\n# still reply to client requests, possibly with out of date data, or the\n# data set may just be empty if this is the first synchronization.\n#\n# 2) if slave-serve-stale-data is set to 'no' the slave will reply with\n# an error \"SYNC with master in progress\" to all the kind of commands\n# but to INFO and SLAVEOF.\n#\nslave-serve-stale-data yes\n\n# You can configure a slave instance to accept writes or not. Writing against\n# a slave instance may be useful to store some ephemeral data (because data\n# written on a slave will be easily deleted after resync with the master) but\n# may also cause problems if clients are writing to it because of a\n# misconfiguration.\n#\n# Since Redis 2.6 by default slaves are read-only.\n#\n# Note: read only slaves are not designed to be exposed to untrusted clients\n# on the internet. It's just a protection layer against misuse of the instance.\n# Still a read only slave exports by default all the administrative commands\n# such as CONFIG, DEBUG, and so forth. To a limited extent you can improve\n# security of read only slaves using 'rename-command' to shadow all the\n# administrative / dangerous commands.\nslave-read-only yes\n\n# Replication SYNC strategy: disk or socket.\n#\n# -------------------------------------------------------\n# WARNING: DISKLESS REPLICATION IS EXPERIMENTAL CURRENTLY\n# -------------------------------------------------------\n#\n# New slaves and reconnecting slaves that are not able to continue the replication\n# process just receiving differences, need to do what is called a \"full\n# synchronization\". An RDB file is transmitted from the master to the slaves.\n# The transmission can happen in two different ways:\n#\n# 1) Disk-backed: The Redis master creates a new process that writes the RDB\n# file on disk. Later the file is transferred by the parent\n# process to the slaves incrementally.\n# 2) Diskless: The Redis master creates a new process that directly writes the\n# RDB file to slave sockets, without touching the disk at all.\n#\n# With disk-backed replication, while the RDB file is generated, more slaves\n# can be queued and served with the RDB file as soon as the current child producing\n# the RDB file finishes its work. With diskless replication instead once\n# the transfer starts, new slaves arriving will be queued and a new transfer\n# will start when the current one terminates.\n#\n# When diskless replication is used, the master waits a configurable amount of\n# time (in seconds) before starting the transfer in the hope that multiple slaves\n# will arrive and the transfer can be parallelized.\n#\n# With slow disks and fast (large bandwidth) networks, diskless replication\n# works better.\nrepl-diskless-sync no\n\n# When diskless replication is enabled, it is possible to configure the delay\n# the server waits in order to spawn the child that transfers the RDB via socket\n# to the slaves.\n#\n# This is important since once the transfer starts, it is not possible to serve\n# new slaves arriving, that will be queued for the next RDB transfer, so the server\n# waits a delay in order to let more slaves arrive.\n#\n# The delay is specified in seconds, and by default is 5 seconds. To disable\n# it entirely just set it to 0 seconds and the transfer will start ASAP.\nrepl-diskless-sync-delay 5\n\n# Slaves send PINGs to server in a predefined interval. It's possible to change\n# this interval with the repl_ping_slave_period option. The default value is 10\n# seconds.\n#\n# repl-ping-slave-period 10\n\n# The following option sets the replication timeout for:\n#\n# 1) Bulk transfer I/O during SYNC, from the point of view of slave.\n# 2) Master timeout from the point of view of slaves (data, pings).\n# 3) Slave timeout from the point of view of masters (REPLCONF ACK pings).\n#\n# It is important to make sure that this value is greater than the value\n# specified for repl-ping-slave-period otherwise a timeout will be detected\n# every time there is low traffic between the master and the slave.\n#\n# repl-timeout 60\n\n# Disable TCP_NODELAY on the slave socket after SYNC?\n#\n# If you select \"yes\" Redis will use a smaller number of TCP packets and\n# less bandwidth to send data to slaves. But this can add a delay for\n# the data to appear on the slave side, up to 40 milliseconds with\n# Linux kernels using a default configuration.\n#\n# If you select \"no\" the delay for data to appear on the slave side will\n# be reduced but more bandwidth will be used for replication.\n#\n# By default we optimize for low latency, but in very high traffic conditions\n# or when the master and slaves are many hops away, turning this to \"yes\" may\n# be a good idea.\nrepl-disable-tcp-nodelay no\n\n# Set the replication backlog size. The backlog is a buffer that accumulates\n# slave data when slaves are disconnected for some time, so that when a slave\n# wants to reconnect again, often a full resync is not needed, but a partial\n# resync is enough, just passing the portion of data the slave missed while\n# disconnected.\n#\n# The bigger the replication backlog, the longer the time the slave can be\n# disconnected and later be able to perform a partial resynchronization.\n#\n# The backlog is only allocated once there is at least a slave connected.\n#\n# repl-backlog-size 1mb\n\n# After a master has no longer connected slaves for some time, the backlog\n# will be freed. The following option configures the amount of seconds that\n# need to elapse, starting from the time the last slave disconnected, for\n# the backlog buffer to be freed.\n#\n# A value of 0 means to never release the backlog.\n#\n# repl-backlog-ttl 3600\n\n# The slave priority is an integer number published by Redis in the INFO output.\n# It is used by Redis Sentinel in order to select a slave to promote into a\n# master if the master is no longer working correctly.\n#\n# A slave with a low priority number is considered better for promotion, so\n# for instance if there are three slaves with priority 10, 100, 25 Sentinel will\n# pick the one with priority 10, that is the lowest.\n#\n# However a special priority of 0 marks the slave as not able to perform the\n# role of master, so a slave with priority of 0 will never be selected by\n# Redis Sentinel for promotion.\n#\n# By default the priority is 100.\nslave-priority 100\n\n# It is possible for a master to stop accepting writes if there are less than\n# N slaves connected, having a lag less or equal than M seconds.\n#\n# The N slaves need to be in \"online\" state.\n#\n# The lag in seconds, that must be <= the specified value, is calculated from\n# the last ping received from the slave, that is usually sent every second.\n#\n# This option does not GUARANTEE that N replicas will accept the write, but\n# will limit the window of exposure for lost writes in case not enough slaves\n# are available, to the specified number of seconds.\n#\n# For example to require at least 3 slaves with a lag <= 10 seconds use:\n#\n# min-slaves-to-write 3\n# min-slaves-max-lag 10\n#\n# Setting one or the other to 0 disables the feature.\n#\n# By default min-slaves-to-write is set to 0 (feature disabled) and\n# min-slaves-max-lag is set to 10.\n\n# A Redis master is able to list the address and port of the attached\n# slaves in different ways. For example the \"INFO replication\" section\n# offers this information, which is used, among other tools, by\n# Redis Sentinel in order to discover slave instances.\n# Another place where this info is available is in the output of the\n# \"ROLE\" command of a masteer.\n#\n# The listed IP and address normally reported by a slave is obtained\n# in the following way:\n#\n# IP: The address is auto detected by checking the peer address\n# of the socket used by the slave to connect with the master.\n#\n# Port: The port is communicated by the slave during the replication\n# handshake, and is normally the port that the slave is using to\n# list for connections.\n#\n# However when port forwarding or Network Address Translation (NAT) is\n# used, the slave may be actually reachable via different IP and port\n# pairs. The following two options can be used by a slave in order to\n# report to its master a specific set of IP and port, so that both INFO\n# and ROLE will report those values.\n#\n# There is no need to use both the options if you need to override just\n# the port or the IP address.\n#\n# slave-announce-ip 5.5.5.5\n# slave-announce-port 1234\n\n################################## SECURITY ###################################\n\n# Require clients to issue AUTH before processing any other\n# commands. This might be useful in environments in which you do not trust\n# others with access to the host running redis-server.\n#\n# This should stay commented out for backward compatibility and because most\n# people do not need auth (e.g. they run their own servers).\n#\n# Warning: since Redis is pretty fast an outside user can try up to\n# 150k passwords per second against a good box. This means that you should\n# use a very strong password otherwise it will be very easy to break.\n#\n# requirepass foobared\n\n# Command renaming.\n#\n# It is possible to change the name of dangerous commands in a shared\n# environment. For instance the CONFIG command may be renamed into something\n# hard to guess so that it will still be available for internal-use tools\n# but not available for general clients.\n#\n# Example:\n#\n# rename-command CONFIG b840fc02d524045429941cc15f59e41cb7be6c52\n#\n# It is also possible to completely kill a command by renaming it into\n# an empty string:\n#\n# rename-command CONFIG \"\"\n#\n# Please note that changing the name of commands that are logged into the\n# AOF file or transmitted to slaves may cause problems.\n\n################################### LIMITS ####################################\n\n# Set the max number of connected clients at the same time. By default\n# this limit is set to 10000 clients, however if the Redis server is not\n# able to configure the process file limit to allow for the specified limit\n# the max number of allowed clients is set to the current file limit\n# minus 32 (as Redis reserves a few file descriptors for internal uses).\n#\n# Once the limit is reached Redis will close all the new connections sending\n# an error 'max number of clients reached'.\n#\n# maxclients 10000\n\n# Don't use more memory than the specified amount of bytes.\n# When the memory limit is reached Redis will try to remove keys\n# according to the eviction policy selected (see maxmemory-policy).\n#\n# If Redis can't remove keys according to the policy, or if the policy is\n# set to 'noeviction', Redis will start to reply with errors to commands\n# that would use more memory, like SET, LPUSH, and so on, and will continue\n# to reply to read-only commands like GET.\n#\n# This option is usually useful when using Redis as an LRU cache, or to set\n# a hard memory limit for an instance (using the 'noeviction' policy).\n#\n# WARNING: If you have slaves attached to an instance with maxmemory on,\n# the size of the output buffers needed to feed the slaves are subtracted\n# from the used memory count, so that network problems / resyncs will\n# not trigger a loop where keys are evicted, and in turn the output\n# buffer of slaves is full with DELs of keys evicted triggering the deletion\n# of more keys, and so forth until the database is completely emptied.\n#\n# In short... if you have slaves attached it is suggested that you set a lower\n# limit for maxmemory so that there is some free RAM on the system for slave\n# output buffers (but this is not needed if the policy is 'noeviction').\n#\nmaxmemory 2GB\n\n# MAXMEMORY POLICY: how Redis will select what to remove when maxmemory\n# is reached. You can select among five behaviors:\n#\n# volatile-lru -> remove the key with an expire set using an LRU algorithm\n# allkeys-lru -> remove any key according to the LRU algorithm\n# volatile-random -> remove a random key with an expire set\n# allkeys-random -> remove a random key, any key\n# volatile-ttl -> remove the key with the nearest expire time (minor TTL)\n# noeviction -> don't expire at all, just return an error on write operations\n#\n# Note: with any of the above policies, Redis will return an error on write\n# operations, when there are no suitable keys for eviction.\n#\n# At the date of writing these commands are: set setnx setex append\n# incr decr rpush lpush rpushx lpushx linsert lset rpoplpush sadd\n# sinter sinterstore sunion sunionstore sdiff sdiffstore zadd zincrby\n# zunionstore zinterstore hset hsetnx hmset hincrby incrby decrby\n# getset mset msetnx exec sort\n#\n# The default is:\n#\nmaxmemory-policy allkeys-lru\n\n# LRU and minimal TTL algorithms are not precise algorithms but approximated\n# algorithms (in order to save memory), so you can tune it for speed or\n# accuracy. For default Redis will check five keys and pick the one that was\n# used less recently, you can change the sample size using the following\n# configuration directive.\n#\n# The default of 5 produces good enough results. 10 Approximates very closely\n# true LRU but costs a bit more CPU. 3 is very fast but not very accurate.\n#\n# maxmemory-samples 5\n\n############################## APPEND ONLY MODE ###############################\n\n# By default Redis asynchronously dumps the dataset on disk. This mode is\n# good enough in many applications, but an issue with the Redis process or\n# a power outage may result into a few minutes of writes lost (depending on\n# the configured save points).\n#\n# The Append Only File is an alternative persistence mode that provides\n# much better durability. For instance using the default data fsync policy\n# (see later in the config file) Redis can lose just one second of writes in a\n# dramatic event like a server power outage, or a single write if something\n# wrong with the Redis process itself happens, but the operating system is\n# still running correctly.\n#\n# AOF and RDB persistence can be enabled at the same time without problems.\n# If the AOF is enabled on startup Redis will load the AOF, that is the file\n# with the better durability guarantees.\n#\n# Please check http://redis.io/topics/persistence for more information.\n\nappendonly no\n\n# The name of the append only file (default: \"appendonly.aof\")\n\nappendfilename \"appendonly.aof\"\n\n# The fsync() call tells the Operating System to actually write data on disk\n# instead of waiting for more data in the output buffer. Some OS will really flush\n# data on disk, some other OS will just try to do it ASAP.\n#\n# Redis supports three different modes:\n#\n# no: don't fsync, just let the OS flush the data when it wants. Faster.\n# always: fsync after every write to the append only log. Slow, Safest.\n# everysec: fsync only one time every second. Compromise.\n#\n# The default is \"everysec\", as that's usually the right compromise between\n# speed and data safety. It's up to you to understand if you can relax this to\n# \"no\" that will let the operating system flush the output buffer when\n# it wants, for better performances (but if you can live with the idea of\n# some data loss consider the default persistence mode that's snapshotting),\n# or on the contrary, use \"always\" that's very slow but a bit safer than\n# everysec.\n#\n# More details please check the following article:\n# http://antirez.com/post/redis-persistence-demystified.html\n#\n# If unsure, use \"everysec\".\n\n# appendfsync always\nappendfsync everysec\n# appendfsync no\n\n# When the AOF fsync policy is set to always or everysec, and a background\n# saving process (a background save or AOF log background rewriting) is\n# performing a lot of I/O against the disk, in some Linux configurations\n# Redis may block too long on the fsync() call. Note that there is no fix for\n# this currently, as even performing fsync in a different thread will block\n# our synchronous write(2) call.\n#\n# In order to mitigate this problem it's possible to use the following option\n# that will prevent fsync() from being called in the main process while a\n# BGSAVE or BGREWRITEAOF is in progress.\n#\n# This means that while another child is saving, the durability of Redis is\n# the same as \"appendfsync none\". In practical terms, this means that it is\n# possible to lose up to 30 seconds of log in the worst scenario (with the\n# default Linux settings).\n#\n# If you have latency problems turn this to \"yes\". Otherwise leave it as\n# \"no\" that is the safest pick from the point of view of durability.\n\nno-appendfsync-on-rewrite no\n\n# Automatic rewrite of the append only file.\n# Redis is able to automatically rewrite the log file implicitly calling\n# BGREWRITEAOF when the AOF log size grows by the specified percentage.\n#\n# This is how it works: Redis remembers the size of the AOF file after the\n# latest rewrite (if no rewrite has happened since the restart, the size of\n# the AOF at startup is used).\n#\n# This base size is compared to the current size. If the current size is\n# bigger than the specified percentage, the rewrite is triggered. Also\n# you need to specify a minimal size for the AOF file to be rewritten, this\n# is useful to avoid rewriting the AOF file even if the percentage increase\n# is reached but it is still pretty small.\n#\n# Specify a percentage of zero in order to disable the automatic AOF\n# rewrite feature.\n\nauto-aof-rewrite-percentage 100\nauto-aof-rewrite-min-size 64mb\n\n# An AOF file may be found to be truncated at the end during the Redis\n# startup process, when the AOF data gets loaded back into memory.\n# This may happen when the system where Redis is running\n# crashes, especially when an ext4 filesystem is mounted without the\n# data=ordered option (however this can't happen when Redis itself\n# crashes or aborts but the operating system still works correctly).\n#\n# Redis can either exit with an error when this happens, or load as much\n# data as possible (the default now) and start if the AOF file is found\n# to be truncated at the end. The following option controls this behavior.\n#\n# If aof-load-truncated is set to yes, a truncated AOF file is loaded and\n# the Redis server starts emitting a log to inform the user of the event.\n# Otherwise if the option is set to no, the server aborts with an error\n# and refuses to start. When the option is set to no, the user requires\n# to fix the AOF file using the \"redis-check-aof\" utility before to restart\n# the server.\n#\n# Note that if the AOF file will be found to be corrupted in the middle\n# the server will still exit with an error. This option only applies when\n# Redis will try to read more data from the AOF file but not enough bytes\n# will be found.\naof-load-truncated yes\n\n################################ LUA SCRIPTING ###############################\n\n# Max execution time of a Lua script in milliseconds.\n#\n# If the maximum execution time is reached Redis will log that a script is\n# still in execution after the maximum allowed time and will start to\n# reply to queries with an error.\n#\n# When a long running script exceeds the maximum execution time only the\n# SCRIPT KILL and SHUTDOWN NOSAVE commands are available. The first can be\n# used to stop a script that did not yet called write commands. The second\n# is the only way to shut down the server in the case a write command was\n# already issued by the script but the user doesn't want to wait for the natural\n# termination of the script.\n#\n# Set it to 0 or a negative value for unlimited execution without warnings.\nlua-time-limit 5000\n\n################################ REDIS CLUSTER ###############################\n#\n# ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++\n# WARNING EXPERIMENTAL: Redis Cluster is considered to be stable code, however\n# in order to mark it as \"mature\" we need to wait for a non trivial percentage\n# of users to deploy it in production.\n# ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++\n#\n# Normal Redis instances can't be part of a Redis Cluster; only nodes that are\n# started as cluster nodes can. In order to start a Redis instance as a\n# cluster node enable the cluster support uncommenting the following:\n#\n# cluster-enabled yes\n\n# Every cluster node has a cluster configuration file. This file is not\n# intended to be edited by hand. It is created and updated by Redis nodes.\n# Every Redis Cluster node requires a different cluster configuration file.\n# Make sure that instances running in the same system do not have\n# overlapping cluster configuration file names.\n#\n# cluster-config-file nodes-6379.conf\n\n# Cluster node timeout is the amount of milliseconds a node must be unreachable\n# for it to be considered in failure state.\n# Most other internal time limits are multiple of the node timeout.\n#\n# cluster-node-timeout 15000\n\n# A slave of a failing master will avoid to start a failover if its data\n# looks too old.\n#\n# There is no simple way for a slave to actually have a exact measure of\n# its \"data age\", so the following two checks are performed:\n#\n# 1) If there are multiple slaves able to failover, they exchange messages\n# in order to try to give an advantage to the slave with the best\n# replication offset (more data from the master processed).\n# Slaves will try to get their rank by offset, and apply to the start\n# of the failover a delay proportional to their rank.\n#\n# 2) Every single slave computes the time of the last interaction with\n# its master. This can be the last ping or command received (if the master\n# is still in the \"connected\" state), or the time that elapsed since the\n# disconnection with the master (if the replication link is currently down).\n# If the last interaction is too old, the slave will not try to failover\n# at all.\n#\n# The point \"2\" can be tuned by user. Specifically a slave will not perform\n# the failover if, since the last interaction with the master, the time\n# elapsed is greater than:\n#\n# (node-timeout * slave-validity-factor) + repl-ping-slave-period\n#\n# So for example if node-timeout is 30 seconds, and the slave-validity-factor\n# is 10, and assuming a default repl-ping-slave-period of 10 seconds, the\n# slave will not try to failover if it was not able to talk with the master\n# for longer than 310 seconds.\n#\n# A large slave-validity-factor may allow slaves with too old data to failover\n# a master, while a too small value may prevent the cluster from being able to\n# elect a slave at all.\n#\n# For maximum availability, it is possible to set the slave-validity-factor\n# to a value of 0, which means, that slaves will always try to failover the\n# master regardless of the last time they interacted with the master.\n# (However they'll always try to apply a delay proportional to their\n# offset rank).\n#\n# Zero is the only value able to guarantee that when all the partitions heal\n# the cluster will always be able to continue.\n#\n# cluster-slave-validity-factor 10\n\n# Cluster slaves are able to migrate to orphaned masters, that are masters\n# that are left without working slaves. This improves the cluster ability\n# to resist to failures as otherwise an orphaned master can't be failed over\n# in case of failure if it has no working slaves.\n#\n# Slaves migrate to orphaned masters only if there are still at least a\n# given number of other working slaves for their old master. This number\n# is the \"migration barrier\". A migration barrier of 1 means that a slave\n# will migrate only if there is at least 1 other working slave for its master\n# and so forth. It usually reflects the number of slaves you want for every\n# master in your cluster.\n#\n# Default is 1 (slaves migrate only if their masters remain with at least\n# one slave). To disable migration just set it to a very large value.\n# A value of 0 can be set but is useful only for debugging and dangerous\n# in production.\n#\n# cluster-migration-barrier 1\n\n# By default Redis Cluster nodes stop accepting queries if they detect there\n# is at least an hash slot uncovered (no available node is serving it).\n# This way if the cluster is partially down (for example a range of hash slots\n# are no longer covered) all the cluster becomes, eventually, unavailable.\n# It automatically returns available as soon as all the slots are covered again.\n#\n# However sometimes you want the subset of the cluster which is working,\n# to continue to accept queries for the part of the key space that is still\n# covered. In order to do so, just set the cluster-require-full-coverage\n# option to no.\n#\n# cluster-require-full-coverage yes\n\n# In order to setup your cluster make sure to read the documentation\n# available at http://redis.io web site.\n\n################################## SLOW LOG ###################################\n\n# The Redis Slow Log is a system to log queries that exceeded a specified\n# execution time. The execution time does not include the I/O operations\n# like talking with the client, sending the reply and so forth,\n# but just the time needed to actually execute the command (this is the only\n# stage of command execution where the thread is blocked and can not serve\n# other requests in the meantime).\n#\n# You can configure the slow log with two parameters: one tells Redis\n# what is the execution time, in microseconds, to exceed in order for the\n# command to get logged, and the other parameter is the length of the\n# slow log. When a new command is logged the oldest one is removed from the\n# queue of logged commands.\n\n# The following time is expressed in microseconds, so 1000000 is equivalent\n# to one second. Note that a negative number disables the slow log, while\n# a value of zero forces the logging of every command.\nslowlog-log-slower-than 10000\n\n# There is no limit to this length. Just be aware that it will consume memory.\n# You can reclaim memory used by the slow log with SLOWLOG RESET.\nslowlog-max-len 128\n\n################################ LATENCY MONITOR ##############################\n\n# The Redis latency monitoring subsystem samples different operations\n# at runtime in order to collect data related to possible sources of\n# latency of a Redis instance.\n#\n# Via the LATENCY command this information is available to the user that can\n# print graphs and obtain reports.\n#\n# The system only logs operations that were performed in a time equal or\n# greater than the amount of milliseconds specified via the\n# latency-monitor-threshold configuration directive. When its value is set\n# to zero, the latency monitor is turned off.\n#\n# By default latency monitoring is disabled since it is mostly not needed\n# if you don't have latency issues, and collecting data has a performance\n# impact, that while very small, can be measured under big load. Latency\n# monitoring can easily be enabled at runtime using the command\n# \"CONFIG SET latency-monitor-threshold \" if needed.\nlatency-monitor-threshold 0\n\n############################# EVENT NOTIFICATION ##############################\n\n# Redis can notify Pub/Sub clients about events happening in the key space.\n# This feature is documented at http://redis.io/topics/notifications\n#\n# For instance if keyspace events notification is enabled, and a client\n# performs a DEL operation on key \"foo\" stored in the Database 0, two\n# messages will be published via Pub/Sub:\n#\n# PUBLISH __keyspace@0__:foo del\n# PUBLISH __keyevent@0__:del foo\n#\n# It is possible to select the events that Redis will notify among a set\n# of classes. Every class is identified by a single character:\n#\n# K Keyspace events, published with __keyspace@__ prefix.\n# E Keyevent events, published with __keyevent@__ prefix.\n# g Generic commands (non-type specific) like DEL, EXPIRE, RENAME, ...\n# $ String commands\n# l List commands\n# s Set commands\n# h Hash commands\n# z Sorted set commands\n# x Expired events (events generated every time a key expires)\n# e Evicted events (events generated when a key is evicted for maxmemory)\n# A Alias for g$lshzxe, so that the \"AKE\" string means all the events.\n#\n# The \"notify-keyspace-events\" takes as argument a string that is composed\n# of zero or multiple characters. The empty string means that notifications\n# are disabled.\n#\n# Example: to enable list and generic events, from the point of view of the\n# event name, use:\n#\n# notify-keyspace-events Elg\n#\n# Example 2: to get the stream of the expired keys subscribing to channel\n# name __keyevent@0__:expired use:\n#\n# notify-keyspace-events Ex\n#\n# By default all notifications are disabled because most users don't need\n# this feature and the feature has some overhead. Note that if you don't\n# specify at least one of K or E, no events will be delivered.\nnotify-keyspace-events \"\"\n\n############################### ADVANCED CONFIG ###############################\n\n# Hashes are encoded using a memory efficient data structure when they have a\n# small number of entries, and the biggest entry does not exceed a given\n# threshold. These thresholds can be configured using the following directives.\nhash-max-ziplist-entries 512\nhash-max-ziplist-value 64\n\n# Lists are also encoded in a special way to save a lot of space.\n# The number of entries allowed per internal list node can be specified\n# as a fixed maximum size or a maximum number of elements.\n# For a fixed maximum size, use -5 through -1, meaning:\n# -5: max size: 64 Kb <-- not recommended for normal workloads\n# -4: max size: 32 Kb <-- not recommended\n# -3: max size: 16 Kb <-- probably not recommended\n# -2: max size: 8 Kb <-- good\n# -1: max size: 4 Kb <-- good\n# Positive numbers mean store up to _exactly_ that number of elements\n# per list node.\n# The highest performing option is usually -2 (8 Kb size) or -1 (4 Kb size),\n# but if your use case is unique, adjust the settings as necessary.\nlist-max-ziplist-size -2\n\n# Lists may also be compressed.\n# Compress depth is the number of quicklist ziplist nodes from *each* side of\n# the list to *exclude* from compression. The head and tail of the list\n# are always uncompressed for fast push/pop operations. Settings are:\n# 0: disable all list compression\n# 1: depth 1 means \"don't start compressing until after 1 node into the list,\n# going from either the head or tail\"\n# So: [head]->node->node->...->node->[tail]\n# [head], [tail] will always be uncompressed; inner nodes will compress.\n# 2: [head]->[next]->node->node->...->node->[prev]->[tail]\n# 2 here means: don't compress head or head->next or tail->prev or tail,\n# but compress all nodes between them.\n# 3: [head]->[next]->[next]->node->node->...->node->[prev]->[prev]->[tail]\n# etc.\nlist-compress-depth 0\n\n# Sets have a special encoding in just one case: when a set is composed\n# of just strings that happen to be integers in radix 10 in the range\n# of 64 bit signed integers.\n# The following configuration setting sets the limit in the size of the\n# set in order to use this special memory saving encoding.\nset-max-intset-entries 512\n\n# Similarly to hashes and lists, sorted sets are also specially encoded in\n# order to save a lot of space. This encoding is only used when the length and\n# elements of a sorted set are below the following limits:\nzset-max-ziplist-entries 128\nzset-max-ziplist-value 64\n\n# HyperLogLog sparse representation bytes limit. The limit includes the\n# 16 bytes header. When an HyperLogLog using the sparse representation crosses\n# this limit, it is converted into the dense representation.\n#\n# A value greater than 16000 is totally useless, since at that point the\n# dense representation is more memory efficient.\n#\n# The suggested value is ~ 3000 in order to have the benefits of\n# the space efficient encoding without slowing down too much PFADD,\n# which is O(N) with the sparse encoding. The value can be raised to\n# ~ 10000 when CPU is not a concern, but space is, and the data set is\n# composed of many HyperLogLogs with cardinality in the 0 - 15000 range.\nhll-sparse-max-bytes 3000\n\n# Active rehashing uses 1 millisecond every 100 milliseconds of CPU time in\n# order to help rehashing the main Redis hash table (the one mapping top-level\n# keys to values). The hash table implementation Redis uses (see dict.c)\n# performs a lazy rehashing: the more operation you run into a hash table\n# that is rehashing, the more rehashing \"steps\" are performed, so if the\n# server is idle the rehashing is never complete and some more memory is used\n# by the hash table.\n#\n# The default is to use this millisecond 10 times every second in order to\n# actively rehash the main dictionaries, freeing memory when possible.\n#\n# If unsure:\n# use \"activerehashing no\" if you have hard latency requirements and it is\n# not a good thing in your environment that Redis can reply from time to time\n# to queries with 2 milliseconds delay.\n#\n# use \"activerehashing yes\" if you don't have such hard requirements but\n# want to free memory asap when possible.\nactiverehashing yes\n\n# The client output buffer limits can be used to force disconnection of clients\n# that are not reading data from the server fast enough for some reason (a\n# common reason is that a Pub/Sub client can't consume messages as fast as the\n# publisher can produce them).\n#\n# The limit can be set differently for the three different classes of clients:\n#\n# normal -> normal clients including MONITOR clients\n# slave -> slave clients\n# pubsub -> clients subscribed to at least one pubsub channel or pattern\n#\n# The syntax of every client-output-buffer-limit directive is the following:\n#\n# client-output-buffer-limit \n#\n# A client is immediately disconnected once the hard limit is reached, or if\n# the soft limit is reached and remains reached for the specified number of\n# seconds (continuously).\n# So for instance if the hard limit is 32 megabytes and the soft limit is\n# 16 megabytes / 10 seconds, the client will get disconnected immediately\n# if the size of the output buffers reach 32 megabytes, but will also get\n# disconnected if the client reaches 16 megabytes and continuously overcomes\n# the limit for 10 seconds.\n#\n# By default normal clients are not limited because they don't receive data\n# without asking (in a push way), but just after a request, so only\n# asynchronous clients may create a scenario where data is requested faster\n# than it can read.\n#\n# Instead there is a default limit for pubsub and slave clients, since\n# subscribers and slaves receive data in a push fashion.\n#\n# Both the hard or the soft limit can be disabled by setting them to zero.\nclient-output-buffer-limit normal 0 0 0\nclient-output-buffer-limit slave 256mb 64mb 60\nclient-output-buffer-limit pubsub 32mb 8mb 60\n\n# Redis calls an internal function to perform many background tasks, like\n# closing connections of clients in timeout, purging expired keys that are\n# never requested, and so forth.\n#\n# Not all tasks are performed with the same frequency, but Redis checks for\n# tasks to perform according to the specified \"hz\" value.\n#\n# By default \"hz\" is set to 10. Raising the value will use more CPU when\n# Redis is idle, but at the same time will make Redis more responsive when\n# there are many keys expiring at the same time, and timeouts may be\n# handled with more precision.\n#\n# The range is between 1 and 500, however a value over 100 is usually not\n# a good idea. Most users should use the default of 10 and raise this up to\n# 100 only in environments where very low latency is required.\nhz 10\n\n# When a child rewrites the AOF file, if the following option is enabled\n# the file will be fsync-ed every 32 MB of data generated. This is useful\n# in order to commit the file to the disk more incrementally and avoid\n# big latency spikes.\naof-rewrite-incremental-fsync yes\n"
},
{
"path": "docs/docker/spug.ini",
"content": "[supervisord]\nnodaemon=true\n\n[program:nginx]\ncommand = nginx -g \"daemon off;\"\nautostart = true\n\n[program:redis]\ncommand = redis-server /etc/redis.conf\nautostart = true\n\n[program:spug-api]\ncommand = sh /data/spug/spug_api/tools/start-api.sh\nautostart = true\nstdout_logfile = /data/spug/spug_api/logs/api.log\nredirect_stderr = true\n\n[program:spug-ws]\ncommand = sh /data/spug/spug_api/tools/start-ws.sh\nautostart = true\nstdout_logfile = /data/spug/spug_api/logs/ws.log\nredirect_stderr = true\n\n[program:spug-worker]\ncommand = sh /data/spug/spug_api/tools/start-worker.sh\nautostart = true\nstdout_logfile = /data/spug/spug_api/logs/worker.log\nredirect_stderr = true\n\n[program:spug-monitor]\ncommand = sh /data/spug/spug_api/tools/start-monitor.sh\nautostart = true\nstartsecs = 3\nstdout_logfile = /data/spug/spug_api/logs/monitor.log\nredirect_stderr = true\n\n[program:spug-scheduler]\ncommand = sh /data/spug/spug_api/tools/start-scheduler.sh\nautostart = true\nstartsecs = 3\nstdout_logfile = /data/spug/spug_api/logs/scheduler.log\nredirect_stderr = true\n"
},
{
"path": "docs/docker/ssh_config",
"content": "Host *\n StrictHostKeyChecking no\n"
},
{
"path": "docs/install.sh",
"content": "#!/bin/bash\n# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\n\nset -e\n\n\nfunction spug_banner() {\n\necho \" \";\necho \" #### ##### # # #### \";\necho \"# # # # # # #\";\necho \" #### # # # # # \";\necho \" # ##### # # # ###\";\necho \"# # # # # # #\";\necho \" #### # #### #### \";\necho \" \";\n\n}\n\n\nfunction init_system_lib() {\n source /etc/os-release\n case $ID in\n centos|fedora|rhel)\n echo \"开始安装/更新可能缺少的依赖: git mariadb-server mariadb-devel python3-devel gcc openldap-devel redis nginx supervisor python36\"\n yum install -y epel-release\n yum install -y git mariadb-server mariadb-devel python3-devel gcc openldap-devel redis nginx supervisor python36\n sed -i 's/ default_server//g' /etc/nginx/nginx.conf\n MYSQL_CONF=/etc/my.cnf.d/spug.cnf\n SUPERVISOR_CONF=/etc/supervisord.d/spug.ini\n REDIS_SRV=redis\n SUPERVISOR_SRV=supervisord\n ;;\n\n debian|ubuntu|devuan)\n echo \"开始安装/更新可能缺少的依赖: git mariadb-server libmariadbd-dev python3-venv libsasl2-dev libldap2-dev redis-server nginx supervisor\"\n apt update\n apt install -y git mariadb-server libmariadbd-dev python3-dev python3-venv libsasl2-dev libldap2-dev redis-server nginx supervisor\n rm -f /etc/nginx/sites-enabled/default\n MYSQL_CONF=/etc/mysql/conf.d/spug.cnf\n SUPERVISOR_CONF=/etc/supervisor/conf.d/spug.conf\n REDIS_SRV=redis-server\n SUPERVISOR_SRV=supervisor\n ;;\n *)\n exit 1\n ;;\n esac\n}\n\n\nfunction install_spug() {\n echo \"开始安装Spug...\"\n mkdir -p /data\n cd /data\n git clone --depth=1 https://gitee.com/openspug/spug.git\n curl -o /tmp/web_latest.tar.gz https://spug.dev/installer/web_latest.tar.gz\n tar xf /tmp/web_latest.tar.gz -C spug/spug_web/\n cd spug/spug_api\n python3 -m venv venv\n source venv/bin/activate\n\n pip install wheel -i https://pypi.doubanio.com/simple/\n pip install gunicorn mysqlclient -i https://pypi.doubanio.com/simple/\n pip install -r requirements.txt -i https://pypi.doubanio.com/simple/\n}\n\n\nfunction setup_conf() {\n\n echo \"开始配置Spug配置...\"\n# mysql conf\ncat << EOF > $MYSQL_CONF\n[mysqld]\nbind-address=127.0.0.1\nEOF\n\n# spug conf\ncat << EOF > spug/overrides.py\nDEBUG = False\nALLOWED_HOSTS = ['127.0.0.1']\n\nDATABASES = {\n 'default': {\n 'ATOMIC_REQUESTS': True,\n 'ENGINE': 'django.db.backends.mysql',\n 'NAME': 'spug',\n 'USER': 'spug',\n 'PASSWORD': 'spug.dev',\n 'HOST': '127.0.0.1',\n 'OPTIONS': {\n 'charset': 'utf8mb4',\n 'sql_mode': 'STRICT_TRANS_TABLES',\n }\n }\n}\nEOF\n\ncat << EOF > $SUPERVISOR_CONF\n[program:spug-api]\ncommand = bash /data/spug/spug_api/tools/start-api.sh\nautostart = true\nstdout_logfile = /data/spug/spug_api/logs/api.log\nredirect_stderr = true\n\n[program:spug-ws]\ncommand = bash /data/spug/spug_api/tools/start-ws.sh\nautostart = true\nstdout_logfile = /data/spug/spug_api/logs/ws.log\nredirect_stderr = true\n\n[program:spug-worker]\ncommand = bash /data/spug/spug_api/tools/start-worker.sh\nautostart = true\nstdout_logfile = /data/spug/spug_api/logs/worker.log\nredirect_stderr = true\n\n[program:spug-monitor]\ncommand = bash /data/spug/spug_api/tools/start-monitor.sh\nautostart = true\nstdout_logfile = /data/spug/spug_api/logs/monitor.log\nredirect_stderr = true\n\n[program:spug-scheduler]\ncommand = bash /data/spug/spug_api/tools/start-scheduler.sh\nautostart = true\nstdout_logfile = /data/spug/spug_api/logs/scheduler.log\nredirect_stderr = true\nEOF\n\ncat << EOF > /etc/nginx/conf.d/spug.conf\nserver {\n listen 80 default_server;\n root /data/spug/spug_web/build/;\n\n location ^~ /api/ {\n rewrite ^/api(.*) \\$1 break;\n proxy_pass http://127.0.0.1:9001;\n proxy_redirect off;\n proxy_set_header X-Real-IP \\$remote_addr;\n }\n\n location ^~ /api/ws/ {\n rewrite ^/api(.*) \\$1 break;\n proxy_pass http://127.0.0.1:9002;\n proxy_http_version 1.1;\n proxy_set_header Upgrade \\$http_upgrade;\n proxy_set_header Connection \"Upgrade\";\n proxy_set_header X-Real-IP \\$remote_addr;\n }\n\n error_page 404 /index.html;\n}\nEOF\n\n\nsystemctl start mariadb\nsystemctl enable mariadb\n\nmysql -e \"create database spug default character set utf8mb4 collate utf8mb4_unicode_ci;\"\nmysql -e \"grant all on spug.* to spug@127.0.0.1 identified by 'spug.dev'\"\nmysql -e \"flush privileges\"\n\npython manage.py initdb\npython manage.py useradd -u admin -p spug.dev -s -n 管理员\n\n\nsystemctl enable nginx\nsystemctl enable $REDIS_SRV\nsystemctl enable $SUPERVISOR_SRV\n\nsystemctl restart nginx\nsystemctl start $REDIS_SRV\nsystemctl restart $SUPERVISOR_SRV\n\n}\n\n\nspug_banner\ninit_system_lib\ninstall_spug\nsetup_conf\n\necho -e \"\\n\\n\\033[33m安全警告：默认的数据库和Redis服务并不安全，请确保其仅监听在127.0.0.1，推荐参考官网文档自行加固安全配置！\\033[0m\"\necho -e \"\\033[32m安装成功！\\033[0m\"\necho \"默认管理员账户：admin 密码：spug.dev\"\necho \"默认数据库用户：spug 密码：spug.dev\"\n"
},
{
"path": "spug_api/.gitignore",
"content": "*.pyc\n/venv/\n__pycache__/\n/.idea/\n/db.sqlite3\nmigrations/\n/access.log\n/repos/*\n/logs/*\n"
},
{
"path": "spug_api/apps/account/__init__.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\n"
},
{
"path": "spug_api/apps/account/history.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom libs.mixins import AdminView\nfrom libs import json_response\nfrom apps.account.models import History\n\n\nclass HistoryView(AdminView):\n def get(self, request):\n histories = History.objects.all()\n return json_response(histories)\n"
},
{
"path": "spug_api/apps/account/management/commands/set.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.core.management.base import BaseCommand\nfrom apps.setting.utils import AppSetting\n\n\nclass Command(BaseCommand):\n help = '系统设置'\n\n def add_arguments(self, parser):\n parser.add_argument('target', type=str, help='设置对象')\n parser.add_argument('value', type=str, help='设置值')\n\n def echo_success(self, msg):\n self.stdout.write(self.style.SUCCESS(msg))\n\n def echo_error(self, msg):\n self.stderr.write(self.style.ERROR(msg))\n\n def print_help(self, *args):\n message = '''\n 系统设置命令用法：\n set mfa disable 禁用登录MFA\n '''\n self.stdout.write(message)\n\n def handle(self, *args, **options):\n target = options['target']\n if target == 'mfa':\n if options['value'] != 'disable':\n return self.echo_error(f'mfa设置，不支持的值【{options[\"value\"]}】')\n AppSetting.set('MFA', {'enable': False})\n self.echo_success('MFA已禁用')\n else:\n self.echo_error('未识别的操作')\n self.print_help()\n"
},
{
"path": "spug_api/apps/account/management/commands/update.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.core.management.base import BaseCommand\nfrom django.conf import settings\nimport subprocess\nimport requests\nimport os\n\n\nclass Command(BaseCommand):\n help = '升级Spug版本'\n\n def handle(self, *args, **options):\n version, is_repair = settings.SPUG_VERSION, False\n res = requests.get(f'https://api.spug.cc/apis/release/latest/?version={version}').json()\n if res['error']:\n return self.stderr.write(self.style.ERROR(f'获取新版本失败：{res[\"error\"]}'))\n if not res['data']['has_new']:\n self.stdout.write(res['data']['extra'])\n is_repair = True\n answer = input(f'\\r\\n当前已是最新版本，是否要进行修复性更新[y|n]？')\n else:\n version = res['data']['version']\n self.stdout.write(res['data']['content'])\n self.stdout.write('\\r\\n')\n self.stdout.write(res['data']['extra'])\n answer = input(f'\\r\\n发现新版本 {version} 是否更新[y|n]？')\n if answer.lower() != 'y':\n return\n\n # update web\n web_dir = os.path.join(settings.BASE_DIR, '../spug_web')\n commands = [\n f'curl -o /tmp/spug_web.tar.gz https://cdn.spug.cc/spug/web_{version}.tar.gz',\n f'rm -rf {web_dir}/build',\n f'tar xf /tmp/spug_web.tar.gz -C {web_dir}'\n ]\n task = subprocess.Popen(' && '.join(commands), shell=True)\n if task.wait() != 0:\n return self.stderr.write(self.style.ERROR('获取更新失败，排除网络问题后请附带输出内容至官方论坛反馈。'))\n\n # update api\n commands = [\n f'cd {settings.BASE_DIR}',\n f'git fetch origin refs/tags/{version}:refs/tags/{version} --no-tags',\n f'git checkout {version}'\n ]\n if is_repair:\n commands.insert(1, f'git tag -d {version}')\n task = subprocess.Popen(' && '.join(commands), shell=True)\n if task.wait() != 0:\n return self.stderr.write(self.style.ERROR('获取更新失败，排除网络问题后请附带输出内容至官方论坛反馈。'))\n\n # update dep\n commands = [\n f'cd {settings.BASE_DIR}',\n 'pip3 install -r requirements.txt -i https://pypi.doubanio.com/simple/'\n ]\n task = subprocess.Popen(' && '.join(commands), shell=True)\n if task.wait() != 0:\n return self.stderr.write(self.style.ERROR('更新依赖包失败，排除网络问题后请附带输出内容至官方论坛反馈。'))\n\n # update db\n apps = [x.split('.')[-1] for x in settings.INSTALLED_APPS if x.startswith('apps.')]\n commands = [\n f'cd {settings.BASE_DIR}',\n f'python3 ./manage.py makemigrations ' + ' '.join(apps),\n f'python3 ./manage.py migrate',\n f'python3 ./tools/migrate.py {settings.SPUG_VERSION}'\n ]\n task = subprocess.Popen(' && '.join(commands), shell=True)\n if task.wait() != 0:\n return self.stderr.write(self.style.ERROR('更新表结构失败，请附带输出内容至官方论坛反馈。'))\n\n self.stdout.write(self.style.SUCCESS('''升级成功，请自行重启服务，如果通过官方文档安装一般重启命令为\n Docker: docker restart spug\n Centos: systemctl restart supervisord \n Ubuntu: systemctl restart supervisor\n '''))\n self.stderr.write(self.style.WARNING(f'最后别忘了刷新浏览器，确保系统设置/关于里的api与web版本一致哦～'))\n"
},
{
"path": "spug_api/apps/account/management/commands/updatedb.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.core.management.base import BaseCommand\nfrom django.core.management import execute_from_command_line\nfrom django.conf import settings\n\n\nclass Command(BaseCommand):\n help = '初始化/更新数据库'\n\n def handle(self, *args, **options):\n args = ['manage.py', 'makemigrations']\n apps = [x.split('.')[-1] for x in settings.INSTALLED_APPS if x.startswith('apps.')]\n execute_from_command_line(args + apps)\n execute_from_command_line(['manage.py', 'migrate'])\n self.stdout.write(self.style.SUCCESS('初始化/更新成功'))\n"
},
{
"path": "spug_api/apps/account/management/commands/user.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.core.management.base import BaseCommand\nfrom django.core.cache import cache\nfrom apps.account.models import User\n\n\nclass Command(BaseCommand):\n help = '账户管理'\n\n def add_arguments(self, parser):\n parser.add_argument('action', type=str, help='执行动作')\n parser.add_argument('-u', required=False, help='账户名称')\n parser.add_argument('-p', required=False, help='账户密码')\n parser.add_argument('-n', required=False, help='账户昵称')\n parser.add_argument('-s', default=False, action='store_true', help='是否是超级用户（默认否）')\n\n def echo_success(self, msg):\n self.stdout.write(self.style.SUCCESS(msg))\n\n def echo_error(self, msg):\n self.stderr.write(self.style.ERROR(msg))\n\n def print_help(self, *args):\n message = '''\n 账户管理命令用法：\n user add 创建账户，例如：user add -u admin -p 123 -n 管理员 -s\n user reset 重置账户密码，例如：user reset -u admin -p 123\n user enable 启用被禁用的账户，例如：user enable -u admin\n '''\n self.stdout.write(message)\n\n def handle(self, *args, **options):\n action = options['action']\n if action == 'add':\n if not all((options['u'], options['p'], options['n'])):\n self.echo_error('缺少参数')\n self.print_help()\n elif User.objects.filter(username=options['u'], deleted_by_id__isnull=True).exists():\n self.echo_error(f'已存在登录名为【{options[\"u\"]}】的用户')\n else:\n User.objects.create(\n username=options['u'],\n nickname=options['n'],\n password_hash=User.make_password(options['p']),\n is_supper=options['s'],\n )\n self.echo_success('创建用户成功')\n elif action == 'enable':\n if not options['u']:\n self.echo_error('缺少参数')\n self.print_help()\n user = User.objects.filter(username=options['u'], deleted_by_id__isnull=True).first()\n if user:\n user.is_active = True\n user.save()\n cache.delete(user.username)\n self.echo_success('账户已启用')\n elif action == 'reset':\n if not all((options['u'], options['p'])):\n self.echo_error('缺少参数')\n self.print_help()\n user = User.objects.filter(username=options['u'], deleted_by_id__isnull=True).first()\n if not user:\n return self.echo_error(f'未找到登录名为【{options[\"u\"]}】的账户')\n user.password_hash = User.make_password(options['p'])\n user.save()\n cache.delete(user.username)\n self.echo_success('账户密码已重置')\n else:\n self.echo_error('未识别的操作')\n self.print_help()\n"
},
{
"path": "spug_api/apps/account/models.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.db import models\nfrom django.core.cache import cache\nfrom libs import ModelMixin, human_datetime\nfrom django.contrib.auth.hashers import make_password, check_password\nimport json\n\n\nclass User(models.Model, ModelMixin):\n username = models.CharField(max_length=100)\n nickname = models.CharField(max_length=100)\n password_hash = models.CharField(max_length=100) # hashed password\n type = models.CharField(max_length=20, default='default')\n is_supper = models.BooleanField(default=False)\n is_active = models.BooleanField(default=True)\n access_token = models.CharField(max_length=32)\n token_expired = models.IntegerField(null=True)\n last_login = models.CharField(max_length=20)\n last_ip = models.CharField(max_length=50)\n wx_token = models.CharField(max_length=50, null=True)\n roles = models.ManyToManyField('Role', db_table='user_role_rel')\n\n created_at = models.CharField(max_length=20, default=human_datetime)\n created_by = models.ForeignKey('User', models.PROTECT, related_name='+', null=True)\n deleted_at = models.CharField(max_length=20, null=True)\n deleted_by = models.ForeignKey('User', models.PROTECT, related_name='+', null=True)\n\n @staticmethod\n def make_password(plain_password: str) -> str:\n return make_password(plain_password, hasher='pbkdf2_sha256')\n\n def verify_password(self, plain_password: str) -> bool:\n return check_password(plain_password, self.password_hash)\n\n def get_perms_cache(self):\n return cache.get(f'perms_{self.id}', set())\n\n def set_perms_cache(self, value=None):\n cache.set(f'perms_{self.id}', value or set())\n\n @property\n def page_perms(self):\n data = self.get_perms_cache()\n if data:\n return data\n for item in self.roles.all():\n if item.page_perms:\n perms = json.loads(item.page_perms)\n for m, v in perms.items():\n for p, d in v.items():\n data.update(f'{m}.{p}.{x}' for x in d)\n self.set_perms_cache(data)\n return data\n\n @property\n def deploy_perms(self):\n data = {'apps': set(), 'envs': set()}\n for item in self.roles.all():\n if item.deploy_perms:\n perms = json.loads(item.deploy_perms)\n data['apps'].update(perms.get('apps', []))\n data['envs'].update(perms.get('envs', []))\n data['apps'].update(x.id for x in self.app_set.all())\n return data\n\n @property\n def group_perms(self):\n data = set()\n for item in self.roles.all():\n if item.group_perms:\n data.update(json.loads(item.group_perms))\n return list(data)\n\n def has_perms(self, codes):\n if self.is_supper:\n return True\n return self.page_perms.intersection(codes)\n\n def __repr__(self):\n return '' % self.username\n\n class Meta:\n db_table = 'users'\n ordering = ('-id',)\n\n\nclass Role(models.Model, ModelMixin):\n name = models.CharField(max_length=50)\n desc = models.CharField(max_length=255, null=True)\n page_perms = models.TextField(null=True)\n deploy_perms = models.TextField(null=True)\n group_perms = models.TextField(null=True)\n created_at = models.CharField(max_length=20, default=human_datetime)\n created_by = models.ForeignKey(User, on_delete=models.PROTECT, related_name='+')\n\n def to_dict(self, *args, **kwargs):\n tmp = super().to_dict(*args, **kwargs)\n tmp['page_perms'] = json.loads(self.page_perms) if self.page_perms else {}\n tmp['deploy_perms'] = json.loads(self.deploy_perms) if self.deploy_perms else {}\n tmp['group_perms'] = json.loads(self.group_perms) if self.group_perms else []\n tmp['used'] = self.user_set.filter(deleted_by_id__isnull=True).count()\n return tmp\n\n def add_deploy_perm(self, target, value):\n perms = {'apps': [], 'envs': []}\n if self.deploy_perms:\n perms.update(json.loads(self.deploy_perms))\n perms[target].append(value)\n self.deploy_perms = json.dumps(perms)\n self.save()\n\n def clear_perms_cache(self):\n for item in self.user_set.all():\n item.set_perms_cache()\n\n def __repr__(self):\n return '' % self.name\n\n class Meta:\n db_table = 'roles'\n ordering = ('-id',)\n\n\nclass History(models.Model, ModelMixin):\n username = models.CharField(max_length=100, null=True)\n type = models.CharField(max_length=20, default='default')\n ip = models.CharField(max_length=50)\n agent = models.CharField(max_length=255, null=True)\n message = models.CharField(max_length=255, null=True)\n is_success = models.BooleanField(default=True)\n created_at = models.CharField(max_length=20, default=human_datetime)\n\n class Meta:\n db_table = 'login_histories'\n ordering = ('-id',)\n"
},
{
"path": "spug_api/apps/account/urls.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.conf.urls import url\n\nfrom apps.account.views import *\nfrom apps.account.history import *\n\nurlpatterns = [\n url(r'^login/$', login),\n url(r'^logout/$', logout),\n url(r'^user/$', UserView.as_view()),\n url(r'^role/$', RoleView.as_view()),\n url(r'^self/$', SelfView.as_view()),\n url(r'^login/history/$', HistoryView.as_view())\n]\n"
},
{
"path": "spug_api/apps/account/utils.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom apps.host.models import Group\nimport re\n\n\ndef get_host_perms(user):\n ids = sub_ids = set(user.group_perms)\n while sub_ids:\n sub_ids = [x.id for x in Group.objects.filter(parent_id__in=sub_ids)]\n ids.update(sub_ids)\n return set(x.host_id for x in Group.hosts.through.objects.filter(group_id__in=ids))\n\n\ndef has_host_perm(user, target):\n if user.is_supper:\n return True\n host_ids = get_host_perms(user)\n if isinstance(target, (list, set, tuple)):\n return set(target).issubset(host_ids)\n return int(target) in host_ids\n\n\ndef verify_password(password):\n if len(password) < 8:\n return False\n if not all(map(lambda x: re.findall(x, password), ['[0-9]', '[a-z]', '[A-Z]'])):\n return False\n return True\n"
},
{
"path": "spug_api/apps/account/views.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.core.cache import cache\nfrom django.conf import settings\nfrom libs.mixins import AdminView, View\nfrom libs import JsonParser, Argument, human_datetime, json_response\nfrom libs.utils import get_request_real_ip, generate_random_str\nfrom libs.push import send_login_code\nfrom apps.account.models import User, Role, History\nfrom apps.setting.utils import AppSetting\nfrom apps.account.utils import verify_password\nfrom libs.ldap import LDAP\nfrom functools import partial\nimport user_agents\nimport ipaddress\nimport time\nimport uuid\nimport json\n\n\nclass UserView(AdminView):\n def get(self, request):\n users = []\n for u in User.objects.filter(deleted_by_id__isnull=True):\n tmp = u.to_dict(excludes=('access_token', 'password_hash'))\n tmp['role_ids'] = [x.id for x in u.roles.all()]\n tmp['password'] = '******'\n users.append(tmp)\n return json_response(users)\n\n def post(self, request):\n form, error = JsonParser(\n Argument('id', type=int, required=False),\n Argument('username', help='请输入登录名'),\n Argument('password', help='请输入密码'),\n Argument('nickname', help='请输入姓名'),\n Argument('role_ids', type=list, default=[]),\n Argument('wx_token', required=False),\n ).parse(request.body)\n if error is None:\n user = User.objects.filter(username=form.username, deleted_by_id__isnull=True).first()\n if user and (not form.id or form.id != user.id):\n return json_response(error=f'已存在登录名为【{form.username}】的用户')\n\n role_ids, password = form.pop('role_ids'), form.pop('password')\n if form.id:\n user = User.objects.get(pk=form.id)\n user.update_by_dict(form)\n else:\n if not verify_password(password):\n return json_response(error='请设置至少8位包含数字、小写和大写字母的新密码')\n user = User.objects.create(\n password_hash=User.make_password(password),\n created_by=request.user,\n **form\n )\n user.roles.set(role_ids)\n user.set_perms_cache()\n return json_response(error=error)\n\n def patch(self, request):\n form, error = JsonParser(\n Argument('id', type=int, help='参数错误'),\n Argument('password', required=False),\n Argument('is_active', type=bool, required=False),\n ).parse(request.body)\n if error is None:\n user = User.objects.get(pk=form.id)\n if form.password:\n if not verify_password(form.password):\n return json_response(error='请设置至少8位包含数字、小写和大写字母的新密码')\n user.token_expired = 0\n user.password_hash = User.make_password(form.pop('password'))\n if form.is_active is not None:\n user.is_active = form.is_active\n cache.delete(user.username)\n user.save()\n return json_response(error=error)\n\n def delete(self, request):\n form, error = JsonParser(\n Argument('id', type=int, help='请指定操作对象')\n ).parse(request.GET)\n if error is None:\n user = User.objects.filter(pk=form.id).first()\n if user:\n if user.type == 'ldap':\n return json_response(error='ldap账户无法删除，请使用禁用功能来禁止该账户访问系统')\n if user.id == request.user.id:\n return json_response(error='无法删除当前登录账户')\n user.is_active = True\n user.deleted_at = human_datetime()\n user.deleted_by = request.user\n user.roles.clear()\n user.save()\n return json_response(error=error)\n\n\nclass RoleView(AdminView):\n def get(self, request):\n roles = Role.objects.all()\n return json_response(roles)\n\n def post(self, request):\n form, error = JsonParser(\n Argument('id', type=int, required=False),\n Argument('name', help='请输入角色名称'),\n Argument('desc', required=False)\n ).parse(request.body)\n if error is None:\n if form.id:\n Role.objects.filter(pk=form.id).update(**form)\n else:\n Role.objects.create(created_by=request.user, **form)\n return json_response(error=error)\n\n def patch(self, request):\n form, error = JsonParser(\n Argument('id', type=int, help='参数错误'),\n Argument('page_perms', type=dict, required=False),\n Argument('deploy_perms', type=dict, required=False),\n Argument('group_perms', type=list, required=False)\n ).parse(request.body)\n if error is None:\n role = Role.objects.filter(pk=form.pop('id')).first()\n if not role:\n return json_response(error='未找到指定角色')\n if form.page_perms is not None:\n role.page_perms = json.dumps(form.page_perms)\n role.clear_perms_cache()\n if form.deploy_perms is not None:\n role.deploy_perms = json.dumps(form.deploy_perms)\n if form.group_perms is not None:\n role.group_perms = json.dumps(form.group_perms)\n role.user_set.update(token_expired=0)\n role.save()\n return json_response(error=error)\n\n def delete(self, request):\n form, error = JsonParser(\n Argument('id', type=int, help='参数错误')\n ).parse(request.GET)\n if error is None:\n role = Role.objects.get(pk=form.id)\n if role.user_set.exists():\n return json_response(error='已有用户使用了该角色，请解除关联后再尝试删除')\n role.delete()\n return json_response(error=error)\n\n\nclass SelfView(View):\n def get(self, request):\n data = request.user.to_dict(selects=('nickname', 'wx_token'))\n return json_response(data)\n\n def patch(self, request):\n form, error = JsonParser(\n Argument('old_password', required=False),\n Argument('new_password', required=False),\n Argument('nickname', required=False, help='请输入昵称'),\n Argument('wx_token', required=False),\n ).parse(request.body)\n if error is None:\n if form.old_password and form.new_password:\n if request.user.type == 'ldap':\n return json_response(error='LDAP账户无法修改密码')\n\n if not verify_password(form.new_password):\n return json_response(error='请设置至少8位包含数字、小写和大写字母的新密码')\n\n if request.user.verify_password(form.old_password):\n request.user.password_hash = User.make_password(form.new_password)\n request.user.token_expired = 0\n request.user.save()\n return json_response()\n else:\n return json_response(error='原密码错误，请重新输入')\n if form.nickname is not None:\n request.user.nickname = form.nickname\n if form.wx_token is not None:\n request.user.wx_token = form.wx_token\n request.user.save()\n return json_response(error=error)\n\n\ndef login(request):\n form, error = JsonParser(\n Argument('username', help='请输入用户名'),\n Argument('password', help='请输入密码'),\n Argument('captcha', required=False),\n Argument('type', required=False)\n ).parse(request.body)\n if error is None:\n handle_response = partial(handle_login_record, request, form.username, form.type)\n user = User.objects.filter(username=form.username, type=form.type).first()\n if user and not user.is_active:\n return handle_response(error=\"账户已被系统禁用\")\n if form.type == 'ldap':\n config = AppSetting.get_default('ldap_service')\n if not config:\n return handle_response(error='请在系统设置中配置LDAP后再尝试通过该方式登录')\n ldap = LDAP(**config)\n is_success, message = ldap.valid_user(form.username, form.password)\n if is_success:\n if not user:\n user = User.objects.create(username=form.username, nickname=form.username, type=form.type)\n return handle_user_info(handle_response, request, user, form.captcha)\n elif message:\n return handle_response(error=message)\n else:\n if user and user.deleted_by is None:\n if user.verify_password(form.password):\n return handle_user_info(handle_response, request, user, form.captcha)\n\n value = cache.get_or_set(form.username, 0, 86400)\n if value >= 3:\n if user and user.is_active:\n user.is_active = False\n user.save()\n return handle_response(error='账户已被系统禁用')\n cache.set(form.username, value + 1, 86400)\n return handle_response(error=\"用户名或密码错误，连续多次错误账户将会被禁用\")\n return json_response(error=error)\n\n\ndef handle_login_record(request, username, login_type, error=None):\n x_real_ip = get_request_real_ip(request.headers)\n user_agent = user_agents.parse(request.headers.get('User-Agent'))\n History.objects.create(\n username=username,\n type=login_type,\n ip=x_real_ip,\n agent=user_agent,\n is_success=False if error else True,\n message=error\n )\n if error:\n return json_response(error=error)\n\n\ndef handle_user_info(handle_response, request, user, captcha):\n cache.delete(user.username)\n key = f'{user.username}:code'\n if captcha:\n code = cache.get(key)\n if not code:\n return handle_response(error='验证码已失效，请重新获取')\n if code != captcha:\n ttl = cache.ttl(key)\n cache.expire(key, ttl - 100)\n return handle_response(error='验证码错误')\n cache.delete(key)\n else:\n mfa = AppSetting.get_default('MFA', {'enable': False})\n if mfa['enable']:\n if not user.wx_token:\n return handle_response(error='已启用登录双重认证，但您的账户未配置推送标识，请联系管理员')\n spug_push_key = AppSetting.get_default('spug_push_key')\n if not spug_push_key:\n return handle_response(error='已启用登录双重认证，但系统未配置推送服务，请联系管理员')\n code = generate_random_str(6)\n send_login_code(spug_push_key, user.wx_token, code)\n cache.set(key, code, 300)\n return json_response({'required_mfa': True})\n\n handle_response()\n x_real_ip = get_request_real_ip(request.headers)\n token_isvalid = user.access_token and len(user.access_token) == 32 and user.token_expired >= time.time()\n user.access_token = user.access_token if token_isvalid else uuid.uuid4().hex\n user.token_expired = time.time() + settings.TOKEN_TTL\n user.last_login = human_datetime()\n user.last_ip = x_real_ip\n user.save()\n verify_ip = AppSetting.get_default('verify_ip', True)\n return json_response({\n 'id': user.id,\n 'access_token': user.access_token,\n 'nickname': user.nickname,\n 'is_supper': user.is_supper,\n 'has_real_ip': x_real_ip and ipaddress.ip_address(x_real_ip).is_global if verify_ip else True,\n 'permissions': [] if user.is_supper else list(user.page_perms)\n })\n\n\ndef logout(request):\n request.user.token_expired = 0\n request.user.save()\n return json_response()\n"
},
{
"path": "spug_api/apps/alarm/__init__.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\n"
},
{
"path": "spug_api/apps/alarm/models.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.db import models\nfrom libs import ModelMixin, human_datetime\nfrom apps.account.models import User\nimport json\n\n\nclass Alarm(models.Model, ModelMixin):\n MODES = (\n ('1', '微信'),\n ('2', '短信'),\n ('3', '钉钉'),\n ('4', '邮件'),\n ('5', '企业微信'),\n ('6', '电话'),\n ('7', '飞书'),\n )\n STATUS = (\n ('1', '报警发生'),\n ('2', '故障恢复'),\n )\n name = models.CharField(max_length=50)\n type = models.CharField(max_length=50)\n target = models.CharField(max_length=100)\n notify_mode = models.CharField(max_length=255)\n notify_grp = models.CharField(max_length=255)\n status = models.CharField(max_length=2, choices=STATUS)\n duration = models.CharField(max_length=50)\n created_at = models.CharField(max_length=20, default=human_datetime)\n\n def to_dict(self, *args, **kwargs):\n tmp = super().to_dict(*args, **kwargs)\n tmp['notify_mode'] = ','.join(dict(self.MODES)[x] for x in json.loads(self.notify_mode))\n tmp['notify_grp'] = json.loads(self.notify_grp)\n tmp['status_alias'] = self.get_status_display()\n return tmp\n\n def __repr__(self):\n return '' % self.name\n\n class Meta:\n db_table = 'alarms'\n ordering = ('-id',)\n\n\nclass Group(models.Model, ModelMixin):\n name = models.CharField(max_length=50)\n desc = models.CharField(max_length=255, null=True)\n contacts = models.TextField(null=True)\n created_at = models.CharField(max_length=20, default=human_datetime)\n created_by = models.ForeignKey(User, models.PROTECT, related_name='+')\n\n def to_dict(self, *args, **kwargs):\n tmp = super().to_dict(*args, **kwargs)\n tmp['contacts'] = json.loads(self.contacts)\n return tmp\n\n def __repr__(self):\n return '' % self.name\n\n class Meta:\n db_table = 'alarm_groups'\n ordering = ('-id',)\n\n\nclass Contact(models.Model, ModelMixin):\n name = models.CharField(max_length=50)\n phone = models.CharField(max_length=20, null=True)\n email = models.CharField(max_length=255, null=True)\n ding = models.CharField(max_length=255, null=True)\n wx_token = models.CharField(max_length=255, null=True)\n qy_wx = models.CharField(max_length=255, null=True)\n feishu = models.CharField(max_length=255, null=True)\n secret = models.TextField(null=True)\n\n created_at = models.CharField(max_length=20, default=human_datetime)\n created_by = models.ForeignKey(User, models.PROTECT, related_name='+')\n\n def __repr__(self):\n return '' % self.name\n\n class Meta:\n db_table = 'alarm_contacts'\n ordering = ('-id',)\n"
},
{
"path": "spug_api/apps/alarm/urls.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.urls import path\n\nfrom .views import *\n\nurlpatterns = [\n path('alarm/', AlarmView.as_view()),\n path('group/', GroupView.as_view()),\n path('contact/', ContactView.as_view()),\n path('test/', handle_test),\n]\n"
},
{
"path": "spug_api/apps/alarm/views.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.views.generic import View\nfrom libs import json_response, JsonParser, Argument, auth\nfrom libs.spug import Notification\nfrom libs.push import get_contacts\nfrom apps.alarm.models import Alarm, Group, Contact\nfrom apps.monitor.models import Detection\nfrom apps.setting.utils import AppSetting\nimport json\n\n\nclass AlarmView(View):\n @auth('alarm.alarm.view')\n def get(self, request):\n alarms = Alarm.objects.all()\n return json_response(alarms)\n\n\nclass GroupView(View):\n @auth('alarm.group.view|monitor.monitor.add|monitor.monitor.edit|alarm.alarm.view')\n def get(self, request):\n groups = Group.objects.all()\n return json_response(groups)\n\n @auth('alarm.group.add|alarm.group.edit')\n def post(self, request):\n form, error = JsonParser(\n Argument('id', type=int, required=False),\n Argument('name', help='请输入组名'),\n Argument('contacts', type=list, help='请选择联系人'),\n Argument('desc', required=False)\n ).parse(request.body)\n if error is None:\n form.contacts = json.dumps(form.contacts)\n if form.id:\n Group.objects.filter(pk=form.id).update(**form)\n else:\n form.created_by = request.user\n Group.objects.create(**form)\n return json_response(error=error)\n\n @auth('alarm.group.del')\n def delete(self, request):\n form, error = JsonParser(\n Argument('id', type=int, help='请指定操作对象')\n ).parse(request.GET)\n if error is None:\n detection = Detection.objects.filter(notify_grp__regex=fr'[^0-9]{form.id}[^0-9]').first()\n if detection:\n return json_response(error=f'监控任务【{detection.name}】正在使用该报警组，请解除关联后再尝试删除该联系组')\n Group.objects.filter(pk=form.id).delete()\n return json_response(error=error)\n\n\nclass ContactView(View):\n @auth('alarm.contact.view|alarm.group.view|schedule.schedule.add|schedule.schedule.edit')\n def get(self, request):\n form, error = JsonParser(\n Argument('with_push', required=False),\n Argument('only_push', required=False),\n ).parse(request.GET)\n if error is None:\n response = []\n if form.with_push or form.only_push:\n push_key = AppSetting.get_default('spug_push_key')\n if push_key:\n response = get_contacts(push_key)\n if form.only_push:\n return json_response(response)\n\n for item in Contact.objects.all():\n response.append(item.to_dict())\n return json_response(response)\n return json_response(error=error)\n\n @auth('alarm.contact.add|alarm.contact.edit')\n def post(self, request):\n form, error = JsonParser(\n Argument('id', type=int, required=False),\n Argument('name', help='请输入联系人姓名'),\n Argument('phone', required=False),\n Argument('email', required=False),\n Argument('ding', required=False),\n Argument('wx_token', required=False),\n Argument('qy_wx', required=False),\n Argument('feishu', required=False),\n Argument('secret', required=False),\n ).parse(request.body)\n if error is None:\n if form.id:\n Contact.objects.filter(pk=form.id).update(**form)\n else:\n form.created_by = request.user\n Contact.objects.create(**form)\n return json_response(error=error)\n\n @auth('alarm.contact.del')\n def delete(self, request):\n form, error = JsonParser(\n Argument('id', type=int, help='请指定操作对象')\n ).parse(request.GET)\n if error is None:\n group = Group.objects.filter(contacts__contains=f'\\\"{form.id}\\\"').first()\n if group:\n return json_response(error=f'报警联系组【{group.name}】包含此联系人，请解除关联后再尝试删除该联系人')\n Contact.objects.filter(pk=form.id).delete()\n return json_response(error=error)\n\n\n@auth('alarm.contact.add|alarm.contact.edit')\ndef handle_test(request):\n form, error = JsonParser(\n Argument('mode', help='参数错误'),\n Argument('value', help='参数错误')\n ).parse(request.body)\n if error is None:\n notify = Notification(None, '1', 'https://spug.cc', 'Spug官网（测试）', '这是一条测试告警信息', None)\n if form.mode == '3':\n notify.monitor_by_dd([(form.value, None)])\n elif form.mode == '4':\n notify.monitor_by_email([form.value])\n elif form.mode == '5':\n notify.monitor_by_qy_wx([form.value])\n elif form.mode == '7':\n notify.monitor_by_fs([(form.value, None)])\n return json_response(error=error)\n"
},
{
"path": "spug_api/apps/apis/__init__.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\n"
},
{
"path": "spug_api/apps/apis/config.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.http.response import HttpResponse\nfrom django_redis import get_redis_connection\nfrom apps.config.models import Environment\nfrom apps.app.models import App\nfrom apps.setting.utils import AppSetting\nfrom apps.config.utils import compose_configs\nimport json\n\n\ndef get_configs(request):\n app, env_id, no_prefix = _parse_params(request)\n if not app or not env_id:\n return HttpResponse('Invalid params', status=400)\n\n configs = compose_configs(app, env_id, no_prefix)\n fmt = request.GET.get('format', 'kv')\n if fmt == 'kv':\n return _kv_response(configs)\n elif fmt == 'env':\n return _env_response(configs)\n elif fmt == 'json':\n return _json_response(configs)\n else:\n return HttpResponse('Unsupported output format', status=400)\n\n\ndef _kv_response(data):\n output = ''\n for k, v in sorted(data.items()):\n output += f'{k} = {v}\\r\\n'\n return HttpResponse(output, content_type='text/plain; charset=utf-8')\n\n\ndef _env_response(data):\n output = ''\n for k, v in sorted(data.items()):\n output += f'{k}={v}\\n'\n return HttpResponse(output, content_type='text/plain; charset=utf-8')\n\n\ndef _json_response(data):\n data = dict(sorted(data.items()))\n return HttpResponse(json.dumps(data), content_type='application/json')\n\n\ndef _parse_params(request):\n app, env_id = None, None\n api_token = request.GET.get('apiToken')\n if api_token:\n rds = get_redis_connection()\n content = rds.get(api_token)\n if content:\n app_id, env_id = content.decode().split(',')\n app = App.objects.filter(pk=app_id).first()\n else:\n api_key = AppSetting.get_default('api_key')\n if api_key and request.GET.get('apiKey') == api_key:\n app_key = request.GET.get('app')\n env_key = request.GET.get('env')\n if app_key and env_key:\n app = App.objects.filter(key=app_key).first()\n env = Environment.objects.filter(key=env_key).first()\n if env:\n env_id = env.id\n return app, env_id, request.GET.get('noPrefix')\n"
},
{
"path": "spug_api/apps/apis/deploy.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.http.response import HttpResponseBadRequest, HttpResponseForbidden, HttpResponse\nfrom apps.setting.utils import AppSetting\nfrom apps.deploy.models import Deploy, DeployRequest\nfrom apps.repository.models import Repository\nfrom apps.deploy.utils import dispatch as deploy_dispatch\nfrom libs.utils import human_datetime\nfrom threading import Thread\nimport hashlib\nimport hmac\nimport json\n\n\ndef auto_deploy(request, deploy_id, kind):\n repo, body = _parse_request(request)\n if not repo:\n return HttpResponseForbidden()\n\n try:\n _, _kind, ref = body['ref'].split('/', 2)\n if kind == 'branch' and _kind == 'heads':\n commit_id = body['after']\n if commit_id != '0000000000000000000000000000000000000000' and ref == request.GET.get('name'):\n message = _parse_message(body, repo)\n Thread(target=_dispatch, args=(deploy_id, ref, commit_id, message)).start()\n return HttpResponse(status=202)\n elif kind == 'tag' and _kind == 'tags':\n Thread(target=_dispatch, args=(deploy_id, ref)).start()\n return HttpResponse(status=202)\n return HttpResponse(status=204)\n except Exception as e:\n return HttpResponseBadRequest(e)\n\n\ndef _parse_request(request):\n api_key = AppSetting.get_default('api_key')\n token, repo, body = None, None, None\n token = request.headers.get('X-Gitlab-Token')\n if 'X-Gitlab-Token' in request.headers:\n token = request.headers['X-Gitlab-Token']\n repo = 'Gitlab'\n elif 'X-Gitee-Token' in request.headers:\n token = request.headers['X-Gitee-Token']\n repo = 'Gitee'\n elif 'X-Codeup-Token' in request.headers:\n token = request.headers['X-Codeup-Token']\n repo = 'Codeup'\n elif 'X-Gogs-Signature' in request.headers:\n token = request.headers['X-Gogs-Signature']\n repo = 'Gogs'\n elif 'X-Hub-Signature-256' in request.headers:\n token = request.headers['X-Hub-Signature-256'].replace('sha256=', '')\n repo = 'Github'\n elif 'X-Coding-Signature' in request.headers:\n token = request.headers['X-Coding-Signature'].replace('sha1=', '')\n repo = 'Coding'\n elif 'token' in request.GET: # Compatible the old version of gitlab\n token = request.GET.get('token')\n repo = 'Gitlab'\n\n if repo in ['Gitlab', 'Gitee', 'Codeup']:\n if token != api_key:\n return None, None\n elif repo in ['Github', 'Gogs']:\n en_api_key = hmac.new(api_key.encode(), request.body, hashlib.sha256).hexdigest()\n if token != en_api_key:\n return None, None\n elif repo in ['Coding']:\n en_api_key = hmac.new(api_key.encode(), request.body, hashlib.sha1).hexdigest()\n if token != en_api_key:\n return None, None\n else:\n return None, None\n\n body = json.loads(request.body)\n if repo == 'Gogs' and not body['ref'].startswith('refs/'):\n body['ref'] = 'refs/tags/' + body['ref']\n\n return repo, body\n\n\ndef _parse_message(body, repo):\n message = None\n if repo in ['Gitee', 'Github', 'Coding']:\n message = body.get('head_commit', {}).get('message', '')\n elif repo in ['Gitlab', 'Codeup', 'Gogs']:\n if body.get('commits'):\n message = body['commits'][0].get('message', '')\n else:\n raise ValueError(f'repo {repo} is not supported')\n return message[:20].strip()\n\n\ndef _dispatch(deploy_id, ref, commit_id=None, message=None):\n deploy = Deploy.objects.filter(pk=deploy_id).first()\n if not deploy:\n raise Exception(f'no such deploy id for {deploy_id}')\n\n req = DeployRequest(\n type='3',\n status='0' if deploy.is_audit else '2',\n deploy=deploy,\n spug_version=Repository.make_spug_version(deploy.id),\n host_ids=deploy.host_ids,\n created_by=deploy.created_by\n )\n\n if commit_id: # branch\n req.version = f'{ref}#{commit_id[:6]}'\n req.name = message or req.version\n if deploy.extend == '1':\n req.extra = json.dumps(['branch', ref, commit_id])\n else: # tag\n req.version = ref\n req.name = ref\n if deploy.extend == '1':\n req.extra = json.dumps(['tag', ref, None])\n\n req.save()\n if req.status == '2':\n req.do_at = human_datetime()\n req.do_by = deploy.created_by\n req.save()\n deploy_dispatch(req)\n"
},
{
"path": "spug_api/apps/apis/urls.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.urls import path\n\nfrom apps.apis import config\nfrom apps.apis import deploy\n\nurlpatterns = [\n path('config/', config.get_configs),\n path('deploy///', deploy.auto_deploy)\n]\n"
},
{
"path": "spug_api/apps/app/__init__.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\n"
},
{
"path": "spug_api/apps/app/models.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.db import models\nfrom django.conf import settings\nfrom libs import ModelMixin, human_datetime\nfrom apps.account.models import User\nfrom apps.config.models import Environment\nimport subprocess\nimport json\nimport os\n\n\nclass App(models.Model, ModelMixin):\n name = models.CharField(max_length=50)\n key = models.CharField(max_length=50, unique=True)\n desc = models.CharField(max_length=255, null=True)\n rel_apps = models.TextField(null=True)\n rel_services = models.TextField(null=True)\n sort_id = models.IntegerField(default=0, db_index=True)\n created_at = models.CharField(max_length=20, default=human_datetime)\n created_by = models.ForeignKey(User, on_delete=models.PROTECT)\n\n def to_dict(self, *args, **kwargs):\n tmp = super().to_dict(*args, **kwargs)\n tmp['rel_apps'] = json.loads(self.rel_apps) if self.rel_apps else []\n tmp['rel_services'] = json.loads(self.rel_services) if self.rel_services else []\n return tmp\n\n def __repr__(self):\n return f''\n\n class Meta:\n db_table = 'apps'\n ordering = ('-sort_id',)\n\n\nclass Deploy(models.Model, ModelMixin):\n EXTENDS = (\n ('1', '常规发布'),\n ('2', '自定义发布'),\n )\n app = models.ForeignKey(App, on_delete=models.PROTECT)\n env = models.ForeignKey(Environment, on_delete=models.PROTECT)\n host_ids = models.TextField()\n extend = models.CharField(max_length=2, choices=EXTENDS)\n is_audit = models.BooleanField()\n is_parallel = models.BooleanField(default=True)\n rst_notify = models.CharField(max_length=255, null=True)\n created_at = models.CharField(max_length=20, default=human_datetime)\n created_by = models.ForeignKey(User, models.PROTECT, related_name='+')\n updated_at = models.CharField(max_length=20, null=True)\n updated_by = models.ForeignKey(User, models.PROTECT, related_name='+', null=True)\n\n @property\n def extend_obj(self):\n cls = DeployExtend1 if self.extend == '1' else DeployExtend2\n return cls.objects.filter(deploy=self).first()\n\n def to_dict(self, *args, **kwargs):\n deploy = super().to_dict(*args, **kwargs)\n deploy['app_key'] = self.app_key if hasattr(self, 'app_key') else None\n deploy['app_name'] = self.app_name if hasattr(self, 'app_name') else None\n deploy['host_ids'] = json.loads(self.host_ids)\n deploy['rst_notify'] = json.loads(self.rst_notify)\n deploy.update(self.extend_obj.to_dict())\n return deploy\n\n def delete(self, using=None, keep_parents=False):\n deploy_id = self.id\n super().delete(using, keep_parents)\n repo_dir = os.path.join(settings.REPOS_DIR, str(deploy_id))\n build_dir = os.path.join(settings.BUILD_DIR, f'{deploy_id}_*')\n subprocess.Popen(f'rm -rf {repo_dir} {repo_dir + \"_*\"} {build_dir}', shell=True)\n\n def __repr__(self):\n return '' % (self.app_id, self.env_id)\n\n class Meta:\n db_table = 'deploys'\n ordering = ('-id',)\n\n\nclass DeployExtend1(models.Model, ModelMixin):\n deploy = models.OneToOneField(Deploy, primary_key=True, on_delete=models.CASCADE)\n git_repo = models.CharField(max_length=255)\n dst_dir = models.CharField(max_length=255)\n dst_repo = models.CharField(max_length=255)\n versions = models.IntegerField()\n filter_rule = models.TextField()\n hook_pre_server = models.TextField(null=True)\n hook_post_server = models.TextField(null=True)\n hook_pre_host = models.TextField(null=True)\n hook_post_host = models.TextField(null=True)\n\n def to_dict(self, *args, **kwargs):\n tmp = super().to_dict(*args, **kwargs)\n tmp['filter_rule'] = json.loads(self.filter_rule)\n return tmp\n\n def __repr__(self):\n return '' % self.deploy_id\n\n class Meta:\n db_table = 'deploy_extend1'\n\n\nclass DeployExtend2(models.Model, ModelMixin):\n deploy = models.OneToOneField(Deploy, primary_key=True, on_delete=models.CASCADE)\n server_actions = models.TextField()\n host_actions = models.TextField()\n require_upload = models.BooleanField(default=False)\n\n def to_dict(self, *args, **kwargs):\n tmp = super().to_dict(*args, **kwargs)\n tmp['server_actions'] = json.loads(self.server_actions)\n tmp['host_actions'] = json.loads(self.host_actions)\n return tmp\n\n def __repr__(self):\n return '' % self.deploy_id\n\n class Meta:\n db_table = 'deploy_extend2'\n"
},
{
"path": "spug_api/apps/app/urls.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.urls import path\n\nfrom .views import *\n\nurlpatterns = [\n path('', AppView.as_view()),\n path('kit/key/', kit_key),\n path('deploy/', DeployView.as_view()),\n path('deploy//versions/', get_versions),\n]\n"
},
{
"path": "spug_api/apps/app/utils.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.conf import settings\nfrom apps.app.models import Deploy\nfrom apps.setting.utils import AppSetting\nfrom libs.gitlib import Git\nimport shutil\nimport os\n\n\ndef parse_envs(text):\n data = {}\n if text:\n for line in text.split('\\n'):\n fields = line.split('=', 1)\n if len(fields) != 2 or fields[0].strip() == '':\n raise Exception(f'解析自定义全局变量{line!r}失败，确认其遵循 key = value 格式')\n data[fields[0].strip()] = fields[1].strip()\n return data\n\n\ndef fetch_versions(deploy: Deploy):\n git_repo = deploy.extend_obj.git_repo\n repo_dir = os.path.join(settings.REPOS_DIR, str(deploy.id))\n pkey = AppSetting.get_default('private_key')\n with Git(git_repo, repo_dir, pkey) as git:\n return git.fetch_branches_tags()\n\n\ndef fetch_repo(deploy_id, git_repo):\n repo_dir = os.path.join(settings.REPOS_DIR, str(deploy_id))\n pkey = AppSetting.get_default('private_key')\n with Git(git_repo, repo_dir, pkey) as git:\n return git.fetch_branches_tags()\n\n\ndef remove_repo(deploy_id):\n shutil.rmtree(os.path.join(settings.REPOS_DIR, str(deploy_id)), True)\n"
},
{
"path": "spug_api/apps/app/views.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.views.generic import View\nfrom django.db.models import F\nfrom libs import JsonParser, Argument, json_response, auth\nfrom apps.app.models import App, Deploy, DeployExtend1, DeployExtend2\nfrom apps.config.models import Config, ConfigHistory, Service\nfrom apps.app.utils import fetch_versions, remove_repo\nfrom apps.setting.utils import AppSetting\nimport json\nimport re\n\n\nclass AppView(View):\n def get(self, request):\n form, error = JsonParser(\n Argument('id', type=int, required=False)\n ).parse(request.GET)\n if error is None:\n if request.user.is_supper:\n apps = App.objects.all()\n else:\n ids = request.user.deploy_perms['apps']\n apps = App.objects.filter(id__in=ids)\n\n if form.id:\n app = apps.filter(pk=form.id).first()\n return json_response(app)\n return json_response(apps)\n return json_response(error=error)\n\n @auth('deploy.app.add|deploy.app.edit|config.app.add|config.app.edit')\n def post(self, request):\n form, error = JsonParser(\n Argument('id', type=int, required=False),\n Argument('name', help='请输入服务名称'),\n Argument('key', help='请输入唯一标识符'),\n Argument('desc', required=False)\n ).parse(request.body)\n if error is None:\n if not re.fullmatch(r'\\w+', form.key, re.ASCII):\n return json_response(error='标识符必须为字母、数字和下划线的组合')\n\n app = App.objects.filter(key=form.key).first()\n if app and app.id != form.id:\n return json_response(error='该识符已存在，请更改后重试')\n service = Service.objects.filter(key=form.key).first()\n if service:\n return json_response(error=f'该标识符已被服务 {service.name} 使用，请更改后重试')\n if form.id:\n App.objects.filter(pk=form.id).update(**form)\n else:\n app = App.objects.create(created_by=request.user, **form)\n app.sort_id = app.id\n app.save()\n return json_response(error=error)\n\n @auth('deploy.app.edit|config.app.edit_config')\n def patch(self, request):\n form, error = JsonParser(\n Argument('id', type=int, help='参数错误'),\n Argument('rel_apps', type=list, required=False),\n Argument('rel_services', type=list, required=False),\n Argument('sort', filter=lambda x: x in ('up', 'down'), required=False)\n ).parse(request.body)\n if error is None:\n app = App.objects.filter(pk=form.id).first()\n if not app:\n return json_response(error='未找到指定应用')\n if form.rel_apps is not None:\n app.rel_apps = json.dumps(form.rel_apps)\n if form.rel_services is not None:\n app.rel_services = json.dumps(form.rel_services)\n if form.sort:\n if form.sort == 'up':\n tmp = App.objects.filter(sort_id__gt=app.sort_id).last()\n else:\n tmp = App.objects.filter(sort_id__lt=app.sort_id).first()\n if tmp:\n tmp.sort_id, app.sort_id = app.sort_id, tmp.sort_id\n tmp.save()\n app.save()\n return json_response(error=error)\n\n @auth('deploy.app.del|config.app.del')\n def delete(self, request):\n form, error = JsonParser(\n Argument('id', type=int, help='请指定操作对象')\n ).parse(request.GET)\n if error is None:\n if Deploy.objects.filter(app_id=form.id).exists():\n return json_response(error='该应用在应用发布中已存在关联的发布配置，请删除相关发布配置后再尝试删除')\n # auto delete configs\n Config.objects.filter(type='app', o_id=form.id).delete()\n ConfigHistory.objects.filter(type='app', o_id=form.id).delete()\n for app in App.objects.filter(rel_apps__isnull=False):\n rel_apps = json.loads(app.rel_apps)\n if form.id in rel_apps:\n rel_apps.remove(form.id)\n app.rel_apps = json.dumps(rel_apps)\n app.save()\n App.objects.filter(pk=form.id).delete()\n return json_response(error=error)\n\n\nclass DeployView(View):\n @auth('deploy.app.view|deploy.request.view')\n def get(self, request):\n form, error = JsonParser(\n Argument('app_id', type=int, required=False)\n ).parse(request.GET, True)\n if not request.user.is_supper:\n perms = request.user.deploy_perms\n form.app_id__in = perms['apps']\n form.env_id__in = perms['envs']\n deploys = Deploy.objects.filter(**form) \\\n .annotate(app_name=F('app__name'), app_key=F('app__key')) \\\n .order_by('-app__sort_id')\n return json_response(deploys)\n\n @auth('deploy.app.edit')\n def post(self, request):\n form, error = JsonParser(\n Argument('id', type=int, required=False),\n Argument('app_id', type=int, help='请选择应用'),\n Argument('env_id', type=int, help='请选择环境'),\n Argument('host_ids', type=list, filter=lambda x: len(x), help='请选择要部署的主机'),\n Argument('rst_notify', type=dict, help='请选择发布结果通知方式'),\n Argument('extend', filter=lambda x: x in dict(Deploy.EXTENDS), help='请选择发布类型'),\n Argument('is_parallel', type=bool, default=True),\n Argument('is_audit', type=bool, default=False)\n ).parse(request.body)\n if error is None:\n deploy = Deploy.objects.filter(app_id=form.app_id, env_id=form.env_id).first()\n if deploy and deploy.id != form.id:\n return json_response(error='应用在该环境下已经存在发布配置')\n form.host_ids = json.dumps(form.host_ids)\n form.rst_notify = json.dumps(form.rst_notify)\n if form.extend == '1':\n extend_form, error = JsonParser(\n Argument('git_repo', handler=str.strip, help='请输入git仓库地址'),\n Argument('dst_dir', handler=str.strip, help='请输入发布部署路径'),\n Argument('dst_repo', handler=str.strip, help='请输入发布存储路径'),\n Argument('versions', type=int, filter=lambda x: x > 0, help='请输入发布保留版本数量'),\n Argument('filter_rule', type=dict, help='参数错误'),\n Argument('hook_pre_server', handler=str.strip, default=''),\n Argument('hook_post_server', handler=str.strip, default=''),\n Argument('hook_pre_host', handler=str.strip, default=''),\n Argument('hook_post_host', handler=str.strip, default='')\n ).parse(request.body)\n if error:\n return json_response(error=error)\n extend_form.dst_dir = extend_form.dst_dir.rstrip('/')\n extend_form.filter_rule = json.dumps(extend_form.filter_rule)\n if form.id:\n extend = DeployExtend1.objects.filter(deploy_id=form.id).first()\n if extend.git_repo != extend_form.git_repo:\n remove_repo(form.id)\n Deploy.objects.filter(pk=form.id).update(**form)\n DeployExtend1.objects.filter(deploy_id=form.id).update(**extend_form)\n else:\n deploy = Deploy.objects.create(created_by=request.user, **form)\n DeployExtend1.objects.create(deploy=deploy, **extend_form)\n elif form.extend == '2':\n extend_form, error = JsonParser(\n Argument('server_actions', type=list, help='请输入执行动作'),\n Argument('host_actions', type=list, help='请输入执行动作')\n ).parse(request.body)\n if error:\n return json_response(error=error)\n if len(extend_form.server_actions) + len(extend_form.host_actions) == 0:\n return json_response(error='请至少设置一个执行的动作')\n extend_form.require_upload = any(x.get('src_mode') == '1' for x in extend_form.host_actions)\n extend_form.server_actions = json.dumps(extend_form.server_actions)\n extend_form.host_actions = json.dumps(extend_form.host_actions)\n if form.id:\n Deploy.objects.filter(pk=form.id).update(**form)\n DeployExtend2.objects.filter(deploy_id=form.id).update(**extend_form)\n else:\n deploy = Deploy.objects.create(created_by=request.user, **form)\n DeployExtend2.objects.create(deploy=deploy, **extend_form)\n return json_response(error=error)\n\n @auth('deploy.app.del')\n def delete(self, request):\n form, error = JsonParser(\n Argument('id', type=int, help='请指定操作对象')\n ).parse(request.GET)\n if error is None:\n deploy = Deploy.objects.get(pk=form.id)\n if deploy.deployrequest_set.exists():\n return json_response(error='已存在关联的发布记录，请删除关联的发布记录后再尝试删除发布配置')\n for item in deploy.repository_set.all():\n item.delete()\n deploy.delete()\n return json_response(error=error)\n\n\n@auth('deploy.app.config|deploy.repository.add|deploy.request.add|deploy.request.edit')\ndef get_versions(request, d_id):\n deploy = Deploy.objects.filter(pk=d_id).first()\n if not deploy:\n return json_response(error='未找到指定应用')\n if deploy.extend == '2':\n return json_response(error='该应用不支持此操作')\n branches, tags = fetch_versions(deploy)\n return json_response({'branches': branches, 'tags': tags})\n\n\n@auth('deploy.app.config|deploy.app.edit')\ndef kit_key(request):\n form, error = JsonParser(\n Argument('key', filter=lambda x: x in ('api_key', 'public_key'), help='参数错误')\n ).parse(request.body)\n if error is None:\n api_key = AppSetting.get_default(form.key)\n return json_response(api_key)\n return json_response(error=error)\n"
},
{
"path": "spug_api/apps/config/__init__.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\n"
},
{
"path": "spug_api/apps/config/models.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.db import models\nfrom libs import ModelMixin, human_datetime\nfrom apps.account.models import User\n\n\nclass Environment(models.Model, ModelMixin):\n name = models.CharField(max_length=50)\n key = models.CharField(max_length=50)\n desc = models.CharField(max_length=255, null=True)\n sort_id = models.IntegerField(default=0, db_index=True)\n created_at = models.CharField(max_length=20, default=human_datetime)\n created_by = models.ForeignKey(User, on_delete=models.PROTECT)\n\n def __repr__(self):\n return f''\n\n class Meta:\n db_table = 'environments'\n ordering = ('-sort_id',)\n\n\nclass Service(models.Model, ModelMixin):\n name = models.CharField(max_length=50)\n key = models.CharField(max_length=50, unique=True)\n desc = models.CharField(max_length=255, null=True)\n created_at = models.CharField(max_length=20, default=human_datetime)\n created_by = models.ForeignKey(User, on_delete=models.PROTECT)\n\n def __repr__(self):\n return f''\n\n class Meta:\n db_table = 'services'\n ordering = ('-id',)\n\n\nclass Config(models.Model, ModelMixin):\n TYPES = (\n ('app', 'App'),\n ('src', 'Service')\n )\n type = models.CharField(max_length=5, choices=TYPES)\n o_id = models.IntegerField()\n key = models.CharField(max_length=50)\n env = models.ForeignKey(Environment, on_delete=models.PROTECT)\n value = models.TextField(null=True)\n desc = models.CharField(max_length=255, null=True)\n is_public = models.BooleanField(default=False)\n updated_at = models.CharField(max_length=20)\n updated_by = models.ForeignKey(User, on_delete=models.PROTECT)\n\n def __repr__(self):\n return f''\n\n class Meta:\n db_table = 'configs'\n ordering = ('-key',)\n\n\nclass ConfigHistory(models.Model, ModelMixin):\n ACTIONS = (\n ('1', '新增'),\n ('2', '更新'),\n ('3', '删除')\n )\n type = models.CharField(max_length=5)\n o_id = models.IntegerField()\n key = models.CharField(max_length=50)\n env_id = models.IntegerField()\n value = models.TextField(null=True)\n desc = models.CharField(max_length=255, null=True)\n is_public = models.BooleanField()\n old_value = models.TextField(null=True)\n action = models.CharField(max_length=2, choices=ACTIONS)\n updated_at = models.CharField(max_length=20)\n updated_by = models.ForeignKey(User, on_delete=models.PROTECT)\n\n def __repr__(self):\n return f''\n\n class Meta:\n db_table = 'config_histories'\n ordering = ('key',)\n"
},
{
"path": "spug_api/apps/config/urls.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.urls import path\n\nfrom .views import *\n\nurlpatterns = [\n path('', ConfigView.as_view()),\n path('parse/json/', parse_json),\n path('parse/text/', parse_text),\n path('diff/', post_diff),\n path('environment/', EnvironmentView.as_view()),\n path('service/', ServiceView.as_view()),\n path('history/', HistoryView.as_view()),\n]\n"
},
{
"path": "spug_api/apps/config/utils.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom apps.config.models import Config, Service\nfrom apps.app.models import App\nimport json\n\n\ndef compose_configs(app, env_id, no_prefix=False):\n configs = dict()\n # app own configs\n for item in Config.objects.filter(type='app', o_id=app.id, env_id=env_id).only('key', 'value'):\n key = item.key if no_prefix else f'{app.key}_{item.key}'\n configs[key] = item.value\n\n # relation app public configs\n if app.rel_apps:\n app_ids = json.loads(app.rel_apps)\n if app_ids:\n id_key_map = {x.id: x.key for x in App.objects.filter(id__in=app_ids)}\n for item in Config.objects.filter(type='app', o_id__in=app_ids, env_id=env_id, is_public=True) \\\n .only('key', 'value'):\n key = item.key if no_prefix else f'{id_key_map[item.o_id]}_{item.key}'\n configs[key] = item.value\n\n # relation service configs\n if app.rel_services:\n src_ids = json.loads(app.rel_services)\n if src_ids:\n id_key_map = {x.id: x.key for x in Service.objects.filter(id__in=src_ids)}\n for item in Config.objects.filter(type='src', o_id__in=src_ids, env_id=env_id).only('key', 'value'):\n key = item.key if no_prefix else f'{id_key_map[item.o_id]}_{item.key}'\n configs[key] = item.value\n return configs\n"
},
{
"path": "spug_api/apps/config/views.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.views.generic import View\nfrom django.db.models import F\nfrom libs import json_response, JsonParser, Argument, auth\nfrom apps.app.models import Deploy, App\nfrom apps.repository.models import Repository\nfrom apps.config.models import *\nimport json\nimport re\n\n\nclass EnvironmentView(View):\n def get(self, request):\n query = {}\n if not request.user.is_supper:\n query['id__in'] = request.user.deploy_perms['envs']\n envs = Environment.objects.filter(**query)\n return json_response(envs)\n\n @auth('config.env.add|config.env.edit')\n def post(self, request):\n form, error = JsonParser(\n Argument('id', type=int, required=False),\n Argument('name', help='请输入环境名称'),\n Argument('key', help='请输入唯一标识符'),\n Argument('desc', required=False)\n ).parse(request.body)\n if error is None:\n if not re.fullmatch(r'\\w+', form.key, re.ASCII):\n return json_response(error='标识符必须为字母、数字和下划线的组合')\n\n env = Environment.objects.filter(key=form.key).first()\n if env and env.id != form.id:\n return json_response(error=f'唯一标识符 {form.key} 已存在，请更改后重试')\n if form.id:\n Environment.objects.filter(pk=form.id).update(**form)\n else:\n env = Environment.objects.create(created_by=request.user, **form)\n env.sort_id = env.id\n env.save()\n return json_response(error=error)\n\n @auth('config.env.edit')\n def patch(self, request):\n form, error = JsonParser(\n Argument('id', type=int, help='参数错误'),\n Argument('sort', filter=lambda x: x in ('up', 'down'), required=False)\n ).parse(request.body)\n if error is None:\n env = Environment.objects.filter(pk=form.id).first()\n if not env:\n return json_response(error='未找到指定环境')\n if form.sort:\n if form.sort == 'up':\n tmp = Environment.objects.filter(sort_id__gt=env.sort_id).last()\n else:\n tmp = Environment.objects.filter(sort_id__lt=env.sort_id).first()\n if tmp:\n tmp.sort_id, env.sort_id = env.sort_id, tmp.sort_id\n tmp.save()\n env.save()\n return json_response(error=error)\n\n @auth('config.env.del')\n def delete(self, request):\n form, error = JsonParser(\n Argument('id', type=int, help='请指定操作对象')\n ).parse(request.GET)\n if error is None:\n if Deploy.objects.filter(env_id=form.id).exists():\n return json_response(error='该环境已关联了发布配置，请删除相关发布配置后再尝试删除')\n if Repository.objects.filter(env_id=form.id).exists():\n return json_response(error='该环境关联了构建记录，请在删除应用发布/构建仓库中相关记录后再尝试')\n # auto delete configs\n Config.objects.filter(env_id=form.id).delete()\n ConfigHistory.objects.filter(env_id=form.id).delete()\n Environment.objects.filter(pk=form.id).delete()\n return json_response(error=error)\n\n\nclass ServiceView(View):\n @auth('config.src.view')\n def get(self, request):\n form, error = JsonParser(\n Argument('id', type=int, required=False)\n ).parse(request.GET)\n if error is None:\n if form.id:\n service = Service.objects.get(pk=form.id)\n return json_response(service)\n services = Service.objects.all()\n return json_response(services)\n return json_response(error=error)\n\n @auth('config.src.add|config.src.edit')\n def post(self, request):\n form, error = JsonParser(\n Argument('id', type=int, required=False),\n Argument('name', help='请输入服务名称'),\n Argument('key', help='请输入唯一标识符'),\n Argument('desc', required=False)\n ).parse(request.body)\n if error is None:\n if not re.fullmatch(r'\\w+', form.key, re.ASCII):\n return json_response(error='标识符必须为字母、数字和下划线的组合')\n\n service = Service.objects.filter(key=form.key).first()\n if service and service.id != form.id:\n return json_response(error='该标识符已存在，请更改后重试')\n app = App.objects.filter(key=form.key).first()\n if app:\n return json_response(error=f'该标识符已被应用 {app.name} 使用，请更改后重试')\n if form.id:\n Service.objects.filter(pk=form.id).update(**form)\n else:\n Service.objects.create(created_by=request.user, **form)\n return json_response(error=error)\n\n @auth('config.src.del')\n def delete(self, request):\n form, error = JsonParser(\n Argument('id', type=int, help='请指定操作对象')\n ).parse(request.GET)\n if error is None:\n rel_apps = []\n for app in App.objects.filter(rel_services__isnull=False):\n rel_services = json.loads(app.rel_services)\n if form.id in rel_services:\n rel_apps.append(app.name)\n if rel_apps:\n return json_response(\n error=f'该服务在配置中心已被 \"{\", \".join(rel_apps)}\" 依赖，请解除依赖关系后再尝试删除。')\n # auto delete configs\n Config.objects.filter(type='src', o_id=form.id).delete()\n ConfigHistory.objects.filter(type='src', o_id=form.id).delete()\n Service.objects.filter(pk=form.id).delete()\n return json_response(error=error)\n\n\nclass ConfigView(View):\n @auth('config.src.view_config|config.app.view_config')\n def get(self, request):\n form, error = JsonParser(\n Argument('id', type=int, help='未指定操作对象'),\n Argument('type', filter=lambda x: x in dict(Config.TYPES), help='缺少必要参数'),\n Argument('env_id', type=int, help='缺少必要参数'),\n ).parse(request.GET)\n if error is None:\n form.o_id, data = form.pop('id'), []\n for item in Config.objects.filter(**form).annotate(update_user=F('updated_by__nickname')):\n tmp = item.to_dict()\n tmp['update_user'] = item.update_user\n data.append(tmp)\n return json_response(data)\n return json_response(error=error)\n\n @auth('config.src.edit_config|config.app.edit_config')\n def post(self, request):\n form, error = JsonParser(\n Argument('o_id', type=int, help='缺少必要参数'),\n Argument('type', filter=lambda x: x in dict(Config.TYPES), help='缺少必要参数'),\n Argument('envs', type=list, filter=lambda x: len(x), help='请选择环境'),\n Argument('key', help='请输入Key'),\n Argument('is_public', type=bool, help='缺少必要参数'),\n Argument('value', type=str, default=''),\n Argument('desc', required=False)\n ).parse(request.body)\n if error is None:\n form.value = form.value.strip()\n form.updated_at = human_datetime()\n form.updated_by = request.user\n envs = form.pop('envs')\n for env_id in envs:\n cf = Config.objects.filter(o_id=form.o_id, type=form.type, env_id=env_id, key=form.key).first()\n if cf:\n raise Exception(f'{cf.env.name} 中已存在该Key')\n Config.objects.create(env_id=env_id, **form)\n ConfigHistory.objects.create(action='1', env_id=env_id, **form)\n return json_response(error=error)\n\n @auth('config.src.edit_config|config.app.edit_config')\n def patch(self, request):\n form, error = JsonParser(\n Argument('id', type=int, help='缺少必要参数'),\n Argument('value', type=str, default=''),\n Argument('is_public', type=bool, help='缺少必要参数'),\n Argument('desc', required=False)\n ).parse(request.body)\n if error is None:\n form.value = form.value.strip()\n config = Config.objects.filter(pk=form.id).first()\n if not config:\n return json_response(error='未找到指定对象')\n config.desc = form.desc\n config.is_public = form.is_public\n if config.value != form.value:\n old_value = config.value\n config.value = form.value\n config.updated_at = human_datetime()\n config.updated_by = request.user\n ConfigHistory.objects.create(\n action='2',\n old_value=old_value,\n **config.to_dict(excludes=('id',)))\n config.save()\n return json_response(error=error)\n\n @auth('config.src.edit_config|config.app.edit_config')\n def delete(self, request):\n form, error = JsonParser(\n Argument('id', type=int, help='未指定操作对象')\n ).parse(request.GET)\n if error is None:\n config = Config.objects.filter(pk=form.id).first()\n if config:\n ConfigHistory.objects.create(\n action='3',\n old_value=config.value,\n value='',\n updated_at=human_datetime(),\n updated_by=request.user,\n **config.to_dict(excludes=('id', 'value', 'updated_at', 'updated_by_id'))\n )\n config.delete()\n return json_response(error=error)\n\n\nclass HistoryView(View):\n @auth('config.src.view_config|config.app.view_config')\n def post(self, request):\n form, error = JsonParser(\n Argument('o_id', type=int, help='缺少必要参数'),\n Argument('env_id', type=int, help='缺少必要参数'),\n Argument('type', filter=lambda x: x in dict(Config.TYPES), help='缺少必要参数')\n ).parse(request.body)\n if error is None:\n data = []\n for item in ConfigHistory.objects.filter(**form).annotate(update_user=F('updated_by__nickname')):\n tmp = item.to_dict()\n tmp['action_alias'] = item.get_action_display()\n tmp['update_user'] = item.update_user\n data.append(tmp)\n return json_response(data)\n return json_response(error=error)\n\n\n@auth('config.src.view_config|config.app.view_config')\ndef post_diff(request):\n form, error = JsonParser(\n Argument('o_id', type=int, help='缺少必要参数'),\n Argument('type', filter=lambda x: x in dict(Config.TYPES), help='缺少必要参数'),\n Argument('envs', type=list, filter=lambda x: len(x), help='缺少必要参数'),\n ).parse(request.body)\n if error is None:\n data, form.env_id__in = {}, form.pop('envs')\n for item in Config.objects.filter(**form).order_by('key'):\n if item.key in data:\n data[item.key][item.env_id] = item.value\n else:\n data[item.key] = {'key': item.key, item.env_id: item.value}\n return json_response(list(data.values()))\n return json_response(error=error)\n\n\n@auth('config.src.edit_config|config.app.edit_config')\ndef parse_json(request):\n form, error = JsonParser(\n Argument('o_id', type=int, help='缺少必要参数'),\n Argument('type', filter=lambda x: x in dict(Config.TYPES), help='缺少必要参数'),\n Argument('env_id', type=int, help='缺少必要参数'),\n Argument('data', type=dict, help='缺少必要参数')\n ).parse(request.body)\n if error is None:\n data = form.pop('data')\n _parse(request, form, data)\n return json_response(error=error)\n\n\n@auth('config.src.edit_config|config.app.edit_config')\ndef parse_text(request):\n form, error = JsonParser(\n Argument('o_id', type=int, help='缺少必要参数'),\n Argument('type', filter=lambda x: x in dict(Config.TYPES), help='缺少必要参数'),\n Argument('env_id', type=int, help='缺少必要参数'),\n Argument('data', handler=str.strip, help='缺少必要参数')\n ).parse(request.body)\n if error is None:\n data = {}\n for line in form.pop('data').split('\\n'):\n line = line.strip()\n if not line or line[0] in ('#', ';'):\n continue\n fields = line.split('=', 1)\n if len(fields) != 2 or fields[0].strip() == '':\n return json_response(error=f'解析配置{line!r}失败，确认其遵循 key = value 格式')\n data[fields[0].strip()] = fields[1].strip()\n _parse(request, form, data)\n return json_response(error=error)\n\n\ndef _parse(request, query, data):\n for item in Config.objects.filter(**query):\n if item.key in data:\n value = _filter_value(data.pop(item.key))\n if item.value != value:\n old_value = item.value\n item.value = value\n item.updated_at = human_datetime()\n item.updated_by = request.user\n item.save()\n ConfigHistory.objects.create(\n action='2',\n old_value=old_value,\n **item.to_dict(excludes=('id',)))\n else:\n ConfigHistory.objects.create(\n action='3',\n old_value=item.value,\n value='',\n updated_at=human_datetime(),\n updated_by=request.user,\n **item.to_dict(excludes=('id', 'value', 'updated_at', 'updated_by_id'))\n )\n item.delete()\n for key, value in data.items():\n query.key = key\n query.is_public = False\n query.value = _filter_value(value)\n query.updated_at = human_datetime()\n query.updated_by = request.user\n Config.objects.create(**query)\n ConfigHistory.objects.create(action='1', **query)\n\n\ndef _filter_value(value):\n if isinstance(value, (str, int)):\n value = str(value).strip()\n else:\n value = json.dumps(value)\n return value\n"
},
{
"path": "spug_api/apps/deploy/__init__.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\n"
},
{
"path": "spug_api/apps/deploy/helper.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.template.defaultfilters import filesizeformat\nfrom libs.utils import human_datetime, render_str, str_decode\nfrom libs.spug import Notification\nfrom apps.host.models import Host\nfrom functools import partial\nimport subprocess\nimport json\nimport os\n\n\nclass SpugError(Exception):\n pass\n\n\nclass Helper:\n def __init__(self, rds, key):\n self.rds = rds\n self.key = key\n self.callback = []\n\n @classmethod\n def make(cls, rds, key, host_ids=None):\n if host_ids:\n counter, tmp_key = 0, f'{key}_tmp'\n data = rds.lrange(key, counter, counter + 9)\n while data:\n for item in data:\n counter += 1\n print(item)\n tmp = json.loads(item.decode())\n if tmp['key'] not in host_ids:\n rds.rpush(tmp_key, item)\n data = rds.lrange(key, counter, counter + 9)\n rds.delete(key)\n if rds.exists(tmp_key):\n rds.rename(tmp_key, key)\n else:\n rds.delete(key)\n return cls(rds, key)\n\n @classmethod\n def _make_dd_notify(cls, url, action, req, version, host_str):\n texts = [\n f'**申请标题：** {req.name}',\n f'**应用名称：** {req.deploy.app.name}',\n f'**应用版本：** {version}',\n f'**发布环境：** {req.deploy.env.name}',\n f'**发布主机：** {host_str}',\n ]\n if action == 'approve_req':\n texts.insert(0, '## %s ## ' % '发布审核申请')\n texts.extend([\n f'**申请人员：** {req.created_by.nickname}',\n f'**申请时间：** {human_datetime()}',\n '> 来自 Spug运维平台'\n ])\n elif action == 'approve_rst':\n color, text = ('#008000', '通过') if req.status == '1' else ('#f90202', '驳回')\n texts.insert(0, '## %s ## ' % '发布审核结果')\n texts.extend([\n f'**审核人员：** {req.approve_by.nickname}',\n f'**审核结果：** {text}',\n f'**审核意见：** {req.reason or \"\"}',\n f'**审核时间：** {human_datetime()}',\n '> 来自 Spug运维平台'\n ])\n else:\n color, text = ('#008000', '成功') if req.status == '3' else ('#f90202', '失败')\n texts.insert(0, '## %s ## ' % '发布结果通知')\n if req.approve_at:\n texts.append(f'**审核人员：** {req.approve_by.nickname}')\n do_user = req.do_by.nickname if req.type != '3' else 'Webhook'\n texts.extend([\n f'**执行人员：** {do_user}',\n f'**发布结果：** {text}',\n f'**发布时间：** {human_datetime()}',\n '> 来自 Spug运维平台'\n ])\n data = {\n 'msgtype': 'markdown',\n 'markdown': {\n 'title': 'Spug 发布消息通知',\n 'text': '\\n\\n'.join(texts)\n },\n 'at': {\n 'isAtAll': True\n }\n }\n Notification.handle_request(url, data, 'dd')\n\n @classmethod\n def _make_wx_notify(cls, url, action, req, version, host_str):\n texts = [\n f'申请标题： {req.name}',\n f'应用名称： {req.deploy.app.name}',\n f'应用版本： {version}',\n f'发布环境： {req.deploy.env.name}',\n f'发布主机： {host_str}',\n ]\n\n if action == 'approve_req':\n texts.insert(0, '## %s' % '发布审核申请')\n texts.extend([\n f'申请人员： {req.created_by.nickname}',\n f'申请时间： {human_datetime()}',\n '> 来自 Spug运维平台'\n ])\n elif action == 'approve_rst':\n color, text = ('info', '通过') if req.status == '1' else ('warning', '驳回')\n texts.insert(0, '## %s' % '发布审核结果')\n texts.extend([\n f'审核人员： {req.approve_by.nickname}',\n f'审核结果： {text}',\n f'审核意见： {req.reason or \"\"}',\n f'审核时间： {human_datetime()}',\n '> 来自 Spug运维平台'\n ])\n else:\n color, text = ('info', '成功') if req.status == '3' else ('warning', '失败')\n texts.insert(0, '## %s' % '发布结果通知')\n if req.approve_at:\n texts.append(f'审核人员： {req.approve_by.nickname}')\n do_user = req.do_by.nickname if req.type != '3' else 'Webhook'\n texts.extend([\n f'执行人员： {do_user}',\n f'发布结果： {text}',\n f'发布时间： {human_datetime()}',\n '> 来自 Spug运维平台'\n ])\n data = {\n 'msgtype': 'markdown',\n 'markdown': {\n 'content': '\\n'.join(texts)\n }\n }\n Notification.handle_request(url, data, 'wx')\n\n @classmethod\n def _make_fs_notify(cls, url, action, req, version, host_str):\n texts = [\n f'申请标题： {req.name}',\n f'应用名称： {req.deploy.app.name}',\n f'应用版本： {version}',\n f'发布环境： {req.deploy.env.name}',\n f'发布主机： {host_str}',\n ]\n\n if action == 'approve_req':\n title = '发布审核申请'\n texts.extend([\n f'申请人员： {req.created_by.nickname}',\n f'申请时间： {human_datetime()}',\n ])\n elif action == 'approve_rst':\n title = '发布审核结果'\n text = '通过' if req.status == '1' else '驳回'\n texts.extend([\n f'审核人员： {req.approve_by.nickname}',\n f'审核结果： {text}',\n f'审核意见： {req.reason or \"\"}',\n f'审核时间： {human_datetime()}',\n ])\n else:\n title = '发布结果通知'\n text = '成功 ✅' if req.status == '3' else '失败 ❗'\n if req.approve_at:\n texts.append(f'审核人员： {req.approve_by.nickname}')\n do_user = req.do_by.nickname if req.type != '3' else 'Webhook'\n texts.extend([\n f'执行人员： {do_user}',\n f'发布结果： {text}',\n f'发布时间： {human_datetime()}',\n ])\n data = {\n 'msg_type': 'post',\n 'content': {\n 'post': {\n 'zh_cn': {\n 'title': title,\n 'content': [[{'tag': 'text', 'text': x}] for x in texts] + [[{'tag': 'at', 'user_id': 'all'}]]\n }\n }\n }\n }\n Notification.handle_request(url, data, 'fs')\n\n @classmethod\n def send_deploy_notify(cls, req, action=None):\n rst_notify = json.loads(req.deploy.rst_notify)\n host_ids = json.loads(req.host_ids) if isinstance(req.host_ids, str) else req.host_ids\n if rst_notify['mode'] != '0' and rst_notify.get('value'):\n url = rst_notify['value']\n version = req.version\n hosts = [{'id': x.id, 'name': x.name} for x in Host.objects.filter(id__in=host_ids)]\n host_str = ', '.join(x['name'] for x in hosts[:2])\n if len(hosts) > 2:\n host_str += f'等{len(hosts)}台主机'\n if rst_notify['mode'] == '1':\n cls._make_dd_notify(url, action, req, version, host_str)\n elif rst_notify['mode'] == '2':\n data = {\n 'action': action,\n 'req_id': req.id,\n 'req_name': req.name,\n 'app_id': req.deploy.app_id,\n 'app_name': req.deploy.app.name,\n 'env_id': req.deploy.env_id,\n 'env_name': req.deploy.env.name,\n 'status': req.status,\n 'reason': req.reason,\n 'version': version,\n 'targets': hosts,\n 'is_success': req.status == '3',\n 'created_at': human_datetime()\n }\n Notification.handle_request(url, data)\n elif rst_notify['mode'] == '3':\n cls._make_wx_notify(url, action, req, version, host_str)\n elif rst_notify['mode'] == '4':\n cls._make_fs_notify(url, action, req, version, host_str)\n else:\n raise NotImplementedError\n\n def add_callback(self, func):\n self.callback.append(func)\n\n def parse_filter_rule(self, data: str, sep='\\n', env=None):\n data, files = data.strip(), []\n if data:\n for line in data.split(sep):\n line = line.strip()\n if line and not line.startswith('#'):\n files.append(render_str(line, env))\n return files\n\n def _send(self, message):\n self.rds.rpush(self.key, json.dumps(message))\n\n def send_info(self, key, message):\n if message:\n self._send({'key': key, 'data': message})\n\n def send_error(self, key, message, with_break=True):\n message = f'\\r\\n\\033[31m{message}\\033[0m'\n self._send({'key': key, 'status': 'error', 'data': message})\n if with_break:\n raise SpugError\n\n def send_step(self, key, step, data):\n self._send({'key': key, 'step': step, 'data': data})\n\n def clear(self):\n self.rds.delete(f'{self.key}_tmp')\n # save logs for two weeks\n self.rds.expire(self.key, 14 * 24 * 60 * 60)\n self.rds.close()\n # callback\n for func in self.callback:\n func()\n\n def progress_callback(self, key):\n def func(k, n, t):\n message = f'\\r {filesizeformat(n):<8}/{filesizeformat(t):>8} '\n self.send_info(k, message)\n\n self.send_info(key, '\\r\\n')\n return partial(func, key)\n\n def local(self, command, env=None):\n if env:\n env = dict(env.items())\n env.update(os.environ)\n task = subprocess.Popen(command, env=env, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)\n message = b''\n while True:\n output = task.stdout.read(1)\n if not output:\n break\n if output in (b'\\r', b'\\n'):\n message += b'\\r\\n' if output == b'\\n' else b'\\r'\n message = str_decode(message)\n self.send_info('local', message)\n message = b''\n else:\n message += output\n if task.wait() != 0:\n self.send_error('local', f'exit code: {task.returncode}')\n\n def remote(self, key, ssh, command, env=None):\n code = -1\n for code, out in ssh.exec_command_with_stream(command, environment=env):\n self.send_info(key, out)\n if code != 0:\n self.send_error(key, f'exit code: {code}')\n\n def remote_raw(self, key, ssh, command):\n code, out = ssh.exec_command_raw(command)\n if code != 0:\n self.send_error(key, f'exit code: {code}, {out}')\n"
},
{
"path": "spug_api/apps/deploy/models.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.db import models\nfrom django.conf import settings\nfrom libs import ModelMixin, human_datetime\nfrom apps.account.models import User\nfrom apps.app.models import Deploy\nfrom apps.repository.models import Repository\nimport json\nimport os\n\n\nclass DeployRequest(models.Model, ModelMixin):\n STATUS = (\n ('-3', '发布异常'),\n ('-1', '已驳回'),\n ('0', '待审核'),\n ('1', '待发布'),\n ('2', '发布中'),\n ('3', '发布成功'),\n )\n TYPES = (\n ('1', '正常发布'),\n ('2', '回滚'),\n ('3', '自动发布'),\n )\n deploy = models.ForeignKey(Deploy, on_delete=models.CASCADE)\n repository = models.ForeignKey(Repository, null=True, on_delete=models.SET_NULL)\n name = models.CharField(max_length=100)\n type = models.CharField(max_length=2, choices=TYPES, default='1')\n extra = models.TextField()\n host_ids = models.TextField()\n desc = models.CharField(max_length=255, null=True)\n status = models.CharField(max_length=2, choices=STATUS)\n reason = models.CharField(max_length=255, null=True)\n version = models.CharField(max_length=100, null=True)\n spug_version = models.CharField(max_length=50, null=True)\n plan = models.DateTimeField(null=True)\n fail_host_ids = models.TextField(default='[]')\n\n created_at = models.CharField(max_length=20, default=human_datetime)\n created_by = models.ForeignKey(User, models.PROTECT, related_name='+')\n approve_at = models.CharField(max_length=20, null=True)\n approve_by = models.ForeignKey(User, models.PROTECT, related_name='+', null=True)\n do_at = models.CharField(max_length=20, null=True)\n do_by = models.ForeignKey(User, models.PROTECT, related_name='+', null=True)\n\n @property\n def is_quick_deploy(self):\n if self.type in ('1', '3') and self.deploy.extend == '1' and self.extra:\n extra = json.loads(self.extra)\n return extra[0] in ('branch', 'tag')\n return False\n\n def delete(self, using=None, keep_parents=False):\n super().delete(using, keep_parents)\n if self.repository_id:\n if not DeployRequest.objects.filter(repository=self.repository).exists():\n self.repository.delete()\n if self.deploy.extend == '2':\n try:\n os.remove(os.path.join(settings.REPOS_DIR, str(self.deploy_id), self.spug_version))\n except FileNotFoundError:\n pass\n\n def __repr__(self):\n return f''\n\n class Meta:\n db_table = 'deploy_requests'\n ordering = ('-id',)\n"
},
{
"path": "spug_api/apps/deploy/urls.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.urls import path\n\nfrom .views import *\n\nurlpatterns = [\n path('request/', RequestView.as_view()),\n path('request/info/', get_request_info),\n path('request/ext1/', post_request_ext1),\n path('request/ext1/rollback/', post_request_ext1_rollback),\n path('request/ext2/', post_request_ext2),\n path('request/upload/', do_upload),\n path('request//', RequestDetailView.as_view()),\n]\n"
},
{
"path": "spug_api/apps/deploy/utils.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django_redis import get_redis_connection\nfrom django.conf import settings\nfrom django.db import close_old_connections\nfrom libs.utils import AttrDict, human_time, render_str\nfrom apps.host.models import Host\nfrom apps.config.utils import compose_configs\nfrom apps.repository.models import Repository\nfrom apps.repository.utils import dispatch as build_repository\nfrom apps.deploy.models import DeployRequest\nfrom apps.deploy.helper import Helper, SpugError\nfrom concurrent import futures\nfrom functools import partial\nimport json\nimport uuid\nimport os\n\nREPOS_DIR = settings.REPOS_DIR\nBUILD_DIR = settings.BUILD_DIR\n\n\ndef dispatch(req, fail_mode=False):\n rds = get_redis_connection()\n rds_key = f'{settings.REQUEST_KEY}:{req.id}'\n if fail_mode:\n req.host_ids = req.fail_host_ids\n req.fail_mode = fail_mode\n req.host_ids = json.loads(req.host_ids)\n req.fail_host_ids = req.host_ids[:]\n helper = Helper.make(rds, rds_key, req.host_ids if fail_mode else None)\n\n try:\n api_token = uuid.uuid4().hex\n rds.setex(api_token, 60 * 60, f'{req.deploy.app_id},{req.deploy.env_id}')\n env = AttrDict(\n SPUG_APP_NAME=req.deploy.app.name,\n SPUG_APP_KEY=req.deploy.app.key,\n SPUG_APP_ID=str(req.deploy.app_id),\n SPUG_REQUEST_ID=str(req.id),\n SPUG_REQUEST_NAME=req.name,\n SPUG_DEPLOY_ID=str(req.deploy.id),\n SPUG_ENV_ID=str(req.deploy.env_id),\n SPUG_ENV_KEY=req.deploy.env.key,\n SPUG_VERSION=req.version,\n SPUG_BUILD_VERSION=req.spug_version,\n SPUG_DEPLOY_TYPE=req.type,\n SPUG_API_TOKEN=api_token,\n SPUG_REPOS_DIR=REPOS_DIR,\n )\n # append configs\n configs = compose_configs(req.deploy.app, req.deploy.env_id)\n configs_env = {f'_SPUG_{k.upper()}': v for k, v in configs.items()}\n env.update(configs_env)\n\n if req.deploy.extend == '1':\n _ext1_deploy(req, helper, env)\n else:\n _ext2_deploy(req, helper, env)\n req.status = '3'\n except Exception as e:\n req.status = '-3'\n raise e\n finally:\n close_old_connections()\n DeployRequest.objects.filter(pk=req.id).update(\n status=req.status,\n repository=req.repository,\n fail_host_ids=json.dumps(req.fail_host_ids),\n )\n helper.clear()\n Helper.send_deploy_notify(req)\n\n\ndef _ext1_deploy(req, helper, env):\n if not req.repository_id:\n rep = Repository(\n app_id=req.deploy.app_id,\n env_id=req.deploy.env_id,\n deploy_id=req.deploy_id,\n version=req.version,\n spug_version=req.spug_version,\n extra=req.extra,\n remarks='SPUG AUTO MAKE',\n created_by_id=req.created_by_id\n )\n build_repository(rep, helper)\n req.repository = rep\n extras = json.loads(req.extra)\n if extras[0] == 'repository':\n extras = extras[1:]\n if extras[0] == 'branch':\n env.update(SPUG_GIT_BRANCH=extras[1], SPUG_GIT_COMMIT_ID=extras[2])\n else:\n env.update(SPUG_GIT_TAG=extras[1])\n if req.deploy.is_parallel:\n threads, latest_exception = [], None\n max_workers = max(10, os.cpu_count() * 5)\n with futures.ThreadPoolExecutor(max_workers=max_workers) as executor:\n for h_id in req.host_ids:\n new_env = AttrDict(env.items())\n t = executor.submit(_deploy_ext1_host, req, helper, h_id, new_env)\n t.h_id = h_id\n threads.append(t)\n for t in futures.as_completed(threads):\n exception = t.exception()\n if exception:\n latest_exception = exception\n if not isinstance(exception, SpugError):\n helper.send_error(t.h_id, f'Exception: {exception}', False)\n else:\n req.fail_host_ids.remove(t.h_id)\n if latest_exception:\n raise latest_exception\n else:\n host_ids = sorted(req.host_ids, reverse=True)\n while host_ids:\n h_id = host_ids.pop()\n new_env = AttrDict(env.items())\n try:\n _deploy_ext1_host(req, helper, h_id, new_env)\n req.fail_host_ids.remove(h_id)\n except Exception as e:\n helper.send_error(h_id, f'Exception: {e}', False)\n for h_id in host_ids:\n helper.send_error(h_id, '终止发布', False)\n raise e\n\n\ndef _ext2_deploy(req, helper, env):\n extend, step = req.deploy.extend_obj, 1\n host_actions = json.loads(extend.host_actions)\n server_actions = json.loads(extend.server_actions)\n env.update({'SPUG_RELEASE': req.version})\n if req.version:\n for index, value in enumerate(req.version.split()):\n env.update({f'SPUG_RELEASE_{index + 1}': value})\n\n if not req.fail_mode:\n helper.send_info('local', f'\\033[32m完成√\\033[0m\\r\\n')\n for action in server_actions:\n helper.send_step('local', step, f'{human_time()} {action[\"title\"]}...\\r\\n')\n helper.local(f'cd /tmp && {action[\"data\"]}', env)\n step += 1\n\n for action in host_actions:\n if action.get('type') == 'transfer':\n action['src'] = render_str(action.get('src', '').strip().rstrip('/'), env)\n action['dst'] = render_str(action['dst'].strip().rstrip('/'), env)\n if action.get('src_mode') == '1': # upload when publish\n extra = json.loads(req.extra)\n if 'name' in extra:\n action['name'] = extra['name']\n break\n helper.send_step('local', step, f'{human_time()} 检测到来源为本地路径的数据传输动作，执行打包... \\r\\n')\n action['src'] = action['src'].rstrip('/ ')\n action['dst'] = action['dst'].rstrip('/ ')\n if not action['src'] or not action['dst']:\n helper.send_error('local', f'Invalid path for transfer, src: {action[\"src\"]} dst: {action[\"dst\"]}')\n if not os.path.exists(action['src']):\n helper.send_error('local', f'No such file or directory: {action[\"src\"]}')\n is_dir, exclude = os.path.isdir(action['src']), ''\n sp_dir, sd_dst = os.path.split(action['src'])\n contain = sd_dst\n if action['mode'] != '0' and is_dir:\n files = helper.parse_filter_rule(action['rule'], ',', env)\n if files:\n if action['mode'] == '1':\n contain = ' '.join(f'{sd_dst}/{x}' for x in files)\n else:\n excludes = []\n for x in files:\n if x.startswith('/'):\n excludes.append(f'--exclude={sd_dst}{x}')\n else:\n excludes.append(f'--exclude={x}')\n exclude = ' '.join(excludes)\n tar_gz_file = f'{req.spug_version}.tar.gz'\n helper.local(f'cd {sp_dir} && tar -zcf {tar_gz_file} {exclude} {contain}')\n helper.send_info('local', f'{human_time()} \\033[32m完成√\\033[0m\\r\\n')\n helper.add_callback(partial(os.remove, os.path.join(sp_dir, tar_gz_file)))\n break\n helper.send_step('local', 100, '')\n\n if host_actions:\n if req.deploy.is_parallel:\n threads, latest_exception = [], None\n max_workers = max(10, os.cpu_count() * 5)\n with futures.ThreadPoolExecutor(max_workers=max_workers) as executor:\n for h_id in req.host_ids:\n new_env = AttrDict(env.items())\n t = executor.submit(_deploy_ext2_host, helper, h_id, host_actions, new_env, req.spug_version)\n t.h_id = h_id\n threads.append(t)\n for t in futures.as_completed(threads):\n exception = t.exception()\n if exception:\n latest_exception = exception\n if not isinstance(exception, SpugError):\n helper.send_error(t.h_id, f'Exception: {exception}', False)\n else:\n req.fail_host_ids.remove(t.h_id)\n if latest_exception:\n raise latest_exception\n else:\n host_ids = sorted(req.host_ids, reverse=True)\n while host_ids:\n h_id = host_ids.pop()\n new_env = AttrDict(env.items())\n try:\n _deploy_ext2_host(helper, h_id, host_actions, new_env, req.spug_version)\n req.fail_host_ids.remove(h_id)\n except Exception as e:\n helper.send_error(h_id, f'Exception: {e}', False)\n for h_id in host_ids:\n helper.send_error(h_id, '终止发布', False)\n raise e\n else:\n req.fail_host_ids = []\n helper.send_step('local', 100, f'\\r\\n{human_time()} ** 发布成功 **')\n\n\ndef _deploy_ext1_host(req, helper, h_id, env):\n helper.send_step(h_id, 1, f'\\033[32m就绪√\\033[0m\\r\\n{human_time()} 数据准备... ')\n host = Host.objects.filter(pk=h_id).first()\n if not host:\n helper.send_error(h_id, 'no such host')\n env.update({'SPUG_HOST_ID': h_id, 'SPUG_HOST_NAME': host.hostname})\n extend = req.deploy.extend_obj\n extend.dst_dir = render_str(extend.dst_dir, env)\n extend.dst_repo = render_str(extend.dst_repo, env)\n env.update(SPUG_DST_DIR=extend.dst_dir)\n with host.get_ssh(default_env=env) as ssh:\n base_dst_dir = os.path.dirname(extend.dst_dir)\n code, _ = ssh.exec_command_raw(\n f'mkdir -p {extend.dst_repo} {base_dst_dir} && [ -e {extend.dst_dir} ] && [ ! -L {extend.dst_dir} ]')\n if code == 0:\n helper.send_error(host.id, f'检测到该主机的发布目录 {extend.dst_dir!r} 已存在，为了数据安全请自行备份后删除该目录，Spug 将会创建并接管该目录。')\n if req.type == '2':\n helper.send_step(h_id, 1, '\\033[33m跳过√\\033[0m\\r\\n')\n else:\n # clean\n clean_command = f'ls -d {extend.deploy_id}_* 2> /dev/null | sort -t _ -rnk2 | tail -n +{extend.versions + 1} | xargs rm -rf'\n helper.remote_raw(host.id, ssh, f'cd {extend.dst_repo} && {clean_command}')\n # transfer files\n tar_gz_file = f'{req.spug_version}.tar.gz'\n try:\n callback = helper.progress_callback(host.id)\n ssh.put_file(\n os.path.join(BUILD_DIR, tar_gz_file),\n os.path.join(extend.dst_repo, tar_gz_file),\n callback\n )\n except Exception as e:\n helper.send_error(host.id, f'Exception: {e}')\n\n command = f'cd {extend.dst_repo} && rm -rf {req.spug_version} && tar xf {tar_gz_file} && rm -f {req.deploy_id}_*.tar.gz'\n helper.remote_raw(host.id, ssh, command)\n helper.send_step(h_id, 1, '\\033[32m完成√\\033[0m\\r\\n')\n\n # pre host\n repo_dir = os.path.join(extend.dst_repo, req.spug_version)\n if extend.hook_pre_host:\n helper.send_step(h_id, 2, f'{human_time()} 发布前任务... \\r\\n')\n command = f'cd {repo_dir} && {extend.hook_pre_host}'\n helper.remote(host.id, ssh, command)\n\n # do deploy\n helper.send_step(h_id, 3, f'{human_time()} 执行发布... ')\n helper.remote_raw(host.id, ssh, f'rm -f {extend.dst_dir} && ln -sfn {repo_dir} {extend.dst_dir}')\n helper.send_step(h_id, 3, '\\033[32m完成√\\033[0m\\r\\n')\n\n # post host\n if extend.hook_post_host:\n helper.send_step(h_id, 4, f'{human_time()} 发布后任务... \\r\\n')\n command = f'cd {extend.dst_dir} && {extend.hook_post_host}'\n helper.remote(host.id, ssh, command)\n\n helper.send_step(h_id, 100, f'\\r\\n{human_time()} ** \\033[32m发布成功\\033[0m **')\n\n\ndef _deploy_ext2_host(helper, h_id, actions, env, spug_version):\n helper.send_info(h_id, '\\033[32m就绪√\\033[0m\\r\\n')\n host = Host.objects.filter(pk=h_id).first()\n if not host:\n helper.send_error(h_id, 'no such host')\n env.update({'SPUG_HOST_ID': h_id, 'SPUG_HOST_NAME': host.hostname})\n with host.get_ssh(default_env=env) as ssh:\n for index, action in enumerate(actions):\n helper.send_step(h_id, 1 + index, f'{human_time()} {action[\"title\"]}...\\r\\n')\n if action.get('type') == 'transfer':\n if action.get('src_mode') == '1':\n try:\n dst = action['dst']\n command = f'[ -e {dst} ] || mkdir -p $(dirname {dst}); [ -d {dst} ]'\n code, _ = ssh.exec_command_raw(command)\n if code == 0: # is dir\n if not action.get('name'):\n raise RuntimeError('internal error 1002')\n dst = dst.rstrip('/') + '/' + action['name']\n callback = helper.progress_callback(host.id)\n ssh.put_file(os.path.join(REPOS_DIR, env.SPUG_DEPLOY_ID, spug_version), dst, callback)\n except Exception as e:\n helper.send_error(host.id, f'Exception: {e}')\n helper.send_info(host.id, 'transfer completed\\r\\n')\n continue\n else:\n sp_dir, sd_dst = os.path.split(action['src'])\n tar_gz_file = f'{spug_version}.tar.gz'\n try:\n callback = helper.progress_callback(host.id)\n ssh.put_file(os.path.join(sp_dir, tar_gz_file), f'/tmp/{tar_gz_file}', callback)\n except Exception as e:\n helper.send_error(host.id, f'Exception: {e}')\n\n command = f'mkdir -p /tmp/{spug_version} && tar xf /tmp/{tar_gz_file} -C /tmp/{spug_version}/ '\n command += f'&& rm -rf {action[\"dst\"]} && mv /tmp/{spug_version}/{sd_dst} {action[\"dst\"]} '\n command += f'&& rm -rf /tmp/{spug_version}* && echo \"transfer completed\"'\n else:\n command = f'cd /tmp && {action[\"data\"]}'\n helper.remote(host.id, ssh, command)\n\n helper.send_step(h_id, 100, f'\\r\\n{human_time()} ** \\033[32m发布成功\\033[0m **')\n"
},
{
"path": "spug_api/apps/deploy/views.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.views.generic import View\nfrom django.db.models import F\nfrom django.conf import settings\nfrom django.http.response import HttpResponseBadRequest\nfrom django_redis import get_redis_connection\nfrom libs import json_response, JsonParser, Argument, human_datetime, human_time, auth\nfrom apps.deploy.models import DeployRequest\nfrom apps.app.models import Deploy, DeployExtend2\nfrom apps.repository.models import Repository\nfrom apps.deploy.utils import dispatch, Helper\nfrom apps.host.models import Host\nfrom collections import defaultdict\nfrom threading import Thread\nfrom datetime import datetime\nimport subprocess\nimport json\nimport os\n\n\nclass RequestView(View):\n @auth('deploy.request.view')\n def get(self, request):\n data, query, counter = [], {}, {}\n if not request.user.is_supper:\n perms = request.user.deploy_perms\n query['deploy__app_id__in'] = perms['apps']\n query['deploy__env_id__in'] = perms['envs']\n for item in DeployRequest.objects.filter(**query).annotate(\n env_id=F('deploy__env_id'),\n env_name=F('deploy__env__name'),\n app_id=F('deploy__app_id'),\n app_name=F('deploy__app__name'),\n app_host_ids=F('deploy__host_ids'),\n app_extend=F('deploy__extend'),\n rep_extra=F('repository__extra'),\n do_by_user=F('do_by__nickname'),\n approve_by_user=F('approve_by__nickname'),\n created_by_user=F('created_by__nickname')):\n tmp = item.to_dict()\n tmp['env_id'] = item.env_id\n tmp['env_name'] = item.env_name\n tmp['app_id'] = item.app_id\n tmp['app_name'] = item.app_name\n tmp['app_extend'] = item.app_extend\n tmp['host_ids'] = json.loads(item.host_ids)\n tmp['fail_host_ids'] = json.loads(item.fail_host_ids)\n tmp['extra'] = json.loads(item.extra) if item.extra else None\n tmp['rep_extra'] = json.loads(item.rep_extra) if item.rep_extra else None\n tmp['app_host_ids'] = json.loads(item.app_host_ids)\n tmp['status_alias'] = item.get_status_display()\n tmp['created_by_user'] = item.created_by_user\n tmp['approve_by_user'] = item.approve_by_user\n tmp['do_by_user'] = item.do_by_user\n if item.app_extend == '1':\n tmp['visible_rollback'] = item.deploy_id not in counter\n counter[item.deploy_id] = True\n data.append(tmp)\n return json_response(data)\n\n @auth('deploy.request.del')\n def delete(self, request):\n form, error = JsonParser(\n Argument('id', type=int, required=False),\n Argument('mode', filter=lambda x: x in ('count', 'expire', 'deploy'), required=False, help='参数错误'),\n Argument('value', required=False),\n ).parse(request.GET)\n if error is None:\n if form.id:\n deploy = DeployRequest.objects.filter(pk=form.id).first()\n if not deploy or deploy.status not in ('0', '1', '-1'):\n return json_response(error='未找到指定发布申请或当前状态不允许删除')\n deploy.delete()\n return json_response()\n\n count = 0\n if form.mode == 'count':\n if not str(form.value).isdigit() or int(form.value) < 1:\n return json_response(error='请输入正确的保留数量')\n counter, form.value = defaultdict(int), int(form.value)\n for item in DeployRequest.objects.all():\n counter[item.deploy_id] += 1\n if counter[item.deploy_id] > form.value:\n count += 1\n item.delete()\n elif form.mode == 'expire':\n for item in DeployRequest.objects.filter(created_at__lt=form.value):\n count += 1\n item.delete()\n elif form.mode == 'deploy':\n app_id, env_id = str(form.value).split(',')\n for item in DeployRequest.objects.filter(deploy__app_id=app_id, deploy__env_id=env_id):\n count += 1\n item.delete()\n return json_response(count)\n return json_response(error=error)\n\n\nclass RequestDetailView(View):\n @auth('deploy.request.view')\n def get(self, request, r_id):\n req = DeployRequest.objects.filter(pk=r_id).first()\n if not req:\n return json_response(error='未找到指定发布申请')\n hosts = Host.objects.filter(id__in=json.loads(req.host_ids))\n outputs = {x.id: {'id': x.id, 'title': x.name, 'data': f'{human_time()} 读取数据... '} for x in hosts}\n response = {'outputs': outputs, 'status': req.status}\n if req.is_quick_deploy:\n outputs['local'] = {'id': 'local', 'data': ''}\n if req.deploy.extend == '2':\n outputs['local'] = {'id': 'local', 'data': f'{human_time()} 读取数据... '}\n response['s_actions'] = json.loads(req.deploy.extend_obj.server_actions)\n response['h_actions'] = json.loads(req.deploy.extend_obj.host_actions)\n if not response['h_actions']:\n response['outputs'] = {'local': outputs['local']}\n rds, key, counter = get_redis_connection(), f'{settings.REQUEST_KEY}:{r_id}', 0\n data = rds.lrange(key, counter, counter + 9)\n while data:\n for item in data:\n counter += 1\n item = json.loads(item.decode())\n if item['key'] in outputs:\n if 'data' in item:\n outputs[item['key']]['data'] += item['data']\n if 'step' in item:\n outputs[item['key']]['step'] = item['step']\n if 'status' in item:\n outputs[item['key']]['status'] = item['status']\n data = rds.lrange(key, counter, counter + 9)\n response['index'] = counter\n if counter == 0:\n for item in outputs:\n outputs[item]['data'] += '\\r\\n\\r\\n未读取到数据，Spug 仅保存最近2周的日志信息。'\n\n if req.is_quick_deploy:\n if outputs['local']['data']:\n outputs['local']['data'] = f'{human_time()} 读取数据... ' + outputs['local']['data']\n else:\n outputs['local'].update(step=100, data=f'{human_time()} 已构建完成忽略执行。')\n return json_response(response)\n\n @auth('deploy.request.do')\n def post(self, request, r_id):\n form, _ = JsonParser(Argument('mode', default='all')).parse(request.body)\n query = {'pk': r_id}\n if not request.user.is_supper:\n perms = request.user.deploy_perms\n query['deploy__app_id__in'] = perms['apps']\n query['deploy__env_id__in'] = perms['envs']\n req = DeployRequest.objects.filter(**query).first()\n if not req:\n return json_response(error='未找到指定发布申请')\n if req.status not in ('1', '-3'):\n return json_response(error='该申请单当前状态还不能执行发布')\n\n host_ids = req.fail_host_ids if form.mode == 'fail' else req.host_ids\n hosts = Host.objects.filter(id__in=json.loads(host_ids))\n message = f'{human_time()} 等待调度... '\n outputs = {x.id: {'id': x.id, 'title': x.name, 'step': 0, 'data': message} for x in hosts}\n req.status = '2'\n req.do_at = human_datetime()\n req.do_by = request.user\n req.save()\n Thread(target=dispatch, args=(req, form.mode == 'fail')).start()\n if req.is_quick_deploy:\n if req.repository_id:\n outputs['local'] = {'id': 'local', 'step': 100, 'data': f'{human_time()} 已构建完成忽略执行。'}\n else:\n outputs['local'] = {'id': 'local', 'step': 0, 'data': f'{human_time()} 建立连接... '}\n if req.deploy.extend == '2':\n outputs['local'] = {'id': 'local', 'step': 0, 'data': f'{human_time()} 建立连接... '}\n s_actions = json.loads(req.deploy.extend_obj.server_actions)\n h_actions = json.loads(req.deploy.extend_obj.host_actions)\n for item in h_actions:\n if item.get('type') == 'transfer' and item.get('src_mode') == '0':\n s_actions.append({'title': '执行打包'})\n if not h_actions:\n outputs = {'local': outputs['local']}\n return json_response({'s_actions': s_actions, 'h_actions': h_actions, 'outputs': outputs})\n return json_response({'outputs': outputs})\n\n @auth('deploy.request.approve')\n def patch(self, request, r_id):\n form, error = JsonParser(\n Argument('reason', required=False),\n Argument('is_pass', type=bool, help='参数错误')\n ).parse(request.body)\n if error is None:\n req = DeployRequest.objects.filter(pk=r_id).first()\n if not req:\n return json_response(error='未找到指定申请')\n if not form.is_pass and not form.reason:\n return json_response(error='请输入驳回原因')\n if req.status != '0':\n return json_response(error='该申请当前状态不允许审核')\n req.approve_at = human_datetime()\n req.approve_by = request.user\n req.status = '1' if form.is_pass else '-1'\n req.reason = form.reason\n req.save()\n Thread(target=Helper.send_deploy_notify, args=(req, 'approve_rst')).start()\n return json_response(error=error)\n\n\n@auth('deploy.request.add|deploy.request.edit')\ndef post_request_ext1(request):\n form, error = JsonParser(\n Argument('id', type=int, required=False),\n Argument('deploy_id', type=int, help='参数错误'),\n Argument('name', help='请输入申请标题'),\n Argument('extra', type=list, help='请选择发布版本'),\n Argument('host_ids', type=list, filter=lambda x: len(x), help='请选择要部署的主机'),\n Argument('type', default='1'),\n Argument('plan', required=False),\n Argument('desc', required=False),\n ).parse(request.body)\n if error is None:\n deploy = Deploy.objects.get(pk=form.deploy_id)\n form.spug_version = Repository.make_spug_version(deploy.id)\n if form.extra[0] == 'tag':\n if not form.extra[1]:\n return json_response(error='请选择要发布的版本')\n form.version = form.extra[1]\n elif form.extra[0] == 'branch':\n if not form.extra[2]:\n return json_response(error='请选择要发布的分支及Commit ID')\n form.version = f'{form.extra[1]}#{form.extra[2][:6]}'\n elif form.extra[0] == 'repository':\n if not form.extra[1]:\n return json_response(error='请选择要发布的版本')\n repository = Repository.objects.get(pk=form.extra[1])\n form.repository_id = repository.id\n form.version = repository.version\n form.spug_version = repository.spug_version\n form.extra = ['repository'] + json.loads(repository.extra)\n else:\n return json_response(error='参数错误')\n\n form.extra = json.dumps(form.extra)\n form.status = '0' if deploy.is_audit else '1'\n form.host_ids = json.dumps(sorted(form.host_ids))\n if form.id:\n req = DeployRequest.objects.get(pk=form.id)\n is_required_notify = deploy.is_audit and req.status == '-1'\n DeployRequest.objects.filter(pk=form.id).update(created_by=request.user, reason=None, **form)\n else:\n req = DeployRequest.objects.create(created_by=request.user, **form)\n is_required_notify = deploy.is_audit\n if is_required_notify:\n Thread(target=Helper.send_deploy_notify, args=(req, 'approve_req')).start()\n return json_response(error=error)\n\n\n@auth('deploy.request.do')\ndef post_request_ext1_rollback(request):\n form, error = JsonParser(\n Argument('request_id', type=int, help='请选择要回滚的版本'),\n Argument('name', help='请输入申请标题'),\n Argument('host_ids', type=list, filter=lambda x: len(x), help='请选择要部署的主机'),\n Argument('desc', required=False),\n ).parse(request.body)\n if error is None:\n req = DeployRequest.objects.get(pk=form.pop('request_id'))\n requests = DeployRequest.objects.filter(deploy=req.deploy, status__in=('3', '-3'))\n versions = list({x.spug_version: 1 for x in requests}.keys())\n if req.spug_version not in versions[:req.deploy.extend_obj.versions + 1]:\n return json_response(error='选择的版本超出了发布配置中设置的版本数量，无法快速回滚，可通过新建发布申请选择构建仓库里的该版本再次发布。')\n\n form.status = '0' if req.deploy.is_audit else '1'\n form.host_ids = json.dumps(sorted(form.host_ids))\n new_req = DeployRequest.objects.create(\n deploy_id=req.deploy_id,\n repository_id=req.repository_id,\n type='2',\n extra=req.extra,\n version=req.version,\n spug_version=req.spug_version,\n created_by=request.user,\n **form\n )\n if req.deploy.is_audit:\n Thread(target=Helper.send_deploy_notify, args=(new_req, 'approve_req')).start()\n return json_response(error=error)\n\n\n@auth('deploy.request.add|deploy.request.edit')\ndef post_request_ext2(request):\n form, error = JsonParser(\n Argument('id', type=int, required=False),\n Argument('deploy_id', type=int, help='缺少必要参数'),\n Argument('name', help='请输申请标题'),\n Argument('host_ids', type=list, filter=lambda x: len(x), help='请选择要部署的主机'),\n Argument('extra', type=dict, required=False),\n Argument('version', default=''),\n Argument('type', default='1'),\n Argument('plan', required=False),\n Argument('desc', required=False),\n ).parse(request.body)\n if error is None:\n deploy = Deploy.objects.filter(pk=form.deploy_id).first()\n if not deploy:\n return json_response(error='未找到该发布配置')\n extra = form.pop('extra')\n if DeployExtend2.objects.filter(deploy=deploy, host_actions__contains='\"src_mode\": \"1\"').exists():\n if not extra:\n return json_response(error='该应用的发布配置中使用了数据传输动作且设置为发布时上传，请上传要传输的数据')\n form.spug_version = extra['path']\n form.extra = json.dumps(extra)\n else:\n form.spug_version = Repository.make_spug_version(deploy.id)\n form.name = form.name.replace(\"'\", '')\n form.status = '0' if deploy.is_audit else '1'\n form.host_ids = json.dumps(form.host_ids)\n if form.id:\n req = DeployRequest.objects.get(pk=form.id)\n is_required_notify = deploy.is_audit and req.status == '-1'\n form.update(created_by=request.user, reason=None)\n req.update_by_dict(form)\n else:\n req = DeployRequest.objects.create(created_by=request.user, **form)\n is_required_notify = deploy.is_audit\n if is_required_notify:\n Thread(target=Helper.send_deploy_notify, args=(req, 'approve_req')).start()\n return json_response(error=error)\n\n\n@auth('deploy.request.view')\ndef get_request_info(request):\n form, error = JsonParser(\n Argument('id', type=int, help='参数错误')\n ).parse(request.GET)\n if error is None:\n req = DeployRequest.objects.get(pk=form.id)\n response = req.to_dict(selects=('status', 'reason'))\n response['fail_host_ids'] = json.loads(req.fail_host_ids)\n response['status_alias'] = req.get_status_display()\n return json_response(response)\n return json_response(error=error)\n\n\n@auth('deploy.request.add')\ndef do_upload(request):\n repos_dir = settings.REPOS_DIR\n file = request.FILES['file']\n deploy_id = request.POST.get('deploy_id')\n if file and deploy_id:\n dir_name = os.path.join(repos_dir, deploy_id)\n file_name = datetime.now().strftime(\"%Y%m%d%H%M%S\")\n command = f'mkdir -p {dir_name} && cd {dir_name} && ls | sort -rn | tail -n +11 | xargs rm -rf'\n code, outputs = subprocess.getstatusoutput(command)\n if code != 0:\n return json_response(error=outputs)\n with open(os.path.join(dir_name, file_name), 'wb') as f:\n for chunk in file.chunks():\n f.write(chunk)\n return json_response(file_name)\n else:\n return HttpResponseBadRequest()\n"
},
{
"path": "spug_api/apps/exec/__init__.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\n"
},
{
"path": "spug_api/apps/exec/executors.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django_redis import get_redis_connection\nfrom libs.utils import human_seconds_time\nfrom libs.ssh import SSH\nimport threading\nimport socket\nimport json\nimport time\n\n\ndef exec_worker_handler(job):\n job = Job(**json.loads(job))\n threading.Thread(target=job.run).start()\n\n\nclass Job:\n def __init__(self, key, name, hostname, port, username, pkey, command, interpreter, params=None, token=None,\n term=None):\n self.ssh = SSH(hostname, port, username, pkey, term=term)\n self.key = key\n self.command = self._handle_command(command, interpreter)\n self.token = token\n self.rds = get_redis_connection()\n self.env = dict(\n SPUG_HOST_ID=str(self.key),\n SPUG_HOST_NAME=name,\n SPUG_HOST_HOSTNAME=hostname,\n SPUG_SSH_PORT=str(port),\n SPUG_SSH_USERNAME=username,\n SPUG_INTERPRETER=interpreter\n )\n if isinstance(params, dict):\n self.env.update({f'_SPUG_{k}': str(v) for k, v in params.items()})\n\n def _send(self, message):\n self.rds.publish(self.token, json.dumps(message))\n\n def _handle_command(self, command, interpreter):\n if interpreter == 'python':\n attach = 'INTERPRETER=python\\ncommand -v python3 &> /dev/null && INTERPRETER=python3'\n return f'{attach}\\n$INTERPRETER << EOF\\n# -*- coding: UTF-8 -*-\\n{command}\\nEOF'\n return command\n\n def send(self, data):\n self._send({'key': self.key, 'data': data})\n\n def send_status(self, code):\n self._send({'key': self.key, 'status': code})\n\n def run(self):\n if not self.token:\n with self.ssh:\n return self.ssh.exec_command(self.command, self.env)\n flag = time.time()\n self.send('\\r\\n\\x1b[36m### Executing ...\\x1b[0m\\r\\n')\n code = -1\n try:\n with self.ssh:\n for code, out in self.ssh.exec_command_with_stream(self.command, self.env):\n self.send(out)\n human_time = human_seconds_time(time.time() - flag)\n self.send(f'\\r\\n\\x1b[36m** 执行结束，总耗时：{human_time} **\\x1b[0m')\n except socket.timeout:\n code = 130\n self.send('\\r\\n\\x1b[31m### Time out\\x1b[0m')\n except Exception as e:\n code = 131\n self.send(f'\\r\\n\\x1b[31m### Exception {e}\\x1b[0m')\n raise e\n finally:\n self.send_status(code)\n"
},
{
"path": "spug_api/apps/exec/management/commands/runworker.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.core.management.base import BaseCommand\nfrom django.conf import settings\nfrom django.db import connections\nfrom django_redis import get_redis_connection\nfrom concurrent.futures import ThreadPoolExecutor\nfrom apps.schedule.executors import schedule_worker_handler\nfrom apps.monitor.executors import monitor_worker_handler\nfrom apps.exec.executors import exec_worker_handler\nfrom apps.notify.models import Notify\nfrom threading import Thread\nimport logging\nimport time\nimport os\n\nEXEC_WORKER_KEY = settings.EXEC_WORKER_KEY\nMONITOR_WORKER_KEY = settings.MONITOR_WORKER_KEY\nSCHEDULE_WORKER_KEY = settings.SCHEDULE_WORKER_KEY\n\nlogging.basicConfig(level=logging.WARNING, format='%(asctime)s %(message)s')\n\n\nclass Worker:\n def __init__(self):\n self.rds = get_redis_connection()\n self._executor = ThreadPoolExecutor(max_workers=max(100, os.cpu_count() * 50))\n\n def job_done(self, future):\n connections.close_all()\n\n def queue_monitor(self):\n counter = 0\n while True:\n time.sleep((counter or 1) ** 3 * 10)\n qsize = self._executor._work_queue.qsize()\n if qsize > 0:\n if counter > 0:\n content = '请检查监控、任务计划或批量执行等避免长耗时任务，必要时可重启服务清空队列。'\n try:\n Notify.make_system_notify(f'执行队列堆积（{qsize}）', content)\n except Exception as e:\n logging.warning(e)\n finally:\n connections.close_all()\n logging.warning(f'!!! 执行队列堆积（{qsize}）')\n counter += 1\n else:\n counter = 0\n\n def run(self):\n logging.warning('Running worker')\n Thread(target=self.queue_monitor, daemon=True).start()\n self.rds.delete(EXEC_WORKER_KEY, MONITOR_WORKER_KEY, SCHEDULE_WORKER_KEY)\n while True:\n key, job = self.rds.blpop([EXEC_WORKER_KEY, SCHEDULE_WORKER_KEY, MONITOR_WORKER_KEY])\n key = key.decode()\n if key == SCHEDULE_WORKER_KEY:\n future = self._executor.submit(schedule_worker_handler, job)\n elif key == MONITOR_WORKER_KEY:\n future = self._executor.submit(monitor_worker_handler, job)\n elif key == EXEC_WORKER_KEY:\n future = self._executor.submit(exec_worker_handler, job)\n else:\n continue\n future.add_done_callback(self.job_done)\n\n\nclass Command(BaseCommand):\n help = 'Start worker process'\n\n def handle(self, *args, **options):\n w = Worker()\n w.run()\n"
},
{
"path": "spug_api/apps/exec/models.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.db import models\nfrom libs import ModelMixin, human_datetime\nfrom apps.account.models import User\nimport json\n\n\nclass ExecTemplate(models.Model, ModelMixin):\n name = models.CharField(max_length=50)\n type = models.CharField(max_length=50)\n body = models.TextField()\n interpreter = models.CharField(max_length=20, default='sh')\n host_ids = models.TextField(default='[]')\n desc = models.CharField(max_length=255, null=True)\n parameters = models.TextField(default='[]')\n created_at = models.CharField(max_length=20, default=human_datetime)\n created_by = models.ForeignKey(User, models.PROTECT, related_name='+')\n updated_at = models.CharField(max_length=20, null=True)\n updated_by = models.ForeignKey(User, models.PROTECT, related_name='+', null=True)\n\n def __repr__(self):\n return '' % self.name\n\n def to_view(self):\n tmp = self.to_dict()\n tmp['host_ids'] = json.loads(self.host_ids)\n tmp['parameters'] = json.loads(self.parameters)\n return tmp\n\n class Meta:\n db_table = 'exec_templates'\n ordering = ('-id',)\n\n\nclass ExecHistory(models.Model, ModelMixin):\n user = models.ForeignKey(User, on_delete=models.CASCADE)\n template = models.ForeignKey(ExecTemplate, on_delete=models.SET_NULL, null=True)\n digest = models.CharField(max_length=32, db_index=True)\n interpreter = models.CharField(max_length=20)\n command = models.TextField()\n params = models.TextField(default='{}')\n host_ids = models.TextField()\n updated_at = models.CharField(max_length=20, default=human_datetime)\n\n def to_view(self):\n tmp = self.to_dict()\n tmp['host_ids'] = json.loads(self.host_ids)\n if self.template:\n tmp['template_name'] = self.template.name\n tmp['interpreter'] = self.template.interpreter\n tmp['parameters'] = json.loads(self.template.parameters)\n tmp['command'] = self.template.body\n return tmp\n\n class Meta:\n db_table = 'exec_histories'\n ordering = ('-updated_at',)\n\n\nclass Transfer(models.Model, ModelMixin):\n user = models.ForeignKey(User, on_delete=models.CASCADE)\n digest = models.CharField(max_length=32, db_index=True)\n host_id = models.IntegerField(null=True)\n src_dir = models.CharField(max_length=255)\n dst_dir = models.CharField(max_length=255)\n host_ids = models.TextField()\n updated_at = models.CharField(max_length=20, default=human_datetime)\n\n def to_view(self):\n tmp = self.to_dict()\n tmp['host_ids'] = json.loads(self.host_ids)\n return tmp\n\n class Meta:\n db_table = 'exec_transfer'\n ordering = ('-id',)\n"
},
{
"path": "spug_api/apps/exec/transfer.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.views.generic import View\nfrom django.conf import settings\nfrom django.db import close_old_connections\nfrom django_redis import get_redis_connection\nfrom apps.exec.models import Transfer\nfrom apps.account.utils import has_host_perm\nfrom apps.host.models import Host\nfrom apps.setting.utils import AppSetting\nfrom libs import json_response, JsonParser, Argument, auth\nfrom libs.utils import str_decode, human_seconds_time\nfrom concurrent import futures\nfrom threading import Thread\nimport subprocess\nimport tempfile\nimport uuid\nimport json\nimport time\nimport os\n\n\nclass TransferView(View):\n @auth('exec.transfer.do')\n def get(self, request):\n records = Transfer.objects.filter(user=request.user)\n return json_response([x.to_view() for x in records])\n\n @auth('exec.transfer.do')\n def post(self, request):\n data = request.POST.get('data')\n form, error = JsonParser(\n Argument('host', required=False),\n Argument('dst_dir', help='请输入目标路径'),\n Argument('host_ids', type=list, filter=lambda x: len(x), help='请选择目标主机'),\n ).parse(data)\n if error is None:\n if not has_host_perm(request.user, form.host_ids):\n return json_response(error='无权访问主机，请联系管理员')\n host_id = None\n token = uuid.uuid4().hex\n base_dir = os.path.join(settings.TRANSFER_DIR, token)\n if form.host:\n host_id, path = json.loads(form.host)\n if not path.strip('/'):\n return json_response(error='请输入正确的数据源路径')\n host = Host.objects.get(pk=host_id)\n with host.get_ssh() as ssh:\n code, _ = ssh.exec_command_raw(f'[ -d {path} ]')\n if code != 0:\n return json_response(error='数据源路径必须为该主机上已存在的目录')\n os.makedirs(base_dir)\n with tempfile.NamedTemporaryFile(mode='w') as fp:\n fp.write(host.pkey or AppSetting.get('private_key'))\n fp.flush()\n target = f'{host.username}@{host.hostname}:{path}'\n command = f'sshfs -o ro -o ssh_command=\"ssh -p {host.port} -i {fp.name}\" {target} {base_dir}'\n task = subprocess.run(command, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)\n if task.returncode != 0:\n os.system(f'umount -f {base_dir} &> /dev/null ; rm -rf {base_dir}')\n return json_response(error=task.stdout.decode())\n else:\n os.makedirs(base_dir)\n index = 0\n while True:\n file = request.FILES.get(f'file{index}')\n if not file:\n break\n with open(os.path.join(base_dir, file.name), 'wb') as f:\n for chunk in file.chunks():\n f.write(chunk)\n index += 1\n Transfer.objects.create(\n user=request.user,\n digest=token,\n host_id=host_id,\n src_dir=base_dir,\n dst_dir=form.dst_dir,\n host_ids=json.dumps(form.host_ids),\n )\n return json_response(token)\n return json_response(error=error)\n\n @auth('exec.transfer.do')\n def patch(self, request):\n form, error = JsonParser(\n Argument('token', help='参数错误')\n ).parse(request.body)\n if error is None:\n task = Transfer.objects.get(digest=form.token)\n Thread(target=_dispatch_sync, args=(task,)).start()\n return json_response(error=error)\n\n\ndef _dispatch_sync(task):\n rds = get_redis_connection()\n threads = []\n max_workers = max(10, os.cpu_count() * 5)\n with futures.ThreadPoolExecutor(max_workers=max_workers) as executor:\n for host in Host.objects.filter(id__in=json.loads(task.host_ids)):\n t = executor.submit(_do_sync, rds, task, host)\n t.token = task.digest\n t.key = host.id\n threads.append(t)\n for t in futures.as_completed(threads):\n exc = t.exception()\n if exc:\n rds.publish(\n t.token,\n json.dumps({'key': t.key, 'status': -1, 'data': f'\\x1b[31mException: {exc}\\x1b[0m'})\n )\n if task.host_id:\n command = f'umount -f {task.src_dir} && rm -rf {task.src_dir}'\n else:\n command = f'rm -rf {task.src_dir}'\n subprocess.run(command, shell=True)\n close_old_connections()\n\n\ndef _do_sync(rds, task, host):\n token = task.digest\n rds.publish(token, json.dumps({'key': host.id, 'data': '\\r\\n\\x1b[36m### Executing ...\\x1b[0m\\r\\n'}))\n with tempfile.NamedTemporaryFile(mode='w') as fp:\n fp.write(host.pkey or AppSetting.get('private_key'))\n fp.write('\\n')\n fp.flush()\n\n flag = time.time()\n options = '-azv --progress' if task.host_id else '-rzv --progress'\n argument = f'{task.src_dir}/ {host.username}@{host.hostname}:{task.dst_dir}'\n command = f'rsync {options} -h -e \"ssh -p {host.port} -o StrictHostKeyChecking=no -i {fp.name}\" {argument}'\n task = subprocess.Popen(command, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)\n message = b''\n while True:\n output = task.stdout.read(1)\n if not output:\n break\n if output in (b'\\r', b'\\n'):\n message += b'\\r\\n' if output == b'\\n' else b'\\r'\n message = str_decode(message)\n if 'rsync: command not found' in message:\n data = '\\r\\n\\x1b[31m检测到该主机未安装rsync，可通过批量执行/执行任务模块进行以下命令批量安装\\x1b[0m'\n data += '\\r\\nCentos/Redhat: yum install -y rsync'\n data += '\\r\\nUbuntu/Debian: apt install -y rsync'\n rds.publish(token, json.dumps({'key': host.id, 'data': data}))\n break\n rds.publish(token, json.dumps({'key': host.id, 'data': message}))\n message = b''\n else:\n message += output\n status = task.wait()\n if status == 0:\n human_time = human_seconds_time(time.time() - flag)\n rds.publish(token, json.dumps({'key': host.id, 'data': f'\\r\\n\\x1b[32m** 分发完成，总耗时：{human_time} **\\x1b[0m'}))\n rds.publish(token, json.dumps({'key': host.id, 'status': task.wait()}))\n"
},
{
"path": "spug_api/apps/exec/urls.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.conf.urls import url\n\nfrom apps.exec.views import *\nfrom apps.exec.transfer import TransferView\n\nurlpatterns = [\n url(r'template/$', TemplateView.as_view()),\n url(r'do/$', TaskView.as_view()),\n url(r'transfer/$', TransferView.as_view()),\n]\n"
},
{
"path": "spug_api/apps/exec/views.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.views.generic import View\nfrom django_redis import get_redis_connection\nfrom django.conf import settings\nfrom libs import json_response, JsonParser, Argument, human_datetime, auth\nfrom apps.exec.models import ExecTemplate, ExecHistory\nfrom apps.host.models import Host\nfrom apps.account.utils import has_host_perm\nimport uuid\nimport json\n\n\nclass TemplateView(View):\n @auth('exec.template.view|exec.task.do|schedule.schedule.add|schedule.schedule.edit|\\\n monitor.monitor.add|monitor.monitor.edit')\n def get(self, request):\n templates = ExecTemplate.objects.all()\n types = [x['type'] for x in templates.order_by('type').values('type').distinct()]\n return json_response({'types': types, 'templates': [x.to_view() for x in templates]})\n\n @auth('exec.template.add|exec.template.edit')\n def post(self, request):\n form, error = JsonParser(\n Argument('id', type=int, required=False),\n Argument('name', help='请输入模版名称'),\n Argument('type', help='请选择模版类型'),\n Argument('body', help='请输入模版内容'),\n Argument('interpreter', default='sh'),\n Argument('host_ids', type=list, handler=json.dumps, default=[]),\n Argument('parameters', type=list, handler=json.dumps, default=[]),\n Argument('desc', required=False)\n ).parse(request.body)\n if error is None:\n if form.id:\n form.updated_at = human_datetime()\n form.updated_by = request.user\n ExecTemplate.objects.filter(pk=form.pop('id')).update(**form)\n else:\n form.created_by = request.user\n ExecTemplate.objects.create(**form)\n return json_response(error=error)\n\n @auth('exec.template.del')\n def delete(self, request):\n form, error = JsonParser(\n Argument('id', type=int, help='请指定操作对象')\n ).parse(request.GET)\n if error is None:\n ExecTemplate.objects.filter(pk=form.id).delete()\n return json_response(error=error)\n\n\nclass TaskView(View):\n @auth('exec.task.do')\n def get(self, request):\n records = ExecHistory.objects.filter(user=request.user).select_related('template')\n return json_response([x.to_view() for x in records])\n\n @auth('exec.task.do')\n def post(self, request):\n form, error = JsonParser(\n Argument('host_ids', type=list, filter=lambda x: len(x), help='请选择执行主机'),\n Argument('command', help='请输入执行命令内容'),\n Argument('interpreter', default='sh'),\n Argument('template_id', type=int, required=False),\n Argument('params', type=dict, handler=json.dumps, default={})\n ).parse(request.body)\n if error is None:\n if not has_host_perm(request.user, form.host_ids):\n return json_response(error='无权访问主机，请联系管理员')\n token, rds = uuid.uuid4().hex, get_redis_connection()\n form.host_ids.sort()\n if form.template_id:\n template = ExecTemplate.objects.filter(pk=form.template_id).first()\n if not template or template.body != form.command:\n form.template_id = None\n\n ExecHistory.objects.create(\n user=request.user,\n digest=token,\n interpreter=form.interpreter,\n template_id=form.template_id,\n command=form.command,\n host_ids=json.dumps(form.host_ids),\n params=form.params\n )\n return json_response(token)\n return json_response(error=error)\n\n @auth('exec.task.do')\n def patch(self, request):\n form, error = JsonParser(\n Argument('token', help='参数错误'),\n Argument('cols', type=int, required=False),\n Argument('rows', type=int, required=False)\n ).parse(request.body)\n if error is None:\n term = None\n if form.cols and form.rows:\n term = {'width': form.cols, 'height': form.rows}\n rds = get_redis_connection()\n task = ExecHistory.objects.get(digest=form.token)\n for host in Host.objects.filter(id__in=json.loads(task.host_ids)):\n data = dict(\n key=host.id,\n name=host.name,\n token=task.digest,\n interpreter=task.interpreter,\n hostname=host.hostname,\n port=host.port,\n username=host.username,\n command=task.command,\n pkey=host.private_key,\n params=json.loads(task.params),\n term=term\n )\n rds.rpush(settings.EXEC_WORKER_KEY, json.dumps(data))\n return json_response(error=error)\n\n\n\n"
},
{
"path": "spug_api/apps/file/__init__.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\n"
},
{
"path": "spug_api/apps/file/urls.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.urls import path\n\nfrom .views import *\n\nurlpatterns = [\n path('', FileView.as_view()),\n path('object/', ObjectView.as_view()),\n]\n"
},
{
"path": "spug_api/apps/file/utils.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.http import FileResponse\nimport stat\nimport time\nimport os\n\nKB = 1024\nMB = 1024 * 1024\nGB = 1024 * 1024 * 1024\nTB = 1024 * 1024 * 1024 * 1024\n\n\nclass FileResponseAfter(FileResponse):\n def __init__(self, callback, *args, **kwargs):\n super().__init__(*args, **kwargs)\n self.callback = callback\n\n def close(self):\n super().close()\n self.callback()\n\n\ndef parse_mode(obj):\n if obj.st_mode:\n mt = stat.S_IFMT(obj.st_mode)\n if mt == stat.S_IFIFO:\n kind = 'p'\n elif mt == stat.S_IFCHR:\n kind = 'c'\n elif mt == stat.S_IFDIR:\n kind = 'd'\n elif mt == stat.S_IFBLK:\n kind = 'b'\n elif mt == stat.S_IFREG:\n kind = '-'\n elif mt == stat.S_IFLNK:\n kind = 'l'\n elif mt == stat.S_IFSOCK:\n kind = 's'\n else:\n kind = '?'\n code = obj._rwx(\n (obj.st_mode & 448) >> 6, obj.st_mode & stat.S_ISUID\n )\n code += obj._rwx(\n (obj.st_mode & 56) >> 3, obj.st_mode & stat.S_ISGID\n )\n code += obj._rwx(\n obj.st_mode & 7, obj.st_mode & stat.S_ISVTX, True\n )\n return kind + code\n else:\n return '?---------'\n\n\ndef format_size(size):\n if size:\n if size < KB:\n return f'{size}B'\n if size < MB:\n return f'{size / KB:.1f}K'\n if size < GB:\n return f'{size / MB:.1f}M'\n if size < TB:\n return f'{size / GB:.1f}G'\n return f'{size / TB:.1f}T'\n else:\n return ''\n\n\ndef fetch_dir_list(host, path):\n with host.get_ssh() as ssh:\n objects = []\n for item in ssh.list_dir_attr(path):\n code = parse_mode(item)\n kind, is_link, name = '?', False, getattr(item, 'filename', '?')\n if stat.S_ISLNK(item.st_mode):\n is_link = True\n try:\n item = ssh.sftp_stat(os.path.join(path, name))\n except FileNotFoundError:\n pass\n if stat.S_ISREG(item.st_mode):\n kind = '-'\n elif stat.S_ISDIR(item.st_mode):\n kind = 'd'\n if (item.st_mtime is None) or (item.st_mtime == int(0xffffffff)):\n date = '(unknown date)'\n else:\n date = time.strftime('%Y/%m/%d %H:%M:%S', time.localtime(item.st_mtime))\n objects.append({\n 'name': name,\n 'size': '' if kind == 'd' else format_size(item.st_size or ''),\n 'date': date,\n 'kind': kind,\n 'code': code,\n 'is_link': is_link\n })\n return objects\n"
},
{
"path": "spug_api/apps/file/views.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.views.generic import View\nfrom django_redis import get_redis_connection\nfrom apps.host.models import Host\nfrom apps.account.utils import has_host_perm\nfrom apps.file.utils import FileResponseAfter, fetch_dir_list\nfrom libs import json_response, JsonParser, Argument, auth\nfrom functools import partial\nimport os\n\n\nclass FileView(View):\n @auth('host.console.list')\n def get(self, request):\n form, error = JsonParser(\n Argument('id', type=int, help='参数错误'),\n Argument('path', help='参数错误')\n ).parse(request.GET)\n if error is None:\n if not has_host_perm(request.user, form.id):\n return json_response(error='无权访问主机，请联系管理员')\n host = Host.objects.get(pk=form.id)\n if not host:\n return json_response(error='未找到指定主机')\n objects = fetch_dir_list(host, form.path)\n return json_response(objects)\n return json_response(error=error)\n\n\nclass ObjectView(View):\n @auth('host.console.list')\n def get(self, request):\n form, error = JsonParser(\n Argument('id', type=int, help='参数错误'),\n Argument('file', help='请输入文件路径')\n ).parse(request.GET)\n if error is None:\n if not has_host_perm(request.user, form.id):\n return json_response(error='无权访问主机，请联系管理员')\n host = Host.objects.filter(pk=form.id).first()\n if not host:\n return json_response(error='未找到指定主机')\n filename = os.path.basename(form.file)\n ssh_cli = host.get_ssh().get_client()\n sftp = ssh_cli.open_sftp()\n f = sftp.open(form.file)\n return FileResponseAfter(ssh_cli.close, f, as_attachment=True, filename=filename)\n return json_response(error=error)\n\n @auth('host.console.upload')\n def post(self, request):\n form, error = JsonParser(\n Argument('id', type=int, help='参数错误'),\n Argument('token', help='参数错误'),\n Argument('path', help='参数错误'),\n ).parse(request.POST)\n if error is None:\n if not has_host_perm(request.user, form.id):\n return json_response(error='无权访问主机，请联系管理员')\n file = request.FILES.get('file')\n if not file:\n return json_response(error='请选择要上传的文件')\n host = Host.objects.get(pk=form.id)\n if not host:\n return json_response(error='未找到指定主机')\n rds_cli = get_redis_connection()\n callback = partial(self._compute_progress, rds_cli, form.token, file.size)\n with host.get_ssh() as ssh:\n ssh.put_file_by_fl(file, f'{form.path}/{file.name}', callback=callback)\n return json_response(error=error)\n\n @auth('host.console.del')\n def delete(self, request):\n form, error = JsonParser(\n Argument('id', type=int, help='参数错误'),\n Argument('file', help='请输入文件路径')\n ).parse(request.GET)\n if error is None:\n if not has_host_perm(request.user, form.id):\n return json_response(error='无权访问主机，请联系管理员')\n host = Host.objects.get(pk=form.id)\n if not host:\n return json_response(error='未找到指定主机')\n with host.get_ssh() as ssh:\n ssh.remove_file(form.file)\n return json_response(error=error)\n\n def _compute_progress(self, rds_cli, token, total, value, *args):\n percent = '%.1f' % (value / total * 100)\n rds_cli.publish(token, percent)\n"
},
{
"path": "spug_api/apps/home/__init__.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\n"
},
{
"path": "spug_api/apps/home/models.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.db import models\nfrom libs.mixins import ModelMixin\nimport json\n\n\nclass Notice(models.Model, ModelMixin):\n title = models.CharField(max_length=100)\n content = models.TextField()\n is_stress = models.BooleanField(default=False)\n read_ids = models.TextField(default='[]')\n sort_id = models.IntegerField(default=0, db_index=True)\n created_at = models.DateTimeField(auto_now_add=True)\n\n def to_view(self):\n tmp = self.to_dict()\n tmp['read_ids'] = json.loads(self.read_ids)\n return tmp\n\n class Meta:\n db_table = 'notices'\n ordering = ('-sort_id',)\n\n\nclass Navigation(models.Model, ModelMixin):\n title = models.CharField(max_length=64)\n desc = models.CharField(max_length=128)\n logo = models.TextField()\n links = models.TextField()\n sort_id = models.IntegerField(default=0, db_index=True)\n created_at = models.DateTimeField(auto_now_add=True)\n\n def to_view(self):\n tmp = self.to_dict()\n tmp['links'] = json.loads(self.links)\n return tmp\n\n class Meta:\n db_table = 'navigations'\n ordering = ('-sort_id',)\n"
},
{
"path": "spug_api/apps/home/navigation.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.views.generic import View\nfrom libs import json_response, JsonParser, Argument\nfrom apps.home.models import Navigation\nimport json\n\n\nclass NavView(View):\n def get(self, request):\n navs = Navigation.objects.all()\n return json_response([x.to_view() for x in navs])\n\n def post(self, request):\n form, error = JsonParser(\n Argument('id', type=int, required=False),\n Argument('title', help='请输入导航标题'),\n Argument('desc', help='请输入导航描述'),\n Argument('logo', help='请上传导航logo'),\n Argument('links', type=list, filter=lambda x: len(x), help='请设置导航链接'),\n ).parse(request.body)\n if error is None:\n form.links = json.dumps(form.links)\n if form.id:\n Navigation.objects.filter(pk=form.id).update(**form)\n else:\n nav = Navigation.objects.create(**form)\n nav.sort_id = nav.id\n nav.save()\n return json_response(error=error)\n\n def patch(self, request):\n form, error = JsonParser(\n Argument('id', type=int, help='参数错误'),\n Argument('sort', filter=lambda x: x in ('up', 'down'), required=False),\n ).parse(request.body)\n if error is None:\n nav = Navigation.objects.filter(pk=form.id).first()\n if not nav:\n return json_response(error='未找到指定记录')\n if form.sort:\n if form.sort == 'up':\n tmp = Navigation.objects.filter(sort_id__gt=nav.sort_id).last()\n else:\n tmp = Navigation.objects.filter(sort_id__lt=nav.sort_id).first()\n if tmp:\n tmp.sort_id, nav.sort_id = nav.sort_id, tmp.sort_id\n tmp.save()\n nav.save()\n return json_response(error=error)\n\n def delete(self, request):\n form, error = JsonParser(\n Argument('id', type=int, help='参数错误')\n ).parse(request.GET)\n if error is None:\n Navigation.objects.filter(pk=form.id).delete()\n return json_response(error=error)\n"
},
{
"path": "spug_api/apps/home/notice.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.views.generic import View\nfrom libs import json_response, JsonParser, Argument\nfrom apps.home.models import Notice\nimport json\n\n\nclass NoticeView(View):\n def get(self, request):\n notices = Notice.objects.all()\n return json_response([x.to_view() for x in notices])\n\n def post(self, request):\n form, error = JsonParser(\n Argument('id', type=int, required=False),\n Argument('title', help='请输入标题'),\n Argument('content', help='请输入内容'),\n Argument('is_stress', type=bool, default=False),\n ).parse(request.body)\n if error is None:\n if form.is_stress:\n Notice.objects.update(is_stress=False)\n if form.id:\n Notice.objects.filter(pk=form.id).update(**form)\n else:\n notice = Notice.objects.create(**form)\n notice.sort_id = notice.id\n notice.save()\n return json_response(error=error)\n\n def patch(self, request):\n form, error = JsonParser(\n Argument('id', type=int, help='参数错误'),\n Argument('sort', filter=lambda x: x in ('up', 'down'), required=False),\n Argument('read', required=False)\n ).parse(request.body)\n if error is None:\n notice = Notice.objects.filter(pk=form.id).first()\n if not notice:\n return json_response(error='未找到指定记录')\n if form.sort:\n if form.sort == 'up':\n tmp = Notice.objects.filter(sort_id__gt=notice.sort_id).last()\n else:\n tmp = Notice.objects.filter(sort_id__lt=notice.sort_id).first()\n if tmp:\n tmp.sort_id, notice.sort_id = notice.sort_id, tmp.sort_id\n tmp.save()\n if form.read:\n read_ids = json.loads(notice.read_ids)\n read_ids.append(str(request.user.id))\n notice.read_ids = json.dumps(read_ids)\n notice.save()\n return json_response(error=error)\n\n def delete(self, request):\n form, error = JsonParser(\n Argument('id', type=int, help='参数错误')\n ).parse(request.GET)\n if error is None:\n Notice.objects.filter(pk=form.id).delete()\n return json_response(error=error)\n"
},
{
"path": "spug_api/apps/home/urls.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.urls import path\n\nfrom .views import *\nfrom apps.home.notice import NoticeView\nfrom apps.home.navigation import NavView\n\nurlpatterns = [\n path('statistic/', get_statistic),\n path('alarm/', get_alarm),\n path('deploy/', get_deploy),\n path('request/', get_request),\n path('notice/', NoticeView.as_view()),\n path('navigation/', NavView.as_view()),\n]\n"
},
{
"path": "spug_api/apps/home/views.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom apps.app.models import App\nfrom apps.host.models import Host\nfrom apps.schedule.models import Task\nfrom apps.monitor.models import Detection\nfrom apps.alarm.models import Alarm\nfrom apps.deploy.models import Deploy, DeployRequest\nfrom apps.account.utils import get_host_perms\nfrom libs.utils import json_response, human_date, parse_time\nfrom libs.parser import JsonParser, Argument\nfrom libs.decorators import auth\nfrom datetime import datetime, timedelta\nimport json\n\n\n@auth('dashboard.dashboard.view')\ndef get_statistic(request):\n if request.user.is_supper:\n app = App.objects.count()\n host = Host.objects.count()\n else:\n deploy_perms, host_perms = request.user.deploy_perms, get_host_perms(request.user)\n app = App.objects.filter(id__in=deploy_perms['apps']).count()\n host = len(host_perms)\n data = {\n 'app': app,\n 'host': host,\n 'task': Task.objects.count(),\n 'detection': Detection.objects.count()\n }\n return json_response(data)\n\n\n@auth('dashboard.dashboard.view')\ndef get_alarm(request):\n form, error = JsonParser(\n Argument('type', required=False),\n Argument('name', required=False)\n ).parse(request.GET, True)\n if error is None:\n now = datetime.now()\n data = {human_date(now - timedelta(days=x + 1)): 0 for x in range(14)}\n for alarm in Alarm.objects.filter(status='1', created_at__gt=human_date(now - timedelta(days=14)), **form):\n date = alarm.created_at[:10]\n if date in data:\n data[date] += 1\n data = [{'date': k, 'value': v} for k, v in data.items()]\n return json_response(data)\n return json_response(error=error)\n\n\n@auth('dashboard.dashboard.view')\ndef get_request(request):\n form, error = JsonParser(\n Argument('duration', type=list, help='参数错误')\n ).parse(request.body)\n if error is None:\n s_date = form.duration[0]\n e_date = (parse_time(form.duration[1]) + timedelta(days=1)).strftime('%Y-%m-%d')\n data = {x.id: {'name': x.name, 'count': 0} for x in App.objects.all()}\n for req in DeployRequest.objects.filter(created_at__gt=s_date, created_at__lt=e_date):\n data[req.deploy.app_id]['count'] += 1\n data = sorted(data.values(), key=lambda x: x['count'], reverse=True)[:20]\n return json_response(data)\n return json_response(error=error)\n\n\n@auth('dashboard.dashboard.view')\ndef get_deploy(request):\n host = Host.objects.filter(deleted_at__isnull=True).count()\n data = {x.id: {'name': x.name, 'count': 0} for x in App.objects.all()}\n for dep in Deploy.objects.all():\n data[dep.app_id]['count'] += len(json.loads(dep.host_ids))\n data = filter(lambda x: x['count'], data.values())\n return json_response({'host': host, 'res': list(data)})\n"
},
{
"path": "spug_api/apps/host/__init__.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\n"
},
{
"path": "spug_api/apps/host/add.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom libs import json_response, JsonParser, Argument, auth\nfrom apps.host.models import Host, HostExtend, Group\nfrom apps.host import utils\nimport json\n\n\n@auth('host.host.add')\ndef get_regions(request):\n form, error = JsonParser(\n Argument('type', filter=lambda x: x in ('ali', 'tencent'), help='参数错误'),\n Argument('ak', help='请输入AccessKey ID'),\n Argument('ac', help='请输入AccessKey Secret'),\n ).parse(request.GET)\n if error is None:\n response = []\n if form.type == 'ali':\n for item in utils.fetch_ali_regions(form.ak, form.ac):\n response.append({'id': item['RegionId'], 'name': item['LocalName']})\n else:\n for item in utils.fetch_tencent_regions(form.ak, form.ac):\n response.append({'id': item['Region'], 'name': item['RegionName']})\n return json_response(response)\n return json_response(error=error)\n\n\n@auth('host.host.add')\ndef cloud_import(request):\n form, error = JsonParser(\n Argument('type', filter=lambda x: x in ('ali', 'tencent'), help='参数错误'),\n Argument('ak', help='请输入AccessKey ID'),\n Argument('ac', help='请输入AccessKey Secret'),\n Argument('region_id', help='请选择区域'),\n Argument('group_id', type=int, help='请选择分组'),\n Argument('username', help='请输入默认SSH用户名'),\n Argument('port', type=int, help='请输入默认SSH端口号'),\n Argument('host_type', filter=lambda x: x in ('public', 'private'), help='请选择连接地址'),\n ).parse(request.body)\n if error is None:\n group = Group.objects.filter(pk=form.group_id).first()\n if not group:\n return json_response(error='未找到指定分组')\n if form.type == 'ali':\n instances = utils.fetch_ali_instances(form.ak, form.ac, form.region_id)\n else:\n instances = utils.fetch_tencent_instances(form.ak, form.ac, form.region_id)\n\n host_add_ids = []\n for item in instances:\n instance_id = item['instance_id']\n host_name = item.pop('instance_name')\n public_ips = item['public_ip_address'] or []\n private_ips = item['private_ip_address'] or []\n item['public_ip_address'] = json.dumps(public_ips)\n item['private_ip_address'] = json.dumps(private_ips)\n if HostExtend.objects.filter(instance_id=instance_id).exists():\n HostExtend.objects.filter(instance_id=instance_id).update(**item)\n else:\n if form.host_type == 'public':\n hostname = public_ips[0] if public_ips else ''\n else:\n hostname = private_ips[0] if private_ips else ''\n host = Host.objects.create(\n name=host_name,\n hostname=hostname,\n port=form.port,\n username=form.username,\n created_by=request.user)\n HostExtend.objects.create(host=host, **item)\n host_add_ids.append(host.id)\n if host_add_ids:\n group.hosts.add(*host_add_ids)\n return json_response(len(instances))\n return json_response(error=error)\n"
},
{
"path": "spug_api/apps/host/extend.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.views.generic import View\nfrom libs import json_response, JsonParser, Argument, human_datetime, auth\nfrom apps.host.models import Host, HostExtend\nfrom apps.host.utils import check_os_type, fetch_host_extend\nimport json\n\n\nclass ExtendView(View):\n @auth('host.host.add|host.host.edit')\n def get(self, request):\n form, error = JsonParser(\n Argument('host_id', type=int, help='参数错误')\n ).parse(request.GET)\n if error is None:\n host = Host.objects.filter(pk=form.host_id).first()\n if not host:\n return json_response(error='未找到指定主机')\n if not host.is_verified:\n return json_response(error='该主机还未验证')\n with host.get_ssh() as ssh:\n response = fetch_host_extend(ssh)\n return json_response(response)\n return json_response(error=error)\n\n @auth('host.host.add|host.host.edit')\n def post(self, request):\n form, error = JsonParser(\n Argument('host_id', type=int, help='参数错误'),\n Argument('instance_id', required=False),\n Argument('os_name', help='请输入操作系统'),\n Argument('cpu', type=int, help='请输入CPU核心数'),\n Argument('memory', type=float, help='请输入内存大小'),\n Argument('disk', type=list, filter=lambda x: len(x), help='请添加磁盘'),\n Argument('private_ip_address', type=list, filter=lambda x: len(x), help='请添加内网IP'),\n Argument('public_ip_address', type=list, required=False),\n Argument('instance_charge_type', default='Other'),\n Argument('internet_charge_type', default='Other'),\n Argument('created_time', required=False),\n Argument('expired_time', required=False)\n ).parse(request.body)\n if error is None:\n host = Host.objects.filter(pk=form.host_id).first()\n form.disk = json.dumps(form.disk)\n form.public_ip_address = json.dumps(form.public_ip_address) if form.public_ip_address else '[]'\n form.private_ip_address = json.dumps(form.private_ip_address)\n form.updated_at = human_datetime()\n form.os_type = check_os_type(form.os_name)\n if hasattr(host, 'hostextend'):\n extend = host.hostextend\n extend.update_by_dict(form)\n else:\n extend = HostExtend.objects.create(host=host, **form)\n return json_response(extend.to_view())\n return json_response(error=error)\n"
},
{
"path": "spug_api/apps/host/group.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.views.generic import View\nfrom django.db.models import F\nfrom libs import json_response, JsonParser, Argument, auth\nfrom apps.host.models import Group\nfrom apps.account.models import Role\n\n\ndef fetch_children(data, with_hosts):\n if data:\n sub_data = dict()\n for item in Group.objects.filter(parent_id__in=data.keys()):\n tmp = item.to_view(with_hosts)\n sub_data[item.id] = tmp\n data[item.parent_id]['children'].append(tmp)\n return fetch_children(sub_data, with_hosts)\n\n\ndef merge_children(data, prefix, childes):\n prefix = f'{prefix}/' if prefix else ''\n for item in childes:\n name = f'{prefix}{item[\"title\"]}'\n item['name'] = name\n if item.get('children'):\n merge_children(data, name, item['children'])\n else:\n data[item['key']] = name\n\n\ndef filter_by_perm(data, result, ids):\n for item in data:\n if 'children' in item:\n if item['key'] in ids:\n result.append(item)\n elif item['children']:\n filter_by_perm(item['children'], result, ids)\n\n\nclass GroupView(View):\n def get(self, request):\n with_hosts = request.GET.get('with_hosts')\n data, data2 = dict(), dict()\n for item in Group.objects.filter(parent_id=0):\n data[item.id] = item.to_view(with_hosts)\n fetch_children(data, with_hosts)\n if not data:\n grp = Group.objects.create(name='Default', sort_id=1)\n data[grp.id] = grp.to_view()\n if request.user.is_supper:\n tree_data = list(data.values())\n else:\n tree_data, ids = [], request.user.group_perms\n filter_by_perm(data.values(), tree_data, ids)\n merge_children(data2, '', tree_data)\n return json_response({'treeData': tree_data, 'groups': data2})\n\n @auth('admin')\n def post(self, request):\n form, error = JsonParser(\n Argument('id', type=int, required=False),\n Argument('parent_id', type=int, default=0),\n Argument('name', help='请输入分组名称')\n ).parse(request.body)\n if error is None:\n if form.id:\n Group.objects.filter(pk=form.id).update(name=form.name)\n else:\n group = Group.objects.create(**form)\n group.sort_id = group.id\n group.save()\n return json_response(error=error)\n\n @auth('admin')\n def patch(self, request):\n form, error = JsonParser(\n Argument('s_id', type=int, help='参数错误'),\n Argument('d_id', type=int, help='参数错误'),\n Argument('action', type=int, help='参数错误')\n ).parse(request.body)\n if error is None:\n src = Group.objects.get(pk=form.s_id)\n dst = Group.objects.get(pk=form.d_id)\n if form.action == 0:\n src.parent_id = dst.id\n dst = Group.objects.filter(parent_id=dst.id).first()\n if not dst:\n src.save()\n return json_response()\n form.action = -1\n src.parent_id = dst.parent_id\n if src.sort_id > dst.sort_id:\n if form.action == -1:\n dst = Group.objects.filter(sort_id__gt=dst.sort_id).last()\n Group.objects.filter(sort_id__lt=src.sort_id, sort_id__gte=dst.sort_id).update(sort_id=F('sort_id') + 1)\n else:\n if form.action == 1:\n dst = Group.objects.filter(sort_id__lt=dst.sort_id).first()\n Group.objects.filter(sort_id__lte=dst.sort_id, sort_id__gt=src.sort_id).update(sort_id=F('sort_id') - 1)\n src.sort_id = dst.sort_id\n src.save()\n return json_response(error=error)\n\n @auth('admin')\n def delete(self, request):\n form, error = JsonParser(\n Argument('id', type=int, help='参数错误')\n ).parse(request.GET)\n if error is None:\n group = Group.objects.filter(pk=form.id).first()\n if not group:\n return json_response(error='未找到指定分组')\n if Group.objects.filter(parent_id=group.id).exists():\n return json_response(error='请移除子分组后再尝试删除')\n if group.hosts.exists():\n return json_response(error='请移除分组下的主机后再尝试删除')\n if not Group.objects.exclude(pk=form.id).exists():\n return json_response(error='请至少保留一个分组')\n role = Role.objects.filter(group_perms__regex=fr'[^0-9]{form.id}[^0-9]').first()\n if role:\n return json_response(error=f'账户角色【{role.name}】的主机权限关联该分组，请解除关联后再尝试删除')\n group.delete()\n return json_response(error=error)\n"
},
{
"path": "spug_api/apps/host/models.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.db import models\nfrom libs import ModelMixin, human_datetime\nfrom apps.account.models import User\nfrom apps.setting.utils import AppSetting\nfrom libs.ssh import SSH\nimport json\n\n\nclass Host(models.Model, ModelMixin):\n name = models.CharField(max_length=100)\n hostname = models.CharField(max_length=50)\n port = models.IntegerField(null=True)\n username = models.CharField(max_length=50)\n pkey = models.TextField(null=True)\n desc = models.CharField(max_length=255, null=True)\n is_verified = models.BooleanField(default=False)\n created_at = models.CharField(max_length=20, default=human_datetime)\n created_by = models.ForeignKey(User, models.PROTECT, related_name='+')\n\n @property\n def private_key(self):\n return self.pkey or AppSetting.get('private_key')\n\n def get_ssh(self, pkey=None, default_env=None):\n pkey = pkey or self.private_key\n return SSH(self.hostname, self.port, self.username, pkey, default_env=default_env)\n\n def to_view(self):\n tmp = self.to_dict()\n if hasattr(self, 'hostextend'):\n tmp.update(self.hostextend.to_view())\n tmp['group_ids'] = []\n return tmp\n\n def __repr__(self):\n return '' % self.name\n\n class Meta:\n db_table = 'hosts'\n ordering = ('-id',)\n\n\nclass HostExtend(models.Model, ModelMixin):\n INSTANCE_CHARGE_TYPES = (\n ('PrePaid', '包年包月'),\n ('PostPaid', '按量计费'),\n ('Other', '其他')\n )\n INTERNET_CHARGE_TYPES = (\n ('PayByTraffic', '按流量计费'),\n ('PayByBandwidth', '按带宽计费'),\n ('Other', '其他')\n )\n host = models.OneToOneField(Host, on_delete=models.CASCADE)\n instance_id = models.CharField(max_length=64, null=True)\n zone_id = models.CharField(max_length=30, null=True)\n cpu = models.IntegerField()\n memory = models.FloatField()\n disk = models.CharField(max_length=255, default='[]')\n os_name = models.CharField(max_length=50)\n os_type = models.CharField(max_length=20)\n private_ip_address = models.CharField(max_length=255)\n public_ip_address = models.CharField(max_length=255)\n instance_charge_type = models.CharField(max_length=20, choices=INSTANCE_CHARGE_TYPES)\n internet_charge_type = models.CharField(max_length=20, choices=INTERNET_CHARGE_TYPES)\n created_time = models.CharField(max_length=20, null=True)\n expired_time = models.CharField(max_length=20, null=True)\n updated_at = models.CharField(max_length=20, default=human_datetime)\n\n def to_view(self):\n tmp = self.to_dict(excludes=('id',))\n tmp['disk'] = json.loads(self.disk)\n tmp['private_ip_address'] = json.loads(self.private_ip_address)\n tmp['public_ip_address'] = json.loads(self.public_ip_address)\n tmp['instance_charge_type_alias'] = self.get_instance_charge_type_display()\n tmp['internet_charge_type_alisa'] = self.get_internet_charge_type_display()\n return tmp\n\n class Meta:\n db_table = 'host_extend'\n\n\nclass Group(models.Model, ModelMixin):\n name = models.CharField(max_length=50)\n parent_id = models.IntegerField(default=0)\n sort_id = models.IntegerField(default=0)\n hosts = models.ManyToManyField(Host, related_name='groups')\n\n def to_view(self, with_hosts=False):\n response = dict(key=self.id, value=self.id, title=self.name, children=[])\n if with_hosts:\n def make_item(x):\n return dict(title=x.name, hostname=x.hostname, key=f'{self.id}_{x.id}', id=x.id, isLeaf=True)\n\n response['children'] = [make_item(x) for x in self.hosts.all()]\n return response\n\n class Meta:\n db_table = 'host_groups'\n ordering = ('-sort_id',)\n"
},
{
"path": "spug_api/apps/host/urls.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.urls import path\n\nfrom apps.host.views import *\nfrom apps.host.group import GroupView\nfrom apps.host.extend import ExtendView\nfrom apps.host.add import get_regions, cloud_import\n\nurlpatterns = [\n path('', HostView.as_view()),\n path('extend/', ExtendView.as_view()),\n path('group/', GroupView.as_view()),\n path('import/', post_import),\n path('import/cloud/', cloud_import),\n path('import/region/', get_regions),\n path('parse/', post_parse),\n path('valid/', batch_valid),\n]\n"
},
{
"path": "spug_api/apps/host/utils.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django_redis import get_redis_connection\nfrom libs.helper import make_ali_request, make_tencent_request\nfrom libs.ssh import SSH, AuthenticationException\nfrom libs.utils import AttrDict, human_datetime\nfrom libs.validators import ip_validator\nfrom apps.host.models import HostExtend\nfrom apps.setting.utils import AppSetting\nfrom collections import defaultdict\nfrom datetime import datetime, timezone\nfrom concurrent import futures\nimport ipaddress\nimport json\nimport math\nimport os\n\n\ndef check_os_type(os_name):\n os_name = os_name.lower()\n types = ('centos', 'coreos', 'debian', 'suse', 'ubuntu', 'windows', 'freebsd', 'tencent', 'alibaba', 'fedora')\n for t in types:\n if t in os_name:\n return t\n return 'unknown'\n\n\ndef check_instance_charge_type(value, supplier):\n if supplier == 'ali':\n if value in ('PrePaid', 'PostPaid'):\n return value\n else:\n return 'Other'\n if supplier == 'tencent':\n if value == 'PREPAID':\n return 'PrePaid'\n if value == 'POSTPAID_BY_HOUR':\n return 'PostPaid'\n return 'Other'\n\n\ndef check_internet_charge_type(value, supplier):\n if supplier == 'ali':\n if value in ('PayByTraffic', 'PayByBandwidth'):\n return value\n else:\n return 'Other'\n if supplier == 'tencent':\n if value == 'TRAFFIC_POSTPAID_BY_HOUR':\n return 'PayByTraffic'\n if value in ('BANDWIDTH_PREPAID', 'BANDWIDTH_POSTPAID_BY_HOUR'):\n return 'PayByBandwidth'\n return 'Other'\n\n\ndef parse_utc_date(value):\n if not value:\n return None\n s_format = '%Y-%m-%dT%H:%M:%SZ'\n if len(value) == 17:\n s_format = '%Y-%m-%dT%H:%MZ'\n date = datetime.strptime(value, s_format).replace(tzinfo=timezone.utc)\n return date.astimezone().strftime('%Y-%m-%d %H:%M:%S')\n\n\ndef fetch_ali_regions(ak, ac):\n params = dict(Action='DescribeRegions')\n res = make_ali_request(ak, ac, 'http://ecs.aliyuncs.com', params)\n if 'Regions' in res:\n return res['Regions']['Region']\n else:\n raise Exception(res)\n\n\ndef fetch_ali_disks(ak, ac, region_id, page_number=1):\n data, page_size = defaultdict(list), 20\n params = dict(\n Action='DescribeDisks',\n RegionId=region_id,\n PageNumber=page_number,\n PageSize=page_size\n )\n res = make_ali_request(ak, ac, 'http://ecs.aliyuncs.com', params)\n if 'Disks' in res:\n for item in res['Disks']['Disk']:\n data[item['InstanceId']].append(item['Size'])\n if len(res['Disks']['Disk']) == page_size:\n page_number += 1\n new_data = fetch_ali_disks(ak, ac, region_id, page_number)\n data.update(new_data)\n return data\n else:\n raise Exception(res)\n\n\ndef fetch_ali_instances(ak, ac, region_id, page_number=1):\n data, page_size = {}, 20\n params = dict(\n Action='DescribeInstances',\n RegionId=region_id,\n PageNumber=page_number,\n PageSize=page_size\n )\n res = make_ali_request(ak, ac, 'http://ecs.aliyuncs.com', params)\n if 'Instances' not in res:\n raise Exception(res)\n for item in res['Instances']['Instance']:\n if 'NetworkInterfaces' in item:\n network_interface = item['NetworkInterfaces']['NetworkInterface']\n else:\n network_interface = []\n data[item['InstanceId']] = dict(\n instance_id=item['InstanceId'],\n instance_name=item['InstanceName'],\n os_name=item['OSName'],\n os_type=check_os_type(item['OSName']),\n cpu=item['Cpu'],\n memory=item['Memory'] / 1024,\n created_time=parse_utc_date(item['CreationTime']),\n expired_time=parse_utc_date(item['ExpiredTime']),\n instance_charge_type=check_instance_charge_type(item['InstanceChargeType'], 'ali'),\n internet_charge_type=check_internet_charge_type(item['InternetChargeType'], 'ali'),\n public_ip_address=item['PublicIpAddress']['IpAddress'],\n private_ip_address=[x['PrimaryIpAddress'] for x in network_interface if x.get('PrimaryIpAddress')],\n zone_id=item['ZoneId']\n )\n if len(res['Instances']['Instance']) == page_size:\n new_data = fetch_ali_instances(ak, ac, region_id, page_number + 1)\n data.update(new_data)\n if page_number != 1:\n return data\n for instance_id, disk in fetch_ali_disks(ak, ac, region_id).items():\n if instance_id in data:\n data[instance_id]['disk'] = disk\n return list(data.values())\n\n\ndef fetch_tencent_regions(ak, ac):\n params = dict(Action='DescribeRegions')\n res = make_tencent_request(ak, ac, 'cvm.tencentcloudapi.com', params)\n if 'RegionSet' in res['Response']:\n return res['Response']['RegionSet']\n else:\n raise Exception(res)\n\n\ndef fetch_tencent_instances(ak, ac, region_id, page_number=1):\n data, page_size = [], 20\n params = dict(\n Action='DescribeInstances',\n Region=region_id,\n Offset=(page_number - 1) * page_size,\n Limit=page_size\n )\n res = make_tencent_request(ak, ac, 'cvm.tencentcloudapi.com', params)\n if 'InstanceSet' not in res['Response']:\n raise Exception(res)\n for item in res['Response']['InstanceSet']:\n data_disks = list(map(lambda x: x['DiskSize'], item['DataDisks']))\n internet_charge_type = item['InternetAccessible']['InternetChargeType']\n data.append(dict(\n instance_id=item['InstanceId'],\n instance_name=item['InstanceName'],\n os_name=item['OsName'],\n os_type=check_os_type(item['OsName']),\n cpu=item['CPU'],\n memory=item['Memory'],\n disk=[item['SystemDisk']['DiskSize']] + data_disks,\n created_time=parse_utc_date(item['CreatedTime']),\n expired_time=parse_utc_date(item['ExpiredTime']),\n instance_charge_type=check_instance_charge_type(item['InstanceChargeType'], 'tencent'),\n internet_charge_type=check_internet_charge_type(internet_charge_type, 'tencent'),\n public_ip_address=item['PublicIpAddresses'],\n private_ip_address=item['PrivateIpAddresses'],\n zone_id=item['Placement']['Zone']\n ))\n if len(res['Response']['InstanceSet']) == page_size:\n page_number += 1\n new_data = fetch_tencent_instances(ak, ac, region_id, page_number)\n data.extend(new_data)\n return data\n\n\ndef fetch_host_extend(ssh):\n public_ip_address = set()\n private_ip_address = set()\n response = {'disk': []}\n code, out = ssh.exec_command_raw('nproc')\n if code != 0:\n code, out = ssh.exec_command_raw(\"grep -c '^processor' /proc/cpuinfo\")\n if code == 0:\n response['cpu'] = int(out.strip())\n\n code, out = ssh.exec_command_raw(\"cat /etc/os-release | grep PRETTY_NAME | awk -F \\\\\\\" '{print $2}'\")\n if '/etc/os-release' in out:\n code, out = ssh.exec_command_raw(\"cat /etc/issue | head -1 | awk '{print $1,$2,$3}'\")\n if code == 0:\n response['os_name'] = out.strip()[:50]\n\n code, out = ssh.exec_command_raw('hostname -I')\n if code == 0:\n for ip in out.strip().split():\n if len(ip) > 15: # ignore ipv6\n continue\n if ipaddress.ip_address(ip).is_global:\n if len(public_ip_address) < 10:\n public_ip_address.add(ip)\n elif len(private_ip_address) < 10:\n private_ip_address.add(ip)\n\n ssh_hostname = ssh.arguments.get('hostname')\n if ip_validator(ssh_hostname):\n if ipaddress.ip_address(ssh_hostname).is_global:\n if ssh_hostname in public_ip_address:\n public_ip_address.remove(ssh_hostname)\n public_ip_address = [ssh_hostname] + list(public_ip_address)\n else:\n if ssh_hostname in private_ip_address:\n private_ip_address.remove(ssh_hostname)\n private_ip_address = [ssh_hostname] + list(private_ip_address)\n\n code, out = ssh.exec_command_raw('lsblk -dbn -o SIZE -e 11 2> /dev/null')\n if code == 0:\n disks = []\n for item in out.strip().splitlines():\n item = item.strip()\n size = math.ceil(int(item) / 1024 / 1024 / 1024)\n if size > 10:\n disks.append(size)\n response['disk'] = disks[:10]\n\n code, out = ssh.exec_command_raw(\"dmidecode -t 17 | grep -E 'Size: [0-9]+' | awk '{s+=$2} END {print s,$3}'\")\n if code == 0:\n fields = out.strip().split()\n if len(fields) == 2 and fields[1] in ('GB', 'MB'):\n size, unit = out.strip().split()\n if unit == 'GB':\n response['memory'] = size\n else:\n response['memory'] = round(int(size) / 1024, 0)\n if 'memory' not in response:\n code, out = ssh.exec_command_raw(\"cat /proc/meminfo | grep 'MemTotal' | awk '{print $2}'\")\n if code == 0:\n response['memory'] = math.ceil(int(out) / 1024 / 1024)\n\n response['public_ip_address'] = list(public_ip_address)\n response['private_ip_address'] = list(private_ip_address)\n return response\n\n\ndef batch_sync_host(token, hosts, password=None):\n private_key, public_key = AppSetting.get_ssh_key()\n threads, latest_exception, rds = [], None, get_redis_connection()\n max_workers = max(10, os.cpu_count() * 5)\n with futures.ThreadPoolExecutor(max_workers=max_workers) as executor:\n for host in hosts:\n if hasattr(host, 'password'):\n password = host.password\n t = executor.submit(_sync_host_extend, host, private_key, public_key, password)\n t.host = host\n threads.append(t)\n for t in futures.as_completed(threads):\n exception = t.exception()\n if exception:\n rds.rpush(token, json.dumps({'key': t.host.id, 'status': 'fail', 'message': f'{exception}'}))\n else:\n rds.rpush(token, json.dumps({'key': t.host.id, 'status': 'ok'}))\n t.host.is_verified = True\n t.host.save()\n rds.expire(token, 60)\n\n\ndef _sync_host_extend(host, private_key=None, public_key=None, password=None, ssh=None):\n if not ssh:\n kwargs = host.to_dict(selects=('hostname', 'port', 'username'))\n with _get_ssh(kwargs, host.pkey, private_key, public_key, password) as ssh:\n return _sync_host_extend(host, ssh=ssh)\n form = AttrDict(fetch_host_extend(ssh))\n form.disk = json.dumps(form.disk)\n form.public_ip_address = json.dumps(form.public_ip_address)\n form.private_ip_address = json.dumps(form.private_ip_address)\n form.updated_at = human_datetime()\n form.os_type = check_os_type(form.os_name)\n if hasattr(host, 'hostextend'):\n extend = host.hostextend\n extend.update_by_dict(form)\n else:\n extend = HostExtend.objects.create(host=host, **form)\n return extend\n\n\ndef _get_ssh(kwargs, pkey=None, private_key=None, public_key=None, password=None):\n try:\n ssh = SSH(pkey=pkey or private_key, **kwargs)\n ssh.get_client()\n return ssh\n except AuthenticationException as e:\n if password:\n with SSH(password=str(password), **kwargs) as ssh:\n ssh.add_public_key(public_key)\n return _get_ssh(kwargs, private_key)\n raise e\n"
},
{
"path": "spug_api/apps/host/views.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.views.generic import View\nfrom django.db.models import F\nfrom django.http.response import HttpResponseBadRequest\nfrom libs import json_response, JsonParser, Argument, AttrDict, auth\nfrom apps.setting.utils import AppSetting\nfrom apps.account.utils import get_host_perms\nfrom apps.host.models import Host, Group\nfrom apps.host.utils import batch_sync_host, _sync_host_extend\nfrom apps.exec.models import ExecTemplate\nfrom apps.app.models import Deploy\nfrom apps.schedule.models import Task\nfrom apps.monitor.models import Detection\nfrom libs.ssh import SSH, AuthenticationException\nfrom paramiko.ssh_exception import BadAuthenticationType\nfrom openpyxl import load_workbook\nfrom threading import Thread\nimport socket\nimport uuid\n\n\nclass HostView(View):\n def get(self, request):\n hosts = Host.objects.select_related('hostextend')\n if not request.user.is_supper:\n hosts = hosts.filter(id__in=get_host_perms(request.user))\n hosts = {x.id: x.to_view() for x in hosts}\n for rel in Group.hosts.through.objects.filter(host_id__in=hosts.keys()):\n hosts[rel.host_id]['group_ids'].append(rel.group_id)\n return json_response(list(hosts.values()))\n\n @auth('host.host.add|host.host.edit')\n def post(self, request):\n form, error = JsonParser(\n Argument('id', type=int, required=False),\n Argument('group_ids', type=list, filter=lambda x: len(x), help='请选择主机分组'),\n Argument('name', help='请输主机名称'),\n Argument('username', handler=str.strip, help='请输入登录用户名'),\n Argument('hostname', handler=str.strip, help='请输入主机名或IP'),\n Argument('port', type=int, help='请输入SSH端口'),\n Argument('pkey', required=False),\n Argument('desc', required=False),\n Argument('password', required=False),\n ).parse(request.body)\n if error is None:\n if not _do_host_verify(form):\n return json_response('auth fail')\n\n group_ids = form.pop('group_ids')\n other = Host.objects.filter(name=form.name).first()\n if other and (not form.id or other.id != form.id):\n return json_response(error=f'已存在的主机名称【{form.name}】')\n if form.id:\n Host.objects.filter(pk=form.id).update(is_verified=True, **form)\n host = Host.objects.get(pk=form.id)\n else:\n host = Host.objects.create(created_by=request.user, is_verified=True, **form)\n host.groups.set(group_ids)\n response = host.to_view()\n response['group_ids'] = group_ids\n return json_response(response)\n return json_response(error=error)\n\n @auth('host.host.add|host.host.edit')\n def put(self, request):\n form, error = JsonParser(\n Argument('id', type=int, help='参数错误')\n ).parse(request.body)\n if error is None:\n host = Host.objects.get(pk=form.id)\n with host.get_ssh() as ssh:\n _sync_host_extend(host, ssh=ssh)\n return json_response(error=error)\n\n @auth('admin')\n def patch(self, request):\n form, error = JsonParser(\n Argument('host_ids', type=list, filter=lambda x: len(x), help='请选择主机'),\n Argument('s_group_id', type=int, help='参数错误'),\n Argument('t_group_id', type=int, help='参数错误'),\n Argument('is_copy', type=bool, help='参数错误'),\n ).parse(request.body)\n if error is None:\n if form.t_group_id == form.s_group_id:\n return json_response(error='不能选择本分组的主机')\n s_group = Group.objects.get(pk=form.s_group_id)\n t_group = Group.objects.get(pk=form.t_group_id)\n t_group.hosts.add(*form.host_ids)\n if not form.is_copy:\n s_group.hosts.remove(*form.host_ids)\n return json_response(error=error)\n\n @auth('host.host.del')\n def delete(self, request):\n form, error = JsonParser(\n Argument('id', type=int, required=False),\n Argument('group_id', type=int, required=False),\n ).parse(request.GET)\n if error is None:\n if form.id:\n host_ids = [form.id]\n elif form.group_id:\n group = Group.objects.get(pk=form.group_id)\n host_ids = [x.id for x in group.hosts.all()]\n else:\n return json_response(error='参数错误')\n for host_id in host_ids:\n regex = fr'[^0-9]{host_id}[^0-9]'\n deploy = Deploy.objects.filter(host_ids__regex=regex) \\\n .annotate(app_name=F('app__name'), env_name=F('env__name')).first()\n if deploy:\n return json_response(error=f'应用【{deploy.app_name}】在【{deploy.env_name}】的发布配置关联了该主机，请解除关联后再尝试删除该主机')\n task = Task.objects.filter(targets__regex=regex).first()\n if task:\n return json_response(error=f'任务计划中的任务【{task.name}】关联了该主机，请解除关联后再尝试删除该主机')\n detection = Detection.objects.filter(type__in=('3', '4'), targets__regex=regex).first()\n if detection:\n return json_response(error=f'监控中心的任务【{detection.name}】关联了该主机，请解除关联后再尝试删除该主机')\n tpl = ExecTemplate.objects.filter(host_ids__regex=regex).first()\n if tpl:\n return json_response(error=f'执行模板【{tpl.name}】关联了该主机，请解除关联后再尝试删除该主机')\n Host.objects.filter(id__in=host_ids).delete()\n return json_response(error=error)\n\n\n@auth('host.host.add')\ndef post_import(request):\n group_id = request.POST.get('group_id')\n file = request.FILES['file']\n hosts = []\n ws = load_workbook(file, read_only=True)['Sheet1']\n summary = {'fail': 0, 'success': 0, 'invalid': [], 'skip': [], 'repeat': []}\n for i, row in enumerate(ws.rows, start=1):\n if i == 1: # 第1行是表头 略过\n continue\n if not all([row[x].value for x in range(4)]):\n summary['invalid'].append(i)\n summary['fail'] += 1\n continue\n data = AttrDict(\n name=row[0].value,\n hostname=row[1].value,\n port=row[2].value,\n username=row[3].value,\n desc=row[5].value\n )\n if Host.objects.filter(hostname=data.hostname, port=data.port, username=data.username).exists():\n summary['skip'].append(i)\n summary['fail'] += 1\n continue\n if Host.objects.filter(name=data.name).exists():\n summary['repeat'].append(i)\n summary['fail'] += 1\n continue\n host = Host.objects.create(created_by=request.user, **data)\n host.groups.add(group_id)\n summary['success'] += 1\n host.password = row[4].value\n hosts.append(host)\n token = uuid.uuid4().hex\n if hosts:\n Thread(target=batch_sync_host, args=(token, hosts)).start()\n return json_response({'summary': summary, 'token': token, 'hosts': {x.id: {'name': x.name} for x in hosts}})\n\n\n@auth('host.host.add')\ndef post_parse(request):\n file = request.FILES['file']\n if file:\n data = file.read()\n return json_response(data.decode())\n else:\n return HttpResponseBadRequest()\n\n\n@auth('host.host.add')\ndef batch_valid(request):\n form, error = JsonParser(\n Argument('password', required=False),\n Argument('range', filter=lambda x: x in ('1', '2'), help='参数错误')\n ).parse(request.body)\n if error is None:\n if form.range == '1': # all hosts\n hosts = Host.objects.all()\n else:\n hosts = Host.objects.filter(is_verified=False).all()\n token = uuid.uuid4().hex\n Thread(target=batch_sync_host, args=(token, hosts, form.password)).start()\n return json_response({'token': token, 'hosts': {x.id: {'name': x.name} for x in hosts}})\n return json_response(error=error)\n\n\ndef _do_host_verify(form):\n password = form.pop('password')\n if form.pkey:\n try:\n with SSH(form.hostname, form.port, form.username, form.pkey) as ssh:\n ssh.ping()\n return True\n except BadAuthenticationType:\n raise Exception('该主机不支持密钥认证，请参考官方文档，错误代码：E01')\n except AuthenticationException:\n raise Exception('上传的独立密钥认证失败，请检查该密钥是否能正常连接主机（推荐使用全局密钥）')\n except socket.timeout:\n raise Exception('连接主机超时，请检查网络')\n\n private_key, public_key = AppSetting.get_ssh_key()\n if password:\n try:\n with SSH(form.hostname, form.port, form.username, password=password) as ssh:\n ssh.add_public_key(public_key)\n except BadAuthenticationType:\n raise Exception('该主机不支持密码认证，请参考官方文档，错误代码：E00')\n except AuthenticationException:\n raise Exception('密码连接认证失败，请检查密码是否正确')\n except socket.timeout:\n raise Exception('连接主机超时，请检查网络')\n\n try:\n with SSH(form.hostname, form.port, form.username, private_key) as ssh:\n ssh.ping()\n except BadAuthenticationType:\n raise Exception('该主机不支持密钥认证，请参考官方文档，错误代码：E01')\n except AuthenticationException:\n if password:\n raise Exception('密钥认证失败，请参考官方文档，错误代码：E02')\n return False\n except socket.timeout:\n raise Exception('连接主机超时，请检查网络')\n return True\n"
},
{
"path": "spug_api/apps/monitor/__init__.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\n"
},
{
"path": "spug_api/apps/monitor/executors.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django_redis import get_redis_connection\nfrom apps.host.models import Host\nfrom apps.monitor.utils import handle_notify\nfrom socket import socket\nimport subprocess\nimport platform\nimport requests\nimport logging\nimport json\nimport time\nimport re\n\nlogging.captureWarnings(True)\nregex = re.compile(r'Failed to establish a new connection: (.*)\\'\\)+')\n\n\ndef site_check(url, limit):\n try:\n res = requests.get(url, timeout=30)\n if limit:\n duration = int(res.elapsed.total_seconds() * 1000)\n if duration > int(limit):\n return False, f'响应时间 {duration}ms 大于 {limit}ms'\n return 200 <= res.status_code < 400, f'返回HTTP状态码 {res.status_code}'\n except Exception as e:\n error = e.__str__()\n exps = re.findall(regex, error)\n if exps:\n error = exps[0]\n return False, error\n\n\ndef port_check(addr, port):\n try:\n sock = socket()\n sock.settimeout(5)\n sock.connect((addr, int(port)))\n sock.close()\n return True, '端口状态检测正常'\n except Exception as e:\n return False, f'异常信息：{e}'\n\n\ndef ping_check(addr):\n try:\n if platform.system().lower() == 'windows':\n command = f'ping -n 1 -w 3000 {addr}'\n else:\n command = f'ping -c 1 -W 3 {addr}'\n task = subprocess.run(command, shell=True, stdout=subprocess.PIPE)\n if task.returncode == 0:\n return True, 'Ping检测正常'\n else:\n return False, 'Ping检测失败'\n except Exception as e:\n return False, f'异常信息：{e}'\n\n\ndef host_executor(host, command):\n try:\n with host.get_ssh() as ssh:\n exit_code, out = ssh.exec_command_raw(command)\n if exit_code == 0:\n return True, out or '检测状态正常'\n else:\n return False, out or f'退出状态码：{exit_code}'\n except Exception as e:\n return False, f'异常信息：{e}'\n\n\ndef monitor_worker_handler(job):\n task_id, tp, addr, extra, threshold, quiet = json.loads(job)\n target = addr\n if tp == '1':\n is_ok, message = site_check(addr, extra)\n elif tp == '2':\n is_ok, message = port_check(addr, extra)\n elif tp == '5':\n is_ok, message = ping_check(addr)\n elif tp not in ('3', '4'):\n is_ok, message = False, f'invalid monitor type for {tp!r}'\n else:\n command = f'ps -ef|grep -v grep|grep {extra!r}' if tp == '3' else extra\n host = Host.objects.filter(pk=addr).first()\n if not host:\n is_ok, message = False, f'unknown host id for {addr!r}'\n else:\n is_ok, message = host_executor(host, command)\n target = f'{host.name}({host.hostname})'\n\n rds, key, f_count, f_time = get_redis_connection(), f'spug:det:{task_id}', f'c_{addr}', f't_{addr}'\n v_count, v_time = rds.hmget(key, f_count, f_time)\n if is_ok:\n if v_count:\n rds.hdel(key, f_count, f_time)\n if v_time:\n logging.warning('send recovery notification')\n handle_notify(task_id, target, is_ok, message, int(v_count) + 1)\n return\n v_count = rds.hincrby(key, f_count)\n if v_count >= threshold:\n if not v_time or int(time.time()) - int(v_time) >= quiet * 60:\n rds.hset(key, f_time, int(time.time()))\n logging.warning('send fault alarm notification')\n handle_notify(task_id, target, is_ok, message, v_count)\n\n\ndef dispatch(tp, addr, extra):\n if tp == '1':\n return site_check(addr, extra)\n elif tp == '2':\n return port_check(addr, extra)\n elif tp == '5':\n return ping_check(addr)\n elif tp == '3':\n command = f'ps -ef|grep -v grep|grep {extra!r}'\n elif tp == '4':\n command = extra\n else:\n raise TypeError(f'invalid monitor type: {tp!r}')\n host = Host.objects.filter(pk=addr).first()\n return host_executor(host, command)\n"
},
{
"path": "spug_api/apps/monitor/management/commands/runmonitor.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.core.management.base import BaseCommand\nfrom apps.monitor.scheduler import Scheduler\nimport logging\n\nlogging.basicConfig(level=logging.WARNING, format='%(asctime)s %(message)s')\n\n\nclass Command(BaseCommand):\n help = 'Start monitor process'\n\n def handle(self, *args, **options):\n s = Scheduler()\n s.run()\n"
},
{
"path": "spug_api/apps/monitor/models.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.db import models\nfrom libs import ModelMixin, human_datetime\nfrom apps.account.models import User\nimport json\n\n\nclass Detection(models.Model, ModelMixin):\n TYPES = (\n ('1', '站点检测'),\n ('2', '端口检测'),\n ('3', '进程检测'),\n ('4', '自定义脚本'),\n ('5', 'Ping检测'),\n )\n STATUS = (\n (0, '正常'),\n (1, '异常'),\n )\n name = models.CharField(max_length=50)\n type = models.CharField(max_length=2, choices=TYPES)\n group = models.CharField(max_length=255, null=True)\n targets = models.TextField()\n extra = models.TextField(null=True)\n desc = models.CharField(max_length=255, null=True)\n is_active = models.BooleanField(default=True)\n rate = models.IntegerField(default=5)\n threshold = models.IntegerField(default=3)\n quiet = models.IntegerField(default=24 * 60)\n fault_times = models.SmallIntegerField(default=0)\n notify_mode = models.CharField(max_length=255)\n notify_grp = models.CharField(max_length=255)\n latest_run_time = models.CharField(max_length=20, null=True)\n\n created_at = models.CharField(max_length=20, default=human_datetime)\n created_by = models.ForeignKey(User, models.PROTECT, related_name='+')\n updated_at = models.CharField(max_length=20, null=True)\n updated_by = models.ForeignKey(User, models.PROTECT, related_name='+', null=True)\n\n def to_view(self):\n tmp = self.to_dict()\n tmp['type_alias'] = self.get_type_display()\n tmp['notify_mode'] = json.loads(self.notify_mode)\n tmp['notify_grp'] = json.loads(self.notify_grp)\n tmp['targets'] = json.loads(self.targets)\n return tmp\n\n def __repr__(self):\n return '' % self.name\n\n class Meta:\n db_table = 'detections'\n ordering = ('-id',)\n"
},
{
"path": "spug_api/apps/monitor/scheduler.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom apscheduler.schedulers.background import BackgroundScheduler\nfrom apscheduler.executors.pool import ThreadPoolExecutor\nfrom apscheduler.triggers.interval import IntervalTrigger\nfrom django_redis import get_redis_connection\nfrom django.conf import settings\nfrom django.db import connections\nfrom django.db.utils import DatabaseError\nfrom apps.monitor.models import Detection\nfrom libs import AttrDict, human_datetime\nfrom datetime import datetime, timedelta\nfrom random import randint\nimport logging\nimport json\n\nMONITOR_WORKER_KEY = settings.MONITOR_WORKER_KEY\n\n\nclass Scheduler:\n timezone = settings.TIME_ZONE\n\n def __init__(self):\n self.scheduler = BackgroundScheduler(timezone=self.timezone, executors={'default': ThreadPoolExecutor(30)})\n\n def _dispatch(self, task_id, tp, targets, extra, threshold, quiet):\n Detection.objects.filter(pk=task_id).update(latest_run_time=human_datetime())\n rds_cli = get_redis_connection()\n for t in json.loads(targets):\n rds_cli.rpush(MONITOR_WORKER_KEY, json.dumps([task_id, tp, t, extra, threshold, quiet]))\n connections.close_all()\n\n def _init(self):\n self.scheduler.start()\n try:\n for item in Detection.objects.filter(is_active=True):\n now = datetime.now()\n trigger = IntervalTrigger(minutes=int(item.rate), timezone=self.timezone)\n self.scheduler.add_job(\n self._dispatch,\n trigger,\n id=str(item.id),\n args=(item.id, item.type, item.targets, item.extra, item.threshold, item.quiet),\n next_run_time=now + timedelta(seconds=randint(0, 60))\n )\n connections.close_all()\n except DatabaseError:\n pass\n\n def run(self):\n rds_cli = get_redis_connection()\n self._init()\n rds_cli.delete(settings.MONITOR_KEY)\n logging.warning('Running monitor')\n while True:\n _, data = rds_cli.brpop(settings.MONITOR_KEY)\n task = AttrDict(json.loads(data))\n if task.action in ('add', 'modify'):\n trigger = IntervalTrigger(minutes=int(task.rate), timezone=self.timezone)\n self.scheduler.add_job(\n self._dispatch,\n trigger,\n id=str(task.id),\n args=(task.id, task.type, task.targets, task.extra, task.threshold, task.quiet),\n replace_existing=True\n )\n elif task.action == 'remove':\n job = self.scheduler.get_job(str(task.id))\n if job:\n job.remove()\n"
},
{
"path": "spug_api/apps/monitor/urls.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.urls import path\n\nfrom .views import *\n\nurlpatterns = [\n path('', DetectionView.as_view()),\n path('overview/', get_overview),\n path('test/', run_test),\n]\n"
},
{
"path": "spug_api/apps/monitor/utils.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.db import close_old_connections\nfrom apps.alarm.models import Alarm\nfrom apps.monitor.models import Detection\nfrom libs.spug import Notification\nimport json\n\n\ndef seconds_to_human(seconds):\n text = ''\n if seconds > 3600:\n text = f'{int(seconds / 3600)}小时'\n seconds = seconds % 3600\n if seconds > 60:\n text += f'{int(seconds / 60)}分钟'\n seconds = seconds % 60\n if seconds:\n text += f'{seconds}秒'\n return text\n\n\ndef _record_alarm(det, target, duration, status):\n Alarm.objects.create(\n name=det.name,\n type=det.get_type_display(),\n target=target,\n status=status,\n duration=duration,\n notify_grp=det.notify_grp,\n notify_mode=det.notify_mode)\n\n\ndef handle_notify(task_id, target, is_ok, out, fault_times):\n close_old_connections()\n det = Detection.objects.get(pk=task_id)\n duration = seconds_to_human(det.rate * fault_times * 60)\n event = '2' if is_ok else '1'\n _record_alarm(det, target, duration, event)\n grp = json.loads(det.notify_grp)\n notify = Notification(grp, event, target, det.name, out, duration)\n notify.dispatch_monitor(json.loads(det.notify_mode))\n"
},
{
"path": "spug_api/apps/monitor/views.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.views.generic import View\nfrom django.conf import settings\nfrom django_redis import get_redis_connection\nfrom libs import json_response, JsonParser, Argument, human_datetime, auth\nfrom apps.monitor.models import Detection\nfrom apps.monitor.executors import dispatch\nfrom apps.setting.utils import AppSetting\nfrom datetime import datetime\nimport json\n\n\nclass DetectionView(View):\n @auth('dashboard.dashboard.view|monitor.monitor.view')\n def get(self, request):\n detections = Detection.objects.all()\n groups = [x['group'] for x in detections.order_by('group').values('group').distinct()]\n return json_response({'groups': groups, 'detections': [x.to_view() for x in detections]})\n\n @auth('monitor.monitor.add|monitor.monitor.edit')\n def post(self, request):\n form, error = JsonParser(\n Argument('id', type=int, required=False),\n Argument('name', help='请输入任务名称'),\n Argument('group', help='请选择任务分组'),\n Argument('targets', type=list, filter=lambda x: len(x), help='请输入监控地址'),\n Argument('type', filter=lambda x: x in dict(Detection.TYPES), help='请选择监控类型'),\n Argument('extra', required=False),\n Argument('desc', required=False),\n Argument('rate', type=int, default=5),\n Argument('threshold', type=int, default=3),\n Argument('quiet', type=int, default=24 * 60),\n Argument('notify_grp', type=list, help='请选择报警联系组'),\n Argument('notify_mode', type=list, help='请选择报警方式'),\n ).parse(request.body)\n if error is None:\n if set(form.notify_mode).intersection(['1', '2', '6']):\n if not AppSetting.get_default('spug_push_key'):\n return json_response(error='报警方式微信、短信、电话需要配置推送服务（系统设置/推送服务设置），请配置后再启用该报警方式。')\n\n form.targets = json.dumps(form.targets)\n form.notify_grp = json.dumps(form.notify_grp)\n form.notify_mode = json.dumps(form.notify_mode)\n if form.id:\n Detection.objects.filter(pk=form.id).update(\n updated_at=human_datetime(),\n updated_by=request.user,\n **form)\n task = Detection.objects.filter(pk=form.id).first()\n if task and task.is_active:\n form.action = 'modify'\n rds_cli = get_redis_connection()\n rds_cli.lpush(settings.MONITOR_KEY, json.dumps(form))\n else:\n dtt = Detection.objects.create(created_by=request.user, **form)\n form.action = 'add'\n form.id = dtt.id\n rds_cli = get_redis_connection()\n rds_cli.lpush(settings.MONITOR_KEY, json.dumps(form))\n return json_response(error=error)\n\n @auth('monitor.monitor.edit')\n def patch(self, request):\n form, error = JsonParser(\n Argument('id', type=int, help='请指定操作对象'),\n Argument('is_active', type=bool, required=False)\n ).parse(request.body, True)\n if error is None:\n Detection.objects.filter(pk=form.id).update(**form)\n if form.get('is_active') is not None:\n if form.is_active:\n task = Detection.objects.filter(pk=form.id).first()\n message = {'id': form.id, 'action': 'add'}\n message.update(task.to_dict(selects=('targets', 'extra', 'rate', 'type', 'threshold', 'quiet')))\n else:\n message = {'id': form.id, 'action': 'remove'}\n rds_cli = get_redis_connection()\n rds_cli.lpush(settings.MONITOR_KEY, json.dumps(message))\n return json_response(error=error)\n\n @auth('monitor.monitor.del')\n def delete(self, request):\n form, error = JsonParser(\n Argument('id', type=int, help='请指定操作对象')\n ).parse(request.GET)\n if error is None:\n task = Detection.objects.filter(pk=form.id).first()\n if task:\n if task.is_active:\n return json_response(error='该监控项正在运行中，请先停止后再尝试删除')\n task.delete()\n return json_response(error=error)\n\n\n@auth('monitor.monitor.add|monitor.monitor.edit')\ndef run_test(request):\n form, error = JsonParser(\n Argument('type', help='请选择监控类型'),\n Argument('targets', type=list, filter=lambda x: len(x), help='请输入监控地址'),\n Argument('extra', required=False)\n ).parse(request.body)\n if error is None:\n is_success, message = dispatch(form.type, form.targets[0], form.extra)\n return json_response({'is_success': is_success, 'message': message})\n return json_response(error=error)\n\n\n@auth('monitor.monitor.view')\ndef get_overview(request):\n response = []\n rds = get_redis_connection()\n for item in Detection.objects.all():\n data = {}\n for key in json.loads(item.targets):\n key = str(key)\n data[key] = {\n 'id': f'{item.id}_{key}',\n 'group': item.group,\n 'name': item.name,\n 'type': item.get_type_display(),\n 'target': key,\n 'desc': item.desc,\n 'status': '0',\n 'latest_run_time': item.latest_run_time,\n }\n if item.is_active:\n if item.latest_run_time:\n data[key]['status'] = '1'\n else:\n data[key]['status'] = '10'\n if item.is_active:\n for key, val in rds.hgetall(f'spug:det:{item.id}').items():\n prefix, key = key.decode().split('_', 1)\n if key in data:\n val = int(val)\n if prefix == 'c':\n if data[key]['status'] == '1':\n data[key]['status'] = '2'\n data[key]['count'] = val\n elif prefix == 't':\n date = datetime.fromtimestamp(val).strftime('%Y-%m-%d %H:%M:%S')\n data[key].update(status='3', notified_at=date)\n response.extend(list(data.values()))\n return json_response(response)\n"
},
{
"path": "spug_api/apps/notify/__init__.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\n"
},
{
"path": "spug_api/apps/notify/models.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.db import models\nfrom django.core.cache import cache\nfrom libs import ModelMixin, human_datetime\nfrom libs.channel import Channel\nimport hashlib\n\n\nclass Notify(models.Model, ModelMixin):\n TYPES = (\n ('1', '通知'),\n ('2', '待办'),\n )\n SOURCES = (\n ('monitor', '监控中心'),\n ('schedule', '任务计划'),\n ('flag', '应用发布'),\n ('alert', '系统警告'),\n )\n title = models.CharField(max_length=255)\n source = models.CharField(max_length=10, choices=SOURCES)\n type = models.CharField(max_length=2, choices=TYPES)\n content = models.TextField(null=True)\n unread = models.BooleanField(default=True)\n link = models.CharField(max_length=255, null=True)\n\n created_at = models.CharField(max_length=20, default=human_datetime)\n\n @classmethod\n def make_system_notify(cls, title, content):\n cls._make_notify('alert', '1', title, content)\n\n @classmethod\n def make_monitor_notify(cls, title, content):\n cls._make_notify('monitor', '1', title, content)\n\n @classmethod\n def make_schedule_notify(cls, title, content):\n cls._make_notify('schedule', '1', title, content)\n\n @classmethod\n def make_deploy_notify(cls, title, content):\n cls._make_notify('flag', '1', title, content)\n\n @classmethod\n def _make_notify(cls, source, type, title, content):\n tmp_str = f'{source},{type},{title},{content}'\n digest = hashlib.md5(tmp_str.encode()).hexdigest()\n unique_key = f'spug:notify:{digest}'\n if not cache.get(unique_key): # 限制相同内容的发送频率\n cache.set(unique_key, 1, 3600)\n cls.objects.create(source=source, title=title, type=type, content=content)\n Channel.send_notify(title, content)\n\n def __repr__(self):\n return '' % self.title\n\n class Meta:\n db_table = 'notifies'\n ordering = ('-id',)\n"
},
{
"path": "spug_api/apps/notify/urls.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.urls import path\n\nfrom .views import *\n\nurlpatterns = [\n path('', NotifyView.as_view()),\n]\n"
},
{
"path": "spug_api/apps/notify/views.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.views.generic import View\nfrom apps.notify.models import Notify\nfrom libs import json_response, JsonParser, Argument\n\n\nclass NotifyView(View):\n def get(self, request):\n notifies = Notify.objects.all()\n return json_response(notifies)\n\n def patch(self, request):\n form, error = JsonParser(\n Argument('ids', type=list, help='参数错误')\n ).parse(request.body)\n if error is None:\n Notify.objects.filter(id__in=form.ids).update(unread=False)\n return json_response(error=error)\n"
},
{
"path": "spug_api/apps/repository/__init__.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\n"
},
{
"path": "spug_api/apps/repository/models.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.db import models\nfrom django.conf import settings\nfrom libs.mixins import ModelMixin\nfrom apps.app.models import App, Environment, Deploy\nfrom apps.account.models import User\nfrom datetime import datetime\nimport json\nimport os\n\n\nclass Repository(models.Model, ModelMixin):\n STATUS = (\n ('0', '未开始'),\n ('1', '构建中'),\n ('2', '失败'),\n ('5', '成功'),\n )\n app = models.ForeignKey(App, on_delete=models.PROTECT)\n env = models.ForeignKey(Environment, on_delete=models.PROTECT)\n deploy = models.ForeignKey(Deploy, on_delete=models.PROTECT)\n version = models.CharField(max_length=100)\n spug_version = models.CharField(max_length=50)\n remarks = models.CharField(max_length=255, null=True)\n extra = models.TextField()\n status = models.CharField(max_length=2, choices=STATUS, default='0')\n created_at = models.DateTimeField(auto_now_add=True)\n created_by = models.ForeignKey(User, on_delete=models.PROTECT)\n\n @staticmethod\n def make_spug_version(deploy_id):\n return f'{deploy_id}_{datetime.now().strftime(\"%Y%m%d%H%M%S\")}'\n\n def to_view(self):\n tmp = self.to_dict()\n tmp['extra'] = json.loads(self.extra)\n tmp['status_alias'] = self.get_status_display()\n if hasattr(self, 'app_name'):\n tmp['app_name'] = self.app_name\n if hasattr(self, 'env_name'):\n tmp['env_name'] = self.env_name\n if hasattr(self, 'created_by_user'):\n tmp['created_by_user'] = self.created_by_user\n return tmp\n\n def delete(self, using=None, keep_parents=False):\n super().delete(using, keep_parents)\n try:\n build_file = f'{self.spug_version}.tar.gz'\n os.remove(os.path.join(settings.BUILD_DIR, build_file))\n except FileNotFoundError:\n pass\n\n class Meta:\n db_table = 'repositories'\n ordering = ('-id',)\n"
},
{
"path": "spug_api/apps/repository/urls.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.urls import path\n\nfrom .views import *\n\nurlpatterns = [\n path('', RepositoryView.as_view()),\n path('/', get_detail),\n path('request/', get_requests),\n]\n"
},
{
"path": "spug_api/apps/repository/utils.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django_redis import get_redis_connection\nfrom django.conf import settings\nfrom django.db import close_old_connections\nfrom libs.utils import AttrDict, human_time, render_str\nfrom apps.repository.models import Repository\nfrom apps.app.utils import fetch_repo\nfrom apps.config.utils import compose_configs\nfrom apps.deploy.helper import Helper\nimport json\nimport uuid\nimport os\n\nREPOS_DIR = settings.REPOS_DIR\nBUILD_DIR = settings.BUILD_DIR\n\n\ndef dispatch(rep: Repository, helper=None):\n rep.status = '1'\n alone_build = helper is None\n if not helper:\n rds = get_redis_connection()\n rds_key = f'{settings.BUILD_KEY}:{rep.spug_version}'\n helper = Helper.make(rds, rds_key)\n rep.save()\n try:\n api_token = uuid.uuid4().hex\n helper.rds.setex(api_token, 60 * 60, f'{rep.app_id},{rep.env_id}')\n helper.send_info('local', f'\\033[32m完成√\\033[0m\\r\\n{human_time()} 构建准备... ')\n env = AttrDict(\n SPUG_APP_NAME=rep.app.name,\n SPUG_APP_KEY=rep.app.key,\n SPUG_APP_ID=str(rep.app_id),\n SPUG_DEPLOY_ID=str(rep.deploy_id),\n SPUG_BUILD_ID=str(rep.id),\n SPUG_ENV_ID=str(rep.env_id),\n SPUG_ENV_KEY=rep.env.key,\n SPUG_VERSION=rep.version,\n SPUG_BUILD_VERSION=rep.spug_version,\n SPUG_API_TOKEN=api_token,\n SPUG_REPOS_DIR=REPOS_DIR,\n )\n # append configs\n configs = compose_configs(rep.app, rep.env_id)\n configs_env = {f'_SPUG_{k.upper()}': v for k, v in configs.items()}\n env.update(configs_env)\n\n _build(rep, helper, env)\n rep.status = '5'\n except Exception as e:\n rep.status = '2'\n raise e\n finally:\n helper.local(f'cd {REPOS_DIR} && rm -rf {rep.spug_version}')\n close_old_connections()\n if alone_build:\n helper.clear()\n rep.save()\n return rep\n elif rep.status == '5':\n rep.save()\n\n\ndef _build(rep: Repository, helper, env):\n extend = rep.deploy.extend_obj\n extras = json.loads(rep.extra)\n git_dir = os.path.join(REPOS_DIR, str(rep.deploy_id))\n build_dir = os.path.join(REPOS_DIR, rep.spug_version)\n tar_file = os.path.join(BUILD_DIR, f'{rep.spug_version}.tar.gz')\n if extras[0] == 'branch':\n tree_ish = extras[2]\n env.update(SPUG_GIT_BRANCH=extras[1], SPUG_GIT_COMMIT_ID=extras[2])\n else:\n tree_ish = extras[1]\n env.update(SPUG_GIT_TAG=extras[1])\n env.update(SPUG_DST_DIR=render_str(extend.dst_dir, env))\n fetch_repo(rep.deploy_id, extend.git_repo)\n helper.send_info('local', '\\033[32m完成√\\033[0m\\r\\n')\n\n if extend.hook_pre_server:\n helper.send_step('local', 1, f'{human_time()} 检出前任务...\\r\\n')\n helper.local(f'cd {git_dir} && {extend.hook_pre_server}', env)\n\n helper.send_step('local', 2, f'{human_time()} 执行检出... ')\n command = f'cd {git_dir} && git archive --prefix={rep.spug_version}/ {tree_ish} | (cd .. && tar xf -)'\n helper.local(command)\n helper.send_info('local', '\\033[32m完成√\\033[0m\\r\\n')\n\n if extend.hook_post_server:\n helper.send_step('local', 3, f'{human_time()} 检出后任务...\\r\\n')\n helper.local(f'cd {build_dir} && {extend.hook_post_server}', env)\n\n helper.send_step('local', 4, f'\\r\\n{human_time()} 执行打包... ')\n filter_rule, exclude, contain = json.loads(extend.filter_rule), '', rep.spug_version\n files = helper.parse_filter_rule(filter_rule['data'], env=env)\n if files:\n if filter_rule['type'] == 'exclude':\n excludes = []\n for x in files:\n if x.startswith('/'):\n excludes.append(f'--exclude={rep.spug_version}{x}')\n else:\n excludes.append(f'--exclude={x}')\n exclude = ' '.join(excludes)\n else:\n contain = ' '.join(f'{rep.spug_version}/{x}' for x in files)\n helper.local(f'mkdir -p {BUILD_DIR} && cd {REPOS_DIR} && tar zcf {tar_file} {exclude} {contain}')\n helper.send_step('local', 5, f'\\033[32m完成√\\033[0m')\n helper.send_step('local', 100, f'\\r\\n\\r\\n{human_time()} ** \\033[32m构建成功\\033[0m **')\n"
},
{
"path": "spug_api/apps/repository/views.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.views.generic import View\nfrom django.db.models import F\nfrom django.conf import settings\nfrom django_redis import get_redis_connection\nfrom libs import json_response, JsonParser, Argument, human_time, AttrDict, auth\nfrom apps.repository.models import Repository\nfrom apps.deploy.models import DeployRequest\nfrom apps.repository.utils import dispatch\nfrom apps.app.models import Deploy\nfrom threading import Thread\nimport json\n\n\nclass RepositoryView(View):\n @auth('deploy.repository.view|deploy.request.add|deploy.request.edit')\n def get(self, request):\n apps = request.user.deploy_perms['apps']\n deploy_id = request.GET.get('deploy_id')\n data = Repository.objects.filter(app_id__in=apps).annotate(\n app_name=F('app__name'),\n env_name=F('env__name'),\n created_by_user=F('created_by__nickname'))\n if deploy_id:\n data = data.filter(deploy_id=deploy_id, status='5')\n return json_response([x.to_view() for x in data])\n\n response = dict()\n for item in data:\n if item.app_id in response:\n response[item.app_id]['child'].append(item.to_view())\n else:\n tmp = item.to_view()\n tmp['child'] = [item.to_view()]\n response[item.app_id] = tmp\n return json_response(list(response.values()))\n\n @auth('deploy.repository.add')\n def post(self, request):\n form, error = JsonParser(\n Argument('deploy_id', type=int, help='参数错误'),\n Argument('version', help='请输入构建版本'),\n Argument('extra', type=list, help='参数错误'),\n Argument('remarks', required=False)\n ).parse(request.body)\n if error is None:\n deploy = Deploy.objects.filter(pk=form.deploy_id).first()\n if not deploy:\n return json_response(error='未找到指定发布配置')\n form.extra = json.dumps(form.extra)\n form.spug_version = Repository.make_spug_version(deploy.id)\n rep = Repository.objects.create(\n app_id=deploy.app_id,\n env_id=deploy.env_id,\n created_by=request.user,\n **form)\n Thread(target=dispatch, args=(rep,)).start()\n return json_response(rep.to_view())\n return json_response(error=error)\n\n @auth('deploy.repository.add|deploy.repository.build')\n def patch(self, request):\n form, error = JsonParser(\n Argument('id', type=int, help='参数错误'),\n Argument('action', help='参数错误')\n ).parse(request.body)\n if error is None:\n rep = Repository.objects.filter(pk=form.id).first()\n if not rep:\n return json_response(error='未找到指定构建记录')\n if form.action == 'rebuild':\n Thread(target=dispatch, args=(rep,)).start()\n return json_response(rep.to_view())\n return json_response(error=error)\n\n @auth('deploy.repository.del')\n def delete(self, request):\n form, error = JsonParser(\n Argument('id', type=int, help='请指定操作对象')\n ).parse(request.GET)\n if error is None:\n repository = Repository.objects.filter(pk=form.id).first()\n if not repository:\n return json_response(error='未找到指定构建记录')\n if repository.deployrequest_set.exists():\n return json_response(error='已关联发布申请无法删除')\n repository.delete()\n return json_response(error=error)\n\n\n@auth('deploy.repository.view')\ndef get_requests(request):\n form, error = JsonParser(\n Argument('repository_id', type=int, help='参数错误')\n ).parse(request.GET)\n if error is None:\n requests = []\n for item in DeployRequest.objects.filter(repository_id=form.repository_id):\n data = item.to_dict(selects=('id', 'name', 'created_at'))\n data['host_ids'] = json.loads(item.host_ids)\n data['status_alias'] = item.get_status_display()\n requests.append(data)\n return json_response(requests)\n\n\n@auth('deploy.repository.view')\ndef get_detail(request, r_id):\n repository = Repository.objects.filter(pk=r_id).first()\n if not repository:\n return json_response(error='未找到指定构建记录')\n rds, counter = get_redis_connection(), 0\n if repository.remarks == 'SPUG AUTO MAKE':\n req = repository.deployrequest_set.last()\n key = f'{settings.REQUEST_KEY}:{req.id}'\n else:\n key = f'{settings.BUILD_KEY}:{repository.spug_version}'\n data = rds.lrange(key, counter, counter + 9)\n response = AttrDict(data='', step=0, s_status='process', status=repository.status)\n while data:\n for item in data:\n counter += 1\n item = json.loads(item.decode())\n if item['key'] == 'local':\n if 'data' in item:\n response.data += item['data']\n if 'step' in item:\n response.step = item['step']\n if 'status' in item:\n response.status = item['status']\n data = rds.lrange(key, counter, counter + 9)\n response.index = counter\n if repository.status in ('0', '1'):\n response.data = f'{human_time()} 建立连接... ' + response.data\n elif not response.data:\n response.data = f'{human_time()} 读取数据... \\r\\n\\r\\n未读取到数据，Spug 仅保存最近2周的构建日志。'\n else:\n response.data = f'{human_time()} 读取数据... ' + response.data\n return json_response(response)\n"
},
{
"path": "spug_api/apps/schedule/__init__.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\n"
},
{
"path": "spug_api/apps/schedule/builtin.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.db import connections\nfrom django.conf import settings\nfrom apps.account.models import History, User\nfrom apps.alarm.models import Alarm\nfrom apps.schedule.models import Task, History as TaskHistory\nfrom apps.deploy.models import DeployRequest\nfrom apps.app.models import DeployExtend1\nfrom apps.exec.models import ExecHistory, Transfer\nfrom apps.notify.models import Notify\nfrom apps.deploy.utils import dispatch\nfrom apps.repository.models import Repository\nfrom libs.utils import parse_time, human_datetime, human_date\nfrom datetime import datetime, timedelta\nfrom threading import Thread\nfrom collections import defaultdict\nfrom pathlib import Path\nimport time\nimport os\n\n\ndef auto_run_by_day():\n try:\n date_7 = human_date(datetime.now() - timedelta(days=7))\n date_30 = human_date(datetime.now() - timedelta(days=30))\n History.objects.filter(created_at__lt=date_30).delete()\n Notify.objects.filter(created_at__lt=date_7, unread=False).delete()\n Alarm.objects.filter(created_at__lt=date_30).delete()\n for item in DeployExtend1.objects.all():\n index = 0\n for req in DeployRequest.objects.filter(deploy_id=item.deploy_id, repository_id__isnull=False):\n if index > item.versions and req.repository_id:\n req.repository.delete()\n index += 1\n\n timer = defaultdict(int)\n for item in ExecHistory.objects.all():\n if timer[item.user_id] >= 10:\n item.delete()\n else:\n timer[item.user_id] += 1\n\n timer = defaultdict(int)\n for item in Transfer.objects.all():\n if timer[item.user_id] >= 10:\n item.delete()\n else:\n timer[item.user_id] += 1\n\n for task in Task.objects.all():\n try:\n record = TaskHistory.objects.filter(task_id=task.id)[50]\n TaskHistory.objects.filter(task_id=task.id, id__lt=record.id).delete()\n except IndexError:\n pass\n\n timestamp = time.time() - 2 * 3600\n for item in Path(settings.TRANSFER_DIR).iterdir():\n if item.name != '.gitkeep':\n if item.stat().st_atime < timestamp:\n transfer_dir = item.absolute()\n os.system(f'umount -f {transfer_dir} &> /dev/null ; rm -rf {transfer_dir}')\n finally:\n connections.close_all()\n\n\ndef auto_run_by_minute():\n try:\n now = datetime.now()\n for req in DeployRequest.objects.filter(status='2'):\n if (now - parse_time(req.do_at)).seconds > 3600:\n req.status = '-3'\n req.save()\n\n for rep in Repository.objects.filter(status='1'):\n if (now - parse_time(rep.created_at)).seconds > 3600:\n rep.status = '2'\n rep.save()\n\n for req in DeployRequest.objects.filter(status='1', plan__lte=now):\n req.status = '2'\n req.do_at = human_datetime()\n req.do_by = req.created_by\n req.save()\n Thread(target=dispatch, args=(req,)).start()\n finally:\n connections.close_all()\n"
},
{
"path": "spug_api/apps/schedule/executors.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom libs.ssh import AuthenticationException\nfrom django.db import close_old_connections, transaction\nfrom apps.host.models import Host\nfrom apps.schedule.models import History, Task\nfrom apps.schedule.utils import send_fail_notify\nimport subprocess\nimport socket\nimport time\nimport json\n\n\ndef local_executor(command):\n code, out, now = 1, None, time.time()\n task = subprocess.Popen(command, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)\n try:\n code = task.wait(3600)\n out = task.stdout.read() + task.stderr.read()\n out = out.decode()\n except subprocess.TimeoutExpired:\n # task.kill()\n out = 'timeout, wait more than 1 hour'\n return code, round(time.time() - now, 3), out\n\n\ndef host_executor(host, command):\n code, out, now = 1, None, time.time()\n try:\n with host.get_ssh() as ssh:\n code, out = ssh.exec_command_raw(command)\n except AuthenticationException:\n out = 'ssh authentication fail'\n except socket.error as e:\n out = f'network error {e}'\n return code, round(time.time() - now, 3), out\n\n\ndef dispatch_job(host_id, interpreter, command):\n if interpreter == 'python':\n attach = 'INTERPRETER=python\\ncommand -v python3 &> /dev/null && INTERPRETER=python3'\n command = f'{attach}\\n$INTERPRETER << EOF\\n# -*- coding: UTF-8 -*-\\n{command}\\nEOF'\n if host_id == 'local':\n code, duration, out = local_executor(command)\n else:\n host = Host.objects.filter(pk=host_id).first()\n if not host:\n code, duration, out = 1, 0, f'unknown host id for {host_id!r}'\n else:\n code, duration, out = host_executor(host, command)\n return code, duration, out\n\n\ndef schedule_worker_handler(job):\n history_id, host_id, interpreter, command = json.loads(job)\n code, duration, out = dispatch_job(host_id, interpreter, command)\n\n close_old_connections()\n with transaction.atomic():\n history = History.objects.select_for_update().get(pk=history_id)\n output = json.loads(history.output)\n output[str(host_id)] = [code, duration, out]\n history.output = json.dumps(output)\n if all(output.values()):\n history.status = '1' if sum(x[0] for x in output.values()) == 0 else '2'\n history.save()\n if history.status == '2':\n task = Task.objects.get(pk=history.task_id)\n send_fail_notify(task)\n"
},
{
"path": "spug_api/apps/schedule/management/commands/runscheduler.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.core.management.base import BaseCommand\nfrom apps.schedule.scheduler import Scheduler\nimport logging\n\nlogging.basicConfig(level=logging.WARNING, format='%(asctime)s %(message)s')\n\n\nclass Command(BaseCommand):\n help = 'Start schedule process'\n\n def handle(self, *args, **options):\n s = Scheduler()\n s.run()\n"
},
{
"path": "spug_api/apps/schedule/models.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.db import models\nfrom libs import ModelMixin, human_datetime\nfrom apps.account.models import User\nimport json\n\n\nclass History(models.Model, ModelMixin):\n STATUS = (\n (0, '执行中'),\n (1, '成功'),\n (2, '失败'),\n )\n task_id = models.IntegerField()\n status = models.SmallIntegerField(choices=STATUS)\n run_time = models.CharField(max_length=20)\n output = models.TextField()\n\n def to_list(self):\n tmp = super().to_dict(selects=('id', 'status', 'run_time'))\n tmp['status_alias'] = self.get_status_display()\n return tmp\n\n class Meta:\n db_table = 'task_histories'\n ordering = ('-id',)\n\n\nclass Task(models.Model, ModelMixin):\n TRIGGERS = (\n ('date', '一次性'),\n ('calendarinterval', '日历间隔'),\n ('cron', 'UNIX cron'),\n ('interval', '普通间隔')\n )\n name = models.CharField(max_length=50)\n type = models.CharField(max_length=50)\n interpreter = models.CharField(max_length=20, default='sh')\n command = models.TextField()\n targets = models.TextField()\n trigger = models.CharField(max_length=20, choices=TRIGGERS)\n trigger_args = models.CharField(max_length=255)\n is_active = models.BooleanField(default=False)\n desc = models.CharField(max_length=255, null=True)\n latest = models.ForeignKey(History, on_delete=models.PROTECT, null=True)\n rst_notify = models.CharField(max_length=255, null=True)\n\n created_at = models.CharField(max_length=20, default=human_datetime)\n created_by = models.ForeignKey(User, models.PROTECT, related_name='+')\n updated_at = models.CharField(max_length=20, null=True)\n updated_by = models.ForeignKey(User, models.PROTECT, related_name='+', null=True)\n\n def to_dict(self, *args, **kwargs):\n tmp = super().to_dict(*args, **kwargs)\n tmp['targets'] = json.loads(self.targets)\n tmp['latest_status'] = self.latest.status if self.latest else None\n tmp['latest_run_time'] = self.latest.run_time if self.latest else None\n tmp['latest_status_alias'] = self.latest.get_status_display() if self.latest else None\n tmp['rst_notify'] = json.loads(self.rst_notify) if self.rst_notify else {'mode': '0'}\n if self.trigger == 'cron':\n tmp['trigger_args'] = json.loads(self.trigger_args)\n return tmp\n\n def __repr__(self):\n return '' % self.name\n\n class Meta:\n db_table = 'tasks'\n ordering = ('-id',)\n"
},
{
"path": "spug_api/apps/schedule/scheduler.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom apscheduler.schedulers.background import BackgroundScheduler\nfrom apscheduler.executors.pool import ThreadPoolExecutor\nfrom apscheduler.triggers.interval import IntervalTrigger\nfrom apscheduler.triggers.date import DateTrigger\nfrom apscheduler.triggers.cron import CronTrigger\nfrom django_redis import get_redis_connection\nfrom django.db import connections\nfrom django.db.utils import DatabaseError\nfrom apps.schedule.models import Task, History\nfrom apps.schedule.builtin import auto_run_by_day, auto_run_by_minute\nfrom django.conf import settings\nfrom libs import AttrDict, human_datetime\nimport logging\nimport json\n\nSCHEDULE_WORKER_KEY = settings.SCHEDULE_WORKER_KEY\n\n\nclass Scheduler:\n timezone = settings.TIME_ZONE\n week_map = {\n '/': '/',\n '-': '-',\n ',': ',',\n '*': '*',\n '7': '6',\n '0': '6',\n '1': '0',\n '2': '1',\n '3': '2',\n '4': '3',\n '5': '4',\n '6': '5',\n }\n\n def __init__(self):\n self.scheduler = BackgroundScheduler(timezone=self.timezone, executors={'default': ThreadPoolExecutor(30)})\n\n @classmethod\n def covert_week(cls, week_str):\n return ''.join(map(lambda x: cls.week_map[x], week_str))\n\n @classmethod\n def parse_trigger(cls, trigger, trigger_args):\n if trigger == 'interval':\n return IntervalTrigger(seconds=int(trigger_args), timezone=cls.timezone)\n elif trigger == 'date':\n return DateTrigger(run_date=trigger_args, timezone=cls.timezone)\n elif trigger == 'cron':\n args = json.loads(trigger_args) if not isinstance(trigger_args, dict) else trigger_args\n minute, hour, day, month, week = args['rule'].split()\n week = cls.covert_week(week)\n return CronTrigger(minute=minute, hour=hour, day=day, month=month, day_of_week=week,\n start_date=args['start'], end_date=args['stop'])\n else:\n raise TypeError(f'unknown schedule policy: {trigger!r}')\n\n def _init_builtin_jobs(self):\n self.scheduler.add_job(auto_run_by_day, 'cron', hour=1, minute=20)\n self.scheduler.add_job(auto_run_by_minute, 'interval', minutes=1)\n\n def _dispatch(self, task_id, interpreter, command, targets):\n output = {x: None for x in targets}\n history = History.objects.create(\n task_id=task_id,\n status='0',\n run_time=human_datetime(),\n output=json.dumps(output)\n )\n Task.objects.filter(pk=task_id).update(latest_id=history.id)\n rds_cli = get_redis_connection()\n for t in targets:\n rds_cli.rpush(SCHEDULE_WORKER_KEY, json.dumps([history.id, t, interpreter, command]))\n connections.close_all()\n\n def _init(self):\n self.scheduler.start()\n self._init_builtin_jobs()\n try:\n for task in Task.objects.filter(is_active=True):\n trigger = self.parse_trigger(task.trigger, task.trigger_args)\n self.scheduler.add_job(\n self._dispatch,\n trigger,\n id=str(task.id),\n args=(task.id, task.interpreter, task.command, json.loads(task.targets)),\n )\n connections.close_all()\n except DatabaseError:\n pass\n\n def run(self):\n rds_cli = get_redis_connection()\n self._init()\n rds_cli.delete(settings.SCHEDULE_KEY)\n logging.warning('Running scheduler')\n while True:\n _, data = rds_cli.brpop(settings.SCHEDULE_KEY)\n task = AttrDict(json.loads(data))\n if task.action in ('add', 'modify'):\n trigger = self.parse_trigger(task.trigger, task.trigger_args)\n self.scheduler.add_job(\n self._dispatch,\n trigger,\n id=str(task.id),\n args=(task.id, task.interpreter, task.command, task.targets),\n replace_existing=True\n )\n elif task.action == 'remove':\n job = self.scheduler.get_job(str(task.id))\n if job:\n job.remove()\n"
},
{
"path": "spug_api/apps/schedule/urls.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.urls import path\n\nfrom .views import *\n\nurlpatterns = [\n path('', Schedule.as_view()),\n path('/', HistoryView.as_view()),\n path('run_time/', next_run_time),\n]\n"
},
{
"path": "spug_api/apps/schedule/utils.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom libs.utils import human_datetime\nfrom libs.spug import Notification\nfrom libs.push import push_server\nfrom apps.setting.utils import AppSetting\nimport json\n\n\ndef send_fail_notify(task, msg=None):\n rst_notify = json.loads(task.rst_notify)\n mode = rst_notify.get('mode')\n url = rst_notify.get('value')\n if mode != '0' and url:\n _do_notify(task, mode, url, msg)\n\n\ndef _do_notify(task, mode, url, msg):\n if mode == '1':\n texts = [\n '## 任务执行失败通知 ## ',\n f'**任务名称：** {task.name} ',\n f'**任务类型：** {task.type} ',\n f'**描述信息：** {msg or \"请在任务计划执行历史中查看详情\"} ',\n f'**发生时间：** {human_datetime()} ',\n '> 来自 Spug运维平台'\n ]\n data = {\n 'msgtype': 'markdown',\n 'markdown': {\n 'title': '任务执行失败通知',\n 'text': '\\n\\n'.join(texts)\n },\n 'at': {\n 'isAtAll': True\n }\n }\n Notification.handle_request(url, data, 'dd')\n elif mode == '2':\n data = {\n 'task_id': task.id,\n 'task_name': task.name,\n 'task_type': task.type,\n 'message': msg or '请在任务计划执行历史中查看详情',\n 'created_at': human_datetime()\n }\n Notification.handle_request(url, data)\n elif mode == '3':\n texts = [\n '## 任务执行失败通知',\n f'任务名称： {task.name}',\n f'任务类型： {task.type}',\n f'描述信息： {msg or \"请在任务计划执行历史中查看详情\"}',\n f'发生时间： {human_datetime()}',\n '> 来自 Spug运维平台'\n ]\n data = {\n 'msgtype': 'markdown',\n 'markdown': {\n 'content': '\\n'.join(texts)\n }\n }\n Notification.handle_request(url, data, 'wx')\n elif mode == '4':\n data = {\n 'msg_type': 'post',\n 'content': {\n 'post': {\n 'zh_cn': {\n 'title': '任务执行失败通知',\n 'content': [\n [{'tag': 'text', 'text': f'任务名称： {task.name}'}],\n [{'tag': 'text', 'text': f'任务类型： {task.type}'}],\n [{'tag': 'text', 'text': f'描述信息： {msg or \"请在任务计划执行历史中查看详情\"}'}],\n [{'tag': 'text', 'text': f'发生时间： {human_datetime()}'}],\n [{'tag': 'at', 'user_id': 'all'}],\n ]\n }\n }\n }\n }\n Notification.handle_request(url, data, 'fs')\n elif mode == '5':\n spug_push_key = AppSetting.get_default('spug_push_key')\n if not spug_push_key:\n return\n data = {\n 'source': 'schedule',\n 'token': spug_push_key,\n 'targets': url,\n 'dataset': {\n 'name': task.name,\n 'type': task.type,\n 'message': msg or '请在任务计划执行历史中查看详情',\n }\n }\n Notification.handle_request(f'{push_server}/spug/message/', data, 'spug')\n"
},
{
"path": "spug_api/apps/schedule/views.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.views.generic import View\nfrom django_redis import get_redis_connection\nfrom apscheduler.schedulers.background import BackgroundScheduler\nfrom apscheduler.triggers.cron import CronTrigger\nfrom apps.schedule.scheduler import Scheduler\nfrom apps.schedule.models import Task, History\nfrom apps.schedule.executors import dispatch_job\nfrom apps.host.models import Host\nfrom django.conf import settings\nfrom libs import json_response, JsonParser, Argument, human_datetime, auth\nimport json\n\n\nclass Schedule(View):\n @auth('schedule.schedule.view')\n def get(self, request):\n tasks = Task.objects.all()\n types = [x['type'] for x in tasks.order_by('type').values('type').distinct()]\n return json_response({'types': types, 'tasks': [x.to_dict() for x in tasks]})\n\n @auth('schedule.schedule.add|schedule.schedule.edit')\n def post(self, request):\n form, error = JsonParser(\n Argument('id', type=int, required=False),\n Argument('type', help='请输入任务类型'),\n Argument('name', help='请输入任务名称'),\n Argument('interpreter', help='请选择执行解释器'),\n Argument('command', help='请输入任务内容'),\n Argument('rst_notify', type=dict, help='请选择执行失败通知方式'),\n Argument('targets', type=list, filter=lambda x: len(x), help='请选择执行对象'),\n Argument('trigger', filter=lambda x: x in dict(Task.TRIGGERS), help='请选择触发器类型'),\n Argument('trigger_args', help='请输入触发器参数'),\n Argument('desc', required=False),\n ).parse(request.body)\n if error is None:\n form.targets = json.dumps(form.targets)\n form.rst_notify = json.dumps(form.rst_notify)\n if form.trigger == 'cron':\n args = json.loads(form.trigger_args)['rule'].split()\n if len(args) != 5:\n return json_response(error='无效的执行规则，请更正后再试')\n minute, hour, day, month, week = args\n week = '0' if week == '7' else week\n try:\n CronTrigger(minute=minute, hour=hour, day=day, month=month, day_of_week=week)\n except ValueError:\n return json_response(error='无效的执行规则，请更正后再试')\n if form.id:\n Task.objects.filter(pk=form.id).update(\n updated_at=human_datetime(),\n updated_by=request.user,\n **form\n )\n task = Task.objects.filter(pk=form.id).first()\n if task and task.is_active:\n form.action = 'modify'\n form.targets = json.loads(form.targets)\n rds_cli = get_redis_connection()\n rds_cli.lpush(settings.SCHEDULE_KEY, json.dumps(form))\n else:\n Task.objects.create(created_by=request.user, **form)\n return json_response(error=error)\n\n @auth('schedule.schedule.edit')\n def patch(self, request):\n form, error = JsonParser(\n Argument('id', type=int, help='请指定操作对象'),\n Argument('is_active', type=bool, required=False)\n ).parse(request.body, True)\n if error is None:\n task = Task.objects.get(pk=form.id)\n if form.get('is_active') is not None:\n task.is_active = form.is_active\n task.latest_id = None\n if form.is_active:\n message = {'id': form.id, 'action': 'add'}\n message.update(task.to_dict(selects=('interpreter', 'trigger', 'trigger_args', 'command', 'targets')))\n else:\n message = {'id': form.id, 'action': 'remove'}\n rds_cli = get_redis_connection()\n rds_cli.lpush(settings.SCHEDULE_KEY, json.dumps(message))\n task.save()\n return json_response(error=error)\n\n @auth('schedule.schedule.del')\n def delete(self, request):\n form, error = JsonParser(\n Argument('id', type=int, help='请指定操作对象')\n ).parse(request.GET)\n if error is None:\n task = Task.objects.filter(pk=form.id).first()\n if task:\n if task.is_active:\n return json_response(error='该任务在运行中，请先停止任务再尝试删除')\n task.delete()\n History.objects.filter(task_id=task.id).delete()\n return json_response(error=error)\n\n\nclass HistoryView(View):\n @auth('schedule.schedule.view')\n def get(self, request, t_id):\n task = Task.objects.filter(pk=t_id).first()\n if not task:\n return json_response(error='未找到指定任务')\n\n h_id = request.GET.get('id')\n if h_id:\n h_id = task.latest_id if h_id == 'latest' else h_id\n return json_response(self._fetch_detail(h_id))\n histories = History.objects.filter(task_id=t_id)\n return json_response([x.to_list() for x in histories])\n\n @auth('schedule.schedule.edit')\n def post(self, request, t_id):\n task = Task.objects.filter(pk=t_id).first()\n if not task:\n return json_response(error='未找到指定任务')\n outputs, status = {}, 1\n for host_id in json.loads(task.targets):\n code, duration, out = dispatch_job(host_id, task.interpreter, task.command)\n if code != 0:\n status = 2\n outputs[host_id] = [code, duration, out]\n\n history = History.objects.create(\n task_id=task.id,\n status=status,\n run_time=human_datetime(),\n output=json.dumps(outputs)\n )\n return json_response(history.id)\n\n def _fetch_detail(self, h_id):\n record = History.objects.filter(pk=h_id).first()\n outputs = json.loads(record.output)\n host_ids = (x for x in outputs.keys() if x != 'local')\n hosts_info = {str(x.id): x.name for x in Host.objects.filter(id__in=host_ids)}\n data = {'run_time': record.run_time, 'success': 0, 'failure': 0, 'duration': 0, 'outputs': []}\n for host_id, value in outputs.items():\n if not value:\n continue\n code, duration, out = value\n key = 'success' if code == 0 else 'failure'\n data[key] += 1\n data['duration'] += duration\n data['outputs'].append({\n 'name': hosts_info.get(host_id, '本机'),\n 'code': code,\n 'duration': duration,\n 'output': out})\n data['duration'] = f\"{data['duration'] / len(outputs):.3f}\"\n return data\n\n\n@auth('schedule.schedule.view|schedule.schedule.add|schedule.schedule.edit')\ndef next_run_time(request):\n form, error = JsonParser(\n Argument('rule', help='参数错误'),\n Argument('start', required=False),\n Argument('stop', required=False)\n ).parse(request.body)\n if error is None:\n try:\n minute, hour, day, month, week = form.rule.split()\n week = Scheduler.covert_week(week)\n trigger = CronTrigger(minute=minute, hour=hour, day=day, month=month, day_of_week=week,\n start_date=form.start, end_date=form.stop)\n except (ValueError, KeyError):\n return json_response({'success': False, 'msg': '无效的执行规则'})\n scheduler = BackgroundScheduler(timezone=settings.TIME_ZONE)\n scheduler.start()\n job = scheduler.add_job(lambda: None, trigger)\n run_time = job.next_run_time\n scheduler.shutdown()\n if run_time:\n return json_response({'success': True, 'msg': run_time.strftime('%Y-%m-%d %H:%M:%S')})\n else:\n return json_response({'success': False, 'msg': '无法被触发'})\n return json_response(error=error)\n"
},
{
"path": "spug_api/apps/setting/__init__.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\n"
},
{
"path": "spug_api/apps/setting/models.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.db import models\nfrom apps.account.models import User\nfrom libs import ModelMixin\nimport json\n\nKEYS_DEFAULT = {\n 'MFA': {'enable': False},\n 'verify_ip': True,\n 'bind_ip': True,\n 'ldap_service': {},\n 'spug_key': None,\n 'api_key': None,\n 'mail_service': {},\n 'private_key': None,\n 'public_key': None,\n 'spug_push_key': None,\n}\n\n\nclass Setting(models.Model, ModelMixin):\n key = models.CharField(max_length=50, unique=True)\n value = models.TextField()\n desc = models.CharField(max_length=255, null=True)\n\n def to_view(self):\n tmp = self.to_dict(selects=('key',))\n tmp['value'] = self.real_val\n return tmp\n\n @property\n def real_val(self):\n if self.value:\n return json.loads(self.value)\n else:\n return KEYS_DEFAULT.get(self.key)\n\n def __repr__(self):\n return '' % self.key\n\n class Meta:\n db_table = 'settings'\n\n\nclass UserSetting(models.Model, ModelMixin):\n user = models.ForeignKey(User, on_delete=models.CASCADE)\n key = models.CharField(max_length=32)\n value = models.TextField()\n\n class Meta:\n db_table = 'user_settings'\n unique_together = ('user', 'key')\n"
},
{
"path": "spug_api/apps/setting/urls.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\n# from django.urls import path\nfrom django.conf.urls import url\nfrom apps.setting.views import *\nfrom apps.setting.user import UserSettingView\n\nurlpatterns = [\n url(r'^$', SettingView.as_view()),\n url(r'^user/$', UserSettingView.as_view()),\n url(r'^ldap_test/$', ldap_test),\n url(r'^email_test/$', email_test),\n url(r'^mfa/$', MFAView.as_view()),\n url(r'^about/$', get_about),\n url(r'^push/bind/$', handle_push_bind),\n url(r'^push/balance/$', handle_push_balance),\n]\n"
},
{
"path": "spug_api/apps/setting/user.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.views.generic import View\nfrom libs import JsonParser, Argument, json_response\nfrom apps.setting.models import UserSetting\n\n\nclass UserSettingView(View):\n def get(self, request):\n response = {}\n for item in UserSetting.objects.filter(user=request.user):\n response[item.key] = item.value\n return json_response(response)\n\n def post(self, request):\n form, error = JsonParser(\n Argument('key', help='参数错误'),\n Argument('value', help='参数错误'),\n ).parse(request.body)\n if error is None:\n UserSetting.objects.update_or_create(\n user=request.user,\n key=form.key,\n defaults={'value': form.value}\n )\n return self.get(request)\n return json_response(error=error)\n"
},
{
"path": "spug_api/apps/setting/utils.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom functools import lru_cache\nfrom apps.setting.models import Setting, KEYS_DEFAULT\nfrom libs.ssh import SSH\nimport json\n\n\nclass AppSetting:\n @classmethod\n @lru_cache(maxsize=64)\n def get(cls, key):\n info = Setting.objects.filter(key=key).first()\n if not info:\n raise KeyError(f'no such key for {key!r}')\n return info.real_val\n\n @classmethod\n def get_default(cls, key, default=None):\n info = Setting.objects.filter(key=key).first()\n if not info:\n return default\n return info.real_val\n\n @classmethod\n def set(cls, key, value, desc=None):\n if key in KEYS_DEFAULT:\n value = json.dumps(value)\n Setting.objects.update_or_create(key=key, defaults={'value': value, 'desc': desc})\n else:\n raise KeyError('invalid key')\n\n @classmethod\n def delete(cls, key):\n Setting.objects.filter(key=key).delete()\n\n @classmethod\n def get_ssh_key(cls):\n public_key = cls.get_default('public_key')\n private_key = cls.get_default('private_key')\n if not private_key or not public_key:\n private_key, public_key = SSH.generate_key()\n cls.set('private_key', private_key)\n cls.set('public_key', public_key)\n return private_key, public_key\n"
},
{
"path": "spug_api/apps/setting/views.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nimport django\nfrom django.core.cache import cache\nfrom django.conf import settings\nfrom libs import JsonParser, Argument, json_response, auth\nfrom libs.utils import generate_random_str\nfrom libs.mail import Mail\nfrom libs.push import get_balance, send_login_code\nfrom libs.mixins import AdminView\nfrom apps.setting.utils import AppSetting\nfrom apps.setting.models import Setting, KEYS_DEFAULT\nfrom copy import deepcopy\nimport platform\nimport ldap\n\n\nclass SettingView(AdminView):\n def get(self, request):\n response = deepcopy(KEYS_DEFAULT)\n for item in Setting.objects.all():\n if item.key == 'spug_push_key':\n response[item.key] = f'{item.real_val[:8]}********{item.real_val[-8:]}'\n else:\n response[item.key] = item.real_val\n return json_response(response)\n\n def post(self, request):\n form, error = JsonParser(\n Argument('data', type=list, help='缺少必要的参数')\n ).parse(request.body)\n if error is None:\n for item in form.data:\n AppSetting.set(**item)\n return json_response(error=error)\n\n\nclass MFAView(AdminView):\n def get(self, request):\n if not request.user.wx_token:\n return json_response(\n error='检测到当前账户未配置推送标识（账户管理/编辑），请配置后再尝试启用MFA认证，否则可能造成系统无法正常登录。')\n spug_push_key = AppSetting.get_default('spug_push_key')\n if not spug_push_key:\n return json_response(error='检测到当前账户未绑定推送服务，请在系统设置/推送服务设置中绑定推送助手账户。')\n code = generate_random_str(6)\n send_login_code(spug_push_key, request.user.wx_token, code)\n cache.set(f'{request.user.username}:code', code, 300)\n return json_response()\n\n def post(self, request):\n form, error = JsonParser(\n Argument('enable', type=bool, help='参数错误'),\n Argument('code', required=False)\n ).parse(request.body)\n if error is None:\n if form.enable:\n if not form.code:\n return json_response(error='请输入验证码')\n key = f'{request.user.username}:code'\n code = cache.get(key)\n if not code:\n return json_response(error='验证码已失效，请重新获取')\n if code != form.code:\n ttl = cache.ttl(key)\n cache.expire(key, ttl - 100)\n return json_response(error='验证码错误')\n cache.delete(key)\n AppSetting.set('MFA', {'enable': form.enable})\n return json_response(error=error)\n\n\n@auth('admin')\ndef ldap_test(request):\n form, error = JsonParser(\n Argument('server'),\n Argument('port', type=int),\n Argument('admin_dn'),\n Argument('password'),\n ).parse(request.body)\n if error is None:\n try:\n con = ldap.initialize(\"ldap://{0}:{1}\".format(form.server, form.port), bytes_mode=False)\n con.simple_bind_s(form.admin_dn, form.password)\n return json_response()\n except Exception as e:\n error = eval(str(e))\n return json_response(error=error['desc'])\n return json_response(error=error)\n\n\n@auth('admin')\ndef email_test(request):\n form, error = JsonParser(\n Argument('server', help='请输入邮件服务地址'),\n Argument('port', type=int, help='请输入邮件服务端口号'),\n Argument('username', help='请输入邮箱账号'),\n Argument('password', help='请输入密码/授权码'),\n ).parse(request.body)\n if error is None:\n try:\n mail = Mail(**form)\n server = mail.get_server()\n server.quit()\n return json_response()\n except Exception as e:\n error = f'{e}'\n return json_response(error=error)\n\n\n@auth('admin')\ndef get_about(request):\n return json_response({\n 'python_version': platform.python_version(),\n 'system_version': platform.platform(),\n 'spug_version': settings.SPUG_VERSION,\n 'django_version': django.get_version()\n })\n\n\n@auth('admin')\ndef handle_push_bind(request):\n form, error = JsonParser(\n Argument('spug_push_key', required=False),\n ).parse(request.body)\n if error is None:\n if not form.spug_push_key:\n AppSetting.delete('spug_push_key')\n return json_response()\n\n try:\n res = get_balance(form.spug_push_key)\n except Exception as e:\n return json_response(error=f'绑定失败：{e}')\n\n AppSetting.set('spug_push_key', form.spug_push_key)\n return json_response(res)\n return json_response(error=error)\n\n\n@auth('admin')\ndef handle_push_balance(request):\n token = AppSetting.get_default('spug_push_key')\n if not token:\n return json_response(error='请先配置推送服务绑定账户')\n res = get_balance(token)\n return json_response(res)\n"
},
{
"path": "spug_api/consumer/__init__.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\n"
},
{
"path": "spug_api/consumer/consumers.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.conf import settings\nfrom django_redis import get_redis_connection\nfrom asgiref.sync import async_to_sync\nfrom apps.host.models import Host\nfrom consumer.utils import BaseConsumer\nfrom apps.account.utils import has_host_perm\nfrom libs.utils import str_decode\nfrom threading import Thread\nimport time\nimport json\n\n\nclass ComConsumer(BaseConsumer):\n def __init__(self, *args, **kwargs):\n super().__init__(*args, **kwargs)\n token = self.scope['url_route']['kwargs']['token']\n module = self.scope['url_route']['kwargs']['module']\n if module == 'build':\n self.key = f'{settings.BUILD_KEY}:{token}'\n elif module == 'request':\n self.key = f'{settings.REQUEST_KEY}:{token}'\n elif module == 'host':\n self.key = token\n else:\n raise TypeError(f'unknown module for {module}')\n self.rds = get_redis_connection()\n\n def disconnect(self, code):\n self.rds.close()\n\n def get_response(self, index):\n counter = 0\n while counter < 30:\n response = self.rds.lindex(self.key, index)\n if response:\n return response.decode()\n counter += 1\n time.sleep(0.2)\n\n def receive(self, text_data='', **kwargs):\n if text_data.isdigit():\n index = int(text_data)\n response = self.get_response(index)\n while response:\n index += 1\n self.send(text_data=response)\n response = self.get_response(index)\n self.send(text_data='pong')\n\n\nclass SSHConsumer(BaseConsumer):\n def __init__(self, *args, **kwargs):\n super().__init__(*args, **kwargs)\n self.id = self.scope['url_route']['kwargs']['id']\n self.chan = None\n self.ssh = None\n\n def loop_read(self):\n is_ready, data = False, b''\n while True:\n out = self.chan.recv(32 * 1024)\n if not out:\n self.close(3333)\n break\n data += out\n try:\n text = data.decode()\n except UnicodeDecodeError:\n try:\n text = data.decode(encoding='GBK')\n except UnicodeDecodeError:\n time.sleep(0.01)\n if self.chan.recv_ready():\n continue\n text = data.decode(errors='ignore')\n\n if not is_ready:\n self.send(text_data='\\033[2J\\033[3J\\033[1;1H')\n is_ready = True\n self.send(text_data=text)\n data = b''\n\n def receive(self, text_data=None, bytes_data=None):\n data = text_data or bytes_data\n if data and self.chan:\n data = json.loads(data)\n # print('write: {!r}'.format(data))\n resize = data.get('resize')\n if resize and len(resize) == 2:\n self.chan.resize_pty(*resize)\n else:\n self.chan.send(data['data'])\n\n def disconnect(self, code):\n if self.chan:\n self.chan.close()\n if self.ssh:\n self.ssh.close()\n\n def init(self):\n if has_host_perm(self.user, self.id):\n self.send(text_data='\\r\\n正在连接至主机 ...')\n host = Host.objects.filter(pk=self.id).first()\n if not host:\n return self.close_with_message('未找到指定主机，请刷新页面重试。')\n\n try:\n self.ssh = host.get_ssh().get_client()\n except Exception as e:\n return self.close_with_message(f'连接主机失败: {e}')\n\n self.chan = self.ssh.invoke_shell(term='xterm')\n self.chan.transport.set_keepalive(30)\n Thread(target=self.loop_read).start()\n else:\n self.close_with_message('你当前无权限操作该主机，请联系管理员授权。')\n\n\nclass NotifyConsumer(BaseConsumer):\n def init(self):\n async_to_sync(self.channel_layer.group_add)('notify', self.channel_name)\n\n def disconnect(self, code):\n async_to_sync(self.channel_layer.group_discard)('notify', self.channel_name)\n\n def receive(self, **kwargs):\n self.send(text_data='pong')\n\n def notify_message(self, event):\n self.send(text_data=json.dumps(event))\n\n\nclass PubSubConsumer(BaseConsumer):\n def __init__(self, *args, **kwargs):\n super().__init__(*args, **kwargs)\n self.token = self.scope['url_route']['kwargs']['token']\n self.rds = get_redis_connection()\n self.p = self.rds.pubsub(ignore_subscribe_messages=True)\n self.p.subscribe(self.token)\n\n def disconnect(self, code):\n self.p.close()\n self.rds.close()\n\n def receive(self, **kwargs):\n response = self.p.get_message(timeout=10)\n while response:\n data = str_decode(response['data'])\n self.send(text_data=data)\n response = self.p.get_message(timeout=10)\n self.send(text_data='pong')\n"
},
{
"path": "spug_api/consumer/routing.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.urls import path\nfrom channels.routing import URLRouter\nfrom consumer.consumers import *\n\nws_router = URLRouter([\n path('ws/ssh//', SSHConsumer),\n path('ws/subscribe//', PubSubConsumer),\n path('ws///', ComConsumer),\n path('ws/notify/', NotifyConsumer),\n])\n"
},
{
"path": "spug_api/consumer/utils.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.db import close_old_connections\nfrom channels.generic.websocket import WebsocketConsumer\nfrom apps.account.models import User\nfrom apps.setting.utils import AppSetting\nfrom libs.utils import get_request_real_ip\nfrom urllib.parse import parse_qs\nimport time\n\n\ndef get_real_ip(headers):\n decode_headers = {k.decode(): v.decode() for k, v in headers}\n return get_request_real_ip(decode_headers)\n\n\nclass BaseConsumer(WebsocketConsumer):\n def __init__(self, *args, **kwargs):\n super(BaseConsumer, self).__init__(*args, **kwargs)\n self.user = None\n\n def close_with_message(self, content):\n self.send(text_data=f'\\r\\n\\x1b[31m{content}\\x1b[0m\\r\\n')\n self.close()\n\n def connect(self):\n self.accept()\n close_old_connections()\n query_string = self.scope['query_string'].decode()\n x_real_ip = get_real_ip(self.scope['headers'])\n token = parse_qs(query_string).get('x-token', [''])[0]\n if token and len(token) == 32:\n user = User.objects.filter(access_token=token).first()\n if user and user.token_expired >= time.time() and user.is_active:\n if x_real_ip == user.last_ip or AppSetting.get_default('bind_ip') is False:\n self.user = user\n if hasattr(self, 'init'):\n self.init()\n return None\n self.close_with_message('触发登录IP绑定安全策略，请在系统设置/安全设置中查看配置。')\n self.close_with_message('用户身份验证失败，请重新登录或刷新页面。')\n"
},
{
"path": "spug_api/libs/__init__.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom .parser import JsonParser, Argument\nfrom .decorators import *\nfrom .validators import *\nfrom .mixins import *\nfrom .utils import *\n"
},
{
"path": "spug_api/libs/channel.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom channels.layers import get_channel_layer\nfrom asgiref.sync import async_to_sync\nimport uuid\n\nlayer = get_channel_layer()\n\n\nclass Channel:\n @staticmethod\n def get_token():\n return uuid.uuid4().hex\n\n @staticmethod\n def send_notify(title, content):\n message = {\n 'type': 'notify.message',\n 'title': title,\n 'content': content\n }\n async_to_sync(layer.group_send)('notify', message)\n"
},
{
"path": "spug_api/libs/decorators.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom functools import wraps\nfrom .utils import json_response\n\n\ndef auth(perm_list):\n def decorate(view_func):\n codes = perm_list.split('|')\n\n @wraps(view_func)\n def wrapper(*args, **kwargs):\n user = None\n for item in args[:2]:\n if hasattr(item, 'user'):\n user = item.user\n break\n if user and user.has_perms(codes):\n return view_func(*args, **kwargs)\n return json_response(error='权限拒绝')\n\n return wrapper\n\n return decorate\n"
},
{
"path": "spug_api/libs/gitlib.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom git import Repo, RemoteReference, TagReference, InvalidGitRepositoryError, GitCommandError\nfrom tempfile import NamedTemporaryFile\nfrom datetime import datetime\nimport shutil\nimport os\n\n\nclass Git:\n def __init__(self, git_repo, repo_dir, pkey=None):\n self.git_repo = git_repo\n self.repo_dir = repo_dir\n self.repo = None\n self.pkey = pkey\n self.fd = None\n self.env = {}\n\n def archive(self, filepath, commit):\n with open(filepath, 'wb') as f:\n self.repo.archive(f, commit)\n\n def fetch_branches_tags(self):\n self.fetch()\n branches, tags = {}, {}\n for ref in self.repo.references:\n if isinstance(ref, RemoteReference):\n if ref.remote_head != 'HEAD':\n branches[ref.remote_head] = self._get_commits(f'origin/{ref.remote_head}', 30)\n elif isinstance(ref, TagReference):\n tags[ref.name] = {\n 'id': ref.tag.hexsha,\n 'author': ref.tag.tagger.name,\n 'date': self._format_date(ref.tag.tagged_date),\n 'message': ref.tag.message.strip()\n } if ref.tag else {\n 'id': ref.commit.binsha.hex(),\n 'author': ref.commit.author.name,\n 'date': self._format_date(ref.commit.authored_date),\n 'message': ref.commit.message.strip()\n }\n tags = sorted(tags.items(), key=lambda x: x[1]['date'], reverse=True)\n return branches, dict(tags)\n\n def fetch(self):\n kwargs = dict(f=True, p=True)\n if self.repo.git.version_info >= (2, 17, 0):\n kwargs.update(P=True)\n try:\n self.repo.remotes.origin.fetch(**kwargs)\n except GitCommandError as e:\n if self.env:\n self.repo.remotes.origin.fetch(env=self.env, **kwargs)\n else:\n raise e\n\n def _get_repo(self):\n if os.path.exists(self.repo_dir):\n try:\n return Repo(self.repo_dir)\n except InvalidGitRepositoryError:\n if os.path.isdir(self.repo_dir):\n shutil.rmtree(self.repo_dir)\n else:\n os.remove(self.repo_dir)\n try:\n repo = Repo.clone_from(self.git_repo, self.repo_dir)\n except GitCommandError as e:\n if self.env:\n repo = Repo.clone_from(self.git_repo, self.repo_dir, env=self.env)\n else:\n raise e\n return repo\n\n def _get_commits(self, branch, count=10):\n commits = []\n for commit in self.repo.iter_commits(branch):\n if len(commits) == count:\n break\n commits.append({\n 'id': commit.hexsha,\n 'author': commit.author.name,\n 'date': self._format_date(commit.committed_date),\n 'message': commit.message.strip()\n })\n return commits\n\n def _format_date(self, timestamp):\n if isinstance(timestamp, int):\n date = datetime.fromtimestamp(timestamp)\n return date.strftime('%Y-%m-%d %H:%M')\n return timestamp\n\n def __enter__(self):\n if self.pkey:\n self.fd = NamedTemporaryFile()\n self.fd.write(self.pkey.encode())\n self.fd.flush()\n self.env = {'GIT_SSH_COMMAND': f'ssh -o StrictHostKeyChecking=no -i {self.fd.name}'}\n self.repo = self._get_repo()\n return self\n\n def __exit__(self, exc_type, exc_val, exc_tb):\n if self.fd:\n self.fd.close()\n"
},
{
"path": "spug_api/libs/helper.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom urllib.parse import quote, urlencode\nfrom datetime import datetime\nfrom pytz import timezone\nimport requests\nimport hashlib\nimport base64\nimport random\nimport time\nimport hmac\nimport uuid\n\n\ndef _special_url_encode(value) -> str:\n if isinstance(value, (str, bytes)):\n rst = quote(value)\n else:\n rst = urlencode(value)\n return rst.replace('+', '%20').replace('*', '%2A').replace('%7E', '~')\n\n\ndef _make_ali_signature(key: str, params: dict) -> bytes:\n sorted_str = _special_url_encode(dict(sorted(params.items())))\n sign_str = 'GET&%2F&' + _special_url_encode(sorted_str)\n sign_digest = hmac.new(key.encode(), sign_str.encode(), hashlib.sha1).digest()\n return base64.encodebytes(sign_digest).strip()\n\n\ndef _make_tencent_signature(endpoint: str, key: str, params: dict) -> bytes:\n sorted_str = '&'.join(f'{k}={v}' for k, v in sorted(params.items()))\n sign_str = f'POST{endpoint}/?{sorted_str}'\n sign_digest = hmac.new(key.encode(), sign_str.encode(), hashlib.sha1).digest()\n return base64.encodebytes(sign_digest).strip()\n\n\ndef make_ali_request(ak, ac, endpoint, params):\n params.update(\n AccessKeyId=ak,\n Format='JSON',\n SignatureMethod='HMAC-SHA1',\n SignatureNonce=uuid.uuid4().hex,\n SignatureVersion='1.0',\n Timestamp=datetime.now(tz=timezone('UTC')).strftime('%Y-%m-%dT%H:%M:%SZ'),\n Version='2014-05-26'\n )\n params['Signature'] = _make_ali_signature(ac + '&', params)\n return requests.get(endpoint, params).json()\n\n\ndef make_tencent_request(ak, ac, endpoint, params):\n params.update(\n Nonce=int(random.random() * 10000),\n SecretId=ak,\n Timestamp=int(time.time()),\n Version='2017-03-12'\n )\n params['Signature'] = _make_tencent_signature(endpoint, ac, params)\n return requests.post(f'https://{endpoint}', data=params).json()\n"
},
{
"path": "spug_api/libs/ldap.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nimport ldap\n\n\nclass LDAP:\n def __init__(self, server, port, rules, admin_dn, password, base_dn):\n self.server = server\n self.port = port\n self.rules = rules\n self.admin_dn = admin_dn\n self.password = password\n self.base_dn = base_dn\n\n def valid_user(self, username, password):\n try:\n conn = ldap.initialize(\"ldap://{0}:{1}\".format(self.server, self.port), bytes_mode=False)\n conn.simple_bind_s(self.admin_dn, self.password)\n search_filter = f'({self.rules}={username})'\n ldap_result_id = conn.search(self.base_dn, ldap.SCOPE_SUBTREE, search_filter, None)\n result_type, result_data = conn.result(ldap_result_id, 0)\n if result_type == ldap.RES_SEARCH_ENTRY:\n conn.simple_bind_s(result_data[0][0], password)\n return True, None\n else:\n return False, None\n except Exception as error:\n args = error.args\n return False, args[0].get('desc', '未知错误') if args else '%s' % error\n"
},
{
"path": "spug_api/libs/mail.py",
"content": "from email.header import Header\nfrom email.mime.text import MIMEText\nfrom email.utils import formataddr\nimport smtplib\n\n\nclass Mail:\n def __init__(self, server, port, username, password, nickname=None):\n self.host = server\n self.port = int(port)\n self.user = username\n self.password = password\n self.nickname = nickname\n\n def get_server(self):\n if self.port == 465:\n server = smtplib.SMTP_SSL(self.host, self.port)\n elif self.port == 587:\n server = smtplib.SMTP(self.host, self.port)\n server.ehlo()\n server.starttls()\n else:\n server = smtplib.SMTP(self.host, self.port)\n server.login(self.user, self.password)\n return server\n\n def send_text_mail(self, receivers, subject, body):\n server = self.get_server()\n msg = MIMEText(body, 'plain', 'utf-8')\n msg['Subject'] = Header(subject, 'utf-8')\n msg['From'] = formataddr((self.nickname, self.user)) if self.nickname else self.user\n server.sendmail(self.user, receivers, msg.as_string())\n server.quit()\n"
},
{
"path": "spug_api/libs/middleware.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.utils.deprecation import MiddlewareMixin\nfrom django.conf import settings\nfrom .utils import json_response, get_request_real_ip\nfrom apps.account.models import User\nfrom apps.setting.utils import AppSetting\nimport traceback\nimport time\n\n\nclass HandleExceptionMiddleware(MiddlewareMixin):\n \"\"\"\n 处理试图函数异常\n \"\"\"\n\n def process_exception(self, request, exception):\n traceback.print_exc()\n return json_response(error='Exception: %s' % exception)\n\n\nclass AuthenticationMiddleware(MiddlewareMixin):\n \"\"\"\n 登录验证\n \"\"\"\n\n def process_request(self, request):\n if request.path in settings.AUTHENTICATION_EXCLUDES:\n return None\n if any(x.match(request.path) for x in settings.AUTHENTICATION_EXCLUDES if hasattr(x, 'match')):\n return None\n access_token = request.headers.get('x-token') or request.GET.get('x-token')\n if access_token and len(access_token) == 32:\n x_real_ip = get_request_real_ip(request.headers)\n user = User.objects.filter(access_token=access_token).first()\n if user and user.token_expired >= time.time() and user.is_active:\n if x_real_ip == user.last_ip or AppSetting.get_default('bind_ip') is False:\n request.user = user\n user.token_expired = time.time() + settings.TOKEN_TTL\n user.save()\n return None\n response = json_response(error=\"验证失败，请重新登录\")\n response.status_code = 401\n return response\n"
},
{
"path": "spug_api/libs/mixins.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.views.generic import View\nfrom .utils import json_response\n\n\n# 混入类，提供Model实例to_dict方法\nclass ModelMixin(object):\n __slots__ = ()\n\n def to_dict(self, excludes: tuple = None, selects: tuple = None) -> dict:\n if not hasattr(self, '_meta'):\n raise TypeError('<%r> does not a django.db.models.Model object.' % self)\n elif selects:\n return {f: getattr(self, f) for f in selects}\n elif excludes:\n return {f.attname: getattr(self, f.attname) for f in self._meta.fields if f.attname not in excludes}\n else:\n return {f.attname: getattr(self, f.attname) for f in self._meta.fields}\n\n def update_by_dict(self, data):\n for key, value in data.items():\n setattr(self, key, value)\n self.save()\n\n\nclass AdminView(View):\n def dispatch(self, request, *args, **kwargs):\n if hasattr(request, 'user') and request.user.is_supper:\n return super().dispatch(request, *args, **kwargs)\n else:\n return json_response(error='权限拒绝')\n"
},
{
"path": "spug_api/libs/parser.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nimport json\n\nfrom .utils import AttrDict\n\n\n# 自定义的解析异常\nclass ParseError(BaseException):\n def __init__(self, message):\n self.message = message\n\n\n# 需要校验的参数对象\nclass Argument(object):\n \"\"\"\n :param name: name of option\n :param default: default value if the argument if absent\n :param bool required: is required\n \"\"\"\n\n def __init__(self, name, default=None, handler=None, required=True, type=str, filter=None, help=None):\n self.name = name\n self.default = default\n self.type = type\n self.required = required\n self.filter = filter\n self.help = help\n self.handler = handler\n if not isinstance(self.name, str):\n raise TypeError('Argument name must be string')\n if filter and not callable(self.filter):\n raise TypeError('Argument filter is not callable')\n\n def parse(self, has_key, value):\n if not has_key:\n if self.required and self.default is None:\n raise ParseError(\n self.help or 'Required Error: %s is required' % self.name)\n else:\n return self.default\n elif value in [u'', '', None]:\n if self.default is not None:\n return self.default\n elif self.required:\n raise ParseError(self.help or 'Value Error: %s must not be null' % self.name)\n elif self.help:\n raise ParseError(self.help)\n else:\n return value\n try:\n if self.type:\n if self.type in (list, dict) and isinstance(value, str):\n value = json.loads(value)\n assert isinstance(value, self.type)\n elif self.type == bool and isinstance(value, str):\n assert value.lower() in ['true', 'false']\n value = value.lower() == 'true'\n elif not isinstance(value, self.type):\n value = self.type(value)\n except (TypeError, ValueError, AssertionError):\n raise ParseError(self.help or 'Type Error: %s type must be %s' % (\n self.name, self.type))\n\n if self.filter:\n if not self.filter(value):\n raise ParseError(\n self.help or 'Value Error: %s filter check failed' % self.name)\n if self.handler:\n value = self.handler(value)\n return value\n\n\n# 解析器基类\nclass BaseParser(object):\n def __init__(self, *args):\n self.args = []\n for e in args:\n if isinstance(e, str):\n e = Argument(e)\n elif not isinstance(e, Argument):\n raise TypeError('%r is not instance of Argument' % e)\n self.args.append(e)\n\n def _get(self, key):\n raise NotImplementedError\n\n def _init(self, data):\n raise NotImplementedError\n\n def add_argument(self, **kwargs):\n self.args.append(Argument(**kwargs))\n\n def parse(self, data=None, clear=False):\n rst = AttrDict()\n try:\n self._init(data)\n for e in self.args:\n has_key, value = self._get(e.name)\n if clear and has_key is False and e.required is False:\n continue\n rst[e.name] = e.parse(has_key, value)\n except ParseError as err:\n return None, err.message\n return rst, None\n\n\n# Json解析器\nclass JsonParser(BaseParser):\n def __init__(self, *args):\n self.__data = None\n super(JsonParser, self).__init__(*args)\n\n def _get(self, key):\n return key in self.__data, self.__data.get(key)\n\n def _init(self, data):\n try:\n if isinstance(data, (str, bytes)):\n self.__data = json.loads(data) if data else {}\n else:\n assert hasattr(data, '__contains__')\n assert hasattr(data, 'get')\n assert callable(data.get)\n self.__data = data\n except (ValueError, AssertionError):\n raise ParseError('Invalid data type for parse')\n"
},
{
"path": "spug_api/libs/push.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom apps.setting.utils import AppSetting\nimport requests\n\npush_server = 'https://push.spug.cc'\n\n\ndef get_balance(token):\n res = requests.get(f'{push_server}/spug/balance/', json={'token': token})\n if res.status_code != 200:\n raise Exception(f'status code: {res.status_code}')\n res = res.json()\n if res.get('error'):\n raise Exception(res['error'])\n return res['data']\n\n\ndef get_contacts(token):\n try:\n res = requests.post(f'{push_server}/spug/contacts/', json={'token': token})\n res = res.json()\n if res['data']:\n return res['data']\n except Exception:\n return []\n\n\ndef send_login_code(token, user, code):\n url = f'{push_server}/spug/message/'\n data = {\n 'token': token,\n 'targets': [user],\n 'source': 'mfa',\n 'dataset': {\n 'code': code\n }\n }\n res = requests.post(url, json=data, timeout=15)\n if res.status_code != 200:\n raise Exception(f'status code: {res.status_code}')\n res = res.json()\n if res.get('error'):\n raise Exception(res['error'])\n"
},
{
"path": "spug_api/libs/spug.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom apps.alarm.models import Group, Contact\nfrom apps.setting.utils import AppSetting\nfrom apps.notify.models import Notify\nfrom libs.mail import Mail\nfrom libs.utils import human_datetime\nfrom libs.push import push_server\nimport requests\nimport json\nimport time\nimport hmac\nimport hashlib\nimport base64\nfrom urllib.parse import urlencode\n\n\ndef _gen_dd_sign(secret):\n timestamp = str(int(time.time() * 1000))\n string_to_sign = f'{timestamp}\\n{secret}'\n hmac_code = hmac.new(secret.encode('utf-8'), string_to_sign.encode('utf-8'), digestmod=hashlib.sha256).digest()\n sign = base64.b64encode(hmac_code).decode('utf-8')\n return timestamp, sign\n\n\ndef _gen_fs_sign(secret):\n timestamp = str(int(time.time()))\n string_to_sign = f'{timestamp}\\n{secret}'\n hmac_code = hmac.new(string_to_sign.encode('utf-8'), b'', digestmod=hashlib.sha256).digest()\n sign = base64.b64encode(hmac_code).decode('utf-8')\n return timestamp, sign\n\n\nclass Notification:\n def __init__(self, grp, event, target, title, message, duration):\n self.grp = grp\n self.event = event\n self.title = title\n self.target = target\n self.message = message\n self.duration = duration\n self.spug_push_key = AppSetting.get_default('spug_push_key')\n\n @staticmethod\n def handle_request(url, data, mode=None):\n try:\n res = requests.post(url, json=data, timeout=15)\n except Exception as e:\n return Notify.make_system_notify('通知发送失败', f'接口调用异常: {e}')\n if res.status_code != 200:\n return Notify.make_system_notify('通知发送失败', f'返回状态码：{res.status_code}, 请求URL：{res.url}')\n\n if mode in ['dd', 'wx']:\n res = res.json()\n if res.get('errcode') == 0:\n return\n elif mode == 'spug':\n res = res.json()\n if not res.get('error'):\n return\n elif mode == 'fs':\n res = res.json()\n if res.get('StatusCode') == 0:\n return\n else:\n raise NotImplementedError\n Notify.make_system_notify('通知发送失败', f'返回数据：{res}')\n\n def monitor_by_email(self, users):\n mail_service = AppSetting.get_default('mail_service', {})\n body = [\n f'告警名称：{self.title}',\n f'告警对象：{self.target}',\n f'{\"告警\" if self.event == \"1\" else \"恢复\"}时间：{human_datetime()}',\n f'告警描述：{self.message}'\n ]\n if self.event == '2':\n body.append('故障持续：' + self.duration)\n if mail_service.get('server'):\n event_map = {'1': '监控告警通知', '2': '告警恢复通知'}\n subject = f'{event_map[self.event]}-{self.title}'\n mail = Mail(**mail_service)\n mail.send_text_mail(users, subject, '\\r\\n'.join(body) + '\\r\\n\\r\\n自动发送，请勿回复。')\n else:\n Notify.make_monitor_notify(\n '发送报警信息失败',\n '未配置报警服务，请在系统管理/系统设置/报警服务设置中配置邮件服务。'\n )\n\n def monitor_by_dd(self, users):\n texts = [\n '## %s ## ' % ('监控告警通知' if self.event == '1' else '告警恢复通知'),\n f'**告警名称：** {self.title} ',\n f'**告警对象：** {self.target} ',\n f'**{\"告警\" if self.event == \"1\" else \"恢复\"}时间：** {human_datetime()} ',\n f'**告警描述：** {self.message} ',\n ]\n if self.event == '2':\n texts.append(f'**持续时间：** {self.duration} ')\n data = {\n 'msgtype': 'markdown',\n 'markdown': {\n 'title': '监控告警通知',\n 'text': '\\n\\n'.join(texts) + '\\n\\n> ###### 来自 Spug运维平台'\n },\n 'at': {\n 'isAtAll': True\n }\n }\n for url, secret in users:\n if secret:\n timestamp, sign = _gen_dd_sign(secret)\n url = f'{url}&{urlencode({\"timestamp\": timestamp, \"sign\": sign})}'\n self.handle_request(url, data, 'dd')\n\n def monitor_by_fs(self, users):\n title = '监控告警通知' if self.event == '1' else '告警恢复通知'\n content = [\n [{'tag': 'text', 'text': f'告警名称：{self.title}'}],\n [{'tag': 'text', 'text': f'告警对象：{self.target}'}],\n [{'tag': 'text', 'text': f'{\"告警\" if self.event == \"1\" else \"恢复\"}时间：{human_datetime()}'}],\n [{'tag': 'text', 'text': f'告警描述：{self.message}'}],\n ]\n if self.event == '2':\n content.append([{'tag': 'text', 'text': f'持续时间：{self.duration}'}])\n content.append([{'tag': 'text', 'text': '来自 Spug运维平台'}])\n for url, secret in users:\n data = {\n 'msg_type': 'post',\n 'content': {\n 'post': {\n 'zh_cn': {\n 'title': title,\n 'content': content\n }\n }\n }\n }\n if secret:\n timestamp, sign = _gen_fs_sign(secret)\n data['timestamp'] = timestamp\n data['sign'] = sign\n self.handle_request(url, data, 'fs')\n\n def monitor_by_qy_wx(self, users):\n color, title = ('warning', '监控告警通知') if self.event == '1' else ('info', '告警恢复通知')\n texts = [\n f'## {title}',\n f'**告警名称：** {self.title} ',\n f'**告警对象：** {self.target}',\n f'**{\"告警\" if self.event == \"1\" else \"恢复\"}时间：** {human_datetime()} ',\n f'**告警描述：** {self.message} ',\n ]\n if self.event == '2':\n texts.append(f'**持续时间：** {self.duration} ')\n data = {\n 'msgtype': 'markdown',\n 'markdown': {\n 'content': '\\n'.join(texts) + '\\n> 来自 Spug运维平台'\n }\n }\n for url in users:\n self.handle_request(url, data, 'wx')\n\n def monitor_by_spug_push(self, targets):\n if not self.spug_push_key:\n Notify.make_monitor_notify(\n '发送报警信息失败',\n '未绑定推送服务，请在系统管理/系统设置/推送服务设置中绑定推送助手账户。'\n )\n return\n data = {\n 'source': 'monitor',\n 'token': self.spug_push_key,\n 'targets': list(targets),\n 'dataset': {\n 'title': self.title,\n 'target': self.target,\n 'message': self.message,\n 'duration': self.duration,\n 'event': self.event\n }\n }\n self.handle_request(f'{push_server}/spug/message/', data, 'spug')\n\n def dispatch_monitor(self, modes):\n u_ids, push_ids = [], []\n for item in Group.objects.filter(id__in=self.grp):\n for x in json.loads(item.contacts):\n if isinstance(x, str) and '_' in x:\n push_ids.append(x)\n else:\n u_ids.append(x)\n\n targets = set()\n for mode in modes:\n if mode == '1':\n wx_mp_ids = set(x for x in push_ids if x.startswith('wx_mp_'))\n targets.update(wx_mp_ids)\n elif mode == '2':\n sms_ids = set(x for x in push_ids if x.startswith('sms_'))\n targets.update(sms_ids)\n elif mode == '3':\n contacts = Contact.objects.filter(id__in=u_ids, ding__isnull=False)\n users = []\n for c in contacts:\n sec = None\n if c.secret:\n sec = json.loads(c.secret).get('ding')\n users.append((c.ding, sec))\n if not users:\n Notify.make_monitor_notify(\n '发送报警信息失败',\n '未找到可用的通知对象，请确保设置了相关报警联系人的钉钉。'\n )\n continue\n self.monitor_by_dd(users)\n elif mode == '4':\n mail_ids = set(x for x in push_ids if x.startswith('mail_'))\n targets.update(mail_ids)\n users = set(x.email for x in Contact.objects.filter(id__in=u_ids, email__isnull=False))\n if not users:\n if not mail_ids:\n Notify.make_monitor_notify(\n '发送报警信息失败',\n '未找到可用的通知对象，请确保设置了相关报警联系人的邮件地址。'\n )\n continue\n self.monitor_by_email(users)\n elif mode == '5':\n users = set(x.qy_wx for x in Contact.objects.filter(id__in=u_ids, qy_wx__isnull=False))\n if not users:\n Notify.make_monitor_notify(\n '发送报警信息失败',\n '未找到可用的通知对象，请确保设置了相关报警联系人的企业微信。'\n )\n continue\n self.monitor_by_qy_wx(users)\n elif mode == '6':\n voice_ids = set(x for x in push_ids if x.startswith('voice_'))\n targets.update(voice_ids)\n elif mode == '7':\n contacts = Contact.objects.filter(id__in=u_ids, feishu__isnull=False)\n users = []\n for c in contacts:\n sec = None\n if c.secret:\n sec = json.loads(c.secret).get('feishu')\n users.append((c.feishu, sec))\n if not users:\n Notify.make_monitor_notify(\n '发送报警信息失败',\n '未找到可用的通知对象，请确保设置了相关报警联系人的飞书。'\n )\n continue\n self.monitor_by_fs(users)\n\n if targets:\n self.monitor_by_spug_push(targets)\n"
},
{
"path": "spug_api/libs/ssh.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom paramiko.client import SSHClient, AutoAddPolicy\nfrom paramiko.rsakey import RSAKey\nfrom paramiko.auth_handler import AuthHandler\nfrom paramiko.ssh_exception import AuthenticationException, SSHException\nfrom paramiko.py3compat import b, u\nfrom io import StringIO\nfrom uuid import uuid4\nimport time\nimport re\n\n\ndef _finalize_pubkey_algorithm(self, key_type):\n if \"rsa\" not in key_type:\n return key_type\n if re.search(r\"-OpenSSH_(?:[1-6]|7\\.[0-7])\", self.transport.remote_version):\n pubkey_algo = \"ssh-rsa\"\n if key_type.endswith(\"-cert-v01@openssh.com\"):\n pubkey_algo += \"-cert-v01@openssh.com\"\n\n self.transport._agreed_pubkey_algorithm = pubkey_algo\n return pubkey_algo\n my_algos = [x for x in self.transport.preferred_pubkeys if \"rsa\" in x]\n if not my_algos:\n raise SSHException(\n \"An RSA key was specified, but no RSA pubkey algorithms are configured!\" # noqa\n )\n server_algo_str = u(\n self.transport.server_extensions.get(\"server-sig-algs\", b(\"\"))\n )\n if server_algo_str:\n server_algos = server_algo_str.split(\",\")\n agreement = list(filter(server_algos.__contains__, my_algos))\n if agreement:\n pubkey_algo = agreement[0]\n else:\n err = \"Unable to agree on a pubkey algorithm for signing a {!r} key!\" # noqa\n raise AuthenticationException(err.format(key_type))\n else:\n pubkey_algo = \"ssh-rsa\"\n if key_type.endswith(\"-cert-v01@openssh.com\"):\n pubkey_algo += \"-cert-v01@openssh.com\"\n self.transport._agreed_pubkey_algorithm = pubkey_algo\n return pubkey_algo\n\n\nAuthHandler._finalize_pubkey_algorithm = _finalize_pubkey_algorithm\n\n\nclass SSH:\n def __init__(self, hostname, port=22, username='root', pkey=None, password=None, default_env=None,\n connect_timeout=10, term=None):\n self.stdout = None\n self.client = None\n self.channel = None\n self.sftp = None\n self.exec_file = None\n self.term = term or {}\n self.eof = 'Spug EOF 2108111926'\n self.default_env = default_env\n self.regex = re.compile(r'Spug EOF 2108111926 (-?\\d+)[\\r\\n]?')\n self.arguments = {\n 'hostname': hostname,\n 'port': port,\n 'username': username,\n 'password': password,\n 'pkey': RSAKey.from_private_key(StringIO(pkey)) if isinstance(pkey, str) else pkey,\n 'timeout': connect_timeout,\n 'allow_agent': False,\n 'look_for_keys': False,\n 'banner_timeout': 30\n }\n\n @staticmethod\n def generate_key():\n key_obj = StringIO()\n key = RSAKey.generate(2048)\n key.write_private_key(key_obj)\n return key_obj.getvalue(), 'ssh-rsa ' + key.get_base64()\n\n def get_client(self):\n if self.client is not None:\n return self.client\n self.client = SSHClient()\n self.client.set_missing_host_key_policy(AutoAddPolicy)\n self.client.connect(**self.arguments)\n return self.client\n\n def ping(self):\n return True\n\n def add_public_key(self, public_key):\n command = f'mkdir -p -m 700 ~/.ssh && \\\n echo {public_key!r} >> ~/.ssh/authorized_keys && \\\n chmod 600 ~/.ssh/authorized_keys'\n exit_code, out = self.exec_command_raw(command)\n if exit_code != 0:\n raise Exception(f'add public key error: {out}')\n\n def exec_command_raw(self, command, environment=None):\n channel = self.client.get_transport().open_session()\n if environment:\n channel.update_environment(environment)\n channel.set_combine_stderr(True)\n channel.exec_command(command)\n code, output = channel.recv_exit_status(), channel.recv(-1)\n return code, self._decode(output)\n\n def exec_command(self, command, environment=None):\n channel = self._get_channel()\n command = self._handle_command(command, environment)\n channel.sendall(command)\n out, exit_code = '', -1\n for line in self.stdout:\n match = self.regex.search(line)\n if match:\n exit_code = int(match.group(1))\n line = line[:match.start()]\n out += line\n break\n out += line\n return exit_code, out\n\n def _win_exec_command_with_stream(self, command, environment=None):\n channel = self.client.get_transport().open_session()\n if environment:\n channel.update_environment(environment)\n channel.set_combine_stderr(True)\n channel.get_pty(width=102)\n channel.exec_command(command)\n stdout = channel.makefile(\"rb\", -1)\n out = stdout.readline()\n while out:\n yield channel.exit_status, self._decode(out)\n out = stdout.readline()\n yield channel.recv_exit_status(), self._decode(out)\n\n def exec_command_with_stream(self, command, environment=None):\n channel = self._get_channel()\n command = self._handle_command(command, environment)\n channel.sendall(command)\n exit_code, line = -1, ''\n while True:\n line = self._decode(channel.recv(8196))\n if not line:\n break\n match = self.regex.search(line)\n if match:\n exit_code = int(match.group(1))\n line = line[:match.start()]\n break\n yield exit_code, line\n yield exit_code, line\n\n def put_file(self, local_path, remote_path, callback=None):\n sftp = self._get_sftp()\n sftp.put(local_path, remote_path, callback=callback, confirm=False)\n\n def put_file_by_fl(self, fl, remote_path, callback=None):\n sftp = self._get_sftp()\n sftp.putfo(fl, remote_path, callback=callback, confirm=False)\n\n def list_dir_attr(self, path):\n sftp = self._get_sftp()\n return sftp.listdir_attr(path)\n\n def sftp_stat(self, path):\n sftp = self._get_sftp()\n return sftp.stat(path)\n\n def remove_file(self, path):\n sftp = self._get_sftp()\n sftp.remove(path)\n\n def _get_channel(self):\n if self.channel:\n return self.channel\n\n counter = 0\n self.channel = self.client.invoke_shell(**self.term)\n command = '[ -n \"$BASH_VERSION\" ] && set +o history\\n'\n command += '[ -n \"$ZSH_VERSION\" ] && set +o zle && set -o no_nomatch\\n'\n command += 'export PS1= && stty -echo\\n'\n command = self._handle_command(command, self.default_env)\n self.channel.sendall(command)\n out = ''\n while True:\n if self.channel.recv_ready():\n out += self._decode(self.channel.recv(8196))\n if self.regex.search(out):\n self.stdout = self.channel.makefile('r')\n break\n elif counter >= 100:\n self.client.close()\n raise Exception('Wait spug response timeout')\n else:\n counter += 1\n time.sleep(0.1)\n return self.channel\n\n def _get_sftp(self):\n if self.sftp:\n return self.sftp\n\n self.sftp = self.client.open_sftp()\n return self.sftp\n\n def _make_env_command(self, environment):\n if not environment:\n return None\n str_envs = []\n for k, v in environment.items():\n k = k.replace('-', '_')\n if isinstance(v, str):\n v = v.replace(\"'\", \"'\\\"'\\\"'\")\n str_envs.append(f\"{k}='{v}'\")\n str_envs = ' '.join(str_envs)\n return f'export {str_envs}'\n\n def _handle_command(self, command, environment):\n new_command = commands = ''\n if not self.exec_file:\n self.exec_file = f'/tmp/spug.{uuid4().hex}'\n commands += f'trap \\'rm -f {self.exec_file}\\' EXIT\\n'\n\n env_command = self._make_env_command(environment)\n if env_command:\n new_command += f'{env_command}\\n'\n new_command += command\n new_command += f'\\necho {self.eof} $?\\n'\n self.put_file_by_fl(StringIO(new_command), self.exec_file)\n commands += f'. {self.exec_file}\\n'\n return commands\n\n def _decode(self, content):\n try:\n content = content.decode()\n except UnicodeDecodeError:\n content = content.decode(encoding='GBK', errors='ignore')\n return content\n\n def __enter__(self):\n self.get_client()\n transport = self.client.get_transport()\n if 'windows' in transport.remote_version.lower():\n self.exec_command = self.exec_command_raw\n self.exec_command_with_stream = self._win_exec_command_with_stream\n return self\n\n def __exit__(self, exc_type, exc_val, exc_tb):\n self.client.close()\n self.client = None\n"
},
{
"path": "spug_api/libs/utils.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom django.http.response import HttpResponse\nfrom django.db.models import QuerySet\nfrom datetime import datetime, date as datetime_date\nfrom decimal import Decimal\nfrom string import Template\nimport string\nimport random\nimport json\n\n\n# 转换时间格式到字符串\ndef human_datetime(date=None):\n if date:\n assert isinstance(date, datetime)\n else:\n date = datetime.now()\n return date.strftime('%Y-%m-%d %H:%M:%S')\n\n\n# 转换时间格式到字符串(天)\ndef human_date(date=None):\n if date:\n assert isinstance(date, datetime)\n else:\n date = datetime.now()\n return date.strftime('%Y-%m-%d')\n\n\ndef human_time(date=None):\n if date:\n assert isinstance(date, datetime)\n else:\n date = datetime.now()\n return date.strftime('%H:%M:%S')\n\n\ndef str_decode(data):\n try:\n data = data.decode()\n except UnicodeDecodeError:\n try:\n data = data.decode(encoding='GBK')\n except UnicodeDecodeError:\n data = data.decode(errors='ignore')\n return data\n\n\n# 解析时间类型的数据\ndef parse_time(value):\n if isinstance(value, datetime):\n return value\n if isinstance(value, str):\n if len(value) == 10:\n return datetime.strptime(value, '%Y-%m-%d')\n elif len(value) == 19:\n return datetime.strptime(value, '%Y-%m-%d %H:%M:%S')\n raise TypeError('Expect a datetime.datetime value')\n\n\n# 传两个时间得到一个时间差\ndef human_seconds_time(seconds):\n text = ''\n if seconds >= 3600:\n text += '%d小时' % (seconds / 3600)\n seconds = seconds % 3600\n if seconds >= 60:\n text += '%d分' % (seconds / 60)\n seconds = seconds % 60\n if seconds > 0:\n if text or isinstance(seconds, int):\n text += '%.d秒' % seconds\n else:\n text += '%.1f秒' % seconds\n return text\n\n\n# 字符串模版渲染\ndef render_str(template, datasheet):\n return Template(template).safe_substitute(datasheet)\n\n\ndef json_response(data='', error=''):\n content = AttrDict(data=data, error=error)\n if error:\n content.data = ''\n elif hasattr(data, 'to_dict'):\n content.data = data.to_dict()\n elif isinstance(data, (list, QuerySet)) and all([hasattr(item, 'to_dict') for item in data]):\n content.data = [item.to_dict() for item in data]\n return HttpResponse(json.dumps(content, cls=DateTimeEncoder), content_type='application/json')\n\n\n# 继承自dict，实现可以通过.来操作元素\nclass AttrDict(dict):\n def __setattr__(self, key, value):\n self.__setitem__(key, value)\n\n def __getattr__(self, item):\n try:\n return self.__getitem__(item)\n except KeyError:\n raise AttributeError(item)\n\n def __delattr__(self, item):\n self.__delitem__(item)\n\n\n# 日期json序列化\nclass DateTimeEncoder(json.JSONEncoder):\n def default(self, o):\n if isinstance(o, datetime):\n return o.strftime('%Y-%m-%d %H:%M:%S')\n elif isinstance(o, datetime_date):\n return o.strftime('%Y-%m-%d')\n elif isinstance(o, Decimal):\n return float(o)\n\n return json.JSONEncoder.default(self, o)\n\n\n# 生成指定长度的随机数\ndef generate_random_str(length: int = 4, is_digits: bool = True) -> str:\n words = string.digits if is_digits else string.ascii_letters + string.digits\n return ''.join(random.sample(words, length))\n\n\ndef get_request_real_ip(headers: dict):\n x_real_ip = headers.get('x-forwarded-for')\n if not x_real_ip:\n x_real_ip = headers.get('x-real-ip', '')\n return x_real_ip.split(',')[0]\n"
},
{
"path": "spug_api/libs/validators.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nimport ipaddress\nfrom datetime import datetime\n\n\n# 判断是否是ip地址\ndef ip_validator(value):\n try:\n ipaddress.ip_address(value)\n return True\n except ValueError:\n return False\n\n\n# 判断是否是日期字符串，支持 2018-04-11 或 2018-04-11 14:55:30\ndef date_validator(value: str) -> bool:\n value = value.strip()\n try:\n if len(value) == 10:\n datetime.strptime(value, '%Y-%m-%d')\n return True\n elif len(value) == 19:\n datetime.strptime(value, '%Y-%m-%d %H:%M:%S')\n return True\n except ValueError:\n pass\n return False\n"
},
{
"path": "spug_api/logs/.gitkeep",
"content": ""
},
{
"path": "spug_api/manage.py",
"content": "#!/usr/bin/env python\n# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\n\"\"\"Django's command-line utility for administrative tasks.\"\"\"\nimport os\nimport sys\n\n\ndef main():\n os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'spug.settings')\n try:\n from django.core.management import execute_from_command_line\n except ImportError as exc:\n raise ImportError(\n \"Couldn't import Django. Are you sure it's installed and \"\n \"available on your PYTHONPATH environment variable? Did you \"\n \"forget to activate a virtual environment?\"\n ) from exc\n execute_from_command_line(sys.argv)\n\n\nif __name__ == '__main__':\n main()\n"
},
{
"path": "spug_api/repos/.gitkeep",
"content": ""
},
{
"path": "spug_api/repos/build/.gitkeep",
"content": ""
},
{
"path": "spug_api/requirements.txt",
"content": "apscheduler==3.7.0\nDjango==2.2.28\nasgiref==3.2.10\nchannels==2.3.1\nchannels_redis==2.4.1\nparamiko==2.11.0\ndjango-redis==4.10.0\nrequests==2.32.0\nGitPython==3.1.41\npython-ldap==3.4.0\nopenpyxl==3.0.3\nuser_agents==2.2.0"
},
{
"path": "spug_api/spug/__init__.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\n"
},
{
"path": "spug_api/spug/asgi.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\n\"\"\"\nASGI entrypoint. Configures Django and then runs the application\ndefined in the ASGI_APPLICATION setting.\n\"\"\"\n\n\nimport os\nimport django\nfrom channels.routing import get_default_application\n\nos.environ.setdefault('DJANGO_SETTINGS_MODULE', 'spug.settings')\ndjango.setup()\napplication = get_default_application()\n"
},
{
"path": "spug_api/spug/routing.py",
"content": "# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nfrom channels.routing import ProtocolTypeRouter\nfrom consumer import routing\n\napplication = ProtocolTypeRouter({\n 'websocket': routing.ws_router\n})\n"
},
{
"path": "spug_api/spug/settings.py",
"content": "\"\"\"\n# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\n\nDjango settings for spug project.\n\nGenerated by 'django-admin startproject' using Django 2.2.7.\n\nFor more information on this file, see\nhttps://docs.djangoproject.com/en/2.2/topics/settings/\n\nFor the full list of settings and their values, see\nhttps://docs.djangoproject.com/en/2.2/ref/settings/\n\"\"\"\n\nimport os\nimport re\n\n# Build paths inside the project like this: os.path.join(BASE_DIR, ...)\nBASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))\n\n# Quick-start development settings - unsuitable for production\n# See https://docs.djangoproject.com/en/2.2/howto/deployment/checklist/\n\n# SECURITY WARNING: keep the secret key used in production secret!\nSECRET_KEY = 'vk0do47)egwzz!uk49%(y3s(fpx4+ha@ugt-hcv&%&d@hwr&p7'\n\n# SECURITY WARNING: don't run with debug turned on in production!\nDEBUG = True\n\nALLOWED_HOSTS = ['127.0.0.1']\n\n# Application definition\n\nINSTALLED_APPS = [\n 'apps.account',\n 'apps.host',\n 'apps.setting',\n 'apps.exec',\n 'apps.schedule',\n 'apps.monitor',\n 'apps.alarm',\n 'apps.config',\n 'apps.app',\n 'apps.deploy',\n 'apps.notify',\n 'apps.repository',\n 'apps.home',\n 'channels',\n]\n\nMIDDLEWARE = [\n 'django.middleware.security.SecurityMiddleware',\n 'django.middleware.common.CommonMiddleware',\n 'libs.middleware.AuthenticationMiddleware',\n 'libs.middleware.HandleExceptionMiddleware',\n]\n\nROOT_URLCONF = 'spug.urls'\n\nWSGI_APPLICATION = 'spug.wsgi.application'\nASGI_APPLICATION = 'spug.routing.application'\n\n# Database\n# https://docs.djangoproject.com/en/2.2/ref/settings/#databases\n\nDATABASES = {\n 'default': {\n 'ATOMIC_REQUESTS': True,\n 'ENGINE': 'django.db.backends.sqlite3',\n 'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),\n }\n}\n\nCACHES = {\n \"default\": {\n \"BACKEND\": \"django_redis.cache.RedisCache\",\n \"LOCATION\": \"redis://127.0.0.1:6379/1\",\n \"OPTIONS\": {\n \"CLIENT_CLASS\": \"django_redis.client.DefaultClient\",\n }\n }\n}\n\nCHANNEL_LAYERS = {\n \"default\": {\n \"BACKEND\": \"channels_redis.core.RedisChannelLayer\",\n \"CONFIG\": {\n \"hosts\": [(\"127.0.0.1\", 6379)],\n \"capacity\": 1000,\n \"expiry\": 120,\n },\n },\n}\n\nTEMPLATES = [\n {\n 'BACKEND': 'django.template.backends.django.DjangoTemplates',\n 'DIRS': [],\n 'APP_DIRS': False,\n },\n]\n\nTOKEN_TTL = 8 * 3600\nSCHEDULE_KEY = 'spug:schedule'\nSCHEDULE_WORKER_KEY = 'spug:schedule:worker'\nMONITOR_KEY = 'spug:monitor'\nMONITOR_WORKER_KEY = 'spug:monitor:worker'\nEXEC_WORKER_KEY = 'spug:exec:worker'\nREQUEST_KEY = 'spug:request'\nBUILD_KEY = 'spug:build'\nREPOS_DIR = os.path.join(os.path.dirname(os.path.dirname(BASE_DIR)), 'repos')\nBUILD_DIR = os.path.join(REPOS_DIR, 'build')\nTRANSFER_DIR = os.path.join(BASE_DIR, 'storage', 'transfer')\n\n# Internationalization\n# https://docs.djangoproject.com/en/2.2/topics/i18n/\n\nLANGUAGE_CODE = 'en-us'\n\nTIME_ZONE = 'Asia/Shanghai'\n\nUSE_I18N = True\n\nUSE_L10N = True\n\nUSE_TZ = False\n\nAUTHENTICATION_EXCLUDES = (\n '/account/login/',\n '/setting/basic/',\n re.compile('/apis/.*'),\n)\n\nSPUG_VERSION = 'v3.3.3'\n\n# override default config\ntry:\n from spug.overrides import *\nexcept ImportError:\n pass\n"
},
{
"path": "spug_api/spug/urls.py",
"content": "\"\"\"spug URL Configuration\n# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\n\nThe `urlpatterns` list routes URLs to views. For more information please see:\n https://docs.djangoproject.com/en/2.2/topics/http/urls/\nExamples:\nFunction views\n 1. Add an import: from my_app import views\n 2. Add a URL to urlpatterns: path('', views.home, name='home')\nClass-based views\n 1. Add an import: from other_app.views import Home\n 2. Add a URL to urlpatterns: path('', Home.as_view(), name='home')\nIncluding another URLconf\n 1. Import the include() function: from django.urls import include, path\n 2. Add a URL to urlpatterns: path('blog/', include('blog.urls'))\n\"\"\"\nfrom django.urls import path, include\n\nurlpatterns = [\n path('account/', include('apps.account.urls')),\n path('host/', include('apps.host.urls')),\n path('exec/', include('apps.exec.urls')),\n path('schedule/', include('apps.schedule.urls')),\n path('monitor/', include('apps.monitor.urls')),\n path('alarm/', include('apps.alarm.urls')),\n path('setting/', include('apps.setting.urls')),\n path('config/', include('apps.config.urls')),\n path('app/', include('apps.app.urls')),\n path('deploy/', include('apps.deploy.urls')),\n path('repository/', include('apps.repository.urls')),\n path('home/', include('apps.home.urls')),\n path('notify/', include('apps.notify.urls')),\n path('file/', include('apps.file.urls')),\n path('apis/', include('apps.apis.urls')),\n]\n"
},
{
"path": "spug_api/spug/wsgi.py",
"content": "\"\"\"\n# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\nWSGI config for spug project.\n\nIt exposes the WSGI callable as a module-level variable named ``application``.\n\nFor more information on this file, see\nhttps://docs.djangoproject.com/en/2.2/howto/deployment/wsgi/\n\"\"\"\n\nimport os\n\nfrom django.core.wsgi import get_wsgi_application\n\n\nos.environ.setdefault('DJANGO_SETTINGS_MODULE', 'spug.settings')\n\napplication = get_wsgi_application()\n"
},
{
"path": "spug_api/storage/transfer/.gitkeep",
"content": ""
},
{
"path": "spug_api/tools/migrate.py",
"content": "import django\nimport sys\nimport os\n\nBASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))\nsys.path.append(BASE_DIR)\n\nos.environ.setdefault(\"DJANGO_SETTINGS_MODULE\", \"spug.settings\")\ndjango.setup()\n\nfrom django.conf import settings\nimport subprocess\nimport shutil\nimport sys\nimport os\nimport re\n\n\nclass Version:\n def __init__(self, version):\n self.version = re.sub('[^0-9.]', '', version).split('.')\n\n def __gt__(self, other):\n if not isinstance(other, Version):\n raise TypeError('required type Version')\n for v1, v2 in zip(self.version, other.version):\n if int(v1) == int(v2):\n continue\n elif int(v1) > int(v2):\n return True\n else:\n return False\n return False\n\n\nif __name__ == '__main__':\n old_version = Version(sys.argv[1])\n now_version = Version(settings.SPUG_VERSION)\n if old_version < Version('v3.0.2'):\n old_path = os.path.join(settings.BASE_DIR, 'repos')\n new_path = os.path.join(settings.REPOS_DIR)\n if not os.path.exists(new_path):\n print('执行 v3.0.1-beta.8 repos目录迁移')\n shutil.move(old_path, new_path)\n task = subprocess.Popen(f'cd {settings.BASE_DIR} && git checkout -- repos', shell=True)\n if task.wait() != 0:\n print('repos目录迁移失败，请联系官方人员')\n"
},
{
"path": "spug_api/tools/start-api.sh",
"content": "#!/bin/bash\n# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\n# start api service\n\ncd $(dirname $(dirname $0))\nif [ -f ./venv/bin/activate ]; then\n source ./venv/bin/activate\nfi\nexec gunicorn -b 127.0.0.1:9001 -w 2 --threads 8 --access-logfile - spug.wsgi\n"
},
{
"path": "spug_api/tools/start-monitor.sh",
"content": "#!/bin/bash\n# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\n# start monitor service\n\ncd $(dirname $(dirname $0))\nif [ -f ./venv/bin/activate ]; then\n source ./venv/bin/activate\nfi\n\nif command -v python3 &> /dev/null; then\n PYTHON=python3\nelse\n PYTHON=python\nfi\n\nexec $PYTHON manage.py runmonitor\n"
},
{
"path": "spug_api/tools/start-scheduler.sh",
"content": "#!/bin/bash\n# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\n# start schedule service\n\ncd $(dirname $(dirname $0))\nif [ -f ./venv/bin/activate ]; then\n source ./venv/bin/activate\nfi\n\nif command -v python3 &> /dev/null; then\n PYTHON=python3\nelse\n PYTHON=python\nfi\n\nexec $PYTHON manage.py runscheduler\n"
},
{
"path": "spug_api/tools/start-worker.sh",
"content": "#!/bin/bash\n# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\n# start worker service\n\ncd $(dirname $(dirname $0))\nif [ -f ./venv/bin/activate ]; then\n source ./venv/bin/activate\nfi\n\nif command -v python3 &> /dev/null; then\n PYTHON=python3\nelse\n PYTHON=python\nfi\n\nexec $PYTHON manage.py runworker\n"
},
{
"path": "spug_api/tools/start-ws.sh",
"content": "#!/bin/bash\n# Copyright: (c) OpenSpug Organization. https://github.com/openspug/spug\n# Copyright: (c) \n# Released under the AGPL-3.0 License.\n# start websocket service\n\ncd $(dirname $(dirname $0))\nif [ -f ./venv/bin/activate ]; then\n source ./venv/bin/activate\nfi\nexec daphne -p 9002 spug.asgi:application\n"
},
{
"path": "spug_api/tools/supervisor-spug.ini",
"content": "[program:spug-api]\ncommand = bash /data/spug/spug_api/tools/start-api.sh\nautostart = true\nstdout_logfile = /data/spug/spug_api/logs/api.log\nredirect_stderr = true\n\n[program:spug-ws]\ncommand = bash /data/spug/spug_api/tools/start-ws.sh\nautostart = true\nstdout_logfile = /data/spug/spug_api/logs/ws.log\nredirect_stderr = true\n\n[program:spug-worker]\ncommand = bash /data/spug/spug_api/tools/start-worker.sh\nautostart = true\nstdout_logfile = /data/spug/spug_api/logs/worker.log\nredirect_stderr = true\n\n[program:spug-monitor]\ncommand = bash /data/spug/spug_api/tools/start-monitor.sh\nautostart = true\nstdout_logfile = /data/spug/spug_api/logs/monitor.log\nredirect_stderr = true\n\n[program:spug-scheduler]\ncommand = bash /data/spug/spug_api/tools/start-scheduler.sh\nautostart = true\nstdout_logfile = /data/spug/spug_api/logs/scheduler.log\nredirect_stderr = true"
},
{
"path": "spug_web/.gitignore",
"content": "# See https://help.github.com/articles/ignoring-files/ for more about ignoring files.\n\n# dependencies\n/node_modules\n/.pnp\n.pnp.js\n/.idea/\n\n# testing\n/coverage\n\n# production\n/build\n\n# misc\n.DS_Store\n.env.local\n.env.development.local\n.env.test.local\n.env.production.local\n\nnpm-debug.log*\nyarn-debug.log*\nyarn-error.log*\n"
},
{
"path": "spug_web/README.md",
"content": "spug web"
},
{
"path": "spug_web/config-overrides.js",
"content": "/**\n * Copyright (c) OpenSpug Organization. https://github.com/openspug/spug\n * Copyright (c) \n * Released under the AGPL-3.0 License.\n */\nconst {override, addDecoratorsLegacy, addLessLoader} = require('customize-cra');\n\nmodule.exports = override(\n addDecoratorsLegacy(),\n addLessLoader({\n lessOptions: {\n javascriptEnabled: true,\n modifyVars: {\n '@primary-color': '#2563fc'\n }\n }\n }),\n);\n"
},
{
"path": "spug_web/jsconfig.json",
"content": "{\n \"compilerOptions\": {\n \"baseUrl\": \"src\"\n },\n \"include\": [\n \"src\"\n ]\n}\n"
},
{
"path": "spug_web/package.json",
"content": "{\n \"name\": \"spug_web\",\n \"version\": \"3.0.0\",\n \"private\": true,\n \"dependencies\": {\n \"@ant-design/icons\": \"^4.3.0\",\n \"ace-builds\": \"^1.4.13\",\n \"antd\": \"4.21.5\",\n \"axios\": \"^0.21.0\",\n \"bizcharts\": \"^3.5.9\",\n \"history\": \"^4.10.1\",\n \"lodash\": \"^4.17.19\",\n \"mobx\": \"^5.15.6\",\n \"mobx-react\": \"^6.3.0\",\n \"moment\": \"^2.24.0\",\n \"react\": \"^16.13.1\",\n \"react-ace\": \"^9.5.0\",\n \"react-dom\": \"^16.13.1\",\n \"react-router-dom\": \"^5.2.0\",\n \"react-scripts\": \"3.4.3\",\n \"xterm\": \"^4.6.0\",\n \"xterm-addon-fit\": \"^0.5.0\"\n },\n \"scripts\": {\n \"start\": \"react-app-rewired start\",\n \"build\": \"GENERATE_SOURCEMAP=false react-app-rewired build\",\n \"test\": \"react-app-rewired test\",\n \"eject\": \"react-scripts eject\"\n },\n \"eslintConfig\": {\n \"extends\": \"react-app\"\n },\n \"browserslist\": {\n \"production\": [\n \">0.2%\",\n \"not dead\",\n \"not op_mini all\"\n ],\n \"development\": [\n \"last 1 chrome version\",\n \"last 1 firefox version\",\n \"last 1 safari version\"\n ]\n },\n \"devDependencies\": {\n \"@babel/plugin-proposal-decorators\": \"^7.10.5\",\n \"customize-cra\": \"^1.0.0\",\n \"http-proxy-middleware\": \"0.19.2\",\n \"less\": \"^3.12.2\",\n \"less-loader\": \"^7.1.0\",\n \"react-app-rewired\": \"^2.1.6\"\n }\n}\n"
},
{
"path": "spug_web/public/index.html",
"content": "\n\n \n \n \n \n \n Spug \n \n \n \n \n \n \n\n"
},
{
"path": "spug_web/public/manifest.json",
"content": "{\n \"short_name\": \"Spug\",\n \"name\": \"Spug\",\n \"icons\": [\n {\n \"src\": \"favicon.ico\",\n \"sizes\": \"64x64 32x32 24x24 16x16\",\n \"type\": \"image/x-icon\"\n },\n {\n \"src\": \"logo.png\",\n \"type\": \"image/png\",\n \"sizes\": \"192x192\"\n },\n {\n \"src\": \"logo.png\",\n \"type\": \"image/png\",\n \"sizes\": \"512x512\"\n }\n ],\n \"start_url\": \".\",\n \"display\": \"standalone\",\n \"theme_color\": \"#000000\",\n \"background_color\": \"#ffffff\"\n}\n"
},
{
"path": "spug_web/public/robots.txt",
"content": "User-agent: *\nDisallow: /"
},
{
"path": "spug_web/src/App.js",
"content": "/**\n * Copyright (c) OpenSpug Organization. https://github.com/openspug/spug\n * Copyright (c) \n * Released under the AGPL-3.0 License.\n */\nimport React, { Component } from 'react';\nimport {Switch, Route} from 'react-router-dom';\nimport Login from './pages/login';\nimport WebSSH from './pages/ssh';\nimport Layout from './layout';\n\nclass App extends Component {\n render() {\n return (\n \n \n );\n }\n}\n\nexport default App;\n"
},
{
"path": "spug_web/src/components/ACEditor.js",
"content": "/**\n * Copyright (c) OpenSpug Organization. https://github.com/openspug/spug\n * Copyright (c) \n * Released under the AGPL-3.0 License.\n */\nimport React from 'react';\nimport Editor from 'react-ace';\nimport 'ace-builds/src-noconflict/mode-sh';\nimport 'ace-builds/src-noconflict/mode-text';\nimport 'ace-builds/src-noconflict/mode-json';\nimport 'ace-builds/src-noconflict/mode-space';\nimport 'ace-builds/src-noconflict/mode-python';\nimport 'ace-builds/src-noconflict/theme-tomorrow';\n\nexport default function (props) {\n const style = {fontFamily: 'Source Code Pro, Courier New, Courier, Monaco, monospace, PingFang SC, Microsoft YaHei', ...props.style}\n return (\n \n * Released under the AGPL-3.0 License.\n */\nimport React from 'react';\nimport { Link as ALink } from 'react-router-dom';\nimport { Divider, Button as AButton } from 'antd';\nimport { hasPermission } from 'libs';\n\nfunction canVisible(auth) {\n return !auth || hasPermission(auth)\n}\n\nclass Action extends React.Component {\n static Link(props) {\n return )\n data.push(el)\n }\n }\n\n render() {\n const children = [];\n if (Array.isArray(this.props.children)) {\n this.props.children.forEach(el => this._handle(children, el))\n } else {\n this._handle(children, this.props.children)\n }\n\n return \n {children}\n \n }\n}\n\nexport default Action\n"
},
{
"path": "spug_web/src/components/AppSelector.js",
"content": "/**\n * Copyright (c) OpenSpug Organization. https://github.com/openspug/spug\n * Copyright (c) \n * Released under the AGPL-3.0 License.\n */\nimport React, { useState, useEffect } from 'react';\nimport { Link } from 'react-router-dom';\nimport { observer } from 'mobx-react';\nimport { Modal, Menu, Spin, Input } from 'antd';\nimport { OrderedListOutlined, BuildOutlined, SearchOutlined } from '@ant-design/icons';\nimport { includes, http } from 'libs';\nimport styles from './index.module.less';\nimport envStore from 'pages/config/environment/store';\nimport lds from 'lodash';\n\nexport default observer(function AppSelector(props) {\n const [fetching, setFetching] = useState(false);\n const [env_id, setEnvId] = useState();\n const [search, setSearch] = useState();\n const [deploys, setDeploys] = useState([]);\n\n useEffect(() => {\n setFetching(true);\n http.get('/api/app/deploy/')\n .then(res => setDeploys(res))\n .finally(() => setFetching(false))\n if (!envStore.records.length) {\n envStore.fetchRecords().then(_initEnv)\n } else {\n _initEnv()\n }\n }, [])\n\n function _initEnv() {\n if (envStore.records.length) {\n setEnvId(envStore.records[0].id)\n }\n }\n\n let records = deploys.filter(x => x.env_id === Number(env_id));\n if (search) records = records.filter(x => includes(x['app_name'], search) || includes(x['app_key'], search));\n if (props.filter) records = records.filter(x => props.filter(x));\n return (\n \n \n * Released under the AGPL-3.0 License.\n */\nimport React from 'react';\nimport { Button } from 'antd';\nimport { hasPermission } from 'libs';\n\n\nexport default function AuthButton(props) {\n let disabled = props.disabled;\n if (props.auth && !hasPermission(props.auth)) {\n disabled = true;\n }\n return disabled ? null : \n}\n"
},
{
"path": "spug_web/src/components/AuthCard.js",
"content": "/**\n * Copyright (c) OpenSpug Organization. https://github.com/openspug/spug\n * Copyright (c) \n * Released under the AGPL-3.0 License.\n */\nimport React from 'react';\nimport {Card} from 'antd';\nimport { hasPermission } from 'libs';\n\n\nexport default function AuthCard(props) {\n let disabled = props.disabled === undefined ? false : props.disabled;\n if (props.auth && !hasPermission(props.auth)) {\n disabled = true;\n }\n return disabled ? null : {props.children} \n}\n"
},
{
"path": "spug_web/src/components/AuthDiv.js",
"content": "/**\n * Copyright (c) OpenSpug Organization. https://github.com/openspug/spug\n * Copyright (c) \n * Released under the AGPL-3.0 License.\n */\nimport React from 'react';\nimport { hasPermission } from 'libs';\n\n\nexport default function AuthDiv(props) {\n let disabled = props.disabled === undefined ? false : props.disabled;\n if (props.auth && !hasPermission(props.auth)) {\n disabled = true;\n }\n return disabled ? null : \n * Released under the AGPL-3.0 License.\n */\nimport React from 'react';\nimport { hasPermission } from 'libs';\n\n\nexport default function AuthFragment(props) {\n return hasPermission(props.auth) ? {props.children} : null\n}\n"
},
{
"path": "spug_web/src/components/Breadcrumb.js",
"content": "/**\n * Copyright (c) OpenSpug Organization. https://github.com/openspug/spug\n * Copyright (c) \n * Released under the AGPL-3.0 License.\n */\nimport React from 'react';\nimport { Breadcrumb } from 'antd';\nimport styles from './index.module.less';\n\n\nexport default class extends React.Component {\n static Item = Breadcrumb.Item\n\n render() {\n let title = this.props.title;\n if (!title) {\n const rawChildren = this.props.children;\n if (Array.isArray(rawChildren)) {\n title = rawChildren[rawChildren.length - 1].props.children\n } else {\n title = rawChildren.props.children\n }\n }\n\n return (\n \n * Released under the AGPL-3.0 License.\n */\nimport React from 'react'\n\n\nfunction Link(props) {\n return (\n \n {props.title}\n )\n}\n\nexport default Link"
},
{
"path": "spug_web/src/components/LinkButton.js",
"content": "/**\n * Copyright (c) OpenSpug Organization. https://github.com/openspug/spug\n * Copyright (c) \n * Released under the AGPL-3.0 License.\n */\nimport React from 'react';\nimport { Button } from 'antd';\nimport { hasPermission } from 'libs';\n\n\nexport default function LinkButton(props) {\n let disabled = props.disabled;\n if (props.auth && !hasPermission(props.auth)) {\n disabled = true;\n }\n return \n}\n"
},
{
"path": "spug_web/src/components/NotFound.js",
"content": "import React from 'react';\nimport styles from './index.module.less';\n\nexport default function NotFound() {\n return (\n store.f_status = e.target.value}>\n 全部 \n 报警发生 \n 报警恢复 \n \n ]}\n pagination={{\n showSizeChanger: true,\n showLessItems: true,\n showTotal: total => `共 ${total} 条`,\n pageSizeOptions: ['10', '20', '50', '100']\n }}\n columns={this.columns}/>\n )\n }\n}\n\nexport default ComTable\n"
},
{
"path": "spug_web/src/pages/alarm/alarm/index.js",
"content": "/**\n * Copyright (c) OpenSpug Organization. https://github.com/openspug/spug\n * Copyright (c) \n * Released under the AGPL-3.0 License.\n */\nimport React from 'react';\nimport { observer } from 'mobx-react';\nimport { SyncOutlined } from '@ant-design/icons';\nimport { Input, Button } from 'antd';\nimport { SearchForm, AuthDiv, Breadcrumb } from 'components';\nimport ComTable from './Table';\nimport store from './store';\n\nexport default observer(function () {\n return (\n \n \n 首页 \n 报警中心 \n 报警历史 \n \n \n \n store.f_name = e.target.value} placeholder=\"请输入\"/>\n \n \n \n \n \n \n )\n})\n"
},
{
"path": "spug_web/src/pages/alarm/alarm/store.js",
"content": "/**\n * Copyright (c) OpenSpug Organization. https://github.com/openspug/spug\n * Copyright (c) \n * Released under the AGPL-3.0 License.\n */\nimport { observable, computed } from 'mobx';\nimport http from 'libs/http';\n\nclass Store {\n @observable records = [];\n @observable isFetching = false;\n\n @observable f_name;\n @observable f_status = '';\n\n @computed get dataSource() {\n let records = this.records;\n if (this.f_name) records = records.filter(x => x.name.toLowerCase().includes(this.f_name.toLowerCase()));\n if (this.f_status) records = records.filter(x => x.status === this.f_status);\n return records\n }\n\n fetchRecords = () => {\n this.isFetching = true;\n http.get('/api/alarm/alarm/')\n .then(res => this.records = res)\n .finally(() => this.isFetching = false)\n };\n}\n\nexport default new Store()\n"
},
{
"path": "spug_web/src/pages/alarm/contact/Form.js",
"content": "/**\n * Copyright (c) OpenSpug Organization. https://github.com/openspug/spug\n * Copyright (c) \n * Released under the AGPL-3.0 License.\n */\nimport React, { useState, useMemo } from 'react';\nimport { observer } from 'mobx-react';\nimport { Modal, Form, Input, Tooltip, Checkbox, Divider, message } from 'antd';\nimport { ThunderboltOutlined, LoadingOutlined } from '@ant-design/icons';\nimport http from 'libs/http';\nimport store from './store';\n\nconst channelConfig = [\n {\n key: 'email',\n label: '邮箱',\n fields: [\n { name: 'email', label: '邮箱地址', placeholder: '请输入邮箱地址', testMode: '4' }\n ]\n },\n {\n key: 'ding',\n label: '钉钉',\n fields: [\n { name: 'ding', label: 'Webhook', placeholder: 'https://oapi.dingtalk.com/robot/send?access_token=xxx', testMode: '3' },\n { name: 'ding_secret', label: 'Secret', placeholder: 'SECxxxxxxxx', extra: '可选，机器人安全设置中的加签密钥' }\n ],\n help: { text: '钉钉收不到通知？请参考', link: 'https://ops.spug.cc/docs/use-problem#use-dd', linkText: '官方文档' }\n },\n {\n key: 'feishu',\n label: '飞书',\n fields: [\n { name: 'feishu', label: 'Webhook', placeholder: 'https://open.feishu.cn/open-apis/bot/v2/hook/xxx', testMode: '7' },\n { name: 'feishu_secret', label: 'Secret', placeholder: 'xxxxxxxx', extra: '可选，机器人安全设置中的签名校验密钥' }\n ]\n },\n {\n key: 'qy_wx',\n label: '企业微信',\n fields: [\n { name: 'qy_wx', label: 'Webhook', placeholder: 'https://qyapi.weixin.qq.com/cgi-bin/webhook/send?key=xxx', testMode: '5' }\n ]\n }\n];\n\nexport default observer(function () {\n const [form] = Form.useForm();\n const [loading, setLoading] = useState(false);\n const [testLoading, setTestLoading] = useState('0');\n\n const initialChannels = useMemo(() => ({\n email: !!store.record.email,\n ding: !!store.record.ding,\n feishu: !!store.record.feishu,\n qy_wx: !!store.record.qy_wx,\n }), []);\n\n const [channels, setChannels] = useState(initialChannels);\n\n function handleChannelToggle(key, checked) {\n setChannels(prev => ({ ...prev, [key]: checked }));\n if (!checked) {\n const channel = channelConfig.find(c => c.key === key);\n if (channel) {\n const resetFields = channel.fields.map(f => f.name);\n form.resetFields(resetFields);\n }\n }\n }\n\n function handleSubmit() {\n setLoading(true);\n const formData = form.getFieldsValue();\n formData['id'] = store.record.id;\n const secret = {};\n if (formData.ding_secret) secret.ding = formData.ding_secret;\n if (formData.feishu_secret) secret.feishu = formData.feishu_secret;\n delete formData.ding_secret;\n delete formData.feishu_secret;\n formData.secret = Object.keys(secret).length ? JSON.stringify(secret) : null;\n http.post('/api/alarm/contact/', formData)\n .then(res => {\n message.success('操作成功');\n store.formVisible = false;\n store.fetchRecords()\n }, () => setLoading(false))\n }\n\n function handleTest(mode, name) {\n const value = form.getFieldValue(name)\n if (!value) return message.error('请输入后再执行测试')\n setTestLoading(mode)\n http.post('/api/alarm/test/', {mode, value})\n .then(() => {\n message.success('执行成功')\n })\n .finally(() => setTestLoading('0'))\n }\n\n function Test(props) {\n return testLoading === props.mode ? (\n \n handleTest(props.mode, props.name)}/>\n \n );\n }\n\n return (\n store.formVisible = false}\n confirmLoading={loading}\n onOk={handleSubmit}>\n \n \n )\n})\n"
},
{
"path": "spug_web/src/pages/alarm/contact/Table.js",
"content": "/**\n * Copyright (c) OpenSpug Organization. https://github.com/openspug/spug\n * Copyright (c) \n * Released under the AGPL-3.0 License.\n */\nimport React from 'react';\nimport { observer } from 'mobx-react';\nimport { Table, Modal, message } from 'antd';\nimport { PlusOutlined } from '@ant-design/icons';\nimport { Action, TableCard, AuthButton } from 'components';\nimport { http, hasPermission } from 'libs';\nimport store from './store';\n\n@observer\nclass ComTable extends React.Component {\n componentDidMount() {\n store.fetchRecords()\n }\n\n handleDelete = (text) => {\n Modal.confirm({\n title: '删除确认',\n content: `确定要删除【${text['name']}】?`,\n onOk: () => {\n return http.delete('/api/alarm/contact/', {params: {id: text.id}})\n .then(() => {\n message.success('删除成功');\n store.fetchRecords()\n })\n }\n })\n };\n\n render() {\n return (\n }\n onClick={() => store.showForm()}>新建\n ]}\n pagination={{\n showSizeChanger: true,\n showLessItems: true,\n showTotal: total => `共 ${total} 条`,\n pageSizeOptions: ['10', '20', '50', '100']\n }}>\n \n \n \n \n \n \n \n {hasPermission('alarm.contact.edit|alarm.contact.del') && (\n (\n \n store.showForm(info)}>编辑 \n this.handleDelete(info)}>删除 \n \n )}/>\n )}\n \n )\n }\n}\n\nexport default ComTable\n"
},
{
"path": "spug_web/src/pages/alarm/contact/index.js",
"content": "/**\n * Copyright (c) OpenSpug Organization. https://github.com/openspug/spug\n * Copyright (c) \n * Released under the AGPL-3.0 License.\n */\nimport React from 'react';\nimport { observer } from 'mobx-react';\nimport { Input } from 'antd';\nimport { SearchForm, AuthDiv, Breadcrumb } from 'components';\nimport ComTable from './Table';\nimport ComForm from './Form';\nimport store from './store';\n\nexport default observer(function () {\n return (\n \n \n 首页 \n 报警中心 \n 报警联系人 \n \n \n \n store.f_name = e.target.value} placeholder=\"请输入\"/>\n \n \n \n );\n})\n"
},
{
"path": "spug_web/src/pages/alarm/contact/store.js",
"content": "/**\n * Copyright (c) OpenSpug Organization. https://github.com/openspug/spug\n * Copyright (c) \n * Released under the AGPL-3.0 License.\n */\nimport { observable, computed } from 'mobx';\nimport http from 'libs/http';\n\nclass Store {\n @observable records = [];\n @observable record = {};\n @observable isFetching = false;\n @observable formVisible = false;\n\n @observable f_name;\n\n @computed get dataSource() {\n let records = this.records;\n if (this.f_name) records = records.filter(x => x.name.toLowerCase().includes(this.f_name.toLowerCase()))\n return records\n }\n\n fetchRecords = () => {\n this.isFetching = true;\n return http.get('/api/alarm/contact/')\n .then(res => this.records = res)\n .finally(() => this.isFetching = false)\n };\n\n showForm = (info = {}) => {\n this.formVisible = true;\n this.record = info\n }\n}\n\nexport default new Store()\n"
},
{
"path": "spug_web/src/pages/alarm/group/Form.js",
"content": "/**\n * Copyright (c) OpenSpug Organization. https://github.com/openspug/spug\n * Copyright (c) \n * Released under the AGPL-3.0 License.\n */\nimport React, {useEffect, useState} from 'react';\nimport {observer} from 'mobx-react';\nimport {Modal, Form, Input, Transfer, Spin, message} from 'antd';\nimport http from 'libs/http';\nimport store from './store';\n\nexport default observer(function () {\n const [form] = Form.useForm();\n const [contacts, setContacts] = useState([]);\n const [loading, setLoading] = useState(false);\n const [fetching, setFetching] = useState(false);\n\n useEffect(() => {\n setFetching(true)\n http.get('/api/alarm/contact/?with_push=1')\n .then(res => setContacts(res))\n .finally(() => setFetching(false))\n }, []);\n\n function handleSubmit() {\n setLoading(true);\n const formData = form.getFieldsValue();\n formData['id'] = store.record.id;\n http.post('/api/alarm/group/', formData)\n .then(res => {\n message.success('操作成功');\n store.formVisible = false;\n store.fetchRecords()\n }, () => setLoading(true))\n }\n\n return (\n store.formVisible = false}\n confirmLoading={loading}\n onOk={handleSubmit}>\n \n \n )\n})"
},
{
"path": "spug_web/src/pages/alarm/group/Table.js",
"content": "/**\n * Copyright (c) OpenSpug Organization. https://github.com/openspug/spug\n * Copyright (c) \n * Released under the AGPL-3.0 License.\n */\nimport React from 'react';\nimport { observer } from 'mobx-react';\nimport { Table, Modal, message } from 'antd';\nimport { PlusOutlined } from '@ant-design/icons';\nimport { Action, TableCard, AuthButton } from 'components';\nimport { http, hasPermission } from 'libs';\nimport store from './store';\nimport contactStore from '../contact/store';\n\n@observer\nclass ComTable extends React.Component {\n constructor(props) {\n super(props);\n this.state = {\n contactMap: {}\n }\n }\n\n componentDidMount() {\n store.fetchRecords();\n if (contactStore.records.length === 0) {\n contactStore.fetchRecords().then(this._handleContacts)\n } else {\n this._handleContacts()\n }\n }\n\n _handleContacts = () => {\n const tmp = {};\n for (let item of contactStore.records) {\n tmp[item.id] = item\n }\n this.setState({contactMap: tmp})\n };\n\n handleDelete = (text) => {\n Modal.confirm({\n title: '删除确认',\n content: `确定要删除【${text['name']}】?`,\n onOk: () => {\n return http.delete('/api/alarm/group/', {params: {id: text.id}})\n .then(() => {\n message.success('删除成功');\n store.fetchRecords()\n })\n }\n })\n };\n\n render() {\n return (\n }\n onClick={() => store.showForm()}>新建\n ]}\n pagination={{\n showSizeChanger: true,\n showLessItems: true,\n showTotal: total => `共 ${total} 条`,\n pageSizeOptions: ['10', '20', '50', '100']\n }}>\n \n `${value.length}个`}/>\n \n {hasPermission('alarm.group.edit|alarm.group.del') && (\n (\n \n store.showForm(info)}>编辑 \n this.handleDelete(info)}>删除 \n \n )}/>\n )}\n \n )\n }\n}\n\nexport default ComTable\n"
},
{
"path": "spug_web/src/pages/alarm/group/index.js",
"content": "/**\n * Copyright (c) OpenSpug Organization. https://github.com/openspug/spug\n * Copyright (c) \n * Released under the AGPL-3.0 License.\n */\nimport React from 'react';\nimport { observer } from 'mobx-react';\nimport { Input } from 'antd';\nimport { SearchForm, AuthDiv, Breadcrumb } from 'components';\nimport ComTable from './Table';\nimport ComForm from './Form';\nimport store from './store';\n\nexport default observer(function () {\n return (\n \n \n 首页 \n 报警中心 \n 报警联系组 \n \n \n \n store.f_name = e.target.value} placeholder=\"请输入\"/>\n \n \n \n );\n})\n"
},
{
"path": "spug_web/src/pages/alarm/group/store.js",
"content": "/**\n * Copyright (c) OpenSpug Organization. https://github.com/openspug/spug\n * Copyright (c) \n * Released under the AGPL-3.0 License.\n */\nimport { observable, computed } from 'mobx';\nimport http from 'libs/http';\n\nclass Store {\n @observable records = [];\n @observable record = {};\n @observable isFetching = false;\n @observable formVisible = false;\n\n @observable f_name;\n\n @computed get dataSource() {\n let records = this.records;\n if (this.f_name) records = records.filter(x => x.name.toLowerCase().includes(this.f_name.toLowerCase()));\n if (this.f_type) records = records.filter(x => x.type.toLowerCase().includes(this.f_type.toLowerCase()));\n return records\n }\n\n fetchRecords = () => {\n this.isFetching = true;\n return http.get('/api/alarm/group/')\n .then(res => this.records = res)\n .finally(() => this.isFetching = false)\n };\n\n showForm = (info = {}) => {\n this.formVisible = true;\n this.record = info\n }\n}\n\nexport default new Store()\n"
},
{
"path": "spug_web/src/pages/config/app/Form.js",
"content": "/**\n * Copyright (c) OpenSpug Organization. https://github.com/openspug/spug\n * Copyright (c) \n * Released under the AGPL-3.0 License.\n */\nimport React, { useState } from 'react';\nimport { observer } from 'mobx-react';\nimport { Modal, Form, Input, message } from 'antd';\nimport http from 'libs/http';\nimport store from './store';\n\nexport default observer(function () {\n const [form] = Form.useForm();\n const [loading, setLoading] = useState(false)\n\n function handleSubmit() {\n setLoading(true);\n const formData = form.getFieldsValue();\n formData['id'] = store.record.id;\n http.post('/api/app/', formData)\n .then(res => {\n message.success('操作成功');\n store.formVisible = false;\n store.fetchRecords()\n }, () => setLoading(false))\n }\n\n return (\n store.formVisible = false}\n confirmLoading={loading}\n onOk={handleSubmit}>\n \n \n )\n})"
},
{
"path": "spug_web/src/pages/config/app/Rel.js",
"content": "/**\n * Copyright (c) OpenSpug Organization. https://github.com/openspug/spug\n * Copyright (c) \n * Released under the AGPL-3.0 License.\n */\nimport React from 'react';\nimport { observer } from 'mobx-react';\nimport { Modal, Form, Transfer, message, Tabs } from 'antd';\nimport { http, hasPermission } from 'libs';\nimport serviceStore from '../service/store';\nimport store from './store';\n\n@observer\nclass Rel extends React.Component {\n constructor(props) {\n super(props);\n this.state = {\n loading: false,\n services: [],\n apps: store.records.filter(x => x.id !== store.record.id).map(x => ({...x, key: x.id, _key: x.key}))\n }\n }\n\n componentDidMount() {\n if (serviceStore.records.length === 0) {\n serviceStore.fetchRecords().then(this._updateRecords)\n } else {\n this._updateRecords()\n }\n }\n\n _updateRecords = () => {\n const services = serviceStore.records.map(x => {\n return {...x, key: x.id, _key: x.key}\n });\n this.setState({services})\n };\n\n handleSubmit = () => {\n this.setState({loading: true});\n const {app, service} = store.confRel;\n http.patch('/api/app/', {id: store.record.id, rel_apps: app, rel_services: service})\n .then(res => {\n message.success('操作成功');\n store.relVisible = false;\n store.fetchRecords()\n }, () => this.setState({loading: false}))\n };\n\n render() {\n return (\n store.relVisible = false}\n confirmLoading={this.state.loading}\n footer={hasPermission('config.app.edit_config') ? undefined : null}\n onOk={this.handleSubmit}>\n \n \n \n store.confRel.app = keys}\n render={item => `${item.name}(${item._key})`}/>\n \n \n \n \n store.confRel.service = keys}\n render={item => `${item.name}(${item._key})`}/>\n \n \n \n \n )\n }\n}\n\nexport default Rel\n"
},
{
"path": "spug_web/src/pages/config/app/Table.js",
"content": "/**\n * Copyright (c) OpenSpug Organization. https://github.com/openspug/spug\n * Copyright (c) \n * Released under the AGPL-3.0 License.\n */\nimport React from 'react';\nimport { observer } from 'mobx-react';\nimport { Table, Modal, message } from 'antd';\nimport { PlusOutlined } from '@ant-design/icons';\nimport { Action, TableCard, AuthButton } from 'components';\nimport { http, hasPermission, history } from 'libs';\nimport store from './store';\n\n@observer\nclass ComTable extends React.Component {\n componentDidMount() {\n store.fetchRecords()\n }\n\n handleDelete = (text) => {\n Modal.confirm({\n title: '删除确认',\n content: `确定要删除【${text['name']}】?`,\n onOk: () => {\n return http.delete('/api/app/', {params: {id: text.id}})\n .then(() => {\n message.success('删除成功');\n store.fetchRecords()\n })\n }\n })\n };\n\n toConfig = (info) => {\n store.record = info;\n history.push(`/config/setting/app/${info.id}`)\n }\n\n render() {\n let data = store.records;\n if (store.f_name) {\n data = data.filter(item => item['name'].toLowerCase().includes(store.f_name.toLowerCase()))\n }\n return (\n }\n onClick={() => store.showForm()}>新建\n ]}\n pagination={{\n showSizeChanger: true,\n showLessItems: true,\n showTotal: total => `共 ${total} 条`,\n pageSizeOptions: ['10', '20', '50', '100']\n }}>\n \n \n \n {hasPermission('config.app.edit|config.app.del|config.app.view_config') && (\n (\n \n store.showForm(info)}>编辑 \n store.showRel(info)}>依赖 \n this.toConfig(info)}>配置 \n this.handleDelete(info)}>删除 \n \n )}/>\n )}\n \n )\n }\n}\n\nexport default ComTable\n"
},
{
"path": "spug_web/src/pages/config/app/index.js",
"content": "/**\n * Copyright (c) OpenSpug Organization. https://github.com/openspug/spug\n * Copyright (c) \n * Released under the AGPL-3.0 License.\n */\nimport React from 'react';\nimport { observer } from 'mobx-react';\nimport { Input } from 'antd';\nimport { SearchForm, AuthDiv, Breadcrumb } from 'components';\nimport ComTable from './Table';\nimport ComForm from './Form';\nimport Rel from './Rel';\nimport store from './store';\n\nexport default observer(function () {\n return (\n \n \n 首页 \n 配置中心 \n 应用配置 \n \n \n \n store.f_name = e.target.value} placeholder=\"请输入\"/>\n \n \n \n )\n})\n"
},
{
"path": "spug_web/src/pages/config/app/store.js",
"content": "/**\n * Copyright (c) OpenSpug Organization. https://github.com/openspug/spug\n * Copyright (c) \n * Released under the AGPL-3.0 License.\n */\nimport { observable } from \"mobx\";\nimport http from 'libs/http';\n\nclass Store {\n @observable records = [];\n @observable record = {};\n @observable confRel = {};\n @observable isFetching = false;\n @observable formVisible = false;\n @observable relVisible = false;\n\n @observable f_name;\n\n fetchRecords = () => {\n this.isFetching = true;\n return http.get('/api/app/')\n .then(res => this.records = res)\n .finally(() => this.isFetching = false)\n };\n\n showForm = (info = {}) => {\n this.formVisible = true;\n this.record = info\n };\n\n showRel = (info) => {\n this.relVisible = true;\n this.record = info;\n this.confRel = {\n app: info['rel_apps'],\n service: info['rel_services']\n }\n }\n}\n\nexport default new Store()\n"
},
{
"path": "spug_web/src/pages/config/environment/Form.js",
"content": "/**\n * Copyright (c) OpenSpug Organization. https://github.com/openspug/spug\n * Copyright (c) \n * Released under the AGPL-3.0 License.\n */\nimport React, { useState } from 'react';\nimport { observer } from 'mobx-react';\nimport { Modal, Form, Input, message } from 'antd';\nimport http from 'libs/http';\nimport store from './store';\n\nexport default observer(function () {\n const [form] = Form.useForm();\n const [loading, setLoading] = useState(false);\n\n function handleSubmit() {\n setLoading(true);\n const formData = form.getFieldsValue();\n formData['id'] = store.record.id;\n http.post('/api/config/environment/', formData)\n .then(res => {\n message.success('操作成功');\n store.formVisible = false;\n store.fetchRecords()\n }, () => setLoading(false))\n }\n\n return (\n store.formVisible = false}\n confirmLoading={loading}\n onOk={handleSubmit}>\n \n \n )\n})"
},
{
"path": "spug_web/src/pages/config/environment/Table.js",
"content": "/**\n * Copyright (c) OpenSpug Organization. https://github.com/openspug/spug\n * Copyright (c) \n * Released under the AGPL-3.0 License.\n */\nimport React, { useEffect } from 'react';\nimport { observer } from 'mobx-react';\nimport { Table, Modal, Divider, message } from 'antd';\nimport { PlusOutlined, UpSquareOutlined, DownSquareOutlined } from '@ant-design/icons';\nimport { Action, TableCard, AuthButton } from 'components';\nimport { http, hasPermission } from 'libs';\nimport store from './store';\n\nfunction ComTable() {\n useEffect(() => {\n store.fetchRecords()\n }, [])\n\n function handleDelete(text) {\n Modal.confirm({\n title: '删除确认',\n content: `确定要删除【${text['name']}】?`,\n onOk: () => {\n return http.delete('/api/config/environment/', {params: {id: text.id}})\n .then(() => {\n message.success('删除成功');\n store.fetchRecords()\n })\n }\n })\n }\n\n function handleSort(info, sort) {\n store.fetching = true;\n http.patch('/api/config/environment/', {id: info.id, sort})\n .then(store.fetchRecords, () => store.fetching = false)\n }\n\n return (\n }\n onClick={() => store.showForm()}>新建\n ]}\n pagination={{\n showSizeChanger: true,\n showLessItems: true,\n showTotal: total => `共 ${total} 条`,\n pageSizeOptions: ['10', '20', '50', '100']\n }}>\n (\n \n \n \n {hasPermission('config.env.edit|config.env.del') && (\n (\n \n store.showForm(info)}>编辑 \n handleDelete(info)}>删除 \n \n )}/>\n )}\n \n )\n}\n\nexport default observer(ComTable)\n"
},
{
"path": "spug_web/src/pages/config/environment/index.js",
"content": "/**\n * Copyright (c) OpenSpug Organization. https://github.com/openspug/spug\n * Copyright (c) \n * Released under the AGPL-3.0 License.\n */\nimport React from 'react';\nimport { observer } from 'mobx-react';\nimport { Input } from 'antd';\nimport { SearchForm, AuthDiv, Breadcrumb } from 'components';\nimport ComTable from './Table';\nimport ComForm from './Form';\nimport store from './store';\n\nexport default observer(function () {\n return (\n \n \n 首页 \n 配置中心 \n 环境管理 \n \n \n \n store.f_name = e.target.value} placeholder=\"请输入\"/>\n \n \n \n )\n})\n"
},
{
"path": "spug_web/src/pages/config/environment/store.js",
"content": "/**\n * Copyright (c) OpenSpug Organization. https://github.com/openspug/spug\n * Copyright (c) \n * Released under the AGPL-3.0 License.\n */\nimport { observable, computed } from \"mobx\";\nimport http from 'libs/http';\n\nclass Store {\n @observable records = [];\n @observable record = {};\n @observable idMap = {};\n @observable isFetching = false;\n @observable formVisible = false;\n\n @observable f_name;\n\n @computed get dataSource() {\n let records = this.records;\n if (this.f_name) records = records.filter(x => x.name.toLowerCase().includes(this.f_name.toLowerCase()));\n return records\n }\n\n fetchRecords = () => {\n this.isFetching = true;\n return http.get('/api/config/environment/')\n .then(res => {\n this.records = res;\n for (let item of res) {\n this.idMap[item.id] = item\n }\n })\n .finally(() => this.isFetching = false)\n };\n\n showForm = (info = {}) => {\n this.formVisible = true;\n this.record = info\n }\n}\n\nexport default new Store()\n"
},
{
"path": "spug_web/src/pages/config/service/Form.js",
"content": "/**\n * Copyright (c) OpenSpug Organization. https://github.com/openspug/spug\n * Copyright (c) \n * Released under the AGPL-3.0 License.\n */\nimport React, { useState } from 'react';\nimport { observer } from 'mobx-react';\nimport { Modal, Form, Input, message } from 'antd';\nimport http from 'libs/http';\nimport store from './store';\n\nexport default observer(function () {\n const [form] = Form.useForm();\n const [loading, setLoading] = useState(false);\n\n function handleSubmit() {\n setLoading(true);\n const formData = form.getFieldsValue();\n formData['id'] = store.record.id;\n http.post('/api/config/service/', formData)\n .then(res => {\n message.success('操作成功');\n store.formVisible = false;\n store.fetchRecords()\n }, () => setLoading(false))\n }\n\n return (\n store.formVisible = false}\n confirmLoading={loading}\n onOk={handleSubmit}>\n \n \n )\n})"
},
{
"path": "spug_web/src/pages/config/service/Table.js",
"content": "/**\n * Copyright (c) OpenSpug Organization. https://github.com/openspug/spug\n * Copyright (c) \n * Released under the AGPL-3.0 License.\n */\nimport React from 'react';\nimport { observer } from 'mobx-react';\nimport { Table, Modal, message } from 'antd';\nimport { PlusOutlined } from '@ant-design/icons';\nimport { Action, TableCard, AuthButton } from 'components';\nimport { http, hasPermission, history } from 'libs';\nimport store from './store';\n\n@observer\nclass ComTable extends React.Component {\n componentDidMount() {\n store.fetchRecords()\n }\n\n handleDelete = (text) => {\n Modal.confirm({\n title: '删除确认',\n content: `将会同步删除服务的配置信息，确定要删除服务【${text['name']}】? `,\n onOk: () => {\n return http.delete('/api/config/service/', {params: {id: text.id}})\n .then(() => {\n message.success('删除成功');\n store.fetchRecords()\n })\n }\n })\n };\n\n toConfig = (info) => {\n store.record = info;\n history.push(`/config/setting/src/${info.id}`)\n }\n\n render() {\n return (\n }\n onClick={() => store.showForm()}>新建\n ]}\n pagination={{\n showSizeChanger: true,\n showLessItems: true,\n showTotal: total => `共 ${total} 条`,\n pageSizeOptions: ['10', '20', '50', '100']\n }}>\n \n \n \n {hasPermission('config.src.edit|config.src.del|config.src.view_config') && (\n (\n \n store.showForm(info)}>编辑 \n this.toConfig(info)}>配置 \n this.handleDelete(info)}>删除 \n \n )}/>\n )}\n \n )\n }\n}\n\nexport default ComTable\n"
},
{
"path": "spug_web/src/pages/config/service/index.js",
"content": "/**\n * Copyright (c) OpenSpug Organization. https://github.com/openspug/spug\n * Copyright (c) \n * Released under the AGPL-3.0 License.\n */\nimport React from 'react';\nimport { observer } from 'mobx-react';\nimport { Input } from 'antd';\nimport { SearchForm, AuthDiv, Breadcrumb } from 'components';\nimport ComTable from './Table';\nimport ComForm from './Form';\nimport store from './store';\n\nexport default observer(function () {\n return (\n \n \n 首页 \n 配置中心 \n 服务配置 \n \n \n \n store.f_name = e.target.value} placeholder=\"请输入\"/>\n \n \n \n );\n})\n"
},
{
"path": "spug_web/src/pages/config/service/store.js",
"content": "/**\n * Copyright (c) OpenSpug Organization. https://github.com/openspug/spug\n * Copyright (c) \n * Released under the AGPL-3.0 License.\n */\nimport { observable, computed } from 'mobx';\nimport http from 'libs/http';\n\nclass Store {\n @observable records = [];\n @observable record = {};\n @observable isFetching = false;\n @observable formVisible = false;\n\n @observable f_name;\n\n @computed get dataSource() {\n let records = this.records;\n if (this.f_name) records = records.filter(x => x.name.toLowerCase().includes(this.f_name.toLowerCase()));\n return records\n }\n\n fetchRecords = () => {\n this.isFetching = true;\n return http.get('/api/config/service/')\n .then(res => this.records = res)\n .finally(() => this.isFetching = false)\n };\n\n showForm = (info = {}) => {\n this.formVisible = true;\n this.record = info\n }\n}\n\nexport default new Store()\n"
},
{
"path": "spug_web/src/pages/config/setting/DiffConfig.js",
"content": "/**\n * Copyright (c) OpenSpug Organization. https://github.com/openspug/spug\n * Copyright (c) \n * Released under the AGPL-3.0 License.\n */\nimport React from 'react';\nimport { observer } from 'mobx-react';\nimport { ArrowLeftOutlined } from '@ant-design/icons';\nimport { Modal, Form, Table, Row, Col, Checkbox, Button, Alert } from 'antd';\nimport http from 'libs/http';\nimport envStore from '../environment/store';\nimport styles from './index.module.css';\nimport store from './store';\n\n@observer\nclass Record extends React.Component {\n constructor(props) {\n super(props);\n this.state = {\n loading: false,\n records: [],\n envs: [],\n page: 0,\n hideSame: false\n }\n }\n\n handleEnvCheck = (env) => {\n const index = this.state.envs.indexOf(env);\n if (index !== -1) {\n this.state.envs.splice(index, 1);\n } else {\n this.state.envs.push(env);\n }\n this.setState({envs: this.state.envs})\n };\n\n handleNext = () => {\n this.setState({page: this.state.page + 1, loading: true});\n const envs = this.state.envs.map(x => x.id);\n http.post('/api/config/diff/', {type: store.type, o_id: store.id, envs})\n .then(res => this.setState({records: res}))\n .finally(() => this.setState({loading: false}))\n };\n\n getColumns = () => {\n const columns = [{title: 'Key', dataIndex: 'key'}];\n for (let env of this.state.envs) {\n columns.push({title: `${env.name} (${env.key})`, dataIndex: env.id})\n }\n return columns\n };\n\n render() {\n let records = this.state.records;\n const {loading, envs, page, hideSame} = this.state;\n if (hideSame) {\n records = records.filter(item => new Set(envs.map(x => item[x.id])).size !== 1)\n }\n return (\n store.diffVisible = false}\n footer={null}>\n
\n
\n \n )\n})\n"
},
{
"path": "spug_web/src/components/AuthButton.js",
"content": "/**\n * Copyright (c) OpenSpug Organization. https://github.com/openspug/spug\n * Copyright (c) \n \n ({key: x.id, label: x.name, title: x.name}))}\n onSelect={({selectedKeys}) => setEnvId(selectedKeys[0])}/>\n \n
\n\n \n \n \n
\n \n
\n {lds.get(envStore.idMap, `${env_id}.name`)}
\n \n {records.map(item => (\n
\n props.onSelect(item)}>\n {item.extend === '1' ?
\n ))}\n {records.length === 0 &&\n {item.app_name}
\n {item.app_key}
\n 该环境下还没有可发布或构建的应用哦，快去应用管理创建应用发布配置吧。
}\n {props.children}
\n}\n"
},
{
"path": "spug_web/src/components/AuthFragment.js",
"content": "/**\n * Copyright (c) OpenSpug Organization. https://github.com/openspug/spug\n * Copyright (c) \n \n {this.props.children}\n \n {this.props.extra ? (\n
\n )\n }\n}"
},
{
"path": "spug_web/src/components/Link.js",
"content": "/**\n * Copyright (c) OpenSpug Organization. https://github.com/openspug/spug\n * Copyright (c) \n {title}\n {this.props.extra}\n
\n ) : null}\n \n \n * Released under the AGPL-3.0 License.\n */\nimport React from 'react';\nimport { Row, Col, Form } from 'antd';\nimport styles from './index.module.less';\n\nexport default class extends React.Component {\n static Item(props) {\n return (\n \n {props.children}\n \n \n )\n }\n\n render() {\n return (\n \n * Released under the AGPL-3.0 License.\n */\nimport React from 'react';\nimport { Card, Col, Row } from 'antd';\nimport lodash from 'lodash';\nimport styles from './index.module.less';\n\n\nclass StatisticsCard extends React.Component {\n static Item = (props) => {\n return (\n \n \n {items.map((item, index) => (\n
\n \n )\n }\n}\n\nexport default StatisticsCard\n"
},
{
"path": "spug_web/src/components/TableCard.js",
"content": "/**\n * Copyright (c) OpenSpug Organization. https://github.com/openspug/spug\n * Copyright (c) \n * Released under the AGPL-3.0 License.\n */\nimport React, { useState, useEffect, useRef } from 'react';\nimport { Table, Space, Divider, Popover, Checkbox, Button, Input, Select } from 'antd';\nimport { ReloadOutlined, SettingOutlined, FullscreenOutlined, SearchOutlined } from '@ant-design/icons';\nimport styles from './index.module.less';\n\nlet TableFields = localStorage.getItem('TableFields')\nTableFields = TableFields ? JSON.parse(TableFields) : {}\n\nfunction Search(props) {\n let keys = props.keys || [''];\n keys = keys.map(x => x.split('/'));\n const [key, setKey] = useState(keys[0][0]);\n return (\n \n {columns.map((item, index) => (\n {item.title} \n ))}\n \n )\n }\n\n function handleCheckAll(e) {\n if (e.target.checked) {\n onFieldsChange(columns.map((_, index) => index))\n } else {\n onFieldsChange([])\n }\n }\n\n function handleFullscreen() {\n if (props.rootRef.current && document.fullscreenEnabled) {\n if (document.fullscreenElement) {\n document.exitFullscreen()\n } else {\n props.rootRef.current.requestFullscreen()\n }\n }\n }\n\n return (\n \n * Released under the AGPL-3.0 License.\n */\nimport StatisticsCard from './StatisticsCard';\nimport SearchForm from './SearchForm';\nimport LinkButton from './LinkButton';\nimport AuthButton from './AuthButton';\nimport AuthFragment from './AuthFragment';\nimport AuthCard from './AuthCard';\nimport AuthDiv from './AuthDiv';\nimport ACEditor from './ACEditor';\nimport Action from './Action';\nimport TableCard from './TableCard';\nimport Breadcrumb from './Breadcrumb';\nimport AppSelector from './AppSelector';\nimport NotFound from './NotFound';\nimport Link from './Link';\n\nexport {\n StatisticsCard,\n AuthFragment,\n SearchForm,\n LinkButton,\n AuthButton,\n AuthCard,\n AuthDiv,\n ACEditor,\n Action,\n TableCard,\n Breadcrumb,\n AppSelector,\n NotFound,\n Link,\n}\n"
},
{
"path": "spug_web/src/components/index.module.less",
"content": ".searchForm {\n padding: 24px 24px 0 24px;\n background-color: #fff;\n border-radius: 2px;\n margin-bottom: 16px;\n\n :global(.ant-form-item) {\n display: flex;\n }\n\n :global(.ant-form-item-control-wrapper) {\n flex: 1;\n }\n\n :global(.ant-form-item-label) {\n padding-right: 8px;\n }\n\n :global(.ant-form-item-control) {\n overflow: hidden;\n }\n}\n\n.statisticsCard {\n position: relative;\n text-align: center;\n\n span {\n color: rgba(0, 0, 0, .45);\n display: inline-block;\n line-height: 22px;\n margin-bottom: 4px;\n }\n\n p {\n font-size: 32px;\n line-height: 32px;\n margin: 0;\n }\n\n em {\n background-color: #e8e8e8;\n position: absolute;\n height: 56px;\n width: 1px;\n top: 0;\n right: 0;\n }\n}\n\n.tableCard {\n border: 1px solid #f0f0f0;\n background: #fff;\n border-radius: 2px;\n padding: 24px;\n\n .toolbar {\n display: flex;\n align-items: center;\n justify-content: space-between;\n margin-bottom: 20px;\n\n .title {\n flex: 1;\n font-weight: 500;\n font-size: 16px;\n opacity: 0.8;\n margin-right: 24px;\n }\n\n .option {\n display: flex;\n align-items: center;\n justify-content: flex-end;\n\n .icons {\n :global(.anticon) {\n font-size: 16px;\n margin-left: 8px;\n }\n }\n }\n }\n}\n\n.tableFields {\n :global(.ant-popover-title) {\n padding: 10px 16px;\n display: flex;\n justify-content: space-between;\n align-items: center;\n }\n\n :global(.ant-popover-inner-content) {\n padding: 8px 0;\n\n :global(.ant-checkbox-group) {\n display: flex;\n flex-direction: column;\n }\n\n :global(.ant-checkbox-wrapper) {\n height: 30px;\n line-height: 30px;\n margin: 0;\n padding: 0 16px;\n }\n\n :global(.ant-checkbox-wrapper):hover {\n background: rgba(0, 0, 0, 0.025)\n }\n }\n}\n\n.tableFooter {\n position: fixed;\n right: 0;\n bottom: 0;\n display: flex;\n align-items: center;\n height: 48px;\n width: calc(100% - 208px);\n padding: 0 24px;\n background: #fff;\n\n .left {\n flex: 1;\n\n span {\n color: #1890ff;\n font-weight: 600;\n }\n }\n}\n\n.breadcrumb {\n margin: -24px -24px 24px -24px;\n padding: 16px 24px;\n background: #fff;\n border-bottom: 1px solid #e8e8e8;\n\n .title {\n display: flex;\n justify-content: space-between;\n align-items: center;\n font-size: 20px;\n margin-top: 8px;\n }\n}\n\n.appSelector {\n display: flex;\n background-color: #fff;\n padding: 16px 0;\n min-height: 500px;\n\n .left {\n flex: 220px;\n border-right: 1px solid #e8e8e8;\n overflow: hidden;\n }\n\n .right {\n flex: 580px;\n padding: 8px 40px;\n }\n\n .title {\n display: flex;\n justify-content: space-between;\n align-items: center;\n margin-bottom: 24px;\n color: rgba(0, 0, 0, .85);\n font-weight: 500;\n font-size: 20px;\n line-height: 28px;\n\n .text {\n padding-right: 12px;\n width: 300px;\n overflow: hidden;\n white-space: nowrap;\n text-overflow: ellipsis;\n }\n }\n\n .appItem {\n display: flex;\n align-items: center;\n padding: 12px 16px;\n margin-bottom: 8px;\n background: #fafafa;\n border-radius: 2px;\n\n :global(.anticon) {\n color: #1890ff;\n margin-right: 16px;\n }\n\n .body {\n flex: 1;\n overflow: hidden;\n text-overflow: ellipsis;\n white-space: nowrap;\n }\n }\n\n .appItem:hover {\n cursor: pointer;\n background: #e6f7ff;\n }\n\n .tips {\n margin-top: 32px;\n color: #888;\n }\n}\n\n\n.notFound {\n display: flex;\n height: 80%;\n align-items: center;\n\n .imgBlock {\n flex: 0 0 62.5%;\n width: 62.5%;\n zoom: 1;\n padding-right: 88px;\n\n .img {\n float: right;\n height: 360px;\n width: 100%;\n max-width: 430px;\n background-size: contain;\n background: url('./404.svg') no-repeat 50% 50%;\n }\n }\n\n .title {\n color: #434e59;\n font-size: 72px;\n font-weight: 600;\n line-height: 72px;\n margin-bottom: 24px;\n }\n\n .desc {\n color: rgba(0, 0, 0, .45);\n font-size: 20px;\n line-height: 28px;\n margin-bottom: 16px;\n }\n}"
},
{
"path": "spug_web/src/gStore.js",
"content": "/**\n * Copyright (c) OpenSpug Organization. https://github.com/openspug/spug\n * Copyright (c) \n * Released under the AGPL-3.0 License.\n */\nimport { observable } from 'mobx';\nimport http from 'libs/http';\nimport themes from 'pages/ssh/themes';\n\nclass Store {\n isReady = false;\n @observable terminal = {\n fontSize: 16,\n fontFamily: 'Courier',\n theme: 'dark',\n styles: themes['dark']\n };\n\n _handleSettings = (res) => {\n if (res.terminal) {\n const terminal = JSON.parse(res.terminal)\n const styles = themes[terminal.theme]\n if (styles) {\n terminal.styles = styles\n } else {\n terminal.styles = themes['dark']\n terminal.theme = 'dark'\n }\n this.terminal = terminal\n }\n }\n\n fetchUserSettings = () => {\n if (this.isReady) return\n http.get('/api/setting/user/')\n .then(res => {\n this.isReady = true\n this._handleSettings(res)\n })\n };\n\n updateUserSettings = (key, value) => {\n return http.post('/api/setting/user/', {key, value})\n .then(res => {\n this.isReady = true\n this._handleSettings(res)\n })\n }\n}\n\nexport default new Store()"
},
{
"path": "spug_web/src/index.js",
"content": "/**\n * Copyright (c) OpenSpug Organization. https://github.com/openspug/spug\n * Copyright (c) \n * Released under the AGPL-3.0 License.\n */\nimport React from 'react';\nimport ReactDOM from 'react-dom';\nimport { Router } from 'react-router-dom';\nimport { ConfigProvider } from 'antd';\nimport zhCN from 'antd/es/locale/zh_CN';\nimport './index.less';\nimport App from './App';\nimport moment from 'moment';\nimport 'moment/locale/zh-cn';\nimport * as serviceWorker from './serviceWorker';\nimport { history, updatePermissions } from 'libs';\n\nmoment.locale('zh-cn');\nupdatePermissions();\n\nReactDOM.render(\n \n document.fullscreenElement || document.body}>\n \n ,\n document.getElementById('root')\n);\n\n// If you want your app to work offline and load faster, you can change\n// unregister() to register() below. Note this comes with some pitfalls.\n// Learn more about service workers: https://bit.ly/CRA-PWA\nserviceWorker.unregister();\n"
},
{
"path": "spug_web/src/index.less",
"content": "@import '~antd/dist/antd.less';\n\nbody {\n margin: 0;\n font-family: -apple-system, BlinkMacSystemFont, Helvetica Neue, PingFang SC, Microsoft YaHei, Source Han Sans SC, Noto Sans CJK SC, WenQuanYi Micro Hei, sans-serif;\n -webkit-font-smoothing: antialiased;\n -moz-osx-font-smoothing: grayscale;\n overflow: hidden;\n}\n\ncode {\n font-family: Source Code Pro, Menlo, Monaco, Consolas, Courier New, monospace, Courier, PingFang SC, Microsoft YaHei;\n}\n\n.ant-form-item-extra {\n font-size: 13px;\n padding-top: 6px;\n}\n\n/* Common CSS style */\n.none {\n display: none;\n}\n\n.btn {\n color: #2563fc;\n cursor: pointer;\n}"
},
{
"path": "spug_web/src/layout/Footer.js",
"content": "/**\n * Copyright (c) OpenSpug Organization. https://github.com/openspug/spug\n * Copyright (c) \n * Released under the AGPL-3.0 License.\n */\nimport React from 'react';\nimport { Layout } from 'antd';\nimport { CopyrightOutlined, GithubOutlined } from '@ant-design/icons';\nimport styles from './layout.module.less';\n\n\nexport default function () {\n return (\n \n \n \n )\n}\n"
},
{
"path": "spug_web/src/layout/Header.js",
"content": "/**\n * Copyright (c) OpenSpug Organization. https://github.com/openspug/spug\n * Copyright (c) \n * Released under the AGPL-3.0 License.\n */\nimport React from 'react';\nimport { Link } from 'react-router-dom';\nimport { Layout, Dropdown, Menu, Avatar, Divider } from 'antd';\nimport { MenuFoldOutlined, MenuUnfoldOutlined, UserOutlined, LogoutOutlined, CodeOutlined, DownOutlined } from '@ant-design/icons';\nimport { AuthDiv } from 'components';\nimport Notification from './Notification';\nimport styles from './layout.module.less';\nimport http from '../libs/http';\nimport history from '../libs/history';\nimport avatar from './avatar.png';\n\nexport default function (props) {\n\n function handleLogout() {\n history.push('/');\n http.get('/api/account/logout/')\n }\n\n function openTerminal() {\n window.open('/ssh')\n }\n\n const UserMenu = (\n \n );\n\n const ToolsMenu = (\n \n );\n\n return (\n \n \n )\n}"
},
{
"path": "spug_web/src/layout/Notification.js",
"content": "import React, { useState, useEffect } from 'react';\nimport { Menu, List, Dropdown, Badge, Button, notification } from 'antd';\nimport {\n NotificationOutlined,\n MonitorOutlined,\n FlagOutlined,\n ScheduleOutlined,\n AlertOutlined\n} from '@ant-design/icons';\nimport { http, X_TOKEN } from 'libs';\nimport moment from 'moment';\nimport styles from './layout.module.less';\n\nlet ws = {readyState: 3};\nlet timer;\n\n\nfunction Icon(props) {\n switch (props.type) {\n case 'monitor':\n return \n \n )\n}"
},
{
"path": "spug_web/src/layout/index.js",
"content": "/**\n * Copyright (c) OpenSpug Organization. https://github.com/openspug/spug\n * Copyright (c) \n * Released under the AGPL-3.0 License.\n */\nimport React, { useState, useEffect } from 'react';\nimport { Switch, Route } from 'react-router-dom';\nimport { Layout, message } from 'antd';\nimport { NotFound } from 'components';\nimport Sider from './Sider';\nimport Header from './Header';\nimport Footer from './Footer'\nimport routes from '../routes';\nimport { hasPermission, isMobile } from 'libs';\nimport styles from './layout.module.less';\n\nfunction initRoutes(Routes, routes) {\n for (let route of routes) {\n if (route.component) {\n if (!route.auth || hasPermission(route.auth)) {\n Routes.push()\n }\n } else if (route.child) {\n initRoutes(Routes, route.child)\n }\n }\n}\n\nexport default function () {\n const [collapsed, setCollapsed] = useState(false)\n const [Routes, setRoutes] = useState([]);\n\n useEffect(() => {\n if (isMobile) {\n setCollapsed(true);\n message.warn('检测到您在移动设备上访问，请使用横屏模式。', 5)\n }\n const Routes = [];\n initRoutes(Routes, routes);\n setRoutes(Routes)\n }, [])\n\n return (\n \n \n \n setCollapsed(!collapsed)}/>\n \n \n {Routes}\n \n \n \n \n \n \n )\n}\n"
},
{
"path": "spug_web/src/layout/layout.module.less",
"content": ".header {\n display: flex;\n flex-direction: row;\n justify-content: space-between;\n padding: 0 12px 0 0;;\n height: 48px;\n line-height: 48px;\n background: #fff;\n box-shadow: 0 1px 4px rgba(0, 21, 41, 0.08);\n z-index: 2;\n\n .right {\n display: flex;\n flex-direction: row;\n align-items: center;\n\n .link {\n color: #333333;\n cursor: pointer;\n padding: 0 16px;\n\n &:hover {\n background: rgba(0, 0, 0, 0.025);\n }\n }\n }\n\n .terminal {\n padding: 0 12px;\n cursor: pointer;\n line-height: 48px;\n display: flex;\n justify-content: center;\n align-items: center;\n\n &:hover {\n background: rgba(0, 0, 0, 0.025);\n }\n }\n\n .user {\n .action {\n cursor: pointer;\n padding: 0 12px;\n display: inline-block;\n transition: all 0.3s;\n height: 100%;\n }\n\n .action:hover {\n background: rgba(0, 0, 0, 0.025);\n }\n }\n\n .trigger {\n cursor: pointer;\n transition: all 0.3s, padding 0s;\n padding: 0 12px;\n }\n\n .trigger:hover {\n background: rgba(0, 0, 0, 0.025);\n }\n}\n\n.notify {\n width: 350px;\n padding: 0;\n\n .item {\n align-items: center;\n cursor: pointer;\n padding: 12px 24px;\n }\n\n .item:hover {\n background-color: rgb(233, 247, 254);\n }\n\n .btn {\n line-height: 46px;\n text-align: center;\n cursor: pointer;\n border-top: 1px solid #e8e8e8;\n }\n}\n\n.notify :global(.ant-dropdown-menu-item:hover) {\n background-color: #fff;\n}\n\n.content {\n display: flex;\n flex-direction: column;\n justify-content: space-between;\n padding: 24px;\n overflow-y: scroll;\n}\n\n.content::-webkit-scrollbar {\n width: 0;\n height: 0;\n}\n\n.sider {\n height: 100%;\n width: 208px;\n min-height: 100vh;\n box-shadow: 2px 0 8px 0 rgba(29, 35, 41, 0.05);\n overflow: hidden;\n\n .menus {\n overflow: auto;\n }\n\n .menus::-webkit-scrollbar {\n width: 0;\n height: 0;\n }\n\n .logo {\n height: 64px;\n line-height: 64px;\n overflow: hidden;\n text-align: center;\n }\n\n .logo img {\n height: 30px;\n }\n}\n\n.footer {\n width: 100%;\n padding: 20px;\n font-size: 14px;\n text-align: center;\n display: flex;\n flex-direction: column;\n\n .links {\n margin-bottom: 7px;\n\n .item {\n margin-right: 40px;\n }\n }\n}"
},
{
"path": "spug_web/src/libs/functools.js",
"content": "/**\n * Copyright (c) OpenSpug Organization. https://github.com/openspug/spug\n * Copyright (c) \n * Released under the AGPL-3.0 License.\n */\nlet Permission = {\n isReady: false,\n isSuper: false,\n permissions: []\n};\n\nexport let X_TOKEN;\nexport const isMobile = /Android|iPhone/i.test(navigator.userAgent)\n\nexport function updatePermissions() {\n X_TOKEN = localStorage.getItem('token');\n Permission.isReady = true;\n Permission.isSuper = localStorage.getItem('is_supper') === 'true';\n try {\n Permission.permissions = JSON.parse(localStorage.getItem('permissions') || '[]');\n } catch (e) {\n\n }\n}\n\n// 前端页面的权限判断(仅作为前端功能展示的控制，具体权限控制应在后端实现)\nexport function hasPermission(strCode) {\n const {isSuper, permissions} = Permission;\n if (!strCode || isSuper) return true;\n for (let or_item of strCode.split('|')) {\n if (isSubArray(permissions, or_item.split('&'))) {\n return true\n }\n }\n return false\n}\n\nexport function clsNames(...args) {\n return args.filter(x => x).join(' ')\n}\n\nfunction isInclude(s, keys) {\n if (!s) return false\n if (Array.isArray(keys)) {\n for (let k of keys) {\n k = k.toLowerCase()\n if (s.toLowerCase().includes(k)) return true\n }\n return false\n } else {\n let k = keys.toLowerCase()\n return s.toLowerCase().includes(k)\n }\n}\n\n// 字符串包含判断\nexport function includes(s, keys) {\n if (Array.isArray(s)) {\n for (let i of s) {\n if (isInclude(i, keys)) return true\n }\n return false\n } else {\n return isInclude(s, keys)\n }\n}\n\n// 清理输入的命令中包含的\\r符号\nexport function cleanCommand(text) {\n return text ? text.replace(/\\r\\n/g, '\\n') : ''\n}\n\n// 数组包含关系判断\nexport function isSubArray(parent, child) {\n for (let item of child) {\n if (!parent.includes(item.trim())) {\n return false\n }\n }\n return true\n}\n\n// 用于替换toFixed方法，去除toFixed方法多余的0和小数点\nexport function trimFixed(data, bit) {\n return String(data.toFixed(bit)).replace(/0*$/, '').replace(/\\.$/, '')\n}\n\n// 日期\nexport function human_date(date) {\n const now = date || new Date();\n let month = now.getMonth() + 1;\n let day = now.getDate();\n return `${now.getFullYear()}-${month < 10 ? '0' + month : month}-${day < 10 ? '0' + day : day}`\n}\n\n// 时间\nexport function human_time(date) {\n const now = date || new Date();\n const hour = now.getHours() < 10 ? '0' + now.getHours() : now.getHours();\n const minute = now.getMinutes() < 10 ? '0' + now.getMinutes() : now.getMinutes();\n const second = now.getSeconds() < 10 ? '0' + now.getSeconds() : now.getSeconds();\n return `${hour}:${minute}:${second}`\n}\n\nexport function human_datetime(date) {\n return `${human_date(date)} ${human_time(date)}`\n}\n\n// 生成唯一id\nexport function uniqueId() {\n return 'xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx'.replace(/[xy]/g, function (c) {\n const r = Math.random() * 16 | 0;\n return (c === 'x' ? r : (r & 0x3) | 0x8).toString(16)\n });\n}"
},
{
"path": "spug_web/src/libs/history.js",
"content": "/**\n * Copyright (c) OpenSpug Organization. https://github.com/openspug/spug\n * Copyright (c) \n * Released under the AGPL-3.0 License.\n */\nimport {createBrowserHistory} from 'history';\n\nexport default createBrowserHistory()\n"
},
{
"path": "spug_web/src/libs/http.js",
"content": "/**\n * Copyright (c) OpenSpug Organization. https://github.com/openspug/spug\n * Copyright (c) \n * Released under the AGPL-3.0 License.\n */\nimport http from 'axios'\nimport history from './history'\nimport { X_TOKEN } from './functools';\nimport { message } from 'antd';\n\n// response处理\nfunction handleResponse(response) {\n let result;\n if (response.status === 401) {\n result = '会话过期，请重新登录';\n if (history.location.pathname !== '/') {\n history.push('/', {from: history.location})\n } else {\n return Promise.reject()\n }\n } else if (response.status === 200) {\n if (response.data.error) {\n result = response.data.error\n } else if (response.data.hasOwnProperty('data')) {\n return Promise.resolve(response.data.data)\n } else if (response.headers['content-type'] === 'application/octet-stream') {\n return Promise.resolve(response)\n } else if (!response.config.isInternal) {\n return Promise.resolve(response.data)\n } else {\n result = '无效的数据格式'\n }\n } else {\n result = `请求失败: ${response.status} ${response.statusText}`\n }\n message.error(result);\n return Promise.reject(result)\n}\n\n// 请求拦截器\nhttp.interceptors.request.use(request => {\n request.isInternal = request.url.startsWith('/api/');\n if (request.isInternal) {\n request.headers['X-Token'] = X_TOKEN\n }\n request.timeout = request.timeout || 30000;\n return request;\n});\n\n// 返回拦截器\nhttp.interceptors.response.use(response => {\n return handleResponse(response)\n}, error => {\n if (error.response) {\n return handleResponse(error.response)\n }\n const result = '请求异常: ' + error.message;\n message.error(result);\n return Promise.reject(result)\n});\n\nexport default http;\n"
},
{
"path": "spug_web/src/libs/index.js",
"content": "/**\n * Copyright (c) OpenSpug Organization. https://github.com/openspug/spug\n * Copyright (c) \n * Released under the AGPL-3.0 License.\n */\nimport _http from './http';\nimport _history from './history';\n\nexport * from './functools';\nexport * from './router';\nexport const http = _http;\nexport const history = _history;\nexport const VERSION = 'v3.3.3';\n"
},
{
"path": "spug_web/src/libs/libs.module.css",
"content": ".container {\n display: flex;\n height: 80%;\n align-items: center;\n}\n\n.imgBlock {\n flex: 0 0 62.5%;\n width: 62.5%;\n zoom: 1;\n padding-right: 88px;\n}\n\n.img {\n float: right;\n height: 360px;\n width: 100%;\n max-width: 430px;\n background-size: contain;\n background: url('./404.svg') no-repeat 50% 50%;\n}\n\n.title {\n color: #434e59;\n font-size: 72px;\n font-weight: 600;\n line-height: 72px;\n margin-bottom: 24px;\n}\n\n.desc {\n color: rgba(0, 0, 0, .45);\n font-size: 20px;\n line-height: 28px;\n margin-bottom: 16px;\n}"
},
{
"path": "spug_web/src/libs/router.js",
"content": "/**\n * Copyright (c) OpenSpug Organization. https://github.com/openspug/spug\n * Copyright (c) \n * Released under the AGPL-3.0 License.\n */\nimport React, { Suspense } from 'react';\nimport { Switch, Route } from 'react-router-dom';\nimport moduleRoutes from '../routes';\nimport styles from './libs.module.css';\n\n\n// 创建单个路由\nexport function makeRoute(path, component) {\n return {subPath: path, component}\n}\n\n// 创建模块路由\nexport function makeModuleRoute(prefix, routes) {\n return {prefix, routes}\n}\n\n// 404 页面\nfunction NotFound() {\n return (\n \n {this.routes.map(route => \n \n \n )\n }\n}\n"
},
{
"path": "spug_web/src/pages/alarm/alarm/Table.js",
"content": "/**\n * Copyright (c) OpenSpug Organization. https://github.com/openspug/spug\n * Copyright (c) \n * Released under the AGPL-3.0 License.\n */\nimport React from 'react';\nimport { observer } from 'mobx-react';\nimport { Radio, Tag, Tooltip } from 'antd';\nimport { QuestionCircleOutlined } from '@ant-design/icons';\nimport { TableCard } from 'components';\nimport store from './store';\nimport groupStore from '../group/store';\n\n@observer\nclass ComTable extends React.Component {\n constructor(props) {\n super(props);\n this.state = {\n groupMap: {}\n }\n }\n\n componentDidMount() {\n store.fetchRecords();\n if (groupStore.records.length === 0) {\n groupStore.fetchRecords().then(this._handleGroups)\n } else {\n this._handleGroups()\n }\n }\n\n _handleGroups = () => {\n const tmp = {};\n for (let item of groupStore.records) {\n tmp[item.id] = item.name\n }\n this.setState({groupMap: tmp})\n };\n\n columns = [{\n title: '任务名称',\n dataIndex: 'name',\n }, {\n title: '监控类型',\n dataIndex: 'type',\n }, {\n title: '监控对象',\n dataIndex: 'target'\n }, {\n title: '状态',\n dataIndex: 'status',\n render: value => value === '1' ? 报警发生 : 故障恢复 \n }, {\n title: '持续时间',\n dataIndex: 'duration',\n }, {\n title: '通知方式',\n dataIndex: 'notify_mode',\n }, {\n title: '通知对象',\n dataIndex: 'notify_grp',\n render: value => value.map(id => this.state.groupMap[id]).join(',')\n }, {\n title: '发生时间',\n dataIndex: 'created_at'\n }];\n\n render() {\n return (\n \n \n \n
\n )}\n loading={store.isFetching}\n dataSource={store.dataSource}\n onReload={store.fetchRecords}\n actions={[\n \n
\n )\n}"
},
{
"path": "spug_web/src/components/SearchForm.js",
"content": "/**\n * Copyright (c) OpenSpug Organization. https://github.com/openspug/spug\n * Copyright (c) \n
\n \n
\n 404
\n抱歉，你访问的页面不存在
\n \n \n
\n )\n }\n}\n"
},
{
"path": "spug_web/src/components/StatisticsCard.js",
"content": "/**\n * Copyright (c) OpenSpug Organization. https://github.com/openspug/spug\n * Copyright (c) \n {props.title}\n
\n )\n };\n\n render() {\n let items = lodash.get(this.props, 'children', []);\n if (!lodash.isArray(items)) items = [items];\n const span = Math.ceil(24 / (items.length || 1));\n return (\n {props.value}
\n {props.bordered !== false && }\n\n \n {actions.map((item, index) => (\n {item} \n ))}\n \n
\n ) : null\n}\n\nfunction Header(props) {\n const columns = props.columns || [];\n const actions = props.actions || [];\n const fields = props.fields || [];\n const onFieldsChange = props.onFieldsChange;\n\n const Fields = () => {\n return (\n 已选择 {length} 项
\n \n
\n )\n}\n\nfunction TableCard(props) {\n const rootRef = useRef();\n const batchActions = props.batchActions || [];\n const selected = props.selected || [];\n const [fields, setFields] = useState([]);\n const [defaultFields, setDefaultFields] = useState([]);\n const [columns, setColumns] = useState([]);\n\n useEffect(() => {\n let [_columns, _fields] = [props.columns, []];\n if (props.children) {\n if (Array.isArray(props.children)) {\n _columns = props.children.filter(x => x.props).map(x => x.props)\n } else {\n _columns = [props.children.props]\n }\n }\n let hideFields = _columns.filter(x => x.hide).map(x => x.title)\n if (props.tKey) {\n if (TableFields[props.tKey]) {\n hideFields = TableFields[props.tKey]\n } else {\n TableFields[props.tKey] = hideFields\n localStorage.setItem('TableFields', JSON.stringify(TableFields))\n }\n }\n for (let [index, item] of _columns.entries()) {\n if (!hideFields.includes(item.title)) _fields.push(index)\n }\n setFields(_fields);\n setColumns(_columns);\n setDefaultFields(_fields);\n // eslint-disable-next-line react-hooks/exhaustive-deps\n }, [])\n\n function handleFieldsChange(fields) {\n setFields(fields)\n if (props.tKey) {\n TableFields[props.tKey] = columns.filter((_, index) => !fields.includes(index)).map(x => x.title)\n localStorage.setItem('TableFields', JSON.stringify(TableFields))\n }\n }\n\n return (\n {props.title}
\n \n \n {actions.map((item, index) => (\n {item} \n ))}\n \n {actions.length ? : null}\n \n \n 列展示,\n onFieldsChange(props.defaultFields)}>重置\n ]}\n overlayClassName={styles.tableFields}\n trigger=\"click\"\n placement=\"bottomRight\"\n content={\n \n
\n \n \n fields.includes(index))}\n dataSource={props.dataSource}\n rowSelection={props.rowSelection}\n expandable={props.expandable}\n pagination={props.pagination}/>\n {selected.length ?
\n )\n}\n\nTableCard.Search = Search;\nexport default TableCard"
},
{
"path": "spug_web/src/components/index.js",
"content": "/**\n * Copyright (c) OpenSpug Organization. https://github.com/openspug/spug\n * Copyright (c) \n {props.collapsed ?
\n \n \n \n 工具服务 \n \n \n \n
\n window.open('https://spug.cc/')}>官网
\n window.open('https://ops.spug.cc/docs/about-spug/')}>文档
\n \n \n \n \n
\n {content}
;\n const btn = ;\n notification.warning({message: title, description, btn, key, top: 64, duration: null})\n } catch (e) {\n\n }\n }\n }\n }\n\n function handleVisible(visible) {\n if (visible) {\n fetch()\n }\n }\n\n function handleRead(e, item) {\n e.stopPropagation();\n if (reads.indexOf(item.id) === -1) {\n reads.push(item.id);\n setReads([...reads])\n http.patch('/api/notify/', {ids: [item.id]})\n }\n }\n\n function handleReadAll() {\n const ids = notifies.map(x => x.id);\n setReads(ids);\n http.patch('/api/notify/', {ids})\n }\n\n const count = notifies.length - reads.length;\n return (\n \n \n \n (\n handleRead(e, item)}>\n }\n title={{item.title}}\n description={[\n \n )}/>\n {notifies.length !== 0 && (\n \n \n )}>\n \n
\n )\n}"
},
{
"path": "spug_web/src/layout/Sider.js",
"content": "import React, { useState, useEffect } from 'react';\nimport { Layout, Menu } from 'antd';\nimport { hasPermission, history } from 'libs';\nimport styles from './layout.module.less';\nimport routes from '../routes';\nimport logo from './logo-spug-white.png';\n\nlet selectedKey = window.location.pathname;\nconst OpenKeysMap = {};\nfor (let item of routes) {\n if (item.child) {\n for (let sub of item.child) {\n if (sub.title) OpenKeysMap[sub.path] = item.title\n }\n } else if (item.title) {\n OpenKeysMap[item.path] = 1\n }\n}\n\nexport default function Sider(props) {\n const [openKeys, setOpenKeys] = useState([]);\n const [menus, setMenus] = useState([]);\n\n useEffect(() => {\n const tmp = []\n for (let item of routes) {\n const menu = handleRoute(item)\n tmp.push(menu)\n }\n setMenus(tmp)\n // eslint-disable-next-line react-hooks/exhaustive-deps\n }, [])\n\n function handleRoute(item) {\n if (item.auth && !hasPermission(item.auth)) return\n if (!item.title) return;\n const menu = {label: item.title, key: item.path, icon: item.icon}\n if (item.child) {\n menu.children = []\n for (let sub of item.child) {\n const subMenu = handleRoute(sub)\n menu.children.push(subMenu)\n }\n }\n return menu\n }\n\n const tmp = window.location.pathname;\n const openKey = OpenKeysMap[tmp];\n if (openKey) {\n selectedKey = tmp;\n if (openKey !== 1 && !props.collapsed && !openKeys.includes(openKey)) {\n setOpenKeys([...openKeys, openKey])\n }\n }\n return (\n {item.content}
,\n {moment(item['created_at']).fromNow()}
\n ]}/>\n 全部 已读
\n )}\n \n 0 ? count : 0}>\n \n
\n \n 
\n
\n \n history.push(menu.key)}/>\n
\n \n Loading...
}>\n \n
\n )\n}\n\n// 组合路由\nexport class Router extends React.Component {\n constructor(props) {\n super(props);\n this.routes = [];\n this.initialRoutes();\n }\n\n initialRoutes() {\n for (let moduleRoute of moduleRoutes) {\n for (let route of moduleRoute['routes']) {\n route['path'] = moduleRoute['prefix'] + route['subPath'];\n this.routes.push(route)\n }\n }\n }\n\n render() {\n return (\n \n
\n \n
\n 404
\n抱歉，你访问的页面不存在
\n 报警历史记录
\n \n handleSort(info, 'up')}\n style={{cursor: 'pointer', color: '#1890ff'}}/>\n \n handleSort(info, 'down')}\n style={{cursor: 'pointer', color: '#1890ff'}}/>\n
\n )}/>\n \n \n {envStore.records.map((item, index) => (\n this.handleEnvCheck(item)}\n style={{cursor: 'pointer', borderTop: index ? '1px solid #e8e8e8' : ''}}>\n x.id).includes(item.id)}/>\n \n \n \n \n
\n \n \n this.setState({hideSame: !hideSame})}>隐藏相同配置 \n \n \n \n );\n }\n}\n\nexport default Record\n"
},
{
"path": "spug_web/src/pages/config/setting/Form.js",
"content": "/**\n * Copyright (c) OpenSpug Organization. https://github.com/openspug/spug\n * Copyright (c)